Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?
Authors: Francesco Innocenti
El Mehdi Achour
Ryan Singh
Christopher Buckley
Venue: NeurIPS 2024 poster (2024)
Keywords: Predictive Coding
Backpropagation
Deep Neural Networks
Loss Landscape
Saddle Points
Gradient Descent
Vanishing Gradients
Local Learning
Inference Learning
Primary area: neuroscience_and_cognitive_science
TL;DR: Predictive coding inference makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.

Abstract: Predictive coding (PC) is an energy-based learning algorithm that performs iterative inference over network activities before updating weights. Recent work suggests that PC can converge in fewer learning steps than backpropagation thanks to its inference procedure. However, these advantages are not always observed, and the impact of PC inference on learning is not theoretically well understood. To address this gap, we study the geometry of the PC weight landscape at the inference equilibrium of the network activities. For deep linear networks, we first show that the equilibrated PC energy is equal to a rescaled mean squared error loss with a weight-dependent rescaling. We then prove that many highly degenerate (non-strict) saddles of the loss including the origin become much easier to escape (strict) in the equilibrated energy. Experiments on both linear and non-linear networks strongly validate our theory and further suggest that all the saddles of the equilibrated energy are strict. Overall, this work shows that PC inference makes the loss landscape of feedforward networks more benign and robust to vanishing gradients, while also highlighting the fundamental challenge of scaling PC to very deep models.

Only Strict Saddles in the Energy Landscape of Predictive Coding Networks?
Francesco Innocenti
El Mehdi Achour
School of Engineering and Informatics University of Sussex
RWTH Aachen University
Aachen, Germany
F. Aamocenti@susseee ac.uk .k
achour@mathc.rwthe wwth--acheh... de
Ryan Singh
Christopher L. Buckley
School of Engineering and Informatics University of Sussex rs7730sussex. ac uk
School of Engineering and Informatics University of Sussex VERSES
c.1. www.kkeyyyssssee.ccooooo ac uk
Abstract
Predictive coding (PC) is an energy-based learning algorithm that performs iterative inference over network activities before updating weights. Recent work suggests that PC can converge in fewer learning steps than backpropagation thanks to its inference procedure. However, these advantages the impact of PC inference on learning not theoretically well understood. To address this gap, study the geometry of the PC weight landscape the inference equilibrium of the network activities. For deep linear networks, we first show that the equilibrated PC energy is equal weight-depender rescaling. We then prove that many highly degenerate (non-strict) saddles the loss including the origin become much easier escape (strict) in the equilibrated energy. Experiments on both linear and non-linear networks strongly validate our theory and further suggest that all the saddles of the equilibrated energy are strict. Overall, this work shows that PC inference makes the loss landscape of feedforward networks more benign and robust vanishing gradients, while also highlighting the fundamental challenge of scaling PC to very deep models.
not always observed, and
rescaled mean squared error loss with
Introduction
Originating as a general principle of brain function, predictive coding (PC) has in recent years been developed into local learning algorithm that could provide biologically plausible alternative backpropagation (BP) [32, 31, 43]. Deep neural networks (DNNs) trained with PC have shown comparable performance to BP standard small-to-medium machine learning tasks, including classification, generation and memory association [31, 43, 41]. PC networks (PCNs) are also highly versatile, allowing for arbitrary computational graphs [45, 10], hybrid and causal inference [44, 59], and temporal prediction [35].
In contrast BP, and similar other energy mergy-based algorithms [e.g. 49, 38], PC performs iterative (approximately Bayesian) inference over network activities before weight updates. This has been recently described fundamentally different principle of credit assignment for learning in the brain called "prospective configuration [54], where weights follow activities (rather than the other way around). While the inference process key PC incurs additional computational cost, it has been suggested to provide many benefits for learning including faster convergence [54, 3, 18]. However,
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

these speed-upe ups are not consistently observed across datasets, models and optimisers [3]. and the impact of PC inference on learning more generally not theoretically well understood (see $A.2.1).
To address this gap, we study the geometry of the effective landscape on which PC learns: the weight landscape at the inference equilibrium of the network activities (defined in $2.2). Our theory considers deep linear networks (DLNs), the standard model for theoretical studies of the loss landscape (see $A.2). Despite being able to learn only linear representations, DLNs have non-convex loss landscapes with non-linear learning dynamics that have proved to be a useful model for understanding non-linear networks [e.g. 48]. In contrast to previous theories of PC [3. 2, 18), we do not make any additional assumptions or approximations (see $A.2), and we empirically verify that our linear theory holds for non-linear networks.
For DINs, we first show that, at the inference equilibrium, the PC energy is equal to a rescaled mean squared error (MSH) loss with non-trivial, weight-dependent rescaling (Theorem 1). We then compare saddle points of the loss, which have been recently characterised [23, 1J. to those of the cquilibrated energy. Such saddles, which are ubiquitous in the loss landscape of neural networks [11, IJ. can be of two main types: "strict", where the Hessian is indefinite (Def. 1); and "non-strict", where an escape direction is found in higher-order derivatives [15, 23, 1]. Non-strict saddles are particularly problematic for first-order methods like (stochastic) gradient descent (SGD) since they are by definition at least second-order critical points. While SGD can be exponentially slowed in the vicinity of strict saddles [12], it can effectively get stuck in non-strict ones [47, 7] (see $A.2 for review). This is the phenomenon of vanishing gradients viewed from a landscape perspective [39, 6].
By contrast, here we prove that many non-strict saddles of the MSE loss, specifically saddles of rank zero, become strict in the equilibrated energy of any DLN (Theorems 2 & 3). These saddles include the origin, whose degeneracy (i.e. flatness) in the loss grows with the number of hidden layers. Our theoretical results are strongly validated by experiments on both linear and non-linear networks, and additional experiments suggest that other (higher-rank) non-strict saddles of the loss are strict in the equilibrated energy. Based on these results, we conjecture that all the saddles of the equilibrated energy are strict. Overall, this work suggests that PC inference makes the loss landscape more benign and robust to vanishing gradients, while also highlighting the fundamental challenge of speeding up PC inference on deeper networks.
The rest of the paper is structured follows. After introducing the setup ($2), present our theoretical results for DLNs ($3), including some illustrative examples and thorough empirical verifications of each result. We then report experiments non-linear networks supporting our theory and more general conjecture ($4). We conclude by discussing the implications and limitations of our work, as well as potential future directions ($5). Appendix A includes review of related work, derivations, experiment details and supplementary results. Code to reproduce all the experiments is available at https: com com/francesco- innoccnniiiiiiieiieiieiiieeniiii /pc- saddles.
1.1 Summary of contributions
We derive exact solution for the PC energy of DLNs at the inference equilibrium (Theorem 1), which turns out to be a rescaled MSE loss with a weight-dependent rescaling. This corrects previous mistake in the literature that the MSE loss is equal to the output energy [34] (which holds only at the feedforward pass) and enables further studies of the PC energy landscape. We find an excellent match between our theory and experiment (Figure 1).
Based this result, we prove that, in contrast to the MSE, the origin of the equilibrated energy of DI.Ns is a strict saddle independent of network depth. We provide an explicit characterisation of the Hessian the origin of the equilibrated energy (Theorem 2), which is perfectly validated by experiments on linear networks (Figures 3, 4 & 8).
We further prove that other non-strict saddles of the MSE than the origin, specifically saddles of rank zero, become strict in the equilibrated energy of DLNs (Theorem 3). We provide an empirical verification of one of these saddles as an example (Figures 9 & 10).
We empirically show that our linear theory holds for lonelinear networks, including con- volutional architectures, trained on standard image classification tasks. In particular, when initialised close to non-strict saddles of the MSE covered by Theorem 3, we find that SGD on the equilibrated energy escapes much faster than on the loss given the same learning rate (Figures 5 & 12). In contrast to BP, PC exhibits vanishing gradients (Figure 11).

---

We perform additional experiments, again on both linear and non-linear networks, showing that PC quickly escapes other (higher-rank) non-strict saddles of the MSE that we do not address theoretically (Figure 6), supporting our conjecture that all the saddles of the equilibrated energy are strict.
2 Preliminaries
Notation. We use the following shorthand W kid W.... We for E E I,....,.. denoting the identity matrix, while O,
total product of weight matrices W L:1 W L L.... W1. I, is the denotes either the n-zero vector or the n x n null matrix, and n will be omitted when clear from context. || denotes the & norm, and @ is the Kronecker product between two matrices. We will consider the gradient and Hessian of an objective f only with respect to the network weights 0 and sometimes abbreviate them as g/ Vof and H Vgl, respectively, omitting the independent variable for simplicity. The largest and smallest eigenvalues of the Hessian are Amax (H3) and Aii (H (), with max and min their associated eigenvectors. See $A.1 for
general notation.
Definition 1. Strict saddle. Following [15] and later work, any critical point O of f()) where g)( = 0 is defined as a strict saddle when the Hessian at that point has at least one negative eigenvalue, Amin (H,(
Any other critical point with a positive semi-definite Hessian and at least one negative eigenvalue in a higher-order derivative is said to be a non-strict saddle.
2.1 Deep Linear Networks (DLNs)
We consider DLNs with one or more hidden layers = L 1 defining the linear mapping L:1 WL. : _d, + Rd where We I ( with layer widths {me) and input and output dimensions Nu = dz, nu = dy. We ignore biases for simplicity. The standard MSE loss for DLNs then be written
N
C
Σ 1=1
ly. - WL.1x:1??
2N
for a dataset of N examples yz) where x E Rd, y E RRd, The total number of weights
is given by D/ Mefl-11, and we will denote the set of all network parameters as O = veo W....., WL) RP. For brevity, we will often refer to the MSE loss as simply the loss.
2.2 Predictive coding (PC)
DNNs trained with PC typically assume a hierarchical Gaussian model with identity covariances, so we will adopt this formulation for linear fully connected layers 2, (WW where the mean activity of each layer zę is linear function of the previous layer. Under some other common assumptions about the generative model, we can derive an energy function, often referred to as the variational free energy, which a sum of squared prediction errors across layers [9]:
F
[zz Wi2
2N
Note that this objective defines an energy for every neuron, highlighting the locality of the algorithm. To train a PCN, the last layer clamped to some data, ZL = y,. which could be a label for classification input, Zo,; = x,, The energy (Eq. 2) is then minimised in two phases, first w.r.t the activities (inference) and then w.r.t. the weights (learning):
image for generation. In supervised task, the first layer is also fixed to some
OF
OF
Inference: Az
33)
Learning: AW
(4)
Zng
OW;
where we omit the data index i for simplicity. In practice, the inference dynamics (Eq. 3) are often run to convergence until Az, ~ 0, before performing weight (e.g. GD) update (Eq. 4). Importantly, the effective weight landscape on which PC learns therefore energy the inference equilibrium Fi o (0), which will refer to the equilibrated energy sometimes simply the energy.
3

---

=
H
10
MNIST
theory
experiment
Energy
0.5
theory
Energy
theory
Training Iteration
Training
Training
CIFAR-10 Fashion-M
theory
theory
theory
Energy
0.5
Energy
50
100
100
50
Training Iteration
Training Iteration
Training iteration
thoory experiment
theory experiment
theory
Energy
Energy
Energy
0.5
50
100
50
Training Iteration
Training
Training
Figure I: Empirical verification of the theoretical equilibrated energy of deep linear networks (Theorem 1). For different datasets, plot the energy (Eq. 2) the numerical inference equilibrium F Flaz/Ozzso for DLNs with different number of hidden layers H E {2, 5, 10} (see $A.4 for more details), observing excellent match with the theoretical prediction (Eq. 5).
Theoretical results
3.1 Equilibrated energy as rescaled MSE
As explained in $2.2, the weights of PCN are typically updated once the activities have converged to an equilibrium. The equilibrated energy Flaf/az=g(0 which we will abbreviate as (*),,, is therefore the effective learning landscape navigated by PC and the object we interested in studying. turns out that we can derive closed-fome solution for the equilibrated energy DLNs, which will be the basis of our subsequent results.
Theorem (Equilibrated energy of DLNs). For any DLN parameterised by = (W1,-. WL) with input and output (x), the PC energy (Eq. 2) the exact infer- ence equilibrium OF 0F /0z = 0 is the following rescaled MSE loss (see $A.3.2 for derivation)
F
y: WLX,, WLAX,)
(5)
2N
where the rescaling S = Id, + EL (WL.)
The proof relies on unfolding the hierarchical Gaussian model assumed by PC to work out the full, implicit generative model of the output, and the rescaling S comes from the variance modelled by PC at each layer (see $A.3.2 for details). Figure shows an excellent empirical validation of the theory.
Intuitively, the PC inference process (Eq. 3) then be thought of reshaping the (MSE) loss landscape to take some layer-wise, weight-dependent variance into account. This immediately raises the question: how does the equilibrated energy landscape (0) differ from the loss landscape L())? the rescaling- and so the layer variance modelled by PC PC-useful for learning? Below we provide partial positive answer to this question by comparing the saddle point geometry of the two objectives.
3.2 Analysis of the origin saddle (0 0)
Here prove that, in contrast to the MSE loss, the origin of the equilibrated energy (Eq. 5, where all the weights are zero, 0 0) is a striet saddle (Def. 1) for DI.Ns of any depth. To do so, we derive an explicit expression for the Hessian at the origin of the equilibrated energy. For intuitive

---

M22
Vmax
M12
Vmax
Fiin
M12
-2
RP
n.5
nP
RP
Lirain
izain
Lirain
15
50
100
10 10 10 10*
Training lteration
Training Iteration
Training Beration
Figure 2: Toy examples illustrating the (Theorem 2) result that the saddle at the origin of the equilibrated energy is strict independent of network depth. We plot the MSE loss L()) (lop) and equilibrated energy landscape (0) (middle) around the origin for 3 linear networks trained with SGD on toy problem (see $A.4 for details). We also show the training losses for representative run with initialisation close to the origin (bottom). For the one-dimensional networks, we visualise the landscape around the origin as well as the SGD updates. For the wide network, we project the landscape onto the maximum and minimum eigenvectors of the Hessian, following [7]. Note that in this case the projection of the loss is flat because the Hessian the origin zero for H (Eq. 6).
comparison, we first briefly recall the known results that, the origin, the loss Hessian is indefinite for one-hidden-layer networks and zero for any deeper network (see $A.3.1 for re-derivation)
& I
==1
In. & Eyx
0
H = 0) =
(6)
Op:
H>1
where following previous works Exy 1 Σ x,y?" is the empirical input-output covariance. One-hidden-layer networks H = 1 are a special case where the origin saddle of the loss is strict (Def. 1) and was studied in detail by [48] (see left panel Figure for an example). For deeper networks H > 1, the saddle is non-strict as first shown by [23]:
N
Amin (H 0)) < 0, H =1 [strict saddle]
Amia HH 0)) 0, H>1 [non-strict saddle]
More specifically, the origin saddle of the loss is order H! becoming increasingly degenerate (flat) and harder escape with depth, especially for first-order methods like SGD (see middle and right panels of Figure 2).
By contrast, we show that the origin saddle of the equilibrated energy is strict for DLNs of any number of hidden layers. Figure shows a few toy examples illustrating the result. In brief, we
'The otder of a saddle simply indicates the (rath11) 1) derivative where the first negative (escape) direction is found. So, for example, first-order (strict) saddle has a zero gradient and an indefinite Hessian, while second-order (non-strict) saddle has zero Hessian but third derivative with negative direction.

---

H (0)
(0)
Hal0 ()
iffrrnce teraiin Sp.
Informne Iteation
Hffence on
Infornce Berotion w 50
H. 0)
H.-0
Daessy Eng
Danee FoR
ewip (4003
seep shoog
evegy cheort
enaoy mhaent
0.5
Figure 3: Empirical verification of the Hessian at the origin of the equilibrated energy for DLNs tested on toy data. We show the Hessian and its eigenspectrum at the origin of the MSE loss (top) and equilibrated energy (middle) for DLNs with Gaussian target y -x where (11, 0.1) (see $A.4 for details). Note that purple bars show overlapping loss and energy Hessian eigendensity. In the right panel, we vary the origin saddle in the equilibrated energy and observe an excellent numerical validation of our theoretical Hessian (Eq. 8). Figure 8 shows the same results for one-dimensional networks, and Figure 4 shows similar results for more realistic datasets.
of the output dimensions be 1/2 22- We confirm the strictness of
observe that, when initialised close to the origin saddle, SGD takes increasingly more time to escape from the loss than the energy function of depth (for the same learning rate). Now we state the result more formally. The Hessian at the origin of the equilibrated energy turns out to be (see $A.3.3 for derivation)
0
Exy @ In
==1
Inn
Σ Eyx
Eyy & Inn-1
H = 00
(8)
H>1
Eyy & Inc-
where Σyy = + Σ; y:y!' is the empirical output covariance. We see that, in contrast to the loss Hessian (Eq. 6). the energy Hessian has zon-zero last diagonal block given by 82 /0// for any number hidden layers H. It then straightforward to show that the energy Hessian has always negative cigenvalues, since the output covariance is positive definite.
Theorem 2 (Strictness of origin saddle of the equilibrated energy). The Hessian at the origin of the equilibrated energy (Eq. 5) for any DLN has at least one negative eigenvalue (see SA.3.3 for proof)
Amin HF. (0 = 0)) < O, IH 2
[strict saddle, Def. 1].
(9)
Figures 3 & 4 show a perfect match between the theoretical (Eq. 8) and numerical Hessian at the origin of the equilibrated energy, which we computed for a range of DL.Ns on a random batch of toy well more realistic datasets.

---

H
1
H
H
=
MNIST-II MNIST
Densty (lg)
ereig frumor))
starge gumeno)
Demsity (leg)
energ (uarenc)
10
ereig [haog)
Density
10"
starge (hoag)
10
energ (heer))
10
10
Hession eigenvalue
Hessian digenvalue
Hessian cigervalue
Ioxt
Ioot
Densty (log)
erergy (turess)
eneryy (uuesn))
Denskyy (log)
enery (uureec) energy (heer))
erergy (heog)
(heary)
Density
10
10
-0.15
-0.05
eigenvalue
eigenvalue
eigervalue
Figure 4: Empirical verification of the Hessian eigenspectrum at the origin of the equilibrated energy for DLNs tested on more realistic datasets. This shows similar results to Figure for the more realistic datasets MNIST and MNIST-1D [16] (see $A.4 for details). We again find perfect match between theory and experiment for DLNS with different number of hidden layers H 2,,2,4). confirming the strictness of the origin saddle of the equilibrated energy.
Theorem proves that the origin is strict saddle of the equilibrated energy for DLNs of any depth. This is in stark contrast to the MSE loss where is only true for one- one-hidden-layer networks (Eq. 7). The result predicts that, near the origin, (SJGD should escape the saddle faster equilibrated energy than on the loss given the same learning rate, and increasingly so as a function of depth. Figure 2 confirms this prediction for some toy linear networks, and Figures & 6 in $4 clearly show that holds for non-linear networks as well.
1 the
3.3 Analysis of other saddles
the origin a special case where the equilibrated energy has an easier-lo-escape saddle than the loss? Or is this result pointing to something more general? Here we consider a specific type of non-strict saddle of the loss (of which the origin is one) and show that indeed they also become strict in the equilibrated energy. We address other saddle types experimentally in $4 and leave their theoretical study for future work.
Specifically, we consider saddles of rank zero, which for the MSE can be identified as critical points where the product of weight matrices is W L:1 = 0 [1]. For the equilibrated energy (Eq. 5), consider the critical points 0" (WL = O, W L-11:1 = 0), since the last weight matrix needs to be null in order for the energy gradient be zero (see $A.3.3 for explanation). It turns out that these critical points there exists a direction of negative curvature.
Theorem 3 (Strictness of zero-rank saddles of the equilibrated energy). Consider the set of critical points of the equilibrated energy (Eq. 5) 0" (WW,. = o, WT.11 = 0) where g.- (0") O. The Hessian at these points has at least negative eigenvalue (see $A.3.6 for proof)
3X(H==(00) < [strict saddles, Def. 1].
(10)
Note that Theorem 2 can now be seen corollary of Theorem although for the origin we derived the full Hessian. This result also stands in contrast to the (MSE) loss, where many of the considered critical points (specifically when or more weight matrices are zero) are non-strict saddles as proved by [1]. The prediction is again that, in the vicinity of any of these saddles, PC should escape faster than with (S)GD given the same learning rate. For space reasons, the subsequent experiments focus only the origin
example of saddle covered by Theorem 3 (and Theorem 2), but $A.5 includes an empirical validation of another (zero-rank) strict saddle of the equilibrated energy (Figures 9, 10 & 12). Our code also makes it relatively casy to test for other saddles.
7

---

4 Experiments
Here we report experiments on linear and non-linear networks supporting our theoretical results as well as more general conjecture that all the saddles of the equilibrated energy are strict. In all the experiments, we trained networks with BP and PC using (S)GD with the same learning rate, since the goal is to test our theory of the saddle geometry of the equilibrated energy landscape. Code reproduce all the results is available at https:/ /ttt::://githaie /gggttthbbb.... cow pc- saddlele
First, we compared the training loss (MSE) dynamics of linear and non-linear networks, including convolutional architectures, on standard image classification tasks with SGD initialised close the origin (see $A.4 for details). For computational reasons, we did not run the BP-trained networks to convergence, underscoring the point that the origin saddle of the loss is highly degenerate and particularly hard to escape for first-order methods like SGD. In all cases, we observe that PC escapes the origin saddle substantially faster than BP (Figure 5), and Figure 11 shows that PC exhibits no vanishing gradients. We find practically the same results when initialising close to another non-strict saddle of the loss covered by Theorem 3 (Figure 12). These findings support our theoretical results beyond the linear case.
Linear
Tanh
ReLU
CIFAR-10 Fashion-M MNIST
0.1
BP
0.1
BP
0.1
BP
train
0.00
train
train
0.00
0.05
0.04
2343
4086
6081
12182
6091
12182
Training iteration
Training iteration
Training tteration
0.1
BP
0.1
BP
0.1
BP
train
0.08
train
0.00
train
0.06
0.02
0.02
3280
8500
3748
7497
6091
12182
Training iteration
Training iteration
Training heeraiion
0.1
BP
0.1
0.1
BP
Ltaiin
0.08
train
0.00
train
0.c9
0.00
0.08
0.07
2734
5468
2734
5468
3515
7050
Training iteration
Training iteration
Training iteration
Figure 5: PC escapes the origin saddle much faster than BP with SGD on non-linear networks. We plot the training loss (MSE) for a representative run of BP and PC for linear and non-linear networks trained on standard image classification tasks (see $A.4 for details). All networks were initialised close to the origin with scale a = 5e -), and trained with SGD and learning rate = le -3 The networks trained on MNIST and Fashion-MNIST had 5 fully connected layers, while those trained on CIFAR- 10 had a convolutional architecture. See Figure 1 for the corresponding weight gradient norms during training. Results were consistent across different random seeds.
From Figure 5, we also observe second plateau in the loss dynamics of PCNs, suggesting saddle of higher rank (presumably rank 1). This is consistent with the saddle-to-saddle dynamics described for DLNs by [19], where for small initialisation GD transitions through sequence of saddles, each representing a solution of increasing rank.
To explicitly test for higher-rank, non-strict saddles of the loss that we did not study theoretically, we replicated one of the experiments by [19, ef. Figure 1] on a matrix completion task. In particular, networks were trained to fit a rank-3 matrix, which meant that starting near origin GD visited 3 saddles (of successive rank 0, and 2) before converging to a rank-3 solution as shown in Figure We find that, when initialised near any of the saddles visited by BP, PC escapes quickly and does not show vanishing gradients (Figure 6), supporting the conjecture that all the saddles of the equilibrated energy are strict.
8

---

Linear
Tanh
ReLU
Train loss (log)
BP
Train loss (log)
Train loss (log)
1
1
PC
10"
10"
10
10
10
46709
93418
12907
29975
31584
63180
Training iteration
Training iteration
Training iteration
BP
BP
8P
|00 2
8
180
50
|00
50
46709 B3418
12887 25975
31584
631.8
Training iteration
Training iteration
Training iteration
Figure 6: PC quickly escapes higher-rank saddles visited by BP with GD on a matrix completion task. We plot the training loss (top) and corresponding weight gradient norms of the loss (BP) and cquilibrated energy (PC) (bottom) for networks (H 3, ne = 100) trained with full-batch GD to fit a random rank-3 matrix as studied by [19]. Br-ttni networks were initialised near the origin with scale a = 5 while PCNs were initialised at each saddle visited by BP (see $A.4 for details). Results were consistent across different random seeds.
5 Discussion
In summary, we took first step in characterising the effective landscape on which PC learns-the energy landscape at the inference equilibrium. For DLNs, we first showed that the equilibrated energy is equal to a rescaled MSE loss with weight-dependent rescaling (Theorem 1). This result corrects a previous mistake in the literature that the MSE loss is equal to the output energy [34] and that the total energy (Eq. 2) can therefore be decomposed into the loss and the other (hidden) energies (a relationship that only holds at the feedforward activity values). As we expand on below, Eq. also enables further studies of the PC learning landscape.
We then proved that many non-strict saddle points of the MSE loss, specifically zero-rank saddles, become strict in the equilibrated energy of any DLN (Theorems & 3). These saddles include the origin, making PC effectively more robust vanishing gradients (Figures 6 & 11). We thoroughly validated our theory with experiments on both linear and non-linear architectures, and provided empirical support for the strictness of higher-rank saddles of the equilibrated energy. Based on these results, we conjecture that all the saddles of the equilibrated energy are strict. Overall, the PC inference process can therefore be interpreted as making the loss landscape more benign.
5.1 Implications
Our work goes significantly beyond existing theories of PC in terms of both explanatory and predictive power. Most previous works make non-standard assumptions or loose approximations that result in non-specific experimental predictions. For example, the interpretation of PC as implicit GD by [3] holds only for small batch sizes and specific layerwise rescalings of the activities and parameter learning rates. ([2) generalised this result to remove the activity rescalings but not the learning rate ones.) By contrast, linearity the only major assumption made our theory, and we empirically verify that all the results hold for non-linear networks. Similarly, both [2] and [18] make second-order approximations of the energy to argue that PC makes use of Hessian information. However, our results clearly show that PC can leverage much higher-order information, turning highly degenerate, Il Horrder saddles into strict (first-order) ones.
Previous theories have also struggled to explain why faster learning convergence with PC is not always observed depending on the task, model and optimiser [3, 54]. Our landscape analysis, while incomplete (more on this below), acknowledges these factors and their interplay. helping to explain inconsistent findings and predict when speed-ups can and cannot be expected. All things being equal, PC should converge faster on deep and nurrow networks (though perhaps not too deep we discuss below), since the distance between the origin saddle and standard initialisations scales with the network width [39]. This likely explains the speed-up reported by [54] narrow (nz = 64)

---

15-layer fully connected network. However, in practice all things are not equal, and everything from not reaching an inference equilibrium to different datasets, architectures and optimisers all interact to determine convergence. This raises the question of whether minimising the equilibrated energy could be faster than the loss or lead to better performance, which we return to below.
More broadly, our landscape theory closely relates to the work of [56], who showed that learning in linear physical systems with equilibrium propagation [49, 50J has beneficial effects on the activity (rather than weight) Hessian. Studying these connections- and more generally the benefits of inference for learning in energy-based systems- could be an interesting future direction.
Our work has also implications for theories of credit assignment in the brain. In particular, our results put the recent principle of prospective configuration [54] for energy- learning on more solid theoretical footing, showing that PC inference can indeed facilitate learning by using high-order information. At the same time, they suggest that the claim of universally faster learning convergence with PC may have been overstated [54].
5.2 Limitations
We conclude by addressing the main limitations of our work. First, the strictness of the energy saddles we studied holds, by derivation, only at the exact inference equilibrium. We note that one does not need to reach equilibrium to improve the degeneracy of the loss saddles, and in this sense PC could be seen resource. However, in practice PC inference requires increasingly more iterations to converge on deeper networks, which aligns with our landscape theory since the loss saddles become more and more degenerate with depth. Our results therefore highlight the fundamental challenge of speeding up PC inference on deeper models if its benefits for learning are to be realised on large-scale tasks [40].
Even if this challenge is overcome, there seem to two interlinked questions that ultimately matter for the practical training of deep networks. First, are there conditions under which the equilibrated energy can be minimised faster than the loss in a more compute- or memory-efficien manner, with at least equal performance? Optimisation tools such Adam [24] and skip connections [17], for example, help to deal with the origin saddle at an increased memory cost. Could this trade off with the compute cost of PC inference? Characterising the inference cost of PC more formally would be useful step in this direction.
Second, could there be scenarios where PC is slower or less efficient but at the benefit of significantly better performance? This hard question to address since we are far from having theory of generalisation in deep learning [63, 20]. Given our origin saddle result (Theorem 2), however, it is interesting to note that on problems where low-rank bias is useful (e.g. matrix completion, Figure 6), GD with small initialisations can converge to sester-generalisingg solutions compared to standard initialisation [19].
Finally, understanding the overall convergence behaviour of PC would also require characterising other critical points of the equilibrated energy, especially its minima [14]. Our work, and Eq. 5 in particular, enables this. In $A.3.7, we present a preliminary investigation showing that, for linear chains, the global minima of the equilibrated energy are flatter than those of the MSE loss. This
result potentially explains the common observation that PC convergence tends to slow down towards the end of training, but we leave its full implications for future work.
Acknowledgements
F. I. is funded by the Sussex Neuroscience 4-year PhD Programme. E. M. A. acknowledges funding by the Deutsche Forschungsgemmenncchatt (DFG, German Research Foundation)- - Project number 442047500 through the Collaborative Research Center "Sparsity and Singular Structures" (SFB 1481). R. S. was supported by the Leverhulme Trust through the be. Doctoral Scholarship Programme in biomimetic embodied AL C. L. B. was partially supported by the European Innovation Council (EIC) Pathfinder Challenges, Project METATOOL with Grant Agreement (ID: 101070940).
10

---

References
[1] E. M. Achour, F. Malgouyres, and S. Gerchinovitz. The loss landscape of deep linear neural networks: second-order analysis. Journal of Machine Learning Research, 25(242):1-76, 2024.
[2] N. Alonso, J. Krichmar, and E. Neftci. Understanding and improving optimization in predictive coding networks, arXiv preprint arXiv:2305. 2023.
[3] N. Alonso, B. Millidge, J. Krichmar, and E. O. Neftci. A theoretical framework for inference learning. Advances in Neural Information Processing Systems, 35:37335-37748, 2022.
[4] A. Anandkumar and R. Ge. Efficient approaches for escaping higher order saddle points in non-convex optimization. In Conference on learning theory, pages 81-102. PMLR, 2016.
[5] P. Baldi and K. Hornik. Neural networks and principal component analysis: Learning from examples without local minima. Neural networks, 2(1):53-58, 1989.
[6] Y. Bengio, P. Simard, and P. Frasconi. Leamiing long-term dependencies with gradient descent difficult. IEEE transactions neural networks, 5(2):157-166, 1994.
[7] L. Böttcher and G. Wheeler. Visualizing high digensiimnnional loss landscapes with hessian direc- tions. Journal of Statistical Mechanics; Theory and Experiment, 2024(2)023401, 2024.
[8] A. J. Bray and D. S. Dean. Statistics of critical points of gaussian fields large-dimensional spaces. Physical review letters, 98(
2007.
[9] C. L. Buckley, C. S. Kim, S. McGregor, and A. K. Seth. The free energy principle for action and perception: A mathematical review. Journal of Mathematical Psychology, 81:55-79, 2017.
[10] B. Byiringiro. T. Salvatori, and T. Lukasiewicz. Robust graph representation learning via predictive coding. arXiv preprint arXiv:2212.0466 2022.
[11] Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. Advances in neural information processing systems, 27, 2014.
[12] S. S. Du, Jin, J. D. Lee, M. I. Jordan, A. Singh, and B. Poczos. Gradient descent can take exponential time to escape saddle points. Advances in neural information processing systems, 30, 2017.
[13] S. Frieder and T. Lukasiewicz. (non-) convergence results for predictive coding networks. In International Conference on Machine Learning, pages 6793-6810 10. PMLR, 2022.
[14] S. Frieder, L. Pinchetti, and T. Lukasiewicz. Bad minima of predictive coding energy functions. In The Second Tiny Papers Track at ICLR 2024, 2024.
[15] R. Ge, F. Huang, C. Jin, and Y. Yuan. Escaping from saddle points online stochastic gradient for tensor decomposition. In Conference on learning theory, pages 797-842. PMLR, 2015.
[16] S. Greydanus, Scaling down deep learning, arXiv preprint arXi::2011.14433 2020.
[17] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778, 2016.
[18] Innocenti, Singh, and Buckley, Understanding predictive coding second-order trust-region method. In ICML Workshop on Localized Learning (LLW), 2023.
[19] A. Jacot, F. Ged, B. Şimşek, C. Hongler, and F. Gabriel. Saddle-to-sadde dynamics in deep linear networks: Small initialization training, symmetry, and sparsity. arXiv preprint arXiv:2006. 15933, 2021.
[20] Y. Jiang, Neyshabur, H. Mobahi, D. Krishnan, and S. Bengio. Fantastic generalization 2019.
measures and where to find them. arXiv preprint
11

---

[21] C. Jin, R. Ge. P. Netrapalli, S. M. Kakade, and M. I. Jordan. How to escape saddle points efficiently. In International conference on machine learning, pages 1724-1732. PMLR, 2017.
[22] C. Jin, P. Netrapalli, R. Ge, S. M. Kakade, and M. Jordan. On nonconvex optimization for machine learning: Gradients, stochasticity, and saddle points. Journal of the ACM (JACM), 68(2):1-29, 2021.
[23] K. Kawaguchi. Deep learning without poor local minima. Advances in neural information processing systems, 29, 2016.
[24] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[25] T. Laurent and J. Brecht. Deep linear networks with arbitrary loss: All local minima are global. In International conference on machine learning, pages 2902- 2907. PMLR, 2018.
[26] J. D. Lee, I. Panageas, G. Piliouras, M. Simchowitz, M. I. Jordan, and B. Recht. First-order methods almost always avoid strict saddle points. Mathematical programming, 176:311-337, 2019.
[27] J. D. Lee, M. Simchowitz, M. I. Jordan, and B. Recht. Gradient descent only converges to minimizers. In Conference on learning theory, pages 1246-1257. PMLR, 2016.
[28] H. Li, Z. Xu, G. Taylor, C. Studer, and T. Goldstein. Visualizing the loss landscape of neural nets. Advances in neural information processing systems, 31, 2018.
[29] H. Lu and K. Kawaguchi. Depth creates no bad local minima. arXiv preprint 772.00100..08580 2017.
[30] A. Meulemans, F. Carzaniga, J. Suykens, J. Sacramento, and B. F. Grewe. A theoretical frame- work for target propagation. Advances Neural Information Processing Systems, 33:20024 20036, 2020.
[31] B. Millidge, T. Salvatori, Y. Song, R. Bogacz, and T. Lukasiewicz. Predictive coding: towards a future of deep learning beyond backpropagation? arXiv preprint arXiv:2202.09467, 2022.
[32] B. Millidge, A. Seth, and C. L.. Buckley. Predictive coding: a theoretical and experimental review. arXiv preprint #rXiv:207.11979, 2021.
[33] B. Millidge, Y. Song, T. Salvatori, T. Lukasiewicz, and R. Bogacz. Backpropagation at the infinitesimal inference limit of energy-based models: Unifying predictive coding, equilibrium propagation, and contrastive hebbian learning. arXiv preprint arXiv:2206. 02629, 2022.
[34] B. Millidge, Y. Song, T. Salvatori, T. Lukasiewicz, and R. Bogacz. A theoretical framework for inference and learning in predictive coding networks. arXiv preprint arXiv:2207.12316. 2022.
[35] B. Millidge, M. Tang, M. Osanlouy, N. S. Harper, and R. Bogacz. Predictive coding networks for temporal prediction. PLOS Computational Biology, 20(4)=e001118) 2024.
[36] B. Millidge, A. Tschantz, and C. L. Buckley. Predictive coding approximates backprop along arbitrary computation graphs. Neural Computation, 34(6):1329-136) 2022.
[37] M. Nouiched and M. Razaviyayn. Learning deep models: Critical points and local openness. INFORMS Journal on Optimization, 4(2):148-173, 2022.
[38] R. C. O'Reilly. Biologically plausible error-driven learning using local activation differences: The generalized recirculation algorithm. Neural computation, 8(5):895 90095-999,,,1 1996.
[39] A. Orvieto, J. Kohler, D. Pavllo, T. Hofmann, and A. Lucchi. Vanishing curvature in randomly initialized deep relu networks. In International Conference on Artificial Intelligence and Statistics, pages 7942-7975. PMLR, 2022.
[40] L. Pinchetti, C. Qi O. Lokshyn, G. Olivers, C. Emde, M. Tang, A. M'Charrak, S. FFiederr B. Menzat, R. Bogacz, et Benchmarking predictive coding networks-mad simple. arXiv 2024.
preprint
12

---

[41] L. Pinchetti, T. Salvatori, Y. Yordanov, B. Millidge, Y. Song, and T. Lukasiewicz. Predictive coding beyond gaussian distributions. arXiv preprint arXiv:2211.0348,, 2022.
[42] R. Rosenbaum, On the relationship between predictive coding and backpropagation. Plos one, 17(3)ee2261022, 2022.
[43] Salvatori, A. Mali, C. L. Buckley, T. Lukasiewicz, R. P. Rao, K. Friston, and A. Ororbia. Brain-
inspired computational intelligence via predictive coding, arXiv preprint arXiv:2308.07870, 2023.
[44] Salvatori, L.. Pinchetti, M'Charrak, B. Millidge, and Lukasiewicz. Causal inference via predictive coding, arXiv preprint arXiv:2306. 15479, 2023.
[45] T. Salvatori, I.. Pinchetti, B. Millidge, Y. Song, T. Bao, R. Bogacz, and T. Lukasiewicz. Learning on arbitrary graph topologies via predictive coding. Advances in neural information processing systems,
2022.
[46] Salvatori, Y. Song. T. Lukasiewicz, R. Bogacz, and Z. Xu. Predictive coding can do exact back- propagation on convolutional and recurrent neural networks. arXiv preprint arXiv:2103.03725, 2021.
[47] A. R. Sankar and V. N. Balasubramanian. Are saddles good enough for neural networks. In Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, pages 37-45, 2018.
[48] A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6122 2013.
[49] B. Scellier and Y. Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
[50] B. Scellier, M. Ernoult, J. Kendall, and S. Kumar. Energy-based learning algorithms for analog computing: a comparative study. Advances in Neural Information Processing Systems, 36, 2024.
[51] O. Shamir. Exponential convergence time of gradient descent for one-dimensional deep linear neural networks. In Conference on Learning Theory, pages 2691-2713, PMLR, 2019.
[52] S. P. Singh, G. Bachmann, and T. Hofmann. Analytic insights into structure and rank of neural network hessian maps. Advances in Neural Information Processing Systems, 14:23914-2399) 2021.
[53] Y. Song. T. Lukasiewicz, Z. Xu, and R. Bogacz. Can the brain do backpropagatton? im- plementation of backpropagation in predictive coding networks. Advances in neural information processing systems, 2257799. 2020.
[54] Y. Song, B. Millidge, T. Salvatori, T. Lukasiewiez. Z. Xu, and R. Bogacz. Inferring neural activity before plasticity: A foundation for learning beyond backpropagation. bioRxiv, pages 1022-05, 2022.
[55] M. Staib, S. Reddi, S. Kale, S. Kumar, and S. Sra. Escaping saddle points with adaptive gradient methods. In International Conference on Machine Learning, pages 5956-5965. PMLR, 2019.
[56] M. Stern, A. J. Liu, and V. Balasubramanian. Physical effects of learning. Physical Review E, 109(2):024311, 2024.
[57] R. Sun. Optimization for deep learning: theory and algorithms. arXiv preprint 1rXiv:1912.08957, 2019.
[58] R. Sun, D. Li. S. Liang. T. Ding, and R. Srikant. The global landscape of neural networks: An overview. IEEE Signal Processing Magazine, 3755):95-108, 2020.
[59] A. Tschantz, B. Millidge, A. K. Seth, and C. L.. Buckley. Hybrid predictive coding: Inferring, fast and slow. arXiv preprint arXiv:2204.02166 2022.
13

---

[60] J. C. Whittington and R. Bogacz. An approximation of the error backpropagation algorithm a predictive coding network with local hebbian synaptic plasticity. Neural computation. 1229-1222-1 2017.
[61] C. Yun, S. Sra, and A. Jadbabaie. Global optimality conditions for deep neural networks. arXiv preprint 1rXiv:1777.02444 2017.
[62] U. Zahid, Q. Guo, and Z. Fountas. Predictive coding as a neuromorphic alternative to backprop- agation: A critical evaluation. Neural Computation, 35(12):1881-190) 2023.
[63] C. Zhang, S. Bengio, M. Hardt, Recht, and O. Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107-115, 2021.
[64] Y. Zhou and Y. Liang. Critical points of linear neural networks: Analytical forms and landscape properties. International conference on learning representations, 2018.
[65] Z. Zhu, D. Soudry, Y. C. Eldar, and M. B. Wakin, The global optimization geometry of shallow linear neural networks. Journal of Mathematical Imaging and Vision, 62(3):279-25 29799992222222 2020.
[66] L. Ziyin, B. Li, and X. Meng. Exact solutions of deep linear network. Advances in Neural Information Processing Systems, 35:24446 -24458, 2022.
14

---

A
Appendix
Contents
A.1 General notation and definitions
15
A.2 Related work
15
A.2.1 Theories of predictive coding
15
A.2.2 Saddle points and neural networks
16
A.3
Proofs and derivations
16
A.3.1
Loss Hessian for DLNS
16
A.3.2
Equilibrated energy for DLNs
17
A.3.3
Hessian of the equilibrated energy for DLNs
18
A.3.4
Example: 1-hidden layer linear network
20
A.3.5
Hessian of the equilibrated energy for linear chains
A.3.6
Strictness of zero-rank saddles of the equilibrated energy
22
A.3.7
Flatter global minima of the equilibrated energy (linear chains)
23
A.4
Experimental details
23
A.5
Supplementary results...
25
A.1 General notation and definitions
Matrices, vectors and scalars are denoted with bold capitals A. bold lower-case characters and non-bold characters or U, respectively. All vectors are by default column vectors [-] E Rex and vcc,-(-) denotes the row-wise vec operator. Following [52], unless otherwise stated we define matrix-by mymaixx derivatives by row-vectorisation, using the numerator or Jacobian layout
aA òB
1 vec,, (A)]
(11)
1 vec, (B)");
such that the result is matrix rather than 4D tensor. Following from this, we will also use the rules
θABC
A CT
(12)
ØB
0AB
Im
B",
A DERMX Be R
(13)
0A
A.2
Related work
A.2.1 Theories of predictive coding
PC and BP. [60] where the first to show that PC can approximate BP on multi-layer perceptrons when influence of the input is upweighted relative to that the output. [36] generalised this result to arbitrary computational graphs including convolutional and recurrent neural networks under the so-called "fixed prediction assumption". variation of PC where weights are updated preciscly timed inference steps was later shown compute exactly the same gradients as BP multi-layer perceptrons [53], a result further generalised by [46] and [42]. [33] unified these and other approximation results from energy-based modelling perspective. [62] proved that the time complexity of all of these PC variants is lower- bounded by BP.
PC and other algorithms. [13] provided an in-depth dynamical systems analysis of the inference convergence for PC variants approximating BP. [34] showed that for linear networks the PC inference equilibrium can be interpreted as an average of BP's feedforward pass values and the local targets computed by target propagation [30]. [54] proposed that PC and other energy-basee algorithms implement fundamentally different principle of credit assignment called "prospective configuration",
15

---

in that neurons first change their activity to align with the target and then update their weights to consolidate that activity pattern. For mini-batches of size one, [3] proved that PC approximates implicit gradient descent under specific layer-wise rescalings of the activities and parameter learning rates. [2] further showed that when this approximation holds, PC can be sensitive to Hessian information. Similarly, recent work cast PC as second-order trust-region method [18].
A.2.2 Saddle points and neural networks
Here we review some relevant theoretical and empirical work (i) saddle points in the loss landscape of neural networks and (ii) the behaviour of different learning algorithms, especially (S)GD, near saddles. For more general reviews on the loss landscape and optimisation of neural networks, see [57] and [58].
Saddles in the neural loss landscape. This work began with [5] showing that for linear networks with one hidden layer, all critical points of the MSE loss are either global minima or striet saddle points (Def. 1). For the same model, [48] later showed saddle-to-saddlee learning transitions for small initialisation and characterised the GD dynamics under specific assumptions on the data. [11] highlighted the prevalence of saddles, relative to local minima, in the high-dimensional non-convex loss of neural networks. In particular, they empirically demonstrated a qualitative similarity between the landscape of networks and random Gaussian error functions, where the higher the error a critical point is associated with, the more exponentially likely it is to be a saddle [8].
[23] famously generalised the [5] result that all local minima are global to arbitrarily deep linear networks (DLNs) under some weak assumptions on the data. This was simplified as well as extended under less strict assumptions by [29]. Importantly, [23] was the first to show that for neural networks with one hidden layer = 1 all saddle points are strict (or first-order), while deeper networks have non-strict (H-order) saddles (for example at the origin where all the weights are zero). Several variations and extensions of this set of results have since been formulated [61, 64, 25, 65, 37, 66). For our purposes, one important extension was made by [1], who characterised all the critical points of the MSE loss for DLNs to second-ordes including strict and non-strict saddles.
Learning near saddles. This work can be traced to [15] who showed that SGD with added noise can converge in polynomial time on strict saddle functions. [27] proved similar result that GD without any noise asymptotically escapes strict saddles for almost all initialisations. This was later generalised to other first-order methods [26]. [21] proved that another noisy version of GD converges with high probability to second-order critical point in poly-logarithmic time depending on the dimension. For a review of these and other convergence results GD and its variants, see [22]. [4] showed (i) that a further GD variant can be proved to converge to third-order critical point and escape second-order saddles but at a high computational cost and (ii) that finding higher- order critical points is Na-hard.
[12] proved the important result that while standard GD with common initialisations will eventually escape strict saddles, it can take an exponential time do so. This is in contrast to the perturbed GD versions mentioned above, which converge in polynomial time. Similarly, [51] proved that for linear chains or dieeneinniioall networks with unit width, the convergence time of GD scales exponentially with the depth. [39] analysed similar models and showed that both the gradients and curvature vanish with network depth unless the width is appropriately scaled. [39] suggested that this in part explains the success of adaptive gradient optimisers like Adam [24] which can adapt to flat curvature. Similarly, [55] showed that adaptive methods can escape saddle points faster by rescaling the gradient noise near critical points to be isotropic.
[19] conjectured a saddle-to-sade -addee dynamic where GD visits a sequence of saddles of increasing rank before converging to a sparse global minimum. A few works have also shown that in practice SGD can converge to second-orda critical points that are non-strict saddles rather than minima [47, 7].
A.3 Proofs and derivations
A.3.1 Loss Hessian for DLNS
Here we derive the Hessian of the MSE loss (Eq. 1) with respect to the weights of arbitrary DLNs ($2. 1); this is essentially re-derivation of [52] with slightly different notation." 2 We then show how
In particular, unlike [52] make transposes of weight matrix products explicit.
16

---

the Hessian and its eigenspectrum at the origin (0 = 0) changes as function of the number of hidden layers H. We start from the gradient of the loss for given weight matrix
Oc OW. (Wit+1) (W LAI - v)
(14)
(Wee+1) ( (. Exx (We E RT
(15)
where following previous works we take the empirical mean over the data matrices Σxx = N Σi x,x;" and Σyx == * 2. y.x," For networks with at least one hidden layer, the ori- gin a critical point since the gradient zero go 0) To characterise this point second order, we look at the Hessian. Starting with the diagonal blocks of size (nene-1) x (nene-1).
0cC aw?
(WL.++) W. WI-1Ex (W (We1n1),
(16)
is straightforward to see that at the origin this term collapses to the null matrix for any 1.3 To compute the (n x (nene-1) dif-diagon blocks, we follow [52] and write the distinct contributions as follows
0 C
Vk + E, H
= (WLe+1) " WL & WI-1EEx (W
(17)
OW
0CC
Vk > e
Hc
(Wi-1s+1) & We-nn L.1 Exx - Σxx) W
(18)
OW ÖWe
UC
4k <l,
H = ow.ow, (W (Wr. ENx
@ (W (We-1&41
(19)
At the origin, these blocks always vanish except for networks with hidden layer, where as shown by [48] they are characterised by the empirical input-output covariance, e.g. for k < E, O C/// can now write the full loss Hessian the origin for different number of hidden layers:
= 0) = - Exy & In. H = 1. Putting the above facts together, we
Exy 3 In
H = [strict saddle]
In
yx
(20)
0
0
Op-
H > [non-strict saddle]
0
For me-hidden--laer networks, the Hessian is indefinite, with positive and negative eigenvalues given by the empirical input-output covariance, as described by [48]. For any DLN with more than one hidden layer, the Hessian zero, and the origin therefore a second-order critical point. In the general case, this point is non-strict saddle because some higher-order derivative of the loss depending on the network depth will contain at least one negative escape direction. More specifically, for network with L layers, all the L derivatives vanish, and negative directions will be found in the derivatives of order 2 L.
A.3.2 Equilibrated energy for DLNs
Here we derive an exact solution to the PC energy (Eq. 2) of DLNs at the inference equilibrium (Theorem 1, Eq. 5), FFaaF///---a- /0z=0: which we will abbreviate as F This turns out be a non-trivial rescaled MSE loss where the rescaling depends on covariances of the network weight matrices. To highlight the difference with the loss, recall that the standard MSE (Eq. 1) for DLN implicitly defines the following generative model
Y~ V~NWWLL.E (WLxI, E)
(21)
TTo he precise, this is true for any network with least one hidder layer H 2 1. For zero-hidden-laye networks (--whic are equivalent lincar regression problem- origin is not critical point, go 0) Eyx. and the Hessian is constant Hc La, Σxx.
17

---

where the target is modelled as Gaussian with mean given by the network function and some covariance Σ. In PC network, by contrast, the activity of each hidden layer-and not just the output-is modelled as Gaussian (see $2.2)
Zi (W
(22)
where Zo := x and Zz. := y. Now, to work out the generative model for the target implied by this hierarchical Gaussian model, we can simply "unfold" the model each layer. Specifically, we can reparameterise the activity of each hidden layer as a noisy function of the previous layer and so on recursively up to the first layer
Wizo 2g + L1
(23)
Z2 W221 + & W. xx WzE, +
(24)
23 W322 + E3 W2W2W1x WoW2f, + W + &3
(25)
where E (N(0 Ig) is white Gaussian noise. The last layer then be written
z = W WL + &
(26)
= W + E WWL.ee-- + &
(27)
8=2
We can now derive the implicit generative model for the target by taking the expectation and variance of Eq. 27:
V-N(WANR WWLIX (WLIX, I (WWr.)
I.
y~N
(28)
E=2
We therefore observe that, in contrast to the loss (Eq. 21), PC implicitly models the target with a non-identity covariance depending
chained covariance of the previous layers which in turns depends only on the weights. follows that, at the exact inference equilibrium where that implicit generative model holds, the PC energy is simply the following rescaled MSE loss
F
1
Σ
ly: WLIX)'SS (y. WL.X,))
(29)
2N
where the rescaling is S Li, + + L.E) and plugging the solution back into the energy.
(W LWe L.)T.. One can also arrive at this expression
by explicitly solving for the activities OF/0z although the calculation becomes much more involved. Note that generative model with non-ine non-identity covariances at each layer would lead different rescaling, but we do not consider this here to remain close possible to what is done in practice.
A.3.3 Hessian of the equilibrated energy for DLNs
Here we derive the Hessian the origin of the equilibrated energy for DLNs, following the calculation of the loss Hessian ($A.3.1). Section A.3.5 shows an equivalent derivation for one one-dimensional linear networks, which preserves the key the intuitions and easier to follow. We start from the equilibrated energy we derived previously for DLNs ($A.3.2, Eq. 29), which turned out to be the following rescaled MSE loss
F
Σ 2N
1
TS
(30)
where S Ia, + Ele
(W (W L.)) L. and denote the residual error for given data sample as := (y: Wt.1x;). In the general case, both the residual and the rescaling depend on W., so to take the gradient of the equilibrated energy we need the product rule. For simplicity, and similar to the characterisation of the off-diagonal blocks of the loss Hessian ($A.3.1), write the two
18

---

contributions separately, as follows
1
d S Òr;
A
(W (W (W. (W
(31)
2N
oW f
OWe
1 2N
N as
1 N
Øs
B
T; = N _ S S
r OW
(32)
OW:
where in Eq. 32 as aS/OW; 4D tensor, and
the rule Oa -x- The The first term A is simply rescaled loss gradient, while the second term B depends on the derivative of the rescaling. Note that for Wi the gradient collapses to the first term since the rescaling does not depend on it, OF /0W, = (Wr.2) S S/-SSS-WWW... Exx Σyx).
As an aside relevant to the zero-rank saddles analysed in $3.3, we note that, in contrast to the loss, WL L 0 is necessary (though not sufficient) condition for the energy gradient be zero. This is because the derivative of the rescaling dS /oW, needs to be zero in order for the gradient term B to vanish, and has one term linear in the last weight matrix.
As for the loss ($A.3.1), J, the origin is critical point of the energy since g (0 = 0) = 0. For B, this is because while the rescaling zero is the identity, the derivative of the rescaling vanishes since is linear with respect to any weight matrix:
S 1 (0 = 0) = IT,, as
(33)
ow, = 0) 0.
(34)
Calculating the Hessian involves multiple application of the product rule, for simplicity analyse the contribution the derivative of each term (Eqs. 31 & 32) at the origin. Because the first term simply a rescaling of the loss, and given Eq. 33, its second derivative at zero always zero with respect to the same weight matrix,
AA
==6
0 = 0)
#21.
(35)
OW
As for the loss, this term is also zero with respect to some other weight matrix + except for the case of a one-hidden-layer network
Im, & Σyxi k 1,H
AA ow:
k #,
e
0)
Exy Inp k k< =1.
(36)
0,
H>1 1
The second derivative of B requires 5-fold application of the product rule, involving the first derivative of the residual (and its transpose) and the first and second derivatives the rescaling. As shown above (Eq. 34), the first derivative of the rescaling the origin is zero, and the derivative of the residual with respect to any weight matrix at zero is always zero for any network with one or more hidden layers, fr/OWs(f non-zero for the special case of the last weight matrix with respect to itself:
0, H 2 1. The second derivative of the rescaling, however,
Ing-1
t=k=
S (0 0)
(37)
oW,
0
else
which means that at zero B takes the following form
Eyy 8 I
In
t= k
L
0 = 0)
(38)
oW
0,
else
19

---

where Eyy= = N Σn y.y is the empirical output covariance matrix. Drawing all these observations together, we can write the full Hessian at the origin of the equilibrated energy for different number of hidden layers:
0
- Exy @ In, Eyy @0 Ing 1
H == [strict saddle]
Ing 2 Eyx
Hy. 10-00-0) - 0)
(39)
0
H>1 [strict saddle]
Byy 2
We that, compared the loss Hessian (Eq. 20), the energy Hessian has non-zero last diagonal block given for any H. We note, but do not investigate in any depth, the potential connection with target propagation [30, 34). The me-hidden-laa layer case is fully derived in the next section ($A.3.4). It is straightforward to show that these matrices have negative eigenvalues
H M 1, Amin HH- ( 0)) < 0, Vy, + 0
(40)
since AA T is positive definite AA * O. The origin is therefore a strict saddle (Def. 1) of the equilibrated energy. This is in stark contrast to the MSE loss, which has a strict origin saddle only for coo-hidden-lans general case H > the negative curvature of the energy Hessian is given only by the output-output covariance Σyy. This means that, in the vicinity of the origin saddle, GD steps equal size on the equilibrated energy will escape the saddle faster (at rate depending the output structure) than on the loss, and increasingly so as a function of depth. In $4, we empirically verify this prediction experimentally linear well as non-linear architectures (including convolutional) trained on different datasets.
networks and non-strict saddle of order H for any deeper network. For the
A.3.4 Example: 1-hidden layer linear network
Here we show an example calculation comparing the Hessian at the origin of the loss and equilibrated energy for DLNs with single hidden layer Hl 1. For this case, the MSE loss and equilibrated energy are
L
Σ
y,, W2x1x/1
(41)
2N
N
F
1
Σ
(y:- (rk--2Wix)) (Io, + W2W,) 'y; W2
(42)
2N
where x E Rd,,y y E Rd, Wi E Ruxds W₂ E Rd, We now show the weight gradients, first of the loss
ac
OW W W.W.W, W
(43)
Oc W.W, E W - Eyx WT.
(44)
aW,
and then of the equilibrated energy
aF W/S- W Eyx
(45)
aw,
1
OF
-S- WS E.. Eyx) -S- -V---' W
(46)
aW,
where we denote the empirical mean of the residual as V 1 Σン r.f The origin is critical
point of the both the loss and the equilibrated energy since ge 10 = 0) = 85-(0- = 0) 0. We now
compute the Hessian blocks, expressing the off-diagonals at the origin for simplicity. again first for
20

---

the loss
W W2 & Σxx
(47)
OW?
20
Ia, & W Ex W
(48)
OW
0 L
00 = 0) I 0 Eyx:
(49)
ow
ow2dW1
and then for the energy
FF W W2 @ Σxx
(50)
dW7
FF
SS & W W xx W -S- In
(51)
Ow?
2
F
0 0) In & yx
(52)
aw WWoWW
At the origin, the Hessians become
0
Exy & In
Hc 0)
(53)
-In Eyx
0
Σ 3 In -Ey & In
Hy- (0=
(54)
-I, & Eyx
A.3.5 Hessian of the equilibrated energy for linear chains
Here include derivation the Hessian of the equilibrated energy (as well its eigenstructure at the origin) for linear chains or networks of unit width #L.1 where ng = =.. n2 1. This follows the derivation for the wide case (SA.3.3), but it reveals all the key insights and is easier to follow. For the scalar case, the implicit generative model of the target defined by PC (see $A.3.2) is
y N
WLAE.
1+
Σ (WL:)))) 2=2
(55)
leading to the following rescaled loss
F C/s, s= 1+ Σ (WL
(56)
where -= 1 Σ, (y: WL.12))2 The weight gradient of the equilibrated energy is
a Ow;
8 Twe
(57)
Oc
C
Os Sws
i >> 1
8 Ow:
where the loss gradient OC/Ow; - WL with residual error (y w1.1x). As shown in $A.3.2, The origin is critical point of both the loss and the equilibrated energy since their gradients are zero gf 0) = 0, gr. 0 0) O. We now show the Hessians, first of the loss
(WL,1z))22²
i=j
8CC
(58)
aw,Ow,
(WIILHH:) ry), i
21

---

and then of the energy
8² BwyDhmy
i=j=1
F Ow,Ow,
0 Sw;
0C 0s Ow,, Jwg
i= j>1.
1 a=c s Binghrn
ac as Bhe, Bwy
ac as 32 Jmy Bhs
3
0²x Bwgchu
3.
C
as Dur
i.j >
(59)
Generalising the one-hidden-unit case shown by [18], the origin the Hessians reduce to
0
ry
==1 1
[strict saddle]
ry
0
H 0)
(60)
:
Op 1
[non-strict saddle]
0
0
xy y²
H == [better-conditioned strict saddle]
xy
Hy. (0 =
=
(61)
H >>> 1 [strict saddle]
0
y2
For one-hidden-layer networks H = 1, the Hessian eigenvalues of the loss and energy are A(Ha(0
0)) = try, A(H,- ( 0)) = (-y² + yv /4x2 + y²)/22, respectively. In this case, the eigenvalues of the energy turn out to be smaller than those of the loss,
==1, (H = 0)) ((H2(0 = O)), y
(62)
following from the fact that the square root of sum is smaller than the sum of the square roots, Va² + 12 < va² + /b2 Va, b * O. This means that, in this particular case, the strict saddle of the equilibrated energy is better conditioned (i.e. easier to escape) than that of the loss. For deeper networks, the Hessian of the loss is zero, and it is easy to see that that of the energy has zero cigenvalues of multiplicity L - and a single negative eigenvalue given by the target squared
H > 1,
Amin (0- = 0)) = -
(63)
A.3.6 Strictness of zero-rank saddles of the equilibrated energy
Here we prove the strictness of the zero-rank saddles of the equilibrated energy (Theorem 3). It is easy check via Eqs. 31 & 32 that any point 0" such that (W, O, W 0) is critical point. Now let's prove that the Hessian at 0" has a negative eigenvalue. To do this, we rely on the Taylor expansion of the function around 0" Since gF. (0") = O, we have for any i and any + 0,
0 T H -
F - (0) = F (0") + 2
(64)
Hence by unicity of the Taylor expansion, if we can find θ such that F (0 - = cfa ((53) where e > 0, this would mean that 0" Hy- (0" 0 -2c < 0 and therefore that it is strict saddle point. By considering the direction of perturbation θ (I. o....... 0). we have
F* + 50) F* (SI, W WL-1.... W1)
(65)
N
L-1 1+
WL-1A WI-1:) )) yi.
Σx 1+
Σ
(66)
i=1
2=2
22

---

Denoting by A : I+ Ei=? W 1-14 W we have when 8 - 0
s-1 = (I+ A) - -
0(6)).
(67)
Hence
N Wi) = Σ (I *A+000))))
(MI, W
(68)
Σ yTy: 8 Σ
T y, Ay: + 0
(69)
=F (WL. WL-1.... Wi) - c8 - -
(70)
where c Et= of Ay, because A is symmetric definite positive and there exists j such that
Uj # 0. Hence
= F"" + +60 = c82 + 0669)
(71)
which concludes the proof.
A.3.7 Flatter global minima of the equilibrated energy (linear chains)
Here we present a preliminary investigation into the minima of the equilibrated energy compared to the MSE loss. For linear chains ($A.3.5), we show that global minima of the equilibrated energy flatter than those of the MSE loss. More precisely, the energy global minima turn out be scaled down versions of those of the loss by the same rescaling factor of the equilibrated energy (SA.3.2). This generalises the result of [18] for linear chains with a single hidden unit.
The proof has only that we are at a global minimum of loss when we perfectly the data W 1. = y. since L C( = y) 0. This is also true of the equilibrated energy, F (Wr,, Ø y) = O. We can check that these are critical points by seeing that the weight gradient of the loss is null, VoL = y) = O. which follows from the fact that the residual of the energy, Vo (WLa1 = y) =0 O.
steps and does not require explicit calculation of the Hessian. First, we know
when perfectly fit the data. Again, the same true
The second and last step is to realise that, at these minima, the terms of the energy Hessian (Eq. 59)
collapse to those of a rescaled loss Hessian (Eq. 58):
8 C Bw,dory
i=j=1
82
(WI.1 = -1)
82 8 Bu.
i= 1, j>1,
(72)
C "uudway"
i.j >
where the rescaling is the same that of the equilibrated energy (Eq. 56). Factoring the rescaling
Hy. (WLL y) He( = y)/s
(73)
H.. = y) He( = y),
(74)
we observe that the minima of the equilibrated energy are simply rescaled version of those of the loss. As we saw in $A.3.2, the rescaling is positive, so it follows that the global minima of the equilibrated energy are flatter or, to put it another way, PC inference has the effect of flattening the global minima of the MSE loss (at least for linear chains).
A.4 Experimental details
Code to reproduce all the experiments is available at https://github. com/ wwwriiiinncnttcceeeeeeeeeee
Unless otherwise stated, for all PC networks stan- dard Fuler integration with step size di 0.1 was used to run the inference dynamics equilibrium ($2.2, Eq. 3), with the number of iterations depending the problem.
23

---

Theoretical energy (Figure 1). We trained DLNs with different number of hidden layers H E (2, 5, 10} on standard image classification datasets (MNIST, Aushion-MNISS and CIFARIO). At every training step, we compared the total energy (Eq. 2) at the numerical inference equilibrium Flanso with the theoretical prediction (Eq. 5). The following hyperparameters were used for all networks: 300 hidden units and SGD with learning rate 1) = le -3 and batch size b = 64. We used a second-order explicit Runge- Kutta ODE solver (Heun) with a maximum upper integration limit T = 300 and an adaptive ienootiional-nteraal--eeee le ) to ensure convergence of the PC inference dynamics (Eq. 3). Results were consistent across different random initialisations.
controller (absolute and relative tolerances:
Toy examples (Figure 2). All networks were linear and trained on a toy regression problem using the MSE loss (Eq. 1) and energy (Eq. 2) with output y = -x,x ~ W(1,0.1), Weights were initialised close to the origin Waj ~ (O, o²) with a < 1. For the chains, the initialisation scale was chosen to be a = 5e -2, while for the wide network it was increased to a = le-1 1 in order to make escape from the saddle faster but still visible. For PC, T = 20 inference iterations were used for chains and 50 for the wide network. All networks were trained with SGD and batch size b = 64. Learning rate 17 = 0.4 was used for the chains and 1e-3 for the wide network. Training was stopped when it was determined that convergence had been effectively reached, to allow for intuitive visualisation of the loss dynamics.
The landscapes were sampled on the training loss or energy with 30 x 30 resolution and domain E [-2, for the 2-hidden node chain and E [-1, 1] for the other networks. For the wide network, the landscape was projected onto the maximum and minimum eigenvectors of the Hessian at the origin 0" 0. s(O" + V min + mmx) since as shown by [7] random directions [28] can fail to identify saddle points. The energy landscape was plotted at the numerical equilibrium FO)).. Figure 2 displays results for an example run, and Figure 8 shows the statistics of the training and test losses as well as gradient norms for random initialisations.
Hessian eigenspectra (Figure 3-4). For different linear network architectures, we computed the Hessian of the loss and equilibrated energy at the origin on a random batch (size b = 64) of a given dataset. The datasets used were (i) toy Gaussian with 3D input and output with the same statistics used for experiments in Figure 2, (ii) MNIST and (iii) MNIST-ID [16]. a procedurally generated, one-dimensional dataset smaller than MNIST with higher model discriminability. The depth, width and data dimensions of the networks tested on the Gaussian data are clear from the vignettes in Figure 3. Figure 9 shows the same results for linear chains. For MNIST and MNIST- networks with H hidden layers (1. 2, 3} had ny widths (10, 10,5) and (100, 50, 10). respectively. Note that the MNIST networks were relatively narrow allow for tractable computation of the Hessian. The Hessian matrices for the Gaussian data were normalised between and -1, and the Hessian of the energy was computed after T 50 inference iterations. For the theoretical eigenspectra of the energy Hessian, we computed the eigenvalues of Eq. 8. Figures 3 and 4 show results for an example run, and we found practically indistinguishable results for different seeds. Figures 9 & 10 show a similar analysis for a zero-rank saddle covered by Theorem 3 other than the origin.
Experiments (Figure 5-6). For the first set of experiment, we trained and tested linear, Tanh and ReLU networks on standard image classification tasks. Networks tested on MNIST and Fashion- MNIST had 5 fully connected (FC) layers with 500 hidden units, while those trained on CIFAR-10 had a convolutional architecture consisting of 3 blocks (with a convolution and max pooling operation) followed by two FC layers (with the last one always being linear). For PC, T = 50 inference iterations were used. Similar to the experiments for Figure 2, all networks were initialised close to the origin W.y ~ (0, o²) with o = 5e SGD with batch size 64 and learning rate 77 = Le -3 was used for all networks. PC networks were trained until the training loss reached tolerance threshold Lrain < 1e -3, For computational reasons, the BP-trained networks were not trained until convergence. Instead, training was stopped at as many iterations as it took PC to converge. We do report the full saddle escape dynamic for the toy examples in Figure 2 and the matrix completion experiment in Figure 6. All hyperparameters except for the initialisation remained unchanged for the other (zero-rank) saddle experiment shown in Figure 12.
For the matrix completion task (Figure 6), we attempted to replicate the experiment by [19, Figure 1] as closely as possible. Networks of depth H = and width n, = 100 were trained with GD and learning rate 17 = le -2 to fit 10x10 matrix of rank 3. The target matrix was generated by
24

---

multiplying two i..d. matrices of size 10x3 with standard Gaussian entries, and 20% of these entries were masked during training. The networks trained with PC were initialised each saddle visited by BP, which was determined numerically by computing the rank of the network map. The origin initialisation had the same scale a 5e used in the previous experiments.
A.5
Supplementary results
Ltaain
train
troin
PC
35
75
150
10 10 10 10*
Training iteration
Training iteration
Training iteration (log)
B6
0.5
BP
8P
Lest
Lest
test
Training Iteration
Training ieeration
Epoch
0.2
PC
50
PC
100
00
[88
15
75
150
10 10a 102 10"
Treining iteration
Training iteration
Training iteration (log)
Figure 7: Training and test statistics for linear networks of Figure 2. For each network, we plot the mean and 11 standard deviation of the training loss, test loss and gradient initialisations. For the wide network, the test loss is evaluated once every epoch (rather than for each batch), and the training metrics are plotted on a log axis for easier visualisation. For the chain with two hidden units, the multiple loss plateaus and corresponding gradient spikes are due to different escape times from the saddle for different runs.
5 random
25

---

H
H
H 10
12/0 0)
H 01
Hy-0 0))
H::0 0)
Ocnoky (og)
tog)
Deroty
owne (lonr)
Oursky
(hcor)
enricc Wffied
2074
Hozaaa
HISS
MessiGn
Figure 8: Empirical verification of the Hessian at the origin of the equilibrated energy for linear chains. This shows the same results of Figure 3 for networks of unit width nig = ... nz = (see $A.4 for details). Again, we observe perfect match between theory and experiment (see in particular Eq. 61).
Inference iteration 50
Inference iteration 50
Demsity (log)
energy (numeric)
Demsity (tog)
energy (numeri))
10"
-
-2
-8
Hessian eigenvalue
Hessian eigenvalue
Figure 9: Empirical verification of a strict zero-rank saddle of the equilibrated energy other than the origin for DLNs tested on a toy dataset. We show the Hessian eigenspectrum of the MSE loss and equilibrated energy at a strict saddle other than the origin covered by Theorem 3, specifically for critical point where all weight matrices except the penultimate are zero 0" (W, = O, Vi & L 1). We do not show the loss Hessians because they are zero for H (Fq. 6). The target is the same as used for Figure 3, and the right panel one of the output dimensions is varied to y/2 x2. Figure 10 shows results for the same critical point on MNIST and MNIST-ID.
26

---

H = 3
H =
MNIST-1D MNIST
l0ss
loss
Deneity (log)
energy (humenic)
Density (log)
energy (numeric)
1
1
10
10
Hessian eigenvalue
Hessian eigenvalue
Density (log)
energy (numeeic))
Demsity (log)
energy (numeeic)
10
10
10"3
10-2
-0.4
-0.3
-0.
-01
0
-03
-02
-01
Hessian eigenvalue
Hessian eigenvaaue
Figure 10: Empirical verification of strict zero-rank saddle of the equilibrated energy other than the origin for DLNs tested more realistic datasets. This shows similar results Figure for the more realistic datasets MNIST and MNIST-ID.
Linear
Tanh
ReLU
MNIST
BP
BP
|,00
180012
2342
4585
6090
12181
0090
12101
Training iteration
Training iteratioo
Training iteration
Fashion-M, M
BP
BP
BP
|000|
[800
1000
32/79
Bo08
3//8
/496
B0RD
12101
Training iteration
Training iteration
Training iteration
CIFAR-10
UP
UP
|00 l
10
|00
10
[80
10
27333
5467
2733
S467
3514
7028
Training iteration
Training iteration
Training iteration
Figure 11: No vanishing gradients for PC starting near the origin. Weight gradient norms ||00||2 of the loss (BP) and equilibrated energy (PC) for the experiments in Figure 5.
27

---

Lincar
Tanh
ReLU
MNIST
0.1
0.1
0.1
8P
train
0.00
train
train
0.00
0.00
0.04
1874
3748
5154
10308
4217
8434
Training lteration
Training iteration
Training iteration
CIFAR-10 Fashion-M
0.1
0.1
0,1
8P
Eraai
0.08
train
0.08
train
0.06
0.02
2343
4686
2811
5628
4217
8434
Training iteration
Training iteration
Training lteration
0.1
0.1
0.1
8P
Ltrain
0.08
train
0.00
train
0.00
0.08
2734
5468
2734
5468
2734
5468
Training iteration
Training ieeration
Training ieeraion
Figure 12: PC escapes another non-strict saddle of the loss much faster than BP with SGD on non-linear networks. This shows the same results as Figure for the same saddle analysed in Figures 9 & 10 (see $A.4 for details). We show results for example run they were practically indistinguishable across different random seeds.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We clearly state our claims in the abstract and introduction, based on both theoretical and empirical results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We dedicate a full section in the discussion ($5.2) to the main limitations of this work. We highlight the most important limitation in the abstract.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
29

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: As emphasise throughout the paper, linearity (of the activation function of deep neural networks) is the only major assumption made by our theoretical analysis, and we perform thorough experiments showing that the theory holds for non-linear networks. We provide proof' and derivations of all the theoretical results in the Appendix ($A.3) and give intuition for the proof's in the main text. We also include some pedagogical derivations ($ ($A.3,4, $A.3.5).
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all the details necessary reproduce all the experimental results in the Appendix (A.4).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
30

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: We provide access to the code that be used reproduce all the experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
wtt.uuubiisssooooiioooooooooooie for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
/ mhips.cc/publl. wwwpc//g////eeeeeee CoodeSubeeeeeeee wataammissiiooooiiiiiiioi
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We specify important specifications of the experiments the main text and all other details in the Appendix ($A.4).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: In most cases we do not report error bars because results were consistent or practically indistinguishable across runs and including them would not enhance (or even confuse) understanding, Figure captions always specify whether results shown for an
31

---

example run or seed. When error bars are included (Figure 7), we use #1 standard deviation, over different runs or random seeds. Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification: Most experimental results can be reproduced in a few hours on a CPU, with the exception of those related to Figures & 12 which were run GPU (typically A100).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Fthics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurd hww.rpsu....///e
Answer: [Yes]
Justification:
The research conforms to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
32

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work does not have any direct societal impact, positive negative.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not existing assets.
33

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
34

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
35


Reviews:
[{"review_id": "q5mbiF1PjQ", "reviewer": "Reviewer_hPaX", "summary": "This work looks at the (equilibrated) loss landscape of predictive coding networks, and analyzes the nature of saddle points therein --- especially in comparison to that obtained for backpropagation based deep linear networks with MSE loss. They find that several non-strict saddle points in the latter turn to strict saddle points (with positive and negative eigenvalues) in the former. This is argued to lead to better convergence properties, and is shown empirically for some simple networks.", "questions": "See the above section", "limitations": "^^", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Studying the loss landscapes for predictive coding is a very interesting direction, and could be useful to better understand the benefits imparted by it, and also circumvent any idiosyncratic problems that might arise. \n\n- An interesting theoretical analysis of the Hessian for predictive coding is carried out, by considering the equilibrated loss, which can also be useful for future theoretical works in this space. \n\n- Non-strict saddles at origin or zero-rank saddles are shown as strict saddles in predictive coding, which afford for better convergence when training starts near such a saddle as compared to that with usual deep nets where these are non-strict saddles. The experiments support the point and associated visualizations are quite interesting.", "weaknesses": "Some of the claims might a bit exaggerated: In the conclusion, the authors write that PC networks have only strict saddles. But, I don't think there is anywhere in the paper it is shown that there are only two kinds of saddles: zero and zero-rank, both of which are shown as strict saddles. No other saddles are analyzed, or shown to be non-existent, and the zero-rank saddle is not too dissimilar to the zero saddle. Likewise, there are statements saying that they prove non-strict saddles other than origin to be zero in PC network setting, but I think they are just talking about the zero-rank saddles. \n\nExperiments are somewhat restricted: The experiments are primarily with simple MLPs. It is also unclear when the backprop'ed networks are trained if their convergence results are well-tuned. In other words, if they use GD vs SGD, or if the learning rates or batch sizes are optimal, adaptive gradient methods, and the kind. (PC doesn't have to outperform them, but it will be good to gauge how well it fares in context of all these things that practitioners would normally employ)  \n\nSection 3.2 analysis is more or less a corollary of Singh et al, 2021: The authors only somewhat acknowledge this in the appendix, but I think this should be properly highlighted and attributed in the main text of the paper as well. \n\nLoss Landscape of PCNs could have been studied a bit better: The technical part is largely based on Achour et al and Singh et al, and some more aspects of the loss landscape could have been carried out. The paper has a good contribution, but I am unsure if this is sufficient compared to a normal NeurIPS paper of 9 pages. \n\nOthers:\n- Some of the terminology is loose and confusing: Often in their paper the talk about saddles of BP in reference to that of MSE. I think this is quite confusing, as what they mean by MSE is networks trained with Backprop on MSE. A better acronym would be BP-MSE or BP, as otherwise it is quite vague.\n- Line 49, Phenomenon of vanishing gradients has the wrong reference. The right one is Bengio 1994 https://www.comp.hkbu.edu.hk/~markus/teaching/comp7650/tnn-94-gradient.pdf, while the currently mentioned one is more about curvature.\n- Could the authors better describe the steps from eqn 70 to 71 in proof of proposition 1, and where all things are under approximation and where equality? It's a bit loosely written at the moment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. Below we address the points made by the reviewer one by one.\n\nWe tried to be very careful to not overstate our claims based on the results. In the abstract, introduction and conclusion, we state that “we provided theoretical and empirical evidence that the effective energy landscape of PC networks has only strict saddles”. Nevertheless, we agree with the reviewer that this could be misinterpreted to overstate our results, so we removed this sentence from the abstract and edited it more conservatively in the introduction and discussion. To avoid overstating our results, we also note that throughout the paper we used word *conjecture* to emphasise that we do not prove that all the saddles of the equilibrated energy are strict. For the same reason, we included a question mark in the title of the paper.\n\nThe reviewer is correct that when we say “other non-strict saddles than the origin” we are referring to the zero-rank saddles. It is true that the origin is one of these saddles as we point out. However, there is no a priori reason to believe that these zero-rank saddles (other than the origin) should also be strict in equilibrated energy. Nevertheless, we agree with the reviewer that we did not study other types (higher-rank) of saddles, theoretically or empirically. Therefore, as we explain in our global response, we now also include two additional sets of experiments (in the attached PDF), one looking at another zero-rank saddles than the origin and the other supporting the strictness of higher-rank saddles of the equilibrated energy.\n\nThe reviewer points out that our “experiments are primarily with simple MLPs”. This is true, although we note that we also performed experiments on convolutional networks too as noted in the caption of Figure 5. \n\nWe completely agree with the reviewer that the convergence results were not at all tuned. As we explain in our global rebuttal, our main goal was to characterise the intrinsic geometry of the equilibrated energy landscape, which had not been done before. Studying under which conditions, such different learning rates and optimisers, the energy landscape might be faster to optimise than the loss landscape is beyond the scope of our work. We refer to the global rebuttal for our argument and note that our work facilitates the study of such conditions.\n\nWe strongly disagree that Section 3.2 is a corollary of Singh et al. (2021). Singh et al. (2021) derived expressions for the Hessian blocks of arbitrary deep linear networks with MSE loss. They did not focus on characterising the landscape. We simply use the notation employed by Singh et al. (2021) to derive the Hessian of the equilibrated energy, which is non-trivial and significantly goes beyond this work. The reason why we include a re-derivation of the Hessian (with slightly different notation) in the appendix, and its structure at the origin in section 3.2 (which as we note was first derived by Kawaguchi, 2016, with a different Hessian derivation) is for intuitive comparison with that of the equilibrated energy.\n\nWe agree with the reviewer that we do not provide a full picture of the landscape, including all saddles and minima. However, we strongly disagree that “the technical part is largely based on Achour et al and Singh et al”. As explained above, we use mainly the notation of Singh et al. and re-derive the Hessian of the loss for comparison only. Achour et al. characterised all the critical points of the MSE for DLNs to second order, and we simply use their characterisation for comparison (i.e. to check what happens to these critical points in the equilibrated energy). Importantly, we would like to emphasise that showing whether the critical points of the MSE loss (characterised by Achour et al.) are also critical points of the energy and, if so, what kind of points, are additional questions that require a completely separate technical analysis of the gradient and Hessian of the equilibrated energy, which as our calculations show are non-trivial to derive. As we also review in the related work section, it is worth emphasising that the landscape of DLNs for the MSE loss took many papers before being understood to the degree that it is today. Our paper can be seen as a first step into understanding the landscape of the equilibrated energy. Indeed, our novel derivation of the equilibrated energy enables further studies into the landscape.\n\nWe agree with the confusions in terminology pointed out by the reviewer and made changes to the text accordingly.\n\nWe agree with the reviewer that the Bengio et al. (1994) paper is the first to introduce the vanishing gradient problem. We made reference to the other paper (Orvieto et al., 2022) because this was the first (to the best of our knowledge) to connect the phenomenon of vanishing gradients to vanishing curvature and therefore the non-strict saddle at the origin addressed in part by our work. We now refer to both papers to also acknowledge Bengio et al. (1994).\n\nWe agree with the reviewer that our original proof of Theorem 3 was loosely written. We now update the paper with a full proof including all steps and clearly separating the equalities from the approximations. We thank the reviewer for pointing out this imprecision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8tkyDNXoOO", "reviewer": "Reviewer_cEwT", "summary": "This paper explores how the energy landscape in deep linear networks trained with predictive coding compares to the loss landscape of DLNs trained with backpropogation. The authors prove that the energy of PC is a rescaled version of the MSE loss. The authors then show that, unlike BP trained DLNs, multiple points in the energy landscape of strict saddles. This suggests a way in which training with PC can escape poor initializations faster than BP. To support these theoretical results, the authors provide numerical results.", "questions": "1. Does PC still show an advantage over BP if a different learning rate is used for BP (so as to match the effective learning rate of PC)? \n\n2. Why must $\\textbf{W}_L =0$ for the saddles considered at the end of the paper?\n\n3. Why would having strict saddles be useful in cognitive science?", "limitations": "The authors did a nice job addressing their limitations.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper was well written and well motivated. It was clear what the problem the authors were tackling was, why it was important, and what the main results of the paper were. \n\n2. The combination of theoretical and numerical results that collectively contributed to a strong, compelling story was great.", "weaknesses": "Major C1: Given that the authors prove that, for DLNs, the equilibriated energy is a scaled version of the MSE, I believe an alternative explanation for why training with PC is able to achieve good performance faster than BP is because it has a different effective learning rate. Is that assessment correct? If so, did the authors explore whether changing the learning rate for BP led to changes in learning speed in the DLN and DNN numerical examples? \n\nMajor C2: While there is technical discussion of PC in Section 2, it was not clear how that maps to the idea of PC in neuroscience. More discussion, even in the Appendix, would be useful for those familiar with PC more from the neuroscience perspective. \n\nMinor C1: It was unclear to me why $\\textbf{W}_L$ had to equal 0 for the equilibrated energy (line 235). I read the subsection in the Appendix that is referenced and still felt like the comment was a little abrupt. \n\nMinor C2: Given that the authors have some remaining space, it would be interesting and good to hear more about the neuroscience perspective, other than learning speed.  \n\nVery minor C1: $\\textbf{g}_f$ is used before it is defined (line 154). \n\nVery minor C2: Why is there a 1/2 in the MSE loss? To the best of my knowledge, this is not \"standard\" as is quoted in line 160.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback. Below we address each point raised by the reviewer.\n\nGiven that the equilibrated energy turns out to be a rescaled version of the MSE loss, the reviewer asks if PC could be interpreted as having a different effective learning rate than BP. Though pointing in the right direction, we think that this is an insufficient explanation. First, the rescaling depends on the weights and so it will change at every weight update. In this sense, the gradient magnitude (or effective learning rate) of PC could be seen as adaptive, specifically higher for small weights and smaller for bigger weights. This is easy to see for the simple case of a network with two weights, where the equilibrated energy is just $\\mathcal{L}/1+w_2^2$. More to the point, however, the rescaling not only affects the magnitude of the gradient, but also the curvature and other higher-order derivatives of the energy. To see this, consider an initialisation near the origin, where the weights are approximately 0 as can be seen from the toy model above, and the rescaling is therefore close to the identity or 1, $\\mathcal{F} \\approx \\mathcal{L}$. If PC had the same effective learning rate as BP, then it would also show slow training dynamics near the origin (for networks with more than one hidden layer where the saddle is non-strict). However, this is not what we observe, theoretically and empirically.\n\nChanging the learning rate of BP to match the “effective learning rate” of PC would therefore not work, in the sense that there is no learning rate for which the two algorithms will show exactly the same (S)GD dynamics. Nevertheless, we agree with the reviewer that it is an interesting question whether higher learning rates would allow BP with SGD to escape the saddle faster than PC. However, as we explain in our global response, such an analysis was beyond the scope of this paper, and we refer the reviewer to that rebuttal for our argument.\n\nWe agree with the reviewer that we did not particularly motivate our work from a neuroscience perspective and we discussed only superficially its implications for the field. There are many excellent reviews of PC we cite (the most recent being Salvatori et al., 2023) in the first sentence of the paper that cover the neuroscientific origins and connections of PC. \n\nWe now expand on the neuroscience implications of our work based on Song et al. (2024; https://www.nature.com/articles/s41593-023-01514-1), who proposed that energy-based algorithms like PC rely on a fundamentally different principle of credit assignment for learning in the brain called “prospective configuration”—where essentially weights follow activities (rather than the other way around). Based on a range of empirical results, Song et al. (2024) argued that PC confers many benefits for learning including faster convergence. Our theoretical results suggest that this latter claim may have been overstated. In particular, our results clearly show that PC and BP operate on two different landscapes and that therefore convergence speed will depend on many factors including network depth, width, dataset (specifically output covariance), initialisation, learning rate, and optimiser–all of which affect the landscape geometry or the learning dynamics. Any universal claim about faster convergence of PC should therefore control for all these factors. Song et al. (2024) claimed that PC learns faster than BP mainly based on an experiment with a deep ($L=15$) but narrow ($n_\\ell=64$) network controlling for learning rate. However, our work reveals the importance of the initialisation of the weights, which interacts non-trivially with the network width. In particular, Orvieto et al. (2022) showed, in brief, that the narrower the network, the closer one will start from the origin saddle for standard initialisations. This insight likely explains the speed-up observed by Song et al. (2024). We add the above point to the discussion and thank the reviewer for highlighting this deficiency and allowing us to improve the reach of the paper.\n\nWe agree with the reviewer that it is not very clear why $W_L = 0$ in order for the gradient of the equilibrated energy to be zero. We can see no simple or intuitive explanation, as it simply turns out to be a property of the energy gradient. In particular, this is because of the rescaling term $S = I_ +W_LW_L^T + …$, which as we highlight contains a term quadratic in the last weight matrix. The derivative of the rescaling $\\partial S / \\partial W_L$ needs to be zero in order for the gradient of the equilibrated energy to be zero. Because as just noted the rescaling is quadratic in $W_L$, the derivative will be linear in $W_L$ and so the last weight matrix will have to be zero for the gradient to be zero. We now clarify and expand on the explanation of this property in the appendix, which as pointed out by the reviewer was previously only noted as a side remark.\n\nThe gradient $g_f$ is now defined before being used, and we thank the reviewer for pointing out this imprecision.\n\nThe ½ in the MSE loss (and energy) is often used as a theoretical convenience to cancel out the factor of 2 when taking derivatives of squared losses. It does not affect any of the derivations or theoretical results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jP8h7hrrI6", "reviewer": "Reviewer_KSTi", "summary": "This paper studies the properties of the loss function of predictive coding (PC) networks. In PC networks, the loss function that is optimized is not a typical loss such as the MSE, but the “equilibrated energy” (or “PC energy”). The paper argues that the PC energy has some better theoretical properties than the MSE loss. Specifically, the paper conjectures that all saddle points of the PC energy are strict, in contrast with the MSE loss that has non strict saddle points. To support this conjecture, the paper compares the properties of the MSE loss function and the PC energy function around the point theta=0 in deep linear networks. In particular, they compare the spectral properties of the Hessian of these functions at theta=0. They also prove that non-strict saddle points of the MSE loss (other than theta=0) become strict in the PC energy. The paper also provides some empirical evidence that PC escapes the saddle point theta=0 of the PC energy faster than BP escapes the saddle point theta=0 of the MSE loss, even in nonlinear networks.", "questions": "1. Line 25: “PC uses only local information available to each neuron to update weights…”\nIs it really the case though? To my understanding, it feels like PC needs both the transpose of the weights (in the feedback path) and knowledge of the derivative of the activation function (in nonlinear networks). It might be a very basic question, but I would like to understand what problems of BP are solved by PC networks from a hardware (neuromorphic) perspective. Could you clarify this point?\n\n2. Is there a consensus in the deep learning literature that the strict/non-strict saddle point issue is what determines whether an optimization problem is easy/hard to solve? (It is fine if there is no consensus, it is a reasonable assumption and this question should be studied anyway).\n\n3. Is it readily clear that the point theta=0 is a critical point of both the MSE loss and the PC energy? Is there a straightforward argument to see that the gradients of these functions at theta=0 vanish?\n\n4. It is shown that theta=0 is a non-strict saddle point of the MSE loss. Is this also true for other loss functions? (e.g. the cross-entropy)\n\n5. In the literature on Predictive Coding in general, it seems to be standard practice to clamp the outputs to the desired outputs. Is there a fundamental reason for doing that? Ref [1] below shows that pushing the outputs away from desired outputs (“negative nudging”) actually works significantly better than pulling them towards desired outputs (“positive nudging”) or clamping them. Note: the simulations of Ref [1] are performed on deep convolutional Hopfield networks, not PC networks, but the theory applies to PC networks too.\n\n6. Related to Question 5 above, can the theoretical analysis performed in this work be directly transferred to the case when using negative nudging as in Ref [1] ? Or does it only work when clamping outputs to desired output values?\n\n7. Could the analysis of the present paper be extended to other energy-based networks (not necessarily PC networks) ? Ref [2] below studies the effect of learning via Equilibrium Propagation on the Hessian of (linear) energy-based physical networks, which looks (at least superficially) similar to the theoretical study of this paper.\n\n\n\nReferences\n\n[1] Scellier, Benjamin, et al. \"Energy-based learning algorithms for analog computing: a comparative study.\" Advances in Neural Information Processing Systems 36 (2024).\n\n[2] Stern, Menachem, Andrea J. Liu, and Vijay Balasubramanian. \"Physical effects of learning.\" Physical Review E 109.2 (2024): 024311.", "limitations": "The paper focuses on a specific feature of loss functions, namely the presence/absence of strict/non-strict saddle points. I would expect, however, that there are more important features than this one to effectively train PC networks. See my questions above.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly presented.\n\nThe topic is important (PCNs and related energy-based networks deserve attention as we are looking for alternative, more energy-efficient, computing paradigms.)\n\nThe question raised is quite interesting.\n\nEven though there is a lot of extrapolation, the argumentation to support the conjecture is well presented and sound, too.\n\nAlthough the paper focuses specifically on PC networks that employ full clamping, my impression is that the analysis can likely be extended to other settings as well, with different energy functions (i.e. different “energy-based networks”), and alternative, more effective ways to do training (e.g. by nudging the output, like in Equilibrium Propagation, instead of fully clamping the output). See my questions below.", "weaknesses": "The theory is restricted to linear networks.\n\nSimulations are performed only around the saddle point theta=0.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their feedback and are glad that they share our excitement about the work. Below we address each point and question raised by the reviewer.\n\nThe reviewer points out that our analysis of deep linear networks (DLNs) is a weakness of our theory. We would like to highlight that DLNs are the standard model for theoretical studies of the geometry of the loss landscape (see our review of related work). This is in part because of the mathematical challenges of analysing non-linear networks. Moreover, and perhaps more importantly, DLNs can be seen as a good minimal model for understanding the optimisation properties of their non-linear counterparts. While non-linearities clearly affect the geometry of the landscape, as we note in the introduction DLNs retain all the most important properties of non-linear models including non-convexity and non-linear learning dynamics.\n\nWe would also like to highlight that our work goes significantly beyond previous theoretical studies on PC (reviewed in the related work section). In particular, previous theories make either non-standard assumptions or loose approximations. For example, both Alonso et al. (2023) and Innocenti et al. (2023) make second-order approximations of the energy to argue that PC makes use of Hessian information. However, our results clearly show that PC can leverage much higher-order information, turning highly degenerate, $H$-order saddles into strict ones. We now add this point of discussion to the appendix.\n\nWe appreciate the point that we performed experiments only around the origin saddle. As we explain in the global rebuttal, we now include two additional sets of experiments (see attached PDF), and we refer the reviewer to that for a full explanation. In brief, the first set of experiments (Figure 13) tests another zero-rank saddle than the origin covered by Theorem 3, and the second investigates higher-rank saddles which our theory does not address (Figure 14).\n\nWhether “PC uses only local information available to each neuron to update weights” depends on what one means by “local” and “available”. We acknowledge that we were being loose with words in this particular sentence and now remove it from the paper as not critical to the main message. Nevertheless, to answer the more specific question, the reviewer is correct that standard PC needs both the transpose of the weights and the derivative of the nonlinearity, though we note that Millidge et al. (2020; https://arxiv.org/pdf/2010.01047) showed that on similar datasets one can remove these constraints without significantly harming performance. How PC could be implemented on alternative (neuromorphic) hardware is largely unknown, and we know of no published work on the topic. This is clearly a very important question for the future.\n\nThe reviewer asks whether there is a consensus on how the difficulty of an optimisation problem maps onto the strict vs non-strict saddle distinction. The short answer is “yes”. We summarise the results in the introduction (paragraph 4) and provide an in-depth review in the related work section. At a very high level, strict saddles are benign for even first-order algorithms like GD as long as one does not initialise too close to them or uses a particularly small learning rate, while non-strict ones can effectively trap (S)GD in the phenomenon of vanishing gradients since they are very flat. As we note in the paper, adaptive gradient methods like Adam have been argued to better deal with vanishing gradients and curvature in this work, but and their behaviour near non-strict saddles is much less understood than that of (S)GD.\n\nThe reviewer asks whether there is a straightforward way of seeing that the gradient of both the MSE and the PC energy vanish at the origin ($\\theta=0$). The simplest case where this can be seen is for a network with two weights, where the loss is $\\mathcal{L} = ½(y- w_2 w_1 x)^2$ and the (equilibrated) energy is $\\mathcal{F}^* = \\mathcal{L} / 1+w_2^2$. Because the residual $r = (y- w_2 w_1 x)$ is quadratic in the weights, the gradient is zero. Specifically, $\\partial \\mathcal{L} / \\partial w_1 = -w_2xr$ and $\\partial \\mathcal{L} / \\partial w_2 = -w_1xr$. For the energy, the expression of the gradient is a little more involved and has basically 2 terms. One is the gradient of the loss scaled by $1+w_2^2$ which will therefore also vanish at 0. The other term depends on the derivative of the rescaling, in this case $\\partial (1+w_2^2)/ \\partial w_i$. The rescaling does not depend on the first weight so this vanishes, and for the second weight is also zero because it is again quadratic in that weight.\n\nThe reviewer asks whether the origin can be a non-strict saddle for other loss functions than the MSE. This is an interesting point, and it turns out to be true for any generic convex, twice-differentiable cost as soon as we have two or more hidden layers as shown by Jacot et al. (2021, last paragraph before Section 5.1; https://arxiv.org/pdf/2106.15933).\n\nThe reviewer asks about the relationship between PC and equilibrium propagation (EQ). As shown by Millidge et al. (2022; https://arxiv.org/pdf/2206.02629), PC can be seen as an EQ algorithm where the free phase is minimised with a feedforward sweep (i.e.  the squared energy of the last layer is equivalent to the nudging term). This avoids the need for two phases and is arguably more biologically plausible. It is not clear to us whether the theory could be generalised to the case of negative nudging or other energy-based networks, but this could be an interesting future direction. We thank the reviewer for directing us to reference [2] which we agree could potentially open interesting connections.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all of the reviewers for taking the time to read our paper and their feedback, which we believe improved the clarity, scope and contributions of the paper. In this global response, we address points that were raised by more than one reviewer and outline other relatively minor changes we made to the paper to address specific points made by individual reviewers. 

We identified two important, related concerns raised by the reviewers:
1. **Whether the reported faster SGD convergence of PC when initialised near the origin (Figure 5) holds under different hyperparameters, such as learning rates and optimisers (e.g. Adam).**

It was beyond the scope of this work—which was mainly concerned with the intrinsic geometry of the (equilibrated) energy landscape—to address this issue, and we completely agree with the reviewers that the original writing of the paper gave the (wrong) impression that this was one of our aims. We now make our goal clear throughout the paper. As we now explain in the discussion, studying all the different conditions under which the energy landscape might be faster to optimise than the loss landscape is a difficult question requiring further study. This is for a few reasons. First, one can always make the initialisation scale small enough such that BP will not escape (even with Adam) one of the considered saddles to show that PC is faster. The question then becomes whether this will have benefits for performance or generalisation and whether there might be a trade-off with learning speed. Second, the behaviour of adaptive gradient optimisers like Adam near non-strict saddles is not as well understood as that of (S)GD. Finally, characterising the geometry of minima of the equilibrated energy—which our work enables—would also be probably necessary to understand the overall convergence behaviour of PC.

We also highlight that our theoretical results do not necessarily suggest that PC will always be faster. In fact, we now argue that our results explain the speed-ups observed by Song et al. (2024; https://www.nature.com/articles/s41593-023-01514-1) on a deep network because of the small network width used, which as we explain in response to one reviewer determines how closely one starts from the origin for standard initialisations (as used in that paper).

2. **Limited experiments.**

Reviewers pointed out that our experiments were quite limited. This was in part motivated by the above concern about the impact of different hyperparameters on convergence speed, which as we hope we now clarified is beyond the scope of our work.

Nevertheless, setting aside the question of hyperparameter tuning, we acknowledge that we performed experiments on non-linear networks only when initialising close to the origin saddle. For that reason, we now include two additional sets of experiments (see attached PDF). The first (attached Figure 13) is simply a replica of the experiments in Figure 5 for another non-strict zero-rank saddle of the MSE that we proved to be strict for the equilibrated energy (Theorem 3). We also note that our submitted code makes it easy to test for other zero-rank saddles by simply changing the initialisation.

The second set of experiments (attached Figure 14) numerically investigates the question of higher-rank saddles which our theory did not address and was a specific concern raised by one of the reviewers. We compared the training loss of BP and PC with GD on a matrix completion task studied by Jacot et al. (2021, Figure 1; https://arxiv.org/pdf/2106.15933), where as they show if one starts near the origin, GD visits a sequence of saddles, each representing a solution of higher rank. As the attached Figure 14 shows, PC quickly escapes all the saddles (specifically of rank 0, 1 and 2) visited by BP. We refer to the Figure caption for more details. These results go beyond our theory (restricted to zero-rank saddles) to support our general conjecture that all the saddles of the equilibrated energy are strict.

Besides generally improving the clarity of the writing, we made other minor changes to address points made by individual reviewers:
* We emphasise the contribution of our theoretical result of the equilibrated energy as a rescaled MSE loss. We argue that this is important because (i) it corrects a previous mistake in the literature that the MSE loss is equal to the output energy as we now discuss in the paper, and (ii) it enables further studies into the energy landscape (e.g. minima) as we note in the discussion.
* We clarify and expand on the implications of the paper. We now include the most important implications and limitations in the abstract and expand on them in the discussion. In particular, we emphasise the point–which was previously only secondary–that the strictness of the origin saddle makes PC more robust to vanishing gradients, providing supporting plots of the norms of the weight gradients for all the experiments. In addition, we discuss how our results suggest that previous claims of faster convergence of PC may have been overstated and highlight the limitation of scaling PC to deeper architectures where inference becomes difficult to solve.
* To make space for the additional experiments and expanded discussion, we moved the related work section to the appendix.
* In the introduction, we better motivate the use of deep linear networks as our theoretical model, which is the standard for studies of the loss landscape geometry. We also explain how our analysis goes beyond most previous theoretical works on PC and elaborate on this point in the related work section.
* We fix some imprecise or unclear notations and derivations pointed out by reviewers. We also provide an expanded proof of the strictness of the zero-rank saddles (Theorem 3) in the appendix as requested by one reviewer.

Meta-review: This paper studies the optimization landscape of predictive coding for training deep linear networks. The authors prove that the energy of predictive coding is a rescaled version of the MSE loss and then show that several non-strict saddle points in standard training become strict saddle points in predictive coding training. This could potentially lead to better convergence.

Studying the loss landscapes for predictive coding could be useful in better understanding the benefits of predictive coding. However, as pointed out by reviewer, this paper only studies two kinds of saddles—zero and zero-rank—not all saddle points, which should be clearly stated.
Decision: Accept (poster)