Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations
Authors: Axel Laborieux
Friedemann Zenke
Venue: NeurIPS 2022 Accept (2022)
Keywords: Equilibrium propagation
credit assignment
bio-plausible deep learning
implicit differentiation

Abstract: Equilibrium propagation (EP) is an alternative to backpropagation (BP) that allows the training of deep neural networks with local learning rules. It thus provides a compelling framework for training neuromorphic systems and understanding learning in neurobiology. However, EP requires infinitesimal teaching signals, thereby limiting its applicability to noisy physical systems. Moreover, the algorithm requires separate temporal phases and has not been applied to large-scale problems. Here we address these issues by extending EP to holomorphic networks. We show analytically that this extension naturally leads to exact gradients for finite-amplitude teaching signals. Importantly, the gradient can be computed as the first Fourier coefficient from finite neuronal activity oscillations in continuous time without requiring separate phases. Further, we demonstrate in numerical simulations that our approach permits robust estimation of gradients in the presence of noise and that deeper models benefit from the finite teaching signals. Finally, we establish the first benchmark for EP on the ImageNet $32 \times 32$ dataset and show that it matches the performance of an equivalent network trained with BP. Our work provides analytical insights that enable scaling EP to large-scale problems and establishes a formal framework for how oscillations could support learning in biological and neuromorphic systems.

Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations
Axel Laborieux'
Friedemann Zenke
(firstname lastnamm)@@mit ch
Friedrich Miescher Institute for Biomedical Research, Basel, Switzerland 2 Faculty of Natural Sciences, University of Basel, Basel, Switzerland
Abstract
Equilibrium propagation (EP) is an alternative to backpropagation (BP) that allows the training of deep neural networks with local learning rules. It thus provides a compelling framework for training neuromorphic systems and understanding learning in neurobiology. However, EP requires infinitesimal teaching signals, thereby limiting its applicability in noisy physical systems. Moreover, the algorithm requires separate temporal phases and has not been applied to large-scale problems. Here address these issues by extending EP to holomorphic networks. We show analytically that this extension naturally leads to exact gradients even for finite-amplitude teaching signals. Importantly, the gradient can be computed the first Fourier coefficient from finite neuronal activity oscillations in continuous time without requiring separate phases. Further, demonstrate in numerical simulations that our approach permits robust estimation of gradients in the presence of noise and that deeper models benefit from the finite teaching signals. Finally, we establish the first benchmark for EP on the ImageNet 32 x 32 dataset and show that it matches the performance of equivalent network trained with BP. Our work provides analytical insights that enable scaling EP to large-scale problems and establishes formal framework for how oscillations could support learning biological and neuromorphic systems.
Introduction
The backpropagation (BP) error algorithm [1] underpins the ability of state-of-the-art deep neural networks to learn useful representations from structured data such as speech, vision, and text [2]. BP stands out the most successful algorithm to solve the credit assignment problem in artificial neural networks [3, 4], which can be defined by the following question: How should synaptic connection be modified in order to improve the global performance of the network perform task, as measured by some objective function? This is difficult question since individual synapse may have a complicated influence on downstream processing. BP solves credit assignment through the chain rule of differentiation [1]. Although BP is efficiently implemented in software, it is difficult to conceive how BP could plausibly be implemented in biological systems. The problematic aspects are BP's use of symmetric connections and the need for two separate phases: A nonlinear forward pass that propagates neuronal activity and a linear backward pass that carries signed gradient signals [3]. These two types of processing should be handled by the same circuit, and explicitly propagating errors does not harness the device mismatches typical of neuromorphic hardware [5, 6]. Despite its implausibility, representations learned with BP match representations of in-vivo data [7] better than networks trained with purely delogiccall-mooivvtee learning rules such STDP [8, 9]. This discrepancy raises the question to
also inconvenient for training physical neural networks, since both
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

whether and how neural dynamics could implement gradient-based credit assignment, and whether it could be as effective as BP to learn useful representations [3].
Equilibrium propagation (EP) [10] is an alternative algorithm for performing credit assignment in dynamical systems that converge fixed point, such energy- models [11]. EP also proceeds in two phases: In the first phase the dynamical system is presented with static input data until the units settle into an equilibrium or fixed point. We refer to this state as the free equilibrium. In second phase, a teaching signal slightly nudges designated output units towards a target value until the dynamics settle into a second equilibrium that is called the nudged equilibrium. EP estimates loss gradients by comparing the neuronal activity between the two equilibria. EP is appealing because the resulting learning rule is spatially local when the energy functionn consists of two-body interactions, as for instance in continuous Hopfield networks [11]. Furthermore EP provably approximates the
true gradient in the limit of vanishing nudging [10]. More generally, the implicit differentiation carried out by EP makes it suitable for meta learning [12], where explicitly backpropagating errors through an inner optimization loop becomes prohibitive due to the high memory requirement of storing intermediate time steps for regular automatic differentiation.
Nevertheless, classic EP [10, 13, 14] has several limitations. First, EP estimates only approach the actual Ioss gradient in the limit of a vanishing nudging or teaching signal. This requirement makes impractical for noisy neuromorphic systems where noise can confound small amplitude teaching signals and also unrealistic as model for learning in the brain where feedback strongly modulates processing. Moreover, the mechanisms by which biological circuits could satisfy the requirement for separate phases remains elusive. Finally, while EP can train deep networks on CIFAR-10 10 [14], it has remained an open question whether it can be scaled up to larger and more complex tasks [4].
In this article, we show that by extending EP with holomorphic network dynamics it naturally estimates exact gradients for finite teaching signals. Mathematically, the exact gradients are encoded as a Fourier coefficient of adiabatic neural oscillations. This finding suggest a natural way of estimating the gradients online through suitable synaptic filtering operations which dispenses with the need for separate phases, in a similar spirit to Baldi and Pineda [15] for contrastive Hebbian learning [16]. Our main contributions are the following:
We develop the theory of holomorphic EP (hEP) and prove that this allows computing exact gradients locally at synapses from finite teaching signal amplitudes of adiabatic oscillations.
We numerically quantify the accuracy of our estimate and show that it outperforms classic EP, especially in the presence of substrate noise and in deep neural networks.
We demonstrate learning with an always-on oscillating teaching signal, thereby alleviating the need for separated phases.
Finally, we show that hEP achieves the same performance as BP in deep convolutional neural networks (CNNs) trained on CIFAR-10/100 [17], and ImageNet 32 x 32 [18].
2 Background and previous work
Equilibrium propagation (EP). EP [10] allows training convergent dynamical systems optimize a loss function. We denote neuronal unit activity by the vector s, and the learnable parameters such as weights and biases by 0 (Fig. la). The system's dynamics are given as the gradient of scalar energy function E(0,s): s):
ds dt Os (0.s).
OE
(I)
As consequence, EP can train any energy-based models, e.g., Hopfield networks [11] to perform classification [10, 13, 14]. In classic EP, training proceeds in two phases. First, subset of units are clamped to the input x and the system goes to a 'free' fixed point denoted by sj- Second, the loss function E(0, s, y), with target y. is scaled with a small positive nudging factor & and added to the energy function É which yields the total energy F(0, s, 8, := E + (E(0,, (0,,s,,y y). This added teaching signal causes the system to reach second equilibrium sy. again by minimizing the total energy F. Although we write that l takes all units as argument, in practice typically only output units which encode the target label and thus serve as inputs for teaching signals are considered (Fig. lb). The learning objective of the system is to optimize the loss function E at the free fixed point,

---

a)
b;
8
b)
Wly
Teaching signal
Input
X
(()) y)
da: di
Σ mige(ss)) bi)
Figure 1: a) Schematic of the neuron model of a continuous Hopfield network [11] with holomorphic activation function a. The neuron 8; receives input both from upstream and downstream neurons 8j plus a bias current 6;- b) In typical supervised learning the input x is clamped, causing the network dynamics settle into a fixed point. A complex-valuee valued oscillating teaching signal added to the output causes neuronal activity to fluctuate around this fixed point.
which is defined by x(0,x,y) == =((0, S0: y). Scellier and Bengio [10] showed that:
1 OOF
2 (0, s/. 0.)))
0F
dc
00 (0, s B) B-+ B
lim
00
d0
This result requires F to be twice continuously differentiable and assumes that can apply the implicit function theorem to the equilibrium equation F,F sy. 0) = (the y argument is hereafter omitted for clarity), that / + s/ is continuously differentiable map [10]. In practice, the left-hand side of Eq. (2) is estimated by finite differences [10. 13. 14, 19]. The appeal of EP for biological plausibility [3, 20] and neuromorphic hardware [15, 21-23] arises from the fact that (i) the system only needs to propagate neural activities (Fig. la) and (ii) in layered neural networks with a Hopfield energy [11] (Fig. lb), the left-hand side Eq. (2) can be computed by a Hebbian-like learning rule as the product of pre- and postsynaptic activity. In summary, EP implicitly propagates error signals through differences of neuronal activity during the two phases, whereas BP propagates error gradients explicitly [3, 20]. However, Eq. (2) only holds in the limit B 0 where activity differences vanish, which can pose a problem in the presence of noise or when activity differences vanish with network depth, which is related the problem of vanishing gradients. In the following, we introduce holomorphic EP (hEP) which avoids these issues by estimating exact gradients with finite B, and thus from finite amplitude teaching signals.
3 Theoretical results
Our main contribution is to show that if F is holomorphic, i.e., differentiable in the sense of complex variables (see Appendix A. I for the definition), hEP computes the gradient of the objective function L for finite 3, i.e., without requiring the vanishing nudging signals (cf. Eq. (2)). To accomplish this hEP requires non-vanishing teaching signal that evolves 'adiabatically" in the complex plane with respect to the dynamics of the system (Fig. lb). In other words, we require the dynamical system to relax to equilibrium on much shorter timescale than the timescale of the nudge.
Derivation of holomorphic EP. To show that hEP yields an unbiased gradient through finite adiabatic nudging, we use the same notation as in Section 2. Specifically, we extend the theory by Scellier and Bengio [10] to the complex case and to dynamical systems, networks whose scalar function governing the dynamics holomorphic. In line with classic EP we assume that the dynamical system has free fixed point described above.
Lemma 1 (Holomorphic Equilibrium Propagation). Let F be scalar function governing the dynamics, so that the holomorphic implicit function theorem can be applied the fixed point equation F,F sú, 0) = 0, then the gradient formula of equilibrium propagation (Eq. (2)) holds in the sense of complex differentiation.
Proof. The proof is extension of the one provided by [10] for the real nudging case. The holomorphic implicit function theorem ensures that there exists an open set U E C including 0 such that the implicit map B U $2 is holomorphic defined on U (see Fig. 2b for how this area looks for a toy example). The proof proceeds in two steps. First we show that the total derivatives of with respect to 0 and can still be interchanged for complex variables by virtue of the Schwarz theorem. Second show that, at the fixed point, the
U. In particular, the fixed point sØ,s is
3

---

total derivative of F with respect to 3 (0) is still equal to the partial derivatives with respect to / (0). To that end, we apply the chain rule of complex differentiation in:
dF dB
OF
OF 0s
aF
ds
0 s0, 3)
0 s0.3. 3) + Os
0g 0.
00.
03 0 0.
0
05
where s denotes the complex conjugate of s. At equilibrium, the second term on the right hand side cancels by definition of the fixed point, and the third term is zero because F is holomorphic, i.e., its derivative with respect to the conjugate variable is zero according to the Cauchy- condition [24]. The same argument holds for the derivative with respect A. Therefore, interchanging the total derivatives of F' with respect to 3 and 0, and replacing the inner total derivatives by the partial derivatives, we obtain that the EP gradient formula (Eq. (2)) still holds, but for complex differentiation (Appendix A.
OF S0,
do 03 Sa 3) dc
d OF
30
(4)
dB3
30
ao
A=0
which concludes the proof (a more detailed version is in Appendix A.2).
We can now evaluate the left hand side of Eq. (4) using a Cauchy integral (Appendix A.1):
Theorem 1 (Exact gradient from finite teaching signals). Assuming that the conditions of Lemma I are met and let |3] > be the radius of a circular path around 0 in C contained in the open set U on which the fixed point So.s is defined. Further assume that this path is parameterized by t te [0, + ((t) = [3] e3(e2imt eiirt/T, where is the imaginary unit. Then the loss gradient is given by: OF 00 ((1))) e- -Bintt/T dt.
dC
1
ao
T13
The full proof is given in Appendix A.3. Theorem guarantees that given holomorphic dynamics we can dispense with the requirement of vanishing teaching signal | -> 0 in the limit of 'adiabatic" nudging which corresponds to integrating over infinitely many fixed points with a finite [3). Note, that complex-valued teaching signals 3 E y produce fixed points in the complex plane computed through the same equations as in the real case (see Appendix B). In particular, multi-layered neural networks (Fig. 1b) can be trained by using the continuous Hopfield dynamics [11]. The trainable parameters are the weights and biases O = (Wiy, b,) and the total energy function F is given by:
1 2
Σ
S
1 Σ
(xsoo()))((ss)) - Σ bi:)(s:) + Be(
(6)
2
i
If the activation function a is holomorphie, which is true in the case of sigmoid functions, the same F (Eq. (6)) can be evaluated with complex B, and we can apply Eq. (5) to obtain:
dc
[-" ((0))(()))-22)) -Bwe/// dt, T|
1
dwiy
where (f = (s) Therefore, assuming T T-periodic teaching signal (Fig. 1b), the gradient proportional to the first exponential Fourier coefficient of the product of the oscillating activities. Although this formulation assumes complex neuronal output, we show in Appendix A.4 that the gradient in Eq. (7) can be expressed in terms of the real part or imaginary part only. The complex teaching signal is therefore best thought of as a way to produce unbiased neuronal oscillations on the nudging timescale. the next section, we numerically estimate the Fourier coefficient of Eq. (5) with a fixed number of points N on the circle which we use to train networks in the subsequent experiments and to compare to the actual loss gradient computed with automatic differentiation.
Numerical estimation of the loss gradient as a Fourier coefficient. Next, we explain how to estimate the gradient from the corresponding Fourier coefficient (Eq. (5)). In practice, we use Riemann sum to compute the integral numerically. We fix the nudging radius |&]| > 0 such that the cireular path lies in the domain U in which equilibria exist as described Theorem 1. We sample the path with N 2 2 nudging points {.B, == aik/NN NNN k [O,..., - 1]}. and define the estimator:
1 N OF
V(N) =
Σ
(0 s/y 3B)) 2in//N
(8)
N|3
0e
K=
4

---

We have that V(N)
- de N-000 de
and the remaining bias term when using N points is:
dc
13p
V(N)
Σ PEO (N)
do
(p + 1)!
where Cp is the p-th derivative in 0 of the function [ B (0, s/g, B) (see Appendix A.5 for the proof). The bias term in Eq. 9 converges to zero with increasing N because it is sub-sum of the (NV + 1)-th order remainder of the series expansion of |3 + (0, s3, 3) in O. The rate of convergence depends on the Cp coefficients and the radius |3). In the case N = 2, the estimate of Eq. (8) coincides with the 'symmetric' estimate of Laborieux al. [14]. However, the bias term on the right hand side of Eq. (9) is only valid when the dynamics are holomorphic. Next, we illustrate the approach in a toy experiment, and list three practical improvements brought by hEP.
4 Experiments
In all the experiments, we used the discrete setting of convergent recurrent neural networks of [13]. and the readout scheme of [14] for optimizing the cross-entropy loss function with EP (Appendix B). All simulations were implemented in Jax [25] and Haiku [26] (Apache License 2.0). The datasets were obtained from the Tensorflow datasets library [27]. Our code is publicly available on GitHub'.
The details of simulations and hyperparameters can be found in Appendix E and F.
Demonstration of holomorphic Equilibrium Propagation on a single data point. To provide the first numerical validation of Theorem 1 while also allowing to gain intuitions about dynamics of individual neurons, we implemented a small-scale multi-layer perceptron (MLP) with layer dimensions 6-4-4-4, including input and output layers. The activation function was a shifted sigmoid, which is holomorphic (Appendix B). The network was fed with a single datapoint, namely a randomly sampled point from Gaussian and random one-hot target. The dark blue region in Figure 2b shows the map of complex 3 for which the network settles to a fixed point after 200 time steps. We found experimentally that the area in the complex plane where stable fixed points exist strongly depends on the activation function and the weight initialization (see Appendix D). As 3 evolves on the circle of radius 0.1 (N 24) hidden layer neurons settle into different equilibrium points in the complex plane (Fig. 2a). We observed that while the teaching signal (t) |B|e 2itt/T was purely sinusoidal, the non-linearity of the network induces neural oscillations that are not purely sinusoidal. Nevertheless, the gradient is contained in the first mode of these non-linear oscillations (Fig. 2c).
To understand how the gradient computed accurately when the magnitude of the teaching signal is increased, we recorded the adiabatic product of activities of (t)o (t) for one pair of neurons (dark blue) over one teaching period and for increasing values of [33] (Fig. 2c). In the case |/|| 0.001, the perturbation induced by the sinusoidal teaching signal is also purely sinusoidal, and the gradient magnitude is simply the radius of the circle. However, when [3] is increased to 0.01, the linear approximation of the perturbation becomes less accurate, because higher powers |3 become significant in the series expansion around the free fixed point. The gradient could still be well approximated by taking the mean of the two radii corresponding to real positive and negative 3. as done by Laborieux et al. [14]. Increasing | further to 0.1 and 0.5 yields deformed perturbation, but the gradient still correctly contained in the first Fourier coefficient of the perturbation. hEP breaks down when 33 reaches amplitudes for which the corresponding path intersects with areas in which no stable equilibrium exists (cf. Fig. 2b, light areas, and Fig. 2d). However, as we will see in the next sections, the finite teaching amplitudes are beneficial when the neuronal dynamics are subject to noise and when training deep neural networks.
more
Holomorphic EP can estimate the gradient in continuous time. Our theoretical findings allow us to revisit in a principled way an idea introduced by Baldi and Pineda [15] for a continuous-time- implementation of contrastive Hebbian learning [16] and learning rules looking to maximize slowness [28-30]. In this context, the oscillating teaching signal is always-on, and the dynamical network governed by three mechanisms acting on distinct timescales. The smallest timescale Tayo is the typical time needed by the network to reach its fixed point. The second timescale is the period of
http://tthbbbccm//llaaiiiuuuuuu/ Axel http://bwo...oo...aaoooooooooo..
5

---

(1)
b)
|
x11I
5
Re(a(())
0,5
Im(
0-
Bele
-5-
-0,3
20
40
Time steps
0.0 Re(
0.5
d)
0
0.001
|
0.01
| = 0.1
[ = 0.5
Cosine similarity
1.00
0.99
True gradient
Classic EP
Im
Holamorshic EP
Re
<100e100
X Holo. EP (Eq. (7))
+ True gradient
0.0
0.3
0.6
0.9
[8]
[88|
Figure Overview of hEP for a small MLP. a) Neural oscillations sampled N 24 points the complex plane relative to the free fixed-point in the center (left, point sizes increasing with time). The corresponding real part values over two periods (right). b) Map of the euclidean norm between two consecutive steps of the dynamics in the complex plane spanned by 3. The map describes the network's ' dynamical stability. Dark blue corresponds to regions with stable fixed points, while light blue indicates lack of stability. c) Adiabatic correlations between two neurons for different [8] values (dark blue dots). Filtering the mode over the period T the teaching oscillation gives the light blue dots, the temporal mean of which (orange cross) is the gradient (grey plus sign). d) Cosine similarity with the true gradient of hEP (orange), and classic EP [10] (blue). The marks the teaching radius |3| of the path in panel b). hEP breaks down when the path passes through unstable (light) regions.
b)
1.00
Cosine similaaity
1- 100m
10
Cosine similarity
godert
I-10 10
Mochine procsien
1001
0.95
Clissc EP Hilomeepic
10-1
1- 1-00-__
T
ssc
100
30C
303
10:3 5-
0.90
0
20
30
40
200
400
0
2
3
Oscillation periods
Time steps for period
[
Figure 3: a) Cosine similarity between the true gradient obtained with BP through time and the online estimate (N = 10) as function of oscillation periods. Different curves correspond to different oscillation periods (darker color indicates larger Tose). Tptas 107 is enough accurately estimate the gradient. b) Measure of the residual convergence of the network as function of the oscillation period Tose, showing that Tayn 400/ /00100 40 time steps. e) Cosine similarity of hEP (N 15) and classic EP with the true gradient as a function of [ with neuronal output noise. All panels used the same MLP with two hidden layers of 256 neurons each, fed with a minibatch of MNIST samples.
teaching oscillation Toxe and the third timescale the number of periods after which synaptic plasticity occurs Tplas. The gradient can be estimated online by:
(T = Tplas [8| oz(1)o,(1)e (1)e)),())22ix)) 2istt/ Bistt//iie dl,
1
I Tim
(10)
which converges to the gradient if Tayn Time < Tpim (see Appendix A.6). This expression is similar to Eq. (5.2) in [15]. However their teaching signal oscillates discretely between and 1, and therefore produces biased estimate of the gradient. To test the influence of the oscillation timescale Tase on the online estimate of Eq. (10), we compared the online estimation of the gradient over several periods between several values of Tese this end, we used MLP with two hidden layers with 256 units each, which we fed with a minibatch of MNIST data [31]. We observed that the gradient could be accurately estimated a few periods for high enough Time (Fig. 3a, dark curves). However, when the oscillations were too fast, vanishiniigg bias remained in the gradient estimates even for many

---

Last fully connected layer
Total gradient
b)
Cosine similantt
1- 10
Cosine limiority
10
0.15
100-1
Medtess preduton
Traiting
0.30
10
600
9.05
[2]
0..00
10
18]
10
10 -
[P]
10
Number of points
Non holo.
(N-22
Hele. (N-22)
Hola. (Nov))
Figure 4: a) Cosine similarity between hEP and the true gradient in holomorphic (plain lines) seven-layer VGG-like CNN [32], and a non-holomorphic version using max pooling and ReLUs (dashed lines). Input data is minibatch of ImageNet 32 x 32 [18] consisting of 10 images. The left plot shows a comparison of the gradient with respect to the parameters of network's 's output layer. The right plot takes into account the gradient with respect to all network parameters. b) Cosine similarity in function of N for three teaching amplitude values. Increasing N is only required for higher amplitudes. e) Average training error on CIFAR-10. The average is calculated over three random initializations and error bars correspond to two standard deviations.
periods (Fig. 3a, lighter curves). This bias is in all likelihood due to the inability of the system to reach the fixed point (Fig. 3b). Finally, we found that given appropriate period timings, hEP used in the online setting can train network MNIST (Table 1). Importantly, the online formulation of hEP allows to dispense with the requirement of strictly separate learning phases by replacing them with separate plasticity mechanisms acting on different timescales.
Table I:
MNIST validation errors in % for classic EP [10], hEP, and online hEP, with and without noise, Results are averages (n = 3) + stddev. For training errors see Table 4 in Appendix E.
Noise
Class. EP, | = 0.1 Class. EP, | = 0.4
hEP, [3 = 0.4
Online hEP
Noise-free
1.87 = 0.01
2.24 : 0.05
1.97 + 0.08
2.05 + 0.02
With noise
88.7 + 0.0
3.01 + 0.1
1.96 + 0.07
1.91 + 0.16
Finite size teaching oscillations provide robustness to noise. To analyze hEP's robustness to noise, we injected a small-amplitude zero-mean Gaussian noise to each neuron in the network in addition to the input from other neurons. We then used a single minibatch from the MNIST dataset to compute gradient estimates using classic EP and hEP. The latter was computed by using one realisation using N- = 15 points, whereas the classic EP estimate was computed using the free and nudged fixed points each averaged over [N/2] to provide a fair comparison. We found that for small 3 when noise amplitudes were comparable to the activity changes caused by teaching oscillations the gradient estimate diverged from the true gradient of the noise-free system (Fig. 3c). To some extent this effect could be mitigated by choosing a finite teaching signal (8 > 0) [12]. However, since also increases the bias for classic EP this creates a trade-off between choosing 3 either too small or too large. Importantly, even for the optimal choice of B, classic EP did not accurately approximate the gradient of the noise-free system. In contrast, hEP thanks to its robustness to finite teaching signals did provide an accurate estimate of the gradient despite the noise. We verified that hEP is indeed more robust to noise than classic EP when used to train the network (Table 1). Thus, hEP combined with finite teaching amplitudes provides an effective way for training noisy computational substrates.
Holomorphic EP matches BP performance on large-scale vision benchmarks. To test hEP's ability train deep neural networks, we first investigated the influence of the number of fixed points N and the teaching amplitude |3] on the approximating quality (NN) of the loss gradient in seven-layer VGG-like architecture ([32]; Fig. 4a,b). We ensured holomorphic dynamics by using softmax pooling layers [23] instead of the holomoooep max pooling, and by relying on sigmoid weighted linear units (dSiLU) [33] (see Appendix B) instead of standard ReLUs. We first considered a minibatch of ten images from the ImageNet 32 x 32 dataset [18] and computed the gradient using hEP as well as BPTT for reference. Here, our estimate (Eq. (8)) was computed by first letting the network settle to the free fixed (3 = 0), and then running the phases with complex B3. We found that the change to holomorphic dynamics already improved upon the gradient estimates used in previous
7

---

work [14, 19). Moreover, we observed that for the last layer increasing N only extended the range of usable teaching magnitudes 1,81, but did not improve the quality of the gradient estimate. This phenomenon can be understood from Eq. (9), since higher N reduces the bias term considerably, which accommodates higher teaching magnitude [3]. However, larger amplitudes tended to improve the total gradient, particularly in deep layers where small teaching magnitudes were not enough to produce sufficient error signals (see Appendix C for details). Additionally, larger N were only required when using& higher teaching amplitude (Fig. 4b). Finally, we tested how the gradient quality impacts the network training accuracy on CIFAR-10. We observed that the non-holomorphic VGG was unable to reach low training error (Fig. 4c), which is consistent with the poor gradient quality (Fig 4a). Changing to a holomorphic architecture with the same number of points resulted in substantial improvement of training accuracy, which was further boosted when training with N 4 consistent with our theory.
Table 2: Validation accuracy of BP and hEP. All values are averages (n = 3) + stddev.
CIFAR-10
100
ImageNet 32 x 32
Top-1 (%)
Top-1 (%)
Top-5 (%)
Top-1 (%)
Top-5 (%)
BP
88.3 + 0.1
62.0 + 0.5
86.2 + 0.1
37.2 + 0.4
60.9 + 0.1
hEP
88.6 + 0.2
61.6 + 0.1
86.0 + 0.1
36.5 + 0.3
60.8 + 0.4
Finally, we wondered whether hEP could train deep neural networks on large-scale datasets, which has remained an open problem for most if not all alternative algorithms to BP [4]. To this end, we trained a five-layer CNN based on the VGG architecture [32] on multiple vision benchmarks including CIFAR-IO, CIFAR-100 [17], and the 32 x 32 pixel version of ImageNet [18], which contains 1.2 million data points and 1000 classes like the full ILSVRC dataset [34]. In all cases we found that the validation accuracy reached by networks trained with BP and hEP using only two fixed points (N = 2) were identical within their uncertainties (Table 2). Note that the networks trained with BP were the feed-forward equivalent of the holomorphic networks used with EP, but with ReLUs instead of dSiL.U, which did not give satisfactory results. Thus, hEP permits training deep CNNs on ImageNet 32 x 32 to comparable performance levels as standard BP.
5 Discussion
We have introduced hEP which extends classic EP by computing exact loss gradients through integration over finite size adiabatic neuronal oscillations caused by a teaching signal (Section 3). Importantly, such integration can be accomplished online with purely local learning rules which makes an exciting theoretical framework for studying learning in the brain where oscillations are ubiquitously observed [35, 36]. In practice we found that numerically evaluating small number of points during one oscillation cycle provides an excellent gradient approximation that outperforms classic EP and thanks to the finite oscillation amplitude is robust to noise, which is advantage for training neuromorphic hardware systems [21, 22, 37). Additionally, the possibility of using finite teaching signals is conducive for training deep CNNs, where infinitesimal teaching signals used by classic EP, may vanish (Section 4).
A body of previous work has attempted to reconcile BP with neurobiology [3, 38). EP is most closely related to classic theories of predictive coding (PC) [39-45] which similarly assumes convergent network dynamics cast into an energy minimization problem. PC further assumes that errors are encoded in neuronal dynamics, dedicated dendritic compartments, or separate temporal phases [20, 40, 46, 47]. In similar vein, Target Propagation (TP) [48-50] assumes locally encoded error signals which in some cases are obtained by iterating approximate inverses, a property reminiscent of EP [51-54] which comes with theoretical guarantees [55]. However, all previous EP studies and most of the works above, with two notable exceptions [38, 50], were all limited to small-scale problems [4]. In contrast, we demonstrate in this article that hEP scales to ImageNet 32 x 32.
While the ability to run hEP online makes it an appealing model for credit assignment in biological neural networks, this interpretation has several notable shortcomings. First, hEP requires complex- valued neuronal outputs and a holomorphic dynamical system which precludes the use of max pooling and ReLUs and hampers direct comparison to neurobiology. However, we found that holomorphic
8

---

alternatives exists which empirically yield comparable performance. Moreover, complex outputs have tradition in computational neuroscience where they appear in variations of Hopfield networks [56-58], in the framework of theta neurons [59], and phasor networks [60] where they are used to describe oscillatory neuronal dynamics. It possible to interpret hEP within such frameworks. For instance, it is straightforward to interpret complex neuronal output as oscillating activity with a defined amplitude and relative phase to some reference signal accessible to the entire neuronal population. Such signal could be provided by neuromodulators such acetylcholine which has been implicated neural oscillations [61]. Within our framework, the oscillatory teaching signal then corresponds to slow phase precession between the neuronal activity and the reference. Importantly, such a mechanism implies a hierarchy of oscillation frequencies. Such different oscillations are known to exist in the brain, c.g., theta (4-8 Hz) and gamma (30-70 Hz), but their precise purpose remains elusive. While establishing formal circuit-level equivalences with hEP will require future work, the principled link hetween oscillatory activity, learning and memory developed in this article seems promising. Like other algorithms hEP requires symmetric synaptic connectivity between layers which seems biologically implausible. While theoretical guarantees for exact gradient computation are lost without strict symmetry [62]. it may not be required for learning [63-65]. Alternatively, symmetry may be acquired through plastic feedback connections [66, 67]. Although our approach neither uses time-varying input nor neuronal spiking dynamics, spiking extensions to EP have been proposed [68, 69]. However, applying our theory to time-varying tasks requiring memory will require additional architectural modifications and theoretical concepts [70] and establishing links to present spike-based approaches [38, 71-73].
a
Our work augments classic EP with desirable properties for potential neuromorphic applications, which promise power-efficient and equitable artificial intelligence (AI) the edge and in loT devices [74-77]. While current software implementation of EP are generally slow compared to backprop, its appeal lies in its potential for training physical networks on future neuromorphic mixed-signal devices that are incompatible with backprop, but achieve settling times on the order of nanoseconds (21,37) As exciting as such developments are, they also risk negative societal impacts, e.g., through mass surveillance or allowing AI systems with potentially discriminatory biases to permeate our everyday lives further. A transparent research strategy and taking into account ethical considerations early during product design will be essential to avoid such adverse outcomes. On the upside, our theoretical work further consolidates EP as a conceptual framework for understanding the brain, fundamental requirement to inform future biomedical research targeted at nervous system disorders.
Acknowledgements
We thank all members of the Zenke Group for comments and discussions. We also thank the anonymous reviewers for their useful feedback. This project was supported by the Swiss National Science Foundation [grant number PCEFP3_20296 and the Novartis Research Foundation.
References
[1] David E Rumelhart, Geoffrey E. Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, (23(688)) 1986.
[2] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436 2015.
[3] Timothy Lillicrap, Adam Santoro, Luke Marris, Colin Akerman, and Geoffrey Hinton. Backpropagation and the brain. Nature Reviews Neuroscience, 21(6))335-346, 2020.
[4] Sergey Bartunov, Adam Santoro, Blake Richards, Luke Marris, Geoffrey Hinton, and Timothy Lillicrap. Assessing the scalability of leltoogccallymmotvviit deep learning algorithms and architectures. Advances in neural information processing systems, 31, 2018.
[5] Pritish Narayanan, Alessandro Fumarola, Lucas L. Sanches, Kohji Hosokawa, Scott C Lewis, Robert M Shelby, and Geoffrey W Burr. Toward on-chip acceleration of the backpropagation algorithm using nonvolatile memory. IBM Journal of Research and Development, 61 (4/5)::111,, 2017.

---

[6] Stefano Ambrogio, Pritish Narayanan, Hsinyu Tsai, Robert M Shelby, Irem Boybat, Carmelo Di Nolfo, Severin Sidler, Massimo Giordano, Martina Bodini, Nathan CP Farinha, al. Fquivalent-accuracy accelerated neural-network training using analogue memory. Nature, 558(7708)660-67, 2018.
[7] Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James J DiCarlo. Porformance-opiminn hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 11 111(23):619-8622 861--8244 2014.
[8] Yang Dan and Mu-ming Poo. Spike timing-dependent plasticity of neural circuits. Neuron, 44 (1):23-30, 2004.
[9] Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothée Masquelier, and Anthony Maida. Deep learning in spiking neural networks. Neural networks, 111:47-63, 2019.
[10] Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
[11] John J Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the national academy of sciences, 31110):08883092 1984.
[12] Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, and João Sacramento. A contrastive rule for meta-learning. arXiv preprint arXiv:2104.01677, 2021.
[13] Maxence Ernoult, Julie Grollier, Damien Querlioz, Yoshua Bengio, and Benjamin Scellier. Updates of equilibrium prop match gradients of backprop through time in an rnn with static input. Advances in neural information processing systems, 32, 2019.
[14] Axel Laborieux, Maxence Emoult, Benjamin Scellier, Yoshua Bengio, Julie Grollier, and Damien Querlioz. Scaling equilibrium propagation deep convnets by drastically reducing its gradient estimator bias. Frontiers in neuroscience, 15:129, 2021.
[15] Pierre Baldi and Fernando Pineda. Contrastive learning and neural oscillations. Neural computation, 3(4):526-55) -555, 1991.
[16] Javier R Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connec- tionist models, pages 10-17. Elsevier, 1991.
[17] Alex Krizhevsky, Learning multiple layers of features from tiny images. Technical report, 2009.
[18] Patryk Chrabaszez, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenet as an alternative to the cifar datasets. arXiv preprint arXiv: 11077707778899 2017.
[19] Artur Luczak, Bruce L McNaughton, and Yoshimasa Kubo. Neurons learn by predicting future activity. Nature Machine Intelligence, pages 1-11, 2022.
[20] James CR Whittington and Rafal Bogacz. Theories of error back pack-propngttion in the brain. Trends in cognitive sciences, 23(3):235-250) 2019.
[21] Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio, and Benjamin Scellier. Training end-to-end analog neural networks with equilibrium propagation. arXiv preprint Stt1:200600198// 2020.
[22] Gianluca Zoppo. Francesco Marrone, and Fernando Corinto. Equilibrium propagation for memristor-basse recurrent neural networks. Frontiers in neuroscience, 14:240. 2020.
[23] A Stergiou, R Poppe, and G Kalliatakis. Refining activation downsampling with softpool. arxiv 2021. arXiv preprint arXiv:2101.0m)
[24] Walter Appel. Mathematics for physics and physicists. 2007.
10

---

[25] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Gunderman-Milne and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http://githnn
goon//oogle//ar
[26] Tom Hennigan, Trevor Cai, Tamara Norman, and Igor Babuschkin. Haiku: Sonnet for JAX, 2020. URL. http://gt hhw.tggg.ttttt.tte com comm/eepmindddeee haiku.
[27] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jazefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large- scale machine learning on heterogeneous systems, 2015. URL sttp:://tenoofflo..or Software available from tensorflow.org.
[28] Manu Srinath Halvagal and Friedemann Zenke. The combination of hebbian and predictive plasticity learns invariant object representations in deep sensory networks. bioRxiv, 2022.
[29] Laurenz Wiskott and Terrence Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural computation, 14(4):715-770, 2002.
[30] David Lipshutz, Charles Windolf, Siavash Golkar, and Dmitri Chklovskii. biologically plausible neural network for slow feature analysis. Advances in neural information processing systems, 11.14986-149999 2020.
[31] Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-12,, 2012.
[32] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv: 1556, 2014.
[33] Stefan Elfwing. Eiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107:3-11, 2018
[34] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (ICV), 115(3):21115 2015. doi: 10. 10.1007/s11263-0000 10000//11223-001-08811....
[35] Juergen Fell and Nikolai Axmacher. The role of phase synchronization in memory processes. Nature reviews neuroscience, 1252):105-118, 2011.
[36] Andreas K Engel, Pascal Fries, and Wolf Singer. Dynamic predictions: oscillations and synchrony in top-down processing. Nature Reviews Neuroscience, 2(10):04-776, 2001.
[37] Menachem Stern, Daniel Hexner, Jason W Rocks, and Andrea J Liu. Supervised learning in physical networks: From machine learning to learning machines. Physical Review X, 11(2): 021045, 2021.
[38] Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience, 24(7):1010-1099, 2021.
[39] Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-fiele effects. Nature neuroscience, 2(1):79-87, 1999.
[40] Georg B Keller and Thomas Mrsic-Flogel. Predictive processing: a canonical cortical computation. Neuron, 100(2):424-435, 2018.

---

[41] Yanping Huang and Rajesh PN Rao. Predictive coding. Wiley Interdisciplinary Reviews: Cognitive Science, 2(5):580-593, 2011.
[42] James CR Whittington and Rafal Bogacz. An approximation of the error backpropagation algo- rithm in a predictive coding network with local hebbian synaptic plasticity. Neural computation, 29(5):1229 1262, 2017.
[43] Beren Millidge, Alexander Tschantz, and Christopher L. Buckley. Predictive coding approxi- mates backprop along arbitrary computation graphs. arXiv preprint arXiv:2006.0m 222200000048880 04/22, 2020.
[44] Robert Rosenbaum. On the relationship between predictive coding and backpropagation. Plos one, 173))e0266102, 2022.
[45] Karl Friston. The free-energy principle: a unified brain theory? Nature reviews neuroscience, 11(2):127-33,, 2010.
[46] João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. Advances in neural information processing systems, 31, 2018.
[47] Paul Haider, Benjamin Ellenberger, Laura Kriener, Jakob Jordan, Walter Senn, and Mihai A Petrovici. Latent equilibrium: Arbitrarily fast computation with arbitrarily slow neurons. Advances in Neural Information Processing Systems, 34: 34:17839-17851, 2021.
[48] Yoshua Bengio. How auto entodenccders could provide credit assignment in deep networks via target propagation. arXiv preprint (00:::4077700 2014.
[49] Dong- Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propaga- tion. In Joint european conference on machine learning and knowledge discovery in databases, pages 498-515. Springer, 2015.
[50] Maxence Emoult, Fabrice Normandin, Abhinav Moudgil, Sean Spinney, Eugene Belilovsky, Irina Rish, Blake Richards, and Yoshua Bengio. Towards scaling difference target propagation by learning backprop targets. arXiv preprint arXiv:2201.134155 2022.
[51] Nasir Ahmad, Marcel van Gerven, and Luca Ambrogioni. Gait-prop: A biologically plausible learning rule derived from backpropagation of error. Advances in Neural Information Processing Systems, 33:10913-11023, 2020.
[52] Bill Podlaski and Christian K Machens. Biological credit assignment through dynamic inversion of feedforward networks. Advances in Neural Information Processing Systems, 33: 10065-1006. 2020.
[53] Yoshua Bengio. Deriving differential target propagation from iterating approximate inverses. arXiv preprini arXiv:2007.15139 2020.
[54] Adam A Kohan, Edward A Rietman, and Hava T Siegelmann. Error forward-propagation Reusing feedforward connections to propagate errors in deep learning. arXiv preprint 13X1v:1808.03337, 2018.
[55] Alexander Meulemans, Francesco Carzaniga, Johan Suykens, João Sacramento, and Benjamin F Grewe. A theoretical framework for target propagation. Advances in Neural Information Processing Systems, 33:20024-20036, 2020.
[56] Akira Hirose. Dynamics of fully complex-valued neural networks. Electronics letters, 28(16): 1492-1494, 1992.
[57] Stanislaw Jankowski, Andrzej Lozowski, and Jacek M Zurada. Complex-valued multistate neural associative memory. IEEE Transactions neural networks, 199-1499 1996.
[58] E Paxon Frady and Friedrich T Sommer. Robust computation with rhythmic spike patterns. 2019.
Proceedings of the National Academy of Sciences,
[59] Bard Ermentrout and Nancy Kopell. Parabolic bursting in an excitable system coupled with a slow oscillation. SIAM journal applied mathematics, (11)):33-2523 1986.
12

---

[60] Connor Bybee, E Paxon Frady, and Friedrich T Sommer. Deep learning in spiking phasor neural networks, arXiv preprint arXiv:2204.00507, 2022.
[61] M. Hasselmo. The role of acetylcholine in learning and memory. Curr. Opinion Neurobiol, 16: 710-715, 2006.
[62] Benjamin Scellier, Anirudh Goyal, Jonathan Binas, Thomas Mesnard, and Yoshua Ben- gio. Generalization of equilibrium propagation to vector field dynamics. arXiv preprint 1808::1800.04833:. 2018.
[63] Timothy P Lillierap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synap- tic feedback weights support error backpropagation for deep learnin.. Nature communications 1(1):1-10, 2016.
[64] Julien Launay, lacopo Poli, François Boniface, and Florent Krzakala. Direct feedback alignment scales to modern deep learning tasks and architectures. Advances neural information processing systems, 9099366-6660,,, 2020.
[65] Arild Nøkland. Direct feedback alignment provides learning in deep neural networks. Advances in neural information processing systems, 29, 2016.
[66] Mohamed Akrout, Collin Wilson, Peter Humphreys, Timothy Lillicrap, and Douglas B Tweed. Deep learning without weight transport. Advances in neural information processing systems,
32, 2019.
[67] Yali Amit. Deep learning with asymmetric connections and hebbian updates. Frontiers in computational neuroscience, 13:18, 2019.
[68] Peter O'Connor, Efstratios Gavves, and Max Welling. Training a spiking neural network with cquilibrium propagation. In The 22nd international conference artificial intelligence and statistics, pages 1516-1523. PMLR, 2019.
[69] Erwann Martin, Maxence Ernoult, Jérémic Laydevant, Shuai Li, Damien Querlioz. Teodora Petrisor, and Julie Grollier. Eqspike: spike- equilibrium propagation for neuromorphic implementations. Iscience, 2423):00222,, 2021.
[70] Jack Kendall. A gradient estimator for varying electrical networks with non-lincar dissipa- tion. arXiv preprint arXhv:2103.05634 2021.
[71] Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 30(6)1511-15441 2018.
[72] Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Leg- enstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1):1-15, 2020.
[73] Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep continuous local learning (decolle). Frontiers in Neuroscience, 14:424, 2020.
[74] Carver Mead. Neuromorphic electronic systems. Proceedings of the IEEE, 78(10):169-11666 1990.
[75] Giacomo Indiveri, Bernabé -arreee,arranco, Tara Julia Hamilton, André van Schaik, Ralph Stienne-Cumminn Tobi Delbruck, Shih- Chii Liu, Piotr Dudek, Philipp Häfliger, Sylvie Renaud, et al. Neuromorphic silicon neuron circuits. Frontiers in neuroscience, 5:73, 2011.
[76] Catherine D Schuman, Thomas E Potok, Robert M Patton, Douglas Birdwell, Mark E Dean, Garrett S Rose, and James S Plank. A survey of neuromorphic computing and neural networks hardware. arXiv preprint arXiv: 1rXiv:1705.06663, 2017.
[77] Julie Grollier, Damien Querlioz, and Mark D Stiles. Spintronic nanodevices for bioinspired computing. Proceedings of the IEEE, 104(10):004-2033, 2016.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] The theoretical claims refers to Section 3, and the experiments claims to Section 4.
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 5.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] Consent to use ImageNet data was obtained from the official website.
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [Yes] The assumption of Theorem are stated along with the result in Section 3, and in the extended proof's in Appendix A.
(b) Did you include complete proofs of all theoretical results? [Yes] The proof of Theorem
1 is summarized in Section 3 and detailed version is in Appendix A.2. The proof of the bias term Eq. (9) is in Appendix A.5. The proof of online estimate is in Appendix A.6
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] The code is included link to the public GitHub repository, where instructions reproduce the results are given (Section 4), and in the supplemental.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] The details and hyperparameters of experiments are reported in Appendix E.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] Error bars for training experiments are reported in Table
I and 2, and in Fig. 4c.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Section 4 and Appendix F.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you cite the creators? [Yes] The creator of the software libraries used for simulations are cited in the beginning of Section 4. The datasets used are cited when mentioned Section 4.
(b) Did you mention the license of the assets? [Yes] See Section 4.
(c) Did you include any new assets either in the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're 'rere using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "S5qzQ_ZJQ4W", "reviewer": "Reviewer_8a15", "summary": "By restricting (can also be viewed as expanding) to holomorphic neural networks, the authors are able to use a contour integral in the complex plane for finite $\\beta$ to evaluate a differential used in equilibrium propagation (EP) that is only valid as an approximation of the backprop gradient in the limit $\\beta \\to 0$. The authors interpret this use of the contour integral as a time integral of an oscillating teaching signal. The authors show that this holomorphic EP surpasses EP in performance especially in the presence of noise.\n", "questions": "Perhaps the authors can speculate further in the discussion how such a scheme may be implemented bio-plausibly.\n\n[Update: since the authors have included this, I have enhanced my rating from 7 to 8.]\n\nMinor corrections:\nFigure 2 legend for panel d \"circular path in panel d)\" -> \"circular path in panel b\".\ninconsistency in lines 85-86: $l(y,\\theta,s)$ on line 85 vs $l(\\theta,s,y)$ on line 86, also line 91.\n", "limitations": "Yes", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors incorporate a cool idea to convert a differential to a contour integral in the complex plane and then interpret this integral as a time integral of an oscillating teaching signal. They also show that allowing for finite beta enables robustness to noise, unlike EP which is only vaild in the lim $\\beta \\to 0$. However, the authors do not provide any pointers as to how this may be implemented in biology except mentioning that there may be connections to theta neurons and phasor networks. Still this is an interesting idea that possibly deserves to be disseminated to stimulate further progress in the search for bio-plausible back-prop.\nDisclaimer: I have not verified any proofs / derivations.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "FCnCZtXOUJG", "reviewer": "Reviewer_YMzn", "summary": "Equilibrium propagation is a promising framework to learn neural networks in a biologically plausible way. However, it is sensitive to noise, which makes it hard to scale to large machine learning tasks, and requires two phases, limiting its interpretability as a biologically plausible learning framework. The authors solve those two problems by introducing a new variant of equilibrium propagation that relies on complex analysis and holomorphic functions, and derive an online unbiased version of the corresponding learning rule. They show that this new algorithm is more robust to noise than traditional equilibrium propagation and scales to large vision classification tasks such as ImageNet.", "questions": "The suggestions below aim at improving the main weakness of the paper:\n\n- It is hard to understand how $s_\\beta^*$ is computed when $\\beta$ is a complex variable. A naive interpretation would be that both the real and imaginary parts of $\\partial_s F$ have to be 0. However, this does not require $s$ to be a complex variable, which seems to be the case in the paper (cf. Eq. 3 or Fig.2.a). More details on how $F$ is defined, how $s^*_\\beta$ is computed, and which signal the real and imaginary parts of $s$ encode would be very valuable.\n\n- In the paper and in the appendix, the authors assume that $\\partial_s F$ is a holomorphic function. How big of an assumption is this? Do we have it for free by construction or are there some additional assumptions compared to the real version of equilibrium propagation?\n\n- Throughout the paper, a basic understanding of complex analysis is assumed. A short background section in the appendix containing the tools (holomorphic function, Cauchy-Riemann equation, holomorphic IFT...) needed to appreciate the theoretical contributions of the paper would be useful for the reader unfamiliar with the topic.\n\nMinor points:\n\nSection 3 - Theoretical results\n- line 122: \"The proof is an extension of the one provided for the real nudging case [10]\"\n\n- In Equation 3, are the * superscript missing on $s_\\beta$? the notation $s_\\beta(\\theta, s_\\beta, \\beta)$ is quite misleading, can you clarify it?\n\n- For the Cauchy integral of equation 5 you need the implicit function $s_\\beta^*$ to be defined on the path $\\gamma$, which is not in the neighborhood of 0. So Theorem 1 should be more general and ensure that the IFT conditions are defined on the entire path. Can you make this point clearer, either in the main text or in the appendix?\n\nSection 4 - Experiments\n- In Table 1, the comparison between classic EP and holo EP is not fair in the noiseless case: cEP uses 200 + 1x100 steps per gradient when hEP uses 200 + 10x100. Increasing the total number of steps of cEP to 1200 (and decreasing $\\beta$ to benefit from this extra number of steps) would make the comparison fairer.\n\n- In Table 1, the cEP estimate used in the noisy regime uses 1 point when $\\beta$ is zero and 10 for non-zero $\\beta$. This results in a very biased negative update but unbiased positive update. One other strategy would be to use 5 points for the two phases to reduce this \"bias imbalance\". I would suspect this to improve the quality of the cEP estimate in the noisy regime.\n\n- Figure 3.a) and 4.a) plot \"cosine dissimilarity\" when Figure 2.d) and 3.c) use \"cosine similarity\". Using the same measure would make navigation in the figures easier.\n\n- Legend Figure 3.a), are $T_\\text{dyn}$ and $T_\\text{plas}$ fixed? If so, to which values?\n\n- Line 236-238: refer to Zucchet et al. 2021, which theoretically analyzed this behavior?\n\n- It is unclear to me how important is the choice of the dSiLU activation function. On one side, it seems necessary theoretically to have holomorphic dynamics. On the other, it seems to allow for a more flexible choice of $\\beta$ (see Fig. 7), which could suggest that it makes convergence to equilibrium faster. I’m therefore wondering how crucial is this activation for the good empirical results displayed in the paper.\n\nAppendix\n- Equation (12): with the current brackets it is not directly that the product is 0 because $\\partial_s F$ and $\\partial_{\\bar{s}} F$ are. Moving them under the partials would make the proof easier to read.", "limitations": "I found the discussion of the limitations of the paper adequate.", "rating": 9, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ Significant theoretical advances on equilibrium propagation\n+ Careful empirical study of the impact of the different parameters ($N$, $\\beta$) on the quality of the gradient approximation\n+ Strong empirical results (robustness to noise, scale to ImageNet) while being arguably more plausible than previous equilibrium propagation versions\n+ The paper is overall very well written\n-  (Weakness) A detailed description of the algorithm is lacking, along with an interpretation in terms of neural dynamics", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "5k8p0jGShhQ", "reviewer": "Reviewer_mSnz", "summary": "The authors begin by showing that the classic equilibrium propagation can be extended to the case where the network layers are holomorphic dynamical systems. The essential result which follows is that the gradient with respect to parameters can be transformed to a contour integral on a circle around zero, which is equivalent to the first Fourier coefficient of the derivative of the nudged energy function with respect to network parameters. The central claim is that this equivalence eliminates the requirement that the \"nudge size\" go to zero. The authors define a straightforward numerical approximation to the gradients (equivalent to discrete Fourier transform), and show that the bias of this approximation goes to 0 as the sample rate of the Fourier transform goes to infinity.\n\nThe authors' toy experiment shows that holomorphic EP is capable of learning and that the cosine similarity of the approximated gradients is significantly higher than with classic EP.\n\nThe authors' theoretical investigations also reveal that holomorphic EP can be adapted to an online learning algorithm, assuming that the time scale of the layer dynamics is much less than the time scale of the oscillations computing the Fourier transform, which is in turn much less than the time scale of the weight updates. The authors validate this result with the MNIST task and show that the online approximation of the gradients are still significantly more accurate than the classic EP algorithm. They also compare classic, homlomorphic, and online homlomorphic EP on the MNIST task and claim that holomorphic EP has greater robustness to noise.\n\nFinally, the authors validate their algorithm by adapting a CNN architecture to be holomorphic and showing that the holomorphic EP training yields virtually the same results as BP training on a standard CNN.", "questions": "A major numerical drawback of holomorphic functions is the presence of poles and unbounded behavior. Both the sigmoid and dSilU activation functions will have poles where ever the exponential function takes the value -1. Why do these poles not cause any kind of instability in the numerical implementation?\n\nSince the learning algorithm can be adapted to be online, it was a bit of a disappointment to not see any time-domain tasks. The authors mention this as a limitation in their discussion. However, why would the algorithm still not work if the time scale of the task was much slower than the time scale $T_{dyn}$ of the layer dynamics? Was this ever tested?\n\n", "limitations": "The authors adequately addressed potential societal impact and some of the limitations of their work. However, only in the final appendix did I find a note that holomorphic EP was *much* slower at learning than BP. Especially since the authors mention edge and IoT devices in their discussion, I think a discussion of computational cost should also be moved to the discussion. Just how much longer should we expect holomorphic EP to take to train and evaluate? Was it only for ImageNet, and why would that be the case? It was not made clear whether or not 64 bits was required for each real and imaginary part or for the entire complex number, this is an important factor which would certainly affect performance.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper presents a very novel idea which greatly increases the power of a local learning rule using the elegance of complex analysis.\n\nThe proof of theorem 1 in the main text could be presented more clearly. It seemed as though the authors were implying that the logic of the complex chain rule needed to be applied to $\\theta$ as well as $\\beta$. The proof in the appendix is much more clear, I think that one thing the authors could do to improve presentation in the main text would be to denote $s_{\\theta,\\beta}$ instead of $s_{\\beta}$.\n\nThe derivation of the weight gradients presented was only with respect to a mean-squared error loss. However, for the computer vision tasks I would assume cross entropy loss would be required. Was an equivalent version of Eq. 8 derived for cross entropy anywhere? Eq. 8 seemed out of place since the main tasks are classification. If I understood correctly, Eq. 8 is the basis for the derivation of the online learning algorithm, is the use of MSE in the online learning essential or inconsequential? It would be strange if the online training of MNIST used MSE. Was the JAX autodiff engine used to compute the gradient of the nudged energy function with respect to parameters, in the case where MSE was not used?\n\nThe toy task was not adequately explained in the main text, what exactly are the inputs and targets? The way it sounded was that the network is given noise as input and is supposed to output a single value. This is very trivial, even for a toy task. Were there multiple Gaussians? Was the task to classify which samples came from which distribution? How many classes were there?\n\nAnother minor clarity issue is the labelling of Figures 2d and 3c versus 4a and S6. Why did the authors choose to present 1 - cos similarity instead of just the similarity itself? This made the plots look very surprising at first.\n\nOverall, this is a very interesting paper which I would very strongly accept on the condition that its clarity issues are fixed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "38nUcdkAxb", "reviewer": "Reviewer_EuCr", "summary": "The authors present `holomorphic equilibrium propagation' (holomorphic EP), an extension of equilibrium propagation for networks with holomorphic activation functions, which estimates the gradient of the loss using an approximation of the complex Cauchy integral. This avoids the need for an infinitesimal teaching signal, making the method more robust to noise.", "questions": "For someone who is not an expert, Figure 2 was challenging for me to interpret. Perhaps the figures can be connected more directly to the equations?\n\nMinor questions/comments:\n\nLine 143: it's not clear to me why $\\text{d}\\beta=(2\\text{i}\\pi\\beta/T)\\text{d}t$. Is there a $e^{2\\text{i}\\pi t/T}$ term missing?\n\nLine 210: The NeurIPS 2020 paper \"A biologically plausible neural network for slow feature analysis\" may be relevant here.\n\nLine 216: should \"discreetly\" be \"discretely\"?", "limitations": "As stated above, it is not clear to what extend the method of holomorphic equilibrium propagation is useful for understanding neural computation. The authors address this to some degree in the discussion, but not in detail. I think this is fine for this paper, but it is an important potential limitation.", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nI think the authors have presented a beautiful extension of equilibrium propagation for energy-based models with holomorphic activation functions. I found their presentation to be clear and as far as I can tell, their work is original (though I am not an expert in this area). The problem of solving the credit assignment problem using local learning rules is an important challenge in neuroscience and I believe the authors have put forth an interesting and elegant solution.\n\nWeaknesses\n\nDespite the elegance of the authors' solution, it is not clear if the results here are actually useful for understanding neural computation, which is in part the motivation for studying local learning rules. However, I think it is reasonable to leave such considerations for future work.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept