Energy-based learning algorithms for analog computing: a comparative study
Authors: Benjamin Scellier
Maxence Ernoult
Jack Kendall
Suhas Kumar
Venue: NeurIPS 2023 poster (2023)
Keywords: energy-based learning algorithm
contrastive learning
equilibrium propagation
coupled learning
convolutional Hopfield network
TL;DR: Equilibrium propagation is more effective than contrastive Hebbian learning and coupled learning in both theory and practice. Moreover, negative perturbations are essential for optimal performance.

Abstract: Energy-based learning algorithms have recently gained a surge of interest due to their compatibility with analog (post-digital) hardware. Existing algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL), all consisting in contrasting two states, and differing in the type of perturbation used to obtain the second state from the first one. However, these algorithms have never been explicitly compared on equal footing with same models and datasets, making it difficult to assess their scalability and decide which one to select in practice. In this work, we carry out a comparison of seven learning algorithms, namely CL and different variants of EP and CpL depending on the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100). We find that, while all algorithms yield comparable performance on MNIST, important differences in performance arise as the difficulty of the task increases. Our key findings reveal that negative perturbations are better than positive ones, and highlight the centered variant of EP (which uses two perturbations of opposite sign) as the best-performing algorithm. We also endorse these findings with theoretical arguments. Additionally, we establish new SOTA results with DCHNs on all five datasets, both in performance and speed. In particular, our DCHN simulations are 13.5 times faster with respect to Laborieux et al. (2021), which we achieve thanks to the use of a novel energy minimisation algorithm based on asynchronous updates, combined with reduced precision (16 bits).

Energy-Based Learning Algorithms for Analog Computing: A Comparative Study
Benjamin Scellier Rain AI benjaminOrain.at ai
Maxence Ernoult
Jack Kendall Rain AI jack@rain ai
Suhas Kumar Rain Al suhas@rain. ai
Rain AI
maxencedrain.e ai
Abstract
Energy-based learning algorithms have recently gained surge of interest due their compatibility with analog (post-digital) hardware. Existing algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learn- ing (CpL), all consisting in contrasting two states, and differing in the type of perturbation used to obtain the second state from the first one. However, these algorithms have never been explicitly compared on equal footing with same models and datasets, making it difficult to assess their scalability and decide which one to select in practice. algorithms, namely CL and different variants of EP and CpL. depending the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) five vision tasks (MNIST, F-MNIST, SVHN, CIFAR and CIFAR-100), We find that, while all algorithms yield com- MNIST, important differences in performance arise the
this work, we carry out a comparison seven learning
parable performance difficulty of the task increases. Our key findings reveal that negative perturbations better than positive ones, and highlight the centered variant of EP (which uses two perturbations of opposite sign) as the best best-performing algorithm. We also endorse these findings with theoretical arguments. Additionally, we establish new SOTA results with DCHNS on all five datasets, both in performance and speed. In particular, DCHN simulations et al.] (202T), which we achieve thanks to the use novel energy minimisation algorithm based on asynchronous updates, combined with reduced precision (16 bits).
13.5 times faster with respect Laborieux
Introduction
Prior to the dominance of backpropagation-bassee machine learning, Hopfield, Hinton and others proposed an alternative ennegy--bsed" learning (EBL) approach [Hopfield] 1984 Hinton et al. 1984). In this approach, the learning model is described by state variable whose dynamics is governed by an energy function. An EBL model can be used to compute as follows: 1) clamp the model input, 2) let the model settle equilibrium (that is, the configuration of lowest energy), and 3) read the model output. The objective is to modify the weights of the model so that it computes a desired input-to-output function. This achieved thanks to EBL algorithm. One of the earliest EBL algorithms was constrastive learning (CI.) (ovelllnn) 1991 Baldi and Pineda 1991]. which adjusts the model weights by contrasting two states: "free" state where the model outputs are free, and perturbed (or 'clamped') ) state where the model outputs are clamped to desired values. In the machine learning literature, interest in EBL algorithms has remained limited due the widespread success of backpropagation running on graphics processing units (GPUs). However, EBL algorithms have more recently revived interest as promising learning framework for analog learning machines [Kendall
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

et al., 2020 Stern et al. 2021]. The benefit of these algorithms is that they use single computational circuit or network for both inference and training. and they rely on local learning rules (i.e. the weight updates are local). The locality of computation and learning makes these algorithms attractive for
training adaptive physical systems in general [Stern and Murugan [023], and for building energy- efficient analog Al in particular (Kendall et al., 2020]] Small-scale EBL -BL-iiii variable resistor networks have already been built [Dillavou et al. 2022 2023 Yi et al. 2023), projecting a possible x improvement in energy efficiency compared to GPU-based training of deep neural networks
Yi al. 2023
In recent years, various EBL algorithms have been proposed, such as equilibrium propagation (EP) [Scellier and Bengio, (00n], the centered variant of EP [Laborieux et al. [021] and coupled learning (CpL) [Stern et al., 202T]. These algorithms, which are variants of CL with modified perturbation methods, are often evaluated on different models and different datasets without being compared to CL. and to one another Due to the lack of explicit comparison between these algorithms, and since the algorithmic differences between them are small, they are often gathered under the "contrastive learning' (or "contrastive Hebbian learning') umbrella name (Dillavou et al., 2022 Stern and Murugan 2023 Peterson and Lavin 2022 Lillicrap et al. 2020 Luczak et al. 2022 Høier et al. 2023]. Consequently, many recent follow-up works in energy based learning indifferent pick one of these algorithms without considering alternatives Dillavou et al. 2022 Wycoff et al., 2022 Stern
let al. 2022, Kiraz et al., 2022 Watfa al.) [023 Yi et al. 2023] Dillavou et al. [203] Altman et al. 2023]. The main contribution of the present work is to provide an explicit comparison of the above- mentioned EBL algorithms and highlight the important differences arising when the difficulty of the task increases. Nonetheless, comparing these algorithms comes with another challenge: simulations typically very slow. Due this slowness, EBL algorithms have often been used train small networks on small datasets (by deep learning standards). F Likewise, experimental realizations of EBL algorithms on analog hardware ("physical learning machines") have thus far been performed on small systems only.
In this work, we conduct study to compare seven EBL algorithms, including the four above- mentioned and three new ones. Depending on the sign of the perturbation, we distinguish between "positively-peretie perturbed' (P-). 'negatively -eetturred (N-) and 'centered' (C-) algorithms. To avoid the problem of slow simulations of analog circuits, we conduct our comparative study on deep convolutional Hopfield networks (DCHNs)(Eman)
et al.] 2019]. energy-basse network that has been emulated on Ising machines [Laydevant et al. [003]] and for which simulations are faster and previously demonstrated scale to tasks such CIFAR- 10 [Laborieux et al., [021] and Imagenet 32x32 [Laborieux and Zenke 2022 Our contributions include the following:
We train DCHNs with each of the seven EBL algorithms on five vision tasks: MNIST, MNShI, SVHN, CIFAR-IO and CIFAR-100 100. We find that all these algorithms perform well on MNIST, but as the difficulty of the task increases, important behavioural differences start emerge. Perhaps counter-intuitivlly we find that N-type algorithms outperform P-type algorithms by a large margin on most tasks. The C-EP algorithm emerges as the best performing one, outperforming the other six algorithms on the three hardest tasks (SVHN, CIFAR-10 and CIFAR-100).
We state novel theoretical results for EP, adapted from those of 'agnostic EP' (Scellier et al. ]022], that support our empirical findings. While EP is often presented an algorithm that approximates gradient descent on the cost function [Scellier and Bengio 2017 Laborieux al. 2022], J, we provide a more precise and stronger statement: EP performs (exact) gradient descent on surrogate function that approximates the (true) cost function. The surrogate
Foor example, Kendall et al. 20200 and Watfa et al. 2023 train layered nonlinear resistive networks using EPP while et al. [222]] train randomly-connectee elastic and flow networks using CpI..
Kendall et al. 2020 use SPICE to simulate the training of a one-hidden-layer network (with 100 'hidden
nodes') ) on MNIST, which takes one week for only ten epochs of training. Watfa et al, [2023) train resistive
networks with EP on Fashion-MNIST. Wine, and Iris. Similarly, [Stern al. [2021] simulate the training of
disordered networks of up to 2048 nodes on a subset of 200 images the MNIST dataset.
"With some exceptions, e.g. Laborieux and Zenke 2022 perform simulations of energy-based based convolutional
networks (DCHNs) on ImageNet 32x32.
Dillavou et al. 2022) train a resistive network of 9 nodes and 16 edges on the IRIS dataset. Yi et al. 2023
train a 24-33-7 memristive network on a 64 x 64 array of memristors to classify Braille words. Laydevant et al.
2023
train 784- 784-120-40 network (Ising machine) on subset of 1000 images of the MNIST dataset.
2

---

function of N-EP is an upper bound of the cost function, whereas the one of P-EP is lower bound (Theorem 2. Moreover, the surrogate function of C-EP approximates the true cost function at the second order in the perturbation strength (Theorem 30, whereas the ones of P-EP and N-EP are first-order approximations.
We achieve state-of-the-an DCHN simulations on all five datasets, both in terms of perfor- mance (aceuracy) and speed. For example, a full run of 100 epochs on CIFAR-I0 yields a test error rate of 10.4% and takes 3 hours 18 minutes this 13.5x faster than the simulations of Laborieux et al. [021] on the same hardware (a A100 GPU). With further training (300 epochs), the test be compared with 11.4% reported in Laborieux and Zenke 2022] Our simulation speedup is enabled thanks to the use of novel energy minimization procedure for DCHNs based on asynchronous updates and the use of 16 bit precision.
single A100 GPU show that
goes down to 9.7% to
We note that work also bears similarities with the line of works on energy-based models [Grath wohl et al. 2019 Nijkamp al., 2019 Du and Mordatch 2019 Geng et al. [021]. Like our approach, these studies involve the minimization of an energy function within the activation space (or input space) of a network. However, unlike our approach, they do not typically exclude the use of the backpropagation algorithm, employing not only to compute the parameter gradients, but also to execute gradient descent within the network's activation space (or input space). In contrast, the primary motivation of our work is to eliminate the need for backpropagation, and to perform inference and learning by leveraging locally computed quantities, with the long term goal building energy- efficient processors dedicated model optimization. Hence, motivation diverges significantly from traditional 'energy-baas -based models'.
Contrastive learning (CL)
Positively-perturbeed algorithms (P-EP, P-CpL)
energy function Ele. h.
energy function Ete.x
0.
G.
output variable
output variable
Negetivellyperttrbbed algorithms (N-EP, N-CpL.
Centered algorithms (C-EP, C-CpL)
energy function F(B. h.
energy function Ere. h.
0-.
Og output variable
O-g
output varlable
Figure 1: Cartoon illustrating the seven energy- learning (EBL) algorithms: contrastive learn- ing (CL), positively- -ertturre algorithms (P-), regatively-perturie algorithms (N-) and centered algorithms (C-). EP and CpL stand for equilibrium propagation and coupled learning, respectively. The desired output is The model prediction is o, i.c. the output configuration minimizing the energy function. The strength of the perturbation is B. A positive perturbation pulls the model output (os) towards A negative perturbation pushes the model output (0_.) away from y. Arrows indicate the weight update: green (resp. red) arrows decrease (resp. increase) the energy value of the corresponding configuration.

---

2 Energy-based learning algorithms
This section introduces the concepts and notations, presents the seven energy-based learning algo- rithms and states the theoretical results.
We consider the setting of image classification. In this setting, energy-based learning (EBL) model composed of an input variable (I), a parameter variable (#) a hidden variable (h) and an output variable (o). A scalar function E called energy function assigns each tuple (6, h, o) a real number E(0, z, h, o). Given 8 and x, among all possible configurations (h, o), the effective configuration of the model is the equilibrium state (or steady state), denoted (h,,,,) 0+) which is implicitly defined minimum of the energy function,
o.,, arg min E(0, h)
(h,o)
The equilibrium value (or steady state value) of output variables, 0., represents prediction of the model, which we also denote o(0, æ²) = o. to emphasize that it depends on the input æ and the model parameter 0.
The goal of training an EBL model is to adjust & so that, for any input æ, the output o0,, x) coincides with a desired output y (the label associated to z). We refer to an algorithm for training an EBL model as an EBL algorithm.
Remark. In the terminology of Stern and Murugan (2023). h and o are the 'physical degrees of freedom', 0 is the set of 'learning degrees of freedom' and E is the 'physical cost function'.
2.1 Contrastive learning
Contrastive learning (CL) is the earliest EBL algorithm [Movellan] 1991]. The CL algorithm proceeds in two phases. In the first phase, input variables are clamped, while the hidden and output variables are free to stabilize to the energy minimum (/,,,, o.) as in (T. In the second phase, the output variables now also clamped to the desired output y, and the hidden variables h are free to stabilize to second energy minimum, denoted h CL. characterized by
h == arg min E x, h, y).
(2)
The contrastive learning rule for the model parameters reads
O (0, h4 h,,
aE
(0, :y) w)
A C 17 00
(3)
0e
where 1j a learning rate. The CL rule is illustrated in Figure
2.2 Equilibrium propagation
Equilibrium propagation (EP) is another EBL algorithm [Scellier and Bengio 2017). One notable difference in EP is that one explicitly introduces cost function Clo, y) that represents the discrepancy between the output o and desired output In its original formulation, EP is variant of CL which also consists of contrasting two states. In the first phase, similar to CL, imput variables are clamped while hidden and output variables are free to settle to the free state (h,, o,) characterized by (I). In the second phase, in contrast with CL, EP proceeds by only perturbing (or nudging) the output variables rather than clamping them. This is achieved by augmenting the model's energy by a term 3C(o, v), where 3 E is a scalar - the nudging parameter. The model settles another equilibrium state, the perturbed state, characterized by
EP EP = arg min [E(0, x. h, (C(o.))) (h og
(hao)
In its classic form, the learning rule of EP is similar to CL:
EP 0
(0, h4 o.) 09 (0,
aE
hg 00
EP EP
(5)
09
The EP learning rule 5 comes in two variants depending on the sign of 3. Scellier and Bengio 2017, EP is introduced in the variant with 3 > 0, which we call here positively- EP (P-EP).

---

In this work, we introduce the variant with 8 < 0. which we call prgeatively-perturbe EP (N-EP), We also consider the variant of EP introduced by Laborieux
[2021] whose learning rule reads
C- C-EP
"
jE 00
00 r. h p.
EP) 2,r.hap
OE
(0, )
(6)
23
de
which we call centered EP (C-EP). The EP learning rules are illustrated in Figure]
2.3 Coupled learning
Coupled learning (CpL) is another variant contrastive learning [Stern et al. 2021] In the first phase, similar to CL and EP, input variables are clamped, while hidden and output variables are free to settle to their free state value (h,, o,) characterized by (T). In the second phase, output variables clamped to weighted mean of and y. and the hidden variables are allowed to settle to their new equilibrium value. Mathematically, the new equilibrium state is characterized by the following formulas, where the weighted mean output (o Cpl. is parameterized by factor 3 E RR {0):
CpL arg min E(O, h, h, C argmin h
GOL (1 3)0, By.
Similarly to CL and EP, the learning rule of CpL reads
7 ( 3 00 (0, h, o.)
OF 20 CoL . .
A Cppg
(8)
In particular, for 1, one recovers the contrastive learning algorithm (CL). their original formulation, Sternn al. [2021] use 3 > 0 here we refer to this algorithm poxitively-eerturbee CpL (P-CpL). Similarly to EP, we also introduce negatively-pertut CpL (N-CpL, with B < 0) as well as centered CpL (C-CpL):
A C-Cpll
1 (E
(0,
CpL CpL. OE h , o
CpL (0. x, h .0
(9)
2/
00
Figure T depicts P-CpL, N-CpL and C-CpL.
2.4 Theoretical results
The EBL algorithms presented above have diffferent theoretical properties. We start with CL.
Theorem (Contrastive learning). The contrastive learning rule (3) performs one step of gradient descent on the so-called contrastive function LCU.
dLCL
CL = -1 00
(6, (0,x,0) U),
LC x. y) = F. E0,00 CL ,.) - F (0, 0.)-
(10)
Theorem [ is proved in Movellan 1991. However, it is not clear that the contrastive function LeL has the desirable properties of objective function from machine learning perspective.
The equilibrium propagation (EP) learning rules have better theoretical properties. EP often presented algorithm that approximates the gradient of the cost function, e.g. in Scellier and Bengio 2017]. Laborieux et al. (2021]. In this work, we provide more precise and stronger statement: EP computes the exact gradient of surrogate function that approximates the cost function. Moreover, N-EP, the surrogate function is an upper bound of the cost function (Theorem 2) and in C-EP, the surrogate function approximates the cost function at the second order in 3 (Theorem Theorems and are adapted from Scellier et al. 2022 see Appendix A for proofs.
$ Another variant EP proposed in Scellier Bengio [007 combines N-EP P-EP, where the sign of the nudging parameter (9 is chosen at random for each example in the training set. However, N-EP wasn't 't used and tested as is.
IIn 'agnostic equilibrium propagation' (AEP) SScellier et al. 2022 J. the function CEP proved be Lyapunov function for AEP, but AEP does perform exact gradient descent CRP In contrast, EP performs exact gradient descent CET, but L is not Lyapunov function for EP.
5

---

Theorem (Equilibrium propagation). There exists some function L such that the learning rule performs one step of gradient descent on L
3C3
(0, x.
(11)
00
The function CB is a lower bound of the 'true' cost function if B > 0 (P-EP), and an upper bound if (N-EP), i.e.
CEP (0, v) S C (o(0,=),1) S (=,(0,,,)), (0,
vB 0,
(12)
where o(0,z) is the free equilibrium value of output variables given 0 and as in (I Furthermore, L approximates the 'true' cost cost function up to 0(3) when B + 0,
CB (0, x. (o(0, 2), y) O(3).
(13)
Theorem 3 (Centered EP). There exists some function L B such that the learning rule (6)
performs one step of gradient descent on C
B:+88
0L 9:+ -9:
C-EEr EP
(0, x. y)-
(14)
00
Moreover, the function CEP
B approximates the 'true' cost function up to O(3²) when 3 0,
CEP 50+ x, C' (o(0, x). + O(3²).
(15)
Finally, the analysis of the coupled learning rules (P-CpL, N-CpL and C-CpL) is more complicated.
Stern et al. [2021] introduce the following function candidate loss function for coupled learning:
P.G 0o2 (0, x, (0,, (v- G(O,2,0) min E(O, h,
Cnt. (0, x. v) ()))
(16)
Stern et al. 2021 argue that the P-CpL rule ( optimizes both LCpt. and the squared error LMSE (,, x, y) == (o(0, x) - 3) Besides, Stern et al. 2022 2023) argue that the learning rule performs gradient descent
(2) CpL = 3
1 (E(0, COL .0g COL) E(O, x. o.))
(17)
In Appendix B we demonstrate that the coupled learning rules and 9 do not optimize CpL or (2)
LMSE, and do not perform gradient descent on C
3 Deep convolutional Hopfield networks
To compare the EBL algorithms presented in section we consider the EBL model of Ernoult et al. 2019), which we call deep convolutional Hopfield network (DCHN), We consider specifically the network architecture of Laborieux et al. 2021
Network architecture. The network has input layer, four hidden layers, and an output layer. Since we consider classification tasks, the output layer has units, where the number of categories for the task. Successive layers are interconnected by convolutional interactions with kemnll size 3x3, padding 1, and max pooling. Except for the last hidden layer and the output layer, which interconnected by dense interaction.
Energy funcciin. We denote the state the network s (so. $1, $2, $3, 81, St), where æ = so is the input layer, h = ($1, $2, $3) is the hidden variable and a is the output variable. The energy function of the network is
E(O,s) = Σ
|k 2 +
D 1=1
Exony (wk, Sk-1, Sk) + Eg Edenss (ws, 84, 8s) +
E (bg, Sk). (18)

---

where 0 = fuok by 1 k I 5] is the set of model parameters, and Ex Ex energy terms defined
Ex and Exin are
Exony == 5Kk P (wk- * Sk-1), Ek
Ex
E == -8 Wy Sk-1 Ebias == by. 8k-
(19)
Specifically, F.g is the energy function of a convolutional interaction between layers 1 and k. parameterized by the kernel we (the weights), where is the convolution operation, P is the max pooling operation, and o is the scalar product for pairs of tensors. bx is the energy of dense interaction between layers k - 1 and k, parameterized by the dim((x-1) x dim(s:) matrix is the energy term of by. the bias of layer k.
wg. Finally, Ex
Cost function. For the equilibrium propagation learning rules, we use the squared error cost function C(o, y) = l|o yl where 85 is the output layer and is the one-hot code of the label (associated image z).
State space. Given an input the steady state of the model is (h., 0.) = arg min E(O, x. h o),
(h.ojeS
where S is the state space, i.e. the space of configurations for the hidden and output variables. We use S = II: [0, 1]dim(ss) [gdim(ss), where [0, 1] is the closed interval of IR with bounds and and dim 84) is the number of units in layer k.
Energy minimization via asynchronous updates. In order to compute the steady states (free state and perturbed states) of the learning algorithms presented in section 2 we use novel procedure for DCHNs using "asynchronous updates' for the state variables, as opposed to the 'synchronous updates' used in other works [Frnoult et al. 2019 Laborieux et al., 2021 Laydevant et al. 2021 Laborieux and Zenke 2022]. This asynchronous procedure proceeds as follows: at every iteration, we first update the layers of even indices in parallel (the first half of the layers) and then we update the layers of odd indices (the other half the layers). Updating all the layers once (first the even layers, then the odd layers) constitutes one 'iteration'. We repeat many iterations necessary until convergence to the steady state. See Appendix [foor a detailed explanation of the asynchronous procedure, and for comparison with the synchronous procedure used in other works. Although do not have a proof of convergence of this asynchronous procedure for DCHNs when P is the max pooling operation. we find experimentally that it converges faster than the synchronous procedure (for which there is no proof of convergence anyway).
Simulations
4.1 Comparative study of EBL algorithms
Setup. We compare with simulations the seven energy-based learning (EBL) algorithms of section 2 on the deep convolutional Hopfield network (DCHN) of section To do this, we train DCHN on MNIST, Fashion-MNIST, SVHN, CIFAR- 10 and CIFAR- 100 using each of the seven EBL algorithms. We also compare the performance of these algorithms two baselines: recurrent backpropagation (RBP) and truncated backpropagation (TBP), detailed below. For each simulation, the DCHN is trained for 100 epochs. Each run performed on a single A100 GPU. A run on MNIST and MNSHionMNIST takes 3 hours 30 minutes run SVHN takes 4 hours 45 minutes and run on CIFAR- 10 and CIFAR- 100 takes 3 hours. All these simulations are performed with the same network using the same initialization scheme and the same hyperparameters. Details of the training experiments are provided in Appendix D and the results are reported in Table
First baseline: recurrent backpropagation (RBP). As the equilibrium state o, of EBL model is defined implicitly as the minimum of an energy function, the gradient of the cost C(o., y) can be computed via implicit differentiation, more specifically via algorithm called recurrent backpropagation [Almeida 1987 Pineda 1987].
Second baseline: truncated backpropagation (TBP). Our second baseline uses automatic differ-
entiation (i.e. backpropagation) as follows. First, we compute the free state (h,, o.). Then, starting
The code is available httpt/pp//gggttt....maaaaaarrrre ctt.:/i..ooaaaaaa
based Meadddeerning

---

from the free state (b).00) (h,, we perform K iterations of the fixed point dynamics we arrive at (hx, which also equal to the free state. We then compute the gradient of C'ox y) with respect to the parameters (f), without backpropagating through (ho, on). This is truncated version of backpropagation, where we only backpropagate through the last K iterations but not through the minimization of the energy function. This baseline is also used by Ernoult et al. [019
Table Results obtained by training the deep convolutional Hopfield network (DCHN) of Section with the EBL algorithms of Section 2 contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL). EP and CpL are tested in their positively-pertutee (N-) and centered (C-) variants. We also report two baselines: truncated backpropagation (TBP) and recurrent backpropagation (RBP). Train and Test refer to the training and test error rates, %. For each of these 45 experiments, perform three runs and report the mean values. See Appendix [ for the complete results with std values. The hyperparameters used for this study are reported in Table
(P-), pncgatively-perturie
MNIST
FashionMNIST
SVHN
CIFAR-10
CIFAR-
Test
Train
Test
Train
Test
Train
Test
Train
Test
Train
TBP
0.42
0.23
6.12
3.09
3.76
2.37
10.1
3.1
33.4
17.2
RBP
0.44
0.33
6.28
3.70
3.87
3.43
10.7
5.2
34.4
18.2
CL
0.61
2.39
10.10
15.49
6.1
15.8
31.4
45.2
71.4
88.6
P-EP
1.66
2.29
90.00
89.99
83.9
81.9
72.6
79.5
89.4
95.5
N-EP
0.42
0.19
6.22
3.87
80.4
81.1
11.9
6.2
44.7
40.1
C-EP
0.44
0.20
6.47
4.01
3.51
3.01
11.1
5.6
37.0
26.0
P-CpL. 0.66
0.59
64.70
65.31
40.1
50.8
46.9
57.7
77.9
91.0
N-CpL 0.50
0.88
6.86
6.27
80.4
81.1
13.5
10.2
51.9
50.6
C-CpL
0.44
0.38
6.91
5.29
4.23
5.05
14.9
14.6
46.5
37.9
We draw several lessons from Table
Algorithms perform alike on MNIST. Little difference observed the test performance of the algorithms on MNIST, ranging from 0.42% 0.66% test eиrоr rate for six of the seven EBL algorithms. This result confirms the current trend in the literature, often skewed towards simple tasks, which sometimes goes
all EBL algorithms as one and the same.
Weak positive perturbations perform worse than strong ones. P-EP fails on most datasets (Fashion-MNISTT, SVHN, CIFAR-10 and CIFAR-100). P-Cpl. (the other weakly vositively-perturiee algorithm) performs better than P-EP on all tasks, but also fails on CIFAR- 100 (91% training error). It is noteworthy that CL, which employs full clamping to the desired output (i.c. strong positive perturbation), performs better than these two algorithms on all tasks, sometimes by very large margin (Fashion-MNISS and SVHN).
Negatíve perturbations yield better results than positive ones. On Fashion-MNIST. CIFAR- 10 and CIFAR-100, algorithms employing negative perturbation (N-EP and N-CpL) perform significantly better than those employing positive perturbation (CL, P-EP and P-CpL). Theorem sheds light on why N-EP performs better than P-EP: N-EP optimizes an upper bound of the cost function, whereas P-EP optimizes lower bound. We note however that on SVHN, the results obtained with N-EP and N-CpL are much worse than CL in Appendix E we perform additional simulations where we change the weight initialization of the network find that N-EP and N-CpL generally perform much better than CL, P-EP and P-CpL, supporting our conclusion.
Two-sided perturbations (i.e. centered algorithms) yield better results than one-sided per- turbations. While little difference in performance between the centered (C-EP and C-CpL) and negatively-perturbee (N-EP and N-CpL) algorithms is observed CIFAR-I0, 10, the centered algorithms unlock training on SVHN and significantly improve the test erro rate CIFAR-100 (by 2 5.4%). Theorem sheds light why C-EP performs better than N-EP: the loss function EPP
MNIST, FashionMNIST and
8:+9 of C-EP better approximates the cost function (up to O(32)) than the loss function CEP of N-EP (up O(3)).

---

The EP perturbation method yields better results than the one of CpL. While little difference in performance is observed between C-EP and C-CpL on MNIST and FashionMNIST. C-EP significantly outperforms C-CpL. on CIFAR-10 and CIFAR-100, both in terms of training error rate (2 11 11.9% difference) and test error rate (2 3.8% difference). The same observations hold between N-EP and N-CpL.. We also note that the Cpl. learning rules seem to be less theoretically grounded than the EP learning rules, as detailed Appendix
4.2 State-of-the-art DCHN simulations (performance and speed)
The comparative study conducted in the previous subsection highlights C-EP as the best EBL algo- rithm among the seven algorithms considered in this work. Using C-EP, we then perform additional simulations on MNIST, CIFAR-10 10 and CIFAR- 100, where we optimize the hyperparameters of train- ing (weight initialization, initial learning rates, number of iterations, value of the nudging parameter and weight decay) to yield the best performance. We use the hyperparameters reported in Table 5 (Appendix D We report in Table Oorr fastest simulations (100 epochs) well the ones that yield the lowest test error rate (300 epochs), averaged over three runs each. We also compare our results with existing works on deep convolutional Hopfield networks (DCHNs).
Table 2:
We achieve state-of-the-arr results (both in terms of speed and accuracy) with C-EP-trained DCHNs on MNIST, CIFAR-IO and CIFAR-I00. The results are averaged over 3 runs, and compared with the existing literature on DCHNs. Topl (resp. Top5) refers to the test error rate for the Topl (resp. Top5) classification task. Wall-clock time (WCT) reported as HH:MM. The hyperparameters used for these simulations are reported in Table 5 (Appendix D)
MNIST
CIFAR-10
CIFAR-100
Top1
WCT
Top1
WCT
Topl
Top5
WCT
Ernoult et al.
2019
1.02
8:58
Laborieux cl al.
2021
11.68
N.A.
Laydevant et al. 2021
0.85
N.A.
13.78
120:00
Luczak el al.
2022
20.03
N.A.
Laborieux and Zenke
[2022
11.4
24:00
38.4
14.0
24:00
This work (100 epochs) This work (300 epochs)
0.44
3:30
10.40
3:18
34.2
14.2
3:02
9.70
9:54
31.6
14.1
9:04
Table shows that we achieve better simulation results than existing works on DCHNs all three datasets, both in terms of performance and speed. For instance, our 100-epoch simulations on
CIFAR-10 10 take 3 hours 18 minutes, which is 7 times faster than those reported in [Laborieux and
Zenke 2022 (1 day), and 36 times faster than those reported in Laydevant et al. [202]] (5 days), and CIFAR- 10 yield 9. 9,70% test error rate, which is significanlly lower than [Laborieux and Zenke [022] (11.4%). On CIFAR- 100, it is interesting to note that our 300-epoch
our 300-epoch simulations
simulations yield a significantly better Top-1 eтror rate than Laborieux and Zenke [2022 (31.6% vs 38.4%), but a slightly worse Top-5 error rate (14, % vs 14.0%): we speculate that this might be due our choice of using the mean-squared error instead of the cross-entropy loss used in Laborieux and
Zenke [2022]. Since no earlier work on DCHNs has performed simulations on MaSSion-MNIST and SVHN, our results reported in Table are state-of-the-an on these datasets as well.
Our important speedup comes from our novel minigyyziiiiiiiiinnn procedure based on "asynchronous updates". combined with 60 iterations at inference (free phase) and the use of 16-bit precision. In comparison, Laborieux et al. [2021] used "synchronous updates" with 250 iterations and 32-bit precision. We show in Appendix that these changes result in 13.5x speedup the same device (a A100 GPU) without degrading the performance (test error rate).
We also stress that there still exists an important gap between our SOTA DCHN results and SOTA computer vision models. For example, Dosoviiskiy et al. 20200 report 0.5% test error rate on CIFAR-10 10, and 5.45% top-1 test error rate CIFAR-100.

---

5
Conclusion
Our comparative study of energy-based learning (EBL) algorithms delivers a few key take-aways: I) simple tasks such as MNIST may suggest that all EBL algorithms work equally well, but more difficult tasks such as CIFAR-I00 magnify small algorithmic differences, 2) negative perturbations yield better results than positive ones, 3) two-sided (centered) perturbations perform better than one-sided perturbations, and 4) the perturbation technique of equilibrium propagation yields better results than the one of coupled learning. These findings highlight the centered variant of equilibrium propagation (C-EP) as the best EBL algorithm among those considered in the present work, outperforming the second-best algorithm on CIFAR- 1-000 (N-EP) by significant margin. Our results also challenge some common views. First, it is noteworthy that CL (which uses clamping to the desired output, L.e. strong positive perturbation) yields better results than P-CpI. and P-EP (which use weak positive perturbations): this is contrary to the prescription of Stern et al. [2021] to use small positive perturbations, and opens up questions in the strong perturbation regime [Meulemans et al. 2022) Second, it is interesting to note that CL [Movellan 1991 Baldi and Pineda 1991.. EP [Scellier and Bengio 2017] and CpL (Stern al. 2021] were originally introduced around the same intuition of bringing the model output values closer to the desired outputs the fact that the four best-performing algorithmss of our study (C-EP, N-EP, C-CpL and N-CpL) all require a negative perturbation suggests on the contrary that the crucial part in EBL algorithms is not to pull the model outputs towards the desired outputs, but to push away from the desired outputs. Third, Laborieux et al. 2021 explained the very poor performance of P-EP by it approximating the gradient of the cost function up to ((8), but our observation that N-EP considerably outperforms P-EP while also possessing a bias of order ((8) shows that this explanation is incomplete Theorem 2 provides a complementary explanation: P-EP optimizes a lower bound of the cost function, whereas N-EP optimizes an upper bound.
Our work also establishes new state-of-the-a thert results for deep convolutional Hopfield networks (DCHNs) on all five datasets, both in terms of performance (accuracy) and speed. In particular, thanks to the use of novel "asynchronous" energy serrgymmnimization procedure for DCHNs, we manage to reduce the number of iterations required to converge to equilibrium to 60 compared to 250 iterations used in Laborieux et al. [202T] Combined with the use of 16-bit precision (instead of 32-bit), this leads our simulations to be 13.5 times faster than those of Laborieux et al. [202T] when run on the same hardware (a A100 GPU).
While our theoretical and simulation results seem conclusive, we also stress some of the limiting aspects of our study. First, our comparative study of EBL algorithms was conducted only on the DCHN model; ; further studies will be required confirm whether or not the conclusions that we have drawn extend to broader class of models such as resistive networks [Kendall et al. (2200) and clastic networks (Stern et al. 2021]. Second, in our simulations of DCHNs, the performance of the different EBL methods depends on various hyperparameters such as the perturbation strength (3), the number of iterations performed to converge to steady state, the weight initialization scheme, the learning rates, the weight decay, etc. Since a different choice of these hyperparameters can lead to different performance, not excluded that a deeper empirical study could lead to slightly different conclusions.
Ultimately, while our work is concerned with simulations, EBL algorithms are most promising for training actual analog machines: we believe that our findings can inform the design of such hardware as well [Dillavou et al. 2022 2023 Yi et al., 2023] Altman al., 2023). Finally, we contend that our theoretical insights and experimental findings may also help better understand or improve novel
EBL algorithms [Anisetti et al., 2022 Laborieux and Zenke, 2022 Hexner 2023 Anisetti al.
2023 and more generally guide the design of better learning algorithms for livel optimization
problems [Zucchet and Sacramento [022]. including the training of Lagrangian systems [Kendall
2021 SSeellirr [202]] meta- meta-learning [Zucchet et al., [022] Park and Simeone| [2022]. direct loss
minimization [Song et al., 2016) and predictive coding [ [Whitington and Bogacz, 2019 Millidge
et al. 2022
Acknowledgments and Disclosure of Funding
The authors thank Mohammed Fouda, Ludmila Levkova, Nicolas Zucchet, Axel Laborieux, Nachi Stern, Sam Dillavou and Arvind Murugan for discussions. This work was funded by Rain AI. Rain AI has an interest in commercializing technologies based on brain-inspired learning algorithms.
10

---

References
L. B. Almeida. A learning rule for asynchronous perceptrons with feedback in a combinatorial environment. In Proceedings, 1st First International Conference on Neural Networks, volume pages 609-618. IEEE, 1987.
L. E. Altman, M. Stern, J. J. Liu, and D. J. Durian. Experimental demonstration of coupled learning in elastic networks. arXiv preprint arXiv:2311.00170 2023.
V. R. Anisetti, A. Kandala, B. Scellier, and J. Schwarz. Frequency propagation: Multi-mechanism learning in nonlinear physical networks. arXiv preprint arXiv:2208. 088-:282..... 2022.
V. R. Anisetti, B. Scellier, and J. M. Schwarz. Learning by non-interfering feedback chemical signaling in physical networks. Physical Review Research, 5(2):023024, 2023.
P. Baldi and F. Pineda. Contrastive learning and neural oscillations. Neural Computation, 3(4): 526-545, 1991.
S. Dillavou, M. Stem, J. Liu, and D. Durian. Demonstration of decentralized physics-driven learning. Physical Review Applied, 18(1):014040. 2022.
S. Dillavou, B. Beyer, M. Stern, M. Z. Miskin, A. J. Liu, and D. J. Durian. Circuits that train themselves: decentralized, physics-driven learning. Al and Optical Data Sciences IV, volume 12438, pages 115-117. SPIE, 2023.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Y. Du and I. Mordatch. Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems, 32, 2019.
M. Ernoult, J. Grollier, D. Querlioz, Y. Bengio, and B. Scellier. Updates of equilibrium prop match gradients of backprop through time in an mnn with static input. In Advances in Neural Information Processing Systems, pages 7079-7089, 2019.
C. Geng. J. Wang, Z. Gao, J. Frellsen, and S. Hauberg. Bounds all around: training energy-based models with bidirectional bounds. Advances in Neural Information Processing Systems, 34: 19821, 2021.
W. Grathwohl, K.-C. Wang. J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky. Your classifier secretly an energy based model and you should treat like one. arXiv preprint arXiv:1912.03263. 2019.
D. Hexner. Training precise stress patterns. Soft Matter, (9(11)22120-211 2126, 2023.
G. E. Hinton, T. J. Sejnowski, and D. H. Ackley. Boltzmann machines: Constraint satisfaction networks that learn. Carnegie- University, Department Computer Science Pittsburgh, PA, 1984.
R. Høier, D. Staudt, and C. Zach. Dual propagation: Accelerating contrastive hebbian learning with dyadic neurons. In Proceedings of the 40th International Conference Machine Learning, pages 13141-13156, 2023.
J. Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the national academy of sciences, 81(10)308883302, 1984.
Kendall. A gradient estimator for varying electrical networks with -ineir dissipation. arXiv preprint arXiv:2103.056:: 2021.
Kendall, Pantone, K. Manickavasagam, Bengio, and Scellier. Training end-to-end analog 2020.
neural networks with equilibrium propagation. arXiv preprint
11

---

F. Z. Kiraz, D.-K. G. Pham, and P. Desgreys. Impacts of Feedback Current Value and Learning Rate on Equilibrium Propagation Performance. 2022 20th IEEE Interregional NEWCAS Conference (NEWCAS), pages 519-523, Quebec City, Canada, June 2022. IEFF. doi: 10.1 N0.100//EWCAS5266. 2022.9842178. htipsttps:: attpp//ttteeaaaaaaaaeaaaaaaaaaaa
A. Krizhevsky, G. Hinton, et al. Learning multiple layers features from tiny images. 2009. URL https://www. toronto........... edweettoeeduueee wttp:///eaaainngeatttueeeee
2009-000000.000 pdf
A. Laborieux and F. Zenke. Holomorphic equilibrium propagation computes exact gradients through finite size oscillations. Advances in Neural Information Processing Systems, 35:12950-12963, 2022.
A. Laborieux, M. Ernoult, B. Scellier, Y. Bengio, J. Grollier, and D. Querlioz. Scaling equilibrium propagation to deep convnets by drastically reducing its gradient estimator bias. Frontiers in neuroscience, 15:129, 2021.
J. Laydevant, M. Emoult, D. Querlioz, and J. Grollier. Training dynamical binary neural networks with equilibrium propagation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4640-4649, 2021.
J. Laydevant, D. Markovic, and J. Grollier. Training an ising machine with equilibrium propagation. arXiv preprint arXiv:2305. 18321, 2023.
Y. LeCun, I.. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11):2278-22222 1998.
T. P. Lillicrap, A. Santoro, L. Marris, C. J. Akerman, and Hinton. Backpropagation and the brain. Nature Reviews Neuroscience, pages 1-12, 2020.
A. Luezak, B. L. McNaughton, and Y. Kubo. Neurons learn by predicting future activity. Nature Machine Intelligence, 4(1):22-72, 2022.
A. Meulemans, N. Zucchet, S. Kobayashi, J. Von Oswald, and J. Sacramento. The least-control principle for local learning equilibrium. Advances in Neural Information Processing Systems, 35:33603-33617, 2022.
B. Millidge, Y. Song, T. Salvatori, T. Lukasiewicz, and R. Bogacz. Backpropagation at the infinitesi- mal inference limit of svergy-basees models: Unifying predictive coding, equilibrium propagation, and contrastive hebbian learning. arXiv preprint 2XX::22006006229 2022.
J. R. Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connectionist Models, pages 10-17. Elsevier, 1991.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. 2011.
E. Nijkamp, M. Hill, S.-C. Zhu, and Y. N. Wu. Learning non-convergent non-persistent short-run meme toward energy based model. Advances in Neural Information Processing Systems, 32, 2019.
S. Park and O. Simeone. Predicting flat-fading channels via meta-learned closed-form linear filters and equilibrium propagation. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8817-8821 IEEE, 2022.
A. Paszke, S. Gross, S. Chintala, G. Chanan, Yang, De Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017.
E. Peterson and A. Lavin. Physical computing for materials acceleration platforms. Matter, 5(11): 3586-3596, 2022.
J. Pineda. Generalization of back- propagaoonn to recurrent neural networks, Physical review letters. 59(19)2222, 1987.
B. Scellier. deep learning theory for neural networks grounded in physics. PhD thesis, Université de Montréal, 2021.
12

---

B. Scellier and Y. Bengio. Equilibrium propagation: Bridging the gap between energy-basee models and backpropagation. Frontiers computational neuroscience, 11:24, 2017.
B. Scellier, S. Mishra, Y. Bengio, and Y. Ollivier. Agnostic physics-driven deep learning. arXiv preprint arXiv:2205. 2022.
Y. Song, A. Schwing, R. Urtasun, et al. Training deep neural networks via direct loss minimization. In International conference on machine learning, pages 2169-2177. PMLR, 2016.
M. Stern and A. Murugan. Learning without neurons in physical systems. Annual Review of
Condensed Matter Physics, 14:417-441, 2023.
M. Stern, D. Hexner, W. Rocks, and A. Liu. Supervised learning in physical networks: From machine learning to learning machines. Physical Review X, 11(2):221045, 2021.
M. Stern, S. Dillavou, M. Z. Miskin, J. Durian, and A. J. Liu. Physical learning beyond the quasistatic limit. Physical Review Research, 4(2):L022037, 2022.
M. Sterm, S. Dillavou, D. Jayaraman, J. Durian, and A. J. Liu. Physical learning of power-efficient solutions. arXiv preprint arXiv:2310.1043: 2023.
M. Watfa, A. Garcia-Ortiz and G. Sassatelli. Energy-based analog neural network framework. Frontiers in Computational Neuroscience, 17:1114651, 2023.
J. C. Whittington and R. Bogacz. Theories of error back-propagation in the brain. Trends in cognitive sciences, 23(5):235-250, 2019.
Wycoff, S. Dillavou, M. Stern, A. Liu, and D. Durian. Desynchronous learning a physics-driven learning network. The Journal of Chemical Physics, (56(14):14490))) 2022.
H. Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms. arXiv preprint 17Xiv:1708.07747 2017.
S.-i. Yi, J. D. Kendall, R. S. Williams, and S. Kumar. Activity-difference training of deep neural networks using memristor crossbars. Nature Electronics, 6(1):45-51, 2023.
N. Zucchet and J. Sacramento. Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation. Neural Computation, 34(12):2309-2) 2022.
N. Zucchet, S. Schug. J. Von Oswald, D. Zhao, and Ja A contrastive rule for meta-learning. Advances in Neural Information Processing Systems, 35:25921-25935 2022.
13

---

OF A Loss functions of the equilibrium propagation learning rules
In this appendix, we prove Theorems 2 and To do so, we state and prove Theorems 4 and 5 below, of which Theorems 2
and 3 are direct consequences.
In this appendix, the input and label y are fixed, we omit them in the notation for simplicity. For fixed 8, we define for any value of | the minimum the augmented energy when and o are free:
F(3,0) inf ( (0,0,1,,) BC(o))
(20)
h.o
We further define the contrastive functions (or 'surrogate functions')
C (0) F(3.0) F(3.8)-F(00.) F(0,0)
B R \ {0),
(21)
3
and
CEP B (0) = F(3,0) 0) - F)-3,0)
8> 0.
(22)
2
We also recall the definition of the steady state values of output variables:
o0) :- arg min min E(O, h, o).
(23)
h
Finally, we recall that the equilibrium propagation (EP) perturbed state 44 is defined and that the EP learning rules
EP EP arg min {E(8, BC(o)) (h
og
(24)
(h.o)
p and 6
read
EP 8
17
JE
(0, N o0)
EP
0E
(0, KE o0
BP)
(25)
B
00
00
and
0E
C- EP
(0,
EP
EP
0-3)
0
EP
(0, hg .03 EP)
(26)
23
00
00
partieular, BP = o() by definition.
Theorem 4. The derivative of the function 3 F)3,0) at 3 0 is equal C(o(O)). In particular, as B + O, we have
C (0) (()(()))=))))))))) ((3)
and
CEP (0) C(((() (
(27)
Furthermore, we have for any B > 0
EP (0) S C(o(O)) S (O).
(28)
Theorem 5. The equilibrium propagation learning rules satisfy
OCEP
JC 88+8 (0).
A EP 0 = 11 00
(0),
C-EP -1] 30
(29)
Theorem 4 is borrowed from (Sellier et 2022 where these statements are proved in the context of 'agnostic equilibrium propagation' Theorem is a new contribution. For clarity to ensure the manuscript self-contained, we provide here proofs for both Theorem 4 and Theorem
First, we introduce a new notation: we introduce the function
F (3, o) (0, 3C(().
(30)
Using this notation, for every 3 E R. the first order conditions for the steady state (h
EP
FPP) read:
OF
EP (8, 19.0.15
EP = O, do
(3,0, h o = 0,
EP
(31)
0h
and the function
20 rewrites
=F (3,0
EP EP
.03
(32)
14

---

Proof of Theorem 4
Using the chain rule of differentiation, we have
0F
OF
OT
EP
OF
8 EP
(3,0)
(3
(8,0.1g
h 0g
(B,0,
,
03
33
Dh
03
do
03
(33)
Therefore
0F 03
OF
(3, 0)
EP FP) C (o))) 03 (3,0, (8,0n1g ,0
(34)
Evaluating this expression at B 0, and using of =()) by definition, we get
OF 03
(0, 0) C (o()))
(35)
which is the first statement of Theorem Using a Taylor expansion of F(3, 0) around / = 0, we
have
F (3,0) = F(O, 0) + 3C (o(0) + 0(8*).
(36)
Subtracting F(0,0 0) on both sides and dividing by 3, we get
(0) F (3,0) F(0. 0) C (o(0)) + +0()),
(37)
3
which the first part of equation Similarly, we write Taylor expansions for F(B, 0) and F)-
1
F
F (3, F(0,0) BC (o(0))
82
(0, + 0(B9,
(38)
2
0.2
1 02
F (-330) F(O,0) 3C (o(0))
(0, + 0(8)). 32
(39)
2 0.2
Subtracting the second equation from the first one, and dividing by 28, we get
F (3,0) F(-3,6)
CP
(0)
C. (o(0) + 0(3))
(40)
233
which is the second part of equation 27
Next, we prove the upper bound and lower bound properties. To this end, we write for any IR,
F(3,0) inf (E(0, h. 0) 3C(()) h,o < inf (F h, o()) + 30 (o(O)) F(0,0) 8C (o(0)), h
(41)
(42)
where o(f) the free state value of output variables. Subtracting F(0, 0) on both sides we get
F(B,0) F(0,0) < BC(o(0))
(43)
Next we divide by B. For 3 > O, we get
F(3,0)-F((. C(o(6))
EP (0)
(44)
B
and for we get
CB CEP (0) F(3,0) - F(0,0)
2 C(o(0))-
(45)
Hence the result:
c (0) S C(o(0) s CEP (O),
v3 0.
(46)
Similarly, it can be shown more generally that the function B + CEP is non-increasing, or cquivalently that the function 8 + F(B, 0) is concave.
15

---

Proof of Theorem 5
First, using 21
we have
oF
OF
00 (3,0)
(0.0)
(47)
00
3
00
Similar to the proof of Theorem using the chain rule of differentiation, we have for every 3 E
R
OF
OF
OF
OF
00 (6, 0) (5, 0 hg
EP
EP
(8, 0.
EP h
EP
h
EP
EP
D0
B
Dh
00
do
B 3
00
(48)
EP Here again, we have used the first-order steady state conditions for a
and 03
EP Therefore
OF
OF
0E
(3, 0)
(3,0,15
EP
0g
EP) =
(0, h
EP EP
(49)
00
00
00
Injecting
49 in 47
we get
Oc FP
1 DE
dE
EP
EP
(0, ho .6 (P)
(50)
00
00
We conclude using the definitions of the equilibrium propagation learning rule ( We prove similarly the case of centered EP with Rc
(0).
16

---

B On the coupled learning rules
In this appendix, prove some theoretical results related to the coupled learning rules.
Recall the definition of the free state
(h., 0.) == arg min E o),
(51)
(h,o)
the definition of the perturbed state of coupled learning (Cpl.)
3 = arg min E(O, I. h, 03
Cpl.
Cpl.),
Cpt.
= 3)a + By.
(52)
3
h
and the definition of the cenativvely-errtuhhee Cpl. rule OE
A 0
(0,
N1, 0+)
DE
(0, n :0B ghi))
Cpt.
(53)
3
a0
Similar to the contrastive function feL of contrastive learning (T0) and the surrogate loss L of
equilibrium propagation 21). Stern et al. [022 define the contrast Cpl.
CpL
(2) 1 E(0, x, h3 .03
o E(0, 2, k.,,
(54)
B
and argue that the CpL. rule performs gradient descent on ( Besides, Stern al. 2021 argue that the CpL rule optimizes both the function
OºG
(1) CPL (0, xr v) = (v-0(0,x)) x)) 0o2 (0)))
G(0,2,0) = min E(O,, xr h, o)
(55)
and the squared error loss
(S x. v) = (o(0,x) -y)²
(56)
where (0,, x) is the free output value.
In this appendix, we show that the CpL algorithm does not perform gradient descent on Cpt. (Appendix [B.1]. We also show that, in general, the CpL rule optimizes neither the squared error CMSE. nor the function CepL. To this end, we present two examples: in the first one, the function fAEE increases after single step of the coupled learning rule (Appendix B.2 the second one, the function cpp increases (Appendix B.3
(2)
B.1
Coupled learning does not perform gradient descent on the contrast Cpt.
Stern et al. 2022 note that C is non-negative and achieves its minimum possible value (clp) 0) when o, y. which motivates them to minimize c(2) by gradient descent. However, show
here, the coupled learning rule does not perform gradient descent CpL Indeed, while the free state is characterized by
OE (0. 1.) =
OE
(6, x, h,, 0,) 0,
(57)
Oh
do
the perturbed state of Cpl. only satisfies
OE
(,, x, h
OpL OpL)
.0 0.
(58)
Th
The partial derivative
Cpt. 0E (0, xr h .0 CpI.) does not vanish in general, in contrast with the perturbed
Do
state of equilibrium propagation (EP), which satisfies (31). Thus, we have
d dB
OF
(0,2,,,,, o.)
(8, x. Nr. 0.)
(59)
00
aE
Oh. aE
do.
h., (0, h, 0.)
00 do (0, 1, 10, k.,
(60)
ah
00
=0
aE
(0, x, hr.
(61)
09
17

---

and
d
E (0, 20
.0
Cpt. cpt.)
.0g
0E
( I, hg 00
Cpt. Cpt.)
(62)
d0
00
C ah B
+ 0h
0F
0, a, B
CpL
CpL
dE
0,
hig
CpL
CpL
00
do
00
#0 in general
(63)
0E
11 .0g .ogit)
CpL
+ (1 3)
OE
0,
hg
CpL
00
CpL
0o.
00
do
00
#0 in general
(64)
As result, the gradient of L
CPL
(E
(0, x. 1 00
CpL Cpt.
0E
(0, h,)
(65)
00
30
00
+
1 - - 0E
0, x, 1r 3 00
Opt.
Cpt.
do.
(66)
B do
00
40 general
The last term does not vanish in general, unless B = 1. We note that the case 3 = corresponds to contrastive learning (CL, section 2.1 hence we have recovered Theorem [[ known since Movellan 1991].
Therefore, contrary CL that performs gradient descent on LCL and EP that performs gradient descent on LEP for any value of B, the CpL. rule does not perform gradient descent on 4(2) for C
8+1 1.
B.2
One step coupled learning may increase the squared error LMSE
Next, we show in an example that CpL. does not optimize LMSE. Let A be the 2 x 2 square matrix
-1 2
A =
(67)
-1 2
and b: IR2 >> RR2 the function defined by
1 0₂
b(01,0z) 02) = 01 + 20
0,,02 ER.
(68)
We define the energy function E: :R? x IRa + 12 by
E(O,0) :: ((0) (0- A(o-b(0))
(69)
For simplicity, there is no input (x) and no hidden variable (h) in our example. Since the matrix is symmetric positive defimite, the minimum of o + E(O, o) is achieved at the point = b(0), where E(0, o.) = 0. Next, let's assume that the 'desired output' is y == (0, 0) e IR.. The perturbed state of the CpL. algorithm is
CpL = (1 B)o, By (1 B)b((),
(70)
where 3 E R is the nudging parameter, and the P-CpL rule is
A (0, o.)
0E 0g (0, og gpL))
A Cpl g = B 0e
(71)
where 7) is the learning rate. Using 69 we calculate
dE
(0, 6'(0) A b(8))
(72)
18

---

In particular,
OE
(0, o.)
(73)
00
and
DE
00 (0, 10.00/00 CpL) = ('(0) T A
0B CpL b(0) = - (0) Ab(0),
(74)
where we have used (70) So the P-CpL. rule (71 rewrite B
A Cpllo = -nb( TAb(0).
(75)
Next, we define the squared error between the prediction and the 'desired output" (0,
MSE (0) = llo. - yl1 = [1(0)]1
(76)
To conclude, show that for 7) small enough, LMSE + A 0 LMSE(()) (0o) at the point 8 = (0, 0) independently of the value the nudging parameter 3 E R. To show this, first we calculate 8CMEE A Cpt.g. We have
80
ÔLMSE ((0)Tb(0),
(77)
00
so that
MLMS 00
A CpLg -bb(0) (ab(0)Tb' (') (0) Ab(0).
(78)
Let us calculate:
1 2 )
1
6'(0)=
(79)
and
1
0
1 2
) (
1
(0) A
(80)
2
1
2
-1 2
(81)
5
-1
-1
3
(82)
-3
8
Since the point 0s (0 we have
b(8w) (
(83)
it follows that
b(o)) 88 6' ((00))((6))) (0o) Ab(()) = -1
(84)
Therefore, at the point 0, (0,
A CpL. = 17.
(85)
a0
Finally,
aLMSE
(MSB(60 + CpLg) LMSE(A)) (0v)
A + O(II (0L0111 17 O(7²).
(86)
00
Thus, provided that the learning rate 1 > 0 is small enough, the loss value LMSE (fl + Ad) after one step of coupled learning is larger than LMSE (6v). This holds for any nudging value 3 E R.
We note that the learning rules of N-CpL and C-CpL are identical this example.
19

---

B.3 One step of coupled learning may increase c(1)
CpL
Finally, we show that CpL does not optimize Cpt. To this end, we consider a second example, where we define the energy function E: 50,5 lR
R by
4A E (0,5/4), E E R,
E 0) 2 - 10) - -0)2
(87)
2
The minimum of o E(0,0) o) is obtained at the point o+ = 0, where F( o,) = 0. As in the
previous example, we assume that the 'desired output' is y == 0, so the perturbed state of Cpl.
CpL (1 -3]0, + By = (1 3)0
(88)
and the CpL rule is
Cpt.g
7 00
(0,0.) 0.) 00
0E
00, oc)
(89)
3 08
We calculate
0E
(0, 0) (5 20 20) 0).
(90)
00
In partieular,
0E
00 (0, 0.) =0
(91)
and
OE 00 (0, (0,0g Cpl.
(5 - 20 20 (og CpL. - 0) =
(5-0-))-00))000000)) -5--0 + 2,0)330
(92)
so the CpL rule reads
A CppL = (5 40 + 2,38)0
(93)
Now, recall the definition of the function CpL of (16) In this example, it is equal to
E
CpL (0) 0+ 0o2
(0, o.) = (5 - -5--000²,
(94)
and its derivative (or 'gradient") is
(C = 100 1202.
(95)
At the point 0, = we have (c (Oo) = 2, and A CpL0 = =-1(1 + 28). This means that (C (6o) < 0, and Appo < 0 for any nudging value B > -1/2. Thus, for a small and any 3 > 11/2, the function c(), increases after a single step of the rule.
rule.
20

---

C
Energy minimization procedure for deep convolutional Hopfield networks via asynchronous updates
Prior works on deep convolutional Hopfield networks [Ernoult et al. 2019. Laborieux et al. 2021
Laydevant et al. 2021 Laborieux and Zenke [022] minimized the energy function (18) using
'synchronous update' scheme, where at every iteration, all the layers are updated synchronously. In
this work, we use an "asynchronous update' scheme.
Recall the form of the energy function of a deep convolutional Hopfield network (DCHN), given by (88) Ernoult et al. [2019] write the energy function in the form
E(O, s) - 1 $),,8),
1
(96)
where ||+|| = Eimn || 2? and t is the so-called 'primitive function', defined as
=
5
((6,s) = Σ Sk P (wuk+ * Sk-1) + 85 S3 + by 8k-
(97)
In this expression, s (sp. $1, 82, 83, $1, $3) is the state of the network,
sy is the input layer. 85 is the output variable, wa and by are the kernel
h = (#1, 82, 83; 84) is the hidden variable, (the weights) and bias of layer k, * the convolution operation, P is the max pooling operation, the scalar product for pairs of tensors, and 0 = {wx, by s h: S 5) is the set of model parameters.
Emoult et al. [2019] define the dynamics
((++1) = (30
0Φ 0s (0, ()))
(98)
where (.)) = max(0, min(: (. 1)) is the 'hard sigmoid function', which they assume to converge to a fixed point s, characterized by
( ④ Os
=.=0
( s.).
(99)
Importantly, the fixed point 8, is also a critical point of the energy function, 8 + E(O,,) 8). since aE 8s (0, 8.) = 8. 85 (0, s.) = O. This dynamics is what we call here "synchronous update' scheme, 5) are updated synchronously.
0
because every iteration (or 'time step') f, all the layers 83 (1
In this work, we use an 'asynchronous update scheme', where at each iteration, we first update the layers of even indices, and then we update the layers of odd indices. Denoting 8,, (s1, 85) the configuration of odd layers, and 8e = (82, 84) the configuration of even layers, the asynchronous update scheme reads
(t+1) 0o
04
(t) , s!()) )
(100)
Üs.
where a. is the activation function of odd layers, and
8 (++1) =
(0, (0.
(
(101)
as
where de the activation function of even layers. For the layers of index i such that < k: < the activation function is (-(-) max(0, min(-, 1)), i.c. the 'hard sigmoid function', and 03 is the identity (the 'linear activation function').
In this work, we use the max pooling operation P Pmax- Although for this choice of pooling operation we do not have a proof of convergence of either of the two schemes ('synchronous' and 'asyn- chronous"), we find experimentally that the asynchronous scheme converges faster. This allows us to reduce the number of iterations in the free and perturbed phases and thus get significant speedup, as explained in the next subsection.
In the case where P is the average (mean) pooling operation, however, decreases at each step of the asynchronous update scheme. Indeed, for the average pooling operation Paug. the primitive & is linear function of se, i.c. the energy E is of the form
prove that the energy
(s.)) = ||s.112
B. St + C.
(102)
21

---

for some constants Be and Ce that do not depend on Specifically
BB : Wk e * )
+ wk+1 * P-1 (skt1) + by
(103)
for every layer k. It is easily seen that the minimum of such function E(se) in IRdim(())) Rddi(())) obtained the point se Be. also easily seen that the minimum of Ess.) in [0, 1]dim(sa)
[0, 1]dim(sa) is obtained the point Se max(0, min min(Be, 1)). Similar expressions hold for the configuration of odd layers 8, that minimizes the energy function given the state of even layers fixed. Given that & is a linear function of 8o. we have that B. de,
eD ab (0, 8), and similarly B, as. (0, s).
Thus, we conclude that the energy function decreases at each step of the asynchronous update scheme (T00-(101).
For the max pooling operation used in this work, the primitive function O is not linear function of Se So, and the above argument no longer holds.
C.1 13.5x simulation speedup compared to Laborieux et al. [2021]
We compare in simulations settings:
1. In the first setting, we use the asynchronous update scheme with 60 iterations at inference, 20 iterations in the perturbed phases, and 16 bit precision. This is the setting of the simulations performed for the comparative study in the present work.
2. In the second setting, we use the synchronous update scheme with 250 iterations at inference, 30 iterations in the perturbed phase, and 32 bits precision. This (up to minor differences) the setting of the simulations of Laborieux et al. [2021].
We perform the simulations using the C-EP algorithm on CIFAR-10, and we use the hyperparameters reported as 'SOTA results on CIFAR10' in T We perform three runs in the first setting, and one run in the second setting, and we report the results Table 3 We see that the first setting is 13.5 times faster than the second setting, requiring only 3 hours and 18 minutes complete on a single A-100 GPU (compared to the 44 hours 29 minutes required in the second setting). Furthermore, the first setting also performs marginally better in terms of training and test error rates than the second one.
Table 3: Comparison of the performance and the wall-clock-tin time between the simulation setting of
Laborieux et al. [ and ours. The simulations Laborieux et al. [202T] the synchronous
update scheme, in conjunction with 32 bits precision, 250 iterations at inference (free phase), and
30 iterations in the perturbed phase. Our asynchronous update scheme is used in conjunction with 16 bits precision, 60 iterations at inference and 15 iterations in the perturbed phase. Simulations are performed on CIFAR-10 10 using the C-EP algorithm for training. We perform 100 epochs and use the hyperparameters of the 'SOTA CIFAR-I0 simulations' provided in Table We report the test error rate (Test Error), the training error rate (Train Error) and the wall-clock time (WCT). WCT is reported as HH:MM.
Test Error
Train Error
WCT
Synchronous - 32 bits 250 iterations (1 run) Asynchronous bits iterations (3 runs)
10.85%
3.58%
44:29
10.40 = 0.10%
3.48 + 0.14%
03:18
22

---

D Simulation details
Datasets. We perform simulations on the MNIST, Fashion-MNIST, SVHN, CIFAR-10 and CIFAR- 100 datasets.
The MNIST dataset is composed of images of handwritten digits [LeCun et al. 1998 Each image in the dataset is 28 x 28 gray-scaled image and comes with label y E {0, 1, .,. 9) indicating the digit that the image represents. The dataset contains 60,000 training images and 10,000 test images.
The Fashion-MNIST dataset [Xiao et al. 2017] shares the same image size, data format and the same structure of training and testing splits MNIST. It comprises training set of 60,000 images and test set of 10,000 images. Each example a 28 x 28 grayscale image from ten categories of fashion products.
The SVHN dataset [Ntterr al. 2011 contains color images 32 x 32 pixels, derived from house numbers in Google Street View images. Like in MNIST, each image comes with a label corresponding to digit from to 9. The number of images per class varies due to the natural distribution of digits in the real world. The dataset contains a training set with 73, 257 images, and test set containing 26, 032 images. The dataset also contains an additional set 531, 131 other (somewhat less difficult) images, which we do not use in our simulations.
The CIFAR-10 10 dataset (Krizhevsky al. 2009] consists of 60, 000 colour images of 32 32 pixels. These images are split in 10 classes (each corresponding to an object or animal), with 6, 000 images per class. The training set consists of 50, 000 images and the test set of 10, 000 images.
Similarly, the CIFAR- 100 dataset [Krizhessky et al., 2009) also comprises 60, 000 color images with resolution of 32 x 32 pixels, featuring a diverse set of objects and animals. These images categorized into 100 distinct classes, each containing 600 images. Like CIFAR-10, the dataset is divided into a training set with 50, 000 images and test set containing the remaining 10, 000 images.
Data pre-processing and data augmentation.
The data is normalized as Table
Since the deep convolutional Hopfield network used in our simulations takes as input 32x32-pixel image, we augment each image of MNIST and Fashion-MNIST to a 32x32-pixel image by adding two pixels at the top, two pixels the bottom, two pixels to the left and two pixels to the right.
On the training sets of flipping. (Random flipping is not used on the training sets of MNIST and SVHN, and it is never used at test time either.)
CIFAR-10 and CIFAR- we also use random horizontal
Table 4: Data normalization. We normalize the input images using the recommended mean (μ) and std (a) values for each dataset. The MNIST and Fashion-MNIST images are gray-scale, i.c. they have unique channel. The SVHN, CIFAR-10 10 and CIFAR- 100 images color images, i.c. they have three channels.
mean (())
std (a)
MNIST Pashion-MNIST
0.1307
0.3081
0.2860
0.3530
SVHN
(0.4377, 0.4438, 0.4728)
(0.1980, 0,2010, 0.1970)
CIFAR-10
(0.4914, 0.4822, 0.4465)
(3*0.2023,
1994, 3*022010) (0.2675, 0,2565, 0.2761)
CIFAR-100
(0.5071, 0.4867, 0.4408)
Network architecture and hyperparameters. Table contains the architectural details of the network, as well as the hyperparameters used to obtain the results presented in Table T Table 2 and T The hard-sigmoid activation function is defined as (h)) == max(0, min(h, 1))-
Weight initialization. according to
We initialize the weights of dense interactions and convolutional interactions
Wij ~ U(-c, +c),
c= a
(104)
fan
23

---

Table 5: The top table indicates the architecture of the deep convolutional Hopfield network (DCHN). We denote nin the number input filters (either or depending on the dataset) and Nout the number of output units (either 10 or 100 depending on the dataset). The bottom table indicates the hyper- syper-parameters used for initializing and training the network. Ir means learning rate. The first two columns (Comparative study) provide the values used for the extensive comparison of learning algorithms (Table and Table 6). The other columns (SOTA results) provide values used to obtain the best performances on MNIST, CIFAR-10 and CIFAR-100, after 100 epochs and 300 epochs (Table
Layer shapes
Weight shapes
Bias shapes
Activation functions
Layer 0 (inputs)
X32x33
Layer
128x16x16
(28xmmx3xx3
128
Hard-sigmoid
Layer 2
256x8x8
256x 256x128x3x3
256
Hard-sigmoid
Layer 3
512x4x4
512x256x3x33
512
Hard-sigmoid
Layer 4
512x2x2
512x512x3x3
512
Hard-sigmoid
Layer 5 (outputs)
Rout
512x2x2xMout
10
Identity
Comparative study
SOTA results (Table 2
Table
Table 6
MNIST
CIFAR-10
CIFAR-100
nudging (3)
0.25
0.25
0.1
0.25
num. iterations at inference (T)
60
60
60
60
num. iterations at training (K)
15
15
20
15
gain conv-weight (a.)
0.5
0.7
0.5
0.4
0.5
gain conv-weight (a2)
0.5
0.7
0.5
0.7
0.4
gain conv-weight (a:))
0.5
0.7
0.5
0.6
0.5
gain conv-weight (04)
0.5
0.7
0.5
0.3
0.8
gain dengh- 5 (oc)
0.5
0.7
0.5
0.4
0.5
Ir conv-weight & bias (7))
0,0625
0.0625
0.03
0.03
Ir conv-weight 2 & bias 2 (72)
0.0375
0.0375
0.03
0.04
conv-weight 3 & bias (7s)
0.025
0.025
0.03
0.04
Ir conv-weight 4 & bias 4 (77)
0.02
0.02
0.03
0.04
Ir dense-weight bias (775)
0.0125
0.0125
0.03
0.025
momentum
0.9
0.9
0.9
0.9
weight decay (1e-4)
3.0
3.0
2.5
3.5
mini-batch size
128
128
128
128
number of epochs
100
100
300
100
300
100
T maa Tmin (le-6)
100
100
300
100
300
100
2.0
2.0
2.0
2.0
which the 'Kaiming uniform' scheme rescaled by a factor a, that call the "gain' here (i.e. scaling number). For dense weights of shape (sizepre- sizepost). ) we have fan;, = size for convolutional weights shape (channelms channelout. height, width), we have fan; = channel,, x height x width. See Table for the choice of the gains.
Energy minimization via asynchronous updates. To compute the steady state of the network, we use the "asynchronous update' scheme described in appendix C at every iteration, we first update the layers of even indices (the first half the layers) and then we update the layers of odd indices (the other half of the layers). Relaxing all the layers once (first the even layers, then the odd layers) constitutes one 'iteration'. We repeat as many iterations as is necessary until convergence to the steady state. In T we denote T the number of iterations performed at inference (free phase), and we denote the number iterations performed in the perturbed phase.
Training procedure. We train our networks with seven energy based learning (EBL) algorithms, namely: contrastive learning (CL), positively-pertubbee equilibrium propagation (P-EP), negatively- perturbed EP (N-EP), centered EP (C-EP), positively-perturie
coupled learning (P-CpL.), negatively- perturbed CpL. (N-CpL.) and centered CpL. (C-Cpl.). Given an EBL. algorithm, each training step of SGD, proceed as follows. First we pick mini-batch of samples in the training set,

---

and their corresponding labels, y. Then we set the nudging parameter to 0 and we perform free phase of T asynchronous iterations. This phase allows in particular to measure the training loss and training error rate for the current mini-batch, to monitor training. We also store the free state (h,, 0.). Next, let's call 31 and 82 the two nudging values used for training (which depend on the EBL algorithm). First, we set the nudging parameter to 31 and we perform new relaxation phase of asynchronous iterations. Next, we reset the state of the network to the free state (h,, Ox), then we set the nudging parameter to the second nudging value 32, and we perform the last relaxation phase for asynchronous iterations. Finally, we update all the parameters simultaneously single 'parameter update' phase.
Optimizer and scheduler. We use mini-bateh gradient descent (SGD) with momentum and weight decay. We also use cosine-annealing scheduler for the learning rates with hyperparameters Tax and Mimn
Computational resources. The code for the simulations uses PyTorch 1.13.1 and Torch Vision 0.14.1. Paszke et al. [2017]. The simulations were carried on Nvidia A100 GPUs. For the comparative study, we used five GPUs (one GPU per dataset) and let them run for six days to complete all the simulations.
25

---

E Additional simulations on SVHN for the comparative study
The results of Table I show that most algorithms (P-EP, N-EP, P-CpL and N-CpL) perform very poorly on SVHN: the training process often gets stuck at 81.08% training error and 80.41% test error rate. These results seem go against our conclusion that N-EP and N-CpL. generally perform (much) better than C-EP and C-CpL and CL. We hypothesize that this poor performance on SVHN due to the network being poorly initialized for this classification task, and we perform additional simulations SVHN where we change the initialisation scheme of the weights of the network see T for the new hyperparameters used for these additional experiments. As for the comparative study of Section 4.1 with this new set of hyperparameters,
compare the seven EBL algorithms of section by training the DCHN of section 3 on SVHN. We also compare the performance these algorithms to recurrent backpropagation (RBP) and truncated backpropagation (TBP). The results of these additional simulations are reported in Table 6
Table [ shows that the N-EP and P-CpL. simulations have large variance, we also report the results for each run. The three runs of N-EP gave the following results: 3.73% test error, 2) 3.87% test error, and 3) 80.41% % test error. The third N-EP run was indeed also blocked at 81.08% training error and 80.41 test error (like many other runs). The three runs of P-CpL. gave the following results: I) 71.05% test error, 2) 15.35% test error, and 3) 15.57% test error.
The results of Table are consistent with our conclusions of Section 4.1 1) strong positive pertur- bations (CL) yield better results than weak positive perturbations (P-CpL and P-EP). 2) negative perturbations (N-EP, N-CpL) yield better results than positive perturbations (CL, C-CpL, C-EP), 3) two-sided perturbations (C-EP, C-CpL) perform better than one-sided perturbations (N-EP, N-CpL, CL, P-EP), and 4) the EP perturbation technique yields better results than the CpL perturbation technique.
Table 6: Additional experiments on SVHN using a different set of hyperparameters. We compare the seven EBL algorithms and benchmark them against truncated backpropagation (TBP) and recurrent backpropagation (RBP). We report the training and test error rates, averaged over three runs. The hyperparameteers used for this study are detailed in Table
SVHN
Test Error (%)
Train Error (%)
TBP
3.66 + 0.05
2.36 + 0.04
RBP
3.91 + 0.05
3.29 + 0.04
CL
9.64 1.59
27.08 = 5.68
P-EP
74.88 + 10.60
77.32 + 5.84
N-EP
49.34+36.11
29.01 36.82
C-EP
3.58 + 0.07
3.00 0.05
P-CpL.
26.211 33.99 + 26.21
47.28 26.52
N-Cpl.
4.53 + 0.05
5.27 0.05
C-Cpl.
4.10 + 0.04
4.68 0.10
In Table 7 we report the results of Table I together with the test std values.
26

---

Table 7: Results of Table reported with the test std values.
MNIST
FashionMNIST
SVHN
CIFAR-10
CIFAR-100
Test
Train
Test
Train
Test
Train
Test
Train
Test
Train
TBP
0.42=0.02
0.23
6.
3.09
3.76
2.37
10.1
3.1
33.4
17.2
RBP
0.44#0.01
0.33
6.28
3.70
3.87#0.09
3.43
10.7#0.1
5.2
34.4
18.2
CL
0.602
2.39
10.10
15.49
6.10+0.13
15.8
31.4+222
45.2
71.4
88.6
P-EP
1.66=0.80
2.29
90.00+0.00
89.98
83.9#1.9
81.9
72.6+0.7
79.5
89.4
95.5
N-EP
0.42 10.01
0.19
6.22 10.10
3.87
80.4=0.0
81.1
11.9+0.3
6.2
44.7+0.3
40.1
C-EP
0.44=0.03
0.20
6.47#0.13
4.01
3.51=0.07
3.01
11.1
5.6
37.0
26.0
P-CpL
0.60=0.13
0.59
64.70=35.77
65.31
40.1+28.6
50.8
46.9+3.8
57.7
77.9
91.0
N-CpL
0.50=0.04
0.88
6.86
6.27
80.4
81.1
13.5
10.2
51.941.3
50.6
C-CpL
0.44
0.38
6.91
5.29
4.23
5.05
14.9
14.6
46,5
37.9
27


Reviews:
[{"review_id": "G0phFfAray", "reviewer": "Reviewer_dBi8", "summary": "This work reviews and compares recent EBL methods on classic image classification benchmarks. The paper first reviews a variety of EBL methods including CL, EP, P-EP, N-EP, C-EP, CpL, P-CpL, N-CpL, and C-CpL. Next, two theorems are presented which show that P/N/C EP approximate gradient descent on the cost function. The experiment section presents a large-scale comparison of the EBL methods along with Backprop methods for image classification on image datasets whose complexity ranges from MNIST to CIFAR-100. The study finds that N-type EBL variants outperform P-type EBL variants, and that C-EP performs best overall. The study also finds that MNIST is not a suitable dataset for identifying differences between method outcomes, and that more complex datasets like CIFAR-100 reveal important differences. A discussion of possible explanations for the results is presented.", "questions": "* Can the ideas in this paper be used to propose a new EBL algorithm which draws from the useful properties of the best current models?", "limitations": "Limitations are not discussed, ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper is a very useful resource for understanding recent EBL methods.\n* Theorems 2 and 3 very nicely corroborate the empirical observations about the effectiveness of P-type over N-type, and the overall superiority of C-type.\n* A large-scale study of EBL methods gives crucial context for understanding which EBL techniques are useful and which are not. The findings could serve to motivation directions for developing improved EBL algorithms. ", "weaknesses": "* The primary weakness of this paper is a lack of novelty. It provides a very useful survey of the literature and a much-needed large scale comparison of EBL methods. However, no new techniques are proposed, and the experimental evaluation is somewhat routine.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We have addressed the novelty factor in the common rebuttal (see above). We would like to emphasize that our asynchronous update method to compute equilibrium states of DCHNs (i.e. to compute minima of the energy function) is novel, and that this technique unlocked significant speedup – indeed, a 13.5x speedup compared to Laborieux et al (2021). We refer to Appendix C in the supplementary material for details.\n\nWe have also clarified in the common rebuttal that NEP, NCpL and CCpL are novel, and that  these EBL algorithms are tested for the first time in our work. We hope that this answers the reviewer’s question about new EBL algorithms.\n\nWe have also explained in the common rebuttal that we will discuss the limitations of our comparative study (limited to DCHNs) in the conclusion section of our manuscript.\n\nFinally, since the reviewer appreciates the usefulness of our large-scale study of EBL methods, we would like to point out that the significant (13.5x) speedup enabled by our novel “asynchronous update method” is also the reason why our extensive comparative study was possible at all. We carried out 135 simulations (5 datasets * 9 algorithms * 3 runs) on five A100 GPUs for about one week, each run (100 epochs) taking between 3 and 5 hours. While not impossible in theory, without this speedup the same study conducted on 5 A100 GPUs would have taken 3 months to complete (instead of one week). Thus, our novel asynchronous update method (one of the essential novelties of our work) played an important role in making this study possible/affordable.\n"}, {"review_id": "12uENZ2Et0", "reviewer": "Reviewer_JWNX", "summary": "This work conducts an extensive comparison of several energy-based learning (EBL) algorithms, including contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL). Depending on the type of perturbation used, 9 variants of EP and CpL are examined. Deep Hopfield networks (DHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100) are trained, and different EBL algorithms are compared.", "questions": "see above.", "limitations": "see above.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Investigating energy-based learning algorithms is interesting, especially for their compatibility with analog (post-digital) hardware. The paper is generally well-written. The concepts and algorithms are mostly clearly presented. Experiments are detailed with good analysis.", "weaknesses": "Not clear that Theorem 1-3 are new, or known from existing literatures.\n\nThe paper is good for a comparison study of different EBL algorithms, but the algorithm contribution is limited.\n\nThe performance of EBL-trained DCHNs on vision tasks (e.g., CIFAR-10) are far behind modern neural networks. It would be better to add such information and comparison.\nhttps://paperswithcode.com/sota/image-classification-on-cifar-10", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Regarding the question on the novelty of the theorems, we would like to clarify that Theorem 1 isn’t new ; it is presented and proved e.g. in Movellan (1991).\nOn the other hand, Theorems 2 and 3 are new in the literature: no prior work had shown that EP (resp. CEP) performs gradient descent on the surrogate loss function L_beta (resp. L_{-beta,+beta}).\n\nWe have explained in the common rebuttal the algorithmic novelties in our work. While the network architecture (the deep convolutional Hopfield network, DCHN) isn’t novel, we introduce in our work three novel EBL algorithms (NEP, NCpL and CCpL, to compute the parameter gradients) and a novel energy minimization algorithm (the “asynchronous update method” to compute the equilibrium states of DCHNs). Importantly, our novel energy minimization algorithm leads to a 13.5x speedup compared to Laborieux et al. (2021).\n\nFinally, we agree that the performance of DCHNs on the tasks considered in this work lags behind that of SOTA deep learning models. We propose to report the results of SOTA algorithms for comparison, as suggested by the reviewer.\n"}, {"review_id": "7Pm0djdVZ8", "reviewer": "Reviewer_5nrK", "summary": "The paper focuses on exploring and comparing various energy-based training methods for deep convolutional Hopfield networks. The performance of contrastive learning, positively-perturbed, negatively-perturbed, and centered versions of Equilibrium propagation and Coupled learning algorithms are evaluated. The paper establishes state-of-the-art (SOTA) results for some of these algorithms.", "questions": "What factors motivated the authors to choose the Deep Convolutional Hopfield Network (DCHN) model as the framework for comparing the training methods in the paper?", "limitations": "The paper does not include a discussion on the limitations.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The set of experiments conducted in this paper provides a comprehensive comparison of the performance of different training methods on a variety of datasets. This extensive evaluation contributes to a robust understanding of the performance of deep convolutional Hopfield networks (DCHN) and their applicability across different datasets.", "weaknesses": "1) One issue with this paper is the lack of citation and discussion of recent works on Energy Based Models (EBMs) that are relevant to the topic, such as Grathwohl et al. (2019), Nijkamp et al. (2019b;a), and Du & Mordatch (2019). These papers analyze the application of EBMs to image classification tasks, propose different training techniques, and compare their performance. It would be beneficial to include a discussion of these works and how they relate to the research presented in this paper.\n\n2) Subsection 2.5, which presents the theoretical results, appears disconnected from the main body of the paper. There is no discussion of the implications of these theorems or how they relate to the research findings. It is important to establish a clear connection between this theoretical section and the rest of the paper to provide a cohesive narrative.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "First, we thank the reviewer for their remark on recent works on energy-based models (EBMs). We understand that the term “energy-based model” is ambiguous and refers to different lines of works with very different motivations, which we would like to clarify here. For clarity, we start by highlighting the common points between our work and the referenced works, and then we discuss the differences and implications of these differences.\n\nBoth in our work and in the referenced works, the model is defined by an energy function, which is a scalar function of the weights and activations of the network. Inference and training in these EBMs involve minimizing this energy function in the activation space. For instance, in Grathwohl et al (2020), they perform gradient descent in the input space:\n$$ x \\leftarrow x - \\alpha \\partial_x E(f_\\theta(x)) $$\nwhere $f_θ(x)$ represents the model logits given an input $x$. Similarly, in our work we minimize an energy function (the Hopfield energy) thanks to our “asynchronous update procedure” in the activation space.\n\nThere is, however, a crucial difference between our work and the referenced works. Our work is concerned with models in which inference and learning are achieved using local update rules, with the long-term motivation of building new, highly-efficient (low-power), hardware for AI. The whole purpose of our line of work is to totally obviate the use of the backpropagation algorithm and perform model optimization (inference and learning) by leveraging locally computed quantities, with the longer term goal in mind to design energy-efficient processors dedicated to model optimization.\nIn our work, the asynchronous update procedure to minimize the energy function of a Hopfield network reads:\n\n$$ s_i \\leftarrow \\sum_j w_{ij} s_j + b_i $$\n\nand the learning rule of CL/EP/CpL in such Hopfield networks reads:\n\n$$\\Delta w_{ij} \\propto \\left( s_i^{\\rm perturbed} s_j^{\\rm perturbed} -  s_i^{\\rm free} s_j^{\\rm free} \\right)$$\n\nImportantly, both the energy minimization procedure and the learning rule require solely locally available information to update the state variables $s_i$ and weights $w_{ij}$. This feature makes our model amenable to highly efficient implementation on dedicated (analog) hardware. Yi et al (2023) have shown that such Hopfield networks can be built in memristive networks (analog networks) and trained using 10,000x less energy compared to DNNs trained on GPUs.\n\nIn the referenced works, the motivation is very different from ours. For example, Grathwohl et al (2020) aim to scale EBM training to build large, well-calibrated and adversarially robust discriminative and generative models. Their algorithms do not preclude the use of the backpropagation algorithm, which they use not only for parameter gradient computation, but also to run stochastic gradient Langevin dynamics. Because they make use of the backpropagation algorithm, which in turn requires that their models runs on digital processors such as GPUs, it is unclear if their model could be useful to build low-power hardware for AI, which is the motivation of our work.\n\nTo avoid the confusion between the EBMs of the referenced works and the models considered in our work, we propose to change the name of our approach to “energy-driven models” instead of “energy-based models”, and to change the title to “Energy-driven learning algorithms for analog computing: a comparative study”. We will also explain in the introduction section that the energy-driven approach of our work differs from this other line of work on energy-based models. We hope that this clarifies the motivation of our work.\n\n\nSecond, we respectfully disagree with the reviewer’s remark that “Section 2.5, which presents the theoretical results, appears disconnected from the main body of the paper”. We discuss the implications of our theorems and how they relate to the research findings in the discussion section (section 4.2 of the paper). For example, we write: “algorithms employing a positive perturbation (P-EP and P-CpL) perform significantly worse than those employing a negative perturbation (N-EP and N-CpL). [...] Theorem 2 sheds light on this observation: N-EP optimizes an upper bound of the cost function, whereas P-EP optimizes a lower bound”. As pointed out by reviewer dBi8, “Theorems 2 and 3 very nicely corroborate the empirical observations”.\n\n\nFinally, our motivation for choosing DCHNs for comparing the different learning algorithms is that DCHNs are suitable for building low-power AI. As explained in the common rebuttal (see above), among all energy-based architectures compatible with analog hardware, DCHNs are the best-performing models to date.\n\nReference:\nYi, S. I., Kendall, J. D., Williams, R. S., & Kumar, S. (2023). Activity-difference training of deep neural networks using memristor crossbars. Nature Electronics, 6(1), 45-51."}, {"review_id": "bOGtAXlGe7", "reviewer": "Reviewer_AFqD", "summary": "This paper aims to investigate existing Energy-based learning algorithms on equal footing with same models and datasets. Energy-based algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL) have been carried out for comparison. The experiments conducted based on deep Hopfield networks (DHNs) show that the centered variant of EP is the best-performing algorithm.", "questions": "1.\tIn this paper, what’s the reason that deep convolutional Hopfield network (DCHN) is suitable for generic comparison? The authors should present detailed illustrations.\n2.\tThe experimental results show that equilibrium propagation performs better than coupled learning. Does this conclusion hold true for large-scale data training? The authors need to present deeper analyses or proofs.\n3.\tIt seems that this paper does not provide new solutions for efficient energy design/ learning. According to the conclusion presented in this paper, how can we design powerful EBL models with more complex network architectures?", "limitations": "See weaknesses.\n", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This work provides systematic comparison of the different energy-based learning algorithms. The idea is meaningful and the presentation is satisfying.", "weaknesses": "it does not present new solutions for efficient energy design/ learning, the generalizability of the comparison (not limited on DCHN) should be further discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We have explained in the common rebuttal the novelties in our work. In particular, our work introduces a novel asynchronous update scheme for accelerating the convergence of the energy-minimization process in DCHNs. This asynchronous update scheme allows us to achieve a 13.5x speedup compared to Laborieux et al. (2021) in our overall simulations. (We refer to Appendix C in the supplementary material for details.)\n\nWe have also clarified in the common rebuttal why we chose DCHNs for comparing the different training algorithms. Among the energy-based architectures compatible with analog computing, we note that DCHNs achieve the highest reported accuracies for energy-based learning algorithms on image-recognition tasks, which is the focus of this work. While other architectures have not been studied, we believe that DCHNs are the best choice for this reason.\n\nWe agree that there is no obvious reason why DCHNs should be generic for the comparison of EBL algorithms, and that we have no evidence that our findings would generalize beyond this setting. We will explain in the conclusion section that our experimental study is limited to DCHNs and that our findings might not hold in the context of other network models. However, we note that our Theorems 2 and 3 indicate that our findings are likely to generalize beyond the setting of DCHNs.\n\nFinally, while there is no certainty that our findings would hold on large-scale data, we emphasize that our comparative study is the very first such study for comparing EBL algorithms.\n"}]

Author rebuttal:
We thank the reviewers for their time and comments. Several criticisms of the manuscript by the reviewers fall into the following two categories:

1. Lack of novelty

2. The choice of DCHNs for comparing the seven EBL algorithms, and the lack of discussion of the limitations of our study

We address 1 and 2 in this order.


To address 1, we would like to point out the following novelties in our work.

First, while the network architecture that we consider (the deep convolutional Hopfield network, DCHN) is not novel, our energy minimization method for DCHN (the “asynchronous update method”) is a novel contribution to the literature, as opposed to the “synchronous update method” used e.g. in Laborieux et al (2021). Thanks to our asynchronous update method, using reduced precision (16 bits instead of 32 bits) and fewer iterations (60 iterations instead of 250), we get an overall 13.5x speedup compared to Laborieux et al (2021). We refer to Appendix C in the supplementary material for details. Importantly, the code of Laborieux et al (2021) yields the expected performance only with 32 bit precision ; when running their code with 16 bit precision the performance collapses! This observation suggests that switching from their “synchronous update method” to our “asynchronous update method” improves the quality of convergence to equilibrium (the minimum of the energy function), which in turn allows us the use of 16 bits and reduced number of iterations. Thus, our “asynchronous update method” was critical to get the overall 13.5x speedup.

Second, our work introduces novel training algorithms (EBL algorithms), namely NEP, NCpL and CCpL. NEP is new and tested in our work for the first time. Our study revealed the not obvious fact that our NEP algorithm performs much better than the original PEP algorithm introduced by Scellier and Bengio (2017), and often performs as well as the CEP algorithm introduced by Laborieux et al (2021). Similarly, while Coupled Learning was introduced by Stern et al (2021) in the positively-perturbed version (PCpL), our work introduces CCpL and NCpL and tests them for the first time. Our study reveals that both CCpL and NCpL perform better than the original PCpL algorithm.

Additionally, we provide new theoretical results (Theorems 2 and 3) on how these different EBL algorithms optimize objective functions. In particular, we show that N-EP optimizes an upper bound on the cost function, whereas P-EP optimizes a lower bound.


Next, we address 2. The reason we chose DCHNs for comparison in this manuscript is that, among the energy-based architectures compatible with analog hardware (memristive networks), DCHNs are the best performing architectures in the literature. Given this, and the fact that these are some of the largest scale simulations of these learning algorithms to date, we believe that these results will scale to larger datasets and more complex architectures. We will explain in the conclusion section that our experimental findings are limited to DCHNs, that they might not hold beyond these networks, and that this will be left for future work to investigate.


References:

1. Laborieux, A., Ernoult, M., Scellier, B., Bengio, Y., Grollier, J., & Querlioz, D. (2021). Scaling equilibrium propagation to deep convnets by drastically reducing its gradient estimator bias. Frontiers in neuroscience, 15, 633674.

2. Scellier, B., & Bengio, Y. (2017). Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience, 11, 24.

3. Stern, M., Hexner, D., Rocks, J. W., & Liu, A. J. (2021). Supervised learning in physical networks: From machine learning to learning machines. Physical Review X, 11(2), 021045.

Meta-review: The paper received mixed ratings before the rebuttal. The reviewers raised shared concerns regarding the clarity of the contribution and the model performance. The authors’ response clarified some of the shared concerns and also steered the reviewer with the most negative opinion about the paper towards positive. 

The ACs agree with the reviewers and appreciate the paper’s effort in providing a systematic and clearly presented study of energy-based learning algorithms, thus recommending acceptance. As suggested by reviewers and promised in the authors’ rebuttal, in the final version, the clarity should be further improved by incorporating well-articulated arguments to emphasize the contribution and difference with EBMs.
Decision: Accept (poster)