SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Energy-Based Learning Algorithms for Analog Computing: A Comparative Study
Benjamin Scellier
Rain AI
benjamin@rain.ai
Maxence Ernoult
maxence@rain.ai
Jack Kendall
jack@rain.ai
Suhas Kumar
suhas@rain.ai
Energy-based learning algorithms have recently gained a surge of interest due to their compatibility with analog (post-digital) hardware. Existing algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL), all consisting in contrasting two states, and differing in the type of perturbation used to obtain the second state from the first one. However, these algorithms have never been explicitly compared on equal footing with same models and datasets, making it difficult to assess their scalability and decide which one to select in practice. In this work, we carry out a comparison of seven learning algorithms, namely CL and different variants of EP and CpL depending on the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100). We find that, while all algorithms yield comparable performance on MNIST, important differences in performance arise as the difficulty of the task increases. Our key findings reveal that negative perturbations are better than positive ones, and highlight the centered variant of EP (which uses two perturbations of opposite sign) as the best-performing algorithm. We also endorse these findings with theoretical arguments. Additionally, we establish new SOTA results with DCHNs on all five datasets, both in performance and speed. In particular, our DCHN simulations are 13.5 times faster with respect to Laborieux et al. [2021], which we achieve thanks to the use of a novel energy minimisation algorithm based on asynchronous updates, combined with reduced precision (16 bits).
Prior to the dominance of backpropagation-based machine learning, Hopfield, Hinton and others proposed an alternative 'energy-based' learning (EBL) approach [Hopfield, 1984, Hinton et al., 1984]. In this approach, the learning model is described by a state variable whose dynamics is governed by an energy function. An EBL model can be used to compute as follows: 1) clamp the model input, 2) let the model settle to equilibrium (that is, the configuration of lowest energy), and 3) read the model output. The objective is to modify the weights of the model so that it computes a desired input-to-output function. This is achieved thanks to an EBL algorithm. One of the earliest EBL algorithms was constructive learning (CL)[Movellan, 1991, Baldi and Pineda, 1991], which adjusts the model weights by contrasting two states: a 'free' state where the model outputs are free, and a perturbed (or 'clamped') state where the model outputs are clamped to desired values. In the machine learning literature, interest in EBL algorithms has remained limited due to the widespread success of backpropagation running on graphics processing units (GPUs). However, EBL algorithms have more recently revived interest as a promising learning framework for analog learning machines [Kendall
et al., 2020, Stern et al., 2021]. The benefit of these algorithms is that they use a single computational circuit or network for both inference and training, and they rely on local learning rules (i.e. the weight updates are local). The locality of computation and learning makes these algorithms attractive for training adaptive physical systems in general [Stern and Murugan, 2023], and for building energy-efficient analog AI in particular [Kendall et al., 2020]. Small-scale EBL-trained variable resistor networks have already been built [Dillavou et al., 2022, 2023, Yi et al., 2023], projecting a possible  $10,000 \times$  improvement in energy efficiency compared to GPU-based training of deep neural networks [Yi et al., 2023].
In recent years, various EBL algorithms have been proposed, such as equilibrium propagation (EP) [Scellier and Bengio, 2017], the centered variant of EP [Laborieux et al., 2021] and coupled learning (CpL) [Stern et al., 2021]. These algorithms, which are variants of CL with modified perturbation methods, are often evaluated on different models and different datasets without being compared to CL and to one another. Due to the lack of explicit comparison between these algorithms, and since the algorithmic differences between them are small, they are often gathered under the 'contrastive learning' (or 'contrastive Hebbian learning') umbrella name [Dillavou et al., 2022, Stern and Murugan, 2023, Peterson and Lavin, 2022, Lillicrap et al., 2020, Luczak et al., 2022, Høier et al., 2023]. Consequently, many recent follow-up works in energy-based learning indifferently pick one of these algorithms without considering alternatives [Dillavou et al., 2022, Wycoff et al., 2022, Stern et al., 2022, Kiraz et al., 2022, Watfa et al., 2023, Yi et al., 2023, Dillavou et al., 2023, Altman et al., 2023]. The main contribution of the present work is to provide an explicit comparison of the above-mentioned EBL algorithms and highlight the important differences arising when the difficulty of the task increases. Nonetheless, comparing these algorithms comes with another challenge: simulations are typically very slow. Due to this slowness, EBL algorithms have often been used to train small networks on small datasets (by deep learning standards). Likewise, experimental realizations of EBL algorithms on analog hardware ("physical learning machines") have thus far been performed on small systems only.
In this work, we conduct a study to compare seven EBL algorithms, including the four above-mentioned and three new ones. Depending on the sign of the perturbation, we distinguish between 'positively-perturbed' (P-), 'negatively-perturbed' (N-) and 'centered' (C-) algorithms. To avoid the problem of slow simulations of analog circuits, we conduct our comparative study on deep convolutional Hopfield networks (DCHNs)[Ernoult et al., 2019], an energy-based network that has been emulated on Ising machines [Laydevant et al., 2023], and for which simulations are faster and previously demonstrated to scale to tasks such as CIFAR-10 [Laborieux et al., 2021] and Imagenet 32x32 [Laborieux and Zenke, 2022]. Our contributions include the following:
- We train DCHNs with each of the seven EBL algorithms on five vision tasks: MNIST, Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100. We find that all these algorithms perform well on MNIST, but as the difficulty of the task increases, important behavioural differences start to emerge. Perhaps counter-intuitively, we find that N-type algorithms outperform P-type algorithms by a large margin on most tasks. The C-EP algorithm emerges as the best performing one, outperforming the other six algorithms on the three hardest tasks (SVHN, CIFAR-10 and CIFAR-100).
- We state novel theoretical results for EP, adapted from those of 'agnostic EP' [Scellier et al., 2022], that support our empirical findings. While EP is often presented as an algorithm that approximates gradient descent on the cost function [Scellier and Bengio, 2017, Laborieux et al., 2021], we provide a more precise and stronger statement: EP performs (exact) gradient descent on a surrogate function that approximates the (true) cost function. The surrogate
For example, Kendall et al. [2020] and Watfa et al. [2023] train layered nonlinear resistive networks using EP, while Stern et al. [2021] train randomly-connected elastic and flow networks using CpL.
$^{2}$ Kendall et al. [2020] use SPICE to simulate the training of a one-hidden-layer network (with 100 'hidden nodes') on MNIST, which takes one week for only ten epochs of training. Watfa et al. [2023] train resistive networks with EP on Fashion-MNIST, Wine, and Iris. Similarly, Stern et al. [2021] simulate the training of disordered networks of up to 2048 nodes on a subset of 200 images of the MNIST dataset.
<sup>3</sup>With some exceptions, e.g. Laborieux and Zenke [2022] perform simulations of energy-based convolutional networks (DCHNs) on ImageNet 32x32.
$^{4}$ Dillavou et al. [2022] train a resistive network of 9 nodes and 16 edges on the IRIS dataset. Yi et al. [2023] train a 24-33-7 memristive network on a  $64 \times 64$  array of memristors to classify Braille words. Laydevant et al. [2023] train a 784-120-40 network (Ising machine) on a subset of 1000 images of the MNIST dataset.
function of N-EP is an upper bound of the cost function, whereas the one of P-EP is a lower bound (Theorem 2). Moreover, the surrogate function of C-EP approximates the true cost function at the second order in the perturbation strength (Theorem 3), whereas the ones of P-EP and N-EP are first-order approximations.
- We achieve state-of-the-art DCHN simulations on all five datasets, both in terms of performance (accuracy) and speed. For example, a full run of 100 epochs on CIFAR-10 yields a test error rate of  $10.4\%$  and takes 3 hours 18 minutes on a single A100 GPU; we show that this is  $13.5\mathrm{x}$  faster than the simulations of Laborieux et al. [2021] on the same hardware (a A100 GPU). With further training (300 epochs), the test error rate goes down to  $9.7\%$  - to be compared with  $11.4\%$  reported in Laborieux and Zenke [2022]. Our simulation speedup is enabled thanks to the use of a novel energy minimization procedure for DCHNs based on asynchronous updates and the use of 16 bit precision.
We note that our work also bears similarities with the line of works on energy-based models [Grathwohl et al., 2019, Nijkamp et al., 2019, Du and Mordatch, 2019, Geng et al., 2021]. Like our approach, these studies involve the minimization of an energy function within the activation space (or input space) of a network. However, unlike our approach, they do not typically exclude the use of the backpropagation algorithm, employing it not only to compute the parameter gradients, but also to execute gradient descent within the network's activation space (or input space). In contrast, the primary motivation of our work is to eliminate the need for backpropagation, and to perform inference and learning by leveraging locally computed quantities, with the long term goal of building energy-efficient processors dedicated to model optimization. Hence, our motivation diverges significantly from traditional 'energy-based models'.
Figure 1: Cartoon illustrating the seven energy-based learning (EBL) algorithms: contrastive learning (CL), positively-perturbed algorithms (P-), negatively-perturbed algorithms (N-) and centered algorithms (C-). EP and CpL stand for equilibrium propagation and coupled learning, respectively. The desired output is  $y$ . The model prediction is  $o_{\star}$  i.e. the output configuration minimizing the energy function. The strength of the perturbation is  $\beta$ . A positive perturbation pulls the model output  $(o_{\beta})$  towards  $y$ . A negative perturbation pushes the model output  $(o_{-\beta})$  away from  $y$ . Arrows indicate the weight update: green (resp. red) arrows decrease (resp. increase) the energy value of the corresponding configuration.
2 Energy-based learning algorithms
This section introduces the concepts and notations, presents the seven energy-based learning algorithms and states the theoretical results.
We consider the setting of image classification. In this setting, an energy-based learning (EBL) model is composed of an input variable  $(x)$ , a parameter variable  $(\theta)$  a hidden variable  $(h)$  and an output variable  $(o)$ . A scalar function  $E$  called energy function assigns to each tuple  $(\theta, x, h, o)$  a real number  $E(\theta, x, h, o)$ . Given  $\theta$  and  $x$ , among all possible configurations  $(h, o)$ , the effective configuration of the model is the equilibrium state (or steady state), denoted  $(h_{\star}, o_{\star})$  which is implicitly defined as a minimum of the energy function,
$$
\left(h _ {\star}, o _ {\star}\right) := \underset {(h, o)} {\arg \min } E (\theta , x, h, o). \tag {1}
$$
The equilibrium value (or steady state value) of output variables,  $o_{\star}$ , represents a prediction of the model, which we also denote  $o(\theta, x) = o_{\star}$  to emphasize that it depends on the input  $x$  and the model parameter  $\theta$ .
The goal of training an EBL model is to adjust  $\theta$  so that, for any input  $x$ , the output  $o(\theta, x)$  coincides with a desired output  $y$  (the label associated to  $x$ ). We refer to an algorithm for training an EBL model as an EBL algorithm.
Remark. In the terminology of Stern and Murugan [2023],  $h$  and  $o$  are the 'physical degrees of freedom',  $\theta$  is the set of 'learning degrees of freedom' and  $E$  is the 'physical cost function'.
2.1 Contrastive learning
Contrastive learning (CL) is the earliest EBL algorithm [Movellan, 1991]. The CL algorithm proceeds in two phases. In the first phase, input variables  $x$  are clamped, while the hidden and output variables are free to stabilize to the energy minimum  $(h_{\star}, o_{\star})$  as in (1). In the second phase, the output variables  $o$  are now also clamped to the desired output  $y$ , and the hidden variables  $h$  are free to stabilize to a second energy minimum, denoted  $h_{\star}^{\mathrm{CL}}$ , characterized by
$$
h _ {\star} ^ {\mathrm {C L}} := \underset {h} {\arg \min } E (\theta , x, h, y). \tag {2}
$$
The contrastive learning rule for the model parameters reads
$$
\Delta^ {\mathrm {C L}} \theta = \eta \left(\frac {\partial E}{\partial \theta} (\theta , x, h _ {\star}, o _ {\star}) - \frac {\partial E}{\partial \theta} (\theta , x, h _ {\star} ^ {\mathrm {C L}}, y)\right), \tag {3}
$$
where  $\eta$  is a learning rate. The CL rule is illustrated in Figure 1.
2.2 Equilibrium propagation
Equilibrium propagation (EP) is another EBL algorithm [Scellier and Bengio, 2017]. One notable difference in EP is that one explicitly introduces a cost function  $C(o,y)$  that represents the discrepancy between the output  $o$  and desired output  $y$ . In its original formulation, EP is a variant of CL which also consists of contrasting two states. In the first phase, similar to CL, input variables are clamped while hidden and output variables are free to settle to the free state  $(h_{\star},o_{\star})$  characterized by (1). In the second phase, in contrast with CL, EP proceeds by only perturbing (or nudging) the output variables rather than clamping them. This is achieved by augmenting the model's energy by a term  $\beta C(o,y)$ , where  $\beta \in \mathbb{R}$  is a scalar – the nudging parameter. The model settles to another equilibrium state, the perturbed state, characterized by
$$
\left(h _ {\beta} ^ {\mathrm {E P}}, o _ {\beta} ^ {\mathrm {E P}}\right) = \underset {(h, o)} {\arg \min } \left[ E (\theta , x, h, o) + \beta C (o, y) \right]. \tag {4}
$$
In its classic form, the learning rule of EP is similar to CL:
$$
\Delta^ {\mathrm {E P}} \theta = \frac {\eta}{\beta} \left(\frac {\partial E}{\partial \theta} (\theta , x, h _ {\star}, o _ {\star}) - \frac {\partial E}{\partial \theta} (\theta , x, h _ {\beta} ^ {\mathrm {E P}}, o _ {\beta} ^ {\mathrm {E P}})\right). \tag {5}
$$
The EP learning rule (5) comes in two variants depending on the sign of  $\beta$ . In Scellier and Bengio [2017], EP is introduced in the variant with  $\beta > 0$ , which we call here positively-perturbed EP (P-EP).
In this work, we introduce the variant with  $\beta < 0$ , which we call negatively-perturbed EP (N-EP). We also consider the variant of EP introduced by Laborieux et al. [2021] whose learning rule reads
$$
\Delta^ {\mathrm {C} - \mathrm {E P}} \theta = \frac {\eta}{2 \beta} \left(\frac {\partial E}{\partial \theta} (\theta , x, h _ {- \beta} ^ {\mathrm {E P}}, o _ {- \beta} ^ {\mathrm {E P}}) - \frac {\partial E}{\partial \theta} (\theta , x, h _ {\beta} ^ {\mathrm {E P}}, o _ {\beta} ^ {\mathrm {E P}})\right), \tag {6}
$$
which we call centered EP (C-EP). The EP learning rules are illustrated in Figure 1.
2.3 Coupled learning
Coupled learning (CpL) is another variant of contrastive learning [Stern et al., 2021]. In the first phase, similar to CL and EP, input variables are clamped, while hidden and output variables are free to settle to their free state value  $(h_{\star},o_{\star})$  characterized by (1). In the second phase, output variables are clamped to a weighted mean of  $o_{\star}$  and  $y$ , and the hidden variables are allowed to settle to their new equilibrium value. Mathematically, the new equilibrium state is characterized by the following formulas, where the weighted mean output  $(o_{\beta}^{\mathrm{CpL}})$  is parameterized by a factor  $\beta \in \mathbb{R} \setminus \{0\}$ :
$$
h _ {\beta} ^ {\mathrm {C p L}} := \underset {h} {\arg \min } E (\theta , x, h, o _ {\beta} ^ {\mathrm {C p L}}), \quad o _ {\beta} ^ {\mathrm {C p L}} := (1 - \beta) o _ {\star} + \beta y. \tag {7}
$$
Similarly to CL and EP, the learning rule of CpL reads
$$
\Delta^ {\mathrm {C p L}} \theta = \frac {\eta}{\beta} \left(\frac {\partial E}{\partial \theta} (\theta , x, h _ {\star}, o _ {\star}) - \frac {\partial E}{\partial \theta} \left(\theta , x, h _ {\beta} ^ {\mathrm {C p L}}, o _ {\beta} ^ {\mathrm {C p L}}\right)\right). \tag {8}
$$
In particular, for  $\beta = 1$ , one recovers the contrastive learning algorithm (CL). In their original formulation, Stern et al. [2021] use  $\beta > 0$ ; here we refer to this algorithm as positively-perturbed CpL (P-CpL). Similarly to EP, we also introduce negatively-perturbed CpL (N-CpL, with  $\beta < 0$ ) as well as centered CpL (C-CpL):
$$
\Delta^ {\mathrm {C} - \mathrm {C p L}} \theta = \frac {\eta}{2 \beta} \left(\frac {\partial E}{\partial \theta} \left(\theta , x, h _ {- \beta} ^ {\mathrm {C p L}}, o _ {- \beta} ^ {\mathrm {C p L}}\right) - \frac {\partial E}{\partial \theta} \left(\theta , x, h _ {\beta} ^ {\mathrm {C p L}}, o _ {\beta} ^ {\mathrm {C p L}}\right)\right). \tag {9}
$$
Figure 1 also depicts P-CpL, N-CpL and C-CpL.
2.4 Theoretical results
The EBL algorithms presented above have different theoretical properties. We start with CL.
Theorem 1 (Contrastive learning). The contrastive learning rule (3) performs one step of gradient descent on the so-called contrastive function  $\mathcal{L}_{\mathrm{CL}}$
$$
\Delta^ {\mathrm {C L}} \theta = - \eta \frac {\partial \mathcal {L} _ {\mathrm {C L}}}{\partial \theta} (\theta , x, y), \quad \mathcal {L} _ {\mathrm {C L}} (\theta , x, y) := E (\theta , x, h _ {\star} ^ {\mathrm {C L}}, y) - E (\theta , x, h _ {\star}, o _ {\star}). \tag {10}
$$
Theorem 1 is proved in Movellan [1991]. However, it is not clear that the contrastive function  $\mathcal{L}_{\mathrm{CL}}$  has the desirable properties of an objective function from a machine learning perspective.
The equilibrium propagation (EP) learning rules have better theoretical properties. EP is often presented as an algorithm that approximates the gradient of the cost function, e.g. in Scellier and Bengio [2017], Laborieux et al. [2021]. In this work, we provide a more precise and stronger statement: EP computes the exact gradient of a surrogate function that approximates the cost function. Moreover, in N-EP, the surrogate function is an upper bound of the cost function (Theorem 2), and in C-EP, the surrogate function approximates the cost function at the second order in  $\beta$  (Theorem 3). Theorems 2 and 3 are adapted from Scellier et al. [2022]  $^6$  - see Appendix A for proofs.
Another variant of EP proposed in Scellier and Bengio [2017] combines N-EP and P-EP, where the sign of the nudging parameter  $\beta$  is chosen at random for each example in the training set. However, N-EP wasn't used and tested as is.
$^6$ In 'agnostic equilibrium propagation' (AEP) [Scellier et al., 2022], the function  $\mathcal{L}_{\beta}^{\mathrm{EP}}$  is proved to be a Lyapunov function for AEP, but AEP does not perform exact gradient descent on  $\mathcal{L}_{\beta}^{\mathrm{EP}}$ . In contrast, EP performs exact gradient descent on  $\mathcal{L}_{\beta}^{\mathrm{EP}}$ , but  $\mathcal{L}_{\beta}^{\mathrm{EP}}$  is not a Lyapunov function for EP.
Theorem 2 (Equilibrium propagation). There exists some function  $\mathcal{L}_{\beta}^{\mathrm{EP}}$  such that the learning rule (5) performs one step of gradient descent on  $\mathcal{L}_{\beta}^{\mathrm{EP}}$ :
$$
\Delta^ {\mathrm {E P}} \theta = - \eta \frac {\partial \mathcal {L} _ {\beta} ^ {\mathrm {E P}}}{\partial \theta} (\theta , x, y). \tag {11}
$$
The function  $\mathcal{L}_{\beta}^{\mathrm{EP}}$  is a lower bound of the 'true' cost function if  $\beta > 0$  (P-EP), and an upper bound if  $\beta < 0$  (N-EP), i.e.
$$
\mathcal {L} _ {\beta} ^ {\mathrm {E P}} (\theta , x, y) \leq C (o (\theta , x), y) \leq \mathcal {L} _ {- \beta} ^ {\mathrm {E P}} (\theta , x, y), \quad \forall \beta > 0, \tag {12}
$$
where  $o(\theta, x) = o_{\star}$  is the free equilibrium value of output variables given  $\theta$  and  $x$ , as in (1). Furthermore,  $\mathcal{L}_{\beta}^{\mathrm{EP}}$  approximates the 'true' cost function up to  $O(\beta)$  when  $\beta \to 0$ ,
$$
\mathcal {L} _ {\beta} ^ {\mathrm {E P}} (\theta , x, y) = C (o (\theta , x), y) + O (\beta). \tag {13}
$$
Theorem 3 (Centered EP). There exists some function  $\mathcal{L}_{-\beta; +\beta}^{\mathrm{EP}}$  such that the learning rule (6) performs one step of gradient descent on  $\mathcal{L}_{-\beta; +\beta}^{\mathrm{EP}}$ :
$$
\Delta^ {\mathrm {C} - \mathrm {E P}} \theta = - \eta \frac {\partial \mathcal {L} _ {- \beta ; + \beta} ^ {\mathrm {E P}}}{\partial \theta} (\theta , x, y). \tag {14}
$$
Moreover, the function  $\mathcal{L}_{-\beta; +\beta}^{\mathrm{EP}}$  approximates the 'true' cost function up to  $O(\beta^{2})$  when  $\beta \to 0$ ,
$$
\mathcal {L} _ {- \beta ; + \beta} ^ {\mathrm {E P}} (\theta , x, y) = C (o (\theta , x), y) + O \left(\beta^ {2}\right). \tag {15}
$$
Finally, the analysis of the coupled learning rules (P-CpL, N-CpL and C-CpL) is more complicated. Stern et al. [2021] introduce the following function as a candidate loss function for coupled learning:
$$
\mathcal {L} _ {\mathrm {C p L}} ^ {(1)} (\theta , x, y) := (y - o (\theta , x)) ^ {\top} \cdot \frac {\partial^ {2} G}{\partial o ^ {2}} (\theta , x, o (\theta , x)) \cdot (y - o (\theta , x)), \quad G (\theta , x, o) := \min  _ {h} E (\theta , x, h, o). \tag {16}
$$
Stern et al. [2021] argue that the P-CpL rule (8) optimizes both  $\mathcal{L}_{\mathrm{CpL}}$  and the squared error  $\mathcal{L}_{\mathrm{MSE}}(\theta, x, y) \coloneqq (o(\theta, x) - y)^2$ . Besides, Stern et al. [2022, 2023] argue that the learning rule (8) performs gradient descent on
$$
\mathcal {L} _ {\mathrm {C p L}} ^ {(2)} := \frac {1}{\beta} \left(E (\theta , x, h _ {\beta} ^ {\mathrm {C p L}}, o _ {\beta} ^ {\mathrm {C p L}}) - E (\theta , x, h _ {\star}, o _ {\star})\right). \tag {17}
$$
In Appendix B, we demonstrate that the coupled learning rules (8) and (9) do not optimize  $\mathcal{L}_{\mathrm{CpL}}^{(1)}$  or  $\mathcal{L}_{\mathrm{MSE}}$ , and do not perform gradient descent on  $\mathcal{L}_{\mathrm{CpL}}^{(2)}$ .
3 Deep convolutional Hopfield networks
To compare the EBL algorithms presented in section 2, we consider the EBL model of Arnoult et al. [2019], which we call deep convolutional Hopfield network (DCHN). We consider specifically the network architecture of Laborieux et al. [2021].
Network architecture. The network has an input layer, four hidden layers, and an output layer. Since we consider classification tasks, the output layer has  $M$  units, where  $M$  is the number of categories for the task. Successive layers are interconnected by convolutional interactions with kernel size  $3 \times 3$ , padding 1, and max pooling. Except for the last hidden layer and the output layer, which are interconnected by a dense interaction.
Energy function. We denote the state of the network  $s = (s_0, s_1, s_2, s_3, s_4, s_5)$ , where  $x = s_0$  is the input layer,  $h = (s_1, s_2, s_3, s_4)$  is the hidden variable and  $o = s_5$  is the output variable. The energy function of the network is