Title: Energy-based learning algorithms for analog computing: a comparative study
Venue: NeurIPS 2023
Accepted: yes
Keywords: energy-based learning algorithm

contrastive learning

equilibrium propagation

coupled learning

convolutional Hopfield network
Abstract: Energy-based learning algorithms have recently gained a surge of interest due to their compatibility with analog (post-digital) hardware. Existing algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL), all consisting in contrasting two states, and differing in the type of perturbation used to obtain the second state from the first one. However, these algorithms have never been explicitly compared on equal footing with same models and datasets, making it difficult to assess their scalability and decide which one to select in practice. In this work, we carry out a comparison of seven learning algorithms, namely CL and different variants of EP and CpL depending on the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100). We find that, while all algorithms yield comparable performance on MNIST, important differences in performance arise as the difficulty of the task increases. Our key findings reveal that negative perturbations are better than positive ones, and highlight the centered variant of EP (which uses two perturbations of opposite sign) as the best-performing algorithm. We also endorse these findings with theoretical arguments. Additionally, we establish new SOTA results with DCHNs on all five datasets, both in performance and speed. In particular, our DCHN simulations are 13.5 times faster with respect to Laborieux et al. (2021), which we achieve thanks to the use of a novel energy minimisation algorithm based on asynchronous updates, combined with reduced precision (16 bits).

Energy-Based Learning Algorithms for Analog Computing: A Comparative Study

Abstract

Energy-based learning algorithms have recently gained a surge of interest due to their compatibility with analog (post-digital) hardware. Existing algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL), all consisting in contrasting two states, and differing in the type of perturbation used to obtain the second state from the first one. However, these algorithms have never been explicitly compared on equal footing with same models and datasets, making it difficult to assess their scalability and decide which one to select in practice. In this work, we carry out a comparison of seven learning algorithms, namely CL and different variants of EP and CpL depending on the signs of the perturbations. Specifically, using these learning algorithms, we train deep convolutional Hopfield networks (DCHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100). We find that, while all algorithms yield comparable performance on MNIST, important differences in performance arise as the difficulty of the task increases. Our key findings reveal that negative perturbations are better than positive ones, and highlight the centered variant of EP (which uses two perturbations of opposite sign) as the best-performing algorithm. We also endorse these findings with theoretical arguments. Additionally, we establish new SOTA results with DCHNs on all five datasets, both in performance and speed. In particular, our DCHN simulations are 13.5 times faster with respect to Laborieux et al. [2021], which we achieve thanks to the use of a novel energy minimisation algorithm based on asynchronous updates, combined with reduced precision (16 bits).

1 Introduction

Prior to the dominance of backpropagation-based machine learning, Hopfield, Hinton and others proposed an alternative 'energy-based' learning (EBL) approach [Hopfield, 1984, Hinton et al., 1984]. In this approach, the learning model is described by a state variable whose dynamics is governed by an energy function. An EBL model can be used to compute as follows: 1) clamp the model input, 2) let the model settle to equilibrium (that is, the configuration of lowest energy), and 3) read the model output. The objective is to modify the weights of the model so that it computes a desired input-to-output function. This is achieved thanks to an EBL algorithm. One of the earliest EBL algorithms was constrastive learning (CL)[Movellan, 1991, Baldi and Pineda, 1991], which adjusts the model weights by contrasting two states: a 'free' state where the model outputs are free, and a perturbed (or 'clamped') state where the model outputs are clamped to desired values. In the machine learning literature, interest in EBL algorithms has remained limited due to the widespread success of backpropagation running on graphics processing units (GPUs). However, EBL algorithms have more recently revived interest as a promising learning framework for analog learning machines [Kendall

et al., 2020, Stern et al., 2021]. The benefit of these algorithms is that they use a single computational circuit or network for both inference and training, and they rely on local learning rules (i.e. the weight updates are local). The locality of computation and learning makes these algorithms attractive for training adaptive physical systems in general [Stern and Murugan, 2023], and for building energyefficient analog AI in particular [Kendall et al., 2020]. Small-scale EBL-trained variable resistor networks have already been built [Dillavou et al., 2022, 2023, Yi et al., 2023], projecting a possible 10,000 × improvement in energy efficiency compared to GPU-based training of deep neural networks [Yi et al., 2023].

In recent years, various EBL algorithms have been proposed, such as equilibrium propagation (EP) [Scellier and Bengio, 2017], the centered variant of EP [Laborieux et al., 2021] and coupled learning (CpL) [Stern et al., 2021]. These algorithms, which are variants of CL with modified perturbation methods, are often evaluated on different models and different datasets without being compared to CL and to one another. 1 Due to the lack of explicit comparison between these algorithms, and since the algorithmic differences between them are small, they are often gathered under the 'contrastive learning' (or 'contrastive Hebbian learning') umbrella name [Dillavou et al., 2022, Stern and Murugan, 2023, Peterson and Lavin, 2022, Lillicrap et al., 2020, Luczak et al., 2022, Høier et al., 2023]. Consequently, many recent follow-up works in energy-based learning indifferently pick one of these algorithms without considering alternatives [Dillavou et al., 2022, Wycoff et al., 2022, Stern et al., 2022, Kiraz et al., 2022, Watfa et al., 2023, Yi et al., 2023, Dillavou et al., 2023, Altman et al., 2023]. The main contribution of the present work is to provide an explicit comparison of the abovementioned EBL algorithms and highlight the important differences arising when the difficulty of the task increases. Nonetheless, comparing these algorithms comes with another challenge: simulations are typically very slow. Due to this slowness, EBL algorithms have often been used to train small networks on small datasets (by deep learning standards). 2 3 Likewise, experimental realizations of EBL algorithms on analog hardware ('physical learning machines') have thus far been performed on small systems only. 4

In this work, we conduct a study to compare seven EBL algorithms, including the four abovementioned and three new ones. Depending on the sign of the perturbation, we distinguish between 'positively-perturbed' (P-), 'negatively-perturbed' (N-) and 'centered' (C-) algorithms. To avoid the problem of slow simulations of analog circuits, we conduct our comparative study on deep convolutional Hopfield networks (DCHNs)[Ernoult et al., 2019], an energy-based network that has been emulated on Ising machines [Laydevant et al., 2023], and for which simulations are faster and previously demonstrated to scale to tasks such as CIFAR-10 [Laborieux et al., 2021] and Imagenet 32x32 [Laborieux and Zenke, 2022]. Our contributions include the following:

We train DCHNs with each of the seven EBL algorithms on five vision tasks: MNIST, Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100. We find that all these algorithms perform well on MNIST, but as the difficulty of the task increases, important behavioural differences start to emerge. Perhaps counter-intuitively, we find that N-type algorithms outperform P-type algorithms by a large margin on most tasks. The C-EP algorithm emerges as the best performing one, outperforming the other six algorithms on the three hardest tasks (SVHN, CIFAR-10 and CIFAR-100).

We state novel theoretical results for EP, adapted from those of 'agnostic EP' [Scellier et al., 2022], that support our empirical findings. While EP is often presented as an algorithm that approximates gradient descent on the cost function [Scellier and Bengio, 2017, Laborieux et al., 2021], we provide a more precise and stronger statement: EP performs (exact) gradient descent on a surrogate function that approximates the (true) cost function. The surrogate

function of N-EP is an upper bound of the cost function, whereas the one of P-EP is a lower bound (Theorem 2). Moreover, the surrogate function of C-EP approximates the true cost function at the second order in the perturbation strength (Theorem 3), whereas the ones of P-EP and N-EP are first-order approximations.

We achieve state-of-the-art DCHN simulations on all five datasets, both in terms of performance (accuracy) and speed. For example, a full run of 100 epochs on CIFAR-10 yields a test error rate of 10.4% and takes 3 hours 18 minutes on a single A100 GPU ; we show that this is 13.5x faster than the simulations of Laborieux et al. [2021] on the same hardware (a A100 GPU). With further training (300 epochs), the test error rate goes down to 9.7% - to be compared with 11.4% reported in Laborieux and Zenke [2022]. Our simulation speedup is enabled thanks to the use of a novel energy minimization procedure for DCHNs based on asynchronous updates and the use of 16 bit precision.

We note that our work also bears similarities with the line of works on energy-based models [Grathwohl et al., 2019, Nijkamp et al., 2019, Du and Mordatch, 2019, Geng et al., 2021]. Like our approach, these studies involve the minimization of an energy function within the activation space (or input space) of a network. However, unlike our approach, they do not typically exclude the use of the backpropagation algorithm, employing it not only to compute the parameter gradients, but also to execute gradient descent within the network's activation space (or input space). In contrast, the primary motivation of our work is to eliminate the need for backpropagation, and to perform inference and learning by leveraging locally computed quantities, with the long term goal of building energyefficient processors dedicated to model optimization. Hence, our motivation diverges significantly from traditional 'energy-based models'.

Contrastivelearning(CL)

Positively-perturbed algorithms(P-EP,P-CpL)

energy function E(θ, x, h, o)

energy function E(0, x, h, o)

y

0.

‘

output variable o

output variable o

Negatively-perturbed algorithms (N-EP, N-CpL)

Centeredalgorithms(C-EP,C-CpL)

energy function E(0,x, h, o)

energy function E(0, x, h, o)

0.

8-o

do

8-o

output variable o

output variable o

2 Energy-based learning algorithms

This section introduces the concepts and notations, presents the seven energy-based learning algorithms and states the theoretical results.

We consider the setting of image classification. In this setting, an energy-based learning (EBL) model is composed of an input variable ( x ), a parameter variable ( θ ) a hidden variable ( h ) and an output variable ( o ). Ascalar function E called energy function assigns to each tuple ( θ, x, h, o ) a real number E ( θ, x, h, o ) . Given θ and x , among all possible configurations ( h, o ) , the effective configuration of the model is the equilibrium state (or steady state), denoted ( h ⋆ , o ⋆ ) which is implicitly defined as a minimum of the energy function,

The equilibrium value (or steady state value) of output variables, o ⋆ , represents a prediction of the model, which we also denote o ( θ, x ) = o ⋆ to emphasize that it depends on the input x and the model parameter θ .

The goal of training an EBL model is to adjust θ so that, for any input x , the output o ( θ, x ) coincides with a desired output y (the label associated to x ). We refer to an algorithm for training an EBL model as an EBL algorithm .

Remark. In the terminology of Stern and Murugan [2023], h and o are the 'physical degrees of freedom', θ is the set of 'learning degrees of freedom' and E is the 'physical cost function'.

2.1 Contrastive learning

Contrastive learning (CL) is the earliest EBL algorithm [Movellan, 1991]. The CL algorithm proceeds in two phases. In the first phase, input variables x are clamped, while the hidden and output variables are free to stabilize to the energy minimum ( h ⋆ , o ⋆ ) as in (1). In the second phase, the output variables o are now also clamped to the desired output y , and the hidden variables h are free to stabilize to a second energy minimum, denoted h CL ⋆ , characterized by

The contrastive learning rule for the model parameters reads

where η is a learning rate. The CL rule is illustrated in Figure 1.

2.2 Equilibrium propagation

Equilibrium propagation (EP) is another EBL algorithm [Scellier and Bengio, 2017]. One notable difference in EP is that one explicitly introduces a cost function C ( o, y ) that represents the discrepancy between the output o and desired output y . In its original formulation, EP is a variant of CL which also consists of contrasting two states. In the first phase, similar to CL, input variables are clamped while hidden and output variables are free to settle to the free state ( h ⋆ , o ⋆ ) characterized by (1). In the second phase, in contrast with CL, EP proceeds by only perturbing (or nudging ) the output variables rather than clamping them. This is achieved by augmenting the model's energy by a term βC ( o, y ) , where β ∈ R is a scalar - the nudging parameter . The model settles to another equilibrium state, the perturbed state, characterized by

In its classic form, the learning rule of EP is similar to CL:

The EP learning rule (5) comes in two variants depending on the sign of β . In Scellier and Bengio [2017], EP is introduced in the variant with β > 0 , which we call here positively-perturbed EP (P-EP).

In this work, we introduce the variant with β < 0 , which we call negatively-perturbed EP (N-EP). 5 We also consider the variant of EP introduced by Laborieux et al. [2021] whose learning rule reads

which we call centered EP (C-EP). The EP learning rules are illustrated in Figure 1.

2.3 Coupled learning

Coupled learning (CpL) is another variant of contrastive learning [Stern et al., 2021]. In the first phase, similar to CL and EP, input variables are clamped, while hidden and output variables are free to settle to their free state value ( h ⋆ , o ⋆ ) characterized by (1). In the second phase, output variables are clamped to a weighted mean of o ⋆ and y , and the hidden variables are allowed to settle to their new equilibrium value. Mathematically, the new equilibrium state is characterized by the following formulas, where the weighted mean output ( o CpL β ) is parameterized by a factor β ∈ R \ { 0 } :

Similarly to CL and EP, the learning rule of CpL reads

In particular, for β = 1 , one recovers the contrastive learning algorithm (CL). In their original formulation, Stern et al. [2021] use β > 0 ; here we refer to this algorithm as positively-perturbed CpL (P-CpL). Similarly to EP, we also introduce negatively-perturbed CpL (N-CpL, with β < 0 ) as well as centered CpL (C-CpL):

Figure 1 also depicts P-CpL, N-CpL and C-CpL.

2.4 Theoretical results

The EBL algorithms presented above have different theoretical properties. We start with CL.

Theorem 1 (Contrastive learning) . The contrastive learning rule (3) performs one step of gradient descent on the so-called contrastive function L CL ,

Theorem 1 is proved in Movellan [1991]. However, it is not clear that the contrastive function L CL has the desirable properties of an objective function from a machine learning perspective.

The equilibrium propagation (EP) learning rules have better theoretical properties. EP is often presented as an algorithm that approximates the gradient of the cost function, e.g. in Scellier and Bengio [2017], Laborieux et al. [2021]. In this work, we provide a more precise and stronger statement: EP computes the exact gradient of a surrogate function that approximates the cost function. Moreover, in N-EP, the surrogate function is an upper bound of the cost function (Theorem 2), and in C-EP, the surrogate function approximates the cost function at the second order in β (Theorem 3). Theorems 2 and 3 are adapted from Scellier et al. [2022] 6 - see Appendix A for proofs.

Theorem 2 (Equilibrium propagation) . There exists some function L EP β such that the learning rule (5) performs one step of gradient descent on L EP β :

The function L EP β is a lower bound of the 'true' cost function if β > 0 (P-EP), and an upper bound if β < 0 (N-EP), i.e.

where o ( θ, x ) = o ⋆ is the free equilibrium value of output variables given θ and x , as in (1) . Furthermore, L EP β approximates the 'true' cost function up to O ( β ) when β → 0 ,

Theorem 3 (Centered EP) . There exists some function L EP -β ;+ β such that the learning rule (6) performs one step of gradient descent on L EP -β ;+ β :

Moreover, the function L EP -β ;+ β approximates the 'true' cost function up to O ( β 2 ) when β → 0 ,

Finally, the analysis of the coupled learning rules (P-CpL, N-CpL and C-CpL) is more complicated. Stern et al. [2021] introduce the following function as a candidate loss function for coupled learning:

Stern et al. [2021] argue that the P-CpL rule (8) optimizes both L CpL and the squared error L MSE ( θ, x, y ) := ( o ( θ, x ) -y ) 2 . Besides, Stern et al. [2022, 2023] argue that the learning rule (8) performs gradient descent on

In Appendix B, we demonstrate that the coupled learning rules (8) and (9) do not optimize L (1) CpL or L MSE , and do not perform gradient descent on L (2) CpL .

3 Deep convolutional Hopfield networks

To compare the EBL algorithms presented in section 2, we consider the EBL model of Ernoult et al. [2019], which we call deep convolutional Hopfield network (DCHN). We consider specifically the network architecture of Laborieux et al. [2021].

Network architecture. The network has an input layer, four hidden layers, and an output layer. Since we consider classification tasks, the output layer has M units, where M is the number of categories for the task. Successive layers are interconnected by convolutional interactions with kernel size 3 × 3 , padding 1 , and max pooling. Except for the last hidden layer and the output layer, which are interconnected by a dense interaction.

Energy function. We denote the state of the network s = ( s 0 , s 1 , s 2 , s 3 , s 4 , s 5 ) , where x = s 0 is the input layer, h = ( s 1 , s 2 , s 3 , s 4 ) is the hidden variable and o = s 5 is the output variable. The energy function of the network is

where θ = { w k , b k | 1 ≤ k ≤ 5 } is the set of model parameters, and E conv k , E dense k and E bias k are energy terms defined as

Specifically, E conv k is the energy function of a convolutional interaction between layers k -1 and k , parameterized by the kernel w k (the weights), where ⋆ is the convolution operation, P is the max pooling operation, and · is the scalar product for pairs of tensors. E dense k is the energy of a dense interaction between layers k -1 and k , parameterized by the dim( s k -1 ) × dim( s k ) matrix w k . Finally, E bias k is the energy term of b k , the bias of layer k .

Cost function. For the equilibrium propagation learning rules, we use the squared error cost function C ( o, y ) = ∥ o -y ∥ 2 , where o = s 5 is the output layer and y is the one-hot code of the label (associated to image x ).

State space. Given an input x , the steady state of the model is ( h ⋆ , o ⋆ ) = arg min ( h,o ) ∈S E ( θ, x, h, o ) , where S is the state space, i.e. the space of configurations for the hidden and output variables. We use S = ∏ 4 k =1 [0 , 1] dim( s k ) × R dim( s 5 ) , where [0 , 1] is the closed interval of R with bounds 0 and 1 , and dim( s k ) is the number of units in layer k .

Energy minimization via asynchronous updates. In order to compute the steady states (free state and perturbed states) of the learning algorithms presented in section 2, we use a novel procedure for DCHNs using 'asynchronous updates' for the state variables, as opposed to the 'synchronous updates' used in other works [Ernoult et al., 2019, Laborieux et al., 2021, Laydevant et al., 2021, Laborieux and Zenke, 2022]. This asynchronous procedure proceeds as follows: at every iteration, we first update the layers of even indices in parallel (the first half of the layers) and then we update the layers of odd indices (the other half of the layers). Updating all the layers once (first the even layers, then the odd layers) constitutes one 'iteration'. We repeat as many iterations as necessary until convergence to the steady state. See Appendix C for a detailed explanation of the asynchronous procedure, and for a comparison with the synchronous procedure used in other works. Although we do not have a proof of convergence of this asynchronous procedure for DCHNs when P is the max pooling operation, we find experimentally that it converges faster than the synchronous procedure (for which there is no proof of convergence anyway).

4 Simulations

4.1 Comparative study of EBL algorithms

Setup. We compare with simulations the seven energy-based learning (EBL) algorithms of section 2 on the deep convolutional Hopfield network (DCHN) of section 3. To do this, we train a DCHN on MNIST, Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100 using each of the seven EBL algorithms. We also compare the performance of these algorithms to two baselines: recurrent backpropagation (RBP) and truncated backpropagation (TBP), detailed below. For each simulation, the DCHN is trained for 100 epochs. Each run is performed on a single A100 GPU. A run on MNIST and FashionMNIST takes 3 hours 30 minutes ; a run on SVHN takes 4 hours 45 minutes ; and a run on CIFAR-10 and CIFAR-100 takes 3 hours. All these simulations are performed with the same network using the same initialization scheme and the same hyperparameters. Details of the training experiments are provided in Appendix D and the results are reported in Table 1. 7

First baseline: recurrent backpropagation (RBP). As the equilibrium state o ⋆ of an EBL model is defined implicitly as the minimum of an energy function, the gradient of the cost C ( o ⋆ , y ) can be computed via implicit differentiation, or more specifically via an algorithm called recurrent backpropagation [Almeida, 1987, Pineda, 1987].

Second baseline: truncated backpropagation (TBP). Our second baseline uses automatic differentiation (i.e. backpropagation) as follows. First, we compute the free state ( h ⋆ , o ⋆ ) . Then, starting

from the free state ( h 0 , o 0 ) = ( h ⋆ , o ⋆ ) we perform K iterations of the fixed point dynamics ; we arrive at ( h K , o K ) , which is also equal to the free state. We then compute the gradient of C ( o K , y ) with respect to the parameters ( θ ), without backpropagating through ( h 0 , o 0 ) . This is a truncated version of backpropagation, where we only backpropagate through the last K iterations but not through the minimization of the energy function. This baseline is also used by Ernoult et al. [2019].

We draw several lessons from Table 1.

Algorithms perform alike on MNIST. Little difference is observed in the test performance of the algorithms on MNIST, ranging from 0.42% to 0.66% test error rate for six of the seven EBL algorithms. This result confirms the current trend in the literature, often skewed towards simple tasks, which sometimes goes to treat all EBL algorithms as one and the same.

Weak positive perturbations perform worse than strong ones. P-EP fails on most datasets (Fashion-MNIST, SVHN, CIFAR-10 and CIFAR-100). P-CpL (the other weakly positively-perturbed algorithm) performs better than P-EP on all tasks, but also fails on CIFAR-100 (91% training error). It is noteworthy that CL, which employs full clamping to the desired output (i.e. a strong positive perturbation), performs better than these two algorithms on all tasks , sometimes by a very large margin (Fashion-MNIST and SVHN).

Negative perturbations yield better results than positive ones. On Fashion-MNIST, CIFAR10 and CIFAR-100, algorithms employing a negative perturbation (N-EP and N-CpL) perform significantly better than those employing a positive perturbation (CL, P-EP and P-CpL). Theorem 2 sheds light on why N-EP performs better than P-EP: N-EP optimizes an upper bound of the cost function, whereas P-EP optimizes a lower bound. We note however that on SVHN, the results obtained with N-EP and N-CpL are much worse than CL - in Appendix E we perform additional simulations where we change the weight initialization of the network ; we find that N-EP and N-CpL generally perform much better than CL, P-EP and P-CpL, supporting our conclusion.

Two-sided perturbations (i.e. centered algorithms) yield better results than one-sided perturbations. While little difference in performance between the centered (C-EP and C-CpL) and negatively-perturbed (N-EP and N-CpL) algorithms is observed on MNIST, FashionMNIST and CIFAR-10, the centered algorithms unlock training on SVHN and significantly improve the test error rate on CIFAR-100 (by ≥ 5 . 4% ). Theorem 3 sheds light on why C-EP performs better than N-EP: the loss function L EP -β ;+ β of C-EP better approximates the cost function (up to O ( β 2 ) ) than the loss function L EP -β of N-EP (up to O ( β ) ).

The EP perturbation method yields better results than the one of CpL. While little difference in performance is observed between C-EP and C-CpL on MNIST and FashionMNIST, C-EP significantly outperforms C-CpL on CIFAR-10 and CIFAR-100, both in terms of training error rate ( ≥ 11 . 9% difference) and test error rate ( ≥ 3 . 8% difference). The same observations hold between N-EP and N-CpL. We also note that the CpL learning rules seem to be less theoretically grounded than the EP learning rules, as detailed in Appendix B.

4.2 State-of-the-art DCHN simulations (performance and speed)

The comparative study conducted in the previous subsection highlights C-EP as the best EBL algorithm among the seven algorithms considered in this work. Using C-EP, we then perform additional simulations on MNIST, CIFAR-10 and CIFAR-100, where we optimize the hyperparameters of training (weight initialization, initial learning rates, number of iterations, value of the nudging parameter and weight decay) to yield the best performance. We use the hyperparameters reported in Table 5 (Appendix D). We report in Table 2 our fastest simulations (100 epochs) as well as the ones that yield the lowest test error rate (300 epochs), averaged over three runs each. We also compare our results with existing works on deep convolutional Hopfield networks (DCHNs).

Table 2 shows that we achieve better simulation results than existing works on DCHNs on all three datasets, both in terms of performance and speed. For instance, our 100-epoch simulations on CIFAR-10 take 3 hours 18 minutes, which is 7 times faster than those reported in [Laborieux and Zenke, 2022] (1 day), and 36 times faster than those reported in Laydevant et al. [2021] (5 days), and our 300-epoch simulations on CIFAR-10 yield 9.70% test error rate, which is significantly lower than [Laborieux and Zenke, 2022] (11.4%). On CIFAR-100, it is interesting to note that our 300-epoch simulations yield a significantly better Top-1 error rate than Laborieux and Zenke [2022] (31.6% vs 38.4%), but a slightly worse Top-5 error rate (14.1% vs 14.0%) ; we speculate that this might be due to our choice of using the mean-squared error instead of the cross-entropy loss used in Laborieux and Zenke [2022]. Since no earlier work on DCHNs has performed simulations on Fashion-MNIST and SVHN, our results reported in Table 1 are state-of-the-art on these datasets as well.

Our important speedup comes from our novel energy-minimization procedure based on 'asynchronous updates', combined with 60 iterations at inference (free phase) and the use of 16-bit precision. In comparison, Laborieux et al. [2021] used 'synchronous updates' with 250 iterations and 32-bit precision. We show in Appendix C that these changes result in a 13.5x speedup on the same device (a A100 GPU) without degrading the performance (test error rate).

We also stress that there still exists an important gap between our SOTA DCHN results and SOTA computer vision models. For example, Dosovitskiy et al. [2020] report 0.5% test error rate on CIFAR-10, and 5.45% top-1 test error rate on CIFAR-100.

5 Conclusion

Our comparative study of energy-based learning (EBL) algorithms delivers a few key take-aways: 1) simple tasks such as MNIST may suggest that all EBL algorithms work equally well, but more difficult tasks such as CIFAR-100 magnify small algorithmic differences, 2) negative perturbations yield better results than positive ones, 3) two-sided (centered) perturbations perform better than one-sided perturbations, and 4) the perturbation technique of equilibrium propagation yields better results than the one of coupled learning. These findings highlight the centered variant of equilibrium propagation (C-EP) as the best EBL algorithm among those considered in the present work, outperforming the second-best algorithm on CIFAR-100 (N-EP) by a significant margin. Our results also challenge some common views. First, it is noteworthy that CL (which uses clamping to the desired output, i.e. a strong positive perturbation) yields better results than P-CpL and P-EP (which use weak positive perturbations): this is contrary to the prescription of Stern et al. [2021] to use small positive perturbations, and opens up questions in the strong perturbation regime [Meulemans et al., 2022]. Second, it is interesting to note that CL [Movellan, 1991, Baldi and Pineda, 1991], EP [Scellier and Bengio, 2017] and CpL [Stern et al., 2021] were originally introduced around the same intuition of bringing the model output values closer to the desired outputs ; the fact that the four best-performing algorithms of our study (C-EP, N-EP, C-CpL and N-CpL) all require a negative perturbation suggests on the contrary that the crucial part in EBL algorithms is not to pull the model outputs towards the desired outputs, but to push away from the desired outputs. Third, Laborieux et al. [2021] explained the very poor performance of P-EP by it approximating the gradient of the cost function up to O ( β ) , but our observation that N-EP considerably outperforms P-EP while also possessing a bias of order O ( β ) shows that this explanation is incomplete ; Theorem 2 provides a complementary explanation: P-EP optimizes a lower bound of the cost function, whereas N-EP optimizes an upper bound.

Our work also establishes new state-of-the-art results for deep convolutional Hopfield networks (DCHNs) on all five datasets, both in terms of performance (accuracy) and speed. In particular, thanks to the use of a novel 'asynchronous' energy-minimization procedure for DCHNs, we manage to reduce the number of iterations required to converge to equilibrium to 60 - compared to 250 iterations used in Laborieux et al. [2021]. Combined with the use of 16-bit precision (instead of 32-bit), this leads our simulations to be 13.5 times faster than those of Laborieux et al. [2021] when run on the same hardware (a A100 GPU).

While our theoretical and simulation results seem conclusive, we also stress some of the limiting aspects of our study. First, our comparative study of EBL algorithms was conducted only on the DCHN model ; further studies will be required to confirm whether or not the conclusions that we have drawn extend to a broader class of models such as resistive networks [Kendall et al., 2020] and elastic networks [Stern et al., 2021]. Second, in our simulations of DCHNs, the performance of the different EBL methods depends on various hyperparameters such as the perturbation strength ( β ), the number of iterations performed to converge to steady state, the weight initialization scheme, the learning rates, the weight decay, etc. Since a different choice of these hyperparameters can lead to different performance, it is not excluded that a deeper empirical study could lead to slightly different conclusions.

Ultimately, while our work is concerned with simulations , EBL algorithms are most promising for training actual analog machines ; we believe that our findings can inform the design of such hardware as well [Dillavou et al., 2022, 2023, Yi et al., 2023, Altman et al., 2023]. Finally, we contend that our theoretical insights and experimental findings may also help better understand or improve novel EBL algorithms [Anisetti et al., 2022, Laborieux and Zenke, 2022, Hexner, 2023, Anisetti et al., 2023], and more generally guide the design of better learning algorithms for bi-level optimization problems [Zucchet and Sacramento, 2022], including the training of Lagrangian systems [Kendall, 2021, Scellier, 2021], meta-learning [Zucchet et al., 2022, Park and Simeone, 2022], direct loss minimization [Song et al., 2016] and predictive coding [Whittington and Bogacz, 2019, Millidge et al., 2022].

Acknowledgments and Disclosure of Funding

The authors thank Mohammed Fouda, Ludmila Levkova, Nicolas Zucchet, Axel Laborieux, Nachi Stern, Sam Dillavou and Arvind Murugan for discussions. This work was funded by Rain AI. Rain AI has an interest in commercializing technologies based on brain-inspired learning algorithms.

References

B. Almeida. A learning rule for asynchronous perceptrons with feedback in a combinatorial environment. In Proceedings, 1st First International Conference on Neural Networks , volume 2, pages 609-618. IEEE, 1987.

E. Altman, M. Stern, A. J. Liu, and D. J. Durian. Experimental demonstration of coupled learning in elastic networks. arXiv preprint arXiv:2311.00170 , 2023.

R. Anisetti, A. Kandala, B. Scellier, and J. Schwarz. Frequency propagation: Multi-mechanism learning in nonlinear physical networks. arXiv preprint arXiv:2208.08862 , 2022.

R. Anisetti, B. Scellier, and J. M. Schwarz. Learning by non-interfering feedback chemical signaling in physical networks. Physical Review Research , 5(2):023024, 2023.

Baldi and F. Pineda. Contrastive learning and neural oscillations. Neural Computation , 3(4): 526-545, 1991.

Dillavou, M. Stern, A. J. Liu, and D. J. Durian. Demonstration of decentralized physics-driven learning. Physical Review Applied , 18(1):014040, 2022.

Dillavou, B. Beyer, M. Stern, M. Z. Miskin, A. J. Liu, and D. J. Durian. Circuits that train themselves: decentralized, physics-driven learning. In AI and Optical Data Sciences IV , volume 12438, pages 115-117. SPIE, 2023.

Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Du and I. Mordatch. Implicit generation and modeling with energy based models. Advances in Neural Information Processing Systems , 32, 2019.

Ernoult, J. Grollier, D. Querlioz, Y. Bengio, and B. Scellier. Updates of equilibrium prop match gradients of backprop through time in an rnn with static input. In Advances in Neural Information Processing Systems , pages 7079-7089, 2019.

Geng, J. Wang, Z. Gao, J. Frellsen, and S. Hauberg. Bounds all around: training energy-based models with bidirectional bounds. Advances in Neural Information Processing Systems , 34: 19808-19821, 2021.

Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03263 , 2019.

Hexner. Training precise stress patterns. Soft Matter , 19(11):2120-2126, 2023.

E. Hinton, T. J. Sejnowski, and D. H. Ackley. Boltzmann machines: Constraint satisfaction networks that learn . Carnegie-Mellon University, Department of Computer Science Pittsburgh, PA, 1984.

Høier, D. Staudt, and C. Zach. Dual propagation: Accelerating contrastive hebbian learning with dyadic neurons. In Proceedings of the 40th International Conference on Machine Learning , pages 13141-13156, 2023.

J. Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the national academy of sciences , 81(10):3088-3092, 1984.

Kendall. A gradient estimator for time-varying electrical networks with non-linear dissipation. arXiv preprint arXiv:2103.05636 , 2021.

Kendall, R. Pantone, K. Manickavasagam, Y. Bengio, and B. Scellier. Training end-to-end analog neural networks with equilibrium propagation. arXiv preprint arXiv:2006.01981 , 2020.

Z. Kiraz, D.-K. G. Pham, and P. Desgreys. Impacts of Feedback Current Value and Learning Rate on Equilibrium Propagation Performance. In 2022 20th IEEE Interregional NEWCAS Conference (NEWCAS) , pages 519-523, Quebec City, Canada, June 2022. IEEE. doi: 10.1109/NEWCAS52662. 2022.9842178. URL https://hal.telecom-paris.fr/hal-03779416 .

Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009. URL https://www.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf .

Laborieux and F. Zenke. Holomorphic equilibrium propagation computes exact gradients through finite size oscillations. Advances in Neural Information Processing Systems , 35:12950-12963, 2022.

Laborieux, M. Ernoult, B. Scellier, Y. Bengio, J. Grollier, and D. Querlioz. Scaling equilibrium propagation to deep convnets by drastically reducing its gradient estimator bias. Frontiers in neuroscience , 15:129, 2021.

Laydevant, M. Ernoult, D. Querlioz, and J. Grollier. Training dynamical binary neural networks with equilibrium propagation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4640-4649, 2021.

Laydevant, D. Markovic, and J. Grollier. Training an ising machine with equilibrium propagation. arXiv preprint arXiv:2305.18321 , 2023.

LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998.

P. Lillicrap, A. Santoro, L. Marris, C. J. Akerman, and G. Hinton. Backpropagation and the brain. Nature Reviews Neuroscience , pages 1-12, 2020.

Luczak, B. L. McNaughton, and Y. Kubo. Neurons learn by predicting future activity. Nature Machine Intelligence , 4(1):62-72, 2022.

Meulemans, N. Zucchet, S. Kobayashi, J. Von Oswald, and J. Sacramento. The least-control principle for local learning at equilibrium. Advances in Neural Information Processing Systems , 35:33603-33617, 2022.

Millidge, Y. Song, T. Salvatori, T. Lukasiewicz, and R. Bogacz. Backpropagation at the infinitesimal inference limit of energy-based models: Unifying predictive coding, equilibrium propagation, and contrastive hebbian learning. arXiv preprint arXiv:2206.02629 , 2022.

R. Movellan. Contrastive hebbian learning in the continuous hopfield model. In Connectionist Models , pages 10-17. Elsevier, 1991.

Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. 2011.

Nijkamp, M. Hill, S.-C. Zhu, and Y. N. Wu. Learning non-convergent non-persistent short-run mcmc toward energy-based model. Advances in Neural Information Processing Systems , 32, 2019.

Park and O. Simeone. Predicting flat-fading channels via meta-learned closed-form linear filters and equilibrium propagation. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 8817-8821. IEEE, 2022.

Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017.

Peterson and A. Lavin. Physical computing for materials acceleration platforms. Matter , 5(11): 3586-3596, 2022.

J. Pineda. Generalization of back-propagation to recurrent neural networks. Physical review letters , 59(19):2229, 1987.

Scellier. A deep learning theory for neural networks grounded in physics . PhD thesis, Université de Montréal, 2021.

Scellier and Y. Bengio. Equilibrium propagation: Bridging the gap between energy-based models and backpropagation. Frontiers in computational neuroscience , 11:24, 2017.

Scellier, S. Mishra, Y. Bengio, and Y. Ollivier. Agnostic physics-driven deep learning. arXiv preprint arXiv:2205.15021 , 2022.

Song, A. Schwing, R. Urtasun, et al. Training deep neural networks via direct loss minimization. In International conference on machine learning , pages 2169-2177. PMLR, 2016.

Stern and A. Murugan. Learning without neurons in physical systems. Annual Review of Condensed Matter Physics , 14:417-441, 2023.

Stern, D. Hexner, J. W. Rocks, and A. J. Liu. Supervised learning in physical networks: From machine learning to learning machines. Physical Review X , 11(2):021045, 2021.

Stern, S. Dillavou, M. Z. Miskin, D. J. Durian, and A. J. Liu. Physical learning beyond the quasistatic limit. Physical Review Research , 4(2):L022037, 2022.

Stern, S. Dillavou, D. Jayaraman, D. J. Durian, and A. J. Liu. Physical learning of power-efficient solutions. arXiv preprint arXiv:2310.10437 , 2023.

Watfa, A. Garcia-Ortiz, and G. Sassatelli. Energy-based analog neural network framework. Frontiers in Computational Neuroscience , 17:1114651, 2023.

C. Whittington and R. Bogacz. Theories of error back-propagation in the brain. Trends in cognitive sciences , 23(3):235-250, 2019.

Wycoff, S. Dillavou, M. Stern, A. Liu, and D. Durian. Desynchronous learning in a physics-driven learning network. The Journal of Chemical Physics , 156(14):144903, 2022.

Xiao, K. Rasul, and R. Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 , 2017.

S.-i. Yi, J. D. Kendall, R. S. Williams, and S. Kumar. Activity-difference training of deep neural networks using memristor crossbars. Nature Electronics , 6(1):45-51, 2023.

Zucchet and J. Sacramento. Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation. Neural Computation , 34(12):2309-2346, 2022.

Zucchet, S. Schug, J. V on Oswald, D. Zhao, and J. Sacramento. A contrastive rule for meta-learning. Advances in Neural Information Processing Systems , 35:25921-25936, 2022.

A Loss functions of the equilibrium propagation learning rules

In this appendix, we prove Theorems 2 and 3. To do so, we state and prove Theorems 4 and 5 below, of which Theorems 2 and 3 are direct consequences.

In this appendix, the input x and label y are fixed, so we omit them in the notation for simplicity. For fixed θ , we define for any value of β the minimum of the augmented energy when h and o are free:

We further define the contrastive functions (or 'surrogate functions')

and

We also recall the definition of the steady state values of output variables:

Finally, we recall that the equilibrium propagation (EP) perturbed state (4) is defined as

and that the EP learning rules (5) and (6) read

and

In particular, o EP 0 = o ( θ ) by definition.

Theorem 4. The derivative of the function β → F ( β, θ ) at β = 0 is equal to C ( o ( θ )) . In particular, as β → 0 , we have

Furthermore, we have for any β > 0

Theorem 5. The equilibrium propagation learning rules satisfy

Theorem 4 is borrowed from Scellier et al. [2022], where these statements are proved in the context of 'agnostic equilibrium propagation'. Theorem 5 is a new contribution. For clarity to ensure the manuscript is self-contained, we provide here proofs for both Theorem 4 and Theorem 5.

First, we introduce a new notation: we introduce the function

Using this notation, for every β ∈ R , the first order conditions for the steady state ( h EP β , o EP β ) read:

and the function F (20) rewrites

Proof of Theorem 4. Using the chain rule of differentiation, we have

Therefore and for β < 0 we get

Hence the result:

Evaluating this expression at β = 0 , and using o EP 0 = o ( θ ) by definition, we get

which is the first statement of Theorem 4. Using a Taylor expansion of F ( β, θ ) around β = 0 , we have

Subtracting F ( θ, 0) on both sides and dividing by β , we get

which is the first part of equation (27). Similarly, we write Taylor expansions for F ( β, θ ) and F ( -β, θ ) ,

Subtracting the second equation from the first one, and dividing by 2 β , we get

which is the second part of equation (27).

Next, we prove the upper bound and lower bound properties. To this end, we write for any β ∈ R ,

where o ( θ ) is the free state value of output variables. Subtracting F ( θ, 0) on both sides we get

Next we divide by β . For β > 0 , we get

Similarly, it can be shown more generally that the function β → L EP β ( θ ) is non-increasing, or equivalently that the function β ↦→ F ( β, θ ) is concave.

Proof of Theorem 5. First, using (21), we have

Similar to the proof of Theorem 4, using the chain rule of differentiation, we have for every β ∈ R

Here again, we have used the first-order steady state conditions for h EP β and o EP β . Therefore

Injecting (49) in (47), we get

We conclude using the definitions of the equilibrium propagation learning rule (5). We prove similarly the case of centered EP with ∂ L EP -β ;+ β ∂θ ( θ ) .

B On the coupled learning rules

In this appendix, we prove some theoretical results related to the coupled learning rules.

Recall the definition of the free state

the definition of the perturbed state of coupled learning (CpL)

and the definition of the (positively-perturbed) CpL rule

Similar to the contrastive function L CL of contrastive learning (10) and the surrogate loss L EP β of equilibrium propagation (21), Stern et al. [2022] define the contrast

and argue that the CpL rule performs gradient descent on L (2) CpL . Besides, Stern et al. [2021] argue that the CpL rule optimizes both the function

and the squared error loss

where o ( θ, x ) = o ⋆ is the free output value.

In this appendix, we show that the CpL algorithm does not perform gradient descent on L (2) CpL (Appendix B.1). We also show that, in general, the CpL rule optimizes neither the squared error L MSE , nor the function L (1) CpL . To this end, we present two examples: in the first one, the function L MSE increases after a single step of the coupled learning rule (Appendix B.2) ; in the second one, the function L (1) CpL increases (Appendix B.3).

B.1 Coupled learning does not perform gradient descent on the contrast L (2) CpL

Stern et al. [2022] note that L (2) CpL is non-negative and achieves its minimum possible value ( L (2) CpL = 0 ) when o ⋆ = y , which motivates them to minimize L (2) CpL by gradient descent. However, as we show here, the coupled learning rule does not perform gradient descent on L (2) CpL . Indeed, while the free state is characterized by

the perturbed state of CpL only satisfies

The partial derivative ∂E ∂o ( θ, x, h CpL β , o CpL β ) does not vanish in general, in contrast with the perturbed state of equilibrium propagation (EP), which satisfies (31). Thus, we have

and

As a result, the gradient of L (2) CpL is

̸

The last term does not vanish in general, unless β = 1 . We note that the case β = 1 corresponds to contrastive learning (CL, section 2.1) ; hence we have recovered Theorem 1, known since Movellan [1991].

̸

Therefore, contrary to CL that performs gradient descent on L CL and EP that performs gradient descent on L EP β for any value of β , the CpL rule does not perform gradient descent on L (2) CpL for β = 1 .

B.2 One step of coupled learning may increase the squared error L MSE

Next, we show in an example that CpL does not optimize L MSE . Let A be the 2 × 2 square matrix

and b : R 2 → R 2 the function defined by

We define the energy function E : R 2 × R 2 → R by

For simplicity, there is no input ( x ) and no hidden variable ( h ) in our example. Since the matrix A is symmetric positive definite, the minimum of o ↦→ E ( θ, o ) is achieved at the point o ⋆ = b ( θ ) , where E ( θ, o ⋆ ) = 0 . Next, let's assume that the 'desired output' is y := (0 , 0) ∈ R 2 . The perturbed state of the CpL algorithm is

where β ∈ R is the nudging parameter, and the P-CpL rule is

where η is the learning rate. Using (69) we calculate

̸

̸

In particular, and

where we have used (70). So the P-CpL rule (71) rewrites 8

Next, we define the squared error between the prediction o ⋆ and the 'desired output' y = (0 , 0) ,

To conclude, we show that for η small enough, L MSE ( θ 0 + ∆ CpL θ ) > L MSE ( θ 0 ) at the point θ 0 = (0 , 0) independently of the value of the nudging parameter β ∈ R . To show this, first we calculate ∂ L MSE ∂θ · ∆ CpL θ . We have

so that

Let us calculate:

and

Since at the point θ 0 = (0 , 0) we have

Therefore, at the point θ 0 = (0 , 0) ,

Finally,

Thus, provided that the learning rate η > 0 is small enough, the loss value L MSE ( θ 0 +∆ θ ) after one step of coupled learning is larger than L MSE ( θ 0 ) . This holds for any nudging value β ∈ R .

it follows that

B.3 One step of coupled learning may increase L (1) CpL

Finally, we show that CpL does not optimize L (1) CpL . To this end, we consider a second example, where we define the energy function E : (0 , 5 / 4) × R → R by

The minimum of o ↦→ E ( θ, o ) is obtained at the point o ⋆ = θ , where E ( θ, o ⋆ ) = 0 . As in the previous example, we assume that the 'desired output' is y := 0 , so the perturbed state of CpL is

and

so the CpL rule reads and the CpL rule is

We calculate

In particular,

Now, recall the definition of the function L (1) CpL of (16). In this example, it is equal to

and its derivative (or 'gradient') is

At the point θ 0 = 1 we have ( L (1) CpL ) ′ ( θ 0 ) = -2 , and ∆ CpL θ = -η (1 + 2 β ) . This means that ( L (1) CpL ) ′ ( θ 0 ) < 0 , and ∆ CpL θ < 0 for any nudging value β > -1 / 2 . Thus, for a small η and any β > -1 / 2 , the function L (1) CpL increases after a single step of the CpL rule.

C Energy minimization procedure for deep convolutional Hopfield networks via asynchronous updates

Prior works on deep convolutional Hopfield networks [Ernoult et al., 2019, Laborieux et al., 2021, Laydevant et al., 2021, Laborieux and Zenke, 2022] minimized the energy function (18) using a 'synchronous update' scheme, where at every iteration, all the layers are updated synchronously. In this work, we use an 'asynchronous update' scheme.

Recall the form of the energy function of a deep convolutional Hopfield network (DCHN), given by (18). Ernoult et al. [2019] write the energy function in the form

where ∥ s ∥ 2 := ∑ 5 k =1 ∥ s k ∥ 2 and Φ is the so-called 'primitive function', defined as

In this expression, s = ( s 0 , s 1 , s 2 , s 3 , s 4 , s 5 ) is the state of the network, x = s 0 is the input layer, h = ( s 1 , s 2 , s 3 , s 4 ) is the hidden variable, o = s 5 is the output variable, w k and b k are the kernel (the weights) and bias of layer k , ⋆ is the convolution operation, P is the max pooling operation, · is the scalar product for pairs of tensors, and θ = { w k , b k | 1 ≤ k ≤ 5 } is the set of model parameters. Ernoult et al. [2019] define the dynamics

where σ ( · ) = max(0 , min( · , 1)) is the 'hard sigmoid function', which they assume to converge to a fixed point s ⋆ characterized by

Importantly, the fixed point s ⋆ is also a critical point of the energy function, s ↦→ E ( θ, s ) , since ∂E ∂s ( θ, s ⋆ ) = s ⋆ -∂ Φ ∂s ( θ, s ⋆ ) = 0 . This dynamics is what we call here 'synchronous update' scheme, because at every iteration (or 'time step') t , all the layers s k ( 1 ≤ k ≤ 5 ) are updated synchronously.

In this work, we use an 'asynchronous update scheme', where at each iteration, we first update the layers of even indices, and then we update the layers of odd indices. Denoting s o = ( s 1 , s 3 , s 5 ) the configuration of odd layers, and s e = ( s 2 , s 4 ) the configuration of even layers, the asynchronous update scheme reads

where σ o is the activation function of odd layers, and

where σ e is the activation function of even layers. For the layers of index k such that 1 ≤ k ≤ 4 , the activation function is σ k ( · ) = max(0 , min( · , 1)) , i.e. the 'hard sigmoid function', and σ 5 is the identity (the 'linear activation function').

In this work, we use the max pooling operation P max . Although for this choice of pooling operation we do not have a proof of convergence of either of the two schemes ('synchronous' and 'asynchronous'), we find experimentally that the asynchronous scheme converges faster. This allows us to reduce the number of iterations in the free and perturbed phases and thus get a significant speedup, as explained in the next subsection.

In the case where P is the average (mean) pooling operation, however, we can prove that the energy decreases at each step of the asynchronous update scheme. Indeed, for the average pooling operation P avg , the primitive Φ is a linear function of s e , i.e. the energy E is of the form

for some constants B e and C e that do not depend on s e . Specifically

for every layer k . It is easily seen that the minimum of such a function E ( s e ) in R dim( s 2 ) × R dim( s 4 ) is obtained at the point s e = B e . It is also easily seen that the minimum of E ( s e ) in [0 , 1] dim( s 2 ) × [0 , 1] dim( s 4 ) is obtained at the point s e = max(0 , min( B e , 1)) . Similar expressions hold for the configuration of odd layers s o that minimizes the energy function given the state of even layers fixed. Given that Φ is a linear function of s o , we have that B o = ∂ Φ ∂s o ( θ, s ) , and similarly B e = ∂ Φ ∂s e ( θ, s ) . Thus, we conclude that the energy function decreases at each step of the asynchronous update scheme (100)-(101).

For the max pooling operation used in this work, the primitive function Φ is not a linear function of s e or s o , and the above argument no longer holds.

C.1 13.5x simulation speedup compared to Laborieux et al. [2021]

We compare in simulations two settings:

In the first setting, we use the asynchronous update scheme with 60 iterations at inference, 20 iterations in the perturbed phases, and 16 bit precision. This is the setting of the simulations performed for the comparative study in the present work.

In the second setting, we use the synchronous update scheme with 250 iterations at inference, 30 iterations in the perturbed phase, and 32 bits precision. This is (up to minor differences) the setting of the simulations of Laborieux et al. [2021].

We perform the simulations using the C-EP algorithm on CIFAR-10, and we use the hyperparameters reported as 'SOTA results on CIFAR10' in Table 5. We perform three runs in the first setting, and one run in the second setting, and we report the results in Table 3. We see that the first setting is 13.5 times faster than the second setting, requiring only 3 hours and 18 minutes to complete on a single A-100 GPU (compared to the 44 hours 29 minutes required in the second setting). Furthermore, the first setting also performs marginally better in terms of training and test error rates than the second one.

Table 3: Comparison of the performance and the wall-clock-time between the simulation setting of Laborieux et al. [2021] and ours. The simulations of Laborieux et al. [2021] use the synchronous update scheme, in conjunction with 32 bits precision, 250 iterations at inference (free phase), and 30 iterations in the perturbed phase. Our asynchronous update scheme is used in conjunction with 16 bits precision, 60 iterations at inference and 15 iterations in the perturbed phase. Simulations are performed on CIFAR-10 using the C-EP algorithm for training. We perform 100 epochs and use the hyperparameters of the 'SOTA CIFAR-10 simulations' provided in Table 5. We report the test error rate (Test Error), the training error rate (Train Error) and the wall-clock time (WCT). WCT is reported as HH:MM.

D Simulation details

Datasets. We perform simulations on the MNIST, Fashion-MNIST, SVHN, CIFAR-10 and CIFAR100 datasets.

The MNIST dataset is composed of images of handwritten digits [LeCun et al., 1998]. Each image x in the dataset is a 28 × 28 gray-scaled image and comes with a label y ∈ { 0 , 1 , . . . , 9 } indicating the digit that the image represents. The dataset contains 60,000 training images and 10,000 test images.

The Fashion-MNIST dataset [Xiao et al., 2017] shares the same image size, data format and the same structure of training and testing splits as MNIST. It comprises a training set of 60,000 images and a test set of 10,000 images. Each example is a 28 × 28 grayscale image from ten categories of fashion products.

The SVHN dataset [Netzer et al., 2011] contains color images of 32 × 32 pixels, derived from house numbers in Google Street View images. Like in MNIST, each image comes with a label corresponding to a digit from 0 to 9 . The number of images per class varies due to the natural distribution of digits in the real world. The dataset contains a training set with 73 , 257 images, and a test set containing 26 , 032 images. The dataset also contains an additional set of 531 , 131 other (somewhat less difficult) images, which we do not use in our simulations.

The CIFAR-10 dataset [Krizhevsky et al., 2009] consists of 60 , 000 colour images of 32 × 32 pixels. These images are split in 10 classes (each corresponding to an object or animal), with 6 , 000 images per class. The training set consists of 50 , 000 images and the test set of 10 , 000 images.

Similarly, the CIFAR-100 dataset [Krizhevsky et al., 2009] also comprises 60 , 000 color images with a resolution of 32 × 32 pixels, featuring a diverse set of objects and animals. These images are categorized into 100 distinct classes, each containing 600 images. Like CIFAR-10, the dataset is divided into a training set with 50 , 000 images and a test set containing the remaining 10 , 000 images.

Data pre-processing and data augmentation. The data is normalized as in Table 4.

Since the deep convolutional Hopfield network used in our simulations takes as input a 32x32-pixel image, we augment each image of MNIST and Fashion-MNIST to a 32x32-pixel image by adding two pixels at the top, two pixels at the bottom, two pixels to the left and two pixels to the right.

On the training sets of Fashion-MNIST, CIFAR-10 and CIFAR-100, we also use random horizontal flipping. (Random flipping is not used on the training sets of MNIST and SVHN, and it is never used at test time either.)

Table 4: Data normalization. We normalize the input images using the recommended mean ( µ ) and std ( σ ) values for each dataset. The MNIST and Fashion-MNIST images are gray-scale, i.e. they have a unique channel. The SVHN, CIFAR-10 and CIFAR-100 images are color images, i.e. they have three channels.

Network architecture and hyperparameters. Table 5 contains the architectural details of the network, as well as the hyperparameters used to obtain the results presented in Table 1, Table 2 and Table 6. The hard-sigmoid activation function is defined as σ ( h ) := max(0 , min( h, 1)) .

Weight initialization. We initialize the weights of dense interactions and convolutional interactions according to

which is the 'Kaiming uniform' scheme rescaled by a factor α , that we call the 'gain' here (i.e. a scaling number). For dense weights of shape ( size pre , size post ), we have fan in = size pre ; for convolutional weights of shape ( channel in , channel out , height , width ), we have fan in = channel in × height × width . See Table 5 for the choice of the gains.

Energy minimization via asynchronous updates. To compute the steady state of the network, we use the 'asynchronous update' scheme described in appendix C: at every iteration, we first update the layers of even indices (the first half of the layers) and then we update the layers of odd indices (the other half of the layers). Relaxing all the layers once (first the even layers, then the odd layers) constitutes one 'iteration'. We repeat as many iterations as is necessary until convergence to the steady state. In Table 5, we denote T the number of iterations performed at inference (free phase), and we denote K the number of iterations performed in the perturbed phase.

Training procedure. We train our networks with seven energy-based learning (EBL) algorithms, namely: contrastive learning (CL), positively-perturbed equilibrium propagation (P-EP), negativelyperturbed EP (N-EP), centered EP (C-EP), positively-perturbed coupled learning (P-CpL), negativelyperturbed CpL (N-CpL) and centered CpL (C-CpL). Given an EBL algorithm, at each training step of SGD, we proceed as follows. First we pick a mini-batch of samples in the training set, x ,

and their corresponding labels, y . Then we set the nudging parameter to 0 and we perform a free phase of T asynchronous iterations. This phase allows us in particular to measure the training loss and training error rate for the current mini-batch, to monitor training. We also store the free state ( h ⋆ , o ⋆ ) . Next, let's call β 1 and β 2 the two nudging values used for training (which depend on the EBL algorithm). First, we set the nudging parameter to β 1 and we perform a new relaxation phase of K asynchronous iterations. Next, we reset the state of the network to the free state ( h ⋆ , o ⋆ ) , then we set the nudging parameter to the second nudging value β 2 , and we perform the last relaxation phase for K asynchronous iterations. Finally, we update all the parameters simultaneously in a single 'parameter update' phase.

Optimizer and scheduler. We use mini-batch gradient descent (SGD) with momentum and weight decay. We also use a cosine-annealing scheduler for the learning rates with hyperparameters T max and η min .

Computational resources. The code for the simulations uses PyTorch 1.13.1 and TorchVision 0.14.1. Paszke et al. [2017]. The simulations were carried on Nvidia A100 GPUs. For the comparative study, we used five GPUs (one GPU per dataset) and let them run for six days to complete all the simulations.

E Additional simulations on SVHN for the comparative study

The results of Table 1 show that most algorithms (P-EP, N-EP, P-CpL and N-CpL) perform very poorly on SVHN: the training process often gets stuck at 81.08% training error and 80.41% test error rate. These results seem to go against our conclusion that N-EP and N-CpL generally perform (much) better than C-EP and C-CpL and CL. We hypothesize that this poor performance on SVHN is due to the network being poorly initialized for this classification task, and we perform additional simulations on SVHN where we change the initialisation scheme of the weights of the network - see Table 5 for the new hyperparameters used for these additional experiments. As for the comparative study of Section 4.1, with this new set of hyperparameters, we compare the seven EBL algorithms of section 2 by training the DCHN of section 3 on SVHN. We also compare the performance of these algorithms to recurrent backpropagation (RBP) and truncated backpropagation (TBP). The results of these additional simulations are reported in Table 6.

Table 6 shows that the N-EP and P-CpL simulations have large variance, so we also report the results for each run. The three runs of N-EP gave the following results: 1) 3.73% test error, 2) 3.87% test error, and 3) 80.41% test error. The third N-EP run was indeed also blocked at 81.08% training error and 80.41% test error (like many other runs). The three runs of P-CpL gave the following results: 1) 71.05% test error, 2) 15.35% test error, and 3) 15.57% test error.

The results of Table 6 are consistent with our conclusions of Section 4.1: 1) strong positive perturbations (CL) yield better results than weak positive perturbations (P-CpL and P-EP), 2) negative perturbations (N-EP, N-CpL) yield better results than positive perturbations (CL, C-CpL, C-EP), 3) two-sided perturbations (C-EP, C-CpL) perform better than one-sided perturbations (N-EP, N-CpL, CL, P-CpL, P-EP), and 4) the EP perturbation technique yields better results than the CpL perturbation technique.

Table 6: Additional experiments on SVHN using a different set of hyperparameters. We compare the seven EBL algorithms and benchmark them against truncated backpropagation (TBP) and recurrent backpropagation (RBP). We report the training and test error rates, averaged over three runs. The hyperparameters used for this study are detailed in Table 5.

In Table 7, we report the results of Table 1 together with the test std values.

Peer reviews:
comments: SUMMARY:
This paper aims to investigate existing Energy-based learning algorithms on equal footing with same models and datasets. Energy-based algorithms include contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL) have been carried out for comparison. The experiments conducted based on deep Hopfield networks (DHNs) show that the centered variant of EP is the best-performing algorithm.

STRENGTHS:
This work provides systematic comparison of the different energy-based learning algorithms. The idea is meaningful and the presentation is satisfying.

WEAKNESSES:
it does not present new solutions for efficient energy design/ learning, the generalizability of the comparison (not limited on DCHN) should be further discussed.

QUESTIONS:
1.	In this paper, what’s the reason that deep convolutional Hopfield network (DCHN) is suitable for generic comparison? The authors should present detailed illustrations.
2.	The experimental results show that equilibrium propagation performs better than coupled learning. Does this conclusion hold true for large-scale data training? The authors need to present deeper analyses or proofs.
3.	It seems that this paper does not provide new solutions for efficient energy design/ learning. According to the conclusion presented in this paper, how can we design powerful EBL models with more complex network architectures?

LIMITATIONS:
See weaknesses.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on exploring and comparing various energy-based training methods for deep convolutional Hopfield networks. The performance of contrastive learning, positively-perturbed, negatively-perturbed, and centered versions of Equilibrium propagation and Coupled learning algorithms are evaluated. The paper establishes state-of-the-art (SOTA) results for some of these algorithms.\n\nSTRENGTHS:\nThe set of experiments conducted in this paper provides a comprehensive comparison of the performance of different training methods on a variety of datasets. This extensive evaluation contributes to a robust understanding of the performance of deep convolutional Hopfield networks (DCHN) and their applicability across different datasets.\n\nWEAKNESSES:\n1) One issue with this paper is the lack of citation and discussion of recent works on Energy Based Models (EBMs) that are relevant to the topic, such as Grathwohl et al. (2019), Nijkamp et al. (2019b;a), and Du & Mordatch (2019). These papers analyze the application of EBMs to image classification tasks, propose different training techniques, and compare their performance. It would be beneficial to include a discussion of these works and how they relate to the research presented in this paper.\n\n2) Subsection 2.5, which presents the theoretical results, appears disconnected from the main body of the paper. There is no discussion of the implications of these theorems or how they relate to the research findings. It is important to establish a clear connection between this theoretical section and the rest of the paper to provide a cohesive narrative.\n\nQUESTIONS:\nWhat factors motivated the authors to choose the Deep Convolutional Hopfield Network (DCHN) model as the framework for comparing the training methods in the paper?\n\nLIMITATIONS:\nThe paper does not include a discussion on the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
This work conducts an extensive comparison of several energy-based learning (EBL) algorithms, including contrastive learning (CL), equilibrium propagation (EP) and coupled learning (CpL). Depending on the type of perturbation used, 9 variants of EP and CpL are examined. Deep Hopfield networks (DHNs) on five vision tasks (MNIST, F-MNIST, SVHN, CIFAR-10 and CIFAR-100) are trained, and different EBL algorithms are compared.

STRENGTHS:
Investigating energy-based learning algorithms is interesting, especially for their compatibility with analog (post-digital) hardware. The paper is generally well-written. The concepts and algorithms are mostly clearly presented. Experiments are detailed with good analysis.

WEAKNESSES:
Not clear that Theorem 1-3 are new, or known from existing literatures.

The paper is good for a comparison study of different EBL algorithms, but the algorithm contribution is limited.

The performance of EBL-trained DCHNs on vision tasks (e.g., CIFAR-10) are far behind modern neural networks. It would be better to add such information and comparison.
https://paperswithcode.com/sota/image-classification-on-cifar-10

QUESTIONS:
see above.

LIMITATIONS:
see above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

comments: SUMMARY:
This work reviews and compares recent EBL methods on classic image classification benchmarks. The paper first reviews a variety of EBL methods including CL, EP, P-EP, N-EP, C-EP, CpL, P-CpL, N-CpL, and C-CpL. Next, two theorems are presented which show that P/N/C EP approximate gradient descent on the cost function. The experiment section presents a large-scale comparison of the EBL methods along with Backprop methods for image classification on image datasets whose complexity ranges from MNIST to CIFAR-100. The study finds that N-type EBL variants outperform P-type EBL variants, and that C-EP performs best overall. The study also finds that MNIST is not a suitable dataset for identifying differences between method outcomes, and that more complex datasets like CIFAR-100 reveal important differences. A discussion of possible explanations for the results is presented.

STRENGTHS:
* The paper is a very useful resource for understanding recent EBL methods.
* Theorems 2 and 3 very nicely corroborate the empirical observations about the effectiveness of P-type over N-type, and the overall superiority of C-type.
* A large-scale study of EBL methods gives crucial context for understanding which EBL techniques are useful and which are not. The findings could serve to motivation directions for developing improved EBL algorithms.

WEAKNESSES:
* The primary weakness of this paper is a lack of novelty. It provides a very useful survey of the literature and a much-needed large scale comparison of EBL methods. However, no new techniques are proposed, and the experimental evaluation is somewhat routine.

QUESTIONS:
* Can the ideas in this paper be used to propose a new EBL algorithm which draws from the useful properties of the best current models?

LIMITATIONS:
Limitations are not discussed,
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-07