# Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks

Victor Cook, Gilles Sassatelli, Marina Reyboz, Thierry Gil, Mohamed Watfa

► To cite this version:

Victor Cook, Gilles Sassatelli, Marina Reyboz, Thierry Gil, Mohamed Watfa. Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks. 2026. lirmm-05525004

HAL Id: lirmm-05525004

https://hal-lirmm.ccsd.cnrs.fr/lirmm-05525004v1

Preprint submitted on 24 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY-SA 4.0 - Attribution - ShareAlike - International
License
# Generative Replay for Equilibrium Propagation:
Toward Edge-Friendly Continual Learning in
Hopfield Networks

Victor Cook*, Gilles Sassatelli*, Marina Reyboz†, Thierry Gil*, Mohamed Watfa*

*LIRMM, Univ. Montpellier / CNRS, Montpellier, France

†Univ. Grenoble Alpes, CEA, List, Grenoble, France

{victor.cook, gilles.sassatelli, thierry.gil, mohamed.watfa}@lirmm.fr

marina.reyboz@cea.fr

***Abstract***—Continual learning at the edge requires models that can adapt to non-stationary data streams under strict memory and energy constraints. While Equilibrium Propagation (EqProp) in Hopfield Networks offers a biologically grounded and hardware-aligned alternative to backpropagation, its behavior in continual learning settings remains largely unexplored. In particular, standard ER relies on stored samples, which conflicts with analog and neuromorphic implementations where external memory is costly or unavailable.

This work addresses the problem of catastrophic forgetting in EqProp-trained Hopfield Networks without relying on data buffers or architectural additions. We introduce an Energy-Based Generative Replay strategy (EB-GenReplay) that exploits the intrinsic generative dynamics of Hopfield Networks, allowing a single model to act as both classifier and replay generator. Past task samples are regenerated by relaxing a frozen copy of the network, enabling rehearsal through internally generated pseudo-data.

We evaluate the proposed method on the Split-MNIST benchmark and compare it against FineTune and Experience Replay (ER). The results show that EB-GenReplay significantly improves stability while preserving plasticity, achieving competitive average accuracy (76% for EB-GenReplay vs. 85% for ER) without the need for external memory. These findings position EqProp-trained Hopfield Networks as a compact and edge-friendly foundation for continual learning.

***Index Terms***—Dynamic Neural Networks, Efficient and Tiny Neural Networks, Physics-Inspired Neural Networks and Neural Operators, Continual Learning

## I. INTRODUCTION

Continual learning (CL) studies how a model can acquire new knowledge from a non-stationary data stream while retaining previously learned information [1]–[3]. This setting arises naturally in long-lived systems deployed in changing environments. In contrast to the standard offline assumption of a fixed dataset and a single training phase, continual learning requires models to balance *plasticity*, the ability to learn new tasks, with *stability*, the ability to preserve past knowledge. Standard neural networks trained with gradient descent fail in this regime due to *catastrophic forgetting*, where learning new tasks rapidly degrades performance on earlier ones [4].

A variety of approaches have been proposed to mitigate catastrophic forgetting, including parameter regularization [5],

[6], dynamic or task-expanding architectures [7], and replay-based strategies [8], [9]. While effective, these methods typically rely on explicit memory, auxiliary models, or architectural growth, which limits their applicability in resource-constrained and edge-oriented settings.

A few works have evaluated the ability of energy-based models (EBMs) to perform continual learning. Li et al. shows that an EBM can naturally mitigate forgetting only using contrastive training instead of softmax-based training [10]. However, their results are obtained using contrastive divergence and backpropagation to shape the energy landscape. As such, they characterize the continual learning behavior of EBMs under globally optimized training rules, leaving open how energy-based objectives behave when learning is governed by *local*, dynamical update rules. Such rules are of particular interest as they align with biological learning mechanisms and avoid global gradient transport, which is known to dominate energy and data movement costs in large-scale and hardware-constrained systems [11].

A natural candidate to study this regime is Equilibrium Propagation (EqProp) [12], a training algorithm explicitly designed for energy-based networks and defined through the system's relaxation dynamics. EqProp replaces backpropagation with a local learning rule obtained by comparing two nearby equilibrium states of the same network, a free relaxation and a weakly perturbed one. When applied to Hopfield Networks, EqProp couples learning directly to the same relaxation dynamics used for inference, allowing the energy landscape that encodes learned attractors to be shaped without introducing separate training mechanisms or side networks.

While EqProp has been thoroughly studied as a training rule for energy-based networks [13], [14], its behavior in continual learning settings has received little attention. In particular, the effect of shaping the energy landscape through local equilibrium-based updates on stability and forgetting across tasks remains largely unexplored.

In this work, we show that Hopfield Networks trained with EqProp naturally support generative replay. Because learned tasks are encoded as stable attractors, relaxing the network from unconstrained or weakly perturbed states produces co-
herent configurations that reflect previously learned structures. We exploit this property to introduce *Energy-Based Generative Replay* (or EB-GenReplay), a continual learning strategy in which a frozen copy of the currently training network generates pseudo-examples by relaxation and interleaves them between current data examples. This enables replay using the same dynamics required for inference and learning, without stored data, auxiliary models, or architectural modifications.

The contributions of this work are as follows:

* We introduce EB-GenReplay, a replay mechanism that exploits the intrinsic generative dynamics of EqProp-trained Hopfield Networks and requires no stored data or auxiliary generative models.

* We empirically evaluate the proposed approach on the Split-MNIST benchmark, comparing it against FineTune and ER to quantify stability-plasticity trade-offs.

The remainder of this paper is organized as follows. Section II reviews related work. Section III describes the models, datasets, and experimental methodology. Section IV details the proposed EB-GenReplay approach. Experimental results are presented in Section V and further analyzed in Section VI.

## II. RELATED WORKS

Catastrophic forgetting is a phenomenon which affects all neural network models when they are sequentially trained on non-identically and independently distributed (i.i.d) data [4]. This scenario might be particularly prevalent at the edge, giving EBMs a double advantage in a setting which has remained largely unexplored as of today.

a) *Continual Learning Algorithms:* Numerous continual learning methods have been proposed, among which we focus on Experience Replay (ER) [1], a simple replay-based approach. ER mitigates catastrophic forgetting by approximating offline training: samples from previous tasks are stored and replayed together with data from the current task. Training batches are augmented with a subset of past examples, preserving performance at the cost of additional memory. Despite its simplicity, ER remains a strong baseline in continual learning.

b) *Generative Replay/Pseudo-Rehearsal:* The "stability-plasticity dilemma" is commonly addressed through pseudo-rehearsal, where a model preserves prior knowledge by training on internally generated data instead of stored samples. Early foundational work [15], [16] showed that pseudo-items, synthesized by passing random noise through a previous network state, stabilize synaptic weights against interference from new tasks without requiring an external memory buffer.

Modern approaches have extended this idea through Deep Generative Replay (DGR). Shin et al. [17] introduced the Scholar framework, which uses GANs to sample from the distribution of past tasks. To improve efficiency, more recent work has moved toward Latent Feature Replay [18], which replays high-level embeddings instead of raw inputs.

c) *Hopfield Networks:* Hopfield networks [19] are a type of recurrent neural network that function as associative memory, allowing them to store and retrieve patterns. They consist of a set of neurons connected by bidirectional connections, and

they are designed to converge to stable states that represent stored patterns. Follow up works upgraded them from discrete to continuous state spaces [20], then modified the energy function to improve the quality and number of stored patterns [21], [22].

d) *Learning Rule:* In [19], Hopfield used Hebbian learning to train his models on toy problems. Hinton and Sejnowski (1983) introduced Contrastive Hebbian Learning to enable training Hopfield Networks for supervised tasks which required non-linear modeling capabilities [23]. Eventually, [12] introduced Equilibrium Propagation with the promise to bring Hopfield networks to the performance level of conventional neural networks. Later works proved that it computed an approximation of the BPTT gradient [13] and even improved its performances by increasing the precision of said approximation [14], [24].

e) *Continual Learning and EBMs:* The question of training EBMs in a continual learning scenario has already been treated a few times in the literature. Notably, [10] showed that Neural EBMs divergence were much better at alleviating catastrophic forgetting than conventional neural networks (designated as "Softmax-Based Classifiers").

References [25] and [26] both implemented methods based on a "wake-sleep" cycle, where the wake phase is a regular learning phase, meant to acquire new knowledge fast, and the sleep phase is a consolidation phase, where the model reinforces its own knowledge. Although the results presented in [10], [25] were promising, their backpropagation-trained, feed-forward Neural EBM architectures lacked the biological plausibility and analog hardware friendliness which drew us towards Hopfield Networks. Conversely, [26] applied the "wake-sleep" cycle to an EqProp-trained Hopfield Network, although their algorithm required the Hopfield Network to transform into a spiking neural network for the sleep phase. It remains unclear whether this operation would be feasible on analog hardware.

Earlier work also explored pseudo-rehearsal in Hopfield Networks trained with Contrastive Hebbian Learning, albeit on small-scale toy problems (with a maximum of 10 examples per task, 2-5 tasks learned sequentially) with now outdated training algorithms. This last paper's work remains the closest to our own among every papers cited here.

## III. METHOD

We study the behavior of a Hopfield Network trained with Equilibrium Propagation under non-stationary data. Performance is evaluated on Split-MNIST, an incremental learning benchmark derived from MNIST. In addition to our proposed EB-GenReplay method, we compare against two standard continual learning baselines: FineTune (FT) and ER (ER).

### A. Problem setting

a) *Benchmarks:* Catastrophic forgetting has been studied across a wide range of machine learning settings, but empirical evaluations are most commonly conducted on incremental variants of standard image classification benchmarks such
as MNIST, CIFAR-10, and ImageNet. Among these, Split-MNIST is a widely used benchmark for small and medium-scale models. It is constructed by partitioning MNIST (60,000 labeled 28 × 28 grayscale images of digits 0 to 9) into a sequence of binary classification tasks (e.g., 0–1, 2–3, 4–5, etc.).

In this work, we focus on class-incremental learning using the Split-MNIST benchmark. Accordingly, our analysis is restricted to sequential classification problems with clearly defined task boundaries and independently and identically distributed (i.i.d) samples within each task.

b) *Incremental learning performance metrics:* We consider a sequence of tasks $T_1 \to T = (T_1, \dots, T_T)$, where each task $T_k$ is associated with a dataset $\mathcal{D}^{(k)}$ and learned sequentially. Let $a_{i,j}$ denote the test accuracy on task $T_j$ after training the model on tasks $T_1$ through $T_i$. The quantity $a_{i,i}$ measures performance on the current task immediately after learning $\mathcal{D}^{(i)}$, while $a_{i,j}$ for $j < i$ reflects retention of previously learned tasks.

We summarize performance over the full task sequence using two standard metrics:

*   **Learning Accuracy:** $LAcc = \frac{1}{T} \sum_{i=1}^{T} a_{i,i}$, which reflects plasticity, and

*   **Average Accuracy:** $AAcc = \frac{1}{T} \sum_{i=1}^{T} a_{T,i}$, which reflects stability.

## B. Continual Learning Baselines

As a frame of reference to evaluate our model’s continual performance, we use two baselines: FineTune and ER.

a) *FineTune:* The most naive strategy for learning a sequence of tasks consists in training each task independently, as if it were part of a single offline dataset. After completing task $\mathcal{D}^{(k)}$, the resulting parameters $W^{(k)}$ are used to initialize training on the next task $\mathcal{D}^{(k+1)}$. This approach is known to suffer from severe catastrophic forgetting, as no mechanism is in place to preserve knowledge acquired from previous tasks. We therefore use FineTune as a lower bound on acceptable stability.

b) *Experience Replay (ER):* ER is used as an upper bound for continual learning performance; see Section II for details.

## C. Hopfield Network and Equilibrium Propagation

a) *Hopfield Networks:* A Hopfield network is a recurrent neural network that stores information in the form of attractors of an energy landscape. It consists of $N$ units with continuous or binary states collected in a state vector $\mathbf{s} = (s_1, \dots, s_N)^\top$, connected through a symmetric weight matrix $\mathbf{W} = [w_{ij}]_{i,j=1}^N$, with $w_{ij} = w_{ji}$ and $w_{ii} = 0$ (no self-connections).

The network’s dynamics minimizes an energy function $E(\mathbf{s}, \mathbf{W})$ via a gradient descent in the state space. In its simplest form, the Hopfield energy is given by

$$E(\mathbf{s}, \mathbf{W}) = -\frac{1}{2} \sum_{i,j=1}^{N} w_{ij} s_i s_j, \quad (1)$$

optionally augmented with bias and regularization terms.

The associated relaxation dynamics follow

$$s_i[t+1] = \sum_{j=1}^{N} w_{ij} s_j[t], \quad (2)$$

which closely resembles inference in conventional neural networks.

b) *Equilibrium Propagation:* Equilibrium Propagation is a supervised learning algorithm for EBMs that computes weight updates using only local information gathered from two successive relaxation phases. This eliminates the need for explicit gradient storage or a separate backward pass, making EqProp both biologically and hardware plausible.

Training proceeds in two phases. In the *free phase*, the input variables are clamped to a data sample $\mathbf{x}$, while the remaining state variables are left free. The system relaxes according to the energy dynamics until it reaches a stable equilibrium state $\mathbf{s}^0$.

In the *nudged phase*, the input remains clamped and a small perturbation is applied to the output variables by augmenting the energy with a task loss. The network energy becomes

$$E^{\beta}(\mathbf{s}, \mathbf{W}) = E(\mathbf{s}, \mathbf{W}) + \beta \mathcal{L}_{\text{out}}(\mathbf{y}, \mathbf{t}), \quad (3)$$

where $\beta > 0$ is a small scalar and $\mathbf{t}$ denotes the target. The system then relaxes again to a new equilibrium $\mathbf{s}^0$.

The resulting weight update is given by

$$\Delta W \propto -\frac{1}{\beta} (\nabla_W E(\mathbf{s}^\beta, \mathbf{W}) - \nabla_W E(\mathbf{s}^0, \mathbf{W})), \quad (4)$$

which approximates gradient descent on the task loss.

For an individual synaptic weight $w_{ij}$, substituting the Hopfield energy yields

$$\Delta w_{ij} \propto \frac{1}{\beta} (s_i^\beta s_j^\beta - s_i^0 s_j^0), \quad (5)$$

showing that EqProp relies only on local differences between pre- and post-synaptic activities across the two phases. These quantities can be accumulated locally, making the learning rule particularly well suited for on-chip and analog implementations.

## IV. ENERGY-BASED GENERATIVE REPLAY

Hopfield Networks trained with EqProp define an explicit energy landscape in which learned tasks correspond to stable attractors. This allows the same network to be used for both prediction and generation by varying which state variables are constrained during relaxation, enabling replay directly through the network dynamics without auxiliary models or stored data.
## A. Sample Generation from EqProp-Trained Energy Landscapes

After learning a sequence of tasks with EqProp, we assume that the energy function $E(s, W)$ contains multiple stable local minima, each corresponding to a task-specific attractor. While any energy function formally defines a Boltzmann distribution $p_W(s) \propto \exp(-E(s, W))$, we do not directly sample from it. Instead, we generate samples by exploiting the network's relaxation dynamics, which is similar to using a gradient sampling method.

Starting from an initial state $s_0$ drawn from an initialization distribution $q(s_0)$, the network evolves according to gradient descent on the energy, $\dot{s} = -\nabla_s E(s, W)$, and converges to a stable equilibrium $s^* = \lim_{t \to \infty} \Phi_t(s_0)$, where $\Phi_t$ denotes the relaxation flow. This induces an implicit generative distribution over equilibria given by the pushforward of $q$ through the asymptotic flow map, $p_{eq}(s) := \int q(s_0) \delta(s - \Phi_\infty(s_0)) ds_0$, whose mass is determined by the basins of attraction of the energy minima.

Sample generation in EqProp-trained Hopfield Networks is performed by releasing all state variables and allowing the system to relax to equilibrium. The resulting states $(x^*, y^*)$ correspond to stable attractors of the learned energy landscape and encode previously learned input-output associations.

However, purely deterministic relaxation tends to produce a limited set of recurrent attractors, resulting in low sample diversity and poor coverage of the underlying task distributions. To promote diversity and improve the informational content of generated samples, stochasticity is introduced during generation. This can be achieved either by injecting noise at each relaxation step, yielding a discrete-time Langevin-like dynamics, or by terminating the relaxation after a finite number of steps, in which case variability arises from the random initialization $s_0$, typically sampled uniformly over the admissible state range.

The resulting generated configurations provide a diverse set of surrogate samples that can be interleaved with new-task data during subsequent training phases, enabling generative replay without storing past data or introducing auxiliary generative models.

## B. Why Input Reconstruction Must Be Explicitly Learned

Initial experiments showed that standard EqProp training does not produce models capable of generating meaningful input samples (see Fig. 1). When all state variables are initialized freely and the system relaxes under the learned energy $E(s, W)$, the resulting equilibria collapse to simplistic or degenerate input patterns that fail to capture the structure of the data.

This behavior follows directly from the EqProp training objective. During learning, input units $x$ are clamped to observed data, and the task loss $\mathcal{L}$ depends only on the output variables $y$. As a result, the energy landscape is shaped to produce correct outputs conditioned on the input, but not to assign low energy to input configurations themselves. In probabilistic terms, EqProp constrains the conditional distribution

$p_W(y, h | x)$ while leaving the marginal distribution over inputs unconstrained.

When inputs are released during generation, the energy therefore provides no incentive for $x$ to converge toward data-like configurations. The relaxation dynamics can settle in arbitrary input states that remain compatible with low-energy hidden-output configurations but do not lie on the data manifold. Explicitly training input reconstruction is therefore necessary to enable meaningful input generation.

## C. Hybrid Training for Input Reconstruction

To enable input generation, we extend the EqProp objective to explicitly train input reconstruction. During learning, a subset of input units is released and allowed to evolve under the same relaxation dynamics as the hidden and output units. Let $\tilde{x}$ denote the released input variables and $x^{\text{data}}$ their corresponding ground-truth values.

We introduce an input reconstruction loss

$$ \mathcal{L}_{\text{in}}(\tilde{x}, x^{\text{data}}) = \|\tilde{x} - x^{\text{data}}\|^2, \quad (6) $$

which is applied during the nudged phase of EqProp. The perturbed energy becomes

$$ E^{\beta}(s, W) = E(s, W) + \beta (\mathcal{L}(y, t) + \lambda \mathcal{L}_{\text{in}}(\tilde{x}, x^{\text{data}})), \quad (7) $$

where $\lambda$ controls the relative importance of input reconstruction.

This hybrid objective encourages low-energy equilibria to jointly encode consistent input-output configurations. As a result, the learned energy landscape supports meaningful relaxation-based generation of both inputs and outputs, which is essential for generative replay in continual learning.

## D. Energy-Based Generative Replay Algorithm

Algorithm 1 details the proposed continual learning procedure. Tasks are learned sequentially using EqProp on batches that combine current-task data with pseudo-samples generated by a frozen copy of the model. Replay is implemented directly through the relaxation dynamics of the learned energy function, without storing past data or introducing auxiliary generative models.

# V. RESULTS

Based on the methodology introduced in Sections III and IV, we report results on hybrid EqProp training, pseudo-example generation, and their use for EB-GenReplay.

## A. Input Images Regeneration

We first evaluate the input reconstruction and generation mechanism described in Section IV-A for networks are trained under four different regimes:

* an untrained network,

* training without input damage,

* training with 1% input damage and an input reconstruction cost,
**Algorithm 1** Continual Learning with Equilibrium Propagation and EB-GenReplay

**Require:** Sequence of tasks $\{\mathcal{D}^{(k)}\}_{k=1}^T$, where $\mathcal{D}^{(k)} = \{(x, t)\}$ and $t \in \{0, 1\}^C$.

**Require:** EqProp hyperparameters: $\beta$, learning rate $\eta$, step size $\Delta t$, time constant $\tau$, free/nudged steps $K_{\text{free}}$, $K_{\text{nudged}}$, tolerance $\epsilon$.

**Require:** Replay hyperparameters: pseudo ratio $r$, mask fraction $\rho$, noise scale $\sigma$, sampling steps $K_{\text{gen}}$.

**Require:** Hopfield energy: $E(s, W)$ over $s = (x, h, y)$. Losses: $\mathcal{L}_{\text{out}}(y, t)$, $\mathcal{L}_{\text{in}}(\tilde{x}, x)$, weight $\lambda$.

1: **Initialize parameters** $W \sim \text{Init}(\cdot)$

2: **for** $k = 1, \dots, T$ **do**

▷ **Iterate over tasks**

3: Set teacher $W_{\text{teach}} \leftarrow W$ and freeze it

4: **for** each minibatch $\mathcal{B} = \{(x_b, t_b)\}_{b=1}^B$ from $\mathcal{D}^{(k)}$ **do**

**(A) Pseudo-example generation (teacher model)**

5: Initialize $\mathcal{B}_{\text{ps}} \leftarrow \emptyset$

6: **for** $m = 1, \dots, rB$ **do**

7: Initialize state $s_0 \sim \text{Unif}(\text{state range})$

8: Release all state variables $(x, h, y)$

9: **for** $t = 0, \dots, K_{\text{gen}} - 1$ **do**

10: $s_{t+1} \leftarrow s_t - \frac{\Delta t}{\tau} \nabla_s E(s_t, W_{\text{teach}}) + \sqrt{\frac{2\Delta t}{\tau}} \sigma \xi_t \triangleright \xi_t \sim \mathcal{N}(0, I)$

11: **end for**

12: Extract $(x^{\text{ps}}, y^{\text{ps}})$ from $s_{K_{\text{gen}}}$

13: Add $(x^{\text{ps}}, y^{\text{ps}})$ to $\mathcal{B}_{\text{ps}}$

14: **end for**

15: Form mixed batch $\mathcal{B}_{\text{mix}} \leftarrow \mathcal{B} \cup \mathcal{B}_{\text{ps}}$

**(B) EqProp training on mixed batch**

16: Initialize accumulated update $\Delta W \leftarrow 0$

17: **for** each $(x, t) \in \mathcal{B}_{\text{mix}}$ **do**

18: Sample mask $M$ and define $x^{\text{cl}}$

19: Initialize state $s$ and clamp $x^{\text{cl}}$

*Free phase*

20: **for** $i = 1, \dots, K_{\text{free}}$ **do**

21: $s \leftarrow s - \frac{\Delta t}{\tau} \nabla_s E(s, W)$

22: **end for**

23: Store $s^0 \leftarrow s$

*Nudged phase*

24: Define $E^\beta = E + \beta(\mathcal{L} + \lambda \mathcal{L}_{\text{in}})$

25: **for** $i = 1, \dots, K_{\text{nudged}}$ **do**

26: $s \leftarrow s - \frac{\Delta t}{\tau} \nabla_s E^\beta(s, W)$

27: **end for**

28: Store $s^\beta \leftarrow s$

29: Accumulate $\Delta W \leftarrow \Delta W - \frac{1}{\beta} (\nabla_W E(s^\beta, W) - \nabla_W E(s^0, W))$

30: **end for**

31: Update parameters $W \leftarrow W + \eta \Delta W / |\mathcal{B}_{\text{mix}}|$

32: **end for**

33: **end for**

* training with 10% input damage and an input reconstruction cost.

Input generation is evaluated by fully releasing the input units and allowing the full network state to relax (here without noise). Representative samples generated under each training regime are shown in Fig. 1. Offline training performance for these models is reported in the next subsection.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Input samples generated after full input release for different training regimes.</figcaption></figure>

When trained without input reconstruction, the network relaxes toward a diffuse, centered activation pattern that only captures coarse properties of MNIST, such as spatial extent and centering. While this reflects some dataset statistics, it provides no meaningful digit structure.

In contrast, models trained with partial input release and an explicit input reconstruction cost produce significantly more structured patterns. Although these patterns do not resemble human-interpretable MNIST digits, this is not necessarily problematic: deep models often encode useful information in non-human-readable representations (such as filters used in CNNs). To assess whether these generated patterns carry task-relevant information, we need to evaluate their effectiveness for knowledge transfer, which we do in an online setting when testing EB-GenReplay.

**B. Offline Hybrid Training**

The generated inputs above are obtained using the hybrid EqProp training scheme described in Section IV-C. Figure 2 reports training accuracy and input reconstruction error for models trained on MNIST with 10% partial input release. This fraction is kept low to limit interference with output learning, ensuring both accurate predictions and reliable input-output associations.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Hybrid EqProp training with and without an input reconstruction cost. Curves are averaged over five runs. Final input reconstruction error is $10.88 \pm 0.2$ without input cost and $0.15 \pm 0.002$ with input cost.</figcaption></figure>

Adding an input reconstruction objective substantially improves input generation but at the expense of classification accuracy. After two epochs (10 training steps), the model trained with this additional cost reaches 85% accuracy with an average input error of 0.15. In contrast, a comparable model trained with the standard EqProp algorithm typically attains 97-98% accuracy, corresponding to an accuracy drop of about 10% due solely to the added objective.

When training without the input reconstruction cost, input generation quality degrades markedly compared to the previous setting. Nevertheless, the model still achieves around 90% accuracy and even surpasses the version trained with the input cost after a slower initial phase. This unexpected gap in performance may result from competition between the two optimization objectives, namely input reconstruction and classification. Without an explicit input cost, imperfect input reconstruction, affecting only about 10% of the input, remains a minor issue and may even act as a form of regularization.

### C. Continual Learning with Generative Replay

We next evaluate whether the generated samples can effectively support continual learning. Using the experimental protocol from Section III, we train a layered Hopfield Network on Split-MNIST with five tasks, comparing FineTune (FT), Experience Replay (ER), and Energy-Based Generative Replay (EB-GenReplay).

The obtained results, averaged over 5 runs for each CL algorithm, are summarized in Table V-C.

The table shows that ER performed the best in terms of stability, with 85% average accuracy, followed by EB-GenReplay, at 76% average accuracy. FineTune performed the worst with 19% average accuracy, which is expected given that it is the naive continual learning approach.

Figure 3 shows a complete history of the task-specific validation accuracies during training for each continual method. The average accuracies reported in the table correspond to

<table><caption>TABLE I
AVERAGE ACCURACY (AACC) AND LEARNING ACCURACY (LACC) OF EQPROP-TRAINED MODELS AFTER LEARNING SPLIT MNIST WITH FT, ER AND EB-GENREPLAY</caption><thead><tr><th rowspan="2">learning rule</th><th colspan="2">EqProp based</th></tr><tr><th>AAcc</th><th>LAcc</th></tr></thead><tbody><tr><td>CL strat</td><td></td><td></td></tr><tr><td>FT</td><td>0.19 ± 0.00</td><td>0.98 ± 0.00</td></tr><tr><td>ER</td><td>0.85 ± 0.01</td><td>0.96 ± 0.00</td></tr><tr><td>EB-GenReplay</td><td>0.76 ± 0.01</td><td>0.92 ± 0.01</td></tr></tbody></table>

the average of the per-task validation accuracies at the end of training, while learning accuracies correspond to the average of the task-specific validation accuracies measured right after a given model finished learning said task (supposedly where the accuracy in question reaches its best value).

<figure><img src="image_4.png" /><figcaption>Fig. 3. Mean accuracy evolution during training across three continual learning methods (FT, EB-GenReplay and ER) on Split-MNIST. Shaded regions indicate one standard deviation across 5 runs.</figcaption></figure>

The FineTune figure shows that each task-specific accuracy instantly drops to zero as soon as the model switches task, indicating absolute and brutal forgetting. EB-GenReplay sees a significant drop in plasticity after learning the first task, with the last task barely reaching 80% maximum accuracy, while the first task maintains a high accuracy until the end. This indicates that EB-GenReplay strongly advantaged the first task, to the point where it hurts the performance of all following tasks. We see a similar phenomenon with ER, although to a lesser degree.

### D. EB-GenReplay vs ER

Although ER performed better than EB-GenReplay in this experiment, the two algorithms need to be compared on equal grounds in terms of edge deep learning. In particular, the main appeal of our base model for the edge setting is that it could be implemented directly on analog hardware with minimal digital overhead, a quality shared by EB-GenReplay. On the other
hand, the buffer used by ER is a much less analog-friendly solution, either necessitating a large array of on-chip storage devices or an external digital memory, which runs counter to the goal of an analog-friendly hardware implementation. Although we do not present analog hardware simulations comparing both solutions, we compare the compactness of both algorithms in the same digital simulation setup used until now. We show the relation between ER's average accuracy and its replay-related memory overhead (the memory size of its replay buffer) in Fig. 4. The EB-GenReplay average accuracy obtained previously is added at the abscissa corresponding to its own memory overhead (the memory size of the copied network used for generating pseudo-examples).

<figure><img src="image_5.png" /><figcaption>Fig. 4. ER average accuracy for different memory buffer memory footprint. EB-GenReplay is represented by a single point as its own replay-related memory footprint corresponds to its base model size.</figcaption></figure>

We can see that for an equivalent excess memory footprint, EB-GenReplay outperforms ER by almost 10%. ER only takes its advantage back after augmenting its excess memory footprint roughly by a factor of ×1.5.

## VI. ANALYSIS

We introduce EB-GenReplay, a continual learning solution which allows EqProp-trained Hopfield Networks to approach state-of-the-art continual learning performances without sacrificing their appeal to edge deep learning. Our method uses the base model, a Hopfield Network, both as a classifier and a generator, saving weights learned for task i and using them to generate pseudo-examples which it uses alongside real examples from task i + 1 to train the new weights. We use a hybrid input/output training extension of EqProp as well as noisy generation methods to improve the quality of the pseudo-examples. We tested our algorithm against two baselines: FineTune and ER. By the end of training, Finetune had entirely forgotten every previous task, which confirmed the idea that EqProp-trained Hopfield Networks are just as sensitive to catastrophic forgetting as conventional neural networks. ER maintained a good stability throughout learning with 85% average accuracy, and EB-GenReplay followed with 76% average accuracy.

a) Forgetting in EBMs: A Hopfield Network defines an EBM over a global state s and parameterized by a weight matrix W that evolves by minimizing an energy function E(s, W). Under standard assumptions, this energy induces a Boltzmann distribution over states,

$$p_W(s) = \frac{1}{Z(W)} \exp(-E(s, W)), \quad (8)$$

where Z(W) is the partition function.

Classical energy-based learning methods, such as contrastive divergence, approximate maximum likelihood estimation by globally shaping the model distribution $p_W(s)$. This leads to the objective

$$\min_W \underbrace{\mathbb{E}_{s^+ \sim p_{\text{data}}} [E(s^+, W)]}_{\text{positive phase}} - \underbrace{\mathbb{E}_{s^- \sim p_W} [E(s^-, W)]}_{\text{negative phase}}. \quad (9)$$

which lowers the energy of data-supported configurations while raising the energy of states sampled from the current model distribution.

In a continual learning setting with task-specific data distributions {$p_{\text{data}}^{(k)}$}$_{k=1}^T$, optimizing this objective for task k redistributes probability mass toward $p_{\text{data}}^{(k)}$. Because samples drawn from $p_W$ include low-energy configurations associated with previously learned tasks j < k, the negative phase raises the energy of these configurations as well. As a result, energy minima encoding earlier tasks are progressively eroded, leading to catastrophic forgetting.

While EqProp substantially deviates from this method, it still functions with a first negative phase which characterises the model's preconceptions. In consequence, it also suffers from catastrophic forgetting.

b) Replay-based Continual Learning and Edge Friendliness: After showing that ER performed better than EB-GenReplay in terms of stability, we provided evidences that EB-GenReplay became better when provided with equal excess memory budget. Although no actual hardware simulation was conducted, we used the digital memory footprint overhead as a proxy for the hypothetical silicon area overhead. We assume that the actual area and power required by EB-GenReplay (most likely memristors) would be much smaller than that required by ER for the same amount of stored data. We consider this assumption reasonable as low area and low power is the objective of in-memory analog computing. Given this assumption, proving the higher compactness of EB-GenReplay in digital simulation reinforces its advantage in resource constrained environments.

However, the previous analysis only accounted for static memory usage. During the experiments, ER systematically used 1 replay sample per real example, whereas for EB-GenReplay this number grew up to 4 replay samples per real sample. Generating and learning those extra samples would either imply much higher latency or the use of buffers for temporarily storing the pseudo-examples, where ER adds little latency and already has a buffer accounted for.
c) *Perspectives:* In the future, we would like to replicate the presented results on more challenging datasets such as CIFAR 10/100 or Tiny ImageNet. Since a pool of one comparison baseline is very limited, we would like to make a broader comparison with the continual learning state of the art, including GAN-based Generative Replay, applied to BPTT-trained convergent RNNs. It is also necessary to conduct a more in depth analysis of the potential compactness of EB-GenReplay and how it compares to other analog hardware-compatible approaches. Finally, we believe that there is room for improving the current version of Energy-Based GenReplay, by testing other weight topologies or trying out sample selection strategies.

## REFERENCES

[1] L.-J. Lin, "Self-improving reactive agents based on reinforcement learning, planning and teaching," *Machine learning*, vol. 8, no. 3, pp. 293-321, 1992.

[2] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, "Continual lifelong learning with neural networks: A review," *Neural Networks*, vol. 113, pp. 54-71, 2019.

[3] M. D. Lange, G. van de Ven, and T. Tuytelaars, "Continual evaluation for lifelong learning: Identifying the stability gap," 2023.

[4] "Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem," in *Psychology of Learning and Motivation*, vol. 24, pp. 109-165, Academic Press, Jan. 1989.

[5] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell, "Overcoming catastrophic forgetting in neural networks," *Proceedings of the National Academy of Sciences*, vol. 114, no. 13, pp. 3521-3526, 2017.

[6] F. Zenke, B. Poole, and S. Ganguli, "Continual Learning Through Synaptic Intelligence," June 2017.

[7] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, "Progressive neural networks," 2022.

[8] D. Lopez-Paz and M. A. Ranzato, "Gradient episodic memory for continual learning," in *Advances in Neural Information Processing Systems* (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017.

[9] H. Shin, J. K. Lee, J. Kim, and J. Kim, "Continual Learning with Deep Generative Replay," in *Advances in Neural Information Processing Systems*, vol. 30, Curran Associates, Inc., 2017.

[10] S. Li, Y. Du, G. van de Ven, and I. Mordatch, "Energy-Based Models for Continual Learning," in *Proceedings of The 1st Conference on Lifelong Learning Agents*, pp. 1-22, PMLR, Nov. 2022.

[11] V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, "Efficient processing of deep neural networks: A tutorial and survey," *Proceedings of the IEEE*, vol. 105, 12 2017.

[12] B. Scellier and Y. Bengio, "Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation," *Frontiers in Computational Neuroscience*, vol. 11, May 2017.

[13] M. Ernoult, J. Grollier, D. Querlioz, Y. Bengio, and B. Scellier, "Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input," in *Advances in Neural Information Processing Systems*, vol. 32, Curran Associates, Inc., 2019.

[14] A. Laborieux, M. Ernoult, B. Scellier, Y. Bengio, J. Grollier, and D. Querlioz, "Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing Its Gradient Estimator Bias," *Frontiers in Neuroscience*, vol. 15, Feb. 2021.

[15] "Catastrophic Forgetting, Rehearsal and Pseudorehearsal: Connection Science: Vol 7, No 2."

[16] B. Ans and S. Rousset, "Avoiding catastrophic forgetting by coupling two reverberating neural networks," vol. 320, no. 12, pp. 989-997.

[17] H. Shin, J. K. Lee, J. Kim, and J. Kim, "Continual learning with deep generative replay," 2017.

[18] X. Liu, C. Wu, M. Menta, L. Herranz, B. Raducanu, A. D. Bagdanov, S. Jui, and p. u. family=Weijer, given=Joost, "Generative Feature Replay for Class-Incremental Learning," pp. 226-227.

[19] J. J. Hopfield and J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," vol. 79, no. 8.

[20] J. J. Hopfield, "Neurons with graded response have collective computational properties like those of two-state neurons," *Proceedings of the National Academy of Sciences*, vol. 81, pp. 3088-3092, May 1984.

[21] D. Krotov and J. J. Hopfield, "Dense Associative Memory for Pattern Recognition," in *Advances in Neural Information Processing Systems*, vol. 29, Curran Associates, Inc., 2016.

[22] H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandve, V. Greiff, D. Kreil, M. Kopp, G. Klambauer, J. Brandstetter, and S. Hochreiter, "Hopfield Networks is All You Need," Apr. 2021.

[23] G. E. Hinton and T. J. Sejnowski, "Optimal Perceptual Inference,"

[24] A. Laborieux and F. Zenke, "Holomorphic Equilibrium Propagation Compute Exact Gradients Through Finite Size Oscillations," *Advances in Neural Information Processing Systems*, vol. 35, pp. 12950-12963, Dec. 2022.

[25] V. Singh, A. Choromanska, S. Li, and Y. Du, "Wake-Sleep Energy Based Models for Continual Learning," in *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, pp. 4118-4127, 2024.

[26] Y. Kubo, J. E. Delanois, and M. Bazhenov, "Toward Lifelong Learning in Equilibrium Propagation: Sleep-like and Awake Rehearsal for Enhanced Stability," Aug. 2025.