# Enhancing temporal learning in recurrent spiking networks for neuromorphic applications

Ismael Balafrej, Soufiyan Bahadi, Jean Rouat, Fabien Alibart

► To cite this version:

Ismael Balafrej, Soufiyan Bahadi, Jean Rouat, Fabien Alibart. Enhancing temporal learning in recurrent spiking networks for neuromorphic applications. Neuromorphic Computing and Engineering, 2025, 5 (2), pp.024008. 10.1088/2634-4386/add293. hal-05380518

HAL Id: hal-05380518

https://hal.science/hal-05380518v1

Submitted on 25 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
PAPER • OPEN ACCESS

# Enhancing temporal learning in recurrent spiking networks for neuromorphic applications

To cite this article: Ismael Balafrej et al 2025 Neuromorph. Comput. Eng. 5 024008

View the article online for updates and enhancements.

You may also like

- *Historical perspective and opportunity for computing in memory using floating-gate and resistive non-volatile computing including neuromorphic computing*
Jennifer Hasler and Arindam Basu

- *Leveraging dendritic complexity for neuromorphic computing*
Suma G Cardwell, Mark Plagge, Luke Parker et al.

- *Spike-based local synaptic plasticity: a survey of computational models and neuromorphic circuits*
Lyes Khacef, Philipp Klein, Matteo Cartiglia et al.
<figure><img src="image_2.png" /></figure>

CrossMark

OPEN ACCESS

RECEIVED

11 January 2025

REVISED

25 March 2025

ACCEPTED FOR PUBLICATION

30 April 2025

PUBLISHED

8 May 2025

Original Content from
this work may be used
under the terms of the
Creative Commons
Attribution 4.0 licence.

Any further distribution
of this work must
maintain attribution to
the author(s) and the title
of the work, journal
citation and DOI.

<figure><img src="image_3.png" /></figure>

PAPER

# Enhancing temporal learning in recurrent spiking networks for neuromorphic applications

Ismael Balafrej¹,²,*, Soufiyan Bahadi¹,², Jean Rouat¹,² and Fabien Alibart²,³

¹ NECOTIS Research Lab, Université de Sherbrooke, Sherbrooke, J1K 2R1, Canada

² Institut Interdisciplinaire d'Innovation Technologique (3IT), Université de Sherbrooke, 3000 Boulevard de l'université, Sherbrooke, J1K OA5 Québec, Canada

³ Laboratoire Nanotechnologies Nanosystèmes (LN2)-IRL3463, CNRS, Université de Sherbrooke, INSA Lyon, École Centrale de Lyon, Université Grenoble Alpes, Sherbrooke, J1K 0A5 Québec, Canada

* Author to whom any correspondence should be addressed.

E-mail: ismael.balafrej@usherbrooke.ca

**Keywords:** spiking neural network, synaptic delays, surrogate gradient descent, neural dynamics

Supplementary material for this article is available [online](#)

## Abstract

Training Recurrent Spiking Neural Networks (RSNNs) with binary spikes for tasks of extended time scales presents a challenge due to the amplified vanishing gradient problem during backpropagation through time. This paper introduces three crucial elements that significantly enhance the memory and capabilities of RSNNs, with a strong emphasis on compatibility with hardware and neuromorphic systems. Firstly, we incorporate neuron-level synaptic delays, which not only allow the gradient to skip time steps but also reduce the overall neuron population's firing rate. Subsequently, we apply a biologically inspired branching factor regularization rule to stabilize the network's dynamics and make training easier by incorporating a time-local error in the loss function. Lastly, we modify a commonly used surrogate gradient function by increasing its support to facilitate learning over longer timescales when using binary spikes. By integrating these three innovative elements, we not only resolve several complex benchmarks but also achieve state-of-the-art results on the spiking permuted sequential MNIST task (psMNIST), showcasing the practicality and relevance of our approach for digital and analog neuromorphic systems.

## 1. Introduction

Biologically inspired spiking neurons have emerged as an energy-efficient alternative to standard artificial neurons because of their event-based transmission. Similar to biological neurons, a spiking neuron only emits a binary output, or spike, when the accumulated temporal information reaches a threshold. This temporal sparsity allows spiking neural networks to be implemented in hardware at a fraction of the energy cost of traditional ANNs (Blouw et al 2019, Davies et al 2021). Another critical aspect of the brain that is less frequently mentioned is its three-dimensional organization and the consequential implications of transmission delays. Delays can introduce a novel dimension of neural dynamics into the system which can be leveraged by the temporal nature of spiking neurons. Moreover, delays are straightforward to implement in hardware, as evidenced by the numerous neuromorphic processors that support this feature (Davies et al 2018, D'Agostino et al 2024). The use of delays has been highlighted in existing literature for many purposes. However, their potential has not been explored with recurrent spiking neural networks (RSNNs) when trained with surrogate backpropagation. This study aims to shed light on this uncharted territory, providing fresh insights into how synaptic delays can enhance the memory capabilities of hardware-friendly RSNNs.

### 1.1. Background

Recurrent neural networks (RNN) process sequences one step at a time, with an internal hidden state to represent past information. They have a constant memory requirement, and they scale linearly when trained with backpropagation through time (BPTT), which is the most common way to train RNNs (Schmidhuber
<figure><img src="image_4.png" /><figcaption>**Figure 1.** Schematic of a recurrent spiking neural network with leaky-integrate-and-fire (LIF) neurons. $W_R$, $W_I$, and $W_O$ are the synaptic weights for the recurrent, input, and output layers, respectively. $I$ is the identity matrix. $W_R$, $\tau_O$ and $\tau_R$ are parameters that impact the length of memory storage, both forward and backward. See text for details.</figcaption></figure>

2015) in a supervised manner. They are notoriously difficult to train because of the vanishing and exploding gradient problems (Bengio et al 1994). The spiking equivalent of an RNN typically involves substituting the neuron model with a spiking version, such as the Leaky-Integrate-and-Fire (LIF) neuron. The LIF neuron is the most commonly used model for spiking neurons, due to its simplicity and ease of hardware implementation (Han et al 2022). Unfortunately, the temporally sparse and binary network activity of this neuron model adds to the training difficulty (Neftci et al 2019). The LIF model consists of a membrane potential $u(t)$, decaying exponentially by a time constant of $\tau$:

$$ \frac{du(t)}{dt} = \frac{-u(t)}{\tau} + x(t) \qquad (1) $$

where $x(t)$ is the sum of all the input currents and biases being fed to that neuron. When $u(t)$ reaches a threshold $u_{th}$, the neuron emits a spike. This behavior can be modeled using the Heaviside step function as an activation function for the neuron:

$$ n(t) = \mathcal{H}(u(t) - u_{th}) \qquad (2) $$

where $n(t)$ is the output of the neuron at time $t$. Figure 1 presents a schematic overview of a RSNN with LIF neurons. To train a spiking network using gradient descent, it is common practice to replace the derivative of the spike function—the Dirac delta function—with a surrogate function with a wider support. We refer to Neftci et al (2019) for a review of commonly used surrogate functions and Zenke and Vogels (2021), Herranz-Celotti and Rouat (2022) for empirical comparisons between them. The drawback is that training can only be done on a limited number of layers in a feed-forward network, or a limited amount of time steps in a RSNN. The temporal sparsity added by the spiking neurons in this scenario severely increases the vanishing gradient problem of standard RNNs.

Liquid state machines (Maass et al 2002) and other types of reservoirs with spiking neurons (Ivanov and Michmizos 2021, Balafrej et al 2022) have been thought to be an alternative to training RSNN with gradient descent. In these networks, the recurrent weights are not trained using a supervised gradient-based algorithm, but rather sampled from a random distribution that attempts to maximize their theoretical memory, with optionally added biologically plausible plasticity rules in more recent works (Jin et al 2016, Ivanov and Michmizos 2021, Balafrej et al 2022, Payvand et al 2022). In many cases, they still fall short in terms of scalability and machine learning performance to their back propagated spiking neural network counterparts.

Other approaches circumvent the limitations of the surrogate function by increasing the time constant $\tau$ of the LIF neuron to the time scale of the task (Frenkel and Indiveri 2022) or by adding a secondary state with a long time constant such as an adaptive threshold (Bellec et al 2020). The disadvantage of such methods is that they limit the implementability of the LIF neuron in analog hardware. Indeed, one of the most sought upon aspect of the LIF neuron is that the exponential decay can be implemented with a single capacitor working in an analog regime. A long exponential decay therefore requires a large and costly capacitor that cannot subsequently be scaled down. Other research area involve changing the technology to overcome the scalability problem of capacitors, such as using the resistivity drift of Phase Change Memories (Demirag et al 2021). By contrast, leveraging synaptic delays preserves the simplicity of LIF neurons, ensuring they remain scalable and feasible for analog hardware implementations.
<figure><img src="image_5.png" /></figure>

**Figure 2.** Simplistic demonstration of the relationship between neuronal distance and spike frequency using two distinct three-neuron subnetworks extracted from a connectome. The subnetworks are identical in configuration, but they differ in neuronal distance. Subfigure (A) presents biological neurons selected based on distances $L_1$, $L_2$, and $L_3$. Subfigure (B) depicts the graphs of two subnetworks selected from figure 1(A), and subfigure (C) presents the spike raster plots for the two neural graphs. In this example, one spike worth of information is stored and goes through both subnetwork from $n_1$ to $n_2$ to $n_3$ and back to $n_1$. To retain that information in the form of a spike, each neuron must directly cause the next neuron to spike after a delay strictly dependent on the distance between the two neurons. By choosing a subnetwork in the connectome with longer interneuron distance, the system reduced its overall spiking frequency. If we were to increase the number of spikes stored in these two subnetworks, the one with the longest delays will store more information as it can spike more often during the waiting period caused by the delays, i.e. there is more opportunity to increase the spike count in subfigure (C) when the delays are longer.

### 1.1.1. Axonal length and synaptic transmission delay

In the brain, delays are known to be highly dynamical. The myelination process can change the transmission delay between pre- and post-synaptic spikes and is dependent on activity (Xin and Chan 2020). The variation in delays also differs significantly across various subsystems and species, ranging from a few microseconds to over 100 ms in very long, non-myelinated axons (Swadlow and Waxman 2012). Delays have been widely used in feed-forward neural network to enable precise spike timing (Shrestha and Song 2016, Wang et al 2019, Zhang et al 2020, Luo et al 2023), to add memory buffers (Waibel et al 1989, D'Agostino et al 2024) and to increase the frequency selectivity (Gilson et al 2012) of neural networks. They have also been added to create oscillations (Izhikevich 2006) which are thought to be a key ingredient in the brain for long-term memory and learning (Buzsáki and Draguhn 2004). In a perfectly trained RSNN, there is no explicit need for transmission delays as the network can in theory reorganize itself with synfire chains (Abeles 1991) to delay spikes. Nonetheless, training a RSNN over a long period of time is hard with current training algorithms, and delay can significantly help by reducing the scale of a temporal problem. Moreover, in a neural network, choosing long-distance neuron pairs can help reduce the overall spiking frequency of the system as showcased in figure 2.

In standard digital RSNNs, there is an implicit delay of 1 timestep during spike transmission. When a neuron spikes, all the connected neurons will receive a weighted version of the spike in the next time step. Some digital neuromorphic processors, such as Loihi (Davies et al 2018), have the ability to increase the transmission delay by a few time steps using a circular memory buffer. Novel Resistive Random Access Memory technologies can also be used to achieve delays in analog neuromorphic chips (D'Agostino et al 2024). This demonstrates that a RSNN approach with delays would be compatible with current and emerging hardware technologies for both digital and analog neuromorphic processors. Figure 3 depicts the computational graph that is used during BPTT for three timesteps. In this work, we take advantage of such mechanism to increase the memory capacity of the network. By using synaptic delays, we allow the spike transmission, and similarly the backpropagated gradient, to skip timesteps. Indeed, once a recurrent network is unravelled during backpropagation, it becomes a very deep feed forward network, with each layer representing the same recurrent layer at a different timestep. When delaying the spike transmission by a certain amount of timestep, the spike skips a few layers and acts as if there was a direct connection between these non-consecutive layers. This mechanism is conceptually similar to residual connections in ResNets (He et al 2015a), but with shared weights. Residual connections played a key role in increasing the number of layers that a feed-forward network could be trained on, and a kindred concept is used here.
<figure><img src="image_6.png" /><figcaption>**Figure 3.** Computational graph of a digital recurrent neural network with input $\vec{x}$, input weights $W_I$, recurrent weights $W_R$ and output weights $W_O$. The membrane potentials of the recurrent layer $\vec{u}$ integrate both the input, and the weighted spikes of the recurrent neurons in the same layer. When a synaptic delay is added -here, a delay of 2 clock periods ($2\delta t$)- the spikes are forwarded in time during inference. When the network is unrolled during backpropagation through time, the delayed connection creates a shortcut that reduces the effective path length between the output at time $t$ and the network activity at time $t-2$. This idea is conceptually similar to residual connections in ResNets (He et al 2015a). As a result, the amplitude of the gradient that reaches the “early” layer increases. Moreover, since the surrogate derivative of the spike function is imperfect, the gradient that skips some of the unrolled layers is less noisy.</figcaption></figure>

## 2. Method

All implementations are based on a standard spiking RNN with three layers: input, recurrent and output. The input maps incoming spikes or current to the LIF neurons in the recurrent layers, which are also connected to themselves with delayed connections. Finally, there are leaky integrators in the output layer connected to the neurons of the recurrent layer from which they receive the spikes. The implementation for this work is available at https://github.com/NECOTIS/TemporalLearningInRSNNs.

### 2.1. Delays

Ideally, each pair of connections in the recurrent layer could have a propagation delay based on the distance between the pre- and post-synaptic neurons, and could potentially be optimized further with e.g. brain-inspired activity-dependent myelination (Xin and Chan 2020). In clock-based software simulations and analog systems, synapse-level delays are somewhat expensive, depending on the system. We therefore added neuron-level delays, where each neuron is associated with a fixed propagation delay, where the instant at which the neuron spikes and the instant at which the spike is propagated is delayed by a fixed constant $d$.

#### 2.1.1. Computational advantage of neuron-level delays

For a recurrent layer of $n$ neurons, neuron-level delays result in at most $n$ possible delays, whereas synapse-level delays could require up to $n^2$ different delays. In most neuromorphic processors, this distinction leads to a lower memory requirement for implementing delays in hardware. Some analog neuromorphic devices, such as DenRAM (D'Agostino et al 2024), implement delays using specific devices, meaning that neuron-level delays reduce both the cost and footprint over synapse-level delays. For some digital neuromorphic processors, such as Loihi, the delay is implemented using a ring-buffer-like structure. While the delays are stored per synapse, they are still implemented at the neuron level, meaning that neuron-level delays do not have a computational advantage other than the reduced SRAM requirements.

Another clear computational advantage of neuron-level delays is that some neuromorphic hardware supports a refractory period at the neuron level (Dalgaty et al 2019). The mechanism responsible for the refractory period could be repurposed to implement the proposed neuron-level delays, resulting in an almost negligible additional cost.

#### 2.1.2. Organization of the delays

We chose to organize the delays based on an arbitrary 3-dimensional organization of the neurons. To do so, we placed $n$ neurons in a 3-dimensional space, where each neuron is positioned in a cubic lattice with positional noise uniformly sampled between 0 and 0.5. The distance between each neuron and the center reflects the delay associated with that neuron. The delays are fixed during an experiment.
### 2.1.3. Implementation of the delays

The delays are implemented using a sequence of discrete delay values $d$ and a set of delay masks $M_d \in \mathbb{R}^{n \times n}$, which are applied to the recurrent weight matrix $W_R \in \mathbb{R}^{n \times n}$. Each mask $M_d$ is a binary matrix where $(M_d)_{ij} = 1$ if neuron $j$ has a synaptic delay of $d$ when projecting to neuron $i$, and 0 otherwise.

At each time step $t$, the recurrent input current $I_R(t)$ is computed by accumulating the contributions from past spike activities weighted by $W_6R$, incorporating the appropriate delay masks:

$$I_R[t] = W_1 x[t] + \sum_{d \in D} W_R \odot M_d \cdot \vec{n}[t-d] \quad (3)$$

where $\odot$ denotes the Hadamard (element-wise) product. This formulation ensures that spikes arriving from recurrent connections are delayed by the corresponding amount before influencing the target neuron's membrane potential. If a neuron has no delay for a given connection, it contributes at the next timestep (implicit $d=1$). This approach is particularly efficient when the number of unique delays is small, such as the three unique delays we use in our work.

## 2.2. Regularization with a biologically inspired branching factor

Activity regulation is important in RSNN as to balance the regime of the network. Other works typically add activity regularization (Bellec et al 2018, Rossbroich et al 2022) to increase sparseness and to target or upper-bound a fixed firing rate. This firing rate is a hyperparameter that depends on the input data and potentially the number of neurons and layers. Our approach takes further inspiration from the brain, with the use of a branching factor based regularization mechanism. RNNs are thought to operate in one of three different regimes: subcritical, critical or supercritical (Beggs 2007). In terms of spike transmission, each spike transmitted should average one spike produced to maintain the activity of the network and is referred to as the critical regime (Haldeman and Beggs 2005, Beggs 2007). Subcriticality and supercriticality occur when the average branching factor is smaller or larger than one, respectively. The brain is reported to work in a slightly subcritical regime (Priesemann et al 2014). To some extent, branching factor regularization can also replace homeostatic plasticity regularization (Rossbroich et al 2022) used to lower-bound the spiking activity, as the minimum spiking activity in the recurrent pool is dictated by the input spike train when using branching factor regularization. Astrocytes in the brain are also thought to be maintainers of this branching factor, by modulating the activity (Todd et al 2006, Ivanov and Michmizos 2021, Balafrej et al 2022). Synaptic plasticity-enabled models have been known to improve RSnns without supervised learning (Ivanov and Michmizos 2021, Balafrej et al 2022).

### 2.2.1. Implementation

In this work, we compute a global branching factor based on the mean-squared error between the total spike count of each time steps:

$$\xi_{bf} = \sum_{t} \left( \sum_{i} (n_{i}[t] - n_{i}[t-1]) \right)^{2} \quad (4)$$

The branching factor loss sums the difference in spiking activity of all recurrent neurons $n_i$ and over the total duration of each sample. This loss is added to the loss of the network with a regularization parameter $\beta$:

$$\xi_{tot} = \xi_{net} + \beta \xi_{bf} \quad (5)$$

The network loss $\xi_{net}$ is the standard cross-entropy loss (Goodfellow et al 2016) between the target output and the output neurons.

We employ the branching factor learning rule as a foundational element for transmitting information through time. Without it, the loss associated with the network's task would rapidly diminish should the spiking activity cease. To prevent this, we integrate a branching factor regularization of lesser magnitude into the loss function. This adjustment encourages the optimizer to reduce the overall loss by ensuring steady activity for the duration of an input signal. Maintaining this input activity makes it easier for the task-specific loss to increase in magnitude with more activity. This process can be seen as a two-step optimization: first, the branching factor regularization promotes stable network activity, helping to preserve temporal information; then, the task-specific loss fine-tunes the network for its intended function. By incorporating both terms into the loss function, we ensure that these two objectives are achieved simultaneously.
## 2.3. Avoiding vanishing gradient with large surrogate function

To mitigate the effect of the very temporally sparse activation function $\mathcal{H}$ and it is surrogate derivative $\mathcal{H}'$, we hypothesize that the upper-bound $\gamma$ of $\mathcal{H}'$, such that $|\mathcal{H}'(x)| < \gamma$ for $x \in \mathcal{R}$, should be equal to one and that it is support$^4$ must be as large as possible. This idea is similar to how the ReLU activation function has a derivative of 1 for positive values. Moreover, if the membrane potential $u$ of a neuron is ever enough to transmit a spike, i.e. $u > u_{th}$, then the input of this neuron had an impact on the downstream loss function and the gradient should be propagated during backpropagation. We define $\mathcal{H}'$ as:

$$ \mathcal{H}' = \min \{ \max \{ u - u_{th} + 1, 0 \}, 1 \}. \qquad (6) $$

This surrogate derivative always passes the gradient backward, except if the membrane potential is highly inhibited, i.e. $u - u_{th} + 1 < 0$. To mitigate gradient explosion that could be increased by this surrogate derivative, we add standard gradient norm clipping (Pascanu et al 2013) with a max norm of 0.01 during BPTT. In some cases, gradient explosion can still occur abruptly which can be solved by reverting back a previous checkpoint and reducing the learning rate.

## 2.4. Initialization strategy

We use the Kaiming Uniform initialization scheme (He et al 2015b) on the recurrent weights $w_{r_{ij}} \in W_R$, such that the weights are sampled from a uniform distribution $-\frac{\sqrt{6}}{N} < w_{r_{ij}} < \frac{\sqrt{6}}{N}$ with $N$ the number of neurons in the recurrent pool. The diagonal of $W_R$ (self-connections) is set to zero and forced to zero during training, i.e. $w_{r_{ii}} = 0$. Fluctuation-driven initialization (Rossbroich et al 2022) is known to help enhance the memory of deep SNNs trained with surrogate gradient descent, but unnecessary in our work. The initial LIF neuron thresholds $\text{Thresh}_R$ are set at 0.5 and the initial time constant for all neurons is 20 ms.

## 2.5. Training strategy

For all experiments, we use the MADGRAD optimizer (Defazio and Jelassi 2022) with a learning rate of $10^{-3}$ in PyTorch (Paszke et al 2019). All network parameters are trainable ($W_R$, $W_I$, $W_O$, $\tau_R$, $\tau_O$, $\text{Thresh}_R$). Time leak time constants $\tau_R$ for the recurrent neurons and $\tau_O$ for the output neurons are clamped between 0.1 ms and 50 ms after each batch to avoid relying on them for memory. The leak time constants $\tau_R$ and $\tau_O$ are shared amongst the neurons of their respective layers, while the thresholds can be learned per neuron.

## 2.6. Computational Requirements

All experiments were conducted on a computing cluster featuring 4 virtual cores from an Intel Gold 6148 Skylake processor (2.4 GHz) and 1 Nvidia V100 SXM2 GPU, running on a Linux environment with Python 3.8.10. The initial benchmark, cue-accumulation, was executed using CPU resources only. Under this configuration, each epoch required approximately 25 min to complete.

# 3. Results

We tested our approach on several memory-intensive tasks. The first task is named the cue-accumulation task as implemented in Bellec et al (2020). We then present two novel generated benchmarks based on the first task, but with an increased demand for memory capacity as to highlight the limitations of the current state-of-the-art approach. Additionally, we examine the performance of our approach on the permuted-sequential MNIST task, a widely recognized benchmark across both spiking and non-spiking RNNs, to assess their memory capabilities. Finally, we present and test a new benchmark created for this evaluation that we name DN-MNIST, which is based on the widespread vision spiking dataset, N-MNIST (Orchard et al 2015).

## 3.1. Cue-accumulation task

The cue-accumulation task simulates a rodent travelling a virtual T-maze with left or right visual cues. A reward is given depending on the cue count and the direction of the rodent. This benchmark was first considered in Morcos and Harvey (2016), Engelhard et al (2019) and reinterpreted as an artificial benchmark in Bellec et al (2020). The generated benchmark consists of 7 left-or-right cues expressed as groups of neurons spiking and generated every 150 ms, with a 50 ms pause in between the cues. After all the cues, a waiting period of 1200 ms occurs. Then, a recall signal is provided and the artificial network must make its decision between left or right, based on the cue count of each respective class. We train the network over 109

<footnote>$^4$ $x_{\text{max}} - x_{\text{min}}$ such that $x_{\text{min}} = \min_x |\mathcal{H}'(x)| > 0$ and $x_{\text{max}} = \max_x |\mathcal{H}'(x)| > 0$.</footnote>
<figure><img src="image_7.png" /></figure>

epochs with 1024 randomly generated training samples per epoch with a batch size of 64. We then test on 2048 newly generated samples. The branching factor regularization factor $\beta = 0.001$.

A visual representation of the task, and a sample of the generated input sequence, is presented in figure 4(A). This problem requires a minimum of 1200 ms + 7 · 150 ms = 2.25 seconds of memory to solve the task. The cues are population encoded with 10 neurons per side, another 10 neurons for background noise and finally 10 neurons that provide a recall cue, to indicate the time at which a decision must be made. A sample of this task is presented in figure 4(B). Our approach solves the basic version of this task with 98% accuracy. The network showcases oscillatory patterns that are dependent on the chosen delays. After training, the network makes use of the long-term delays to create oscillations to store the input spikes. We present in figure 4(E) the Fourier transform of the spike rate summed for all recurrent neurons and centered at 0. The patterns observed are directly representative of the information being stored in the network. We hypothesize that the network learns to minimize its spike count by promoting communication between neurons that are further apart (which is modeled by longer delays). These long-range spike message will eventually cycle back between long-range neurons if the information is adequately stored, creating these oscillations in the spike pattern. In this experiment, there is a total of 3 possible delays which are 0 ms for neuron within 20% of the maximum radius, 80 ms for neurons within 30% radius and 100 ms for the rest. These categories of delay were arbitrarily distributed into 17.6%, 42.4% and 40.0% of the total neuron population, respectively.

We performed an ablation study to evaluate the impact of the added delays and the branching factor regularization independently. Removing either of those components prevents convergence above random
<figure><img src="image_8.png" /><figcaption>**Figure 5.** Accuracy of our approach for an increased waiting period (A) and an increased number of cues (B) during inference. The wait-period showcases the limitation of an RSNN trained using the LSNN strategy of increasing the time constants to hold the information for longer periods of times. For the increasing number of cues, a probabilistic model is used as a baseline and is defined in the main text.</figcaption></figure>

accuracy. When greatly simplifying the task to 3 cues and a 50 ms pause (instead of the original 1500 ms), the network with branching factor regularization without delays was able to achieve 75% accuracy, while the delay-only network did not converge above random performance.

## 3.2. Generated memory intensive benchmarks

The performance on the standard cue-accumulation task is not enough to showcase the network ability. We therefore created two more difficult tasks to test the limit of our work, based on the cue-accumulation task. We first compare the performance of the network when increasing the waiting time after presentation of the cues. If adequate activity-based memory is achieved, the waiting time should not affect the accuracy of the network. Figure 5(A) presents the accuracy of our network for various waiting times, compared with the state-of-the-art approach, as implemented in Frenkel (2022) with the LSNN, using LIF neurons with a long $\tau_R$ = 2000ms. The accuracy of the latter network quickly degrades when increasing the waiting time, as the fixed 2 s time constant is not enough for such long memory period. On the other hand, the performance of our network with synaptic delays is not influenced when increasing the waiting period. Both methods are compared using the same number of neurons, i.e. 100.

We also test the ability of our approach to generalize over an increasing number of cues, without retraining. The performance is presented in figure 5(B). To validate whether our network can effectively remember beyond 7 cues, we contrasted its performance with that of a probabilistic model. We define this model as a simple function that perfectly retains the last 7 cues and makes the best decision based on that knowledge. For example, if the last 7 cues are all left cues, there is a higher likelihood that the overall sequence is mostly left cues since all cues are independent events. As expected, the probabilistic model can obtain above random accuracy for sequences of more than 7 cues, but the degradation of accuracy is steep. Surpassing the accuracy of the probabilistic model suggests that our network is capable of retaining information about more than 7 cues.

## 3.3. Permuted sequential MNIST

The permuted sequential MNIST (psMNIST) task, as introduced in Le *et al* (2015), involves taking the 784 input pixels of the MNIST handwritten digit dataset, shuffling the order of the pixels in a random but fixed manner, and presenting them sequentially, one at a time, to a RNN over 784 time steps. We benchmarked our approach on this difficult task using 216 recurrent neurons using the normalized value of the pixels as an input current directly fed to the LIF neurons. In this experiment, the delays were optimized using a random hyperparameter search to 0, 25 and 86 ms for the 3 regions of neurons. We train the network for 100 epochs with a batch size of 64 using a branching factor regularization constant $\beta = 10^{-4}$. The random hyperparameter search ran 50 times the experiment, with random initial weights and random delays between 15 and 30 ms for delay one, and 75 and 120 ms for delay two. We used a subset of 3000 images from the training set as validation data to select the best network, and finally measured the test accuracy. Momentum and learning rate were randomly chosen from the discrete sets {0.9, 0.5, 0.0} and {0.001, 0.0001}, with the best model using 0.9 and 0.001, respectively. As shown in table 1, our approach showed state-of-the-art accuracy for this task compared to other spiking neural networks (Yin *et al* 2020). There is still a small gap
<table><caption>**Table 1.** Comparison of test accuracy on psMNIST for both spiking and non-spiking neural networks. Best overall accuracy is in bold, best accuracy with a spiking model is underlined.</caption><thead><tr><th>Model</th><th>Spiking</th><th>Param. Count</th><th>Accuracy</th></tr></thead><tbody><tr><td>SMPCConv (Kim and Park 2023)</td><td>No</td><td>373k</td><td><strong>99.10</strong></td></tr><tr><td>M-LMU (Chilkuri and Eliasmith 2021)</td><td>No</td><td>165k</td><td>98.49</td></tr><tr><td>LSTM (reported in Voelker et al (2019))</td><td>No</td><td>165k</td><td>89.86</td></tr><tr><td>Adaptive RSNN (Yin et al 2020)</td><td>Yes</td><td>138k</td><td>91.0</td></tr><tr><td>This work</td><td>Yes</td><td>49k</td><td><u>94.74</u></td></tr></tbody></table>

with non-spiking networks due to the binary spike constraint, although our network also contains much fewer parameters than all other approaches as we target hardware efficiency.

## 3.4. Delayed-Neuromorphic MNIST

To address the limitations of standard neuromorphic benchmarks for memory-dependent RSNNs, we introduce the Delayed-Neuromorphic MNIST (DN-MNIST) dataset. While N-MNIST Orchard et al (2015) is a robust benchmark for event-based vision, its design, where each MNIST digit is presented with three saccades over approximately 300 ms, provides ample information even within a smaller subset of time. We found that using less than 30 ms suffices for human recognition of the digits, see supplementary figure 1. DN-MNIST extends the temporal gap between sensory input and classification by adding a wait period after the original DVS input during which 10 Hz random Poisson spike noise is injected on all neurons. For example, a 350 ms sample duration in DN-MNIST represents 300 ms of DVS input from the N-MNIST dataset, followed by 50 ms of 10 Hz Poisson spike noise. The task is to maximize the duration of the DN-MNIST sample duration while maintaining a classification accuracy above 90%. This criterion challenges the model's capacity for memory retention and its ability to generalize over extended intervals, a scenario not met by conventional N-MNIST. We believe that DN-MNIST validates our approach's real-world applicability and sets a new standard for benchmarking memory in neuromorphic computing systems. In our experiments, only the ON polarity events from N-MNIST were selected.

To test our approach on this new challenging benchmark, we train our RSNN on DN-MNIST for 1 epoch using the same hyperparameters which were optimized on the psMNIST task with the same optimizer, namely, MADGRAD (Defazio and Jelassi 2022). We kept the size of the network to 216 neurons. We used a batch size of 96 samples. Furthermore, we created different variants to perform an ablation study, where the delays and/or the branching factor regularization were removed, to showcase the impact of each component. The results are presented in figure 6. In this new benchmark, branching factor regularization alone did not seem to help increase the duration of the sample beyond 325 ms. The best resulting network contained both branching factor regularization and synaptic delays and was able to solve the DN-MNIST task with 150 ms of noisy pause after the presentation of the digit (i.e. a total sample duration of 450 ms), demonstrating prolonged memory.

As in the other benchmarks, the membrane potential decay constant, $\tau_R$, was capped at 50 ms in all experiments. Interestingly, when training the RSNN on the DN-MNIST task of length 325 ms, the network without delay or branching factor regularization learned to rely on the exponential decay, with a final trained

<figure><img src="image_9.png" /><figcaption>**Figure 6.** Accuracy of our approach on the DN-MNIST task. Each point represents the test accuracy after training a recurrent spiking network for one epoch. Each line color represents a variation of our work, with different components being removed, namely the branching factor regularization (BFR) and the neuron-level synaptic delays (SD). All networks were tested with 25 ms intervals from 300 to 500 ms. The best performing network on the DN-MNIST task is the one that performs above 90% test accuracy for the longest sample duration.</figcaption></figure>
value of $\tau_R$ = 48.45 ms. In contrast, the network with delays and branching factor regularization resulted in a significantly lower final value of $\tau_R$ = 19.33 ms for the same experiment. Without this maximum value, the baseline network could potentially learn to solve DN-MNIST with a longer pause. However, this would require a large $\tau_R$, which in turn would limit the feasibility of an efficient analog hardware implementation.

# 4. Discussion

We presented a new spiking RNN architecture that enables learning and provides solutions to a set of challenging benchmarks focused on memory. Unlike previous approaches, the modifications that we propose are hardware and neuromorphic-friendly. We do not require, for example, overly large capacitors to store information. Our approach relies on BPTT for learning in a digital system, but the learned neural network could be used for inference on an analog system with no issue—which is the core problem that we are trying to tackle.

## 4.1. Synaptic delays

We started by adding neuron-level synaptic delays. Synaptic delays have been used extensively in feed-forward networks using backpropagation as a way to add a memory buffer. In RNNs, the addition of synaptic delays is not always beneficial since the information can already propagate through time. Nonetheless, we showed that, during training, delays can allow the gradient to skip time steps and reach further up in time. Additionally, we observed that training networks with synaptic delays alters the way memory is maintained. Instead of relying solely on the exponentially diminishing states of individual neurons, the network begins to store information through recurrent activity distributed across its connections. These delays effectively introduce loops within the network's dynamics, allowing past signals to persist longer and influence future states. As a result, memory is no longer confined to the rapid decay of individual neuron activations but is instead sustained through the network's ongoing activity, leading to an extended memory capacity. To achieve this, it is important that the range of possible delays be limited and well below the time scale of the task. Feed-forward neural networks with delays, on the other hand, would need to scale the number and length of delays with the time scale of the task, as the delays are used to join features across time. Only two possible values of non-zero delays were added in our experiment, which are trivial to optimize using any black-box method such as random search. The additional cost in training is therefore negligible. Lastly, the energy overhead of implementing synaptic delays is also minimal, accounting for less than 3% of the network's total energy consumption in Patino-Saucedo et al (2024). To further assess this impact, we measured the energy and latency on Intel's Loihi 2 neuromorphic processor for a simple two-neuron system, both with and without delays ranging from 1 to 20 ms. Our results indicate that synaptic delays have no statistically significant effect on these metrics. Detailed findings are provided in the supplementary material (figure 2).

## 4.2. Branching factor regularization

We then integrated a regularization rule that is based on the branching factor as described in biology. Interestingly, the branching factor is a simple rule that shares common ground with other regularization mechanisms, yet, it enables the network to learn over long time scales. The purpose of the branching factor is to offer an objective for the backpropagation algorithm to facilitate the propagation of activity through time. If the network succeeds in preserving spikes, there's a slight reduction in loss. When the network consistently remembers spikes for the entire duration of the task, this allows the primary objective (the cross-entropy loss) to become the focus, guiding the adjustment of weights to effectively address the task at hand.

## 4.3. Surrogate gradient function

Finally, we increased the support of the surrogate gradient spiking function, and its maximum value. This modification mitigates the vanishing gradient problem at the cost of a possibly exploding gradient. Although, the exploding gradient problem is easily solvable using standard gradient clipping.

## 4.4. Generalization properties

Increasing the wait time or the number of cues in the cue-accumulation task are inherently challenging problems to solve. These tasks are not embedded in the original loss function of the network. However, since the branching-factor regularization helps maintain spike activity and creates oscillations in combination with the synaptic delays, there is no reason why the activity would die off. Oscillations have long been observed in the brain and are known to be connected to memory. When considering the brain as three-dimensional, delays are present due to its nature, and reducing delays is costly in terms of energy. It is
therefore only logical that for mechanisms such as memory, these delays can be used proactively, resulting in back-and-forth oscillations of spiking activity.

The combination of branching factor regularization and synaptic delays can also permit the creation of a network that could be fine-tuned to other tasks. We can train a foundational RSNN model whose job is to store information using spikes. This network could then be adapted and finetuned to other tasks with greater ease than training from scratch and solving the hard temporal credit assignment problem.

**4.5. Limitations and future work**

We demonstrated the importance of adding delays in RSNNs. Although better than the alternatives, delays can still be expensive to implement in hardware. We mitigated this problem by using neuron-level delays and limiting the number of unique delays to three in all the experiments. Having three unique delays can still yield interesting patterns of activity by combining those delays while being trivial to select using any hyperparameter selection framework.

Another important consideration is the scalability of our approach as network size increases. While we focused on maintaining hardware efficiency, larger networks may require alternative delay distribution strategies to balance computational cost and performance. Our current method was designed for practical implementation, but exploring more adaptive or learned delay assignment mechanisms could further enhance scalability. Additionally, while we used the LIF neuron model due to its prevalence in neuromorphic hardware, future work could investigate the impact of delays in more complex neuron models, such as burst-coding neurons and multi-compartment models, to assess their potential benefits.

A promising direction for future research involves incorporating online learning capabilities through synaptic plasticity mechanisms. While our current approach relies on offline training, integrating biologically plausible plasticity rules could enable continuous adaptation to changing input statistics and potentially improve generalization capabilities. Furthermore, we acknowledge that training these networks remains challenging, with convergence issues occasionally arising during optimization. Specifically, we observed that excessive branching factor regularization can lead to no activity in the network. Finding the optimal balance in regularization strength combined with a good set of initial weights remains an important consideration for successful implementation. Finally, given that RSNNs excel in low-cost real-time inference scenarios, applying our delay-based methodology to audio processing tasks represents a particularly compelling application area. Audio tasks typically involve temporal processing that could benefit significantly from the temporal dynamics enabled by our approach, while maintaining the energy efficiency advantages inherent to spiking neural networks.

**Data availability statement**

No new data were created or analysed in this study.

**Acknowledgments**

We thank Damien Querlioz and Adrien Vincent for the fruitful discussions over the ideas behind this work. We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC), funding Reference Number 559730. Fabien Alibart acknowledges support from IONOS-ERC Project #773228 and the *Chaire Neuromorphique Ministère de l'Économie et de l'innovation du Québec, Université de Sherbrooke*. We thank the Fonds de recherche du Québec-nature et technologies (FRQNT) for the Chistera UNICO Project #287330. This research was enabled in part by support provided by Calcul Québec (calculquebec.ca) and the Digital Research Alliance of Canada (alliancecan.ca). We thank Intel Inc. for access to the Loihi 2 neuromorphic processor.

**Author contribution**

I B, J R, and F A designed the research. I B conducted the experiments, with S B contributing by creating the power and latency experiment on Loihi. I B, J R, and F A wrote the paper.

**ORCID iDs**

Ismael Balafrej https://orcid.org/0000-0001-6730-0794

Soufiyan Bahadi https://orcid.org/0000-0003-0216-0457

Jean Rouat https://orcid.org/0000-0002-9306-426X

Fabien Alibart https://orcid.org/0000-0002-9591-220X
References

Abeles M 1991 *Corticonics: Neural Circuits of the Cerebral Cortex* (Cambridge University Press) (https://doi.org/10.1017/CBO9780511574566)

Balafrej J, Alibart F and Rouat J 2022 P-critical: A reservoir autoregulation plasticity rule for neuromorphic hardware *Neuromorph. Comput. Eng.* **2** 024007

Beggs J M 2007 The criticality hypothesis: how local cortical networks might optimize information processing *Phil. Trans. R. Soc. A* **366** 329–43

Bellec G et al 2018 Long short-term memory and learning-to-learn in networks of spiking neurons (arXiv:1803.09574)

Bellec G, Scherr F, Subramoney A, Hajek E, Salaj D, Legenstein R and Maass W 2020 A solution to the learning dilemma for recurrent networks of spiking neurons *Nat. Commun.* **11** 1–15

Bengio Y, Simard P and Frasconi P 1994 Learning long-term dependencies with gradient descent is difficult *IEEE Trans. Neural Netw.* **5** 157–66

Blouw P, Choo X, Hunsberger E and Eliasmith C 2019 Benchmarking keyword spotting efficiency on neuromorphic hardware *Proc. 7th Annual Neuro-Inspired Computational Elements Workshop (ACM)*

Buzsáki G and Draguhn A 2004 Neuronal oscillations in cortical networks *Science* **304** 1926–9

Chilkuri N R and Eliasmith C 2021 Parallelizing Legendre Memory Unit Training *Proc. 38th Int. Conf. on Machine Learning (Proc. Machine Learning Research vol 139)* ed M Meila and T Zhang (PMLR) pp 1898–907

D'Agostino S, Moro F, Torchet T, Demirağ Y, Grenouillet L, Castellani N, Indiveri G, Vianello E and Payvand M 2024 DenRAM: neuromorphic dendritic architecture with RRAM for efficient temporal processing with delays *Nat. Commun.* **15** 3446

Dalgaty T, Payvand M, Moro F, Ly D R B, Pebay-Peyroula F, Casas J, Indiveri G and Vianello E 2019 Hybrid neuromorphic circuits exploiting non-conventional properties of RRAM for massively parallel local plasticity mechanisms *APL Mater.* **7** 081125

Davies M et al 2018 Loihi: a neuromorphic manycore processor with on-chip learning *IEEE Micro* **38** 82–99

Davies M, Wild A, Orchard G, Sandamirskaya Y, Guerra G A F, Joshi P, Plank P and Risbud S R 2021 Advancing neuromorphic computing with loihi: a survey of results and outlook *Proc. IEEE* **109** 911–34

Defazio A and Jelassi S 2022 A momentumized, adaptive, dual averaged gradient method *J. Mach. Learn. Res.* **23** 1–34

Demirag Y, Moro F, Dalgaty T, Navarro G, Frenkel C, Indiveri G, Vianello E and Payvand M 2021 PCM-trace: scalable synaptic eligibility traces with resistivity drift of phase-change materials (arXiv:2102.07260)

Engelhard B et al 2019 Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons *Nature* **570** 509–13

Frenkel C 2022 PyTorch e-prop implementation (available at: https://github.com/ChFrenkel/eprop-PyTorch/)

Frenkel C and Indiveri G 2022 ReckOn: a 28nm Sub-mm2 Task-Agnostic spiking recurrent neural network processor enabling on-chip learning over second-long timescales *2022 IEEE Int. Solid- State Circuits Conf. (ISSCC)* vol 65 pp 1–3

Gilson M, Bürck M, Burkitt A N and van Hemmen J L 2012 Frequency selectivity emerging from spike-timing-dependent plasticity *Neural Comput.* **24** 2251–79

Goodfellow I, Bengio Y and Courville A 2016 *Deep Learning* (MIT Press)

Haldeman C and Beggs J M 2005 Critical branching captures activity in living neural networks and maximizes the number of metastable states *Phys. Rev. Lett.* **94** 1–4

Han J-K, Yun S-Y, Lee S-W, Yu J-M and Choi Y-K 2022 A review of artificial spiking neuron devices for neural processing and sensing *Adv. Funct. Mater.* **32** 2204102

He K, Zhang X, Ren S and Sun J 2015a Deep residual learning for image recognition (arXiv:1512.03385)

He K, Zhang X, Ren S and Sun J 2015b Delving deep into rectifiers: surpassing human-level performance on imagenet classification (arXiv:1502.01852)

Herranz-Celotti L and Rouat J 2022 Stabilizing spiking neuron training (arXiv:2202.00282v4)

Ivanov V and Michmizos K 2021 Increasing liquid state machine performance with edge-of-chaos dynamics organized by astrocyte-modulated plasticity *Advances in Neural Information Processing Systems* p 34

Izhikevich E M 2006 Polychronization: computation with spikes *Neural Comput.* **18** 245–82

Jin Y, Liu Y and Li P 2016 SSO-LSM: a sparse and self-organizing architecture for liquid state machine based neural processors *Proc. 2016 IEEE/ACM Int. Symp. on Nanoscale Architectures, NANOARCH vol 2016* pp 55–60

Kim S and Park E 2023 SMPCong: self-moving point representations for continuous convolution (arXiv:2304.02330)

Le Q V, Jaitly N and Hinton G E 2015 A simple way to initialize recurrent networks of rectified linear units (arXiv:1504.00941)

Luo X, Qi H, Wang Y, Yi Z, Zhang J and Zhang M 2023 Supervised learning in multilayer spiking neural networks with spike temporal error backpropagation *IEEE Trans. Neural Netw. Learn. Syst.* **34** 10141–53

Maass W, Natschläger T and Markram H 2002 Real-time computing without stable states: a new framework for neural computation based on perturbations *Neural Comput.* **14** 2531–60

Morcos A S and Harvey C D 2016 History-dependent variability in population dynamics during evidence accumulation in cortex *Nat. Neurosci.* **19** 1672–81

Neftci E O, Mostafa H and Zenke F 2019 Surrogate gradient learning in spiking neural networks: bringing the power of gradient-based optimization to spiking neural networks *IEEE Signal Process. Mag.* **36** 51–63

Orchard G, Jayawant A, Cohen G K and Thakor N 2015 Converting static image datasets to spiking neuromorphic datasets using saccades *Front. Neurosci.* **9** 437

Pascanu R, Mikolov T and Bengio Y 2013 On the difficulty of training recurrent neural networks *Proc. 30th Int. Conf. on Machine Learning (Proc. of Machine Learning Research vol 28)* ed S Dasgupta and D McAllester (PMLR) pp 1310–8

Paszke A et al 2019 PyTorch: an imperative style, high-performance deep learning library *Advances in Neural Information Processing Systems* vol 32, ed H Wallach, H Larochelle, A Beygelzimer, F d'Alché Buc, E Fox and R Garnett (Curran Associates, Inc.)

Patino-Saucedo A, Meijer R, Yousefzadeh A, Gomony M-D, Corradi F, Detteter P, Garrido-Regife L, Linares-Barranco B and Sifalakis M 2024 Hardware-aware training of models with synaptic delays for digital event-driven neuromorphic processors (arXiv:2404.10597)

Payvand M, Moro F, Nomura K, Dalgaty T, Vianello E, Nishi Y and Indiveri G 2022 Self-organization of an inhomogeneous memristive hardware for sequence learning *Nat. Commun.* **13** 5793

Priesemann V, Wibral M, Valderrama M, Pröpper R, Le Van Quyen M, Geisel T, Triesch J, Nikolić D and Munk M H J 2014 Spike avalanches *in vivo* suggest a driven, slightly subcritical brain state *Front. Syst. Neurosci.* **8** 108

Rossbroich J, Gygax J and Zenke F 2022 Fluctuation-driven initialization for spiking neural network training *Neuromorph. Comput. Eng.* **2** 044016
Schmidhuber J 2015 Deep learning in neural networks: an overview *Neural Netw.* **61** 85–117

Shrestha S B and Song Q 2016 Adaptive delay learning in SpikeProp based on delay convergence analysis *2016 Int. Joint Conf. on Neural Networks (IJCNN) (IEEE)* pp 277–84

Swadlow H A and Waxman S G 2012 Axonal conduction delays *Scholarpedia J.* **7** 1451

Todd K J, Serrano A, Lacaille J-C and Robitaille R 2006 Glial cells in synaptic plasticity *J. Physiol.* **99** 75–83

Voelker A, Kajić I and Eliasmith C 2019 Legendre memory units: continuous-time representation in recurrent neural networks *Advances in Neural Information Processing Systems* p 32

Waibel A, Hanazawa T, Hinton G, Shikano K and Lang K J 1989 Phoneme recognition using time-delay neural networks *IEEE Trans. Acoust. Speech Signal Process.* **37** 328–39

Wang X, Lin X and Dang X 2019 A Delay learning algorithm based on spike train kernels for spiking neurons *Front. Neurosci.* **13** 252

Xin W and Chan J R 2020 Myelin plasticity: sculpting circuits in learning and memory *Nat. Rev. Neurosci.* **21** 682–94

Yin B, Corradi F and Bohté S M 2020 Effective and efficient computation with multiple-timescale spiking recurrent neural networks *Int. Conf. on Neuromorphic Systems 2020 (ICONS 2020)* (Association for Computing Machinery) pp 1–8

Zenke F and Vogels T P 2021 The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks *Neural Comput.* **33** 899–925

Zhang M, Wu J, Belatreche A, Pan Z, Xie X, Chua Y, Li G, Qu H and Li H 2020 Supervised learning in spiking neural networks with synaptic delay-weight plasticity *Neurocomputing* **409** 103–18