Exploring the Limitations of Layer Synchronization in Spiking Neural Networks

## Abstract

Abstract Currently, neural-network processing in machine learning applications relies on layer synchronization, whereby neurons in a layer aggregate incoming currents from all neurons in the preceding layer, before evaluating their activation function. This is practiced even in artificial Spiking Neural Networks (SNNs), which are touted as consistent with neurobiology, in spite of processing in the brain being, in fact asynchronous. A truly asynchronous system however would allow all neurons to evaluate concurrently their threshold and emit spikes upon receiving any presynaptic current. Omitting layer synchronization is potentially beneficial, for latency and energy efficiency, but asynchronous execution of models previously trained with layer synchronization may entail a mismatch in network dynamics and performance. We present a study that documents and quantifies this problem in three datasets on our simulation environment that implements network asynchrony , and we show that models trained with layer synchronization either perform sub-optimally in absence of the synchronization, or they will fail to benefit from any energy and latency reduction, when such a mechanism is in place. We then “make ends meet” and address the problem with unlayered backprop , a novel backpropagation-based training method, for learning models suitable for asynchronous processing. We train with it models that use different neuron execution scheduling strategies, and we show that although their neurons are more reactive, these models consistently exhibit lower overall spike density (up to 50%), reach a correct decision faster (up to 2x) without integrating all spikes, and achieve superior accuracy (up to ∼ similar-to \sim ∼ 10% higher).
Our findings suggest that asynchronous event-based (neuromorphic) AI computing is indeed more efficient, but we need to seriously rethink how we train our SNN models, to benefit from it.

## 1 Introduction

Artificial Neural Networks (ANNs) are the foundation behind many of the recently successful developments in AI, such as in computer vision [, ] and natural language processing [, ].
To match the complexity of the ever more demanding tasks, networks have grown in size, with advanced large language models having billions of parameters [].
With this the power consumption exploded [], limiting the deployment to large data centers.
In an effort to learn from our brain’s superior power efficiency, and motivated in neuroscience research, SNNs [] bolster as an alternative. They use discrete binary or graded spikes (events) for communication, are suited for processing sparse features [], and when combined with asynchronous event-based processing are assumed to enhance latency and energy efficiency (and scalability). Sparsity leads to fewer synaptic operations, resulting in low energy consumption, and asynchronous operation potentiates concurrent evaluation of all neurons in the network purely event-driven, leading to low latency.

Conventional highly parallel ANN compute accelerators, such as Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), which are primarily optimized for dense vectorized matrix operations, face inherent challenges in exploiting sparsity for improving their energy efficiency. Targeting the commonplace practiced way of executing ANNs layer-after-layer has left them with poor support for asynchronous processing too (for improving latency). At best, they parallelize processing within a layer and/or pipeline processing across layers. This leaves an exploration space for neuromorphic processors that try to excel in handling the event-driven nature of SNNs and leverage asynchronous concurrent processing, offering efficiency advantages in various tasks [, , ].

However, despite this advancement, the training of SNNs today very often conveniently relies on conventional end-to-end ANN training methods for performance [],
which organize/synchronize computations per-layer rather than event-driven per neuron [].
Specifically, at any given discrete timestep within a neuron layer, first, the total of all presynaptic currents (from the preceding layer) must be computed and integrated, before postsynaptic neurons in the current layer update their state and evaluate their activation function (i.e. emitting new spikes). For consistency, neuron evaluation in one layer must thus complete and synchronize before proceeding to evaluate neurons of a next layer.
This *breadth-first* processing approach (with per layer synchronization), while it facilitates use of vectorized computing hardware (such as GPUs) during training, it introduces dependencies on synchronization that could impact model accuracy if altered during inference. Consequently, even asynchronous neuromorphic processors, such as Loihi [], have integrated mechanisms to ensure (and enforce) layer synchronization.

This leaves a crucial (efficiency) aspect of SNNs relatively unexplored: the ability to allow spiking completely asynchronously across the network without having separate phases for integration and firing, just like in our brain []. In that neurophysical modus operandi, neurons can fire and receive currents anywhere in the network at any time, completely independent, a concept we term *"network asynchrony"*.
Allowing network asynchrony can be advantageous [], as we confirm in our results. Spike activity can quickly propagate deep into the network without being bound by synchronization barriers, reducing latency.
Furthermore, adhering to layer synchronization could lead to increased computational overhead as the network scales, as suggested by Amdahl’s law []. This implies that the overhead grows non-linearly by adding more computational units to a group that needs to be synchronized at some point in time []. With network asynchrony, such groups can be kept smaller, and the number of synchronization moments can be minimized, potentially reducing the waiting time.

In this paper, we demonstrate this problem and explore solutions.
Using a simulator that implements the concept of network asynchrony, we provide quantitative results based on 3 benchmark datasets (with different spatio-temporal structures) that show the performance degradation and latency/energy inefficiency resulting from changes in model dynamics when trained with layer synchronization and later deployed for asynchronous inference.
We then explore potential solutions by proposing a novel gradient (backpropagation-based) training method, that can be parameterized with various neuron execution scheduling strategies for asynchronous processing, and vectorization abilities that may be available in different neuromorphic processors. We show that using this training method, it is possible to prepare models that achieve superior accuracy and, at the same time, can save energy and inference latency under asynchronous processing (when compared to the conventional breadth-first processing in GPUs). This work opens a space with unique potential for design space explorations aimed to bridge the efficiency gap between neural network model training and asynchronous processor design.

## 2 Related work

The term “asynchronous processing” is often used whenever neurons can be active in parallel and communicate asynchronously, even if some synchronization protocol is enforced to control the order of spikes and synaptic current processing [, , , ].
Another kind of asynchrony is related to input coding[].
In this context, synchronizing means grouping spikes (events) into frames, a topic that has extensively been researched [, , , , ].
Neither of the two, however, is the focus of this work. Here, asynchrony refers solely to activity within the network not being artificially bound by any order restriction. This has been researched for simulating biologically accurate neural networks [, , ], but remains under-explored in the context of executing SNNs for machine learning. In this context, grouping and processing in layers is just one convenient way [].

Few event-driven neuromorphic processors, such as $\upmu$Brain [] and Speck [], are fully event-driven and lack any layer synchronization mechanism. This makes them notorious for training out-of-the-box with mainstream model training tools that rely on per-layer synchronization (e.g., PyTorch). Speck developers propose to train their models with hardware in-the-loop [] to reduce the mismatch between the training algorithms and the inference hardware. However this method does not provide a general solution for training asynchronous neural networks. Others, like SpiNNaker [] and TrueNorth [], use timer-based synchronization, or Loihi [] and Seneca [] use barrier synchronization between layers, in order to warrant that the asynchronous processing dynamics on hardware are aligned at layer boundaries with models trained in software (with layer synchronization). This, however, entails an efficiency penalty, as we will show.

Functionally, the most suitable type of model for end-to-end asynchronous processing is probably rate-coded SNN models, whereby neurons can integrate state and communicate independently from any other neuron. These models are trained like ANNs [, , ] or converted from pre-trained ANNs [, ]. Therefore, to be accurate under asynchronous processing, it is required to run the inference for a long time, reducing the latency and energy benefit of using SNNs [].

Alternative to rate-coding models are temporal-coding models, with time-to-first spike (TTFS) [, , , ] or order encoding []. They are very sparse (hence energy efficient) but very cumbersome and elaborate to convert from ANNs [, ] or train directly [, ], less tolerant to noise [], and their execution so far, while event-based, requires some form of synchronization or a reference time. Efficiency is thus only attributable to the reduced number of spikes, all of which need to be evaluated in order before a decision is reached. Other alternative encodings for event-based processing of SNNs include phase-coding [] and burst coding [], which are, however, no more economical than rate coding and have not been shown, to our knowledge, to attain competitive performance.

The approach presented in this paper is, in fact, unique in enabling the trainability of models for event-based asynchronous execution, providing efficiency from processing only a subset of spikes, and delivering consistent performance and tolerance to noise. Also relevant to the works in this paper are SparseProp [] and EventProp [] on efficient event-based simulation and training, respectively. EventProp[] is potentially more economical than discrete-time backpropagation for training event-based models for asynchronous processing (and fully compatible with the work in this paper), but it has not been shown how its complexity scales beyond 1 hidden layer. SparseProp [] proposes an efficient neuron execution scheduling strategy for asynchronous processing in software simulation. An effective hardware implementation of this scheduling strategy can be found in [], which has inspired the herein proposed *“momentum schedule”*. Moreover, we advance this research by demonstrating how to train SNN models using this event scheduler.

## 3 Methods for simulating and training of asynchronous SNNs

In this section, we provide an overview of our methodology. We first introduce our simulator for SNN processing with network asynchrony and then show how this is used for training.

### 3.1 Simulating asynchronous SNNs

The SNNs used in this work consist of $L$ layers of Leaky Integrate and Fire (LIF) neurons [], with each layer $l$ for $1\leq l\leq L$ having $N^{(l)}$ neurons and being fully-connected to the next layer $l+1$, except for the output layer. Each input feature is connected to all neurons in the first layer. Every connection has a synaptic weight. Using these weights, we can compute the incoming current $x$ per neuron resulting from spikes, as per equation 3.

Each LIF neuron has a membrane potential exponentially decaying over time based on some membrane time constant $\tau_{m}$. We use the analytical solution (see equation 4) to compute the decay. By keeping track of the membrane potential $u[t]$ and the elapsed time $\Delta t$ since time $t$, it is possible to precisely calculate the membrane potential for $t+\Delta t$. Therefore, computations are required only when $x[t+\Delta t]>0$.

To determine if a neuron spikes, a threshold function $\Theta$ checks if the membrane potential exceeds a threshold $U_{\text{thr}}$, following equation 5. If $\Theta(u)=1$, the membrane potential is immediately hard reset to 0 0. Soft reset is another option in the simulator often defended in the literature [] as well as refractoriness []. However for simplicity here we only experiment with hard reset.

#### 3.1.1 Event-driven state updates

We can vectorize the computations introduced in the previous section to perform them on a per-layer basis. This is the "layered" inference approach, which implies layer synchronization, since all neurons of one layer evaluate their state at the same instant.
An alternative to this is an event-driven approach, where the computations for state updates are applied in response to new spike arrivals. Spikes can be emitted at any time by any neuron, affecting the states of postsynaptic neurons independently of others. Such an approach can achieve a true representation of network asynchrony.

When using event-driven updates, the network dynamics evolve with each spike. A single spike can generate multiple currents, each linked to a specific time $t$, determined by the initiating input activity. Let $K_{t}$ represent the atomic computational unit that is executed for an input current received at time $t$ and that will update the state of a single neuron and evaluate its threshold function (activation). The exact computations for a LIF neuron can be found in section A.2.
The order in which $K_{t}$ is executed across neurons inside the entire network (and not just in one layer) affects the output of the network due to the non-linear nature of $K_{t}$ (see figure 1 for an example of the effect this can have). For simplicity, it is assumed that the currents resulting from the same spike are processed in a group at once. This processing is done when an emitted spike is selected for processing, i.e., when the spike is propagated. This may not necessarily be the same moment as when the spike was emitted due to asynchrony. Because of these assumptions, analyzing the spike propagation order gives the same insights as analyzing the execution order of $K_{t}$.

Unlike the layered approach, with event-driven state updates neurons can theoretically fire more than once at the same time $t$.
We prevent this by inducing a neuron to enter a refractory state for the remainder of time $t$. In this state, the neuron keeps its membrane potential at 0. This makes the model causally simpler, more tractable, and more "economic" when energy consumption is coupled to spike communication. These aspects also give an explicit retrospectively justification for the neurophysically observed refractoriness [].

Figure: Figure 1: With network asynchrony, currents from neurons $x_{i}$ in layer $l$ may come to neuron $y$ in layer $l+1$ at any time in any order. It may spike before all inputs have been received in a discrete timestep. Illustrated is the exceedance of the firing threshold at point $A$. All inputs, and thus information after $A$ is missed, i.e., the spiking decision is made based on partial information. In this example, if layer synchronization is enforced, neuron $y$ would wait for 100% of the input and will not spike.
Refer to caption: extracted/5783407/figures/async_dynamics.png

##### The concept of time

We apply input framing in discrete timesteps as typically done for inputs from a DVS sensor. The *timestep size* refers to the duration within which all input spikes are collected into a single matrix (also called frame), before being propagated throughout the network collectively in what is termed a *forward pass*. This input framing process precedes any network activity and follows the completion of all activity from the last forward pass. The timestamps of input spikes are rounded down to the nearest multiple of the timestep size.

The timestamps on the input spikes provide a meaningful measure of time. Upon arrival, they initiate network activity, making all new spikes contingent on their timing. Thus, they serve as the markers for the progression of time in timesteps, while the order of spike propagation though the network establishes another concept of “time” within a forward pass (or “micro-time” between timesteps).

#### 3.1.2 Vectorized network asynchrony

The event-driven (neuron state) update rules for network asynchrony as introduced in the previous section can be vectorized by selecting multiple spikes for processing at the same time.
This allows us to consider the entire spectrum of possibilities between full layer synchronization at one extreme, and “complete” asynchrony at the other extreme, where each spike event is processed entirely independently of all others.
Additionally, vectorization makes acceleration possible by exploiting the parallelization features and vector pipelines of accelerators, where these models execute, leading to pragmatic simulation of network asynchrony.

During the simulation, the states of all the $N=\sum_{l=1}^{L}N^{(l)}$ neurons in the network are stored in vectors. The vector $\mathbf{x}\in\mathbb{R}^{N}$ tracks the computed input currents for the neurons, $\mathbf{u}\in\mathbb{R}^{N}$ the membrane potential of the neurons, $\mathbf{s}\in\mathbb{N}^{N}$ the emitted spikes queue, and $\mathbf{c}\in\mathbb{N}^{N}$ which neurons have spiked in the current forward pass.
For any of those vectors, the indices from $\sum_{k=1}^{l-1}N^{(k)}$ to $\sum_{k=1}^{l}N^{(k)}$ represent the values for the neurons in layer $l$, for $1\leq l\leq L$ where $N^{(0)}=0$.

Algorithm 1 outlines the processing during a forward pass (propagation of the spikes of an input frame through the network). Before starting, the input spikes from time $t_{0}$ to $t_{1}$ are grouped, resulting in an input vector $\mathbf{s_{\text{in}}}\in\mathbb{N}^{N_{\text{in}}}$ where $N_{\text{in}}$ is the number of input features. Then, the input spikes are passed as one single batch, triggering a forward pass. Each *forward pass* consists of *forward steps* (the code within the while loop), which update the state of the neurons based on the spikes selected for propagation. A complete list of parameters can be found in section A.5.

Figure: Algorithm 1 Vectorized network asynchrony forward pass

The forward pass always ends when all spike activitations have been processed, a stop condition referred to as "On spiking done". Additionally, EarlyStop is used to interrupt a forward pass if any neuron in the output layer has spiked zero, one, or more (default is one) forward steps ago. This is termed the "On output" stop condition.

The SelectSpikes function defines how to select a subset of the emitted spikes for propagation. The function selects $F$ spikes at a time, or less if there are less than $F$ remaining spikes ready to be propagated. The method of spike selection is determined by a scheduling policy. For the experiments, two policies are considered. The first, Random Scheduling (RS), randomly picks spikes from the entire network. The second, Momentum Scheduling (MS), prioritizes spikes from neurons based on their membrane potential upon exceeding their threshold.

The neuron model-specific (LIF) behavior is expressed in the NeuronDecay and NeuronForward functions. This entails the computations for state updates (see section 3.1) for all neurons in the network (NeuronDecay) or for only those neurons receiving input currents in the forward step (NeuronForward), with added restriction that spiking is only allowed once per forward pass.

The network architecture is defined by the InputLayerForward and NetworkLayersForward functions. These functions compute the values of synaptic currents from spikes. For a network with only fully-connected layers, this follows from equation 3.

##### Imitating neuromorphic accelerator hardware

Neuromorphic processors come in various architectures, but most of them have a template architecture that involves interconnecting many tiny processing cores with each other. This design enables a scalable architecture that supports fully distributed memory and compute systems. In this template, each processing core has its own small synchronous domain, but the cores are asynchronous among each other. For instance, the architectures of Loihi [], SENECA [], SpiNNaker [], Speck [], TrueNorth [], POETS [] and $\upmu$Brian [] all follow this template. Our simulator imitates the behavior of a generalized neuromorphic processor by simulating the asynchronous interactions of several tiny synchronous domains and can be further customized to reproduce more intrinsic behavior of many of those processors (see table 4).

### 3.2 Training asynchronous SNNs

We use backpropagation to estimate the gradients of a loss function with respect to the weights.
If multiple forward passes are needed, such as for sequential or temporal data, then Backpropagation Through Time (BPTT) can be used for training. Given that spike trains are temporal data as well, BPTT also applies to SNNs, and by extension, to the research presented here.

To address the issue of the non-differentiability of the threshold function, the surrogate gradient method is used []. An arctan function [], (see section A.3 for details) provides a continuous and smooth approximation of the threshold function.

Class prediction is based on softmaxed spike counts over time, as described in section A.4.
The loss is minimized using the Adam optimizer, with $\beta_{1}=0.9$ and $\beta_{2}=0.999$.

#### 3.2.1 Unlayered backpropagation

Training of SNNs with per-layer synchronization is by now straightforward using backpropagation [] so long as the forward processing is differentiable. We refer to this as "layered backpropagation".

The vectorized network asynchronous processing approach is differentiable as well, and can be used with backpropagation. We refer to this as "unlayered backpropagation".
Combined with BPTT unrolling, this method implies a two level unrolling. In the outer level, unrolling is based on discretization of time in timesteps (when new inputs are provided); within the inner level, unrolling is based on the $F$-grouping of spikes, given a scheduling policy, and the activity of neurons up-until the output is read.
For a single *backward pass*, which can be used in BPTT in a similar way as the layered backpropagation equivalent, it is given that:

$$ $\frac{\partial L_{t}}{\partial W}=\frac{\partial L_{t}}{\partial c_{t}}\sum_{i =1}^{N_{t}}(\frac{\partial c_{t}}{\partial s_{i}}\frac{\partial s_{i}}{ \partial W})$ (1) $$

where $t$ is the time(step) of the *forward pass*, $W$ can be a weight on any of the synapses, $L_{t}$ is the loss, $c_{t}$ is the spike count at the end of the forward pass, and $s_{i}$ are the spikes in the queue at the end of the forward step $i$, with $N_{t}$ *forward steps* in total within the forward pass (referring to concept of (micro)-time in 3.1.1). The number of steps scales linearly with the number of spikes processed in the forward pass. It is important to understand that due to asynchronous processing the number of spikes processed until the evaluation of the loss function may be (well) less than the total number of spikes emitted.
Since for every forward step, the computations are repeated, the time complexity scales linearly with the number of forward steps: $O(N_{t})$. The same applies to the space complexity.

During the backward pass, the spikes which are not selected can skip the computation $f$ for that step:

$$ $\frac{\partial s_{i}}{\partial W}=\frac{\partial f(s_{i-1;\text{selected}},u_{ i-1},x_{t},W)}{\partial W}+\frac{\partial s_{i-1;\text{not selected}}}{ \partial W}$ (2) $$

Skip connections have been researched in deep ANNs and identified as a major contributor to the stability of the training process []. This may apply to the skip in unlayered backpropagation as well. To what extent this is the case is not explored in this work.

#### 3.2.2 Regularization techniques

During training, we use regularization to prevent overfitting and/or enhance model generalization. These techniques are not used during inference.

Input spike dropout.
Randomly omits input spikes with a given probability. The decision to drop each spike is independent according to a Bernoulli distribution.

Weight regularization.
Adds weight decay to the loss function:
$L_{\lambda_{W}}(\mathbf{W})=L(\mathbf{W})+\lambda_{W}\|\mathbf{W}\|_{2}^{2}$
where $\lambda_{W}$ is the regularization coefficient, $L$ is the loss given the weights, and $\mathbf{W}$ are all the weights.

Refractory dropout.
With some probability, do not apply the refractory effect, allowing a neuron to fire again within the same forward pass.

Momentum noise.
When using momentum scheduling, noise sampled from $U(0,1)$ and multiplied by some constant $\lambda_{\text{MS}}$ is added to the recorded membrane potential while selecting spikes.

## 4 Results

### 4.1 Experimental setup

We experimented with trained SNN models for asynchronous execution in three common benchmarking datasets: N-MNIST [], SHD [], and DVS gesture []. Each of them has different structure. N-MNIST has purely spatial structure, SHD purely temporal, and DVS gesture combines both spatial and temporal (i.e. input framing in DVS gesture is done such that an entire gesture motion and contour is not revealed within a single frame). More details on the datasets, can be found in section A.6.1.
Table 1, summarizes the parameterization of the experiments and the reported results. The network architecture and hyperparameters are given in table 5. State-of-the art performance for these tasks can be achieved with reasonably shallow and wide models. We chose however to train narrow, but deeper network architectures so that the effects of absence of layer synchronization can be revealed in the comparison (our aim is not to compete the state-of-the art). In section A.7, details on how the forward group size, refractory dropout, and momentum noise during training affect accuracy are provided.

**Table 1: Parameterization of experiments and results.**
| Parameterization | Description |
| --- | --- |
| Training method | Training with layered backpropagation is marked as "Layered" and with unlayered backpropagation as "Unlayered [scheduling strategy]". |
| Inference method | Inference with layer synchronization is marked as "Layered" and with network asynchrony as "Async [scheduling strategy]". |
| Scheduling strategy | How to select spikes from the queue. Can be random ("RS"), or based on the membrane potential just before spiking ("MS"). |
| Forward group size $F$ | Number of spikes to select for processing at the same time. Default is 8, both during training and inference. |
| Stop condition | Forward pass terminates: either when all network activity drains ("On spiking done") or one forward step after the first spike is emitted by the output layer ("On output"). The default is "On spiking done" during training and "On output" during inference. |

### 4.2 Network asynchrony increases neuron reactivity

As observed in figure 2 (top row) during asynchronous inference, neurons are more reactive, i.e. a neuron can spike after integrating only a small number of incoming currents. With layer synchronization, this effect is averaged out as neurons are always required to consider all presynaptic currents (which are more likely to cancel each other out). For inference with $F=8$, artifacts in the number of currents to spike can be observed, because each forward step propagates currents resulting from 8 spikes, causing neurons to integrate more currents than necessary for firing. Similar artifacts might also occur in neuromorphic chips equipped with fixed-width vector processing pipelines; making these observations insightful into the behavior of such hardware.

Figure: Figure 2: (Top row) Number of currents integrated by a neuron before spiking, recorded per neuron and per forward pass for all samples and neurons (excluding the neurons in the input layer). The Y-axis shows the relative frequency of the number of currents integrated before spiking. (Bottom row) Mean number of spikes per neuron during inference of all samples in the test. Error bars show the 25th and 75th percentiles. (Models in this figure were trained with layered backpropagation.)
Refer to caption: extracted/5783407/figures/network_dynamics_spike_count.png

Interestingly while one expects that more reactive neurons imply higher activation density, surprisingly this is not the case for the models trained for asynchronous inference! In figure 2 (bottom row) we see that if we wait for the network to “drain” of spike activity the total number of spikes will indeed be higher, but if the forward pass terminates as soon as a decision is made, asynchronous models are consistently sparser. This is because asynchronous processing allows spike activity to freely flow through to the output and not be blocked at every layer for synchronization.

### 4.3 Unlayered backpropagation increases accuracy and sparsity

Network asynchrony negatively affects the performance of the models trained with layered backpropagation in all three datasets (table 2). This is likely in general the “Achilles’ heel” of neuromorphic AI today. However, we observe that the accuracy loss is remediated when using unlayered backpropagation, which also significantly increases sparsity (by about $2$x). In fact, the accuracy of models trained and executed asynchronously is consistently superior under both scheduling policies (with momentum scheduling having consistently an edge). This result alone suffices to attest that neuromorphic AI is competitive and more energy efficient.

Figure 3 reveals another interesting result. It depicts how accuracy evolves as we allow more forward steps in the forward pass after the initial output during asynchronous inference. We see that because of the free flow of key information depth-first, models trained with unlayered backpropagation obtain the correct predictions as soon as the output layer gets stimulated. Activity that is likely triggered by “noise” in the input is integrated later on. Momentum scheduling is particularly good at exploiting this to boost accuracy.

**Table 2: Accuracy and activation density results. More details about these metrics in section A.6.2.**
|  |  | N-MNIST | SHD | DVS gesture |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Training | Inference | Acc. | Density | Acc. | Density | Acc. | Density |
| Layered | Layered | 0.949 | 3.987 | 0.783 | 14.386 | 0.739 | 46.473 |
| Layered | Async RS | 0.625 | 3.652 | 0.750 | 13.905 | 0.701 | 44.140 |
| Unlayered RS | Async RS | 0.956 | 1.504 | 0.796 | 5.100 | 0.777 | 25.140 |
| Unlayered MS | Async MS | 0.963 | 1.476 | 0.816 | 6.224 | 0.856 | 26.686 |

Figure: Figure 3: Accuracy as function of forward steps after the first spike in the output layer. Models trained with layer synchronization or random scheduling, are processed with random scheduling. Models trained with momentum scheduling, are processed with momentum scheduling. Given that $F=8$, each extra forward step processes another 8 spikes, assuming enough spikes are available. Dashed lines show the accuracy after all spike activity has been “drained” out of the network.
Refer to caption: extracted/5783407/figures/max_forward_steps_after_output.png

### 4.4 Unlayered backpropagation decreases latency

The final and equally important result that we report is that under asynchronous network processing, models trained with unlayered backpropagation have significantly lower latency than those trained with layer synchronization. Assuming that, without parallel processing (worst case), latency is a function of the number of spikes processed until a decision is made, figure 4 shows a distribution of the inference latencies across the entire test-set. It should be clear by now that this is because “the important spikes” in these models quickly reach the output layer, uninhibited by layer synchronization.

Figure: Figure 4: Latency per forward pass for all samples, in number of spikes until a decision in the output layer is reached. The Y-axis shows the relative frequency of recorded latencies.
Refer to caption: extracted/5783407/figures/latency.png

### 4.5 Unlayered backpropagation is resource-intensive

The current implementation of unlayered backpropagation is costly when training with a smaller $F$ and a larger dataset. When $F$ is halved, the time and memory requirements approximately double, consistent with the linear time and space complexity discussed in section 3.2.1. The detailed results can be found in section A.8.

## 5 Discussion

We have introduced a variation of backpropagation training that does not assume per layer synchronization, and we use it to train models. Under asynchronous processing, these models exceed the performance of conventionally trained models and deliver the anticipated energy and latency efficiency from event-driven execution. Our findings show that unless we revise our training methods (taking into account asynchronous processing dynamics), and designs choices of neuromorphic accelerators (layer synchronization primitives considered harmful), SNNs and brain-inspired computing will likely fail to deliver either in performance or efficiency.

This study has merely scratched the surface. A key realization (that confirms neurobiology) is that trainable dynamics related to network asynchrony allow neurons (and models overall) to use partial input for decision making. The extent and generality of this effect on network architecture/structure, neuron models, and data types deserves further exploration. Our results also encourage a dynamics driven design space exploration for neuromorphic processors with regard to event processing scheduling and granularity of synchronization primitives (or lack thereof), which is almost absent today. The third exploration dimension is undoubtedly the resource efficiency and scalability of unlayered backpropagation (or analogous training framework) to facilitate very big models and datasets.

## Appendix A Appendix / supplemental material

### A.1 Spiking neural networks

#### A.1.1 Incoming current

For every neuron layer, all the synaptic weights on its inbound connections are kept in a weight matrix $\mathbf{W}^{(l)}\in\mathbb{R}^{N^{(l)}\times N^{(l-1)}}$, where $N^{(0)}=$ number of input features $N_{\text{in}}$. Using these weight matrices, the total incoming current $x$ for a neuron $i$ in the next layer can be computed using:

$$ $x_{i}^{(l+1)}[t]=\sum^{N^{(l)}}_{j=1}W_{ij}^{(l+1)}s_{j}^{(l)}[t]$ (3) $$

where $s_{j}^{(l)}[t]$ is $1$ if neuron $j$ in layer $l$ has emitted a spike at time $t$, otherwise 0 0.

#### A.1.2 Membrane potential decay

The decay of the membrane potential is governed by a linear Ordinary Differential Equation (ODE). The analytical solution can be used to compute the decay:

$$ $u[t]=u[t-\Delta t]\cdot e^{-\frac{\Delta t}{\tau_{m}}}+x[t]$ (4) $$

where $\tau_{m}$ is the membrane time constant and $\Delta$t the elapsed time.

#### A.1.3 Threshold function

$$ $\Theta(u)=\begin{cases}1&\text{if $u>U_{\text{thr}}$}\\ 0&\text{otherwise}\end{cases}$ (5) $$

where $U_{\text{thr}}$ is the membrane potential threshold required for spiking.

### A.2 Event-driven state update rule for LIF neuron

When an input current is received at time $t$ by a neuron with a previous state $u[t_{0}]$ at some time $t_{0}\leq t$, an atomic set of computations is executed.
Start with computing the decayed membrane potential $u[t_{-}]=u[t_{0}]\cdot e^{\frac{-(t-t_{0})}{\tau_{m}}}$, then update the membrane potential $u[t_{+}]=u[t_{-}]+x[t]$ with the input current $x[t]$, and finally set $u[t]=0$ and emit a spike if $\Theta(u[t_{+}])=1$; otherwise $u[t]=u[t_{+}]$ without emitting a spike.

### A.3 Arctan surrogate gradient

$$ $\Theta(u)=\frac{1}{\pi}\text{arctan}(\pi u\frac{\alpha}{2})$ (6) $$

where $\alpha$ is a hyperparameter modifying the steepness of the function.

### A.4 Class prediction

Class prediction involves first calculating the output rates as follows:

$$ $c_{i}=\sum_{t\in T}s[i,t]$ (7) $$

where $c_{i}$ is the spike count for class $i$, $T$ are all the timesteps for which a forward pass occurred, and $s[i,t]$ is the output of the neuron representing class $i$ in the output layer at the end of the forward pass at time $t$.
These values are subsequently used as logits within a softmax function:

$$ $p_{i}=\frac{e^{c_{i}}}{\sum_{j=1}^{N_{C}}e^{c_{j}}}$ (8) $$

where $N_{C}$ is the total number of classes. The resulting probabilities are then used to compute the cross-entropy loss:

$$ $L=\sum_{i=1}^{N_{C}}y_{i}\text{log}(p_{i})$ (9) $$

where $y\in\{0,1\}^{N_{C}}$ is the target class in a one-hot encoded format.

### A.5 Parameters for the simulator

**Table 3: Overview of all current simulator parameters. If the text is italic, then the parameter was not used for the experiments in this paper.**
| Name | Value range | Description |
| --- | --- | --- |
| Forward group size | $\mathbb{N}_{>0}$ | 3.1.2 |
| Scheduling policy | RS/MS | 3.1.2 |
| Prioritize input | True/False | If input spikes are propagated before any spikes from inside the network. |
| Stop condition | On spiking done / On output | 3.1.2 |
| Forward steps after output | $\mathbb{N}_{\geq 0}$ | 3.1.2 (only for "On output" stop condition) |
| Refractory dropout | $[0.0,1.0]$ | 3.2.2 |
| Momentum noise | $\mathbb{R}_{\geq 0}$ | 3.1.2 (only for "MS" scheduling policy) |
| Membrane time constant | $\mathbb{R}_{>0}$ | 3.1 |
| Input spike dropout | $[0.0,1.0]$ | 3.2.2 |
| Network spike dropout | $[0.0,1.0]$ | The same as input spike dropout, but for spikes from inside the network, applied per forward step. |
| Membrane potential threshold | $\mathbb{R}_{>0}$ | 3.1 |
| Timestep size | $\mathbb{N}_{>0}$ | 3.1.1 |
| Synchronization threshold | $\mathbb{N}_{\geq 0}$ | Have neurons wait for a number of input currents (including barrier messages) before being allowed to fire. Can be set per neuron. |
| Emit barrier messages | True/False | Have neurons emit a barrier message if they have retrieved exactly the number of input currents to exceed the synchronization threshold, but not enough to exceed the membrane potential threshold. |

**Table 4: Simulator parameters for simulating neuromorphic processors**
| Synchronization type | Neuromorphic processors | Forward group size | Synchronization threshold [emit barrier messages] |
| --- | --- | --- | --- |
| Barrier-based / layer | Loihi [], SENECA [], POETS [] | # neu/layer | # neu/layer [True] |
| Timer-based / core | SENECA [], SpiNNaker [], TrueNorth [], POETS [] | # neu/core | # neu/core [True] |
| Asynchronous | Speck [], $\upmu$Brian [] | # neu/layer | 1 [False] |

### A.6 Details on the experimental setup

#### A.6.1 Datasets and network architectures

The Neuromorphic MNIST (N-MNIST) dataset captures the MNIST digits using a Dynamic Vision Sensor (DVS) camera. It presents minimal temporal structure []. It consists of 60000 training samples and 10000 test samples. Each sample spans approximately 300 ms, divided into three 100 ms camera sweeps over the same digit. Only the initial 100 ms segment of each sample is used in this study.

The Spiking Heidelberg Digits (SHD) dataset [] is composed of auditory recordings with significant temporal structure. It consists of 8156 training samples and 2264 test samples. Each sample includes recordings of 20 spoken digits transformed into spike sequences using a cochlear model, capturing the rich dynamics of auditory processing. The 700 cochlear model output channels are downsampled to 350 channels.

The DVS gesture dataset [] focuses on different hand and arm gestures recorded by a DVS camera. Like the SHD dataset, it has significant temporal structure. It focuses on 11 different hand and arm gestures recorded by a DVS camera. It consists of 1176 training samples and 288 test samples. The $128\times 128$ input frame is downsampled to a $32\times 32$ frame.

For all three datasets, input events are assigned a timestamp and an index. In the case of N-MNIST, the index corresponds to a position within a $34\times 34$ pixel frame, with each pixel having a binary polarity value (either 1 or 0), leading to a total of $34\times 34\times 2=2312$ distinct input indices. For SHD, the index denotes one of the 350 output channels. For DVS gesture, the $128\times 128$ input frame is downsampled to a $32\times 32$ frame. Like N-MNIST, each pixel has a binary polarity value, so in total this gives $32\times 32\times 2=2048$ input indices.

**Table 5: Network architecture and hyperparameters. The architecture is given as [input features] - [neurons in hidden layers $\times$ number of hidden layers] - [neurons in output layer].**
|  | N-MNIST | SHD | DVS gesture |
| --- | --- | --- | --- |
| Architecture | 2312-[64$\times$3]-10 | 350-[128$\times$3]-20 | 2048-[128$\times$3]-11 |
| Timestep size | 10 ms | 10 ms | 20 ms |
| Batch size | 256 | 32 | 32 |
| Epochs | 50 | 100 | 70 |
| Learning rate | 5e-4 | 7e-4 | 1e-4 |
| Membrane potential threshold $U_{\text{thr}}$ | 0.3 | 0.3 | 0.3 |
| Weight decay constant $\lambda_{W}$ | 1e-5 | 1e-4 | 1e-5 |
| Membrane time constant $\tau_{m}$ | 1 ms | 100 ms | 100 ms |
| Surrogate steepness constant $\alpha$ | 2 | 10 | 10 |
| Input spike dropout | 0.25 | 0.2 | 0.2 |
| Forward group size $F$ | 8 | 8 | 8 |
| Refractory dropout | 0.8 | 0.8 | 0.8 |
| Momentum noise $\lambda_{\text{MS}}$ | 1e-6 | 0.1 | 0.1 |

#### A.6.2 Performance metrics

To evaluate accuracy, output rates $c_{i}$ for each class $i$ are first calculated as outlined in equation 7. The predicted class corresponds to the one with the highest output rate. Accuracy is then quantified as the ratio of correctly predicted outputs to the total number of samples.

Spike density is computed using:

$$ $\text{density}=\frac{1}{N_{\text{samples}}\cdot N_{\text{neurons}}}\sum_{i=1}^ {N_{\text{samples}}}N_{\text{spikes}}[i]$ (10) $$

where $N_{\text{samples}}$ is the number of samples, $N_{\text{neurons}}$ is the number of neurons in the hidden layers, and $N_{\text{spikes}}[i]$ is the total number of spikes during inference of the sample $i$.

### A.7 Results on hyperparameters

Choosing a smaller $F$ (i.e., with a more asynchronous system), may improve accuracy, particularly benefiting models with mechanisms that rely on network asynchrony such as momentum scheduling. However, reducing $F$ also has its drawbacks. It significantly raises resource demands (discussed in section A.8), and there is a risk of reducing the effectiveness or even stalling the training process, as observed for SHD and DVS gesture.

Refractory dropout, can positively affect training outcomes. An explanation for this is that it increases the gradient flow by allowing more spiking activity. However, using full refractory dropout can also reduce performance, likely due to the inability to generalize to inference with refractoriness.

The momentum noise helps by introducing a slight stochastic element into the spike selection process, helping to avoid potential local minima that a purely deterministic selection method is prone to get stuck in. This seems to do little for N-MNIST, but for more complex datasets like SHD and DVS gesture it has a significant effect.

Figure: Figure 5: Results from inference with network asynchrony for different hyperparameters used during training. For SHD and DVS gesture with random scheduling, training $F$s smaller than 6 are not included due to the training failing to converge.
Refer to caption: extracted/5783407/figures/hyperparams.png

### A.8 Results on resource usage

The increase in processing time is less pronounced for the N-MNIST and DVS gesture datasets compared to the SHD dataset. This discrepancy could be due to computational optimizations that apply specifically to the N-MNIST and DVS gesture datasets (both being vision-based datasets).

**Table 6: Resource usage compared between layered and unlayered backpropagation during the second epoch of training on an NVIDIA Quadro RTX 5000.**
| Method | Time per epoch (s) | VRAM use (MB) |
| --- | --- | --- |
| N-MNIST |  |  |
| Layered | 14 | 210 |
| Unlayered RS, training $F=16$ | 19 | 412 |
| Unlayered RS, training $F=8$ | 24 | 556 |
| Unlayered RS, training $F=4$ | 37 | 986 |
| SHD |  |  |
| Layered | 38 | 214 |
| Unlayered RS, training $F=16$ | 118 | 2220 |
| Unlayered RS, training $F=8$ | 292 | 4844 |
| Unlayered RS, training $F=4$ | 681 | 10574 |
| DVS gesture |  |  |
| Layered | 120 | 244 |
| Unlayered RS, training $F=16$ | 150 | 3802 |
| Unlayered RS, training $F=8$ | 177 | 6984 |
| Unlayered RS, training $F=4$ | 245 | 13914 |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. **Input Encoding in Asynchronous Processing Framework**: How is event data encoded as input into the network within the asynchronous processing framework? Although lines 210-215 provide an explanation, I am still unclear about the form of the input data. Could the authors provide some equations for clarification? Perhaps an example with a single input event could be illustrated step-by-step, or a small code snippet could be submitted to demonstrate how a single input event is encoded and processed within the asynchronous framework.

2. **UNLAYERED BACKPROPAGATION**: In UNLAYERED BACKPROPAGATION, due to the dynamic nature of asynchronous processing, the backpropagation may occur across layers rather than layer-by-layer in a chained manner. Does this training approach lead to faster convergence during network training?
soundness: 4
strengths: 1. **Importance of the Research Content**: Although the original intention of SNN research was for asynchronous computation, most current SNN work is time-driven (i.e., layer-synchronized as mentioned in the paper). However, the asynchronous design of SNNs is crucial for applications in event-driven neuromorphic processors. This paper could contribute significantly to the SNN research community.

2. **Novelty of the Results**: The experiments presented in the paper showcase many intriguing advantages of the asynchronous design, such as enhanced neuronal activity while the overall network remains sparser (Figure 2); key information flowing freely during asynchronous inference (Figure 3); and reduced inference latency under asynchronous network processing (Figure 4). These results provide strong support for the future development of asynchronous computation.
summary: Current spike neural networks (SNNs) must compute and integrate all presynaptic currents from the previous layer before performing calculations for the neurons in the next layer. This dependence on layer synchronization deviates significantly from the original intention of asynchronous SNN design. Ideally, neurons should be able to emit and receive spike currents at any time and at any location within the network. In this paper, the authors address this issue and explore potential solutions. They propose a generalized approach for gradient training that allows for scheduling strategies using asynchronously processing neurons. Experiments demonstrate that this method can save energy and improve latency under asynchronous processing.

The authors highlight a key issue within the SNN research community: most SNNs are layer-synchronized, i.e., time-driven, rather than achieving the original goal of implementing asynchronous event-driven mechanisms. The authors discuss and conduct some experiments on asynchronous networks, and the results are intriguing.

At this stage, my rating is "6: Marginally above acceptance threshold." I would be very willing to raise my score if the authors could address and clarify the following issues.
weaknesses: 1. **Accuracy of Unlayered Method**: I reviewed the accuracy of the Unlayered method (Table 2), and it generally falls below that of the traditional Layered method. What is the network architecture of the Layered methods compared in Table 2? Can the performance of the Unlayered method be demonstrated using the network architecture from this work [1]? Because these networks are more used by SNN researchers, it would be more convincing to compare the methods in the same network structure.
2. **Network Sparsity and Energy Consumption**: The paper presents many instances of network sparsity; however, the neuronal activity has also increased, and these two factors have opposing effects on energy consumption. Which of these factors is dominant?   Could experiments be conducted to analyze the energy trade-off between the increased neuronal activity and network sparsity, further investigating the impact of each factor on energy consumption within the asynchronous framework? Table 3 shows the energy efficiency of asynchronous computation, but there is no significant reduction, and it does not even decrease by an order of magnitude. I find this outcome somewhat unsatisfactory. Could the authors provide some clarification?

[1]: Stsc-snn: Spatio-temporal synaptic connection with temporal convolution and attention for spiking neural networks, Frontiers in Neuroscience, 2022

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Check the Weaknesses.
soundness: 3
strengths: The paper is well-written and easy to follow. 

Highlighted an important aspect of training SNNs (asynchronous computation) which is generally ignored while training SNN models. 

The related work is very well explained, with their contributions and limitations. 

Through empirical results, the paper demonstrates the effectiveness of the proposed algorithm in terms of sparsity and accuracy.
summary: The paper highlights an important aspect of SNN, that is asynchronous computation of spikes, resulting in energy efficiency.  This work points out that, the training of SNN uses GPU and when deployed on asynchronous neuromorphic processes, the performance may be reduced during inference if asynchronous computation is adopted.

An generalized backpropagation algorithm is introduced which exploits the vectorized computation of the hardware and also allows asynchronous computation during inference without compromising the accuracy.
weaknesses: It's commendable that the authors have highlighted the asynchronous aspect of SNNs. Could the authors provide some comparative empirical results in terms of accuracy and sparsity with some previous works? 

For example, the state of the art directly trained SNN models. Such as, [1,2,3], I believe they belong to the class of synchronous computation of spikes. 

[1] Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting

[2] GLIF: A Unified Gated Leaky Integrate-and-Fire Neuron for Spiking Neural Networks

[3] Membrane Potential Batch Normalization for Spiking Neural Networks

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. I am not sure about Algorithm 1. Does $\boldsymbol s$ represent whether there is a spike or the timing of the spike? If it represents the timing, why it is an integer? If it represents whether there is a spike, how to determine the arrival order of input spikes, which is critical in asynchronous simulation?
2. In SelectSpikes() in Algorithm 1, the two scheduling policies seem not to consider the difference in input spike arrival times. How the asynchornization is achieved then?
3. Due to the above reasons, I am not sure whether Algorithm 1 reflects the real asynchronous property of SNNs. If not so, the comparisons in the experiments are not fair since layered training with async RS inference is not a practical situation.
4. How the classification result is determined by the network? What do 'on output', 'on spiking done', and 'forward steps after output' in Figure 2 and Figure 3 mean?
soundness: 3
strengths: This paper considers a critical problem for SNNs that they are expected to run in an asynchronous way, while most current SNN training strategies train them synchronously.
summary: This paper points out most current works train SNNs in a synchronous way, while SNNs should be running in an asynchronous environment, which leads to a gap. Then this paper quantifies this gap and develops a training method better suited for asynchronous inference. Experiments show the effectiveness of this method in accuracy, inference speed, and energy consumption.
weaknesses: 1. See questions.
2. There are some word and grammar mistakes and some figures should be improved. For example, 'compex' in line 363 should be 'complex', 'To what extent this is the case is not explored in this work.' in line 334 is incoherent, the legends in Figure 4 stretches over two subfigures.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: I am not sure that I really understand the methods in this paper. Thus, I have the following questions. Please feel free to clarify my wrong understanding. 

What is the role of `c` in Algorithm 1? It is only used as the output.

I am not sure if the key characteristic of the proposed asynchronous method is "selecting spikes". More specifically, suppose a layer has `N` inputs, the previous synchronous simulation method uses the spike tensor `S` with `N` elements as inputs. The proposed method will firstly apply a scheduling strategy to create a `mask` to select elements (spikes) in `S`, and use `S[mask]` as inputs. If so, it seems that this characteristic is trying to mimic the asynchronous spike firing in chips, which do not have a global clock. The role of the global clock is similar to "time-step" in SNNs simulated by the synchronous method. Then, what is the relation between the asynchronous spike firing in chips and the randomly selecting in simulation? I guess (note that I am not the expert of neuromorphic hardware) that the events (spikes) are processed (chronologically) orderly in chips, and this behavior is more similar to the synchronous simulation with a large number of time-steps. While the randomly selecting is disordered.
soundness: 3
strengths: As we know, most (or all?) existing software frameworks for deep SNNs use a (layer-wise) synchronous simulation method. The idea of the asynchronous simulation in this paper is very interesting. The source codes in the supplementary materials indicate that the authors have developed an asynchronous primary software framework, which will benefit the SNN community.
summary: This paper proposes an asynchronous simulation method for deep SNNs. The network structure, vectorized simulation algorithm, backward methods, and experiment results on some simple datasets are provided.
weaknesses: Unfortunately, this paper is written unclearly. The difference between synchronous and asynchronous simulation methods is not explained clearly. It is better to illustrate the difference by a figure, i.e., an example to show thow the synchronous/asynchronous method simulates an SNN.


The advantages of using the asynchronous simulation method are not shown. I guess that this simulation method is more similar to how the asynchronous neuromorphic chip works. However, I believe that not all of SNN researchers are familiar to the hardware, and it is hard for them to understand the advantages of asynchronous simulation without the knowledge about neuromorphic chips I suggest that the authors add more background knowledge about asynchronous neuromorphic chips (such as uBrain and Speck), and why the existing simulation methods are not so good for them such as the difference between synchronous simluation and asynchronous on-chip inference.

Meta-review: This paper examines the possibility of using asynchronous processing for spiking neural networks (SNNs). In brief, the authors are interested in eliminating the constraint that is often imposed in multi-layer SNNs of having every cell on a layer accumulate its inputs and spike (or not) at some clocked time. The authors are interested in relaxing this constraint as it could potentially reduce the latency for inference and increase the energy efficiency. There are several challenges with asynchronous processing, though, one of which is that standard backprop is not well suited to training in the asynchronous regime. The authors propose a solution for this (a form of backrop adapted for asynchronous events) and they explore various scheduling strategies for asynchronous SNNs. They claim that these strategies improve the accuracy of asynchronous networks, putting them on par or better than synchronous networks, and that they reduce the energy footprint of inference.

The strengths of this paper are that it is exploring an important, but under-examined issue in SNNs, and provides some novel solutions. The weaknesses are that the clarity of the paper could be much improved, and the actual improvements provided by the solutions on offer are fairly limited - the authors find that their asynchronous approach scales poorly in terms of both accuracy and efficiency (e.g. it does not work on VGG models). As such, the claims on performance and efficiency improvements only apply in limited, relatively toy models, and do not appear to be relevant to the ML community more broadly. This paper is really best framed as an initial exploration of the limitations of asynchronous processing. In fairness, the authors try to be clear about this with their choice of title. However, arguably, the high bar for acceptance at ICLR requires more than a well-executed study of limitations in a class of model.

Given these considerations, a decision of reject was reached.

Decision: Reject