Exploring the Limitations of Layer Synchronization in Spiking Neural Networks
Authors: Roel Koopman
Amirreza Yousefzadeh
Mahyar Shahsavari
Guangzhi Tang
Manolis Sifalakis
Venue: Accepted by TMLR ()

Abstract: Neural-network processing in machine learning applications relies on layer synchronization. This is practiced even in artificial Spiking Neural Networks (SNNs), which are touted as consistent with neurobiology, in spite of processing in the brain being in fact asynchronous. A truly asynchronous system however would allow all neurons to evaluate concurrently their threshold and emit spikes upon receiving any presynaptic current. Omitting layer synchronization is potentially beneficial, for latency and energy efficiency, but asynchronous execution of models previously trained with layer synchronization may entail a mismatch in network dynamics and performance. We present and quantify this problem, and show that models trained with layer synchronization either perform poorly in absence of the synchronization, or fail to benefit from any energy and latency reduction, when such a mechanism is in place. We then explore a potential solution direction, based on a generalization of backpropagation-based training that integrates knowledge about an asynchronous execution scheduling strategy, for learning models suitable for asynchronous processing. We experiment with 2 asynchronous neuron execution scheduling strategies in datasets that encode spatial and temporal information, and we show the potential of asynchronous processing to use less spikes (up to 50\%), complete inference faster (up to 2x), and achieve competitive or even better accuracy (up to $\sim$10\% higher). Our exploration affirms that asynchronous event-based AI processing can be indeed more efficient, but we need to rethink how we train our SNN models to benefit from it. (Source code available at: \url{https://github.com/RoelMK/asynctorch})

Published in Transactions on Machine Learning Research (09/2025)
Exploring the Limitations of Layer Synchronization in Spiking Neural Networks
Roel Koopman
voel.koopman@cui..tl
Machine Leorning Group, Centrum Wiskunde & Informatica (CWT)
Amsterdam, The Netherlands
Amirreza Yousefzadch
a. Srgoussfaadehhhuuuueeeeee.
Facully of Electrical Engineering, Mathematics & Computer Science, University of Twente Enschede, The Netherlands
Mahyar Shahsavari
Donders Centre for Cognition, Radboud University Nijmegen, The Netherlands
Guangzhi Tang
guangzhi. haamgpgh..aon@gaaassssaaaneeeiir
Deportment of Advanced Computing Sciences, Maastricht University Muastrichl, The Netherlands
Manolis Sifalakis
Innatera Nonosystems BV Rijswijk, The Netherlands
Reviewed OpenReview: /ttps://openreviiee net/ forum? id=mfmAVutMII
Abstract
Neural-network processing in machine learning applications relies on layer synchronization. This is practiced even in artificial Spiking Neural Networks (SNNs), which are touted as consistent with ncurobiology, in spite of processing in the brain being fact asynchronous. truly asynchronous system however would allow neurons to evaluate concurrently their threshold and emit spikes upon receiving any presynaptic current. Omitting layer synchronization is potentially beneficial, for latency and energy efficiency, but asynchronous execution of models previously trained with layer synchronization may entail mismatch in network dynamics and performance. We present and quantify this problem, and show that models trained with layer synchronization either perform poorly in absence of the synchro- nization, or fail to benefit from any energy and latency reduction, when such a mechanism in place. We then explore a potential solution direction, based on generalization of backproopaaation-basdd training that integrates knowledge about an asynchronous execution scheduling strategy, for learning models suitable for asynchronous processing. We experiment with 2 asynchronous neuron execution scheduling strategies in datasets that encode spatial and temporal information, and we show the potential of asynchronous processing to use less spikes (up to 50%), complete inference faster (up to 2x), and achieve competitive or even bet- ler accuracy (up to ~00% higher). Our exploration affirms that asynchronous event-based AT processing can be indeed more efficient, but we need to rethink how we train our SNN models to benefit from it. (Source code available at: /ttpps//giite http///gtttuu.... httpghhhw..c//oollol//aaoooooooo
1 Introduction
Artificial Neural Networks (ANNs) are the foundation behind many of the recently successful developments in AI, such in computer vision (Szegedy et al. 2017 Voulodimos et. al. 2018) and natural language processing

---

Published in Transactions on Machine Learning Research (09/2025)
Vaswani et al. 2017 Brown et al. 2020 To match the complexity of the ever more demanding tasks, works have grown in size, with ad advanced large language models having billions of parameters (Zhao et al. 2024h. With this, the power consumption has exploded (Luccioni et al. 2023), limiting the deployment to large data centers. In an effort to learn from our brain's superior power efficiency, and motivated by neuroscience research, SNNs (Maass) 1997] bolster as an alternative. They use discrete binary or graded spikes (events) for communication, are suited for processing sparse features (He et al., 2020), and when combined with event-based asyuchronous processing are assumed to reduce latency and improve energy efficiency (Zeki) 2015). Sparsity can synaptic operations, thus lowering energy consumption, and asynchronous operation enables concurrent evaluation of all neurons anywhere in the network purely event-driven, leading to low latency.
In an event-based processing system (figure IA), a spike or event an SNN (or non-zero activation in an ANN) can independently and locally trigger some downstream computationn (that, may may not have global effect). Event-based processing manifests essen- tially by comparison to vector-based processing, which groups (activation) computations in batches.
Forward Inference in one timestep
(A)
By asynchronous processing we understand the fact that spikes (or activations) can be processed at any time (or in any order), and also anywhere in the net- work, independently of any other spikes and unhindered by artificial conditions that delay their immediate prop- agation (such as waiting for all other neurons in the same layer to integrate their currents or evaluate their thresholds). Typically, for a large network of ncurons only the approximate timing of events plays a role in asynchrony (London et al., 2010),, and the resulting order is in often the main information-carrying signal (according to neuroscience literature that underpins order codings) (Thorpe et al. 2001). Approximate or relative timing makes asynchronous processing vi- able in digital accelerators even though they discretize time. Asynchronous processing, as perceived in this work, illustrated (and visually represented through an analogy) in figure [
(B)
(C)
Layer-synchronous (higher latency)
Asynchronous No synchronization barriers
Figure 1: (A) Event-bascd processing. (B) Event- based processing with per-layer synchronization: la- tency a function of the events in the system well the number of layers (synchronization bar- riers). (C) Asynchronous (end-to-end) event-based processing: event processing does not encounter syn- chronization barriers, and so inference latency is determined solely by the subset of the total mumber of events processed until a decision made at the output layer. Color codes indicate at which layer events were fired. The x-axis of the plots represents time/order order that events were processed. The vertical planes in (B) represent layor-synchronization barriers (also indicating bottlenecks in network routing and memory 1/O). The difference between (B) and (C) can be understood through an analogy: bikes racing in a circuit versus on a public road with a sequence of traffic lights.
It needs to be meticulously clarified that asynchronous processing within the network, and at the hardware processing system level (which is the focus of this work), is distinct from what is customarily referred to as asynchrony at the model input. Specifically, at the model level RNNs, we colloquially refer to timesteps the discretization ion of continuous external stimuli into sequence of consecutive steps for input to the network model. Asynchrony in the latter sense (as commonly seen in SNN literature) refers to the fact that the stimulus is sampled across the input dimension and can be presented to the model not in a single step, but rather distributed i..d. across several such discrete timesteps. To limit ambiguity, call this
2

---

Published in Transactions on Machine Learning Research (09/2025)
input asynchrony and we distinguish it from (in-)network asynchrong which we study in this paper, namely the relative propagation of spatial signals inside the network within individual timesteps. Moreover, with imput asynchrony, by common convention, processing of the stimulus in one timestep inside the network has been assumed until now to be as taking place "atomically" through a deterministic process and sequence: breadth-first across neurons one-layer-after-The-olll that spikes (activation state) from two distinct timestops, but also between layers, do not interfere with one another. Forward inference in two timesteps interacts only through "neuron state caching" in each layer, by means of (not resetting) the membrane voltages (hidden state in RNNs). But, from the viewpoint of this work, this type of processing inside the network is still synchronous (despite input asynchrony) as depicted in figure BB. Moreover, we argue this per-layer synchronization dulls the dynamics of the stimulus (data-generating process); is not quite what happens in the brain (SNNs); and neither is the type of processing neuromorphic processors are primarily aimed for.
This convention serves the purpose
2015) has been connected to energy and latency efficiency because information dynamics can express freely and propagate signals promptly through neuron clusters triggering early outputs and leveraging fast decisions even before other neuron clusters of slower dynamics have finished processing the input stimulus (and generated downstream activations). Specifically, there is no known process in the brain that consistently and uniformly forces neurons in one layer (or column) to finish their computations before the neurons in another downstream layer assume processing of their input currents. On the contrary, the more intensely the stimulus manifests in some regions (subgroups of neurons), the faster dynamics are reinforced, leading to a non- non-uniform (unaligned) and non-synchronized propagation of spikes across regions. These dynamics are also likely modulating and/or are modulated along certain pathways by heterogencity in axonal or synaptic delays and different state decay procossos across ncurons.
Asynchronous processing in the brain Zeki
In other words, through asynchronous processing, in-network expressed dynamics should lead to computation economy. The work in this paper puts this hypothesis to test. It presents a first of its kind study of how current synchronous training impacts such an asynchronons modus operandi, how can we possibly train good performing robust models for asynchronous processing, and if energy and latency elficiency stein from such a modus operandi.
Using a simulation environment that implements the concept of network asynchrony and can emulate the processing behavior of number of neuromorphic accelerators, we provide quantitative results on benchmark datasets (with different ppatio-tempora information content) and network topologies of two or more hidden layers, which show performance degradation and latency /energy inefficiency resulting from changes in model dynamics when trained with layer synchronization and later deployed for asynchronous inference (processing). Next, we explore a potentially promising solution by proposing a generalization of gradient. (backpropagattoo--aaee))
training, that can be parameterized with various neuron execution scheduling strategies for asynchronous processing, and vectorization abilities present in various neuromorphic processors. We show that using this training method, it is possible not only to recover the compromised accuracy, but also to fulfil the expectation of saving energy and improving latency under asynchronous procossing (when compared to the conventional breadth-first processing in GPUs). This work opens a path for design space explorations aimed to bridge the efficiency gap between neural network model training and asynchronous processor design.
2 Related work
Conventional highly parallel ANN compute accelerators, such Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), which are primarily optimized for dense vectorized matrix operations, face inherent challenges in exploiting unstructured and temporal sparsity for improving their energy efficiency.
'Notably few approaches, such as in Zhu et al. 2202 quote as asynchrony the manipulation through learning of the shape and distribution of stimulus (imput spikes) across timesteps with the goal of improving model performance (accuracy), assuming however execution with layerssynctronizel processing (event-bused or vectorized alike).
AAs will be seen later, also need notion of discretization of (processing) steps through the network, which can be variable and decoupled from the number of layers. We refer to these steps as forward steps. Thus, while timestops discretize processing (in complete forward model evaluations) across stimulus presentations, forward steps discretize processing within timestep from the onset. of (one timestep's) s) stimulus presentation until the output neurons conclude the model evalution for that timestep.
3

---

Published in Transactions on Machine Learning Research (09/2025)
Targeting the common practice of executing ANNs with per-layer synchronization bottlenecks has left them with poor support for asynchronous processing (for improving latency or energy consumption). At best, they parallelize processing within layer and/or pipeline processing across layers. This leaves an exploration space for neuromorphic processors that try to excel in handling the event-driven nature of SNNs and leverage asynchronous concurrent processing, offering efficiency advantages in various tasks.
However, the training of SNNs today more and more often relies conveniently, for performance, on conventional end-to-end ANN training methods (Dampfhoffer et al. 2023), based on synchronizing computations per-layer rather than event-driven per neuron (Guo et al. 2023a). First, the total of all presynaptic currents from the precoding layer must be computed and integrated before postsynaptic neurons in the current layer update their state and evaluate their activation function (i.e. emit new spikes). For consistency, neuron evaluation in one layer must thus complete and synchronize before proceeding to evaluate neurons of the next layer.
Some event-driven nouromorphic processors, such as μBrain (Stuijt et al. 2021), Speck (Caccavella et al., 2023), and Pulsar's (Inn 2024] SNN sub-system, are fully event-driven and lack any layer synchronization mechanism. They are capable of fully asynchronous processing but this makes them notoriously difficult to train out-of-the-bot box with methods for ANNs that rely on per-layer synchronization (e.g. (Dampfhoffer et al., 2023)). Speck developers propose to train their models with hardware in-the-loop (Liu et al. 2024) to reduce the mismatch between the training algorithms and the inference hardware. However, this met does not provide a general solution for training asynchronous neural networks. Others, like SpiNNaker (Furber et al. 2014) and TrueNorth (Akopyan et al. 2015), use limer-based synchronization, or Loihi (Davies et al.) 2018) and Seneca. (Tang et al.) (023) use barrier synchronization between layers, in order to ensure consistency and alignment with models trained in software (with layer synchronization). As we will show, this entails efficiency penalty.
Functionally, the most suitable type of model for end-to-end asynchronous processing is probably rate- coded SNN models, whereby neurons can integrate state and comminicate independently from any other neuron. These models are trained like ANNs (Zenke & Vogels 2021 Lin et al. 2018 Diehl et al. 2015) or converted from pre-trained ANNs (Rueckauer et al.) 2017 Kim al. 2020). Therefore, to be accurate under asynchronous processing, it is required to run the inference for a long time, reducing the latency and energy benefit of using SNNs (Sengupta et al. 2019
Alternative to rate-coding models are temporal-coding models, with time-to-first spike (TTFS) Park et al.
2020: Srivatsa et al.) 2020 Kheradpisheh & Masquelier 2020: Comşa et al. (022) or order encoding (Bonilla ct al. 2022). They are very sparse (hence energy efficient) but very cumbersome and claborate to convert
Trom ANN (Pannkras al. 2013: Srivatsa el al., 2020) or train directly (Kheradpisheh & Masquelier 2020
Comşa et al. 2022), less tolerant to noise (Comga et al. 2022), ; and their exccution so far, while event-based
requires some form of synchronization or a reference time. Efficiency is thus only attributable to the reduced
number of spikes, all of which need to be evaluated in order before a decision is reached. Other alternative encodings for event-based processing of SNNs include phase-coding (Kim et al.) 2018) and burst coding (Park] ct al. 2019), which are, however, no more cconomical than rate coding and have not been shown, to our Snowledge to attain competitive performance.
The approach presented in this paper is, in fact, unique in enabling the trainability of models for event-based asynchronous execution, providing efficiency from processing only a subset of spikes, and delivering consistent performance and lolerance to noise. Also relevant. to the works in this paper are SparseProp (Engelken) 2024
and EventProp (Wunderlich & Pehle 2021b) efficient weett simulation and training, respectively. EventProp (Wunderlich & Pehle 2021b) for training event-based models for asynchronous processing (and fully compatible with the work in this paper), but it has not been shown how its complexity scales beyond 1-2 hidden layers. SparseProp (Engelken) 2024] proposes an efficient neuron execution scheduling strategy for asynchronous processing in software simulation. An effective hardware implementation of this scheduling strategy can be found in (Monti et al., 2017), which has inspired the herein proposed "momentum schedule". Moreover, we advance this research by demonstrating how to train SNN models using this event scheduler.
b) is potentially more economical than discrete-time backpropagation
the fact that neurons evaluate their threshold immediately upon receiving a synaptic input (spike) from each fan-in synapse independently (as opposed to aggregating all synaptic inputs and the evaluating the threshold
4

---

Published in Transactions on Machine Learning Research (09/2025)
once where they can often cancel each other out); the relaxation of forcing synchronization of processing at every layer; and the adoption of dynamic scheduling of neuron execution across the entire network which continuously adapts (stochastically) to flow dynamics of stimulus /actuulttactiiatiinns or neuron membrane states. The last last two "ingredients" allow neurons deep inside the network to fire before processing at early layers has finished (if the flow dynamics encourage it).
3 Methods for simulating and training of asynchronous SNNs
In this section, we detail our methodology. First, we detail (section 3.1 how simulate the unconstrained behavior of various (digital) neuromorphic event-based accelerators capable of asynchronous SNN processing inside the neural network (i.e., implement networkk usymchrony), by leveraging the following principles: (a) allow neurons to evaluate their thresholds independently for every received synaptic current; (b) abolish the constraint to synchronize activations at each layer; (c) adopt dynamic scheduling of neuron execution across the entire network that contimuously adapts to the flow dynamics of activations/stimmuu sstimulus or relative neuron states; and (d) support limited-length vectorization. (Details about the accelerators we considered, and the parameterizability of the simulator to support them, are given in the tables in appendix A.5
Then in section 3.2 we explain how these principles can be applied to backpropagation training (leading to an approach we call unlayered backpropagation) to prepare good performing, robust and efficient asynchronous models.
3.1 Simulating asynchronous SNNs
The SNNs used in this work consist of L layers of Leaky Integrate-and-Fire (LIF) neurons (He et al. 2020
with each layer for 1 < I. having N() neurons and being fully-connected to the next layer 1 1+ 1, except for the output layer. Each input feature is connected to all neurons in the first layer. Every connection has a synaplic weight. Using these weights, we can compute the incoming current x per neuron resulting from spikes, as per equation in appendix A.1
Each LIF neuron has a membrane potential exponentially decaying over time basod on some membrane time constant Tm- We use the analytical solution (see equation 4 in appendix [A.1] to compute the decay. By keeping track of the membrane potential u(t) and the elapsed time At since time t, it is possible to precisely calculate the membrane potential for L + A.. Therefore, computations are required only when x[t + Ar) > 0.
To determine if a neuron spikes, a threshold function θ checks if the membrane potential exceeds a threshold Uur, following equation in appondix A.1 If @(u) = 1, the membranc potential is hard reset to 0. Soft reset is another option (Guo et al. 2022 as well as refractoriness (Sanaullah al. 2023). However, for simplicity here we only experiment with hard reset.
3.1.1 Asynchronous inference with event-driven state updates
We can vectorize the computations introduced in the previous section per-layer basis. This is herein referred to as the 'layered' inference approach, which implies layer synchronization, since all neurons of one layer need to evaluate their state, before any neuron in down-stream layer does the same. An alternative to this situation is an event-driven approach, where the computations for state updates are applied in response to new spike arrivals. If additionally spikes can be emitted (threshold evaluation) at any point time (e.g response to any individual synaptic current), by any neuron the network, affecting the states of postsynaptic neurons independently of others, then (we assume) this approach can achieve a true representation of network asynchrony.
When using network asynchrony, the network dynamics evolve with each spike. Any single spike can generate multiple currents rewm, linked to a specific stimuli at limestep 4, determined by the initiating input activity. Let K2 represent the atomic computation that executed at one neuron in the network, in response to input synaptic currents and triggered by stimulus presented at the network at timestep t, and which will update the state of the neuron and evaluate its threshold function (activation). The exact computations for a LIF neuron can be found in section
5

---

Published in Transactions on Machine Learning Research (09/2025)
Membrane potental
Fraction of input received by
Layer synchronized processing
Asynchronous processing
Membrone potentaly
Z--1
Fraction of input received by
Figure 2: Layer synchronization ensures that activations from all neurons (2:) in layer are available simultancously to any post-synaptic neuron (y) in the next layer. Hence they are integrated together and the threshold of the post-synaptic neuron is evaluated only once (see top-left graph). As a result the currents arriving from different synapses often cancel each other out given opposite sign weights, resulting in fewer spikes and a different firing pattern than when each synaptic current is processed independently (see bottom-left graph), where each input more likely to trigger a spike, is the case in asynchronous processing. Thus, in SNNs, layer synchronization leads to the volume of spikes being reduced and the actívation flow dynamics being dulled. This is not issue (at least not grave) in ANNs because activations communicate relative magniludes.
Notice that whether K, executes in response input current from each synapse independently, or in response to the aggregate synaptic current from all fan-in synapses collectively, leads to different activation behavior/pattene and hence neuron dynamics (see figure 2 Also, the relative order in which K₁ executes among neurons across the entire network (as opposed to within just one layer due to synchronization) affects the output of the network due to the non-linearity entailed in Kr. Here, for simplicity, it is assumed that the fan-out currents resulting from a spike generation are propagated as a group at once (e.g. no heterogeneous propagation delays exist.). Because of network asynchrony however the recipient neurons will not necessarily process them simultancously or in fixed order, but rather opportunistically as dictated by (dynamic) scheduling algorithm. This dynamic scheduling captures the asynchronous hardware behaviour/capaaillie (more about this is given in section 3.1.2). Because of these assumptions, analyzing the spike propagation order gives the same insights as analyzing the execution order of K, (across neurons in the network).
Overall, unlike the layered approach, in this opduunoperandi because of the per-input current evaluation of the firing threshold neurons can fire more than once at the same timestep t (in response to the same input stimulus). We can prevent this by inducing a ncuron to enter a refractory state for the remainder of time In this state, the neuron keeps its membrane potential at 0. This makes the model causally simpler, more tractable, and more "economie' when energy consumption is coupled to spike communication. These aspects may also give explicit retrospective justification for the neurophysically observed refractoriness (Berry & Meister 1997). However, the observations we report in the results hold true both when we allowed neurons in the network to enter refractoriness after firing a spike, as well as when there is no refractoriness.
Two resolutions of time We apply input framing as typically done for inputs from a DVS sensor, accumulating timestamped events in discrete time-bins, and providing these frames inputs to the nor

---

Published in Transactions on Machine Learning Research (09/2025)
Algorithm 1 Vectorized network asynchrony forward pass
Input: input spikes Sin E NNi,, previous forward pass time IR, current forward pass time E R, neuron state u e RR at time to, forward group size F E Nyo
Output:
spike count N
At 1
u Decron At)
InputLayerForvar((ss.)
0% ON
Vector with zeros of length
while (Sum(x) * or (s) + #0) and -EarlyStop(s) do Snew, NeuronForward(xx.u
S s t Snow
> Enqueue new spikes Update spike count
Snew
Spelected + SelectSpikes F)
Syelected
Dequeue selected spikes
X + BettorrLLayersFovvarrssddd..
end while
in discrete timesteps. The limestep size refers to the time-interval between the limestamps of the first (or last) events in the time-bins of two consecutive timesteps. Timesteps correspond one concept of timing (resolution) with same semantics in RNNs, and which we refer to here macro-time. At cach timestep, input event frames initiate network activity, in so-called forword pass, making all new spikes contingent on the timestep.
The order of spikes propagated though the network during a timestep establishes another notion of "time" in a forward pass (we call "micro-lime"), which is discrelized in so-called forward sleps. A forward step is associated with the processing of one or a group of events (in case of vectorization), and the number of forward steps measures time complete inference. One can realize that in case of layered inference the number of forward steps is fixed and equals the number of layers, but this is not the case for asynchronous processing.
3.1.2 Vectorized network asynchrony
The event-driven (neuron state) update rules for network asynchrony introduced in the previous section can be vectorized by selecting a number of spikes for processing them simultaneously. This allows us to consider the entire spectrum of possibilities between per-layer synchronization at one extreme (by assuming a vector size equal to a layer size), and "complete" asynchrony at the other extreme, where each spike event is processed ent irely independently of all others. Additionally, veclorization makes acceleration possible by exploiting the parallelization features and vector pipelines of accelerators, where these models execute, leading to pragmatic simulation of network asynchrony.
During the simulation, the states of all the N N Dt NO neurons in the network are stored in vectors. Vector x E RN tracks the computed imput currents for the neurons, u E g the membrane potential of the neurons, 8 E NN the emitted spikes awaiting processing, and с E NW which neurons have spiked in the current forward pass. For any of those vectors, the indices from E (1)k) to Ei (K) represent the values for the neurons layer 1, for < < L. where N(O) 0.
Algorithm | outlines the processing during forward pass (propagation of the spikes of an input frame through the network). Input spikes available in Sin E N.. (not to be confused with s) where Ni is the number of input features. Each forward pass consists of forward steps (the code within the while loop), which update the state of a set of neurons based on the spikes selected for propagation. A complete list of parameters can be found in section A.5
The forward pass ends either when all spike activations have boen processed ('On spiking done' stop condition), or when any neuron the output layer has spiked one or more (default one) forward steps ago ('On output" stop condition). The latter is checked in the EarlyStop function.
7

---

Published in Transactions on Machine Learning Research (09/2025)
The SelectSpikes function defines how to select subset of the emitted spikes for propagation. The function selects F spikes at time, or less if there are less than F remaining spikes ready to be propagated. The method of spike selection is determined by a scheduling policy.
For the experiments here two policies are exercised. The intent is that different scheduling strategies can be tested for their effectiveness in capturing tractably asynchronous processing dynamics. The first, Random Scheduling (RS), randomly picks spikes from the entire network. The second, Momentum Scheduling (MS), prioritizes spikes from neurons based their membrane potential upon exceeding their threshold.
The neuron model-specific (LIF) behavior is expressed in the NeuronDecay and NeuronForward functions. This entails the computations for state updates (see section 3.1 for all neurons in the network (NeuronDecay) or for only those neurons receiving inpul currents in the forward step (NeuronForward), with added restriction that spiking is only allowed once per forward pass.
The network architecture is defined by the Fopaaaay and NetworkLayyrsForwaad functions. These functions compute the values of synaptic currents from spikes. This follows from equation
Imitating neuromorphic accelerator hardware Neuromorphic processors come in a number of variations, but most of them have a template architecture that interconnects many tiny processing cores with each other. This design enables a scalable architecture that supports fully distributed memory and compute systems. In this template, each processing core has its own small synchronous exccution domain, but the cores operate asynchronously among each other. Archilectures such as those cited in the Sect.oo 2 all follow this template. Our asynchronous processing simulation environment captures the intrinsics of the synchronous domain (e.g. in the forward group size F) while simulating the asynchronous interactions among them (e.g. scheduling policy) and can be also configured to reproduce more specialized intrinsic behaviors of many of those processors (table 5
3.2 Training asynchronous SNNs
We use backpropagation to train the model weights, and specifically when stimulous is presented across multiple timesteps (forward passes), such as for sequential or temporal data, then this is Backpropagation Through Time (BPTT). Following common practice to address the non-differentiabiiiyy of the threshold function, the surrogate gradient method is used (Zenke & Vogell 2021). Specifically the arctan function (Fang et al. 2021) (see section function.
A.3 for details) provides a continuous and smooth approximation of the threshold
Class prediction is based on the softmaxed membrane potentials (for CIFAR-10) or spike counts over time (for the other datasets) of the neurons in the output. layer, as described in section A.4 The loss is minimized using the Adam optimizer, with 3 0.9 and 32 0.999.
3.2.1 Unlayered backpropagation
We refer to "conventional" SNN training with per-layer synchronization using backpropagation Dampfhoffer
et al. 2023) as "layered backpropagation..
The vectorized network asynchronous processing approach differentiable as well, and can be used with backpropagation. We refer to this as "unlayered backpropagation'. Combined with BPTT unrolling, this method implies a two level unrolling. At the outer level, unrolling is based on discretization of time and thus the input across (as usually fixed number of) timesteps. At the inner level, unrolling is based the F-grouping (and vectorized processing) of spikes in forward steps, subject to the scheduling policy, applied to the emitted spikes by neurons; from the beginning of the current timestep until the output is read.
Note, that the dependence on the scheduling policy, applied on a variable mumber of activations (across timesteps and data samples), in F-sized groups the fundamental difference from layered backpropagation, and leads to different gradient state being built up in the computation graph. This state now captures the dynamics of asynchronous processing. For single backward pass in timestep of BPTT, which is applied in
8

---

Published in Transactions on Machine Learning Research (09/2025)
similar way as the layered backpropagation equivalent, it is given that:
aL.
aLC
N
dc 3s:
(1)
aw
de,
3s; aw
where & is the time(step) of the forward pass, W refers to the trainable weights, L is the loss, a is the spike count at the end of the forward pass, and s; is the emitted spikes vector at the end of the forward step i. The overall gradient (state) depends on the total number of forward steps N, in the forward pass and the spikes processed in each forward step. The number of steps scales linearly with the number of spikes processed in the forward pass, and it is important to understand that due to asynchronous processing the number of spikes processed until the evaluation of the loss function may be (well) less than the total number of spikes emitted throughout the forward pass. Since for every forward step, the computations repeated, the time complexity scales lincarly with the number of forward steps, O(N)). The same applics to the space complexity.
During the backward pass, the spikes in s, that are not selected for processing can skip the computation f for that step:
as: (f U4-1, W) OW
Ds:-1:not selected
(2)
OW
aW
Skip connections have been explored in deep ANNs and identified contributor to the stability of the training process (Orhan & Pitkow] 2018). This may apply to the skip in unlayered backpropagation, as well. The extent to which this is the case remains unexplored in this work.
Note that under this generalization, layered backpropagation corresponds to F-group the size of a layer, populated with a static round- Robii execution schedule following neuron index order within a layer, and re-resct across consocutive layers).
Finally, to leverage some visual insight into what makes unlayered backpropagation different from canonical (layered) backpropagation forward and backward paths in the two cases (the forward path in figure depicts essentially the control-flow of our simulator). The most notable difference is that because of the way activation flow is generated in a variable number of forward steps (which, unless F-group has equal size to the number of neurons in a layer, is often more than the number of layers in the network), the unrolled compute graph the backward pass will be different at every timestep, which contrasts with the fixed lattice-liss like structure of the compute graph in canonical backpropagation. This results in (fundamentally) different gradient flow and learning different. model (more robust for asynchronous inference, our experiments will show).
consequence of the ramifications we have proposed, figure 3 illustrates the
3.2.2 Regularization techniques
During training, we use regularization to prevent overfitting and/or enhance model generalization. These techniques are not used during inference.
Input spike dropout. Randomly omits input spikes with a given probability. The dccision to drop cach spike is independent according to Bernoulli distribution.
Weight regularization. Adds weight decay to the loss function: Lxw (W) = L(W) + Aw W where Aw is the regularization coefficient, L is the loss given the weights, and W are all the weights
Refractory dropout. With some probability, do not apply the refractory effect, allowing a neuron to fire again within the same forward pass.
Momentum noise. When using momentum scheduling, noise sampled from (O, and multiplied by some constant AMs is added to the recorded membrane potential while selecting spikes.
9

---

Published in Transactions on Machine Learning Research (09/2025)
Backward
layer syne
syme
ndected
Forward osyne
U
U
Paleated
Salected
To
Ko
Day D Ungg
Sabeted
Figure& (a) The forward pass with layer synchronization, where layer 0 S j < receives spike input sy creating currents that integrated into membrane potentials új which carried over to the next timestep, and potentially emitting new spikes to the next layer if exceeding the spike threshold. (b) The backward pass of single timestep for layered backpropagation. The structure of the computational graph is fixed across timesteps for a network with constant depth. (c) The forward pass for unlayered backpropagation, where s² selected represents a selection of spikes from anywhere in the network propagated during the ith forward step, and U denotes the membrane potentials of all neurons in the network. The membrane potentials are carried over the next timestep at the end of the forward pass. Spikes from the last layer are pulled out of the loop as output. (d) The backward pass for unlayered backpropagation of a single timestep that consisted of n+1 forwards steps. Unlike layered backpropagation, the structure of the computational graph varies across timesteps.
4 Results
4.1
Experimental setup
We carried out our experiments SNN models trained primarily in three common benchmarking datasets, each of them has a different structure: N-MNIST (Orchard et al. 2015). SHD (Cramer et al. 2020 and DVS gestures (Amir et al. 2017). N-MNIST has purely spatial structure, SHD purely temporal, and DVS grss combines both spatial and temporal (input framing in DVS gesture is done such that an entire gesture motion and contour not revealed within a single frame). These three datasets were processed with fully connected network topologies, without explicit lateral connectivity. The input was provided sequentially across number of timesteps, allowing the networks operate recurrently, exploiting their statefulness (implicit recurrence) and neuron state decay, both of which needed for temporal tasks. We also repeated some of the experiments with fourth dataset, CIFAR-10 (Krizhevsky) 2012). in more complex and deep convolutional VGG-style network. In this case, as the input consists of static images, it was fed to the network in a single limestep. Our experiments involved primarily models with rate coding without and with refractoriness forcing neurons down to New/single spikes per timestep. We did not experiment with population coding. More details on the datasets and network topologies can be found sections A.6.1IA.10] Table [ summarizes the parameterization of the experiments and the reported results. The network architecture and hyprrparameters given in table State-of-the art performance for these tasks can be achieved with
10

---

Published in Transactions on Machine Learning Research (09/2025)
Table Parameterization of experiments and results.
Parameterization
Description
Training method
Training with layered backpropagation is marked as "Layered" and with unlayered backpropagation as 'Unlayered [scheduling strategy]".
Inference method
Inference with layer synchronization is marked as 'Layered" and with network asyn- chrony as "Asyne [scheduling strategy]".
Scheduling strategy
How to select spikes from the queue. Can be random ('RS"), based on the membrane potential just before spiking ("MS").
Forward group size F
Number of spikes to select for processing at the same time. Defanlt is 8, both during training and inference.
Stop condition
Forward pass terminates: If all network activity drains ('On spiking done") or forward step after the first spike is emitted by the output layer ('On output"). The default is "On spiking done' during training and "On output" during inference.
reasonably shallow and wide models. We chose however to train narrow, but deeper network architectures so that the effects of absence of layer synchronization can be revealed in the comparison (because of this bias in network topology the accuracy results shown can vary from the top state of the art). In section A.7 we also provide an additional ablation with regard to how various training hyperparameters affect accuracy (forward group size, refractory dropout, and momentum noise during training). Results on error convergence are provided in section A.9
4.2 Network asynchrony increases neuron reactivity
As observed in figure (top row) during asynchronous inference, neurons are more reactive, i.e. a neuron can spike after integrating only a small number of incoming currents. With layer synchronization, this effect is averaged out as neurons are always required to consider all presynaptic currents (which are more likely cancel each other out). For inference with F 8, artifacts in the number of currents spike can be observed, because each forward step propagates currents resulting from 8 spikes, causing neurons integrate more currents than necessary for firing. Similar artifacts might also occur in neuromorphic chips equipped with fixcd-width vector procossing pipclines; making these observations insightful into the behavior of such hardware.
One expects that more reactive neurons imply higher activation density, Interestingly this not necessarily the case for the models trainod for asynchronous inference! In figure 4 (bottom row) we see that we wait for the network to "drain" of spike activity during forward pass the total number of spikes will indeed be higher. But if the forward pass terminates as soon as decision is made, asynchronous models are consistently sparser. This is because asynchronous processing allows spike activity to freely flow through to the output and not be blocked at every layer for synchronization.
4.3 Unlayered backpropagation recovers accuracy and increases sparsity
Network asynchrony negatively affects the performance of the models trained with layered backpropagation in all three datasets (table 2. This likely the " "Achilles' heel" of neuromorphic processing today. However, we observe that the accuracy loss remodiated when training takes into account asynchronous procossing dynamics, which also significantly increases sparsity (by about 2x). We witnessed that the accuracy of models trained and executed asynchronously is consistently superior under the two scheduling policies considered. This result conjectures that neuromorphic AT is competitive and more computationally efficient.
Figure 5 reveals another interesting result. It depicts how accuracy evolves as we allow more forward steps in the forward pass after the initial output during asynchronous inference. We see that because of the free flow of key information depth-first, models trained with unlayered backpropagation obtain the correct predictions as soon the output layer gets stimulated. Activity that likely triggered by "noise" in the input is integrated later Momentum scheduling is particularly good at exploiting this to boost accuracy.
11

---

Published in Transactions on Machine Learning Research (09/202))
WNISS
SHD
DVS gesture
0.10
Aoynee
0.20
0.32
Asyncc RS inference, F
Layered
0.01
0.05
0.00
20
10
20
20
40
60
00
100
Number
Number
Number
2.5-
Layered inference
Poyncc
0.0 output
output
Stop sprking
spilknng cone
Stop condition
Stop
Figure 4: (Top row) Number of currents integrated by neuron before spiking, recorded per neuron and per forward pass for all samples and neurons (excluding the neurons in the input layer). The Y-axis shows the relative frequency of the number of currents integrated before spiking. (Bottom row) Mean number of spikes per neuron during inference of all samples in the test. Error bars show the 25th and 75th percentiles. (Models in this figure were trained with layered backpropagation.)
Table 2: Accuracy and activation density results. More details about these metrics in section A.6.2
N-MNIST
SHD Acc.
DVS gesture
Training
Inference
Acc.
Density
Density
Acc.
Density
Layered
Layered
0.949
3.987
0.783
14.386
0.739
46.473
Layered
Async RS
0.625
3.652
0.750
13.905
0.701
44.140
Unlayered RS
Asyne RS
0.956
1.504
0.796
5.100
0.777
25.140
Unlayered MS
Asyne MS
0.963
1.476
0.816
6.224
0.856
26.686
N-MNIST
SHD
DVS gesture
0.80
0.86
0.80
0.83
Layered training
Unlayered Rt training
0.74
0.74
Unleyered training
0.72
0.00
Dn spiking done
0.72
10
0
8
10
10
Forward after
Forward after
after
Figure 5: Accuracy as function of forward steps after the first spike in the output layer. Given that F = 8, each extra forward step processes another 8 spikes, assuming enough spikes are available. Dashed lines show the accuracy after all spike activity has been "drained" out of the network.
4.4 Network asynchrony and energy efficiency
To quantify the energy savings from asynchronous processing in the trained models, table 3 presents the mean number of synaptic operations and projected energy consumption on the μBrain neuromorphic chip (Stuijt et al. (021) per sample. The reported reduction under asynchronous trained models with unlaycred backpropagation account for savings in dynamic power. Additional details also on static power savings in a digital (neuromorphie) accelerator provided in appendix A.11
12

---

Published in Transactions on Machine Learning Research (09/2025)
Table Mean num of Synaptic Operations (SOs x x10*) and energy consumption (IJ) for classification of one sample, using 26 pJ/SO as measured for yBrain (Stuitt et al., 2021); ignoring static power, (Numbers are based on projections based on energy reported in literature, not from actual mn-hardware measurement.)
N-MNIST
SHD
DVS gesture
Training
Inference
SOs
Energy
SOs
Energy
SOs
Energy
Layered
Layered
3.521
0.9155
50.82
13.21
158.8
41.29
Layered
Async RS
3.225
0.8386
49.12
12.77
150.9
39.22
Unlayered RS
Async RS
1.328
0.3454
18.02
4.684
85.92
22.34
Unlayered MS
Asyne MS
1.304
0.3389
21.99
5.717
91.2
23.71
N-MNIST
SHD
DVS gesture
0.08
0.03
0.03
Layered training
0.06
Asyne RS inference
Unlayered
0.04
Async RS inference
Unlayered MS training Asyyc MS inference
0.01
0.01
0.02
0.00
0.00
0.00
20
40
60
80
100
50
100
50
100
150
Latency
Latency
Latency
Figure 6: Latency per forward pass for all samples, in number of spikes until a decision in the output layer is reached. The Y-axis shows the relative frequency of recorded latencies.
4.5 Network asynchrony reduces latency
An oqually important result concerns the inference latency reduction under asynchronous network processing. The models trained with unlayered backpropagation have significantly lower latency than those trained with layer synchronization. Assuming unit latency for processing one spike, the comparatively worst case latency will be given under sequential processing, function of the number of spikes processed until an inference decision is made. Figure D shows distribution of the inference latencies across the entire test-sct. It should be clear by now that this is because "the important spikes" " in these models quickly reach the output layer, uninhibited by layer synchronization.
4.6 Unlayered backpropagation is (currently) iecource-intessive
We also tried confirm the results in a scaled up setup, namely with deeper VGG-7 like network (details in A.10 trained on the CIFAR-10 dataset. Figure 7 confirms the problem when training with layered backpropagation and the rumning asynchronous inference. When try to train asynchronous models with unlayered backpropagation and the simpler RS scheduling policy the memory cost however becomes prohibitive unless we substantially increase the forward group size F. Nevertheless, even with as large as 512 (during training, the smallest possible with an NVIDIA RTX 5000 GPU) and tested with F = 64, accuracy steeply recovers to 71%, while activation density drops to about 1/4 (confirming our previous obscrvations). We anticipate that with much smaller F during training (higher degree of network asynchrony), the accuracy can be completely recovered.
Unfortunately, the computational cost of unlayered backpropagation, in the current framework (and imple- mentation) is rather high, espccially when training with small F. Each time F is halved, the time and memory requirements approximately double as discussed in sections 3.2. and A.8) While this computational cost imposes obvious scalability constraints in the current implementation of mollyeree-backproopae training very deep models with complex structures, it does not burden however inference on-chip. But it creates trade-off, since the costlier computationally the training becomes, the more robust asynchronous inference will be when the resulting model is deployed on-chip.
for
13

---

Published in Transactions on Machine Learning Research (09/2025)
Thxt C1588 10, tranned sectemmbation and
CIFAR-10
Training
Inference
Acc.
Density
(a)
(b)
Layered
Layered
0.805
0.0089
Layered
Asyne RS
0.183
0.0111
Asyne RS
Async RS
0.712
0.0026
Layer UWX
F-M
Figure 7: (a) Impact of network asynchrony CIFAR-10 trained VGG-style model. A larger decrease in accuracy occurs with smaller forward group sizes accompanied by an increase in activations. (b) Accuracy and activation density results on CIFAR-10.
5 Discussion & Future outlook
We pinpoint crux for neuromorphic AI processing in training SNN models conventionally with backpropaga- tion and naively assuming they will execute consistently, and energy and latency efficiently in neuromorphic processors. The problem lies in neglecting the dynamics of asynchronous processing, and we found way to factor them in the gradient training process. Resulting models not only recover the affected performance (and even exceed it) but also exhibit the energy and latency benefits touted by neuromorphic computing.
Asynchronous processing mainly about neurons acting independently of other neurons (also across layer boundaries) based locally emerging flow dynamics, which is feasible with rate-like codings. So long these dynamics are unobstructed by synchronization barriers, there is no need for explicit time- time-tracking, Temporal coding schemes requiring explicit time tracking (timestamps) typically do so due to the presence of synchronization points in event processing. Moreover, order coding and TTFS schemes which may use timestamps to enforce strict per/ace per/acros--laye ordering, can also do without them by working with queues. The overarching assumption that exact timing relevant mainly to the extent that it facilitates the relative ordering of events.
From this viewpoint, in our framework synaptic currents arriving at a neuron trigger instantaneous evaluation of the membrane threshold independently of other incoming currents (i.c. there is integration time interval al the neuron level). This eliminates the need to keep explicit global clock for spike times. The instantaneous membrane voltage of neurons then reflect the temporal dynamics of the incoming activations at the neuron level. The use of dynamic scheduling allows neuron order dynamics to be reflected in spike exchanges without synchronization delays al layer boundaries. In other words, these modeling ranifications reflect relative temporal (order) dynamics among neurons across layers, unfolding inside the entire network under the influence of (a) the input distribution, and (b) the intrinsic operation of the underlying computing substrate (c.g., AI accelerator).
Accounting for neuron execution scheduling strategies in-training is likely a missing link for between neural algsrithms models and neuromorphic hardware architecture co-optimization. This study merely scratched the surface of a research exploration in this direction, but shows that unless we rethink our training methods account for asynchronous dynamics and redesign neuromorphic accelerators to move away from layer synchronization primitives, SNNs and brain-inspired computing could fail to deliver both performance and efficiency.
It is worth noting that the discrepancy between the dynamics encoded in neurons' membranes and the way these dynamics are propagated across the neural network in current SNNs, was also independently identified and published only recently in (Yao el al., 2024 2025). To address this perceived as "modeling flaw" in how SNNs work, they propose training ANN models and converting them to SNNs by translating quantized integer activations to spike bursts of equivalent cardinality; or directly training bespoke SNN models where neurons fire not single spikes but spike bursts proportional their membrane potentials. While these approaches at least preserve ANN performance, they only suited for models of integrate-and-fiee
14

---

Published in Transactions on Machine Learning Research (09/2025)
(IF) neurons, since membrane leakage in LIF neurons would become ineffective with bursts of this sort. More importantly, the spike rates increase dramatically with adverse effect on energy consumption (which is a function of the spike density). By contrast, in the herein presented exploration we identify the 'flaw' in the per-layer synchronization mechanism (which is a baggage from ANNs, but not inherently native to SNNs) that blocks these dynamics from expressing freely throughout the network. We show that casting away layer synchronization and enabling dynamics driven, adaptive neuron-cxecution scheduling during training per-se, we can train models aware of various asynchronous hardware, that finish inference faster than hardware agnostic ones, and provide correct predictions by processing only a subset of the spikes (improving both on energy and latency efficiency). Either approach however is in infancy, deserving more research and engineering for addressing scalability and demonstrating practical testing and efficiency with modern large deep models (e.g. as proposed in Yao et al.
2025 on neuromorphic accelerators that can accommodate
them (e.g. Davies et al. 2018 Furber et al.
2014 Tang et al. 2023
This motivates our follow-up work on challenges along parallel diroctions: (a) improving on the computational
resource limitations of asynchronous training with unlayered backpropagation, and (b) bridging this work
to more complex contemporary model structures and constructs (e.g. (Deng et al. 2022 Yao et al. 2022
Guo et al. 2023b Yu et al.) ))22 Regarding the former direction, recent advances with event-
back-propagation Wunderlich & Pehle 2021a) and sparse gradient learning (Lohoff et al. 2025 1 show promise for addressing these challenges.
Acknowledgments
The herein rescarch was partially sponsored and funded by Imee Netherlands, and the EU's Horizon Europe Research and Innovation programme (under Grant Agreement, 101070679). Guangzhi Tang is partially funded by the Dutch Research Council's programme AiNed XS Europe programme (under Grant agreement 1009.2692.3..
attps://doi.or 110.66666/MYMM22 Research Council (under Grant agreement KICHL.STO44.2.021),
Roel Koopman is funded by the Dutch
References
Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. Truenorth: Design and tool flow of a 65 nw 1 million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems, 15877)77577 2015.
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event- based gesture recognition system. In Proceedings of the IEEE conference on compuler vision and pallern recognition, pp. 7243 7252, 2017.
Michael Berry and Markus Meister. Refractoriness and neural precision. Advances in neural information processing systems, 10, 1997.
Lina Bonilla, Jacques Gautrais, Simon Thorpe, and Timothée Masquelier. Analyzing dime-to-first-pile coding schemes: A theoretical approach. Frontiers in Neuroscience, 16:971937, 09 2022. doi: 10.3389 fnins. 2022.971937.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Caterina Caccavella, Federico Paredes- Vallés, Marco Cannici, and Lyes Khacef. Low-power event-based face detection with asynchronous neuromorphic hardware. arXiv preprint arXiv:2312.14261 2023.
Iulia-Maria Comşa, Krzysztof Potempa, Luca Versari, Thomas Fischbacher, Andrea Gesmundo, and Jyrki Alakuijala. Temporal coding spiking neural networks with alpha synaptic function: Learning with backpropagation. IEEE Transactions on Neural Networks and Learning Systems, 33(10))5939-56)) -9522, 2022. doi: 10. N..1109/TNNTS..O 22222001117797 3071976.
15

---

Published in Transactions on Machine Learning Research (09/2025)
Benjamin Cramer, Yannik Stradmann, Johannes Schemmel, and Friedemann Zenke. The heidelberg spiking data sets for the systematic evaluation of spiking neural networks. IEEF Transactions on Neural Networks and Learning Systems, 33(7):2744 2757, 2020.
Manon Dampfhoffer, Thomas Mesquida, Alexandre Valentian, and Lorena Anghel. Backpropagatton- based learning techniques for deep spiking neural networks: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Mike Davies, Narayan Srinivasa, Tsung- Han Lin, Gautham Chinya, Yonggiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38(1):82-99, 2018.
Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural net- work via gradient re-weighting. In Proceedings of the International Conference on Learning Representation, 02 2022. doi: 10.48550/ (18550//rXiv.... www.iv.222..11.1.
Peter U. Dichl, Danicl Neil, Jonathan Binas, Matthew Cook, Shih-Chii Liu, and Michael Pfeiffer. Fast- classifying, high-accuracy spiking deep networks through weight and threshold balancing. In 2015 Interna- tional Joint Conference on Newral Networks (IJCNN), pp. 1-8, 2015. doi: 10. D0.109/LICCN,,C WOW.CCNNN..1...8800.000
Rainer Engelken. Sparseprop: Efficient event-based simulation and training of sparse recurrent spiking neural networks. Advances in Newral Information Processing Systems, 36, 2024.
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF international conference computer vision, pp. 2661-2671, 2021.
S. B. Furber, F. Galluppi, S. Temple, and L. A. Plana. The spinnaker project. Proceedings of the IEEE, 102 (5):652-665, 2014. doi:
Yufei Guo, Yuanpei Chen, Liwen Zhang, YingLei Wang, Xiaode Liu, Xinyi Tong, Yuanyuan Ou, Xuhui Huang, and Zhe Ma. Reducing information loss for spiking neural networks. In European Conference on Computer Vision, pp. 36- 12-52.66 Springer, 2022.
Yufei Guo, Xuhui Huang, and Zhe Ma. Direct learning-based deep spiking neural networks: a review. Frontiers in Neuroscience, 17:1209795, 2023a.
Yufei Guo, Yuhan Zhang, Yuanpei Chen, Weihang Peng, Xiaode Liu, Liwen Zhang, Xuhni Huang, and Zhe Ma. Membrane potential batch normalization for spiking neural networks. In 2023 IEEE/CVT International Conference on Computer Vision (ICCV), pp. 19363 19373, 2023b. doi: LLL00/1ICVV511002002.....
Jesse Hagenaars, Federico MaredesVVallss and Guido de Croon. Self-supeviised learning of event-based optical flow with spiking neural networks. Advances in Neural Information Processing Systems, 34, 2021.
Weihua He, YuJie Wu, Lei Deng, Guoqi Li, Haoyu Wang, Yang Tian, Wei Ding, Wenhui Wang, and Yuan Xie. Comparing smns and rmns on neuromorphic vision datasets: Similarities and differences. Neural Networks, 132:108 120, 2020.
Spiking Neural Processor PULSAR. Innatera Nanosystems BV, Rijswijk, The Netherlands, 2024. Available at http:ss///mmatterr.ccom///////
Laxmi R Iyer, Yansong Chua, and Haizhou Li. Is neuromorphic mnist neuromorphic? analyzing the discriminative power of neuromorphic datasets in the time domain. Frontiers in neurosciencs, 15:608567, 2021.
Sacod Reza Kheradpisheh and Timothée Masquelier. Temporal backpropagation for spiking neural networks with one spike per neuron. International Journal of Neural Systems, 30, 03 2020. doi: 10.1442/5012900555550777
16

---

Published in Transactions on Machine Learning Research (09/2025)
Jachyun Kim, Heesu Kim, Subin Huh, Jinho Lee, and Kiyoung Choi. Deep neural networks with weighted spikes. Newrocomepuling, 311:373 386, 2018. ISSN 0925-2312. doi: http:///oooorg//1.0000//moooooo 05.087. URL
Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: Spiking neural network for energy-efficien object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 34:
04 2020. doi: 10.1609 11.609/aaai.v34i10 v34107.6777 .6787.
Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto, 05 2012.
Chit-Kwan Lin, Andreas Wild, Gautham N. Chinya, Yongqiang Cao, Mike Davies, Daniel M. Lavery, and Hong Wang. Programming spiking neural networks on intel's loihi. Computer, 51(3):52 61, 2018. doi:
(A11009/MC201181110
Yuhang Liu, Tingyu Liu, Yalun Hu, Wei Liao, Yannan Xing, Sadique Sheik, and Ning Qiao. Chip-in-loop smn proxy learning: new method for efficient training of spiking neural networks. Frontiers in Neuroscience, 17:1323121, 2024.
Jamie Lohoff, Anil Kaya, Florian Assmuth, and Emre Neftci. A truly sparse and general implementation of gradient-based synaptic plasticity. In Neuro-inspireed Compulational Elements, NICE 2025, 2025.
Michael London, Arnd Roth, Lisa Beeren, Michael Häusser, and Peter E Latham. Sensitivity to perturbations vivo implies high noise and suggests rate coding cortex. Nature, 166(7302):123-11) 2010. doi: 10.1038/ 1011038/nature09055
Alexandra Sasha Luccioni, Sylvain Viguier, and Anne-Laure Ligozat. Estimating the carbon footprint of bloom, 176b parameter language model. Journal of Machine Learning Research, 24(253):1 15, 2023.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Newral networks, 10(9):1659 1671, 1997.
Massimo Monti, Manolis Sifalakis, Christian F. Tschudin, and Marco Luise. On hardware programmable network dynamies with chemistry-inspired abstraction. IEEE/ACM Transactions in Networking, 25 aug 2017. ISSN 1063-6692. doi: 1109/TNET. 2017 2674690
URL https:://oo htw//oo..or/////
Garrick Orchard, Ajinkya Jayawant, Gregory K Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in neuroscience, 9:437, 2015.
Emin Orhan and Xaq Pitkow. Skip connections climinate singularities. In International Conference on Learning Represenlations, 2018.
Eustace Painkras, Luis A Plana, Jim Garside, Steve Temple, Francesco Galluppi, Cameron Patterson, David R Lester, Andrew D Brown, and Steve B Furber. Spinnaker: A 1-w 18-core system-on-chip for massively-paralle neural network simulation. IEEE Journal of Solid-Stale Cirewits, 48(8):1943 1953, 2013.
Seongsik Park, Seijoon Kim, Hyeokjun Choe, and Sungroh Yoon. Fast and efficient information transmission with burst spikes deep spiking neural networks. In 2019 56th ACM/TEEE Design Automation Conference (DAC), pp. 6, 2019.
Seongsik Park, Seijoon Kim, Byunggook Na, and Sungroh Yoon. T2fsnn: Deep spiking neural networks with tine-to-first-sikke coding, 57th ACM/IEEE Design Aulomation Conference (DAC), pp. 1 6, 2020. doi:
Bodo Rueckauer, Aleia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chi Liu. Conversion of contimuous-valued doep networks efficient event-driven networks for image classification. Frontiers in Neuroscience, 11, 2017. doi: 10.3389 fninininiiiiiini 2017.00682 URL wtwp//irr.....nttiriiiiiii. wwt.:.rrrtttrri..oooooooooaaaaa. org/journals/
17

---

Published in Transactions on Machine Learning Research (09/2025)
Sanaullah, Shamini Koravuna, Ulrich Rückert, and Thorsten Jungeblut. Exploring spiking neural networks: comprehensive analysis of mathematical models and applications. Fronliers in Compulational Neuroscience, 17:1215824, 2023.
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience, 13:95, 2019.
Mahyar Shahsavari, Jonathan Beaumont, David Thomas, and Andrew D. Brown. POETS: A Parallel Cluster Architecture for Spiking Neural Network. International Journal of Machine Learning and Computing, 11 (4):281-285, August 2021. ISSN 20103700. doi: 10.18178/ http:/mm..222...........
P Srivatsa, Kyle Timothy Ng Chu, Yaswanth Tavva, Jibin Wu, Malu Zhang, Haizhou Li, and Trevor E. Carlson. You only spike once: Improving energy-efficien neuromorphic inference to ann-level accuracy. ArXie, has/2006.099822 2020. doi: 10.48550/arXiv.00000
Jan Stuijt, Manolis Sifalakis, Amirreza Yousefzadeh, and Federico Corradi. pbrain: An event-driven and fully synthesizable architecture for spiking neural networks. Prontiers in neuroscience, 15:664208, 2021.
Christian Szegedy, Sergey Loffe, Vincent Vanhoucke, and Alexander Alemi. Inception-v4, inception-resnee and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
Guangzhi Tang, Kanishkan Vadivel, Yingfu Xu, Refik Bilgic, Kevin Shidqi, Paul Detterer, Stefano Traferro, Mario Konijnenburg, Manolis Sifalakis, Gert-Jan van Schaik, and Amirreza Yousefzadch. Sencca: building a fully digital neuromorphic processor, design trade-offs and challenges. Frontiers in Neuroscience, 17, 2023. ISSN 1662-453X. doi: 15389/ffins.2011 1187252.
Simon Thorpe, Arnaud Delorme, and Rufin Van Rullen. Spike- based strategies for rapid processing. Neural Networks, 14(6-7):715 725, 2001. doi:
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis. Deep learning for computer vision: A brief review. Computational intelligence and newroscience, 2018, 2018.
Timo Wunderlich and Christian Pehle. Event- backpropagation can compute exact gradients for spiking neural networks. Nature Scientific Reports, 11(12829), 2021a. doi: 011/8///41590001--9919
Timo C Wunderlich and Christian Pehle. Event-based backpropagation can compute exact gradients for spiking neural networks. Scientific Reports, 11(1):1282) 2021b.
Man Yao, Ole Richter, Guangshe Zhao, Ning Qiao, Yannan Xing, Dingheng Wang, Tianxiang Hu, Wei Fang, Tugba Demirci, Michele De Marchi, Lei Deng, Tianyi Yan, Carsten Nielsen, Sadique Sheik, Chenxi Wu, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-basod dynamic computing with asynchronous sensing- computing neuromorphic chip. Nature Comm., 15(),, 2024. doi: 10.1038/s41440 46-1144---2---7781---
Man Yao, Xuerui Qiu, Tianxiang Hu, Jiakui Hu, Yuhong Chou, Keyu Tian, Jianxing Liao, Luziwei Leng, Bo Xu, and Guoqi Li. Scaling spike-driven transformer with efficient spike firing approximation training. IEFE Transactions on Pallern Analysis and Machine Intelligence, 47(4):2973 2990, 2025. doi: 10.1109/
202MM.20253333226
Xingting Yao, Fanrong Li, Zitao Mo, and Jian Cheng. Glif: a unified gated leaky iinerraee-ad-ffie neuron for spiking neural networks. Tn Proceedings of the 36th International Conference on Newral Information Processing Systems, NIPS '22, 2022.
Amirreza Yousefzadeh, Gert-Jan Van Schaik, Mohammad Tahghighi, Paul Detterer, Stefano Traferro, Martijn Hijdra, Jan Stuijt, Federico Corradi, Manolis Sifalakis, and Mario Konijnenburg. Sencca: Scalable energy- efficient neuromorphic computer architecture. In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 371 374. IEEE, 2022.
18

---

Published in Transactions on Machine Learning Research (09/2025)
Chengting Yu, Zheming Gu, Da Li, Gaoang Wang, Aili Wang, and Erping Li. Stsc-snn: Opatio-tempora synaptic connection with temporal convolution and attention for spiking neural networks. Frondiers
in Neuroscience, 16, 2022. ISSN 1662-453X. doi: 10.3389 10.3389/fning /tt///iiiin.22222 10793377 URL https://wwy
frontiersin htapaaaa........ooaaaa//////nnns 3389//ninss 2022.107933
Semir Zeki. A massively asynchronous, parallel brain. Philosophical Transactions of the Royal Society B:
Biological Sciences, 370(1666):201401777 2015.
Friedemann Zenke and Tim P Vogels. The remarkable robustness of surrogate gradient learning for instilling complex function in spiking neural networks. Neural computation, 33(4):899-925, 2021.
Haiyan Zhao, Hanjic Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawci Yin, and Mengnan Du. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1-38, 2024.
Yaoyu Zhu, Zhaofei Yu, Wei Fang, Xiaodong Xie, Tiejun Huang, and Timothée Masquelier. Training spiking neural networks with event-driven backpropagation. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, 2022.
A Appendix / supplemental material
A.1 Spiking neural networks
A.1.1
Incoming current
For every neuron layer, all the synaptic weights on its inbound connections are kept in a weight matrix W( E R xx(0) where ((() = number of input features Ni. Using these weight matrices, the total incoming current x for a neuron in the next layer can be computed using:
(1+
Σ
(1+1) 1
(3)
where (t) is if neuron j in layer I has emitted spike at time t, otherwise O.
A.1.2
Membrane potential decay
The decay of the membrane potential is governed by linear Ordinary Differential Equation (ODE). The analytical solution can be used to compute the decay:
u[[] ull Al] + x
(4)
where Tm is the membrane time constant and At the elapsed time.
A.1.3 Threshold function
f if u > Uchr otherwise
Ouu) where Uthr is the membrane potential threshold required for spiking.
(5)
A.2 Event-driven state update rule for LIF neuron
When an input current received at time by a neuron with a previous state u[to] at some time to S t, an atomic set of computations is executed. Start with computing the decayed membrane potential u[t_] ulto]
-(t-to) then update the membrane potential u[t+] ult_] + z[t] with the input current x(t), and finally set ull]] and emit, spike if ((u((+)) 1; otherwise un[]] w([+] without emitting spike.
19

---

Published in Transactions on Machine Learning Research (09/2025)
Note that the evaluation of the firing threshold takes place independently for every incoming synaptic current (instead of synchronizing and accumulating all currents first, and then evaluating the threshold). This allow to consider and preserve the temporal dynamics among the input currents (different arrival times) in the temporal derivative of the membrane potential of each neuron and propagate them accordingly.
A.3 Arctan surrogate gradient
(u) arctan (Tu a 2
(6)
where a is a hyperparameter modifying the steepness of the function.
A.4 Class prediction
Class prediction involves first calculating the output rates as follows:
C
Σ s(i, t]
(7)
SET
where & the spike count for class i, 7 are all the timesteps for which a forward pass occurred, and 8(1, I] is the output of the neuron representing class in the output layer at the end of the forward pass at time t. For CIFAR-10, instead the value of a is equal to the membrane potential of the output neuron for class i at the end of having processed all the spikes. These values are subsequently used as logits within softmax function:
e
Pi
No j==1 e
(8)
where Ne is the total number of classes. The resulting probabilities are then used to compute the cross-entropy loss:
Nc
log(p;)
(9)
where y E (0, No is the target class in one-hot encoded format.
20

---

Published in Transactions on Machine Learning Research (09/2025)
A.5 Parameters for the simulator
Table 4: Overview of all current simulator parameters. If the text is italic, then the parameter used for the experiments in this paper.
Name
Value range
Description
Forward group size
Noo
3.1.2
Scheduling policy
RS/MB
3.1.2
Prioritize input
True/False
If input spikes are propagated before any spikes from inside the network.
Stop condition
On spiking done
3.1.2
On output
Forward steps after output
Nz0 1.0)
3.1.2
(only for 'On output' stop condition)
Refractory dropout Momentum noise
3.2.2
R20
3.1.2
(only for 'MS" scheduling policy)
Membrane time constant Input spike dropout
Ry0
3.1
[0.0, 1.0) [0.0, 1.0)
3.2.2
Network spike dropout
The same as input spike dropout, but for spikes from inside the net applied per forward
step.
Membranc potential threshold
R>0
3.
Timestep size Synchronization threshold
Nso
Ngo
Have neurons wait for number input cur- rents (including barrier messages) before being allowed fire. Can be set per neuron.
Emit barrier messoges
True/Falst
Have neurons emit barrier message if they have retrieved exactly the number of input currents to exceed the synchronization thresh- old, but not enough to exceed the membrane potential threshold.
Table 5: Simulator parameters for simulating neuromorphic processors
Synchronization type
Neuromorphic processors
Forward group size
Synchronization threshold
[emit
barrier messages)
Barrier-based
/ layer
Loihi (Davies et. al. 2018).
SENECA POETS
neu/layer
neu/layer [True]
Yousefzadeh et al. 2022
Shahsavari et al. 2021)
Timer-based based core
SENECA (Yousefzadeh et al.
2022)
# neu/core
# neu/core [True]
SpiNNaker (Painkras et al.
2013).
TrueNorth (Akopyan et al.) 2015 PO-
ETS (Shahsavari al. 2021
Asynchronous
Speck (Caccavella et al.
2023
nBran
neu/layer
1 [False]
(Stuijt al. 20221)
A.5.1 Neuron execution scheduling policy & asynchronous processing dynamics
A scheduling policy in the simulation environment aims capture the biases introducod by the operating principle of an AI accelerator (class) on data dynamics, and provide a means reflect them in the model training process (see Unlayered Backpropagation). We experimented with two scheduling policies that capture
21

---

Published in Transactions on Machine Learning Research (09/2025)
two different aspect of order dynamics, aiming to highlight their effects and importance but they are not the only possible. It is a topic of open exploration for the future, and co-design for dataflow AT accelerators.
Random Scheduling, reflects the fact that the spike propagation and neuron integration process is or can be inherently noisy (either epistemically or aleatorically), thus affecting the temporal/rank order of spikes. It can be seen as annealing noise or dropout noise, which beyond certain level breaks the system down but in small enough quantities (relevant to the vector pipelining of an accelerator F-group size, arbitration between cores, network-on-chip routing, etc), makes the system more robust. Note that it is applied across the entire network, not within each layer.
Momentun Scheduling, puts emphasis on the relative order dynamics in neuron evaluation as reflectod in their membranes (i.e. which neuron is likely to fire first next relevant to TTFS and rank-order codes for the importance of the first spike(s) even though we work with rate codes). Note again this takes place across all layers, and remains unbiased by artificial layer-synchronization barriers.
A.6 Details on the experimental setup
A.6.1 Datasets and network architectures
The Neuromorphic MNIST (N-MNIST) dataset captures the MNIST digits using Dynamic Vision Sensor (DVS) camera. It presents minimal temporal structure (Tyer et al. 2021 b. It consists of 60000 training samples and 10000 test samples. Each sample spans approximately 300 ms, divided into three 100 ms camera sweeps over the same digit. Only the initial 100 ms segment of each sample is used in this study.
The Spiking Heidelberg Digits (SHD) dataset (Cramer et al. 2020 is composed of auditory recordings with significant temporal structure. It consists of 8156 training samples and 2264 test samples. Each sample includes recordings of 20 spoken digits transformed into spike sequences using cochlear model, capturing the rich dynamics of auditory processing. The 700 cochlear model output channels are downsampled to 350 channels.
The DVS gesture dataset (Amir et al. 2017) focuses on different hand and arm gestures recorded by a DVS camera. Like the SHD dataset, it has significant temporal structure. It focuses on 11 different hand and arm gestures recorded by a DVS camera. It consists of 1176 training samples and 288 test samples. The 128 128 input frame is downsampled a 32 x 32 frame.
2012 is widely-used image classification dataset consisting of 60000 32x32 color images across 10 classes, including animals and vehicles. It has no temporal structure. Unlike the other datasets, all imput is provided in one single timestep.
CIFAR-10 (Krizhevsky
For all four datascts, events of a data point belong to a contimuous stream where they have a timestamp and an index position (see figure R). In the case of N-MNIST, the index corresponds to a position within a 34 x 34 pixel frame, with each pixel having a binary polarity value (either or 0), leading to total of 34 x 34 x 2 2312 distinct input indices. For SHD, the index denotes one of the 350 frequency channels. For DVS gesture, the 128 x 128 input frame is downsampled to 32 x 32 frame. Like N-MNIST, each pixel has a binary polarity value, so in total this gives 32 x 32 x 2 = 2048 input indices. Unlike the other datasets, for CIFAR-10, the input events present a continuous value (so they are currents instead of spikes), while also still being assigned an index and timestamp (although the timestamp is irrelevant). For encoding them as inputs to the models the stream of events is sliced in time-bins (eR, .x.),...) along the temporal dimension and the timestamps are discretized to the time-bin index. The time- bins then construct time-frames whereby all the events at each spatial index are accunulated (counted). Each time-frame is then used as input to the model at discrete subsequent timesteps. This is one of the common-place input encoding for SNNs.
A.6.2 Performance metrics
To evaluate accuracy, output rates e for each class i are first calculated as outlined equation 7 The predicted class corresponds to the one with the highest output rate. Accuracy is then quantified as the ratio of correctly predicted outputs to the total number of samples.
22

---

Published in Transactions on Machine Learning Research (09/2025)
Table Network architecture and hyperparameters. The architecture is given as [neurons in hidden layers x number of hidden layers] [ncurons in output layer].
N-MNIST
SHD
DVS gesture
CIFAR-10
Architecture
[64x3]-10
x28x33-20
[128x3]-11
see
A.10
Timestep size
10 ms
10 ms
20 ms
N/A
Batch size
256
32
32
64
Epochs
50
100
70
150
Learning rate
5e-4
7e-4
le-4
2e-4
Membrane threshold Utr
0.3
0.3
0.3
0.2
Weight decay constant Aw
1e-5
1c-4
1e-5
Membrane time constant Tm
1 ms
100
100
N/A
Surrogate steepness
2
10
10
2
Input spike dropout
0.25
0.2
0.2
0
Forward group size
8
8
8
512
Refractory dropout
0.8
0.8
0.8
0.7
Mo noise AMS
1e-6
0.1
0.1
see
A.10
Event
H
[6]
Gpol
Event representation
Figure 8: Encoding of a continuous temporal stream of input events into frames Hagenaars et al. 2021
Spike density computed using:
N Nanmpleu
density
Napikes
(10)
Nsamples
N Nueurous
where Nsamples is the number of samples, Nneurons is the number of neurons in the hidden layers, and Napites is the total number of spikes during inference of the sample i.
[i]
A.7 Results hyperparameters
Choosing smaller (i.c., with a more asynchronous system), may improve accuracy, particularly benefiting models with mechanisms that rely network asynchrony such momentum scheduling. However, reducing
23

---

Published in Transactions on Machine Learning Research (09/2025)
F also has its drawbacks. It significantly raises resource demands (discussed in section A.8 and there is risk of reducing the elfectiveness or even stalling the training process, as observed for SHD and DVS gesture.
Refractory dropout, can positively affect training outcomes. An explanation for this is that it increases the gradient flow by allowing more spiking activity. However, using full refractory dropout can also reduce performance, likely due to the inability generalize to inference with refractoriness.
The momentum noise helps by introducing slight stochastic element into the spike selection process, helping to avoid potential local minima that purely deterministic selection method is prone to get stuck in. This seems to do little for N-MNIST, but for more complex datasets like SHD and DVS gesture it has significant. effect.
0.95
0.90
0.8
Accuracy
0.85
0.7
0.80
N-MNIST
N-MNIST RS
SHD MS
0.75
0.4
DVS MS
0.70
DVS gesture RS
15
1.0
0.0
0.2
1.0
Training
Momentum
Refractory dropout
Figure 9: Results from inference with network asynchrony for different hyperparameters used during training. For SHD and DVS gesture with random scheduling, training Fs smaller than are not inchuded due to the training failing to converge.
A.8 Results on resource use
Table 7:
Resource usage compared between layered and unlayered backpropagation during the second epoch of training NVIDIA Quadro RTX 5000.
Method
Time per epoch (s)
VRAM use (MB)
N-MNIST
Layered
14
210
Unlayered RS, training = 16
19
412
Unlayered RS, training F
556
Unlayered RS, training
37
986
SHD
Laycred
38
214
Unlayered RS, training F
118
2220
Unlayered RS, training
292
4844
Unlayered RS, training F
681
10574
DVS gesture
Layered
120
244
Unlayered RS, training F
150
3802
Unlayered RS, training
177
6984
Unlayered RS, training F
245
13914
24

---

Published in Transactions on Machine Learning Research (09/2025)
The increase in processing time is less pronounced for the N-MNIST and DVS gesture datasets compared to the SHD dataset. This discrepancy could be due to computational optimizations that apply specifically to the N-MNIST and DVS gesture datasets (both being usion-based datasets).
A.9 Results convergence
Both unlaycred training with random schoduling and momentum schoduling achieve lower loss values comparod to layered training, shown in figure T0. At the start of training, random scheduling shows a slightly less steep loss curve, while momentum scheduling shows steepness comparable to that of layered training. For CIFAR-10, error convergence is shown in figure 12
N-MNIST
SHD
DVS gesture
2.75
Layered training
2.2
3.0
Unlayered RS training
2.50
Unlayered MS training
2.0
2.8
2.25
1.8
2.00
2.6
1.6
1.75
1.4
1.50
1.25
1.2
1.00
0
10
20
30
40
50
20
40
60
80
100
20
40
60
Epoch
Epoch
Epoch
Figure 10: Error convergence for different synchronization methods.
A.10 Details on CIFAR-10 dataset with VGG experiments
The details of network structure of the VGG model that we used to train on the CIFAR-10 dataset is shown in figure 11 We have removed (for simplicity) the batch normalization and dropout layers (that explicitly necessitate layer synchronization).
To reduce the memory overhead for training with unlayered backpropagation and make it feasible NVIDIA RTX 5000 we provided all the input of each sample in single timestep (thus eliminating the unfolding across timesteps), and we adopted forward group size F' = 512 for training and F = 64 for testing. Additionally we deployed Integrate-and-Fire (IF) neurons that do not leak.
Output predictions were based on the membrane potential accumulated at the output neurons. Because of the very large F-group size, MS schoduling policy was difficult train requiring rather large amounts of annealing noise (the range of values tested are shown in figure 12 Note that large F-group is typically adverse for supporting network asynchrony, but in this experiment were limited by the training resources given the currently high computational cost of unlayered backpropagation. Nonetheless, the results in figure 7 show that even with such large value range for F-group, there monotonic trend for accuracy recovery in asynchronous inference, as we decrease F.
The training error evolution reported figure 12 For momentum schoduling with AMS or Ams 2.5, training was unstable after initial convergence and was terminated as soon as the error started to diverge. This not the case for random scheduling or larger values of Aas-
25

---

Published in Transactions on Machine Learning Research (09/202))
(3,32,32)
Com22djn-3, out+64)
For lavers: kemee_sizen,, stiid=2, paddinge1
For pool layers:
(64.22.32)
AugPool2d
kemel stridee2, paddinge0
(64,6,116)
Conv26jin-44, out-128)
Number neurons: 115722 Number parameters. 21664664
(128,16,16)
AugPoolld
(128.8,8)
Conv26(nn128, out-128)
(128,8,8)
AugPool22
(128,4,4)
Conv2d/jn=128 128, out-256)
Conv2d(in256,
(256.4.4)
AugPool2d
(256,22)
Linear[i== 1024, ou1-1024)
(1024))
Linear(n==100 out-10)
Figure 11: Simplified VGG network topology without batch normalization and dropout layers.
CIFAR-10
2.25
Layered training
fm 512 (RS)
2.00
512 (MS,
512 (MS,
1.25
1.00
0.75
30
120
150
Epoch
Figure 12: Training error convergence for CIFAR-10 for different synchronization methods and amounts of momentum noise in the MS scheduling policy.
A.11 Neuromorphic processing and energy efficiency
the forefront of asynchronous spiking neural network (SNN) hardware development, showcasing significant benefits over traditional clock-based architectures. Both chips employ an event-driven, fully asynchronous, architecture, eliminating the need for global or per-layer synchronization. In both cases, all neurons can fire a spike immediately once their membrane potential crosses a specific threshold response to integrating an incoming current/spike (without waiting for synchronization signal an interval timer to expire). Such fully event-driven inference allows independent. and instantaneous neuron processing, reducing computational
Neuromorphic chips such pBrain (Stuijt al. 2021 and Speck (Caccavella et al. 2023
26

---

Published in Transactions on Machine Learning Research (09/2025)
overhead and latency, and directly matches the operational principles of SNNs. However, neurons with fully event-driven asynchronous computation are sensitive to the sub-microsecond timing and exact order dynamics of the incoming spikes, which time-stepped training algorithm is not sensitive to. As a result, there will be a significant accuracy drop if the deployed SNN these chips is trained with a timo-stepped algorithm.
Traditional synchronous simulation methods for SNNs introduce limitations when mapping to these asyn- chronous neuromorphic chips. In synchronous simulations, time is discretized into uniform steps, and all neurons in layer updated simultancously, leading to excessive idle computations and artificial delays that are not representative of "real-time" temporal dynamics. In contrast, asynchronous simulation methods align more closcly with the intrinsic event-driven nature of neuromorphic hardware. They allow all ncurons to react as events occur, mirroring the operational principle of chips like pBrain and Speck. This results in lower latency and more efficient mapping of SNN models onto the hardware.
In the absence of common-place asynchronous training and simulation methods, many other neuromorphic chips (e.g. SpiNNaker (Furber al. 2014), TrueNorth (Akopyan al.) 2015), Loihi (Davies al. 2018), Seneca (Tang et al. 2023 resort to explicit layer synchronization primitives (timer-based current integration explicit signaling) for maintaining consistent performance with synchronous simulated and trained models. Under this constraint these systems execute models event-based but not end-to-end asynchronously (even though they could). This comes at the cost of increased latency of execution and energy consumption.
Typically, energy consumption on such digital neuromorphic accelerators has two components. baseline static one that present even when the system idle so long as it powered, and which relates to its hardware attributes and components (memory leakage, clock frequency, manufacturing technology, and other). The sccond is dynamic component that relates the operation of the system for executing model. The latter has to do with actual computations the system performs, memory I/O (lypically the source of the Von Neummann bottleneck), and communication (particularly in multicore systemss that entail a network-on-chipp on-chip). In many architectures, a technique called power-gating helps save substantial energy from static power by switching off parts of the system that are idle.
Model execution is directly related energy consumption due to dynamic power (e.g. when fetching data from memory and exccuting arithmetic operations), but also indirectly due to static power (as a function of the inference latency that forces the system to stay powered).
27


Reviews:
[{"review_id": "0110VYXbqm", "reviewer": "Reviewer_CxyB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe motivation of the paper is strong. The paper explores a critical aspect of spiking neural networks—asynchronous event-based execution—which is fundamental to their energy and power efficiency.\n\nWeaknesses:\n1) How asynchronous computation is achieved in the experimental setup is not clearly highlighted and explained. \n2) The authors do not perform any on-chip implementation. Furthermore, the simulation framework used is also not well established. I believe the authors should explain the simulation framework used in more details.\n3) Architectures used are fairly simplistic (feedforward layers). It would be good to see some experiments using convolutional layers to understand how information aggregation will happen in those layers in an asynchronous setting.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors in this paper highlight the challenge of executing models asynchronously when they were originally trained with layer synchronization, showing that this mismatch leads to degraded performance or diminished energy and latency benefits. They further propose a solution that incorporates knowledge of asynchronous execution scheduling strategy, enabling the development of models optimized for asynchronous processing. The experiments conducted highlights that  event-based asynchronous  models can be an efficient alternative to conventional AI solutions.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) More details should be given around how asynchronous computation is achieved for the experiments. Fig. 2 is not entirely convincing. The authors should provide a more illustrative figure for explaining asynchronous communication. Preferably code should also be attached. \n\n2) More details regarding the simulations should be provided. \n\n3) Good to have: Additional experiments on convolutional architectures.", "broader_impact_concerns": "There are no strong concerns on the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "auIooNBx88", "reviewer": "Reviewer_t4gC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- The subject of this investigation is of clear interest to the audience of this journal\n- The paper is well structured, clearly written, and easy to follow\n- To the best of my knowledge, the presented algorithm is novel\n- Results showing the role of training asynchronous SNN to improve performance across two axes (both accuracy and energy efficiency) are convincing and could stimulate further research in this direction\n\n**Weaknesses**\n- Compared to conventional SSN, asynchronous SSN show more spikes when the network activity fully drains, but fewer if measured at the time the first spike is emitted by the last layer (Figure 3). While this observation is relevant for various SSN tasks, other tasks require measuring spike rate (rate coding) or the behavior of groups of spike (population coding), so I wonder how fair the \"on output\" metrics is. Can the authors comment on this point? In particular, how does the number of spike (~ energy efficiency) compares, as more outputs are taken into account, asymptotically approaching the \"in spiking done\" state?\n- Table 3 shows some energy estimates; it is my understanding that the energy numbers are derived by simply multiplying the number of synaptic operations (estimated using the asynchronous algorithm) by a fixed value of energy consumption per operation (extracted from a paper). If this is correct, it should be made more explicit that these numbers aren't on-chip measurements.\n- The main limitation of this algorithm is the significant overhead in training time (as properly acknowledged by the authors), which hinders its application to relatively small networks or restricted training configuration. What is the impact on inference time, both in terms of simulation and potential on-chip deployment?\n\n**Other comments**\n- DVS acronym (as in \"DVS sensor\") is first mentioned in the text in section 3.1.1 but only defined later in the Appendix", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an investigation into the impact of asynchronous processing in Spiking Neural Networks (SNNs). The authors set up a framework to simulate the behavior of SNNs in which neurons can emit spikes at any time, in an event-drive process that is independent on other conditions (such as waiting for all spikes to arrive at other neurons) that could delay the propagation. A two-stage backpropagation algorithm is used for training, which combines a conventional BPTT (backpropagation through time) approach, with a second unrolling that processes groups of emitted spikes. It is shown that, in such asynchronous SNNs, neurons tend to fire earlier and to emit fewer spikes than in conventional (layered) SNNs, provided the process is terminated when the first output is emitted. It is also highlight the importance of training the networks with an asynchronous process (instead of running inference on conventionally trained networks) to improve accuracy and lower spike density. Experiments are provided on 4 benchmarks (N-MNIST, SHD, DVS, CIFAR10), presenting spatial, temporal, or spatial-temporal features.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See previous section.", "broader_impact_concerns": "Not addressed in the manuscript. No concerns on my end.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JfCEsbWgPn", "reviewer": "Reviewer_d2oa", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: The work is clearly written, and the proposed method leads to several improvements in SNNs.\n\nWeaknesses: my biggest concern with this work is that the problem the authors postulate actually already has better solutions. For example, synchronous integer training and asynchronous spike inference. That is, assuming that spiking neurons emit integer spikes during training, they can be equivalently converted into asynchronous spike trains during inference. Speck's software Sinabs supports this approach (page 13 of the paper [1]). In addition, recent work has analyzed the advantages of synchronous integer training and asynchronous spike inference, which can simultaneously bring improvements in five aspects [2]: training consumption, inference power consumption, scalability, performance, and support for asynchronous neuromorphic chips. More importantly, doing so can support the training of large-scale SNNs and help them perform very well [3,4]. In this work, the authors' experiments are still tiny. What do the authors think of this?\n\n---\n[1] Spike-based dynamic computing with asynchronous sensing-computing neuromorphic chip. In Nature Communications, 2024.\n\n[2] Scaling Spike-driven Transformer with Efficient Spike Firing Approximation Training. In IEEE TPAMI, 2025.\n\n[3] Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-performance and Energy-efficient Object Detection. In ECCV 2024.\n\n[4] Spike2Former: Efficient Spiking Transformer for High-performance Image Segmentation. In AAAI 2025.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work presents and quantifies the Layer Synchronization problem in SNNs, then explores a potential solution direction, based on a generalization of backpropagation-based training that integrates knowledge about an asynchronous execution scheduling strategy for learning models suitable for asynchronous processing. Experimental results show that the proposed method can improve performance, inference speed, and energy efficiency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please see weaknesses.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)