Synchrony-Gated Plasticity with Dopamine Modulation for Spiking Neural Networks
Authors: Yuchen Tian
Samuel Tensingh
Jason Eshraghian
Nhan Duy Truong
Omid Kavehei
Venue: Accepted by TMLR ()

Abstract: While surrogate backpropagation proves useful for training deep spiking neural networks (SNNs), incorporating biologically inspired local signals on a large scale remains challenging. This difficulty stems primarily from the high memory demands of maintaining accurate spike-timing logs and the potential for purely local plasticity adjustments to clash with the supervised learning goal. To effectively leverage local signals derived from spiking neuron dynamics, we introduce Dopamine-Modulated Spike-Synchrony-Dependent Plasticity (DA-SSDP), a synchrony-based rule that is sensitive to loss and brings a synchrony-based local learning signal to the model. DA-SSDP condenses spike patterns into a synchrony metric at the batch level. An initial brief warm-up phase assesses its relationship to the task loss and sets a fixed gate that subsequently adjusts the local update's magnitude. In cases where synchrony proves unrelated to the task, the gate settles at one, simplifying DA-SSDP to a basic two-factor synchrony mechanism that delivers minor weight adjustments driven by concurrent spike firing and a Gaussian latency function. These small weight updates are only added to the network`s deeper layers following the backpropagation phase, and our tests showed this simplified version did not degrade performance and sometimes gave a small accuracy boost, serving as a regularizer during training. The rule stores only binary spike indicators and first-spike latencies with a Gaussian kernel. Without altering the model structure or optimization routine, evaluations on benchmarks like CIFAR-10 (+0.42\%), CIFAR-100 (+0.99\%), CIFAR10-DVS (+0.1\%), and ImageNet-1K (+0.73\%) demonstrated reliable accuracy gains, accompanied by a minor increase in computational overhead.

Published in Transactions on Machine Learning Research (12/2025)
Synchrony-Gated Plasticity with Dopamine Modulation for Spiking Neural Networks
Yuchen Tian"
sechen.tionnssydneee edw.eeeeeeddd.....
School of Biomedical Engineering, The University of Sydney, Sydney, NSW, Australia
Samuel Tensingh
samuel. wew.leeeeeeennnnghssssee.e...ee edw.eeeeeeeda..o
School of Biomedicnl Engineering, The University of Sydney, Sydney, NSW, Australia
Jason K. Eshraghian
jsn edu
Dept. of Electrionl and Computer Engineering, University of California, Sonta Cruz, CA, USA CIFAR Fellow
Nhan Duy Truong
whan@brrimnonneet.comm
School of Biomedical Engineering, The University of Sydney, Sydney, NSW, Australia
Omid Kavehei
omidkkaveheiBByynne...d edu.on
School of Biomedical Engineering, The University of Sydney, Sydney, NSW, Australio
Reviewed on OpenReview: https: o/perreveee net/ ftrmm? id- Ga4Qk6NtEP
Abstract
While surrogate backpropagation proves useful for training deep spiking neural networks (SNNs), incorporating biologically inspired local signals on large scale remains challenging. This difficulty stems primarily from the high memory demands of maintaining accurate spike-timing logs and the potential for purely local plasticity adjustments to clash with the supervised learning goal. To effectively leverage local signals derived from spiking neuron dynamics, we introduce Dopamine- Modulltee SSDP), a synchrony-based rule that sensitive to loss and brings a synchrony-based local learning signal to the model. DA-SSDP condenses spike patterns into a synchrony metric at the batch level. An initial brief warm-up phase assesses its relationship the task loss and sets fixed gate that subsequently adjusts the local update's magnitude. In cases where synchrony proves unrelated to the task, the gate settles onc, simplifying DA-SSDP to basic two-factor synchrony mechanism that delivers minor weight adjustments driven by concurrent spike firing and a Gaussian latency function. These small weight updates are only addod to the network's deeper layers following the backpropagation phase, and our lests showed this simplified version did not degrade performance and sometimes gave a small accuracy boost, serving as a regularizer during training- The rule stores only binary spike indicators and first-spike latencies with Gaussian kernel. Without altering the model structure optimization routine, evaluations benchmarks like CIFAR-10 (+0.42%), CIFAR-100 (+0.99%), CIFAR1O-DVS (+0.1%), and ImageNet-11 (+0.73%) demonstrated consistent accuracy gains, accompanied by a minor increase in computational overhead. Our code is available at
Plasticity (DA-
1 Introduction
Spiking neural networks (SNNs) depicted the third generation of artificial neural networks MMass 1997). Driven by the growing demand for energy-efficient computing and the rising interest in brain-inspired architectures, SNNs have gained traction in recent years with neuromorphic hardware and biologically
"Corresponding author.

---

Published in Transactions on Machine Learning Research (12/2025)
grounded research communities because of their event-driven nature and potential for significantly lowering
energy consumption (Amir et al. 2017 Davies et al. 2018 Friedmann et al. 2016
Merolla et al. 2014
By transmitting information through discrete spikes, SNNs enable sparse and asynchronous processing that aligns closely with the characteristics of brain-inspired hardware (Furber et al. 2014 Morolla et al. 2014
That said, their performance on complex and large-scale benchmarks still lags behind conventional artilicial neural networks (ANNs) (Hu et al. 2021 Shi et al.) 2024; Zhou et al.] 2022). This gap has led researchers to explore whether architectural techniques from ANNs, such as attention mechanisms from transformers, can be adapted to improve performance in spiking models.
Efforts to combine spiking dynamics with transformer-style attention mechanisms have recently gained momentum (Yao et al. 2024 Zhou et al. 2022). Several research groups have started to rework components of the transformer architecture, particularly self-attention, to better fit the characteristics of spike-based computation. One early example is Spikformer (Zhou et al. 2022), which replaces operations like softmax with compatile alternatives, resulting in competitive accuracy on image classification tasks. Building this foundation, Yao et al. (2024) proposed fully spike-driven transformer, while Spiking (Shi et al. 2024) introduced residual convolutional blocks into spiking attention framework, achieving 79.40% Top- 1 accuracy on ImageNet using only four time steps, narrowing the gap between SNNs and standard ANNs.
Despite rapid architectural progress, current spiking transformers are trained almost exclusively with surrogate backpropagation. What is missing is a scalable, duructure-wware local signal that can be injected during training without altering the architecture. Conventional regularizers (e.g., weight decay (Krogh & Hertz, 1991), label smoothing (Szegedy et al., 2016), dropout (Srivastava al. 2014) Joperate on weights real- valued activations and largely ignore the event structure that determines computation in SNNs. In contrast, timing-baseen baste plasticity rules accurately capture certain neural behaviors. However, when simply adapted to deep, multi-layer networks, they demand detailed temporal tracking and extra memory for states, Although Tian et al. (2025) provides a preliminary exploration of synchrony-based signals, its adjustments are not aligned with the supervised learning signal, which hinders a safe and scalable training-tine implementation.
In contrast to most artificial systems, biological brains combine both supervised and unsupervised forms of learning (Storrs et al.] 2021 Hennig al. 2021]. Synaptic changes are typically driven by local spike activity patterns [Feldman 2012), and these changes can also be shaped by neuromodulatory signals. Dopamine, for instance, plays a central role in linking synaptic modification to reward feedback (Frémaux & Gerstner 2016 Speranza et al. 2021). Inspired by this mechanism, we propose integrating biologically motivated, self self-organizing plasticity into spiking transformers to enhance their learning capabilities and generalization. Previous research indicates that Hebbian plasticity (Hebb) 1949) can effectively support continual learning and unsupervised feature extraction in SNNs (Xiao et al. (022)). Yet, current spiking transformer implementations do not incorporate such biologically inspired learning mechanisms.
In this work, we present a novel training strategy for spiking transformers, integrating supervised gradient- based training with an unsupervised, moduuddddd plasticity mechanism. We name this method Plasticity (DA-SSDP). The fundamental idea is to monitor taikee-tmminggao synchrony during training and dynamically adjust synaptic weights in response to task-specific loss. Essentially, DA-SSDP encourages synapses that contribute to synchronized spikes correlatod with correct predictions, and penalizes those associated with poorly timed spikes, mimicking the modulatory role of dopamine observed in shaping neural coordination and learning. Through this additional plasticity mechanism, aim to (i) capture subtle spike synchronous correlations potentially overlooked by gradient descent, (ii) regularize neural activity to reduce uncoordinated spiking, and (ii) explore how synchrony-based local plasticity learning mechanisms can be integrated with global learning signals to improve generalization without imposing additional computational cost.
This paper presents the following primary contributions:
(A) Bringing synchrony- signal into model learning: DA-SSDP injects a batch-level, bynchrony-based complementary learning signal during training. DA-SSDP departs from Spike- Timing-Dependent Plasticity (STDP) by screening for co-firing synchrony before any update; only
2

---

Published in Transactions on Machine Learning Research (12/2025)
then is a bounded Hebbian adjustment applied, independent of pre/post order. A brief warm-up phase derives dopanine-inspired scaling factor from synchrony and loss. After that, the gate stays fixed and simply adjusts the size of a Hebbian-style local update for each batch. When synchrony is task-irrelovant, the gate collapses to unity and the rule safely reverts to the two-factor bascline (no online loss), providing a scalable, structure-aware signal without architectural changes.
(B) Scalable training-time synchrony update for deep SNN-transffrmors: DA-SSDP requires only binary spike indicators and first-spike latencies, adding only D(CoueCin); where Cin refers to the number of pre-synaptic channels and Cout refers to the number of post-synaptic channels of the module to which DA-SSDP is attached, with per-batch clement-wise operations and no inference-time cost. We provide a drop-in implementation that tweaks just the final classification part and a simple late-stage feature projector, keeping the architecture and training schedule (epochs, batch size, optimizer) unchanged. This approach saves a lot of memory compared with storing detailed spike-timing records.
(C) Quantitative validation and robustness analysis with safe degradation: Across CIFAR- 10/100, ImageNet-1KK, and an event stream (CIFARIO-DVS), DA-SSDP shows consistent accuracy gains under the same training setup. On the test sel, the median batch synchrony S, increases from 3x10 to 1.0x10-2 (~ 33x).
2 Related Works
Spiking transformers and training-time cost: Recent SNN transformers (Zhou et al., 2022
2023
Shi
ct al. 2024 Yao al., 2024 Zhou et al. 2024a Yao et al., 2024) primarily target inference-time time efficiency by exploiting sparse spikes and few time steps while maintaining competitive accuracy on image benchmarks. However, training remains almost entirely surrgatee backprop (Zhou et al., 2024a Gygax & Zenke 2025 Hu
ct al. 2024), whose unrolling, state storage, and memory traffic can be comparable to, or even higher than ANN counterparts (Zhou et al.) 2024b). Architectural progress, therefore, narrows the inference gap but does not by itself provide a scalable, structure-aware training signal that interacts with spike statistics.
Local plasticity at scale: Biologically motivaled rules range from two-factor timing/ /race updates to three-factor modulated variants (Mazurek al. 2025). In deep, multi-layer neural networks, fiming-based plasticity rules run into two ongoing problems. First, scalability is hindered by keeping detailed records of spike timings and assigning /blame between pairs of neurons over time, which consumees a large amount of memory and processing bandwidth. Secondly, the update directions are not coordinated with the supervised loss and may conflict with gradient signals. Consequently, most demonstrations remain on small-scale tasks or online/RL settings [Mazurek et al.) 2025 Zhou et al., 2024b). Along two useful axes, local signal (pairwise timing 88. population synchrony) and global modulation (absent vs reward/ paarr//predettio-erroo largely emphasizes pairwise timing, sometimes gated by a neuromodulator, while underusing population-level coordination. Yet neuroscience highlights population synchrony a description of coordinated activity Vinck et al. 2023: Majhi et al. 2024), whereas synchrony has rarely been used scalable, supervised training-time signal in deep SNNs.
prior work
We fill this gap by using batch-level spike synchrony as a low-memory substitute. It relies on binary spike indicators and first-spike latencies with (CCott per batch. We then adjust its local influence using a dopamine-inspired gate, derived from synchrony and loss during a brief warm-up period. Once set, this modulation remains constant and merely scales a Hebbiau-style local update. As a result, the rule enhances backpropagation without modifying the forward pass dynamics. When syuchrony is uninformative, the gate converges to one, simplifying the update to harmless two-factor baseline that's applied after the main updates in deeper layers. This provides the scalable training signal that prior spiking transformers have been missing. The formulation and implementttion are detailed next.
3 Method
This section introduces the proposed DA-SSDP mechanism and its role in enhancing the training of spiking transformer models. We first describe the DA-SSDP rule itself, then explain how it is integrated into the
3

---

Published in Transactions on Machine Learning Research (12/2025)
a
b
Synchrony Gate (A)
Dopamine
Nauron-1
synchrony pais An1,
post
Neuron-2
Neuron-3
asynchrony pair AnD,
Neuron-4
Spike Synchrony
AAW
Silent Neuron
Spiking Neuron
AW
DA-S5DP
A
.... SSDP
co-fire
potentiation
Loss
co-fire
depression
Figure 1: Overview of DA-SSDP (a) Pre-poss layer and binary activity indicators from the current mini-batch. (b) Synchrony gate X - A pre/post pair that co-fires in the sample sets À = and is potentiated (red), no co-firing sets À = 0 and yields depression (cyan). The first-spike lag A is passed through a Gaussian window = exp(- - At? (2)")) which scales the magnitude only and the rule is arder-invariant (synchrony decides the sign, timing sets the strength). (c) Dopamine modulation During a short warm-up, a single scalar slope is fit from the empirical synchrony-loss corrclation and after warm-up, the slope is frozen. Therealter, the gate depends only on batch synchrony and simply rescales the local SSDP update (solid blue: DA-SSDP; dashed: ungated SSDP).
SpikingResformer backbone and trained in a two-stage manner. Fig- D summarizes DA-SSDP. For each pre/post pair within sample, binary synchrony gate X E 1) indicates whether both neurons fired at least once, co-firing (A 1) leads to potentiation and co-firing (A =) 0) leads to depression (Fig, I [(,b)). The absolute difference in first-spike times, |At), feeds into a Gaussian function g = exp(-Atf /220((20)) (2a")) to adjust only the magnitude of the update. The sign comes from the synchrony itself, ensuring the rule does not depend on the order of spikes. Summing up Agb.i across all connections produces a synchrony measure for the entire batch. During a short warm-up, we a single negative slope that captures the inverse relationship between synchrony and loss, and then freeze it. From then on, this factor serves as scalar that rescales the local SSDP update without changing its direction (Fig. 1 [c).
3.1 Soaamine-Modulateer SSDP Mechanism
We use mini-batch of size B, where samples are indexed by E (1...., B}. For a hooked module with On input channels and Cout output channels, we index pre-synaptic channels as j E ......,,m] and postsynaptic channels E (1,..., ). Cout) Let P be the binary pre-synaptic spike matrix of shape P E {0, Bx and Q the binary postsynaptic spike matrix of shape Q E {0, BxBx for a mini-batch of size B (each row resembles a sample, and a column represents a channel). In the implementation, these indicators are obtained by thresholding layer activations during the forward pass and keeping only whether a channel fired at least once in the window, which yields a stable binary co-activation signal that is cheap to cache and process at scale. For every (j,i) we also record the first-spike latency Als.i. /post spre Only the first spike time is stored, minimizing memory use while retaining the core temporal signal. Channels that remain silent throughout the window receive à t= T, where T is the window length in time steps. The first spike has been shown to carry most of the critical information, whereas subsequent spikes contribute little additional content and may even introduce redundancy (Thorpe al.) 20011 yan Rullen & Thorpe 20011] Gollisch & Meister 2008). The forward pass provides these timestamps directly as tensors frrepnost enabling on-the-spot computation of At without. needing to rerun the sequence.
4

---

The per-sample plasticity change for a given sample and connection pair (i.j) is defined Published in Transactions on Machine Learning Research (12/2025)
We now outline the fundamental update mechanism of DA-SSDP. For mini-batch, the weight adjustment between pre-synaplic j and and post-synaplic i is given by
AW clip
Σ
Go Ihiid (Atdiis (1-1-Aap )osss) -1, 1)
(1)
B
where Ab,ú, = QbiPos E {0,1) signals co-spiking between the post- and pre-synaptic channels, Ib.i.j exp [- 10-A2/(22) [ is Gaussian kernel with learnable bandwidth o > 0, and A+, A_ > 0 control the degrees of potentiation and depression, respectively. The batch-specific scalar gate G. is detailed further in the warm-up calibration section (Eq. 10
This equation highlights three key elements of DA-SSDP. First, the binary synchrony gate ensures that only coincident pre/post events contribute to plasticity, grounding the rule firmly in spike synchrony. Second, the temporal kermel g(At) prioritizes updates for near synchronous events while suppressing those with larger delays, reflecting how spike timing influences synaptic strength. Third, the Loss-aware modulation G. injects loss-dependent signal, enabling the local plasticity rule to remain compatible with global task objectives. Together, these elements produce weight increments that are clipped to a bounded rango, preventing instability and cnsuring stable training in deep architcctures.
3.1.1 Instantaneous Potentiation and Depression
Gaussian weighting: For each pre- pee-post pair, we apply a continuous Gaussian kernel
Jb.i.j = cxp [
Ar 20
(2)
where a is a learnable bandwidth parameter that's shared across the module. Since decreases with | updates are strongest for nearly synchronous spikes and taper off as the pre-poss latency widens. Using a continuous kernel avoids fragile outcomes tied to rigid thresholds and maintains effective gradients during combined training with backpropagation.
Synchrony gate: Co-activation is represented by binary mask
Ab,s.j = Qb, E 10,
(3)
Here A computed via an outer product QPT for each sample (i.e., for sample b, Qo E Ru.. and x1,,,, we transpose P. 1 Op to form the outer product Q&P,T yielding Cout On mask) OCCout Cin) complexity and no need for temporal scanning, because first-spike times computed once per channel (not per pair) and At is formed by broadcasting (Post and prre, avoiding any O(TCout Cin) per-pair search. Consequently, the combined factor Ab.i. functions as gentle synchrony sensor, and it remains strictly positive only when the pair spikes together within the window, and its value adjusts according to temporal closeness through g.
Weight update:
as
Awb.i.j = Sh. (AA + A-) Ao,is - -A.).
(4)
lcading to potentiation when À = 1 and depression when X = 0. In cases where both units spike, A + Aw = + g. Spikes that are closely timed (A/x0) produce the strongest, potentiation, with bigger delays diminishing it. When only one unit spikes, À = 0 + Au = - g. For both silent: = and p -7 = At 0, so g == and A A_. Coactive synapses are reinforced according their temporal alignment, while inactive pairs are mildly suppressod, which limits accidental co-firing and improves generalization without maintaining eligibility traces for each synapse.
5

---

Published in Transactions on Machine Learning Research (12/2025)
3.1.2 Batch-level Synchrony Score
We combine spike occurrences into a single scalar synchrony value
Sb Cout Cin
-QodPas=(00P3))) (QuP)))
Dos.Poo = 2.
(5)
which ranges in [0, 1] and measures how synchronous the two populations are sample b. In the implementa- tion, this is calculated as the average of the outer product over the [i,j) dimensions. The division by Cout ensures Sd can be meaningfully compared between different layers or architectures with varying channel sizes. The definition is permutation-invariin with respect to channel ordering and depends only on binary co-firing, which makes robust to activation scale and batch normalization statistics. Relying on co-firing rather than full At details, the signal maintains low variance over time windows, while At still influences the individual updates through the Gaussian kernel.
3.1.3 Warm-up and Gate Calibration
During the Ewarm epochs, DA-SSDP does not modify weights. Instead, it collects per-batch pairs (Sb, (b), where S₁ is the batch-wise synchrony metric and is the supervised loss for that batch. These pairs are aggregated across all warm-up batches to form two aligned sequences.
Let {5 and {6 represent the collected sequences from the warm-up phase (length N). We calculate their means and standard deviations as follows,
Standardization:
1
1
Us
Sb
Ss us)?
(6)
N
N
N
1
14
Pb:
(es 12e)2
(7)
N
N
1=1
b=1
and derive the normalized values
So
Sb
us
8
Eb kee
(8)
ØS
σt
Slope fitting: The dopamine slope sot by the negative empirical correlation between standardized synchrony and loss:
k = -E Eo]
1 N
Σ S,L.
where Es|:] signifies the empirical mean across the warm-up collection. In essence, a pattern higher synchrony pairing with reduced loss produces 0, boosting weights for future high-S, batches. feeble connection yields k as 0, rendering the gate essentially impartial.
Gate at training time: After warm-up, (us,os, k) remain fixed. For each new batch, compute the scalar gate.
G. = clip So ps .02).
(10)
0s
which is uniformly applied to all synaptic pairs (i.j) within the batch, scaling the per-sample term in Eq. 1 The clip(æ, b) == min (max (ar a), b} keeps the modulation bounded and prevents explosive adjustments.
Robustness: If the warm-up data oflers limited insight, for example, if the batch synchrony Sn has a very small dynamic range (leading to small os) or negligible correlation with loss (so ze 0), we fall back to a neutral gate via k=0 with standardized S.. This socurely deactivates the dopamine component without disrupting the overall training flow. For a practical demonstration, consider the CIFAR1O-DVSS example in where S₃ exhibits minimal variation during warm-up, yielding fitted slope k 0 with G, ss
Sec. 4.2

---

Published in Transactions on Machine Learning Research (12/2025)
Token
Projection
Conv
MatMul1
MatMul2
Wproj 1x1 Conv
DA PSPS model
BN
Pepprd
Prologue
DSSA
GW-FFN
DSSA
GW-FFN
DSSA (DA- PSDP
GW-FFN
Avg-Pool
classifier (DA (DA-SSDP)
Stage
Stage
Stage
Avg-Pool
Classifter
DA-SSDP model
recordipost
Figure 2: DA-SSDP integration points in SpikingResformer The DA-SSDP module is inserted at two locations (1) the 1x1 projection convolution of the last DSSA block in Stage (2) the linear classifier following global average pooling. During training, each hook records pre-/post-spike activity and applies the DA-SSDP update after the warm-up phase, operating alongside standard back-propagation.
3.1.4 Complexity and Stability
All DA-SSDP computations reduce to broadcast outer products, element-wise exponential for the Gaussian kernel, scalar standardization, and simple reductions over Boolean tensors. Overall, the effort grows O(B Cout Cin) for each mini-batch, primarily involving efficient vectorized tensor operations that align seamlessly with GPU acceleration. No synapse-specific cligibility traces persist between steps, with only first-spike timings and binary indicators stored from the forward pass; thus, the maximum memory burden consists the batch-level accumulator AW alongside minor per-batch values (Sp, Ps, og.k). After calibration, the gate parameters (k, JAs. os)) are fixed, whereas A+, A-, and a continue be trainable. This separation ensures numerical reliability in the dopamine-ni dopamine-modulated pathway and prevents fluctuations towards the end the force behind convergence, while DA-SSDP contributes additional, task-relevant information by coupling local population synchrony So to the global objective through the dopamine gate G,, This coupling reinforces synchrony only when it predicts lower loss, guiding the network toward more stable firing regimes. Otherwise, the gate neutralizes, and the update reduces to the two-factor SSDP bascline.
of training. Backpropagation contimues
3.2 Implementation Details
Integration of DA-SSDP into SpikingResformer We integrated DA-SSDP into the SpikingResformer architecture, illustrated in Fig. 2 The model follows hierarchical backbone comprising (i) front-end downsampling stem (Prologue) that maps the input image to an initial spike fcature map, (ii) spiking self-attention layers (DSSA), (iii) grouped convolutional feed-forward blocks (GWFFN), and (iv) a spike-based classifier at the output stage. We insert two DA-SSDP modules as light- post-update hooks without changing the forward path (i) the linear classifier and (ii) the 1x1 projection in the last DSSA block. At this hook, we consider the DSSA input channels as presynaptic and the projection outputs as postsynaptic. In every training iteration, we begin with the usual surrogate-gradient update. Next, using the current.

---

Published in Transactions on Machine Learning Research (12/2025)
mini-batch, generate binary activity markers and per-channel first-spike timings at the two hook locations, invoke DA-SSDP to calculate additive weight adjustment AW, and apply it directly to the relevant. weights. During the warm-up phase, < Éwarm where denotes the current epoch and Ewarm denotes the maximum epoch of the warm-up phase for computing the DA gate, DA-SSDP only records statistics without making any weight modifications. For the classifier hook, we extract pre- or post-activity from the per-step readout activations, and for the DSSA hook, channel counts as active a time step if any of its spatial positions is active, with the first-event timing to the earliest step where the indicator activates. The two DA-SSDP instances maintain separate parameters and warm-up data. This setup preserves attention maps and all forward-pass operations intact, with DA-SSDP simply providing minor post-update adjustment driven by batch-level synchrony.
Table 1: Top-1 Classification accuracy, Values are reported as mean (#) standard deviation over five runs.
Energy is the estimation of energy consumption, same in Appendix A.2
Yao al. 2024 Zhou al. 2022 Shi et al. 2024
Dataset
Method
Architecture
Param (M)
Energy (md)
TimeStop
Accuracy (%)
CIFARIO
Splidormer ZZhon et nl.
2022
Spikformer 4-384
9.32
90.51
Spilcingforreer Zhou
Spitingformer
96.61
Spike-drive Wmmatteme
Spile-ddilvnn Transformnet
95.8
SpikingResaarmn Shi ml
A021
10.83
96.05
SpikingResfarmee +SSDP 177
DA
Spiking Restome:-CIFAR
10.83
96.16 # 0.09
Spikinggessormeer-
CIFARION
CIFARIO-DVS
Spileformer
Spikformen-43364
0.32
16
80.6
Spikingformer
Spikingformer
16
81.3
Transformer
Tranformer
Tronsformet-44 4-384
9.32
81.022
Spilaingeesbrmerr
Spiking/tesformee CIFAR
11.31
2.403
10
84..4
SpikingResformer
SSDP DA)
Spiking Resooroeoooorreeeeeee
17.31
Spikinggesformee-H
Spikinng Spiking Ree
17.31
2.458
CIFAR100
Spiidormer
Spikformer 4-384
9.32
77.
Splkingfarmer
Spikingloruxe
71:001
Spile-driven Transformer
Spiko-driven Transformer
78.4
Tranffrmmer
Thareformen-43366
9.32
81 .02
SpilinggRessmmer
Spiking/Reffomme CIFAR
10.63
0.493
78. 78.75
SplkinggRestoreer DA)
SpikingRestormees
79.20
Spikinggessorrmee--AAA
Spiking/esformee CIFAR
10.83
0.494
79.48 0.24
ImageNet
Spiicing ResNet ef 2021
ResNet 50
25.56
70.934
350
72:75
Tresfones (Deavisaly
Tharaformer---11
Splidarmer
Spilformes 8-384
18.81
1.734
70.24
Spikformer
Spikformer--- 768
66.34
21.477
74.81
Spikinngfrmer
$pikingformee-8881
T4.70
Spikingfarmer
$plkingforme--8-000
13.68
Spilae driven Transformer
Spike driven Transformer-3
68.34
6.09
76.32
SpiringRRessmmer
SpikingResdorme Redformer-1,
60.36
8.76
78.77
Splking/Testorroe:: DA)
SpikingResformes
DolkingResfomsees
60.38
8.76
79.29 =: 0.21
4
Experiments
We design our experiments to systematically evaluate the effectiveness of DA-SSDP in improving performance, refining temporal feature encoding, and energy cost in spiking transformer models. All experiments are conducted using publicly available vision datasets. Each experiment. repeated five times to estimate the variance. To maintain fairness, we preserve identical model structures and training regimens across variants, except where otherwise specified.
4.1 Experimental Setup
Our primary baseline is SpikingResformee Shi al. 2024). We keep the architecture and training schedule identical to this bascline and only attach DA-SSDP during training, so the comparison isolates the effect of our method. For a broader conlext, , we also list. representative results from earlier SNN-transformer variants
(Zhou et al. 2023 2022 Yao et al. 2024 Shi et al. 2024) as published. DA-SSDP uses a warm-up of Evarm =00 for CIFARIO-DVS, Ewarm 80) epochs
the gate, and the fitted gate is then kept fixed for the remaining epochs. Kernel parameters are A,=1.5x10 10-3 =11.00x00 and a learnable a. Backbones
follow Spikingresformer with T=4 steps CIFAR-10/100 (Krizhevsky et al. 2009) and ImageNet-1k (Deng

---

Published in Transactions on Machine Learning Research (12/2025)
et al. 2009 and T=10 on CIFAR1O-DVS (Li et al 2017 More training details are in Appendix con configuration is repeated five times with different seeds.
A.1
Each
4.2 Results
Table summarizes the results. With identical model structures and comparable training configurations, DA-SSDP yields consistent performance boosts: +0.42% CIFAR-10, +0.99% CIFAR-100, and +0.73% on ImageNet-lk. Model size and inference cost are unchanged; the rule adds only train-time computation at the hooked layers. We further isolate the regularization effect with a 200-epoch (100-epoch warm-up + 100-epoch gated phasc); see Appendix A.3 for gap and validation-advantage analyses and Appendix A.9 for further experiments.
On CIFARIO-DVS: CIFARIO-DVS created by showing static CIFAR-10 images to a DVS sensor while moving the screen camera, which produces event streams dominated by edge-related contrast changes. After the first convolution and pooling layers, the neural activity becomes quite sparse in time, and many pre- and post-channel pairs never fire together within the T' steps, the synchrony gate À is often zero. Even when both sides do fire, their first spikes are usually far apart, making Att large and the Gaussian weight ((() close to zero. Channels that stay silent. contribute small negative updates through the -A-g term, which pushes weights down even further. At the batch level, the synchrony score S, varies only slightly across batches, so its correlation with the loss weak. This makes the fitted slope close to zero, and the dopamine gate essentially neutral (G, = 1). In practice, DA-SSDP then behaves almost like its basic two-factor version, with only small net updates. This explains why the improvement CIFARIO-DVS is modest compared to based datasets, and shows that the rule is most effective when the data provide strong, task-relevant spike synchrony.
4.3 Ablation Study
To quantify the contribution of the proposed rule, we compare
Baseline: the spiking transformer trained only with surrogate backprop.
SSDP (tctoractor): local synchrony-based updates without dopamine gating.
DA-SSDP Sturee-factor). the full method with a dopamine- gate.
4.3.1 DA-SSDP Integration in the Model
We further examine the rule's optimal placement by applying it to (i) the linear classifier only, (ii) the DSSA block only, or (iii) both components at the same time. No further hyperparameter adjustments are made for each configuration.
Under identical hyperparameters, the baseline reaches 78.73% Top-1. Table 2 reports the effect of where the local rule is attached. Applying the update in the Prologue convolution destabilized training in our setting (diverged under the same schedule). By contrast, attaching the rule to the last DSSA block yields the largest gain (+0.58% for two- factor SSDP: +0.87% with the dopamine gate), while adding it only to the linear classifier gives a smaller but positive gain (+0.23%). Using both hooks gives the best result. (79.72%, +0.99%). No extra tuning was performed per placement, and the warm-up- firttup-fitted gate kept fixed thereafter.
These observations are consistent with the mechanism of DA-SSDP. The rule acts post-update, synchrony- based local adjustment whose benefit depends on the presence of task-aligned co-activation. Such co-activation is more reliably expressed in mid-to-late representations than in very early feature extractors, where activity is sparser and more input-driven, which explains why the last DSSA benefits most while the Prologue is unstable under the same hyperparameters. Across placements, the dopamine gate further improves over the two-factor variant, indicating that scaling the local update by the synchrony-loss relationship learned during warm-up is helpful for learning. Overall, DA-SSDP enhances model performance by injecting local synchrony signals into deeper layers and scaling them through the correlation with the global backpropagation loss.
9

---

Published in Transactions on Machine Learning Research (12/2025)
Table 2: Effects of integrating SSDP into different parts of the model on the CIFAR-100 dataset. Adding SSDP to the DSSA module yiclds the largest accuracy gain (+0.58%), whereas integrating it into the classifier improves performance by +0.23%. When placed in the Prologue, however, the model fails to converge.
Position
Accuracy (%)
Improvement (%)
Baseline (No SSDP)
78.73
Prologue+SSDP (w/o DA)
Divergent
Prologue+DA-SSDII
Divergent
DSSA (w/o DA)
79.31
+0.58
DSSA-DA-SSDP
79.60
+0.87
Clasiffer+SSDD (w/o DA)
78.96
+0.23
Classffer=DD--SDDP
79.12
+0.39
DSSA+Classifer+ (w/o DA)
79.48
+0.75
DSSA+ Classifier classiCCe DA-SSDP
79.72
+0.99
Table 3: Performance under different A4 and A_ settings. The highest improvement is achieved with A, 0.00015 and A_ 0.00005.
DA
A+
A_
Accuracy (%)
X
0.0015
0.0005
78.86
0.00015
0.00005
79.48
X
0.00010
0.00010
79.03
0.001
0.001
Divergent
0.0015
0.0005
79.72
0.00015
0.00005
79.40
V
0.00015
0.0001
79.22
0.001
0.001
79.21
4.3.2 Impact of SSDP parameters on Convergence and Performance
To better understand how DA-SSDP handles paramcter variation, we compared it against both the bascline and a variant using SSDP alone. We vary the polentiation polentiation//leprssiio /depression amplitudes (A+, A-) and compare the factor SSDP to the three factor DA-SSDP with a fixed gate (learned during warm-up and then frozen). No additional hyperparameter retuning performed per sctting.
Hyperparameter Sensitivity to A-+,A-): Table highlights three key patterns (i) under high am- plitudes AA+=A==10 3), the two-factor SSDP fails converge, but DA-SSDP achieves 79.21% accuracy, demonstrating better stability with extreme hyperparameter; (ii) moderate amplitudes, DA-SSDP matches slightly exceeds SSDP (c.g., 79.48% 79.40% at 1.5x10 11111=1110011 (fii) the best accuracy is obtained with larger potentiation setting (A_=1.5x10- A_=5x 10-A), where DA-SSDP reaches 79.72% versus 78.86% for SSDP.
Interpretation: These findings align with the dopamine gate's function in our implementation. After warm-up, it delivers a stable, batch-specifie rescaling G,=1 + k: Sr, for the local adjustment, which adjust the per-batch update magnitude and reduces sensitivity to overly large (A..A_) without changing the update direction or the forward computation. When k=00 (weak synchrony-loss correlation), the method reduces the two-factor baseline. Overall, DA-SSDP enables safe use of larger potentiation rottiaationddpeesssion depression amplitudes and achieves equivalent or better accuracy within an identical training configuration.
10

---

Published in Transactions on Machine Learning Research (12/2025)
0 t=1
À feature per-patch
0
0.
1
-4
2
3
3
4
5
6
0.2
7
-1
0
2
4
6
(a) Bascline attention maps (t=1~4)
(b) Per-patch feature gain induced by DA-SSDP
Figure 3: Spatial effect of DA-SSDP in the last DSSA stage (a) Bascline attention maps (averaged over heads; t= 1~44 on the same test image. (b) Per-patch feature-gain A(u, v), computed as the absolute change of the post-projection activation between the DA-SSDP and baseline checkpoints and averaged over time and channels (brighter larger change). Hotspots in A align with high-attention areas from (a), suggesting a re-weighting of already attended tokens rather than wholesale redirection.
4.4 Visualization and Activity Analysis
To explore the effect of DA-SSDP on temporal feature organization, we used several types analysis. A qualitative (-SNE view the lest embeddings, with setup and plots, is provided Appendix [A.4
Spatial feature gain analysis: We examine how DA-SSDP modifies spatial evidence within the final DSSA block using typical test image. Fig. 3(a) shows the baseline attention maps (averaged across heads and time steps). Fig. b) depicts per-patch feature gain map, calculated as the absolute difference in post-projection activations between the final DA-SSDP model and the baseline version, as detailed in Eq. 11
A(u, v) TC
1
Σ Ne
(WDA (PAVAA) (WBage ABsso VBBaso) Base proj
(u,
(11)
Hotspots in A(u, correspond to high attention zones in Fig. B(),, aligning with our design. The local update connects the attention layer, thereby preferentially recalibrating tokens with already prominent signals. In our implementation, DA-SSDP is attached only to the last-stage 1x1 projection Wproj (and to the classifier); the attention operators remain unchanged. The update is computed at the channel level after aggregating spikes across spatial locations, so the gate G5 and the synchrony signal do not encode where the image a spike occurred. Consequently, the spatial map A(u, larger regions where the pre-projection features A,V, alrcady have higher magnitude (i.c., higher attention mass): the learned change in Wrroj reweights the existing channel mixture, and this effect is amplified where features strong. This explains the co-location of hotspots with object regions in CIFAR-100 and supports claim that DA-SSDP re-weights already attended tokens rather than redirecting attention.
demonstrates how DA-SSDP reorganizes the network's spiking behaviors. the baseline network, spike frequencies vary broadly: many neurons fire two or three times within the four-step window, and small subset fire all four steps, leading dense spike patterns. After introducing DA-SSDP, neurons crucial for decisions become more prone to simultaneous activation, and as a result, these channels reinforced and maintain elevated spiking rates, while most other neurons emit. at.
Spike Activity and Rate Analysis:
Fig. f
11

---

Published in Transactions on Machine Learning Research (12/2025)
Raster (Baseline)
Raster (DA-SSDP)
500
500
400
400
Nounsn
200
200
0.0
1.0
Tinestep
2,0
3.0
0.0
1.0
Tinestop
2.0
3.0
Deseire DA-00P
Spike per neuron
Spla
100
200
400
500
Figure Temporal spike patterns shaped by DA-SSDP Raster plots (top) and per neuron spike counts (bottom) for the first 500 LIF /PIIF channels over T = 4 time steps, evaluated representative CIFAR-100 sample. Baseline activity is dense and widely spread, with many neurons firing several of the four time steps and a small subset reaching the maximum count of four spikes. After DA-SSDP training, a compact synchronous burst emerges around indices 30 120, whereas the majority of channels emit at most a single spike or remain silent, yielding a markedly sparser profilc.
Baseline (+0.0028)
DA-SSDP (+0.01)
0.5
0.95
0.2
Normalized Sp
0.1
0.9
0.05
0.85
0.02
0.01
0.8
0.75
0.7
Figure 5: Batch-level synchrony boxplot Each dot is one mini-batch. For batch b, we mark a channel active if it fires at least. once within the window T, and define the batch synchrony score S, as the fraction of channel pairs that spike the same time bin. Relative to the vanilla baseline, DA-SSDP shifts the distribution upward and tightens its spread (median 3x + Ix10-2, <33x). This pattern is consistent. with the loss-aware gate Gg, which promotes fask-aligned spike coincidences and suppresses asynchronous activity, leading to more coordinated and stable population activity.
12

---

Published in Transactions on Machine Learning Research (12/2025)
most one spike or stay inactive. Taken together with the results in Table DA-SSDP strengthens neurons that convey useful information while suppressing those offering minimal value or just adding noise.
These results confirm that the proposed plasticity rule not only reshapes temporal dynamics but does so in a manner that is aligned with the model's performance.
Loss-aligned modulation via synchrony covariance: Using the gate Go in Eq. 10 with the slope k from Eq. g DA-SSDP is expected to up-weight batches whose synchrony S. predicts lower loss and to stay neutral otherwise. Fig. provides the mechanism-level signature of this behavior. Relative to the vanilla model, the entire distribution of Sy on the test set shifts upward and contracts, with the median increasing from 3x to 1x10-2 (x=33x). This pattern indicates (i) more frequent task-aligned co-activation and (ii) reduced batch-to-batch variability in synchrony. Across random seeds, we observe the same qualitative shift. Taken together, the plot validates that the learned gate injects population-level information into the update stream and steers the network toward temporally stable firing regimes.
5 Discussion and Conclusion
Loss-modulated synchrony as scalable training-time signal. DA-SSDP incorporates a group-level signal into the training updates. Batch synchrony S acts as rough gauge of task-relevant simultaneous firing and impacts training exclusively via the gate G₁ and is inherently tuned to the supervised goal. As such, backpropagation continues as the core optimizer, whereas DA-SSDP adds a supportive inclination towards reliable, low-noise spiking dynamics in channels prone join activation on well-predicted examples. This inclination disappears at inference time and preserves the forward-pass operations intact.
Effect size tracks task-aligned co-activation in deeper layers. The rule most effective in mid late representations where task-aligned co-activation reliably expressed. Applying the update to very early layers can be brittle under the same hyperparameters, consistent with sparser, input-driven activity in those stages. The dopamine- like gate fitted during warm-up rescales the local update per batch, which improves stability and enables the safe use of larger potentiation and depression amplitudes.
Boundary conditions and stability under weak synchrony: The gains rely the availability of synchrony linked to class-specific patterns. When the synchrony-loss correlation is weak, as observed on CIFARIÓ-DVS, the fitted slope collapses toward zero, the gate becomes neutral, and DA-SSDP reduces to its two-factor baseline with minimal elfect. These properties clarify where the mechanism proves useful and ensure a safe degradation when synchrony offers little insight.
Deployment in neuromorphic hardware or custom ASICs. DA-SSDP serves as a training-ting regularizer and is ideally suited for offline use on GPUs to generate fixed weights ready for deployment. Although a straightforward silicon implementation would demand floating-poit operations, latency storage, and custom accumulators, these components align well with neuromorphic hardware concepts. Local aspects, like detecting spike coincidences, function on per-synapse or per-neuron-pale basis, relying on straightforward binary spike markers and time offsets-thase can be processed efficiently neuromorphic processors, capitalizing on their built-in capabilities for event-triggored synaptic modifications. Additionally, the global modulators resemble biological neuromodulatory transmissions, like dopamine releases, which can theoretically be realized and optimized on ASICs through basic scalar multiplications. By concentrating spikes into fewer, more synchronized channels and reducing incidental activity, the learned weights can indirectly improve efficiency on event-driven hardware. Detailed assumptions and quantitative estimates are provided in Appendix A.5
Limitations and future directions. While DA-SSDP shows promising improvements in accuracy, temporal coding, and estimated energy efficiency, a number of limitations remain. We outline some limitations of our proposed DA-SSDP, which currently have under investigation and improvement.
13

---

Published in Transactions on Machine Learning Research (12/2025)
(A) While our experiments demonstrate performance gains standard vision benchmarks (reported mean and standard deviation over five runs), a broader validation with larger sample sizes is needed to strictly establish statistical significance. In particular, evaluations on neuromorphic hardware platforms like Loihi or SpiNNaker could provide stronger evidence of generalization and practical utility.
(B) On datasets such as DVS, spike counts are extremely sparse and often distributed across wide temporal spans. Reliable performance typically requires larger T to capture sufficient information. In such regimes, the syuchrony signal of DA-SSDP is suppressed by the Gaussian kernel, leading near-zero incremental benefit over weights.
(C) We acknowledge that while DA-SSDP is motivated by biological plausibility, the introduction of direct loss-derived global signal diminishes strict biological aligmment. An interesting direction for future work is to explore alternative modulatory signals inspired by biological dopaminergie pathways, which may better balance biological compatibility with task-driven optimization (Kurniawan et al., 2011).
Future work includes extending synchrony-aware plasticity to self-supervised pretraining regimes, exploring additional hook placements in attention pathways, and designing curriculum schedules that shape synchrony over time. A second direction is to couple DA-SSDP with online neuromodulatory signals in continual or reinforcement learning scenarios. Finally, we plan to quantify hardware effects on real accelerators by measuring event traffic, latency, and energy under exidh deployment.
References
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 7243 7252, 2017.
Louis Bouvier, Angeliki Steger, Jonathan Binas, Christian Pehle, et al. Spiking neural networks hardware implementations and challenges: A survey. ACM Journal on Emerging Technologies in Computing Systems, 15(2):1 -3, 2019. doi:
Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. JEEE Miero, 38(1):82-900 99, 2018.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognétion, pp. 248 255. Iece, 2009.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesníkov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference Learning Represenlations, 2021.
Benjamin Evans, Ming Li, Richard Reeve, et al. Reinforcement learning in a neurally controlled robot using dopamine modulated STDP. arXiv preprint Xrxi::1002 160600000009, 2015. URL
06096
Daniel E Feldman. The spike-timing dependence of plasticity. Neuron, 75(4):556-57 571, 2012.
Nicolas Frémaux and Wulfram Gerstner. Neuromodulated taikee-iming-ddppooe plasticity, and theory of three-factor learning rules. Frontiers in Neural Cireuils, 9:85, 2016.
Simon Friedmann, Johannes Schemmel, Andreas Grübl, Andreas Hartel, Matthias Hock, and Karlheinz Meier. Demonstrating hybrid learning in flexible neuromorphic hardware system. IEFE Transactions on Biomedical Circuits and Systems, 11(1):128 142, 2016.
14

---

Published in Transactions on Machine Learning Research (12/2025)
Steve B Furber, Francesco Galluppi, Steve Temple, and Luis A Plana. The spinnaker project. Proceedings of the IEEE, 102(5):652-66) 665, 2014.
T. Gollisch and M. Meister. Rapid neural coding in the retina with relative spike latencies. Science, 319 (5866):1108 1111, 2008.
Julia Gygax and Friedemann Zenke. Elucidating the theoretical underpinnings of surrogate gradient learning in spiking neural networks. Neural Computation, 37(5):886 925, 2025. doi: 10.1162/neco a 01752.
Donald O Hebb. The first stage of perception: growth of the assembly. The Organization of Behavior, 4(60): 78 60, 1949.
Jay A Hennig, Emily R Oby, Darby M Losey, Aaron P Batista, Byron M Yu, and Steven M Chase. How learning unfolds in the brain: toward an optimization view. Neuron, (09 3730-3333333 2021.
Yangfan Hu, Huajin Tang, and Gang Pan. Spiking deep residual networks. IEEE Transactions on Newral Networks and Learning Systems, 34(8):5200 5205, 2021.
Yangfan Hu, Qian Zheng, Guoqi Li, Huajin Tang, and Gang Pan. Toward large-scale spiking neural
networks: A comprchensive survey and future directions. arXiv preprint arxiv:2409 021111 2024. URL
wrw./ba////////////////// 1 02111
Eric Hunsberger and Chris Eliasmith. Training spiking deep networks for neuromorphic hardware. arXiv preprinl Xrx 1xXi::611.00111 0.1 2016. URL httpp::/aarxiv.orr/abb///// 015100000/01111111111111111 05141
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Department of Computer Science, 04 2009.
Anders Krogh and John Hertz. A simple weight decay can improve generalization. Aduances in Neural Information Processing Systems, 4, 1991.
Irma T Kurniawan, Mare Guitart-Masip, and Ray Dolan. Dopamine and effort-based decision making. Frontiers in Neuroscience, 5:9616, 2011.
C. Lee, Tobi Delbruck, Michael Pfeiffer, Shih-Chii Liu, et al. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10:508, 2016. doi: 10.3389 /eaiis/ 1ttp://fiin.....0055..3.. 2016.00508.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. CIEAR1O-DVS: an event-stream dataset for object classification. Frontiers in Neuroscience, 11:309, 2017.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Newral Networks, 1069):1659-1611, 1997.
Soumen Majhi, Samali Ghosh, Palash Kumar Pal, Suvam Pal, Tapas Kumar Pal, Dibakar Ghosh, Jernej Završnik, and Matjaž Perc. Patterns of neuronal synchrony in higher-order networks. Physics of Life Reviews, 2024.
Szymon Mazurek, Jakub Caputa, Jan K Argasiński, and Maciej Wielgosz. Three-factor learning in spiking neural networks: An overview of methods and trends from machine learning perspective. orXiv preprint ar XrXiv:2504.05341 2025.
Paul A Merolla, John V Arthur, Rodrigo Alvarez-lcaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan, Bryan L Jackson, Nabil Tmam, Chen Guo, Yutaka Nakamura, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):668 673, 2014.
K. E. Nikiruy, A. V. Emelyanov, V. A. Demin, A. V. Sitnikov, A. A. Minnekhanov, V. V. Rylkov, P. K. Kashkarov, and M. V. Kovalchuk. Dopamino-like STDP modulation in nanocomposite memristors. AIP Advances, 966)01651, 2019. doi: 10. 10.106//55111033.
Michael Pfeiffer and Thomas Pfeil. Deep learning with spiking neurons: Opportunitics and challenges. Frontiers in Neuroscience, 12:774, 2018. doi: (0.3389//iis.20118 00774.
15

---

Published in Transactions on Machine Learning Research (12/2025)
Yusuke Sakemi, Kakei Yamamoto, Takeo Hosomi, and Kazuyuki Aihara. Sparse-firigg regularization methods for spiking neural networks with time-to-firsttspike coding. Scientific. Reports, 13(1):2289)) 2023. doi: 00000888/511508002-50000000
Xinyu Shi, Zecheng Hao, and Zhaofei Yu. Spikingresformer: Bridging resnet and vision transformer in spiking neural networks. Proceedings of the IEEE/CVF Conference on Compuler Vision and Pultern Recognition, pp. 5610 5610--569.--1 2024.
Luisa Speranza, Umberto Di Porzio, Davide Viggiano, Antonio de Donato, and Floriana Volpicelli. Dopamine: the neuromodulator of long-term synaptic plasticity, reward and movement control. Cells, 10(4):735, 2021.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, 1lya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1): 1929-1958, 2014.
Katherine Storrs, Barton Anderson, and Roland W Fleming, Unsupervised learning predicts human perception and misperception of gloss. Nature Human Behaviour, 5(10):1402-1411 2021.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Compuler Vision and Pattern Recognition, pp. 2818-2828 2016.
S. Thorpe, A. Delorme, and van Rullen. Spike-based strategies for rapid processing. Neural Networks, (6):715 725, 2001.
Yuchen Tian, Assel Kembay, Nhan Duy Truong, Jason K Eshraghian, and Omid Kavehei. Beyond pairwise plasticity: Group-level spike synchrony facilitates efficient learning in spiking neural networks. arXiv preprint Xie:22500 14841, 2025.
R. van Rullen and Thorpe. Rate coding versus temporal order coding: what the retinal ganglion cells tell the visual cortex. Newral Computation, 13(::12555 1283, 2001.
Martin Vinck, Cem Uran, Georgios Spyropoulos, Irene Onorato, Ana Clara Broggini, Marius Schneider, and An- dres Canales-Johnson, Principles of large-scale neural interactions. Newron, 111(7):987-1000) 2023. doi: 10. N116/j.neuron.22233 2023.0..0 URL. httpps://wii http::/wwwwwwlllll.. cell httpa///////e.eooae/eteeeee///// /ttp////lleeeet/006666222222 6273 (23) 00211-8
022)0111
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Hebbian learning based
orthogonal projection for continual learning of spiking neural networks. arXie preprint 0rXiv:2402.11984 2024.
Kashu Yamazaki, Viet-Khoa Vo-Ho, Darshan Bulsara, and Ngan Le. Spiking neural networks and their applications: A review. Brain Sciences, 12(7):863, 2022.
Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer. Advances in Neural Information Processing Systems, 36, 2024.
Hanle Zheng, Zhong Zheng, Rui Hu, Bo Xiao, Yujie Wu, Fangwen Yu, Xue Liu, Guoqi Li, and Lei Deng. Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics. Nature Communications, 15(1):277, 2024.
Chenlin Zhou, Liutao Yu, Zhaokun Zhou, Zhengyu Ma, Han Zhang, Huihui Zhou, and Yonghong Tian. Spikingformer: Spike-driven residual learning for transformer-based spiking neural network. arxiv preprint arXiv:234,.1195,, 2023.
Chenlin Zhou, Han Zhang, Zhaokun Zhou, Liutao Yu, Liwei Huang, Xiaopeng Fan, Li Yuan, Zhengyu Ma, Huihui Zhou, and Yonghong Tian. Qkformer: Hierarchical spiking transformer using Q-K attention. In Advances in Neural Information Processing Systems, 2024a. doi: 10.48550/amd (ar//00///ii///2 240...2.001.6522 URL Spotlight.
https://aaas attps://arxiv.. www.wbo//////..e.. 16552
16

---

Published in Transactions on Machine Learning Research (12/2025)
Yihan Zhou, Rui Xu, Yuhan Li, Qiang Zhang, Kaiqi Tan, Jianing Ding, Yongpan Zhao, Huazhong Yang, Shaojie You, Wei Ding, and Hong Li. Direct training high-peeformance deep spiking neural networks: review of theories and methods. Frontiers in Neuroscience, 18:1383844, 2024b. doi: 10.3389/fnins 20441484
Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng Yan, Yonghong Tian, and Li Yuan. Spikformer: When spiking neural network meets transformer. arXiv preprint arXiv:2209. 15425, 2022.
A Appendix
A.1
Training Configurations
Setup and reproducibility: All experiments were run from a unified (raining script. The setup involved workstation with dual NVIDIA RTX 3090 GPUs, employing Py 1.12.1 with CUDA 11.3 and NumPy 1.24.4.
Model variants and spike simulation: We use the Spikingresforrer architecture with time steps T set per dataset. Variants are chosen to match the input resolution: spikingreefformercciiie _ifar for CIFAR-100 and CIFARIO-DVS, and spikingresformer for ImageNet-1K.
Datasets and preprocessing: CIFAR-10/100 10/100 and ImageNet-1K use bicubic interpolation and per-dataset normalization. CIFAR-10 uses mean (0.4914, 0.4822, 0. 0.4465) and std (0.2023, 0.1994, 0.2010), CIFAR-100 follows the common statistics from prior work and ImageNet-1K uses mcan (0.485, 0.456, 0.406) and std (0.229, 0.224, 0.225). Event-based CIFARIO-DVS is loaded from SpikingJelly in frame representation with T frames per sample and split 90% for train/teste Frames are resized the configured input size and optionally passed through DVS-specific angmentation pipeline.
Data augmentation and regularization: For static image classification, we incorporate Augmentt and, if activated, Random Erasing along with mixup. CIFAR-100 employs the rand-m7-nl-mstt - configuration, while ImageNet-1K utilizes probability in constant mode whenever cutout is enabled. Mixup is handled with FasttoollateMixu. Label smoothing of 0.1 is applied in the absence of soft labels. For CIFAR10-DVS, a lightweight DVS augmentation is used upon request.
Random Frasing operates 0.25
Optimization and schedules: All runs AdamW with the dataset-specifie learning rate and weight decay reported below. A cosine learning-rate schedule is used for the entire training with a 3-epoch warm-up from le-5 and a minimum learning rate of le-5. We add a 10-epoch cooldown at the end of training. Unless otherwise stated, no gradient clipping or accumulation is used.
When mixup is enabled, we use Soft TatTTTarttrrosssnntnn Otherwise, we use cross-entropy with label smoothing. Losses are wrapped in the script's CriterionWrapper. Temporal Efficient Training (TET) is wired but disabled in all reported runs.
Losses:
Evaluation and compute reporting: Top-1 accuracy is computed on the validation set after every epoch. MACs are measured with thop using custom counters for convolution, linear, and spike-aware matrix multiply. SOPs are monitored during inference with SOPMonitor. For step-mode s models, reported SOPs are multiplied by T for comparability.
Per-dataset hyperparameters:
CIFAR-100: Model: spikingresformer__ee cifar; input: 3x32x32; epochs: 600; batch size: 200; T: 4; optimizer: AdamW with 5c-4 and weight decay 0.01; augmentation: RandAngment rand-mm7-n1-mstdd05--- mixup: on; cutout: off; label smoothing: 0.1; AMP: on; SyneBN: off.
17

---

Published in Transactions on Machine Learning Research (12/2025)
CIFARIO-DVS: Model: spikingresformer__ s; imput: 3x 3x128x128; epochs: 100; batch size: 64; T: 10 frames; optimizer: AdamW with Ir Ir= 4e-5 and weight decay 0.01; mixup: on; cutout: off; label smoothing: 0.1; AMP: on; SyncBN: off.
ImageNet-1K: Model: spikingresformer I; input: 3x224x224; epochs: 320; batch size: 16; T: 4; optimizer: AdamW with Ir = 5e-4 and weight decay 0.01; augmentation: RandAugment rand-m9- n3-mstd0.5-incc with mixup a=0.2 and cutmix a=1.0; label smoothing: 0.1; AMP: on; SyncBN:
on.
Implementation notes: For CIFAR experiments, the data loader applies bicubic interpolation, random horizontal flipping, and skips extra cropping aside from RandAngment. ImageNet employs random resized cropping with a scale range of [0.08, 1.0] and aspect ratios between [3/4, 4/3]. For DVS setups, the training sampler accounts for distributed processing, while the test sampler proceeds sequentially. Gradients are cleared at each iteration, and spiking states are reinitialized following the synchrony adjustment. During training, we store checkpoints for both the highest Top-1 performing model and the most recent one.
A.2
Computation of MACs, SOPs, and Estimated Energy
We follow the standard accounting used in prior SNN works Yao et al. 2024 Zhou et al. 2022 Shi et al. 2024 All quantitics are reported per image, per inference and denotes the number of simulation steps.
Synaptic operations. For block or layer I in SpikingResformer, the synaptic operation count is related to its static arithmetic cost and the activity level of its inputs:
SOPs(1) = f+(1) x T x MACs(I,,
(12)
where f,()) is the average input firing rate of block T is the simulation length, and MACs(() is the multpply-accmmulte count of obtained by structural profiling (thop with custom counters). In our implementation, SOPs(I) are measured dynamically by a forward-hook monitor, which consistent with the scaling in Eq. 12
Model-level estimated energy. Assuming per-operation energy coefficients on a 45 nm process, EMAC 4.6 PJAMCC and FAC 0.9 pJ/SOP, the energy estimate of SpikingResformer is
EMAC MACs (FCCs(FLSSER (FL.SNN-Cow))
EspshingReaformer =
(13)
+ EAC
Σ
SOPs SONN--NCC
Σ m=1
SOPs
SSNN-FC + Σ SOPSOSSA
where FLSNN-ConV denotes the first convolution that converts the static RGB image into spikes; the subsequent spiking convolution blocks, spiking fully-connected blocks, and DSSA blocks contribute activity-dependent SOPs.
Table 4: SOPs and estimated (Est) energy per image (Img) on Spiking CIFAR. Estimated energy follows the mapping described in Appendix A.2
Method SOPs (G) Est. energy per image (nJ (uJ/Img)
(%) us. Base
Baseline 0.54846
493.62
DA-SSDP
0.56425
494.82
+0.24
Block-wise form.
For arbitrary block b,
Estimated energy (b/( 4.6pJ MACs(b),
Estimated energysNs( 0.9 SOPs(b),
(14)
where MACs(b) SOPs(b) expressed in billions (G), which results in the mJ/s estimation of energy.
18

---

Published in Transactions on Machine Learning Research (12/2025)
78.
topp iteestine (meonmy 158
27.5
Coep SSDP jowan
Epoch (ndative last 103)
Suseinn D4-880P
Baseine
DA NW-88DP
Figure 6: DA-SSDP reduces the generalization gap and improves validation accuracy. We trained the same CIFAR-100 model for 200 epochs with identical hyperparameters and seeds. The first 100 epochs served as warm-up during which the DA-SSDP gate learned slope linking batch-level synchrony to loss reduction; no DA-SSDP weight updates were applied in this phase. In the last 100 epochs, DA-SSDP was enabled alongside backpropagation. (a) SSDP yields smaller generalization gap (training minus validation accuracy) across most epochs, with the mean gap roduced from 7.68 6.27 percentage points. (b) Validation accuracy is higher under DA-SSDP (mean 78.49% 77.94%). (c) The gap distribution shifts lower under DA-SSDP, consistent with milder training fit and better generalization.
A.3 Regularization Effects of DA-SSDP
We trained the same CIFAR-100 model for 200 epochs with the same hyperparameters and scods. The first 100 epochs served a warm-up period, during which the DA-SSDP gate learned slope that links batch-level synchrony to loss reduction. However, no update was applied to the weights, and accuracy did not change. In the remaining 100 epochs, turned on DA-SSDP as training-time update alongside backpropagation showen in Fig 6
Across the last. 100 epochs, training accuracy with DA-SSDP was a bit lower than the baseline, while validation accuracy was higher through the mid-to-late stage and at the end. DA-SSDP won with an average validation gain of about 0.56%. The generalization gap, defined as training minus validation accuracy per epoch, dropped from about 7.68 to about 6.27 percentage points the same window. We observed the same trend across seeds.
These findings confirı that DA-SSDP serves regularizer by modifying weights based on synchrony signals, encouraging learning of feature groups that consistently fire together while avoiding easily overfitted patterns that lack generalization. This leads to less aggressive training data fitting, a smaller gap between training and validation curves, and a small but steady increase in validation and test accuracy.
A.4 Additional Visualization: t-SNE of Test Embeddings
Fig. 7 compares the hidden state embeddings across the two models. In the baseline model, each class clusters into a compact, circular hub, resulting in wide, vacant spaces between hubs. It results in configuration that commonly signals overfitting to the 'simpler" examples near class centroids. On the other hand, in DA-SSDP, after applying the adaptive gate in Eq. g and Eq. 10 the original compact clusters extend into wider, sometimes stretched shapes. Samples belonging to the same class remain grouped, but sparse connections occasionally form between similar categorics, creating smoother, rather than sharp class boundarics. Both panels use the same (-SNE configuration (identical hyperaarameters and random seed) applied to the same test features, and thus the differences reflect the learned representations rather than projection randomness.
The reason is that the gate only adjusts the strength of the original SSDP update, and it enhances potentiation for samples showing higher synchrony and lower loss (S. > μs and k > 0), while for samples with below- average synchrony, reduces both potentiation and depression. As a result, the network reinforces typical firing patterns without forcing all data points into overly tight clusters. Samples near decision boundaries or with unusual features receive gentler updates, creating a more flexible decision boundary. Although the resulting geometry appears less organized in two dimensions, it corresponds to the improved Top-1 accuracy
19

---

Published in Transactions on Machine Learning Research (12/2025)
achieved by DA-SSDP (79.66% us. 78.73% baseline). In essence, this seeming drop in visual tightness embodics synchrony-driven regularization that enhances the model's ability to generalize.
b
80
60-
40-
40
-40-
60
-35
50
50
-90
-00 20
80
Baseline t-SNE at test accuracy (78.73%)
DA-SSDP t-SNE at test accuracy (79.66%)).
Figure 7: (a) t-SNE visualization of hidden-layer embeddings after the final attention stage for the baseline model, exhibiting compact, well-separated clusters. (b) I-SNE visualization for the DA-SSDP model, exhibiting broader intra-class manifolds and smooth, low-density "bridges" that connect semantically related classes, indicating softer inter-class boundaries. Each dot is one sample after two-dimensional t-SNE projection and its color is the class label. A cluster a patch of the same-color dots that lie close logether. The cluster area gauges how far samples of one class spread inside the feature space (small area = high tightness). The gap between clusters reflects the distance between class centers (large gap = clear separation).
A.5 Hardware Perspective
A.5.1 Neuromorphic hardware/ASIC implementation
Parts of DA-SSDP are hardware-friendly and biologically inspired in ways that align with neuromorphic/ANl ASIC designs. Neuromodulators like dopamine are highly efficient in ASICs because they can be implemented simple multiplier or broadcast operation, rather than complex per-synapse computations. Neuromorphic hardware often supports global signals for modulation, making DA-SSDP more feasible for on-chip learning than a purely software-hasse based regularizer. Its primary focus is on offline training, but it has components that. could be adapted for hardware, emphasizing biological plausibility and efficiency.
Training needs batch statistics (synchrony S, and loss &x), dopamine-slope calibration, and operations such outer products, exponential, and clipping. Supporting this would add extra compute and storage chip (e.g., accumulators for S₁ and buffers for At). That increases design complexity, memory traffic, and power. The exact overhead depends on the archilecture.
A.5.2 Energy and spike statistics
In our results, DA-SSDP changes spike patterns by concentrating task-relevant activity into short, synchronous bursts. We observe a 33x increase in median synchrony S, (Fig. 5). On event-driven hardware, power consumption often correlates with the number of synaptic events and the routing they require. If DA-SSDP lowers effective event counts in deployment, energy can go down. However, our software estimates are almost. unchanged (c.g., 493 JJ us. 494 μJ on CIFAR-100: Table g and so we do not make numeric energy claims. Any sparsity effect is task-dependent and should be checked by counting events directly.
20

---

Published in Transactions on Machine Learning Research (12/2025)
A.5.3 Platforms and feasibility evidence
Prior work on supervised deep SNN training and threshold regularization addresses activity balance and stability, and surveys describe neuromorphic ASTCs and training pipelines that manage sparsity and noise
(Sakemi al. 2023 Lee et al. 2016 Bouvier al. 2019 Hunsberger & Eliasmith 2016 Pfeiffer & Pfeil 2018
Evans et al. 2015). Bursty activity concentrates mainly switching, power delivery, boockngg/syychrongg design choices, and thermal margins need extreme care. Storing spike latencies can sparse address-event representation (AER) and buffer hierarchies tuned to the workload (Yamazaki et al.] 2022 IBM TrueNorth demonstrated large-scale, event-driven inference with off-chip training (Merolla et al. 2014), while Intel Loihi supports programmable on-chip plasticity, including reward/three-ffe tthhce-ffcttrr mechanisms with evidence that neuromodulated learning rules are realizable in silicon (Davies et al.) 2018). Memristive devices have also shown dopamine-like modulation of STDP windows at the device level, suggesting potential hybrid ASIC memristor pathways (Nikiruy et al.) 2019 If the full DA-SSDP is too complex for direct hardware implementation, simplified versions using local-only rules work on-chip.
A.6 Theoretical Analysis: Complementary Information and Regularization
Let
A 17 Vw (up AW,
(15)
where A: B E Air and > 0 absorbs the scale in Eq. 1 Ignoring only the poament-wiset clipping (which clips magnitudes but not signs), write Eq. 1 as
AW
1 B
E U..
[blks go.sd ((A++A))).. -A-).
(16)
where 90,65 g(Mt,,))) and Ao.i. Po. exactly those in Eqs. 14 of the gate in Eq. 10
Using the lincar (unclipped) form
S, S us
G. = 1 A:S,
(17)
os
Then we have the exact identity
E [CO) = E/E [V Loup U + tn Sa (Vw Soup (W) Us)
1 Ú = B
U..
(18)
In the implementation, the gate entering Eq. 1 is the clipped quantity clip(1 k:S,, 0.2). Using the linear
gate in Eq. 17 only enlarges magnitudes; therefore, any upper bound obtained from Eq. 18 continues to hold (conservatively) after clipping. For any E R
clip(1 + kz, 0,2, - 1 s |kz|.
(19)
There exists 3 > 0 such that, conditional on Sb.
E
V w Caup (W)) U. S -3 (S, -us).
(20)
Equivalently, by the law of total expectation and So.
E Caw Ub)]
- _ (Sb ps) (V (Vw (W)) s -Bos.
(21)
0g
Combining this assumption with Eq. 18 yields
E
< nL W (W): 0 k83os
(22)
Hence:
21

---

Published in Transactions on Machine Learning Research (12/2025)
Informative synchrony (complementary information). If corr(S, E) 0 in warm-up, then k: > 0 by Eq. g and the gated component contributes a striclly negalive first order term -njk30s in Eq. 22 The net sign depends on the baseline term W[V U], but the increment due to gating negative and proportional to k3og-
Weak synchrony (regularizer). If corr(S, 6) R 0 that k O, the gated contribution in Eq. 18 vanishes to first order and the update reduces, in expectation, to the loss-agnostic baseline U. With element-wise clipping in Eq. [ this baseline acts as bounded regularizer shaping oo-aciivation statistics; we do not assume favorable sign for E[V Loup U]. This basclinc docs not depend on & and remains loss-agnostic by construction.
A.7 Complexity of timing logs and updates
Consider hooked module with Cn imputs, Cout outputs, mini-batch size B, and window length T. We denote the average spikes per sample per channel by Upre (input) and post (output). Let fbytes be the bytes used to store one spike time (4 for FP32, 2 for FP16), and bbool = byte for Boolean.
DA-SSDP (ours). From the implementation, we cache only the first spike time per channel frre gpost and form 2o,5, = Qb, and Ato.., = 1
pre by broadcasting. The extra memory is bj
MDA-SSDP = B(Cnn + Cout bbytes.
(23)
The per-batch time consists of extracting first spikes (no pairwise loop)
Tristt
( Cou));
(24)
and forming the update tensor once by outer product and broadcasting
Tupdate Cont
(25)
Warm-up statistics (us, as. k)) use two scalars per batch and can be streamed with O(1) memory.
Conventional local rules for comparison. We contrast three usual variants:
(i) Full multi-spike logging (pairwise STDP): store all spike times reconstruct latencies.
M = B Cin (rr loytes) + Cour Cour (Tpost loytes)
Tpair = O Cin).
(26)
(ii) Neuron traces (factorized STDP): maintain per-neuron exponential traces and update with Q.PT at every step 1. This avoids storing all spike times but still does a pairwise outer product per step,
(BCii + Cout)),
Terace = OBTCout Cin)
(27)
trace
(iii) Eligibility traces per synapse (e-prop style): keep an eligibility state for each (i,j).
M elig
(
Telig Con
(28)
DA-SSDP replaces either O(BT(Cnn + Cout)) storage (full traces) or O(BC Cin) state (per-synapse eligibility) with constant-size first-spike cache O(B (Cin Cout)), and removes the T-factor from the pairwise update cost:
Combining Eqs. 23
DA-SSDP: (B + Cont)
BContCin)
STDP/c-propp
OBTConn
(29)
For deep modules where Cous Cin dominates and for longer windows T, the saving is substantial in both memory and time.
22

---

Published in Transactions on Machine Learning Research (12/2025)
Algorithm DA-SSDP (post-step correction; used in our experiments)
Require: Mini-batch {(Pb, Qb. t/ :t bo)] hooked weight W RCCon A+, A_, 0; warm-up
Ewarm; statistics (us, k) fitted after warm-up.
Supervised step: forward + loss é= = π Ex l backprop + optimizer.step()
2: Warm-up: compute per-sample Sp.-
3: if epoch < Ewarm then record (Sb, ()) and return
4: end
epoch = Ewarm and (us, os, k) not fitted then standardize Sb, Eo; fit
end if
for b to B do
Go clip + k (S, us)/os,,00 0.2)
end for
10: For each (i.j), set [Ublis t Ib.i. ((A++A-) Ab,i. - -A)
11: AW - clip B E GbU,, 11,1
12: Post-step correction: W t W + AW
decoupled from optimizer moments
A.8 Algorithmic Specification
Algorithm [ |pecifies the post-step integration used our experiments. After the standard supervised update, DA-SSDP computes the per-sample synchrony S₁, the per-sample gates Gb, and the local matrices Ug. The aggregated, element-wise clipped correction AW is then applied. Because this correction is added after the optimizer step, it is decoupled from optimizer state: Adam/ AdamW's first and second moments continue to track only the supervised gradient, while the DA-SSDP correction remains a small, bounded parameter adjustment. This decoupling is precisely why the method optimizer-agnostic.. For completeness, we note that one could alternatively inject the same local term into the gradient before the optimizer step if one wished adaptive moments absorb it; however, do not use that variant in this work.
A.9 Supplementary Experiments
To assess optimizer compatibility and architectural generality, we integrated DA-SSDP into the DH-SRNN backbone Zheng al. (2024) trained with Adam on SHD. DA-SSDP is attached to the classifier post-step correction. No hyperparameters were retuned.
Result. On SHD, DH-SRNN with Adam reaches 88.1% accuracy; adding DA-SSDP at the classifier improves the average accuracy to 89.3% (+1.2%).
Table 5: DH-SRNN on SHD with Adam, Method
Optimizer Acc. (%)
DH-SRNN (baseline)
Adam 88.1
DA-SSDP
Adam 89.3
The improvement. with Adam and non-transformer backbone indicates that DA-SSDP is optimizer-agnostic and transfers beyond Seskren
A.10 Declaration of Generative Al
The authors used LLMs in some parts of the paper in order check grammar mistakes.
23


Reviews:
[{"review_id": "li9zE8cZ01", "reviewer": "Reviewer_6cPD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary\nSpiking transformers have been mostly trained using surrogate backpropagation. The paper proposes a lightweight local update rule, DA-SSDP, that can complement surrogate backpropagation in the training of spiking transformers. Different from spike-timing-dependent plasticity (STDP), which induces much overhead when tracking spike times/traces, the proposed update rule depends on co-firing synchrony and only requires binary spike indicators and first-spike latencies, thus being more memory efficient than STDP, enabling its application in large-scale deep SNNs. This paper also incorporates dopamine modulation to align the local updates with the supervised learning signal. The paper demonstrates the utility of the proposed rule in image classification tasks.\n\n\n### Strengths\n- The proposed rule is lightweight, easy to implement, and induces little overhead during training.\n- The proposed update rule could spark interest in synchrony-based learning rules, which are relatively under-explored.\n- Extensive analyses of the proposed rule are performed\n\n### Weaknesses\n1. Statistics like confidence interval/standard deviation are not provided for experimental results. \n2. Writing/Polish can be significantly improved\n  - a) Structure of Introduction can be improved\n    - The details of the proposed method are introduced in the second half of the third paragraph but the proposed method is only introduced later in the fifth paragraph.\n  - b) Structure of Experiments can be improved\n    - Figure 5 is described in Section 4.2, which is supposed to discuss the results in Table 1. Figure 5 itself is discussed later in much more detail in Section 4.4, so why include it in Section 4.2.\n    - The results are discussed in Section 4.2 but the baselines and metrics are only introduced later in Section 4.3. Also, more discussion on the choice of the baselines (and perhaps metrics) is appreciated. For example, some sentence like: \"We mainly compare with SpikingResformer (Shi et al, 2024), but performances of earlier SSN transformer variants \\[references] are also included for reference.\"\n  - c) No Background section\n    - Section 3.1 and the description of the SpikingResformer architecture better belong to a separate Background section. Including them in the Methods section gives the impression that they are part of this paper's contribution.\n  - d) Lack of consistency\n    - Equations 6 and 8 are supposed to be components of Equation 5 but use slightly different notations, e.g., $g_{b, i, j}$ vs $g(\\Delta t_{b, i, j})$\n    - The paper sometimes writes $\\lambda=0$ and sometimes $\\lambda$=0. The part \"=0\" should also be included in the dollar signs for the equation. This happens a lot throughout the paper.\n    - The paper sometimes capitalize \"Transformer\" and sometimes do not. Similar for \"Classifier\".\n  - e) Lack of polish for figures\n    - In Figure 3a, the color bar overlaps with the figure.\n    - In Figure 4 (top), the horizontal axis ticks 0.5, 1.5, and 2.5 are not necessary. Plus the horizontal axis label is missing.\n    - In Figure 5, why do you report the data divided by 0.0028 for the baseline?\n  - f) The paper does not distinguish between the use of \\citet and \\citep.\n  - g) Grammar\n    - For example, in the second paragraph of Related Works (first letter of \"works\" should be capitalized by the way): \"First, scalability by keeping super detailed records\nof spike timings and assigning credit/blame between pairs of neurons over time, using up a huge amount of\nmemory and processing bandwidth\" is not a complete sentence. Also, \"super\" is not a word formal enough to be used in research papers. \n    - Another example, in the caption of Figure 5: \"making population activity becomes more coordinated and stable\"\n  - h) References\n    - \"An image is worth 16 x 16 words\" is published in ICLR 2021, so the paper should cite that version instead of the arXiv version. Also, the author list for this paper is wrong. Alexey Dosovitskiy is not the sole author of this paper.\n    - For \"Rethinking the inception architecture for computer vision\", \"recognition\" of CVPR should be capitalized.\n    - Why does \"Spike-driven transformer\" appear twice in the references, both published in NeurIPS but with a different year?\n    - After \"Advances in Neural Processing Systems\", the paper sometimes include \"(NeurIPS)\" and sometimes do not.\n    - I just did a brief scan of the references. There might be other issues.\n  - i) the use of acronyms without first introducing the full term, e.g., LIF and STDP\n  - j) In Section 4.3.2, There's a line consisting of only two words \"Hyperparameter sensitivity\". Is that a mistake?\n3. The proposed method bears similarity to Gaussian Decay-based Weight Update Mechanism in the cited paper \"Synchrony-Gated Plasticity with Dopamine Modulation for Spiking Neural Networks\", but the connections are not discussed.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Claims are supported in general, with some concerns about experiment results.\n\n- Claim 1 (bringing synchrony-based signal into model learning) is supported by the detailed description of the proposed learning rule in Section 3. Claim 2 (memory-efficient, scalable implementation for deep SSN-Transformers) is supported likewise.\n- Claim 3 (quantitative validation and robust analysis with safe degradation) is supported by results in Section 4. However, there are several concerns:\n  - The lack of metrics of statistical significance (e.g., confidence intervals or standard deviation) casts doubts on whether the improvements reported in Table 1 occurred by chance. The paper mentions running 5 trials for every experiment, so relevant statistics should be easy to compute and report.\n  - Why is there no Prologue+DA-SSDP and Classifier+DA-SSDP in Table 2? It is strange that the paper includes the results for all variants apart from these two.\n  - In the interpretation paragraph of Section 4.3.2, the paper writes that the dopamine gate \"delivers a stable, batch-specific rescaling ... which compresses the per-batch update magnitude\". But according to Eq. 14, isn't the range of the dopamine gate [0, 2]? So it does not only compress, but could increase the magnitude.\n  - In Section 4.4, the paper writes \"In the baseline network, spike frequencies vary broadly, and most neurons fire every time step (four spikes in total).\" However, judging from the bottom part of Figure 4, very few neurons fire every time step (spike count = 4), seemingly contradicting with the statement.", "audience_yn": "Yes", "audience_explanation": "A lightweight local update rule that is easy to implement and improves the performance of SNN transformers is of interest to researchers in this area.", "requested_changes": "Critical:\n- Report statistical significance (e.g., confidence intervals) for Table 1\n\nHighly recommended: \n- Improve writing/polish (see Weaknesses above)", "broader_impact_concerns": "N/A", "additional_comments": "Clarifying Questions\n1. Is $N=B E_{warm}$? \n2. In Equation 5, when only one neuron of a pair spikes, the one that didn't spike has $t=T$, then $\\Delta t$ is equal to the difference between time of the one that spiked and $T$. Is this intended? If yes, what's the intuition?\n3. It seems that SSDP is introducing the inductive bias that, if two neurons co-fire together more often than not, then they should be encouraged to co-fire consistently across all batches of data. Similarly, if two neurons co-fire less often than not, their connections are weakened. Is this an accurate description? If yes, is there an explanation of why this should improve performance? \n4. Is is possible for $k$ to be negative? Also, is it possible that after some updates, the correlation between synchrony score and supervised loss change? For example, originally high synchrony is correlated with low loss, but over time, due to updates by surrogate backprop, low synchrony becomes correlated with low loss. How would the fixed ($\\mu_S, \\sigma_S, k$) still be relevant?\n5. Can you provide more explanation for Figure 3?\n6. In Section 4.4, the paper writes \"After introducing DA-SSDP, neurons crucial for decision become more prone to simultaneous activation and as a result, these channels are reinforced and maintained elevated spiking rates, while more neurons emit at most one spike or stay inactive.\" This is very hard to see from Figure 4. Wouldn't it be much easier to demonstrate the claim simply by comparing the histograms of spike counts of the baseline and DA-SSDP?\n7. Is there a reason to break eq 15 into two lines?", "extra_scores": {}, "extra_text": {}}, {"review_id": "rzkupcpY3s", "reviewer": "Reviewer_nwUT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Authors propose a special training (or regularisation) strategy for spiking neural networks. In addition to surrogate backpropagation they apply an update based on neuron synchrony. Given that the update rule is also related to Hebbian plasticity, authors made an analogy with synaptic modifications produced by dopamine release. The resulting method is called DA-SSDP.\n\nAuthors apply DA-SSDP in two variants for the training of SpikingResformer. On four classification problems DA-SSDP led to the improved accuracy.", "claims_and_evidence_yn": "No", "claims_explanation": "Authors claim several contributions:\n1. Introduction of the synchrony-based method.\n2. Scalable implementation of spiking transformer.\n3. Validation that shows DA-SSDP provides consistent accuracy improvements.\n\nI am uncertain about the correctness of the second and the third contributions.", "audience_yn": "Yes", "audience_explanation": "Researchers in spiking neural networks and other biologically inspired approaches may be interested in the current contribution.", "requested_changes": "I have several questions about a plasticity mechanism that authors propose in the paper. My main concern is that I can not understand what kind of problem authors are trying to solve with DA-SSDP.\n\n**Scalable implementation of SNN-Transformer.**\n\nOn pages 2 and 3 authors claim \"(B) Memory-efficient, scalable implementation for deep SNN-Transformers: DA-SSDP requires only binary spike indicators and first-spike latencies (...) We provide a drop-in implementation that tweaks just the final classification part and a simple late-stage feature projector, keeping the architecture and training schedule (epochs, batch size, optimizer) unchanged. This approach saves a lot of memory compared with storing detailed spike-timing records.\"\n\nThe title of the contribution (given in bold font in the text of the article) suggests that authors provide a scalable implementation for deep SNN-Transformers. Is this the case? If I am not mistaken, authors used SpikingResformer https://arxiv.org/abs/2403.14302 which they altered in several places by introducing DA-SSDP plasticity applied only on the training stage. Moreover, authors claim that their approach keeps \"all forward-pass operations intact\" (page 5). If this is the case, the implementation of SpikingResformer remains largely the same. Can the authors please clarify what they mean by scalable implementation of deep SNN-Trainsormers?\n\n**Consistent accuracy improvements.**\n\nAccuracies of SpikingResformer, SpikingResformer+SSDP, SpikingResformer+DA-SSDP are within . It seems that the improvement is within $2\\sigma$. Is it \"statistically significant\"? For example, if one applies a different train-test split, does the difference in performance survive? Can the authors provide accuracy with $\\pm 2\\sigma$ \"confidence intervals\"?\n\n**Many components of proposed approach are not biologically plausible:**\n\n1. To define plasticity authors use batch size.\n2. There is a warmup state where rules are different from the rest of training.\n3. There are train and deployment stages.\n4. Gradient descent is used for training.\n5. $G_b$ is computed using global information (this is acknowledged in limitations).\nCan the authors comment on the biological plausibility of their approach? Is it important to have biological plausibility if surrogate backpropagation is used to train a spiking neural network?", "broader_impact_concerns": "na", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gaP4hG0OgR", "reviewer": "Reviewer_728F", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This study targets improving spiking neural networks. The authors proposed a post-update rule, which applies an update on weight using a Gaussian kernel with a dopamine gate. Experiments such as CIFAR and ImageNet datasets verify the improvements. Several qualitative analyses are presented in the appendix.", "claims_and_evidence_yn": "No", "claims_explanation": "I think current version is not sufficient. Please see the Requested Changes below.", "audience_yn": "Yes", "audience_explanation": "Researchers on spiking neural networks would find certain values in this study.", "requested_changes": "- The proposed method applies update after backpropagation. This practice is only compatible with stochastic gradient descent; for example, the Adam(W) optimizer uses the second moment and does not directly apply gradients on parameters.\n\n- Eq. 9 computes the outer product with division by channel sizes, but mini-batch size is not considered and might affect the result. The authors should clarify the definition as per-sample or per-batch.\n\n- “This difficulty stems primarily from the high memory demands of maintaining accurate spike-timing logs and the potential for purely local plasticity adjustments to clash with the supervised learning goal. To effectively leverage local signals derived from spiking neuron dynamics, …” This part requires clear empirical or theoretical evidence. The authors should quantify and measure this behavior more concretely.\n\n- I think that 79% accuracy on ImageNet-1K is a solid baseline, and achieving improvements on this baseline is impressive. The ablation study was also conducted faithfully, and I think that the overall experimental results are promising. One comment is that the authors should consider other models beyond SpikingResformer.\n\n- Nevertheless, this manuscript lacks theoretical analysis. I encourage the authors to perform and formulate some theoretical results.\n\n- Please avoid excessive spacing management such as the narrow space between “clip(” in Eq. 14. Avoid writing in parentheses, such as the two parentheses in the Robustness paragraph and Section 4.3.1. Avoid excessive italics, such as in the Per-dataset hyperparameters paragraph. Avoid “and/or” and write precisely. Honestly speaking, I feel that several parts across this paper were written by GPT. Although it is not bad to be helped by GPT, I encourage the authors to clarify the use of LLM, at least in the appendix.\n\n- Writing should be improved.\n    - Check parenthesis for “Conventional regularizers (e.g., weight decay …” on page 2.\n    - “as showed” → “as shown”\n    - CriterionWarpper → CriterionWrapper\n    - “we fall back to a neural gate” → I think it should be a neutral gate, right?\n    - “making population activity becomes more coordinated and stable.” → “making population activity more coordinated and stable.”\n    - There is a duplicate entry for the paper “Spike-driven transformer.”\n    - “Hebb (2005)” → “Hebb (1949)”\n    - Please use \\citep and \\citet separately.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)