Robustify Spiking Neural Networks via Dominant Singular Deflation under Heterogeneous Training Vulnerability
Authors: Desong Zhang
Jia Hu
Geyong Min
Venue: ICLR 2026 Poster (2026)
Keywords: Neuromophic computing
Spiking neural networks
robustness
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Spiking Neural Networks (SNNs) process information via discrete spikes, enabling them to operate at remarkably low energy levels. However, our experimental observations reveal a striking vulnerability when SNNs are trained using the mainstream method—direct encoding combined with backpropagation through time (BPTT): even a single backward pass on data drawn from a slightly different distribution can lead to catastrophic network collapse. We refer to this phenomenon as the heterogeneous training vulnerability of SNNs. Our theoretical analysis attributes this vulnerability to the repeated inputs inherent in direct encoding and the gradient accumulation characteristic of BPTT, which together produce an exceptional large Hessian spectral radius. To address this challenge, we develop a hyperparameter-free method called Dominant Singular Deflation (DSD). By orthogonally projecting the dominant singular components of gradients, DSD effectively reduces the Hessian spectral radius, thereby preventing SNNs from settling into sharp minima. Extensive experiments demonstrate that DSD not only mitigates the vulnerability of SNNs under heterogeneous training, but also significantly enhances overall robustness compared to key baselines, providing strong support for safer SNNs.

Published as a conference paper at ICLR 2026
ROBUSTIFY SPIKING NEURAL NETWORKS VIA DOMINANT SINGULAR DEFLATION UNDER HETEROGENEOUS TRAINING VULNERABILITY
Desong Zhang, Jia Hu; and Geyong Min Department of Computer Science, University of Exeter, Exeter, U.K. {dz288, J. J.Hu, G.Min]eexetez acc.uk .uk
ABSTRACT
Spiking Neural Networks (SNNs) process information via discrete spikes, enabling them to operate remarkably low energy levels. However, our experimental observations reveal a striking vulnerability when SNNs are trained using the mainstream method- direct encoding combined with backpropagation through time (BPTT): even single backward pass data drawn from a slightly differennt distribution can lead to catastrophic network collapse. We refer to this phenomenon the heterogeneous training vulnerability of SNNs. Our theoretical analysis attributes this vulnerability to the repeated inputs inherent in direct encoding and the gradient accumulation characteristic of BPTT, which together produce exceptional large Hessian spectral radius. To address this challenge, we develop hyperparameterr-fee method called Dominant Singular Deflation (DSD). By orthogonally projecting the dominant singular components of gradients, DSD effectively reduces the Hessian spectral radius, thereby preventing SNNS from settling into sharp minima. Extensive experiments demonstrate that DSD not only mitigates the vulnerability of SNNs under heterogeneous training, but also significantly enhances overall robustness compared to key baselines. Code available at httt:ps//ggEhbbb..om/////222222/
INTRODUCTION
As an emerging brain-inspired computational paradigm, Spiking Neural Networks (SNNs) leverage event-driven, discrete spike streams for feature representation (Maass) 1997p. By eliminating the need for pervasive and computationally intensive matrix multiplications of traditional Artificial Neural Networks (ANNs). SNNs achieve remarkable computational efficiency and significantly lower energy consumption (Pei et al.
2019 Meng et al. 2023). Owing to these inherent advantages, SNNs have been applied across diverse array of application domains, such autonomous driving (Zhu et al.
2024 Shalumov et al.
2021 Viale et al., 2021), edge computing (Liu et al.) 2024a] Zhang et al.
2024
image process (Lu et 2025 Pan al. 2024), and robot control (Jiang al. 2025).
In the practical deployment of SNNs, safety and reliability are of paramount importance, particularly in terms of robustness against perturbations. Even subtle perturbations in the input data that imperceptible to human senses can trigger severely adverse and unpredictable network responses (Ding et al. 2024a). To enhance the robustness of SNNs, existing studies predominantly adopt homogeneous training paradigm, where models are trained on data drawn from single, uniform distribution for instance, vanilla training using only clean samples (Ding et al., 2024b a Geng & Li] 2023,DDmm et al. 2022), or adversarial training where all inputs are perturbed with equal intensity (Ding et al. 2024b Geng Li 2023 Liu et al. 2024b). However, such training settings are idealized and do not reflect the variability and complexity of real-world data. In practical scenarios, models are often required to learn from inherently unpredictable and heterogencous data distributions, as adversaries may employ wide range of poisoning strategies deliberately disrupt distributional homogeneity. We refer to this more realistic paradigm as heterogeneous training (hetero-training). Notably, from the perspective of the attacker, when the number of manipulable samples is limited,
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Homogereeuss Training (Vamilla)
Homogeneous Training (Adversaria)
Haterogensous Training
Heterogansous Training
Laat Daza Bece
Preurted Dats
Bocarr Data Dittae Duta
Clear Data EiPetareed Data
A
Boach
Baich
Baich
Train normally
Train normally
Model Collapses
Model Collapses
Figure 1: The vulnerability of SNNs in heterogeneous training.
concentrating these perturbed samples as batch, rather than dispersing them sporadically throughout the dataset, often leads to a more pronounced degradation of model performance (Zou et al. 2022).
When exposed to batch-level heterogeneity in the training data, we observe:
Observation 1. In SNN training phase, even a single backward pass with a slightly differently- distributed batch can trigger complete model collapse. As depicted in Fig. [ SNNs trained on homogeneous datasets-whether comprised solely of clean samples or perturbed ones-exhibit stable training trajectory. However, introducing just one batch of perturbed data into a cleun dataset, or vice versa, leads to abrupt and catastrophic model collapse. We refer to this phenomenon as the heterogeneous training vulnerability of SNNs. (Sec. 3.7 presents a comprehensive analysis of the experimental results regarding the Observation
This observation reveals a fundamental security risk SNNs when dealing with training data that is inherently unpredictable and cannot be predefined a scenario often encountered in real-world adversarial contexts (Goodfellow et al. 2014, Kurakin al., 2018). This prompts these questions:
1. Why do SNNs experience model collapse in hetero-training?
2. Without relying on input data manipulation, how to design an approach for SNNs that effectively mitigates the model collapse induced by hetero-training and enhance robustness?
Motivated by these questions, we propose a novel training method that enhances the robustness of SNNs under both homogeneous and heterogeneous training conditions. Specifically,
We theoretically show that BPTT yields a Gauss-Newton Hessian with at most linear spectral growth, and that direct encoding makes this bound tight, explaining the abnormally large spectral radius underlying SNN hetero- hetero-training training vulnerability.
Building on these theoretical insights, we develop hyperparameter-free Dominant Singular Deflation (DSD) method. By explicitly deflate the dominant singular components of gradi- ents, DSD effectively reduces the Hessian spectral radius, thereby preventing the network from falling into sharp minima.
Extensive experimental results demonstrate that DSD mitigates SNN vulnerabilities and significantly enhances robustness under both homogeneous and heterogeneous training conditions, outperforming key baselines and thereby ensuring greater safety in deployment.
PRELIMINARY
Spiking neuron dynamic. In SNNs, neurons emulate the spiking behavior of biological neurons to facilitate information transmission. One of the most prevalent nonlinear spiking neuron models in SNNs is the Leaky dilerrate-ad--Fe (LIF) neuron (Xu al. [2022] Fang et al.) 2021 Ding et al. 2022). The dynamics of LIF neuron are described by Eq. ( where 7,, V,, and S. represent the input current, membrane potential, and spike output at time i, respectively. Here, 7 denotes the membrane time constant, Vn, is the potential threshold, and @ corresponds to the Heaviside function.
dV. dt
V Jo = Vvn)
(1)
Ad Adversarial attack. Given input with label y, adversarial examples are generated by finding a perturbation 8 within an ly no-m ball of radius e that maximizes the loss [(h(z - 8). v). This optimization problem is formally expressed as:
arg max L(s(x + +),
| Sc
2

---

Published as a conference paper at ICLR 2026
(a) Collapee Comparison
Rea: Realts-15 15
VOG VGG-11
(b) BPTT VGG+D Test Accuracy
(c) BPTT VGG+D D Test Loss
R. Rate encoding
DI brost smooding
Hamosgareeaa Traniig Heverogenoou Tranking
8.8.0
0/0.30
ACC. (55)
19
12
130
10
VISSH VSSHO IVvo--yyso
200
Network Architecture Encoding Method
Epoch
Epoch
SNN model degradation under hetero-training and collapse curves. In (a), homogeneous training uses clean CIFAR-10 10 data, while hetero-training employs perturbed CIFAR-10 10 data. In (b) and (c), the "clean/+perturbationn. CIFAR- 10 for the first 9 epochs, followed by netero-training starting from epoch 10, during which one random batch per epoch is poisoned with perturbed data. Conversely, the "p/+c" setting denotes homogeneous training perturbed data, with the heterogeneous phase poisoning one random batch per epoch using clean data. All perturbations are generated with FGSM on CIFAR-10 10 using c 2.
Figure 2:
(c/+p_10)" setting denotes homogeneous training on clean
In this paper, we employ two widely adopted adversarial attack methods in main experiments Fast Gradient Sign Method (FGSM) (Goodfellow al.) 2014) and Projected Gradient Descent (PGD) (Madry et al. 2017). Hyperparameter configurations are provided in Appendix A
3 ANALYSIS AND METHOD
In this section, we first present experimental results that demonstrate the model collapse phenomenon described in Observation and analyze why SNNs exhibit disconcerting vulnerability in hetero- training (Sec. 3.1). Building on this analysis, we introduce the Dominant Singular Deflation method and provide theoretical analysis explaining how our approach mitigates the vulnerabilities associated with hetero- trairoottaiinn while simultancously enhancing network robustness (Sec. 3.2
3.1 PRELIMINARY EXPERIMENT: WHY SNN MODEL COLLAPSES IN HETERO- -RANRRO-
We conduct controlled experiments to examine network collapse in SNNs under heterogencous training across three core factors: training paradigm, encoding method, and network architecture. For training paradigms, consider BPTT and SLTT (Spatial Learning Through Time (Meng et al.. ). BPTT backpropagates gradients through temporal multiplication, whereas SLTT eliminates the multiplicative terms BPTT. For encoding methods, we evaluate direct encoding and rate encoding (Poisson). For network architectures, we adopt two widely used backbones: ResNet- 18 and VGG-11. Further experimental settings are provided in Appendix B.1 Fig. 2 a) presents all combinations of these three factors, alongside ANN baselines, and compares their collapse behaviors under hetero- training.. The most severe collapses: BPTT+ResNet- 18ddrrect encoding and BPTT+VGG-I +ddirect encoding, are highlighted with red boxes, and both collapse to the point of exhibiting almost no effective training. From these comparisons, we draw the following observations:
Observation 2. (i). Under hetero-training, ANNs exhibit moderate degradation yet far from collapse, SLTT suffers much less collapse compared with BPTT. (üi). Direct encoding leads to far more severe collapse than rate encoding. (ii). Regarding architectures, both ResNet- 18 and VGG-11 I show comparable levels of degradation or collapse, suggesting that collapse is not tied to architecture.
To further illustrate this phenomenon, Fig. bbb and (c) display training curves of the BPTT+VGG- 1 1+Direct Encoding combination when switching abruptly from homogeneous to heterogeneous training different stages. In all cases, such switch induces catastrophic collapse, manifested by an immediate drop in accuracy, a complete degradation in loss, and unstable oscillations thereafter. Taken together, these results suggest that the damage caused by hetero-training is independent of network architecture and training stage, but strongly dependent on the training paradigm (BPTT) and the encoding method (direct encoding).
This phenomenon suggests that, within a single epoch, the BPTT+direct encoding combination may intermittently drive the network parameters into extremely sharp local minima, characterized by an
3

---

Published as a conference paper at ICLR 2026
abnormally large spectral radius of the loss Hessian (Cheng al. 2022), To interpret this behavior, we conduct an amalysis in the context of SNN ENN-specific properties as follows.
Theorem (Layer-wist wise GN spectral bound under BPTT with LIF neuron dynamics). For the 1-th layer parameters W the BPTT gradient expansion (Xiao et al.
2022
Huang el al. 2024) is
0L dW
as!-1 0L
av
OV4+1 ++1
o as!+1
OV owi
EII
+
(3)
ast
OV
aw
oV
as
ov
D.
JU
By bounded surrogate derivatives and the contractive LIF dynamics (See Appendix for detailed derivations for all bounds mentioned in this analysis), there exist constants Ca, CD < and
0V 88
(0, 1), independent of and T, such that ||G=|l S Cc. D CD. and
89,
Ov.
P.
Then, we have || D + Exet Dx <
CD = CJ, therefore l| = [[ W S
I-p
C.C,. Hence each per-step gradient contribution is O(1). The Gauss-Newtor (GN) Hessian block with respect to W satisfies
H(W") R E = 0,
(4)
where H1 Bi Hg is the effective Hessian with respect to the membrane potential V,+1 Bi H indicates the output-layer Hessian at time and B, - denotes the readout Jacobian from the membrane potential to the output logits z, Since H₂ ž and |Bllll S CB. where C2 sup; Amax Hz,)) < OC, we have Helll S CzC2, and therefore
T
Amax (H(*)) S C Σ SWMPSGECCC
(5)
constant
Thus, the largest eigenvalue (equivalently, the spectral radius) of H H('') is linearly bounded in T. Theorem (Direct encoding makes the GN bound tight). Consider the same setting Theorem 7
with the GN block (W') (J H₂J > 0, where H. B, Hr,. Under direct encoding. the per-step inputs are stationary, and the recurrent Jacobians become nearly time- invariant up to bounded perturbations (Zenke & Ganguli 2018 Bellec al.) [2020). Consequently, by the power-iteration effect of repeatedly applying contructive operators, the BPTT Jacobians (JW concentrate along a common dominant singular direction. Formally, there exist unit vectors (output space) and b (parameter space), scalars oi and residual terms R2 such that for all I we have
J a ab RRe
3 < M,
Σ E=1
|| =I(T),(=o(T),
oo
(6)
where m. M > O denote finite constants independent of t and T, and we use the Landau notation o()) to denote a sublinear term, i.e., f(T o()) if f(T)/T - T 30. Moreover, suppose the output-layer curvature along has a strictly positive time-average, T a H.a 2 c > 0,
where c denotes a uniform positive lower bound, serving as the lower- bound counterpart of C.- Then have equation as follows, where O(.) denotes the standard asymplotic order notation.
T OTT)
Amasd (anxs(((W)) e
Σ
a
Together, Theorem I and Theorem 2
establish two-stage picture of the SNNs. The first result shows that under LIF dynamics the per-step BPTT contributions are uniformly bounded, and consequently the spectral radius of the Hessian grows most linearly in T. The second
curvature in
4

---

Published as a conference paper at ICLR 2026
result demonstrates that when direct encoding is used, the stationarity of inputs and the time-invariant structure of the recurrent operators lead to a power-iteration effect, aligning the Jacobians J along common rank-one component. This alignment ensures that the lower bound grows the same linear rate, thereby making the Gauss-Newton bound asymptotically tight. In summary, BPTT establishes the (TT) upper bound for spectral radius, and direct encoding sharpens it ( This spectral pathology becomes particularly severe under heterogeneous training, where even small fraction of distributional variation sufficient excite the sharp directions amplified by the O(T) curvature growth. Because the dominant Hessian eigenmodes scale linearly with T, perturbations from mismatched batches accumulate disproportionately along these fragile directions, causing instabilities that can quickly lead to collapse. Hence, the vulnerability of SNNs under hetero-training can traced to the same mechanism identified above: the alignment of BPTT Jacobians and the resulting unbounded growth of the spectral radius.
Furthermore, we conduct experiments vali- date our theoretical analysis. As shown in Fig. 3 we train on CIFAR-IO with the VGG-11 archi- tecture under direct encoding at different time step scales for 200 epochs (additional experi- mental details are provided in Appendix [B. The results indicate that network degradation under
100
Homogenecus
Degraded Acouracy
80
60
becomes increasingly se- vere as the number of time steps grows. This confirms that time step scaling indeed one of the factors affecting robustness, thereby support- ing the reliability of our earlier analysis. How- ever, robustness against hetero-training cannot be achieved simply by reducing the number of time steps, since doing so causes a drastic performance drop under homogeneous training as Fig. This motivates the need for mechanism that can tively suppress the dominant curvature growth, thereby enhancing the robustness and hetero- teteoottraning resistance without sacrificing the efficiency of direct encoding.
Time Step
Figure 3: Degradation comparison.
3.2 REDUCING SPECTRAL RADIUS VIA DOMINANT SINGULAR DEFLATION
Dominant Singular Deflation. To mitigate the pathological spectral growth identified above, that is, suppress the excessive enlargement of the Hessian spectral radius, we propose a deterministic and byperparameter-fiee gradient update technique named Dominant Singular Deflation (DSD). The key idea is to deflate the gradient by explicitly removing its rank-one dominant singular component, thereby directly reducing the maximal singular value that drives curvature amplification.
Formally, for parameter set 0, VoC(O) denote the gradient of loss function, represented as k-dimensional tensor CD) E Rdd xxd To systematically analyze its principal components, we introduce a deterministic matrixization operator M IR x Xd., + gmxa with m dj and
11 dj. Applying singular value decomposition gives
M M(V.C(0) = UVV Eonuo.
min(m,n). σ1 012022 2
0.
(8)
refer l as the dominant singular component. To remove this component, we define projection operator D(A) for any A E Rmxx as
(AA) (A, u1
ui iie
17
where = Bij the Frobenius inner product. Thus, DSD orthogonally projects M(Vo((0)) onto the complement of the dominant singular component, yielding the deflated update
VoL(O)=
- (B(r:CCC(C(DD((((MMMM ((M(VoL(0)))
(10)
Effectiveness of DSD. Under direct encoding, Theorem shows that the GN spectral bound becomes tight: the largest eigenvalue Xmax (W")) is governed by the squared maximal singular value of the
5

---

Published as a conference paper at ICLR 2026
Jacobians, i.c. Amax (W()))
(m=x Hence the curvature growth is dominated
by the rank-one component associated with the leading singular pair (01, u1, U1). By construction, DSD removes this rank-one component 0 "181n from the gradient, yielding deflated Jacobians J Consequently, their maximal singular value satisfies T spectral radius strictly decreases;
(J;) = (J+) 01 (Je), and the Hessian
maa H(W:; VoL(O))) < Amaod A Vol(O)))-
(11)
In other words, under the alignment effect induced by direct encoding, DSD deterministically sup- presses the spectral radius by eliminating its dominant contributor, thereby mitigating the instability that causes vulnerability in SNN training. Besides, reduction in Hessian spectral radius can indi-
rectly lower the upper bound of the network's Lipschitz constant (Nesterov 2013 Yao al. 2020) Ghorbani et al.) 2019), thereby contributing to improved robustness and enhanced generalization
capabilities (Ding al. 2022).
Descent preservation of DSD. Although DSD explicitly modifies the gradient by removing its dominant singular component, it does not compromise the descent property of gradient-based optimization. Formally, DSD in Eq. (TO) rewrites the matrixized gradient as
M(VoL(() M((oL(()) D(M(VoL(()))
(12)
The directional derivative of the loss [d] i.e., the rate of change of L. at 0 along a direction d, is given by the inner product between the gradient and d. Under DSD, for the update direction
= (o)( we obtain
DC(0)Ld = (VoL(0), d) = (M(MOOCC)) (M(0)))),MVVC MV
(13)
Since D is an orthogonal projection operator, we can expand and simplify
MMVOC10))MMM220))) (M(VoL(0)), M(V D(M(VoL()))
[|((V -I (M(V -
(14)
where (+) uses the self-ad reff-adjoinmees and idempotence of the orthogonal projection D with respect to the Frobenius inner product, namely. (4, D ((A)) (D(A), D(A)) = IIP(A) Substituting this into the directional derivative gives Eq. 15 with strict inequality whenever VoC(O) + O.
DC(() = I --
< 0,
(15)
In summary, DSD guarantees that the update direction always yields non-increasing loss, and it is strict descent direction whenever the deflated gradient is non-zero.
Remark 1. This conclusion follows solely from the general mathematical property of orthogonal projections in Hilbert spaces (Horn & Johnson 2012), and does not rely on any special structure of SNNs, BPTT, or the cross-entropy loss. Hence the guarantee of descent preservation is universal and independent of the particular network architecture or loss function.
4 EXPERIMENT
Our experiments are structured into four parts. First, we assess the robustness of DSD in homogeneous training (Sec.
which divided into two settings: vanilla training using clean data and adversarial training (AT) (Kundu et al., 2021) using perturbed data generated by white box FGSM with an of 2/ 2/255. Second, we investigate whether DSD can prevent network collapse in hetero- training (Sec. 4.2). Third, we evaluate the effect of our approach on the Hessian eigenvalue during
inference (See. 44. Finally, we inspect DSD for any instances of gradient obfuscation (Sec. 4.4). In
addition, the extra computational overhead of DSD during training is reported in Appendix while DSD introduces no overhead at inference.
To ensure comprehensive evaluation, we conduct experiments static visual datasets of varying
scales, including CIFAR-10 10 Krizhevsky et 2009). CIFAR-100 (Krizhevsky et al. 2009). Tinylm- ageNet (Deng et al. 2009), and ImageNet (Deng et al., 2009) and Dynamic Vision Sensor (DVS) datasets DVS-CIFARIO (Li et al. 2017) and DVS-Gesture (Amir et al., 2017). Implementation
specifies provided in Appendix B.2

---

Published as a conference paper at ICLR 2026
4.1 COMPARISON WITH
(SOTA) IN HOMOGENEOUS TRAINING
White box attack in static datasets. Table D summarizes DSD accuracies under various homo- geneous training settings, compared with SOTA defenses (StoG (Ding et al. 2024b), DLIF (Ding et al. 2024a), HoSNN (Geng & Li 2023 and FEEL (Xu et al., 2024 Overall, DSD consistently outperforms prior defenses against gradient- gradient-based white box attacks across all datasets and training modes. Notably, it yields over 10% accuracy gains in several cases, including CIFAR-100 with FGSM in vanilla training. CIFAR-10 and ImageNet with FGSM, and IinylmageNet with PGD in AT. In TinyImageNet AT. DSD reaches 30.87% accuracy, striking 22.68% improvement over the baseline SNN's 8.19%. Robustness under vanilla training remains difficult, particularly against PGD. Even so, DSD achieves 8.09% accuracy on CIFAR- 100 under PGD in vanilla training, major improvement over the previous best of 2.04% from FEEL (Xu et al. 2024).
Table I: White box performance comparison (%). The highest accuracy in each column is highlighted in bold. The "Improvement" quantifies the gain of DSD over the other best-performing baseline.
CIFAR-10
CIFAR-100
TinyImageNet
ImageNet
Methods
Clean FGSM
PGD
Clean FGSM
PGD
Clean FGSM PGD
Clean FGSM
PGD
Homogeneous Training: Vanilla Training
SNN
93.75
8.19
0.03
72.39
4.55
0.19
56.82
3.51
0.14
57.84
4.99
0.01
StoG
Ding et al.
2024b
91.64
16.22
0.28
70.44
8.27
0.49
DLIF
Ding al.
2024a
92.22
13.24
0.09
70.79
6.95
0.08
HoSNN
Geng
2023
92.43
54.76
15.32
71.98
13.48
0.19
FEEL
Xu et al.
2024
93.29
44.96
28.35
73.79
9.60
2.04
43.83
9.59
4.53
DSD (Ours)
90.21
55.86
31.44
70.26
23.81
8.09
54.54
19.50
12.02
53.47
14.69
4.30
Improvement
3.54
1.10
A 3.09
3.53 A 10.33
6.05
2.28
A 9.91
A 7.49
4.37
9.70
4.29
Homogeneous Training: Adversarial Training (AT)
SNN
91.16
38.20
14.07
69.69
16.31
8.49
49.91
8.19
2.97
51.00
15.74
6.39
StoG
Ding el al.
2024b
90.13
45.74
27.74
66.37
24.45
14.42
DLIF
Ding al.
2024a
88.91
56.71
40.30
66.33
36.83
24.25
HoSNN
Geng
2023
90.00
63.98
43.33
64.64
26.97
16.66
FEEL
[uu ct al.
2024
69.79
18.67
11.07
DSD (Ours)
86.62
74.43
44.38
64,21
43.91
27.11
46.30
30.87
18.21
49.78
26.83
9.12
Improvement
4.54
10.45 A 1.05
5.58
7.08
A 2.86
w 3.61
22.68
15.24
1.22 A 10.09
2.73
Stronger white box attack. Beyond conven- tional white-box attacks, we further evaluate whether DSD can defend against stronger adver- saries, We employ the APGD attack (Croce & Hein 2020), which incorporates adaptive step- size control and momentum-like update during iterations, enabling more efficient exploration
Table 2: APGD performance (%). The highest accuracy in each column is highlighted in bold.
CIFARIO
CIFAR-100
Methods
APGDCE APGDDER R
APGDCE APGDDLR
Homogeneous Training: Vanilla Training
SNN
0.61
2.06
0.09
0.12
DLIF
0.05
0.03
0.02
0.18
HoSNN
10.35
27.39
2.55
0.02
of the loss landscape and avoiding local op- tima to generate stronger adversarial samples. Two variants of APGD are APGDEA and APGDDLR-with detailed descriptions and hy- perparameter settings provided in Appendix A As shown in Table 2 our method achieves the highest accuracy under all attack settings except for APGDDLR on CIFAR- 100 with AT, where it slightly lags behind DLIF (Ding et al., 2024a). These results demonstrate that DSD remains effective and superior even against stronger white box attacks.
DSD (Ours)
22.19
29.77
5,99
5.98
Homogeneous Training: Adversarial Training (AT)
SNN
10.93
17.72
5.87
6.20
DLIF
35.09
39.85
20.68
24.21
HoSNN
37. $7.94
DSD (Ouns)
47.08
44.62
22,86
23.04
White box attack in DVS datasets. In DVS datasets, we trained the model and performed inference under FGSM and PGD attacks by di- rectly perturbing the preprocessed event frames, as implemented in (Liu et al.) 2024b). It can be seen from Table 4 that method also demon- strates excellent robustness when dealing with the DVS dataset, surpassing SOTA method SR.
Table 3: Performance comparion in DVS datasets (%). he highest accuracy in each column is high- lighted in bold. SR: (Liu et 2024b)
DVS-CIFARIO
DVS-Gesture
Methods
Clean
FGSM
PGD
Clean
FGSM
PGD
SNN
76.30
17.20
5.00
95.49
39.24
9.72
SR
75.50
64.60
61.20
DSD
75.10
65.80
61.20
93.75
90.28 55.21
7

---

Published as a conference paper at ICLR 2026
(b)
TinylmageNet
ImageNet
W1TM Eon FOIM
PGD WhBa Ba
FiGo Vhise Bo
While Don
Test (15
TeetAcz (%5
Teol (%)
x((255)
((255)
(255))
(2255)
Figure 4: Performance comparison in different white box and black box attacks.
Black box attack. Black box adversarial examples generated using the substitute- approach in this experiment. We evaluated DSD's resilience across range of perturbation magnitudes and compared its performance to that of vanilla SNN under the same attack strengths. Fig. 4 visualizes these results, and the complete experimental data are provided in Appendix F Across all four dashed-line baselines, DSD consistently achieves substantially higher accuracy than the vanilla model, demonstrating its effectiveness against diverse black box threat scenarios.
4.2 PERFORMANCE IN HETEROGENROUS TRAINING
Comparison with SOTA methods. Building the effectiveness of DSD under homogencous training, we further examine its behavior under heterogeneous training conditions. We compare DSD against SOTA methods, including RAT (Ding et al. 2022). DLIF (Ding et al. (024a), and FEEL (Xu et al. 2024). The results are represented by float bar figures as Fig. [] with detailed numerical values provided in Appendix F As illustrated in the figure, DSD exhibits the least performance degradation under hetero-training (i.e., it produces the shortest floating bars), and moreover achieves clearly higher absolute accuracy than all competing methods under batch injections with findings demonstrate that DSD effectively mitigates the model collapse induced by hetero- hetero-training, outperforming existing approaches.
These
Clean
FGSM
PGD
70-
80
60-
50-
Acc. (9)
40
40-
20
DUF (a) CIFAR-10, cl+p_0
DSD SNN RAT DLIF FEEL
DSD
DLF (c) CIFAR-100, c/+p_0
DSD SNN RAT DLIF FEEL (d) CIFAR-100, p/+c_0
(b) CIFAR-1D, p/+o_0
Figure 5: Performance degradation comparison in hetero-training. Following the protocol of Sec. 3.1
define two poisoning schemes: c/+p_0 and p/+c 00. In c/+p.0, training with clean data, beginning at epoch 0, we inject 6 = 0, FGSM -ertt-perturbdd (c = 2/255) batches at the end of each epoch; p/+c_0 defined analogously: trained with perturbed data and injected with clean data. Subfigures (a), (b) and (c), (d) are vertically aligned, with each pair sharing the same y-axis. For any floating bar in the figure, the top represents the accuracy at (b 0), the bottom represents the accuracy at (b 1), and the bar length indicates the degree of performance degradation. For some methods under PGD, the floating bars are barely visible because their accuracies both (b=0) and (b=1) are nearly zero.
DSD's resilience in different heterogeneous intensities. Furthermore, we evaluate the maximum poisoning intensity that DSD can tolerate under hetero-training. this experiment, we inject different numbers of heterogeneous batches (b 1, 2, 5) to measure the extent of performance degradation. The results, including the performance degeneration, visualized in Fig. 6 with detailed experimental data in Appendix [ omit the vanilla SNN baselines due to their near-total failure under hetero hetero-training, Remarkably, DSD maintains strong resilience across all settings, with no instance of full collapse. Even at the highest poisoning strength (b 5), DSD sustains 30% accuracy CIFAR- 10 under FGSM inference. demonstrating that DSD can withstand high-intensity hetero-training and further indicates its ability to remain robust under realistic batch-level data poisoning scenarios.
8

---

Published as a conference paper at ICLR 2026
Hetero-fraining: citp_0
Hetaro-training: pi+e_0
Clean
PGD
60
20
CIFAR-10
CIFAR-100
TinylmageNee
ImageNet
CIFAR-10
CIFAR-100
TinylmageNet
ImageNet
Figure 6: DSD performance in hetero-training. All subfigures share the same y-axis.
4.3 HESSIAN EIGENVALUE EVALUATION
In this experiment, compare the Hessian eigenvalue of the DSD-trained model against those of vanilla SNN during inference. Specifically, for each inference batch, we compute two metrics: (i). Amax (H): The spectral radius of the Hessian; (ii). Pr(H): The proportion of Amax(()) within the top-five eigenvalues, serving as an indicator of the overall smoothness of the loss landscape. The details for this experiment are provided in Appendix B.2 Table repoots both metries under three distinct adversarial attack scenarios. Across all cases, DSD consistently achieves a lower Amax (H) and markedly reduces its proportional presence among the top-five eigenvalues. These findings corroborate our theoretical design, demonstrating that DSD indeed reduces the Hessian spectral radius, smooths the Hessian sharpness, and underpins its robustness enhancements.
Table Hessian eigenvalue evaluation. The smaller Amax (H) and Pr(H) are, the better.
CIFAR-10
CIFAR-100
Tiny ImageNet Pr(H)
ImageNet
Methods
Amaa (H)
Pr(H)
Amax
Pi(H)
Amax (H)
Amar (H)
Pr(H)
Clean Inference
SNN
261.94
0.98
187.49
0.41
2077.09
0.60
174.88
0.43
DSD
209.90 52.04
0.35 188
135.55
51.4
0.30 vall
1802.80 234.00 0.51 wam
110.56 54.32
0.33 y0.0
FGSM Inference
SNN
269.90
1.15
190.82
0.46
1998.11
0.61
162.57
0.44
DSD
218.27 $1.5
0.38 w0.7
132.01
0.29 0017
1767.57
0.50
111.77
0.32
PGD Inference
SNN
1.03
200.11
2072.77
174.16
DSD
202.89
62.55
0.35
143.07
0.30 v0.6
1793.12
279.6 0.56 waos
113:73 50.43
0.32 40.0
4.4 INSPECTION OF GRADIENT OBFUSCATION
Next, we examine whether DSD suffers from gradient obfuscation (Athalye et al. 2018). For items presents DSD's accuracy under FGSM and PGD attacks across
(1) and (2) in Table 5 Fig. S range of perturbation bounds. as well as side-by-side comparison of white box versus black box performance. It is clear that DSD consistently more vulnerable to iterative PGD than single-step FGSM, and that white box attacks inflict greater degradation than black box attacks. Items (3) and (4) are also evident in Fig. 3 as the perturbation limit increases, DSD's 's accuracy drops sharply, even reaching 0% under several settings. Fig. | further corroborates this trend, showing that although DSD's performance progressively worsens with more PGD iterations, eventually converges to steady minimum. Item (5) indicates that gradient-based attacks fail to locate adversarial examples; however, our results in Fig. 3 demonstrate the opposite- both FGSM and PGD continue to fool DSD despite the training. In short, DSD does not utilize gradient obfuscation to achieve false robustness.
Table 5: Checklist for identifying gradient obfuscation.
Acc. (%))
80
SNN+AT
DSD
DSDIAT
80
Characteristics to identify gradient obfuscation
Pass?
40
(1) Single-step attack performs better compared to iterative attacks
(2) Black-box attacks perform better compared to white-hox attacks
(3) Increasing perturbation bound can't 't increase attack strength
1520
30
80
(with
s=8/255)
(4) Unbounded attacks can't 't reach 100% success
(S) Adversarial example can be found through random sampling
Figure 7: Acc. under different K.

---

Published as a conference paper at ICLR 2026
5
RELATED WORK
SNN defensive methods. Direct encoding with BPTT is the prevailing paradigm for SNN training
(Wu et al., 2018 Deng et al. 2022. Meng al. 2023 Xiao et al. 2022 Wang al. 2023), which
improves efficiency but sacrifices the robustness inherent to rate encoding. To counteract this
vulnerability. prior studies have explored input noise injection (Kundu et al. (021), adversarial training with Lipschitz regularization (Ding et al.
2022), gradient sparsity regularization (Liu et al.
2024b), stochastic gating mechanisms (Ding et al.
2024b), randomized smoothing coding (Wu et al.
2024), noise sensitivity regularization (Zeng et al.
2023), adaptive leak dynamics (Xu et al. [2024).
and modified objectives to suppress membrane potential perturbations (Ding et al.][2024] While these methods improve robustness empirically, they do not address the underlying mechanism by which direct encoding with BPTT induces vulnerability.
Beyond SNNs. Related ideas of suppressing dominant directions have been studied in the broader context of SNNs. Classical works control spectral growth through Parseval networks (Cisse et al. 2017) or spectral norm regularization (Yohhida & Miyato| 2017). while Lipschitz-margin training (Tsuzuku et al. 2018) and Jacobian regularization (Hoffman et al. 2019) constrain large singular values of Jacobians. Recent efforts further reduce sharpness via SAM (Foret et al., (022)) or project adversarial inputs back to the data manifold using generative defenses (Meng & Chen 2017 Saman gouci et al. 2018). Yet existing methods typically act indirectly, depend on auxiliary models, or require sensitive hyperparameters that hinder deployment. By contrast, DSD operates directly in gradient space, deterministically removing the dominant singular component to suppress the principal curvature contributor while provably preserving descent. Being hyperparameter- free and readily practicable, DSD offers simple yet principled mechanism for enhancing SNN robustness.
6 CONCLUSION AND DISCUSSION
Conclusion. In this paper, we experimentally demonstrate that SNNs trained with direct encoding and BPTT can undergo catastrophic model collapse when hetero- traiiontriinng which is common in real- world scenarios. Through theoretical analysis, we show that the repeated inputs of direct encoding combined with gradient accumulation BPTT induce extremely large spectral radius in the Hessian matrix of the loss function, causing the model parameters to become trapped in precarious local minima. Motivated by these insights, we propose a hyperparameter-ffer Deflation (DSD): by orthogonally projecting gradients to precisely eliminate their dominant singular components, DSD effectively reduces the Hessian spectral radius. Extensive evaluations under both homogeneous and heterogeneous training conditions demonstrate that DSD substantially enhances SNN robustness, paving the way for safer and more reliable deployments.
method named Domaint Singular
Limitation. The gradient-basee adjustment inherent to DSD induces deliberate divergence between the gradients actually applied during training and those that would be obtained under an ideal, unmodified regime. While this adjustment markedly bolsters SNNs' robustness, it unavoidably incurs a slight degradation in accuracy when evaluated on unperturbed data. This limitation is also prevalent in existing SOTA methods (Ding et al. 2024b a. Geng & Li 2023 Xu et al., 2024) according to Table I this calls for more extensive and in-depth future research to overcome.
ACKNOWLEDGMENT
This work was supported in part by UK Research and Innovation (UKRI) Grant No. EP/Y036786/1, and Horizon Europe Grant No. 101 129910.
REPRODUCIBILITY STATEMENT
The complete code with fixed random seed utilized in this work is publicly available at phtttps //github com/ Apple22 6419441999 1194199/7 SNN DSD All datasets employed in this research, including CIFAR-10, CIFAR-100, Tiny- ImageNet, ImageNet, DVS-CIFARIO. 10, and DVS-Gestume are publicly accessible. Details regarding the hardware, coding environment, and hyperparameter settings used in our experiments are also included in the Appendix. We dedicate to enable researchers to reproduce the results presented in this paper using similar computational setups.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7243-7252 2017.
Anish Athalye, Nicholas Carlimi, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference machine learning, pp. 274-283, PMLR, 2018.
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature communications, 11(1):3625, 2020.
Andy Brock, Soham De, Samuel L Smith, and Karen Simonyan. figh-performance large-scale image recognition without normalization. In International conference on machine learning, pp. 1059-1071. PMLR, 2021.
Xu Cheng, Hao Zhang, Yue Xin, Wen Shen, Jie Ren, and Quanshi Zhang. Why adversarial training of relu networks difficult? arXiv preprint arXiv:2205. 15130, 2022.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: Improving robustness to adversarial examples. In International conference on machine learning, pp. 854-863, PMLR, 2017.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206-2216. PMLR, 2020.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. Ieee, 2009.
Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re-weighting. arXiv preprint arXiv:2202.11 1946, 2022.
Terrance Vries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: /7.Xiv::700.00 111.7708..00. 04552, 2017.
Jianhao Ding, Tong Bu, Zhaofei Yu, Tiejun Huang, and Jian Liu. Snn-rat: Robustness- spiking neural network through regularized adversarial training. Advances in Neural Information Processing Systems, 35:24780-24799 2022.
Jianhao Ding, Zhiyu Pan, Yujia Liu, Zhaofei Yu, and Tiejun Huang. Robust stable spiking neural networks. arXiv preprint arXiv:2405.2066 2024a.
Jianhao Ding, Zhaofei Yu, Tiejun Huang, and Jian K Liu. Enhancing the robustness of spiking neural networks with stochastic gating mechanisms. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 492-502, 2024b.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056-21069, 2021.
Wei Fang, Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, and Yonghong Tian. Spikingjelly: An open-source machine learning infrastructure platform for baike-based intelligence. Science Advances, 9(0)) 1480, 2023.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiw:2010.01412, 2020.
Hejia Geng and Peng Li. Hosnn: Adversarially-rouus homeostatic spiking neural networks with adaptive firing thresholds. arXiv preprint arXiv:2308. 10373, 2023.

---

Published as a conference paper at ICLR 2026
Behrooz Ghorbani, Shankar Krishnan, and Ying Xiao. An investigation into neural net optimization via hessian eigenvalue density. In International Conference on Machine Learning. pp. 2232-2241 PMLR. 2019.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 1rXiv:1412.6577, 2014.
Judy Hoffman, Daniel Roberts, and Sho Yaida. Robust learning with jacobian regularization. arXiv preprint arXiv:1908.02770 2019.
Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Yulong Huang, Xiaopeng Lin, Hongwei Ren, Haotian Fu, Yue Zhou, Zunchang Liu, Biao Pan, and Bojun Cheng. Clif: Complementary leaky integrate-and-fit neuron for spiking neural networks. arXiv preprint arXiv:2402.0466) 04663,, 2024.
Xiaoyang Jiang, Qiang Zhang, Jingkai Sun, Jiahang Cao, Jingtong Ma, and Renjing Xu. Fully spiking neural network for legged robots. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Souvik Kundu, Massoud Pedram, and Peter A Beerel. Hire-smn: Harnessing the inherent robustness of energy-efficien deep spiking neural networks by training with crafted input noise. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 5209-5218, 2021.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Artificial intelligence safety and security, pp. 99-112. Chapman and Hall/CRC, 2018.
Chankyu Lee, Syed Shakib Sarwar, Priyadarshini Panda, Gopalakrishnan Srinivasan, and Kaushik Roy. Enabling spike-based backpropagation for training deep neural network architectures. Frontiers in neuroscience, 14:497482, 2020.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 11:244131, 2017.
Shiqiang Liu, Weisheng Li, Dan He, Guofen Wang, and Yuping Huang. Ssefusion: Salient semantic enhancement for multimodal medical image fusion with mamba and dynamic spiking neural networks. Information Fusion, 119: 119:103031, 2025.
Yanzhen Liu, Zhijin Qin, and Geoffrey Ye Li. Energy-efficient distributed spiking neural network for wireless edge intelligence. IEEE Transactions on Wireless Communications, 2024a.
Yujia Liu, Tong Bu, Jianhao Ding, Zecheng Hao, Tiejun Huang, and Zhaofei Yu. Enhancing adversarial robustness in snns with sparse gradients. arXiv preprint arXiv:2405.2035. 2024b.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models.
Neural networks, 10(9):1659-1671, 1997.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: (.Kiv:7006.000 1706.06083, 2017.
Dongyu Meng and Hao Chen. Magnet: a two pro-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pp. 135-147, 2017.
Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo. Towards memory-and time-efficient backpropagation for training spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6166-6176. 2023.
Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013.
12

---

Published as a conference paper at ICLR 2026
Yi Pan, Hanqi Jiang, Junhao Chen, Yiwei Li, Huaqin Zhao, Yifan Zhou, Peng Shu, Zihao Wu, Zhengliang Liu, Dajiang Zhu, et al. Eg-spikeformer: Eye-gaze guided transformer on spiking neural networks for medical image analysis. arXiv preprint arXhv:2410.09674, 2024.
A Paszke. Pytorch: An imperative style, high- perhrereffmmec deep learning library. arXiv preprint 19X20:9912.01033, 2019.
Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, et al. Towards artificial general intelligence with hybrid tianjic chip 2019.
architecture. Nature, 572(7767):
Pouya Samangouei, Maya Kabkab, and R Defense-GAr GAasee-GAAAA Chellappa. Protecting classifiers against adversarial attacks using generative models. arxiv 2018. arXiv preprint 1rXiv:1805.0660 1, 2018.
Albert Shalumov, Raz Halaly, and Elishai Ezra Tsur. Lidar-driven spiking neural network for collision avoidance in autonomous driving. Bioinspiration & Biomimetics, 16(6):06601, 6, 2021.
Yusuke Tsuzuku, Issei Sato, and Masashi Sugiyama. Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks. Advances in neural information processing systems, 31, 2018.
Alberto Viale, Alberto Marchisio, Maurizio Martina, Guido Masera, and Muhammad Shafique. Carsnn: An efficient spiking neural network for event-based autonomous cars on the loihi neuro- morphic research processor. In 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-10. IEEE, 2021.
Jingtao Wang, Zengjie Song, Yuxi Wang, Jun Xiao, Yuran Yang, Shuqi Mei, and Zhaoxiang Zhang. Ssf: Accelerating training of spiking neural networks with stabilized spiking flow. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5982-5991, 2023.
Keming Wu, Man Yao, Yuhong Chou, Xuerui Qiu, Rui Yang. Bo Xu, and Guoqi Li. Rsc-snn: Exploring the trade-off between adversarial robustness and accuracy in spiking neural networks via randomized smoothing coding. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 2748-2756, 2024.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Online training through time for spiking neural networks. Advances in neural information processing systems, 35: 20717-20730, 2022.
Mengting Xu, De Ma, Huajin Tang, Qian Zheng, and Gang Pan. Feel-snn: Robust spiking neural networks with frequency encoding and evolutionary leak factor. Advances in Neural Information Processing Systems, 019999930-99900 2024.
Qi Xu, Yaxin Li, Jiangrong Shen, Pingping Zhang, Jian K Liu, Huajin Tang, and Gang Pan. Hierar- chical spiking- based model for efficient image classification with enhanced feature extraction and encoding. IEEE Transactions on Neural Networks and Learning Systems, 2022.
baased
Zhewei Yao, Amir Gholami, Kurt Keutzer, and Michael W Mahoney. Pyhessian: Neural networks through the lens of the hessian. In 2020 IEEE international conference on big data (Big data), pp. 581-590. IEEE, 2020.
Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint 1cXiv:1705.10941 2017.
Friedemann Zenke and Surya Ganguli. Superspike: Supervised learning in multilayer spiking neural networks. Neural computation, 306))1514-1541, 2018
Guanlei Zhang, Lei Feng, Fanqin Zhou, Zhixiang Yang, Qiyang Zhang, Alaa Saleh, Praveen Kumar Donta, and Chinmaya Kumar Dehury. Spiking neural networks in intelligent edge computing. IEEE Consuner Electronics Magazine, 2024.
13

---

Published as a conference paper at ICLR 2026
Wendong Zheng, Yu Zhou, Gang Chen, Zonghua Gu, and Kai Huang. Towards effective training of robust spiking recurrent neural networks under general input noise via provable analysis. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pp. 1-9. IEEE, 2023.
Rui-Jie Zhu, Ziqing Wang, Leilani Gilpin, and Jason Eshraghian. Autonomous driving with spiking neural networks. Advances in Neural Information Processing Systems, 37:1 37:136782-1368M, 2024.
Tianyuan Zou, Yang Liu, Yan Kang, Wenhan Liu, Yuanqin He, Zhihao Yi, Qiang Yang, and Ya-Qin Zhang. Defending batch-leve label inference and replacement attacks in vertical federated learning. IEEE Transactions Big Data, 2022.
14

---

Published as a conference paper at ICLR 2026
A ADVERSARIAL ATTACK DETAILS AND CONFIGURATIONS
FGSM. FGSM is simple yet effective technique to generate adversarial examples. In FGSM, given an input with its true label perturbation is computed in the direction of the gradient of the loss function with respect x. The perturbation is defined as:
c sign (V (v,(((z),))))
(16)
where controls the magnitude of the perturbation and h(x) represents the model's output. The adversarial example then constructed as x + 6, which is designed to force the model into misclassi- fication.
PGD. PGD an iterative method for generating adversarial examples and can be regarded as an extension of the FGSM. PGD updates the adversarial example iteratively by performing gradient ascent step and then projecting the result back onto the feasible set defined by the -y-norm constraint. Formally, the update rule given by:
x ++1 B((,,)) (x' + a. sign (V..([(z'),
(17)
where a denotes the step size, [(f(x'),)) is the loss function of the model with true label y. and H[((,.)) is the projection operator that projects the perturbed example back into the ball <(x,,)) {a' ||x² ==lp S c). By iterating this process, PGD effectively seeks perturbation that maximizes the loss while ensuring that the adversarial example remains within the specified perturbation budget.
APGD. Auto-PGD (APGD) (Croce & Hein 2020) is an iterative adversarial attack based on PGD, equipped with an adaptive step-size strategy and momentum-like update. Given perturbation budget e, the APGD update is defined as
r+1
II (x* 04 sign(V=L((<=))
(18)
where a denotes the step size at iteration t. APGD further maintains an auxiliary momentum variable:
=' - 1111)
x [+1 11B(2,0) (x' + sign sigu((v (19)
where 3, is the momentum coefficient.
Hyperparameters. For these attack methods, we set c = 8/255 for all experimental cases. For PGD, step size a 0.01 and step number K For APGD, there are loss versions: APGDCE (Cross Entropy loss) and APGDDLR (Difference of Logits Ratio loss), we use the Loouneddded setting with perturbation budget e 8/255, initial step size aig 2/255.
B EXPERIMENTAL SETTING
SETTINGS FOR PRELIMINARY EXPERIMENTS
In this section, we detail the network architectures and hyperparameter settings used in the preliminary experiments of Sec. 3.1 Any configurations not mentioned here are identical to those in the main experiments (Appendix B.2). For Fig. a). we set the number of epochs to 50, adopt Poisson encoding for rate encoding (Lee et al.) 2020), and use standard convolutional neural networks (CNN) for ANN, All experiments in Figs a-cc) employ the hyperparameters listed Table Direct encoding is performed with T 4, while rate encoding uses T 64.
Table 6: Hyperparameter settings for preliminary experiments.
Dataset
Optimizer
LeaningRate
WeightDecay
BatchSize
CIFAR-10
SGD
0.1
5e-5
128
15

---

Published as a conference paper at ICLR 2026
B.2 SETTINGS FOR MAIN EXPERIMENTS
In our main experiments, all training cases are implemented using PyTorch (Paszke 2019) with the SpikingJelly (Fang et al. 2023) framework and executed For each dataset, we utilize the hyperparameters listed as Table 7 consistently employing the SGD optimizer and setting the membrane time constant to 1.1. We leverage the PyHessian framework Yao et al. 2020) to compute Hessian cigenvalues
NVIDIA GeForce RTX 5090 GPU.
Table 7: Hyperparameter settings for experiments. * Brock et al. 2021
Dataset
Model
Lea WecghtDecay Epoch BatchSize TimeStep
CIFAR-10
VGG-11
0.1
5e-5
300
128
4
CIFAR-I00
VGG-11
0.1
5c-4
300
128
4
TinylmageNet
VGG-16
0.1
5e-4
300
128
4
ImageNet
NF-ResNet-18
0.1
le-5
100
512
4
ImageNet (AT)
ResNet-18
0.1
le-5
100
512
4
DVS-CIE DVS-CIFARIO
VGG-11
0.05
5c-4
200
128
10
DVS-Gesture
VGG-I1
0.05
5e-4
200
20
C ANALYSIS OF BOUNDED BPTT FACTORS UNDER LIF DYNAMICS
We make explicit why the constants in the inequalities ||G|]| I Cc. ||Dell S CD.
av!+1 as!+1
exist and are independent of and T.
0s
(i) Bound on G.- Write G, aL as
as av.--. Let Φ(+) be the surrogate nonlinearity for spikes,
with [o((w) S к (e.g., sigmoid with slope B gives K 3/44 piccewise-lineat surrogates have Class uniformly in the logits for softmax and || - 1/ (binary)
fixed maximal slope). For standard losses, at cross-entropy, H2 V²L Diag(p) - pp
2=
gives
| <
or < 2 (multi-class). Since 05!4t
8L
8L 0 0z. as! and the readout weights are kept bounded by
regularizaation/llipinngg 0zt+1
< Auu- Therefore, ass
dL ast+1 as ov!+1 Nouss Ca.
||G|ll
(20)
a uniform bound independent t,
(ii) Bound on D.. LIF layer, V!+ aV/11 + W S/ + b (plus optional input term) in
practical code implementation (Fang et al. 2023), with a = E (0, 1). The local Jacobian w.r.t.
W at time is linear in the presynaptic spikes: D, 0
awr L(S)) where || S /imm Tmax
because spikes are binary and Fmax S 1. Hence there exists Cn such that
| S Cin Cp-
(21)
ov al and as
0v!41
(iii) Contraction of the recurrent Jacobian chain. For LIR
W41 while
as!+1 ov! = ø' (V!++) with | S K. Therefore
avi+1
as.+1 ov
++1 !
$ a |
(22)
os
'Hyperparameters set as default: mar setr/trr ter 100, tol = 1e-3, where mar/ter: maximum iterations used compute each single eigenvalue, fol: the relative tolerance hetween two consecutive eigenvalue computations from power iteration.
16

---

Published as a conference paper at ICLR 2026
Imposing spectral- norm control ||*++|| 1 A with + Rλ
yields a uniform contraction rate
p = a + RΛ < 1,
(23)
so that every time-local Jacobian factor and their products satisfy [|Ja.t-1ll S p t-
(iv) Bound on the readout Jacobian Br. Let 24 denote the logits at time and B,
0z The
readout in SNNs is typically linear w.r.t. a hidden state hs (either the membrane potential V, (+1 spike S );
or the
Wout h4 + b, h1 E { (V(+1, s = o(V(+1) }.
(24)
and ar Σ 24 with or 2 (e.g., or or= = T for averaging). Let Aout Wwt (controlled by weight decay/ / clipping spectral normalization), and let == sup, (d(u)) be the maximal slope of the surrogate nonlinearity Φb
Case (direct-V/ readout). If h4 = V/++1 then B, = 82, ov = Wout, hence
Bell < [oott|] = Aout-
(25)
Case 2 (spike readout). If h4 st+1 @(V(+1), then B.
8s. as as av
Wout o' o(VV++), hence
B.ll S | ll | (V²( I s Aout K.
(26)
Combining the cases, there exists uniform constant
Cв k Nout
(27)
such that |B < CB for all t, independent of T. Consequently, ||H | Hz, C |H,ell S C.
(v) Bound Hz,t- For softmax cross-entropy, Hat Diag(p:) p.p. has HHzell (and S for binary); for squared loss, ||l S 1. Hence there exists global C2 < 00 with || S C2 for all f.
(vi) Bound on J.W. We now show why the bound | S Co = C2 holds. Recall that T-p
||D S Cp for all k and [| S a* with p (0, 1). Using the triangle inequality and the sub- sub-mutiplicatiiityy of the operator norm, we obtain
|| = D. + Σ
Jaa- Dr
ket
< | + Σ 1
kxt
SCD + Σ |
SCD + Σ
1C*CDD CD
(28)
RCt
= -CD(1 (1
1"
CDD Σ p
9=0
CD
1 = C).
Here we re-indexed with q = t - A and used the geometric series bound Σ W || uniformly bounded by Cy, independent of and T.
Eq=0 p² =
1
Thus
T-p-
17

---

Published as a conference paper at ICLR 2026
D DATASET
CIFAR-10. The CIFAR-10 10 dataset (Krizhevsky et al. 2009) consists of 60,000 color images, each of size x2 pixels, divided into 10 different classes, such as airplanes, cars, birds, cats, and dogs. Each class has 6,000 images, with 50,000 images used for training and 10,000 for testing. Normalization, random horizontal flipping, random cropping with 4 padding, and CutOut (DeVries & Taylor 2017) applied for data augmentation.
CIFAR-100. The CIFAR 100 dataset (Krizhessky et al. 2009) consists of 60,000 color images, each of size 32x32 pixels, categorized into 100 different classes. Each class contains 600 images, with 500 used for training and 100 for testing. The same processing methods as for dataset CIFAR-10 are applied to dataset CIFAR-100.
Tiny-ImageNet. The Tiny- ImygeNet dataset is scaled-down version of the ImageNet dataset (Deng al., 2009). It contains 200 different classes, with 500 training images and 50 testing images per class, resulting in a total of 100,000 training images and 10,000 testing images. Each image is resized x4 pixels. Normalization, random horizontal flipping, and random cropping with 4 padding are applied for data augmentation for the Tiny- ImageNet dataset.
ImageNet. We evaluate the ILSVRC-2012 ImageNet dataset (Deng al. 2009). which contains ~1.28M training images and 50,000 validation images spanning 1,000 classes. Images are of variable resolution; following common practice and our implementation, training augmentation includes RandomResizedCrop to 224 224, Randommorrzzotllllip conversion to tensors, and channel-wise normalization. For test, images are resized to have a shorter side of 256 pixels and then center-cropped to 224 224 before applying the same normalization.
DVS-CIFARI0. The DVS-CIFAR-II dataset (Li et al. 2017) is a neuromorphic version of the traditional CIFAR-10 10 dataset. DVS-CIFARIO captures the visual information using Dynamic Vision Sensor (DVS), which records changes the scene as series asynchronous events rather than as sequence of frames. The dataset consists of recordings of 10 object classes, corresponding the original CIFAR-10 categories, with each object presented in front DVS camera under various conditions. The dataset contains 10,000 128x 128 images, of which 9,000 are used the training set and the remaining 1,000 as the test set.
DVS- Gesture. The DVS-Gesture dataset (Amir al. 2017 is neuromorphic dataset, consisting of 1l different hand gesture classes, such as hand clapping, arm rolling, and air guitar, performed by 29 subjects under various lighting conditions. Each gesture is represented by a sequence of events rather than frames. The dataset contains 1,676 training samples and 288 testing samples.
E EVALUATION OF ADDITIONAL TRAINING COMPUTATIONAL OVERHEAD
During training, benchmarked its SVD overhead against a vanilla SNN using VGG-11 framework. Our measurements, as Table show no increase in memory usage, and on an NVIDIA Geforce RTX 4070 Ti, DSD adds only around 0.1s of extra training time per batch, this is nearly neglectable. In summary, although DSD does introduce slight training overhead, the increase is minimal.
Table 8: Computational overhead evaluation.
AvgTime perEpoch
AvgTime perBatch
Dataset
BatchSize
BatchNum
DSD
Memory
X
1.4102GB
34s
0.0872s
CIFAR-10, 100
128
390
/
1.4102GB
63s
0.1616s
0.0743
X
5.1797GB
38s
0.5352s
DVS-CIFARI0
128
71
/
5.1797GB
46s
0.6479s
0.1127
X
8,4727GB
63s
0.8630s
DVS-Gesture
73
/
8,4727GB
70s
0.9589s
0.0959
18

---

Published as a conference paper at ICLR 2026
Table 9: Performance of DSD with different attack methods (%). for Fig.
This detailed experimental results
Attack
0
4
6
16
32
64
128
CIFAR-10
SNN FGSM WB
93.75
17.06
14.22
11.78
8.19
4.21
1.99
0.58
0.00
SNN FGSM BB
93.75
24.13
18.41
13.86
10.26
7.88
3.82
1.48
0.59
SNN PGD WB
93.75
2.37
1.01
0.34
0.03
0.00
0.00
0.00
0.00
SNN PGD BB
93.75
4.01
2.80
1.20
0.89
0.02
0.00
0.00
0.00
DSD FGSM WB
90.21
59.75
58.22
57.00
55.86
50.36
39.82
26.61
8.01
DSD FGSM BB
90.21
81.42
79.37
76.65
74.90
67.18
49.30
31.09
12.81
DSD PGD WB
90.21
40.78
37.46
34.50
31.44
22.04
6.36
0.03
0.00
DSD PGD BB
90.21
57.88
54.09
47.80
43.15
30.73
12.11
0.91
0.24
CIFAR-100
SNN FGSM WB
72.39
9.37
7.42
5.46
4.55
2.35
1.19
0.33
0.00
SNN FGSM BB
72.39
13.26
12.11
10.84
9.16
5.31
2.07
1.47
0.49
SNN PGD WB
72.39
2.53
1.15
0.50
0.19
0.02
0.00
0.00
0.00
SNN PGD BB
72.39
3.65
2.68
1.89
0.78
0.15
0.02
0.00
0.00
DSD FGSM WB
70.26
27.89
26.65
24.83
23.81
14.81
8.38
3.36
0.20
DSD FGSM BB
70.26
40.89
38.67
35.67
31.55
21.63
13.31
9.03
1.01
DSD PGD WB
70.26
20.52
15.78
12.35
8.09
2.37
0.00
0.00
0.00
DSD PGD BB
70.26
27.99
26.01
23.84
18.37
9.01
1.07
0.00
0.00
Tiny ImageNet
SNN FGSM WB
56.82
9.42
7.60
4.82
3.51
1.53
0.67
0.00
0.00
SNN FGSM BB
56.82
16.27
15.59
13.69
12.46
8.20
4.80
2.74
1.28
SNN PGD WB
56.82
2.98
1.46
0.89
0.14
0.00
0.00
0.00
0.00
SNN PGD BB
56.82
4.07
2.99
2.12
1.67
0.99
0.46
0.00
0.00
DSD FGSM WB
54.54
22.80
21.74
21.00
19.50
12.66
6.58
1.58
0.02
DSD FGSM BB
54.54
34.63
31.02
29.40
27.68
20.22
10.35
3.20
1.38
DSD PGD WB
54.54
18.42
15.70
13.22
12.02
5.78
2.96
0.20
0.00
DSD PGD BB
54.54
24.92
20.66
16.82
14.84
6.55
4.47
2.10
0.25
ImageNet
SNN FGSM WB
57.84
10.75
8.59
6.73
4.99
1.56
0.14
0.01
0.00
SNN FGSM BB
57.84
12.13
11.43
10.25
8.46
5.42
3.93
1.25
0.35
SNN PGD WB
57.84
1.02
0.48
0.13
0.01
0.00
0.00
0.00
0.00
SNN PGD BB
57.84
9.19
6.05
4.06
3.67
2.02
1.79
0.05
0.00
DSD FGSM WB
53.47
18.62
17.31
15.65
14.69
10.98
5,30
2.01
0.02
DSD FGSM BB
53.47
27.68
25.58
24.44
22.40
16.24
8.96
4.90
2.16
DSD PGD WB
53.47
11.63
8.12
6.33
4.30
2.97
1.50
0.00
0.00
DSD PGD BB
53.47
16.10
12.22
8.33
5.68
4.09
1.67
0.20
0.04
F DETAILED EXPERIMENTAL RESULT
In this section, we present the full experimental results underlying Figs. and 12 respectively, from the main text.
5 and
as Tables
19

---

Published as a conference paper at ICLR 2026
Table 10: Comparison of hetero-training performance degradation (%). This is detailed experimental results for Fig. where "*" indicates self-implementationn.
CIFAR-10, c/+p.0
CIFAR-10,
p/+_.0
CIFAR-100, c/+p_0
CIFAR-100, p/+c_0
Methods
Clean
FGSM
PGD
Clean
FGSM
PGD
Clean
FGSM
PGD
Clean
FGSM
PGD
1
93.75
8.19
0.03
91.16
38.20
14.07
72.39
4.55
0.19
69.69
16.31
8.49
SNN
12.97
0.34
0.01
18.54
4.07
1.98
0.07
0.01
3.41
1.40
0.20
93.01
12.63*
0.05
90.74
45.23
21.16
71.00"
5.77
0.15
70.89
25.86
10,38
RAT Ding
2022
27.24
8.76
0.03
29.01
27.67
8.00
19.20
2.59
0.04
20.11
10.78
3.81
92.22
13.24
0.09
88.91
56.71
40.30
70.79
6.95
0.08
66.33
36.83
24.25
DLIF Ding ct al.
2024a
18.40
0.02
14.75
19.02
5.74
9.98
1.02
0.02
11.40
8.02
2.97
0
93.29
44.96
28.35
90.20
$9.08
39.87
73.79
9.60
2.04
69.79
18.07
11.07
FEEL
Xu al
2024
1
20.01
8.19
2.77
13.99
17.50
7.07
12.84
2.00
0.10
13.86
9.39
1.89
31
44.38
23.81
9.09
43.91
27.11
DSD
40.58
33.70
22.87
38.61
40.09
20.77
27.40
12.54
3.97
22.08
19.16
13.71
Table 11: DSD performance in hetero-training (%). The data in parentheses represents the difference from the baseline (b = 0). This detailed experimental results for Fig.
Hetero-training: c/+p_0
Hetero-training: p/tc.)
CIFAR-10
CIFAR- 100 ThyImmgeNNet
TmtgeNet
CIFAR-10
CIFAR-100
TiyyIaggNeet
ImageNet
Clean Inference
90.21
70.26
54.54
53.47
86.62
64.21
46.30
49.78
40.58 (-49.63)
27.40 (-42.86)
21.45 (-33.09)
23.54 (-29 93)
38.61 (-48.01)
22.08 (-4.113)
19.30 (-27.00) 00)
22.69 (-27.09)
31.08 (-59.13)
12.76 (-57.50)
9.64 (-44.90)
10.53 (-42.94)
29.95 (-56.67)
11.58 (-52.63)
8.95 (-37.35)
9.03 (-40.75)
12.07 (-78.14)
4.18 (-66.08)
3.98 (-50.56)
5.67 (-47.80)
12.02 (-74.60)
3.77 (-60.44)
2.72 (-43.58)
3.44 (-46.34)
FGSM Inference
23.81
74.43
43.91
30.87
26.83
(-22.16)
(-11.27)
14.67 (-4.33)
(-3.44)
40.09 (:34.34)
(-24.75)
(-18833
1168 (-15.15)
(-22.57)
13.70 (-10.11)
14.78
(-3.4))
38.22 (:36.21)
17.31 (-26.60)
10.11 (-20.76)
8.64
(-21,01)
14.93 (-8.88)
(-3.94)
11.86 (-2.73)
31.58 (-42.85)
8.49 (-35,42)
(-25.07)
(-23.73)
PGD Inference
31,44
9.09
12.02
4,30
44.38
27.11
18.21
9.12
22.87 (-8.57)
3.97 (-5.12)
8.89 (-3.13)
2.67 (-1.63)
20.77 (-23.61)
13:71 (-13.40)
12.44 (-5.77)
4,90 (-4.22)
21.23 (-10.21)
3.34 (-5.75)
8.67 (-3.35)
2.21 (-2.09)
18.72 (-25.66)
11.22 (-15.89)
11.90 (-6.31)
3.38 (-5.74)
21.69 (-9.75)
3.01 (-6.08)
9.02 (-3.00)
1.88 (-2.42)
14.31 (-30.77)
7.93 (-19.18)
8.78 (-9.43)
1.94 (-7.18)
Table 12:
Performance comparison with different PGD step number CIFAR-10 (%). This is detailed experimental results for Fig.
Method
=
7
10
15
20
30
50
60
70
SNN
0.03
0.01
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
SNN (AT)
14.07
13.48
12.99
12.03
11.36
10.79
10.49
10.44
10.42
10.41
DSD
31.44
30.34
29.12
27.66
26.40
25.74
25.50
25.46
25.45
25.44
DSD (AT)
44.38
43.10
42.56
41.29
40.78
40.01
39.80
39.78
39.77
39.77
20

---

Published as a conference paper at ICLR 2026
G STATEMENT OF LARGE LANGUAGE MODEL (LLM) USAGE
In the preparation of this manuscript, an LLM was employed to assist with non-scientific tasks. These included polishing the English writing for clarity and style, providing suggestions for figure design and color schemes, supporting LTEX formatting and typesetting, and drafting this statement.
21


Reviews:
[{"review_id": "fv0Aknf57V", "reviewer": "Reviewer_zz8t", "summary": "This paper investigates the unstable training issue in heterogeneous training of SNNs. To address this, they propose a hyperparameter-free method named Dominant Singular Deflation (DSD). This reduces the Hessian spectral radius, prevents convergence to sharp minima, and enhances robustness under different training conditions. Extensive experiments across multiple datasets (CIFAR, TinyImageNet, ImageNet) demonstrate that DSD improves both robustness and stability without incurring inference overhead.", "questions": "Could the authors please clarify what specific training scenario hetero-training refers to in this paper and what is the difference between hetero and homo-training?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-structured and easy to follow.\n- The proposed method is elegant, hyperparameter-free, and mathematically grounded, making it practical for real-world SNN training.\n- The method is validated across multiple datasets and architectures, including both static (CIFAR, ImageNet) and event-driven (DVS) data, consistently outperforming state-of-the-art baselines.", "weaknesses": "- The experimental validation does not clearly substantiate the theoretical motivation of the proposed method. While the Dominant Singular Deflation (DSD) algorithm is designed to suppress the unbounded growth of the Hessian’s spectral radius, it is unclear how this mechanism directly translates into enhanced robustness of the SNN models?\n- The paper states that DSD is designed to mitigate model collapse during heterogeneous training, but most experimental evaluations emphasize adversarial robustness under homogeneous training. Furthermore, while DSD improves robustness, it also yields the lowest clean-data accuracy in Table 1, which contradicts the paper’s stated goal of achieving stable and reliable training.\n- The theoretical analysis in Theorems 1 and 2 relies on the ****Gauss–Newton (GN) Hessian approximation rather than the true Hessian. Since the GN Hessian is always positive semidefinite, this assumption may limit the generality of the theoretical results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "plVNv17xAo", "reviewer": "Reviewer_peRB", "summary": "This paper indicates that spiking neural networks (SNNs) trained using the backpropagation through time (BPTT) algorithm are inherently susceptible to perturbations from heterogeneous training data (clean and corrupted). The paper analyzes this susceptibility and concludes that it stems from an excessively high Hessian spectral radius. To address this issue, the authors propose Dominant Singular Deflation (DSD), a method that explicitly removes the dominant rank-one singular component from the gradient during training. The author conducted experiments on multiple datasets and demonstrated that their method significantly improves the robustness of SNNs.", "questions": "See the weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. From a novel perspective, this paper identifies a source of robustness vulnerability in SNNs.\n2. A thorough theoretical analysis supports the proposed method.\n3. The experimental results demonstrate the performance advantages of the proposed method.", "weaknesses": "1. Whether the analysis presented in this paper holds true for other training methods, such as other parallel training [1] approaches or single-step SNNs that propagate firing rates [2], and encoding schemes, such as temporal encoding, remains to be seen.\n2. The proposed method produced SNNs that performed significantly worse than other robust methods on clean data—a clear drawback.\n3. As shown in Table 7, the proposed method significantly increases training time. Training time nearly doubles on the static CIFAR10 and CIFAR100 datasets.\n\n```\n[1] Parallel Spiking Neurons with High Efficiency and Long-term Dependencies Learning Ability. NeurIPS. 2023.\n[2] Scaling Spike-Driven Transformer With Efficient Spike Firing Approximation Training. IEEE TPAMI. 2025.\n```", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Zrj8GCvKxM", "reviewer": "Reviewer_h1QE", "summary": "In this work, the authors present a novel method, Dominant Singular Deflation (DSD), to address the vulnerability of Spiking Neural Networks (SNNs) under heterogeneous training conditions. The authors provide both theoretical and empirical evidence to support their claims, demonstrating significant improvements in robustness across multiple datasets and attack scenarios. The work is timely and addresses an interesting issue in the safe deployment of SNNs.", "questions": "Please see weaknesses!", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This manuscript identifies and systematically analyzes the phenomenon of model collapse under heterogeneous training—a realistic yet understudied scenario. The theoretical analysis linking BPTT and direct encoding to the growth of the Hessian spectral radius is rigorous and insightful.\n2. In this work, the proposed DSD method is hyperparameter-free. It effectively reduces the spectral radius of the Hessian and preserves the descent property, ensuring stable training without introducing significant overhead.\n3. The authors conduct extensive experiments across multiple static and neuromorphic datasets, under both homogeneous and heterogeneous training settings, and against a variety of white-box and black-box attacks. The results consistently show that DSD outperforms existing SOTA methods in robustness.", "weaknesses": "1. As reported in Table 1, DSD leads to a noticeable decrease in clean accuracy compared to vanilla SNNs. This trade-off between robustness and clean performance may limit its applicability in certain real-world scenarios where high clean accuracy is required. However, the reviewer's previous research also discovered similar phenomena. So, what thoughts does the author have regarding the improvement of this issue?\n2. While the paper identifies the combination of BPTT and direct encoding as the main culprit for vulnerability, it does not extensively explore how DSD performs with other training paradigms (e.g., SLTT) or encoding methods beyond direct and rate encoding.\n3. The author seems to have overlooked some studies on the robustness of SNNs [1-3] in the related work.\n\n[1] \"Enhancing the robustness of spiking neural networks with stochastic gating mechanisms.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 1. 2024.\n\n[2] \"Towards effective training of robust spiking recurrent neural networks under general input noise via provable analysis.\" 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD). IEEE, 2023.\n\n[3]\"RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Coding.\" Proceedings of the 32nd ACM International Conference on Multimedia. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "V8S6ay1rAw", "reviewer": "Reviewer_RTMm", "summary": "1. Research Question\n- The paper addresses the instability and vulnerability of Spiking Neural Networks (SNNs) under heterogeneous training conditions and adversarial perturbations. \n- Key problem: Why do SNNs collapse under small distribution shifts, and can this be mitigated by stabilizing their spectral dynamics during training?\n\n2. Method Proposed: Dominant Singular Deflation (DSD)\n- DSD is a spectral regularization technique applied during backpropagation. \n- Instead of modifying weights or data, DSD removes the dominant singular component from the gradient matrix (Δ W) at each update step. \n- DSD suppresses the gradient’s most unstable direction, aiming to reduce spectral explosion and improve training stability and robustness without introducing new hyperparameters or architectural changes.\n\n3. Theoretical Part\n- The authors conduct theoretical analysis based on the spectral properties of the Jacobian and Hessian during BPTT.\n- The paper formalizes *heterogeneous training vulnerability* as a consequence of the exponential growth of the Hessian’s spectral radius over time steps.\n- It shows (under simplifying assumptions) that the dominant singular direction of (Δ W) aligns across time, amplifying instability.\n- Removing the dominant singular direction theoretically reduces the Hessian’s largest eigenvalue, ensuring descent and smoother loss curvature.\n\n4. Experimental Part\n- Experiments on CIFAR-10/100, TinyImageNet, and event-based datasets show: DSD stabilizes training (smoother loss, bounded gradient norms).\n- DSD Improves robustness under FGSM, PGD without adversarial training, maintains or slightly improves clean accuracy.\n- Spectral diagnostics (Hessian radius, singular value histograms) support the theoretical mechanism.", "questions": "1.Could the authors explicitly state the key assumptions/conditions in each theorem?   For example:\n  * Under what conditions do the contraction and boundedness assumptions on the Jacobian fail?\n  * Are there cases where the spectral radius of the Hessian would not scale linearly with time steps (T)?\n  * What happens if the “approximate time-invariance” assumption of the Jacobian is violated (e.g., due to noise, dropout, or adaptive thresholds)?\n\n2. Could the authors consider evaluating the robustness under additional perturbations such as Gaussian noise or stronger adversarial attacks e.g., APGD?\n\n3. Could the authors quantify DSD's computational cost (e.g., time per iteration, FLOPs, or scaling with network size)\n\n4. Could the authors discuss potential adaptations or approximations that make DSD compatible with neuromorphic chips?\n\n5.  Have the authors observed any instabilities or performance inconsistencies during DSD training process or when DSO is combined with different optimizers?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This work reminds me of a relevant paper: SNN-RAT (https://openreview.net/forum?id=xwBdjfKt7_W). The two works do have very similar views—both aim to improve robustness by controlling the spectral properties of SNNs (singular values/Lipschitz constants). However, they have subtle but important differences in their conceptual approach, theoretical approach, and technical implementation. Therefore, I will use SNN-RAT as a comparison to discuss this paper.\n\n\n1. Theoretical Part\n\n- Presents a novel *optimization-space* spectral stabilization method, contrasting prior *parameter-space* approaches such as SNN-RAT, which constrain the largest singular value of (W). DSD instead regularizes the gradient matrix Δ W, introducing a fresh angle on robustness grounded in training dynamics.\n- Provides a clear and mathematically coherent link between Hessian spectral growth, gradient alignment, and heterogeneous training instability — a valuable conceptual contribution to understanding SNN optimization behavior.\n- Demonstrates theoretical descent guarantees via spectral deflation, suggesting that DSD maintains convergence while suppressing dominant curvature directions.\n- DSO is elegant, minimal, and free of additional hyperparameters, making it an analytically interpretable robustness mechanism.\n\n2. Experimental Part\n\n- Empirical results show that DSD improves both training stability and adversarial robustness without adversarial training, which is a notable distinction from previous works relying on adversarial data augmentation like SNN-RAT/HoSNN.\n- Consistent improvements across CIFAR-10/100, TinyImageNet, and DVS datasets, while preserving or slightly improving clean accuracy.\n- Spectral diagnostics (gradient singular spectra, Hessian radius) align closely with the theory, strengthening internal validity.\n- The evaluation includes white-box, black-box scenarios, reducing concerns about gradient obfuscation.\n\n3. Method Scalability\n\n- The DSD operation is simple and parameter-free, requiring only rank-1 spectral deflation per layer, theoretically lightweight compared to full spectral regularization in SNN-RAT.\n- The method integrates seamlessly into standard backpropagation and does not alter network architecture, making it easy to adopt in existing SNN frameworks.\n- DSO can reduce the reliance on adversarial samples in SNN adversarial training, which greatly reduces the amount of computation while providing adversarial robustness.", "weaknesses": "1. Theoretical Part\nOverall, the paper's proof follows the path of \"spectral analysis of the Jacobian product → increasing the radius of the Hessian spectrum → shrinking the Deflation spectrum → improving stability.\" This is intuitively reasonable.\n\n- It would be helpful for the rigor and clarity of the paper if the author could clearly state the key assumptions in each theorem. In particular, it would deepen our understanding of the problem by indicating when the core theorems proposed fail.\n\n2. Experimental Part\nOverall, I think the experimental part is clear, comprehensive, and rigorous.\n- More attack methods like Gaussion noise, APGD (https://arxiv.org/pdf/2003.01690) could be tested. Only FGSM and PGD are tested in the current paper. \n\n\n3. Method Scalability\n- The proposed DSD step requires layer-wise SVD or power iteration, which is computationally demanding for large-scale or convolutional SNNs; the paper should quantify this cost in the main paper.\n- DSD depends on full gradient access and is thus incompatible with neuromorphic or local learning implementations, limiting its deployability on real spiking hardware.\n- The direct modification of the principal components of the gradient is a concern. Even though the paper provides good results overall, unpredictable results may occur when interacting with other adaptive optimizer such as Adam and RMSProp.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)