Breaking Gradient Temporal Collinearity for Robust Spiking Neural Networks
Authors: Desong Zhang
Jia Hu
Geyong Min
Venue: ICLR 2026 Poster (2026)
Keywords: Neuromophic computing
Spiking neural networks
robustness
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Spiking Neural Networks (SNNs) have emerged as an efficient neuromorphic computing paradigm, offering low energy consumption and strong representational capacity through binary spike-based information processing. However, their performance is heavily shaped by the input encoding method. While direct encoding has gained traction for its efficiency and accuracy, it proves less robust than traditional rate encoding. To illuminate this issue, we introduce Gradient Temporal Collinearity (GTC), a principled measure that quantifies the directional alignment of gradient components across time steps, and we show—both empirically and theoretically—that elevated GTC in direct encoding undermines robustness. Guided by this insight, we propose Structured Temporal Orthogonal Decorrelation (STOD), which integrates parametric orthogonal kernels with structured constraints into the input layer of direct encoding to diversify temporal features and effectively reduce GTC. Extensive experiments on visual classification benchmarks, show that STOD consistently outperforms state-of-the-art methods in robustness, highlighting its potential to drive SNNs toward safer and more reliable deployment.

Published as a conference paper at ICLR 2026
BREAKING GRADIENT TEMPORAL COLLINEARITY FOR ROBUST SPIKING NEURAL NETWORKS
Desong Zhang, Jia Hu, and Geyong Min Department of Computer Science, University of Exeter, Exeter, U.K. (dz288, J. Hu, G.Min]axxetes .ac.r..c..u....
ABSTRACT
Spiking Neural Networks (SNNs) have emerged as an efficient neuromorphic com- puting paradigm, offering low energy consumption and strong representational capacity through binary spike-based information processing. However, their per- formance is heavily shaped by the input encoding method. While direct encoding has gained traction for its efficiency and accuracy, it proves less robust than tra- ditional rate encoding. To illuminate this issue, we introduce Gradient Temporal Collinearity (GTC), a principled measure that quantifies the directional alignment of gradient components across time steps, and we show both empirically and theo- retically-tha that elevated GTC in direct encoding undermines robustness. Guided by this insight, we propose Structured Temporal Orthogonal Decorrelation (STOD), which integrates parametric orthogonal kernels with structured constraints into the input layer of direct encoding to diversify temporal features and effectively reduce GTC. Extensive experiments on visual classification benchmarks, show that STOD consistently outperforms state-of-te state-of-the-ar art methods in robustness, highlighting its potential to drive SNNs toward safer and more reliable deployment. Code available at htlttpss:// git com/ cwtp:ppeeee1119//////o SNN STOD
INTRODUCTION
Spiking Neural Networks (SNNs), as an emerging neuromorphic computing paradigm, process information over time through binary spikes (Maass 1997), exhibiting outstanding efficiency and
low power consumption in domains such as autonomous driving (Viale et al. 2021 Zhu et al. 2024
medical image processing (Pan al. [025 Liu et al., 2025), edge computing (Zhang al. 2024
Liu et al. (2024a). and robot control (Kumar et al. 2025 Jiang et al. (2025). However, under binary spike-based paradigm, the performance of SNNs is largely determined by the input encoding method, as it directly shapes the network's 's temporal statistics and representational capacity (Auge et al. 2021).
In early studies, rate encoding was the most commonly adopted method (Heeger al. 2000 Lee et al. 2016). Its core idea is to represent inputs by the firing frequency of randomly generated spikes within temporal window, thereby transforming static data into spike trains with temporal attributes (Lee) et al. 2020). However, rate encoding requires substantially long spike sequences to capture features with suffcientt fidelity, and under the mainstream training paradigm of Backpropagation Through Time (BPTT), the increased sequence length dramatically amplifies computational cost, which severely limits efficiency in large-scale applications. To overcome this efficiency bottleneck, direct encoding has been proposed (Rueckauer al. 2017). Unlike rate encoding, direct encoding injects identical original data into the network over only a few time steps, thereby maximally preserving the original features while significantly reducing the required sequence length. This property grants direct encoding distinct advantages in both training and inference performance, establishing it as the mainstream choice for efficient and nigh-performance SNN training (Fang et al. 202T).
In addition to efficiency and high performance, robustness during real-world deployment is equally critical for SNNs (Ding et al. 2024a by 2022 Kundu al. 2021). Due their cumulative membrane- potential dynamics, small input perturbations can be repeatedly propagated and amplified over time, making SNNs inherently more vulnerable to temporal perturbation accumulation than traditional artificial neural networks (Stanojevic et al., 2024).. In many of the safety-critica domains where SNNs are most attractive, such as autonomous driving, robotics, and edge intelligence, this temporal sensitivity makes robustness not an optional enhancement but fundamental requirement.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Despite being the mainstream choice for efficient and regh-performance SNN training, direct encoding exhibits surprisingly worse robustness compared to the more traditional rate encoding (Kundu et al. 2021 Kim et al. 2022). Because identical inputs are repeatedly injected across time steps, membrane potentials accumulate highly correlated signals, causing the network to degenerate into an enlarged static feature extractor rather than exploiting temporal dynamics capture complementary information. As a result, the lack of temporal diversity makes the network highly vulnerable to small input perturbations, which are repeatedly accumulated and amplified across time steps, rendering its representations fragile. In contrast, the randomized spike generation of rate encoding serves as natural feature decorrelation mechanism: spike patterns across different time steps are independent, preventing perturbations from remaining consistent across all steps and thereby mitigating error accumulation while strengthening resistance to adversarial perturbations.
To formalize and gain deeper insight into the robustness differences between the two encoding methods, this paper approaches the problem from the perspective of training dynamics and introduces the concept of Gradient Temporal Collinearity (GTC). By quantifying the directional consistency of gradient components across time steps, GTC establishes a unified lens through which the robustness disparities of different encoding methods can be revealed and their underlying mechanisms rigorously explained. Building on GTC, we propose a novel robustness enhancement strategy, Structured Temporal Orthogonal Decorrelation (STOD), which incorporates a feature decorrelation mechanism inspired by rate encoding into direct encoding. By diversifying input features, STOD effectively reduces the level of GTC, thereby improving network robustness. Specifically, the contributions of this paper can be summarized as follows:
We introduce and quantify gradient temporal collinearity (GTC) in SNNs. GTC pro- vides a principled metric that reveals robustness disparities between direct and rate encoding and elucidates the mechanisms by which these disparities affect network robustness.
We propose Structured Temporal Orthogonal Decorrelation (STOD). STOD intro- duces parametric orthogonal kernels with structured constraints into the input layer, which structurally breaks GTC and enhances the network's ability to resist external perturbations.
Through extensive experiments on datasets of varying scales, we show that STOD consis- tently improves the robustness of SNNs without incurring extra inference overhead, and achieves superior robustness performance compared with existing state-of-the methods.
RELATED WORK
Robustness differences across encoding methods. Rate encoding is widely observed provide higher robustness compared with direct encoding. [undu et al. (202I) explained this advantage through proposed saike-based sparse activation maps, Mukhotty al. (2025) associated it with the injection of noise through rate encoding, while Kim et al. (2022) experimentally demonstrated that rate encoding suffers less performance degradation under attacks. Thus, enhancing the robustness of SNNs under direct encoding remains a pressing challenge of both theoretical and practical importance.
Robustness enhancement under direct encoding. Within the more efficient direct encoding paradigm, several methods have been proposed to strengthen robustness: Ding et al. (2022) introduced a regularized adversarial training scheme to constrain the spiking Lipschitz constant of SNNs and thereby mitigate the impact of perturbations
network outputs; Ding et al. (2024a) proposed modifying LIF neurons and minimizing the mean squared perturbation of membrane potentials to stabilize SNNs; Geng & Li (2023) developed LIF neurons with adaptive thresholds that anchor membrane potentials to a neural dynamic signature, thereby minimizing perturbation errors: Liu et al.) (2024b) enhanced adversarial robustness by incorporating gradient sparsity regularization during training: and Ding et al. (2024b) introduced stochastic gating mechanisms across layers to reduce the Lipschitz constant and suppress error amplification, ultimately strengthening SNN robustness. Summarily, existing methods primarily focus on constraining neuronal dynamics or regularization.
3 PRELIMINARIES
Dynamics of leaky integrate-and-firt neuron. In the field of SNNs, the most widely used neuron model is the Leaky Integrate-and-Fin (LIF), which simulates the charging and discharging process
2

---

Published as a conference paper at ICLR 2026
of biological neurons to achieve activation (Fang et al. 2021). From implementation perspective, the dynamics of single LIF neuron can be expressed as follows:
ult + 1| = } (u(t) (t)(t)) + ctt.
(I)
1111-1H(pe + um),
where uft], s(t], and c[t) denote the membrane potential, the binary spike output, and the input current is the membrane time constant controlling the leak rate, Va
time step f, respectively. Here, the firing threshold, and H. represents the Heaviside function. Training in SNNs is performed using BPTT combined with surrogate gradients, with detailed formulations of the BPTT gradients and the specific surrogate gradient methods provided in Appendix A
Adversarial attacks. Given input with label y. adversarial examples generated by finding perturbation å within an -n-rrrm ball of radius E that maximizes the loss [(f(z + 8), v). that is,
arg max L( 6).9))
[|6] pΣ=
METHOD
In this section, we use empirical observations as the motivation and further introduce our method. Specifically, we first define gradient temporal collinearity measure of the alignment between gradient components across time steps. We then compare gradient temporal collinearity under direct encoding and rate encoding and analyze its impact network robustness (Sec. 4.I). Based on the observed gap in gradient temporal collinearity between different encoding methods, we are inspired develop our method: Structured Temporal Orthogonal Decorrelation (Sec. 4.2
4.1 RATE VS. DIRECT: GRADIENT TEMPORAL COLLINEARITY ANALYSIS
While direct encoding substantially outperforms rate encoding in terms of efficiency and accuracy, its weakness in robustness equally evident (Kundu et al.) 2021 Kim et al. 2022 Mukhoty et al.) 2025).
This naturally raises central question:
Can the feature decorrelation mechanism inherent to rate encoding be borrowed to enhance the robustness of direct encoding without sacrificing its efficiency?
Addressing this question requires more than empirical comparison; it calls for a principled metric that can systematically characterize the robustness gap. From optimization perspective, robustness is closely tied to the spectral radius of the Hessian, which in turn governed by the temporal structure of gradients. Motivated by this connection, we introduce Gradient Temporal Collinearity- new measure that quantifies the directional consistency of gradient components across time steps, providing a rigorous lens through which the robustness of direct encoding can be analyzed and improved. We begin by defining:
Definition 1. (Gradient Temporal Collinearity (GTC)). Let the network parameters be denoted as 0 (W1, W W ...., W.). The gradient can be expressed, according Eq. (10), as the sum of its T components, that is, V = Σt= G[t), where G(t) referred to as the component of the gradient at time step t. For any gradient components Gli), G[j] with i * j. define their collinearity, i.e. gradient temporal collinearity C, as in Eq. (3) where (A, B) = Tr(A B) and || = ((A, (A, A) denote the Frobenius inner product and Frobenius norm, respectively. The value C E [-1,1] quantifies the degree of collinearity between the two matrices: C 1, the two matrices become increasingly collinear, while smaller values indicate weaker collinearity.
C(G GLD = (G G(il) |GDIIE
|G
Definition 2. Saatch-Averaged and Epoch-Averager GTC.) For the b-th batch, we define the batch- averaged GTC, denoted as C5 as the mean collinearity among all pairs of gradient components within that batch. Based this, the epoch-averaged GTC C. is defined as the mean of the batch-averaged
3

---

Published as a conference paper at ICLR 2026
values across all B batches within an epoch. Both of these can be referred Eq. 4 Gi[ denotes the gradient component at time step in the b-th batch.
where
E C(C")),G"V1, C Co T(T 1) 1S1CJST
2
1 Eee co.
Experimental evaluation of GTC. Next, we evaluate the evolution of GTC during training under the two encoding methods by varying the number of time steps across different datasets. The detailed setup for this experiment is provided in Appendix D.I
As shown in Fig. T under different time step settings 7 = 4, 8, 32, the epoch-averaged GTC of direct encoding remains consistently higher than that of rate encoding. For direct encod- ing, the GTC starts around 0.8-0.9 in the early stage of training and gradually decreases while remaining relatively stable overall. In contrast, the GTC of rate encoding stays in lower range of 0.2-0.3, with much larger fluctuations across epochs and without clear decreasing trend dur- ing training. Moreover, within each encoding method, the trend of GTC variation does not depend on the number of time steps, and the overall magnitude of GTC shows no clear positive or negative correlation with the time step setting.
CIFAR-10
CIFAR-100
Epoch- Averaged GTC
0.6
Direct Encoding
Diract Encoding
9.4
Rate Encoding
Rate Encoding
160
60
100
200
Epoch
Epoch
Figure 1: GTC evaluation curves.
Impact of GTC on SNN robustness. While the above experiment highlights clear difference in GTC between encoding methods, it remains to be explained why such a difference crucial for SNN robustness. We address this by analyzing the spectral radius of the parameter Hessian, since robustness is closely tied to its magnitude (Stutz et al. 202I). The key result of our analysis (see Appendix B for the full derivation) is the following structured upper bound:
Xmaxx (lo) 2 T. T-mxxx [|G[)|))) [1+ (T- - max max C(G)),
(5)
where Hg is the parameter Hessian for a single sample and Amax (Ho) indicates the spectral radius of it. This inequality shows that a higher of GTC directly amplifies the Hessian spectral radius, thereby reducing robustness.
Origins of GTC disparities between encoding methods. Having established that GTC affects robustness through its impact on the Hessian spectral radius, we next turn to the question of why direct encoding consistently exhibits higher GTC than rate encoding. The key lies in the temporal structure of the inputs. In direct encoding, every time step receives exactly the same input, leaving little temporal diversity. This produces highly consistent neuronal activations and gradient directions across time, leading to a near low-rank structure and consequently higher and more stable GTC. In contrast, rate encoding introduces stochastic spike sampling at the input, which serves inherent decorrelation mechanism across time steps. This reduces temporal collinearity of gradient components and thus yields much lower GTC, albeit with larger fluctuations across epochs due to the randomness involved. Moreover, from Fig. [ we observe that the GTC curve direct encoding gradually decreases as training progresses. This indicates that the strengthening robustness of direct encoding during training is accompanied by a process of gradient component dispersion, further highlighting the role of GTC in explaining the robustness gap between the two encoding methods.
4.2 STOD: STRUCTURALLY BREAKING GRADIENT TEMPORAL COLLINEARITY
Inspired by the inherent input feature decorrelation mechanism in rate encoding that enhances robust- ness, we propose Structured Temporal Orthogonal Decorrelation (STOD), a strategy that embeds feature decorrelation into the temporal dimension of direct encoding. The objective of STOD is to reduce GTC while preserving as much of direct encoding's 's feature- rettttiineteniion capability as possible, thereby achieving balance between decorrelation and retention and ultimately enhancing the robust- ness of SNNs. STOD is composed of two key components: Patchwise Feature Diversification and Global Orthogonal Regularization, which we describe in detail in the following subsections.
4

---

Published as a conference paper at ICLR 2026
Diversification
Global Orthagonal Regularization
Timee Stop
X [2]
II
Main Network
[33
[4]
x1I
Otthogona Karnel (1= 9)
X(I
X'I
X
Main Network
[33
X
[4]
X[2]
Orthogonal Karnal
X [
(1=
Input Directions
Figure 2: Flowchart of STOD, including PFD and GOR.
4.2. PATCHWISE FEATURE DIVERSIFICATION
The core idea of our method is break the GTC, encouraging directional diversity across time steps so that external perturbations are misaligned in gradient space and their cumulative effect mitigated. Since Sec. 4.1 shows that the high GTC of direct encoding originates from repeatedly injecting identical inputs across time while the low GTC of rate encoding arises from its inherent temporal diversity, natural way to reduce GTC introduce controlled diversity into the temporal inputs of direct encoding. A seemingly intuitive approach would be to inject random noise into the inputs or gradients to disrupt this repetition, but such noise lacks mechanism awareness and offers no guarantee of producing meaningful temporal diversity; moreover, these artificial perturbations do not correspond to the actual variations encountered by the network and may instead cause gradient obfuscation (Athalye et al.) 2018) and reduce explainability. To achieve this reduction in an explainable and stable manner, we first require the temporal inputs to exhibit well-controlled and systematically varied feature directions, rather than random unstructured perturbations, that the resulting gradient components naturally become less aligned across time. Building on this requirement, we introduce a structured input transformation module, Patchwise Feature Diversification (PFD), which applies, each time step, an individual parametric orthogonal kernel to transform the input features.
However, Fig. directly applying transformations to the entire input is computationally expensive and renders optimization unstable. PFD first partitions the input into smaller, non-overlapping patches, enabling localized transformations while reducing complexity. Specifically, take static image data as an example, the input dimension is [(] E RC*H HxW
HxW. and we define a partition operator P that {X*(t)) with any X" [t] E Cxpp and N H. W. where p is a hyperparameter representing the patch size. A detailed discussion on the p p. choice of p provided by ablation study (Sec. 5.2
divides the input into N patches, that is (t)
In an SNN with T time steps, we denote the parametric orthogonal kernels as (011,01))...,0(]]]], where each Olt] e Rdxd with d C' x p². For any time step t, the original input ( is mapped to the transformed input X" E CRCW as Eq. 66) where '0" indicates the Kronecker product.
X'[| = vec (P- (P(XI() @ O(1))-
(6)
In addition, the orthogonal kernels are subject to the following structured constraints, ensuring both feature diversity and effective learning:
Structured Constraint 1: Identity initialization at L 1. The first kernel is initialized as the identity matrix, i.e. Ø[] Ta- This serves as a stable reference point, preventing all temporal inputs from being simultancously distorted at initialization and ensuring that a portion of the original information consistently preserved throughout training.
Structured Constraint 2: Mutual orthogonality at initialization. To maximize inter-step feature diversity from the outset and to avoid instability caused by overlapping gradient components during early training, all orthogonal kernels are initialized be mutually orthogonal. Concretely, Houscholder reflections (Mhammedi et al.] 2017) to enforce this constraint. Let é ....... ea denote construct vectors kj ea ej. for 2, 2,3,...,
adopt
the canonical basis of Rd. For steps >

---

Published as a conference paper at ICLR 2026
(we use the first T canonical basis vectors since T S d in all settings) and define the corresponding Houscholder matrices as Eq. 7 {La, Q(2]....... Q(7])...
then, the initialization set given by [0[1),002 "ai
k kj
Q - la
(7)
Structured Constraint 3: Self-orthogonality during training. During training, each orthogonal kernel is constrained remain orthogonal, i.c., O11jO'] Lo- This ensures that the feature transformation at each time step preserves the energy structure of the imput, namely L2-norm conservation: ||]]| || As a result, only the directional structure of the features is altered, avoiding meaningless scaling distortions or information loss. At the same time, this constraint preserves discriminative information and prevents pixel-intensity drift, since even slight intensity shifts at the pixel level may lead to entirely opposite network predictions (Williams & Lij (023). In code practice, we register each orthogonal kernel as a Mani fo dParameter constrained the Stiefel manifold, and update them using RiemannianSGD optimizer (Kochurov et al. 2020).
4.2.2 GLOBAL ORTHOGONAL REGULARIZATION
Ideally, the orthogonal kernels should also satisfy a fourth structured constraint: maintaining mutual orthogonality throughout training, rather than only initialization. This would ensure persistent feature diversity across time steps and prevent temporal inputs from converging during optimization, which would otherwise reintroduce gradient collinearity. However, enforcing both mutual orthogo- nality and self-orthogonality as hard constraints simultaneously would make the system overly rigid, reducing the flexibility of parameter updates and ultimately hindering training effectiveness. More- over, maintaining nnter-kernel orthogonality during training requires concatenating all kernels and constraining them on a Stiefel manifold of dimension d2T, which incurs prohibitive computational and memory costs, rendering this constraint impractical a hard constraint.
Therefore, we propose Global Orthogonal Regularization (GOR), as Fig. shows, as a soft constraint to achieve this effect. The objective of GOR is guide the transformed inputs produced by the orthogonal kernels toward more diverse directions. We define GOR as:
Lo =
Σ cos2 (x71))
(8)
ISKUST
where X denotes the normalized transformed input. Thus, our final training objective is
L Lce + AoCo.
(9)
where Lc is cross-entropy loss and λ hyperparameter controlling the strength of the regular- ization. A detailed discussion of Xo setting is provided in the ablation study (Sec. 5.2)
5 EXPERIMENT
In this section, we first compare the robustness of our method with state-of-the-art methods 5.1). We then conduct ablation studies analyze the contributions of different components of our method (Sec. 5.2) Finally, we inspect our method for any instances of gradient obfuscation (Sec. 5.3).
5.1 COMPARISON WITH STATE-OF-THE-ART (SOTA) METHODS
We conduct experiments on static visual classification datasets, including the small-scale CIFAR- 10 and C1F (Krizhevsky et al. 2009). as well as the large-scale ImageNet (Deng et al. 2009). and Dynamic Vision Sensor (DVS) datasets DVS-CIFARIN (Li et al. (017)) and DVS-GGesture (Amir al.]]22 (2177) Detailed descriptions of these datasets and all preprocessing procedures are provided in Appendix C For attack settings, we adopt Fast Gradient Sign Method (FGSM) (Goodfellow et al.
2014) and Projected Gradient Descent method (PGD) (Madry et al. 2017). both with 8/255.
Unless otherwise specified, the number of iterations K in PGD is set to 7. The hyperparameter
settings of our method are detailed in Appendix [.2]
White box attacks. White box attacks serve as the most direct means evaluating the robustness of method. Under this setting, we select several SOTA methods, including adversarial training

---

Published as a conference paper at ICLR 2026
(AT) (Kundu et al.] 2021) DLIF (Ding al. 2024a). HoSNN (Geng & Li 2023). FEEL (Xu et al. 2024), and StoG DDigg et al. 2024b). . as baselines for comparison. Since the role of structured orthogonal kernels is primarily regularize training by diversifying temporal feature representations, their effect largely embedded into the learned network parameters after optimization. Therefore, during inference remove the orthogonal kernels to highlight the intrinsic robustness gained from training, while also ensuring efficiency and fairness in comparison with baseline methods. This inference-time variant is denoted as STOD without Orthogonal Kernel (STOD w.o. OK). As shown in T even without using orthogonal kemels at inference, STOD w.o. OK achieves the most robust and balanced performance across all three datasets. For example, on CIFAR-10. 10. HoSNN performs well under FGSM with accuracy of 54.76%, but its performance drops dramatically to 15.32% under PGD. Conversely, FEEL achieves a strong 28.35% under PGD, but only 44.96% under FGSM. In contrast, our STOD w.o. OK consistently outperforms all baselines under both attacks. While the robustness gains on ImageNet are smaller than those on CIFAR datasets, this is expected. ImageNet's higher-resolution inputs and the deeper backbone naturally produce richer and more diverse representations, yielding lower temporal redundancy. As a result, STOD has a narrower margin for improvement, though still provides consistent robustness gains under this challenging setting, Moreover, STOD can be further combined with AT (Kundu et al. 2021) to achieve additional performance gains. The corresponding experimental results provided Appendix D Our method likewise demonstrates significant superiority under AT over existing SOTA approaches.
Table I: White box performance comparison. The highest accuracy in each column is highlighted in bold. ** indicates self-implementation results.
CIFAR-10
CIFAR-100
ImageNet
Method
Clean
FGSM
PGD
Clean
FGSM
PGD
Clean
FGSM
PGD
SNN
93.75
8.19
0.03
72.39
4.55
0.19
57.84
4.99
0.01
AT Kundu al.
2021
91.16
38.20
14.07
69.69
16.31
8.49
51.00
15.74
6.39
DLIF Ding et al.
2024a
92.22
13.24
0.09
70.79
0.08
HoSNN Geng Li
2023
92.43
54.76
15.32
71.98
13.48
0.19
FEEL
Xu et al
2024]
93.29
44.96
28.35
73.79
9.60
2.04
StoG Ding et al.
2024b
91.64
16.22
0.28
70.44
8.27
0.49
STOD w.o. OK (Ours)
91.43
55.80
32.97
71.00
26.26
13.13
54.02
19.08
Admittedly, according to Table 1, the clean accuracy of STOD is slightly lower than that of bascline methods. This because all baselines rely on direct encoding, which repeatedly injects identical clean inputs and therefore maximally preserves clean feature fidelity. In contrast, STOD introduces structured temporal decorrelation at the input stage, replacing repeated clean inputs with diversified representations, which unavoidably results in moderate clean-accuracy drop. But, this reduction is small, and the substantial robustness gains provided by STOD outweigh this minor degradation.
Table 2: White box performance comparison with- /without orthogonal kernels.
FOSM
SAN
Whit
Test Acc. (%)
Black
80
60
Dataset
OK
Clean
FGSM
PGD
40
X
91.43
55.80
32.97
20
CIFAR-10
0
V
90.87
59. 3.6
36.72 3775
CIFAR 8
40
CIFAR-100
X
71.00
26.26
13.13
30
/
70.69
32.15 5.89
15.02
1.80
20
X
54.02
19.08
6.44
10
ImageNet
53.57
20.93, 1.85
8.01
25
(c) ImageNet
20
Table 3: Accuracy comparison under RGA attacks.
10
White Box RGA
Black Box RGA
Dataset
Method
FGSM
PGD
FGSM
PGD)
0
16
32
64
128
SNN
8.01
1.01
29.80
13.09
& (/255)
CIFAR-10
STOD
62.17
49.40
80.77
57.21
SNN
6.89
0.64
18.94
9.20
Figure 3: Performance comparison under white and black box attacks.
CIFAR-100
STOD
34.15
29.80
45.81
36.29
7

---

Published as a conference paper at ICLR 2026
Orthogonal kernels further improve SNN robustness. While removing the orthogonal kernels at inference demonstrates that the robustness gains are intrinsically preserved in the trained parameters, retaining them provides additional option to actively maintain structural decorrelation during deployment. When orthogonal kernels are enabled at inference (denoted STOD), temporal features remain explicitly decorrelated, leading to further robustness improvements under various white box attacks. As shown in Table STOD consistently outperforms STOD w.o. OK under both FGSM and PGD, Importantly, the additional cost remains marginal; for example, under the main hyperparameter setting 7 = 4, 8, STOD introduces only 0.15M extra parameters (Detailed in Appendix FF)
White box attack in DVS datasets. In DVS datasets, we trained the model and performed inference under FGSM and PGD attacks by di- rectly perturbing the preprocessed event frames, as implemented in (Liu al., 2024b). It can be seen from Table 4 that our method also demon- strates excellent robustness when dealing with the DVS dataset, surpassing SOTA method SR. Our method shows the same trend on the DVS dataset on the static image dataset, that is, the inference without OK can surpass SOTA, and if OK is added, the robustness can be further enhanced.
Table 4: Performance comparion in DVS datasets (%). he highest accuracy in each column is high- lighted in bold. SR: (Liu et al. (024b)
Methods
DVS-CIFARD
DVS-Gesture
Clean
FGSM
PGD
Clean 95.49
FGSM
PGD
SNN
76.30
17.20
5.00
39.24
9.72
SR
75.50
64.60
61.20
STOD w.o. OK
74.70
66.10
60.40
93.75
87.15
54.67
STOD
72.90
68.80
62.50
93.05
88.88
56.60
Black box attacks. For black box attacks, adversarial examples are generated using the substitute model. We compare the performance gap of STOD between black box and white box settings and its performance trend under varying attack strengths as Fig. 3 with detailed experimental data provided in Appendix E Table 9 It can be observed that robustness under black box attacks consistently exceeds that under white box attacks, and STOD exhibits clear advantage over the vanilla SNN.
Rate gradient approximation (RGA) attack. RGA (Bu et al. 2023) leverages inter-layer spike rates in SNNs perform more targeted adversarial attacks. Moreover, RGA an incremental approach that must be combined with FGSM and PGD to construct the attacks. We evaluate the robustness of STOD under RGA attacks. As shown in Table 3 the vanilla method collapses severely under different forms of RGA, whereas our method demonstrates strong resistance against all such attacks.
Figure 4: Visualization of origina//ranssormeed input and gradient components. For any given image, the top-left entry in the first row is the original input X while the next four images represent the transformed inputs X'[ obtained through PFD at t = 2,2,3, 3, 4, respectively. top-left image within the red box shows the gradient component under direct encoding. Since the four gradient components of direct encoding are visually indistinguishable, we display only one of them. The remaining four images correspond to the gradient components (G(t) of STOD each time step.
the second row, the
Visualization of transformed inputs and gradient components. To further illustrate why STOD enhances network robustness, we visualize the input features at each time step and the corresponding gradient components (heatmap) in Fig. 4 Although the transformed inputs obtained by STOD are less visually discernible than the original images, its gradient components clearly reveal distinct structural patterns of the image from diverse perspectives. In contrast, the gradient components produced by direct encoding appear highly disordered and noisy, with almost no identifiable information to the human eye. This demonstrates that our method enhances the robustness of SNNs against adversarial attacks by leveraging orthogonal kernels to suppress gradient noise while diversifying gradients across time steps in an interpretable manner.
8

---

Published as a conference paper at ICLR 2026
5.2 ABLATION STUDY
In this subsection, we conduct ablation studies on the two most critical hyperparameters: the patch size p and the regularization strength λo.
60
Acc. (%)
Patch size p. On the CIFAR-10 dataset, we evaluate robustness with T = 2, 4, 6, 8 under
40
different patch sizes p 2, 4, 8, 16, 32. The detailed results are presented in Fig. [5] We ob- serve that, across time steps and regardless of whether orthogonal kernels are used at infer- ence, STOD consistently achieves peak perfor- mance
20
PGD
T=2
T
Time step and patch
8. This indicates that overly small patches fail to capture spatial structure effec- tively, limiting temporal feature diversification, while overly large patches make the orthogonal transformation too coarse, discarding fine-grained local information. In contrast, medium-sizee sized patches (e.g.. 8) strike the best balance by preserving sufficient representational capacity while introducing an appropriate scale of perturbation, thereby achieving the optimal trade-off between temporal feature decorrelation and information fidelity.
Figure 5: Accuracy comparison under different p.
Detailed data provided in Appendix F.
Table 10
(a) CIFAR-10 performance
(b) CIFAR-100 performance
CIFAR-10 GTC
(d) CIFAR-100 GTC
Cleet
FOOM
rcc
Clka
FCSH
PCp
GP14
Test Pat-
Test Aot.Acc.SS (S)
838
0.8
mmmm
0000
Epodho Awwerggedd
0.7
0.6
20-
0.55
Agg
Ag
Epoch
Epach
Figure 6: Performance and GTC curve comparison under different Ao.
Regularization strength Åo. We evaluate our method on CIFAR-10 and CIFAR-100 with Ào = O, 0.01, 0.05, 0.1. As shown in Fig. 6 panels (a) and (b) compare robustness under these settings, while panels (c) and (d) illustrate the corresponding epoch- averaged GTC curves during training. The results show that STOD substantially reduces GTC throughout training, with stronger reductions observed as λo increases. However, greater regularization strength does not always lead to stronger robustness. The best performance is achieved when Åo 0.05. It is important note that STOD) does not reduce GTC to the same extent as rate encoding. This discrepancy stems from the intrinsic nature of direct encoding: unlike rate encoding. which inherently randomizes temporal features, direct encoding must repeatedly preserve the original input to ensure feature fidelity. For this reason, aggressively reducing GTC to the level of encoding would inevitably erode essential information content. Instead, STOD deliberately seeks principled balance cotnncee-itrouucnnn sufficient temporal decorrelation to suppress error accumulation, while retaining the integrity clean representations.
Beyond identifying the optimal values of each hyperparameter, the ablation results collectively indicate that STOD exhibits stable behavior rather than brittle sensitivity across broad range of settings. For patch size p, performance varies smoothly p changes; suboptimal values (c.g., p 4 or = 16) still yield substantial robustness gains over the vanilla SNN, showing that STOD does not collapse when deviating from its optimum. A similar pattern emerges for the regularization strength Ao: although oo 0.05 performs best, all non-zero values consistently reduce GTC and improve robustness compared with the baseline, and no abrupt degradation is observed even when the regularization moderately mis-tuned. This robustness plateau demonstrates that STOD benefits from wide viable hyperparameter window, enabling strong generalization across datasets and architectures without precise tuning.
5.3 INSPECTION OF GRADIENT OBFUSCATION
The seminal work Athalye et al. 2018) critically identified several characteristic behaviors, as summarized in Table that arise when defense attains spurious robustness through gradient

---

Published as a conference paper at ICLR 2026
obfuscation. Accordingly, evaluate DSD against each of these behaviors. Our experiments demonstrate that DSD passes all checks, as detailed follows: Table summarizes the evaluation criteria. For items (1) and (2), it is clear from Fig. and Table that STOD performs significantly better under single-step attacks (FGSM) and black box attacks than under multi-step attacks (PGD) and white box attacks, respectively. For items (3) and (4), Fig. R shows that the performance of STOD drops to zero as the attack strength increases. Fig. (with detailed data in Appendix [ Table further confirms this trend, indicating that although the performance STOD gradually degrades with more PGD iterations, it eventually converges to stable minimum. Finally, item (5) states that gradient-based attacks fail to find adversarial examples; however, our results in Fig. show the opposite- botth FGSM and PGD consistently succeed in fooling STOD despite the training. In conclusion, our method does not utilize gradient obfuscation achieve false robustness.
Table 5: Checklist for identifying gradient obfuscation.
Test Acc. (%)
80
30-
CIFAR-11 CIFAR- 100
Characteristics to identify gradient obfuscation
Pass?
20
Imagahett
(1) Single-stop attack performs better compared to iterative attacks
16
(2) Black box atlacks perform betler compared to white bux atlacks
5.61
(3) Increasing perturbation bound can't 't increase attack strength
(4) Unbounded attacks can't It reach 100% success
K (with 8/2555))
(5) Adversurial example can be found through random sampling
Figure 7: Acc. under different K.
CONCLUSION AND DISCUSSION
Conclusion. In this paper, introduce the concept of Gradient Temporal Collinearity (GTC) and, from both experimental and theoretical perspectives, explain why rate encoding--we its lower stronger robustness than direct encoding, which suffers from higher GTC. Inspired by this phenomenon and its theoretical underpinnings, we borrow the feature decorrelation mechanism of rate encoding and structurally incorporate it into direct encoding, leading to our proposed method: Structured Temporal Orthogonal Decorrelation (STOD). STOD introduces parametric orthogonal kernels with structured constraints into the input layer to diversify input features, thereby breaking the high GTC of direct encoding and enhancing robustness. Extensive experiments demonstrate that our method consistently surpasses SOTA approaches in robustness, while comprehensive ablations confirm the effectiveness and necessity of each module. Together, these results position our method as a strong step toward more reliable and secure application SNNs.
Limitation. Since method reshapes network gradients to favor updates that enhance resistance to perturbations, it inevitably leads to a performance drop when the network is evaluated on clean data. This limitation is common across existing sabustness-oriented studies in SNNs (Kundu et al.,
2021 et al.) 2224a| Geng & Li [023) Xu et al.) 2024 Ding et al.) 2024b) and calls for deeper investigation to effectively addressed.
Broader impact. When both inference modes: STOD with/without orthogonal kernels (OK) surpass existing SOTA methods, approach offers practitioners greater flexibility. Users prioritizing maximum performance can opt for inference with OK, whereas those emphasizing efficiency and lightweight deployment may choose inference without OK. This dual advantage underscores the practical value of our method across diverse application scenarios.
ACKNOWLEDGMENT
This work was supported in part by UK Research and Innovation (UKRI) Grant No. EP/Y036786/1, and Horizon Europe Grant No. 101 129910.
REPRODUCIBILITY STATEMENT
The complete code with fixed random seed utilized in this work is publicly available at https: //qithub. 25419/5K1 _STO All datasets employed this research, including CIFAR-10, CIFAR-100, and ImageNet are publicly accessible. Details regarding the hardware,
10

---

Published as a conference paper at ICLR 2026
coding environment, and hyperparameter settings used in experiments are also included in the Appendix. We dedicate to enable future researchers to reproduce the results presented in this paper using similar computational setups.
REFERENCES
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 7243-7252, 2017.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International conference machine learning, pp. 274-283. PMLR, 2018.
Daniel Auge, Julian Hille, Etienne Mueller, and Alois Knoll. A survey of encoding techniques for signal processing in spiking neural networks. Neural Processing Letters, 53(6))4693-4700, 2021.
Andy Brock, Soham De, Samuel I. Smith, and Karen Simonyan. High-performance large-scale image recognition without normalization. In International conference on machine learning, pp. 1059-1071. PMLR, 2021.
Tong Bu, Jianhao Ding, Zecheng Hao, and Zhaofei Yu. Rate gradient approximation attack threats deep spiking neural networks. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 7896-7906, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248-244880 245-255.. Ieee, 2009.
Terrance Vries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv: 04552, 2017.
Jianhao Ding, Tong Bu, Zhaofei Yu, Tiejun Huang, and Jian Liu. Snn-rat: Robustness-enhand spiking neural network through regularized adversarial training. Advances in Neural Information Processing Systems, 11-27780-247933 2022.
Jianhao Ding, Zhiyu Pan, Yujia Liu, Zhaofei Yu, and Tiejun Huang. Robust stable spiking neural networks. arXiv preprint arXi:: 20694, 2024a.
Jianhao Ding, Zhaofei Yu, Tiejun Huang, and Jian K Liu. Enhancing the robustness of spiking neural networks with stochastic gating mechanisms. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 492-502, 2024b.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 14-10056-21069 2021.
Wei Fang. Yanqi Chen, Jianhao Ding, Zhaofei Yu, Timothée Masquelier, Ding Chen, Liwei Huang, Huihui Zhou, Guoqi Li, and Yonghong Tian. Spikingjelly: An open-source machine learning infrastructure platform for baike-based intelligence. Science Advances, 9(40) 1480, 2023.
Hejia Geng and Peng Li. Hosnn: Adversarially-robust homeostatic spiking neural networks with adaptive firing thresholds. arXiv preprint 2tt.::220..113..... 2023.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014.
David Heeger et al. Poisson model of spike generation. Handout, University of Standford, 5(1-13): 76, 2000.
Xiaoyang Jiang, Qiang Zhang, Jingkai Sun, Jiahang Cao, Jingtong Ma, and Renjing Xu. Fully spiking neural network for legged robots. In ICASSP 2025-2025 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2025.
11

---

Published as a conference paper at ICLR 2026
Youngeun Kim, Hyoungscob Park, Abhishek Moitra, Abhiroop Bhattacharjee, Yeshwanth Venkatesha, and Priyadarshini Panda. Rate coding or direct coding: Which one better for accurate, robust, and energy-efficient spiking neural networks? In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 71-75. IEEE, 2022.
Max Kochurov, Rasul Karimov, and Serge Kozlukov. Geoopt: Riemannian optimization in pytorch. arXiv preprint arXiv:2005.02819 2020.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Aakash Kumar, Lei Zhang, Hazrat Bilal, Shifeng Wang, Ali Muhammad Shaikh, Lu Bo, Avinash Rohra, and Alisha Khalid. Dsqn: Robust path planning of mobile robot based on deep spiking q-network. Neurocomputing, 634: 129916, 2025.
Souvik Kundu, Massoud Pedram, and Peter A Beerel. Hire-snn: Harnessing the inherent robustness of energy-efficient deep spiking neural networks by training with crafted input noise. In Proceedings of the IEEE/CVF international conference computer vision, pp. 5209-5218, 2021.
Chankyu Lee, Syed Shakib Sarwar, Priyadarshini Panda, Gopalakrishnan Srinivasan, and Kaushik Roy. Enabling spike-based backpropagation for training deep neural network architectures. Frontiers in neuroscience, 14:497482, 2020.
Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in neuroscience, 10:508, 2016.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li. and Luping Shi. Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience, 11:244131, 2017.
Shiqiang Liu, Weisheng Li, Dan He, Guofen Wang, and Yuping Huang. Ssefusion: Salient semantic enhancement for multimodal medical image fusion with mamba and dynamic spiking neural networks. Information Fusion, 119:103033 2025.
Yanzhen Liu, Zhijin Qin, and Geoffrey Ye Li. Energy-efficient distributed spiking neural network for wireless edge intelligence. IEEE Transactions on Wireless Communications, 2369):10633-16677, 2024a.
Yujia Liu, Tong Bu, Jianhao Ding, Zecheng Hao, Tiejun Huang, and Zhaofei Yu. Enhancing adversarial robustness in snns with sparse gradients. arXiv preprint -xXiv:2405.20355 2024b.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models.
Neural networks, 10(9):1659-1671 1997.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint (edi:::770.0000 2017.
Zakaria Mhammedi, Andrew Hellicar, Ashfaqur Rahman, and James Bailey. Efficient orthogonal parametrisation of recurrent neural networks using householder reflections. International Conference Machine Learning, pp. 2401-2409. PMLR, 2017.
Bhaskar Mukhoty, Hilal AlQuabeh, and Bin Gu. Improving generalization and robustness snns through signed rate encoding and sparse encoding attacks. In The Thirteenth International Conference on Learning Representations, 2025.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51 1666)) 2019.
Yi Pan, Hanqi Jiang, Junhao Chen, Yiwei Li, Huaqin Zhao, Yifan Zhou, Peng Shu, Zihao Wu, Zhengliang Liu, Dajiang Zhu, al. Eg-spikeformer: Eye-gaze guided transformer spiking neural networks for medical image analysis. In 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI), pp. 1-5. IEEF, 2025.
12

---

Published as a conference paper at ICLR 2026
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
Bodo Rueckauer, Iulia- Alexandaa Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conver- sion of continuous-valune valued deep networks to efficient event-driven networks for image classification. Frontiers in neuroscience, 11:682, 2017.
Ana Stanojevic, Stanislaw Wožniak, Guillaume Bellec, Giovanni Cherubini, Angeliki Pantazi, and Wulfram Gerstner. High-perfornance deep spiking neural networks with 0.3 spikes per neuron. Nature Communications, 15(1):6793, 2024.
David Stutz, Matthias Hein, and Bernt Schiele. Relating adversarially robust generalization to flat minima. In Proceedings of the IEEE/CVF international conference on computer vision, pp.
7807-7817, 2021.
Alberto Viale, Alberto Marchisio, Maurizio Martina, Guido Masera, and Muhammad Shafique. Carsnn: An efficient spiking neural network for event-based autonomous cars on the loihi neuro- morphic research processor. 2021 International Joint Conference on Neural Networks (IJCNN),
pp. 1-10. IEEE, 2021.
Phoenix Neale Williams and Ke Li. Black-box sparse adversarial attack via multi-objective optimisa- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12291-12301, 2023.
Mengting Xu, De Ma, Huajin Tang, Qian Zheng, and Gang Pan. Feel-snn: Robust spiking neural networks with frequency encoding and evolutionary leak factor. Advances in Neural Information Processing Systems, 37:91930-91999 2024.
Guanlei Zhang, Lei Feng, Fanqin Zhou, Zhixiang Yang. Qiyang Zhang, Alaa Salch, Praveen Kumar Donta, and Chinmaya Kumar Dehury. Spiking neural networks in intelligent edge computing. IEEE Consumer Electronics Magazine, 2024.
Rui-Jie Zhu, Ziqing Wang, Leilani Gilpin, and Jason Eshraghian. Autonomous driving with spiking neural networks. Advances Neural Information Processing Systems, 37:136782-136804 2024.
13

---

Published as a conference paper at ICLR 2026
A GRADIENT OF BPTT AND SURROGATE GRADIENT
In this section, we present the gradient formulation in the BPTT process, along with the surrogate gradient method employed.
In BPTT, let W denote the weight matrix between layer / and layer I 1, and let L represent the loss function. The gradient can then be expressed
DC oW
Oc
0z [t] [t)
(u)+
au aut
du Ou [i 1]
as a Oul
[i]
Σ ktt
II
Os
OW
Oul+
0s [1] Ou + [
oW
(10)
where the non-differentiable term ds tl as the surrogate gradient (Fang et al.
the piecewise quadratic function (triangle function) 2023; Neftei et al. 2019), which defined as:
0,
a
g((x)
(11)
a² | +
[x| S
The primitive function is defined as:
0.
I <
(xx) =
a
1 0z
|x| <
(12)
21
1.
where constant is set to 1 for all cases.
B ANALYSIS OF GRADIENT TEMPORAL COLLINEARITY IMPACTS NETWORK ROBUSTNESS
In this section, we analyze what impact GTC will have on network robustness. This analysis is divided into two parts. In the first part, we analyze the impact of GTC on the spectral radius of the network parameter Hessian. In the second part, we analyze the influence of spectral radius on the robustness of the SNN network.
Impact of GTC on Hessian Spectral Radius. For a single sample (æ, y), let z fo(x) be the network output and let the sample loss be E(x, y; 0). Given batch B {(2,, H:) the overall objective is
B (()) Σ
(xx,, ys: 0),
(13)
i=1
where 8 the set of network parameters given by Definition 7
We first recall the
form of the parameter Hessian for single sample (z,v):
Ho Jo(=)) H=(fo(z):)) Jo(x),
(14)
where Jo (f) the parameter Jacobian, and H2 8Ll(z n)/0²² the Hessian of the loss ((2, y) with respect to the network output fo()). Then we analyze the temporal structure
under BPTT. The network output can be written the sum of contributions across time steps fe(x) = Ein f (I), and correspondingly we decompose
T
Jo
Jr,
Jr 01=
(15)
so that the Gauss-Newter Hessian becomes
11.
J.
(16)
Eq. 16) makes explicit that the Hessian contains both diagonal terms (t = t') and cross-terms + ('). where the latter are directly determined by the temporal corrllation of gradient components.
14

---

Published as a conference paper at ICLR 2026
Next, introduce notational substitution Zz H,!2 Je absorb the eurvature weighting into each time block, so that
He
Zt
21):
Amax (Hp)
2
Z1
(17)
1=1
/=
op
We then define the temporal Gram matrix
Te TERT TxT,
To (Z,, Zj)
(18)
so that E Zr11i =
- 11, where 1 indicates vector with all elements of 1. Using the Rayleigh
quotient, have
1 S T (emase())
(19)
and therefore
Amax (Pp) T Xmax (T).
(20)
After obtaining this bound, we continue to further bound Amax Let M E RTxT denote the normalized correlation matrix with
Zj)r
Mij | 11
Ma = 1,
(21)
and let = diag ( [2, (TTlF)) so that T = DMD. Then by the submultiplicatiiiy of the
operator norm have
mmax (T) < mmax 12.112)
Amax(M)
(22)
Applying Gershgorin's circle theorem, we obtain
Amax(M) S + (T max (Magl #f
(23)
Combining Eqs. 20
22 and 23 we arrive at the structured upper bound
Amax Ho)
T
max | + (T-
- 1) max MMy]]
(24)
if)
Finally, we may replace My by GTC C(G)), G(il) and | | by || Hence we obtain
Amax (Ho) 2 T (max [[G|E]|?)
(T- 1) max C(G]), in
(25)
This structured bound makes explicit that the spectral radius of the parameter Hessian is determined jointly by the per-step gradient magnitudes and the temporal gradient similarities. In particular, when C(G GDi)) is close to 1 for all pairs, the gradients are nearly collinear across time steps, leading to near rank-1 structure that amplifies cross-terms and yields quadratic growth of the spectral radius with T; when C(G G(i) is small due to temporal decorrelation, the growth is closer to linear. Therefore, lowering the GTC directly reduces the structured upper bound of Amax (Ha).
Impact of the Hessian Spectral Radius on Network Robustness. The paper (Stutz et al. 2021 presents from an experimental perspective the conclusion that the larger the spectral radius, the less robust the network. We conduct a brief theoretical analysis of this conclusion based on SNN.
From the perspective of single sample, we analyze why an increase in the spectral radius of the network parameters amplifies the effect of input perturbations on the loss function, thereby indicating that the network's robustness deteriorates as the spectral radius grows.
Based on Eq.
13). for any input perturbation 8 with ||||| S c. Taylor's theorem at (ar y: 0) gives ((x + 6, y: 0) E(x, 0) V,l(z, y: 8) 5
Hz(x,v:0) o ((18)11))
(26)
where Hz y; 0) = V 0) is the input Hessian for the sample.
By the Rayleigh-quotieen bound 8 H28 < (mmx((H))lll where Xmaa (-) the spectral radius for symmetric matrices, we obtain the sample-wise worst-case bound
sup (o(x + 9, 0) f(x, v) < s||V 0) + 1/2 Amar( [e
(27)
Near rank-1 refers the dominance of the leading eigenvalue of the temporal correlation matrix.
15

---

Published as a conference paper at ICLR 2026
Next, we should further obtain the relation between input Hessian (2, y: () and overall parameter Hessian Hg. Parameter Hessian can be expressed as
B Ho(0) = V2CC0) Σ (a(a))):0))
(28)
where [z, y; v; (2, yi 0) is the parameter Hessian for a single sample.
First, the input Hessian admits the exact decomposition
(z, 0) Jy(z)' (=(()()): (y(x) + R(x, y;
(29)
where Hz(fo(r); is the output Hessian, and Jy(x) = (f( is the input Jacobian. In the exact decomposition, the residual term R(x, y; 8) arises from the interaction between the loss gradient Val(x, y; 0) and the second derivatives 0² fa)) /åx² of the network with respect to the input. While this term may be nonzero in general, near convergence the loss gradient becomes small, and thus its contribution can be upper- hopude- by a constant 2 0, i... [[][ 0)] S r). For the cross-entropy loss, which is generally used in SNN fields, the softmax Hessian takes the form diag(p) pp which is positive semidefinite in the de-centered subspace orthogonal to the all-ones vector. All nonzero eigenvalues of this Hessian lie in the interval [0. 1]. Therefore, we can take
an upper bound on the spectral norm of H., i.e., H S B, and proceed with the subsequent analysis restricted to this subspace.
On the other hand, for the input Hessian Hz its positive semidefiniteness cannot be guaranteed in general. This because the nonlinearity the network may lead to non-convexity. Consequently, we can conclude that its spectral radius is bounded above by its spectral norm, namely,
Amax H (z, y: 0)) S (H, y: 0)
(30)
By combining Eqs.
29 and 30 with the above analysis, we obtain the relation as follows:
Amax (x
H (I, 0)
[+) |H [[[x, y: 0) 3 ( + (31)
After obtaining the crucial incquality relation as Eq. (3I. We further analyze the relationship between the input Jacobian (x) = (z) /öx and the parameter Jacobian Jo (z) = (z)
In SNN with L-layer feedforward network over T steps, the pre-activation membrane potential. For the 1-th layer at time t, we define
[1] diag (1)))))) Up. sup [d(u) 00;
(32)
where ø is surrogate gradient function, and Ug is the upper bound of surrogate gradient function, which is bounded value for generally used surrogate gradient functions, such as triangle, rectangle, and sigmoid. Based on Eq. (32). we can use chain rule across layer and time steps after unrolling the network and obtain
(x) II WW t=1
Dr Dj (1),))
(33)
hence, by ||]|t]]| < Us, we get
|, < (u%-
II Iwwa))".
(34)
The parameter Jacobian (o(x) stacks the blocks O (z) (Wi) Assuming co(x) > 0 is a non- degeneracy constant that aggregates lower bounds intermediate activations and on surrogate slopes over active neurons across layers/time, such that
|+((1)]| 2
co(x)
(35)
Substituting Eq.
35 into Eq.
34 yields
||,(()]|
(1)(()1
(36)
16

---

Published as a conference paper at ICLR 2026
with the input-dependent factor
U
K((x))
(37)
cul (x()
Next, we link || to Ho(0) via a structural inequality and Gauss-Newton method. On the parameter side, for the sample (z, y) we have
Ao(a,y;0)
Jo(a)) (J)())
+ Ro (2, y:
(38)
with higher-order residual Rg. Gauss-Newton regime, [| || is negligible, and thus we can get Eq. 39), where "z" is Loewner order.
No(a, v:0) = J9(2) (f(
(39)
Let Heff be the minimal eigenvalue of H₂ on the working subspace. Then
| z
Amax (Z,
(40)
Heff
Since Ho(0) = B Ei= (I ye; 0) and each (zi, v; 0) = 0 near minima, we have
Ho( = B Ho y:
Amard (Ho(O)) 2 B
Amarl (Mo,)
(41)
Combining Eq. 44 and Eq. yields
B Mett
[|o(z))11
Amax (Ha
(42)
Using Eq. (36) and Eq. 42
BK(x))
[x)(1)]
Amax Ho Ho)))..
(43)
Pelt
Substituting Eq.
43
into Eq.
3
gives
BBK(x))
Amax H2 (x, 0;
Amar (o)()) +
(44)
Peft
Putting Eq. 44 into the perturbation bound Eq.
27
we obtain the general inequality
0) e (8 (BB((x)² Amard (o +m) (45)
sup ( 0))
V V
Peft
~ 0, the worst-case loss increase reduces to At (near-)stationary points where V , yi 0)
sup (f(x + 0, f(x, y: 0)) [6
(BB((x)
Amax (Ho(0)) + n))
(46)
Heff
Although the terms Peft and KC(x) in Eq. 446 are influenced by the network's input and output, they are not directly coupled with Amax( Ho (0). Hence, they only affect the scaling factor on the right-hand side of the equation and do not alter monotonicity when treating Amax ( (0)) variable. Therefore, we conclude that under fixed perturbation strength the upper bound of the network loss fluctuation increases with the spectral radius Amax( Hgg robustness of the network is weakened as its spectral radius grows.
(0) as the
(H(()), (0)). In other words, the
C DATASET
CIFAR-10. The CIFAR-10 dataset (Kizheessyy et al. 2009) consists of 60,000 color images, each of size 32 32 pixels, divided into 10 different classes, such as airplanes, cars, birds, cats, and dogs. Each class has 6,000 images, with 50,000 images used for training and 10,000 for testing. Normalization. random horizontal flipping, random cropping with 4 padding, and CutOut (DeVries & Taylor 2017) applied for data augmentation.
17

---

Published as a conference paper at ICLR 2026
CIFAR-100. The CIFAR- 100 dataset (Krizhevsky et al. 2009) consists of 60,000 color images, each of size 32x32 pixels, categorized into 100 different classes. Each class contains 600 images, with 500 used for training and 100 for testing. The same processing methods as for dataset CIFAR-10 1- applied to dataset CIFAR-100.
ImageNet. We evaluate the ILSVRC-2012 ImageNet dataset (Deng al. 2009). which contains ~ 1..28M training images and 50,000 validation images spanning 1,000 classes. Images are of variable resolution; following common practice and our implementation, training augmentation includes RandomResizedCrop to 224 x 224,
conversion to tensors, and channel-wise normalization. For test, images are resized have a shorter side of 256 pixels and then center-cropped to 224 x 224 before applying the same normalization.
DVS-CIFARIO. The DVS-CIFAR-I0 dataset (Li et al. 2017 is a neuromorphic version of the traditional CIFAR- 10 dataset. DVS-CIFA 10 captures the visual information using Dynamic Vision Sensor (DVS), which records changes the scene as series of asynchronous events rather than as sequence of frames. The dataset consists of recordings of 10 object classes, corresponding to the original CIFAR-10 categories, with each object presented in front DVS camera under various conditions. The dataset contains 10,000 128x 128 images, of which 9,000 are used the training set and the remaining 1,000 as the test set.
The DVS-Gesture dataset (Amir et al. 2017 is neuromorphic dataset, consisting of 11 different hand gesture classes, such as hand clapping, arm rolling, and air guitar, performed by 29 subjects under various lighting conditions. Each gesture is represented by sequence of events rather than frames. The dataset contains 1, 1,176 training samples and 288 testing samples.
D EXPERIMENTAL SETUP
D.1 EXPERIMENTAL SETUP FOR GTC EVALUATION
In the GTC evaluation experiments, we consider both direct encoding and rate encoding. In direct encoding, identical input data are fed into the network at every time step. For rate encoding, we adopt Poisson encoding (Lee et al. 2020), where input pixel values converted into Poisson-distriuteed spike trains and presented to the network. Typically, direct encoding requires fewer time steps (e.g., evaluate both encoding methods under time steps T = 4, 8, 32. For both methods, the data preprocessing follows Appendix C and the hyperparameters used for evaluation are listed in the following table. All other experimental settings not mentioned are identical to those described in Appendix D.2
or 8), whereas rate encoding requires more (e.g., 32). To ensure rigorous comparison,
Table 6: Hyperparameter settings for GTC evaluation experiments.
Dataset
Optimizer
Model
LeaningRate
Weigh
Epoch
BatchSize
CIFAR-I0
SGD
VGG-11
5e-5
200
128
CIFAR-100
SGD
VGG-11
0.1
5e-4
200
128
D.2 EXPERIMENTAL SETUP FOR MAIN EXPERIMENTS
In our experiments, all training cases are implemented using PyTorch (Paszke et al. 2019) with the SpikingJelly (Fang et al. 2023) framework and executed For each dataset, we utilize the hyperparameters listed as the following table, consistently employing the SGD optimizer for network parameters and RiemannianSGD optimizer for orthogonal kernels and the membrane time constant to 1.1. Unless otherwise specified, the ablation studies are conducted under the same experimental settings as the main experiments.
NVIDIA GeForce RTX 5090 GPU.
E DETAILED EXPERIMENTAL RESULTS
This section first presents comparison of method with SOTA approaches under AT in terms of robustness against white box attacks (Table g). We then provide the complete experimental results
used generate Fig. 3 (Table 9. Fig. 5 (TTble 10 and Fig. (Table [1) in the main text.
18

---

Published as a conference paper at ICLR 2026
Table Hyperparameter settings for main experiments. 'LR" denotes the learning rate, 'LRO' the
learning rate for orthogonal kernels. NF-ResNet-188 (Brock et al. 2021
Dataset
Model
Timestep
LR
LRO
WeighiDecuy
Epoch
ButchSize
CIFAR-I0
VGG-11
0.1
0,05
5e-5
200
128
0.05
CIFAR-100
VGG-11
0.1
5e-4
128
ImageNet
NF-ResNet-18*
0.1
0.05
lc-5
150
512
16
0.05
DVS IVS-CIFAR10
VGG-11
10
0.05
0.05
5e-4
200
128
8
0.05
DVS-Gesture
VGG-11
20
0.05
0.05
5e-5
200
8
0.05
Table 8: White box performance (with AT) comparison. The highest accuracy in each column is highlighted in bold. indicates self-impplmeenttiion results.
CTFAR-10
CIFAR-
ImageNet
Method
Clean
FGSM
PGD
Clean
FGSM
PGD
Clean
FGSM
PGD
SNN AT Kundu
2021
91.16
38.20
14.07
69.69
16.31
8.49
51.00
15.74
6.39
DLIF + AT
Ding ct al.
2024a
88.91
56.71
40.30
66.33
36.83
24.25
HoSNN
Geng & Li 2023
90.00
63.98
43.33
64.64
26.97
16.66
FEEL + AT [Xu et al.
2024
69.79
18.67
11.07
StoG AT Ding al. 2024b
90.13
45.74
27.74
66.37
24.45
14.42
STOD AT w.o. OK (Ours)
89.70
68.79
39.99
68.02
39.76
24.31
51.92
23.86
8.43
STOD AT (Ours)
88.94
73.23
43.54
67.19
41.89
27.94
50.68
26.77
9.80
F ADDITIONAL PARAMETER NUMBER ASSESSMENT
It is important to note that the orthogonal kernels in our method can be discarded during inference, ensuring that additional inference overhead is introduced. Specifically, in Table of the main experiments, we compare STOD without orthogonal kernels (STOD w.o. OK) against other SOTA methods. Table without orthogonal kernels.
further report the performance gap between inference with and
In this section, calculate the additional number of parameters introduced by the orthogonal kernels. Given that the kernel dimension d = C x x p. in an SNN with T time steps, the extra parameters N Npmm can be computed as:
para T(Cp2)2
(47)
Therefore, when C 3 and 7 4, if set p 2, 16, 32, the additional number parameter would be 576, 9216, 0. 0.15M, 2.36M, and 37.75M, respectively. As shown, when p = 8, only 0.15M additional parameters are introduced, which is nearly negligible.
G STATEMENT OF LARGE LANGUAGE MODEL (LLM) USAGE
In the preparation of this manuscript, an LLM was employed to assist with non con-scientific tasks. These included polishing the English writing for clarity and style, providing suggestions for figure design and color schemes, supporting 16TμX formatting and typesetting, and drafting this statement.
19

---

Published as a conference paper at ICLR 2026
Table Performance of STOD with different attack methods. WB and BB denote white and black box, respectively. This detailed experimental results of Fig.
Attack
6
16
32
128
CIFAR-10
SNN FGSM WB
93.75
17.06
14.22
11.78
8.19
4.21
1.99
0.58
0.00
SNN FGSM BB
93.75
24.13
18.41
13.86
10.26
7.88
3.82
1.48
0.59
SNN PGD WB
93.75
2.37
1.01
0.34
0.03
0.00
0.00
0.00
0.00
SNN PGD
93.75
4.01
2.80
1.20
0.02
0.00
0.00
STOD FGSM WB
90.87
63.71
62.08
60.04
59.16
51.49
39.45
27.49
8.17
STOD FGSM BB
90.87
84,36
80.88
77.74
75,42
67.91
50.09
33.18
13.95
STOD PGD WB
90.87
41.10
38.89
37.29
36.72
23.07
6.19
0.04
0.00
STOD PGD BB
90.87
60.42
56.29
48.70
44.24
32.96
14.31
1.07
0.51
CIFAR-100
SNN FGSM WB
72.39
9.37
7.42
5.46
4.55
2.35
1.19
0.33
0.00
SNN FGSM BB
72.39
13.26
12.11
10.84
9.16
5.31
2.07
1.47
0.49
SNN PGD WB
72.39
2.53
1.15
0.50
0.02
0.00
0.00
0.00
SNN PGD BB
72.39
3.65
2.68
1.89
0.78
0.15
0.02
0.00
0.00
STOD FGSM WB
70.69
34.15
33.56
32.99
32.15
18.79
9.44
3,00
0.71
STOD FGSM BB
70.69
49.44
46.30
44.25
41.96
29.22
17.40
10.22
2.27
STOD PGD
70.69
23.42
20.00
17.91
15.02
5.55
0.02
0.00
0.00
STOD PGD BB
70.69
33.60
31.08
28.58
25.90
13.46
2.40
0.46
0.00
ImageNet
SNN FGSM WB
57.84
10.75
8.59
6.73
4.99
1,56
0.14
0.01
0.00
SNN FGSM
57.84
12.13
11.43
10.25
8.46
5,42
3.93
1.25
0.35
SNN PGD WB
57.84
1.02
0.48
0.13
0.01
0.00
0.00
0.00
0.00
SNN PGD BB
57.84
9.19
6.05
4.06
3.67
2.02
1.79
0.05
0.00
STOD FGSM WB
53.57
25.10
23.65
21.99
20.93
12.68
6.43
3.65
0.17
STOD FGSM BB
53.57
30.59
27.75
23.64
22.50
17.94
9.61
5.64
2.70
STOD PGD WB
53.57
13.86
11.38
9.71
8.01
4.52
1,36
0.02
0.00
STOD PGD BB
53.57
18.43
15.69
12.23
10.98
7.98
2.29
0.51
0.02
Table 10: Performance of STOD with different time steps and patch sizes. This is detailed experimen- tal results of Fig. 5
T=2
T=4
T=6
T=8
p=2
16
32
8
RGSM
47.88
51.26
5351
50.58
45665
55.41
55.80
50.54
47.96
89.19
51.27
54. 89
52.35
50.0
47.50
51.26
52.95
56.01
$3.89
56.77
55.50
5133
57,80
16
8584
55
57.10
54.83
96.93
57.00
99.74
60.12
PGD
1989
21.92
2540
21.1
7172
24.35
52.30
92.7
31.11
24
25.32
31.33
36.55
34.99
26.63
25.95
29.56
51.34
33,89
30.40
2632
34.0
36.72
3542
38.19
36,00
Table 11: Performance of STOD with different PGD step number. results of Fig.
This is detailed experimental
Dataset
K
20
50
80
90
CIFAR-I0
36.72
35.44
34.98
33.80
32.17
31.59
31.02
30.97
30.95
30.94
30.93
30.93
CIFAR-100
15.02
14.20
13.50
13.09
12.87
12.62
12.59
12.57
12.56
12.56
12.56
12.56
ImageNet
8.01
7.29
6.77
6.20
5.92
5.71
5.65
5.63
5.62
5.62
5.62
20


Reviews:
[{"review_id": "BkZ0BkUBST", "reviewer": "Reviewer_Tj2T", "summary": "This paper investigates robustness issues in Spiking Neural Networks (SNNs) under direct encoding, a promising alternative to rate encoding due to its efficiency. The authors introduce Gradient Temporal Collinearity (GTC), a metric that quantifies temporal gradient alignment, and theoretically show that high GTC contributes to reduced robustness via analysis linked to the Hessian spectral radius. To mitigate this, the paper proposes Structured Temporal Orthogonal Decorrelation (STOD), which applies parametric orthogonal kernels with structured constraints at the input stage to decorrelate temporal features. STOD is applied only during training and can be removed at inference, incurring no runtime cost. Experiments on visual classification benchmarks show improved robustness over state-of-the-art methods.", "questions": "see weaknesses 2-5", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper proposes GTC and provides a theoretical analysis that connects the encoding strategy with robustness through the derivation of the Hessian spectral radius, demonstrating strong theoretical depth.\n2. STOD performs orthogonal transformation only during the input stage of training; after training it can be removed, resulting in low deployment cost and no impact on inference efficiency.", "weaknesses": "1. In the definition of GTC, several important formulations should be presented in the main text.\n2. The direct connection between GTC and robustness should be further validated experimentally under other encoding schemes such as temporal or latency encoding.\n3. It remains unexplained why the GTC curve of Rate Coding does not decrease during training, whereas that of Direct Coding gradually becomes smoother and decreases.\n4. Although gradient reshaping methods are known to reduce accuracy on clean-dataset, the paper does not analyze why its performance on clean-dataset is lower than other methods or why the degradation is larger.\n5. Since STOD orthogonal transformation is removed during validation, it is unclear whether this might cause overfitting during training.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "wOA7xUsA7V", "reviewer": "Reviewer_PFkG", "summary": "This paper proposes Structured Temporal Orthogonal Decorrelation (STOD), a framework designed to mitigate Gradient Temporal Collinearity (GTC) - the tendency of gradients in Spiking Neural Networks to align their directions across time steps, where high GTC in direct encoding results in vulnerability to adversarial perturbations. \nSTOO combines two complementary modules: Patchwise Feature Diversification (PFD), which enforces local orthogonality in kernel transformations to preserve intra-step feature independence; and Global Orthogonal Regularization (GOR), which introduces a feature-level angular diversity loss — a cosine-based soft constraint (Eq. 8) — to penalize similarity between normalized transformed features across time steps. This combination jointly ensures orthogonal kernel dynamics and temporally diverse representations, leading to robust and decorrelated spatio-temporal learning. \nEmpirical evaluations on CIFAR-10/100 and ImageNet under multiple adversarial attack scenarios demonstrate consistent improvements without additional inference overhead.", "questions": "1. Could you provide ablation results isolating the impact of PFD and GOR individually? It is difficult to evaluate their relative contributions without such analysis. \n2. In Fig. 4, the comparison between direct encoding and STDO shows only marginal visual differences, and the transformed features via PFD across time steps also appear similar. Could you provide quantitative evidence or clearer visualizations demonstrating improved gradient diversity? \n3. Eq.(7) defines the canonical basis $e_1, …, e_d$, yet the index range for $j$ ($[1,T]$ vs. $[1,d]$) remains ambiguous. How is this handled to ensure valid kernel initialization?\n4. The inference phase introduces no additional overhead because learned orthogonal kernels are discarded. Can you clarify whether inference without these kernels still maintains the same level of temporal decorrelation? \n5. The paper focuses exclusively on static natural-image datasets. While this choice aligns with the motivation around direct encoding, have you considered extending STOD to other natural input domains (e.g., sensor time-series, audio, etc.), where gradient temporal collinearity may also naturally arise?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Clearly identifies GTC as a key cause of robustness degradation in SNNs and provides an intuitive explanation of its geometric implications, supported by experiments (in Fig.1).\n- Integrates local (PFD) and global (GOR) components to address temporal alignment of gradients from complementary kernel- and feature-level perspectives, offering a coherent and theoretically consistent framework.\n- Evaluates robustness under diverse attack settings (e.g., FGSM, PGD and both white- and black-box attacks) and across multiple datasets, presenting consistent gains over baselines.\n- Effectively connects temporal gradient decorrelation to robustness improvement while maintaining no inference overhead due to removal of orthogonal kernels at test time.", "weaknesses": "- Some mathematical formulations are ambiguous: in Eq.(7), the index $j$ range in the Householder reflection is unclear ($j \\in [1,T]$ vs. $j \\in [1,d]$), limiting reproducibility and interpretability of the initialization process. \n- Lacks ablation studies analyzing the independent contributions of PFD and GOR, making it difficult to assess which component drives the main performance gains in STOD.\n- Although the motivation contrasts direct and rate encoding, there is no quantitative evidence that STOD-enhanced direct encoding exhibits rate-like decorrelation characteristics. Fig.4 includes comparisons, but those differences are visually small and not statistically analyzed.\n- Experiments are limited to static image benchmarks, which aligns with the focus on input encoding but leaves open whether the proposed decorrelation mechanism generalizes to other domains of natural inputs (such as audio, time-series, etc.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AhZHL9AwTD", "reviewer": "Reviewer_ZQzg", "summary": "The authors study the resistance of spiking neural networks (SNNs) to adversarial attacks (white box and black box).\n\nThey argue that direct input coding, which outperforms Poisson rate coding in terms of accuracy, makes the SNNs less resistant because of strong correlations across timesteps.\n\nThey propose a new method to decorrelate timesteps, and thus increase resistance to adversarial attacks.", "questions": "The size of the orthogonal kernels is d in the text, but c x p x p on Fig 1. So you choose d = c x p x p?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "As far as I know, the method is new, and it outperforms other approaches for white box attacks (Table 1).", "weaknesses": "I am not convinced, because a number of comparisons are missing:\n\n* Comparison to SOTA (Table 1) is limited to white box attacks. How about for black box attacks?\n\n* The authors should also compare their method to Poisson rate coding.\n\n* How about using direct input coding, but adding white noise? This should also decorrelate time steps (similar to Poisson rate coding). One could vary the noise amplitudes. Maybe there is a sweet spot in terms of amplitude, the authors should investigate.\n\n* An alternative to direct input coding has been proposed in\nhttps://dl.acm.org/doi/10.5555/3692070.3692856\nthe idea is simple:\na simple 2d conv is applied to the input image with:\nCin = 3 (RGB channels)\nCout = T\nthe output is [T, W, H] then at each t timestep t we they feed the image [t, :, :] to the SNN.\nThis could also decorrelate timesteps.\nIt would be interesting to compare the proposed approach to this previous one, in terms of accuracy and resistance to attacks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WC0aNSKhOM", "reviewer": "Reviewer_6hhX", "summary": "This paper aims to improve the robustness of spiking neural networks (SNNs) under direct coding. The authors first proposes a metric called Gradient Temporal Collinearity (GTC) to measure robustness by how similar gradients are across time. Then the authors propose Structured Temporal Orthogonal Decorrelation (STOD), which applies orthogonal transformations to input features at each time step via Patchwise Feature Diversification and Global Orthogonal Regularization. Experiments on CIFAR-10, CIFAR-100, and ImageNet show that STOD achieves better robustness against adversarial attacks than baselines.", "questions": "1. The robustness improvements of STOD appear smaller on ImageNet compared to CIFAR datasets. Could the authors explain the underlying reasons for this difference? \n\n2. The experiments on black-box attacks and rate gradient approximation attacks did not include baseline comparisons. Could the authors clarify why these comparisons were omitted and whether STOD maintains its advantages when such baselines are included?\n\n3. What are the key differences between the proposed STOD and the baseline methods? While it is intuitive that STOD improves over vanilla SNNs, it remains unclear why it also outperforms other robust approaches.\n\n4. Could the authors empirically compare the robustness of direct coding with that of rate coding? Is the proposed STOD method still effective when applied to rate coding or other spike encoding schemes? \n\n5. In Figure 1, GTC gradually decreases under direct coding. What reasons contribute to this trend? \n\n6. How exactly is the gradient term G[i] in Eq. (3) computed? Does it aggregate gradients from all network parameters, or only from specific layers or subsets of weights?\n\n7. Why is improving robustness particularly crucial for SNNs compared to ANNs? The authors might consider emphasizing the significance and potential benefits of robustness for SNNs in the Introduction.\n\n8. Could the authors compare STOD with simpler baselines that inject random noise into features or gradients? \n\n9. Why are adversarial attacks not applied at every time step? Would the proposed method remain effective under this more challenging attack setting?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper provides a mathematical analysis linking GTC to the spectral radius of the Hessian, connecting gradient structure directly to network robustness.\n\n2. The proposed STOD framework improves robustness by decorrelating temporal features through orthogonal transformations. It delivers strong robustness gains on CIFAR-10, CIFAR-100, and ImageNet.\n\n3. The paper offers clear Interpretability through visualization.", "weaknesses": "1. The link between the proposed GTC metric and the STOD method appears somewhat indirect. The paper does not clearly articulate how insights from GTC specifically motivate the design of STOD, nor does it provide sufficient theoretical or empirical evidence showing that STOD directly reduces GTC in a quantifiable manner.\n\n2. The paper omits a detailed analysis of the computational overhead introduced by STOD during both training and inference. Without explicit comparisons of time or resource consumption against baseline methods, it is difficult to assess the practical efficiency and scalability of the proposed approach. \n\n3. The method introduces several hyperparameters (e.g., patch size, regularization strength), yet the paper does not investigate whether the optimal settings generalize across different architectures and datasets. This raises concerns about the generalizability of the approach in broader applications.\n\n4. The evaluation focuses primarily on static image datasets. However, SNNs are naturally well-suited for event-based or sequential data. The paper does not explore or discuss how STOD could be applied or adapted to event-based or sequential datasets, which limits the scope and generalizability of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)