Threaten Spiking Neural Networks through Combining Rate and Temporal Information
Authors: Zecheng Hao
Tong Bu
Xinyu Shi
Zihan Huang
Zhaofei Yu
Tiejun Huang
Venue: ICLR 2024 poster (2024)
Keywords: Spiking Neural Networks
Rate and Temporal Information
Hybrid Adversarial Attack
Primary area: applications to neuroscience & cognitive science
TL;DR: We propose a hybrid adversarial attack based on rate and temporal information (HART) for Spiking Neural Networks, whose rate and temporal attribute are dynamically adjustable.

Abstract: Spiking Neural Networks (SNNs) have received widespread attention in academic communities due to their superior spatio-temporal processing capabilities and energy-efficient characteristics. With further in-depth application in various fields, the vulnerability of SNNs under adversarial attack has become a focus of concern. 
In this paper, we draw inspiration from two mainstream learning algorithms of SNNs and observe that SNN models reserve both rate and temporal information. To better understand the capabilities of these two types of information, we conduct a quantitative analysis separately for each. In addition, we note that the retention degree of temporal information is related to the parameters and input settings of spiking neurons. Building on these insights, we propose a hybrid adversarial attack based on rate and temporal information (HART), which allows for dynamic adjustment of the rate and temporal attributes. Experimental results demonstrate that compared to previous works, HART attack can achieve significant superiority under different attack scenarios, data types, network architecture, time-steps, and model hyper-parameters. These findings call for further exploration into how both types of information can be effectively utilized to enhance the reliability of SNNs. Code is available at [https://github.com/hzc1208/HART_Attack](https://github.com/hzc1208/HART_Attack).

Published as a conference paper at ICLR 2024
THREATEN SPIKING NEURAL NETWORKS THROUGH COMBINING RATE AND TEMPORAL INFORMATION
Zecheng Hao', Tong B Xinyu Shi Zihan Huang', Zhaofei Yu! & Tiejun Huang!
School of Computer Science, Peking University
2 Institute for Artificial Intelligence, Peking University
ABSTRACT
Spiking Neural Networks (SNNs) have received widespread attention in academic communities due to their superior spatio-temporal processing capabilities and energy-efficient characteristics. With further in-depth application in various fields, the vulnerability of SNNs under adversarial attack has become a focus of concern. In this paper, we draw inspiration from two mainstream learning algorithms of SNNs and observe that SNN models reserve both rate and temporal information. To better understand the capabilities of these two types of information, we conduct a quantitative analysis separately for each. In addition, we note that the retention degree of temporal information is related to the parameters and input settings of spiking neurons. Building on these insights, we propose hybrid adversarial attack based on rate and temporal information (HART), which allows for dynamic adjustment of the rate and temporal attributes. Experimental results demonstrate that compared to previous works, HART attack can achieve significant superiority under different attack scenarios, data types, network architecture, steesstess,ae and These findings call for further exploration into how both types of information can be effectively utilized to enhance the reliability of SNNs. Code is available at httpt::://// / http///gtttttttt//// thub. com /aac/ AOR/HART, HART_Attack Attack
model hyper-paramette
INTRODUCTION
Due to the unique event-driven property (Bohte et al. 2000) and superior biological plausibility (Ger stner al. 2014), Spiking Neural Networks (SNNs) acclaimed as the third generation of artificial neural networks (Maass 1997) and have received extensive academic attention. Unlike traditional Analog Neural Networks (ANNs), SNNs utilize discrete binary sequences for communication between spiking neurons, with spikes being emitted only when the membrane potential exceeds the firing threshold. This characteristic makes SNNs well-suited for processing
data (Zhang & Li (020) and offers benefits such as reduced power consumption (Tavanaei et al. 2019 Zenke et al. 2021p. In addition to image classification applications (Cao et al. 2015). SNNs have found utility in other areas such as text recognition (Lv et al. 2023) and object detection (Kim et al. 2020b), etc.
With the increasing deployment of SNNs on mobile devices and neuromorphic hardware (Davies Cet al. 2018 DeBole et al. 2019),, ensuring the security and reliability of SNNs has become a critical concern. While SNNs exhibit stronger robustness compared to ANNs (Sharmin et al. 2020 Kundu ct al. 2021 Ding al. 2022), they still susceptible to imperceptible attack samples crafted from learning gradient of SNNs, leading to potentially disastrous consequences, especially in safety-critica scenarios. Currently, both ANN-SNN conversion (Cao et al. 2015 Li et al., (02T) and Spatial- Temporal back-propagation (STBP) (Wu et al. 2018) can be leveraged for training SNNs, which respectively utilize the rate and temporal information of SNNs (Meng et al., [022] Zhu et al. (202)). Here rate information mainly denotes the approximate linear transformation relationship between the average firing rate of adjacent layers, while temporal information refers to the information with time dimensions such as the order of spike sequence and the value of membrane potential at each time step. To achieve better attack performance, which method should be used to calculate the learning gradient? How should we utilize these two types of information reasonably?
Corresponding author; yuzf1 @@pku.edu.cn

---

Published as a conference paper at ICLR 2024
Researchers have discovered that adopting rate or temporal information auxiliary knowledge can further optimize the performance of SNNs (Kim et al. [020aa] Wang et al. [002] Xu et al., (023). In addition, theoretical validations have indicated that both rate and temporal information contribute to the overall characteristics of spike sequences (Panzeri & Schultz 2001). The above findings hint that SNN models have the potential to retain and leverage both types of information, which is significant difference between SNNs and ANNs. However, previous works (Sharmin et al. 2020 Ding et al., 2022] Bu et al. 2023) have not adequately designed gradient calculation methods to effectively integrate rate and temporal information into the adversarial attack framework of SNNs,
thereby keeping the potential safety hazard of SNN models from being fully revealed.
In this paper, we present an in-depth analysis of rate and temporal information in SNNs. To better threaten SNNs, we propose a novel hybrid adversarial attack based on both rate and temporal information (HART). Experiments demonstrate the superiority of our attack method compared to previous strategies. To the best of our knowledge, this is the first work that simultaneously applies rate and temporal information to the adversarial attack of SNNs. Our main contributions are follows:
We redefine the rate information gradient, and the attack based on this gradient achieves better performance than the ANN-SNN Conversion attack. We quantitatively analyze the retention degree of temporal information in SNNs and identify its correlation with factors such as the membrane decay constant and the number of time-steps.
We propose a hybrid adversarial attack based on both rate and temporal information (HART), which performs gradient pruning as well as merging on time dimension, offering an ad- justable surrogate gradient and a pre-calculation property.
We theoretically prove that our method has mathematical equivalence with the rate gradient. Additionally, we demonstrate the ability to control the temporal attribute of HART by adjusting the shape of the surrogate gradient curve.
Extensive experiments validate the effectiveness of HART. Our method achieves state-of- the-art attack success rate (ASR) across various aaper-prrameter settings for both static and neuromorphic datasets.
RELATED WORKS
Learning algorithms for SNNs. ANN-SNN conversion and STBP are currently the most widely adopted mainstream learning methods. Conversion methods are based on the principle of an approx- imately linear relationship between adjacent layers in SNNs, allowing researchers to train source
ANNs and then replace their activation layers with spiking neurons to obtain SNNs (Cao et al. 2015) Rueckauer et al. 2017 Han et al. 2020). Presently, the converted SNNs can achieve comparable
performance to ANNs on large-scale datasets with sufficient number of time-steps (Deng & Gu 2021). However, due to the presence of residual membrane potential, the mapping from ANNs to SNNs is not completely precise, resulting in performance degradation for converted SNNs under ultra-low time latency (Bu et al. 2022. Li et al., 2022] Hao et al., 2023a]b). Inspired by the back- propagation through time training mode in recurrent neural networks, researchers have proposed STBP for training SNNs (Shrestha& & Orchard 2018 Wu et al., 2018 which is a supervised learning algorithm that incorporates time dimension. To tackle the non-differentiable issue during the spike firing procedure, surrogate gradient and various smoothing functions have been introduced (Neftci et al. 2019: Fang et al. 2021). In addition, hybrid training methods that leverage multiple types of information in SNNs have received widespread attention (Mostafa 2017 Kim et al., 2020a| Zhang & Li 2020 Wang et al. 2022). These hybrid approaches offer potential benefits in optimizing memory
overhead and energy consumption in SNNs (Xiao et al., 2022 Rathi & Roy 2023).
Robustness and adversarial attack of SNNs. Compared to ANNs, SNNs are considered to possess
stronger robustness due to their capability to store diverse and rich information (Sharmin et al. 2020).
Previous works aimed improving the robustness of SNNs can be generally divided into two routes:
one strengthens the model's 's robustness by migrating classic defense strategies from ANNs to SNNs, such as certification training (Liang et al., 2022) and Lipschitz analysis (Ding et al., 2022), while the other enhances defense capabilities by exploring some unique encoding and model specific to SNNs, including Poisson coding (Kundu et al., 2021 Leontev et al. 202I) and membrane time constant (El-Allami et al.) 2021). Nevertheless, SNNs remain susceptible to adversarial attacks that exploit gradients learned during training. As both ANN-SNI conversion and STBP methods can be employed for model learning, /Sharmin et al. (2099) explored the attack performance of the gradient
2

---

Published as a conference paper at ICLR 2024
corresponding these methods and found that the STBP attack is more effective. In addition, the STBP attack scheme adapted to neuromorphic data has also been designed (Lin et (022 Marchisio et al. 202Tp. In contrast, [Bu et al. (2023) proposed method based rate gradient approximation, achieving higher attack success rate compared to STBP.
3
PRELIMINARIES
3.1 SPIKING NEURON MODELS
In this paper, adopt the commonly used Integrate- (IF) model and Leaky-Integratee and-Fire (LIF) model (Gerstner & Kistler 20022 Izhikevich) (004). The dynamic equations about membrane potential in discrete form be deseribed as follows (Brette et al. 2007).
m (1) X'v (1- 1) w'ss''(0
(1)
v (1) m'(1) -nn (n)(s)s
(2)
n/ (1) (m' (t) Urea),
hard-reset
soft-reset
(1, (0.
m' (t) > A!
of (t)
(4)
otherwise
At the f-th time-step, we use the notation m m'((t (t) and vt() to denote the membrane potential before and after triggering a spike, respectively. X is the membrane leaky constant. When X 1, the LIF model will degenerate into the IF model. s st)) determines whether to deliver spike and ny (t) indicates the type of reset for the neurons. Specifically, a hard-reset directly resets v'(t) to the resting potential Urest, while soft-reset subtracts the corresponding threshold 0 from v' (t). W denote the weight matrix in the (-th layer, and W (t) represents the input current from layer 1.
3.2 ADVERSARIAL ATTACK
Adversarial attack aims exploit the vulnerability of neural networks by introducing maliciously crafted input data with imperceptible perturbations, causing the target model to make incorrect predictions (Goodfellow al. 2015). Generally, it can be formulated an optimization problem:
arg max [(s(x + 6, W),)), s.t. ||01| <.
Here, 8 represents the adversarial perturbation, and its strength is constrained by L denotes the loss function based on the average firing rate. f()) is the network model under attack, with internal weight parameters W. and y denote the input images and the corresponding output target. In this paper, we consider two representative adversarial attack algorithms: the Fast Gradient Sign Method (FGSM) (Goodfellow al. (015)) and Projected Gradient Descent (PGD) (Kurakin al. (017). White-box and black-box attacks, which respectively refer to the situation that hackers have access to the knowledge about model topology and parameters or not, are considered simultaneously.
The FGSM perturbs the input data linearly in the direction of the sign gradient with respect to the loss function, while PGD is a more powerful variant of FGSM, which enhances the attack effectiveness by optimizing the perturbation route iteratively. These methods can be described as follows:
FGSM: á + sign (V Ca, W)))))
PGD: & TI (2 k-1
sign (V+C(f(** w),
(7)
Here, e limits the disturbance level of the input data, & represents the iteration number, and a is the step size for each iteration. II denotes the constrained e Ip neighborhood projection space for æ.
3.3 RATE INFORMATION SPIKING NEURAL NETWORKS
Rate information in SNNs mainly refers to approximate linear transformation relationship, similar ANNs, between the average firing rate of adjacent layers. Here we use pd r'T) E s (1(1)T to
3

---

Published as a conference paper at ICLR 2024
Casll 55
(')
Cas22
82
07)
Casel
Pate
Cas2
1/2
Cases
Grad
Case3
6C
82/2
W. (T)
10 (02)
Figure 1: A set of examples for utilizing rate information in SNNs.
denote the average firing rate in layer I, with T denoting the total number of time-steps. To simplify the expression, we adopt the soft-reset mechanism and set r' (0) = 0, 0' = for each layer SNNs.
By combining Eqs. [and 2 summing from & 1 to T, and dividing both sides by T, we get:
r (T) w (T) (T) + +1
(8)
T
Considering that (T) non-negative vector, the above equation resembles the calculation rule a' max( W'a' 1- a''aa1 00) ANNs with a' denoting the activation output in layer l. At this point, the residual term (w' (T) E
(L)) becomes the main gap from ANNs SNNs. Specially, when considering the IF model (X' 1), Eq. simplifies to r' (T) W v (T) /T, which forms the core idea ANN-SNN conversion methods (Cao et al. 2015).
The specific value of o' (V) E [1, is data-driven, resulting in varying residual terms for different neurons. When X = 1, the residual term gradually approaches zero as the time-step increases, allowing for the training of nigh-precision SÑNs in ANN-SNN conversion. However, this approach leads to performance degradation under ultra-low time latency conditions. For smaller values of X, the residual term may approach Therefore, from another perspective, we can consider the residual term as the temporal information retained by SNNs, as is the primary characteristic that distinguishes SNNs from ANNs.
T increases, making its impact more significant.
METHODS
In this section, we respectively analyze the influence of rate and temporal information the robust- ness of SNN models. Then we propose a hybrid attack scheme named HART, which comprehensively leverages these two types of information. Additionally, we analyze some unique properties of HART related to gradient calculation.
4.1 RETHINKING RATE INFORMATION GRADIENT IN SNNS
Previous studies have demonstrated the potential of utilizing rate information for training non-leaky IF models and developing adversarial attack strategies that threaten the security of SNNs. However, the determination the value of (wip-(m remains untackled, thus lacking an explicit and effective
2r' (r'(T)
activation gradient for rate information. Considering that a' = max (W'a 1-1 (0//__.0) .) and awia'-1 takes on 0 or 1 in ANNs, aim to investigate the relationship between r ((T) and -' (T).
da
In contrast to ANNs, there is no deterministic relationship between r (T') and (' Fig. illustrates this phenomenon using simple set of examples, assuming soft-reset mechanism and
x = 1. Despite receiving the same average input current w',''1 = 8²/2, Case 1-3 exhibit diverse average firing rates y (T), due the different spike arrival sequence (more details provided in the Appendix). According to Eq, [ the actual gradient for the Rate-Input curve can be expressed as follows, visually represented by pink dashed lines Fig. !
r r'T)
v (T) + MT- 2o
W'r'-10 (T)
W/ >7 T Dm 8 (t)
4

---

Published as a conference paper at ICLR 2024
Table 1: Attack success rate (ASR) of CBA and Ours (bold font) under white-box attack.
Datasets
Time-steps
FGSM. A=0.5
FGSM. A=1.0
PGD. A=0.5
PGD, A=1.0
4
59,93/86.42
64.95/90.28
41.51/99.08
$2.65/98.89
CIFAR-10
8
60,40/88.34
71.76/92.56
42.13/99.47
67.94/99.90
5
42.44/49.92
37.39/55.80
44.58/55.57
42.46/62.90
CITARIO-DVS
10
36.05/51.18
45.39/74.47
38.95/58.03
54.74/89.61
(T)
Considering that w varies across different neurons, we propose Eq. 10 to relate the average firing rate in adjacent layers (solid blue line in Fig.
1.. inspired by the principle of ReLU(.)).
(T) W Wlpd-
grmee -(WW+r-1(T
Ör (T)
E
W
(10)
0,
otherwise
Therefore, the gradient propagation chain based on rate information in SNNs can be described as:
OC ( wi Wi ard (T) Prac r I
dc
OC
OW (IT
(T))
(T) Prate Orl-1(1)
(II)
0rl-1(T)
rate
We use Conversion- Approximation (CBA) (Sharmin et al. 2019 baseline, which is conventional attack method relying on the approximation of the ANN-SNN conversion gradient. As shown in Tab. [ we have demonstrated that our proposed activation gradient outperform CBA in terms of attack performance, This result reinforces the importance of rate information in SNNs and validates the effectiveness of our proposed gradient.
4.2 ANALYZE THE RETENTION DEGREE OF TEMPORAL INFORMATION IN SNNS
As previously mentioned, the average firing rate r' (T) can exhibit wide range of values even when the average input current W', gradient of SNNs solely based E value derived from temporal information statistics, becomes challenging. Therefore, we attempt to measure the retention degree of temporal information SNNs through the following equation.
identical. Consequently, accurately characterizing the W'r"-(T)) which represents the mathematical expectation
(T) (
Var
W =
= z)) dz.
(12)
Eq. 12 first divides spiking neurons into groups based on their average input current and then measures the richness of temporal information according to the expectation of the variance within each group. To facilitate subsequent theoretical analysis, we propose the following assumption.
Considering group of spiking neurons with the same average input cur- rent W (T)) and membrane leaky constant X. we assume that for Vt E T) (1(1) ~ U (g(W)) - h( h(WW)'i-(T)) (((.)) -b(')) ((WW (')) Here g(-) denotes the expectation of the uniform distribution, and h(()) represents the length of the distribution interval.
Assumption 1.
Since Eq. 12 not suitable for direct calculation and analysis, we present Theorem based on Assumption I. Detailed proof is provided in the Appendix.
Theorem 1. If W
U(-c,c), for the soft-reset mechanism, we have x 1 [T- 1)(1 41 dr. Moreover, assuming h(2, X') ar b. will further have GCT
X 222_6² (T 11-11 362
T2
From Theorem it can be found that the retention degree of temporal information is influenced by or T and improves with smaller values of À or T. To verify this conclusion, we conduct experiments static and neuromorphic datasets (CIFAR-10 and CIFAR10-DVV using different values of

---

Published as a conference paper at ICLR 2024
8
82
$
Dal-
0m
larger grad. discrepancy
m
m
mc
BC
0C
W
BC
BC (m)
larger y
04
0m
W
W
W
8
AC
BC
BC
BC 8ml-3
del
Bsics
8mb
Bsi.1
Calculate E advance
Pruning
Grad. Merging
Adjustable Surrogate Grad.
(a)
(b)
Figure 2: Overall algorithm framework for HART. (a): the property of pre-calculation, (b): back- propagation design, (c): adjustable temporal attribute.
and T. Since our attack scheme in Eqs. TO is solely based rate information, higher attack success rate implies a more pronounced rate attribute of SNN models. Conversely, SNN models may exhibit more temporal attributes. As shown in Tab. | it can observed that SNN models with (i) smaller X, (ii) smaller T, and (iii) neuromorphic data exhibit better robustness and potentially retain more temporal information. These results indicate that SNNs still retain certain degree of temporal characteristics, which motivates us incorporate temporal components into SNN adversarial attacks.
4.3 HYBRID ADVERSARIAL ATTACK BY RATE AND TEMPORAL INFORMATION (HART)
Motivation. Based on the preceding discussion, we have recognized the necessity of comprehensively utilizing both rate and temporal information. To effectively leverage these two types of information and enhance the performance of our attack, we propose the HART attack framework, as depicted in Fig. 2 In this framework, we maintain the standard forward-propagation pattern of spiking neurons, while modifying the back-propagation chain by pruning and merging it along the time dimension. This brand-new gradient calculation mode enables HART to capture rate information more effectively. In addition, we propose a surrogate function that can be adjusted based on the degree of temporal information retention in SNN models. This flexibility allows us to dynamically regulate the balance between rate and temporal attributes.
Rate Attribute. Our primary focus for pruning lies in dm (t + 1)/0m² (1) and dm'( + 1)/0s((0) both of which play crucial role in the gradient calculation process of STBP. However, if these two terms exist, OC/èm' /3m (t) will receive contributions from åC/Ōm' + 1), OC/ 0m Om'(()) which leads to strong correlations between gradients calculated at different time-steps, making it chal- lenging to effectively capture rate information. To address this issue, as shown in Fig Fig2(b, we propose canceling dm' (t + 1)/àm'(tt (t) and 0m'(t 1)/0s((t) (t). In addition, ac/0m' (I), VI E [1, before calculating OL/0s/-1(I for all E [1, By employing these operations, the gradient calculation chain of proposed method can be expressed as follows.
further average
T
Oc
3ml (t)
Oc
OC
dm' (t)
V Vwi
(13)
am
ow
ds--1(1)
Om
0sl
t=1
OC Om
ac 0s'(t)
(14)
ds 0m'(t)
At this point, we can demonstrate that the gradient calculation of our method in Eqs. [ lent to that based on rate information in Eq. [Ofrom the perspective of mathematical expectation. This equivalence verifies that our method possesses the attribute of effectively utilizing rate information, as stated Theorem 2. detailed proof is provided in the Appendix.
14 equiva-
Theorem 2. For spiking neurons with W'r'-'(TT) > O, when there exists an approximate propor- tional relationship between gW' 1 p')
(T), X') and l (denoted as k' "'
W
under soft-reset mechanism, if E (e)(()
1)(1 X²) + 1), then we (p--((T)
8m
have E f) = (V w: mee and E DT= Ds (5()(())
9C
Here, E (V W C) and

---

Published as a conference paper at ICLR 2024
Atteck Succoos Rote
Aftric Succes 8.0
04
-4
92
92
80
88
88
8
86
0..
1.0
1.5
2.0
2.5
0.5
1.0
1.5
2.0
2.5
0.5
1.0
1.5
2.0
2.5
0.5
1.0
1.5
2.0
2.5
(a) A=0.5, FGSM
(b) A=1.0, FGSM
(c) X=0.5, PGD
(d) A=1.0, PGD
Figure 3: The performance of HART under different y on CIFAR-10.
Table Comparison between HART and previous works under white-box attack (WBA). * denotes robust target models.
Dataset
Architecture
A
Clean Ace.
Attack
CBA
BPTR
STRP
RGA
Ours
FGSM
60,40
82.67
91.71
93.63
96.28
0.5
91.48
PGD
42.13
99.21
99.95
99.92
100.00
FGSM
70.58
88.36
89.91
94.41
97.24
0.9
93.03
VGG-I1
PGD
55.29
99.45
99.94
99.97
99.98
FGSM
25.49
41.77
55.41
56.76
58.70
CIFAR-10
0.9
89.99
PGD
20.77
74.42
83.54
FGSM
71.76
88.76
86.28
93.74
96.22
1.0
93.06
PGD
67.94
99.63
99.70
99.94
99.97
FGSM
44.29
85.06
84.24
92.93
94.80
ResNet-17
0.9
93.04
PGD
29.76
99.86
99.91
100.00
100,00
0.9
73.28
FGSM
83.73
92.47
92.88
94.72
96,06
PGD
82.91
99.59
99.86
99.92
99.96
VGG-I1
CIFAR-100
0.9
67.21
FGSM
32.69
57.19
70.42
70.24
72.41
PGD
27.57
71.98
86.56
83.35
87.68
FGSM
63.34
86.94
85.66
V2.06
94.54
ResNet-11
0.9
72.05
PGD
45.17
99.65
99.69
99.90
99.96
FGSM
36.05
53.95
0.5
76.0
PGD
38.95
60.00
71.05
62.11
74,08
CIERRI-DVS
VGG-DVS
FGSM
45.39
69.74
76.97
76.05
78.42
1.0
76.0
PGD 54.74
87.11
92.63
89.08
93.03
Et= (A) refer to the gradient obtained after calculating the mean value of the surrogate gradient term in the back-propagation chain.
Temporal Attribute. Similar to Wu al. (2018), we address the non-differentiable issue arising from 0s (t)/ (t) by using surrogate gradient function, as illustrated in Eq. 15 In this equation, the parameter 7 controls the shape of the surrogate function.
Os' (t)
max (m))
(15)
0m'(()
72
By combining Assumption 1 and Eqs 13 15
adjust the expected gradient E [ΣT ac
(())
for a group of neurons with the same average input current by modifying the value of 7. From Eq. 15 can be derived that the slope of the surrogate function is 1/72, and the correspond- ing membrane potential range is [0' 7, + 7]. Moreover, we observe that the position of Σ 7 max (7 |m (t) () in the back-propagation chain is equivalent to the position of curve, wider membrane potential range covered, and smaller discrepancy the activation gradient obtained by groups of neurons with different average input currents or spike arrival sequences. In this scenario, our method can be considered have a gradient with more rate attributes. Conversely, when 7 becomes smaller, HART focuses the subset of spiking neurons belonging to specific membrane potential range in each time-step, allowing us to obtain a gradient with more temporal attributes.
g'uc in Eq. 11 pa' ANNs. Therefore, can consider it as the activation gradient of HART. Consequently. as illustrated in Fig. [cc), larger value of results in smoother surrogate gradient
oW'a'-
Pre-calculation Property. important to note that in the HART framework, the condition ac
02 is actually satisfied. Therefore, according to Eqs 14 we can consider VI, (T (T((T
7

---

1t= (t) during the forward propagation stage, allowing us update the gradients reduce the Published as a conference paper at ICLR 2024
Table Comparison between HART and previous works under black-box attack (BBA). * denotes robust target models.
Dataset
Architecture
A
Clean Acc.
Attack
CBA
BPTR
STBP
RGA
Ours
FGSM
43.04
63.44
77.77
79.65
82.68
0.5
91.48
PGD
23.50
84.21
95.99
95.36
96.74
FGSM
43.45
66.72
73.45
77.28
85.82
0.9
93.03
PGD
23.98
94.69
97.62
VGG-11
FGSM
14.08
25.26
35.83
35.44
38.26
CIFAR-I0
0.9
89.99
PGD
10.63
31.10
46.06
44.42
47.83
FGSM
43.28
64.25
68.03
73.26
80.34
1.0
93.06
PGD
24.75
80.55
90.91
91.36
96.22
FGSM
36.07
69.53
67.11
80.11
84.95
ResNet-17
0.9
93.04
PGD
15.57
93.72
94.30
98.36
99.28
FGSM
68.33
80.10
80.90
84.27
88.51
0.9
73.28
PGD
42.45
88.91
93.65
93.91
97.32
VGG-11
CIFAR-100
0.9
67.21
FGSM
22.59
37.58
47.20
47.94
50.78
PGD
18.24
41.73
54.40
54.78
57.66
FGSM
61.22
75.65
74.30
81.19
85.31
ResNct-17
0.9
72.05
PGD
32.59
91.07
89.13
95.66
98.06
0.5
76.0
FGSM
34.87
44.08
47.89
48.55
49.74
PGD 35.13
47.63
50.53
50.92
53.16
CIFARIO-DVS
VGG-DVS
FGSM
43.03
62.50
66.32
65.79
69.74
1.0
76.0
PGD $2.11
70.92
76.45
75.66
78.03
pre-calculating Σ
8a' (t)
in single operation during the nack-propagation. By leveraging this property, cost of calculating gradients for HART from O to O(1).
Empirical Principles for Selecting 7. We have previously discovered that when using a smaller value of X, SNNs tend to exhibit more temporal characteristics. Therefore, in order to achieve better attack effectiveness, is advisable use smaller value o at this time. This viewpoint is supported by Fig. 3 where it can be seen that CIFAR- 10/1 = 1.0 has a larger while CIFAR- 10AAR110/X = 0.5 has smaller 7 under the condition of achieving optimal attack success rate. Additionally, from Fig. 3 we observe that the curve of ASR with respect to y approximately follows an unimodal distribution. Based on the aforementioned empirical principles, we can approximately determine the optimal value of y by selecting several points with equal intervals within the search range.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
We validate the effectiveness of our proposed attack method on the CIFAR- (Krizhevsky et al.) 2009) and IO-DVS (Li et al. 2017) datasets. We attempt different values of X, time-steps, perturbation degree and network architectures (Simonyan & Zisserman, 2014 Zheng et al. 202T). In experiments, employed FGSM and PGD as the basic attack methods in both white-box and black-box environments. Previous studies (Sharmin et al.) 2019 2020) have pointed out that SNN models trained through STBP exhibit stronger robustness compared ANN-SNN conversion, such as (Ding et al., 2022). Therefore, we select set of SNN models as our attack targets, trained using the STBP method for 8 time-steps (CIFAR-10. 10, CIFAR-100) or 10 time-steps (CIFARIO-DVS). In addition, we also consider the effect of robust training, high-intensity attacks, filter, and encoding scheme our attack framework. More detailed experimental configuration provided Appendix.
5.2 COMPARISON WITH PREVIOUS STATE-OF-THE-TT WORKS
To demonstrate the superiority of our proposed method, we compare it with the current state-of-the-art techniques, including CBA (Sharmin et al., 2019). STBP (Sharmin al. 2020), BPTR (Ding et al.) 2022), and RGA (Bu et al. 2023). We utilize the attack success rate metric to evaluate the effectiveness of these different attacks. As shown in Tabs, 2 and 3 our method consistently achieves optimal performance across all experimental scenarios, whether under white-brr box attack or black-bon attack. This demonstrates the robustness and generalization capability of our approach, considering the different attack environments, data types, network architectures, and À values. Notably,
8

---

Published as a conference paper at ICLR 2024
Table 4: ASR for STBP/RGA/HART with different time-steps on CIFAR-IOVVG-II.
Time-steps
FGSM, WBA
FGSM, BBA
PGD, WBA
PGD, BBA
90.07/93.24/95.66
76.22/78.52880.1 77.77/7.665/8266
99.88/99.855 85/99,98
14.59/94.21/94.66
0.5
91.71/93.63/96.2
99.92/99.92/ 00199999/000000
95.99/95.36/96.77
16
91.86/93.48/955
77.49/79.66/83.44
99.95/99.91/99.99 91999.99
06.122/95.8/97.22
81 $1.89/910331 03399.667
05.22/71.4/76..44
99.17/99
1111999/3/999./// 23/99.40
87.
1.0
86.28/93.74/900 74/96.22
68, 0./33/3. 26/70.346/80.44
99. 99,70/99 94/99.97
90.91/91 00.91/91.36/9622 36/96..22
16
97.49/995.4/96.65
66.89775.07881411
01.88/9.997/99999
90.86/92.67/97.44
Table 5: ASR for STBP/RGA/HART with different perturbation degrees on CIFAR-IO/VGG:
FGSM. WBA
FGSM, BBA
PGD. WBA
PGD. BBA
2/255
49.15/45.76555.
24.67/22.87/266. 51.28/50.05/54.31
66.32/62. 08/78.33
39.30/28.42/30..
0.5
4/255
76.30/76.86/83,.
96.99/95. 0199/9.14//8.99
69.43/68. 12/71.5.
8/255
91.71/93.63/98
77.77/79 65/82.68
99.92/99.92/100.0
95.99/95.36//6.74
36/96.74
2/255
46.41744.46/46.76
99.19/19.62/21.8
65.58/61 44/65.2446522
21.89/11 (189921.960 99622.75
1.0
4/255
01882/75.17778...
41 41.48/42.768 76/47.80 .80
95.28/95.27/96.39
$7.29/55.78/66.00
8/255
86.28/93.74/96.2
68.03/73.26/80.22
01.70/99.94/9.97
90.91/91.36/96.0
method exhibits particular advantages black box attack scenarios. For instance, when 0.9 and employing the FGSM attack, our method outperforms RGA by 2.83% under the white-box attack setting and surpasses RGA by 8.54% under the black-box attack setting. Moreover, since our models trained using the STBP method, the gradient corresponding to the STBP attack can be considered the most accurate gradient estimation. However, our method outperforms the STBP attack, indicating that SNN models preserve both rate and temporal information. Therefore, directly applying the principles of adversarial attacks in ANNs cannot achieve optimal results for SNNs.
5.3 PERFORMANCE OF HART WITH DIFFERENT TIME-STEPS
To account for potential time constraints or the necessity selecting different time-steps in practical attack scenarios, conduct additional experiments evaluate the performance of method across a range of time-steps, from 4 to 16. As mentioned earlier, the retention degree of temporal information in SNN models may vary different time-steps. However, we can dynamically adjust the temporal attribute of the HART attack by tuning the parameter y, Tab, 4 presents the results of these experiments. Our method consistently outperforms both the STBP and RGA techniques in all experimental cases, showcasing its general adaptive capability.
5.4 PERFORMANCE OF HART UNDER DIFFERENT PERTURBATION DEGREE
Our proposed method also exhibits remarkable superiority in adversarial attacks with limited pertur- bations, as demonstrated by the results presented in Tab. For example, when 0.5, c = 2/255, and adopting a white-box attack, the performance gap between HART and previous techniques is 6.76% for the FGSM attack and 12.01% % for the PGD attack. These results highlights the precision of the gradient direction obtained from HART and underscores its ability to pose a significant threat to SNNs even within an extremely small step size range.
6 CONCLUSIONS
In this paper, we first conduct a quantitative analysis of rate and temporal information in SNNs. respectively, and then propose an adversarial attack method that integrates these two types of information. We emphasize the simultaneous preservation of both rate and temporal information in SNNs, and the considerable advantages of our proposed method validate this standpoint. These findings motivate further research on effectively integrating rate and temporal information into the adversarial training of SNNs, which is an interesting research direction includes drawing inspiration from the characteristics of the brain's neural circuits to achieve robustness similar that of the human brain YY et al., 2020 2024). In addition, the potential application of these gradient techniques to SNN training also fascinating direction.

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This work was supported by the National Natural Science Foundation of China (62176003, 62088102) and by Beijing Nova Program (20230484362).
REFERENCES
Sander M Bohte, Joost N Kok, and Johannes A La Poutré. Spikeprop: backpropagation for networks of spiking neurons. In European Symposiun Artificial Neural Networks, 2000.
Romain Brette, Michelle Rudolph, Ted Carnevale, Michael Hines, David Beeman, James Bower, Markus Diesmann, Abigail Morrison, Philip H Goodman, Frederick C Harris, et al. Simulation of networks of spiking neurons: a review of tools and strategies. Journal of Computational Neuroscience, 2007.
Tong Bu, Wei Fang, Jianhao Ding, Penglin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ANN-SNN conversion for high-accuracy and ullra-low-lateng -ltenne spiking neural networks. In International Conference on Learning Representations, 2022.
Tong Bu, Jianhao Ding, Zecheng Hao, and Zhaofei Yu. Rate gradient approximation attack threats deep spiking neural networks. In /EEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2023.
Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, 2015.
Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. IEEE Micro, 2018.
Michael V DeBole, Brian Taba, Amon Amir, Filipp Akopyan, Alexander Andreopoulos, William P Risk, Jeff Kusnitz, Carlos Ortega Otero, Tapan Nayak, Rathinakumar Appuswamy, et al. TrueNorth: Accelerating from zero to 64 million neurons in 10 years. Computer, 2019.
Shikuang Deng and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking neural networks. In International Conference Learning Representations, 2021.
Jianhao Ding, Tong Bu, Zhaofei Yu, Tiejun Huang, and Jian K Liu. SNN-RAT: Robustness-enhanced spiking neural network through regularized adversarial training. In Advances in Neural Information Processing Systems, 2022.
Rida El-Allami, Alberto Marchisio, Muhammad Shafique, and Ihsen Alouani. Securing deep spiking neural networks against adversarial attacks through inherent structural parameters. In Design, Automation Test in Europe Conference Exhibition, 2021.
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In International Conference on Computer Vision, 2021.
Wulfram Gerstner and Werner M Kistler. Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press, 2002.
Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: From single neurons networks and models of cognition. Cambridge University Press, 2014.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. RMP-SNN: Residual membrane potential neuron for enabling deeper high-ace accuracy and low-latency spiking neural network. In IÉEE/CVF Conference Computer Vision and Pattern Recognition, 2020.
10

---

Published as a conference paper at ICLR 2024
Zecheng Hao, Tong Bu, Jianhao Ding. Tiejun Huang, and Zhaofei Yu. Reducing ANN-SNN conversion eπror through residual membrane potential. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023a.
Zecheng Hao, Jianhao Ding, Tong Bu, Tiejun Huang, and Zhaofei Yu. Bridging the gap between ANNs and SNNs by calibrating offset spikes. In International Conference on Learning Represen- tations, 2023b.
Eugene M Izhikevich. Which model to use for cortical spiking neurons? IEEE Trunsactions on Neural Networks, 2004.
Jinseok Kim, Kyungsu Kim, and Jae-Joon Kim. Unifying activation-and timing- learning rules for spiking neural networks. In Advances in Neural Information Processing Systems, 2020a.
Seijoon Kim, Seongsik Park, Byunggook Na, and Sungroh Yoon. Spiking-yolo: Spiking neural network for energy-efficient object detection. Proceedings of the AAAI Conference on Artificial Intelligence, 2020b.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.
Souvik Kundu, Massoud Pedram, and Peter A Beerel. Hire-SNN: Harnessing the inherent robust- ness of energy-effiiient deep spiking neural networks by training with crafted input noise. In International Conference on Computer Vision, 2021.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In International Conference on Learning Representations, Workshop Track Proceedings, 2017.
Mikhail Leontev, Dmitry Antonov, and Sergey Sukhov. Robustness of spiking neural networks against adversarial attacks. In International Conference on Information Technology and Nanotechnology, 2021.
Chen Li, Lei Ma, and Steve Furber. Quantization framework for fast spiking neural networks. Frontiers in Neuroscience, 2022.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifarl0-dvs: event-stream dataset for object classification. Frontiers in Neuroscience, 11, 2017.
Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, and Shi Gu. A free lunch from ANN: Towards efficient, accurate spiking neural networks calibration. In Infernational Conference Machine Learning, 2021.
Ling Liang, Kaidi Xu, Xing Hu, Lei Deng, and Yuan Xie. Toward robust spiking neural network against adversarial perturbation. Advances in Neural Information Processing Systems, 2022.
Xuanwei Lin, Chen Dong, Ximeng Liu, and Yuanyuan Zhang. SPA: An efficient adversarial attack spiking neural networks using spike probabilistic. In IEEE International Symposium on Cluster, Cloud and Internet Computing, 2022.
Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint 1608 160.0366...,383, 2016.
Changze Lv, Jianhan Xu, and Xiaoqing Zheng. Spiking convolutional neural networks for text
classification. In International Conference on Learning Representations, 2023.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models.
Neural Networks, 1099):165--1671, 1997.
Alberto Marchisio, Giacomo Pira, Maurizio Martina, Guido Masera, and Muhammad Shafique. DVS-Attacks: Adversarial attacks on dynamic vision sensors for spiking neural networks. In International Joint Conference on Neural Networks, 2021.
Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhiquan Luo. Training high-pend rech-eefforancce latenyy spiking neural networks by differentiation on spike representation. In IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2022.

---

Published as a conference paper at ICLR 2024
Hesham Mostafa.
Supervised learning based temporal coding in spiking neural networks. IEEE Transactions on Neural Networks and Learning Systems, 29(7):3227-322 2017.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of mundient-basedd optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51-66 2019.
Stefano Panzeri and Simon R Schultz. A unified approach to the study temporal, correlational, and rate coding. Neural Networks, 2001.
Nitin Rathi and Kaushik Roy. DIET-SNN: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Transactions on Neural Networks and Learning Systems, 34:3174-3182, 2023.
Bodo Rueckauer, Iulia- Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conver- sion of continuous- deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience, 2017.
Saima Sharmin, Priyadarshini Panda, Syed Shakib Sarwar, Chankyu Lee, Wachirawit Ponghiran, and Kaushik Roy. A comprehensive analysis on adversarial robustness of spiking neural networks. In International Joint Conference on Neural Networks, 2019.
Saima Sharmin, Nitin Rathi, Priyadarshini Panda, and Kaushik Roy. Inherent adversarial robustness of deep spiking neural networks: Effects of discrete input encoding and non-linear activations. In European Conference on Computer Vision, 2020.
Sumit B Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. In Advances in Neural Information Processing Systems, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:: 1556, 2014.
Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh, Timothée Masquelier, and Anthony Maida. Deep learning in spiking neural networks. Neural networks, 2019.
Ziming Wang, Shuang Lian, Zhang Yuhao, et al. Towards lossless ANN-SNN conversion under ultra-low latency with dual-phase optimization. arXiv preprint -=XXi:2205.00473: 2022.
Yujie Wu, Lei Deng. Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high- sigh-eerformacc spiking neural networks. Frontiers in Neuroscience, 2018.
Mingqing Xiao, Qingyan Meng. Zongpeng Zhang, Di He, and Zhouchen Lin. Online training through time for spiking neural networks. In Advances in Neural Information Processing Systems, 2022.
Qi Xu, Yaxin Li, Jiangrong Shen, Jian K. Liu, Huajin Tang, and Gang Pang. Constructing deep spiking neural networks from artificial neural networks with knowledge distillation. IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
Zhaofei Yu, Jian K Liu, Shanshan Jia, Yichen Zhang, Yajing Zheng, Yonghong Tian, and Tiejun Huang. Toward the next generation of retinal neuroprosthesis: Visual computation with spikes. Engineering, 6(4):449-46) 2020.
Zhaofei Yu, Tong Bu, Yijun Zhang, Shanshan Jia, Tiejun Huang, and Jian K Liu. Robust decoding of rich dynamical visual scenes with retinal spikes. IEEE Transactions on Neural Networks and Learning Systems, 2024.
Friedemann Zenke, Sander M Bohté, Claudia Clopath, Iulia M Comşa, Julian Göltz, Wolfgang Maass, Timothée Masquelier, Richard Naud, Emre O Neftci, Mihai A Petrovici, et al. Visualizing a joint future of neuroscience and neuromorphic engineering. Neuron, 2021.
Wenrui Zhang and Peng Li. Temporal spike sequence learning via backpropagation for deep spiking neural networks. In Advances in Newral Information Processing Systems, 2020.
Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with sreettl-trainne larger spiking neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, 2021.
12

---

Published as a conference paper at ICLR 2024
Yaoyu Zhu, Zhaofei Yu, Wei Fang. Xiaodong Xie, Tiejun Huang, and Timothée Masquelier. Training spiking neural networks with event-driven backpropagation. In Advances in Neural Information Processing Systems, 2022.
A APPENDIX
A.1 PROOF OF THEOREM
Theorem 1. If W'P-1(() ~ U(-c,c). for soft-reset mechanism, we have x S_ [T IX (1 dr. Moreover, assuming h(x, ar b, we will further have 627
2²2 (T- 2
x
36
Proof. Based on Assumption and the preconditions in Theorem 1, by combining Eq. and Eq. 12 we will have
v v'(T) ET- -))0((
Var 1
P = x) dx
W Σラ=1 8 s'
Var (D'(T)) + Σ 11
1=1 2 (v' 2cI2x²
dr de.
[T - 1)(1 212 + (,,
(AI)
2x.
With the assumption that ((x, X/) ax b, we will derive the following equation.
[(T 1)(1 - 1)) + (2,"(2, X
dx
6r
(T 1)(1 21 +
2ab
dx
6c12
I
x
a 222 82 (T 1)(1 - 2)2
+1
(A2)
3
72
Theorem 2. For spiking neurons with Wl,'-1(T) > 0, when there exists an approximate propor- tional relationship between g (W',"' 1 (TT), X') and W (denoted as kt g(W','' '(T),2))
W W
under soft-reset mechanism, if IE IT ae((1)
T k(T 1)(1 X) + 1). then
9m
(e)
have (V wi wi and E
(ET,
OC a -6--1(()
ac (T--1(T)
Here, E (V wi and
vate
refer to the gradient obtained after calculating the mean value of the surrogate gradient term in the back- pack-propggation chain.
8C 08
Proof. Assuming an SNN with L layers, we first consider the case of the output layer (1 = L). We 9c (p"(TT
derive that Σi=1 ac
Ts4(() Σ
+ IT
3c अा-( and ac 0=*(1]
Jr"(T) (=*(e)
(T"(T)
ac ac With the preconditions in Theorem 2. get the following equation (2) ... (04(T))
13

---

Published as a conference paper at ICLR 2024
according to Eqs.
13 15
E V WL L) =E (
T OC
(t
T
3ml
Σ OWL
(t) (t)
1=1
1 0c
0sL(1) (t)
(1((2)
T d7L
0ml
1 OC
as (t)
E T 1r (T)(T)
pp (1)-
dt)
0L
T (IT 1)(1 - -xa) + 1)1 (
(T)
T
ac
v' ('(T) -1=1 DT-(1 (1- (t)
pl (1)
(T)
WL E1=1 81-1((t)
OC
(T)
-T)
(T)
WW'.1-1 (T)
rate
-(V
(A3)
By following similar derivation process in Eq. A3
can obtain E (ΣT= In the HART method, due the presence of merge operations, for any layer in
ac aeh-10 (1)
9C
(T)
rate
SNNs, we have (t'(()
[[ami L.mm
Om (m)++(()
(+) where l 1...,, T. Consequently, we can derive Based on the above conclusions, for any layer in SNNs,
that
ac
ac (2)
Oc (T)
(1)
E D
is already satisfied and each term in the gradient calculation chain is independent of each other, we can further derive the following equation.
Oc (F)
oC (T)
rate
((
dL (L)
T 3ml (1) ))
E w L) =F
T
Os' (t) 0ml (t)
ow
T
Oc
T
ast (1) dml (t)
T E
Σ
xe (T))
ds!
(t)
OC
Ord (T)
(T)
3r'(T)
OW! ((())
(V W w: Dasee
(A4)
Oc
1
dc as)(1)
0m'(()
E
=
Üs (t)
T
as (t) 3ml(t) (t)
as (t)
dc
as'(1)
T
2m (1)
E
Σ
E
ds!
dml (t)
Os (t)
OC
dr (T)
a (T)
(T((T)
OW!p!-I(T)
Ori
UL
-(ar-1(TT)
(A5)
rate
14

---

Published as a conference paper at ICLR 2024
Table AI: ASR about STBP/STBP+opiimll y/Ours for CIFAR-IONGG--I
FGSM. WBA
FGSM, BBA 71.71/77.0088. 68.03/72.78/80.34
PGD. WBA
PGD, BBA
0.5
91.71/91 977996
99.95/99.95/1000 99.70/99.81/992
95.99/96.03/96.74 90.91/94.00/96.22
1.0
26.22//90440/9...
Table A2: High-intensity attack about STBP/RGA//Ors for robust SNNs, X=0.9, VGG-11.
CIFAR- I0, WBA
CIFAR-IO, BBA
CIFAR-I00, WBA
CIFAR-I00, BBA
84.71/81.0188.122
56.82/54.79/58.66
90.61/88.26/91.65 65.94/55.6968.00
A.2 FURTHER EXPERIMENTS & PARAMETER CONFIGURATION
A.2.1 STBP ATTACK WITH OPTIMAL
In previous works (Sharmin et al. 2019 Bu al. 2023), the vanilla STBP attack directly adopted a Sack-propagation mode consistent with the STBP learning algorithm to obtain the disturbance gradients. In this work, we consider more threatening STBP attack that enhances the attack success rate by adjusting the value of y. Tab. pressenss the attack results of our proposed method and two types STBP attacks.
evident that the HART attack framework consistently outperforms both types of STBP attacks in all cases, which further verifies the effectiveness of our gradient pruning and merging techniques.
A.2.2 PGD ATTACK WITH HIGHER INTENSITY
We randomly restart the 10-step PGD attack times and calculate the cumulative attack success rate during this procedure. We set the robust SNNs used in Tab. as our attack targets. The results of this experiment are presented in Tab. A2 It is evident that the HART attack continues to maintain significant advantage in terms of attack success rate.
A.2.3 MEDIAN FILTER
We also consider the impact of filtering on our proposed attack method from the perspective of traditional image processing. During the training and testing process, we chose the median filtered image as the input data. As shown in Tab. A3 compared to STBP and RGA, HART exhibits significant advantages at different time-steps. For example, when adopting 8 time-steps, our method has a performance advantage of at least 3.47% on FGSM attack.
A.2.4 POISSON ENCODING
Poisson encoding is widely used neuromorphic encoding schemne for SNNs, which simulates spike sequences by binarizing each pixel point. The pixel value is proportional to the corresponding spike firing rate. We have verified that HART still achieve superior attack performance under the Poisson encoding environment. As shown in Tab. A4 our method consistently achieves optimal results in various experimental cases.
A.2.5 TRAINING AND INFERENCE PARAMETER SETTING
In Tabs. we set the perturbation strength e = 8/255 for all experimental cases about vanilla SNN models. The step size and the number of iterations for the PGD attack set to 2/255 and respectively. For robust SNN models, we adopt adversarial STBP training for 4 time-steps and the iteration number of PGD set to 10. Regarding neuromorphic data, following the approach described in (Bu et al. 2023), the perturbation obtained through gradient calculation is directly allocated to each preprocessed data frame. Additionally, considering that neuromorphic data may contain abundant temporal features,
also attempt to combination of the gradient obtained from STBP and HART as the final gradient for the attack. We compared the optimal attack performance between the HART gradient and the mixed gradient for neuromorphic datasets.
15

---

(x(()) Published as a conference paper at ICLR 2024
Table A3: ASR about STBP/RGA/Ours with median filter under WBA for CIFAR-IOVVGG-I
steps
FGSM, À 0.9
FGSM, À 1.0
PGD, À 0.9
PGD, 1.0
4
73.88/79,45/80.1
.02/83.00
99.06/99.38/99941
97.78/98.31/99.02
8
78.69/82.49/85.99
99.31/83.48/81 005
0/56/99.68/99.44
99.36/99.69//9.4
16
79,47/83.46/87.64
80.04/85.44/88.81
99.68/99.80/99.99
99.61/99.77/99.89
Table A4: ASR about STBP/RGA/Ours with Poisson encoding for CIFAR-10VVGG-11
X
Time-steps
FGSM, WBA
FGSM, BBA
PGD, WBA
PGD, BBA
16
71.95/64.78777.77
46.46/42.26/500.88
81.33/7.11/85.27
50.01/44.59/5445
0.5
32
72.73/61.73/7.445
52.47744.67/566.2
1L.01/69.00/83..0
57.31/4.73/60.99
16
1/1199 77.15/75.93/80.40 66.22//50.6655512
00.77/44.08/47.06
$7.08786.88887.000
43.04/47.21/50.66 50.05/55.28/59.32
1.0
32
87.39/86.11888377
Optimizer & Scheduler. During the training procedure, chose the Stochastic Gradient Descent Optimizer (SGD) in conjunction with the Cosine Annealing scheduler (Loshchilov & Hutter 2016) the STBP method for 200 epochs. The initial learning rate, momentum, and weight decay are set to 0.1, 0.9, and 5 x 10 respectively.
Attack Scenarios. For white-box attacks, we directly calculate the gradient based on the back- propagation chain of different methods and add attacks, similar to (Bu et al., (002)), our gradient is calculated using the black-box model, which is similar model trained through different random seeds compared to the attack target model.
perturbation to the input data. For black-box
Mixed Gradient. For neuromorphic datasets, to further enhance the temporal properties of our attack scheme, we attempt to combine the gradient obtained from HART and STBP. The mixed gradient can be described as follows:
(Z,, W), MIx (V=L(f(a, W)),))AAKE + (1 (V W), v))
(A6)
In our experiments, we choose a mixture of equal proportions (B 0.5) to validate the attack performance.
A.3 DETAILED ALGORITIM FOR THE HART ATTACK FRAMEWORK
Algorithm Hybrid Adversarial Attack by Rate and Temporal Information (HART).
Require: Input image æ; Weight matrix and spiking neuron layer W Neuron', Vl [1, L): Loss function L; Total simulation time T.
Ensure: Adversarial example image
1: # Forward propagation
2: for = 1 to L do
3: for = 1 to 7 do
4: Update the state of Neuron' based on Eq.
5: Determine whether fire spikes based on Eq.
6: Reset the state of Neuron' based on Eq.
7: Use the pre-calculation property then
8:
Calculate Σi= Dm (t)
0= 0z'(t) by Eq.15)
9: end if
end for 11: end for
# Back-propagation
13: for = L to do
if Not use the pre-caleulation property then
15: for t= 1 to 7 do
16:
Calculate the surrogate gradient (m)( at each time-step by Eq. 15p
17:
end for
16

---

Published as a conference paper at ICLR 2024
18:
Calculate the gradient merging term
0C Om
by Eq. 14
19:
for to do
20:
Calculate 0s'-1()) =
ac
(ac 8m
0m (t)
at each time-step by Eq.013
21:
end for
22:
else
23:
Oc As (1)
oc (T)) here we use Oc
to denote them
Dz
24:
Calculate the gradient merging term
ac ac as'(t) 10m. ds
Ei=1
0m 0m'(1)
25:
Calculate U=
UC W T
BC Tom
26:
end if
end for
28:
Calculate the perturbation gradient V C about the input image æ
For FGSM attack, calculate å esign V V. by Eq.6
30:
For PGD attack, calculate a² = II a + asign V V=L)] by Eq
31: return Adversarial example image a
A.4 THE COMPLEXITY OF GRADIENT COMPUTATION IN HART
As discussed the Pre-calculation Property section, we employ a pre-calculation technique for the spiking neuron layers in the HART framework. This involves calculating the sum of surrogate gradients across multiple time-steps. By doing so, we can compute the gradient of HART in single step during the back proppaapaantioon process, similar to ANNs. Consequently, the time complexity of pre-calculation for spiking neurons is O(T), while the gradient computation complexity for each synaptic layer remains at (1). In the case of deeper SNNs with more synaptic layers, we believe that this property will lead to a more significant reduction in computational cost compared to the vanilla STBP attack.
A.5 SPECIFIC EXPLANATION ABOUT FIGURE
Fig. depicts scene where two presynaptic neurons are connected to one postsynaptic neuron. In all three cases, we set W [30' /2. 8¹ /2] and r' (T) [1/2, 1/2]". In Case 1, positive and negative currents are alternately inputted, resulting in the postsynaptic neuron emitting spikes at moments 1, 3, and moments, leading to an increased number of spikes in the postsynaptic layer. In Case 3, negative currents are inputted during the first three time-steps, resulting in fewer spikes in the postsynaptic neuron.
Case 2, a continuous positive current inputted during the first three
A.6 DETAILED DERIVATION FOR EQUATION
By combining [ and Eq we have:
o'(1) (0o W ('))-1((() - (()ssss
As we adopt the soft-reset mechanism and 6¹ 1, we further have:
o'(t) - - 1) = w - (-((t) (t) - s(()).
By summing the above equation from = 1 to T. and dividing both sides by T, we have:
o' o' (oox-))''(
T 1ー s'-1(()
Σ s((()
w
T
T
T
As we set 00( - r,' = Σt= s (t)/T, finally we get:
x((T) w' (T) w'(T) + T (')
T
17


Reviews:
[{"review_id": "JgvXhO7UdA", "reviewer": "Reviewer_hCfa", "summary": "This paper proposed a hybrid adversarial attack framework, named HART, based on both rate and temporal information. The proposed method offers the flexibility to dynamically adjust the proportion between rate and temporal attributes according to the defined variable known as the retention degree of temporal information. Experiments indicated that HART can significantly improve the attack success rate of SNNs in various attack scenarios.", "questions": "1. In Figure 1, it is unclear why the purple curve appears to be identical to the curve representing case 1.\n\n2.  I kindly request the authors to assess which specific information (rate or temporal) from the SNN was leveraged by several previous attack algorithms, namely CBA, STBP, BPTR, and RGA, in their experimental evaluations. It may shed light on the attack and defense exploration. \n\n3. In Table 1, the authors juxtapose their rate-based gradient attack method with CBA. Given that CBA also employs rate information for its attack strategy, could the authors elucidate the factors that make the proposed rate gradient method superior?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1.The analytical perspective of this paper is novel. The author starts with the propagation mode between SNN layers and summarizes the two types of information: rate and temporal information. Then they explored how to obtain more precise attack gradients by analyzing the optimal integration methods for both types of information.\n\n2.This paper takes an important initial stride in showcasing the potential of combining rate and temporal information to enhance SNN attacks.\n\n3.The theoretical derivation and mathematical analysis encompassed in this paper are solid.\n\n4. The results of extensive experiments effectively demonstrate that HART exhibits remarkable performance improvements under different hyperparameter configurations when compared to previous SOTA works.", "weaknesses": "1. The authors should provide a more detailed analysis on the specific role of gradient pruning and merging within the proposed framework.\n\n2. The figures and legends of the paper can be further improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RSyXGorCTw", "reviewer": "Reviewer_RoZ6", "summary": "This paper proposes an adversarial attack method for SNN, which combines both rate and temporal information. Based on detailed analysis and experiments, this work demonstrates the efficiency of the proposed method.", "questions": "Overall, I think this work provide an interesting attack method, and the experiments are well presented. The theory is not easy to follow, please see my comments for detail.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tIt is interesting to consider both rate and temporal characters in SNN when considering adversarial attack.\n2.\tAuthor provide detailed experiment to demonstrate the effectiveness of the proposed framework.", "weaknesses": "1.\tThe theory is hard to follow, it is better to provide an illustration of how to utilize temporal information during BP (as Fig.1)\n2.\tDoes the pruning has negative effect on the gradient computation? I think it is also interesting to discuss whether the proposed gradient computation can be directly applied to SNN training.\n3.\tIt is better to discuss the complexity of gradient computation in HART. \n4.\tFig.2(c) is hard to understand. In Fig.2(b), it is not clear the meaning of solid arrows.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "83gpdCTv5A", "reviewer": "Reviewer_w1gJ", "summary": "This paper analyzes the temporal information in SNNs and establishes its relationship with the membrane decay constant and the number of time-steps. Then the authors propose a novel approach to combine rate and temproal information in SNNs, leveraging it to generate effective gradients for attacking SNNs. The experimental results demonstrate the proposed method achieve the SOTA attack success rate.", "questions": "Refer to the weaknesses above. \nPlease provide a more comprehensive explanation of Figure 2(c).", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written.\n2. The idea of combining rate and temporal information in SNNs is highly noteworthy. The temporal information in SNNs has not been fully utilized in the current model. \n3. The work is solid. The authors provide a rigorous theoretical analysis of the retention degree of temporal information in SNNs and showcase the influencing factors involved.", "weaknesses": "1. My main concern is about equation 8. I find it hard to derive equation 8 by combining equations 1 and 2. I am wondering about the absence of the threshold in equation 8. The authors should clarify why the threshold is not included in equation 8.\n2. The authors primarily focus on demonstrating how the combination of rate temporal information can enhance the attack of SNNs. How about defense? Can the proposed method also be used to improve the robustness of SNNs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2O8NbTd7qQ", "reviewer": "Reviewer_4VCV", "summary": "The paper presents HART, an adversarial attack for SNNs. An analysis of rate and temporal information in SNNs identifies the importance of rate information in SNNs. The proposed attack combines rate and temporal information. The results show better attack success rates than related works on CIFAR-10 and CIFAR-10-DVS datasets.", "questions": "1. In Table 1, please clarify what results are relative to CBA and what results are relative to the proposed attack.\n\n2. In Section 4.2: “Therefore, we attempt to measure the retention degree of temporal information in SNNs through the following equation.” Please provide more details of the intuitions and design decisions made to develop Eq.12.\n\n3. Please describe the proposed attack through a detailed algorithm that collects all the operations and equations involved. The current description with several equations may be unclear.\n\n4. Section 5 contains the results when varying the parameters of the attack. However, there is very little discussion on the results and reasons why certain values of the parameters achieve higher attack success rates than others. Please discuss it in a more comprehensive manner.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The tackled problem is relevant to the community.\n\n2. The proposed attack is novel.\n\n3. The results outperform related works.", "weaknesses": "Some aspects are not completely clear. Please see the questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)