Adversarially Robust Spiking Neural Networks Through Conversion
Authors: Ozan Ozdenizci
Robert Legenstein
Venue: Accepted by TMLR ()

Abstract: Spiking neural networks (SNNs) provide an energy-efficient alternative to a variety of artificial neural network (ANN) based AI applications. As the progress in neuromorphic computing with SNNs expands their use in applications, the problem of adversarial robustness of SNNs becomes more pronounced. To the contrary of the widely explored end-to-end adversarial training based solutions, we address the limited progress in scalable robust SNN training methods by proposing an adversarially robust ANN-to-SNN conversion algorithm. Our method provides an efficient approach to embrace various computationally demanding robust learning objectives that have been proposed for ANNs. During a post-conversion robust finetuning phase, our method adversarially optimizes both layer-wise firing thresholds and synaptic connectivity weights of the SNN to maintain transferred robustness gains from the pre-trained ANN. We perform experimental evaluations in a novel setting proposed to rigorously assess the robustness of SNNs, where numerous adaptive adversarial attacks that account for the spike-based operation dynamics are considered. Results show that our approach yields a scalable state-of-the-art solution for adversarially robust deep SNNs with low-latency.

Published in Transactions on Machine Learning Research (04/2024)
Adversarially Robust Spiking Neural Networks Through Conversion
Ozan Özdenizei
ozen. ozon. aogn.ozdeeniziiiigl.uguggg
Institute of Theoretioal Computer Science, Graz University of Technology, Graz, Austria TU Gruz SAL Dependable Embedded Systems Lab, Silicon Austria Labs, Graz, Austria
Robert Legenstein
bugruz.at
Institute of Theoretioal Computer Science, Graz University of Technology, Graz, Austria
Reviewed on OpenReview: https: // openreview. net/ forun? id= IBFW
Abstract
Spiking neural networks (SNNs) provide energy-effiiient alternative to variety of artificial neural network (ANN) based AT applications. As the progress in neuromorphic computing with SNNs expands their use in applications, the problem of adversarial robustness of SNNs becomes more pronounced. To the contrary of the widely explored end-to-end adversarial training based solutions, we address the limited progress in scalable robust SNN training methods by proposing an adversarially robust ANN-to-SNN conversion algorithm. Our method provides an efficient approach to embrace various computationally demanding robust learning objectives that have been proposed for ANNs. During a post-conversion robust finetuming phase, our method adversarially optimizes both layer-wise firing thresholds and synaptic connectivity weights of the SNN to maintain transferred robustness gains from the pre-trained ANN. We perform experimental evaluations in a novel setting proposed to rigorously assess the robustness of SNNs, where numerous adaptive adversarial attacks that account for the ppike-basod operation dynamics are considered. Results show that our approach yields a scalable sate-of-the-ate solution for adversarially robust deep SNNs with low-latency.
1 Introduction
Spiking neural networks (SNNs) are powerful models of computation based on principles of biological neuronal networks (Maass, 1997). Unlike traditional artificial neural networks (ANNs), the neurons in an SNN communicate using binary signals (i.e., spikes) elicited across temporal dimension. This characteristic alleviates the need for computationally demanding matrix multiplication operations of ANNs, thus resulting in an advantage of energy-efficiency with event-driven processing capabilities (Roy et al., 2019). Together with the developments in specialized neuromorphic hardware to process information real-time with low-power requirements, SNNs offer promising energy-efficient technology be widely deployed safety-critical real world AI applications (Davics et al., 2021). There are two main streams of applications for SNNs. First, they are applied directly to spiking input streams, e.g., from dynamic vision sensors (Bellec el. al., 2018; Wu el al., 2019; Kim & Panda, 2021a). Second, they are applied to standard machine learning tasks and standard machine learning data sets (Rathi & Roy, 2021; Bu et al., 2022; Jiang et al., 2023). We focus in this article on the latter case. Here, conversion methods that convert a pre-trained ANN to an SNN have been shown to yield SNN models with excellent performance (Diehl et al., 2015; Roy et al., 2019). Post-conversion finetuning can then be used for low-latency and energy-efficient SNNs (Rathi Roy, 2021; Davies et al., 2021).
Numerous studies have explored the susceptibility of traditional ANNs to adversarial attacks (Szegedy et al., 2013), and this security concern evidently extends to SNNs since effective attacks gradient-baser

---

Published in Transactions on Machine Learning Research (04/2024)
and architecture agnostic (Sharmin et al., 2019). Recently, there has been growing interest in achieving adversarial robustness with SNNs, which remains an important open problem (Kundu et al., 2021; Ding et al., 2022; Liang et al., 2022). Temporal operation characteristics of SNNs makes it non-trivial to apply existing defenses developed for ANNs (Madry et al., 2018). Nevertheless, current solutions are mainly end-to-end adversarial training based methods that are adapted for SNNs (Ding et al., 2022). These approaches, however, yield limited robustness for SNNs as opposed to ANNs, both due to their computational limitations (e.g., infeasibility of heavily iterative backpropagation across time); as well as the confounding impact of spike-based end-to-end adversarial training, the SNN gradients, as elaborate later in our evaluations. According to the current literature, the widely accepted assumption is that SNNs directly trained with spike-based backpropagation achieve stronger robustness than SNNs obtained by converting a pre-trained ANN (Sharmin et al., 2020). In this work, we challenge this hypothesis by proposing a novel ad adversarially robust ANN-Lo-SNN conversion algorithm based solution.
We present an ANN-to-SNN conversion algorithm that improves SNN robustness by exploiting adversarially trained baseline ANN weights at initialization followed by an adversarial finetuning method. Our approach allows to integrate any existing robust learning objective developed for conventional ANNs into an ANN- to-SNN conversion algorithm, thus optimally transferring robustness gains into the SNN. Specifically, we convert a robust ANN into an SNN by adversarially finetuning both the synaptic connectivity weights and the layer-wise firing thresholds, which yields significant. robustness benefits. Innovatively, we introduce a highly effective approach to incorporate adversarially pre-trained ANN batch-norm layer parameters which helps facilitating adversarial training of the deep ANN within spatio-temporal SNN batch-norm operations, without the need to omit. these layers from the models as conventionally done in conversion based methods.
Previous works improving adversarial robustness of SNNs conduct attack evaluations in a similar way to ANNs (Sharmin et al., 2020; Kundu et al., 2021; Ding et al., 2022), which are insufficient due to not considering the influence of the spike function surrogate gradient used by the adversary. To that end, we also introduce a novel ensemble SNN robustncss evaluation approach where we rigorously simulate the attacks by constructing adaptive adversaries based on different dilferentiable approximation techniques (see Section 4), accounting for the architectural dynamics of SNNSs that are different from ANNs. By doing so, we adapt recent observations on evaluating ANNs with non-differentiable operations and/or obfuscated gradients (Athalye et al., 2018a; Carlini et al., 2019) the domain of SNNs, and improve SNN robustness evaluations with rigor.
In depth experimental analyses show that our approach achieves a scalable state-of-the-at solution for adver- sarial robustness in deep SNNs with low-latency, outperforming recently introduced end-to-end adversarial training based algorithms with up to 2x larger robustness gains and reduced booustnsss-cccurace trade-offs.
2
Preliminaries and Related Work
2.1 Spiking Neural Networks
Unlike traditional ANNs with continuous valued activations, SNNs use discrete, pulsed signals to transmit information. Discrete time dynamics of lealy-inteerrate-an--rrr (LIF) neurons in food-forward SNNs follow:
- ((=))+v((--)++ w'
(1)
- ((t)=H(v((tt) Vil),
(2)
(())=v(())-)))) s'(())
(3)
where the neurons in the [-th layer receive the output spikes (t) from the preceding layer weighted by the trainable synaptic connectivity matrix W', and 7 denotes the membrane potential leak factor. We represent the neuron membrane potential before and after the firing of a spike at time t by v'(t) and v'(t), respectively. H()) denotes the Heaviside step function. A neuron j in layer generates spike when its membrane polential (t-) reaches the firing threshold V/A. Subsequently, the membrane potential is updated via Eq. (3), which performs a hard-reset. Alternatively, a neuron with soft-reset updates its membrane potential by subtraction, (t))) ((t)) s'(t)V,,, where the residual membrane potential can encode information. Inputs s²()) to the first layer are typically represented either by rale coding, where the input signal intensity determines the
2

---

Published in Transactions on Machine Learning Research (04/2024)
firing rate of an imput Poisson spike train of length T, by direct coding, where the input signal is applied as direct current to (t-) for all T timesteps of the simulation.
Direct Training: One category of algorithms to obtain deep SNNs relies on end-to-end training with spike-based backpropagation (Wu et al., 2018; 2019; Lee et al., 2020). Different from conventional ANNs, gradient-based SNN training backpropagates the error through the network temporally (i.e., backpropagation through time (BPTT)), in order to compute a gradient for each forward pass operation. During BPTT, the discontinuous derivative of H<z (t)) for spiking neuron when z' (t) = v' - V 2 O, is approximated via surrogate gradient functions (Neftci et al., 2019). Such approximate gradient based learning algorithms have reduced the performance gap between deep ANNs and SNNs over the past years (Bellec et al., 2018; Shrestha & Orchard, 2018). Various improvements proposed for ANNs, e.g,, residual connections or batch normalization, have also been adapted to SNNs (Fang et al., 2021; Duan et al., 2022). Notably, recent work introduced batch-norm through time (BNTT) (Kim & Panda; 2021b), and hrrehhld-dependen batch-norm (tdBN) (Zheng ct al., 2021) mechanisms to better stabilize training of deep SNNs.
ANN-to-SNN Conversion: An alternative approach is to convert a structurally equivalent pre- trained ANN into SNN (Cao et al., 2015). This is performed by replacing ANN activation functions with LIF neurons (thus keeping weights W' intact), and then calibrating layer-wise firing thresholds (Dichl et al., 2015; Sengupta et al., 2019). The main goal is to approximately map the continuous-valuue ANN neuron activations to the firing rates of spiking neurons (Rueckauer et al., 2017). While conversion methods leverage prior knowledge from the source ANN, they often need more simulation time steps T than BPTT-based SNNs which can temporally encode information. Recent, hybrid approaches tackle this problem via post-conversion finetuning (Rathi et al., 2020). This was later improved with soft-reset mechanisms (Han et al., 2020), trainable firing thresholds and membrane leak factors during finetuning (Rathi & Roy, 2021). More recent methods redefine the source ANN activations to minimize conversion loss (Li et al., 2021; Bu et al., 2022).
2.2
Adversarial Attacks and Robustness
Deep neural networks are known to be susceptible to adversarial allacks (Szegedy et al., 2013), and various defenses have been proposed to date. Currently, most effective ANN defense methods primarily rely on exploiting adversarial examples during optimization (Athalye et al., 2018a), which is known as adversarial training (AT) (Madry et al., 2018; Goodfellow et al., 2015).
Given samples (x, y) from a training dataset D, a neural network fo is conventionally trained via the natural cross-entropy loss ( v), which follows a maximun likelihood learning rule. In the context of robust optimization, adversarial training objectives can be formulated by:
min (x,))
max Crobust (fo(E), (o))) (x)
(4)
with & denoting the adversarial example crafted during the inner maximization step, which is constrained within Af(x) := {x: : ||X - x S c), defined as the bp- norm ball around x with an e > 0 perturbation budget. We will focus on -orm bounded examples where Ax
Standard AT (Madry et al., 2018) only utilizes iteratively crafted adversarial examples during optimization, i.., Crobust (fe (8). v), thus maximizing log- likelihood solely based on X, and does not use benign samples. Conventionally, the inner maximization is approximated by projected gradient descent (PGD) on a choice of an LPGD loss (e.g,, for standard AT: LPGD = (crr[o((),v))) to obtain k-step adversarial examples fFGD (fox). y))], where Xo = x x 5 with ~ Zd(-c, e), step size n, and (x)) [.] the clipping function onto the e-bal.. Due to its iterative computational load, single-step variants of PGD, such as the fast gradient sign method (FGSM) (Goodfellow et al., 2015) or random-step FGSM (RFGSM) (Tramèr et al., 2018), have been explored as alternatives (Wong et al., 2020).
by: R = (x))
State-of-the-art robust training methods propose regularization schemes to improve the inherent robustness- accuracy trade-off (Tsipras et al., 2019). A powerful robust training loss, TRADES (Zhang et al., 2019a), redefines the training objective as:
TRADES == Lcr((o(x)))) ATRADES DKA. (Jo(3)]][o(x)) robust
(5)
3

---

Published in Transactions on Machine Learning Research (04/2024)
where DKL denotes the Kullback-Leibler divergence, and ATRADES is the trade-off a subsequent work (Wang et al., 2020), MART robust training loss proposes to emphasize the error on misclassified examples via:
In
Coobust = (BCE(fO((), + AMART (1 f1- (x)) DKL (fe (f)(x))
(6)
where AMART is the trade-off parameter, f (x) denotes the probability assigned to class y, and the boosted cross-entropy is defined as, (fc == (cn(Jo()) (x), - log(1 - max (0 (x)), with the second term improving the decision margin for adversarial samples.
2.3 Adversarial Robustness in Spiking Neural Networks
Earlier explorations of adversarial attacks on SNNSs particularly emphasized that direct training exhibits stronger robustness compared to conversion-based SNNs with high latency, and rate input coding SNNs tend to yield higher robustness (Sharmin et al., 2019; 2020). Accordingly, subsequent work mainly focused on end-to-end SNN training methods, yet relying on shallower networks and simpler datasets (Sharmin et al., 2020; Marchisio et al., 2020). Notably, SNNs were also shown to maintain higher resiliency to black-box attacks than their ANN counterparts (Sharmin et al., 2020). Later studies analyzed various components of SNNs and their robustness implications, such as the structural configuration (Fl-Allami el al., 2021; Kim et al., 2022a), or the surrogate gradient used for training (Xu et al., 2022). Another line of work focuses designing attacks for event-based neuromorphic data, with stable BPTT gradient estimates (Marchisio el al., 2021; Liang et al., 2021; Büchel el al., 2022). Until recently, there has been limited progress in scalable robust SNN training algorithms.
A recent work (Liang et al., 2022) explores certified adversarial robustness bounds for SNNs on MNIST- variant datasets, based on an interpretation of methods designed for ANNs (Zhang et al., 2019b). Recently, HIRE-SNN (Kundu et al., 2021) proposed adversarial finctuning of SNNs initialized by converting a naturally trained ANN, using a similar approach to (Rathi & Roy, 2021). For finetuning, HIRE-SNN manipulates inputs temporally with single-step adversarial noise, resulting in marginal robustness benefits relatively small models. The recently proposed SNN-RAT (Ding et al., 2022) algorithm empirically achieved state-of-the-art robustness with SNNs. SNN-RAT interprets the ANN weight orthogonalization methodology to constrain the Lipschitz constant of a feed-forward network (Cisse et al., 2017), in the context of SNNs optimized with single-step adversarial training. Due to its ability to scale to deep SNN architectures, SNN-RAT currently serves as a solid bascline to achieve robustness with SNNs.
3
Adversarially Robust ANN-to-SNN Conversion
3.1
Converting Adversarially Trained Baseline ANNs
Our method converts an adversarially trained ANN into an SNN by initially configuring layer-wise firing thresholds based on the well-known threshold- shresholl-ballaciing approach (Sengupta et al., 2019). Then we propose to adversarially finetune these thresholds together with the weights.
Given an input x, we use a direct input encoding scheme for s()),, where the input signal (e.g., pixel intensity) is applied as a constant current for simulation length of T timesteps. Subsequently, the network follows the SNN dynamics from Eqs. (1), (2), (3). We mainly consider SNNs with integrato-and-fre (IF) neurons, i.c., 1, similar to Ding et al. (2022) (sce Experiments on TinylmageNeet for LIF neuron sinulations). The output layer of our feed-forward SNN with T, layers only accumulates weighted incoming inputs without generating a spike. Integrated non-leaky output neuron membrane potentials v/- (T) for j = I, with N being the number of classes, are then used to estimate normalized probabilities via:
Configuration of the SNN:
N
(x) = =x) Σ Ettt, where v'(1)=v
W g 1- ((), 1= 1...,.,
and fo (x) denotes the probability assigned to class j. In our case, the neural network parameters O to be optimized will include connectivity weights, layer-wise firing thresholds, and batch-norm parameters.
4

---

Published in Transactions on Machine Learning Research (04/2024)
Initializing SNN Weights: We use the weights from baseline ANN that is adversarially trained via Eq. (4); to initialize the SNN synaptic connectivity matrices. Our approach is in principle agnostic to the ANN adversarial training method. For a feed-forward ANN, neuron output activations a' in the L-th layer would be, a' = (W' a'-),, where o() often indicates a ReLU. We essentially maintain W for all layers, and replace a (.) with spiking neuron dynamics.
Conversion of Batch Normalization: Traditional conversion methods either exclude batch-norm from the source ANN (Sengupta et al., 2019), or absorb its parameters into the preceding convolutional layer (Rueckauer et al., 2017). Here, we introduce novel and highly effective approach to directly integrate robustly pre-trained ANN batch normalization affine transformation parameters into SNN threshol--dependentt batch-norm (tdBN) layers (Zheng et al., 2021), and propose to maintain the parameters se and w' within:
S' S'= 3'=''.((''
/
[Varss++ +0) + w', where ='= (1)
(8)
weighted spiking inputs s1(1) = w' (t) are gathered temporally in S 0 is a tiny constant, and B[.] and Var[-] are estimated both across time and mini-batch. This temporal and spatial normalization was found highly effective in stabilizing training. Normalized (1) is then integrated into the neuron membrane potential via Eq. (1). In order to accurately maintain the utility of the preserved of and wt parameters for the SNN, we do not re-use the moving average mean and variance estimates from the ANN, which are often tracked to be used at inference time. Instead, we discard these previous statistics, and re-estimate them during SNN finetuning. Different from its original formulation (Zheng et al., 2021), we do not perform a threshold-based scaling in this normalization, but instead finetune up' and wt together with the firing thresholds.
Initialization of Trainable Firing Thresholds: We generate inputs (t) by direct coding for Te calibration time steps, which we choose to be much larger than T, using a number of training set mini-batches. In order to estimate proportionally balanced per-layer firing thresholds as in (Sengupta et al., 2019), starting from the first layer, for all inputs we record the pre-activation values (i.e., summation of the weighted spiking inputs) observed across Te timesteps, and set the threshold to be the maximun value in the p-percentile of the distribution of these pre-activation values (often > 99%). This is performed similarly to (Lu & Sengupta, 2020; Rathi Roy, 2021), ), in order to accommodate for the stochasticity involved due to the use of sampled training set mini- bathhes (see Suppl. A.2 for more details).
After sctting the firing threshold for 1 1, we perform forward pass through the spiking neuron dynamics of this layer, and move on to the nexi layer with spiking activations. We sequentially set the firing thresholds for all other layers in the same way. During the forward passes, we only use inference-time statistics estimated spatio-temporally on the current batch for normalization via tdBN. After estimating all layer thresholds, we initialize the trainable (V% I--1 values by scaling the initial estimates with a constant factor k < 1, in order to promote a larger number of spikes to flow across layers proportionally to our initial estimates (Tu & Sengupta, 2020) (see Suppl. A.3 for complete algorithm).
3.2 Robust Finetuning of SNN After Conversion
We complement our hybrid conversion approach with spike- backpropagation based adversarial finctuning. We exploit adversarial examples for robustly pre-trained weight finetuning updates, and also to robustly adjust firing thresholds. Our finetuning objective utilizes the TRADES loss (Zhang et al., 2019a):
min E (CE(fo(x),)) + B max (xc(fe())]lfe(x)) 0
(9)
(x)
where 8 is a robustness-accuraay trade-off parameter, O consists of (W", of w/ and (VA and fo(x) and fo(x) are the probability vectors assigned to benign and an adversarial example, estimated by the integrated output neuron membrane potentials over T timesteps. In order to minimize this objective, the regularizer term requires an adversarial example & obtained through an inner maximization step. This adversarial example v E A (x) is obtained via an REGSM based single inner maximization step by:
x = x' + (e a) sign WRRGSMM) where x'= x sign (N(O,
(10)
5

---

Published in Transactions on Machine Learning Research (04/2024)
VGG11
VGGI1
Activations for Clean Imput
#131
3500 10000
#1300
200000
#129
DI SOUToo
C000C1 COO0AL 000S
#128
50000 100000 550000
Couron
Sime.
Activations for Adversarial Imput
#131
1I
20
30
LG time
time
#1 30
Conv2D
Conv2D +LII' Neurous
#128
+LIF Neurons
sime
direct coding
direct coding
Clean
Clean
Adversarial Input
Input
Input
Figure 1: Illustration of the impact of an adversarial atlack leed-forward SNNs with direct input coding. In this setting, the spiking representation of the input is generated after the first convolutional layer, where the input pixel intensities are applied direct current to the neurons for T time steps. We show one Sean/adversarial test case from TinylmageNet against our VGG11 model (T = 30) obtained via conversion. Left side depicts the clean input scenario, and the right, side depicts an adversarial attack scenario. The first layer of this model consists of 64 Conv2D kernels, which outputs 262,144 LIF neuron activations (64 feature maps with 64x64 spatial image resolution). Adversarial perturbations of magnitude values are visualized darker and positive values are visualized brighter) are obtained in the input image pixel domain via spike-based BPTT. The change in the input is hardly visible for the adversarial image. However, this small change leads to noticeable difference of the spiking output of the first convolutional layer on the fine temporal scale (see cyan and red zoom-in boxes). Neuron IDs are vectorized for illustration purposes.
8/255 (negative
Note that only single gradient computation is used for RFGSM, thus it is computationally efficient and yet powerful alternative to PGD (Ding et al., 2022). For RFGSM, differently from its original formulation with cross-entropy loss, we propose to use single-step gradient ascent for CreGSM = DKL (fe (x)),,,,,,,, which maximizes the KL-divergence between the vectors integrated membrane potentials of output class neurons (see Suppl B.3 ablations for alternatives). Objective Eq. (9) is optimized via. BPTT, and a piecewise linear surrogate gradient with unit dampening factor used (Bellec et al., 2018). We present an outline of our BPTT-based parameter updates, well as our overall training algorithm with further details in Suppl. A.3.
4 An Ensemble Evaluation Approach for Adversarial Robustness of SNNs
We focus low-latency SNNs via conversion, which are widely applicable to standard machine learning tasks. Therefore we mainly use SNNs with direct input coding, where the input pixel intensitics are applied as direct current to the first layer LIF neurons, and a spiking representation of the input is generated through their activations. this SNN setting, adversarial attacks operate similar as in the ANN setting, where adversarial perturbations are obtained by computing input gradients through spiko-based backpropagation through time, and then applied the input pixel domain. These adversarial examples then result in a different spiking activation pattern after the first convolutional layer of the network, illustrated in Fig. 1.

---

Published in Transactions on Machine Learning Research (04/2024)
Spike-based backpropagation relies surrogate gradient functions due to the discontimuous derivative of the spike activation function (Neftci et al., 2019). Thus, differently from ANNs, the success of backpropagation based SNN adversarial attack also critically depends on the surrogate gradient used by the adversary for the attack. Previous studies on adversarial robustness of SNNs (Sharmin et al., 2020; Kundu et al., 2021; Ding et al., 2022) conduct naive attack evaluations, by only considering the same surrogate gradient function used during training at the time of attack. Here, we propose to adapt recent observations made on ANNs with non-differentiable operations and/or obfuscated gradients (Brendel & Bethge, 2017; Athalye et al., 2018a; 8a; Carlini et al., 2019) to the domain of SNNs, in order to improve SNN robustness evaluations by potentially relying on more stable gradient estimates with different surrogate gradient choices at the time of attack.
For evaluations, we therefore implemented an ensemble attack strategy. For each attack algorithm, we anticipate an adaptive SNN adversary, as proposed for ANNs (Carlini et al., 2019), that simulates a mumber of attacks for a test sample where the BPTT surrogate gradient varies in each case. We consider variants of the three most common surrogate gradient functions: piocewise lincar (Bellee et al., 2018), exponential (Shrestha & Orchard, 2018), and rectangular (Wu et al., 2018), with different. width, dampening, or steepness parameters (see Suppl. A.4 for parameter details). In the ensemble we also consider straight-through estimation (Bengio et al., 2013), backward pass through rate, and a conversion-bssed approximation (Ding et al., 2022), where we replace spiking neurons with a ReLU during BPTT. We calculate robust accuracies on the whole test set, and consider an attack successful for a test. sample if the model is fooled with any of the attacks from the ensemble. Note that during training we only use the piecewise linear surrogate gradient with a unit dampening factor as in Ding al. (2022).
5 Experiments
5.1 Datasets and Models
We performed experiments with CIFAR-1O, CIFAR-100, SVHN and TinylmageNet datasets. We used VGG11 (Simonyan & Zisserman, 2015), ResNet-20 (He et al., 2016) and WideResNet (Zagoruyko & Komodakis, 2016) architectures with depth 16 and width 4 (i.e., WRN-16-4) in our experiments (see Suppl. A.1 for further details). Unless specified otherwise, SNNs were run for T timesteps as in Ding et al. (2022).
5.2 Robust Training Configurations
We performed adversarially robust ANN-to-SNN conversion with bascline ANNs optimized via state-of-the-art robust training objectives: standard AT (Madry et al., 2018), TRADES (Zhang et al., 2019a), and MART (Wang et al., 2020), as well as ANNs optimized with natural training. We mainly focus on adversarial training with a perturbation bound of E1 = 2/255. In addition, for CIFAR-10 we also performed conversion of ANNs with standard AT under E2 = 4/255 and e3 8/255. This resulted in baseline ANNs with: AT-E1, TRADES-e1, MART-E1, AT- AT-e3-
Bascline ANNs:
Converted SNNs: Robust finetuning was performed for 60 epochs (only for CIFAR-100 we used 80 epochs). Regardless of the baseline ANN training scheme, SNN finetuning was always performed with e bounded pertur- bations. At test time, we evaluated robustness also at higher perturbation levels e2 and e3, similar to Ding et al. (2022). We used / = 2 for CIFAR-10, and 3 = 4 in all other exporiments. Further configurations are dotailed in Suppl. A.2. Our implementations can be found at: https:
5.3 Adversarial Robustness Evaluations
White-box Adversarial Attacks: In accordance with the ensemble setting presented in Section 4, we evaluate SNNs against FGSM (Goodfellow et al., 2015), RFGSM (Tramèr et al., 2018), PGD (Madry et al., 2018), and Auto-PGD attacks based on cross-entropy loss (APGDcr) and difference of logits ratio loss (APGDDlR) (Croce& & Hein, 2020). We also consider a targeted version of this attack (T-APGDeE), where each sample was adversarially perturbed towards 9 target classes from CIFAR-10 or CIFAR-100, until successful. PGD attack step size n determined via 2.5 x e/ #steps (Madry et al., 2018). For baseline ANNs, we present our evaluations via Auto (Croce et al., 2020) at 63 (see Table A1 for extended evaluations).
7

---

Published in Transactions on Machine Learning Research (04/2024)
Table 1: Robustness evaluations (%) of baseline ANNs and our converted SNNs across different datasets and architectures. Robust accuracies of converted SNNs are evaluated separately under e 2/255, 62 4/255 E3 8/255 perturbation budgets, with white-box ensemble SNN attacks. Robust accuracies of baseline ANNs are presented here with AutoAttack for 63 perturbation budget (sce Table A1 for other ANN attacks)
Baseline ANN
Adversarially Robust ANN-to-SNN Conversion
Dataset & Architecture
ANN Training Objective
Clean 6g- Robust Acc.
Clean
61-Robust Acc.
62-Robust Acc.
<g-Robust Acc.
Acc.
FGSM
PGD
FGSM
PGD
FGSM
PGD
Natural
95.10 / 0.00
98.53
66.62
59.22
42.81
22.44
15.71
0.74
CIFAR-10 with VGGI1
AT-E1
93.55 / 16.64
92.14
74.10
70.02
59.11
46.59
33.94
13.18
TRADES-c
92.10 / 27.62
91.39
75.01
72.00
62.64
51.47
41.42
19.65
MART-11
92.68 21.98
91.47
74.83
70.35
62.87
49.51
44.39
17.79
Natural
78.93 / 0.00
70.87
41.16
35.49
26.08
13.87
11.25
1.40
CIFAR-100
AT-61
74.90 / 7.05
70.02
47.91
44.73
31.74
27.41
18.60
7.64
with WRN-16-4
TRADES-C
73.87 8.43
67.84
45.48
42.73
33.62
26.78
18.13
7.79
MART-G
72.61
11.04
67.10
45.99
43.46
34.03
27.40
18.93
8.90
Natural
97.08 / 0.08
94.18
77.38
74.30
59.93
48.09
31.50
10.35
SVHN
AT-c
96.50 / 23.58
91.90
74.15
71.80
60.66
52.68
36,60
21.51
with ResNet-20
TRADES-C
96.29 27.16
92.78
76.38
74.27
62.45
55.59
37.76
23.96
MART-c
96.16 / 27.65
92.71
76.76
74.82
62.60
55.52
37.90
24.11
Black-box Adversarial Attacks: We performed query-based Square Attack (Andriushchenko et al., 2020) evaluations with various numbers of limited queries (presented in Section 6.4). We also evaluated transfer-based attacks (Papernot et al., 2017) to verify that such black-box attacks succeed less often than our white-box evaluations (presented in Suppl. B.1).
6
Experimental Results
6.1 Evaluating Robustness of Adversarially Trained ANN-to-SNN Conversion
We present our results in Table using FGSM and PGD ensemble attacks which are significantly stronger than the attacks used during robust finetuning for models obtained by converting baseline ANNs optimized with natural and different adversarial training objectives. These results show that our approach is compatible with any bascline ANN robust training algorithm, and an initialization from a relatively more robust ANN proportionally transfers its robustness to the converled SNN. Although we use an identical robust. finetuning procedure, converted SNNs that used weaker (natural training) ANNs show minimal robustness under ensemble attacks (c.g., 63- 2GD20 for CIFAR-10, Natural: 0.74, AT-e1: 13.18, TRADES- 61: 19.65, MART-61: 17.79, similar results can be observed with CIFAR-100 and SVHN). Here, conversion of natural ANN and finetuning is analogous to the HIRE-SNN approach by Kundu et al. (2021), where the finetuning objective and optimized paramcters are slightly different, nevertheless the final robustness remains significantly weak.
30
Robust accuracy
25
20
AT-
15
AT
SNN-RAT
AT-és
10
Our finetuning objective and robustly adjusting VA also appear critical to maintain robustness. We claborate this later in our ablation studies in Suppl. B.3. In brief, we observed that natural Lce finetuning (Rathi & Roy, 2021), or using fixed V during robust finetuning, yields worse robustness (c.g., from Table B5, 20 when converting using natural Leg: 10.03, using Eq. (9) but with fixed Vi. 11.51, Ours (trainable V 13.18).
2
3
6
8
Finetuning objective 8
Cg-Robust accuracies under 2GD vs. B, with different baseline ANN standard AT perturbation levels.
Figure 2:
8

---

Published in Transactions on Machine Learning Research (04/2024)
Table 2: Comparisons with the state-of-the-art robust SNN training method SNN-RAT, based on the model checkpoints from Ding et al. (2022). All attacks are norm bounded with c3 = 8/255, and evaluated through an ensemble. Note that FGSM, RFGSM and PGD" (with 17 0.01) are the only attacks evaluated in Ding et al. (2022), which also include in ensemble setting.
CTFAR-10 with VGG11
CTFAR-100 with WRN-16-4
Conversion (Ours)
Conversion (Ours)
SNN- RAT
SNN- RAT
AT-
TRADES-11 MART-11
AT-c1
TRADES-4
MART-1
Clean Acc.
90.74
92.14
91.39
91.47
69.32
70.02
67.84
67.10
FGSM
32.80
33.94
41.42
44.39
19.79
18.60
18.13
18.93
Gr-Robsst Acc.
RFGSM
55.29
57.52
61.42
61.59
32.39
32.55
31.77
32.00
PGD
13.47
15.98
23.63
23.04
7.76
9.53
9.29
10.70
2
10.06
13.18
19.65
17.79
6.04
7.64
7.79
8.90
PGD
9.45
12.67
19.20
16.99
5.84
7.43
7.67
8.48
APGDDLR
11.71
14.07
23.28
20.12
8.81
8.88
8.92
9.68
APGDoR
9.07
10.05
17.63
15.54
5.22
6.62
6.94
7.67
T-APGDcE 7.69
9.19
16.11
13.09
4.72
5.41
5.50
6.43
Fig. 2 shows the impact of baseline ANN adversarial training 6, on the converted SNN, as well as the robustness-accureg trade-off parameter 3, for CIFAR-10. Note that we still perform SNN finetuning with G as in Ding al. (2022), since larger perturbations observe that baseline ANN robustness transfers also in this case, and one can obtain even higher SNN robustness (e.g,, with AT-e3, 3 6: 26.90%) by using method via heavier adversarial training of the ANN where optimization stability is not problem.
BPTT yiclds an unstable finetuning process. We
6.2 Comparisons to State-of-the-Art Robust SNNs with Direct Adversarial Training
In Table 2, we compare our approach with the current state-of-the-art method SNN-RAT, using the models and specifications from Ding et al. (2022), under extensive white-box ensemble attacks². Our models yield both higher clean accuracy and adversarial robustness across various attacks, with some models being almost twice as robust against strong adversaries, e.g., clean/PGD for CIFAR-10, SNN-RAT: 90.74/9. Ours (TRADES-G):: 01.39/19.20 Note that in Table we only compare our models trained through bascline ANNs with é er-perturbations, since SNN-RAT also only uses éi-adversaries (see Fig. 2 as where SNN-RAT stands with respect our models which can also exploit larger perturbation budgets via conversion). Overall, we observe that SNNs gradually get weaker against stronger adversaries, however our models maintain relatively higher robustness, also against powerful APGDgE and T-APGDGE attacks.
We further compare our approach with other direct AT methods (e.g., TRADES), within our ablation studies in Suppl. B.3, which again revealed worse performance than our conversion-basse approach (e.g., from Table B5, clean/ clean/PGD's 2D for CIFAR-10, direct TRADES training: 91.75/6.70, Ours (AT-e1): 92.14/13.18).
Experiments on TinyImageNet: Results are summarized in Fig. Here, we utilize the same VGG11 architecture used for BNTT (Kim & Panda, 2021b), which has claimed robustness benefits through its batch-norm mechanism combined with Poisson input encoding (see Suppl. A.4 for attack details this model). For comparisons, besides our usual IF (T 8) setting, we also converted (AT-e1) ANNs by using LIF neurons (T 0.99) and soft-reset with direct coding.
Results in Fig. 3b show that our model achieves a scalable, state-of-the-art solution for adversorially robust SNNs. Our model remains superior to RFGSM and SNN-RAT based end-to-end AT methods in the usual (T 8) setting with 17.82% robust accuracy. Results with LIF neurons show that our models can improve clean accuracy and robustness with higher T (T 30) whereas RFGSM and SNN-RAT perform generally
'The original work Ding al. (2022) reports BPTT-based PCD7 with 21.16% CIFAR-10, and 11.31% CIFAR-100 g-rooustness. While could confirm these results in their naive BPTT sctting, this particular attack yielded lower SNN-RAT robust accuracies with ensemble attack: 13.47% CIFAR-10, and 7.76% CIFAR-100. FEad-to-ent adversarial SNN training, e.g., with SNN-RAT, for was computationally infeasible using BPTT.

---

Published in Transactions on Machine Learning Research (04/2024)
60
Vanilla
Input
SNN
Adv.
Clean Acc.
<2-Robust Acc.
Robust accuracy (%)
BNTT
Enc.
Config.
Train
FGSM
PGD20
50
RPGSM
SNN-RAT
Vanilla
Direct
IF (T 8)
K
57.29
1.93
0.13
40
Ours
RFGSM
Direct
IF (T
/
54.29
19.21
14.35
SNN-RAT
Direct
(T
52.76
22.39
16.63
30
Ours (AT-6)
Direct
IF (T' 8)
v 55.98
22.91
17882
Vanilla
Direct
LIF 30)
59.62
1.76
0.05
20
BNTT RFGSM
Poisson
LIF (T' = 30)
X
57.33
17.64
11.88
Direct
LIF (T 8)
55.92
17.51
12.42
10
SNN-RAT
Direct
LIF (T 8)
54.47
19.66
11.37
1/255 2/255 3/255 4/255 Perturbation budget (c)
Ours (AT-e1)
Direct
LIF (T 8)
57.21
22.52
16.25
Ours (AT-e1)
Direct
LIF (T 30)
58.36
24.02
19.67
(a) Robustness under PCD attacks.
(b) Comparisons with 1/255 attacks.
Figure 3: Evaluations on TinyImageNes with comparisons to SNN-RAT (Ding et al., 2022), an RFGSM based mixed AT baseline from Ding et. al. (2022), BNTT (Kim & Panda, 2021b), and a vanilla SNN with natural training. SNNs with LIF neurons (r 0.99) use soft-reset as in Kim & Panda (2021b). (a) we compare LIF-ncuron models from the bottom half of the table in (b), using Ours with = 30.
worse with LIF neurons. Results in Fig. 3a also reveal stronger robustness of our LIF-based (T 30) model starting from smaller perturbations, where it consistently outperforms existing methods. Our evaluations of BNTT show that AT is particularly useful for robustness (Ours (T =30): 19.67%, BNTT: 11.88%). Although direct coding does not approximate inputs and attacks become simpler (Kim et al., 2022a), we show that direct coding SNNs can still yield higher resilience when adversarially calibrated.
6.3 Coding Efficiency of Adversarially Robust SNNs
In Fig. 4, we demonstrate higher coding efficiency of our approach. Specifically, VGG11 models consisted of 286,720 720 IF neurons operating for T = 8 timesteps, which could induce a total of approx. 2.3M possible spikes for inference on a single sample. For CIFAR-10, on average for a test sample (ie., mean of histograms in Fig. 4a), the total #spikes clicited across a VGG11 was 148,490 with Ours, 169,566 for SNN-RAT, and 173,887 for a vanilla SNN. This indicates an average of only 6.47% active spiking neurons with our method, whereas SNN-RAT: 7.39%, and vanilla SNN: 7.58%. For CIFAR-100 in Fig. 4b, total #spikes elicited on average was 589,774 with Ours, 616,439 for SNN-RAT, and 592,297 for vanilla. SNN, across approx. 3.8M possible spikes from the 475,136 neurons in WRN-16-4. Here, our model was 15.52% actively spiking, whereas 16.22% with SNN-RAT. We did not observe an efficiency change with our more robust SNNs, and spike activity was similar to the models converted from a baseline ANN with AT-
6.4 Further Experiments
Query-based Black-box Robustness: We evaluated SNNs with adversaries who have no knowledge of the defended model parameters, but can send a limited number of queries to it. Results are shown in
Fig. 5, where our models consistently yield higher robustness (CIFAR-10 at 5000 queries: Ours: 50.2% vs. SNN-RAT: 46.3%, CIFAR-100: 18.8% vs. 17.5%). We also see that after 1000 queries, the baseline ANN shows weaker resilience than our SNNs. This reveals improved black-box robustness upon conversion to a spiking architecture, which is consistent with previous findings (Sharmin et al., 2020).
Transfer-based Black-box Robustness: Previous work (Athalye et al., 2018a) suggests that transfer-based attacks should be considered as a baseline to verify no potential influence of gradient masking or obfuscation. Accordingly, we present our evaluations in Suppl. B.1, where we verify (in Table B1) that fully black-box transfer attacks succeed less often than our ensemble white-box evaluations.
Performance of Ensemble Attacks: We elaborate the effectiveness of our ensemble attack approach in Suppl. B.2. In brief, reveal that changing the shape dampening magnitude of the surrogate gradient.
10

---

Published in Transactions on Machine Learning Research (04/2024)
(Vunilla)
(Vanilla)
# test samples
300
SNN-RAT Ours (AT'-c;)
# test samples
300
SNN-RAT
Ours (AT-oo)
200
200
100
100
0
100K 125K 150K 175K 200K 225K 250K
450K
500K 550K 600K 650K 700K 750K
total # spikes during inference (T=8)
total # spikes during inference (T= = 8)
(a) CIFAR-10 with VGG11
(b) CIFAR-100 with WRN-16-4
Figure 4: Coding efficiency comparisons vanilla and adversarially robust SNNs.
Robust accurwey (36)
80
ANN (XT s))
Robust accuzacy (36)
50
ANN (AT a)
ENN (Vaullla)
SNN (Vaillia)
40
SNN-RAT
SNN-RXT Ours (AT--G)
80
Ours (AT
30
50-
20
10
30
100
250
500
1000
2000
3000
4000
3000
100
250
500
1000
2000
3000
4000
5000
#queries
#queries
(a) CIFAR-10 evaluations with VGG11
(b) CIFAR-100 evaluations with WRN-
Figure Square Attack (Andriushchenko et al., 2020) evaluations with limited number of queries for the baseline ANN (AT-e1), end-to-end trained vanilla SNN, adversarially trained SNN-RAT, and ours (AT-e1).
during BPTT can result in much stronger adversaries. Specifically, these attacks revealed stronger and more accurate robustness evaluations for SNNs directly optimized with AT (Ding et al., 2022). Moreover, we present a comparison of robust baseline ANNs and converted SNNs under analogous attacks in Table B3, where we highlight the need for an ensemble setting for SNNs, since naive evaluations can be misleading relation to ANNs. We argue that our attacks reveal a more reliable robustness quantification for SNNs.
Out-of-distribution Generalization. We evaluated robust SNNs CIFAR-10-C and CIFAR-100-C benchmarks (Hendrycks & Dietterich, 2019) in Suppl. B.4. Our analyses yielded mean CIFAR- accuracies of 83.85% with our model, whereas SNN-RAT: 81.72%, Vanilla SNN: 81.22% (CIFAR-100-C, Ours: 54.06%, SNN-RAT: 51.95%, Vanilla SNN: 46.14%). We also observed that this result was consistent for all corruptions beyond noise (Patel et al., 2019) (see Tables B6 and B7).
7 Discussion
We propose an adversarially robust ANN-to-SNN conversion algorithm that exploits robustly trained baseline ANN weights initialization, to improve SNN resiliency. Our method shows state-of-the-art robustness in food-forward SNNs with low latency, outperforming recent end-to-end AT basod methods in robustness, generalization, and coding efficiency. Our approach embraces any robust. ANN training method to be incorporated in the domain of SNNs, and also adopt further improvements to robustness, such leveraging additional data (Rebuffi et al., 2021). Importantly, we empirically show that SNN evaluations against non-adaptive adversaries can be misleading (Carlini et al., 2019; Pintor et al., 2022), which has been the common practice in the field of SNNs.
We argue that direct adversarial training methods developed for ANNs yield limited SNN robustness both because of their computational limitations with BPTT (c.g., use of single-step inner maximization), and also findings with BPTT-based adversarial training leading to SNNs with bias to the used surrogate gradient.
11

---

Published in Transactions on Machine Learning Research (04/2024)
and vulnerability against adversaries that can leverage this. Therefore propose to harness stronger and stable adversarial training possibility through ANNs, and exploit better robustness gains.
One other innovation in this work is our method of utilizing adversarially pre-trained ANN batch-norm parameters within spiking tdBN layers, without the need to omit these from the models as conventionally done. Although batch-norm layers were found to help facilitating stable adversarial training of the deep ANNs that we consider for conversion, there has also been recent studies on ANNs that particularly explore the necessity of batch normalization layers in adversarial training. In particular, Benz et al. (2021) highlight the increasing adversarial vulnerability of ANNSs that utilize batch normalization layers due to the models' emerging dependence on non-robust features. Similarly, Wang et al. (2022) proposes to improve robustness by removing batch-norm layers altogether during adversarial training, which is in principle restricted to gradients
deep residual network architectures that use stable weight initialization methods for during adversarial training. There are also more recent studies that propose to replace batch-norm layers ANNs with more robust alternatives (c.g., aggregating several normalization layers and treating the model an ensemble of different models (Dong et al., 2022)). Although there has been some findings regarding its positive contribution to robustness (Kim & Panda, 2021b), the role of batch normalization in SNN adversarial robustness is currently not fully explored and to be studied in future work. We use bascline ANNs that contained batch-norm layers in our simulations, since these architectures have been the commonly used ones (e.g., VGG architectures with batch-norm), especially in state-of-the-art. ANN-to-SNN conversion studies (Rathi & Roy, 2021). In principle, our robust conversion algorithm still remains compatible with any of such developments even if the baseline ANN does not contain any batch-norm layers.
There were no observed white-box robustness benefits of SNNs with respect to ANNs. However, under black-box scenarios, converted SNNs were more resilient. Overall, we achieved to significantly improve robustness of deep SNNs, considering their progress in safety-critical settings (Davies et al., 2021). Since SNNs offer a promising energy-efficient technology, we argue that their robustness should be an important concorn to be studied for reliable AI applications.
7.1 Limitations
There are two main application scenarios for SNNs: (1) on event-based data, (2) standard non data sets where one can still achieve significant energy savings via SNNs (Davies el al., 2021). In our study we only focus on the latter, that is on direct coding SNNs with low latency where conventional attacks remain powerful as important open problem. In this case, can simply pre-trained on the same task, which our method requires. On the other hand, DVS adversaries constitute a different problem setting tailored to the event- based domain where a pre-trained ANN is not available. This problem also requires consideration of specialized attacks where adversarial noise should be binary, hence the conventional l norm attacks are not directly applicable (Marchisio et al., 2021; Büchel et al., 2022).
conventional ANN that is robustly
Acknowledgments
This work has been supported by the Graz Center for Machine Learning (GraMT.), and the "University SAL, Labs" initiative of Silicon Austria Labs (SAL) and its Austrian partner universities for applied fundamental research for electronic based systems.
References
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square attack: query-efficient black-box adversarial altack via random search. In European Conference on Computer Vision, pp. 484-501, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In International Conference on Machine Learning, pp. 274-283, 2018a.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. Tn International Conference on Machine Learning, pp. 284 293, 2018b.
12

---

Published in Transactions on Machine Learning Research (04/2024)
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short- term memory and learningtto-leam in networks of spiking neurons. Advances in Newral Information Processing Systems, 31, 2018.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint 3rX4::1308.3432, 2013.
Philipp Benz, Chaoning Zhang, and In So Kweon. Batch normalization increases adversarial vulnerability and decreases adversarial transferability: A non-robust feature perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7818-7827, 2021.
Wieland Brendel and Matthias Bethie. Comment on "biologically inspired protection of deep networks from adversarial attacks" arXiv preprint arXiv: 104.::770 2017.
Tong Bu, Wei Fang, Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks. In International Conference on Learning Representations, 2022.
Tong Bu, Jianhao Ding, Zecheng Hao, and ZZhaofei Yu. Rate gradient approximation attack threats deep spiking neural networks. Proceedings of the SEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7896 7906, 2023.
Julian Büchel, Gregor Lenz, Yalun Hu, Sadique Sheik, and Martino Sorbaro. Adversarial attacks on spiking convolutional neural networks for event-based vision. Frontiers in Neuroscience, 16, 2022.
Yongqiang Cao, Yang Chen, and Deepak Khosla. Spiking deep convolutional neural networks for energy- efficient object recognition. International Journal of Computer Vision, 113:54-66, 2015.
Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness. arXiv preprint arXiv:1902006705, 2019.
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. Parseval networks: improving robustness to adversarial examples. In International Conference on Machine Learning, pp. 854-863, 2017.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In International Conference on Machine Learning, pp. 2206 2216, 2020.
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. Robustbench: a standardized adversarial robustness benchmark. Xrv preprint Xix::22000.0 09670, 2020.
Mike Davies, Andreas Wild, Garrick Orchard, Yulia Sandamirskaya, Gabriel A Fonseca Guerra, Prasad Joshi, Philipp Plank, and Sumedh R Risbud. Advancing neuromorphic computing with Loihi: A survey of results and outlook. Proceedings of the IEEE, 109(5):911-922 1944, 2021.
Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu. Temporal efficient training of spiking neural network via gradient re-weighting. In International Conference on Learning Representations, 2022.
Peter U Diehl, Daniel Neil, Jonathan Binas, Matthew Cook, Shih-Chi Liu, and Michael Pfeiffer. Fast- classifying, high-accuracy spiking deep networks through weight and threshold balancing. In International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2015.
Jianhao Ding, Tong Bu, Zhaofei Yu, Tiejun Huang, and Jian Liu. SNN-RAT: Robustness-enhancee spiking neural network through regularized adversarial training. Advances in Neural Information Processing Systems, 35:24780-24777 2022.
Minjing Dong, Xinghao Chen, Yunhe Wang, and Chang Xu. Random normalization aggregation for adversarial defense. Advances in Neural Information Processing Systems, 35:33676 333676-33688,, 2022.
13

---

Published in Transactions on Machine Learning Research (04/2024)
Chaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu, and Tiejun Huang. Temporal effective batch normalization in spiking neural networks. Advances in Neural Information Processing Systems, 35:34377- 34390, 2022.
Rida El-Allami, Alberto Marchisio, Muhammad Shafique, and Ihsen Alouani. Securing deep spiking neural networks against adversarial allacks through inherent structural parameters. In Design, Aulomation & Test in Europe Conference & Exhibition (DATE), pp. 774 779. IEEE, 2021.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056 21069, 2021.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. RMP-SNN: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pollern Recognition, pp. 13558 13567, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
Sergey Tolle and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pp. 448 06-45,6411 2015.
Haiyan Jiang, Srinivas Anumasa, Giulia De Masi, Huan Xiong, and Bin Gu. A unified optimization framework of ANN-SNN conversion: towards optimal mapping from activation values to firing rates. In International Conference on Machine Learning, pp. 11945-14977 2023.
Hoki Kim. Torchattacks: A Pylorch repository for adversarial attacks. ar preprinl arXi::2010.01950, 2020.
Youngeun Kim and Priyadarshini Panda. Optimizing deeper spiking neural networks for dynamic vision sensing. Neural Networks, 144:686-695, 2021a.
Youngeun Kim and Priyadarshini Panda. Revisiting batch normalization for training low-latency deep spiking neural networks from scratch. Frontiers in Neuroscience, pp. 1638, 2021b.
Youngeun Kim and Priyadarshini Panda. Visual explanations from spiking neural networks using inter-spike intervals. Scientific Reports, 11(1):19037, 2021c.
Youngeun Kim, Hyoungscob Park, Abhishck Moitra, Abhiroop Bhattacharjee, Yeshwanth Venkatesha, and Priyadarshini Panda. Rate coding or direct coding: Which one is better for accurate, robust, and energy- efficient spiking neural networks? In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 71-75, 2022a.
Youngeun Kim, Yeshwanth Venkatesha, and Priyadarshini Panda. PrivateSNN: privacy-preserving spiking neural networks. In Proceedings of the AAAT Conference Arlificial Intelligence, volume 36, pp. 1 2022b.
Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion. On the effectiveness of adversarial training against common corruptions. In Uncertainly Artificial Intelligence, pp. 1012 1021, 2022.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009.
14

---

Published in Transactions on Machine Learning Research (04/2024)
Souvik Kundu, Massoud Pedram, and Peter A Beerel. HIRE-SNN: Harnessing the inherent robustness of energy-efficient deep spiking neural networks by training with crafted input noise. In Proceedings of the LEEE/CVF International Conference on Computer Vision (ICCV), pp. 5209 5218, 2021.
Ya Le and Xuan Yang. Tiny Tmage.eet visual recognition challenge, 2015.
Chankyu Lee, Syed Shakib Sarwar, Priyadarshini Panda, Gopalakrishnan Srinivasan, and Kaushik Roy. Enabling spike-baset based backpropagation for training deep neural network architectures. Frontiers Neuro- science, pp. 119, 2020.
Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, and Shi Gu. A free lunch from ANN: Towards efficient, accurate spiking neural networks calibration. In International Conference on Machine Learning, pp- 6316-638 6325. PMLR, 2021.
Yuhang Li, Youngeun Kim, Hyoungseob Park, and Priyadarshini Panda. Uncovering the representation of spiking neural networks trained with surrogate gradient. Transactions on Machine Learning Research, 2023.
Ling Liang, Xing Hu, Lei Deng, Yujie Wu, Guoqi Li, Yufei Ding, Peng Li, and Yuan Xie, Exploring adversarial attack in spiking neural networks with spike-compatible gradient. IEEE Transactions on Neural Networks and Leurning Systems, 2021.
Ling Liang, Kaidi Xu, Xing Hu, Lei Deng, and Yuan Xic. Toward robust spiking neural network against adversarial perturbation. Advances in Newral Information Processing Systems, 35:10244-100 2022.
Thomas Limbacher, Ozan Özdenizei, and Robert Legenstein. Memory-dependent computation and learning in spiking neural networks through Hebbian plasticity. IEFE Transactions on Newral Networks and Learning Systems, 2023.
Sen Lu and Abhronil Sengupta. Exploring the connection between binary and spiking neural networks Frontiers in Neuroscience, 14:535, 2020.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Newral Networks, 10(9):1659 1671, 1997.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Alberto Marchisio, Giorgio Nanfa, Faiq Khalid, Muhammad Abdullah Hanif, Maurizio Martina, and Muham- mad Shafique. Is spiking secure? a comparative study on the security vulnerabilities of spiking and deep neural networks. In International Joint Conference on Newral Networks (IJCNN), pp. 1 8. IEEE, 2020.
Alberto Marchisio, Giacomo Pira, Maurizio Martina, Guido Masera, and Muhammad Shafique. Dvs-attacks: Adversarial attacks on dynamic vision sensors for spiking neural networks. In International Joint Conference on Nearal Networks (LJCNN), pp. 1-9. IEEE, 2021.
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient- based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):11-63, 2019.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning, 2011.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Compuler and Communications Securily, pp. 506-519, 2017.
Adam Paszke et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32:8026 0003006--377,,, 2019.
15

---

Published in Transactions on Machine Learning Research (04/2024)
Devdhar Patel, Hananel Hazan, Daniel J Saunders, Hava T Siegelmann, and Robert Kozma. Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to atari breakout game. Neural Networks, 120:108 115, 2019.
Maura Pintor, Luca Demetrio, Angelo Sotgiu, Ambra Demontis, Nicholas Carlini, Battista Biggio, and Fabio Roli. Indicators of attack failure: Debugging and improving optimization of adversarial examples. examplees Advances in Neural Information Processing Systems, 35:23063-2306 -3076,, 2022.
Nitin Rathi and Kaushik Roy. DIET-SNN: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization. IEEE Tronsactions on Neural Networks and Learning Systems, pp. 1-9, 2021.
Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. In International Conference on Learning Representations, 2020.
Sylvestro-Alvisc Rebuffi, Sven Gowal, Dan Andrei Calian, Florian Stimberg, Olivia Wiles, and Timothy A Mann. Data augmentation can improve robustness. Advonces in Neural Informalion Processing Systems, 2021.
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607-667, 2019.
Bodo Rueckauer, Tulia-Alexander Lungu, Yuhuang Hu, and Michael Pfeiffer. Theory and tools for the conversion of analog to spiking convolutional neural networks. arXiv preprint (XXw:1612.04052, 2016.
Bodo Rueckauer, Tulia-Alexane Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Frontiers in Neuroscience, 11:682, 2017.
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: VGG and residual archilectures. Fronliers in Nearoscience, 13:95, 2019.
Saima Sharmin, Priyadarshini Panda, Syed Shakib Sarwar, Chankyu Lee, Wachirawit Ponghiran, and Kaushik Roy. A comprehensive analysis adversarial robustness of spiking neural networks. In International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2019.
Saima Sharmin, Nitin Rathi, Priyadarshini Panda, and Kaushik Roy. Inherent adversarial robustness of deep spiking neural networks: Effects of discrete input encoding and non-linear activations. In Europeon Conference Computer Vision, pp. 399-414, 2020.
Sumit Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. Advances in Neural Information Processing Systems, 31, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1812.6199, 2013.
Jihoon Tack, Sihyun Yu, Jongheon Jeong, Minscon Kim, Sung Ju Hwang, and Jinwoo Shin. Consistency regularization for adversarial robustness. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pp. 8414-8422, 2022.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, lan Goodfellow, Dan Boneh, and Patrick McDaniel. En- semble adversarial training: Attacks and defenses. In International Conference Learning Representations, 2018.
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. Tn International Conference on Learning Representations, 2019.
16

---

Published in Transactions on Machine Learning Research (04/2024)
Haotao Wang, Aston Zhang, Shuai Zheng, Xingjian Shi, Mu Li, and Zhangyang Wang. Removing batch normalization boosts adversarial training. In International Conference on Machine Learning, pp. 23433- 23445, 2022.
Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. Improving adversarial bustness requires revisiting misclassified examples. In International Conference Learning Representations, 2020.
Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representalions, 2020.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training performerrre spiking neural networks. Frontiers in Neuroscience, 12:331, 2018
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1311 1318, 2019.
Nuo Xu, Kaleel Mahmood, Haowen Fang, Ethan Rathbun, Caiwen Ding, and Wujic Wen. Securing the spike: On the transferabilty and security of spiking neural nelworks to adversarial examples. ar preprinl ar arXiv:2209.0330 2022.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Mochine Vision Conference, 2016.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International Conference on Machine Learning, pp. 7472-7482, 2019a.
Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsich. Towards stable and efficient training verifiably robust neural networks. arXiv preprint 1906.06316, 2019b.
Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-trained larger spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intellignnce, volume 35, pp. 11062-11070, 2021.
17

---

Published in Transactions on Machine Learning Research (04/2024)
A Details on the Experimental Setup
A.1 Datasets and Models
We experimented with CIFAR-10, CIEAR-100, SVHN and TinylmageNet datasets. CIFAR-10 and CIFAR- 100 datasets both consist of 50,000 training and 10,000 test images of resolution 32x32, from 10 and 100 classes respectively (Krizhevsky, 2009). SVHN datasel consists of 73,257 training and 26,032 test samples of resolution 32x32 from 10 classes (Netzer et al., 2011). TinyImageNet dataset consists of 100,000 training and 10,000 test images of resolution 64x64 from 200 classes (Le & Yang, 2015). We use VGG11 (Simonyan & Zisserman, 2015). ResNet-20 (He et al., 2016) and WideResNet (Zagoruyko & Komodakis, 2016) models with depth 16 and width 4 (i.e., WRN-16-4).
For baseline ANN optimization, we adopted the conventional data augmentation approaches that were found beneficial in robust adversarial training (Rebuffi et al., 2021; Tack et al., 2022). During SNN finetuning after conversion, we follow the simple traditional data augmentation scheme that was also adopted in previous work (Ding ct al., 2022). This procodure involves randomly cropping images into 32x32 dimensions by padding zeros for most 4 pixels around it, and randomly performing horizontal flip. For Tiny Inngee we instead perform a random resized crop to 64x64, also followed by a random horizontal flip. All image pixel values are normalized using the conventional metrics estimated on the training set of each datasct.
A.2 Robust Training Configurations
During robust optimization with standard AT, TRADES, and MART, adversarial examples are itcratively crafted for cach mini bati- (inner maximization step of Eq. (4)), by using 10 PGD steps with random starts under bounded perturbations, and v) 2.5 e/ #steps. For standard AT and MART we perform the inner maximization PGD using LIGD LCr (X), y), whereas for TRADES we use CPCD = DrL (fo (x)) as the inner maximization loss, following the original works (Madry et al., 2018; Zhang et al., 2019a; Wang et al., 2020). We set the trade-off parameter AMART = 4 in all experiments, and ATRADES= = 6 in C-FAR-10 and ATRADES = 3 in CIFAR-100 and SVHN experiments. We present extended robustness evaluations of the baseline ANNs used in our experiments in Table A1.
Baseline ANNs:
During initialization of layer-wise firing thresholds, we use 10 training mini-batches of size 64 with a calibration sequence length of T. = 100 timesteps, and observe the pre-activation values (Rathi & Roy, 2021; Limbacher et al., 2023). We choose the maximum value in the & 99.7% percentile of the distribution ion of observed values. This process is performed similarly to (Lu & Sengupta, 2020), and is a better alternative to simply using the largest observed pre-activation value for threshold balancing (Sengupta et al., 2019), since the procedure involves the use of randomly sampled training batches which might lead to large valued outliers compared to the rest of the distribution. We use x = 0.3 in CIFAR-10 and SVHN, and к = 0.1 in CTFAR-100 and Tmageeee experiments. During BPTT-based finetuning, we used the piecewise linear surrogate gradient in Eq (16) with The 1, as in Ding et al. (2022). For the initial random step of RFGSM, set c1/2 in CIFAR-10 and SVHN, and / in CIFAR-100 and TinylmageNet experiments. For 11 experiments with AT- and AT-E3, we used e/4.
Converted SNNs:
End-to-end trained SNNs used for comparisons were initialized statically at VA = 1 for all layers, as in Ding et al. (2022). During robust finetuning, we impose a lower bound for the trainable firing thresholds such that they remain VH, 2 0.03. At inference-time with our models, for idBN layers, we use the mean and variance moving averages estimated during the robust finctuning process from scratch. Since we performed conversion with baseline ANNS without bias terms (Sengupta et al., 2019; Rueckauer al., 2016), only in CIFAR-10/100 experiments we introduce bias terms during finetuning in the output dense layers to be also optimized, in order to have identical networks to SNN-RAT. Although it slightly increased model performance, we did not find the use of bias terms strictly necessary in our method.
Optimization: We used momentum SGD with a cosine annealing based learning rate scheduler, and a batch size of 64 in all model training settings. All end-to-end trained models (i.c., bascline ANNs and vanilla SNNs) were optimized for 200 epochs with an initial learning rate of 0.1, following the conventional settings (Madry et al., 2018; Zhang et al., 90; Wang et al., 2020). After ANN-to-SNN conversion and
18

---

Published in Transactions on Machine Learning Research (04/2024)
Table A1: Robustness evaluations (%) of used baseline ANN architectures in our experiments. All attacks are evaluated under an norm bounded c3 8/255 perturbation budget. AAx indicates AutoAttack evaluations. PGD attack step sizes were determined by 2.5 e/ #steps. Maximum number of queries for SquareAttack were set to 5000.
Clean Acc.
eg-Robust Accuracies
FGSM
PGI)
PGD)
PGD)
APGD₂E
Square
A
Natural
95.10
20.35
0.13
0.09
0.06
1.07
0.45
0.00
AT-é;
93.55
42.71
21.86
21.15
20.76
18.98
32.99
16.64
CIFAR-10
TRADES-Q
92.10
51.26
33.59
32.94
32.56
29.75
41.90
27.62
with VGG11
MART-e;
92.68
47.21
28.43
27.54
27.43
24.58
37.56
21.98
AT-r2
91.05
51.55
37.00
36.56
36.32
34.62
45.87
32.61
AT-é3
85.07
58.30
53.57
53.38
53.24
51.65
52.98
45.05
Natural
78.93
6.79
0.00
0.00
0.00
0.00
0.00
0.00
CIFAR-100
AT-é;
74.90
20.23
9.85
9.46
9.34
8.49
15.53
7.05
with WRN-16-4
TRADES-c
73.87
20.40
11.07
10.74
10.53
9.64
15.61
8.43
MART-c1
72.61
23.93
15.08
14.72
14.47
13.26
19.51
11.04
Natural
97.08
63.83
0.71
0.15
0.08
1.87
1.38
0.08
SVHN
AT-e1
96.50
52.53
30.36
29.56
29.25
25.86
29.30
23.58
with ResNet-20
TRADES-e1
96.29
54.11
33.43
32.63
32.18
29.13
32.10
27.16
MART-6
96.16
56.66
36.02
35.32
34.82
31.01
31.70
27.65
setting initial throsholds, SNNs were finetuned for 60 epochs on CIFAR-10, SVHN and TinylmageNet, and epochs on CIFAR-100, with an initial learning rale of 0.001. During robust finetuning, we used a weight, decay parameter of 0.001 in CIFAR-10, SVHN and TinylmageNer experiments, whereas used 0.0001 for CIFAR-100. We also used lower weight decay parameters of 0.0005 and 0.0001 during post-conversion finetuning of AT-ez and AT-cs models respectively in our CIFAR-10 experiments.
Computational Overhead: Although our method uses larger amount of total training epochs (baseline ANN training and robust SNN finetuning), the overhead does not differ much from end-to-end AT methods (e.g., SNN-RAT) due to the heavier computational load AT with SNNs which involves the temporal dimension. To illustrate quantitatively (CIFAR-10 with VGG11), Ours: baseline ANN training for 200 epochs takes ~11 hours and robust SNN finetuning for 60 epochs takes 10.5 hours, SNN-RAT: robust end-to-end training for 200 epochs takes 23 hours wall-clock time.
Implementations: All models were implemented with the PyTorrh 1.13.0 (Paszke et al., 2019) li- brary, and experiments were performed using GPU hardware of types NVIDIA A40, NVIDIA Quadro RTX 8000 and NVIDIA Quadro P6000. Adversarial attacks were implemented using the TorchAt- tacks (Kim, 2020) library, with the default attack aaper-paraaeters. Our implementations can be found at: https://github.com com RaaGGGaeeRRbbssSSNNoooaaoroo
A.3 Robust SNN Finetuning Parameter Updates with Surrogate Gradients
Our robust optimization objective after the initial ANN-to- -NN-to-SNNN conversion step follows:
min E (CE(Jo(x),)) 0
max xEA!(x)
Dxa,(J(())](((xx))
(11)
where 0 consist of [W', , "//."")' w t and 1 the adversarial example & E (x) is obtained at the inner maximization via REGSM using LReGSM = (fo((')))
(1) (x)
LoB - Eg0) log(se) (x)) and DKL Es log
(12)
19

---

Published in Transactions on Machine Learning Research (04/2024)
Following the robust optimization objective that minimizes overall loss function expressed in the brackets of Eq. (11), say frohust; the gradient-based synaptic connectivity weight and firing threshold updates in the hidden layers of the SNN [ = fI....,L L -1 1} are determined via BPTT as:
i Crobust θ w
① Crobust (t'(t) (z) (v (t) (z (t aw
A W
Σ
(13)
AV
Σ
θ Crobust
① Crobust 0s (t) (z)
(14)
(1) (z) 2v
where (t) = v V is the input to the Hcaviside step function: st()) = (t)( For BPTT we use the piecewise linear surrogate gradient with a unit dampening factor:
(t)(t) = m0 (z) (z)(()
(15)
Remaining gradient terms can be computed via LIF neuron dynamics. For the layers containing tdBN operations, the parameters s and w' are also updated similarly through normalized st(t), following the original gradient computations from Toffe & Szegedy (2015), which are however gathered temporally (Zheng et al., 2021). Since there is no spiking activity in the output layer, weight updates can be computed via BPTT without surrogate gradients, similar to the definitions from (Kundu et al., 2021; Rathi & Roy, 2021). Our complete robust ANN-to-SNN conversion procedure is outlined in Algorithm 1.
A.4 Ensemble Adversarial Attacks
The success of an SNN attack depends critically on the surrogate gradient used for by the adversary. Therefore we consider adaptive adversaries that perform a number of altacks for any lest sample, where the surrogate gradient used approximate the discontinuity of the spike function during BPTT varies in each case. Specifically, part of the attack ensemble we perform backpropagation by using the piecewise linear (triangular) function (Bellec et al., 2018; Neftci et al., 2019) defined in Eq. (16) with The e {0.25, 0.5, 1.0, 2.0,
0s'(1) 1 (1)
max 7m - -12(()))
(16)
the exponential surrogate gradient function (Shrestha & Orchard, 2018) defined as in Eq. (17) with hyper- parameters (1a.%) {(0.3, 0.5), (0.3, 1.0), (0.3, 2.0), (1.0,0.5), (1.0, 1.0), (L.0,.00).
(t)
= 7/4 exp ( (t)())),
(17)
0 (t)
and the rectangular function (Wu et al., 2018) defined as in Eq. (18) with Yor E
Os'(t) 1 sign 2 [z2(0)] Yw 2
(18)
Zz (1) Yw
For the rectangular and piecewise linear (triangular) functions, Tw indicates the width that determines the input range to activate the gradient. The dampening factor of these surrogate gradients is then inversely proportional to "ou (Ding et al,, 2022; Deng et al., 2022; Wu et al., 2018; 2019). For the exponential function, Na controls the dampening and 7 controls the steepness. A conventional choice for the exponential is (Ya, 76) = (0.3, 1.0) (Shrestha & Orchard, 2018), while we also experimented by varying these parameters. Interestingly, we could verify that varying the shape and parameters of this surrogate gradient function occasionally had an impact on the success of the adversary for various test samples. This scenario, for instance, indicates having approximate gradient information long before the membrane potential reaches the firing threshold (i.e., larger Yw), which can be useful to attack SNNs from the perspective of an adaptive adversary. In Section B.2 present the effectiveness of the individual components of the ensemble.
20

---

Published in Transactions on Machine Learning Research (04/2024)
Algorithm 1 Adversarially robust ANN-LO-SNN conversion
1: Input:
Training dataset D, adversarially pre-trained baseline ANN parameters ( . ا number of calibration samples, calibration sequence length To, percentile Pl threshold scaling factor k, number of finetuning iterations, perturbation strength e and random step a for RFGSM, simulation timesteps T, membrane leak factor T, trade-off parameter B.
Output: Robust SNN with parameters 0 consisting of (W", . w and (V
D Converting the adversarially trained baseline ANN
3: Initialize: Set weights of the SNN f directly from the pre-trained ANN parameters (W", ( J==
4: for = 1 to L - do
5: for c to #calibration_sanee
do 7: r' <- Store pre-activation values at layer during forward pass with direct input coding.
6: for f = to T. do
end for if max(p-percentile of the dist. in r>> Vh, then
10: V max(p-percentile of the distribution in r']
11: end if
12: end for
13: end for
> Initializing trainable firing thresholds
14: for /= to do
15:
Vih <- K 1/
16: end for
D Robust finetuning of SNN after conversion
17: for i= to # f'ineluning iterations do
18: Sample mini-batch of (x,y) D
> Inner maximization for single-step adversarial examples
19:
x' +- +a a sign (O))
20:
fx(x'), fe(x)),eexx) t Forward pass direct coded x' and x via Eqs. (1), (2), (3), (7), with r.
21:
x < x' + (c--opp a)- sign VVx CREGSM) with CRFGSM = Dfo (x)) llo())))))
P Outer optimization with the robust. finetuning objective
22:
fo(x), fo (x) +- Forward pass direct coded x and & via Eqs. (1), (2), (3), (7), with T.
23:
Lrotust f E.. [[cE(J(x)),)) + 3 DKL. (Jo()) ([o())
24:
A AW' + Σ, OW!
25: Ap' + Σ, Laulead
26: AW + Σ,
27: AV th Lx
0 Enatuees
0V
28: end for
In the ensemble, we also considered the straight-throughh estimator (STE) (Bengio et al., 2013), backward pass through rate (BPTR) (Ding al., 2022), and a conversion-basel approximation, where spiking neurons are replaced with ReLU functions during BPTT (Ding et al., 2022). STE uses an identity function as the surrogate gradient for all SNN neurons during backpropagation. BPTR performs differentiable approximation by taking the derivative of the spike functions directly from the average firing rate of the neurons between layers (Ding et al., 2022). In this case the gradient does not accumulate through time, and the complete neuronal dynamic is approximated during backpropagation by a single STE that maintains gradients for neurons that have a non-zero spike rate in the forward pass. We use the same BPTR implementation based on the cvaluations from Ding al. (2022). Conversion-based approximation was one of the carliest attempts design adversarial attacks on SNNs which we also include, although we found to be only minimally effective within the ensemble (Sharmin et al., 2019).
21

---

Published in Transactions on Machine Learning Research (04/2024)
Table B1: Evaluating e = 2/255 and E3 = 8/255 robust accuracies under black-box (BB) transfer attacks (Pa- pernot al., 2017), obtained via adversarial examples crafted with ensemble PGD attacks on a source model. In white-box (WB) setting, the source and target models are identical. BBsNN indicates attacks via souree model that undergoos the same ANN-to-SNN conversion procedure the target model, with different random seed during finetuning. ABAN indicates transfer attacks where the source model is the baseline ANN used for conversion (hence no ensemble needed).
Adversarially Robust ANN-to-SNN Conversion
Clean Acc.
E1 E1-Robust Accuracies
eg-Robust Accuracies
WB
BBSNN
BBANN
WB
BBsNN
BBANN
Natural
93.53
59.22
76.93
82.21
0.74
6.87
21.99
CIFAR-10
AT-é1
92.14
70.02
85.40
82.04
13.18
42.27
29.81
with VGGI1
TRADES-G
91.39
72.00
86.21
82.29
19.65
52.34
38.18
MART-c1
91.47
70.35
84.19
81.43
17.79
41.82
32.26
Natural
70.87
35.49
45.95
64.44
1.40
3.15
48.12
CIFAR-100
AT-
70.02
44.73
54.88
59.36
7.64
11.88
25.52
with WRN-16-4
TRADES-E
67.84
42.73
53.14
58.20
7.79
11.86
27.23
MART-1
67.10
43.46
53.21
57.83
8.90
13.39
28.03
Natural
94.18
74.30
82.21
91.71
10.35
14.96
89.18
SVHN
AT-E1
91.90
71.80
81.47
84.18
21.51
29.94
44.51
with ResNet-20
TRADES-A
92.78
74.27
83.30
85.00
23.96
31.62
43.37
MART-11
92.71
74.82
82.94
85.67
24.11
33.01
44.27
Attacks on the Poisson encoding BNTT model: In our TinylmageNet experiments, for FGSM and PGD20 on the BNTT model (Kim Panda, 2021b), we used the SNN-crafted BPTT-based adversarial input generation methodology by Sharmin et al. (2020). These attacks approximate the non-differentiable Poisson input encoding layer by using the gradients obtained for the first convolution layer activations during the attack, and was found to be powerful substitute. We also used two iterations (Athalye et al., 2018b) while crafting adversarial examples on this model, due to the involved randomness in input encoding (Athalye et al., 2018a). Overall, we use the same surrogate gradient ensemble approach, however with an obligatory approximation of the input encoding and EOT iterations for stable gradient estimates.
trantrornnfooiionn (EOT)
B Additional Experiments
B.1 Black-box Transfer Attacks
In Table B1 we present our transfer-based black-box robustness evaluations (Papernot et al., 2017), using both the baseline ANN, and a similar SNN model obtained with the same conversion procedure using a different random seed. We craft. adversarial examples on the source models with PGD also an ensemble setting. In all models, we observe that white-box robust accuracies are significantly lower than black-box cases. This result holds both for the é perturbation budget where the models were trained and finetuned on, well E3 where investigate robust generalization to higher perturbation bounds. Only in CIFAR-10, observed that adversarially trained bascline ANNs used for conversion were able to craft stronger adversarial examples than the SNNs converted and finetuned in an identical setting (e.g,, eg-robust ness for AT-e1: BBANN 29.81 %, BBSNN: 42.27%). We believe that this is related to our weaker choice of = for finetuning in CIFAR-10 experiments, whereas we used 3 in all other models. On another note, we observed that BB attacks are highly powerful on converted SNNs based on natural baseline ANN training (e.g., CIFAR-10 cg-robuss WB: 0.74%, BBSNN: 6.87%), demonstrating their weakness in the black-box setting.
22

---

Published in Transactions on Machine Learning Research (04/2024)
Table B2: Evaluating e3 = 8/255 FGSM attack robust accuracies from our CIFAR-10 experiments with VGG11, under individual components of the white-box ensemble. Triangular (piecewise lincar). exponential and rectangular surrogate gradients were used as part of BPTT. Underlined values indicate the strongest individual attack within the ensemble.
SNN-
ANN-to-SNN Conversion (Ours)
RAT
Natural
AT-51
TRADES-E
MART-E1
Clean Accuracy Ensemble FGSM
90.74
93.53
92.14
91.39
91.47
32.80
15.71
33.94
41.42
44.39
1.0
45.01
28.16
53.61
71.50
62.16
Triangular
Th = 2.0
39.27
63.06
74.36
82.94
79.27
The 3.0
43.23
77.79
83.59
87.72
86.30
The 0.5
82.20
19.62
39.25
52.09
51.56
The 0.25
90.15
54.15
57.48
51.03
50.16
(1a, Tts) (0.3, 0.5)
48.94
80.75
83.89
87.68
86.70
Exponential
(Na: Ys)) (0.3,
36.91
54.96
69.65
79.66
74.97
(Nd: Ya) = 2.03,
37.65
27.15
51.66
67.60
60.02
(1a: 7+) 0.5)
62.73
89.41
89.01
90.36
89.41
(1a, 7s) 1.0)
41.63
64.45
75.37
83.30
79.34
(Ta Ys) (1.0,2.0)
38.65
32.07
55.96
70.99
62.55
Yow 0.25
90.01
88.14
85.22
71.08
66.57
Rectangular
The 0.5
89.15
26.58
42.51
51.18
53.44
Twe 1.0
65.28
25.10
49.85
68.98
61.68
Ywe 2.0
45.21
63.45
73.59
83.49
80.22
Now 4.0
52.62
88.84
89.42
90.93
89.43
Backward Pass Through Rate
51.46
23.90
41.33
46.68
58.26
Conversion-based Approx.
84.25
39.73
55.35
89.43
87.79
Throughh Estimation
90.14
92.84
90.74
90.68
91.03
On the Analysis of Gradient Obfuscation: Previous SNN robustness studies (Kundu et al., 2021; Ding et al., 2022) analyzes gradient obfuscation (Athalye et al., 2018a) solely based the five principles: (1) iterative attacks should perform better than single-step attacks, (2) white-box attacks should perform better than transfer-based attacks, (3) increasing attack perturbation bound should result in lower robust accuracies, (4) unbounded attacks can nearly reach 100% success?, (5) adversarial examples can not be found through random sampling". the main paper and our results presented here, we could confirm all of these observations in our evaluations. However, we argue that more reliable SNN assessments should ideally go beyond these, we clarify in further depth via our ensemble evaluations (Carlini et al., 2019).
B.2 Evaluations with Alternative Surrogate Gradient Functions
Our detailed results for one particular ensemble attack is demonstrated in Table B2, where we focus on single-step FGSM without an initial randomized start. The third row indicates naive BaTT-based FGSM attack where the adversary uses the same surrogate gradient function utilized during model training (c.g., SNN-RAT: 45.01%, Ours (AT-e1): 53.61%). Importantly, we observe that there are stronger attacks that can be crafted by versatile adversary, by varying the shape or parameters of the used surrogate gradient. Moreover, in this particular case with FGSM attacks, we observe 4-6% difference in attack performance between the strongest individual attack configuration within the ensemble (c.g., underlined SNN-RAT: 36.91%, Ours (AT-e)): 39.25%), versus using ensemble of alternative possibilities (e.g., SNN-RAT: 32.80%,
3 After perturbation bound of 16/255, ensemble attacks on all models reached 1% robust accuracy. GGanssian naise with rg standard deviation only led to 0.4-3.0% decrease from the clean acc. of the models.
23

---

Published in Transactions on Machine Learning Research (04/2024)
Ours (AT-e1): 33.94%). This further em- phasizes the relative strength of our evalu- ations, in comparison to traditional naive evaluation settings.
Table B3: Impact of ensemble SNN attacks on SIFAR-10/100, with comparisons to ANNs under the analogous c3-attack. PGD uses BPTT with training- surrogate gradient.
vanilla
Table B3 shows a comparison of ANNs and SNNs under analogous PGD220 attack settings. Here, SNN evaluations under PGD vanilla indicate naive BPTT-based attacks, where the adversary only uses the same surrogate gradient utilized dur- ing training. Simply running these eval- uations would generally be considered analogous to those for the PGD20 ANN attacks (Ding al., 2022), although they are essentially not identical. Considering PGD vauilla accuracics, results might even
CIFAR-10 with VGG11
ANN
SNN
Clean
PGD
Clean
PGD vanilla
PGD ensemble
AT-c
93.55
21.15
92.14
37.42
13.18
TRADES-c
92.10
32.94
91.39
62.03
19.65
MART-G
92.68
27.54
91.47
49.90
17.79
CIFAR-100 with WRN-16-4
ANN
SNN
Clean
PGD
Clean
PGD
vanilla
PGD
ensomble
AT-c TRADES-C1
74.90
9.46
70.02
12.56
7.64
73.87
10.74
67.84
12.97
7.79
MART-11
72.61
14.72
67.10
14.27
8.90
be (wrongly) perceived as SNNs being more robust than ANNs, as claimed in previous work based on shallow models and naive evaluation settings (Kim & Panda, 2021c; Li et al., 2023)- However, since the SNN attack depends on the surrogate gradient, we also consider the ensemble PGD3 as we did throughout our work. Note that this stronger attack not necessarily equivalent to the ANN attack either, but we believe it is a better basis for comparison.
Comparison to RGA Attacks: We compare our ensemble attack approach also against a re- cently proposed, concurrent work on rate gradi- ent approximation (RGA) attacks on SNNs (Bu et al., 2023). Our results demonstrated in Ta- ble B4 show that the proposed ensemble ap- proach yiclds significantly stronger robust ac- curacy estimates both in our CIFAR-10 and CIFAR-100 simulations, and presents an al- ready significantly stronger evaluation baseline than RGA for the main models considered in this work (e.g., for experiments CIFAR-10 with
Table B4: Comparison of ensemble SNN attacks against RGA attacks (Bu et al., 2023) under eg with PGD
CIFAR-10 with VGG11
Clean
PGD
PGD casemble
SNN-RAT
90.74
17.98
10.06
Ours (Natural)
93.53
1.50
0.74
Ours (AT-e1)
92.14
16.44
13.18
Ours (TRADES-G)
91.39
26.51
19.65
Ours (MART-c)
91.47
24.53
17.79
CIFAR-100 with WRN-16-4
Clean
PGD
RGI
2GD
easemblee
SNN-RAT
69.32
7.72
6.04
Ours (Natural)
70.87
5.50
1.40
VGG11, SNN-RAT under 2GD 20 ensemble: 10.06%
Ours (AT-c1)
70.02
16.81
7.64
v8. PGD RGA: 17.98%, or Ours (TRADES-
Ours (TRADES-+))
67.81
16.16
7.79
a) under 2GD " 19.65% 26.51%). Nevertheless, it is indeed possible to integrate RGA (Bu et al., 2023) as an alternative method to approximate more stable gradients for the adversary, within our proposed ensemble SNN attack framework in future work.
PGDRA
Ours (MART-+1)
67.10
17.43
8.90
B.3 Ablations on the Robust SNN Finetuning Objective
Our ablation studies are presented in Table B5. The first three rows respectively denote comparisons to end-to-end trained SNNs using a natural cross-entropy loss, and using our robust. finetuning objective without. and with trainable thresholds, for 200 epochs. Note that in the third row, we are essentially comparing our ANN-Lo-SNN conversion approach, to simply end-to-end adversarially training the SNN using the TRADES objective. Our method yields approximately twice more robust models than the model in the third row (i.c., ensemble APGDcE: 4.72 versus 10.05), indicating that the weight initialization from an adversarially trained ANN brings significant gains. The fourth model indicates weight initialization from a naturally trained ANN within our conversion approach (previously included in Table 1 as well), which yields nearly no robustness since these weights are inclined to a non-robust configuration during natural ANN training.
Remaining models compare different choices for CRFGSM (e.g., adversarial cross-entropy loss (Tramèr et al., 2018)), and the outer optimization scheme (e.g., standard AT (Madry et al., 2018)), which we have eventually
24

---

Published in Transactions on Machine Learning Research (04/2024)
Table 15:
Ablations for the robust SNN finetuning objective with our CIFAR-10 experiments. Robust accuracies are evaluated via ensemble white-bos attacks. Bottom row indicates our method.
Baseline ANN
Inner Max.
Outer Optimization
Trainable
Clean Acc.
Co Cg-Robust Accuracies
CREGSM
V
FGSM
PGD
APGDcE
X
(CB(fo(x),)) (x((fa(x))lff(x)) DkL(fe(x')llfo(x))
93.73
6.43
0.00
0.00
Eq. (11)
91.90
24.94
7.23
5.32
X
Eq. (11)
91.75
23.83
6.70
4.72
Natural
(xL(fo()))l(fex))
Eq. (11)
98.53
15.71
0.74
0.44
AT-é1
CCE(fe(x),))
92.18
32.31
10.03
8,40
AT-61
(cm(fo(x'),))
(cu(fo(X).))
92.02
34.17
12.19
9.40
AT-é1
(kL(fe((")llfo())
(c
91.59
32.83
11.78
9.26
AT-G
(cx(fo(x'),))
Eq. (11)
92.13
33.15
11.96
9.32
AT-61
Dku(fo('')lfo(xx)
Eq. (11)
91.09
34.03
11.51
9.39
AT-61 (Ours)
(x
Eq. (11)
92.14
33.94
13.18
10.05
Table B6: Evaluations of vanilla. and adversarially robust SNNs on out-offdisttriution generalization to common image corruptions and perturbations using the CIFAR-10-C test set (Hendrycks Dietterich, 2019).
Values indicate averaged accuracies across the complete test set for all five severities.
Cleon Acc.
Clo- (xll)
Clopp
Weother
(w/o suiie)
Shot
Impulse
Liflorus
Zoom
Suow
Rog
Bright
Ocntrast
Elistio
Vanzlla SNN
53.73
81.22
53.04
72.3
70.0
70.5
85.1
153
80.0
83.4
85.3
85.8
91.8
67.5
86.6
68.
38.4
SNN TAT
90.74
81.72
81.00
50.4
85.9
71.
85.2
8533
80.
$3.8
83.5
89.2
723
87.7
56.1
85.3
89.2
58.7
Durs (AT-r.)
92.14
83.85
83.06
36.3
88.0
79.5
86.2
8200
81.9
84.9
$5.8
87.3
74.
90.7
65.3
85.3
9000
80.0
Table B7: Evaluations of vanilla and adversarially robust SNNs on out-of-disttibutioo generalization to common image corruptions and perturbations using the CIFAR-100-C test set (Hendrycks & Dietterich, 2019). Values indicate averaged accuracies across the complete test set for all five severities.
Cloaa Acc.
C100-C (X]]
C100-C
Noise
Blur
Weother
Digital
(w/o noteo)
Tmrgules
Bright
JPRG
76.20
33.2
19.9
SNN-RAT
60.32
$1.95
54.20
30.4
45.4
43.5
80.0
50.8
56.2
57.0
04.6
35.3
02.9
Durs (AT-.)
70002
54.00
65.54
42.5
40.
52.5
59.6
41.
53.0
67.5
38.0
57.5
48.2
66.0
38.4
58.9
02.0
64.1
chosen to follow a similar one to TRADES (Zhang et al., 2019a). We also observed significant robustness benefits of using trainable Vl during post-conversion finetuning as proposed in Rathi & Roy (2021) (e-g., clean/P clean/PGI) 2G 01.09/9.39 vs. 92.14/10.055 Overall, these results show that our design choices were important to achieve better robustness. Nevertheless, several alternatives in Table B5 (e.g., standard AT based outer optimization or LCE(Jo(x').)) based RFGSM) still remained elfective to outperform existing defenses.
Robust ANN Conversion Without Finetuning: In another ablation study, we evaluate our method without the robust finetuning phase altogether. is important to note that in hybrid conversion, finetuning is a critical component to shorten the simulation length such that one can achieve low-latency. Since we also reset the moving average batch statistics of tdBN layers during conversion, these parameters can only be estimated reliably during the robust. finetuning phase. Thus, right after conversion and setting the thresholds, our SNNs only achieve chance-level accuracies with direct input coding for simulation length of T 8.
However, we evaluated our models (CIFAR-10 with VGG-11) that were only finetuned for single epoch, or only two epochs lo exemplify. After one epoch of robust finetuning the SNN could only achieve a clean/PGD* 20 robust acc. 87.25% 11.42%, and with only two epochs robust finetuning achieves 89.15% whercas our model obtains 92.14% 19%/1..11% in the same setting after 60 epochs of finctuning. These results reveal the benefit of reasonable robust finetuning stage, both in terms of obtaining more reliable tdBN batch statistics following conversion, well adversarial finetuning of layer-wise firing thresholds.
25

---

Published in Transactions on Machine Learning Research (04/2024)
B.4 Out-of Out-of-Diste of---ssbbibttiinnn Generalization to Common Image Corruptions
It has been argued that adversarial training can have the potential to improve out-of-distributiion generaliza- tion (Kireev et al., 2022). Tables B6 and B7 presents our evaluations on CIFAR-1O-C and CIFAR-100-C respectively (Hendrycks & Dictterich, 2019). Results show that averaged accuracics across all corruptions (noise, blur, weather and digital) improved with our AT-c model (i.e., CIFAR-10-C: 83.85% and CIFAR-100-C: 54.06%), comparison to SNN-RAT and vanilla SNNs. Importantly, this improvement persists across all individual corruptions, and CIFAR-10/ C00-CC performance without the noise corruptions maintains similar behavior (see w/o noise averages in Tables B6 and B7). Notably for CIFAR-1O-C we observe that SNN-RAT provides noise robustness, but mostly not improving performance in other corruptions (i.e., CIFAR-10-C (all): 81.72%, CIFAR-10-C (w/o noisc): 81.60%).
Overall, our conversion-based approach achieved to significantly improve SNNs in terms of their adversarial robustness and generalization. Nevertheless, there are also various other security aspects that were not in the scope of our current work (c.g., preserving privacy in conversion-based SNNs (Kim al., 2022b)), which requires attention in future studies towards developing trustworthy SNN-based Al applications.
26


Reviews:
[{"review_id": "XTPg8PMTqN", "reviewer": "Reviewer_gqSN", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The conversion algorithm and robust finetuning methodology are well-designed and clearly explained.\n2. The ensemble evaluation strategy enhances the evaluation process, leading to more reliable robustness assessments of SNNs, which could benefit the community.\n3. The experiments are sufficient, which include the evaluation of the proposed method on various datasets and networks.\n\nWeaknesses:\n1. The authors introduce an ensemble evaluation strategy to tackle the issue of obfuscated gradients. However, some relevant effective attacks have not been discussed or compared, such as [a]. In addition, how about utilizing traditional EOT? Will EOT address the obfuscated gradients in SNNs?\n2. It is weird to perform AA attack to baseline ANNs but FGSM/PGD to SNNs in Table 1 since AA is regarded as a stronger attack than FGSM/PGD. In addition, how about the results of baseline ANNs after adversarial training compared with robust finetuning SNNs?\n3. Some relevant works that tackle adversarial robustness via normalization layers, such as [b,c,d], should be discussed.\n\n[a]. Rate Gradient Approximation Attack Threats Deep Spiking Neural Networks. CVPR 2023.\n\n[b]. Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective. ICCV 2021.\n\n[c]. Random normalization aggregation for adversarial defense. NeurIPS 2022.\n\n[d]. Removing batch normalization boosts adversarial training. NeurIPS 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a study on adversarial robustness in spiking neural networks (SNNs). The authors propose an approach to convert adversarially trained artificial neural networks (ANNs) into SNNs and then perform robust finetuning on the SNNs. The conversion algorithm from ANNs to SNNs involves different steps of configuring SNNs, including initializing weights, converting batch normalization, and initializing trainable firing thresholds. They also introduce an ensemble evaluation strategy for assessing the robustness of SNNs against adversarial attacks. The experiments on different datasets demonstrate the effectiveness of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Please discuss the relevant attacks and compare with them if possible.\n2. Please clarify the setting in Table 1 and discuss the results after adversarial training.\n3. Please discuss the relevant works related to batch normalization in adversarial robustness.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pykGh34j8o", "reviewer": "Reviewer_eKJA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, the paper is well-written and organized well. The numerical experiments to show the performance of the proposed conversion methods. \n\n### Major\nIt seems that the SNN is over-parameterized and has redundancy in the conversion. I see the authors adjust $\\phi^l$ and $\\omega^l$ in the batch norm layer, and also adjust the firing threshold in the SNN. Here the adjustable firing threshold seems redundant with $\\phi^l$ and $\\omega^l$. Can the SNN achieve the same performance if we didn't tweak the firing threshold?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This study proposes a way to convert the pre-trained ANN with continuous neurons into spiking neural networks(SNNs) to enhance the adversarial robustness of SNNs. The innovation of this study is converting the batch-norm layer parameters in ANN into the corresponding batch-norm parameters in SNNs. In contrast, conventional methods omit the conversion of batch-norm layer from ANN to SNN.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "There are some mistakes/typos in the manuscript. \n\n- Eq. 7: the $\\mathbf{v}^L(t-1)$ on the RHS misses a factor such as $\\tau$.\n- Text right above Eq.8: the abbreviation \"tdBN\" is undefined and I need to guess.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nKewhwXwuE", "reviewer": "Reviewer_xfzq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The idea described in the paper is relatively simple, so the paper is easy to follow. I have appreciated the care put into the experimental evaluation in terms of baselines, attacks, etc. Some figures of the proposed method could have improved even more the readability.\n\nAs a general comment, I am not sure I understand the motivation of the paper. There is still a sizeable gap in performance between SSNs and standard NNs when applied to these datasets (a gap which is shrinking but still there), so caring about the robustness of these models at this point is a bit strange. The proposed method does not extend to more general \"spiking\" datasets.\n\nIn the experimental part, I noted the lack of clear visualisations of the adversarial attacks. As a sidenote, I am familiar with the literature on adversarial attacks on CNNs but less so on SNNs, so I am not sure about how these visualisations can be achieved. However, I am left wondering whether the resulting adversarial attacks are \"easy to spot\" by eye.\n\nOverall, this is a technically valid paper. From what I know of the literature, it is novel, although for now it is more of an interesting observation (\"SNNs must be initialised from robust NNs to be robust\") than a practical concern.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper deals with the adversarial robustness of spiking neural networks (SNNs). In particular, they consider the setup where a standard NN is first trained (e.g., a CNN on CIFAR-10) and a spiking SNN is then \"distilled\" by copying the weight matrices and initialising the other parameters (e.g., the firing rates). The SNN is then fine-tuned on the same dataset (properly encoded).\n\nThey show that in order to obtain good results, one has to combine adversarial training of the original NN and adversarial fine-tuning of the resulting SNN. This last step is only done on the firing rate thresholds (the weights are kept fixed) and on the parameters of the batch normalization layers.\n\nThey provide an extensive set of results showing the good performance of their method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "In Section 3.2, I do not follow why the authors are using the RFGSM loss if they described another one previously. In general, it is a bit hard to distinguish what is novel in the paper from what has been lifted from previous literature. \n\nAs an example, I would ask a clarification on the novelty of Section \"Conversion of Batch Normalization\". If I understand correctly, the authors are advocating (i) discarding the previous statistics, (ii) re-computing them (across time and batches) for the SNN, and (iii) fine-tune the trainable weights?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers agreed that the paper was generally clearly written and technically sound. They had some questions and points of clarification, which they authors were able to address. As well, this paper is of interest to those researchers who study SNNs. Therefore, an 'accept' decision was reached. The authors now have to upload the final, camera ready version that incorporates the changes included for the reviewers.
Decision: Accept (poster)