Toward Robust Spiking Neural Network Against Adversarial Perturbation
Authors: Ling Liang
Kaidi Xu
Xing Hu
Lei Deng
Yuan Xie
Venue: NeurIPS 2022 Accept (2022)
Keywords: Spiking Neural Network
Certified Training
Adversarial Attack

Abstract: As spiking neural networks (SNNs) are deployed increasingly in real-world efficiency critical applications,  the security concerns in SNNs attract more attention.
Currently, researchers have already demonstrated an SNN can be attacked with adversarial examples. How to build a robust SNN becomes an urgent issue.
Recently, many studies apply certified training in artificial neural networks (ANNs), which can improve the robustness of an NN model promisely. However, existing certifications cannot transfer to SNNs directly because of the distinct neuron behavior and input formats for SNNs. In this work, we first design S-IBP and S-CROWN that tackle the non-linear functions in SNNs' neuron modeling. Then, we formalize the boundaries for both digital and spike inputs. Finally, we demonstrate the efficiency of our proposed robust training method in different datasets and model architectures. Based on our experiment, we can achieve a maximum $37.7\%$ attack error reduction with $3.7\%$ original accuracy loss. To the best of our knowledge, this is the first analysis on robust training of SNNs.

Toward Robust Spiking Neural Network Against Adversarial Perturbation
Ling Liang UC Santa Barbara
Kaidi Xu Drexel University
Xing Hu
SKL of Processors
1 Lingliangeucsb. edu
kx46@drexel. edu
Institute of Computing Technology, CAS
huxingdict.ccm ac. cn
Lei Deng
Yuan Xie
Tsinghua University
Alibaba Group yuanxie@gmail.en
leideng@mail. tsinghua. edu cn
Abstract
As spiking neural networks (SNNs) are deployed increasingly in real-world effi- ciency critical applications, the security concerns in SNNs attract more attention. Currently, researchers have already demonstrated an SNN can be attacked with ad- versarial examples. How build robust SNN becomes urgent issue. Recently, many studies apply certified training in artificial neural networks (ANNs), which can improve the robustness of an NN model promisely. However, existing certifica- tions cannot transfer to SNNs directly because of the distinct neuron behavior and input formats for SNNs. In this work, we first design S-IBP and S-CROWN that tackle the non-line non-linear functions in SNNs' neuron modeling. Then, we formalize the boundaries for both digital and spike inputs. Finally, we demonstrate the efficiency of our proposed robust training method in different datasets and model architec- tures. Based on our experiment, we can achieve maximum 37.7% attack error reduction with 3.7% original accuracy loss. To the best of our knowledge, this is the first analysis on robust training of SNNs.
Introduction
Spiking neural networks (SNNs) are class of models that have the potential ability to simulate the behavior of neuron circuits in the brain. Among the rich biological characters of SNNs, the event- driven information propagation mechanism makes SNNs easy deploy with limited computation resources. Many neuromorphic chips are designed to achieve low-power SNN inference, such as TrueNorth [1] Loihi R. SpinalFlow
23 and H2Learn 20
With more attention the study of SNNs, the security issues also raise concerns in the community. Adversarial attack 27
3 is one of the most intuitive ways to evaluate the robustness of model. In adversarial attacks, the attacker generates adversarial examples to fool model that has wrong prediction. Currently, SNNs are demonstrated can be attacked through adversarial examples
26 19
1
21.
urgent explore an efficient way to improve the robustness of SNN models.
Previously, researchers have investigated the impact of hyperparameter selection 10 and input filtering [21] on the adversarial attack in SNNs. However, these methods do not directly promote the classification behavior of given SNN model. [5] proposes a robust training method that relies on the adversarial attack in parameter space. Unlike SNNs, how to improve the robustness of an artificial neural network (ANN) has been well studied. Recently, trianing neural network model with certified defense methods model's robustness. CROWN-IBP 3N one of the most promising certified training methods with
33 show remarkable guarantee improve the
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

polynomial computational cost compared to natural training. The CROWN-IBP method computes the output boundary for given bounded input. However, the current CROWN-IBP method cannot be applied directly to SNNs. Firstly, the neuron dynamic in SNNs is more complicated. Hence, new boundary functions should be defined. Secondly, SNNs accept both spike and digital inputs, which requires additional boundary generalization for different input types.
Enlightened by certification training in ANNs, designed an end-to-end robust training method to improve the robustness of an SNN model against adversarial attacks. Specifically, our contributions can be summarized as follows:
We design S-IBP and S-CROWN tackle the non-linear fire function and temporal update in SNNs which firstly introduce SNNs to the linear relaxation based verification family.
We formalized f-rmomm and norm boundaries for spike and digital inputs, respectively.
Our proposed methods evaluated MNIST [18] FMNIST B2 and NMNIST [24 I dataset: The experimental results show that we can achieve a maximum 37. .77 attack enror reduction with 3.7% original accuracy loss.
2 Preliminary and Related Work
2.1 Spiking Neural Netowks (SNNs)
Neuron Modeling: Usually, SNNs are designed to simulate the neuron behavior in the brain. In this work, we adopt the well-studied leaky integrated and-fire (LIF) [12 for the neuron modeling Figure aa). The state of neuron each time step is determined by its membrane potential m and its spike status 8. Specifically, a neuron will f'ree a spike to the post-synaptic neurons once its membrane potential is greater than threshold th and its membrane potential will be reset to 0) the same time. An example of a neuron dynamic is shown in Figure lb)) which simulates neuron's ss behavior in time steps. The spike status of neuron can be formulated
[k] fire(melk)) th).
(1)
1, 20
fire(x) = (0 otherwise.
(2)
We use my [k] and sy[k] to represent neurons' membrane potentials and spike status at t-th time step and k-tth layer. fire(-) is the Heaviside step function. The membrane potential of neuron is composed of spatial and temporal update that follows
m. m.[k]
wilk s.lk am1--1
[K] (1 81 [k])
apatial
temporal
The spatial update a Convolution Fully Connected or Pooling operation between the weight and the spike events in previous layer. The temporal update is determined by the neuron's 's membrane potential and spike status in the previous time step. Here, a is scaling factor to control the decay.
Membrano Potensill (m))
Input
Output
m
Fire
SE
Layer
Layer
Layer
Inpul '3'
Spike
Digital Input 3'
Time Steps
Spatial Update
Tempord Updato
Class
3
Fire
Time Steps
1
2
3
Spike Train (5)
0 000
Blollgicr Signel
Class
(a)
(b)
(c)
(d)
Figure 1: Introduction of SNNs: (a) neuron modeling; (b) computing model of a neuron; (c)
spatiotemporal flow of an SNN model; (d) input formats of an SNN.
Network Structure: SNNs are built with an input layer, hidden layers, and an output layer as Figure c). In SNNs, the input layer receives spike inputs. The hidden layers are CONV/FC/POOL that correspond to the spatial update in the neuron modeling. Compared to ANNs, SNNs involve an

---

additional temporal axis that passes the neurons' status along with time steps. The final recognition of an SNN is determined by the spike events of the output layer. In this work, we adopt the rate coding, i.e., the neuron that fires the most spikes indicates the classification result.
Input Format: In this work, focus image recognition tasks. The input of SNN can be spike events captured by DVS devices [25]. Also, SNNs can take a digital image as input with an additional sampling. In this work, we adopt Bernoulli sampling ( for digital inputs that follows
[ 1) = =[1]-
(4)
We use dt to represent the spike input at time step f. 2 is the digital input after normalization to [o, 1]. The example of spike and digital inputs are shown in Figure (d).
Recently, supervised learning via backpropagation through time (BPTT) has been adopted in SNN training [3I]. The BPTT-based training method can achieve higher accuracy and scalability compared to unsupervised learning algorithms. Also, the SNNs trained with BPTT require fewer time steps. which is more hardware friendly in the real deployment. Thus, in this work, focus on the SNNs trained with the BPTT-based learning method.
Training Method:
2.2 Adversarial Attack in SNN
The generation of adversarial examples is one of the most powerful ways of affecting model's robustness (27 / 39 36). In this work, we use adversarial attacks as an assessment tool to evaluate the robustness of an SNN model. Specifically, in this work we adopt gradient-based attack method 19 for SNNs.
Untargeted White-box Gradient-basee Attack: In adversarial attacks, the attacker adds impercepti- ble noise on inputs to fool the model predict wrong results. In untargeted attack, the model predicts the adversarial example as any other classes except the ground-truth, which can be formulated as
@emmnn|00lp s.l. f(x + 8) + f(x).
Here, f()) is the prediction result, æ is the original input, and ô is the noise added on the input. In this work, adopt white-box attack, which means the attacker knows all of the model information.
The gradient-based attacks are the most efficient ways to generate adversarial examples. In SNNs. the gradient-based attack for the spike and digital inputs can be formulated as
i 1/41 Hh + Fl s4 L(O, H'y Morg)],
spike input,
(6)
E. Fl My L(O, H',. Morg)] 11 H/
digital input.
T
We use and a' to represent the spike and digital adversarial examples, i.c., x' = x + 5. L and 0 are the loss function and parameters of the model, respectively. The adversarial example is constructed by adding the gradient of inputs with the original label (Yorg) in loss function. F a filter function that samples, clips, and generates spike compatible noise. During the attack, attackers can compute the adversarial examples iteratively. We use n to denote the attack iteration.
2.3 Certified Training
has been demonstrated to improve the guaranteed ro- bustness of a neural network. In this work, we leverage the CROWN-IBP method [S)] one of the tate-offtheeart certified training that achieve tighter bounds in accepetable training cost. Considering a digital input data bounded with loo -nom,, the goal of CROWN-IBP to identify whether arbitrary input data within the boundary can fool the model. The CROWN-IBP contains two and CROWN B7
Recently, the certified training 13
38 3
parts of bounding methods: IBP [
IBP: In IBP processing, the lower and upper bounds of each layer's 's feature map are computed along the forward propagation. During the bound computation, the linear operation can be easily bounded once we know the maximum and minimum values of input. However, the upper and lower bounds after the non-linear operations need to dedicated to analysis. Once we acquire the bounds of output we can evaluate the robustness of the model for the given input purbation set.
3

---

Unlike IBP, CROWN bounds the model in a backward propagation manner recursively. The goal of CROWN is to formulate the output bounds as a linear equation of input. order to achieve this goal, every operation in an NN model should be bounded by two linear equations.
CROWN:
Although CROWN can achieve very tight bounds, the computational cost is remarkably higher than IBP. So CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and CROWN in backward bounding pass can efficiently and consistently outperforms IBP baselines on training verifiably robust neural networks.
Integrate with Other Approaches: One of the promising characteristics of certified training is that can be integrated with other algorithms. For example, the combination of certified training and adversarial training can further improve the robustness of model 3 [ل] For the operations which can improve the performance of a model and only involve linear operations, the certified training
algorithm can be directly applied without causing additional relaxation loss (8 29 - Currently, the
adversarial training [17 and batch normalization [4 15 have demonstrated their effectiveness
in improving the robustness of an SNN model, which have the potential to be integrated into our proposed design in the future.
Methodology
3.1 S-IBP & S-CROWN
As described in Scction 2.3 the core mission of certification training is find the bounds of IBP and CROWN for each function. The information propagation in SNNs includes fire, temporal update and spatial update as shown in Figure [ In this subsection, we detail the upper bound and lower bound of S-IBP and S S-CROWN for each function. During the bound formulation of a function, we assume the S-IBP bounds for function inputs have been acquired.
Fire: The fire function describes the relationship between the membrane potential m, and spike s of neuron as Equation I [and i.e. once the neuron's membrane potential is greater than threshold th, the neuron will fire a spike. Assume that we have already acquired the S-IBP upper bound m, and lower bound my for membrane potential, the S-IBP bounds for spike can be calculated with
8 = fire(mū -h), sf = fire(m! -th).
During S-CROWN, we need to find two linear equations to bound the fire function. When the S-IBP upper bound of membrane potential m is smaller than the threshold th, we can conclude that the neuron must not fire. Also, when the S-IBP lower bound mi greater than the threshold, we can make sure the neuron must fire. Thus, we mainly need to consider the unstable case, i.e., m/ th < mm. In order to achieve the lowest bound relaxation, we design two boundary systems as Figure 2 a) and (b). We use the red line to represent the fire function. The blue and yellow lines represent the boundary functions. When the S-IBP lower bound of membrane potential mf is much smaller than th, we set the S-CROWN lower bound to s, = 0 and the S-CROWN upper bound line that crosses (m/, 0) and (th, 1). In contrast, when m, is much larger than th, we set the S-CROWN upper bound to s; = 1 and the S-CROWN lower bound is line that passes (th, 0) and (my. 1). The bound can be potentially optimized by solving the decent gradient (55). Overall, the
S-CROWN boundary for the fire function under different cases can be summarized as
0 0 8 S 0
m < th,
1 1 81 < 1,
m/ > th,
(8)
0 8, <
m4- mp-m
< m. 1h < 1h- - mb
mh-th m th
< 81 < 1,
my th 2 th m > 0
Temporal Update: According to Equation 3 the temporal update of membrane potential can be formalized as com4 (1 - 8t). Since a a constant factor. in this subsection we do not include the boundary analysis. Assume 2; m. 82), and the S-IBP upper and lower bounds for membrane potential at time step has been acquired. The relation between m, and 2, can be described through Figure 2 c). Based on the LIF model, the membrane potential will be reset to once its value acrosses

---

St
Zt
my (1 se)
2 m 5s)
ab
(1 (0, (8)
m
m1 md
md 15
St
(0, oh.
m
(L.m(.rr)
(b)
(6)
(d)
Figure 2: S-CROWN upper and lower bounds for the fire function when (a) m is much smaller than th and (b) my much larger than th. (c) temporal part memory potential update. (d) S-CROWN upper and lower bounds for the temporal part of memory potential update (unstable case).
the pre-defined threshold th. Thus, the S-IBP bounds for 2, can be computed with
zt ma, = my, zt z 0, 2t = th, =t = min m²),
m < th, m 2 th,
m < th S m
When m. < th or m4 2 th, there is no boundary relaxation in S-IBP. We only need to consider the boundary when m² < th < m, (unstable case), where 2, E [min(0, m²), th) for unstable case.
During S-CROWN, 2, can be also bounded without relaxation when m < th and m 2 th. The S-CROWN upper bound (blue plane) and lower bound (yellow plane) for 2 when mi th < m are shown in Figure d). Here, we use an additional (1 82) axis to help us build the boundary. Note that the membrane potential related the spike status. Once 3, 1 1 (1-s) s,)
0, me must be greater than th and 2 = O. When 8, 0 - - s²) = 1, my must be smaller than th and 2, my Thus, we need find planes bound these two functions (red lines Figure d)). In summary, the S-CROWN boundary for the temporal update can be formulated as
m
S 2t S MiI1
m < Lh, m 2 th, sm) < < (1 8t)th, m. < th S mm
0
S 2 S O,
(10)
Spatial Update
The spatial update in SNNs is shown in Equation B Similar to ANNs, the spatial update in SNNs is composed of CONV/FC/POOL (in this work we focus on the CONV and FC). In SNNs CONV/FC takes spike events and weight as input. Since the spike events are in binary format, the S-IBP of spatial update can be implemented with
center = [*] @ [|[M] + b[k], spi
center + w[k]* (s)[k) on [A] = 1), spl j: + 1] center w[A] @ [<] 0n s [[]]]] 1).
(11)
We use sp, to represent the result of the spatial update. @ denotes CONV/FC operation. In Equation 11 s [ represents those pre-synaptic neurons in layer k who are must fire. The stable fired pre-synaptic neurons contribute the same for both S-IBP upper and lower bound of spt. The unstabled pre-synaptic neurons can be represented with (s) [k] 0 On s [A: = 1). whose upper and lower bound for spike status are and 0. These unstable pre-synaptic neurons will perform CONV/FC with the positive and negative weights to affect the S-IBP upper and lower bound of spt-
Since the spatial update is a linear operation, the S-CRWON for spatial update does not have relaxation. We do not need to design boundary functions bound the spatial update in SNNs.
3.2 Input Boundary Formalization
In SNNs, the input layer an SNN model only accepts binary spike. The special data format for the input layer leads to different boundary formalizations for spike and digital images.
Spike Input: example of spike input is shown Figure dd). All elements in a spike data are in binary format. For each element 2x[i] in spike input, the boundary of that element can be either
5

---

stable cases: [ 2 = 0; [| [ 1 or unstable case: - = 1, 1. = 0. For spike imput with uncertainty noise, we can only control how many data points the spike input are unstable. Thus, the boundary for spike input can be formulated with fo-norm. Specifically, we can pick (iz)(() x e elements from a spike input and set them as unstable points. Also, the norm boundary can be interpreted as the probability of an unstable element, which can be formulated as
P [ 1, + = 0) = + | #/ S size(() x
(12)
Here, is an arbitrary adversarial example that has at most size(E) x data points different from
Note that we can only certify the robustness of spike input after we have picked the unstable data points. Under our robustness formulation, we cannot guarantee the robustness of a spike input under a given fo-nomm The reason is that the search space for f--norm cannot be bounded.
6/2
e/2
Unatable Region
Stable Region
0.8
0.5
0.1
0.9
0.6
0.2
0.7
0.4
0.0
planntablee
0.8
0.2
0.3
0.7
0.3
0.4
0.5
0.1
0.2
411 401
rand
x2
rand, 01:0
rand, D
0.5
08
0.2
0.5
0
0
0.8
0.0
0.7
0
o
Figure 3: Relation between loo -norm digital input doundary and 6o-norm spike input boundary. In this example, e = 0.2.
Digital Input:
Digital inputs need additional sampling before feeding the data to the input layer as Equation 4 Note that we have defined the fy-norm boundary for a spike input. We need to further explore how to define a boundary for digital input when its sampled data is bounded with En-nomm. Based our analysis, we find that the corresponding digital input boundary can be formalized with Eo -nom as We use rand, to represent rand mask achieve Bernoulli sampling for each time step:
Figure
1,
a|i] > rand [i],
d [
(13)
0, otherwise.
After we bound the digital input with lo -nrmm, the upper bound of digital image becomes 3t f + c/2 and the lower bound is _ = f c/2. For all elements in é, the difference between the upper bound and the lower bound is i.e., .0 - [ c. For each time step, 1 and a' use the identical random map rand, to sample the corresponding spike inputs i and i/. Based on the sampling processing, the probability of element in input layer is unstable be formulated
[ = 1, i = 0) [1 a' =
(14)
From the static perspective, the lo -norm boundary for digital input is equivalent to the fo-norm boundary for spike input.
For digital inputs, although we apply foe -nom on input boundary, we cannot guarantee the robustness of the input under such boundary. Since for digital input, the possible sampled results are equivalent to the entire space (each element in the spike input can be or O), which cannot be bounded.
For both spike and digital inputs, need to carefully select for robust training. An extremely small e will cause the upper and lower bound identical which does not consistent with linear relaxation. A large may cause the following robust training to diverge. In this work, is selected based the noise magnitude in adversarial attacks and some empirical experiments.
3.3 Robust Training Algorithm
In this subsection, we present the end-to-end robust training for an SNN. For the overall flow, our method will first run S-IBP to get the boundary of intermediate data. Then we apply the S-CROWN which needs the S-IBP bound. Finally, apply the certification--asee robust training.

---

Algorithm S-IBP
Imput:
spike input digital input input boundary e: robust training time steps T":
Func:
for T
input boundary formalization
then
generate random map randy:
# (E + c/2) rand; > 0.? 1:71:00 U:
= (a e/2) rand, > 0.11::0.
else
Randomll pick size((ts) x elements from di and label the picked elements with picks; #7 di = ze: #7 [picke] = 1; #5 prcck]] = 0:
* [0] #7: +1(0) = #/:
inital m." [A:] [k] for all layers. WS-IBP
for k to
Rsputial update (E4.
my [o] + spr [R]:
m! = sp
Meemgural update (Ei if < T" then
m²41 [b] = a+ #2: mi+1lM [R] ma:
end if
Wire (Eq M
of [k] (ire(m; [k]
th): [[[+]] = fsre(m) [A]] th):
end for
end for
Return upper and lower hound intermediate data ai, my m., 42.
Flexible Time Steps: We note that for each SNN layer, the parameters shared among different time steps. Thus, we can set arbitrary time steps 7 for the robust training.
S-IBP: During the robust training, the inputs first pass the S-IBP Algorithm At the beginning, the input is bounded according to the input type. Then, the intermediate data are bounded along the forward direction. Finally, the upper and lower bounds of all intermediate data are stored which will be used during the S-CROWN phase.
S-CROWN:
Note that the goal of S-CROWN is to formulate the output bounds of model as linear equations of input's upper and lower bounds. The boundary of S-CROWN is computed from the backward direction, and the details are shown in Algorithm 2 Finally, we can follow the robust training method in 3 and use the lower bound of S-CROWN to train the SNN model.
Algorithm 2 S-CROWN
Input:
aa" 2/, m.. mi. o7 si from S-IBP; robust training time steps T';
Func:
WS-CROWN
build identit / matris: Asa[K = I/T';
K
initial Am, [R] 0 for all time steps
for 0 = 2 to do
Mire (li. [)
build m.[ [K] d1' f 61 < ++( m. [R] d1 61%;
Am, [K] + Asi [K]
"d1" Asilk] d1%
bias
[A] 61 + As. [k] b11:
/temporal update (Eq.
if 1 then
build mi-1 [k] v d2' + (1 - -- [k]) - 03' < Re-1 [k]:
build mi-11 [M] d2
8._ [k]) x d3 ---1 [k]:
Am.-1 [k]
(Am, d2" + Am, [R]
tmp.
(Am, d3 + Am! [M] d3')
Ast-1 [A] tmp.. bias + sumppppp)):
end If
Rspatial update:
As, [A 1] Amy [6] mi(l: 1]: bios + - Am, [8] + b/6
[]:
end for
end for
Return E, (As,(0) + (2 + 4f)/2
[As][o]] (ay #f3/2) + bins

---

4 Experiment
4.1 Experiment Setup
Dataset and Network Structure: In this work, we evaluate our robust training method on three datasets: MNIST [8. FMNIST [2 and NMNIST 244. MNIST and FMNIST are digital datasets and NMNIST is spike dataset. For each dataset, we use two network structures in experiments. The detailed setting for datasets and network structures are shown in Table | We the firing threshold th and decay factor a to 0.25 in Eq. I and B Our experiments are conducted by Pytorch 1.8. The hardware we used one Nvidia RTX3090 GPU and one AMD Ryzen CPU.
Table 1: Datasets and network structure
Input Type
Size
Time Step
PC Acc
CONV Ace
MNIST
digital
1#28*28
10
98.45%
99.09%
FMNIST
digital
1+28*28
10
87.58%
89.53%
NMNIST
spike
2434"34
10
98.30%
99.05%
RC
X-FC512-FC256-FCIII
CONV
X- X-CGKKSSS-CC228832-FFFCM..
In the original training, we adopt BPTT based training B3I. We train 80 epochs for each SNN model. The learning rate is set to 0.01 at the beginning, it decays to 0.001 at the 55th epoch. In robust training, we use the lower bound of S- S-CROWN as the loss function. During robust training, we set e to 0 at the beginning. It will increase linearly to the final e during the first 250 training epochs. In the last 50 training epochs, é is unchanged.
Original and Robust Training:
In this work we adopt the untargeted white-box gradient- based attack [9] in SNN. In our experiment, we select 300 examples for each dataset to apply adversarial attack. In order to bound the noise of adversarial examples, we involve additional constraints during the attack. Specifically, for digital inputs, we clip the adversarial example for each attack iteration to make the adversarial example stay in the boundary. For spike inputs, each attack iteration we limit the amount of changed elements to size(f) ( e/2 and size(ā) e/6 for FC and CONV networks to achieve the highest attack efficiency.
Adversarial Attack:
100
3.7
3.2
2.7
2.2
Robust Training Epoch
Epacning
(4)
(+)
Figure 4: Impact of robust training time steps 7" on (a) original accuracy and (b) S-CROWN loss. (CONV model; MNIST dataset; 0.12)
4.2 Robust Training in SNNs
Table Comparing untargeted white-box gradient- based attack between the original model and the model after robustness training.
(ONNV-ED/01)
arginn atlack
robust
sctwork JSack
czr
stack
Jtick
ataad exr
attakk
0.104
0.37%
19.3%
4.7%
16.6% (227%)
0.840
13,0%
20.75
13.6%
20.056 793
0.0007
3.7%
21.0%
3.75
18.09(-11.00
0.124
0.5%
29
20
13.0%
29.7%
24.96 795
0.009
3.7%
30.7%
12.39
49.7%
38.75 (1.099
18.391-300491
0.130
0.3%
18.7%
377%
8,351 10.455
U140
10.%
19.75
18.055
12.05(00.95)
0100N
1.35
20.05
4.36
7,95K 15 (3%)
CONY
01110
0.140
11.9%
0..9%
40
2010%
4.0%
5.9%
10.7994-10155
9.0091-20.55) 20,053
11880
8.060
10.3%
10.05
25.3%
17.3%
17.05.
23.098/5.399)
80017
0.0000
1.3%
1.3%
33.7%
4.5%
4.9%
11.091-223 756)
(1
(4.9901180065
Selection of Robust Training Time Steps T': We do not follow the original time steps during
the robust training. Because the spatial computations in SNNs share the same parameters between different time steps, we can use arbitrary robust training time steps T". In Figure 4 we analyze the impact of 7" on original accuracy and S-CROWN loss. Note that we would like to keep higher original accuracy but reduce the S-CROWN loss after the robust training. From the result, we can find that I" = 3 gives the optimal solution. It implics that too small I" cannot capture the temporal
8

---

16)
100
96
COOWWN LOG0
Robust (ND
96
90
94
CRRV 6.1
COMP
83
0.00
RAbLST Training Epoch
Robust Training 5ooch
Robust Training
(b)
COWFERTES
20W033005 640
00MA3300
9.25
15000
8
20W/2
E:15
TEOAL
RoDusE Training Epocn
Robust Traning Epoch
Robust Traiiing Bpoch
CONNADES
FC4.1
2.25
26
225
Nobust
8
COVV-4005
55
FC2 200
55
Robuat Traning tpech
Hobus
Robust tpach
Figure 5: Original accuracy, S-CROWN loss and S-CROWN robust error during robust training under (a) MNIST; (b) FMNIST and (C) NMNIST datasets with different network structures and
dynamics of SNNs and a larger T' may cause the boundary functions in S-CROWN to become too loose. Also we find that the robust training time for each epoch is 2.6x, 7.7x, and 12.9x with respect to the original plain training when we set 7" to 1, 3, and 5. Thus, the selection of 7" also affects the robust training efficiency. In the rest of our evaluations, we set T" 3 for all robust training.
The analysis of robust training on different datasets with various and network structures is shown in Figure [ From the result, have the following observations: 1. During the robust training, it is more stable for a fully connected network(yellow curves are smoother than blue curves) because of the simpler network structure. 2. With larger c, the original accuracy is dropped more after the robust training. Also, it is more difficult to achieve a smaller robust error with larger 3. In the MNIST dataset, when the network structure is CONV, the robust training is more stable as e increases. The potential reason is that when e is small, the flipped regions in spike inputs are more diverse. 4. In FMNIST and NMNIST datasets, when the network structure is CONV, the robust training is much more fluctuating. For FMNIST dataset, the unstable may be caused by the more complicated input data (cloths) and the lower convergence in original training (final accuracy is 89.53%). For NMNIST dataset, the unstable may come from the larger input data size. A larger input size indicates the potential input regions that can be attacked become larger. By considering the original accuracy and S-CROWN result after the robust training, we select the e as shown in Table We use FC-6 and CONV-E to represent the noise boundary we selected for different network structures during robust training.
Robust Training on Variaous Datasets:
4.3 Network Robustness Evaluation
We use untargeted white-box adversarial attack to compare the robustness between the original model and the model after robust training. The robustness comparison is shown in Table 2 We use the original error rate to reflect the accuracy of model on the 300 test data. The original error may have variance during the evaluation for the digital image dataset because of the input sampling mechanism. In our experiment, we select the attack to achieve approximately 20%. 30%, 40%, and 50% attack error rate on the original model. From the result, we can find that after the robust training, the models are harder to be attacked with adversarial example in all cases. We also find that the model after robust training is more secure when the attack is larger, even though the for robust training is much smaller. The potential reason that the binary behavior of the spike events causes the boundary propagation to diverge quickly, which makes the final boundary cover more noisy inputs. Finally, we find that model robustness can be improved more when the network structure is CONV, since more parameters can be adjusted under the CONV model. From the result, we find that the CONV model MNIST achieves the highest robustness improvement, i.e., the attack error rate reduces 37.7 7% with 3. 7% original accuracy loss when the attack equals 190

---

Discussion & Conclusion
In this work, we aim to design an efficient robust training method for SNNs based on the certified methods. Specifically, we design S-IBP and S-CROWN to tackle the distinct non-linear neuron behaviors in SNNs. Also, we formulate the input boundary for different input types. Based on our results, we can achieve most 37.7% attack error reduction with 3.7% original accuracy loss, which demonstrates the efficiency of our proposed method.
Border Impact
SNN is class of neural networks that inherit the biological model of the neuron dynamics in the brain. As the accuracy of SNNs becomes higher, the security issue attracts more attention. In this work, design S-IBP and S-CROWN that apply certification-basse -based robust training to improve the robustness of an SNN model. Our work can help to improve the misclassification rate of an SNN in real-world deployments. We believe our work can provide a positive impact on the performance of the SNN models. Also, we encourage researchers to seek other techniques and factors that can potentially affect the performance of an SNN model.
References
[1] Akopyan, F. Sawada, J.. Cassidy, A., Alvarez-Icaza, R., Arthur, J., Merolla, P., Imam, N.. Nakamura, Y., Datta, P., Nam, G.-J., et al. Truenorth: Design and tool flow of a 65 mw I million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems, 34(10):537-1555, 2015.
[2] Athalye, A., Carlini, N., and Wagner, D. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. International conference on machine learning, pp. 274 283. PMLR, 2018.
[3] Balunović, M. and Vechev, M. Adversarial training and provable defenses: Bridging the gap. In 8th International Conference on Learning Representations (ICLR 2020)vvirtual). International Conference on Learning Representations, 2020.
[4] Bhattacharjee. A., Kim, Y., Moitra, A., and Panda, P. Examining the robustness of spiking neural networks non-ideal memristive crossbars. arXiv preprint 2tt.::22000099999 2022.
[5] Büchel, J.. Faber, F., and Muir, D. R. Network insensitivity to parameter noise via adversarial regularization. arXiv preprint arXiv:2106.00 055009 2021.
[6] Büchel, J., Lenz, G., Hu, Y., Sheik, S., and Sorbaro, M. Adversarial attacks spiking convolutional networks for event-based vision. arXiv preprint arXiv:2110.02929 2021.
[7] Carlini, N. and Wagner, D. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
[8] Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al. Loihi: A neuromorphic manycore processor with on-chip learning. leee Micro, 38(1):82-99, 2018
[9] Deng, L., Wu, Y., Hu, X., Liang, L., Ding, Y., Li, G., Zhao, G., Li, P., and Xie, Rethinking the performance comparison between snns and anns. Neural Networks, 121:294-307, 2020.
[10] El-Allami, R., Marchisio, A., Shafique, M., and Alouani, Securing deep spiking neural networks against adversarial attacks through inherent structural parameters. In 202/ Design, Automation & Test in Europe Conference Exhibition (DATE), pp. 774-779. IEEE, 2021.
[11] Fan, J. and Li, W. Adversarial training and provable robustness: tale of two objectives. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7367-7376, 2021.
[12] Gerstner, W., Kistler, W. M., Naud, R., and Paninski, L. Neuronal dynamics: From single neurons networks and models of cognition. Cambridge University Press, 2014.
10

---

[13] Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R.. Mann, T., and Kohli, P. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprinl arXiv: 12715, 2018.
[14] Kim, Y. and Panda, P. Revisiting batch normalization for training low-latency deep spiking neural networks from scratch. Frontiers in neuroscience, pp. 1638, 2020.
[15] Kim, Y. and Panda, P. Visual explanations from spiking neural networks using inter-spike intervals. Scientific reports, 11(1):1-14, 2021.
[16] Kim, Y., Park, H., Moitra, A., Bhattacharjee, A., Venkatesha, Y., and Panda, P. Rate coding or direct coding: Which one is better for accurate, robust, and energy-efficctee spiking neural networks? In ICASSP 2022-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 71-75. IEEE, 2022.
[17] Kundu, S., Pedram, M., and Becrel, P. A. Hire-snn: Harnessing the inherent robustness of energy- efficient deep spiking neural networks by training with crafted input noise. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5209-5218, 2021.
[18) LeCun, Y., Bottou, I.., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE,
1998.
[19] Liang, L., Hu, X., Deng, L., Wu, Y., Li, G., Ding, Y., Li, P., and Xie. Y. Exploring adversarial attack in spiking neural networks with pole--comatibbe gradient. IEEE Transactions on Neural Networks and Learning Systems, 2021.
[20] Liang, L., Qu, Z., Chen, Z. Tu, F., Wu, Y., Deng, L., Li, G., Li, and Xie, Y. H2learn: High- efficiency learning accelerator for high-accuracy spiking neural networks. IEEE Transactions on Computer-Aided Design of Integruted Circuits and Systems, 2021.
[21] Marchisio, A., Pira, G., Martina, M., Masera, G., and Shafique, M. Dvs-attacks: Adversarial attacks on dynamic vision sensors for spiking neural networks. In 2021 International Joint
Conference on Neural Networks (IJCNV), pp. 1-9. IEEE, 2021.
[22] Mirman, M., Gehr, T., and Vechev, M. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning, pp. 3575-3583, 2018.
[23] Narayanan, S., Taht, K., Balasubramonian, R., Giacomin, E., and Gaillardon, P.-E. Spinalflow: an architecture and dataflow tailored for spiking neural networks. In 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), pp. 349-362. IEEE, 2020.
[24] Orchard, G., Jayawant, A., Cohen, G. K., and Thakor, N. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in neuroscience, 9:437, 2015.
(25) Patrick, L., Posch, C., and Delbruck, T. A 128x 128 120 db 15p s latency asynchronous temporal contrast vision sensor. IEEE journal of solid-state circuits, 43:566-576, 2008.
[26] Sharmin, S., Panda, P., Sarwar, S. S., Lee, C., Ponghiran, W., and Roy, K. comprehensive analysis on adversarial robustness of spiking neural networks. In 2019 International Joint
Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2019.
[27] Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D.,, Goodfellow, I., and Fergus, R. Intriguing properties of neural networks. arXiv preprint arXiv: /tfi::/99/9.9 2013.
[28] Tsuzuku, Y., Sato, I., and Sugiyama, M. Lipschitz-margin training: Scalable certification of perturbation invariance for deep neural networks. Advances in neural information processing systems, 31, 2018.
[29] Wang. S., Zhang, H., Xu, K., Lin, X., Jana, S., Hsich, C.-J., and Kolter, J. Z. Beta-crown: Efficient bound propagation with per-neuron split constraints for neural network robustness verification. Advances in Neural Information Processing Systems, (1-29999-29229 2021.
[30] Wong, F. and Kolter, J. 7.. Provable defenses against adversarial examples via the convex outer adversarial polytope. In International Conference on Machine Learning, 2018.
11

---

[31] Wu, Y., Deng. L., Li, G., Zhu, J., and Shi. L. Spatio-temporal backpropagation for training nigh-performance spiking neural networks. Frontiers in neuroscience, 12, 2018.
[32] Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017.
[33] Xu, K., Shi, Z., Zhang, H., Wang, Y., Chang, K.-W., Huang, M., Kailkhura, B., Lin, X., and Hsieh, C.-.. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33, 2020.
[34] Xu, K., Zhang, G., Liu, S., Fan, Q. Sun, M., Chen, H., Chen, P.-Y., Wang, Y., and Lin, X. Adversarial t-shirt! evading person detectors in a physical world. In European Conference on Computer Vision, pp. 665-681. Springer, 2020.
[35] Xu, K., Zhang, H., Wang. S., Wang, Y., Jana, S., Lin, X., and Hsieh, C.-J. Fast and complete: Enabling complete neural network verification with rapid and massively parallel incomplete verifiers. In International Conference on Learning Representations, 2021. URL. https
//openreview.net nett.forrumm.ddd///A........
[36] Yuan, C. and Moghaddam, M. Attribute-awar generative design with generative adversarial networks. IEEE Access, 8:190710-1907211 2020.
[37] Zhang, H., Weng, T.-W., Chen, P.-.,, Hsieh, C.-J,, and Daniel, L. Efficient neural network robustness certification with general activation functions. arXiv preprint 18.00066, 2018.
[38] Zhang, H., Chen, H., Xiao, C., Gowal, S., Stanforth, R., Li, B., Boning, D., and Hsieh, C.-J. Towards stable and efficient training of verifiably robust neural networks. In International Conference on Learning Representations, 2020. URL httptttt//////eeeeee wtwpreeewwiieeeeeffreeeeee net/forum? id=SkxuklrFwB
[39] Zhao, P., Xu, K., Liu, S., Wang, Y., and Lin, X. Admm attack: an enhanced adversarial attack for deep neural networks with undetectable distortions. In Proceedings of the 24th Asia and South Pacific Design Automation Conference, pp. 499-505, 2019.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "r5jNMqCDEKd", "reviewer": "Reviewer_d88H", "summary": "This paper considers that spiking neural networks are not suitable for traditional adversarial robustness analysis methods, and proposes linear relaxations for the membrane potential and spikes of SNNs. These relaxations can be used to provide robust training for SNNs.", "questions": "Is this the first analysis of the robustness of spiking neural networks? [1] is also a paper that copes with the robustness of SNNs. \n\n[1] HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training With Crafted Input Noise\n\n", "limitations": "yes", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\nThis paper is well written.\n\nThis paper gives a linear relaxation scheme for SNNs.\n\nThis solution takes into account not only temporal updates but also spatial updates.\n\n\n\nCons:\n\nI'm not sure if the linear relaxation is too loose. I think an example of MNIST can be used to illustrate the gap.\n\nThe upper and lower bounding strategies used for spike inputs in Sec 3.2 are probabilistic, not linear. Can the introduction of probability be consistent with linear relaxation? If not, i.e. x^u=x^i=x, how will the model work?\n\nPrevious work has shown the robustness of SNN is affected by coding (Poisson coding and directly coding) [1]. What's the way of coding in your paper? \n\nThere are some typos that need to be fixed. And the citation is not standardized. For example: \"leaky integrated-and-fire (LIF) Gerstner et al. (2014)”（Line 56）should be “leaky integrated-and-fire (LIF) (Gerstner et al. 2014)”.\n\n[1] HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training With Crafted Input Noise\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "YuFyGN_3Xnz", "reviewer": "Reviewer_2YgY", "summary": "The paper proposes a methodology to tackle adversarial robustness in SNNs.", "questions": "Please see above comments.", "limitations": "Please see weakness section.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ The authors show that their method is able to resist attacks of different types on small scale datasets.\n- The authors evaluation is pretty limited. In [1, 3], the authors show that BNTT trained SNN models are inherently more robust. Can the authors comment on how their methodology is different from [1,3]. Further, the authors started their paper discussion with SNNs being advatangeous on hardware, so it makes more sense to develop hardware aware robustness. But, the author's method is algorithm-based. Can the authors comment if their robustness will transfer to hadware as is or any modification will be required? In [4], the authors show that adversarial robustness on hardware become pretty low, so they come up with a normalization technique to resist attacks. In [2], the authors show that the type of coding technique plays a role in determining adversarial robustness. I am not sure if teh author's methodology can tarnsfer across differnet coding techniques.\n\n[1] Revisiting batch normalization for training low-latency deep spiking neural networks from scratch Y Kim, P Panda Frontiers in neuroscience, 1638\n\n[2] Rate Coding Or Direct Coding: Which One Is Better For Accurate, Robust, And Energy-Efficient Spiking Neural Networks? Y Kim, H Park, A Moitra, A Bhattacharjee, Y Venkatesha, P Panda ICASSP 2022-2022\n\n[3] Visual explanations from spiking neural networks using interspike intervals Y Kim, P Panda Scientific Reports 11, Article number: 19037 (2021)\n\n[4] Bhattacharjee, Abhiroop, et al. \"Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars.\" arXiv preprint arXiv:2206.09599 (2022).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "XqIcx2EGR0", "reviewer": "Reviewer_6QhU", "summary": "In this paper, a robust training method for SNNs is proposed. It is based on S-IBP and S-CROWN algorithms. The results on 3 different datasets show attack error reduction with some original accuracy loss.", "questions": "1.\tIn Section 1, typo, rewrite the sentence: “Except for the biological character of SNN models, the event-driven based information propagation mechanism makes SNN can be deployed with limited computation resources.\n2.\tIs the methodology for generating adversarial attacks in SNN described in Section 2.2 novel, or is it inspired by related work? If the latter, the references should be included in the paragraph.\n3.\tPlease describe in more detail which are the assumptions made to derive the equations in Section 3.1.\n4.\tIn Section 3.2, typo, correct the sentence: “Also, the l0-norm boundary can be interpreted as the probability of an element is unstable”\n5.\tSection 3.3 only describes the S-IBP and S-CROWN algorithms independently. More details are required to describe the overall structure of the proposed robust training method.\n6.\tWhat is the tool flow used for conducting the experiments? Which packages/libraries are used?\n7.\tIn Section 4.2: “By considering the original accuracy and S-CROWN result after the robust training, we select the ϵ as shown in Table 2.” It is not clear which criteria are used to select ϵ.\n8.\tHow does this work compare with the following article?\nKundu, Souvik et al. “HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise.” 2021 IEEE/CVF International Conference on Computer Vision (ICCV) (2021): 5189-5198.\n", "limitations": "The limitations and societal impact of this work have not been discussed. However, there is no reason to penalize the submission for this, since this work provides only positive impacts.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n1.\tThe contributions of this paper are clear and original.\n2.\tThe achieved results are significant and relevant to advancing the state-of-the-art.\n\nWeaknesses:\n\n1.\tThere are several typos and semantically incorrect sentences throughout the text. It is recommended to conduct thorough proofreading.\n2.\tThe clarity of some key sections can be improved. See comments below.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept