Toward Robust Spiking Neural Network Against Adversarial Perturbation
Abstract
As spiking neural networks (SNNs) are deployed increasingly in real-world efﬁciency critical applications, the security concerns in SNNs attract more attention.
Currently, researchers have already demonstrated an SNN can be attacked with adversarial examples.
How to build a robust SNN becomes an urgent issue.
Recently, many studies apply certiﬁed training in artiﬁcial neural networks (ANNs), which can improve the robustness of an NN model promisely.
However, existing certiﬁcations cannot transfer to SNNs directly because of the distinct neuron behavior and input formats for SNNs.
In this work, we ﬁrst design S-IBP and S-CROWN that tackle the non-linear functions in SNNs’ neuron modeling.
Then, we formalize the boundaries for both digital and spike inputs.
Finally, we demonstrate the efﬁciency of our proposed robust training method in different datasets and model architectures.
Based on our experiment, we can achieve a maximum 37 .
7% attack error reduction with 3 . 7% original accuracy loss.
To the best of our knowledge, this is the ﬁrst analysis on robust training of SNNs.
Introduction
Spiking neural networks (SNNs) are a class of models that have the potential ability to simulate the behavior of neuron circuits in the brain.
Among the rich biological characters of SNNs, the eventdriven information propagation mechanism makes SNNs easy to deploy with limited computation resources.
Many neuromorphic chips are designed to achieve low-power SNN inference, such as
TrueNorth [1], Loihi[8], SpinalFlow [23] and H2Learn [20].
With more attention to the study of SNNs, the security issues also raise concerns in the community.
Adversarial attack [27, 7, 2, 34] is one of the most intuitive ways to evaluate the robustness of a model.
In adversarial attacks, the attacker generates adversarial examples to fool a model that has wrong prediction.
Currently, SNNs are demonstrated can be attacked through adversarial examples [26, 19, 6, 21].
It is urgent to explore an efﬁcient way to improve the robustness of SNN models.
Previously, researchers have investigated the impact of hyperparameter selection [10] and input ﬁltering [21] on the adversarial attack in SNNs.
However, these methods do not directly promote the classiﬁcation behavior of a given SNN model.
[5] proposes a robust training method that relies on the adversarial attack in parameter space.
Unlike SNNs, how to improve the robustness of an artiﬁcial neural network (ANN) has been well studied.
Recently, trianing a neural network model with certiﬁed defense methods [30, 22, 38, 33] show remarkable guarantee to improve the model’s robustness.
CROWN-IBP [38] is one of the most promising certiﬁed training methods with 36th Conference on Neural Information Processing Systems (NeurIPS 2022).
polynomial computational cost compared to natural training.
The CROWN-IBP method computes the output boundary for a given bounded input.
However, the current CROWN-IBP method cannot be applied directly to SNNs.
Firstly, the neuron dynamic in SNNs is more complicated.
Hence, new boundary functions should be deﬁned.
Secondly, SNNs accept both spike and digital inputs, which requires additional boundary generalization for different input types.
Enlightened by certiﬁcation training in ANNs, we designed an end-to-end robust training method to improve the robustness of an SNN model against adversarial attacks.
Speciﬁcally, our contributions can be summarized as follows:
• We design S-IBP and S-CROWN to tackle the non-linear ﬁre function and temporal update in SNNs which ﬁrstly introduce SNNs to the linear relaxation based veriﬁcation family.
• We formalized ` 0 -norm and ` 1 -norm boundaries for spike and digital inputs, respectively.
• Our proposed methods are evaluated on MNIST [18], FMNIST [32] and NMNIST [24] datasets 1 .
The experimental results show that we can achieve a maximum 37 .
7% attack error reduction with 3 . 7% original accuracy loss.
Preliminary and Related Work 2.1 Spiking Neural Netowks (SNNs)
Neuron Modeling : Usually, SNNs are designed to simulate the neuron behavior in the brain.
In this work, we adopt the well-studied leaky integrated-and-ﬁre (LIF) [12] for the neuron modeling as Figure 1(a).
The state of a neuron at each time step is determined by its membrane potential m and its spike status s .
Speciﬁcally, a neuron will fire a spike to the post-synaptic neurons once its membrane potential is greater than a threshold th and its membrane potential will be reset to 0 at the same time.
An example of a neuron dynamic is shown in Figure 1(b) which simulates a neuron’s behavior in 5 time steps.
The spike status of a neuron can be formulated as
[Equation]
, x 0 , otherwise .
We use m t [ k ] and s t [ k ] to represent neurons’ membrane potentials and spike status at t -th time step and k -th layer.
fire ( · ) is the Heaviside step function.
The membrane potential of a neuron is composed of spatial and temporal update that follows
[Equation]
| [Equation] The spatial update is a Convolution or Fully Connected or Pooling operation between the weight and the spike events in previous layer.
The temporal update is determined by the neuron’s membrane potential and spike status in the previous time step.
Here, ↵ is a scaling factor to control the decay.
[Figure]
Network Structure : SNNs are built with an input layer, hidden layers, and an output layer as
Figure1(c).
In SNNs, the input layer receives spike inputs.
The hidden layers are CONV/FC/POOL that correspond to the spatial update in the neuron modeling.
Compared to ANNs, SNNs involve an additional temporal axis that passes the neurons’ status along with time steps.
The ﬁnal recognition of an SNN is determined by the spike events of the output layer.
In this work, we adopt the rate coding, i.e., the neuron that ﬁres the most spikes indicates the classiﬁcation result.
Input Format : In this work, we focus on image recognition tasks.
The input of an SNN can be spike events captured by DVS devices [25].
Also, SNNs can take a digital image as input with an additional sampling.
In this work, we adopt Bernoulli sampling [9] for digital inputs that follows
[Equation]
We use ˙ x t to represent the spike input at time step t .
ˆ x is the digital input after normalization to [0 , 1] .
The example of spike and digital inputs are shown in Figure 1 (d).
Training Method : Recently, supervised learning via backpropagation through time (BPTT) has been adopted in SNN training [31].
The BPTT-based training method can achieve a higher accuracy and scalability compared to unsupervised learning algorithms.
Also, the SNNs trained with BPTT require fewer time steps, which is more hardware friendly in the real deployment.
Thus, in this work, we focus on the SNNs trained with the BPTT-based learning method.
2.2 Adversarial Attack in SNN
The generation of adversarial examples is one of the most powerful ways of affecting a model’s robustness [27, 7, 39, 36].
In this work, we use adversarial attacks as an assessment tool to evaluate the robustness of an SNN model.
Speciﬁcally, in this work we adopt a gradient-based attack method [19] for SNNs.
Untargeted White-box Gradient-based Attack
: In adversarial attacks, the attacker adds imperceptible noise on inputs to fool the model to predict wrong results.
In untargeted attack, the model predicts the adversarial example as any other classes except the ground-truth, which can be formulated as argmin
[Equation]
Here, f ( · ) is the prediction result, x is the original input, and  is the noise added on the input.
In this work, we adopt white-box attack, which means the attacker knows all of the model information.
The gradient-based attacks are the most efﬁcient ways to generate adversarial examples.
In SNNs, the gradient-based attack for the spike and digital inputs can be formulated as
[Equation]
We use ˙ x 0 and ˆ x 0 to represent the spike and digital adversarial examples, i.e., x 0 = x +  .
L and ✓ are the loss function and parameters of the model, respectively.
The adversarial example is constructed by adding the gradient of inputs with the original label ( y org ) in loss function.
F is a ﬁlter function that samples, clips, and generates spike compatible noise.
During the attack, attackers can compute the adversarial examples iteratively.
We use n to denote the attack iteration.
2.3 Certiﬁed Training
Recently, the certiﬁed training [13, 38, 33] has been demonstrated to improve the guaranteed robustness of a neural network.
In this work, we leverage the CROWN-IBP method [38], one of the state-of-the-art certiﬁed training that can achieve tighter bounds in accepctable training cost.
Considering a digital input data bounded with ` 1 -norm, the goal of CROWN-IBP is to identify whether arbitrary input data within the boundary can fool the model.
The CROWN-IBP contains two parts of bounding methods: IBP [13] and CROWN [37].
IBP :
In IBP processing, the lower and upper bounds of each layer’s feature map are computed along the forward propagation.
During the bound computation, the linear operation can be easily bounded once we know the maximum and minimum values of input.
However, the upper and lower bounds after the non-linear operations need to be dedicated to analysis.
Once we acquire the bounds of output we can evaluate the robustness of the model for the given input purbation set.
CROWN : Unlike IBP, CROWN bounds the model in a backward propagation manner recursively.
The goal of CROWN is to formulate the output bounds as a linear equation of input.
In order to achieve this goal, every operation in an NN model should be bounded by two linear equations.
Although CROWN can achieve very tight bounds, the computational cost is remarkably higher than
IBP.
So CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and CROWN in a backward bounding pass can efﬁciently and consistently outperforms IBP baselines on training veriﬁably robust neural networks.
Integrate with Other Approaches :
One of the promising characteristics of certiﬁed training is that it can be integrated with other algorithms.
For example, the combination of certiﬁed training and adversarial training can further improve the robustness of a model [3, 11].
For the operations which can improve the performance of a model and only involve linear operations, the certiﬁed training algorithm can be directly applied without causing additional relaxation loss [28, 29].
Currently, the adversarial training [17] and batch normalization [14, 15, 4] have demonstrated their effectiveness in improving the robustness of an SNN model, which have the potential to be integrated into our proposed design in the future.
Methodology 3.1 S-IBP & S-CROWN
As described in Section 2.3, the core mission of certiﬁcation training is to ﬁnd the bounds of IBP and
CROWN for each function.
The information propagation in SNNs includes fire , temporal update and spatial update as shown in Figure 1(a).
In this subsection, we detail the upper bound and lower bound of S-IBP and S-CROWN for each function.
During the bound formulation of a function, we assume the S-IBP bounds for function inputs have been acquired.
Fire : The fire function describes the relationship between the membrane potential m t and spike s t of a neuron as Equation 1 and 2, i.e. once the neuron’s membrane potential is greater than a threshold th , the neuron will ﬁre a spike.
Assume that we have already acquired the S-IBP upper bound m ut and lower bound m lt for membrane potential, the S-IBP bounds for spike can be calculated with
[Equation]
During S-CROWN, we need to ﬁnd two linear equations to bound the fire function.
When the
S-IBP upper bound of membrane potential m ut is smaller than the threshold th , we can conclude that the neuron must not ﬁre.
Also, when the S-IBP lower bound m lt is greater than the threshold, we can make sure the neuron must ﬁre.
Thus, we mainly need to consider the unstable case, i.e., m lt < th  m ut .
In order to achieve the lowest bound relaxation, we design two boundary systems as Figure 2(a) and (b).
We use the red line to represent the fire function.
The blue and yellow lines represent the boundary functions.
When the S-IBP lower bound of membrane potential m lt is much smaller than th , we set the S-CROWN lower bound to s t = 0 and the S-CROWN upper bound is a line that crosses ( m l t , 0) and ( th, 1) .
In contrast, when m u t is much larger than th , we set the
S-CROWN upper bound to s t = 1 and the S-CROWN lower bound is a line that passes ( th, 0) and ( m ut , 1) .
The bound can be potentially optimized by solving the decent gradient [35].
Overall, the
S-CROWN boundary for the fire function under different cases can be summarized as
[Equation]
[Equation]
Temporal Update :
According to Equation 3, the temporal update of membrane potential can be formalized as ↵ m t (1  s t ) .
Since ↵ is a constant factor, in this subsection we do not include it in the boundary analysis.
Assume z t = m t (1  s t ) , and the S-IBP upper and lower bounds for membrane potential at time step t has been acquired.
The relation between m t and z t can be described through
Figure 2(c).
Based on the LIF model, the membrane potential will be reset to 0 once its value acrosses
# (0,! !" ,0) (0,&ℎ,0) (1,! !
# ) (1,&ℎ,&ℎ) ! )
" − $ ) % ) = ! ) (" − $ ) ) ! ) ! ) ! ) &ℎ &ℎ &ℎ &ℎ 1! !
# $ ) $ ) (a) (b) (c) (d) % ) = ! ) (" − $ ) ) the pre-deﬁned threshold th .
Thus, the S-IBP bounds for z t can be computed with
[Equation]
When m u t < th or m l t  th , there is no boundary relaxation in S-IBP.
We only need to consider the boundary when m lt < th  m ut (unstable case), where z t 2 [ min (0 , m lt ) , th ) for unstable case.
During S-CROWN, z t can be also bounded without relaxation when m ut < th and m lt  th .
The
S-CROWN upper bound (blue plane) and lower bound (yellow plane) for z t when m lt < th  m ut are shown in Figure 2(d).
Here, we use an additional (1  s t ) axis to help us build the boundary.
Note that the membrane potential is related to the spike status.
Once s t = 1 ! (1  s t ) = 0 , m t must be greater than th and z t = 0 .
When s t = [-
-]0 !
(1  s t ) = 1 , m t must be smaller than th and z t = m t .
Thus, we need to ﬁnd two planes to bound these two functions (red lines in Figure 2(d)).
In summary, the S-CROWN boundary for the temporal update can be formulated as
[Equation]
Spatial Update
The spatial update in SNNs is shown in Equation 3.
Similar to ANNs, the spatial update in SNNs is composed of CONV/FC/POOL (in this work we focus on the CONV and FC).
In SNNs CONV/FC takes spike events and weight as input.
Since the spike events are in binary format, the S-IBP of spatial update can be implemented with
[Equation]
We use sp t to represent the result of the spatial update.
⌦ denotes CONV/FC operation.
In Equation 11, s lt [ k ] = 1 represents those pre-synaptic neurons in layer k who are must ﬁre.
The stable ﬁred pre-synaptic neurons contribute the same for both S-IBP upper and lower bound of sp t .
The unstabled pre-synaptic neurons can be represented with ( s lt [ k ] =
\ s ut [ k ] = 1 ), whose upper and lower bound for spike status are 1 and 0.
These unstable pre-synaptic neurons will perform CONV/FC with the positive and negative weights to affect the S-IBP upper and lower bound of sp t .
Since the spatial update is a linear operation, the S-CRWON for spatial update does not have relaxation.
We do not need to design boundary functions to bound the spatial update in SNNs.
3.2 Input Boundary Formalization
In SNNs, the input layer of an SNN model only accepts binary spike.
The special data format for the input layer leads to different boundary formalizations for spike and digital images.
Spike Input : An example of spike input is shown in Figure 1(d).
All elements in a spike data are in binary format.
For each element ˙ x t [ i ] in a spike input, the boundary of that element can be either stable cases: ˙ x ut
[ i ] = ˙ x lt [ i ] = 0 ; ˙ x ut [ i ] = ˙
x lt [ i ] = 1 or unstable case: ˙ x ut
[ i ] = 1 , ˙ x lt [ i ] = 0 .
For a spike input with uncertainty noise, we can only control how many data points in the spike input are unstable.
Thus, the boundary for a spike input can be formulated with ` 0 -norm.
Speciﬁcally, we can pick size ( ˙ x ) ⇥ ✏ elements from a spike input and set them as unstable points.
Also, the ` 0 -norm boundary can be interpreted as the probability of an unstable element, which can be formulated as
[Equation]
Here, ˙ x 0 is an arbitrary adversarial example that has at most size ( ˙ x ) ⇥ ✏ data points different from ˙ x .
Note that we can only certify the robustness of a spike input after we have picked the unstable data points.
Under our robustness formulation, we cannot guarantee the robustness of a spike input under a given ` 0 -norm.
The reason is that the search space for ` 0 -norm cannot be bounded.
[Figure]
̇" !"
= $" " −&'()
[Figure]
̇" !
[Figure]
=
[Figure]
Digital Input :
Digital inputs need an additional sampling before feeding the data to the input layer as Equation 4.
Note that we have deﬁned the ` 0 -norm boundary for a spike input.
We need to further explore how to deﬁne a boundary for a digital input when its sampled data is bounded with ` 0 -norm.
Based on our analysis, we ﬁnd that the corresponding digital input boundary can be formalized with ` 1 -norm as
Figure 3.
We use rand t to represent a rand mask to achieve Bernoulli sampling for each time step:
[Equation]
After we bound the digital input with ` 1 -norm, the upper bound of digital image becomes ˆ x u
= ˆ x + ✏ / 2 and the lower bound is ˆ x l
= ˆ x  ✏ / 2 .
For all elements in ˆ x , the difference between the upper bound and the lower bound is ✏ , i.e., ˆ x u [ i ]  ˆ x l [ i ] = ✏ .
For each time step, ˆ x u and ˆ x l use the identical random map rand t to sample the corresponding spike inputs ˙ x ut and ˙ x lt .
Based on the sampling processing, the probability of an element in input layer is unstable can be formulated as
[Equation]
From the static perspective, the ` 1 -norm boundary for digital input is equivalent to the ` 0 -norm boundary for spike input.
For digital inputs, although we apply ` 1 -norm on input boundary, we cannot guarantee the robustness of the input under such boundary.
Since for an digital input, the possible sampled results are equivalent to the entire space (each element in the spike input can be 1 or 0), which cannot be bounded.
For both spike and digital inputs, we need to carefully select ✏ for robust training.
An extremely small ✏ will cause the upper and lower bound identical which does not consistent with linear relaxation.
A large ✏ may cause the following robust training to diverge.
In this work, ✏ is selected based on the noise magnitude in adversarial attacks and some empirical experiments.
3.3 Robust Training Algorithm
In this subsection, we present the end-to-end robust training for an SNN.
For the overall ﬂow, our method will ﬁrst run S-IBP to get the boundary of intermediate data.
Then we apply the S-CROWN which needs the S-IBP bound.
Finally, we apply the certiﬁcation-based robust training.
[Table]
[Equation]
[Table] [Equation] [Table] Flexible Time Steps : We note that for each SNN layer, the parameters are shared among different time steps.
Thus, we can set arbitrary time steps T 0 for the robust training.
S-IBP : During the robust training, the inputs ﬁrst pass the S-IBP as Algorithm 1.
At the beginning, the input is bounded according to the input type.
Then, the intermediate data are bounded along the forward direction.
Finally, the upper and lower bounds of all intermediate data are stored which will be used during the S-CROWN phase.
S-CROWN : Note that the goal of S-CROWN is to formulate the output bounds of a model as linear equations of input’s upper and lower bounds.
The boundary of S-CROWN is computed from the backward direction, and the details are shown in Algorithm 2.
Finally, we can follow the robust training method in [38] and use the lower bound of S-CROWN to train the SNN model
[Equation]
Algorithm 2 S-CROWN
[Equation]
; end for end for Return
P t ⇣
As t [0] ⇤ (˙ x ut + ˙ x lt ) / 2  | As t [0] | ⇤ (˙ x ut
˙ x lt ) / 2 ⌘ + bias
Experiment
4.1 Experiment Setup
Dataset and Network Structure : In this work, we evaluate our robust training method on three datasets: MNIST [18], FMNIST [32] and NMNIST [24].
MNIST and FMNIST are digital datasets and NMNIST is spike dataset.
For each dataset, we use two network structures in experiments.
The detailed setting for datasets and network structures are shown in Table 1.
We set the ﬁring threshold th and decay factor ↵ to 0.25 in Eq.
(1) and (3).
Our experiments are conducted by Pytorch 1.8.
The hardware we used is one Nvidia RTX3090 GPU and one AMD Ryzen CPU.
[Table]
Original and Robust Training :
In the original training, we adopt BPTT based training [31].
We train 80 epochs for each SNN model.
The learning rate is set to 0.01 at the beginning, it decays to 0.001 at the 55 th epoch.
In robust training, we use the lower bound of S-CROWN as the loss function.
During robust training, we set ✏ to 0 at the beginning.
It will increase linearly to the ﬁnal ✏ during the ﬁrst 250 training epochs.
In the last 50 training epochs, ✏ is unchanged.
Adversarial Attack : In this work we adopt the untargeted white-box gradient-based attack [19] in
SNN.
In our experiment, we select 300 examples for each dataset to apply adversarial attack.
In order to bound the noise of adversarial examples, we involve additional constraints during the attack.
Speciﬁcally, for digital inputs, we clip the adversarial example for each attack iteration to make the adversarial example stay in the boundary.
For spike inputs, each attack iteration we limit the amount of changed elements to size (ˆ x ) ⇤ ✏ / 2 and size (ˆ x ) ⇤ ✏ / 6 for FC and CONV networks to achieve the highest attack efﬁciency [Figure] [Figure] [Figure]
4.2 Robust Training in SNNs
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Table] [Figure]
[Table]
Selection of Robust Training Time Steps T 0 : We do not follow the original time steps during the robust training.
Because the spatial computations in SNNs share the same parameters between different time steps, we can use arbitrary robust training time steps T 0 .
In Figure 4 we analyze the impact of T 0 on original accuracy and S-CROWN loss.
Note that we would like to keep a higher original accuracy but reduce the S-CROWN loss after the robust training.
From the result, we can ﬁnd that T 0 = 3 gives the optimal solution.
It implies that a too small T 0 cannot capture the temporal dynamics of SNNs and a larger T 0 may cause the boundary functions in S-CROWN to become too loose.
Also we ﬁnd that the robust training time for each epoch is 2 .
6 ⇥ , 7 . 7 ⇥ , and 12 . 9 ⇥ with respect to the original plain training when we set T 0 to 1, 3, and 5.
Thus, the selection of T 0 also affects the robust training efﬁciency.
In the rest of our evaluations, we set T 0 = 3 for all robust training.
Robust Training on Variaous Datasets : The analysis of robust training on different datasets with various ✏ and network structures is shown in Figure 5.
From the result, we have the following observations: 1.
During the robust training, it is more stable for a fully connected network(yellow curves are smoother than blue curves) because of the simpler network structure.
With a larger ✏ , the original accuracy is dropped more after the robust training.
Also, it is more difﬁcult to achieve a smaller robust error with larger ✏ . 3.
In the MNIST dataset, when the network structure is CONV, the robust training is more stable as ✏ increases.
The potential reason is that when ✏ is small, the ﬂipped regions in spike inputs are more diverse.
In FMNIST and NMNIST datasets, when the network structure is CONV, the robust training is much more ﬂuctuating.
For FMNIST dataset, the unstable may be caused by the more complicated input data (cloths) and the lower convergence in original training (ﬁnal accuracy is 89.53%).
For NMNIST dataset, the unstable may come from the larger input data size.
A larger input size indicates the potential input regions that can be attacked become larger.
By considering the original accuracy and S-CROWN result after the robust training, we select the ✏ as shown in Table 2.
We use FC- ✏ and CONV- ✏ to represent the noise boundary we selected for different network structures during robust training.
4.3 Network Robustness Evaluation
We use untargeted white-box adversarial attack to compare the robustness between the original model and the model after robust training.
The robustness comparison is shown in Table 2.
We use the original error rate to reﬂect the accuracy of a model on the 300 test data.
The original error may have variance during the evaluation for the digital image dataset because of the input sampling mechanism.
In our experiment, we select the attack ✏ to achieve approximately 20%, 30%, 40%, and 50% attack error rate on the original model.
From the result, we can ﬁnd that after the robust training, the models are harder to be attacked with adversarial example in all cases.
We also ﬁnd that the model after robust training is more secure when the attack ✏ is larger, even though the ✏ for robust training is much smaller.
The potential reason is that the binary behavior of the spike events causes the boundary propagation to diverge quickly, which makes the ﬁnal boundary cover more noisy inputs.
Finally, we ﬁnd that model robustness can be improved more when the network structure is CONV, since more parameters can be adjusted under the CONV model.
From the result, we ﬁnd that the CONV model in MNIST achieves the highest robustness improvement, i.e., the attack error rate reduces 37 .
7% with 3 .
7% original accuracy loss when the attack ✏ equals 0.190.
Discussion & Conclusion
In this work, we aim to design an efﬁcient robust training method for SNNs based on the certiﬁed methods.
Speciﬁcally, we design S-IBP and S-CROWN to tackle the distinct non-linear neuron behaviors in SNNs.
Also, we formulate the input boundary for different input types.
Based on our results, we can achieve at most 37 .
7% attack error reduction with 3 . 7% original accuracy loss, which demonstrates the efﬁciency of our proposed method.
Border Impact
SNN is a class of neural networks that inherit the biological model of the neuron dynamics in the brain.
As the accuracy of SNNs becomes higher, the security issue attracts more attention.
In this work, we design S-IBP and S-CROWN that apply certiﬁcation-based robust training to improve the robustness of an SNN model.
Our work can help to improve the misclassiﬁcation rate of an SNN in real-world deployments.
We believe our work can provide a positive impact on the performance of the SNN models.
Also, we encourage researchers to seek other techniques and factors that can potentially affect the performance of an SNN model