Adversarial Sample Detection via Channel Pruning
Abstract
Adversarial attacks are the main security issue of deep neural networks.
Detecting adversarial samples is an effective mechanism for defending against adversarial attacks.
Previous works on detecting adversarial samples show superior in accuracy but consume too much memory and computing resources.
In this paper, we propose an adversarial sample detection method based on pruned models.
We ﬁnd that pruned neural network models are sensitive to adversarial samples, i.e., the pruned models tend to output labels different from the original model when given adversarial samples.
Moreover, the channel pruned model has an extremely small model size and actual computational cost.
Experiments on CIFAR10 and SVHN show that the FLOPs and size of our generated model are only 24.46% and 4.86% of the original model.
It outperforms the SOTA multi-model based detection method (87.47% and 63.00%) by 5.29% and 30.92% on CIFAR10 and SVHN, respectively, with signiﬁcantly fewer models used.
1. Introduction
Though Deep Neural Networks (DNN) have achieved great success in various applications, e.g., computer vision (Pham et al., 2020), natural language processing (Brown et al., 2020), and speech recognition (Pan et al., 2020), the existence of adversarial samples undermines their use in safety critical areas and raises public concern.
The Machine Learning (ML) community has proposed many approaches to improve DNN robustness against adversarial samples, including data augmentation (Tian et al., 2018; Kurakin et al., 2016), adversarial training (Yu et al., 2018; Li et al., 2020b), and robust optimization (Deng et al., 2020).
These approaches can improve the robustness of the model to a cer- tain extent, but ask for additional data and training, which cost intensive resources, especially for those large models.
The other optional defense strategy is detecting adversarial samples.
The ML community has observed that adversarial samples are different from benign samples in multiple aspects, including data distribution (Li et al., 2020a;
Chen et al., 2020), decision boundary (Yin et al., 2019), and neuron activating path (Ma & Liu, 2019).
The model developer can distinguish adversarial samples by these characteristics and stop them from attacking the model.
The software engineering community proposes the concept of DNN testing that aims to detect bugs in the DNN model (Tian et al., 2018;
Ma et al., 2018; Wang et al., 2020).
Adversarial samples are a kind of bug hidden in the DNN model.
One of these testing methods for detecting adversarial samples is mutation testing (Wang et al., 2019).
It generates multiple models by randomly shufﬂing neuron weights or change the activation state of neurons.
They ﬁnd that the generated models are sensitive to adversarial samples, which means the outputs of generated models are different from the original model.
For an unknown sample, through the label changes of generated models, we can distinguish whether it is adversarial.
We argue that distinguishing adversarial samples using multiple models is a more reliable strategy for adversarial sample detection.
(Huang et al., 2020) shows that many defense methods, including improving model robustness or detecting adversarial samples, are vulnerable to certain attacks.
In extreme cases, such as the attacker grabs both model details and defense strategy, these methods can be even circumvented (Carlini & Wagner, 2017).
Most of the detection methods evaluated in (Carlini & Wagner, 2017) use indicators from a single model.
Since the indicator comes from the victim model and the perturbation updated in the attack also uses the output of the model or its gradient information, it is possible to design an adaptive attack strategy to circumvent these indicators (Carlini & Wagner, 2017) by adding constraints in the attack objective.
For the multiple-model method, the indicator comes from various models, it is hard to guarantee that the perturbation works on all the models.
However, mutation testing is barely practical because the detection requires running dozens of models with almost the same size as the original model.
In this paper, we ﬁnd that models with channels being randomly pruned are more
[Table]
[Figure]
sensitive to adversarial samples.
Moreover, pruning also reduces the model size, making the detection feasible in practice.
Through experiments on CIFAR10 and SVHN, we proved the effectiveness of our method.
It can detect 92.76% and 93.92% adversarial samples on CIFAR10 and SVHN, with 29.24 and 30.68 pruned models that are only 24.46% and 4.86% the size of the original models, respectively.
2. Related Works
As far as we know, Model Mutation Testing (MMT) (Wang et al., 2019) is the only approach to detect adversarial samples using multiple completely different models (not part of the original model).
The other multi-model detection methods use models intercepted from the original model (Wang et al., 2020).
The authors propose to build sub-models with the parameters and structure inherited from the original model and use them to detect adversarial samples.
The number of sub-models can be as many as the number of intermediate layers of the original model.
They argue that a normal sample should be predicted with increasing conﬁdence, which reﬂects on the output of sub-models.
Their method needs to retrain an output layer based on the inherited layers, but the inherited layers are frozen during training.
In MMT (Wang et al., 2019), the authors propose to generate mutated models through four operators, namely weights fuzzing, weights shufﬂing, neuron switch, and neuron activation inverse.
The operators will not cause a signiﬁcant decrease in accuracy, but the generated models are sensitive to adversarial samples.
Their method usually takes dozens of models to complete one detection.
3. Method
We detect adversarial samples by the outputs of pruned models.
As shown in Figure 1, a benign sample with the label burger is still burger in most outputs of additional models, but an adversarial sample makes these models output various labels, e.g., cat, bicycle, bear.
Because pruning and training rebuild the decision boundary of the pruned model, making it different from the original model.
The diversity of outputs can be measured with Label Change Rate (LCR) and used to identify adversarial samples, which is deﬁned as
[Equation]
where x is the input, f ( x ) is the original model output, s ( x ) is the pruned-model output, | S | is the size of used pruned models, C is the number of classes, and E ( · ) is deﬁned as
[Equation]
Instead of using the original model structure and parameters, we exploit random channel pruning to produce submodels.
Compared with the mutated models in Wang et al.
’s
work (Wang et al., 2019), the pruned model has a smaller size and is more sensitive to adversarial samples, which means our method requires fewer models and is also faster.
Channel pruning usually evaluates the importance of different channels of a DNN layer and removes all the input and output connections of the unimportant channels (Gao et al., 2018;
He et al., 2017; Zhuang et al., 2018).
The advantages of channel pruning include reduction of actual parameters and increase of inference speed.
Model pruning aims to ﬁnd the smallest model with the least accuracy loss, while our work focuses on generating a model set for detection.
Thus we use random channel pruning to ﬁnd a set of small models with accuracy close to the original model.
In order to reduce the computational cost while ensuring the pruned models’ accuracy and diversity, we set a ﬁxed overall pruning rate for each model and assign a random number of channels that need to be pruned in each layer.
Speciﬁcally, every several layers with the same number of channels are divided into a group.
A group has an overall pruning rate (e.g., 50%) and layers in a group will be assigned with two random pruning rates (e.g., 30% and 20%), while the sum of which equals the overall pruning rate.
A straightforward way to calculate LCR is using the ﬁxedsize sampling test, i.e., adopting a ﬁxed number of models and counting the outputs that are different from the original model.
To reduce the computational cost, we use the Sequential Probability Ratio Testing (Wald, 2004)
(SPRT) to detect adversarial samples dynamically.
The mutual exclusive hypothesis in the testing is
[Equation]
where ς h is a threshold determined by the LCR of benign samples (calculated by Eq.
SPRT runs the pruned model successively and calculate the probability ratio pr by
[Equation]
[Table]
where z is the number of models that output different labels, n is the total number of used models, p
=
ς h + σ , and p 1 = ς h − σ .
We set a relax scale σ , which means when the
LCR falls in the region ( ς h − σ, ς h + σ ) , neither hypothesis can be denied and the test continues.
The accept LCR and deny LCR are deﬁned as follows
[Equation]
where α and β denote the probability of false positive and false negative, respectively.
The test stops when one of the hypothesis is accepted.
The input is considered as a benign sample if pr ≥ ς a , while it is adversarial otherwise.
4. Experiments 4.1. Dataset and Models
We evaluate our approach on CIFAR10 and SVHN.
The former contains 50,000 images for training and 10,000 images for testing, while the latter are 73,257 and 26,032, respectively.
The image size of both two datasets is 32 × 32 × 3 .
We adopt ResNet18 and VGGNet16 for CIFAR10 and SVHN, and their accuracy is 93.03% and 95.63%, respectively.
4.2. Random Channel Pruning
We set different pruning rates according to the number of channels of the layer in the group.
The details are shown in Table 1, e.g., the layers in a group have 64 channels, the group pruning rate is set to 50%.
Overall, 65% of the channels are pruned for ResNet18 and 70% for VGGNet16.
Under this setting, the pruned model accuracy is above 90%, while the model size is relatively small.
The average model size after pruning is listed in Table 2.
We compare our approach with the MMT (Wang et al., 2019), which is a
SOTA adversary detection algorithm.
There are four mutation operators that can be used to generate mutated models, we choose the best performers for comparison, i.e., Neuron
Activation Inversion (NAI).
We use the best parameter setting, i.e., the mutation rate is 0.007.
Both MMT and our method use SPRT to test the generated models’ outputs.
For a fair comparison, we set the maximum number of available models in SPRT to 100.
[Table] [Table]
4.3. Adversarial Sample Generation
We use six typical adversarial attack methods, includingwhite-box and 2 black-box, each method generates 1, adversarial samples for detection.
The parameters for each attack are summarized as follows:
FGSM:
the scale of perturbation is 0.03; 2.
JSMA:
the maximum distortion is 12%; 3.
CW: adopt L2 attack, the scale coefﬁcient is 0.6 and the iteration number is 1000; 4.
Deepfool (DF): the maximum number of iterations is 50 and the termination criterion is 0.02; 5.
One Pixel Attack (OP): the number of pixels for modiﬁcation is 3 (in order to ensure that enough successful samples are generated) and the differential algorithm runs with a population size of 400 and a max iteration count of 100; 6.
Local Search Attack (LS): the pixel complexity is 1, the perturbation value is 1.5, the half side length of the neighborhood square is 5, the number of pixels perturbed at each round is 5 and the threshold for kmisclassiﬁcation is 1.
In addition to the
above seven attacks, we also regard the normal samples which Wrongly Labeled (WL) by the original model as adversarial samples.
4.4. Metrics
a) AUROC:
Our approach takes the LCR of normal samples as the threshold.
In order to verify whether the feature is suitable for distinguishing adversarial sample from the normal sample, we calculate the area under the ROC curve to determine whether LCR is an appropriate feature (the closer the AUROC is to 1, the better the feature is).
b) Detection accuracy and the number of models used: in addition to detection accuracy, we also evaluate the number of models required for detection.
The higher the accuracy, the fewer models needed, the better the method.
[Table]
4.5. Results
As shown in Table
[Table]
our random pruning strategy greatly reduced the model size from 85 .
35 MB and 112 .
45 MB to 20 .
88 MB and 5 .
47 MB on CIFAR10 and SVHN, respectively.
The FLoating point OPerations (FLOPs) of generated models are listed in Table 3.
The pruning reduces the number of FLOPs from 140.60M and 314.03M to 51.82M and 32.42M on CIFAR10 and SVHN respectively.
Note that the mutation operator does not change the model size nor the
FLOPs, thus models generated by MMT are the same size as the original model.
AUROC scores are summarized in Table 4.
With the best results marked in bold, our approach outperforms MMT with average AUROC 0.9750 and 0.9816 on CIFAR10 and
SVHN respectively, while MMT is 0.9502 and 0.9242.
It shows that our pruned models are better in distinguishing
Figure normal samples and adversarial samples.
Figure 2 and Figure 3 show the adversarial sample detection accuracy and the number of used models.
On average, our approach used only 29.24 and 30.68 models for all attacks, while MMT requires 42.81 and 64.66 models for CIFAR and SVHN, respectively.
In addition to the reduction in model numbers, our pruned model also has advantages in terms of memory footprint and inference speed.
The average detection accuracy on CIFAR10 and SVHN are 92.76% and 93.92%, exceeding MMT (87.47% and 63.00%) 5.29% and 30.92%, which shows the superior of our approach.
5. Conclusion
In this paper, we propose an adversarial sample detection algorithm based on random channel pruning models.
Compared with mutation operators, channel pruning greatly reduces the actual model size and improves the model sensitivity to the adversarial samples.
We use SPRT to test the pruned models outputs and detect adversarial samples through the label changing rate.
Experimental results show that our method outperforms MMT in AUROC, the number of used models, model size, and detection accuracy.
The average AUROC of our method outperforms MMT (0. and 0.9242) 0.0248 and 0.0574 on CIFAR10 and SVHN, respectively.
Using only 29.24 and 30.68 models, the average detection accuracy of our method on CIFAR10 and
SVHN are 92.76% and 93.92%, while MMT needs 42.
and 64.66 models with accuracy 74.74% and 44.79%.
Acknowledgements
This work was supported in part by the National Natural
Science Foundation of China under Grant 61973273, and by the Zhejiang Provincial Natural Science Foundation of
China under Grant LR19F