An Adversarial Attack via Feature Contributive Regions
Authors: Yaguan Qian
Jiamin Wang
Xiang Ling
Zhaoquan Gu
Bin Wang
Chunming Wu
Venue:  (2021)
Keywords: Adversarial example
Feature contributive regions
Local attack

Abstract: Recently, to deal with the vulnerability to generate examples of CNNs, there are many advanced algorithms that have been proposed. These algorithms focus on modifying global pixels directly with small perturbations, and some work involves modifying local pixels. However, the global attacks have the problem of perturbations’ redundancy and the local attacks are not effective. To overcome this challenge, we achieve a trade-off between the perturbation power and the number of perturbed pixels in this paper. The key idea is to find the feature contributive regions (FCRs) of the images. Furthermore, in order to create an adversarial example similar to the corresponding clean image as much as possible, we redefine a loss function as the objective function of the optimization in this paper and then using gradient descent optimization algorithm to find the efficient perturbations. Various experiments have been carried out on CIFAR-10 and ILSVRC2012 datasets, which show the excellence of this method, and in addition, the FCRs attack shows strong attack ability in both white-box and black-box settings.

Under review as a conference paper at ICLR 2021
AN ADVERSARIAL ATTACK VIA FEATURE CONTRIBUTIVE REGIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Recently, to deal with the vulnerability to generate examples of CNNs, there are many advanced algorithms that have been proposed. These algorithms focus modifying global pixels directly with small perturbations, and some work involves modifying local pixels. However, the global attacks have the problem of pertur- bations' redundancy and the local attacks are not effective. To overcome this challenge, we achieve trade-off between the perturbation power and the number of perturbed pixels in this paper. The key idea is to find the feature contribu- tive regions (FCRs) of the images. Furthermore, in order to create an adversarial example similar to the corresponding clean image as much as possible, we re- define a loss function as the objective function of the optimization this paper and then using gradient descent optimization algorithm to find the efficient per- turbations. Our comprehensive experiments demonstrate that FCRs attack shows strong attack ability in both white-box and black-box settings for both CIFAR-10 and ILSVRC2012 datasets.
INTRODUCTION
The development of deep learning technology has promoted the successful application of deep neural networks (DNNs) in various fields, such as image classification (Krizhevsky et al., 2012 Simonyan & Zisserman 2014), computer vision (He et al.] 2016) TTigman al.] (014), natural language processing (Devlin et al. 2018 Goldberg 2017). etc. In particular, convolutional neu- ral networks (CNNs), a typical DNNs, have shown excellent performance applied in image clas- sification. However, many works have shown that CNNs are extremely vulnerable to adversarial examples (Szegedy al. 2013). The adversarial example is crafted from clean example added by well-designed perturbations that are almost imperceptible to human vision, while can fool CNNs. Scholars have proposed a variety of methods to craft adversarial samples, such L-BFGS (Szegedy et al. 2013), FGSM (Goodfellow et al. 2014), I-FGSM (Kurakin et al. 2016). PGD (Madry et al.) 2017 and C&W (Carlini & Wagner 70) These attack strategies can successfully mislead CNNs to make incorrect predictions, restricting the application of CNNs in certain security-sensiinn eas (such as autonomous driving, financial payments based on face recognition, etc.). Therefore, learning how generate adversarial examples is of great significance.
We can categorize these attacks into two categories, i.e., the global attacks and the local attacks, according the region added perturbations. The global attacks tempt to perturb all pixels of the clean image, such as FGSM [Goodfellow et al., 4014), PGD (Madry et al. 2017) and C&W (Carlini & Wagner) 2017); the local attacks only modify some pixels of the clean image. such as one-pixel attacks (Su et al. 2019) and JSMA (Papernot et al. 2016b). At present, the global attacks perturb all pixels on the whole image, which not only fail to destroy the feature contributive regions (the critical semantics of an image), but they also increase the degree of image distortion. We explain in detail in the experimental part. The local attacks seem to able to solve this problem, but the current proposed local attacks don't 't well realize that focus on undermining the image feature contributive regions. Papernot et al. (0016b) proposed a method of crafting adversarial example based on the Jacobian Saliency Map by constraining the to norm of the perturbations, which means that only few pixels in the image are modified. However, this method has the disadvantage of over- over-modifying the value of the pixels, making the added perturbations easily perceptible by the naked eye, and its adversarial strength is weak (Akhtar & Mian 2018). [Su et al. (2019) proposed an extremely adversarial stack-one-piln attack. One-pixel attack can fool CNNs by changing I to pixels, but

---

Under review as a conference paper at ICLR 2021
Eprptinee
cleepan
6d
Origial
FCRs
Top-3
Figure 1: We use Grad-CAM to get the heatmap of the image and the red frame area is the feature contribution regions involved in this work.
this method is better for low-resolution images attack (such as CIFAR-10), and the attack success rate for high-resolution images will be greatly reduced (such as ImageNet), and the cost is very large & distortion (Xu et al. 2018).
In this paper, we propose novel attack method to overcome the redundant perturbations of the global attacks and the poor strength of the proposed local attacks. Inspired by the work of CAM (Zoou et al., (2016) and Grad-CAM (Selvaraju et al., 2017). it is the most effective way to reduce image distortion, high efficiency and reduce computational complexity by adding perturbations to the critical semantics. As we all know, CNN is an end-to-end representation learning model, which starts from simple low-level features and combines them into abstract high-level features layer by layer. Thus, Grad-CAM (Selvaraju al.) (017) uses the gradient information of the last convo- lutional layer as the metric to understand the decision of each neuron for target classification, and explains in visual way that not all image pixels contribute the model classification. Similarly, as shown in Figure D the red area is the main contributive area. Therefore, perturbing the image globally is the most efficient strategy. We propose the FCRs attack strategy, which only adds perturbations in Feature Contributive Regions (FCRs) with the aim of generating sparse and more excellent perturbations. Especially, compared with existing local attacks, our proposed method per- turbs continuous semantic regions rather than discrete pixels. In this work, we use Frad-CAM to locate regions that have a greater impact on the classification decision of CNNs. To ensure the similarity between the adversarial example and the corresponding clean image much as possible, the objective function we optimize is the sum the two parts of the function: the l norm of the perturbations and the loss function of the generated adversarial examples. We thus use the stochastic gradient descent optimization algorithm to find efficient perturbations. In order to avoid the situation where the perturbations do not update when the objective function tends to zero, also introduce inverse temperature under the inspiration of Hinton et al. (013)).
Compared to previous work, the contributions of our work summarized as follows:
We propose attack via feature contributive regions (FCRs) for achieving trade-off between the powerful attack and the small perturbations. More importantly, this work implements effective local attack algorithm by redefining objective function.
Specially, we novelly propose an inverse temperature T, which avoids the situation where the loss function the generated adversarial example tends to zero when the stochastic gradient descent optimization algorithm is used to find the perturbations.
Comprehensive experiments demonstrate that FCRs attack consistently outperforms state- of-the-art methods on the CIFAR-10 and ILSVRC-2012 datasets. In addition, we verify the importance of FCRs by dividing the original clean image into two parts (i.e., FCRs and Non-FCRs).
RELATED WORK
In many cases, the CNNs are vulnerable to adversarial attacks which have caused extensive research in academia. (Szegedy et al. (2033) used the constrained L-BFGS algorithm craft adversarial ex-
2

---

Under review as a conference paper at ICLR 2021
amples. L-BFGS attack has high attack success rate, but the computational cost is also high (Nar-
odytska & Kasiviswanathan 2017). Therefore, Goodfellow et al. (2014) proposed FGSM, which can quickly generate adversarial examples but has low attack success rate. Kurakin et al. (2016)
proposed the Iterative attack method (I-FGSM) on the basis of FGSM and Madry et al. (2017) pro- posed PGD. [Dong al.] (2018) proposed an iterative algorithm based momentum (MI-FGSM)
to improve the transferability of adversarial samples. Xie et al. (2019) combined the input diversity strategy with iterative attacks on I-FGSM and MI-FGSM to further improve the transferability of adversarial examples. The aforementioned attacks belong to the gradient attack family, and they destroy the semantic information of the whole image. Papernot et al.] (2016b) proposed an attack method based on the Jacobian Saliency Map by minimizing the to norm of adversarial perturba- tions and used a greedy algorithm to find salieney pixels. However, this method has the problems of over- weer-odifyinng pixels too much and weak attack intensity. [Su et al. (2019) proposed an adver- sarial attack method based on the differential evolution algorithm. This method also focuses on the number of pixels to be modified, but does not limit the power of single change, thus leading to very large & distortion (Xu et al.) 2018). In this work, we expect to achieve a more effective attack that can be successful as existing attacks but achieves trade-off between the perturbation power and the number of perturbed pixels. We will show that the proposed FCRs attack is able to destroy the feature contribution regions that make attacks successful, but without incurring extra pixel-level perturbations.
Related to our work is Deng & Zeng (201)) who proposed a spatial transformed attack method based on attention mechanism. This work expands the stadv (Xiao et al. 2018) to A-stadv, The purpose of this work to generate adversarial examples with less interference and less visible. The author only conducts experiments on the ImageNet dataset, and does not discuss the black-box attack effect of this method. But while verifying that many pixel-level perturbations are redundant, our work proposes new algorithm to craft perturbations, and demonstrates its white-box and black- box attack effects on the CIFAR- 10 and ILS ILSVRC2012 datasets. In addition, Xu et al.. (2019) used CAM to explain adversarial perturbations but their target is not generate adversarial examples, but to understand and interpret adversarial examples. Zhang et al. (2020) proposed a target-free free method to generate adversarial examples via principal component analysis and made adversarial examples relate to the data manifold, but their experiment showed that the performances of their method were not always better than FGS and C. Here we pay more attention the feature contribution regions and finally, we achieve a trade-off between the powerful attack and the number of perturbed pixels.
3 METHODOLOGY
Inspired by "attention mechanism" (Zagoruyko & Komodakis 2016). we believe the classifier's performance greatly affected by some specific feature regions that is termed as feature contributive regions (FCRs) in this paper. This intuition is also confirmed by Deng & Zeng (201)) proposed A- stady which is an attention based on spatial transformed adversarial example. Therefore, if we find FCRs and add perturbations to them, it will be more effective to fool the classifier with fewer perturbations than previous methods. Our idea to divide an image into two semantic parts: FCRs and Non-FCRs and then perturbs feature contributive regions. The result of fewer perturbations ensures maximumly adversarial effects on local regions of clean images.
3.1 NOTIONS
Deep neural networks (DNNs): A DNN can be expressed as high-dimensional approximation function: f(X, 0) R + R", where X E IR' is the input variable, Y E IR" is the true class, X and 0 represents the model parameters. In this work, we focus on a specific DNN, convolutional neural networks (CNNs) that are typically comprised of convolutional layers with some method of periodic downsampling (either through pooling or strided convolutions). Here, we define the Logits layer. The input before the softmax layer of the CNNs, namely the Logits layer (the penultimate layer): Y3 = wf A, j = 1, 2, ., C. where wff the weight matrix and A is the input vector of the Logits layer, which contains mapping function X + A. Then the softmax function can be expressed as S, = exp / Di= exp Y,, and the final model can be expressed as f(X) = S (w] A). Given an
input X, then the predicted class of X can be expressed V arg max f(X),- The goal
3

---

Under review as a conference paper at ICLR 2021
of model training is to minimize the cross-entropy loss function, which can be expressed as:
J= E YJ log S, = - log Sj
(I)
j=1
where Y is 1 x C' vector and there are C' values in it. Only one value is 1 (corresponding to the true label), and the other C values are all O. For N input-label pairs (X,, Yi), the cross-entropy loss function of the model can be expressed as:
N
N log = N Σ log Sj
L i=1 j=1
(2)
N
Adversarial examples: An adversarial example can be represented as X' = X + 6, where 8 the perturbation. Normally. the perturbation 8 is constrained by the to. Eg or loo norm, that is ||X - -II,, S e. For untargeted attacks, we only need to search for an X' satisfying Y' = arg max; f (X'),. where Y' + Y and we also do not need to specify which class will be misclassified; for targeted attacks, we specify target class Y * Y, so that the target model not only misclassifies the example, but also needs to classify them into the specified class. In general, the targeted attacks are more difficult than untargeted attacks.
3.2 FEATURE CONTRIBUTIVE REGIONS (FCRS)
FCRs refer to the regions in an image that are critical for model prediction. We can utilize Grad- CAM (Selvaraju et al. (017), CAM (Zhou et al. 20166) and c-MWP (Zhang et al.) (018)) to observe FCRs. However, compared with CAM and c-MWP, Grad-CAM is not restricted by specific CNNs architecture. In addition, it can generate better quantitative and qualitative results with less compu- tation. As a result, we use Grad-CAM to search for FCRs in our work.
Suppose the input image X is forward propagated through the CNNs, and the last layer of convo- lutional layer outputs the high-level feature map A the image, where A E Raxx represents the activation of the k-th convolution kernel with the size of u x v. A outputs the score vector Yalso called logits) of each class after passing through a fully connected layer FC, where y represents the logits value of the C-th class. To this end, compute the gradient of y to A i.c. ay / to measure the classification prediction importance of the k-th convolution kernel to the C-th class. Furthermore, we adopt the global average pooling operation calculate the weight Xe of the k-th. convolution kernel:
dYc 1 ΣΣ 2
where Z = Aty is the activation at cell (i, of the k-th convolution kernel. We use the weight XE to perform weighted summation of A and calculate feature activation map Ee XL A for the C-th class. Considering that only the positive value in Σn X will have a positive effect on the final classification result, the final weighted result is reactivated by ReLU to remove the influence of the negative value, and the activation map the C-th class is obtained:
Lxx = U Σ A
We can visualize Lx in the form of heatmap (e.g. Figure Dp. in which the red area is the feature contribution regions (FCRs) to the C-th class.
Since the FCRs are usually irregular, we introduce a masking mechanism to locate. Formally, the mask is a 0-1 matrix with the same size of the input image. The element is 0 in maskx indicates the corresponding pixel in the image is not in the FCRs. On the contrary, the element is indicates the corresponding pixel is in the FCRs. Thus, can obtain the FCRs of the image by simply Hadamard product applied between the mask and the image. For obtaining the mask, a simple threshold mechanism can be utilized:
maskx {
1
(5)
others
where t is a threshold and Lx indicates that the input image X is the C-th class activation map. Our proposed method uses mask to locate the location of the added perturbations ören-
4

---

Under review as a conference paper at ICLR 2021
3.3 GENERATE PERTURBATIONS FOR FCRS
We now turn to our approach to generate adversarial perturbations. To begin, we rely on the initial formulation of adversarial perturbations (Goodfellow al. 2014) and formally define the problem as follows:
min |
(6)
s.1. ʃ(X + 8) 8) イリ
X + ô E 10
where || ||pp is the norm that constrains perturbations 5. The commonly used p-norm is lo. &g coo X is fixed, and the goal is to find the minimal á that can fool the CNNs.
Our method is different in that only perturbs FCRs, solve this problem by formulating it as follows:
min
|0FCRRl
SPER
s.t. f(X + 6FCR) # y:
X ÖFOR [0,
However, the exact and direct computation of ||бссиll is difficult for existing algorithms, as the constraint f(X + 8) # y is highly non-linear. Therefore, we approximate [|öFER|I, in dif- ferent form that is better suited for optimization. f(X + + y. This objective function consists of two parts: (I) loss function for generating adversarial examples, and (2) an lg regularization function to limit the perturbations. In theory, the to and &o norms can also be considered to norm is non-differentiable and cannot be adopted for the standard gradient descent algorithm. In addition, the fo norm only focuses the largest value in ÖFCR, it easy oscillate between the two suboptimal solutions during the gradient descent process (Carlini & Wagner 2017]. Therefore, we use the & norm of the perturbations OFCR as the distance metric. Thus define the objective function as follows:
define an objective function satisfying
regularization function. However, notice that the
F F= 16FCRll2 + (Jo (X + 8ren),
where is hyper-parameter that controls the degree of distortion. For the clean image X, optimization goal is to find the åFCR that maximizes the objective function F when the model is misclassified:
max F 8FCR
s.t.
X + SFCR 10"
Since maximizing and minimizing 1/F are equivalent, we can get the following optimization problem:
min
1/F
(10)
6pen
s.t. X+0rCR E 10
Then we use the stochastic gradient descent (SGD) algorithm solve the ÓFCR. The gradient of 1/F in öpen is Vören (1/ (1/)) F) and it is used to update öpen iteratively:
8FCR (ŌFCR VI (1/Loss) LR) O maskx
(11)
where LR is a hyper-parameter, which is equivalent to the learning rate.
Firstly, generate random perturbation åFCR, and get the initial adversarial example X' X + ÓFCR. From Eq. ( we can know that when S, + 1, Jade + 0, we set P 3 (1/ (/ lly), Jado J (X Ören). Y). At this time 1/F = 1/P, VOrCR continue to use the stochastic gradient descent (SGD) algorithm to update 5yen will not lead to Jadu becoming bigger. In order to avoid this situation, we use the distillation idea introduce the hyper-parameter T (T' < 1). Applying T' will make Jado increase log T, then Eq. [ becomes the following form:
V breg (1/P), then
JT log (S//T)
(12)
5

---

Under review as a conference paper at ICLR 2021
Thus our objective function is modified to:
F= B
|0FOR ll2
+ Jar
adv
(13)
Algorithm Generate Adversarial Examples via FCRs
Input: A clean image X; the iterations N; the learning rate LR; the degree of distortion 3; the threshold t; the inverse temperature T
Output: ÖFCR
initialize ÖFCB // K is the number of feature maps in the last layer of convolution layers ere DAE X
2: 26 2 Ep Eg
k
...... K
3: Lx & ReLU
Ex
A*)
[x 2t
4: maskx = 0
FCRs
others
5: for uriz= .....NN do
6:
X' +- x ÓOCR
7:
1/F 1/ 1/ Jade)
8:
ÖFCR + (ÓFCR VoreR x LR) o maskx // Update ÔFC
9:
end for
4 EXPERIMENTS
We verify the proposed method in Section by experiments: (1) FCRs is an important basis for the final classification decision; (2) FCRs attack will produce less perturbations and reduce the pixel search space; (3) In this section, show the experimental results of white-box attack and black-box attack, which shows that FCRs attack has powerful white-box attack capability and high transferability.
4.1 EXPERIMENT SETUP
Datasets and Models: We validate our method on two benchmark datasets: CIFAR-10 (Krizhevsky
ct al. 2009) and ILSVRC2012 (Russakovsky et al. 2015). The CIFAR- 10 consists of 60,000 images sized. including 10 categories, each with 6,000 images. There are 50,000 images for training and 10,000 for testing. The ILSVRC2012 image classification dataset contains 1,200 thousand images from 1,000 categories, and 50,000 images attacking images that have been misclassified, so the images we use to generate adversarial examples all images that correctly classified by all models. We use VGG (Simonyan & Zisserman 2014) and ResNet (He et al. 2016) series models on the two datasets.
used the validation set. There is no point in
Evaluation indicators:
The evaluation indicators setting in this article the attack success rate (ASR), the image quality assessment index-peak signal-to-noise ratio (PSNR) (Hore & Ziou (010) ideal situation, we need conduct stronger attacks with smaller perturbations, so that higher PSNR and smaller & distortion can be guaranteed.
and the la distortion of perturbations.
4.2 VALIDATE THE IMPORTANCE OF FCRS
This section uses VGG and ResNet model structure to conduct experiments on CIFAR-10 to further illustrate that FCRs are the basis for model classification. We divide into FCRs and Non-FCRs by using (Figure 2(a)). The accuracy rate of the input FCRs is up to 85% and above. However, the accuracy rate of the input Non-FCRs is very low (Figure 2(b)). The experimental results show that FCRs have the greatest semantics to model decision makiin and are the areas that have positive contribution to model classification.
In order to show that global attacks will produce not powerful perturbations and FCRs adding per- turbations is the most efficient way, we improve the FGSM algorithm and add different perturbations to FCRs and Non-FCRs (Appendix AA). We conduct experiments on the CIFAR-10. We add different perturbations on FCRs and Non-FCRs for comparison and the results are summarized Appendix A Obviously, the perturbations only in FCRs hardly reduce the attack success rate, that say, the FCRs are the best areas optimize perturbations in the optimization landscape.

---

Under review as a conference paper at ICLR 2021
Original
Nav-FCR
80%
Hestmap
60%
40%
FCR
20%
Not FCR
05
(b)
Figure 2: (a) The FCRs and Non-FCRR of an original image; (b) Accuracy of different model struc- tures. For the maskx, we use the threshold 0.2.
Table 1: Third column: Accuracy of the clean images different models; Fourth column: ASR of FCRs attack to generate adversarial examples. (Hyper-parameter settings: f = 0.2, = 0.03, LR 10, N 20, 3 1 and / = 0.2, 7 = 0.05, LR 20, N 20, 3 1 the two datasets,
respectively. The introduction of parameters is detailed the Appendix
Datasets
Model
Accuracy
ASR
VGG-11
91.66%
99.70%
CIFAR-10
ResNet-18
92.19%
100,00%
VGG-19
71.02%
100.00%
ILSVRC2012
ResNet-34
72.17%
99.13%
4.3 FCRS ATTACK
We generate adversarial examples on two datasets under the white-box setting. The results in Table D show the classification accuracy of the clean test data and the ASR of the adversarial examples generated by FCRs attack on different models. Figure 3 shows the perturbations and adversarial examples generated by the global attacks and FCRs attack. These examples that can be successfully attacked. It can be seen that the FCRs attack not only generates perturbations in the FCRs but also the adversarial examples ages. However, the images of global attacks are distorted greatly. When we use the same constraint of the l distortion, we observe that the ASR of PGD is 74.33% and 56.50% the two datasets; the ASR of C&W is 72.11% 1% and 45.00%. In contrast, FCRs attack can still have powerful attack performance when it only attacks the local semantics.
randomly selected from the
very close to the corresponding im-
4.4 COMPARISON WITH OTHER METHODS
Table Table reports the ASR, PSNR, and & distortion of different attack methods (it is pointed out here that we are giving the average difference between the adversarial examples and the clean images). We show that the FCRs attack not only generates small perturbations (smaller & distortion) but also has powerful attack performance (higher ASR), and the crafted adversarial examples are more similar to the original images (larger PSNR). Specifically, the distortion performance of C&W is the worst, that le distortion is the largest on the two datasets, and the PSNR also the smallest. Given that JSMA and one-pixel attack both local attacks, do comparative experiment with
Table ASR, PSNR and l distortion for various attacks
Datasets
Attack Methods
ASR
PSNR
Eg
PGD 93.18%
68.03
5.57
C&W
97.44%
57.09
21.01
CIFAR-10
JSMA
90.33%
60.12
19.53
One-pixel
80.77%
65.7
7.03
Ours
100.00 %
79.26
1.56
PGD
97.70%
49.69
373.95
C&W
99.33%
46.86
543.76
ILSVRC2012
JSMA
90.00%
60.08
60.64
One-pixel
40.56%
80.74
9.55
Ours
100.00 %
72.67
42.94
7

---

Under review as a conference paper at ICLR 2021
Porturbations
Hearmap
ImageNer
CIFAR-10
Figure 3:
Comparison perturbations and adversarial examples crafted by PGD, C&W and FCRs attack on the two datasets. Obviously, our proposed method only destroys critical semantics and reduces image distortion. (Our constraint on the bg norm of the three methods 2 and 45.)
these two methods. On the CIFAR-10, the performance of JSMA is lower than the FCRs attack (ASR: 90.33% vs 100.00%); and its & distortion very large. On the ILSVRC2012, our method outperforms it in all metrics. We choose to attack 5 pixels for the one-pixel attack. On the CIFAR- 10, one-pixel attack is not only large in lz distortion, but also has poor attack performance. On the ILSVRC2012, although the & distortion of the one-pixel attack is the smallest, its attack success rate is only 40.56% and we observe that the one-pixel attack requires a lot of memory during the ex- periments. Thus observe that the reduction of attack semanties does not reduce the performance of FCRs attack.
4.5 BLACK-BOX ATTACK
In this section, we explore a more challenging black-box scenario where the attacker first specifies alternative model of the black-box model, and then generates can successfully attack the alternative model. Normally, this set of adversarial examples is consid- ered to have strong transferability, that is, in the of misleading alternative model, it will also mislead the target model (Papernot et al. (2)1 The underlying assumption is that highly transfer- able adversarial examples can achieve similar attack performance many different target models (Papernot et al.) 2017). Therefore, we can expect that transferable adversarial samples will reduce the accuracy of the alternative model and the same time reduce the accuracy the target model, resulting in high black-box attack capabilities. In order to prove the black-bor box attack capability of the FCRs attack, we conduct black-bor attack experiments on different target models and datasets. As shown in Table and the adversarial examples generated by FCRs attack is more transferable most cases.
of adversarial examples that
CONCLUSIONS
This work explores the method of generating perturbations via the feature contribution regions. This article provides evidence to prove that the attack on the local semanticsis the most effective. As our theory and experiments have shown, we have devised more excellent attack method. We conduct extensive experiments with the CIFAR-10 and ILSVRC2012 datasets. The results show that FCRs attack much stronger than existing global attacks (such as PGD and C&W) and local attacks (such as JSMA and Pixel), and the attack based on feature contribution regions may also provide new perspective for future research better defensive methods.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Naveed Akhtar and Ajmal Mian. Threat of adversarial attacks deep learning in computer vision: A survey. IEEE Access, 6:4440-14440, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pp. 39-57. IEEE, 2017.
Ting Deng and Zhigang Zeng. Generate adversarial examples by spatially perturbing on the mean- ingful area. Pattern Recognition Letters, 125:632-6388 2019.
Jacob Devlin, Ming- Ming-Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 1e..18000000555. 2018.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 9185-9193, 2018.
Yoav Goldberg. Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies, 10(1):1-309, 2017.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint 1oXiv:5533.02533 2015.
Alain Hore and Djemel Ziou. Image quality metrics: Psnr ssim. In 2010 20th international conference on pattern recognition, pp. 2366-2369. IEEE, 2010.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Alex Krizhevsky, lyy Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv:1611.01236, 2016.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks, arXiv preprint arXiv: 06083, 2017.
Nina Narodytska and Shiva Kasiviswanathan. Simple black-box adversarial attacks on deep neural networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1310-1318. IEEE, 2017.
Nicolas Papernot, Patrick McDaniel, and Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 2016a.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In 2016 IEEE European sympo- sium on security and privacy (EuroS&P), pp. 372-387. IEEE. 2016b.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow. Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-bor attacks against machine learning. In Proceedings of the 2017 ACM Asia conference computer and communications security, pp. 506-519, 2017.

---

Under review as a conference paper at ICLR 2021
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision. 115(3):11-222,, 2015.
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based local- ization. In Proceedings of the IEEE international conference on computer vision, pp. 618-626, 2017.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1556, 2014.
Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation, 23(5):828 841, 2019.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Yaniv Taigman, Ming Yang, Marc' Aurelio Ranzato, and Lior Wolf. Deepface: Closing the gap to human-level performance face verification. In Proceedings of IEEE conference on computer vision and pattern recognition, pp. 1701-1708. 2014.
Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially trans- 2018.
formed adversarial examples. arXiv preprint arXiv:
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2730-2739, 2019.
Kaidi Xu, Sijia Liu, Pu Zhao, Pin-Yo Yu Chen, Huan Zhang, Quanfu Fan, Deniz Erdogmus, Yanzhi Wang, and Xue Lin, Structured adversarial attack: Towards general implementation and better interpretability. arXiv preprint arXiv: htt::/808.01100 016601 2018.
Kaidi Xu, Sijia Liu, Gaoyuan Zhang, Mengshu Sun, Pu Zhao, Quanfu Fan, Chuang Gan, and Xue Lin. Interpreting adversarial examples by activation promotion and suppression. arXiv preprint arXiv: 02057 2019.
Sergey Zagoruyko and Nikos Komodakis. Paying more attention attention: Improving the perfor- mance of convolutional neural networks via attention transfer. arXiv preprint 1xXiv:/6/2.03928, 2016.
Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. Top-down neural attention by excitation backprop. International Journal of Computer Vision, 126 (10) 2018.
Yonggang Zhang, Xinmei Tian, Ya Li, Xinchao Wang. and Dacheng Tao. Principal component adversarial example. IEEE Transactions on Image Processing. 29:4804-4815, 2020.
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2921-2929, 2016.
10

---

Under review as a conference paper at ICLR 2021
A IMPROVING FGSM VIA FCRs
Here, give the algorithm of combining FGSM and FCRs.
Algorithm 2 FGSM maskx
Input: clean image X; Perturbations E of FCRs; Perturbations eg of Non-FCRs Output: X"
1: X + 2 Eg 8A5,
// K the number of feature maps the last layer convolution layers
2: Lx + ReLU (5x A*)
Lx] 2
3: maskx
// Get FCRs
0
others
4: 81 = 61 x sign (0, maskx), 82 = =2 x sign (Vx.(0,X.,)) massx))
5: X' X ò1 82
6: X = clip (X', min, max)
7: end for
Table 3: The attack success rate of adding different perturbations to two different semantic parts.
Model
Gi (FCRs)
G (Non-FCRs)
ASR
0.03
0.00
35.12%
0.03
0.01
36.43%
0.03
0.03
38.71%
0.03
0.05
40.34%
VGG-11
0.03
0.03
38.71%
0.05
0.03
48.65%
0.05
0.05
49.92%
0.08
0.05
59.62%
0.03
0.00
33.47%
0.03
0.01
34.17%
0.03
0.03
35.24%
0.03
0.05
36.07%
ResNet-
0.03
0.00
33.47%
0.05
0.00
40.58%
0.05
0.03
42.23%
0.05
0.05
43.21%
B ANALYSIS OF HYPER-PARAMERT PARAMETERS
Iteration Times and Inverse Temperature T: and T are the dominant hyper-parameter in the proposed algorithm, and here we explore them effects on ASR. We can observe that both N and T' have positive trends on the ASR (Figure 4(a) 4(b).. As N and T' increase, the ASR also tends increase. When N 30, the ASR of the FCRs attack can reach 100% on both datasets. The ASR increases fastest when N to N = 5, and then it tends to grow slowly until 100%. With the increase of the iteration times. our objective function can better find the global optimal solution, thereby avoiding falling into the local optimal solution. The increase of T also obviously leads to the high ASR because the increase of can make Jr and the regularization function P become smaller, which makes objective function 1/F continue to decrease and is better to find the optimal solution when performing stochastic gradient descent. It needs to be explained here that find that the best situation be achieved when T 0.05, especially the ILSVRC2012 dataset, but the attack effect may be reduced T continues to increase.
Threshold t: The threshold is also the dominant hyper- param-er which size directly determines the size of the maskx. that is, the size the range of adding perturbations. Specifically, we vary t while keeping the other parameters fixing observe the influence of changes on the ASR and
11

---

Under review as a conference paper at ICLR 2021
the to norm of the perturbations. When f O, the ASR both datasets reaches 100%. At the same time to is 2903 and 198402. respectively. As the threshold f continues to increase, the range of perturbations continue to decrease. The direct manifestation is that the norm decreases linearly. When reaching 0.5, the norms drop to 1529 and 24026 on the two datasets, respectively, which are 1/2 and 1/10 of 0. However, the ASR does not drop dramatically, which is reduced by 0.7% the CIFAR-10 and 5.07% on the ILSVRC2012 (Figure (aa) (()). In the experiments of this paper, set the threshold 0.2 on both datasets.
100%
100%
90%
T-0.01
T-0.01
95%
80%
1-0.5
1-0.5
ASR
ASR
60%
85%
40%
80%
10
15
20
25
10
20
25
Iteration times
Ieeration times
(a) CIFAR-10
(b) ILSVRC2012
Figure 4: The effect of iteration times N and hyper-parameter 'I' on the ASR. Using ResNet- 18 network on the CIFAR-IO 0.2, LR 10, 1); using VGG-16 network the ILSVRC2012 (t = 0.2, LR 20, B = 1).
100%
ASR
ASR.
180000
160000
2500
140000
60%
120000
ASR
2250
100000
40%
40%
80000
2000
20%
1750
20%
40000
20000
0%
1500
0%
2400
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.4
0.5
(a) CIFAR-10
(b) ILSVRC2012
Figure 5: The effect of iteration times N and hyper-parameter T on the ASR. Using ResNet- 18 network the CIFAR-10 dataset
0.2, LR 10, 1); using VGG-16 network on the
ILSVRC2012 dataset (t 0.2, LR 20, 3 = 1).
12

---

Under review as a conference paper at ICLR 2021
C THE RESULTS OF BLACK-BOX ATTACK
Table ASR of alternative model and target model under various attack methods on the CIFAR-10.
The diagonal blocks show the white-box attacks, while the off-diagonal blocks indicate black-box
attacks which are much more challenging: PGD iteration number: = 20, e-ball: = 16, step
size: 2), M-DI2- (s 20, 16, 2, the transformation probability:
0.5),
C&W (c = 1, lr = 0.01, iterations = 1000), Ours (f = 0.2, T = 0.03, LR = 10, = 20, 1).
Model
Attack
VGG-16
VGG-11
VGG-13
ResNet- 18
ResNet-34
PGD
90.43%
56.57%
71.72%
58.59%
56.78%
M-DI--FGSM
90.79%
66.78%
83.94%
70.84%
69.38%
VGG-16
C&W
96.97%
51.52%
52.53%
40.40%
48.48%
Ours
96.00%
81.8%%
88.29% %
75.00% %
77.08% %
PGD
63.46%
93.51%
65.17%
55.18%
58.15%
M-DI M-DI--FGSM
90.11%
99.10%
90.69%
87.79%
85.09%
VGG-11
C&W
46.46%
93.94%
45.23%
39.70%
49.25%
Ours
86.97%
99.90%
91.00% %
86.12%
85.69 %
PGD 75.08%
61.33%
95.81%
61.36%
58.91
M-DI--FGSM
80.11%
74.47%
97.70%
79.78%
75.38%
VGG-13
C&W
64.11%
62.13%
95.72%
61.95%
61.98%
Ours
90.39%
79.78%
100.00%
80.38%
80.68 %
PGD
60.92%
60.70%
62.52%
93.18%
64.22%
M-DI'-FGSI
83.42%
72.83%
82.12%
94.60%
84.35%
ResNet-18
C&W
54.88%
61.28%
55.97%
97.44%
56.24%
Ours
85.03%
85.09%
84.18%
100%
89.59
PGD 60.92%
60.28%
64.56%
68.47%
92.72%
M-DI--FGSM
88.89%
78.08%
87.78%
91.39%
98.60%
ResNet-34
C&W 46.00%
57.75%
47.40%
48.59%
90.34%
Ours 89.00%
84.95%
86.29%
90.95%
100% %
Table 5: ASR of alternative model and target model under various attack methods on the
ILSVRC2012. Parameter settings are as follows: PGD ( iteration number: = 20, e-ball: e = 16,
step size: 2), M-DI--FGSM (s 20, 16, a 2, the transformation probability: p 0.5),
C&W (c = 1, lr = 0.01, iterations 1000), Ours (f 0.2, T = 0.05, LR 20. = 20, 1).
Model
Attack
VGG-16
VGG-19
ResNet-34
ResNet-50
ResNet- 101
PGD
99.40%
89.00%
71.70%
70.30%
62.70%
M-DI--FGSS
99.80%
92.00%
70.90%
71.20%
63.30%
VGG-16
C&W 97.00%
85.90%
71.70%
70.70%
68.70%
Ours
100.00%
92.90%
75.80%
76.80%
72.20%
PGD 90.50%
99.00%
71.40%
67.80%
64.80%
M-DI--FGSM
96.00%
99.00%
67.70%
72.60%
64.50%
VGG-19
C&W
83.80%
95.50%
70.90%
69.80%
67.80%
Ours
94.20%
99.20%
80.50%
75.60%
74.20%
PGD
82.60%
82.60%
99.40%
76.20%
67.70%
M-DI--FGSM
78.80%
81.40%
100.00%
86.60%
81.30%
ResNet-34
C&W
81.50%
81.90%
94.50%
70.90%
67.80%
Ours
89.20 %
86.20%
100.00%
90.00%
87.10%
PGD
82.60%
78.80%
72.30%
99.60%
69.60%
M-DI2-FGSMM
77.40%
77.90%
79.80%
99.50%
87.90%
ResNet-50
C&W
81.80%
82.40%
71.90%
95.50%
67.30%
Ours
87.60 %
84.60%
91.00%
99.80%
89.60%
PGD
81.60%
77.40%
70.70%
75.70%
99.60%
M-DI
72.70%
69.30%
77.80%
88.90%
99.70%
ResNet-101
C&W
83.90%
81.90%
72.70%
70.40%
95.50%
Ours
86.30 %
84.90%
90.00%
92.50%
100.00%
13


Reviews:
[{"review_id": "oDrVU-IShtR", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method for adversarial attacking, which focuses on the contributive region of the input image. \nHowever, there are several weaknesses:\n1.\tThe interested-region-based adversarial attacking is not a novel idea, it has been proposed earlier: \nYao et al. Trust Region Based Adversarial Attack on Neural Networks, CVPR 2019.\n2.\tThe proposed method in the paper is spliced by existing methods. The feature contributive regions are generated by Grad-CAM, as the authors mentioned in Section 3.2. The perturbation generation is also similar to general gradient-based methods except for the mask of the feature contributive regions.\n3.\tThe way that the information of the contributive region is leveraged is somewhat crudity. A naïve binary mask is injected into the iteration of perturbation generation. More analysis is desired. For example, what if we use a soft mask of the contributive region, rather than a hard mask, for adversarial attacking?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "OOYBwK9dtdA", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper focuses on the problem of generating sparse l2-adversarial examples in a white-box and surrogate/transfer setting. The authors consider “local attacks” – perturbing on a limited number of pixels while achieving high attack success rate. The main contribution of this work is to define the region to perturb using grad-cam based saliency maps to identify regions that have a greater impact on the classification decision. Having identified this region, the author use SGD to find the adversarial perturbations. The experimental results show that a high attack success rate can be achieved with this method. \n\nPros:\n1)\tThe idea is simple yet affective. Attack success rate seems to be high. \n2)\tPaper is well written and easy to read.\n3)\tThe experimental setup is reasonable.\n\nCons:\n1) My main concern with this paper is that it appears incremental to me. The idea builds upon Grad-cam and existing adversarial example generation approaches.  \n2)\tAlso, the idea is not new implying that there exist several related approaches achieving sparse adversarial examples (see [1,2]). \n3)\tI would encourage authors to perform a more detailed literature survey and compare their results with them in a fair manner. The metric should not just be the attack success rate but also the number of perturbed pixels.  \n4)\tThe robustness analysis of adversarial examples to image transformations or adversarial example detection methods can be interesting.  \n \n\n[1] Kaidi Xu et al., STRUCTURED ADVERSARIAL ATTACK: TOWARDS GENERAL IMPLEMENTATION AND BETTER INTERPRETABILITY, ICLR 2019\n[2] Hai Shu et. al., Adversarial Image Generation and Training for Deep Neural Networks", "extra_scores": {}, "extra_text": {}}, {"review_id": "0BjbRGzEAXr", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors proposed a new adversarial attack method based on feature contributive regions. The authors firstly utilize an off-the-shelf method to extract FCR of the image and then utilize the FCR as constraint for the perturbation. I tend to reject this paper due to following reasons:\n1) The presentation of this paper needs significant improvement, the current manuscript is hard to read. Furthermore, the equations and many mathimatical presentations are unprofessional. \n2) Although I am not familiar with the adversarial attack literature,  I think the method proposed in this paper is quite straightforward and the contribution of this paper is not significant enough for ICLR.\n3) The FCR is estimated for the original input, while, introducing perturbations might change the FCR (but I guess the FCR won't change significantly). Instead of adopting a two-stage approach which utilize fixed FCR, have you thought about generating perturbation and update FCR jointly or alternatively?\n4) I think the authors should analysis the effect of FCR threshold.\n\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an adversarial attack method based on feature contributive regions. In generating adversarial perturbations, Grad-CAM heatmaps are used as constraints for the perturbation. The overall idea is interesting and straightforward. However, as several reviewers raised, similar methods have been proposed in prior literature (Yao et al. CVPR 2019 and Xu et al. ICLR 2019), therefore making the novelty questionable. 

All three reviewers have indicated rejection, and there is no author response. The paper is therefore rejected. 
Decision: Reject