Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation
Authors: Zeyu Qin
Yanbo Fan
Yi Liu
Yong Zhang
Jue Wang
Baoyuan Wu
Venue: ICLR 2022 Submitted (2022)
Keywords: Adversarial Examples
Black-Box Attacks
Adversarial Transferability

Abstract: Deep neural networks (DNNs) have shown to be vulnerable to adversarial examples, which can produce erroneous predictions by injecting imperceptible perturbations. In this work, we study the transferability of adversarial examples, which is of significant due to its threat to real-world applications where model architecture or parameters are usually unknown. Many existing works reveal that the adversarial examples are likely to overfit the surrogate model that they are generated from, limiting its transfer attack performance against different target models. Inspired by the connection between the flatness of loss landscape and the model generalization, we propose a novel attack method, dubbed reverse adversarial perturbation (RAP) to boost the transferability of adversarial examples. Specifically, instead of purely minimizing the adversarial loss at a single adversarial point,  we advocate seeking adversarial examples locating at the low-value and flat region of the loss landscape, through injecting the worst-case perturbation, the reverse adversarial perturbation, for each step of the optimization procedure. The adversarial attack with RAP is formulated as a min-max bi-level optimization problem. Comprehensive experimental comparisons demonstrate that RAP can significantly boost the adversarial transferability. Furthermore, RAP can be naturally combined with many existing black-box attack techniques, to further boost the transferability. When attacking a real-world image recognition system, Google Cloud Vision API, we obtain 22% performance improvement of targeted attacks over the compared method.

Under review as a conference paper at ICLR 2022
BOOSTING THE TRANSFERABILITY OF ADVERSARIAL ATTACKS WITH REVERSE ADVERSARIAL PERTURBA- TION
Anonymous authors
Paper under double-bling blind review
ABSTRACT
Deep neural networks (DNNs) have shown to be vulnerable adversarial exam- ples, which can produce erroneous predictions by injecting imperceptible pertur- bations. this work, we study the transferability of adversarial examples, which is of significant due to its threat to real-world applications where model archi- tecture or parameters are usually unknown. Many existing works reveal that the adversarial examples are likely to overfit the surrogate model that they are gener- ated from, limiting its transfer attack performance against different target models. Inspired by the connection between the flatness of loss landscape and the model generalization, we propose a novel attack method, dubbed reverse adversarial perturbation (RAP) to boost the transferability of adversarial examples. Specif- ically, instead of purely minimizing the adversarial loss at a single adversarial point, we advocate seeking adversarial examples locating the low-value and flat region of the loss landscape, through injecting the worst-case perturbation (i.e., the reverse adversarial perturbation) for each step of the optimization procedure. The adversarial attack with RAP is formulated as min-max bi-level optimization problem. Comprehensive experimental comparisons demonstrate that RAP can significantly boost the adversarial transferability. Furthermore, RAP can be natu- rally combined with many existing black-box attack techniques, to further boost the transferability. When attacking a real-world image recognition system, i.e., Google Cloud Vision API, we obtain 22% performance improvement of targeted attacks over the compared method.
INTRODUCTION
Deep neural networks (DNNs) have been successfully applied in many safety-critical tasks, such as autonomous driving, face recognition and verification, etc. However, it has been shown that DNN models are vulnerable to adversarial examples (Szegedy et al. 2013 Goodfellow al. 2014), which are indistinguishable from natural examples but make a model produce erroneous predictions. For real-world applications, the DNN models are often hidden from users. Therefore, the attackers need to generate the adversarial examples under black-box setting where they do not know any information of the target model. For black-box setting, the adversarial transferability matters since it can allow the attackers to attack target models by using adversarial examples generated on the surrogate models. Therefore, learning how to generate adversarial examples with high transferability has gained more attentions in the literature (Liu et al. 2016; Tramer et al. 2018 Dong et al. 2018 Xie et 20196 Dong al. 2019 Guo et al.) 2020).
Under white-box setting where the complete information of the attacked model (e.g., architecture and parameters) is available, the gradient-
attacks such as FGSM (Goodfellow et al. 2014) and I-FGSM (Kurakin et al., 2018) have demonstrated good attack performance. However, they often exhibit the poor transferiability (Xie et al. 2019b; Dong et al., 2018), í.e., the adversarial examples generated from the surrogate models perform poorly against different target models. The previous works attribute that to the overfitting adversarial examples to the surrogate models (Dong et al.) 2018 Xie al. 2019b; Lin et al. 2020). And various techniques have been proposed improve the transferability, including input transformation (Dong et al. [0199; Xie et al.] (019b)) gradient calibration (Guo al. 2020), feature-level attacks (Huang al. 2019), and generative models

---

Under review as a conference paper at ICLR 2022
(Nsseer al. 2021 ,, etc. However, there still exists large gap of attack performance between the transfer setting and the ideal white-box setting, especially for targeted attack, requiring more efforts for boosting the transferability.
Recalling that several existing works demonstrate that is possible to enhance the standarc/robust generalization of the learned model by seeking flat local minima in both standard and adversarial
training of deep neural networks (Dziugaite & Roy 2017 et al. 2018)Jiang et al. 2020 Izmailov
al. 2018 Foret et al., [021] Wu et al. 2020b). We find that the transferability of adversarial examples between source and target models is somewhat analogous to the generalization of model parameters between training and testing sets: the formal aims to address the overfitting issue of the generated adversarial example to surrogate models, while the latter tends to alleviate the overfitting of the model parameters to the training set. The perspective motivates us to think about the following question: can we alleviate the overfitting to surrogate models by investigating the flatness of loss landscape around the generated adversarial examples?
In this work, we propose a novel attack method called reverse adversarial perturbation (RAP) to boost the transferability of adversarial examples. The key idea is to seek an adversarial example that is not only of low adversarial loss but also locates at local flat region, i.e., the points within the local neighborhood region around this adversarial example should also of similar low loss values. To achieve this goal, we design min-max bi-level optimization problem. The inner maximization aims to find the worst-case perturbation (i.e., that with the largest adversarial loss, and this is why we call it reverse adversarial perturbation) within the local region around the current adversarial example, which can be solved by the projected gradient ascent algorithm. Then, the outer minimization will update the adversarial example to find new point added with the provided reverse perturbation that leads to lower adversarial loss. Besides, we design a late-start variant of RAP (RAP-LS) to further boost the attack effectiveness and efficiency, which doesn't 't insert the reverse perturbation into the optimization procedure in the early stage. Extensive experiments on attacking several DNN mod- els have empirically verified that the adversarial examples generated by the proposed RAP method often locate at flat regions, and show much higher adversarial transferabilityy compared to existing methods. Moreover, from the technical perspective, since the proposed RAP method only introduces one specially designed perturbation onto the adversarial example, one notable advantage of the pro- posed method is that it can be naturally combined with many existing black-box attack techniques to further boost the transferability. For example, when combined with different input transforma- tions (e.g., the random resizing and padding in Diverse Input (Xie et al. (019b)), our RAP method consistently outperforms the counterparts by clear margin.
Our main contributions are four-fold: 1) we advocate considering the flatness of loss landscape around the adversarial example to boost the adversarial tranferability; 2) we propose the reverse ad- versarial perturbation (RAP) attack to explicitly encourage the local flatness around the adversarial example by injecting the worst-case perturbation; 3) we present vigorous experimental study and show that RAP can significant boost the adversarial transferability on both untargeted and targeted attacks: 4) we demonstrate that RAP can be easily combined with existing transfer attack techniques and outperforms the fate-of-thhearr performance by a large margin.
2 RELATED WORK
The black-box attacks can be categorized into two categories: 1) query-based attacks that conduct the attack based on the feedback of iterative queries to target models, and 2) transfer attacks that use the adversarial examples generated on some surrogate models attack the target models. In this work, we focus on the transfer attacks. For surrogate models, existing attack algorithms such as FGSM (Goodfellow et al., 2014) and I-FGSM (Kurakin et al., 2018) could achieve good attack performance. However, they often overfit the surrogate models and thus exhibit poor transferability. Recently, many works are proposed to generate more transferable adversarial examples.
Instead of the I-FGSM, the work of Dong et al. (2018) integrate momentum into the updating strat- egy and Lin et al. (2020) use the Nesterov accelerated gradient to boost the transferability. Data augmentation, which has been shown to be effective in improving model generalization, has also been widely studied in transfer attack, such as randomly resizing and padding (Xie et al., 2019b). randomly scaling (Lin et al.) 2020), and adversarial mixup (Wang et al. 2021b). The work of Dong jet al.] (2019) use a set of translated images to compute gradient and get the better performance against defense models. There are also some model-specific designs to boost the transferability. For
2

---

Under review as a conference paper at ICLR 2022
example, Wu et al. (2020a) find the gradient of skip connections is more crucial to generate more transferable attacks. The work of Guo et al. (2020) also propose LinBP to utilize more gradient of skip connections during the back- pack-propggation. However, these methods tend to be specific to par- ticular model architecture, such as skip connection, and it is nontrivial to extend the findings to other architectures modules. Meanwhile, |Huang et al.] (2019); Inkawhich et al. (2020a bb propose to exploit feature space constraints to generate more transferable attacks. Yet they need to identity the best performing intermediate layers train one-vs-all binary classifies for all attacked classes. Re- cently, Zhao et al. (2020) find iterative attacks with much more iterations and logit loss can achieve relatively high targeted transferability and exceed the feature- based attacks.
Apart from the input-specific adversarial attacks, there have been some methods utlizing the gener- ative models to generate the adversarial perturbations (Poursaeed et al., 2018 Naseer et al., 2019, 2021]. For example, the work of Naseer et al. (202T) propose to train generative model to match the distributions of source and target class, so as to increase the targeted transferability. However, the learning of the perturbation generator is nontrivial, especially on large-scale datasets. In summary, the current performance of transfer attacks is still unsatisfactory, especially for targeted attacks.
Wang et al. 2021a) also explore the min-max framework for producing the attacks, but they are totally different from us in terms of motivation and formulation. They are to generate attacks based on multiple models and samples. And, the variable of inner maximization in WWng al. (202 la) is the probability vector, representing the weight assigned to different models or samples.
METHODOLOGY
3.1 PRELIMINARIES OF TRANSFER ADVERSARIAL ATTACK
Given an benign sample (z. v) E (X,)),, the procedure of transfer adversarial attack firstly constructing the adversarial example add within the neighborhood region B, (z) = {x' || æIIp S }} by attacking the white-box surrogate model M 0) : X + V, then transferring æade to directly attack the black-box target model M Φ) X' - V. The attack goal is to mislead the target model, i.e., (x Φ) * y (untargeted attack), or MM' Φ) = yr (targeted attack)
with zr E 3' indicting the target label. Taking the target attack as example, the general formulation of many existing transfer attack methods can be written as follows:
min ae madii (a)
L(M (9 (z ad"): 0) v.).
The loss function L is often set as the cross entropy (CE) loss (Xie et al. 2019b) or the logit loss (Zhao et al., 2020), which will be specified in later experiments. Besides, the formulation of untar- geted attack can be easily obtained by replacing the loss function L and yı by - C and respectively.
Since M is boxte-box, if G(-) is set as the identity function, then any off-the-shelf white-box ad- versarial attack method can be adopted to solve Problem (I), such as I-FSGM (Kurakin et al.) 2018). MI-FGSM (Dong al. 2018). etc. Meanwhile, some existing works have designed different 9T-) functions and developed the corresponding optimization algorithms, in order to boost the ad adversarial
transferability between the surrogate and target models. For example, G(-) is specified as random resizing and padding (DI) (Xie et al. (009b),, translation transformation (TT) (Dong et al. 2019). scale transformation (SI) (Lin et al. 2020), and adversarial mixup (Admix) (Wang et al. 2021b).
3.2 TRANSFER ADVERSARIAL ATTACK WITH REVERSE ADVERSARIAL PERTURBATION
Adversarial transferability between surrogate to target models somewhat analogous the model generalization between training and testing sets. The former studies the overfitting to the surrogate model when constructing an adversarial example, and the latter focuses on the overfitting to the training set when training model. Inspired by this point, some previous works have attempted to borrow the effective techniques on improving model generalization to enhance the adversarial transferability. For example, inspired by data augmentation, series of transfer attack methods focused on augmenting the adversarial perturbation during the procedure of generating adversarial perturbation, and showed good transferability, such DI (Xie et al. 2019b), TI (Dong al. 2019) SI (Lin et al. 2020), and Admix (Wang et al. 2021b).
Recalling that prior works have related the model generalization to the flatness (or sharpness) of the loss landscape in the weight space, motivating a series of methods on boosting the model general- ization by implicitly or explicitly seeking flatter minima (Dziugaite & Roy 2017 Li et al. 2018
3

---

Under review as a conference paper at ICLR 2022
Boseine
BRS
Baseline+ +-AA-L
Tarpeted Attackk Success (%)
Den-121 [M[]
Den-121 (Admie)
VGG-16 (M4)
VGG--6 (Admix)
50-
20-
15-
20-
33-
20-
10-
335
200
320
430
0
100
335
300
400
0
1.0
220
302
100
382
200
200
400
Itoratio
Iboration
Itoration
Itoration
Figure 1: Targeted attack success rate (%) on Dense-121 and VGG-16. We take the Res-50 as the
surrogate model and take MI and Admix as baseline methods.
Jiang et al. 2020; Izmailov et al. 2018: Foret et al., 202T). Inspired by this point, we assume that the adversarial example locating at flatter regions of the loss landscape of the surrogate model may have better adversarial transferability to other models. Specifically, we encourage that not only æade itself has low loss value, but also the points in the vicinity of ad have similarly low loss values. To achieve this goal, we propose to minimize the maximal loss value within a local neighborhood region around the adversarial example œd The maximal loss is implemented by perturbing æade to maximize the adversarial loss, named Reverse Adversarial Perturbation (RAP). By inserting the RAP into the formulation I we aim to solve the following problem.
min a EB)
(G ado + +ndd""); 0), yt),
(2)
where
n ado
L(M* + n ando 9t),
arg max [radel]
Sen
with n adv indicating the RAP, and en defining its search region. The above formulations Equa- tion (2) and Equation (3) correspond to the targeted attack, and the corresponding untargeted formu- lations can be easily obtained by replacing the loss function C and y/ by -L and y, respectively.
It is a min-max livle optimization problem (Liu et al. 202T), and can be solved by iteratively op- timizing the inner maximization and the outer minimization problem. Specifically, in each iteration, given add the inner maximization w.r.t. n nade is solved by the projected gradient ascent algorithm:
n adv + adv + On sign( (V Vande L(M* + adv; 0), 9)))
(4)
The above update is conducted by T steps, and ar en Then, given n adv the outer minimization T- w.r.t. æade can be solved by any off-the-shelf algorithm that is developed for solving Equation | For example, it can be undated by one step projected gradient descent, as follows:
add f- Clips.
(te)
[a-ade a sign gadu ((M + add) ()))))))
(5)
with Clipg, (w) (a) clipping a into the neighborhood region B.(x). The overall optimization proce- dure is summarized in Algorithm| [ Since the optimization w.r.t. ad can be implemented by any off-the-shelf algorithm for solving Problem Equation ( one notable advantage of the proposed RAP is that it can be naturally combined with any one of them, such as the input transformation methods (Xie et al. 2019b Dong et al. 2019: Lin et al. 2020 Wang et al. (0211))
A Late-Start (LS) Variant of RAP. In our preliminary experiments, we find that RAP requires more iterations to converge and the performance is slightly lower during the initial iterations, com- pared to its baseline transfer attack methods. As shown in Figure [ we combine MI (Dong et al. 2018) and Admix (Wang et al. 2021b) with RAP, and adopt ResNet-50 as the surrogate model. We take the evaluation on 1000 images from ImageNet (see 4.1.. It is observed that the method with RAP (see the orange curves) quickly surpasses its baseline method (see the blue curves) and finally achieves much higher success rate with more iterations, which verify the effect of RAP on enhancing the adversarial transferability. However, it is also observed that the performance of RAP is slightly lower than its baseline method in the early stage. The possible reason is that the early- stage adversarial example is of very weak attack performance to the surrogate model. In this case, it may be waste to pursue better transferable adversarial example by solving the min-mar problem. A better strategy may be only solving the minimization problem Equation (T) in the early stage to
4

---

Under review as a conference paper at ICLR 2022
Algorithm Transfer Adversarial Attack Algorithm with Reverse Adversarial Perturbation (RAP)
Input:
Surrogate model M benign data (a; y), target label y/, loss function L, transformation G. the global iteration number K, the late-start iteration number KLS of RAP, as well as hyper- parameters in optimization (specified in later experiments)
Output: the adversarial example zddv
Initialize ado t- x
2: for k = I....,K K do
3: i: 2 KiS then
4: Initialize n +
5: for l,. do
6: Update n ad using Equation (4
7:
Update addv using Equation
quickly achieve the region of relatively high adversarial attack performance, then starting RAP to further enhance the attack performance and transferability simultaneously. This strategy is denoted as RAP with late-start (RAP-LS), whose effect is preliminarily supported by the results shown in Figure I (see the green curve) and will be evaluated extensively in later experiments.
EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Dataset and Evaluated Models. We conduct the evaluation on the ImageNet-compatinn dataset I comprised of 1,000 images. For the surrogate models, we consider the four widely used network architectures: Inception-v3 (Inc-v3) (Szegedy et al.] 2016). ResNet-50 (Res-50) (He et al.) 2016), DenseNet-121 121 (Dense-121) (Huang et al.) 2017), and VGG-16bn (VGG-16) (Simonyan & Zisser- man 2014). For target models, apart from the above models, we also utilize more diverse architec- tures: Quception-ResNett-V (Inc-Res-v2) (Szegedy et al.] 2017), NASNet-Large (NASNet-L) (Zoph) al. 2018), and ViT-Base/16 (ViT-B/16) (Dosovitskiy al. (021). For defense models, we adopt the two widely used adversarial trained models: adv-Inc-v3 (Inc-v3ade) and ens-adv-Inc-Res-2 (IncRes-v2ens) (Tramer et al. 2018).
Compared Methods. We adopt I-FGSM (Kurakin et al. 2018) (denoted as I), MI (Dong et al. 2018). TI (Dong al., 2219). DI (Xie et 20.9),, ST (Lin al.) 2020). Admix (Wang et al. 2021b), ILA (Huang et al. 2019). LinBP (Guo et al. 2020), and the generative targeted attack method TTP (Naseer et al. 2021). We also consider the combination baseline methods, including MI-TI-DI (MTDI), MI-TI-DI-SI (MTDSI), and MI-TI-DI-Admix (MTDAI).
Implementation Details. For untargeted attack, we adopt the Cross Entropy (CE) loss. For targeted attack, apart from CE, we also experiment with the logit loss, where Zhao et al. (2020) shows it behaves better for targeted attack. The adversarial perturbation e is restricted by & = 16/255. The step size is set as 2/255 and number of iteration K is set 400 for all attacks. In the following, we mainly show the results at K 400 and the results at different value K are shown in supplementary materials. For RAP, set KLs as 100 and on as 2/255. We set Gn 12/255 for and TI in untargeted attack and 16/255 for other attacks in all other settings.
4.2 CLOSER LOOK AT THE FLATNESS OF RAP
We first conduct experiments to study the effect of RAP the flatness of the loss landscape around the adversarial examples. We use Re as surrogate model and conduct the targeted attacks. We take I, MI, DI, and MTDI as baselines and combined them with RAP. We visualize the flatness of the loss landscape around add on surrogate model by plotting the loss variations when we move andv along random direction with different magnitudes a. The details of the calculation are pro- vided in supplementary materials. Figure 2 plots the visualizations. We can see that comparing to the baselines. RAP significantly improves the flatness of loss landscape around ado In the follow- ing, we investigate whether the flatness induced by RAP can boost the transfcrability of adversarial examples or not, on both untargeted and targeted transfer attacks.
Publicly available from http:///////tttooooooooooooaaaaa exampl
master/cleverhane h/tp://rraaaaa......ooeeaaaaa//a
5

---

Under review as a conference paper at ICLR 2022
Beseine-RAP
MTDI
6.25
0.61
0.514
0.010
0.332
E005
0.030
0.500
0.000
0000
0.544
0.001
0.002
0.50
0.000
0.000
01000
0.000
-2
-22
-10
-20
Magnitude
Maghitude
Nagritude
Magnttuds
Figure 2: The flatness visualization of targeted adversarial examples.
Table I: The untargeted attack success rate (%) of baseline attacks with RAP. The results with CE loss are reported. The best results bold and the second best results are underlined.
ResNet-50 VGG-16
DenseNei-121 VGG-16
Dense- 121
Inc.v3
Res-50
Inc-v3
1/4RAP/ / S
79.2/91.5 S: 91.9
19.1/91.1/
92.9
(5.0/7..0/5 57.2
87 4/ 94.2 94.3
85.17 91. 7/22X
/6.5/60.2 /6.1
MI/ -RAP ARRLLS
85.8/95.0/ 96.1
82.4/93.9 94.5
00.3/75.9/
77.4
90.3
87.5 96.0 99.6
59.3/ M0.4 82.8
TI/+ +RAP-LS
82.0/94.1
95.1
81.0/93
93.3
45.5/66.1/
89.6 94.2 /9.88
87.01
54.21 66.7 77000
DI/ -
0110/9969999 99.7
99.0 99/6 99.7
57.7/82.91
K5.0
28.2/ 99.6 99.7
98.1 / /994
67.66 86.6 N66.9
SL/ IRAP/ IRAPALS
94.9/ 98.9/ 99.7
88.6 18.6//95.7 / 97.2
65.9 29.7 84.4
95.1 96.9 9888
97.5 91.9795.0
71.67
83.2 /87.4
Admix /RRAP/-RAP-S
97.9/966/9999
95.8 /97./97.//1
71.7/87.4/9
82.07 89.8/99.
Attack
Res-50
VGG-16 mad
Dease 121
Inc.v3
Res 50
Inc-v3mm
Dense 121
VGG-16
1/ &RAP-LS
53.27530: 54.2
49.1 /0/6/
22.07 24.7/ 24.9
$1.5 62.1 / /62.0
48.7 601.8 60.0
55.1 /5.9/ 6800
MI/ -RAP +RAAPLS
62.5 76.2 76.4
60.5 73.0 73,9
30.0/ 02.7/22.///2 42.2
62.0/ 85.8 84.8
$6.7/ 84.6
63. /3.1/84.91
TIA-RAP/ ARAP-LS
64. 62.8164.8/
65.8
55.9 63.7 62.1
29.1 36.2 37.1
49.3 63.4/61 /6.66
0044/33.//63
58. 58.1 166.6 /69.5
DI/4RAP/ ARAP/-RAPPISS -
72.2/%..07 88.8
68.8 85.0 87.4
29.9 46.6 51.6
68.4 81.7 /818
71.91 85.0 84.0
76. 85.2 / /66.
/1/+RAP/+RATLLS
80.0792.7
/2.1/9488/
45.8
74,9
69.2
Admix /RRA// +RAP-LS
87.3
94.67 96.8
88.27 196.4 97.2
77.6/ 80.8
75.9
81.2 15499
78.5 83.7 /874
74.5 77.2 X3.5
4.3 THE EVALUATION OF UNTARGETED ATTACKS
Baseline Methods. We first evaluate the performance of RAP and RAP-LS with different baseline attacks, including I, MI, DI, TI. SI, and Admix. The results are shown in Table [ For instance, the 'MI/ +RAP/ +RAP-LS' denotes the methods of baseline MI, MI+RAP, and MI-RAP-LS, respec- tively. RAP achieves the significant improvements for all methods on each target model. For average attack success rate of all target models, RAP outperforms the and MI by 9.6% and 16.3%, respec- tively. For TI, DI, SI, and Admix, RAP gets the improvements by 10.2%, 10.9%. 9.3%, and 6.3%. With startstart, RAP-LS further enhance the transfer attack performance for almost all methods
Combinational Methods. Prior works demonstrate the combination of baseline methods could largely boost the adversarial transfcrability (Zhao et al. [2200] Wang et al. 2021b). We also investi- gate of behavior of RAP when incorporated with the combinational attacks. The results are shown in T As shown in the table, there exist the clear improvements of the combinational attacks over all baseline attacks shown Table In addition, our RAP-LS further boosts the average attack suc- cess rate of the three combinational attacks by 6.9%, 2.6%, and 1.7% respectively. Combined with the three combinational attacks, RAP-LS achieves 95.4%, 97.6%, and 98.3% average attack success rate, respectively. These results demonstrate RAP can significantly enhance the transferability.
4.4 THE EVALUATION OF TARGETED ATTACKS
We then evaluate the targeted attack performance of the different methods with RAP. The results with logit loss are presented and the results with CE loss are shown in supplementary materials.
Baseline Methods. The results of RAP with baseline attacks are shown in Table From the results, RAP is also very effective in enhancing the transfcrability in targeted attacks. Taking ResNet-50 and DenseNet- 121 as surrogate models for example, the average performance improvements induced by RAP are 5.0% (I), 8.1% (MI), 4.6% (TT). 10.4% (DI), 18.5% (SI), and 15.1% (Admix), respectively. Comparing to the ResNet-50 and DenseNet- 121, the baseline attacks generally achieve lower trans- ferability when using the VGG-16 or Inception-v3 v3 as the surrogate models, which has also been verified in existing works (Xie et al., 2019b Zhao et al. 2020). However, for Inception-v3 and VGG-16 16 as the surrogate models, RAP also consistently boosts the transferability under all cases. With late-start, RAP-LS could further improve the transferability of RAP for most attacks. The average attack success rate under all attack cases of RAP-IS is 2.6% higher than that of RAP.

---

Under review as a conference paper at ICLR 2022
Table 2: The untargeted attack success rate (%) of combinational methods with RAP. The re- sults with CE loss are reported. The best results are bold and the second best results are underlined.
Attack
Dense-12 121
ResNet-50 VGG-16
Inc-13
Rcs-50
DenseNet-121
VGG-16
Inc-v3
+RAP-LS
1100
99.9
85.7/96.0/ 96.9
99.47 99.8 100
05.1/97.1/99
MIDSI/ IRAP/ IRAP-18
100 / 100: 100
99.9 99.8
97.07 99.1 99.1
99.8
99.9 99.9
1995 99.7
98.1 98.33 98.4
MIDAI/+RA +RAP/ PARRRAAPPPRRR +RAP-LS
100 / 100 100
99.8 99.9 99.9
98.3/99.21 99.8
99.8
99.8 99.9
99.4 99.6/ 99.8
97.9/98.8 98.9
VGG-16
Iac-v==>
Res-50
Dense- 121
Inc-v3
Res-50
Dense- 121
VGG-16
MIDI/ ARAP/ -RAP-IS 90.0/97.2 / 97.7
88.8 /7.0 97.3
86.8/826/
81.4
82.9/ 91.8 91.8/90.6
X5.7 94.2 94.2/93.3
85.1 / 12.7/910
M IRAP/ IRAPLS
/7.6/9888/ 99.4
19.2 99.4
95.2
89.0/912// 92.3
92.01 95.2 9956 96.4796. 99.2 9976
87.6/903/ 92.2
MIDAI/-RAP -RAPLS
97.8/992 99.2
98.9/ 99.5 99.6
89.3/95.0 15.5
91.57 94.1
914/932
Table 3:
The targeted attack success rate (%) of baseline methods with RAP. The results with logit loss are reported. The best results are bold and the second best results are underlined.
ResNet-50 VGG-16
DenseNet-12 121
Dense-12 121
Inc-v3
Res-50
VGGG11
Inc-v3
4/4RAP/ SERRAP/RRAPA +
9.5/ 14.3
2.4/98/ 11.8
0.1/0.1 0.7
5.D/ 12.8
17.9
17.9
2.9 10.1 15..9
D.D/ / 1,2
MI/-RAP +RAP-LS
2.21 14.51:515
16.2 26.5
3.1/13.4/22
0.3/2.033.
TV/ +RAP/ +RAP-LS
7.2/ /10//7.3
4.0%
15.3
0.1/
1.2
8.4/ 13.5 20.8
5.21 12.4 16.4
/2/221133 3.0
DI /RAP/ /
62.6/ 64.9 73.9
57.2/6344
69.3
1.5
10.1
50.2/52.6/ 60.4
32.1 03.1//49:3 /49.5 / 58.9
1.4/8// 10.0
31/+RAP/RRAPE
30.0 53.2/61.
9.51 52.8
1.8/9.3/
8413110/355
1.6/
10.4
Admix / +RAP / +RAP-LS
54.67
68.0/74.8 74,6
26.0/45.4/9
/5.4/516
5.8
17.1
19.6
29.3 53.0 55.22
21.5/ 42.7 48.2
5.07 12.1
11.6
VGG-16 =>
Inc-x3
Attack
Res-50
Dense-121
Inc-v3
Res-50
Dense-121
VGG-16
I/ TRRAP/+RAPLLS
0.1/0./144
02/14/1.7
0.0/0.1/0.
0.2/0.9 11/9/033 0.5
0.2/ 0.6//
0.1 0../05500
MI/-RAP/ +RAP-LS
0./13/119
0.5/1333.0
0.0/0.00.3
0.2/17/ L5
0.6/16/15
0.2/1.3/ L.0
TI /RAP/ &RAPLLS
0.7/12/32
8.8 18//12129
0.0/0.1/0
0.2/0.5 0.7
0.1/0.7/0.6
DI/-RAP/+RAN +RAP-LS
28/7.3/9.7
381884 12.7
0.0/ 1.1
1./46// 6.4
28/58/77.5
2.6/6.3 8.1
SI/ SI/+RAP/-RR -RAPPLS
3.3/988998
7,2/ /
0.2/
0.6 2.5
0.9/27/3.2
0.5/ 2.3
(RAP/ (RAP-LS
56//1.1/119
13.04 20.2 23,6
0.7/24/28
/.5/49/52
20/69/77.
1.333
Combinational Methods. As did in the untargeted attacks, we also evaluate the performance of combinational methods. The results shown in Table [ Similar to the findings in untargeted attacks, the combinational methods obtain significantly improvements over baseline methods. The RAP-LS outperforms all combinational methods by significantly margin. For example, taking the average attack success rate of all target models as evaluation metric, RAP-LS obtains 14.2%, 11.8%, 9.3% improvements over the MTDI, MTDSI and MTDAI, respectively.
4.5 THE COMPARISON WITH OTHER TYPES OF ATTACKS
Apart from the baseline and the combinational methods, we also experiment with more diverse attack methods, including the model-specific attack LinBP (Guo et al.) 2020). the feature-based attack ILA (Huang et al. 2019), and the generative targeted attack TTP (Naseer et al. 2021).
The LinBP depends on the skip connection and the authors only provide the source code about ResNet-50. We use their re- experimer leased code and thus conduct experiments with Res as surrogate model. The results of LibBP and ILA are shown in Ta- 5] where we also implement the vari- ants of LinBP following Guo al. (2020). inlcuding LinBP-ILA, LimBP-ILA-SCOE SGM,
Table 5: The comparison with ILA and LinBP. We use ResNet-50 as surrogate model. The best results are bold.
Attak
Derse 121
Untarged VGG 16
Isc x3
Deuse 121
Targeted VGG 16
lac
IIA
95.0
71.7
2.8
LUBP LEBE-MI-DI-SOM
59.7
25.1
09.5
MI-E--RP
$9.9
LinBP-MI-DI, and LinBP-MI-DI-SGM. We observe that our MI-DI-RAP significantly outperforms
the LinBP and ILA, especially for the targeted attacks. Compared with the second-best method (i.e., LiNBP-MI-DI-SGM) we obtain a large improvement by 33.5% on average ASR of targeted attacks.
TTP Naseer et al. 202T) is the state-of-the-art generative method. To compare with it, we adopt the generators based on ResNet-50 provided by the authors. Since TTP needs to train the perturbation generator for each targeted class, follow their "10-Targets (all-source)" setting, did in Zhao et al. (022)). The results are shown in Table [ where MIDSI+RAP-LS behaves best and outperforms TTP and MTDI by large margins of 1 9.9% and 25.7, respectively.
Table 6: The comparison with TTP on targeted attack. The best results are bold.
Altack
Dense: 121
VGG- 1b
Imc-v3
TTP MIDI MITDI-RAP-ES
70.6
78.6
40,3
74.6
12.7
87.2
35.4
93.2
MTDSSIRRA-LL
59.1
7

---

Under review as a conference paper at ICLR 2022
Table 4: The targeted attack success rate (%) of combinational methods with RAP. The results with logit loss are reported. The best results are bold and the second best results are underlined.
Attack
121
ResNet-50 VGG-16
Inc-v3
Rcs-50
DenseNet- 121= VGG-16
Inc-3
MIDI IRAP/ / -RAP-LS
74.9/ / 28.2 88855
62.8/22.9
81.5
10.9/283/ 33.2
44.91 64.3/74.55
38.5 55.0 /6.55
7.71
277/25./26.
MIDSI FRAP/+RAP-LS
86.3
93.3
70.1 84.7
38.1/518/ 58.0
55.0/112/75 7588
42.07 58,4/223 /62.3
19.8 39.0 39,2
MIDAI/=RAP/ -RAPEES
91.4 22///99//90
79.9 79:0 M6.3
008/57.1/644 64.1
69.1 74.2/ 82.1
/6.3
320/43.5
VGG-16
Inc-v-=
Attack
Res-50
Dense- 121
Inc-v3
Res-50
Dense-121 121
VGG-16
MIDI HTTD//+RA//RANAA
118/167/22.9
27.4
0,7/34/46
18/83/7.5
4.1/
2.0/8.
MIDSS/ +RAP/ +RAP-LS
31,0/353330 38.7
41.7 44.4: 49.6
9.61122/15
56/119/
10.4/21.2/2 20.9
4.289//86
MIDAI/ -RAPLE
36.2/ 39.07 43.1
48.0 45. 1 555.2
11.6/17111 17.6
9.61 13.6 16.7
77.9//2.5/00 /316
8.47 12.0 12.1
Table 7: The evaluation diverse network architectures and defense models.
Attock
IncRes-v2
Untarged NASNet-L
VIT-R/16
IncRes-v2
Targeted NASNet-L.
VIT-B/16
Inc-vJuh
Untarged
IncRess-22..
Targeted IncRox-<2ew.
MIDI
83.4
89.0
27.9
32.1
68. 1
50.9
MTDI-RAP-LS
95.6
97.5
42.7
43,9
62.5
1.7
86.5
72.3
9.7
4.1
MIDSE
95.7
98.0
43.0
45.5
67.9
2.6
90.0
79.6
12.7
6.7
MIDSI-RAP-LS
98.6
99.7
574
64.0
80.4
5,3
96.5
91.5
31,0
22.0
MIDAI
91.3
45.5
92.1
82.7
12.2
MIDAHRAP-LS
99.2
99.8
4.6 THE EVALUATION ON DIVERSE NETWORK ARCHITECTURES AND DEFENSE MODELS
To further demonstrate the efficacy of RAP, we evaluate our method on more diverse network ar- chitectures, including Preptiio-ResNNet-v NASNet-Large and ViT- ViT-Base/16. We adopt ResNet-50 as the surrogate model and the results are shown in Table 7 col 2-7. As shown in the table, the proposed RAP-LS achieves significant improvements for all three combinational methods on all tar- get models, and MTDAI+RAP-LS achieves the best performance for diverse models. For MTDAI, the average performance improvements induced by RAP-LS is 5.9% and 7.8% for untargeted and targeted attacks, respectively. Since ViT is based on the transformer architecture that totally being different from convolution models, the transfer attacks based on Resnet-50 behave relatively poor on it, especially on targeted attacks. Yet our RAP-LS still obtains consistent improvements for all com- pared methods. We also consider the ensemble-mode attack on these diverse network architectures and the results are given in supplementary materials.
Furthermore, we evaluate RAP on attacking defense models adv-Inc-v3 and ans-adv-Ine-- Anc-Res-v2 v2. Following prior works (Wang et al.) 2021b Xie et al. 2019b), we adopt the ensemble- model attack by averaging the logits of different surrogate models, including ResNet-50, ResNet- 101, Inception- v3, and Eiception-ResNet-yy
The transfer attack success rate on defense models are shown in Table 7 col 8-11. We can observe that our RAP-LS further boosts transferability of the bascline methods on both targeted and untargeted attacks. For untargeted attacks, RAP-LS achieves average perfor- mance improvements of 9.% and 14.1% on Inc-v3ado and MuRRes-v2ens- respectively. For targeted attacks, the average performance improvements of RAP-LS are 14.8% and 11.1%, respectively.
4.7 ABLATION STUDY
We conduct ablation study on the borhoods en. the iteration number of inner optimization T' and late-start KLS. We adopt targeted attacks with ResNet-50 as the surrogate model.
of the proposed RAP, including the size of neigh-
We first evaluate the effect of en and T. We consider different values of en including 2/255, 4/255, 8/255, 12/255. 16/255, and 20/255. In Figure 3 (a), we plot the tendency curves of the targeted attack success rate under different values of 6, and T. Note that in Sec. 3.2 we set on = 6n/" Thus for fixed cn, larger T indicating lower stepsize an- The minimum stepsize of o, is set to 2/255. We have the following observation from the plot: for a fixed en the more iterations T. the better attack performance. Thus, we adopt relatively smaller a,, = 2/255 in our experiments. In Figure 3 (b-d), we further plot the results of different attack methods and target models w.r.f 6n. where 0n 2/255. As shown in the plots, the larger Cn generally improves the attack performance. For Inception-v3 and DenseNet- 121, the improvements become mild for even larger 6,, Overall, the value of 12 or 16 could lead satisfactory result under most cases.
8

---

Under review as a conference paper at ICLR 2022
Torgeree iitack Success Rate (8g
Don-121 (Mi: (AP)
1 RAP
1 RAP
DV RAP
15
Incrvd
25
Inceya
30
Den-121
50
Den-121
VCG-15
---
VGG 15
VGG
k,m5255
40-
6.581255
2,112255
15/255
20
200235
4055 toess atess 7. Perturbation ta (c)
12221
Number Attack Steps
Perturbattoo Size of E.
Perturbation Size of &o
[a]
Figure 3: Targeted attack success rate (%) with various T and En. Res-50 is set as surrogate model.
invv3
Dens-121
VGG-16
Targeted Attack Success Rate 1861
[+RAPPLL
MI+RAPPLL
DI-RAPILS
MIDI+ RAPILS
30
20
30
15
22
60-
10
10
10
200
152
202
0
50
100
152
201
50
100
150
200
100
150
200
Late-Start Iteration Number Ka
Late-Start Iteracion Number Kis
Late-Start Iteration Number Kis
Late-Start Iteratioo Number Ks
Figure 4: Targeted attack success rate (%) with various KLS. Res-50 is set as surrogate model.
Then we conduct the ablation study KLs. In Figure we report the targeted attack success rate of I, MI, DI, and MI-TI-DI combined with RAP-LS with KLS 0, 25, 50, 100, 150, 200. Note that the RAP-LS with Kis reduces to RAP. As shown in the plots, the proposed late-start strategy can further boost attack performance of RAP for most cases. In general, the performance improvements increase as KLs increases, and then become mild when Ki.s larger than 100. The suitable value of KLS is relatively consistent among different methods and target models.
4.8 THE TARGETED ATTACK AGAINST GOOGLE CLOUD VISION API
Finally, we conduct the transfer attacks attack practical and widely used image recognition system, Google Cloud Vision API, and in the more challenging targeted attack scenario. MTDAI- RAP-LS behaves the best performance above experiments, so we choose it to conduct the attack. We take the evaluation on randomly selected 500 images and use ResNet-50 as surrogate model. As the API returns 10 predicted labels for each query, to evaluate the attacking performance, we test whether not the target class appears in the returned predictions. Since the predicted label space of Google Cloud Vision API do not fully correspond to the 1000 ImageNet classes, we manually treat classes with similar semantics to be the same classes. In comparison, the baseline MTDAI successfully attacks 232 images against the Google API. Our RAP-LS achieves large improve- ment, successfully attacking 342 images, leading 22.0% performance improvements. These demonstrates the high efficacy of our method to improve transferability on real-world system.
CONCLUSION
In this work, we study the transferability of adversarial examples that is significant for black-box attacks. The transferability of adversarial examples is generally influenced by the overfitting of sur- rogate models. To alleviate this, we propose seeking adversarial examples that locate flatter local regions. That is, instead of optimizing the pinpoint adversarial loss, we aim to obtain a consis- tently low loss at the neighbor regions the adversarial examples. formulate this as a min-max bi-level optimization problem, where the inner maximization aims to inject the worse-case pertur- bation for the adversarial examples. We conduct a rigorous experimental study, covering untargeted attack and targeted attack, standard and defense models, and a weal-world Google Clould Vision API. The experimental results demonstrate that RAP can significantly boost the transferability of adversarial examples.

---

Under review as a conference paper at ICLR 2022
ETHICS STATEMENT
Deep neural networks (DNNs) have been successfully applied in many safety-critical tasks, such as autonomous driving, face recognition and verification, etc. And adversarial samples have posed a serious threat to machine learning systems. For real-world applications, the DNN model as well as the training dataset, are often hidden from users. Therefore, the attackers need to generate the adversarial examples under black-box setting where they do not know any information of the target model. For black-box setting, the adversarial transferability matters since it can allow the attackers to attack target models by using adversarial examples generated on the surrogate models. This work potentially contribute to understanding of transferability of adversarial examples. Besides, the better transferability of adversarial examples calls the machine learning and security communities into action create stronger defenses and robust models against black-box attacks.
REPRODUCIRILITY STATEMENT
Our approach easy to implement and requires only simple modifications to the existing meth- ods, shown in See 3.2 The detailed algorithm of our method is shown in Algorithm [ The implementation details of our method are shown in Sec 4.1 We provide the details to reproduce the experimental results in Sec of main submission and Sec|Aoof of supplementary materials, including datasets in Sec evaluated models in Sec/4.1, baseline methods S The implementation details of baseline methods also shown in Sec A We will release all related codes after the accep- tance of this work.
REFERENCES
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193, 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversar- ial examples by translation-invarinn attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4312 4321, 2019.
Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, and Jun Zhu. Benchmark- ing adversarial robustness on image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 321-331, 2020.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani. Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recogni- tion at scale. In International Conference Learning Representations, 2021. //openrevies ww..:ett./orao diw.rrummmmiiiiiiommomoi
https:
d=Y d=Yicb=cNTTT
Gintare Karolina Dziugaite and Daniel M Roy. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint arXiv: 1008, 2017.
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimiza- tion for efficientl improving generalization. International Conference on Learning Represen- tations, 2021. URLLhttts:////eeeeeeee
net// /er//oouumiii ===ddd htddm=mmmmppp.smmmmmmmmmmmmmmmmm lrM
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Patch-wise attack for fooling deep neural network. In European Conference on Computer Vision, pp. 307-322. Springer, 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572. 2014.
Yiwen Guo, Qizhang Li, and Hao Chen. Backpropagating linearly improves transfcrability of adversarial examples. In H. Larochelle, M. Ranzato, R. Hadsell, M. E. Balean, and H. Lin
10

---

Under review as a conference paper at ICLR 2022
(eds.), Advances in Neural Information Processing Systems, volume 33, pp. 85 85-95 95. Curran As- sociates, Inc., 2020. htL_attpps//prrccciisseeeeeee httpsscccpppppee http://ww...aaassooooo//rr/a///a (bozaffsscbaccceaccccca66660---0
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700-4700 2017.
Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhanc- ing adversarial example transferability with an intermediate level attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.
2019.
Nathan Inkawhich, Kevin Liang, Lawrence Carin, and Yiran Chen. Transferable perturbations of deep feature distributions. In International Conference on Learning Representations, 2020a. URL htttps://pperrett
net Forum? wt
Nathan Inkawhich, Kevin Liang, Binghui Wang. Matthew Inkawhich, Lawrence Carin, and Yiran Chen. Perturbing across the feature hierarchy improve standard and strict blackbox attack transferabilityy. Advances in Neural Information Processing Systems, 33, 2020b.
Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. Averaging weights leads to wider optima and better generalization. In 34th Conference on Un- certainty in Artificial Intelligence 2018, UAI 2018, pp. Artificial Intelligence (AUAI), 2018.
Association For Uncertainty in Representations, 2020. URL https: htttl:ttt:://pttt /ttp://///ppeeeeeeeeeeee net Stweet//oorumt--SS1IPPPP-
Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantas- tic generalization measures and where to find them. In International Conference on Learning
Alexey Kurakin, lan J. Goodfellow, and Samy Bengio. Adversarial examples in the physical world. Artificial Intelligence Safety and Security, pp. 99 12, Jul 2018. doi: 10.1201997813512511388
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss land- scape of neural nets. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 6391-6401, 2018
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In International Conference on Learning Representations, 2020. URL
Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, and Zhouchen Lin. Investigating bi-level opti- mization for learning and vision from unified perspective: A survey and beyond. arXiv preprint arXiv:2101.11517, 2021.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. arXiv preprint arXiv: 161.0700 2016.
Muhammad Muzammal Naseer, Salman Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. Advances in Neural Information Processing Systems, 32: 22:12-1292211 12915, 2019.
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self- supervised approach for adversarial robustness. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 262-271, 2020.
Muzammal Naseer. Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. On generating transferable targeted perturbations. arXiv preprint arXiv:2103. 14641, 2021.
11

---

Under review as a conference paper at ICLR 2022
Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge Belongie. Generative adversarial pertur- bations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4422-4431 2018.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adver- sarially robust imagenet models transfer better? In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 9533-35455 Curran Associates, Inc., 2020. URL http::::pppppceeeeeetopte
newrips cc/ pdd]
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.15566 2014.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zbigniew Wojna. Rethink- ing the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826 2016.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI con- ference artificial intelligence, 2017.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Confer- ence on Learning Representations, 2018. URL https:://ppereeveew.cot www.eeeeerfiiiomoooooo
rk2vSe-RZ
Jingkang Wang, Tianyun Zhang, Sijia Liu, Pin- Yu Chen, Jiacen Xu, Makan Fardad, and Bo Li. Ad- versarial attack generation empowered by min-max optimization. Advances in Neural Information Processing Systems, 34, 202
Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. arXiv preprint arXiv:2102.004336 2021b.
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In International Confer- ence on Learning Representations, 2020a. URL http::ttttpspseseeeeeeee ew net. htt.//ormmmmmmm
B.J R434Fv
Dongxian Wu, Shu-Tao Xia, and Yisen Wang. Adversarial weight perturbation helps robust gener- alization. NewrIPS, 2020b.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations, 2018 8. URL /ttpppps///peereviee net/ htt.://rwweeeeeeeeeoorrrrrrddss- 2ot.
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He, Feature denoising for improving adversarial robustness. In Proceedings of the IEEC/CVF Conference on Computer Vision and Puttern Recognition, pp. 501-509, 2019a.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille. Improving transferabilitty of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2730-2739, 2019b.
Zhengyu Zhao, Zhuoran Liu, and Martha Larson. On success and simplicity: A second look at transferable targeted attacks. arXiv preprint arXiv:2012.11207, 2020.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 8697-8710, 2018
12

---

Under review as a conference paper at ICLR 2022
A IMPLEMENTATION DETAILS
Implementation Details of Evaluated Models. For ResNet-50, DenseNet-121, VGG-16, Inception-v3,, we adopt the pre-trained models provided by torchvision package. For Inception- ResNet-v2, NASNet-Large, ViT-Base/16, adv-Inc-v3, and uns-adv-Inc-Res-e we adopt the pro- vided pre-trained models
Implementation Details of Baseline Attack Methods. We adopt the source code F provided by Zhao al. (2020) implement MI, TI. and DI attacks. The decay factor for MI is set as 1.0. The kernel size is set as for TI attack, following et al. (2020). The transformation probability as 0.7 for DI. For SI and Admix, we adopt the parameters suggested Wang al. (2021b). The number of copies for SI is set as 5. The number of randomly sample and 1j of Admix are set as and 0.2 respectively. For implementation of ILA and LinBP, we utilize the source code provided by Guo et al. 2020). For implementation of TTP, we use the pre-trained generator based on ResNet-50 provided by Naseer et al. (202T).
Implementation Details of Visualization. We visualize the flatness of the loss landscape around add on surrogate model by plotting the loss change when moving ndd along a random direction with different magnitudes. Specially, first sample d from Gaussian distribution and normalize it Then, we calculate the loss change (flatness) f(a) with different on a & unit norm ball, d +- Tall=" magnitudes
f(a) L(M + d): 94) L(M*(9 0), v1)-
Considering d is randomly selected, repeat the above calculation 20 times with different d and take the averaged value to conduct the visualization.
Computation Cost. We conducted all experiments in Nvidia-V100 GPU. Taking MTDI as an example, its computation time is 2 hours. Combining RAP or RAP-LS, the computation time becomes 5.5 hours.
B THE EXPERIMENTAL RESULTS IN DISCUSSION
In this section, show the experiments mentioned in the discussion. The detailed results are shown in the below tables and figures.
Visualization of Untargeted Attacks We add the visualization results about untargeted attacks. The implementation details are also shown Section A.
Baseline
Baselne--RAP
320
345
K0
333
70-
20
20
10
30
10
20 25 10
20
20
10
27
10
20
Hagnituee 4
Magniude
Magn Lude
Magnitu.co de
Figure 5: The flatness visualization of untargeted adversarial examples.
We add I-FGSM with random-star start (Io) in our experiments. The untargeted attack performance of I and I combined with RAP is shown in Table We follow the experimental setting in Section of the main submission. RAP achieves improvements for I on each target model, and with late-start, RAP-IS further enhances the transfer attack performance for I*, getting 6.0% increase in terms of average attack success rate.
13

---

Under review as a conference paper at ICLR 2022
Table 8: The untargeted attack success rate (%) of _ with RAP. The results with CE loss and 400 iterations are reported. The best results are bold and the second best results are underlined.
ResNer-50
Dense-
VGG-16
Inc-v3
Res-50
VGG-16
Incor3
I- P/#RAP/-RAP-L 75.71 80.2 81.2
78.07 80.5 82.1
35.0/47.2/ 48.9
85.6/88 7869
83.7/ 77.6/44./58
47.0/ 17.055.2/55J
Allack
Res-50
Dense-12 121
Inc-v3
Res-50
Dense-121
VG6-16
1" //+RAP/-RAPLLS 52.9/ 56.8/ $7.2
19.9 51.5/55.3
21.7/245/ 25.1
50.9/5522 57.9
48.6 54.0 56.2
54.7/616/ 63.66668
The Evaluation on More Defense Models
Following the reviewers' suggestions, we take the evaluations on more defense models: Feature Denoising (Xie et al. 2019a), Adversarial training
models in ImageNet (Salman et al. 2020). NRP Naseer et al. 2020), R&P (Xie et al. 2018).
For Feature Denoising, we utilize the pre-trained ResNet-152 model. For AT models on ImageNet, adopt the pre- pre-trained ResNet-50 AT models from (Salman et al. 2020). For foo norm, adopt the ResNet-50 AT model with budget 4/255, which ranks first in the RobustBench leaderboard f For l2 norm, adopt the ResNet-50 AT model with budget 0.5. The untargeted attack performance shown in Table We follow the experimental settings in Section of the main submission. We can observe that our RAP-LS further boosts the transferability of baseline methods on these new defense models, getting 5.5% boost for the average attack success rate.
For NRP, we adopt the pre-trained purifiers provided by the authors. Since NRP is offline defense module, we combine it with the two used ensemble AT models and the above two AT models. The untargeted attack performance is shown in Table IO We also follow the experimental settings in Section 4.6 of the main submission. Combining NRP with AT models is much stronger defense mechanism, but RAP-LS still achieves an improvement by 0.8%.
For R&PP, we adopt the source code provided by Dong et al. (2020) to implement it. We also combine R&PP with the two used ensemble AT models and the two new AT models above. The untargeted attack performance is shown in Table I1] We also follow the experimental settings in Section 4.6 of the main submission. For R&P, RAP-LS achieves an 9.1% increase terms average attack success rate.
Table 9: The evaluation of ensemble attacks on AT models and Feature Noising.
Untarged Res-50 AT (foc)
Attack
Res-50 AT (22)
Feature Denoising
MIDI
42.5
32.4
44.1
MIDI+RAP-LS
59.5
34.4
44.4
MTDSI
56.6
35.8
45.0
MTDSI+RAP-LS
70.3
36.6
MTDAI
62.1
35.6
44.2
MTDAI+RAP-LS
73.7
37.7
45.2
Table 10: The evaluation of ensemble attacks on defense models with NRP.
Untarged IncRes-v2ens
Attack
Inc-v3add
Res-50 AT (6x)
Res-50 AT (E.o)
MTDI
23.1
13.5
14.2
25.7
MIDI-RAP-LS
22.7
14.8
14,9
26.3
MIDSI
22.5
14.2
26.1
MIDSI-RAP-LS
24.5
15.3
15.4
26.2
MIDAI
24.1
14.7
14.2
25.9
MTDAI-RAP-ILS
24.9
15.6
15.3
26.1
The Comparison with EOT Attack them to ade in each iteration. We set the standard deviation of Gaussian noise as 15/255 and the
We sample noise ad from Gaussian multiple times and add
httts:://rbbuttbenchh wttp::ttbbbbbb...............
14

---

Under review as a conference paper at ICLR 2022
Table 11: The evaluation of ensemble attacks on defense models with R&P.
Untarged
Attack
Inc-v3edo
IncRes-v2ens
Res-50 AT (fz)
Res-50 (Poo)
MTDI
65.0
46.2
52.3
43.7
MTDI-RAP-LS
82.1
63.2
65,3
45,8
MIDSI
86.5
64.1
45.9
MTDSI-RAP-LS
93.4
84.9
74.0
46.2
MTDAA
88.9
76.5
68.4
46.2
MTDAI+RAP-LS
94.8
87.0
77.7
47.7
number of EOT as 20. We take MTDI as our baseline, ResNet and DenseNet as surrogate models and compare this attack with our method RAP.
The untargeted and targeted attack performance is shown in Table 12 As shown in experimental results, our methods achieve better performance and surpass this EOT attack a large margin on target models for targeted attacks. RAP-LS gets an 8.7% increase for targeted attacks in terms of average success rate. This demonstrates the superiority our methods.
Table 12: The comparison between EOT and our method.
ResNet-50 (Untarged)
ResNett-0 (Taagged)
Attack
Dense-15 121
VGG-16
Inc-v3
Dense 121
VGG-16
Inc-v3
MTDI=EOT MTDI-RAP-LS
99.9/ 100
99.7 99.9
95.3/ =
82.7 / 88.5
73.4 81.5
34.6/33.2
Attack
DenseNet-121 (Untarged) VGG-16
DenseNet-121 (Ttarged) VGG-16
=
Res-50
Inc-v3
Res-50
Inc-v3
MTDI-BOT -ATI-RRAPLS
99.8 / 100
99.7 / 100
96.3 / 97.1
56.6/74.5
49.7 65.5
22.6/ 26.5
C ADDITIONAL EXPERIMENTAL RESULTS
In this section, we first show the evaluation of targeted attacks with CE loss in Sec C. Then we show the results ensemble attacks on more diverse network architectures in See Sec we report the experimental results w.r.f different value of iterations.
C.I THE RESULTS OF TARGETED ATTACKS WITH CE LOsS
Following the settings in main submission, we evaluate the targeted attack performance of the dif- ferent baseline methods with our method on ResNet-50,, DenseNet-121, VGG-16 16, and Inception-v3. The results of combinational methods are shown Table 13 The RAP-LS outperforms all com- binational methods by significantly margin. Taking the average attack success rate of all target models as the evaluation metric, RAP-LS achieves 20.9%, 18.4%. and 15.1% improvements over the MTDI, MTDSI and MTDAI, respectively.
Table 13: The targeted attack success rate (%) of combinational methods with RAP. The results with CE loss and 400 iterations are reported. The best results are bold and the second best results are underlined.
ResNet-50
DenseNet-122 121-y
Attack
Dense-122 121
VGG-16
Isc-v/3
Res-50
VGG-16
Inc-y3
MIDI HRAP/ +RAP-LS
45.5/ 78.3 /85.9
29.8 10.5 76.7
A/
20.0
04.07 62.7
9.9/4112/
48.7
2.67 12.5 18.5
MIDSL/ POOOS//RRA/-OOO RRAP/+RAPLLS
7 230/92 93.7
39.9
69.4 76.7
26.9 25.9/453/4 /50.8
30.5
60.5 69.5
14.97428 $2.8 22.8/40 49.7
/2.6 /26.//2 33,5
MTDAI/-RA//-RAE -RAP-LS
00./9941/4 96.1
73.7 61.8
83,4
17.9/99
55.8
68:1 79,3
/ 64.1
/ /4.4
Inc-x3
Attack
Res-St
Dense-12 121
Inc-43
Res-50
Dense- 121
VGG-16
MTDL +RAP/ -RAP-LS
0.5% 10.4/ 12,1
0.1 110/
0.0
2./44.9 5.9
2.21988 11.0
1.244.9 6.7
MIDSE
16.8
/2.22
2
MIDAL/ IRAP JRAP-LS
11.6/ 22.6/ 26.6
20.6/ 32.1 1391
/9.2/ 9.5
6.7 12.5/ /
14.07 22.9 29.2
4..19./ 13.2
15

---

Under review as a conference paper at ICLR 2022
C.2 THE RESULTS ENSEMBLE ATTACKS ON DIVERSE NETWORK ARCHITECTURES
We also take the evaluation of the ensemble attacks on diverse network architecture (Sec We adopt the ensemble-model attack by averaging the logits of different surrogate models, including ResNet-50, DenseNet-121, 121, VGG-16, and Inception-v3. The transfer attack success rate on diverse models are shown in Table 14 Compared with results single model attack in Table the ensemble attack achieve the better performance. We can observe that our RAP-LS further boosts transfcrability of the baseline methods on both targeted and untargeted attacks. We take VIT as target model for example. For untargeted attacks, RAP-LS achieves average performance improvements of 19.2%. For targeted attacks, RAP-LS achieves average performance improvements of 10.4%.
Table 14: The evaluation of ensemble attacks on diverse network architectures.
Untarged NASNet-L
Targeted NASNet-L
Attack
IncRcs-v2
VIT-B/16
IncRes-v2
ViT-B/16
MTDI
98.6
99.3
46.2
65.7
80.1
2.8
MIDI+RAP-LS
100
100
73.2
84.4
89.7
12.7
MTDSI
99.8
100
68.3
81.7
89.4
15.0
MTDSI+RAP-LS
100
85.0
89.8
92.3
25.1
MTDAI
100
70.7
MTDAI-RAP-IS
100
84.6
90.4
91.8
27.8
C.3 THE EXPERIMENTAL RESULTS w.r. DIFFERENT VALUE OF ITERATIONS
In the main submission, report the evaluations of K 400. Here, further report the per- formance with different values of K for completeness in Table 15 (targeted attack) and Table [66 (untargeted attack). From the results, observe that the attacking performance generally increase as increases for most cases, this is also aligned with prior works (Zhao et al. 2020).
Table 15: The targeted attack success rate (%) of all baseline attacks with our method. The results with logit loss and 10/100/200/3000/400 iterations reported. We highlight the results with K 400 in bold.
ResNet-50 + Inception-v3 +RAP
Baseline
+RAP-LS
0.0/0.1 0.1 /011
0.0/0.2/.3 0.3
00.0..//.4/0.6 /
MI
0.1/0.1 /0.1 /0.1
0.0/0.6/ 1.0/ 1.
0.1/0.1/1.4/ 1.6///1///////.
TI
L0/0.3/02/0../// /0.1
0.0/ 0,9 1.2 0.8
0.0/0.3/ 1.3
1.2
DI
0.2/1.2/1.7 1.5///15//11
0.0/38/6.6/77 /7.9
0,2/1.2/ 10.2 /9.41///////// 10.1
0.3/2.6/2//20
0,2/6.68.22 8.60/////.////// 8.6
65/22669.6/9.33 10,5
Admix
9..95//55996600
0.6/14.6/ 16.6 17.1
1./55.7 19.2
MI-TI-DI
1.5/7.9 10.5 10,9
0.1/12.7/223
15/7.9 30.0//3
MI-TI-DI-SI
34.1 36.7 38.1 38.1
33//4.3//4.9/4999953
8.9/ / /5.88//////885.5
MI-TI-DI-Admix
13.5 49.2 50.5
50.8
5.0/48.155.4/5622
13.5/45.1 /6.4////////3/ 63.0/ 64.1
ResNet-50 + DenseNet-121
Baseline
+RAP
+RAP-LS
0.9/5.3/5.0/555 5.5 4,5
0.0/48/7./888 8
0.9/5.3 /.// 14.0/ 14./ 14.3
3.4663/633
0.2/9.0/ 14.1 15.8
3.4/6.3 /8.9///////2///// 28.9
2.5/86/89//.0/79
0.0/7.1/
2.5/86/16.1
164/
6.454..860.44/66
0.1/40.6/S3.2/99.
8.4/44.6 21..6//70.9//.5555 /7.55/
29.6 /0./299///443...4 30.0
4.2/558/500//5555
67/29.6/60.0//..1////
Admix
23.6/55.6/55. 55.55555.555... 55.6///555/55555 54.6
23//6.//66.//// 68.0
23.6/556/744/7777/222
MI-TI-DI
16.3 /6.3/669//144734///
LB//5.7/71.//6//
16.3/66.78855 /5..7///.2/8.////11 88.5
MI-TI-DI-SI
41.0/ 82,8 84.5 86.2/ 86,3
12.9 00.02/85.7/87.3 88.4
41.0/
MI-TI-DI-Admix
48.0 88.7 99.99999999111
20.3 / 0..3/83.//772/8888
88.4
/7.9/88.5/9..5/988////
47.9
16

---

Under review as a conference paper at ICLR 2022
ResNet-50 - VGG-16 +RAP
Baseline
+RAP-LS
1.//7/226/..1/
0.0/ 8.3 9.8 9.8
1.0/2.7/ 114/ 12.8 11.8
MI
12/2.1/24/22//2
0,1/8.6/ 12.3 14.1 /
1.2/2.1 18.2 20,6
TI
1./48 4,8 4.5
0.1/6.0/9. 9.9 12.9
1,1/4.8 14.2 15.3 15.3
7.6/5100 56.9 56.6/57.2 57.2
24/42.4/55.0/6...
7.6/5100 69.3 69.8 69.3
4.4/10.4889/
1.1/7.8/3111
4./1044/ 35.1 36.0
Admix
10.6/249//250/ 62.8
36/4414/445//488844444
10.6/249/51.7/555
MI-TI-DI
/2.1/55.9///00//33/////
L.5/53.//447770.9/
/21/55.8//8.55/85 81.5
MI-TI-DI-ST
27.6/ 68.5 /9.7
6.2/66.//33./775.21
/6.5/6644// /2.4///////////8 / 84.7
MI-TI-DI-Admix
33.4/75.3/775 78.7
14.6/70.4/6.777 /
85.4 86.4 86.3
DenseNet121 - Inception-v3
Baseline
+RAP
+RAP-LS
I
0.0/0.10.///1.. /0.1/////01///
0.0/0.6/ /
0,0/0.1/ 1.31/
MI
0./00.20.3
//.3 /0.3////3///0.0
0.0/1.2/
2.1 2.0
02/0.2/2.5/3.1 3.4
TI
0.0/0.4 /0.5 0.5/
0.0/0.//.6/3.11
DI
0.3/1./ 1.7///////////
/0/4.1/7.0/7../ 8.8
0.//19/93/999 /9.///9//99//// /99/ 10,0
ST
0.3/1.5 L88 1.6
0.1/7.6/9.2 10.0
0.3/1.5 10.7 10.4
Admix
4./55.0/54/555/555
1.7/5.0/ 18.5 18.2/
MI-TI-DI
/2./6/8/7.//.7/777
0.1/ 19.7 /2.2
1.2/6.7 21. 26.2
MI-TI-DI-SI
5.1 / 18.9 19.8
5.2/ 38.9//8////99
MI-TI-DI-Admix
11.4 30.5 /1.//33/2.0
11.2/312/47222
DenseNet121 ResNet-50 +RAP
Baseline
+RAP-LS
18/65/ 5.6
5.0
0.2/7.7 11.2 12.4/ 12.8
1.86655 18.7 /1.77 19.0 17.9
3./54
4.6
0.3/ 14.3 16.3 16.2
3.4/ 23.54/23./// 26.3 26.5
2.6/8.1
8.4
0.2/788/ 10.9 12.1/1 13.5
2.6/ 19.88/919//// 20.2
30./3
0.4/336/44.1/ 52.6
6.3/30.8
SI
7.3 16.5 15.9
14.8 / 14.2
/ /9,//41444415
7.3/ 44.7 /4,8//334
Admix
16.4 32.6 30.3
28.8
29.3
3.7/ (2..3//55.////4//55
16.4/32.6/0... / 58.2
MI-TI-DI
8.3 41.3 44.6/ 46.3
/4..9
/2/44//////////////
/ 72.8
MI-TI-DI-SI
18.6 / / 55.2
6.6/
18.6
/ 75.5
MI-TI-DI-Admix
27.6 66.3
69.7
69.8
69.1
12.1 / /
27.6 66.4 814// 82.0 82.1
DenseNet121 + VGG-16 +RAP
Baseline
+RAP-LS
1
3.5
0.1/6.2/93/ 10.5 10.1
0.6/3.8/ 14.5 15.7/
MI
1,6/ / 2.6 2.73.1
0,2/86/ 12.2 13.0 13.4
1.6/24/ /9.55/21.7//.
TI
1.1
5.8///
0.1/6.//.1/11.// 12.4
1.1/5./
16.5 17.0/ 16.4
DI
4.1/29.8 32.7 3/.7/33.//3333 33.1/32.1
/4..7/44..7//44 49,55
01/29.9552//566 58.9
SI
2.8/9.8/88/
06//558/28.23314//33
2.8998833.5/ 33.33333/33533.
Admix
2022/23.3/22.1/22 /2/2233//221/113/22522
L.7/39.442../45.00
10.2/23.3/ 49.37//49.911 /9.3/4
MI-TI-DI
6.1/32.4/ 36.22////666/// 39.0/
07/362//499/5..2/555
6.1/ 61.866/61../6
MI-TI-DI-ST
12.4 /4.22///22 /42.2/42.0
4.6/
12.4
MI-TI-DI-Admix
20.0/ 53.2 / 55.7
9.4/448/ 60.1 /61.//////////.. / 63.1
19.9 /9.9/53.1/
/9.7//999///
VGG-16 Inception-v3 +RAP
Bascline
+RAP-LS
0.0/0.0/ /
0.0/0.1 /..0/.10111001//
0.0/0.00..2/ 0.0
0.0/0.0/0.0 /0.0/0.....
0.0/0.//..//0.0/
0.0/0.0/0./
TI
0.0/0.0 /.0/ 0.1 /0.0
0.0/0.1 0.1 /0.1 0.1
20/0.0/0.// 0.4///////44/00
DI
0.0/ 0.0 /.0///////////////00///0/0000//0 /
0.0/0.0/ /0.6
0.0/0.0/ 0.7 0.7 1.1
SI
0.0/0.4/.3/ /..
0.0/2.0/ 1.5
0.0/0.6/ 1,6 1,9 1.8
Admix
0.1 / 0.8 /0.6 0..7
0.0/2.7 2.3
0.1/ 2.3 2.3223333300 3.0
MI-TI-DI
0.1/10/ 00. 1.7 7.7 /9.1 /9,8
0.//8//28/3.0/5
0.1 1/1.9/3.44/..00
MI-TI-DI-SI
0.6/ 12.2//4455/// 13.4//13///// 13.8
/.7/8.6/
12.//221/13.
MI-TI-DI-Adme
3.6 12.4/ 12.2/115/11.
1.//14.5/1 16.1 15.9
3.4/ 11. 17.9/
17

---

Under review as a conference paper at ICLR 2022
VGG-16 - ResNet-50 +RAP
Baseline
+RAP-LS
I
0,2/04/0.3 0.3 / 0.1
00/1.0/0.8/00 0.8//0///077 0.7
0.2/0.57 14/ 1.5/
MI
0,4/0.5 / 0.6 /.55
0./1.1/133 1.3
0.4/0.2/21 20..2/11//4/111 2
TI
0.3/ 0.7 0.9 0.7
0,0/14/ 1,5 14/
03//L//30/33/3/
DI
0.5/ 2.//288
0.0/49/ 6../7.3
0.5/39/9.5/ 10.1
SI
1.4/44/3 3.3
0.4/92/9.0/9.1
144/4./ 10.1
Admix
4.6/7.3 6.7 5.6
0.7/ 10.9//11.
4.7/7.3/11.
MI-TI-DI
188 10.2
11.9
0.0/ 15.6/ 15.7/
20.2/ L.8/9.//002/11.6
MI-TI-DI-SI
30.1 3.81
31.6 31.0
3.2/30.8 / 32.5 33.5
29.5
36.9 38.5 38.7
MI-TI-DI-Admix
15.2 34.6/ 35.1
5.4334.7/333 38.11 /
15.3 /5.3/ 43.2///33//422 43.1
VGG-16 + DenseNet-121
Baseline
+RAP
+RAP-LS
I
2./002/0044 /03//02
0,0/0.7 LT /0.7
0.1/0.3 1.5/
MI
0.3 / 0.6 0.5
0.0/1.1 1.4 2.1 2.3
23/06//2/33.2
TI
0.1 1.1 1.0/ 0.8
0,0/0.9 /66 1.7
0.1/0.9/255/.7111
DI
0.2/3.8/4.8/ /...///////31/.8
/.//5007.6//788 8,4
02/3.7/1199 12.2 12.7
ST
1.3/9.0/8.9/ /.7
0.3/ 15.6 16.4 16.8
1.3/ 17.07 17.4 17.8
Admix
14//14//1///132/
0..9 20.5// 20.2/
4,9/ 14.0 / 24.2 23,6
MI-TI-DI
1,5 13.41/13///11 13.9/
0.1/9.7 15.7 /
1.6/ 24.1 26.3
MI-TI-DI-SI
13.0 38.9 AL.00889/455/422//444
3.8/3788 /220/ 43..//3.///// 44.4
12.8/37.3 48.6 49.8 49.6
MI-TI-DI-Admix
4.8/413 44.8/4544
/ 52.91 55.9/55.2
Inc-v3 -> ResNet-50
Baseline
+RAP
+RAP-LS
0.2/04/03/0.1/0/
0.0/0.200./000 /06
0.2/04/1.0/07 0,5
0.1/..3/.3/0./000
0.0/0.6/ /4/ 1.5/77
0.1/0.3008/1.0 1.6/ 1,5
0.2/33/02/022/02
0.0/02/0.//09//
0.2/0./11//0.//00
0.2/1.5/
0.1/25/43/43/45
0.2/1./5.055.1//
SI Admix
0./1.9/26/22/22/
03/0.3/24/228/ 2.5
1.2/1.9/2.2 1.9
0.6/5.0/4.9/5.///
02/1.9/5.7/5.7//
MI-TI-DI
0.6 1.5/4./555/5./55
60/4.2/6.//7.//88
0.6/ 6.2 7.0/ 7.5
MI-TI-DI-SI
0.7/8.6/ /0.3 /1.9
5.0 / 10.7
MI-TI-DI-Admix
2.8 8.9 9.5 /9.6/
L.4 12.6/ 14.0/ 13.6/ 13.6
2.8 /866/ 14.5/ 15.1 16.7
Inc-v3 + DenseNet-121
Baseline
+RAP
+RAP-LS
00/00/02/0.0/000
0.0/02/0.4 0.6 0,6
0.0/0.0/02/ 0.4
MI
0./00.102/00.1 /0.1
0.1/0.7/1.//..
0,0/0.1 1.0/ 1.1 1.5
TI
0.0/0.3022 /0.0
0.//3//33//33/0 /0.
0.0/0.3/0.9 /.9
DI
0.1/13/2.53.01
00/27/44/54/55.
0.1/13/5.9/7.//
0.2/0.7/09/0.8/00
00//2/3.3//2//2
0.2/0.7/3.2/3..1
Admix
1.1/26/2.5/2.3222
0.5/7.27.7 /7.0///
1.1/2.6/8.
MI-TI-DI
0.5/3./38/455/4.
0,2/54/108
12.6
0.5/3.3/
MI-TI-DI-ST
1.9/9.0/94//.55/00
1./1
19.8
1.9/9.0/ 19.1 /
MI-TI-DI-Admix
1.67 16.8 17.9
24/23.2/ 26.6
26.6
Inc-v3 VGG-16 +RAP
Baseline
+RAP-LS
0.0/0./0.1/000
0.0/022088/0/ 0.6
0.0/0.30.2/055 0,5
A.1/0.1/02/00
0.1/0.4/08//
0.1/0.1/04/08// 1.0
0.1/0.2/02/0.1//
0.1/0.4/0.5 /
0.1/0.2/.6/0.6 0.6
DI
0.//2/28/223/2
0.1/18/4.3/5.2//
0./220/6.8/733 8.1
SI
0.0/0.7006/0444
02/20/115/1.5//
0.0/0.7 2./0.7/1.6/2.3/ 2.3
Admix
0.5/ 1.0/ 1.3
04/3.2/3./4./25.
55/1.6/4.53..8 4.4
MI-TI-DI
0.1/3./6.55773//88
0,3/2.0/8.0/ 2.0.0//../7.////
MI-TI-DI-SI
0.7 /
0.5 0.5/7.67.5/8.5/6
0,7/3.2 /.17//8.//8.
MI-TI-DI-Adma
2.3/6./8.3/8.//
L33 10.5/ 12.0/
2.3 11/0/119//100
18

---

Under review as a conference paper at ICLR 2022
Table 16: The untargeted attack success rate (%) of all baseline attacks with RAP. The results with C' loss and 10/1000/000330/400 iterations are reported. We highlight the results with K = 400 in bold.
ResNet-50 - Inception-v3 +RAP
Baseline
+RAP-LS
I
35./3555/5... 34.7 34.6
34,6
12.3/48.3 54,1 55.5 57.00 57.0
25.7/ 54,1 57.2
MI
53.2/ / 80.6 50.3
26.2/ 68.9 73.4 75,9
63.2/50.7/6433 013/7//6.3773//////
30.0/45.3/ /4.0/ 45.3 44.5
16.4/ 63.9 64.6 66.1
30.0/45.1/6..
67.0
27.31 82.8/
85,0
/0.1/66.0/654 /1./666666666666600) 65.99
60.6 80.5
80,9
49.9 85.2 84,4
Admix
66.6/78.777..2/8000
73.9 83.9/87.6/ 86.8 87.4
67.6/79.4 /2.33/99
MI-TI-DI
/ 255.8/86.4/853. 85.7
61.9 95. / 96.0
82.1 85.8 95,9 / /
MI-TI-DI-SI
94.2 / 96.8 99.. /9.00
92.3 98.9 98.9 99.0
94.2 96.7 99.0 /9.1
MI-TI-DI-Admix
97.3 / 98.6 98.5 98.5 98.3
95.1 / 99.4 / 99.4 99.3 /2993/9992
97.3 98.5 99.8
99.8 99.8
ResNet-50 DemseNet-121 +RAP
Baseline
+RAP-LS
77,4/ 79.1 79.0
79.2
26.7 / 84,8 /9.1 90.8 / 91.5
67.8 80.1 89.8
91.3 91.9/9.99
MI
87.3/ 85.485,4/44 86.4 85.9/ 85,8
45.2 85.3 91.3 93.9 95.0
87.3 85.4 90.8 /0.8 / 96.1
TI
73.2/83.0 82.2 / 81.6/
82.0
30.9 / 87.3 / 93.3 94.1
72.9 /2.4/90.2 /4.2 95.1
DI
92.8/98.9 98.9 / 99.2 /9.0///999//// 99.0/ 99.0
52.6 99.0 92.6/ 99.6 / 99.7 99.6
42.8 99.0/9.0//0 99.6 99.7 99.7
/9.1/95.71 / 99.6
/ 94,9
91.3/96 98.9 /9.0/ 99.2 98.9
89.1/95.7 / 99.7 99.7 99.7
Admix
96.6/ 98.5 98.1 9779
96.2 99.6 99.6 99.6 / 99.6
96.4 99.9 99.9
MI-TI-DI
98.21 99.7 99.7 99.8 99.8 99.8
86.4 99.9/ 100 100
98.2 99.7 99.9 100
MI-TI-DI-SI
100
98.8 100 100
99.8
100
MI-TI-DI-Admix
99.9 100 100
100 100
99.5 100 100 100 100
99.9 100
100
/
ResNet-50 VGG-16
Bascline
+RAP
+RAP-LS
68.2/77.4 77.4 78.0
36.2 84.6/ 89.2
191.
68.4/77.3/ 87.1 /90.
82.5/82.8
82.9 82.7
82.4
53.1 85.5 92.2 93.1
93,9
82.5 82.8 89.3 /93.7 94.5
TI
70.6 80.5 79.8 80.8
81.0
39.3 86.9 90.6 / 92.5 93.1
/1.1/80001
89.0/9900/0 /9.9 /3.3
DI
92.3/ /9.1
99.0
99.0
64.4 99.4 99.7 99.7 99.6
92.3/99.1 99.8 99.9 99.7
SI
82.2/90.0 88.9/ 89.6 88.6
81.3 95.7 95588 95.7
95.7
82.1/89.3/97.3
/9.8
Admix
92.3/95.4/966
95.6 95.8
91.6/97.99 / 97.8 97.7
92.7 98.9 /
MI-TI-DI
97.9
99.7 99.7
99.7 99.8 99.8
85.9 99.5 100 100 100
/ 99.9 99,9
MI-TI-DI-SI
99.1 / 99.8 / 89.899988/ 99.7 99.7
97.4 99.7 99.9 99..99999999999
99.1 99.8 99.8 99.8 99.8
MI-TI-DI-Admi
99.2 99.8
99.8 99.8
98.5 99.7 99.9 99.9 99.9
99.2 99.8 99.9
99.9
DenseNet-121 Inception-v3 +RAP
Baseline
+RAP-LS
31.2
48.5 48.55
46.9 46.3/ 46.5
18.0 54,9 58. 59.8 60.2
31.6/
58.9 61.0 61.1
58.8 59.3 60.6/ 59.3
32.2/ 6222/65.6/10 74.1 78.9 80.4
56.8/
80.0
37.7 55.1
54.6/
20.4/ 6/.4/61./// 64.7 67.3 66.7
/5.2/54.5/65.4//
70.0
51.0/ 66.7
31.4/ 84.0/ 86.8
86.6
51.0 89.0 88.8 86.9
24.7771./77.//70.11
61.1 82.9 83.1 83.5 83.2
53.9/7L0/ 86.4 87.0 87.4
Admix
72.5/ 82.0//2..//////// 82.2/82.0
73.0/ 89.9 90.3 89.5
89.8
71.7 993.9//33.//// / 93,88
MI-TI-DI
81.5 89.7 89.8 89.4 89.1
62.5 / 94.8 96.8 97.1 /97.1
81.5 89.6 96. 96.9 97.1
MI-TI-DI-ST
92.3 95.2 94. 95.1 / 99.1
88.6 97.7 98.0 98.0 98.3
92,4 97.8 98.4
MI-TI-DI-Admix
95.8 97.7 97.2 97.3 97.9
93.2 / 98.6 98.6 99.0
99.6
98.8
95.4 97.6 99.0
98.9
DenseNet-121 - ResNet-50 +RAP
Baseline
+RAP-LS
76.1/88.0/ 87.5
/ 87.4
35.7 90.1 /93.5/ / 94.2
76.1
88.0 91
92.9 94.3
MI
87.7/90.5 99.99 90.8 90.3
55.6 99. 96.2
97.6
87.7 95.4 /90.5 /
97.2 97.9
TI
79.2/ 90.4 90.0 89.9 89.6
36,9 / 90.1 /
94,2
79.0/ 89.8 000.8/90 92.7 /4.3 94.8
DI
91.1/98.00 98.3 98.2 98.2
57.0 98.6 99.3 99.7 19.66
91.1/98.0 99.5 /9.6/ 99.7
99.6/95.2 94.8 99.3/
83.0 96.5 96.7 96.3 /
89.4/95.0/ /
Admix
96.3 /
90.9 98.8 98.8 99.0/99. 99.0
/ 99.3 /9,2
MI-TI-DI
9 / 99.4
99.4
84.4
99.8 99.8/999
96.3/
MI-TI-DI-SI
98.3 99.7 99.8 99.8 99.8
95.8
99.7 99.9 99,9 99.9
98.3/99.7 99.9
MI-iI--I-Admix
99.2 99.7 8 99.8 99.8
97.9 99.9 99.8 / 99.8 99.8
99.0 99.7 99.9 99.9
19

---

Under review as a conference paper at ICLR 2022
DenseNet-121 - VGG-16 +RAP
Baseline
+RAP-LS
75.1/8477 85.2 84.9/ 85.1
42.2/ 87.5 / 90.7 /9.2 201.2//1111 91.7
75.1 /
/9.. /
MI
87.1/87.2 88.6
87.9
87.5
58.4 90.2 / 95.4/9
95.1 96.0
85.1
TI
74.4/ 86.3
86.4 / 87.0
44.2/87.8 87.8 89.6 91.0 92.1
7 93..8/99.399..22 93.3
90.8/ 98.0 98.4 0984//988 / 98.1
63.3/98,8m 99.2 / /9.69999/999 /994
90.8/97.9/
84.2/91.5
/ 94.. / 95.2/95.0
83.9/966/ /9.11.999999///999
Admix
93.5/95.7 /9.0/ / 95.6
87.8/97.//9.55
99.69 98.7
MI-TI-DI
95.1 / 99.2
99.2
84.2 99.1 / /9.5
95.1 /9.00 99.9
100
MI-TI-DI-ST
97.9 99.5 99.4 99.4 99.2
93.3 / 99.0 99.2 99.3
/
99.7 99.7 99.7
MI-TI-DI-Admix
98.4 99.4 99.4
99.5 99.4
96.1 / 99.7 99.7
99.6 99.6
98.3 99.4 99.8 99.7
VGG-16 Inception-v3 +RAP
Baseline
+RAP-LS
/4.3/22.2/220/ 22.2/ 22.0
9.4/238/ 23.7 24.7
14.4/218/
25.4/ 24.9
MI
32.3/31.3/1.001
30.1 30.0
16.4/ /3.4//6////////// 42.0 42.7
32.4/ 35.0 42.2
TI
3
29.7
29.1
11.9/32.1 /33.7//35333339 36.2
18.3/ 29.3 / /6.0/
DI
30.4
29,9
14.2/43.6/46.1 / 46.6
18.0 29.2 50.1 / /5.551551//////
SI
31.0/ 45.1 /46.1
44..
/
458
46.7 06.7/7../72.0/77 74.0
74,3 74.7 31.0/ 73.0 73.0/
Admix
40.21 / 55.5 /
55.5
57.0/78.0/7.66/7..99
41.4/ 80.0 80.0 79,9 80.8
MI-TI-DI
50.7 55.9 57.2 56.7
56.7/ 77.8/ 80.0 81.4
MI-TI-DI-SI
77.6 85.3 85.7
/5.0//8550/00 85.0
/ /99.7//99.9//..22///
78.0 85.0 94.4
MI-TI-DI-Admix
89.4 89.2
89.9/ 89.3
88.4/ 984//449/95.1
/9.4/ 95.5
VGG-16 + ResNet-50 +RAP
Baseline
+RAP-LS
37.2/ 534 53.1//37
17.8/48.51
53.9 / 53.7 53.0
/ $2.4 548 54.2
64.3 63.5 62.0/ 62.5
32.9
67.6 73.1
76.2
60.4 62.0/ 66.3 /3.2
62.3 63.6
62.5
62.8
/9.4/ 63.0 65.6 64.8
46.0/ 62.9 63.5 65.58/658///6
51.5/
72.2
29.6 /
86.0
51.4/73.8
SI Admix
64.6 80.2
80.5 80.0
99.1999./92.311 92/3/92///9999
64.9/ 80.6 95.1 99.3 94.7
76.8 87.5 88.2
88.0
87.3
/ / 94.4 95.2 94.6
77.6 78.6/88.3 96.6 96.8 / 96.8
MI-TI-DI
81.1 89.9 89.8 90.3
90.0
66.7 / 94.6 96.3 96.9 /97.2
81.4 88.5 /
MI-TI-DI-SI
95. 97.6
98.0
97.9 97.6
94.7 / 98.4 98.8 98.9/ 98.8
/
99.3 99.4
MI-TI-DI-Admix
97.2 98.1 98.0/
98.1 97.8
96.1 / 99.1 /9.2/99.99.. /9.2
97.3 98.6/ 99.5 99.6
VGG-16 + DenseNet-121 +RAP
Baseline
+RAP-LS
35.4/ 49.8
48.4
49.1
15.4 46.0/ 49.6/ 50.5 50.6
55.2/50.3/499
/2.9
MI
/2.1/
62.8 61.7 60.5
26.6 51.1 /3..//0.001 73.0
62.6//2.55/ 62. /
TI
58.6/58.6/4
58.7
57.2
55.9
/ / 63.0 63.7
44.3/ 60.33/60.3/// /3.8///33388
DI
48.1/70.2 68.9 70.0/
68.8
79/9999//222 84.22/844.// 84.2 85.0 26.5 26.5/
88.0
47.9/ /
87.2
/ /1..8/./.22///2 22.4//2//2000
82.1
9../99..//3../99... 99.47/9944/9..8
22..2/99550 /5.4//
Admix
79.6/89.4/ 89.4 88.6 88.4 88.2
83.5 83.5/96.1
/95.9 / /9.4
79.2
MI-TI-DI
80.3 87.0/ 88.7 /
62.9 95..9/9999999949....
80.4/ 86.8 96.8
MI-TI-DI-ST
/
4
95.9
/9.2 /9.2
95.4
99.5
MI-TI-DI-Admix
97.1 98.6
98.8 99.1 98.9
97.4 99.4 99.6
99.6 99.5
/ 98.5 99.5
Inc-v3 ResNet-50 +RAP
Bascline
+RAP-LS
34.0/ 58.5/ 60.3 62.0
51.2
60.9
62.1
34.5/49.0
60.5
MI
43.8/
84.0
85.8
58.5/
84.8
TI
33.6/46.9 48.7 48.5
49,3
21.8/ 58.9
60.2 61.7
63.4
33.1/47.2/50 5//44////99//// /1.55/66// 61.6
DI
4 67.2/ 68.4
$3.3/78.58 /
81.4
/81.
81.7
48.3/65.7 / 82.3
81.88
SI
66.2
4 69.4/ 69.5 69.8
/16//6../7..//77 72.8
Admix
56.1 /5..00//5..9/
2..1/73.0//5503
015.00/75.9/669//1 76.9
77.0/77.5/99.0
80.3 80.2
56.3/ / 82.9 84.0 84.9
MI-TI-DI
72.2 /2./7995/88///1999 82,9
61.2 88. 99.99/99//99/99999999
72.2 79.4 89.9
90.6
MI-TI-DI-SI
82.9 88.3 3 //
83.5 / 90.8 91,2 90.6 /91.2
82.8 88. 9919 91.9999..... 92.6/ 92.3
MI- MI-TI-DI-Admi
89.8 99./99.6//91//994//91
89.0 93.9 99.0 940/9999/940///4///99
89.6 92.3 94.1 94.7
20

---

Under review as a conference paper at ICLR 2022
Inc-v3 DenseNet-121 +RAP
Baseline
+RAP-LS
35.2/ /41 46.8
48.7
21.4/
54.3
59.4 60.8
34.9/4.55 58.7 58.8/ 60.0
58.8
57.4/ 56.5 56.8
56.7
42.9/74.0/ 80.1 82.5 84.6
/7.4/562/744/899//
35.8/4886/47.8/
49,4
22.1/59.6/633//66 63.4
35.5/48.7/61.66
36.2/72.1/718/7155/711
35.7/81.9/83.7855
58.2/718884.11
40.6/66..7665.1/6533 65.9
526/72.4/735/74.51
46.6/63.077../7T.
Admix
60.5/76.777.0/799 78.5
MI-TI-DI
/5.77 85.7
65.1 /9..5/ 99..9999/////////// /9.0/ 94.2
MI-TI-DI-SI
99.0/91.9/ /1.9 /1.7 92.0
89.0 / 94.7 / 95.2 95.2
/9.0/914 /9.199449/11//9 /95.1/95.//955
MI-TI-DI-Admix
93.5 95.5 /9.1
93.3 / 96.8 96.9
96.4
96.2
/
Inc-v3 + VGG-16
Baseline
+RAP
+RAP-LS
39.9/53.1 /5./8..//55.1558755555 55.7
29.1/63.0/65
66.9 65.9
39.7/52.6/65.4
68.3 68.0
63.7/66../6.8862.1111 63.1
50.7/76.1
83.6/ 84,9
60.7/62.2/798//
840/
41.6/55.1552255533
31.1/65.9/67.1
68.6
41.5/55.16663//666
$4.9/7.4/745/76.00
44.4/83.4/84777 /5.0//8550//// 85.0/ 85.2
54.9/73.088../88..21
SI
66.7/62.4/444/65.7 66.0
47.4/ /9.666666626/66 68.6 69.2
4633/441172.4/7...
Admix
57.3/7.2/728/7A./ 74.5
71.3/75.475.9//7.5.
55.3/73.4/82.611 82.2 83,5
MI-TI-DI
74.7 82.7 82.7 84.7 84.6/ 85.1
67.7 / 90.0/ / 92.7
74.7/82.5 / 90.8 91.0
MI-TI-DI-SI
79.8 88.0/ 87.5/ 87.6
81.6 89.0 89.4 89.4 90.3
79.7 /1..8//922444 99..//92////2.. /9..5 / /2. 92.9
MI-TI-DI-Admix
87.9
87.0 /2.5//92//2/////// 93.2
87.7 /917 /9..5//994////..66
21


Reviews:
[{"review_id": "wONHWi9SEC", "reviewer": "Reviewer_b3AS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The writing is clear and the proposed method is easy to follow.\n- The idea to seek a flat region of loss landscape to improve the transferability is novel. Instead of purely minimizing the adversarial loss at a single adversarial point, the paper proposes to find a flat region of loss landscape by bi-level max-min optimization, so as to eliminate the overfitting problem of adversarial attacks.\n-  The experimental results demonstrated the effectiveness of the proposed method. By injecting the proposed attack method into the existing attack method, the paper improves the transferability of adversarial examples by a large marge in both untargeted attacks and targeted attacks. \n\nWeaknesses:\n- The evaluation of defense methods is somewhat limited, the paper only considers the ensemble adversarial training, and many advanced defense methods are lost (such as feature denoising [1] and NRP [2]).\n- A drawback of the proposed method is that it requires to conduct T steps update in the inner maximization, which increases the overhead of adversarial attacks.\n\nSuggestions:\n- In section 4.2, the paper plots the visualizations of loss landscapes of targeted attacks,  the loss landscapes of untargeted attacks are also needed.\n\n[1] Feature denoising for improving adversarial robustness. CVPR 2019.\n[2] A self-supervised approach for adversarial robustness. CVPR 2020.", "summary_of_the_paper": "This paper focuses on the transferability of the adversarial examples and proposes to boost the transferability by reversing the adversarial perturbation. The motivation is that the flatness of the loss landscape can help to alleviate the overfitting to surrogate models, and thus improve the transferability. Specifically,  instead of purely minimizing the adversarial loss at a single adversarial point, this paper injects the worst-case perturbation for each step of the optimization procedure. Experimental results show that the proposed method surpasses the SOTA transfer-based attack methods by a clear margin, which demonstrates its effectiveness. ", "summary_of_the_review": "Overall, the paper proposes a novel adversarial attack, which is effective and easy to follow. I think the paper is marginally above the acceptance threshold, although there are some weaknesses.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "pwLEElQmt0", "reviewer": "Reviewer_5ohk", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is easy to follow and the motivation is clear. However, the technical novelty of the paper is limited. The proposed attack method mainly leverage the minmax framework to search for regions that may have high adversarial transferability loss and then minimize the loss for those regions so as to achieve “flat” loss regions. The algorithms to solve the minmax optimization is standard and follows the existing work. There is no convergence, or any transferability guarantee.\n\nEmpirically, from table 3, the targeted attack success rate improvement is usually not significant. Since the paper aims to propose a more transferable attack, it would be interesting to see if it can attack against existing defenses, including the gradient obfuscated ones comparing BPDA attack [1] and adversarially trained models. \n\n[1] Athalye, Anish, Nicholas Carlini, and David Wagner. \"Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples.\" International conference on machine learning. PMLR, 2018.\n", "summary_of_the_paper": "This paper proposes an adversarial attack, RAP, to boost the transferability of adversarial examples, based on the intuition of flatness of loss landscape and model generalization.\nExperimental results show that the proposed attack is more effective against real world APIs.\n\n", "summary_of_the_review": "Overall the paper is well motivated, and the problem statement is clear. However the technical novelty is limited and more baseliens on improving adversarial transferability need to be compared with.\nIt would be interesting to see if the proposed method can boost the adversarial transferability for different tasks such as object detections as well.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bWzAWk4G_Bc", "reviewer": "Reviewer_Mzh9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Paper is well written, evaluation is reasonably good and thorough. Results show that method helps to improve transferability of adversarial examples.\n\nNevertheless there are few things in evaluation that could be improved:\n\n1. Authors use I-FGSM as one of the baseline attacks, but it’s not clear whether they do random restarts (as described in https://arxiv.org/abs/1706.06083 ). It would be useful to clarify this and if authors don’t do random restarts then add an attack with random restarts.\n\n2. Most evaluation is done on undefended models. It would be interesting to study attack performance when source and/or target model is adversarially trained. While authors mentioned few defended models from Tramer at al 2018, it has been shown later that ensemble adversarial training is not particularly strong defence. It’s much better to perform multi-step PGD adversarial training, like in https://arxiv.org/abs/1812.03411. Note that it has been shown that denoising used in https://arxiv.org/abs/1812.03411 is not a good defense, nevertheless authors do produce an adversarially trained model without denoising.", "summary_of_the_paper": "Paper proposes a novel adversarial transferability attack (i.e. an adversarial attack when a surrogate model is used to attack an unknown model). Proposed method works by modifying iterative proceduce to find adversarial examples, such that it tend to find adversarial examples in flatter regions of a loss surface.\nAuthors conduct thorough study of the method.\n", "summary_of_the_review": "Reasonably good paper. There are few potential improvements for evaluation procedure.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_zHtItuvMFP", "reviewer": "Reviewer_PVqn", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n+ The bi-level min-max formulation is novel for generating transferrable attacks. \n+ There exist extensive experiments involving a variety of model architectures and baselines. \n\n\nWeakness:\n\n- The flatness of loss landscape with respect to (w.r.t.) input vs. model generalization should be more carefully studied. My doubt arises from the paper [https://openreview.net/pdf?id=BylKL1SKvr]. This work showed that model transferability (from the source domain and the target domain) relates to the smoothness of loss landscape, but this conclusion holds for loss landscape w.r.t. model parameters. I understood why authors draw a connection between the flatness of loss landscape and model generalization in the context of attack generation. However, has this been well studied and believed as a grounded conclusion?\n\n- The use of a min-max formulation for attack generation has been studied in [https://arxiv.org/pdf/1906.03563.pdf]. This related work should be covered and discussed in Related Work. \n\n- If the min-max formulation (2) is replaced with EoT (expectation over transformation), namely, n^{adv} is randomly generated from e.g., Gaussian distribution, then this will lead to an EoT-type baseline. It will be better to cover this baseline as well to demonstrate the superiority of bi-level formulation.\n\n- Min-max attack generation could be difficult to tune. Thus, the computation time and hyper-parameter setups of the proposed approach should be clearly stated. \n\n- If the n^{adv} is regarded as model noise to flatten the loss landscape, then how does the attack perform compared to RAP? This is another baseline to verify the usefulness of the flatness of loss landscape w.r.t. `input' rather than 'model parameters'.\n\n\n\n\n\n", "summary_of_the_paper": "This paper proposed a min-max formulation to improve attack transferability. The key idea is motivated by the fact that the smoothness of the loss landscape could improve model generalization ability.  Thus, a reverse adversarial noise (for landscape smoothing) is injected so as to compensate for the effect of an adversarial attack. Numerous experiments are provided to demonstrate the effectiveness of reverse adversarial perturbation (RAP). \n\n", "summary_of_the_review": "I think this is an Okay submission, but several technical and experimental questions remain. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies the transferability of adversarial attacks in deep neural networks. In particular, it proposes the reverse adversarial perturbation (RAP) method to boost attack transferability by flattening the landscape of the loss function.

The reviewers acknowledge the strengths of the paper, which include effectiveness of the simple RAP method proposed and the extensive experimentation presented.

However, a number of outstanding concerns still remain. Some of them include the technical novelty of the paper, insufficient theoretical justification of the proposed method, lack of grounded justification between flatness of the loss landscape and model generalization under the specific context of attack transferability, similarity of the optimization problem with some existing work, potential difficulties of the min-max attack generation problem, among others.

As it stands, this is a borderline paper that is reasonably good, but not great. Addressing the outstanding concerns will make the paper more ready for publication in ICLR.
Decision: Reject