Boosting the Transferability of Adversarial Attacks with Reverse Adversarial Perturbation
AffiliationAddress
Abstract
Deep neural networks (DNNs) have been shown to be vulnerable to adversarialexamples, which can produce erroneous predictions by injecting imperceptibleperturbations.
In this work, we study the transferability of adversarial examples,which is significant due to its threat to real-world applications where model architecture or parameters are usually unknown.
Many existing works reveal thatthe adversarial examples are likely to overfit the surrogate model that they aregenerated from, limiting its transfer attack performance against different targetmodels.
To mitigate the overfitting of the surrogate model, we propose a novelattack method, dubbed reverse adversarial perturbation (RAP).
Specifically, instead of minimizing the loss of a single adversarial point, we advocate seekingadversarial example located at a region with unified low loss value, by injectingthe worst-case perturbation ( i.e ., the reverse adversarial perturbation) for each stepof the optimization procedure.
The adversarial attack with RAP is formulated asa
min-max bi-level optimization problem.
By integrating RAP into the iterativeprocess for attacks, our method can find more stable adversarial examples whichare less sensitive to the changes of decision boundary, mitigating the overfittingof the surrogate model.
Comprehensive experimental comparisons demonstratethat RAP can significantly boost adversarial transferability.
Furthermore, RAPcan be naturally combined with many existing black-box attack techniques, tofurther boost the transferability.
When attacking a real-world image recognitionsystem, i.e ., Google Cloud Vision API, we obtain 22% performance improvementof targeted attacks over the compared method.
Introduction
Deep neural networks (DNNs) have been successfully applied in many safety-critical tasks, such asautonomous driving, face recognition and verification, etc .
However, it has been shown that DNNmodels are vulnerable to adversarial examples [7, 25], which are indistinguishable from naturalexamples but make a model produce erroneous predictions.
For real-world applications, the DNNmodels are often hidden from users.
Therefore, the attackers need to generate the adversarial examplesunder black-box setting where they do not know any information of the target model.
For blackbox setting, the adversarial transferability matters since it can allow the attackers to attack targetmodels by using adversarial examples generated on the surrogate models.
Therefore, learning how togenerate adversarial examples with high transferability has gained more attentions in the literature[2, 3, 8, 17, 28, 34].
Under white-box setting where the complete information of the attacked model ( e.g ., architecture andparameters) is available, the gradient-based attacks such as PGD [18] have demonstrated good attackperformance.
However, they often exhibit the poor transferiability [2, 34], i.e ., the adversarial examplex adv generated from the surrogate model M S performs poorly against different target models M T .
The previous works attribute that to the overfitting of adversarial examples to the surrogate models[2, 15, 34].
Figure 1 (b) gives an illustration.
The PGD attack aims to find an adversarial point x pgdwith minimal attack loss, while doesn’t consider the attack loss of the neighborhood regions roundx pgd .
Due to the highly non-convex of deep models, when x pgd locates at a sharply local minimum,a slightly change on model parameters of M S could cause a large increase of the attack loss, makingx pgd fail to attack the perturbed model.
Many techniques have been proposed to mitigate the overfitting and improve the transferability,including input transformation [3, 34], gradient calibration [8], feature-level attacks [11], and generative models [21], etc.
However, there still exists a large gap of attack performance between thetransfer setting and the ideal white-box setting, especially for targeted attack, requiring more effortsfor boosting the transferability.
In this work, we propose a novel attack method called reverse adversarial perturbation (RAP) toalleviate the overfitting of the surrogate model and boost the transferability of adversarial examples.
We encourage that x adv is not only of low attack loss but also locates at a local flat region, i.e .,the points within the local neighborhood region around x adv should also be of low loss values.
Figure 1 (b) illustrates the difference between the sharp minimum and flat minimum.
When themodel parameter of M S has some slight changes, the variation of the attack loss w.r.t.
the flat localminimum is less than that of the sharp one.
Therefore, the flat local minimum is less sensitive to thechanges of decision boundary.
To achieve this goal, we formulate a min-max bi-level optimizationproblem.
The inner maximization aims to find the worst-case perturbation ( i.e ., that with the largestattack loss, and this is why we call it reverse adversarial perturbation) within the local region aroundthe current adversarial example, which can be solved by the projected gradient ascent algorithm.
Then, the outer minimization will update the adversarial example to find a new point added with theprovided reverse perturbation that leads to lower attack loss.
Figure 1 (a) provides an illustration ofthe optimization process.
For t -th iteration and x t , RAP first finds the point x t + n rap with maxattack loss within the neighborhood region of x t .
Then it updates x t with the gradient calculatedby minimizing the attack loss w.r.t.
x t + n rap .
Compared to directly adopting the gradient at x t ,
RAP could help to escape from the sharp local minimum and pursue a relatively flat local minimum.
Besides, we design a late-start variant of RAP (RAP-LS) to further boost the attack effectivenessand efficiency, which doesn’t insert the reverse perturbation into the optimization procedure inthe early stage.
Moreover, from the technical perspective, since the proposed RAP method onlyintroduces one specially designed perturbation onto adversarial attacks, one notable advantage of
RAP is that it can be naturally combined with many existing black-box attack techniques to furtherboost the transferability.
For example, when combined with different input transformations ( e.g ., therandom resizing and padding in Diverse Input [34]), our RAP method consistently outperforms thecounterparts by a clear margin.
Our main contributions are three-fold: 1)
We propose a novel adversarial attack method RAP thatencourages both the adversarial example and its neighborhood region be of low loss value.
2) wepresent a vigorous experimental study and show that RAP can significant boost the adversarialtransferability on both untargeted and targeted attacks; 3) we demonstrate that RAP can be easilycombined with existing transfer attack techniques and outperforms the state-of-the-art performanceby a large margin.
Related Work
The black-box attacks can be categorized into two categories: 1) query-based attacks that conduct theattack based on the feedback of iterative queries to target models, and 2) transfer attacks that use theadversarial examples generated on some surrogate models to attack the target models.
In this work,we focus on the transfer attacks.
For surrogate models, existing attack algorithms such as FGSM [7]and I-FGSM [14] could achieve good attack performance.
However, they often overfit the surrogatemodels and thus exhibit poor transferability.
Recently, many works are proposed to generate moretransferable adversarial examples.
Instead of the I-FGSM, the work of Dong et al.
[2] integrate momentum into the updating strategy and
Lin et al.
[15] use the Nesterov accelerated gradient to boost the transferability.
Data augmentation,which has been shown to be effective in improving model generalization, has also been widely studiedin transfer attack, such as randomly resizing and padding [34], randomly scaling [15], and adversarialmixup [30].
The work of Dong et al.
[3] use a set of translated images to compute gradient and get the better performance against defense models.
There are also some model-specific designs to boostthe transferability.
For example, Wu et al.
[31] find the gradient of skip connections is more crucialto generate more transferable attacks.
The work of Guo et al.
[8] also propose LinBP to utilize moregradient of skip connections during the back-propagation.
However, these methods tend to be specificto a particular model architecture, such as skip connection, and it is nontrivial to extend the findingsto other architectures or modules.
Meanwhile, Huang et al.
[11], Inkawhich et al.
[12, 13] proposeto exploit feature space constraints to generate more transferable attacks.
Yet they need to identitythe best performing intermediate layers or train one-vs-all binary classifies for all attacked classes.
Recently, Zhao et al.
[35] find iterative attacks with much more iterations and logit loss can achieverelatively high targeted transferability and exceed the feature-based attacks.
Apart from the input-specific adversarial attacks, there have been some methods utlizing the generativemodels to generate the adversarial perturbations [19, 21, 22].
For example, the work of Naseer et al.[21] propose to train a generative model to match the distributions of source and target class, so as toincrease the targeted transferability.
However, the learning of the perturbation generator is nontrivial,especially on large-scale datasets.
In summary, the current performance of transfer attacks is stillunsatisfactory, especially for targeted attacks.
Wang et al.
[29] also explore the min-max framework for producing the attacks, but they are totallydifferent from us in terms of motivation and formulation.
They are to generate attacks based onmultiple models and samples.
And, the variable of inner maximization in Wang et al.
[29] is theprobability vector, representing the weight assigned to different models or samples.
Methodology
3.1 Preliminaries of Transfer Adversarial Attack
Given an benign sample
( x , y ) ∈
( X , Y ) , the procedure of transfer adversarial attack is firstlyconstructing the adversarial example x adv within the neighborhood region B ϵ ( x ) = { x ′ :
∥ x ′ −x ∥
p ≤ ϵ } by attacking the white-box surrogate model M s ( x ; θ ) :
X → Y , then transferring x advto directly attack the black-box target model M t ( x ; ϕ ) :
X → Y .
The attack goal is to mislead thetarget model, i.e ., M t ( x adv ; ϕ )
̸
= y (untargeted attack), or M t ( x adv ; ϕ ) =
y t (targeted attack) withy t ∈ Y indicting the target label.
Taking the target attack as example, the general formulation ofmany existing transfer attack methods can be written as follows:
[Equation]
The loss function L is often set as the cross entropy (CE) loss [34] or the logit loss [35], which willbe specified in later experiments.
Besides, the formulation of untargeted attack can be easily obtainedby replacing the loss function L and y t by −L and y , respectively.
Since M s is white-box, if G ( · ) is set as the identity function, then any off-the-shelf white-boxadversarial attack method can be adopted to solve Problem (1), such as I-FSGM [14], MI-FGSM[2], etc .
Meanwhile, existing works have designed different G ( · ) functions and developed thecorresponding optimization algorithms, to boost the adversarial transferability between surrogate andtarget models.
For example, G ( · ) is specified as random resizing and padding (DI) [34], translationtransformation (TI) [3], scale transformation (SI) [15], and adversarial mixup (Admix) [30].
3.2 Reverse Adversarial Perturbation
As discussed above, although having good performance for the white-box setting, the adversarialexamples generated from M S exist poor adversarial transferability on M T , especially for targetedattacks.
The previous works attribute this issue to the overfitting of adversarial attack to M S[1, 2, 3, 28, 32].
As shown in Figure 1 (b), when x pgd locates at a sharp local minimum, it is notstable and is sensitive to changes of M S .
When having some changes on model parameters, x pgdcould results in a high attack loss against M S ′ and lead to a failure attack.
To mitigating the overfitting to M S , we advocate to find x adv located at flat local region.
That meanswe encourage that not only x adv itself has low loss value, but also the points in the vicinity of x advhave similarly low loss values.
To this end, we propose to minimize the maximal loss value within a local neighborhood regionaround the adversarial example x adv .
The maximal loss is implemented by perturbing x adv to
[Figure]
maximize the attack loss, named Reverse Adversarial Perturbation (RAP).
By inserting the RAP intothe formulation (1), we aim to solve the following problem,
[Equation]
where
[Equation]
with n rap indicating the RAP, and ϵ n defining its search region.
The above formulations Equation (2)and Equation (3) correspond to the targeted attack, and the corresponding untargeted formulationscan be easily obtained by replacing the loss function L and y t by −L and y , respectively.
It is a min-max bi-level optimization problem [16], and can be solved by iteratively optimizing theinner maximization and the outer minimization problem.
Specifically, in each iteration, given x adv ,the inner maximization w.r.t.
n rap is solved by the projected gradient ascent algorithm:
[Equation]
The above update is conducted by T steps, and α n = ϵ n T .
Then, given n rap , the outer minimizationw.r.t.
x adv can be solved by any off-the-shelf algorithm that is developed for solving Equation (1).
For example, it can be undated by one step projected gradient descent, as follows:
[Equation]
with Clip B ϵ ( x ) ( a ) clipping a into the neighborhood region B ϵ ( x ) .
The overall optimization procedureis summarized in Algorithm 1.
Moreover, since the optimization w.r.t.
x adv can be implemented byany off-the-shelf algorithm for solving Problem Equation (1), one notable advantage of the proposed
RAP is that it can be naturally combined with any one of them, such as the input transformationmethods [3, 15, 30, 34].
A Late-Start (LS) Variant of RAP.
In our preliminary experiments, we find that RAP requires moreiterations to converge and the performance is slightly lower during the initial iterations, compared toits baseline attack methods.
As shown in Figure 2, we combine MI [2] and Admix [30] with RAP,and adopt ResNet-50 as the surrogate model.
We take the evaluation on 1000 images from ImageNet(see Sec.4.1).
It is observed that the method with RAP (see the orange curves) quickly surpassesits baseline method (see the blue curves) and finally achieves much higher success rate with moreiterations, which verify the effect of RAP on enhancing the adversarial transferability.
However, itis also observed that the performance of RAP is slightly lower than its baseline method in the earlystage.
The possible reason is that the early-stage attack is of very weak attack performance to thesurrogate model.
In this case, it may be waste to pursue better transferable attacks by solving themin-max problem.
A better strategy may be only solving the minimization problem Equation (1)in the early stage to quickly achieve the region of relatively high adversarial attack performance,then starting RAP to further enhance the attack performance and transferability simultaneously.
Thisstrategy is denoted as RAP with late-start (RAP-LS), whose effect is preliminarily supported by theresults shown in Figure 2 (see the green curve) and will be evaluated extensively in later experiments.
Input: Surrogate model M s , benign data ( x , y ) , target label y t , loss function L , transformation G , the global iteration number K late-start iteration number K LS of RAP, as well as hyper-parameters in optimization (specified in later experiments
: the adversarial example x adv
3.3 A Closer Look at RAP
To verify whether RAP can help us find a x adv located at the local flat region or not, we use ResNet-50as surrogate model and conduct the untargeted attacks.
We visualize the loss landscape around x advon M S by plotting the loss variations when we move x adv along a random direction with differentmagnitudes a .
The details of the calculation are provided in supplementary materials .
Figure 3 plotsthe visualizations.
We take I-FGSM [14] (denoted as I), MI [2], DI [34], and MI-TI-DI (MTDI) asbaselines attacks and combined them with RAP.
We can see that comparing to the baselines, RAPcould help find x adv located at the flat region.
Experiments
4.1 Experimental Settings
Dataset and Evaluated Models.
We conduct the evaluation on the ImageNet-compatible dataset 185comprised of 1,000 images.
For the surrogate models, we consider the four widely used networkarchitectures: Inception-v3 (Inc-v3) [26], ResNet-50 (Res-50) [9], DenseNet-121 (Dense-121) [10],and VGG-16bn (VGG-16) [24].
For target models, apart from the above models, we also utilize morediverse architectures: Inception-ResNet-v2 (Inc-Res-v2) [27], NASNet-Large (NASNet-L) [36], and
ViT-Base/16 (ViT-B/16)
For defense models, we adopt the two widely used ensemble adversarialtraining (AT) models: adv-Inc-v3 (Inc-v3 adv ) and ens-adv-Inc-Res-v2 (IncRes-v2 ens ) [28].
Besides,we also test multi-step AT model [23], imput transformation defense [32], feature denoising [33], andpurification defense (NRP) [20].
Compared Methods.
We adopt I-FGSM [14] (denoted as I), MI [2], TI [3], DI [34], SI [15], Admix[30], ILA [11], LinBP [8], and the generative targeted attack method TTP [21].
We also considerthe combination of baseline methods, including MI-TI-DI (MTDI), MI-TI-DI-SI (MTDSI), and
MI-TI-DI-Admix (MTDAI).
Implementation Details.
For untargeted attack, we adopt the Cross Entropy (CE) loss.
For targetedattack, apart from CE, we also experiment with the logit loss, where Zhao et al.
[35] shows itbehaves better for targeted attack.
The adversarial perturbation ϵ is restricted by ℓ ∞ = 16 / 255 .
The step size α is set as 2 / 255 and number of iteration K is set as 400 for all attacks.
In the
[Table]
following, we mainly show the results at K =
and the results at different value of K are shownin supplementary materials .
For RAP, we set K LS as 100 and α n as 2 / 255 .
We set ϵ n as 12 / 255 for
I and TI in untargeted attack and 16 / 255 for other attacks in all other settings.
The computationalcost is shown in supplementary materials .
4.2 The Evaluation of Untargeted Attacks
Baseline Methods.
We first evaluate the performance of RAP and RAP-LS with different baselineattacks, including I, MI, DI, TI, SI, and Admix.
The results are shown in Table 1.
For instance, the‘MI/ +RAP/ +RAP-LS’ denotes the methods of baseline MI, MI+RAP, and MI+RAP-LS, respectively.
RAP achieves the significant improvements for all methods on each target model.
For average attacksuccess rate of all target models, RAP outperforms the I and MI by 9 . 6% and 16 .
3% , respectively.
For TI, DI, SI, and Admix, RAP gets the improvements by 10 . 2% , 10 . 9% , 9 .
3% , and 6 . 3% .
Withlate-start, RAP-LS further enhance the transfer attack performance for almost all methods.
Combinational Methods.
Prior works demonstrate the combination of baseline methods couldlargely boost the adversarial transferability [30, 35].
We also investigate of behavior of RAP whenincorporated with the combinational attacks.
The results are shown in Table 2.
As shown in thetable, there exist the clear improvements of the combinational attacks over all baseline attacks shownin Table 1.
In addition, our RAP-LS further boosts the average attack success rate of the threecombinational attacks by 6 . 9% , 2 . 6% , and 1 . 7% respectively.
Combined with the three combinationalattacks, RAP-LS achieves 95 .
6% , and 98 .
3% average attack success rate, respectively.
Theseresults demonstrate RAP can significantly enhance the transferability.
4.3 The Evaluation of Targeted Attacks
We then evaluate the targeted attack performance of the different methods with RAP.
The results withlogit loss are presented and the results with CE loss are shown in supplementary materials .
Baseline Methods.
The results of RAP with baseline attacks are shown in Table 3.
From theresults, RAP is also very effective in enhancing the transferability in targeted attacks.
Taking ResNet50 and DenseNet-121 as surrogate models for example, the average performance improvementsinduced by RAP are 5 .
0% (I), 8 .
1% (MI), 4 .
6% (TI), 10 .
4% (DI), 18 .
5% (SI), and 15 .
1% (Admix),respectively.
Comparing to the ResNet-50 and DenseNet-121, the baseline attacks generally achievelower transferability when using the VGG-16 or Inception-v3 as the surrogate models, which has alsobeen verified in existing works [34, 35].
However, for Inception-v3 and VGG-16 as the surrogatemodels, RAP also consistently boosts the transferability under all cases.
With late-start, RAP-LS
[Table] [Table]
could further improve the transferability of RAP for most attacks.
The average attack success rateunder all attack cases of RAP-LS is 2 .
6% higher than that of RAP.
Combinational Methods.
As did in the untargeted attacks, we also evaluate the performance ofcombinational methods.
The results are shown in Table 4.
Similar to the findings in untargetedattacks, the combinational methods obtain significantly improvements over baseline methods.
The
RAP-LS outperforms all combinational methods by a significantly margin.
For example, taking theaverage attack success rate of all target models as evaluation metric, RAP-LS obtains 14 .
8% ,9 . 3% improvements over the MTDI, MTDSI and MTDAI, respectively.
4.4 The Comparison with Other Types of Attacks
[Figure]
[Table]
Apart from the baseline and the combinational methods, we also experiment withmore diverse attack methods, includingthe model-specific attack LinBP [8], thefeature-based attack ILA [11], and thegenerative targeted attack TTP [21].
The
LinBP depends on the skip connection andthe authors only provide the source codeabout ResNet-50.
We use their released code and thus conduct experiments with ResNet-50 as
M S .
The results of LibBP and ILA are shown in Table 5, where we also implement the variantsof LinBP following Guo et al.
[8], inlcuding LinBP-ILA, LinBP-ILA-SGM, LinBP-MI-DI, and
LinBP-MI-DI-SGM.
We observe that our MI-DI-RAP significantly outperforms the LinBP and ILA,especially for the targeted attacks.
Compared with the second-best method ( i.e ., LinBP-MI-DI-SGM),we obtain a large improvement by 33 . 5% on average ASR of targeted attacks.
[Table]
TTP [21] is the state-of-the-art generative method to conducttargeted attack.
To compare with it, we adopt the generatorsbased on ResNet-50 provided by the authors.
Since TTPneeds to train the perturbation generator for each targetedclass, we follow their “10-Targets (all-source)" setting, as didin Zhao et al.
The results are shown in Table 6, whereour MTDSI+RAP-LS behaves best and outperforms TTP and MTDI by large margins of 14 . 9% and25 .
7% , respectively.
[Table] [Table]
4.5 The Evaluation on Diverse Network Architectures and Defense Models
To further demonstrate the efficacy of RAP
[Table]
we evaluate our method on more diverse networkarchitectures, including Inception-ResNet-v2, NASNet-Large and ViT-Base/16.
We adopt ResNet-50as the surrogate model and the results are shown in Table 7, col 2-7 .
As shown in the table, theproposed RAP-LS achieves significant improvements for all three combinational methods on alltarget models, and MTDAI+RAP-LS achieves the best performance for diverse models.
For MTDAI,the average performance improvements induced by RAP-LS is 5.9% and 7.8% for untargeted andtargeted attacks, respectively.
Since ViT is based on the transformer architecture that totally beingdifferent from convolution models, the transfer attacks based on Resnet-50 behave relatively poor on it,especially on targeted attacks.
Yet our RAP-LS still obtains consistent improvements for all comparedmethods.
We also consider the ensemble-model attack on these diverse network architectures and theresults are given in supplementary materials.
Furthermore, we evaluate RAP on attacking defense models.
We choose ensemble adversarial training(AT) model, adv-Inc-v3 and ens-adv-Inc-Res-v2 [28], multi-step AT model [23], imput transformationdefense [32], feature denoising [33], and purification defense (NRP) [20].
We only demonstrate theresults of ensemble AT in main submission.
The evaluations of other defense models are shown insupplementary materials .
Following prior works [30, 34], we adopt the ensemble-model attack byaveraging the logits of different surrogate models, including ResNet-50, ResNet-101, Inception-v3,and Inception-ResNet-v2.
The transfer attack success rate on defense models are shown in Table 7,col 8-11 .
We can observe that our RAP-LS further boosts transferability of the baseline methods onboth targeted and untargeted attacks.
For untargeted attacks, RAP-LS achieves average performanceimprovements of 9 . 8% and 14 .
1% on Inc-v3 adv and IncRes-v2 ens , respectively.
For targeted attacks,the average performance improvements of RAP-LS are 14 .
8% and 11 .
1% , respectively.
4.6 Ablation Study
We conduct ablation study on the hyper-parameters of the proposed RAP, including the size ofneighborhoods ϵ n , the iteration number of inner optimization T and late-start K LS .
We adopttargeted attacks with ResNet-50 as the surrogate model.
We first evaluate the effect of ϵ n and T .
We consider different values of ϵ n , including 2 / 255 , 4 / 255 ,8 / 255 , 12 / 255 , 16 / 255 , and 20 / 255 .
In Figure 4 (a), we plot the tendency curves of the targetedattack success rate under different values of ϵ n and T .
Note that in Sec. 3.2, we set α n = ϵ n /T .
Thus for a fixed ϵ n , larger T indicating lower stepsize α n .
The minimum stepsize of α n is set to2 / 255 .
We have the following observation from the plot: for a fixed ϵ n , the more iterations T , thebetter attack performance.
Thus, we adopt a relatively smaller α n = 2 / 255 in our experiments.
In
Figure 4 (b-d), we further plot the results of different attack methods and target models w.r.t .
ϵ n ,where α n = 2 / 255 .
As shown in the plots, the larger ϵ n generally improves the attack performance.
For Inception-v3 and DenseNet-121, the improvements become mild for even larger ϵ n .
Overall, thevalue of 12 or 16 could lead to satisfactory result under most cases.
Then we conduct the ablation study of K LS .
In Figure 5, we report the targeted attack success rate of
I, MI, DI, and MI-TI-DI combined with RAP-LS with K LS = 0 , 25 , 50 , 100 , 150 , 200 .
Note that the
RAP-LS with K LS = 0 reduces to RAP.
As shown in the plots, the proposed late-start strategy canfurther boost attack performance of RAP for most cases.
In general, the performance improvementsincrease as K LS increases, and then become mild when K LS is larger than 100.
The suitable valueof K LS is relatively consistent among different methods and target models.
4.7 The Targeted Attack Against Google Cloud Vision API
Finally, we conduct the transfer attacks to attack a practical and widely used image recognition system,
Google Cloud Vision API, and in the more challenging targeted attack scenario.
MTDAI-RAP-LSbehaves the best performance in above experiments
, so we choose it to conduct the attack.
We takethe evaluation on randomly selected 500 images and use ResNet-50 as surrogate model.
As the APIreturns 10 predicted labels for each query, to evaluate the attacking performance, we test whether ornot the target class appears in the returned predictions.
Since the predicted label space of Google
Cloud Vision API do not fully correspond to the 1000 ImageNet classes, we manually treat classeswith similar semantics to be the same classes.
In comparison, the baseline MTDAI successfullyattacks 232 images against the Google API.
Our RAP-LS achieves a large improvement, successfullyattacking 342 images, leading to a 22.0% performance improvements.
These demonstrates the highefficacy of our method to improve transferability on real-world system.
Conclusion
In this work, we study the transferability of adversarial examples that is significant for black-boxattacks.
The transferability of adversarial examples is generally influenced by the overfitting ofsurrogate models
To alleviate this, we propose to seeking adversarial examples that locate at flatter local regions.
That is, instead of optimizing the pinpoint attack loss we aim to obtain a consistently low loss at the neighbor regions of the adversarial examples.
We formulate this as a min-max bi-level optimization problem, where the inner maximization aims to inject the worse-case perturbation for the adversarial examples.
We conduct a rigorous experimental study, covering untargeted attack and targeted attack, standard and defense models and a real-world Google Clould Vision API.
The experimental results demonstrate that RAP can significantly boost the transferability of adversarialexamples.
[-Here, we further report theperformance with different values of K for completeness in Table 13 (targeted attack) and Table 
-]