Towards Transferable Unrestricted Adversarial Examples with Minimum Changes
Abstract —Transfer-based adversarial example is one of the most important classes of black-box attacks.
However, there is a trade-off between transferability and imperceptibility of the adversarial perturbation.
Prior work in this direction often requires a fixed but large ℓ p -norm perturbation budget to reach a good transfer success rate, leading to perceptible adversarial perturbations.
On the other hand, most of the current unrestricted adversarial attacks that aim to generate semanticpreserving perturbations suffer from weaker transferability to the target model.
In this work, we propose a geometry-aware framework to generate transferable adversarial examples with minimum changes.
Analogous to model selection in statistical machine learning, we leverage a validation model to select the best perturbation budget for each image under both the ℓ ∞ - norm and unrestricted threat models.
We propose a principled method for the partition of training and validation models by encouraging intra-group diversity while penalizing extra- group similarity.
Extensive experiments verify the effectiveness of our framework on balancing imperceptibility and transferability of the crafted adversarial examples.
The methodology is the foundation of our entry to the CVPR’21 Security AI Challenger: Unrestricted Adversarial Attacks on ImageNet , in which we ranked 1st place out of 1,559 teams and surpassed the runner-up submissions by 4.59% and 23.91% in terms of final score and average image quality level, respectively. Code is available at https://github.com/Equationliu/GA-Attack.
I. I NTRODUCTION
Though deep neural networks have exhibited impressive performance in various fields [1], [2], they are vulnerable to adversarial examples [3]–[7], where test inputs that have been modified slightly strategically cause misclassification.
Adversarial examples have posed serious threats to various securitycritical applications, such as autonomous driving [8] and face recognition [9].
Most positive results on adversarial attacks have focused on white-box settings [10], [11].
However, the problem becomes more challenging when it comes to the black-box setting, where the attacker has no information about the model architecture, hyper-parameters, and even the outputs of the black-box model.
In this setting, adversarial examples are typically generated via transfer-based methods [3], [12], [13], e.g. , attacking an ensemble of accessible source models and hoping that the same adversarial examples are able to fool the unknown target/test model [14], [15].
Despite a large amount of work on transfer-based attacks, many fundamental questions remain unresolved.
For example, existing transfer-based attacks [16]–[18] that search for adversarial examples in a fixed-radius ℓ ∞ -norm ball often require a high perturbation budget to reach a satisfactory transfer success rate.
However, such perturbations might be perceptible to humans (see Figs.
1 and 6).
On the other hand, unrestricted attacks that aim to generate minimum human-imperceptible perturbations [19]–[21] suffer from weaker transferability to the target model.
This is in part due to the difference between the decision boundaries of the source and target models.
Given the trade-off between transferability and imperceptibility, one of the long-standing questions is generating transferable adversarial examples by minimum changes of natural examples.
A. Our Methodology and Results
In this work, we propose a novel geometry-aware framework to generate transferable unrestricted adversarial examples with minimum changes.
Our intuition is that the smallest perturbation budgets w.r.t.
distinct images should be different (see
Fig. 1) and should depend on their geometrical relationship with the decision boundary of the target model (see Fig. 2).
Unfortunately, finding transferable minimum-budget adversarial perturbations is an intractable optimization problem (see
Eq.
as the target model is unknown.
We approximately solve this problem by discretizing the continuous space of perturbation radius into a finite set and choosing the minimum perturbation budget that is able to fool the test model.
The main challenge here is to evaluate whether a given perturbation can transfer well to the unknown target model [22], [23].
To overcome this challenge, we split all accessible whitebox source models into training and validation sets, where adversarial perturbations are crafted only on the training set.
We use the validation set to select the smallest perturbation radius for each input that suffices to fool the validation model with a certain confidence level through an early-stopping mechanism.
When the training (or validation) set consists of multiple models, we use their average ensemble [14].
Experimentally, our method yields a significant performance boost on the trade-off (leading to higher S total in Table.
II) between transferability and imperceptibility.
As shown in Fig.
4, the transfer success rate of our method GA-DTMI-FGSM surpasses the baseline DTMI-FGSM (see Eq.
(1)) by up to 16% in absolute value under the same average perturbation reward (see Eq.
Besides, our method GA-DMI-FSA is able to generate semantic-preserving yet transferable unrestricted adversarial examples under the unrestricted threat model (see
Eq.
(3), Figs. 1, 6, 10 and 11).
Rhodesian ridgeback bull mastiff boxer
Great Dane
Norfolk terrier 40% 80% bloodhound bull mastiff
Rhodesian ridgeback redbone boxer 40% 80% bloodhound redbone coonhound otterhound basset hound 40% 80% bloodhound redbone coonhound otterhound basset hound 40% 80% bloodhound bull mastiff
Rhodesian ridgeback redbone
Great Dane
Benign 40% 80% koala wombat maypole golfcart dromedary
DTMI ( =4)
[Table]
% 80% koala wombat maypole golfcart
Lemur
[Table]
DTMI ( =12)
[Table]
catamount wall
[Table]
fox squirrel
[Table] [Table]
wall
[Table]
analog
[Table]
fox squirrel grey fox
GA-DTMI
[Table]
fox squirrel koala hare
[Table]
analog clock 40% 80%
Bedlington terrier
Dandie Dinmont rosehip standard poodle
Sealyham 40%
[Table]
Dandie
[Table]
Sealyham miniature
[Table]
[Table] [Table]
Sealyham
[Table]
Bedlington
[Table] [Table]
[Table]
Sealyham miniature
[Table]
toy
[Table]
[Table] [Table]
Sealyham
Bedlington
[Table]
toy
[Table]
miniature
[Table]
Benign 40% 80% coucal bulbul magpie
Erythrocebus patas jacamar DMI-FSA ( =1.28)
[Figure] [Figure]
% coucal jacamar bulbul bee eater magpie DMI-FSA ( =2.12)
[Table] [Table]
% jacamar water ouzel coucal magpie hummingbird DMI-FSA ( =3.50)
[Table]
% 80% water
[Figure]
• We propose a Geometry-Aware (GA) framework, where fixed-budget attacking methods can be integrated, to generate transferable unrestricted adversarial examples with approximately minimum changes.
To the best of our knowledge, we are the first to explore transfer-based black-box attacks with adaptive perturbation budgets.
• Under ℓ ∞ -norm setting, our GA framework improves the imperceptibility of the crafted adversarial examples by a large margin without the decrease of transfer success rate (see Fig. 4).
By applying our method GA-DTMI-FGSM to the CVPR’21 Security AI Challenger [24], we ranked 1st place out of 1,559 teams and surpassed the runner-up submissions by 4.59% and 23.91% in terms of final score and average image quality level, respectively.
• Under unrestricted setting, we propose a transfer-based unrestricted attack (see Eq.
(3)) by combining the whitebox feature space attack [25] with transfer-based ℓ ∞ - norm attacks to generate semantic-preserving yet transferable adversarial examples (see Figs.
1 and 10).
Moreover, the crafted adversarial examples transfer well to adversarially robust models (see Table.
III, Figs. [-
-]6 and 11).
II. R ELATED W ORK A. ℓ p -norm Adversarial Examples
Existing gradient-based white-box attacks either search for adversarial examples in a fixed ℓ p -norm ball [26], [27], or optimize the perturbation for each image independently to get a minimum-norm solution such as DeepFool [28], CW [29], and fast adaptive boundary attack [30].
However, white-box assumption usually does not hold in real-world scenarios.
In query-based black-box setting, attackers utilize output logits [31], [32] or predicted label [33], [34] of the target model to generate adversarial examples.
But these attacks typically suffer from high query complexity, making it easy to be detected [35].
Transfer-based black-box attacks [36]–[39] can pose serious threats in practice as they need no information about the defense models.
Dong et al.
[16] boosted transferability by integrating momentum into gradient-based methods.
Liu et al.
[14] found that attacking a group of substituted source models simultaneously can improve transferability.
Besides, transferability benefits from input transformations such as input diversity [17] and translation-invariant method [18].
B. Unrestricted Adversarial Examples
The ℓ p -norm distance is not an ideal perceptual similarity metric [40], [41], which oversimplifies the diversity of realworld perturbations.
Unrestricted adversarial examples have received significant attention in recent years [42].
Most of the current unrestricted attacks aim to generate imperceptible adversarial examples under white-box setting, such as geometric transformations [19], [43], [44] and distance metrics beyond ℓ p norm [20], [21].
Color-based attacks [45]– [50] were also proposed to generate large but imperceptible perturbations, however, the modified color can sometimes be unnatural.
Instead of optimizing in the input space, generative approaches [51]–[54] search for adversarial embeddings in the latent space.
Style transfer [50], [55] is inherently an unrestricted attack as it preserves the semantic of the content image.
However, constructing transferable unrestricted adversarial examples is still less explored.
In this work, we will fill this gap by combining the white-box feature space attack [25] with transfer-based ℓ ∞ -norm attacks to generate semanticpreserving yet transferable unrestricted adversarial examples.
C. Adversarial Defenses
There have been long-standing arms races between defenders and attackers.
Adversarial training [4] is one of the most promising defense methods.
Many variants of adversarial training framework were proposed, e.g., ensemble adversarial training [14] for transfer-based attacks, PGD-based adversarial training [26], and TRADES [56] with a new robust loss based on the trade-off between robustness and accuracy.
Geometryaware instance-reweighted adversarial training [57], which is proven falling into gradient masking [58], shares similar insights with us that the importance of distinct inputs in adversarial training should be different.
Laidlaw et al.
[21] integrate adversarial training with Learned Perceptual Image
Patch Similarity (LPIPS) [59], aiming to improve robustness against perturbations that were unseen during training.
Unlike empirical defenses, Certified defenses [60]–[63] could provide robustness guarantee under a certain ℓ p -norm budget.
III. P RELIMINARIES
Notation.
A deep neural network classifier can be described as a function f ( x ; θ ) :
X → R C , parameterized by weights θ , which maps a vector x ∈ X to its output logits.
Given an input x of class y ∈ { 1 , 2 , · · · , C} , the predicted label of f ( x ; θ ) is ˆ f ( x ) :
=
arg max j f j ( x ; θ ) , where f j ( x ; θ ) represents the j -th entry of f ( x ; θ ) .
We use L ( f ( x ; θ ) , y ) to represent the cross-entropy loss and denote the ε -neighborhood of x by B ( x , ε ) :
=
{ x ′ ∈ X : D ( x , x ′ ) ≤ ε } , where D is a distance metric that describes the change between the adversarial example x ′ and the nature example x .
We denote the black-box test model by g , and split the set of accessible source models Φ = { ϕ 1 , ϕ 2 , · · · , ϕ n } into the set of training models f and the set of validation models h .
A. Transfer-based ℓ ∞ -norm Attacks
Existing transfer-based attacks typically search for adversarial examples in a fixed-radius ℓ p -norm ball, i.e.,
D ( x , x ′ ) =
∥ x ′ − x ∥ p ≤ ε .
Various methods were proposed to boost transferability of the generated adversarial examples, such as input Diversity Iterative Fast Gradient Sign
Method (DI-FGSM)
[17], Momentum-based Iterative (MIFGSM) method [16] and Translation-invariant Iterative (TIFGSM) method [18].
We formulate a strong ℓ ∞ -norm baseline
DTMI-FGSM by combining all these techniques, i.e.,
[Equation]
where m
= 0 , W is a pre-defined kernel with a convolution operation ∗ , α is the step size, Π is the projection operator, and γ is the decay factor for the momentum term.
T ( x t , p ) represents the input transformation on x t with probability p .
When γ = 0 , DTMI-FGSM attack degenerates to the DTIFGSM attack.
When p = 0 , DTMI-FGSM attack degenerates to the DMI-FGSM attack.
B. Transfer-based Unrestricted Attack
Inspired by prior work [64] in style transfer, Xu et al.
[25] tries to find stylized adversarial examples by assuming that the image pairs from the same class share consistent content and differ mainly in their styles.
Here we propose to generate semantic-preserving yet transferable unrestricted adversarial examples by combining the Feature Space Attack (FSA) [25] with transfer-based ℓ ∞ -norm attacks [16], [17].
Given an encoder ϕ , we extract the style features of input x as channelwise mean µ ( ϕ ( x )) ∈ R C and channel-wise standard deviation σ ( ϕ ( x )) ∈ R C , Specifically,
[Equation]
where ϕ ( x ) ∈ R C × H × W represents the latent embedding.
Xu et al.
[25] adds adversarial perturbations on µ and σ before projecting ϕ
( x ) back to the input space X with a pre-traineddecoder ϕ − 1 , namely,
[Equation]
where ˜ ϕ
( x ) enlarges or shrinks the mean µ and the standard deviation σ of the embedding ϕ ( x ) by a factor of e τ µ and e τ σ , respectively.
In this way, the distance metric D ( x , x ′ )
= max (cid:0) e ∥
[Equation]
∞ , e ∥
[Equation]
∞ (cid:1) ≤ ε .
In order to preserve the semantic of the unrestricted adversarial example x ′ , a content loss was added during the attacking process, i.e.,
[Equation]
where λ balance the trade-off between adversarial and the content loss.
Following Xu et al.
[25], we set λ
= 128 and use the top-5 margin loss for adversarial attack.
With all above, the unrestricted attack (see Eq.
(3)) can be solved by conventional ℓ ∞ -norm attack on parameters τ µ and τ σ .
Moreover, the same techniques in Sec.
III-A such as input diversity T ( x ′ , p ) (only for the margin loss) and momentum-based method can be integrated to improve transferability, i.e.,
[Equation]
When γ = 0 , the DMI-FSA attack degenerates to the DI-FSA attack.
f g h initial / source example
geometry-aware perturbation path minimum-norm adversarial example of f fixed-radius adversarial example of f norm balls centered at source example decision boundary of training model f decision boundary of validation model h decision boundary of test model g
C. Evaluation Metric for Transfer-based Attack
The imperceptibility of adversarial examples is hard to evaluate due to the lack of precise quantization of human perception [65].
Sharif et al.
[66] found that ℓ p -norm distance is not an ideal perceptual similarity metric and suggest setting adaptive perturbation budget for every sample to ensure that the attacks’ output would be imperceptible.
Therefore, we choose transfer success rate and the perturbation budget under distance metric D as our main evaluation metrics.
Consider a dataset ˆ S
=
( x i , y i ) } Ni =1 and the corresponding adversarial examples ˆ S adv
= { ( x ′ i , y i ) } Ni =1 that are crafted on the training model f .
Let N
= (cid:80) Ni
=1 1 { ˆ g ( x ′ i )
̸ = y i } be the number of misclassified adversarial examples on the test model g .
We define the average total score as:
[Equation]
where S APR is the Average Perturbation Reward of adversarial examples that are misclassified by test model g and the reward function F reward is a decreasing function w.r.t.
metric D ( x , x ′ ) .
IV. M ETHODOLOGY : G EOMETRY -A WARE F RAMEWORK
Eq.
(5) factorizes the average total score as the product of transfer success rate and average perturbation reward, which motivates us to find the adversarial example with minimum changes under metric D , i.e.,
[Equation]
However, direct optimization of problem (6) is intractable, in part due to the lack of information about test model
Algorithm
Require:
Benign input x with label y ; training models f ; validation model h ; number of sub-procedures K ; maximum perturbation size ε and threshold η ; attack algorithm A ;
Ensure:
Transfer-based unrestricted adversarial example x ′ with approximately minimum change; 1: x 0 = x ; 2: for k = 1 , 2 , · · · , K do 3: x k
=
A ( x , x k − 1 , f, kεK ) ; ▷ fixed budget 4: conf ← exp(
[Equation]
(cid:80) j exp(
[Equation]
( x
[Equation] [Equation]
; 5: if conf < η then 6:
Return x k ; ▷ early-stopping in Eq.
(7) 7: end if 8: end for 9: Return x K ; g .
We approximately solve this problem by discretizing the continuous space of perturbation radius into a discrete set and choosing the minimum perturbation budget such that the attack is able to fool the test model g .
However, the challenge is that it is typically difficult to decide whether a given perturbation radius can also fool the test model [22], [23].
This problem is also known as model selection (we view the source models as training data, then the generated adversarial perturbation is the so-called selected model or optimized parameters), and a classic approach to tackle this problem is to have a validation set.
More specifically, we split all accessible source models into training model set and validation model set.
With validation model h , we are able to generate transferable adversarial examples with dynamic radii.
To approximately solve problem (6), we first divide the attack in the ball B ( x , ε ) into K sub-procedures.
In the k -th subprocedure, we re-run a fixed-radius attack algorithm
A such
[Table]
as DMI-FSA (see Eq.
(4)) under the perturbation budget
[Equation]
Each sub-procedure starts from the solution of last subprocedure to accelerate the convergence.
To obtain a minimum-radius solution, we perform an early-stopping mechanism at the end of each sub-procedure if the probability of true class on the validation model h is smaller than a threshold η , i.e.,
[Equation]
Our GA framework is summarized in Algorithm 1 and illustrated in Fig.
Note that the output of GA framework is related to the choice of the training model f and the validation model h .
Thus it is important to figure out which partition of the source models performs better .
We split n pre-trained models Φ = { ϕ 1 , ϕ 2 , · · · , ϕ n } into k training models and n − k validation models.
Instead of traversing all possible partitions to select the optimal split by querying the test model g by C kn times, we propose a query-free approach that only utilizes the information of transferability between the pre-trained models (see Fig.3).
Let w ij ( w ij ̸ = w ji ) be the transfer success rate ( N 0 N in Eq.
(5)) from the source model ϕ
i to the target model ϕ j under a fixedradius attack (e.g., DTMI-FGSM).
Denote the binary partition function as G .
The training set and the validation set can be formulated as T =
{ i | G ( ϕ i ) = 0 } and V = { j | G ( ϕ j )
= 1 } , respectively.
We define the partition loss ℓ G as:
[Equation]
For both the training set loss ℓ G ( ϕ i ) and the validation set loss ℓ G ( ϕ j ) : 1) Minimizing the first formula on the right of
Eq.
(8) encourages intra-group diversity.
To make the decision boundary of the ensemble model ( f or h ) more general and effective, we minimize the transfer success rate between any two pre-trained models inside the group. 2)
Minimizing the second formula on the right of Eq.
(8) is penalizing extragroup similarity.
If f = h , the early-stopping mechanism in
Eq.
(7) will be triggered too early, leading to small adversarial
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
perturbations for all inputs.
Reducing the transferability from the training model f to the validation model h might improve generalization of the adversarial examples to unknown models.
Empirically, we find that the proposed partition loss ℓ G negatively correlates with the average total score S total [Figure]
V. E XPERIMENTS A. Experimental Setup
Datasets & Networks.
Similar to Xie et al.
[17], we randomly select 1,000 images from ILSVRC 2012 validation set [67], which are almost correctly classified by all the attacking models.
All these images are resized to 229 × 229 × 3 beforehand.
We consider eight normally trained models and two ensemble adversarially trained models [15].
The weights of all these models are publicly available [68].
More details about the networks are summarised in Table.
I.
The transferability between these models under ℓ ∞ -norm setting is summarised in
Fig.
It is much easier for the generated adversarial examples to transfer from vision transformers to convolutional neural networks (CNNs), which is consistent with the empirical observation in Shao et al.
Surprisingly, the robustness of naturally trained vision transformers under transfer attack is even on par with two ensemble adversarially trained CNNs.
Implementation Details.
Given the maximum perturbation size ε and number of sub-procedures K (5 as default) in our geometry-aware framework, we set the step size α = 1 .
25 × ε k T in the k -th sub-procedure, where the number of iterations T is set to 10 in the ℓ ∞ setting and 50 in the unrestricted setting.
ε is set to 20 in the ℓ ∞ -norm setting and 3.5 in the unrestricted setting.
When running a fixed-radius baseline at perturbation budget ε k , we set the number of iteration as T 2 (cid:0) 1 + Kε k ε (cid:1) with step size α to keep the same total perturbation budget (the sum of step size across all iterations) as our geometryaware framework for fair comparison.
The reward function
F reward ( ε 0 ) is set to 1 /ε
as smaller perturbation radius exhibits significantly higher image quality.
For the momentum term, we set the decay factor µ = 1 as in Dong et al.
For DI-FGSM [17], we set the transformation probability to p = 0 .
The input is first randomly resized to be an r × r ×image with r ∈
[(1 − γ ) s, (1 + γ ) s ] , and then padded to size (1+ γ ) s × (1+ γ ) s × 3 .
The transformed input is then resized to s × s × 3 for different input size s of various models, i.e., 224, 299 and 384.
We set γ = 0 .
1 as default.
For TI-FGSM [18], we use Gaussian kernel with kernel size 5 × 5 .
B. Balancing Transfer Success Rate and Perturbation Reward
Implementation Details.
Benefiting from the adaptive choice of perturbation budgets, our geometry-aware framework can generate transferable unrestricted adversarial examples with smaller changes.
In this experiment, the training model f and validation model h are an ensemble of models { 2 , 3 , 5 } and { 1 , 4 , 6 } in Table.
I, respectively.
The test model is InceptionResNet-v2.
The optimal threshold η (see Eq (7)) is searched from a finite set ranging from 0.001 to 0.9 by querying 2 the test model g to achieve the best average total score S total .
For each η , we execute our method and compute the average perturbation reward S APR (the x -axis of each red point in
Fig. 4).
Then the corresponding fixed-radius baseline is run at the same x -axis.
We conduct experiments on two threat models.
For the ℓ ∞ -norm setting, we combine our GeometryAware (GA) framework with DI-FGSM, DTI-FGSM, DTMIFGSM, and Admix-DTI-FGSM [38] (limited by the memory of a single NVIDIA RTX 3090, we set the number of admixed images m 1 = 3 and the number of randomly sampled images from other categories m 2 = 2 ), named GA-DI-FGSM, GADTI-FGSM, GA-DTMI-FGSM, and GA-Admix-DTI-FGSM, respectively; For the unrestricted setting, we combine our GA framework with DMI-FSA and DI-FSA, named GA-DMI-FSA and GA-DI-FSA, respectively.
Experimental Results.
We present the contour plot of average total score in Fig.
4, where the improvement of our method upon baselines depends on the choice of hyper-parameter η (leading to different S APR ).
Fixing S APR as 0.115,
our
[Figure]
[Table]
[Figure]
Fixing transfer success rate as 80%, our approach GA-DTMI-FGSM surpasses the baseline DTMI-FGSM (see Eq.
(1)) by up to 43.35% in terms of average perturbation reward.
approach GA-DTMI-FGSM surpasses the baseline DTMIFGSM by up to 16.1% in terms of transfer success rate.
As shown in Table.
II
[Table]
our approach yields a significant performance boost on the average total score S total across various threat models, especially in the ℓ ∞ -norm setting where both the transfer success rate and S APR are improved.
[Table]
C. Case Study: CVPR’21 Security AI Challenger
In the CVPR’21 Security AI Challenger: Unrestricted Adversarial Attacks on ImageNet [24], contestants were asked to submit adversarial examples without any access to the defense models.
The dataset is a subset of ILSVRCvalidation set [67], which consists of 5,000 images withimages per class.
The final score of each submission was manually scored from two aspects: 1) image semantic and 2) quality.
If the semantic of the submitted image changes (judged by human referees), then S s = 0 , otherwise S s = 1 .
The image quality S q (equivalent to our reward function [Figure] i n a l [Figure]
c o r e
[Figure]
1st (Ours) 2nd 3rd 4th 5th 6th 1.
A v e r a g
e I m a g e Q u a li t y L e v e l 2. 5×FinalScore 5000×TransferSuccessRate F reward ) was quantified with five levels S q ∈ { 1 , 2 , 3 , 4 , 5 } by multiple human referees.
The final score is given by (cid:80)
[Equation]
We apply our method GA-DTMI-FGSM ( η = 0 . 01 ) to the competition, where our entry ranked 1st place out of 1,559 teams.
In the adversarial competition, our training and validation models are both an ensemble of eight highperformance models.
We report the final score and average image quality level (equivalent to our average perturbation reward) in Fig.
It shows that our method outperforms other approaches by a large margin.
In particular, we surpass the runner-up submissions by 4.59% and 23.91% in terms of final score and average image quality level, respectively.
D. Transferable Unrestricted Adversarial Examples
Most of current defenses can be easily broken by unseen attacks in a white-box manner.
Adversarial training against multiple ℓ p -norm attacks [76] solved this issue partially, however, at the cost of robustness against single ℓ p -norm attack.
Laidlaw et al.
[21] integrated adversarial training with Learned Perceptual Image Patch Similarity (LPIPS) [59], aiming to improve robustness against perturbations that were unseen during training.
However, the proposed attack [21], similar to other unrestricted attacks [44], [46], suffers from weaker transferability to the target model.
In practice, attackers typically have no information about the defense models and the defenders do not have the ground truth to make pixel-level comparison (perturbation can be large as long as the generated adversarial examples are semantic-preserving).
Therefore, we propose to benchmark classification models on ImageNet under transfer-based unrestricted attacks .
Implementation Details.
For the ReColor attack [46], we set ε = 1 . 0 and iterations T = 100 which achieves 89.7% attack success rate on training model f
(the same as Sec.
V-B) and 9.1% transfer success rate on the test model
Inception-ResNetv2.
For FSA attack [25], we set ε = 3 . 0 and T = 500 which achieves 54.2% attack success rate and 12.4% transfer success rate on the same training and test models (Note that our method GA-DMI-FSA achieves 95.5% attack success rate and 58.3% transfer success rate, indicating that the input diversity and momentum techniques in Eq.
(4) boost both the attacking ability and transferability.).
Besides six adversarially trained and two high-performance classification models, we select two state-of-the-art models on the ImageNet-R dataset [73].
Experimental Results.
From Table.
III, we can conclude the following observations: a) GA Admix-DTI transfers better than
GA DTMI-FGSM .
b) ℓ ∞ -norm transfer attack can hardly break ℓ ∞ -norm adversarially trained models while the unrestricted attack (GA-DMI-FSA) reduces the accuracy of these models by a large margin (see also in Fig. 6).
c) DeepAugment [73], which utilizes semantic-preserving augmentations during training, exhibits non-trivial robustness against GADMI-FSA attack.
d) Efficientnet-l2-ns [74] performs well under all the transfer-based attacks and enjoys 86% accuracy against GA-DMI-FSA attack, showing that the distribution of our generated adversarial examples is close to the natural examples’.
Note that Efficientnet-l2-ns is also the bestperforming model on the ImageNet-V2 dataset [77], [78].
We visualize part of the transfer attack results on adversarially trained Resnext101-DenoiseAll [71] in Figs.
6, where our method GA-DMI-FSA is able to generate semantic-preserving yet transferable unrestricted adversarial examples.
For more visualization results, please see Fig.
11 in Sec.
VII-A.
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
Resnet152-Denoise [71] 72. [Table] [Table] [Table]
[Table]
[Table]
RVT-Tiny [72]
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
DeepAugment+AugMix [73] 96.1%
[Table]
[Table] [Table]
[Table] [Table]
Efficientnet-l2-ns [74]
[Table] [Table]
[Table]
Swin-L/patch4-window-12 [75]
[Table]
[Table]
[Table]
Benign
GA DTMI-
ReColor
FSA
GA DMI-FSA  ε =   ε = 1 .  ε =   ε = 1 .
ε =   ε = 1 .
ε =   ε = 2 .
E. Ablation Studies and Discussions
The optimal η depends on the train-valid splitting.
As declared in the implementation details in Sec.
V-B, the threshold η is searched from a finite set ranging from 0.001 to 0.9.
We now investigate how the hyper-parameter η will affect the average total score S total .
From Fig.
7, we observe that the optimal η ∗ varies across different splittings and can be larger if the transferability from the training model f to the test model g is higher enough.
However, as shown in Table IV, the improvement of our GA framework upon fixed-radius baseline (DTMI-FGSM) is stable and independent of the splitting.
model g is model 7 in Table I
and we select three (two of them are highlighted in Fig.
8a) representative splittings according to the partition loss ℓ G .
For each setting, we repeat the experiment three times and report the mean and the standard deviation (in the parenthesis)
[Table]
[Figure]
[Table]
[Figure]
[Table]
A v e r a g e T o t a l
S c
o r e
S t o t a l f , h , g =(4,5,6),(1,2,3),7 f , h , g =(1,2,3),(4,5,6),7 f , h , g =(2,3,5),(1,4,6),7 f , h , g =(1,2,3,8),(4,5,6,9),7 f , h , g =(1,2,3,4,5,6),(8,9),
The index in the legend corresponds to the model index in
Table.
I.
As the threshold η increases, the generated adversarial examples have higher confidence (probability of the true class) on the validation model h , leading to a lower transfer success rate ( left ).
Besides, the optimal η ∗ that yields the maximum
S total is dependent on the partition ( right ).
The effectiveness of ℓ G under different numbers of pretrained models and different kinds of pre-trained models.
To investigate the robustness of the proposed partition loss ℓ G under different settings, we carefully design controlled experiments in Figs. 8 and 9.
There are total C kn kinds of partitions when selecting k training models from n pre-trained models.
Given n and k , we run our method GA-DTMI-FGSM for all the train-valid splittings and obtain a scatter plot with
C kn points.
We observe a strong negative correlation between
[Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
S t o t a l f ={4,5,6} h
={1,2,3} f ={2,3,5} h ={1,4,6}
Corr( S total , ) =
Linear Regression Bayesian Credible Intervals (a) f ∪ h
= { 1 , · · · , 6 } , ( n, k ) =
[Figure] [Figure]
Linear
[Figure]
Credible Intervals (
[Figure]
∪ h = {
[Figure]
[Figure] [Figure]
( n
[Figure]
k )
=
[Figure] [Figure]
[Figure]
∪ h = {
[Figure] [Figure]
[Figure]
∪{ 8 } , ( n, k ) =
[Figure] [Figure]
S t o t a l
[Figure]
(a) f ∪ h
= { 0 , 1 , · · · , 6 } \ { 1 } .
[Figure]
(b) f ∪ h
= { 0 , 1 , · · · , 6 } \ { 3 } .
[Figure]
(c) f ∪ h
= { [Figure]
[Figure]
the partition loss ℓ G and the average total score S total .
For example, the average Pearson correlation coefficient over the six scatterplots in Figs.
8 and 9 is around -0.82.
Moreover, the negative correlation is significant and consistent across different numbers of pre-trained models (see Fig.
8) and different kinds of pre-trained models (see Fig. 9).
VI. C ONCLUSION
In this work, we propose a geometry-aware framework, where fixed-radius methods can be integrated to generate transferable unrestricted adversarial examples with minimum changes.
Under ℓ ∞ -norm setting, our framework could improve the imperceptibility of the crafted adversarial examples by a large margin without the decrease of transfer success rate.
Besides, we propose a transfer-based unrestricted attack by combining the white-box feature space attack with transferbased ℓ ∞ -norm attacks to generate semantic-preserving yet transferable unrestricted adversarial examples. Acknowledgements.
Fangcheng Liu and Chao Zhang are supported by the National Nature Science Foundation of China under Grant 62071013 and 61671027, and National Key R&D
Program of China under Grant 2018AAA0100300.
Hongyang
Zhang is supported by NSERC Discovery Grant RGPIN-202203215, DGECR-2022-00357.