Towards Transferable Unrestricted Adversarial Examples with Minimum Changes
Anonymous submission
Abstract —Transfer-based adversarial example is one of the most important classes of black-box attacks.
However, there is a trade-off between transferability and imperceptibility of the adversarial perturbation.
Prior work in this direction often requires a fixed but large ℓ p -norm perturbation budget to reach a good transfer success rate, leading to perceptible adversarial perturbations.
On the other hand, most of the current unrestricted adversarial attacks that aim to generate semanticpreserving perturbations suffer from weaker transferability to the target model.
In this work, we propose a geometry-aware framework to generate transferable adversarial examples with minimum changes.
Analogous to model selection in statistical machine learning, we leverage a validation model to select the best perturbation budget for each image under both the ℓ ∞ -norm and unrestricted threat models.
Extensive experiments verify the effectiveness of our framework on balancing imperceptibility and transferability of the crafted adversarial examples.
The methodology is the foundation of our entry to the adversarial competition of a 2021 conference, in which we ranked 1st place out of 1,559 teams and surpassed the runner-up submissions by 4.59% and 23.91% in terms of final score and average image quality level, respectively.
I. I NTRODUCTION
Though deep neural networks have exhibited impressive performance in various fields [1], [2], they are vulnerable to adversarial examples [3]–[7], where test inputs that have been modified slightly strategically cause misclassification.
Adversarial examples have posed serious threats to various securitycritical applications, such as autonomous driving [8] and face recognition [9].
Most positive results on adversarial attacks have focused on white-box settings [10], [11], where the attacker can get full access to the defense models.
However, the problem becomes more challenging when it comes to the black-box setting, where the attacker has no information about the model architecture, hyper-parameters, and even the outputs of the black-box model.
In this setting, adversarial examples are typically generated via transfer-based methods [3], [12], [13], e.g., attacking an ensemble of accessible source models and hoping that the same adversarial examples are able to fool the unknown target/test model [14], [15].
Despite a large amount of work on transfer-based attacks, many fundamental questions remain unresolved.
For example, existing transfer-based attacks [16]–[18] that search for adversarial examples in a fixed-radius ℓ ∞ -norm ball often require a high perturbation budget to reach a satisfactory transfer success rate.
However, such perturbations might be perceptible to humans (see Figs.
1 and 7).
On the other hand, unrestricted adversarial attacks that aim to generate minimum human-imperceptible perturbations [19]–[21] suffer from weaker transferability to the target model.
This is in part due to the difference between the decision boundaries of the source and target models.
Given the trade-off between transferability and imperceptibility, one of the long-standing questions is generating transferable adversarial examples by minimum changes of natural examples.
A. Our Methodology and Results
In this work, we propose a novel geometry-aware framework, into which fixed-budget methods can be integrated to generate transferable unrestricted adversarial examples with minimum changes.
Our intuition is that the smallest perturbation budgets w.r.t.
distinct images should be different (see
Fig. 1) and should depend on their geometrical relationship with the decision boundary of the target model (see Fig. 2).
Unfortunately, finding transferable minimum-budget adversarial perturbations is an intractable optimization problem (see
Eq.
as the target model is unknown.
We approximately solve this problem by discretizing the continuous space of perturbation radius into a finite set and choosing the minimum perturbation budget that is able to fool the test model.
The main challenge here is to evaluate whether a given perturbation can transfer well to the unknown target model [22], [23].
To overcome this challenge, we split all accessible whitebox source models into training and validation sets, where adversarial perturbations are crafted only on the training set.
We use the validation set to select the smallest perturbation radius for each input that suffices to fool the validation model with a certain confidence level through an early-stopping mechanism.
When the training (or validation) set consists of multiple models, we use their average ensemble [14].
Experimentally, our method yields a significant performance boost on the trade-off (leading to higher S total in Table.
II) between transferability and imperceptibility.
As shown in Fig.
4, the transfer success rate of our method GA-DTMI-FGSM surpasses the baseline DTMI-FGSM (see Eq.
(1)) by up to 16% in absolute value under the same average perturbation reward (see Eq.
Besides, our method GA-DMI-FSA is able to generate semantic-preserving yet transferable unrestricted adversarial examples under the unrestricted threat model (see
Eq.
(3), Figs. 1, 6, 7 and 9).
B. Summary of Our Contributions
• We propose a Geometry-Aware (GA) framework, where fixed-budget attacking methods can be integrated, to
Rhodesian ridgeback bull mastiff boxer
Great Dane
Norfolk terrier 40% 80% bloodhound bull mastiff
Rhodesian ridgeback redbone boxer 40% 80% bloodhound redbone coonhound otterhound basset hound 40% 80% bloodhound redbone coonhound otterhound basset hound 40% 80% bloodhound bull mastiff
Rhodesian ridgeback redbone
Great Dane
Benign 40% 80% koala wombat maypole golfcart dromedary
DTMI ( =4)
[Table]
% 80% koala wombat maypole golfcart
Lemur
[Table]
DTMI ( =12)
[Table]
catamount wall
[Table]
fox squirrel
[Table] [Table]
wall
[Table]
analog
[Table]
fox squirrel grey fox
GA-DTMI
[Table]
fox squirrel koala hare
[Table]
analog clock 40% 80%
Bedlington terrier
Dandie Dinmont rosehip standard poodle
Sealyham 40%
[Table]
Dandie
[Table]
Sealyham miniature
[Table]
[Table] [Table]
Sealyham
[Table]
Bedlington
[Table] [Table]
[Table]
Sealyham miniature
[Table]
toy
[Table]
[Table] [Table]
Sealyham
Bedlington
[Table]
toy
[Table]
miniature
[Table]
Benign 40% 80% coucal bulbul magpie
Erythrocebus patas jacamar DMI-FSA ( =1.28)
[Figure] [Figure]
% coucal jacamar bulbul bee eater magpie DMI-FSA ( =2.12)
[Table] [Table]
% jacamar water ouzel coucal magpie hummingbird DMI-FSA ( =3.50)
[Table]
% 80% water
[Figure]
with approximately minimum changes.
To the best of our knowledge, we are the first to explore transfer-based black-box attacks with adaptive perturbation budgets.
• Under ℓ ∞ -norm setting, our geometry-aware framework improves the imperceptibility of the crafted adversarial examples by a large margin without the decrease of transfer success rate (see Fig. 4).
By applying our method GA-DTMI-FGSM to the adversarial competition of a 2021 conference, we ranked 1st place out of 1,559 teams and surpassed the runner-up submissions by 4.59% and 23.91% in terms of final score and average image quality level, respectively.
• Under unrestricted setting, we propose a transfer-based unrestricted attack (see Eq.
(3)) by combining the whitebox feature space attack [24] with the transfer-based ℓ ∞ -norm attacks to generate semantic-preserving yet transferable adversarial examples (see Figs.
1 and 6).
Moreover, the crafted adversarial examples transfer well to adversarially trained robust models (see Table.
III,
Figs.
7 and 9).
II. R ELATED W ORK A. ℓ p -norm Adversarial Examples
Existing gradient-based white-box attacks either search for adversarial examples in a fixed ℓ p -norm ball [25], [26], or optimize the perturbation for each image independently to get a minimum-norm solution such as DeepFool [27], CW [28], and fast adaptive boundary attack [29].
However, white-box assumption usually does not hold in real-world scenarios.
In query-based black-box setting, attackers utilize output logits [30], [31] or predicted label [32], [33] of the target model to generate adversarial examples.
But these attacks typically suffer from high query complexity, making it easy to be detected [34].
Transfer-based black-box attacks [35]–[38] can pose serious threats in practice as they need no information about the defense models.
Dong et al.
[16] boosted transferability by integrating momentum into gradient-based methods.
Liu et al.
[14] found that attacking a group of substituted source models simultaneously can improve transferability.
Besides, transferability benefits from input transformations such as input diversity [17] and translation-invariant method [18].
B. Unrestricted Adversarial Examples
The ℓ p -norm distance is not an ideal perceptual similarity metric [39], [40], which oversimplifies the diversity of realworld perturbations.
Unrestricted adversarial examples have received significant attention in recent years [41].
Most of the current unrestricted attacks aim to generate imperceptible adversarial examples under white-box setting, such as geometric transformations [19], [42], [43] and distance metrics beyond ℓ p norm [20], [21].
Color-based attacks [44]– [49] were also proposed to generate large but imperceptible perturbations, however, the modified color can sometimes be unnatural.
Instead of optimizing in the input space, generative approaches [50]–[53] search for adversarial embeddings in the latent space.
Style transfer [49], [54] is inherently an unrestricted attack as it preserves the semantic of the content image.
However, constructing transferable unrestricted adversarial examples is still less explored.
In this work, we will fill this gap by combining the white-box feature space attack [24] with transfer-based ℓ ∞ -norm attacks to generate semanticpreserving yet transferable unrestricted adversarial examples.
C. Adversarial Defenses
There have been long-standing arms races between defenders and attackers.
Adversarial training [4] is one of the most promising defense methods.
Many variants of adversarial training framework were proposed, e.g., ensemble adversarial training [14] for transfer-based attacks, PGD-based adversarial training [25], and TRADES [55] with a new robust loss based on the trade-off between robustness and accuracy.
Geometryaware instance-reweighted adversarial training [56], which is proven falling into gradient masking [57], shares similar insights with us that the importance of distinct inputs in adversarial training should be different.
Laidlaw et al.
[21] integrate adversarial training with Learned Perceptual Image
Patch Similarity (LPIPS) [58], aiming to improve robustness against perturbations that were unseen during training.
Unlike empirical defenses, Certified defenses [59]–[62] could provide robustness guarantee under a certain ℓ p -norm budget.
III. P RELIMINARIES
Notation.
A deep neural network classifier can be described as a function f ( x ; θ ) :
X → R C , parameterized by weights θ , which maps a vector x ∈ X to its output logits.
Given an input x of class y ∈ { 1 , 2 , · · · , C} , the predicted label of f ( x ; θ ) is ˆ f ( x ) :
=
arg max j f j ( x ; θ ) , where f j ( x ; θ ) represents the j -th entry of f ( x ; θ ) .
We use L ( f ( x ; θ ) , y ) to represent the crossentropy loss and denote the ε -neighborhood of x by B ( x , ε ) :
=
{ x ′ ∈ X : D ( x , x ′ ) ≤ ε } , where D is a distance metric that describes the change between the adversarial example x ′ and the nature example x .
We denote the black-box test model by g , and split the set of accessible source models into the set of training models f and the set of validation models h .
A. Transfer-based ℓ ∞ -norm Attacks
Existing transfer-based attacks typically search for adversarial examples in a fixed-radius ℓ p -norm ball, i.e.,
D ( x , x ′ ) =
∥ x ′ − x ∥ p ≤ ε .
Various methods were proposed to boost transferability of the generated adversarial examples, such as input Diversity Iterative Fast Gradient Sign
Method (DI-FGSM)
[17], Momentum-based Iterative (MIFGSM) method [16] and Translation-invariant Iterative (TIFGSM) method [18].
We formulate a strong ℓ ∞ -norm baseline
DTMI-FGSM by combining all these techniques, i.e.,
[Equation]
where m
= 0 , W is a pre-defined kernel with a convolution operation ∗ , α is the step size, Π is the projection operator, and γ is the decay factor for the momentum term.
T ( x t , p ) represents the input transformation on x t with probability p .
When γ = 0 , DTMI-FGSM attack degenerates to the DTIFGSM attack.
When p = 0 , DTMI-FGSM attack degenerates to the DMI-FGSM attack.
B. Transfer-based Unrestricted Attack
Inspired by prior work [63] in style transfer, Xu et al.
[24] tries to find stylized adversarial examples by assuming that the image pairs from the same class share consistent content and differ mainly in their styles.
Here we propose to generate semantic-preserving yet transferable unrestricted adversarial examples by combining the Feature Space Attack (FSA) [24] with transfer-based ℓ ∞ -norm attacks [16], [17].
Given an encoder ϕ , we extract the style features of input x as channelwise mean µ ( ϕ ( x )) ∈ R C and channel-wise standard deviation σ ( ϕ ( x )) ∈ R C , Specifically,
[Equation]
where ϕ ( x ) ∈ R C × H × W represents the latent embedding.
Xu et al.
[24] adds adversarial perturbations on µ and σ before projecting ϕ
( x ) back to the input space X with a pre-traineddecoder ϕ − 1 , namely,
[Equation]
where ˜ ϕ
( x ) enlarges or shrinks the mean µ and the standard deviation σ of the embedding ϕ ( x ) by a factor of e τ µ and e τ σ , respectively.
In this way, the distance metric D ( x , x ′ )
= max (cid:0) e ∥
[Equation]
, e ∥
[Equation]
∞ (cid:1) ≤ ε .
In order to preserve the semantic of the unrestricted adversarial example x ′ , a content loss was added during the attacking process, i.e.,
[Equation]
where λ balance the trade-off between adversarial and the content loss.
Following Xu et al.
[24], we set λ
= 128 and use the top-5 margin loss for adversarial attack.
With all above, the unrestricted attack (see Eq.
(3)) can be solved by conventional ℓ ∞ -norm attack on parameters τ µ and τ σ .
Moreover, the same techniques in Sec.
III-A such as input diversity T ( x ′ , p ) (only for the margin loss) and momentum-based method can be integrated to improve transferability, i.e.,
[Equation]
When γ = 0 , the DMI-FSA attack degenerates to the DI-FSA attack.
f g h initial / source example
geometry-aware perturbation path minimum-norm adversarial example of f fixed-radius adversarial example of f norm balls centered at source example decision boundary of training model f decision boundary of validation model h decision boundary of test model g
C. Evaluation Metric for Transfer-based Attack
The imperceptibility of adversarial examples is hard to evaluate due to the lack of precise quantization of human perception [64].
Sharif et al.
[65] found that ℓ p -norm distance is not an ideal perceptual similarity metric and suggest setting adaptive perturbation budget for every sample to ensure that the attacks’ output would be imperceptible.
Therefore, we choose transfer success rate and the perturbation budget under distance metric D as our main evaluation metrics.
Consider a dataset ˆ S
= ( x i , y i ) } ni =1 and the corresponding adversarial examples ˆ S adv
= { ( x ′ i , y i ) } ni =1 that are crafted on the training model f .
Let n [-
-]0 = (cid:80) ni
=1 1 { ˆ g ( x ′ i )
̸ = y i } be the number of misclassified adversarial examples on the test model g .
We define the average total score as:
[Equation]
where S APR is the Average Perturbation Reward of adversarial examples that are misclassified by test model g and the reward function F reward is a decreasing function w.r.t.
metric D ( x , x ′ ) .
IV. M ETHODOLOGY : G EOMETRY -A WARE F RAMEWORK
Eq.
(5) factorizes the average total score as the product of transfer success rate and average perturbation reward, which motivates us to find the adversarial example with minimum changes under metric D , i.e.,
[Equation]
However, direct optimization of problem (6) is intractable, in part due to the lack of information about test model g .
We approximately solve this problem by discretizing the
Require:
Benign input x with label y ; training models f ; validation model h ; number of sub-procedures K ; maximum perturbation size ε and threshold η ; attack algorithm A ;
Ensure:
Transfer-based unrestricted adversarial example x ′ with approximately minimum change; 1: x 0 = x ; 2: for k = 1 , 2 , · · · , K do 3: x k
=
A ( x , x k − 1 , f, kεK ) ; ▷ fixed-budget
attack
4: conf ← exp(
[Equation]
j
[Equation]
; 5: if conf < η then 6:
Return x k ; ▷ early-stopping in Eq.
(7) 7: end if 8: end for 9: Return x K ; continuous space of perturbation radius into a discrete set and choosing the minimum perturbation budget such that the attack is able to fool the test model g .
However, the challenge is that it is typically difficult to decide whether a given perturbation radius can also fool the test model [22], [23].
This problem is also known as model selection problem (we view the source models as training data, then the generated adversarial perturbation is the so-called selected model or optimized parameters), and a classic approach to tackle this problem is to split a validation set.
More specifically, we split all accessible source models into training model set and validation model set.
With validation model h , we are able to generate transferable adversarial examples with smaller changes.
To approximately solve problem (6), we first divide the attack in the ball B ( x , ε ) into K sub-procedures.
In the kth sub-procedure, we re-run a fixed-radius attack algorithm A
[Table]
such as DMI-FSA (see Eq.
(4)) under the perturbation budget
[Equation]
We start each sub-procedure from the solution of the last sub-procedure to accelerate the convergence.
To obtain a minimum-radius solution, we perform an early-stopping mechanism at the end of each sub-procedure if the probability of the true class on the validation model h is smaller than a threshold η , i.e.,
[Equation]
Our GA framework is summarized in Algorithm 1 and illustrated in Fig.
V. E XPERIMENTS A. Experimental Setup
Datasets & Networks.
Similar to Xie et al.
[17], we randomly select 1,000 images from ILSVRC 2012 validation set [66], which are almost correctly classified by all the attacking models.
All these images are resized to 229 × 229 × 3 beforehand.
We consider eight normally trained models and two ensemble adversarially trained models [15].
The weights of all these models are publicly available [67].
More details about the networks are summarised in Table.
I.
The transferability between these models under ℓ ∞ -norm setting is summarised in
Fig.
It is much easier for the generated adversarial examples to transfer from vision transformers to convolutional neural networks (CNNs), which is consistent with the empirical observation in Shao et al.
Surprisingly, the robustness of naturally trained vision transformers under transfer attack is even on par with two ensemble adversarially trained CNNs.
Implementation Details.
Given the maximum perturbation size ε and number of sub-procedures K (5 as default) in our geometry-aware framework, we set the step size α = 1 .
25 × ε k T in the k -th sub-procedure, where the number of iterations T is set to 10 in the ℓ ∞ setting and 50 in the unrestricted setting.
ε is set to 20 in the ℓ ∞ -norm setting and 3.5 in the unrestricted setting.
When running a fixed-radius baseline at perturbation budget ε k , we set the number of iterations N = T 2 (cid:0) 1 + Kε k ε (cid:1) with step size α to keep the same total perturbation budget (the sum of step size across all iterations) as our geometryaware framework for fair comparison.
The reward function
F reward ( ε 0 ) is set to 1 /ε
as smaller perturbation radius exhibits significantly higher image quality.
For the momentum term,
V i T - S / 16 V i T - B /
S w i n - B / p a t c h- w i n d o w
R e s N e X
t- 32 x 8 d - s w s l
R e s N e
[Table]
I n
[Table]
p t
[Table]
n
[Table]
n
[Table]
t
[Table]
p t i o n
[Table] E n s [Table] a d v
[Table] n c [Table] p t i o n
[Table] R e [Table] N
[Table]
[Figure]
we set the decay factor µ = 1 as in Dong et al.
For DI-FGSM [17], we set the transformation probability to p = 0 .
The input is first randomly resized to be an r × r ×image with r ∈
[(1 − γ ) s, (1 + γ ) s ] , and then padded to size (1+ γ ) s × (1+ γ ) s × 3 .
The transformed input is then resized to s × s × 3 for different input size s of various models, i.e., 224, 299 and 384.
We set γ = 0 .
1 as default.
For TI-FGSM [18], we use Gaussian kernel with kernel size 5 × 5 .
B. Balancing Transfer Success Rate and Perturbation Reward
Implementation Details.
Benefiting from the adaptive choice of perturbation budgets, our geometry-aware framework can generate transferable unrestricted adversarial examples with smaller changes.
In this experiment, the training model f and validation model h are an ensemble of models { 2 , 3 , 5 } and { 1 , 4 , 6 } in Table.
I, respectively.
The test model is InceptionResNet-v2.
The optimal threshold η (see Eq (7)) is searched from a finite set ranging from 0.001 to 0.9 by querying 2 the
[Figure]
test model g to achieve the best average total score S
For each η , we execute our method and compute the average perturbation reward S APR (the x -axis of each red point in
Fig. 4).
Then the corresponding fixed-radius baseline is run at the same x -axis.
We conduct experiments on two threat models.
For the ℓ ∞ -norm setting, we combine our GeometryAware (GA) framework with DI-FGSM, DTI-FGSM, DTMIFGSM, and Admix-DTI-FGSM [37] (limited by the memory of a single NVIDIA RTX 3090, we set the number of admixed images m 1 = 3 and the number of randomly sampled images from other categories m 2 = 2 ), named GA-DI-FGSM, GADTI-FGSM, GA-DTMI-FGSM, and GA-Admix-DTI-FGSM, respectively; For the unrestricted setting, we combine our GA framework with DMI-FSA and DI-FSA, named GA-DMI-FSA and GA-DI-FSA, respectively.
Experimental Results.
We present the contour plot of average total score in Fig.
4, where the improvement of our method upon baselines depends on the choice of hyper-parameter We report the results when both our approach and baselines achieve highest [Table]
[Figure]
η (leading to different S APR ).
Fixing S APR as 0.115, our approach GA-DTMI-FGSM surpasses the baseline DTMIFGSM by up to 16.1% in terms of transfer success rate;
As shown in Table.
II, our approach yields a significant performance boost on the average total score S total across various threat models, especially in the ℓ ∞ -norm setting where both the transfer success rate and S APR are improved.
C. Case Study: Adversarial Competition of a 2021 Conference
In the adversarial competition of a 2021 conference, contestants were asked to submit adversarial examples without any access to the defense models.
The dataset is a subset of
ILSVRC 2012 validation set [66], which consists of 5, images with 5 images per class.
The final score of each submission was manually scored from two aspects: 1) image semantic and 2) quality.
If the semantic of the submitted image changes (judged by human referees), then S s = 0 , otherwise S s = 1 .
The image quality S q (equivalent to our reward function F reward ) was quantified with five levels S q ∈ { 1 , 2 , 3 , 4 , 5 } by multiple human referees.
The final score is given by (cid:80) i
[Equation]
We apply our method GA-DTMI-FGSM ( η = 0 . 01 ) to the competition, where our entry ranked 1st place out of 1,559 teams.
We report the final score and average image quality level (equivalent to our average perturbation reward) in
Fig.
It shows that our method outperforms other approaches by a large margin.
In particular, we surpass the runner-up submissions by 4.59% and 23.91% in terms of final score and average image quality level, respectively.
D. Transferable Unrestricted Adversarial Examples
Most of current defenses can be easily broken by unseen attacks in a white-box manner.
Adversarial training against multiple ℓ p -norm attacks [75] solved this issue partially, however, at the cost of robustness against single ℓ p -norm attack.
Laidlaw et al.
[21] integrated adversarial training with Learned Perceptual Image Patch Similarity (LPIPS) [58], aiming to improve robustness against perturbations that were unseen during training.
However, the proposed attack [21], similar to other unrestricted attacks [43], [45], suffers from weaker transferability to the target model.
In practice, attackers
[Figure]
Defenses
[Table]
Resnet152-Base [70]
[Table]
Resnext101-DenoiseAll [70]
[Table]
Resnet152-Denoise [70]
[Table]
[Table]
Swin-L/patch4-window
[Table]
typically have no information about the defense models and the defenders do not have the ground truth to make pixel-level comparison (perturbation can be large as long as the generated adversarial examples are semantic-preserving).
Therefore, we propose to benchmark classification models on ImageNet under transfer-based unrestricted attacks .
Implementation Details.
For the ReColor attack [45], we set ε = 1 . 0 and iterations T = 100 which achieves 89.7% attack success rate on training model f
(the same as Sec.
V-B) and 9.1% transfer success rate on the test model
Inception-ResNetv2.
For FSA attack [24], we set ε = 3 . 0 and T = 500 which achieves 54.2% attack success rate and 12.4% transfer success rate on the same training and test models (Note that our method GA-DMI-FSA achieves 95.5% attack success rate and 58.3% transfer success rate, indicating that the input diversity and momentum techniques in Eq. (4) boost both the attacking ability and transferability.).
Besides six adversarially trained and two high-performance classification models, we select two state-of-the-art models on the ImageNet-R dataset [72].
Experimental Results.
From Table.
III, we can conclude the following observations:
a) GA Admix-DTI transfers better than GA DTMI-FGSM . b) ℓ ∞ -norm transfer attack can hardly break ℓ ∞ -norm adversarially trained models while the unrestricted attack (GA-DMI-FSA) reduces the accuracy of these models by a large margin (see also in Fig. 7).
c) DeepAugment [72], which utilizes semantic-preserving augmentations during training, exhibits non-trivial robustness against GADMI-FSA attack.
d) Efficientnet-l2-ns [73] performs well under all the transfer-based attacks and enjoys 86% accuracy against GA-DMI-FSA attack, showing that the distribution of our generated adversarial examples is not too far from the natural examples’.
Note that Efficientnet-l2-ns is also the best-performing model on the ImageNet-V2 dataset [77], [78].
We visualize part of the transfer attack results on adversarially trained model Resnext101-DenoiseAll [70] in Figs.
7, where our method GA-DMI-FSA is able to generate semanticpreserving yet transferable unrestricted adversarial examples.
For more visualization results, please see Fig. 9 in Sec.
VII-A.
Benign
GA DTMI-
ReColor
FSA
GA DMI-FSA  ε =   ε = 1 .  ε =   ε = 1 .
ε =   ε = 1 .
ε =   ε = 2 .
E. Ablation Studies and Discussions
The optimal η depends on the combination of f, h, g .
As illustrated in Fig.
2, the effectiveness of our geometry-aware framework relies on the relationship between the decision boundaries of f , h , and g .
Thus we compared various kinds of partition of training, validation, and test models in Fig.
8, from which we can conclude the following two observations: a) The role of training model f and validation model h is not symmetrical .
The transfer success rate changed a lot when we exchanged the role of models { 4 , 5 , 6 } and { 1 , 2 , 3 } for training and validation.
b) Ensembling more models yields better performance.
When adding models 8 and 9 into f = (2 , 3 , 5) and g = (1 , 4 ,
) respectively, there is a remarkable improvement in the average total score. Guidelines on how to split the source models. A natural question is how to choose the training model f and the validation model h . Firstly, given a group of fixed source models, the optimal train-valid split must be related to the test model g .
Thus we argue that there is not an optimal train-valid split in practice since the test model is unknown. When the test model is queryable, we can traverse all possible partitions to select the best split which achieves the highest score. However, this approach is time-consuming and the maximum number of queries is usually rather limited.
Fortunately, we can extract useful guidelines on how to initialize a satisfactory partition from the experiments through post-hoc explanations: 1) Ensembling as many models as possible. In the adversarial [Figure]
[Table]
[Figure] [Table] [Figure] g =(2,3,5),(1,4,6),7 f , h , g =(1,2,3,8),(4,5,6,9),7 f , h , g =(1,2,3,4,5,6),(8,9), competition, our training and validation model are both an ensemble of disjoint eight high-performance models.
2) f and h cannot be too similar.
As shown in the first group in Table.
IV where f = h , the transfer success rate decreases quickly when K increases, indicating that early stopping on the training model itself will lose transferability to unseen models.
A 29.5% success rate is not acceptable this issue where test model g = { 7 } and η = 0 .
Larger
K means smaller possible perturbation radii, thus leading to higher S APR when η is fixed.
[Table]
VI. C ONCLUSION
In this work
[Table]
we propose a geometry-aware framework, where fixed-radius methods can be integrated to generate transferable unrestricted adversarial examples with minimum changes.
Under ℓ ∞ -norm setting, our framework could improve the imperceptibility of the crafted adversarial examples by a large margin without the decrease of transfer success rate.
Besides
[Table]
we propose a transfer-based unrestricted attack by combining the white-box feature space attack with transferbased ℓ ∞ -norm attacks to generate semantic-preserving yet transferable unrestricted adversarial examples
[Table]
A. More Visualization Results
Benign
GA DTMI-FGSM
ReColor
FSA
GA DMI-FSA  ε =   ε = 1 .  ε =   ε = 2 .
ε =   ε = 1 .
ε =   ε = 2 .
ε =   ε = 1 .
ε =   ε = 2 .
ε =   ε = 1 .  ε =   ε = 1 .
We thank all the reviewers for their detailed and constructive comments, which greatly helped us improve the manuscript.
We have comprehensively addressed these comments in the revision, and we believe the manuscript has been notably improved.
We feel encouraged that our geometry-aware (GA) framework is acknowledged by the reviewers:
• “ The idea behind GA sounds interesting. ”
• “ The paper is well-written.
The ideas are new, wellmotivated and well performing in practice. ”
Below are our responses to each reviewer’s questions:
1) Reviewer #1: • “There is no insight and/or analysis into why and how the current realization of GA framework is (always) capable of finding the minimum change adversarial example with a fixed set of K sub-procedures.
Apart from transferability performance to different target models (like Fig. 4) and some qualitative figures (1 and 6), there is no analysis in this regard.”
[Re: Finding the optimal minimum-norm adversarial perturbation is intractable for deep networks
even
in
white-
box
attacks.
For example,
DeepFool [79] demonstrates its effectiveness by showing that their perturbations have smaller ℓ ∞ -norms
compared
to FGSM when achieving a comparable attack success rate (similar to our Fig. 4).
Our framework GA approximately solves
Eq.
(6) by discretizing the continuous perturbation radius into K candidates and
choosing the minimum candidate
that
suffices to fool
the validation model with a
certain
margin.
Following we provide two insights why GA could achieve a good solution to Eq
: 1) K is capable of controlling the trade-off between
S APR and efficiency.
The early-stopping mechanism on validation model h could make it possible to generate smaller perturbations without a sharp decrease in transfer success rate (TSR).
To
verify
the above two insights, we add three groups of experiments
in
Table
IV.]
• “ In the proposed framework how to partition the available white-box source models into training and validation sets is quite unclear.
There are no clear guidelines.
Furthermore, the ablation studies in Fig.
8 doesn’t explain why certain combination splits work better than others.”
[Re: we provide two guidelines in Sec.
V-E and add detailed description about Fig.
• “The GA framework seems quite sensitive to the eta parameter and is also dependent on the specific partition of source models into training and validations sets (Fig. 8).”
[Re: It’s not GA sensitive to η , but the optimal η ∗ is dependent on the relationship between f, h and g , which might be inevitable in transfer-based black-box attacks.
In practice, η is firstly sampled from a prior distribution and then finetuned by
the
feedback
of
TSR.
Fortunately from Fig.
8, we find it effective to set η ∈ (0 . 02 , 0 . 2)
[Equation]
• “Parts of the notation could be clearer to improve readability of the presented ideas.
Examples for this is the use of lower-case letters for both single models and sets of models.
In addition, while the implementation choices are well documented, making the code base publicly available would allow for much easier reproduction of the results.”
• “ The ideas presented in the paper are new, wellmotivated and well performing in practice, and they can be easily used with other attacks.
The authors show extensive experimental evidence regarding the performance of their method.
Parts where the paper could benefit is more care to finest details in the notation and making the codebase available for easier reproduction.”
• “After reading the rebuttal and other reviewers comments, I agree that more experiments and fairer comparisons to baselines are needed.
Resubmitting it to the next venue would be beneficial for the paper’s contributions and strengths.”
[Re: For the potential fairness concern, we add detailed explanations in Sec. ?? .]
3) Reviewer # • “Lack of SOTA methods of transfer-based attack in the related work, e.g., SIM, SGM, Admix, IR.”
[Re: As also mentioned in the strengths@Reviewer #2, GA allows for easy use of incorporation of arbitrary fixed-budget transfer-based attacks.]
• “Is the C in L180 (the number of classes) and the C in L230 (the number of channels in the feature) the same meaning?
If not, please revise the notation with another alphabet.”
[Re: They do not have the same meaning, we have changed the notation of the number of classes to C .]
• “The concern about the fairness in the experiments.
The proposed GA methods utilize both training models f and validation model h when generating the adversarial examples.
In the baseline method of comparison, whether the ensemble model of both f and h is used?
In
L414 the author says that they choose the threshold eta by querying the test model g. I suppose that test model should be a black-box model and totally can’t be obtained by attackers.
I think this behavior of tuning hyperparameters on the test set is unacceptable.”
[Re: (1) The ensemble model in baseline uses f only but we add detailed explanations in Sec. ?? .
(2) We provide three points of view that querying the test model g for TSR is acceptable in our designed experiments in
Sec.
V-B: 1) We query TSR on the test model to select the optimal η that exhibits the best performance.
For a fair comparison, the baseline method also queries the test model multiple times at the corresponding x -axis to find the best fixed x that performs well.
We only query the TSR as feedback 3 , i.e., the logits and the predict label of single query are still not accessible.
However, conventional query-based attacks may require thousands of queries on logits or predict label for each data. 3)
We might DO NOT need (or only several times) this query procedure in practice by following the guidelines proposed in Sec.
V-E.]