Aug-ILA: More Transferable Intermediate Level Attacks with Augmented References
Authors: Chiu Wai Yan
Dit-Yan Yeung
Venue: ICLR 2022 Submitted (2022)
Keywords: adversarial examples
adversarial transferability
intermediate feature
image augmentation

Abstract: An intriguing property of deep neural networks is that adversarial attacks can transfer across different models. Existing methods such as the Intermediate Level Attack (ILA) further improve black-box transferability by fine-tuning a reference adversarial attack, so as to maximize the perturbation on a pre-specified layer of the source model. In this paper, we revisit ILA and evaluate the effect of applying augmentation to the images before passing them to ILA. We start by looking into the effect of common image augmentation techniques and exploring novel augmentation with the aid of adversarial perturbations. Based on the observations, we propose Aug-ILA, an improved method that enhances the transferability of an existing attack under the ILA framework. Specifically, Aug-ILA has three main characteristics: typical image augmentation such as random cropping and resizing applied to all ILA inputs, reverse adversarial update on the clean image, and interpolation between two attacks on the reference image. Our experimental results show that Aug-ILA outperforms ILA and its subsequent variants, as well as state-of-the-art transfer-based attacks, by achieving $96.99\%$ and $87.84\%$ average attack success rates with perturbation budgets $0.05$ and $0.03$, respectively, on nine undefended models.

Under review as a conference paper at ICLR 2022
AUG-ILA: MORE TRANSFERABLE INTERMEDIATE LEVEL ATTACKS WITH AUGMENTED REFERENCES
Anonymous authors Paper under double-blind review
ABSTRACT
An intriguing property of deep neural networks is that adversarial attacks can transfer across different models. Existing methods such as the Intermediate Level Attack (ILA) further improve black-box transferability by fine-tuning reference adversarial attack, so as to maximize the perturbation on pre-specified layer of the source model. In this paper, we revisit ILA and evaluate the effect of applying augmentation to the images before passing them to ILA. We start by looking into the effect of common image augmentation techniques and exploring novel aug- mentation with the aid of adversarial perturbations. Based on the observations, we propose Aug-ILA, improved method that enhances the transferability of an existing attack under the ILA framework. Specifically, Aug-ILA has three main characteristics: typical image augmentation such random cropping and resiz- ing applied to all ILA inputs, reverse adversarial update on the clean image, and interpolation between two attacks on the reference image. Our experimental re- sults show that Aug-ILA outperforms ILA and its subsequent variants, as well as state-of-the-ar art ransfer-based attacks, by achieving 96.99% and 87. 84% average attack success rates with perturbation budgets 0.05 (13/ (13/255) and 0.03 (8/255), respectively, on nine undefended models.
INTRODUCTION
Recent studies (Szegedy et al., 2013; Goodfellow al.] 2015) showed that deep neural network (DNN) models are vulnerable to adversarial attacks, where perturbations are added to the clean data to fool the models in making erroneous classification. Such adversarial perturbations are usually crafted to be almost imperceptible by humans, yet causing apparent fluctuations in the model output. The effectiveness of adversarial attacks on deep learning models raises concerns in multiple fields, especially for sensrity applications.
Another intriguing phenomenon is that adversarial attacks can transfer across different models (Pa- pernot et al., 2016). One explanation for this phenomenon is the overlapping decision boundaries shared by different models (Liu et al. 2017] Dong et al.) 2018). In many black-box settings where the attacker has access to the internal state of the model, such transferability can be exploited either to generate attacks from surrogate model (Zhou et al. 2018) or to provide proper guidance reduce the number of model queries (Guo al. 2019).
While many methods set their goals to generate highly transferable attacks -eosavi-Dezfoooli et al., 2016 Wu et al. 2020 Guo al. 2020), some attempt improve the transferability of given adversarial example by fine-tuning it. Huang et al. (2019) proposed the Intermediate Level Attack (ILA), technique that takes an existing adversarial example as reference and boosts its black-box transferability by maximizing the projection of the intermediate feature map discrepancies between attacks. After being fine-tuned by ILA, a strong white-box attack can achieve remarkable black-bor transferability, outperforming previous methods based on direct generation of transfer-based attacks (Zhou et al. 2018 Xie et al. 2019). More specifically, ILA takes three input images as references to fine-tune the attack, including clean image, reference attack of the clean image, and the updated image from the previous iteration. Both the clean image and reference attack remain unchanged throughout the fine-tuning process. If we view the fine-tuning tuning process of ILA a generalization task that attempts to make an attack effective for different models, possible direction for improvement increase the diversity of the input references, such through applying data augmentation.

---

Under review as a conference paper at ICLR 2022
Clean Image
I-FGSM I-FGSM (Ours)
Figure 1: Visualization of the generated images among: clean image, IIA, and Aug-II.A (Ours), with the perturbation budget e 0.03. Rather than adding uninterpretable noise, Aug-ILA appears smoothen original fine texture and overwrite new texture on the image.
In this paper, we revisit ILA and explore possible room for improvement. A natural way to promote input diversity is to apply data augmentation techniques to the input references. We first evalu- ate the effect typical image transformation operations in the context of ILA. Then, we look into transformations exploiting the adversarial perturbation, and apply them together with the best image transformation found previously. Specifically, we incorporate a mixture of augmentation opera- tions: random image cropping and resizing all input examples, reverse adversarial update on the clean image, and attack interpolation on the reference attack. Incorporating these adaptations to the original ILA, we propose the Augmented Intermediate Level Attack (Aug-ILA) to further enhance black-bor box transfcrability. By fine-tuning simple baseline attacks such as the Iterative Fast Gradi- ent Sign Method (I-FGSM), Aug-ILA not only outperforms ILA and its subsequent variants, but also state-of-the-aae ahr transfer-based attacks. Visualizations of the adversarial examples fine-tuned by Aug-ILA are illustrated in Figure [ and Appendix N
Our main contributions can be summarized as follows:
We analyze the formulation of ILA and conduct extensive experiments to study the effect some common image transformation operations as well transformations exploiting the adversarial perturbation for augmenting the ILA references.
We propose Aug-I a method that fine-tunes a given adversarial attack to reinforce its attack transferability. Aug-ILA follows the framework of ILA, but with effective augmentations introduced to its references. The resulting black-box transferability of Aug-ILA outperforms the original ILA by 18.97% on average with perturbation budget 0.03( 8/255).
We identify factors affecting the transfer success rate and explain the roles of intermediate feature perturbation and augmentation in a transfer-based attack.
2 RELATED WORK
In the context of adversarial machine learning, the common threat models are the white-box and black-box settings. The white-box setting assumes that one has access to the victim model's internal state, including its gradient, parameters, training dataset, etc. The black-box setting, on the other hand, only allows querying the model with input but prohibits subsequent access to the model information. While more variations such the grey-box and no-box settings exist, they are generally not considered in this work.
Typical white-box attacks exploit the gradient of the model to generate adversarial examples. The Fast Gradient Sign Method (FGSM) (Goodfellow et al. (2015) generates attacks by adding the signed gradient of the loss with respect to the image back to the image, obtaining a higher loss and possibly incorrect prediction. The Iterative Fast Gradient Sign Method (I-FGSM, also known as BIM) (Ku- rakin et al. [2017a performs FGSM iteratively while clipping the difference between the adversarial example and the original image within the perturbation budget. Projected Gradient Descent (PGD) (Madry et al., 2018) first initiates a random start within the perturbation budget and runs I-FGSM to update the attack.
2

---

Under review as a conference paper at ICLR 2022
In the black-box setting, one estimate the model information by repeatedly querying the model with varying inputs (Chen et al., 2017; Brendel et al. 2018). Recently, some novel methods have been proposed in order reduce the number of queries (lyya et al. 2018 Bhagoji al. 2018 Li et al., 2020a). However, the number of queries to the models is still abnormally large, making it easy to be detected by real-world systems. This turns the spotlight back to waasfer-based attacks, where adversarial examples mainly rely on black-box transferability instead of consecutive attempts.
2.1 TRANSFER-BASEE ATTACKS
Since strong gradient-based white-box attacks do not have high transferability by themselves (Ku- rakin et al. 2017b), a number of attempts have been made in the research community to improve transterability apart from increasing the attack strength.
MI-FGSM. Dong et al. (2018) first introduced the Momentum Iterative Fast Gradient Sign Method (MI-FGSM) to incorporate momentum to stabilize the update direction iterative attacks, resulting in more transferable attacks.
DIM. The Diverse Input Method (DI2-FGSM, or DIM) (Xie et al. 2019 applies random resizing and zero-padding with a certain probability to the image in each iteration of I-FGSM.
TIM. Dong et al. (019) pointed out the discretion of attention map between the defended and undefended models and proposed the Translation-Invariant Method (TIM) by attacking an ensemble of translated images. To reduce the computational overhead, was shown that kernel convolution can be applied to the gradient achieve similar effects.
SIM and NI-FGSM. Lin et al. (2020) introduced the inveriinvriannt property of DNN and proposed the Scale-Invare Invariant Method (SIM) to attack ensemble of images with scaling factor of 0.5 on the pixel values. In the same paper, the Nesterov Iterative Fast Gradient Sign Method (NI-FGSM) was introduced, aiming escape from poor local maxima with the Nesterov accelerated gradient.
VMI-FGSM. Wang & He (202I) proposed variance tuning. For each iteration, the gradient is ad- justed with an expectation of gradient sampled from the image's 's neighborhood. Combining variance tuning with the composition of previous methods, denoted as the Composite Transformation Method (CTM), they achieved one of the strongest cransferbbase black-box attacks on defended models.
A more recent work (Wu et al. 2021) replaced all input transformations with a DNN model trained apply distortions that neutralize the adversarial noises. After preparing the denoising model, the attack is generated with the objective to sustain such distortions. All these methods suggested that adversarial transferability could be improved by augmenting the images or tuning the magnitude of gradient updates.
Alternatiely,, some recently proposed methods exposed the surprising nature of CNN model ar- chitectures and utilized them to generate transferable attacks. [Wu et al.) (2020) found that attack transferability can be enhanced by scaling up the gradient in the skip connection and referred the technique as Skip Gradient Method (SGM). Guo et al. (2020) revised the linear property of DNN models and proposed LinBP to propagate backward without considering the non-linear layers.
2.2 INTERMEDIATE LEVEL ATTACK FOR FINE-TUNING ADVERSARIAL EXAMPLES
Consider input image existing adversarial attack x', model F, and function Fi that outputs the feature maps at layer I of the model. The original ILA projection loss proposed by Huang et al. 2019) minimizes the dot product:
(xx,x',x xx)x -- Avi
(1)
where Ay and Ay" are two vectors defined as follows:
By' [xxx) [x()
(2)
Ay Fi(x") - Fx))
(3)
This is equivalent to maximizing Ay" Ay/. The idea behind the formulation is to increase the norm without sacrificing the perturbation direction that causes misclassification. Due to the constraint of the perturbation budget. there is hardly room for fine-tuning in the image space. Instead, projection maximization is carried out in the feature space, specifically at layer of the model.
3

---

Under review as a conference paper at ICLR 2022
Based on the same framework, Li et al. (2020b) proposed another formulation of ILA:
max (Fi(x")
Fi(x)) w
(4)
x""
where w" is pre-computed parameter vector that directly maps the intermediate- discrepancies to predict the adversarial loss, skipping the remaining model layers. One notable trick of the design that the computation of w involves every feature map discrepancy Fi(x,) Fx(x in each iteration
t during the generation of x', We refer to this attack as ILA++ in the rest of this paper.
OUR METHOD
Based on what we have observed from the previous work, adversarial attacks tend exhibit higher transferability when more references are involved. The increase of references can be achieved by data augmentation and reusing the temporal information all three ILA input references x, x' and x" are images, suggesting the possibility of applying image augmentation like that for representation learning to improve model generalization.
multiple iterations. From Equation M.
One natural way to augment the images is to apply image transformation to the input before retriev- ing the intermediate feature maps. Consequently, the function Fi can be substituted by
(5) where T can be any image transformation function. However, unlike attacks that enforce wrong prediction at the logit output, ILA aims to maximize the projection of feature map discrepancies.
(x) = (x)())
This requires perfect pixel alignment of x, x' and x", a tiny shift in the image pixel induces misalignment in the feature map. Spatial transformations such as translation, cropping and rotation are required to be applied identically to all three ILA references. Otherwise, the optimization of ILA ends up being trivial in the sense of distorted feature discrepancies. Our claim is empirically verified in Appendix B where the transformation T is not identical for the images.
Different from augmentation that generalizes model representation, our task is to improve attack transferability, which can be interpreted as generalizing an adversarial example to succeed attack- ing more models. Hence, apart from the common image processing methods, may also explore any transformation associated with the adversarial perturbation. Intuitively, augmentation based on be viewed
the adversarial perturbation the final attack. This kind of pixel-wise augmentation can be applied to the images independently without conflicting with most spatial transformations. Algorithm [ depicts the algorithmic details for applying these transformations together to ILA, where the clip function is to ensure that the first argument within the range of the second and third arguments.
evaluation of multiple attacks before fine-tuning
Algorithm Aug-ILL -LA
Require: x,x', Fr. perturbation budget e. step size a, number iterations v,, sequence of transformation
functions T x" + x
for i + 1 to do
Xee + 2x xx'
reverse adversarial update b to ensure the image augmentations are aligned
Randomly initialize
Ay Fx(((')) F (x kx...)
Ay Fi(T(x")) (T(T(
x"
x" - sign V (- Ay?))
X"
+ clip(x" x, c)
x"
+ clip(x", 00, 0,
||
& |Ay |
X + ox"" + (1 oxxx
b attack interpolation
end for return x"
3.1 REVERSE ADVERSARIAL UPDATE ON THE CLEAN EXAMPLE
Let us reconsider Equation 2 To perform ILA, we need to obtain the feature map discrepancies between the reference attack x' and the unperturbed image x. Although is correctly classified
4

---

Under review as a conference paper at ICLR 2022
by the model, there is no information how confident the model is and how well the input can represent the ground-trutt class. In contrast, if x is not only correctly classified, but it is also done with high confidence, the discrepancy of the feature maps is expected to intrinsically reflect more useful information about the attack.
In order boost the confidence of we propose to add negative perturbation to it, which is exactly opposite to applying an adversarial attack. If we consider the entire model, this operation is expected to decrease the loss of the classifier, making the classification task easier by emphasizing features that are more significant for the task. Thus, we also expect such input to highly activate the intermediate layers, providing a better guidance for image update in ILA. Our preliminary study on the effect of reverse adversarial update can be found in Appendix [A] which verifies the benign behavior brought by the reverse adversarial update. With the reverse adversarial perturbation, as expected, the model gives lower loss and higher confidence on the correct class.
Instead of crafting new adversarial attack which would incur extra computational demand, we can extract the perturbation from the existing reference attack x', turning the transformation into
Tadv (x) = x (x' = 2x x
(6)
The idea of such an update is similar to DeepDream (Mordvintsev et al. 2015). in which the image updated towards the models' interpretation of the target class. After the transformation, we look for the feature discrepancies caused by a bi-directional perturbation the (vgn(V.(( x, y)), with the positive side being the adversarial attack and the negative side being the augmentation.
3.2 ATTACK INTERPOLATION ON THE REFERENCE ATTACK
Besides the clean image the reference attack x' is another factor to remain unchanged throughout all the ILA iterations. Given the image transformation in Equation (5). it is unlikely to work with another image transformation unaligned with the other images. Similar to Section 3.1 instead exploit the adversarial noise to be the augmentation.
According the intuition of Best Transfer Direction (BTD) in Huang et al. (019). the stronger the reference attack is, the better ILA performs. This inspires us to strengthen the reference attack during the iterations of ILA, by interpolating the reference attack itself with stronger attack. The output of the previous iteration, x", is good candidate for strong attack. At iteration t, we set
X axx" + (1-a)x;
where a is a weighting factor in the range [0. 1] to control the proportion the two attacks. This augmentation is similar mixup (Zhang al. 2018), except that examples, rather than two images from different classes. By interpolating two attacks, we can strengthen the transferability of the reference attack while preserving its original characteristics.
mixing two adversarial
Based on our preliminary findings, setting to non-trivial values such as 0.5 suffices to yield sat- isfactory performance. However, a constant value lacks consideration of the behavior of the two attacks. Precisely, we hope to perform an adaptive interpolation depending on the effectiveness of x². If x is weak after a single update, a should bias towards preserving the reference attack more than mixing with the new attack. Since ILA focuses on maximizing the linear projection between feature map discrepancies, the norm of the feature map discrepancy can be good indicator of the performance. Consequently, we set
2
Ayyll + Avil1
If we neglect the transformation in Section what obtained from Equations (2) and respectively. Note that the value of a is recomputed
3.1 for simplicity, Ay' and Ay" will remain the same as
in each iteration before interpolation is applied.
A similar approach of reusing previous attacks in the temporal domain can be applied to the reverse adversarial update in Section The reference attack in Equation (6) can be further extended to a collection of x for every iteration t. This is not the only work to consider the temporary values of the reference attack before its convergence. ILA++ (Li et al.) (020b) also collects losses computed during the generation of the reference attack, achieving superiority over the standard ILA. Different
5

---

Under review as a conference paper at ICLR 2022
from Li et al. (2020b) which uses past losses (with respect to x') to help update x,, make of the past x directly as an augmentation to enrich the information regarding feature discrepancies under different attacks.
4 EXPERIMENTAL RESULTS
In this section, we start by comparing the effect of different image transformations. After combin- ing the transformations that benefit the most, we evaluate the attack transferability of Aug-IIA in comparison with previous methods based on fine-tuning as well as state-of-the-an transfer-basee based at- tack generation methods. Finally, we investigate the selection of parameters and identify influential factors to an attack's 's transferability.
Setup. Following the line of work in transfer-based attacks, we use total of nine classification
models to evaluate attack transferability, including ResNe150 (He et al., 2016). VGG19 (Simonyan & Zisserman 2013). Inception V3 (Inc. V3) (Szegedy al. 2016), Wide ResNet50 2x (WRN) Zagoruyko & Komodakis 2016). DenseNet161 (DenseNet) Huang et al., 2017). ResNeXt101 (ResNeXt) (Xie et 2017, MobileNet V2 (MobileNet) (Sandler et al. 2018). PNASNet5 (PNAS- Net) (Liu et al., 2018) and SENe150 (SENet) (Hu et al.) 2018). Among these models, we use ResNe150, VGGT9 and Inception V3 the source models of the attack. All of the models pretrained on ImageNet (Russakovsky et al. 2015). with the model parameters of PNASNe and S from public repositories and the remaining from Torchvision (Paszke al. 2019 For the choice of the intermediate layer, we opt the layer 3-1 for ResNet50, layer 9 for VGG 19, and layer 6a for Inception V3, where the former two have been shown result in good performance by Li al. (2020b). Due to the ineffectiveness of transfer-based targeted attacks (Liu et al. 2017), we mainly consider untargeted attacks with e values of 0.05 (Rs 13 13/255) and 0.03 ( 8/255) under the lo norm constraint.
To measure the attack success rate, we randomly sample 5000 images from the ILSVRC2012 vali- dation set with all images being classified correctly by the nine models. That means, the original test accuracy before the attack is 100% for every model. The default number iterations of I-FGSM is 10 and the attack step size (learning rate) is set to max( 255 no, ittratioos ). To mount a complete attack, we first run I-FGSM for 10 iterations on the source model, and then pass the example as the reference attack to Aug-ILA for fine-tuning. We use the same model be the source model of I-FGSM and Aug-ILA. We run Aug-ILA and other fine-tuning methods for 50 iterations in all the experiments.
4.1 EFFECTIVENESS OF COMMON IMAGE AUGMENTATION OPERATIONS
We select four image transformation operations that commonly used, including translation, crop- ping, rotation and color jittering. To test our claim regarding the pixel misalignment causing poor transferability in ILA, we also implement unaligned version of translation and cropping, such that the transformations with different randomized values are applied to each of the references x, x' and x". We then report the transfer success rates Inception V3 and VGG19 over varying perturbation budget e. The comparison of different transformations is shown in Appendix B
From the result, translation and cropping yield more transferable attacks than the other transforma- tions, especially when the perturbation budget is small. Under a critical constraint when 0.01 (=s 3/255), augmentations without proper alignment fail to transfer at all, with the attack success rate very close to 0. Between translation and cropping, cropping consistently gives better perfor- mance. further evaluation of attack transferability after the incorporation of reverse adversarial update (denoted as 'adversarial") is reported in Appendix C demonstrating that further incorpora- tion of adversarial reverse update leads to higher transferability. Among different combinations, find that 'adversarial cropping' stably gives outstanding results, while further complement of translation degrades the transferability. Hence, we opt for random cropping together with reverse adversarial update as the default augmentation of Aug-ILA.

---

Under review as a conference paper at ICLR 2022
text in size 8
Method
ResNet50
Inc. v3
WRN
VGG19
PNASNet
I-GSSM
100.0
25.80%
72.12%
48.34%
29.62%
I-FGSM + ILA
100.0% %
51.50%
93.20%
86.60%
65.211%
I-FGSM + ILA+
99.98%
63.68%
96.36%
90.57%
70.34%
0.05
I-FGSM + Aug-ILA (Ours)
99.78%
90.02% %
99.04% %
98,76%
93.18 %
VMI-CT-FGSM
99.96%
69.66%
91.04%
81.78%
73.38%
I-FGSM +
+IAA
100.0%
72.46%
98.88%
96.82%
81.82%
99.96%
14.88%
41.28%
26.36%
17.40%
I-FGSM ILA
99.98%
34.56%
79.88%
66.72%
43.32%
I-FGSM ILA++
99.98%
41.44%
87.14%
75.24%
49.18%
0.03
I-FGSM Aug-ILA (Ours)
99.42%
66.02%
93.92 %
92.90%
75.76 %
VMI-CTFGGSM
98.28%
47.42%
69.94%%
56.46%
48.08%
I-FGSM LinBP SGM ILA
100.0%
42.08%
90.82%
84.08%
53.90%
Method
DenseNet
ResNeXt
MobileNet
SENet
Average
I-FGSM
57.80%
70.90%
54.56%
71.92%
59.01%
I-FGSM + ILA
85.50%
91.00%
84.00%
91.90%
83.21%
I-FGSM + ILA++
92.32%
92.30%
89.34%
96.28%
87.90%
0.05
I-FGSM + Aug-ILA (Ours)
98.40 %
96.90%
98.12%
98.72%
96,99
FMI-CTFFSM
88.50%
82.22%
82.82%
92.64%
84.67%
I-FGSM LinBP SGM ILA
97.64%
95.96%
95.60%
98.96%
13%
I-FGSM
3142%
41.82%
31.88%
44.00%
38.78%
I-FGSM ILA
69.16%
78.48%
67.60%
80.16%
68.87%
I-FGSM ILA++
79.08%
78.46
75.28%
87.6 37.64%
74.83%
I-FGSM Aug-ILA (Ours)
91.04% %
87.76% %
90.38 %
93.40%
87.84 %6%
VMI-CT-FGIM
66.06%
58.02% 80.58%
60.22%
72.84%
64, 15%
I-FGSM + LinBP + SGM ILA
85.24%
80.56%
92.64%
78.88%
The source model used to generate the attack.
Table I: Attack success rates of ImageNet adversarial examples on different models, generated from ResNet50 the untargeted setting.
4.2 BLACK-BOX TRANSFERABILITY AUG-ILA
Our previous experiments only evaluate the transformation function T, without interpolation on the reference attack. In this section, we enable all augmentations discussed previously. forming the complete Aug-IL-s ILA. We first demonstrate the superiority of Aug-ILA over ILA (Huang et al., 2019) and ILA++ (Li et al.) 2020b). We use ResNet50 as the source model and choose layer 3- the intermediate layer the comparison of attacks generated from VGG19 and Inception V3 can be found in Appendix E Moreover, additional experiments with single-step attack (FGSM) and other datasets (CIFAR-IO and C10AR-100) are shown in Appendix and Appendix G respectively.
all three fine-tuned attacks. The result shown in Table [ while
In Table we can see that Aug-ILA outperforms both ILA and ILA++ for all the models except the source model (ResNet50). The apparent improvement in attack transferability reflects the effec- tiveness of augmentation with respect to both image transformation and adversarial transformation. Another observation worth mentioning is that the attack difficulty increases according to how much the model architectures differ. For example, attacks generated from ResNet (with skip connections) tend transfer better models also with skip connections, such as WRN, DenseNet. ResNeXt, comparing to others such as Inception V3 and PNASNet. However, the improvement in transfer- ability brought by Aug-ILA is more significant under such model dissimilarity, reflected in the large gap of attack success rates between Aug- ILA and the baselines for Inception V3 and PNAS- Net. We believe such phenomenon can be attributed to the extensive augmentations that enable Aug-ILA to exploit gradients with better resemblance to different architectures.
In the next step, we show that the supremacy Aug-ILAA not limited to methods based on fine- tuning, but also other state-of-the-art transfer-based attacks. In (Wang & He, 2021), combination of the Composite Transformation Method (CTM) and variance tuning, dubbed VMI-CT-FGSM (or "MI-SI-TI-DI-FGSSM more detailedly), exhibits remarkably high transferability among the state- of-the-art methods. On the other hand, Guo et al. (220)) found that LinBP works favorably together with SGM (Wu et al. 22200) and ILA, with their incorporation further advancing transferability to new level. Using the same setting as before, we also include the attack success rates the two com- bined methods into Table with the hyper-parameters and other details specified in Appendix D

---

Under review as a conference paper at ICLR 2022
layers
intermeiati
intermediate
Junegg 18 Inampsst/
learge (dag MA
Averge 21.44)
62-
50
42-
R00 Suncess
Attack
15
40
54
Meds
(a) ResNet50
VGG19
(c) Inception V3
Figure 2: Attack transferability over different choices of the intermediate layer for ResNet50, VGG19 and Inception V3. dotted curve shows the performance of ILA without augmentation. '3-1' in ResNet50 refers to the first residual block of the third meta-block. The naming of layers of Inception V3 follows the Py PyTorch convention.
Under choices of model architecture, VMI-CT-FGSM does not work well transfer between different model architectures, despite remarkable performance in attacking defended models. Among the baselines, I-FGSM + LinBP SGM ILA achieves the best transferability between undefended models, while it is still slightly outperformed by Aug-ILA.
To further evaluate the performance of Aug-IL.A, we conduct experiments defended models. The details are reported in Appendix H
4.3 EFFECT OF DIFFERENT
In the previous sections, translation and random cropping are found to the two most effective image augmentations be applied to the ILA references. We hope to evaluate the best values for the shifting proportion and cropping size. We test different values of the shifting factor between O to 0.5 inclusively for translation. Similarly, also with the size factor of random cropping between 0.5 and 1.0 inclusively. The experiments for both augmentations are reported in Appendix[] The result indicates that proper syper-parameterd for the transformations contribute greatly attack transferability, with the best value of 0.05 for translation and 0.95 for random cropping.
Next, study the effect of a used for attack interpolation, with fixed values between 0 and in addition to the adaptive choice computed from the norm of the feature map discrepancy. The result shown in Appendix ! Although a non-trivial value of suffices to improve the performance, the adaptive selection using Equation (8) results in optimal or sub-optimal attack transferability in most cases.
Another important factor is the intermediate layer where IIA is performed. We apply IIA and Aug- ILA at each layer of the source models, and plot the attack success rates in Figure 2 The overall fluctuation pattern is similar, but Aug-ILA results in peaks that are more protruded, by virtue of the various augmentations. However, we also observe that for shallow layers, Aug- Aug-ILA turns out have limited performance, sometimes even inferior its baseline. While Huang et al. (2019) interpreted the tendency above in terms of the linearity of the decision boundary, we offer our explanation from the perspective of intermediate feature perturbation.
4.4 ROLES OF INTERMEDIATE LAYER PERTURBATION AND AUGMENTATION
Transfer-based based attacks can be viewed as using one model's gradient to estimate other models' gradi- ents. However, it was shown that the gradient directions of different models are almost orthogonal to each other (Liu et al. 2017). Locally, each layer of the target model can cause weakening effect to the attack, assembling to lead to the gradient orthogonality. The deeper the attack goes, the weaker its influence becomes. For an attack the intermediate feature, since it transits through fewer lay- ers, both forward and backward, is weakened to lesser degree. One problem is, if the attack only perturbs the intermediate feature without considering the remaining layers, it is also less capable
8

---

Under review as a conference paper at ICLR 2022
of altering the model decision, getting demoted to essentially weaker attack. Therefore, in the choice of the ILA layer, one that is either too shallow or too deep decays the attack transferability.
With the same insight, three factors can be identified for black-box transferability:
Perturbation strength
2. Similarity between the predicted gradient and the actual gradient
3. Difference in architecture between the source model and the target model.
The first two factors increase transferability while the last one hinders the attack transfer. Factor is determined by the perturbation budget, but it can be improved implicitly by ILA, such that the intermediate perturbation is also strengthened. However, factor does not dominate the attack transferability, since the victim model does not necessarily have any resembling architecture to the source model. Such difference, which factor 3, induces the weakening effect on the attack. We will show that Aug-ILA accomplishes good balance between factors and 2, leading to less severe weakening effect from factor 3.
Since ILA maximizes the projection of feature discrepancies, the norm of the feature discrepancy should also be larger (factor 1). We visualize the L.2 norm of Fi(x") - Fi(x) at different layers in estimate factor 2 by computing
ResNet50 and compare with different attacks in Figure 3 Also, the cosine distance between the model gradient suggested by the attack (x" x) and the gradient direction of the clean image (f) x. y)), with the result shown in Table
One interesting finding is that the L2 norm of ILA is actually stronger than that of Aug-ILA in shallow layers, which agrees with the poor transferability of Aug-ILA when applied at the first few layers in Figure 2 However, after the selected layer in ILA, the perturbation norm of Aug- ILA surpasses ILA and the situation continues until the output of the model. In the meantime, de- spite the strengthened feature perturbation, ILA causes the gradient direction to be more orthogonal (worse for transferability) the actual gradient direction of the model, comparing I-FGSM. Aug- ILA, nevertheless, slightly improves the gradient estimation without significant loss in perturbation strength. With stronger perturbation strength together with a better predicted model gradient, Aug- ILA facilitates its comeback for the perturbation in deeper layers, and therefore its resulting attack more transferable than the baselines.
IPGOM
Cosine distance
Source
Target
I-FGSM
ILA
Aug-ILA
12
10.
ResNet50
VGG19
0.0110
0.0067
0.0089
Inception V3
0,0016
0.0010
0.0016
ResNet50
0.0024
0.0013
0.0017
Inception V3
VGG19
0,0042
0.0029
0.0026
ResNet50
0.0049
0.0031
0.0035
VGG19
Inception V3
0.0016
0.0013
0.0016
Monel layer
Table 2: Cosine distance between x"
and (.) x, v). A
Figure 3: L.2 norm of Fi(x") (x) at
larger cosine distance refers closer prediction of the gradient direction.
different layers of ResNet50.
5 CONCLUSION
In this paper, present Aug-ILA, which applies extra augmentation to the input references in the ILA framework. We evaluate different image transformations to be applied, and select random cropping to be our base augmentation. The transformation has to be applied consistently to all the input images in order to align the values of the intermediate feature maps. Moreover, we apply two more augmentations exploiting the adversarial perturbation, namely, reverse adversarial update and attack interpolation across iterations. By incorporating all the augmentations introduced, Aug- ILA not only outperforms ILA and its variants, but also the combination state-of-the-ar methods for transfer-based attacks. We have conducted extensive studies to study the effects of different Finally, we provide explanations on the effect of augmentation in terms of the weakening effect to the perturbation strength. As we only perform simple image transformation op- erations to augment the ILA references, one potential improvement is to consider more sophisticated augmentations such as automatic data augmentation, which will be marked for future studies.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Arjun Nitin Bhagoji, Warren He, Bo Li, and Dawn Song. Practical black-box attacks on deep neural networks using efficient query mechanisms. ECCV, 2018.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In ICLR, 2018.
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. ZOO: zeroth order opti- mization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, 2017.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. CVPR, 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by ionsslationinvvriiann attacks. CVPR, 2019.
lan J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. CoRR. abs/ 4b2.6542, 2015.
Yiwen Guo, Ziang Yan, and Changshui Zhang. Subspace attack: Exploiting promising subspaces for query-efficient black-box attacks. In NeurIPS, 2019.
Yiwen Guo, Qizhang Li, and Hao Chen. Backpropagating linearly improves transferability of ad- versarial examples. In NeurIPS, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. CVPR, 2016.
Jie Hu, Li Shen, and Gang Sun. Squeeze-and-exciaation networks. In CVPR, 2018
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing adversarial example transfcrability with an intermediate level attack. In ICCV, 2019.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. ICML,, 2018.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2012.
Alexey Kurakin, lan Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In ICLR, 2017a.
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Adversarial machine learning scale. In ICLR, 2017b.
Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and Bo Li. QEBA: query-efficient boundary- based blackbox attack. In CVPR, 2020a.
Qizhang Li, Yiwen Guo, and Hao Chen. Yet another intermediate-level attack. In ECCV, 2020b.
Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang. Xiaolin Hu, and Jun Zhu. Defense against adversarial attacks using high-level representation guided denoiser. In CVPR, 2018.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hoperoft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In ICLR, 2020.
Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan L. Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. In ECCV, 2018.
10

---

Under review as a conference paper at ICLR 2022
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks. In ICLR, 2017.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In ICLR, 2018.
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. CoRR, abs/1610.08401, 2016.
Alexander Mordvintsev, Christopher Olah, and Mike Tyka. Deepdream a code example for visu- alizing neural networks. Technical report, Google Research, 2015.
Nicolas Papernot, Mcdaniel, and I. Goodfellow. Transferability in machine learning: from phe- nomena to black-box attacks using adversarial samples. ArXiv, abs/1605.072777 2016.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS. 2019.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei- Fei. Imagenet large scale visual recognition challenge. I/CV, 115(3):211 1-222, 2015.
Mark Sandler, Andrew G. Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In CVPR, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recogmition. ICLR, 2015.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. CVPR, 2016.
Florian Tramer, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. In ICLR, 2018.
Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. In CVPR, 2021.
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In ICLR, 2020.
Weibin Wu, Yuxin Su, Michael R. Lyu, and Irwin King. Improving the transferability of adversarial samples with adversarial transformations. In CVPR, 2021.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In ICIR, 2018.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity. In CVPR, 2019.
Saining Xie, Ross B. Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In CVPR, 2017.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In BMVC, 2016.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empiri- cal risk minimization. In ICLR, 2018.
Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang, Xiangqi Huang, Xiang Gan, and Yong Yang. Transferable adversarial perturbations. In ECCV, 2018.
11

---

Under review as a conference paper at ICLR 2022
A EFFECT OF REVERSE ADVERSARIAL UPDATE ON THE MODEL PERFORMANCE
We would like to verify the intuition using reverse adversarial update, specifically, whether in practice such operation can decrease the loss and increase the model confidence. Therefore, randomly sample 5000 images, including those that are misclassified by the model. Then we apply I-FGSMI0 the form of reverse adversarial update to the images, and pass them back the model for classification. We record the loss and apply softmax to the logits to obtain the confidence, and report the average result of all 5000 images. The result is summarized
Table 3
Furthermore, we also inspect the class activation map (CAM) of the source model. Figure visual- izes some of the results. For some images (the first two), updating reversely helps the model focus on more features. For the remaining (the latter two), adversarial reverse update does not exhibit significant change of intermediate layers' contribution to the ground-truth class.
Model
Image
Loss
Confidence
Accuracy
Clean
0.9755
0.7913
75.08%
ResNet50
Reversely updated
0.3070
0.8847
92.20%
Clean
1.1165
0.7339
76.44%
Inception V3
Reversely updated
0.3525
0.8296
94.60%
Clean
1.1431
0.7419
70.54%
VGG19
Reversely updated
0.5756
0.8998
89.98%
Table 3: Changes in loss and confidence after applying reverse adversarial update on the images for ResNet50.
Image
image
Image
CAM (Clean image)
CAM image)
CAM (Clean image)
CAM (Clean image)
CAM (Reversely Updated)
CAM (Reversely Updated)
CAM (Reversely Updated)
CAM (Reversely Updated)
Figure 4: CAM visualization of the images after reverse adversarial update.
12

---

Under review as a conference paper at ICLR 2022
To test the effect of reversely updated examples targeted models, also feed them into the remaining undefended models and observe the change in accuracy. The result reported in T which shows that reverse adversarial update not only benefits the source model, but also many of the other models with similar architecture. It shows that reverse adversarial update can be beneficial many of the target models under proper choices of pepre-prraeeters such the source model.
Target model
Source model
ResNet50
Inception v3
WRN
VGG19
PNASNet
75.08%
76.44%
77.58%
70.54%
71.34%
ResNet50
92.20%
79.48%
82.22%
74.06%
72.84%
Inception V3
78.58%
94.60%
80.70%
73.62%
72.88%
VGG19
78.46%
79.56%
80.06%
89.98%
71.76%
Target model MobileNet
Source model
DenseNet
ResNeXt
SENet
Average
75.86%
78.30%
70.48%
75.36%
74.55%
ResNer50
80.70%
82.70%
73.70%
80.70%
79.84%
Inception V3
80.54%
81.54%
73.04%
78.809
79.37%
VGG19
80.58%
81.02%
74.38%
79.00%
79.42%
Table 4: Top-1 accuracy of models under the example with reverse adversarial update from different source models. The source model "." indicates clean example without any perturbation.
B EFFECT OF DIFFERENT IMAGE TRANSFORMATION OPERATIONS
For translation, we shift the image by a random factor of 0.1, 1, 0.1] of the image size, both vertically and horizontally. Cropping operations consist of random cropping of 95% of the size, followed by an upsampling to the original shape. The rotation degree ranges between [-90", -9%) Color jittering tweaks all the brightness, contrast, saturation by 0.2 and hue by 0.1. Random cropping is found be the one resulting in the highest attack transferability, followed by translation and rotation. Pixel-wise transformations such as color jittering only exhibit limited improvement to transferability comparing to spatial transformations.
Resent50 + Inception V3
Resent50 VGG19
100
100
Attack Success faae (%a)
60
Attack Success
40
Erosping
Cropping
Coter Jiter
Jiter Cropping (Unaligned)
Cropping (Unaligned)
Fansiatio (Unaligne))
Transtation (unaligndd)
0.07
(a) Transfered from ResNet50 Inception V3
(b) Transferred from ResNet50 to VGG19
Figure 5: Attack success rates of applying translation, cropping, rotation and color jittering augmentation
ILA.
13

---

Under review as a conference paper at ICLR 2022
C EFFECT OF REVERSE ADVERSARIAL UPDATE ON ATTACK TRANSFERABILITY
Resent50 Inception V3
Resent50 + WRN
80
100
Attack Succeds Rate (%)
Actack (%)
60
Cropping Traneaation
Cropping
50
Trensaation
Avvertataal Cropping
Advorsarall Cropping
Adversaiial Transiation Adversaria Transaaton Cropping
Adversarial a Transtation
Adversrtts + Transtation Cropling
0.03
0.05
0.03
0.05
(a) Transferred from ResNet50 to Inception V3
(b) Transferred from ResNet50 WRN
Figure 6: Attack success rates of different augmentation, in conjunction with adversarial reverse update (de- noted as adversarial). The performance of simple augmentation without reverse adversarial update is also included baseline.
D HYPER-PARAMETERS USED IN THE BASELINES
We reproduce the algorithms variance tuning. CTM (DIM, TIM, SIM), MI-FGSM and NI-FGSM according to the repository released by Wang & He (02TT. and test the attacks under our experi- mental settings. The list of hyper-parameters used in the experiments is shown in Table 5 which the default values in their corresponding papers. For the implementation of SGM and LinBP, directly use the official implementation released by Guo et al. (020))
Method
ppper-parreeter
Value
MI-FGSM
1.0
NI-FGSM
1.0
Probability
0.5
DIM
Upscale ratio
1.1
TIM
Kernel size
7 X 7
SIM
Scale copies
5 (i
0, 1, 2, 4)
N
20
Variance Tuning
8
1.5/255
SGM
0.5
1- I-FGSM iteration
300
LinBP
Layer
'3-1' for ResNet50
Table 5: Hyperparameters used in the basclines.
14

---

Under review as a conference paper at ICLR 2022
E
ATTACKS USING OTHER SOURCE MODELS
Method
ResNet50
Inc. v3
WRN
VGG19"
PNASNet
I-FGSM ILA
77.32%
45.42%
74.78%
99.40%
70.30%
0.05
I-FGSM ILA++
79.56%
47.08%
76.04%
99.30%
70.76%
I-FGSM Aug-ILA (Ours)
87.36
66.40%
87.00%
99.02%
82.34%
[-FGSM + ILA
53.78%
25.94%
49.54%
99.34%
46.30%
0.03
I-FGSM ILA++
57.88%
27.94%
52.94%
99.30%
49.48%
I-FGSM Aug- ILA (Ours)
66.22%
40.82%
64.90 %
97.82%
60.24% %
Method
DenseNet
ResNeXt
MobileNet
SENet
Average
I-FGSM ILA
67.38%
61.68%
85.34%
72.90%
72.72%
0.05
I-FGSM + ILA+
38.66%
63.56%
85.60%
74.46%
73.89%
I-FGSM Aug- Aug-ILA (Ours)
81.72%
80.30%
93.06 %
85.84%
84.78 %
IFGSM ILA
40.12%
36.26%
66.52%
49.42%
51.91
0.03
I-FGSM ILA++
44.88%
39.70%
68. 68.70%
53.62%
54.94%
I-FGSM Aug-ILA ILA (Ours)
57.38 %
54.02 %
77.58 %
63.32%
64.70 %
The model used generate the attack.
Table 6: Attack success rates of ImageNet adversarial examples on different models, generated from VGG19 the untargeted setting.
Method
ResNet50
Inc.
WRN
VGG19
PNASNet
I-FGMM + ILA
57.26%
97.84%
51.98%
61.70%
54.16%
0.05
I-FGSM ILA+
57.80%
97.60%
51.72%
60.54%
52.86%
I-FSM Aug-IIA (Ours)
85.16% %
98.14%
83.24%
89.16%
81.38%
I-FGMM ILA
34.18%
96.98%
29.62%
37.56%
34.02%
0.03
I-FGSM + ILA+
36.74%
97.18% %
31.90%
38.90%
35.14%%
I-FGSM Aug-ILA (Ours)
57.94 %
91.18%
53.54% %
65.22%
56.38%
Method
DenseNet
ResNeXt
MobileNet
SENet
Average
IFFGM ILA
46.14%
42.24%
64.38%
48,866 48.86%
58.28%
0.05
I-FGSM + ILA+
47.42%
44.08%
62.62%
49.72%
58.26%
I-FGSM Aug-ILA (Ours)
78.88 %
74.54%
89.16% %
80.42%
84.45 %
FFGSM ILA
27.00%
23.72%
43.36%
29.64%
39.56%
0.03
I-FSM ILA++
29.06%
25.68%
43.82%
31.58%
41.1
I-FGSM Aug-II.A (Ours)
47,80 %
43.06%
65.96 %
51.58%
59.18 %
The source model used to generate the attack.
Table 7: Attack success rates of ImageNet adversarial examples on different models, generated from Inception V3 in the untargeted setting.
15

---

Under review as a conference paper at ICLR 2022
F EXPERIMENTS WITH SINGLE-STEP ATTACK
Method
ResNet50"
Inc. v3
WRN
VGGT9
PNASNet
FOSM
80.86%
28.78%
37.08%
46.24%
36.04%
FGSM ILA
93.36%
28.56%
51.94%
58.22%
38.48%
0.05
FGSM ILA+
97.88%
43.14%
74.88%
65.92%
44.04%
FGSM + Aug-ILA (Ours)
96.98%
57.48% %
87.20 %
91.76% %
67.40 %
FGSM
84.68%
20.44%
29.00%
30,96%
23.52%
FGSM ILA
88.60%
19.48%
38.02%
39.50%
24.04%
0.03
FGSM ILA+
97.02 %
29.82%
60.02%
50.64%
30.60%
FGSM Aug-ILA (Ours)
89.72%
32.46% %
65.44 %
72.60 %
40.08 %
Method
DenseNet
ResNeXi
MobileNet
SENet
Average
FGSM
32.48%
30.72%
45.32%
41.42%
42. 10%
FGSM ILA
39.10%
40.12%
52.42%
52.58%
50.53%
0.05
FGSM + ILA+
65.80%
65.48%
66.12%
74.12%
66.38%
FGSM Aug-ILA (Ours)
79.48 %
74.20% %
84.18 %
83.70%
80.26 %
FGSM
25.10%
22.38%
29. 86%
31.70%
33.07%
FGSM ILA
28.50%
29.22%
35.76%
39.56%
38.08%
0.03
FGSM + ILA+
50.54%
50.62 %
52.66%
61.36%
53.70%
FGSM Aug-ILA (Ours)
55.68 %
48.22%
58.74 %
62.56 %
58,39 %
The model used generate the attack.
Table 8: Attack success rates of ImageNet adversarial examples with single-step FGSM the reference attack, generated from ResNet50 in the untargeted setting.
G EXPERIMENTS WITH MORE DATASETS
To show the generalization capability of Aug-ILA, ILA, conduct experiments with two datasets, namely CIFAR-10 10 and CIFAR-106 (Krizhessky) 2012). Partly following Li et al. (2020b), we con- sider four models: VGG19 with batch normalization (Simonyan & Zisserman [2013 Wide ResNet- 28- 10 (WRN) ZZaoruyko & Komodakis 2016 ResNeXt-29 (ResNeXt) (Xie et al. 2017) and DenseNet-190 (DenseNet) (Huang al. (017). The model parameters are obtained from public 5
repository For both datasets, we randomly sample 3000 images from the test set that are classified correctly by all four models and we pick VGG19 to be the source model. The experimental results using CIFAR-10 and CIFAR-100 are shown in Table and Table T0leeTOreeeeeee respectively.
For both CIFAR-10 and CIFAR-I00, 100, the attack success rates of the baselines are much higher than that of ImageNet. Although Aug-ILA mostly attains the highest attack success rate among the baselines, we observe that degree of improvement not as high as that in Table T may be because of the tiny size and low resolution x2x32 for both CIFAR-10 10 and CIFAR 100) of the images, leading to reduction in the effect data augmentation such as random cropping.
Method I-FSMM
VGG19*
WRN
ResNeXt
DenseNet
Average
99.54%
67.91%
67.91%
64.00%
74.84%
I-FGSM + ILA
99.54%
97.85%
98.19%
97.08%
98.16%
0.05
I-FGSM + ILA+
99.57% %
98.00%
98.17%
97.40%
98.29%
I-FGSM Aug-ILA
98.77%
98.22%
98.46 %
98.06 %
98.38%
LFGSM
99.32%
55.27%
55.43%
51.64%
51.64%
I-FSM ILA
99.45% %
88.26%
89.61 %
87.15%
91.12%
0.03
I-FGSM + ILA+
99.13%
89.67%
90.13%
87.40%
91.58%
I-FGSM Aug-ILA
98.31%
93.05%
93.42%
91.45 %
94.06%
The source model used to generate the attack.
Table 9: Attack success rates of CIFAR-10 adversarial examples different models, generated from VGG19 the untargeted setting.
ficaiion
16

---

Under review as a conference paper at ICLR 2022
Method
VGGT9*
WRN
ResNeXt
DenseNet
Average
I-FGSM
99.45% %
45%
39.45%
42.41%
57.37%
I-FGSM + ILA
99.03%
92.73%
88.97%
88.63%
92.34%
0.05
I-FGSM ILA++
99.10%
93.13%
89.53%
89.70%
92.87%
I-FGSM Aug-II.A
96.08%
92.65%
90.56%
90.69%
92.50%
I-FGSM
98.84%
38.91%
31.94%
33.77%
50.87%
I-FGSM ILA
98.91% %
80.77%
72.71%
73.93%
81.58%
0.03
I-FGSM ILA+
98.53%
81.23%
74.77%
74.60%
82.28%
F-GSM Aug-ILA
95.22%
84.37% %
78.33%
78.56%
84.12%
The source model used to generate the attack.
Table Attack success rates of CIFAR-I00 adversarial examples on different models, generated VGGI9 in the untargeted setting.
H EFFECT OF AUG-ILA ON DEFENDED MODELS
In this section, we evaluate the effect of Aug-ILA on defended models. We consider three robust models using ensemble adversarial training (Tramer et al. 2018), including ensemble of 3 ad- versarially trained Inception V3 (Inc-v3enes), an ensemble of adversarially trained Inception (Inc-v3ened), and ensemble of adversarially trained Inception-ResNettv22 ResNet-v2 (IncRes-v2ens). On top of that, we also test the attacks against three defenses: HGD (Liao et al.) 2018), R&P (Xie et al. 2018) and NIPS-r3 which the top 3 defenses in the NIPS 2017 adversarial competition. For all three defenses, we adopt the default models and hyper-parameters used in their corresponding repositories.
We ported the implementation of Aug-ILA ILA into the same experimental setup used in previous works (Dong al. 2018 2019 Wang & He 2021), which includes 1000 sampled test images and the pretrained parameters of the models. Then we compare the baselines, including I-FGSM, MI-FGSM, MI-CT-FGSM, NI-CT-FGSM, and VNI-CT-FGSM, with ILA and Aug-ILA inserted. We pick Inception V3 as the source model, and all attacks are performed with perturbation size <= 16/255 0.0627). For ILA and Aug,II.A, we finetune the attack for 50 iterations by choosing the layer 6a in Inc-V3 an adversarially trained Inception V3 model. The attack success rate against each the defenses reported in Table From preliminary results, ILA cannot improve the attack success rate on the defended models, so it is removed in the experiments with stronger attacks. Nevertheless, Aug-IL ILA able to improve the attack success rate of most of the attacks, except VNI-CT-FGSM.
Attack I-FSMM
Inc-v3ensa
Inc-v3ensd
IncRes-v2ena
HGD
R&P
NIPS-r3
12.1%
10.9%
5,80%
2.70%
4.00%
8.30%
IFGSM ILA
10.1%
10.6%
5,00%
0.10%
2.50%
7.90%
I-FGSM + Aug-ILA
43.0%
33.4%
24.0%
35.1%
23.6%
14.3%
MI-FGSM
14.1%
13.0%
6.60%
4.60%
5.00%
8.30%
MI-FGSM ILA
14.1%
12.5%
6.40%
0.70%
3.60%
5.40%
MI-FGSM Aug-ILA
44.2%
36.3%
24.6%
33.6%
25.3%
28.5%
MI-CT-FGSM
65.5%
62.1%
45.5%
56.6%
44.5%
52.5%
MI-CT-FGSM Aug-ILA
69.0%
65.8%
49.8%
62.1%
51.6%
65.0%
NI-CT-FGSM
58.8%
54.4%
40.0%
49.2%
38.0%
46.1%
NI-CT-FGSM + Aug-ILA
66.8%
62.4%
44.5%
55.8%
46.5%
52.8%
VNI-CT-FGSM
79.1%
77.4%
65.3%
72.7%
63.5%
70.8%
VNI-CT-FGSS Aug-ILA
75.6%
73.2%
58.3%
68.4%
81.6%
64.0%
Table 11: Attack success rates of ImageNet adversarial examples, generated from Inception in the untar- geted setting.
17

---

Under review as a conference paper at ICLR 2022
I
HYPER-PARAMETERS FOR IMAGE TRANSFORMATION OPERATIONS
Random Cropping
Translation
75-
Attck
65
VG019
Redivit
VGG19
RecWelt
0.9
65
Factor Random Cropping
Factor Transiation
(a) Random cropping
(b) Translation
Figure 7: Attack success rates of different hyper-paramcters for random cropping and translation, using ResNet50 as source model with (00(==8/255))
J EFFECT OF a ON ATTACK INTERPOLATION
73
Merack Success Rate 063
(90
72
82.5
80.0
71
77.5
Attack
Adaptive a on VGG15
Acopdive a Mubilenet
70
Food a yau13
Fixed on Mobiennt
Adaptive a on WIN
Adapdive on SENsE
Ade on Average Flaed Average
Floed W9NN
Fixed on SEhe:
70.0
69
(a) VGGI9, WRN, MobileNet and SENet
(b) Average of the 8 models (excluding the source model)
Figure 8: Attack success rates over different values of a, using ResNet50 the source model with 0.03.
K INTERMEDIATE FEATURE DISCREPANCIES ON OTHER MODELS
We extend the experiments examining the L.2 norm of Fx(x") (x) in Figure 3 Using the attacks generated by the same source model (ResNet50), we also test the magnitude of the intermediate feature discrepancies for VGG19 and Inception V3, as shown in Figure 9 The magnitude of the norm depends on the size of the feature maps, so it cannot be compared directly across layers but relatively among different attacks. Agreeing with our explanation in Section 4.4 the L2 norm of Aug-ILa ILA starts with comparable strength to that ILA in shallow layers, but outpaces ILA when the layers get deeper.
18

---

Under review as a conference paper at ICLR 2022
I-GSSM
LFGSM
I-FSMM IIA Aug-LA
25
I-FGSM ILA
600
I-FGSMM Aug-ILA
20
9 Norm
400
L2
10
5-
Model layer
Model layer
(a) VGGI9 (without batch normalization)
(b) Inception
Figure L2 norm of Fx(" Fi(x) at the intermediate layers of VGG19 and Inception V3.
L ABLATION STUDY ON THE THREE AUGMENTATIONS IN AUG-ILA
Aug-ILA can be regarded as generalization of the original ILA, with the extension three major parameters. With the cropping size set to 1.0, conduct ablation study on the effect of the three augmentations
O, and reverse adversarial update disabled,
Aug-ILA degenerates to ILA. by removing each of them from Aug-ILA and reporting the performance of the reduced methods in Table 12 Alternatively, we also observe the change in attack success rates with each of the three augmentation components added into the original ILA, and report the result in Table 13
is found that that cropping contributes the most to attack transferability. followed by the remaining two methods. However, from Table [3 much unless it is applied together with other augmentations.
see that attack interpolation alone does not contribute
E Method
ResNet50
Inc. v3
WRN
VGGT9
PNASNet
I-FGSM Aug-ILA
99.78%
90.02%
99.04% %
98.76%
93.18%
w/o Cropping
99.78%
66.78%
96.92%
95.12%
75.84%
0.05
Attack interpolation
99.96%
77.10%
98.00%
94.60%
82.22%
w/o Reverse adversarial update
99.74%
89.20%
98.54%
98.80%
91.40%
I-FGSM Aug-ILA
99.42%
66.02%
93.92%
92.90%
75.76%
w/o Cropping
99.72%
39.74%
86.64%
79.80%
49.76%
0.03
Attack interpolation
99.96%
54.26%
91.82%
83.46%
62.22%
w/o Reverse adversarial update
98.76%
62.56%
91.46%
92.28%
71.38%
Method
DenseNet
ResNeXt
MobileNet
SENet
Average
I-FGSM Aug-ILA
98.40%
96.90%
98.12%
98.72%
96.99
w/o Cropping
93.96%
92.68%
93.26%
96.98%
90.15%
0.05
Attack interpolation
96.52%
95.74%
94.92%
97.68%
92.97%
w/o Reverse adversarial update
98.22%
96.80%
97.80%
98.44%
96.55%
I-FSM ILA
91.04% %
87.76% %
90.38 %
93.40%
87.84%
w/o Cropping
77.06%
76.08%
75.66%
85.96%
74.49%
0.03
Attack interpolation
87.26%
85.92%
83.52%
92.06%
82.28%
w/o Reverse adversarial update
90.18%
86.32%
88.86%
90.94%
85.86%
The source model used to generate the attack.
Table 12: Comparison of the attack success rates of ImageNet adversarial examples on different models, with each augmentation removed from Aug- LLA.
19

---

Under review as a conference paper at ICLR 2022
Method
ResNet50"
Inc. v3
WRN
VGGT9
PNASNer
I-FGSM + ILA
100.0%
51.50%
93.20%
86.60%
65.20%
w/ Attack interpolation
99. 99.84%
61.88%
96.82%
95.42%
72.12%
0.05
w/ Reverse adversarial update
99. 99.94%
61.36%
96.72%
89.38%
65.94%
w/ Cropping
99.96%
74.30%
97.42%%
94.62%
78.78%
w/ All (Aug-ILA)
99.78%
90.02%
99.04%
98.76%
93.18%
I-FGSM ILA
99.98%
34.56%
79.88%
66.72%
43.32%
w/ Attack interpolation
99.60%
34.22%
85.06%
78.04%
43.14%
0.03
w/ Reverse adversarial updatc
99.98%
37.86%
87.46%
71.76%
42.72%
w/ Cropping
99.96%
50.80%
90.14%
82.26%
56,30%
w/ All (Aug-ILA)
99.42%
66.02%
93.92%
92.90%
75.76%
Method
DenseNet
ResNeXt
MobileNet
SENet
Average
I-FGSM ILA
85.50%
91.00%
84.00%
91.90%
83.21%
Attack interpolation
73.12%
73.14%
74.04%
83.28%
88.96%
0.05
w/ Reverse adversarial update
93.36%
92.28%
92.68%
96.22%
87 87.08%
w/ Cropping
92.68%
92.20%
89.26%
96.20%
91.68%
w/ All (Aug-ILA)
98.40%
96.90%
98.12%
98.72%
96.99%
I-FGSM ILA
69.16%
78.48%
67.60%
80.16%
68.87%
w/ Attack interpolation
73.
73.14%
74.04%
83.28%
71.52%
0.03
w/ Reverse adversarial update
78.08
77.82%
74.36%
87.52%
73.06%
w/ Cropping
83.74%
82.34%
82.04%
89.18%
79.64%
w/ All (Aug-ILA)
91.04%
87.76%
90.38%
93.40%
87.84%
The source model used to generate the attack.
Table 13: Comparison of the attack success rates of ImageNet adversarial examples on different models, with each augmentation added to Aug.ILA.
M STUDY ON THE RUNNING TIME OF THE ALGORITHMS
In this section, compare the running time between ILA and Aug-ILA. We report the running time required to run attack on the 5000 sampled images from the ILSVRC2012 validation set. The setup the same the previous experiments, which we run I-FGSM for 10 iterations followed by ILA for 50 iterations. The batch size is set to 100. All the experiments are performed on machine with an Intel Xeon Silver 4214R CPU and Nvidia GeForce RTX3090 GPU. The result summarized in Table [4
Running Time (8)
Method
Total
Average (per batch)
I-FGSM
336
6.72
I-FGSM ILA
947
18.99 94
I-FGSM + Aug-ILA
1083
21.66
Table 14: Running time comparison between ILA and Aug-ILA.
Comparing with IIA, Aug-ILA has three more augmentations applied. Firstly, random cropping is performed once for every ILA iteration. Secondly, reverse adversarial update requires basic arith- metic on the images. The computation time of these two processes insignificant and hence negli- gible. However, attack interpolation requires the computation of a with the norms. Since need to obtain the intermediate feature output of the new x", an extra forward pass to the model is required per iteration. Such extra pass is the major overhead introduced by ALg. From Table 14 the 50 extra forward passes increases the running time by around 14%, which is still reasonable in terms of generation of adversarial attack.
20

---

Under review as a conference paper at ICLR 2022
N
MORE VISUALIZATIONS ON THE GENERATED EXAMPLES
Clean image
I-FGSM ILA
I-FGSM Auq-ILA (Ours)
Clean image
I-FGSM ILA
1-FGSM Aug-ILA (Ours)
Clean image
FFGSM ILA
1-FGSM Auq-ILA ILA (Ours)
image
Aug (Ours)
Figure 10: Visualization of the generated images among: clean image, ILA, and Aug-ILA (Ours), with the perturbation budget = 0.03( 8/255).
21


Reviews:
[{"review_id": "x5diiyntoan", "reviewer": "Reviewer_JiJd", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n1. This paper is well organized.\n2. The proposed AUG-ILA algorithm is simple, practical, and effective. Experiments on various datasets show that it outperforms both the original ILA algorithms and other state-of-the-art methods with a large margin.\n\n**Weakness**\n1. Lack of novelty. Although the experiment results are impressive, the proposed AIG-ILA algorithm is a very practical extension. No new theories or methods about ILA are proposed.\n2. In Section 3.1, this paper claims that the reverse adversarial update is used to boost the confidence of clean image $x$. The update formulation is $x − (x^\\prime− x) = x − \\epsilon$ where $\\epsilon$ is the adversarial\nperturbation. Although $x − \\epsilon$ will decrease the loss for the original model, it doesn’t always decrease the loss of the target model. So the reverse adversarial update operation that will boost confidence might not be well supported unless the transferability of adversarial examples is proved.\n3. This paper argues that reverse update operation will help to get more useful attack information from discrepancy feature maps. It is better to show some visualization results to support this view.", "summary_of_the_paper": "The paper proposed the augmented Intermediate Level Attack (ILA) algorithm to strengthen the transferability of adversarial examples. Also, it claimed that increasing the diversity of input references could improve the generalization of adversarial examples when attacking different models. Specifically, this paper performs augmentation operations, including common image data augmentations and transformations exploiting adversarial perturbation (e.g., reverse adversarial updates and attack interpolation on the reference attack), before\nmaximizing the projection of intermediate feature map discrepancies.", "summary_of_the_review": "Generally, this paper proposes a simple and practical extension of the ILA algorithm. The experiments on various benchmarks are strong. However, this paper lacks the support of theory and some views might not be well supported.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "YPCMte9btvk", "reviewer": "Reviewer_uNtx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The overall structure of this paper is clear, and  extensive experiment results are provided to support their ideas.\n- Based on the observation from the  previous work, diversity transformation and perturbation interpolation are used to improve the transferability. The experimental results show that the augementation is simple and effective. However, I still have some doubts: how does the adversarial perturbation generated against the transformed examples correspond to the original ones? Especially random cropping with large size causes great damage to the image, making the generated perturbation is extremely incosistent with the original image in pixel. But the adversarial example shown in Figure 1 is natural.\n- The author expect the reverse adversarial update input can highly activate the intermediate layers and provide a better attack direction guidence. However, the reduction of classification loss in the Appendix A seems obvious. \nIs it possible to show the change of feature attention map after the first update?\n- Although the three basic methods proposed are effective and intuitive, there is a problem to be solved: how to choose the index of the attack layer, which seems to have no prior guidance other than referring to existing papers.\n- Beyond experiment results, the author provide explanations on the effect of augmentation in terms of the weakening effect to the perturbation strength.", "summary_of_the_paper": "This paper presents a transfer-based attack method based on Intermediate Level Attack(ILA). Image augementation and reverse adversarial updateare applied to ILA input, to make diverse adversarial references. Moreover, the interpolation of cumlative attacks can maintain a better transfer direction.The paper is clear and well-written and experimental results and analysis support the claims.", "summary_of_the_review": "In summary, I find the experiments satisfying, although the additional experiment show can make the claims more convincing.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "QZjkSfSxk81", "reviewer": "Reviewer_2dLg", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "strengths:\n===========\n- Using more number of reference attacks seems to be a valid solution (as also done in ILA++), and using data augmentation techniques to achieve this goal is logical, providing a simple solution with significant improvements.\n\n- The empirical results consistently show significant improvements to other approaches, specially to closely related methods such as ILA and ILA++.\n\n- The study of hyperparameters supports the choices made\n\n- Ablation result demonstrates the effectiveness of each data augmentation\n\nweaknesses:\n===========\n\n## lack of comparison with published results:\n\n- The configurations used (e.g, perturbation size, dataset combinations, architecture type) in the reported experiments are different from those used in the published results, and it appears all results are reported from experiments carried out by the authors. This raises some concerns on whether the baselines were used justly, with best hyperparameters etc. For example ILA++ [Li et at., 2020] reports results on CIFAR100, ILA [Huang et al,. 2019] on CIFAR10 and ImageNet (but different models than used in this work). Why isn't the models/dataset combinations used in related work (e.g, in [Wang et al., 2021], [Wu et al., 2021]) aren't used? Also it is unclear which methods from [Wang et al., 2021.] are dubbed as VMI-SI-DI-TI-FGSM, and what are the related details regarding how they relate to VMI-SI-DI-TI-FGSM ? Also it seems that in  [Wang et al., 2021.] VNI-FGSM performs better than VNI-FGSM. What's the reason for choosing VMI-FGSM over VNI-FGSM? As for I-FGSM + LinBP + SGM + ILA, and LinBP from [Wu et al., 2020], it is unclear what method from [Wu et al., 2020] is I-FGSM + LinBP + SGM + ILA referring to.\n\n\n\n- The proposed method's empirical performance is only tested on one dataset, and its generalization capabilities onto other datasets with the tuned hyperparameters is not demosntrated. Providing results on other datasets (e.g, CIFAR100) would demonstrate generalisation on their approach. Especially, because the best parameters chosen, are found via the results conducted on the validation set of Imagenet, the same set as model is evaluated, and there os no held-out set to account for overfitting.\n\n- Only  $L_{inf}$  I-FGSM attacks are used, and for example, the effectiveness by using l2, single-step FGSM, was not demonstrated (as done in [Wu et al., 2020]. Also the choices of perturbation sizes are not justified. \n\n- All results are only reported on undefended models, and it is unclear how these attacks would transfer if an adversarially-robust model was used as target (as done in [Wu et al., 2021.].\n\n- results reported only on a single run, and there is no performance report on the effect of random seed. As in [Wu et al., 2020], it is recommended to report mean and std over several runs of the algorithm to demonstrate stability of the attack's convergence.\n\n\n## limitation of ablation studies:\n\n- In ablations reported in Appx. I, results when ignoring a single factor removed; and the two other augmentations are in place, are reported. Based on these, it appears that removing cropping has the largest effect. I would like to know what is the performance of ILA only with each of the augmentations (cropping, reverse update, interpolation). This way we can pinpoint the contribution of each individual part. At the moment, we can only observe a result of combined steps.\n\n\n## computational cost:\n\nPlease discuss the computational complexity of the proposed method in comparison with SOTA and baselines used. E.g, how much additional computation will these 3 data augmentations require compared to ILA?\n\n\nadditional comments:\n====================\n\n- in  $L_{inf}$  attacks, it is common to state perturbation size by $\\epsilon= x/255$. I recommend providing the perturbation size in this format, or at least clarify this point.\n\n- on the reverse adversarial update, it seems that this step will move the sample further away from the decision boundary.\n\n- connection to related work: it would be insightful to discuss connections to related work (ILA, ILA++). Can Aug-ILA be considered a generalisation of any of the previous methods? or a special case? or there is no major connection?\n\n- some theoretical analysis on the convergence/generalisation on Aug-ILA, could shed light on the functionality of such approaches, and is encouraged if it could provide insights. For example, ILA, ILA++, and Aug-ILA could be studied under a simplified setting.\n\n\nrefs:\n=====\n\n[Wu et al., 2021]: Improving the Transferability of Adversarial Samples with Adversarial Transformations\n\n[Wang et al., 2021]: Enhancing the Transferability of Adversarial Attacks through Variance Tuning\n\n[Wu et al., 2020]: Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets\n\n\n \n", "summary_of_the_paper": "This paper introduces improvements over Transferable Intermediate-level Attacks (ILA), by incorporating data augmentation into the attack tuning stage. The data augmentations introduced include 3 kinds: simple transformations (cropping), reverse-adversarial update, and attack interpolation. An empirical study has been conducted on Imagenet, and 3 well-known architectures have been used as source models, and the attack transferability was evaluated onto 9 models using I-FGSM with $L_{inf}$ distance and 2 different perturbation sizes.\nAblation studies have been conducted to demonstrate the effectiveness of hyperparameter choices, and different kinds of data augmentation used, demonstrating the effectiveness of the proposed approach compared to other ILA-based blackbox attacks.\n\n", "summary_of_the_review": "The proposed method is simple, and provides significant improvements. However, I have the following concerns:\n\n- More clarity on some of the baselines and evaluation used is required (see above).\n\n- There is possibility of overfitting on hyperparameters due to using a single dataset for evaluations. Providing additional results will help resolve this issue.\n\n- The generality of the proposed method is not sufficiently demonstrated (only one attack, only one dataset).\n\nPlease see full comments above, and address if possible.\n\n\nAfter rebuttal:\n=====\n\nAfter the rebuttal, results on 2 new datasets, as well as results on defended models, and single-step attacks are provided, comparing AugILA with other ILA variants, which partially addresses my concerns regarding these points. Additionally, some concerns regarding the use of baselines have been resolved by providing responses to my comments.\n\nThe additional results shows that the proposed method consistently and in the majority of the cases, improves over ILA and ILA++, and I am convinced that the proposed approach improves over ILA/ILA++.\nHowever, unfortunately no comparison is provided with other SOTA methods beyond ILA-based approaches, which limits the paper's contributions.\nMore specifically, my comments regarding evaluation on new datasets, single-step attacks, and defended models are not fully addressed due to the fact that the provided results are only comparisons to ILA-based approaches (ILA, ILA++), and baselines such as VMI-CT-FGSM and I-FGSM + LinBP + SGM + ILA, which were among the baseline presented in Table 1 have been missed. \nEspecially, because as reported in Table 1, VMI-CT-FGSM and I-FGSM + LinBP + SGM + ILA have better success rates than of the ILA baseline.\nThis leaves uncertainty on how the proposed method would perform on other datasets, single-step attacks, and defended models, compared to non-ILA approaches. In my opinion, this is the main drawback of the new results.\n\nIn summary, although I believe the paper has been improved after the rebuttal, due to the limitations mentioned above, I keep my score.\nNevertheless, I believe by strengthening the baseline comparisons, this work has the potential to become a valuable contribution for the community.\n\n\n\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "HElz5vYaGT", "reviewer": "Reviewer_TcRw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n1.\tThe paper is well written, and the content is clear, except some minor problems.  \n2.\tThe three techniques effectively enhance ILA.\n3.\tI appreciate the use of reverse adversarial update on the clean example and the automatic parameter selection scheme.\n\n\nWeaknesses (major)\n1.\tRecently, most of transferability methods published in top conferences are evaluated on both undefended and defended models. The authors only evaluated their method on undefended models. If we consider that the evaluation of defended models is a must, the paper is not ready for publish. \n2.\tSome strong transferability methods, such as LinBP, have evaluated on target attacks. However, no such experiment was provided. \n3.\tThe method is very sensitive to the selection of layers in the source model. For example, the Inception V3 in Fig. 2(c) shows that the difference of the attack success rates between the 6a and 6e layer is more than 20% and the VGG19 in Fig. 2(b) shows that the difference of the attack success rates between 9 and 12 layer is around 20%. Once non-optimal layers are selected, the attack performance would drop very significantly. Does the proposed method still outperform other state-of-the-art methods in that situation? In the real black attacks, attackers cannot select the layers, as the authors did, because they don’t have attack accuracy of target models. \n4.\tThe authors mentioned “VMI-SI-TI-DI-FDSM does not work well in terms of black-box attack, despite its remarkable performance in attacking defended models”. The statement has two problems. 1) VMI-SI-TI-DI-FDSM was examined on a black-box setting when their method was evaluated on defended model. 2) Why don’t compare the proposed method with it on the defended models? \n\n\nWeaknesses (others)\n1.\tThe authors method that “The combination of LinBP and SGM also incurs overhead as it requires 300 iterations of I-FGSM on ImageNet, which is 30 times more than the reference attack in Aug-ILA. Thus, statement is inaccurate. In setup, the authors mention that they use 10 iterations for I-FGSM and 50 iterations for Aug -ILA. In the 50 iterations, gradients are needed. Thus, this statement is inaccurate, although speed is not very important for adversarial attack. \n2.\tIn Eq. 1, better to write clearly that dot represents dot product and the outputs of Eq. 2 and 3 are vectors.\n3.\tAppendix F, in the training, do the victim models use copping with 0.9-0.97 as their augmentation? Thus, the curves have similar behaviours. \n4.\tNo experiment shows that the proposed method can work with other methods to enhance overall transferability.\n\n", "summary_of_the_paper": "The authors proposed / used three techniques to enhance the transferability of ILA on undefended models. The techniques include image augmentation, reverse adversarial update on the clean example and reference attack update through interpolation with an automatic parameter selection scheme. The authors evaluated their method on nine undefended models and showed the proposed method outperforms ILA and ILA++ and other state-of-the-art methods ", "summary_of_the_review": "The paper is well-written and the proposed method does improve ILA. However, due to the insufficient experiments, especially evaluation on the defended models and target attacks and the sensitivity of the selected layers, this paper is not ready for publish. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper develops a new method, named Augmented Intermediate Level Attack (Aug-ILA), to improve the transferability of black-box attacks. Specifically, the proposed Aug-ILA contains three key modules: image transformations, reverse-adversarial update, and attack interpolation. 

Overall, the reviewers think it is an interesting paper, but are concerned that the original ablations are not enough to support the effectiveness of the proposed method, including missing strong baseline attacks and defense methods, and only one dataset is considered. During the discussion period, the authors actively provide new results. However, the Reviewer TcRw and the Reviewer 2dLg are not fully convinced by the rebuttal, especially regarding 1) in these additional experiments, no comparison is provided with other SOTA attacks beyond ILA-based approaches; 2) Table 11 shows the proposed method even degrades (rather than improves) the performance of VNI-CT-FGSM on defense models;  3) the attack rate of the proposed method is sensitive to the selection of layers, therefore, need to be carefully tuned in experiments (which could lead to unfair comparisons to other attacks). These concerns are indeed legitimate, and should be addressed carefully before publication.

I encourage the authors to incorporate all the reviewers' comments and make a stronger submission next time.
Decision: Reject