The Ultimate Combo: Boosting Adversarial Example Transferability by Composing Data Augmentations
Authors: Zebin Yun
Achi-Or Weingarten
Eyal Ronen
Mahmood Sharif
Venue: Submitted to ICLR 2023 (2023)
Keywords: Adversarial machine learning
transferability
evasion
black-box attacks

Abstract: Transferring adversarial examples from surrogate (ML) models to evade target models is a common method for evaluating adversarial robustness in black-box settings. Researchers have invested substantial efforts to enhance transferability. Chiefly, attacks leveraging data augmentation have been found to help adversarial examples generalize better from surrogate to target models. Still, prior work has explored a limited set of augmentation techniques and their composition. To fill the gap, we conducted a systematic, comprehensive study of how data augmentation affects transferability. Particularly, we explored ten augmentation techniques of six categories originally proposed to help ML models generalize to unseen benign samples, and assessed how they influence transferability, both when applied individually and when composed. Our extensive experiments with the ImageNet dataset showed that simple color-space augmentations (e.g., color to greyscale) outperform the state of the art when combined with standard augmentations, such as translation and scaling. Additionally, except for two methods that may harm transferability, we found that composing augmentation methods impacts transferability monotonically (i.e., more methods composed $\rightarrow$ $\ge$transferability)---the best composition we found significantly outperformed the state of the art (e.g., 95.6% vs. 90.9% average transferability from normally trained surrogates to other normally trained models). We provide intuitive, empirically supported explanations for why certain augmentations fail to improve transferability.

Under review as a conference paper at ICLR 2023
THE ULTIMATE COMBO: BOOSTING ADVERSARIAL EXAMPLE TRANSFERABILITY BY COMPOSING DATA AUGMENTATIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Transferring adversarial examples (AEs) from surrogate machine-learning (MI. models to evade target models is common method for evaluating adversarial ro- bustness in black-box settings. Researchers have invested substantial efforts enhance transferability. Chiefly, attacks leveraging data augmentation have been found help AEs generalize better from surrogates targets. Still, prior work has explored a limited set of augmentation techniques and their composition. To fill the gap, we conducted systematic study how data augmentation affects trans- ferability. Particularly, originally proposed to help ML models generalize to unseen benign samples, and assessed how they influence transferability, both when applied individually and when composed. Our extensive experiments with the ImageNet and CIFAR-10 dataset showed that simple color-space augmentations (e.g., color to greyscale) outperform the state of the art when combined with standard augmentations, such translation and scaling. Additionally, except for two methods that may harm transferability, we found that composing augmentation methods impacts trans- ferability monotonically (i.e., more methods composed >transferability) the best composition we found significantly outperformed the state of the art (c.g.. 95.6% vs. 92.0% average transferability on ImageNet from normally trained sur- rogates to other normally trained models). We provide intuitive, empirically sup- ported explanations for why certain augmentations fail improve transferability.
explored ten augmentation techniques of six categories
INTRODUCTION
Adversarial examples (AEs)-varian) variants of benign inputs minimally perturbed to induce misclassifi- cation test time have emerged profound challenge to machine learning (ML) (Biggio al.) 2013 Szegedy et al. 2014), calling its use in security- and safety-critical systems into question (e.g.. Eykholt al. (2018)) Many attacks have been proposed generate AEs in white-box set- tings, where adversaries are familiar with all the particularities of the attacked model (Papernot et al.) By contrast, black-box attacks enable evaluating the vulnerability of ML in realistic settings, without access to the model (Papernot et al. 2016).
2016
Attacks exploiting the transferability-rrpere of ABs (Szegedy al., 2014) have received spe- cial attention. Namely, as AEs produced against one model are often misclassified by others, transferability-basse attacks produce AEs against surrogate (a.k.a. substitute) white-box models to mislead black-box ones. To measure the risk of AEs in black-box settings accurately, researchers have proposed varied methods to enhance transferability (e.g., Lin al. 22000;; Liu et al.) (2017)).
Notably, attacks using data augmentation, such as translations (Dong et al. 2019) and scaling of pixel values (Lin et al.) 2020), as means to improve the generalizability of AES across models attacks
have accomplished state-of-the-art transferability rates. Still, previous transferability-bae have studied only four augmentation methods (see Section 3.1, out of many proposed in the data- augmentation literature (Shorten & Khoshgoftaar 2019). primarily for reducing model overfitting. Hence, the extent to which different data- augmentatmntnn types boost transferability, either individu- ally when combined, remains largely unknown.

---

Under review as a conference paper at ICLR 2023
To fill the gap, conducted systematic study of how augmentation methods influence transfer- ability. Specifically, alongside techniques considered in previous work, we studied how ten aug- mentation techniques pertaining six categories impact transferability when applied individually or composed (Section 3). Integrating augmentation methods into attacks via a flexible framework we propose (Algorithm we conducted extensive experiments using
ne-comet-ompaiille dataset, CIFAR-10 10 (Krizhevsky) 2009), and 16 models, and measured transferability in diverse settings, in- cluding with and without defenses (Sections andd 5
Our results offer several interesting insights:
Simple color-space augmentations outperform sate-of-the-arr transferability-bassee attacks when composed with standard augmentations (Section 5.1)
Transferability has a mostly monotonic relationship with data-augmentation techniques. Except for two augmentation methods that may harm transferabilityy, composing additional augmentation methods either improves of preserves transferability (Section 5.2).
Out 2" compositions explored, the best composition we found, ULTIMATECOMBO, outprrooooo state-of-the the-art attacks by large margin (Section 5.3
We show empirical support to conjectures we raise concerning when data-augmennation tech- niques may be counterproductive to transferability (Section 5.4).
BACKGROUND AND RELATED WORK
Evasion Attacks Many evasion attacks assume adversaries have white-box access to models adversaries know models' architectures and weights (c.g., Goodfellow et al. (2015); Szegedy et al. (2014); Carlini & Wagner (2017)) These typically leverage first- second-order optimizations to generate AES models would misclassify. For example, given an input x of class model weights 0, and loss function J, the Fast Gradient Sign method (FGSM) of Goodfellow al. (2015). crafts an AE è using the loss gradients VzJ(r,y.6
$ = x + e sign V J y, 0))
where sign(-) maps real numbers to -1, O, or I, depending on their sign. Following FGSM, re- searchers proposed numerous advanced attacks. Notably, iterative FGSM (IFFGSM) of Kurakin ct al. (2017b) performs multiple gradient-ascent steps, updating & iteratively to evade models:
2++1 = Proj + a sign (VzJJ (6e. (.).
where Proj (-) projects the perturbation into loo- norm e-ball centered at x, a is the step size, and dt The attacks we study in this work are based on I-FGSM.
In practice, adversaries often lack white-bon box access to victim models. Hence, researchers studied black-box attacks in which adversaries may only query models. Certain attack types, such score- and boundary-based attacks perform multiple queries, often around several thousands, to produce AEs (e.g., Brendel et al. (2018): Ilaas al. (2019)) By contrast, attacks leveraging transferabil- ity (e.g.. Goodfellow et al. (2015); Szegedy et al. (2144) avoid querying victim models, and use surrogate white-box models create AES that are likely misclassified by other black-box ones.
Attempts to explain the transferability phenomenon attribute it to gradient norm of the target model (i.e., its susceptibility to attacks), the smoothness of classification boundaries, and, primarily, the alignment of gradient directions between the surrogate and target models (Demontis et al. 2019 Yang et al. 202T). Said differently, for AEs transfer, the gradient directions of surrogates need to be similar to those of target models (i.e., attain high cosine similarity).
Enhancing transferability an active research area. Some methods integrate momentum into attacks such as I-FGSM to avoid surrogate-specific optima and saddle points that may hinder transferabilitty (e.g.. Dong et al. (2018); Wang He (202I). Others employ specialized losses, such as reduc- ing the variance of intermediate activations (Huang et al. 2019) or the mean loss of model ensem- bles (Liu et al.) 2017). 7. to enhance transferability. Lastly, a prominent family of attacks leverages data augmentation to enhance AEs' generalizability between models. For instance, Dong al. (2019) boosted transfcrability by integrating random translations into I-FGSM. Evasion attacks incorpo- rating data augmentation attain state-of-the-art transferability rates (Lin et al. 2020
Wang et al.
2

---

Under review as a conference paper at ICLR 2023
Algorithm 1 MI-FGSM with data augmentation
Input: Benign sample z; ground-truth label y; loss function J()); model parameters 0; iterations T; momentum parameter μi; perturbation size data-augmentation method D(-).
Output: Adversarial example &t
I: a =/TT
2: I
Initialize adversarial example Initialize momentum
3: =0
4: for to T do
5: Jt-1 Ei= Vx (D(())))
Expected loss gradient on augmented samples
6: Jt-1 1 p. et l8e+1
g++1
Gradient with momentum
7: fet1 Projž (&t + a sign (9(+1))
Update adversarial example
8: return /T
202 Nonetheless, prior work has only considered restricted set of augmentation methods for boosting transferability. By contrast, we aim to investigate the role data augmentation at enhanc- ing transferability more systematically, by exploring how more comprchensive set of augmentation types and their compositions affect transferability.
Defenses Various defenses have been proposed mitigate evasion attacks. Adversarial training procedure integrating correctly labeled AEs training-is one of most practical and effective methods for enhancing adversarial robustness (e.g.. Goodfellow al. (2013); Tramer et al. (0188)
Other defense methods sanitize inputs prior to classification (c.g.. Guo et al. (2018)); attempt to detect attacks (see Tramer (022 ); or seek to certify robustness in e-balls around inputs (e.g.. Cohen et al. (2019): [Salman et al. (2019)). Following standard practices in the literature (Wang et al. 202 we evaluate transferability-bass attacks against representative of these defense.
DATA AUGMENTATION FOR ENHANCING TRANSFERABILITY
Data augmentation is traditionally used in training, to reduce overfitting and improve generalizabil- ity (Shorten & Khoshgoftaar 2019). Inspired by this use, transferability-ban attacks adopted data augmentation to limit overfitting to surrogate models and produce AEs likely generalize and be misclassified by victim models. Algorithm depicts a general framework for integrating data aug- mentation into I-FGSM with momentum (MI-FGSM). In the framework, method D--) augments the attack with m variants of the estimated AE at each iteration. Consequently, the adversarial per- turbation found by the attack increases the expected loss transformed counterparts of the benign sample x (i.e., the distribution set by D(-) given 2.). Note that D(-)'s output may include x.
The framework in Algorithm is flexible, and admit any data- auteettgmentttion method. We use it to describe previous attacks employing data augmentation and to systematically explore new ones. Next, we detail previous attacks, describe data augmentation methods adopt for the first time enhance transferability, and explain how these can be combined for best performance.
3.1 PREVIOUS ATTACKS LEVERAGING DATA AUGMENTATION
Previous work explored the following augmentation methods to set D(-).
Translations Using random translations of inputs, Dong al. (2019) proposed translation- invariant attack to promote transfcrability. They also offered an optimization to reduce the attack's 's time and space complexity by simply convolving the model's gradients (w.rt. non-translated inputs) with a Gaussian kernel. While we use this optimization in the implementation for the interest of efficiency, highlight that the attack be well-captured by our framework.
Diverse Inputs Xie et al. (2019) proposed size-invarian attack. Their augmentation procedure samples random crops from 2; that later resized per the model's input dimensionality.
3

---

Under review as a conference paper at ICLR 2023
Scaling Pixels Lin al. (2020) showed that adversarial perturbations invariant to scaling pixel values transfer with higher success between deep neural networks (DNNs). In their case, D()) produces samples such that Dz)) I for i E {0, 1, m 1}. where m=5 by default.
Admix Wang et al. (2021a) assumed that the adversary has a gallery of images from different classes and adopted augmentations similar MixUp (Zhang et al. 2018a). For each sample x' from the gallery, Admix augments attacks with m (typically set to 5) samples, such that D( x'); = Āτ.
(ā, + 1 x'), where E {0, 1,.., m - 1}, and 17 [0, 1] is set to 0.2 by default. Notably, Admix degenerates to pixel scaling when = 0.
attacks compose (1) diverse inputs, scaling, and translations (Lin et al.js (2020) DST-MI-FGSM attack, and Wann & Hels (202I) DST-VMI-FGSM attack that also tunes the gradients' variance); or (2) Admix, diverse inputs, and translation (Wang et al. (2021a) Admix-DT-MI-FGSY attack). We describe how the compositions operate in Section 3.3
The leading transferability-
3.2 NEW AUGMENTATIONS FOR ENHANCING TRANSFERABILITY
While prior work studied the effect of spatial transformations (i.e., translations and diverse inputs), pixel scaling, and mixing on transferability, a substantially wider range of data-augmentation meth- ods exist. Yet, the impact of these on transferability remains unknown. To fill the gap, we exam- ined Shorten & Khoshgoftaar's s (2019) survey on data augmentation for reducing overfitting in deep learning and identified ten representative methods of six categories that may boost transferability. We present them in what follows, one category at time.
Color-space Transformations Potentially the simplest of all augmentation types are those applied in color-space. Given images represented as three-channel tensors, methods in this category manip- ulate pixel values only based on information encoded in the tensors. We evaluate four color-space transformations. First, we consider color jitter (CJ), which applies random color manipulation (Wul ett al., 2015). Specifically, we consider random adjustments of pixel values within a pre-defined range in terms of hue, contrast, saturation, and brightness around original values. Second, eval- uate fancy principle component analysis (fPCA). Used in AlexNet (Krizhvsky et 2017), fPCA adds noise to the image proportionally to the variance in each channel. Given an RGB image, fPCA adds the following quantity to each image pixel:
Pi. P2, 02
where ps and X are the 8hh cigenvector and eigenvalue of the of 3x 3 covariance matrix of RGB pixels, respectively, and a; is sampled once per image from Gaussian distribution (N(outtionnOOO 0.1). Third, we test channel shuffle (CS). Included in ShuffleNet training (Zhang et 2018b). CS simply swaps the orders of the image's 's RGB channels at random. Last, but least, we consider greyscale (GS) augmentations. This simple augmentation converts images into greyscale (replicating it three times to obtain an RGB representation). Mathematically, the conversion is calculated by WR x wG I + WB x=" where æ LG.. and x, correspond to the RGB channels, respectively, and wR. WG, and WB, all E [o, 1]. denote the channel weights, and sum up to
Random Erasing Inspired by dropout regularization, random erasing (RE) helps ML models focus on descriptive features of images and promote robustness to occlusions (Zhong et al.) 2020). To do so, randomly selected rectangular regions in images are replaced by masks composed of random pixel values. Similarly to RE, CutOul masks out regions of inputs to improve DNNs' accuracy (De- Vries& & Taylor 2017). The main difference from e is that CutOut uses fixed masking values, and may perform less aggressive masking when selected regions lie outside the image.
Kernel Filters Convolving images with kernels of different types produce certain effects, such as blurring (via Gaussian kernels), sharpening (via edge filters), or edge enhancement. We study the effect of sharpening (Sharp) transferability with edge-enhancement filters.
Mixing Images As a form of vicinal risk minimization, some augmentation methods mix images together, creating virtual examples for training. MixUp, the cornerstone behind Admix, computes weighted sums of images (Zhang et al. 2018a). By contrast, we consider CurMix, which replaces a region within one image with a region from another image picked from gallery (Yun et al. 2019.
4

---

Under review as a conference paper at ICLR 2023
Neural Transfer Augmentations using neural transfer (NeuTrans) preserve image semantics while changing their style. We use Gatys et al.[s (2013) generative model to transfer image styles to that of Picasso's 1907 self-portrait.
Meta-tearning--nspiredd Augmentations Meta-learning is a subficld of ML studying how ML al- gorithms can optimize other learning algorithms (Hospedales et al. 2021). In the context of data augmentation, algorithms such as AutoAugment have been proposed to train controllers to select an appropriate augmentation method avoid overfitting (Cubuk al. 2019). We use the pre-trained AutoAugment controller, encoded as a recurrent neural network, to select augmentation methods and their magnitude from a set of 13 augmentation methods.
3.3 COMPOSING AUGMENTATIONS
There are two ways to compose data latz-auugmetttiion methods in attacks, namely: parallel and serial composition. Figure I in Appendix A iillustrate both. In parallel composition, each augmentation method is applied independently on the input, and their outputs are aggregated by taking their union to augment attacks (i.e., D-')'s output). By contrast, serial composition applies augmentation methods sequentially, one after the other, where the first method operates on the original sample, and its predecessor's 's outputs. Consequently, serial
each subsequent augmentation function operates composition leads to an exponential growth in the number of samples, while parallel composition leads to linear growth. DST-MI-FGSM and Admix-DT-MI-FGIMM use serial composition. By contrast, we consider a substantially larger number of augmentation methods, which may lead to prohibitive memory and time requirements in the case of serial composition. Additionally, because the order of applying certain augmentations matters (e.g., GS then CutMix leads to different outcome that CutMix followed by GS), exploring a meaningful number of serial compositions (out of an order of 10! possibilities) becomes virtually impossible. Accordingly, we mainly consider parallel composition between data-augmentation methods. We only serially compose translations, scaling, and diverse inputs, for consistency with prior work (e.g.. [Wng et al. (2022)), We tested few serial compositions between new augmentation methods we consider and found they were significantly outperformed by their parallel counterparts. While non-exhaustive, this hints that serially composing augmentations may not be a promising direction for enhancing transferability.
4 EXPERIMENTAL SETUP
Now we turn to the setup of our experiments, including the data, models, and attack configurations.
Data We used an ImageNet-conpatibe (e.g.. (Dong et al. 2019 Yang et al. 2002)). The former contains 1,000 images, originally collected for the NeurlPS 2017 adversarial MI. competition. For the latter, we sampled 1,000 images, roughly balanced between classes, from the test set.
datase and CIFAR-10 for evaluation, per common practice
Models We used 16 DNNs to transfer attacks from (as surrogates) and to (as targets)-six six for CIFAR-10 and ten for ImageNet. All CIFAR-10 models and six of the ImageNet models were normally trained, while the other four ImageNet models were adversarially trained. To facilitate comparison with prior work, we included models that are widely used for assessing transferability (e.g., (Wang et al. 2021a; Yang et al., (02T)). Furthermore, to ensure that our findings are general, we included models covering varied architectures, including Inception, ResNet, VGG, DenseNet, and MobileNet. Appendix B provides more details about the models.
Attack Parameters We tested standard attack configurations, in line with prior work (Wang et al. 2021a Yang et al.) 2 Namely, we evaluated untargeted MI-FGSM-base attacks, bounded in lo norm. We validated findings with varied perturbation norms. For ImageNet, unless stated 16 but also experimented with { 255 2. For CIFAR-10, 255
otherwise, we tested
experimented with e E (0.02,0.04) 0.043. We quantified attack success via transferabiliity rates-the the percentages of attempts at which AEs created against surrogates were misclassified by vietims. As baselines, we used three state-of-the-ar transferability-baae attacks: DST-MI-FGSM, DST-VMI- FGSM, and Admix-DT-MI-FGSM (see Section 3.I. Appendix reports the parameters used in
httpss///bit.ly/3fgee
5

---

Under review as a conference paper at ICLR 2023
attacks and augmentation methods. Appendix D discusses attacks considered but excluded from experiments.
EXPERIMENTAL RESULTS
This section summarizes our findings. We start by evaluating individual augmentation methods and standard combinations with scaling, diverse inputs, and translations (Section 5.1). We then turn to analyzing all possible compositions between different augmentation types to assess whether trans- ferability typically improves when considering additional augmentations (Section Our analysis helped us identify the best performing composition for boosting transferability, denoted by UL- TIMATECOMBO, outperforming state-of-the- thettrrrrt attacks. Section 5.3 reports rigorous comparisons between ULTIMATECOMBO and the baselines, including against defended models. Finally, we help develop intuition for when augmentations may may not help improve transferability (Section
5.1 COLOR-SPARS SAAOR AUGMENTATIONS SIGNIFICANTLY ADVANCE THE STATE OF THE ART
Initially, we evaluated transferability integrating a single augmentation at a time in attacks, or when composing individual augmentations with diverse inputs, scaling, and translation (DST), as is stan- dard (Lin et al. 2020 Wang et al., 2021a). We found that considering each of the ten augmentations individually does not lead to competitive performance with the baselines (Table 9in Appendix (E). However, composing individual augmentations with DST enhanced transferability markedly (Ta- ble TO in Appendix E). Surprisingly, augmentations in color-space fared particularly well, outper- forming the baselines and advanced augmentation methods (e.g., AutoAugment) in most cases.
Composing GS with DST (GS-DST-MI-FGSM attack) performed best in this setting. Table re- ports the transferability rates from four normally trained models to other models on ImageNet (see Table in Appendix for more details). It can be immediately seen that GS-DST-MI-FGSM attains higher transferability than the baselines (93.6% vs. <92.0%. on avg.). This held also when considering different perturbation norms on ImageNet, where GS- DST-MI-FGSSI outperformed the baselines with sometimes higher margin (e.g.. 75.9% <70.8% on avg. with ž Table in Appendix [F) GS-DST-MI-FGSM also outperformed the baselines on CIFAR-10, when trans- ferring ABs to normally trained DNNs of different architectures, with perturbation norms c=0.02 (74.9% vs. <71.5% avg. transferability rate) and c=0.04 (92.1%) 1% vs. <89.6% avg. transfcrability rate). Tables 14 and in Appendix G show the detailed CIFAR-10 results.
Model
Attack
Inc-v3
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
MAXBASELINE
100.0
94.7
90.7
88.9
89.1
92.6
Inc-v3
GS-DST-MI-FGIS
100.0
95.6
93.7
91.8
90.9
94.9
ULTIMATECOMBO
100.0
94.3
92.7
97.1
MAXBASELING
95.3
100.0
91.0
89.9
88.4
93.5
Inc-v4
GS-DST-MI-FGHS MI-FGSM
96.5
100.0
94.
92.5
93.0
95.4
ULTIMATECOMBO
98.1
99.9
94.8
95.0
94.6
96.8
MAXBASELINE
88.3
85.0
97.6
99.9
96.9
87.2
Res-101
ES-DST-MI-PEE MI-FGSM
89.0
84.8
97.6
99.8
97.7
87.6
ULTIMATECOMBO
93,0
90.4
98.1
99.7
97.8
91.8
MAXBASELINE
95.8
94.7
92.9
92.9
99.8
IncRes-v2
GS- SEST-M--FGSM
96.5
95.6
95.5
94.2
94.7
100.0
ULTIMATECOMBO
98.2
97.1
96.3
96.5
95.7
100.0
Table 1: Transferability rates (%) ImageNet, from normally trained surrogates (rows) normally trained targets (columns). All attacks are black-box, except for when the surrogate and target models the same. MAXBASELINE the best performing of the three baselines.
The same trends held when transferring AEs to adversarially trained models. Here, we transferred AEs from individual, normally trained DNNs (Table reports a summary, and Table 12 shows com- plete results), as well as an ensemble DNNs (Table B used to boost transferability further (Liu et 2017)) finding that all, according to paired t-test, the differences between GS- DE----S---SSSSS and the baselines across different surrogate and target models were statistically significant (p <0.01).
attained better transferability than the baselines. Over-

---

Under review as a conference paper at ICLR 2023
Model
Attack
Iad-v3adv
Inc-v3ense enss
Inc-v3ens4
IncRes-v2 v2
MAXBASELINE
84.6
84.8
83.5
70.8
Inc-v3
GS-DST-MI-FGES
87.3
88.5
85.5
72.2
ULTIMATECOMBO
88.2
88.7
86.7
72.6
MAXBASELINE
84.3
86.0
83.0
74.6
Inc-v4
GS-DST-MI-FGSM
87.6
89.5
87.2
78.0
ULTIMATECOMBO
88.6
89.4
88.4
78.2
MAXBASELINE
82.0
83.0
80.9
72.5
Res-101
GS-DST-MI-FGSM
82.3
83.7
81.1
73.2
ULTIMATECOMBO
83.5
86.7
82.8
76.8
MAXBASELINE
89.0
89.0
88.7
87.1
IncRes-v2
GS-DSTTMM-PGGS
91.1
92.2
90.0
88.2
ULTIMATECOMBO
92.2
92.6
92.0
88.5
Table 2: Transferability rates (%) on ImageNet, from normally trained surrogates (rows) to adver- sarially trained targets (columns). MAXBASELINE is the best performing of the three baselines.
Attack
Inc-v3ade Inc-v3ens3
Inc-v3ensa
IneRes-v2ena
DST-MI-FGSM
89.0
90.0
87.6
82.4
Admix Admix-DT-MI--E DT-MII-GSM
90.1
90.5
89.4
84.7
GS- GS-DST-MI-FGSM ULTIMATECOMBO
92.4
93.5
92.5
88.7
93.6
95.2
93.7
91.2
Table 3: Transferability
(%) on ImageNet, from an ensemble of normally trained surrogates (containing Inc-v4, Res-50, Res-101 and Res-152) to adversarially trained target models. DST-VMI- FGSM was excluded due to resource constrains.
Finally, we evaluated attack run-times, finding that, despite investing no effort improve its ef- ficiency, GS-DSTTMM-FGSM is at least x1.14 more time-efficient than -Lmix-DT-MI-FGSM and DST-VMI-FGSM, on avg. (Table [6] Appendix II.. Still, we denote that, since transferability- based attacks generate AEs offline, and only once per surrogate model, as long an attack is not prohibitively slow, attack run-time a marginal consideration for selecting an attack compared to transferability rates.
5.2 THE MONOTONICITY OF TRANSFERABILITY WHEN ADDING AUGMENTATIONS
We wanted evaluate whether transferability is monotonic in the number of augmentation types considered-3se -ic... whether composing more techniques increases, or at least does not harm, trans- ferability. To this end, we selected the best performing augmentation method of each of the categories presented in Section 3.2 as well as DST-MI-FGSM and evaluated all 2 (=128) com- positions possible (per Section 3.3 More precisely, we tested every possible combination of GS, CutOut, Sharp, NeuTrans, Autonugment, Admix, and DST-MI-FGSM. Given a composition, produced AEs against the Inc-v3 ImageNet DNN as surrogate, and computed the expected transfer- ability rate against all other nine ImageNet DNNs, both normally and adversarially trained. Then, for every pair of attacks differing only in whether a single augmentation method was incorporated in the composition, we tested whether adding the augmentation method improved transferability.
The results reflected a mostly monotonic relationship between transferability and augmentations. Except for NeuTrras and Sharp, which sometimes harmed transferability when considered within a composition, adding augmentation method increased or preserved transferability. Figure 2 in Appendix H summarizes the results. Notably, comparing all compositions enabled find that composition of all seven augmentation methods except for NeuTram attained the best transferabilityy We call this composition the ULTIMATTECOMBO.
5.3 THE MOST EFFECTIVE COMBINATION
We evaluated ULTIMATECOMBO extensively, testing transferability to normally and adversarially trained DNNs. As shown in Table 1 ULTTMATECOMBO obtained higher transferability normally

---

Under review as a conference paper at ICLR 2023
trained models than the baselines (95.6% vs. 592.0% avg. transferability) and GS-DST-MI--GSMM when normally trained models were used as surrogates. This holds across different values e (Table 13). and the CIFAR-10 dataset with different architectures (Tables and 15
Furthermore, ULTIMATECOMBO achieved the best performance also when transferring attacks from normally trained to adversarially trained DNNs (Table 2 86.0% <82.7% avg. transferabilitty. Transferring AEs crafted by ULTIMATECOMBO using an ensemble models increased transfer- ability further (Table 3 93.4% avg. transferability). Per paired t-test, the differences between ULTIMATECOMBO and the baselines over all pairs of surrogates and targets considered are statisti- cally significant (p <0.01).
Besides adversarially trained models, we evaluated ULTIMATECOMBO) transferability against five defenses. Two defenses, bit reduction (Bit-Red) (Xu al., 2018) and neural representation pu- rification (NRP) (Naseer et al., 2020). transform inputs to sanitize adversarial perturbations. Two others, randomized smoothing (RS) (Cohen et al. 2019) and randomized smoothing with adversarial training (ARS) (Salman al., 2019) offer provable robustness guarantees. Finally, TRS leverages an ensemble of smooth DNNs trained to have misaligned gradients, to defend attacks (Yang et al.) 2021]. We evaluated all defenses except for TRS on ImageNet. We used the defenses with default parameters (see Appendix D). and transferred AEs crafted against an ensemble of normally trained models. Results shown in Table 4 Similar to other settings, here too, ULTIMATECOMBO out- performed the baselines (66.8% vs. 263.9% avg. transferabilityy. Following Yang et al. (002), we tested TRS on CIFAR- 10 with adversarial perturbation norms (0.02, 0.043. ULTIMATECOMBO did best against this defense as well (T.
Attack
Bit-Red
NRP
RS
ARS
DST-MI- MI-FGSM
85.3
40.7
84.2
39,8
Admix DD-MI-----S-SSSS
86.4
39.4
86.6
43.0
ULTIMATECOMBO
87.5
47.7
88.4
43.5
Table 4: Transferability rates (%) from an ensemble of normally trained surrogates (Inc-v4, Res- 50, Res-101 and Res-152) to models defended by provable methods or input transformations. DST- VMI-FGSM was excluded due to resource constrains.
Epsilon
A'MI--DTT-MI-GGSM
DST-MI-FGSM
DST-VMI-FGSM
GS-DST-MI-FGSM
ULTIMATECOMBO
0.02
21.5
23.1
18.9
25.1
27.4
36.3
Table Transferability rates (%) on CIFAR-10 from a normally trained VGG surrogate DNN to an ensemble of Res DNNs trained via TRS.
Lastly, due to composing more augmentations, ULTIMATECOMBO is slower than DST-MI-FGSM DGSM and AdmiX-DT-MI-FGSM However, x2.44 faster than DST MSI-VMM-FGSM producing AEs (Table 16).
5.4 WHEN Do AUGMENTATIONS FAIL TO IMPROVE TRANSPERABILITY?
While augmentation methods mostly increased transferability, some cases they were counterpro- ductive. Particularly, NeuTrans and Sharp decreased transferability when composed with certain methods. We conducted simple experiments as a preliminary assessment of two conjectures we had concerning when augmentations may harm transferability.
First, we expected augmentation methods that harm model accuracy on benign samples to be less conducive for transferability. As DNNs do not generalize well to benign samples produced by these augmentation methods, we anticipated that adversarial perturbations relying on the augmented samples would also have limited generalizability across models. To support the conjecture, we tested the normally trained DNNs' accuracy on benign samples transformed by each augmentation method. As be seen from Table 66, NeuTrans and Sharp, which often decrease transferability (Section [.2] and Figure 2 harmed the DNN accuracy the most (6.5% 9%5%%..79 lower accuracy than other methods), supporting our conjecture.
8

---

Under review as a conference paper at ICLR 2023
Augmentation
Inc-v3
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
Avg.
None
96.2
97.4
94.5
96.3
95.8
99.8
96.7
CS
94.0
95.7
95.3
94.6
95.4
99.5
95.8
fPCA
91.6
96.8
89.9
92.8
93.6
99.4
94.0
CJ
90.0
92.3
90.3
90.3
91.4
96.8
91.9
Admix
86.7
91.6
86.8
88.9
89.7
94.6
89.7
CutOut
86.5
89.2
85.7
87.2
88.6
92.3
88.2
GS
86.6
90.3
84.7
87.6
92.7
88.1
AutoAugment
82.9
86.2
82.1
84.3
84.4
89.8
85.0
Sharp
69.5
87.3
71.5
76.7
75.5
90.6
78.5
NeuTrans
24.4
25.4
24.2
27.0
24.0
32.7
26.3
Table 6: Benign accuracy (%) after applying data augmentation methods. Rows are sorted descending order of average transferability.
Augmentation
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
Avg.
CutOut
0.568
0.583
0.581
0.574
0.591
0.579
CS
0.565
0.578
0.576
0,570
0.590
0.576
None
0.564
0.575
0.573
0,568
0.586
0.573
Admix
0.563
0.575
0,573
0.567
0.586
0.573
CJ
0.560
0.575
0.573
0.568
0.584
0.572
GS
0.559
0.572
0.569
0.563
0.582
0.569
Automugment
0.558
0.569
0.567
0,562
0,579
0.567
IPCA
0.560
0.568
0.566
0.561
0.578
0.567
NeuTrans 0.546
0.556
0.554
0.549
0.565
0.554
Sharp
0.548
0.548
0.545
0.540
0.558
0.548
Table Cosine similarities between gradients of benign images computed on Inc-v3 after applying augmentation methods composed with DST-MI-FGSM, and gradients of other normally trained models on benign images. Rows are sorted in descending order of average cosine similarity.
Prior work demonstrated that gradient alignment between surrogates and targets is needed for trans- ferability (Demontis et al. 2019). Thus, we expected augmentation methods that estimate target model gradients more accurately increase transferability further. To assess this conjecture, evaluated the cosine similarity between the gradients of the Inc-v3 model while using augmentations composed with DST applied benign samples, and the gradients of other normally trained models on (untransformed) benign samples. The results (Table 7 show some support to the conjecture- NeuTrans and Sharp led to lower cosine similarities with target models' gradients. Yet, the differ- ences in cosine similarities between augmentation methods were small (50.031, avg.).
CONCLUSION AND FUTURE WORK
Our study uncovered mostly monotonic relationship between data-augmentattin methods and transferability, and helped us identify simple yet effective composition of data-augmentation meth- ods, ULTIMATECOMBO, that outperforms previously proposed methods when integrated into tacks. The resulting attack should be considered as a standard baseline in follow-up work trans- ferability. Our work also puts forward conjectures for when augmentation techniques are expected to improve transferability, and offers some empirical support.
In the future, it would be informative to develop theory that formally explains why augmentation methods help increase transferability. Furthermore, instead of relying existing augmentation methods originally developed to improve DNN generalizability, an intriguing research direction would be to develop augmentation techniques tailored specifically for improving transferability. Lastly, in addition for assessing the vulnerability of ML models in black-box settings, would be interesting evaluate whether the ULTIMATECOMBO-ban based attack advances methods leveraging AEs for defensive purposes, by deceiving adversaries (c.g., to attain privacy (Cherepanova et al. 2021 Shetty et al. 2018)).

---

Under review as a conference paper at ICLR 2023
REPRODUCIBILITY STATEMENT
In the interest of reproducibility, we make our code publicly available at the following repository: https://tinyurll hompa//oooo///iiiiiaaaacoooooll
REFERENCES
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Smniić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Proc. ECML/PKDD, 2013.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In Proc. ICLR, 2018.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In Proc. IEEE S&P, 2017.
Valeriia Cherepanova, Micah Goldblum, Harrison Foley, Shiyuan Duan, John Dickerson, Gavin Taylor, and Tom Goldstein. Lowkey: Leveraging adversarial attacks to protect social media users from facial recognition. Proc. ICLR, 2021.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proc. ICML, 2019.
Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. AutoAugment: Learning augmentation policies from data. In Proc. CVPR, 2019.
Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, and Fabio Roli. Why do adversarial attacks transfer? Explaining transfer- ability of evasion and poisoning attacks. In Proc. USENIX Security, 2019.
Terrance Dries and Graham W Taylor. Improved regularization convolutional neural networks with cutout. arXiv preprint 1708. 040800 2017.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proc. CVPR, 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversarial examples by translation-invariant attacks. Proc. CVPR, 2019.
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song. Robust physical-worid attacks on deep learning visual classification. In Proc. CVPR, 2018
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. A neural algorithm of artistic style, arXiv preprint 7508.06576, 2015.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proc. ICLR, 2015.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten. Countering adversarial 2018.
images using input transformations. In Proc. 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Identity mappings in deep residual networks. In Proc. ECCV, 2016.
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE PAMI, 44(9):5149-51111 51699,9511190111111 2021.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Weinberger. Densely connected convolutional networks. In Proc. CVPR, 2017.
Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. Enhancing adversarial example transferability with an intermediate level attack. In Proc. ICCV, 2019.
10

---

Under review as a conference paper at ICLR 2023
Andrew Ilyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversarial attacks with bandits and priors. In Proc. ICLR, 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convo- lutional neural networks. CACM, 60(6):84-90, 2017.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. 2017a.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In Proc. ICLRW. 2017b.
Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E Hoperoft. Nesterov accelerated gradient and scale invariance for adversarial attacks. In Proc. ICLR, 2020.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks, In Proc. ICLR, 2017.
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self-supervised approach for adversarial robustness. In Proc. CVPR, 2020.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In Proc. Euro S&P, 2016.
Hadi Salman, Jerry Li, Ilya Razenshteyn, Pengchuan Zhang, Huan Zhang, Sebastien Bubeck, and Greg Yang. Provably robust deep learning via adversarially trained smoothed classifiers. In Proc. NeurlPS, 2019.
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chich Chen. Mo- bilenetv2: Inverted residuals and linear bottlenecks. In Proc. CVPR, 2018
Rakshith Shetty, Bernt Schiele, and Mario Fritz. A4NT: Author attribute anonymity by adversarial training of neural machine translation. In Proc. USENIX Security, 2018.
Connor Shorten and Taghi M Khoshgoftaar, survey on image data augmentation for deep learning. Journal of big data, 6(1):1-48, 2019.
K. Simonyan and Zisserman. Very deep convolutional networks for large-scale image recogni- tion. In Proc. ICLR, 2015.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In Proc. ICLR, 2014.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Du- mitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proc. CVPR, 2015.
Christian Szegedy, Vincent Vanhoucke. Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proc. CVPR, 2016.
Christian Szegedy, Sergey loffe, Vincent Vanhoucke, and Alexander A Alemi. Inception-v4, Inception-ResNet and the impact of residual connections on learning. In Proc. AAAI, 2017.
Florian Tramer. Detecting adversarial examples is (nearly) as hard as classifying them. In Proc. ICML, 2022.
Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick Mc- Daniel. Ensemble adversarial training: Attacks and defenses. In Proc. ICLR, 2018.
Xiaosen Wang and Kun He. Enhancing the transferability of adversarial attacks through variance tuning. In Proc. CVPR, 2021.
11

---

Under review as a conference paper at ICLR 2023
Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In Proc. ICCV, 2021a.
Xiaosen Wang, Jiadong Lin, Han Hu, Jingdong Wang. and Kun He. Boosting adversarial transfer- ability through enhanced momentum. In Proc. BMVC, 2021b.
Ren Wu, Shengen Yan, Yi Shan, Qingging Dang, and Gang Sun. Deep image: Scaling up image recognition. arXiv preprint 1501.02876, 2015.
Weibin Wu, Yuxin Su, Michael R Lyu, and Irwin King. Improving the transferability of adversarial samples with adversarial transformations. In Proc. CVPR, 2021.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan L Yuille. Improving transferability of adversarial examples with input diversity. In Proc. CVPR, 2019.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. In Proc. NDSS, 2018
Zhuolin Yang, Linyi Li, Xiaojun Xu, Shiliang Zuo, Qian Chen, Pan Zhou, Benjamin Rubinstein, Ce Zhang, and Bo Li. TRS: Transferability reduced ensemble via promoting gradient diversity and model smoothness. In Proc. NeurIPS, 2021.
Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. CutMix: Regularization strategy to train strong classifiers with localizable features. In Proc. ICCV, 2019.
Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In Proc. ICLR, 2018a.
Xiangyu Zhang, Xinyu Zhou, Mengxiao Lin, and Jian Sun. ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In Proc. CVPR, 2018b.
Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmen- tation. In Proc. AAAI, 2020.
A PARALLEL AND SERIAL COMPOSITIONS OF AUGMENTATIONS
Figure Dillustrates how parallel and serial compositions work.
B DNNs USED IN THE EXPERIMENTS
We tested transferability using ten ImageNet DNNs and six CIFAR- DNNs. Ohe ten ImageNet models, six were normally trained, while others were adversarially trained. Specifically, for nor- mally trained models, we selected: Inception-v3 (Inc-v3) (Szegedy et al. 2016); Inception-v4 (Inc- v4);
(IncRes-v2) (Szegedy et al.] (01 ves (Res-50); ResNet-v2- 101 (Res-101); and ResNet-v2-152 (Res-152) (He et al. 2016). For adversarially trained models, we selected: Inception-v3addy (Inc-v3adu)) (Kurakin et al. 2017a); e (Inc-v3mnsa) ens4-Inception-v3 (Ine-v3ans;)) ); and Ins-ady-Inception-Ree -eetoot-- (IncRes-v2ens) (Tramèr et al., 2018). We obtained the models" PyTorch implementations and weights from public GitHub repository Fl All six CIFAR-I0 DNNs were normally trained. For this dataset, we used pre- trained VGG-11 (VGG) (Simonyan & Zisserman 2015),, ResNet-50 (Res) (He et al. 2016). DenseNet- 121 (DenseNet) (Huang et al. 2017), MobileNet-v2 (MobileNet) (Sandler et al. 2018), GoogleNet (Szegedy et al. 2015). and Inception-v3 (Inc) DNNs (Szegedy et al. 2016), also
implemented in PyTorch
htt.a:///thhbbooo///////fttttttt |
12

---

Under review as a conference paper at ICLR 2023
Serial Composition
Paralle Composition
Tr.
DL
Sc.
CutOut
Auto Augmenr
Figure 1: An illustration of serial and parallel compositions. When serially composing augmenta- tions, each augmentation method operates on the output of the previous one. By contrast, parallel composition, each augmentation method operates independently the input (or set of inputs). The number samples grows exponentially in serial composition, whereas it grows linearly in parallel composition. We serial composition when composing diverse inputs (DI), scaling (Sc.), and translations (Tr.). Other augmentation methods are composed parallel.
C ATTACK AND AUGMENTATION METHOD PARAMETERS
Similarly to Wang et al. (2021b) we set the MI-FGSM decay factor p=1.0, and the number of iterations T=10
We mostly used default commonly used parameters of augmentation methods. For CJ, per- formed random adjustments of image hue [-0.5, 0.5), contrast E [0.5, 1.5), saturation E [0.5, 1.5), and brightnessE [0.5, 1.5]. For CutOut, we replaced values in selected regions with zeros, and the portion of masked areas compared image dimensions lied in [0.02, 0.4), with aspect ratios RE, the dimension of masked areas relatively the image dimen- sions lied in [0.02, 0.2), with aspect ratios 00.,3.3). For Sharp, used the following edge- enhancement mask:
[0.4, 2.5). comparison,
0.5
0.5
-0.5
0.5
5.0
-0.5
0.5
0.5
-0.5
For diverse inputs, images were transformed with probability 0.5. For the Admix operation, consis- tently with Wang et al. (2021a), we randomly sampled three images from other categories for mixing attack. However, for the interest of computational efficiency,
part of the Admix-DT-MI-GGAS we use only one image for mixing when composing Admix with other augmentation methods. We did find that mixing with fewer images harmed performance. In fact, it even improved transfer- ability in some cases. Finally, in CutMix, we picked the top left coordinate (ra, Ty), the width, Tw: and height, Th, of the region be replaced, using the formulas:
Tz U(0, W),
Tw W WVI
ry ~ U(0, H), TA = HVH---1 where 24 is the uniform distribution, W is the image width, H is the image height, and À is parameter set to 0.5.
In attempt enhance transferability further, optimized the parameters of few augmentation methods we considered via grid search. Except for the Gaussian kernel's size used translation- invariant attacks (Dong al. 2019), found that the selected parameters had little impact
13

---

Under review as a conference paper at ICLR 2023
transferability. Specifically, for translations, after considering Gaussian kernels of sizes E (5 x 5, 7 x 7,9 x 9}, we set the default to x 7, except for Admix-DT-MI-FGSM. for which the 9 x kernel performed best. Table g shows that choice of Admix parameters (m=1 and Gaussian kernel size of 9 x 9) improves its performance. For GS, we found wr. wG, and wB had little impact on transferability, as long the weight assigned to each channel was >0.1. Accordingly, wG, and w to 0.299. 0.587, and 0.1 0.114, respectively, per commonly used values (c.g., in the Python PyTorch p Finally, for CS, we only swapped the blue and green channels, this led to minor improvement compared to swapping all three channels.
set wn.
Attack
Inc-v3
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
Admix-DT-MI-FGGN (original)
99.5
92.3
88.5
87.0
85.3
90,9
AdmiX-DT-MI-FGSM (ours)
100.0
94.7
91.9
90.3
88.7
93.3
against an Inc-v3 sur- rogate, Our variant sets m=1 and the translation's 's Gaussian kernel to 7 and include original images when calculating gradients, whereas the original work uses m=3 and 9 9 kernel.
Table 8: Transferability rates (%) of AEs crafted via Admix-DT-MI-FGOI
Finally, we clarify that each of our attack combinations emits the original image once, alongside the transformed images. Moreover, when aggregating the gradients, the gradients of the original and transformed images are assigned equal weights. We tested whether weighting the gradients differ- ently (c.g., assigning higher or lower weight to the original sample) can help improve transferability using the GS method. However, we found that equal weights attained the best results.
D ATTACKS CONSIDERED BUT EXCLUDED
Besides the three fate-offthhe-ar baselines we experimented with, we considered including two other attacks in the evaluation. WW et al. | (202I) attack uses a neural network to create adversarial perturbations robust against transformations for enhanced transferability, and achieves competitive transferability rates. However, unfortunately, we were unable to find a publicly available imple- mentation of the attack. Huang et al. (2019) intermediate level attack improve AE transferability by reducing the variance of intermediate activations. We used the official implementation to test the attack CIFAR-10 with e=0.02 and the VGG DenseNet models as surrogates. The results showed that the transferabilityy rates were much less competitive that the three baselines we consid- ered (50.34% vs. 55.00% average transferability with VGG surrogate, and 45.68% vs. 55.56% average transfcrability with a DenseNet surrogate). Therefore, we removed the intermediate level attack for the remaining experiments.
E INDIVIDUAL AUGMENTATIONS
Table presents the transferability rates when integrating individual augmentation methods into MI-FGSM. Table I0 presents the transferability when composing individual augmentation methods with DST. Trasnferability rates were computed on ImageNet, using the Inc-v3 DNN as a surrogate and the other normally trained DNNs as victims (c 255 Notice how composing color-space aug- mentations (specifically, CS, CI, and GS) with DST helps improve transferability over the baselines
(Table 10
F TRANSFERABILITY RATES ON IMAGENET
Tables 11 and 12 detail the trasferability rates on ImageNet, from all ten DNNs to normally and adversarially trained models, respectively. Here, we also consider transferring AEs from adversar- ially trained surrogates. Table shows the transferability rates on ImageNet from Inc-v3 to other normally traiend models with varied perturbation norms (i.e., values e).
https://bit. http://AA//mmmemmaaaaaaeeeeeeee/
https://github.rt
14

---

Under review as a conference paper at ICLR 2023
G TRANSFERABILITY RATES ON CIFAR-10
Tables 14 and 13 report attack tranferability rates from all six normally trained DNNs to all other victim DNNs for ==0.02 and e=0.04, respectively.
H THE MONOTONICITY OF TRANSFERABILITY WHEN ADDING AUGMENTATIONS
Figure 2 depicts a visual summary of the experiment presented in Section 5.2 demonstrating how the relationship between augmentation methods and transferability is mostly monotonic.
I ATTACK RUN-TIME
MI-FGSM's time complexity is predominated by the gradient computation steps. Accordingly, the attacks' run-times are directly affected by the number of samples the augmentation methods create (i.e., samples emitted by D--) in Algorithm I): The more samples emitted by the augmentation method, the more back propaagttotnnn would be required to compute gradients for updating the adver- sarial examples each iteration, thus increasing the AF-generation time. The empirical measure- ments corroborate this intuition (Table T6). Overall, we can see that DST augments MI-FGSM with the least samples, leading to the fastest attack (DST-MI-FGSM), GS-DST- DST--MI-FGSM is the second fastest attack, while ULTIMATECOMBO is slower than Admix-DT-MIL MI-MMMM but substantially faster than DST-VMI-FGSM We note that no particular effort was invested to make GS- GS-DST-MI--GSM and ULTIMATECOMBO more time-efficient (e.g., stacking augmented samples for parallel compu- tation, similarly to DT-MMM-TG--GMM Moreover, since transferability--oee based attacks generate AEs offline, and only per surrogate model, as long as attack is prohibitively slow, attack run-time a marginal consideration for selecting an attack compared to transferability rates.
J DEFENSE PARAMETERS
We used standard parameters when attacking defenses. For RS. we used a normally trained ResNet- 0.25, following Cohen et al. (2019). For ARS, the target model was ResNet-50
50 and trained with isotropic Gaussian-noise augmentations (sampled from (0, 0.25)), and was set to 0.25 during prediction, per (Salman al. (2019). In both cases, used 10,000 noisified samples during inference. For Bit-Red, we used squeezer with bit-depth of one, in accordance with Xu al. (2018). Finally, we used the default NRP parameters and pre-trained model from the official GitHub repository (Naseer et al. 2020).
15

---

Under review as a conference paper at ICLR 2023
Attack
Inc-v3
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
MI-FGSM
100.0
54.7
48.9
43.5
41.1
50.6
JPCA-MI CEA-MI-FOGT F--SMMM
100.0
70.7
65.4
59.6
57.6
69.2
CS-MI-FGSM
100.0
57.5
54.7
49.9
46.3
56.7
CJ-MI-FGSM
100.0
66.4
57.5
54.7
65.4
GS-MI-FGSM
100.0
62.9
56.5
51.8
62.4
RE-MI-FGSM
100.0
55.1
52.1
46.6
43.9
52.2
CutMix-MI-FGSS
63.0
34.7
33.9
30.0
33.1
31.2
CutOut-MI-FGSS
100.0
58.2
54.4
49.6
45.7
55.9
NeuTrans-MI-FGSII MI-FGSM
44.4
35.6
33.6
38.2
Sharp-MI-FGSM
99.3
44.7
41.0
34.3
34.4
40.2
AutoAugment-MI-FGSS
100.0
61.1
56.6
50.1
48.5
58.4
Table 9: Transferability rates (%) on ImageNet from normally trained Inc-v3 surrogate to normally trained target models (columns) when integrating individual augmentation methods into MI-FGSM- based attacks.
Attack
Inc-v3
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
Admix-Die DT-MMI-GGSM
99.5
92.3
88.5
87.0
85.3
90.9
BST-MI-GSSM
100.0
92.9
89.5
87.2
91.2
DST-VMI-FGSM
100.0
94.7
90.7
88.9
89.1
92.6
SCA-DST-MI-FGSST
100.0
94.3
90.3
88.6
87.3
90.8
CS-DST-MI-FGSH
100.0
94.7
92.8
90.3
88.7
93.3
CJ-DST-MI POST-MI--GSST F-GMMM
100.0
94.9
92.3
90.9
GS-DST-MI LDST-MI-FOC MI-FGSM
100.0
95.6
93.7
91.8
90.9
94.9
RE-DST- BE-DST-MI-FGSSS MI-FGSM
100.0
95.3
91.5
89.7
88.2
93.5
SOMMi--DSTMM--FCGE
69.5
61.8
59.6
58.1
56.4
58.6
COODU--DSTMM--FGSSS
100.0
95.0
91.4
89.8
89.1
93.4
NeuTrans- DST-DD-TGSSSSSS
97.8
87.2
82.5
79.8
84.4
Shar--DST-MI-GGSM
99.9
95.6
91.3
89.6
89.4
92.9
BoooAuugment--ST-M---EE
100.0
94.1
91.1
88.6
87.5
92.9
Table 10: Transferability rates (%) on ImageNet from a normally trained Inc-v3 surrogate to nor- mally trained target models (columns) when integrating individual augmentation methods composed with DST into MI-FGSM-based attacks. AdmiX-DT-MI-FGSM. DST-MI-FGSM, and DST-VMI! FGSM are baseline attacks from prior work.
16

---

Under review as a conference paper at ICLR 2023
Model
Attack
Inc-v3
Inc-v4
Res-50
Res-101
Res-152
IncRes-v2
Admix-DT-an
SBmni-DD-MII-GSM
99.5
92.3
88.5
87.0
85.3
90.9
DST-MI-FGSM
100.0
92.9
89.5
87.2
86.4
91.2
Inc-v3
DST-VMI-FGSM
100.0
94.7
90.7
88.9
92.6
GS-DST-MI-FGEE
100.0
95.6
93.7
91.8
90.9
94.9
ULTIMATECOMBO
100.0
98.0
95.1
94.3
92.7
97.1
Admix-DT-MI--GGA
93.7
99.3
86.7
84.9
84.7
89.6
DST-MI-FGSM
94.4
90.2
88.1
92.8
Inc-v4
DST-VMI-FOO FMMM-GGMM
95.3
99.9
91.0
89.9
88.4
93.5
AS-DST-MI-FFGE
96.5
100.0
94.1
92.5
93.0
95.4
ULTIMATECOMBO
98.1
99.9
94.8
95.0
94.6
96.8
Edmix-DT-MI--GSST
82.6
78.1
93.5
97.4
79.3
DST-MIL DST-MI-FGSM
86.7
83.2
97.3
99.9
96.6
84.9
Res-101
SEEVMI-FGSSA
88.3
85.0
97.6
99.9
96.9
87.2
GS-DST-MI-FGSM
89.0
84.8
97.6
99.8
97.7
87.6
ULTTMAAECCOMO
93.0
90.4
98.1
99.7
97.8
91.8
Admix Admix-DT-MII-FEE -DT-MI-FGSM
93.8
91.9
91.1
90.6
89.5
98.9
DST-MI-FGSM
95.8
94.2
93.5
92.0
92.4
99.8
IncRes-v2
AST-VMI-FGSSM
95.8
94.7
94.0
92.9
92.9
99.7
GS-DST-MI-FGSM
96.5
95.6
95.5
94.2
94.7
100.0
ULTIMATECOMBO
98.2
97.1
96.3
96.5
95.7
100.0
Admix-DT-MI-FGGN
83.9
80.0
97.8
93.4
93.2
78.9
DST-MI-FGSM
89.1
84.1
99.9
97.3
96.8
84.3
Res-50
DST-VMI-FGSS
88.7
84.7
100.0
98.1
97.1
86.7
GS-DST-MI-FGSM
90.3
85.7
99.9
98.3
87.5
ULTIMATECOMBO
94.1
91.6
99.9
99.0
98.2
90.9
82.9
79.0
92.5
92.3
96.5
77.9
DST-MI-FGSM
86.7
83.6
95.5
99.8
83.1
Res-152
DST-VMI-FGSM
88.4
85.3
95.6
97.2
99.7
85.6
GS-DST-MI-FGSM
87.7
84.3
96.7
97.6
99.8
85.3
ULTIMATECOMRO
91.8
90.4
97.0
97.3
99.5
90.3
Admix-DT-MIIFGGSM
92.5
87.9
89.0
88.2
86.8
88.3
DST-MI-FGSM
93.6
90.9
91.2
90.6
91.7
Inc-v3ade
DST-VMI-FGSM
95.9
91.7
91.3
91.7
90.3
92.6
GS-DST-MI-FOET
95.2
92.8
93.8
92.8
92.2
94.2
ULTIMATECOMBO
97.1
95.5
95.3
94.6
94.0
95.8
Admix-DT-MI-
87.9
83.4
85.3
84.5
83.9
85.3
DST-MI-FGSM
90.2
85,9
86.7
86.0
85.3
87.3
Ine-v3ensä
DST-VMI-FGSM
90.4
87.1
88.9
88.0
87.1
88.7
GS-DST-MI-FFS
93.2
90.4
90.7
90.0
90.1
90.7
ULTIMATECOMBO
94.5
92.1
93.2
92.3
92.1
92.9
Admix-DT-MI--FSTT
86.0
80.3
81.4
82.3
81.1
80.7
DST-MI-FGSM
88.8
85.6
83.4
85.9
83.7
84.4
Ine-vdensd
DST-VMI-FGSM
89.8
84.8
85.6
86.2
85.2
85.3
LL-DS-MM-FGSS
92.8
88.4
90.2
89.5
87.8
88.5
ULTIMATECOMBO
94.5
90.2
92.0
91.0
91.1
91.5
82.8
80.4
81.8
79.5
80.3
83.5
DST
DST-MI-FGSM
85.5
82.9
84.8
84.5
83.9
88.1
IncRes-v2ens
DST-VMI-FGSS
87.2
84.3
85.3
85.7
88.6
GS-DST-M DE------FSSSA
90.1
88.2
91.0
89.7
88.6
91.6
ULTIMATECOMRO
91.7
90.9
90.9
90.9
89.7
93.3
Table 1I: Transferability rates (%) ImageNet from surrogates (rows) normally trained target models (columns). All attacks are black-box, except for when the surrogate and target models the same.
17

---

Under review as a conference paper at ICLR 2023
Model
Attack
Inc-v3ade
Inc-v3ens3
Inc-v3ensd
IncRes-v2 v2ens
Admix-DT-MI-FGAI
84.3
83.5
70.8
DST-MI-FGSM
81.3
81.2
77.7
61.4
Inc-v3
84.4
84.8
82.9
69.2
GS-DST-MI-FOO
87.3
88.5
85.5
72.2
ULTIMATECOMBO
88.2
88.7
86.7
72.6
AdmiX-DT-MI-
DDTMM--GSMM
82.7
83.3
81.3
73.7
DST-MI-FGSM,
80.6
81.8
80.8
70.5
Inc-v4
DST-VMI-FGSM
84.3
86.0
83.0
74.6
GS-DST-MM-FGSM
87.6
89.5
87.2
78.0
ULTIOATECOMBO
89.4
88.4
78.2
79.5
80.4
78.6
71.2
DST-MI-FGSM
78.9
78.7
76.7
68.7
Res-101
DST-VMI-FGSM
82.0
83.0
80.9
72.5
GS-DST-MI-FGES
82.3
83.7
81.1
73.2
ULTIMATECOMBO
83.5
86.7
82.8
76,8
Admix-DY--MI-CGEE
89.0
89.0
88.7
87.1
DST-MI-FGSM
87.2
89.2
86.4
82.9
IncRes-v2
DST-VMI-FGSM
88.8
90.6
87.8
85.5
GS-DST-MI-FGSM
91.1
92.2
90.0
88.2
ULTIMATECCMMBO
92.2
92.6
92.0
88.5
Admix-DT-MIIFOSMM
79.1
78.0
77.1
68.0
DST-MI-FGSM
77.7
75.6
63.6
Res-50
DST-VMI-FGSM
80.7
80.7
79.5
69.7
GS-DST-MI-FGSM
81.4
83.6
78.9
70.4
ULTIMATECCOMB
85.1
85.7
83.4
74.0
Admix-DT-MI-FGSM
78.2
75.9
71.9
DST-MI-FGSM
75.0
78.4
75.7
70.1
Res-152
DST-VMI-FGSM
79.1
81.0
79.5
72.7
GS-DST-MI-FGET
79.6
82.1
79.0
71.3
ULTIMATECOMBO
83.6
85.0
82.5
74.8
Admix-DT-MMIFGSM
98.3
93.0
92.2
87.4
DST-MI-FGSM
99.7
93.0
93.5
85,4
Inc-v3adu
DST-VMI-FGSM
99.6
94.1
93.7
88.1
GS-DST-MI-FGSM
95.1
94.1
88.5
ULTIMATECOMBO
99.9
96.1
96.2
91.0
88.3
98.2
91.6
84.6
DST-MI-FGSM
87.8
99.9
91.9
83.0
Inc-v3ena3
DST-VMI-FGSM
99.7
92.6
84.8
GS-DST-MI DDST-MI-FGSM
91.6
99.9
93.4
85.8
ULTIMATECOMBO
94.0
99.9
94.7
89.6
Admix-DT-MI-GGSM
88.3
90.3
98.7
83.4
DST-MI-FGSM
88.2
91.8
99.8
83.8
Inc-v4
DST-VMI-FGSM.
89.7
91.7
100.0
85.6
US-DST-MI- MI-FGSM
91.8
94.1
99.9
86.9
ULTIMATRCCMME
93.4
95.4
99.9
90.2
AdmiX-DT-MI-
86.2
87.7
88.6
96.3
DST-MI-FGSM
88.2
89.4
89.8
98.7
IncRes-v2ens
DST-VMI-FGSM
89.4
90.5
91.3
99.3
GS GS-DSSTMI-FGSM
92.0
93.4
92.7
99.1
ULTIMATECOMBO
92.5
93.5
93.1
99.3
Table 12: Transferability rates on ImageNet (%) from ten surrogates (rows) to adversarially trained target models (columns).
18

---

Under review as a conference paper at ICLR 2023
Epsilon
Attack
Inc-v3
Ine-v4
Res-152
IncRes-v2
Res-50
Res-101
Admix-DT-MI-FGSM
98.3
67.8
50.0
59.1
57.4
53.5
DST-MI-FGSM
99.7
77.1
62.9
70.9
69.8
64.8
8/255
DST-VMI-FGSMM
77.9
64.1
72.9
72.2
GS-DST-MI-FGSS
99.5
81.3
71.1
77.6
77.3
72.0
ULTIMATEC
99.7
86.0
75.0
81.8
81.2
76.3
Admix-DT-MI-FGOI
99.8
94.9
88.2
93.1
89.4
88.4
DST-MI-FGSM
100.0
97.8
93.6
96.8
95.1
24/255
DST-VMI-FGSH
100.0
97.9
95.0
97.1
96.2
95.6
GS-DST-MI-FGAS
100.0
98.5
96.7
98.7
97.0
97.0
ULTIMATECOMBO
100.0
99.3
97.4
99.2
97.5
98.4
Table 13: Transfcrability rates (%) on ImageNet, from Inc-v3 surrogate to other normally trained other than the default 11
models, with perturbation norms 8
255
255
Model
Attack
VGG
Res
DenseNet
MobileNet
GoogleNet
Inc
Admix-DT-MI-PGSM
92.1
67.8
66.6
78.8
69.9
67.5
DST-MI-FGSM
93.5
73.5
71.5
80.4
71.6
71.5
VGG
FISM
68.9
52.3
50.5
61.6
54.0
51.6
GS-DST-MI-FGSHI
94.5
76.5
75.7
85.8
78.3
77.3
ULTIMATECOMRO
95.0
77.0
77.7
86.5
80.2
77.9
Sdmix-DT-MI--GSMM
47.1
94.9
74,2
78.9
68.1
64.3
DST-MI-FGSM
52.4
98.5
83.1
86.5
75.9
71.5
EST-VMI-FGSH
40.9
64.7
56.1
54.4
SS-DST-MI-FGGST
58.7
98.4
86.9
90.3
80.3
78.6
ULTIMATECOMBO
60.8
98.3
88.5
91.0
80.7
80.6
Admix-DT-MI-FGSM
50.8
78.4
94.7
81.4
70.8
70.7
DST-MI-FGSM
56.3
86.9
98.5
86.8
76.5
77.4
DenseNet
DST SST-VMI-FGSST
40.6
62.2
77.6
67.1
58.6
54.3
GS-DST-MI- MIS-MI-FGSM
61.5
88.8
98.4
89.1
81.1
81.8
ULTIMATECOMBO
64.3
90.0
98.6
92.2
82.4
84.2
Admix-DT-MI-B
34.6
54.5
50.8
99.8
70.3
64.0
DST-MI-FGSM
37.3
60.0
54.4
100.0
76.8
71.3
MobileNet
DST SEEVVMI-GMMMM
32.6
50.7
46.5
90.6
64.8
59.3
GS-DST-MI-FGSM
42.0
63.6
62.2
100.0
83.1
78.7
ULTIMATECOMRO
44.2
67.9
62.5
100.0
86.4
82.1
Admix-DT-MI-
DT-MI-FGSM
42.8
63.0
59.0
88.2
99.9
78.6
DST-MI-FGSM
45.7
65.8
62.5
92.7
100.0
84.2
GoogleNet
DST-VMI-FGSM
40.5
56.2
55.2
77.7
90.2
67.2
GS-DST-MI-FGEST
47.7
65.3
62.5
93.4
100.0
87.0
ULTIMATECOMBO
49.2
65.0
94.0
90.6
Admix-DTi
MAmix-DDT-IIFGGMM
46.8
65.4
63.3
89.6
85.4
98.0
DST-MI-FGSM
50.9
68.1
70.1
93.1
90.2
99.7
Inc
DST-VMI-FGSM
39.0
51.1
48.6
64.1
59.0
74.5
GS-DST-MI-FGSM
50.8
69.0
68.0
92.6
89.7
99.6
ULTIMATECOMBO
52.6
72.2
70.0
95.3
93.0
99.8
Table 14: Transferability rates (%) CIFAR-10, from normally trained surrogates (rows) to mally trained target models (columns), with perturbation norm 6=0.02.
19

---

Under review as a conference paper at ICLR 2023
Model
Attack
VGG
Res
DenseNet
MobileNet
GoogleNet
Inc
Admix-DT-MI-FGSMM
97.4
91.0
90.1
93.2
89.2
88.8
DST-MI-FGSM
91.7
92.9
94.2
89.8
VGG
DST-VMI-FGSM
90.4
75.5
72.0
79.3
75.6
74.4
GS-DST-MI-FORE
97.9
95.1
94.8
96.1
92.9
92.4
ULTIMATECOMIES
98.2
95.2
95.5
96.8
94.6
92.5
AdmiX-DT-MIFGSM
99.4
92.8
85.3
85.5
DST-MI-FGSM
77.5
100.0
96.9
95.9
91.6
90.9
Res
DST-VMI-FGSM
57.7
88.6
78.0
81.8
75.6
74.2
GS-DST-MI-FGGS
86.1
100.0
98.2
97.9
93.8
94.5
ULTIMATECOMBO
88.7
100.0
99.0
98.9
95.7
96.3
Admix-DT-MI-FGSM
78.8
94.1
98.3
93.9
89.4
90.0
DST-MI-FGSM
85.5
98.0
99.8
97.5
93.5
94.7
DenseNet
DST-VMI-FGSSE
59.8
80.2
88.2
80.8
73.0
72.5
GS-DST-MI-FOL
90.4
97.9
99.9
98.5
95.6
96.1
ULTIMATECOMBO
92.2
99.0
99.9
99.1
95.7
97.1
Admix-
51.5
76.5
70,6
99.9
87.0
86.1
DST-MI-FGSH
59.6
82.4
78.8
100.0
93.3
91.2
MobileNet
DST-VMI-FGHI
48.7
68.3
63.6
94.2
78.4
GS-DSS-MI-FGSS
70.
87.6
100.0
95.2
ULTTIATECCMMA
71.1
90.0
88.2
100.0
96.8
95.4
Admix-DT-MI-FOOSS
69.7
86.4
81.8
97.5
100.0
95.0
DST-MI-FGSM
73.1
88.6
97.8
GoogleNet
DST-VMI-FGSS
61.8
73.8
72.8
86.8
93.1
GS-DST-MI-FGSH
77.6
89.2
88.5
99.0
100.0
98.3
ULTIMATECOMBO
78.1
92.4
90.4
99.0
100.0
98.9
LAmix-DT-MI-FGSSY
74.2
87.1
88.1
97.0
96.0
99.3
DST-MI-FGGS FGSMGMM
79.5
92.2
91.2
98.9
98.4
100.0
Inc
DST-VMI-FGSM
55.4
65.3
63.0
74.7
72.2
81.2
GS-DST-MI-FGSM
79.7
90.6
91.0
97.7
98.3
100.0
ULTIMATECOMBO
82.6
93.8
92.7
98.9
99.3
100.0
Table 15: Transferability rates (%) on CIFAR-10, from normally trained surrogates (rows) to nor-
mally trained target models (columns), with perturbation norm c=0.04.
Augmented samples
Time (s)
Edmix-DT-MI--FSSS DST-MI-FGSM
15
1.68
0.72
DST-VMI-FGSM
105
11.29
GS-DST-MI-FGSM
10
1.47
ULTIMATECOMBO
30
4.63
Table 16: The number samples augmented and the average time of crafting an AE (seconds images) for different attacks. Times were measured on ImageNet, while attacking an Inc-v3 surrogate, and averaged for 1,000 samples. Experiments were executed on an Nvidia A5000 GPU.
20

---

Under review as a conference paper at ICLR 2023
Best viewed after zooming in. Each node represents a composition of augmentation methods. The binary string within a node encodes the composition Each bit, from the most to the least significant. denotes whether Admix (most significant bit), GS, CunOut, AutoAugment
S-T-MI-FGSMM Neu' Neu'Trans, and Sharp (least significant bit), respectively, is included (1) or excluded (0) from the composition An edge from node u to t is included if v includes exactly one more ferabiiiyyy rate per composition on ImageNet. from an Inc-v3 surrogate DNN to the remaining nine models as vietim models. An edge (u. v) is colored in black (resp. red) if v's composition achieves higher or equal (resp. lower) average transferability rates than when integrated into MI-FGSM. We faded away nodes containing NeuTrans or Sharp and their corresponding edges. Notice how all the remaining (unfaded) edges are black, showing that the relationship between the (average) transferabilityy and the remaining augmentations is monotonic (i.e., more augmentations composed 2 traanfferbbllity). Said differently, only NeuTramm and Sharp harm transferabilityy in some cases.
augmentation method compared to As explained in Section 5.2 we computed the average
Figure 2:
21


Reviews:
[{"review_id": "R9J1H94EFN", "reviewer": "Reviewer_DKMA", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Driven by the success of data augmentation in improving the transferability of adversarial samples, this paper conducts an empirical investigation about the best combination of different data augmentations towards boosting the adversarial transferability. Experiments across ten data augmentations lead to a new composition that outperforms previous data augmentation methods.", "summary_of_the_review": "Considering the limited novelty and the lack of necessary experiments as elaborated in the weakness section, I tend to reject this paper.", "strength_and_weaknesses": "**Strength**\n\n1. An empirical investigation about the combination of the best practices in data augmentations for boosting adversarial transferability can provide useful insights for the community.\n\n**Weakness**\n\n1. The novelty of this paper is very limited. It manually combines existing data augmentations to generate a better augmentation strategy without providing theoretical analysis or an automatic solution, which is more like a technical report. The technical contributions cannot match the bar of ICLR.\n\n2. The paper wastes too much contents on background knowledge about adversarial attacks and data augmentations, and starts to introduce the experimental results, which are the major contributions of this paper, from the 6th page. The presentation style can be better organized.\n\n3. The experimental results are not solid enough. Only one dataset with 1000 images is considered, while the datasets like CIFAR-10/100/SVHN/ImageNet, which are commonly adopted in the literature, are not included. In addition, only one perturbation strength (16/255) is considered. It is highly desired to conduct experiments across different datasets, perturbation strengths, and defensive methods, otherwise the accuracy gap between the proposed methods and baselines in Table 1/2 may be overturned via tuning the hyperparameters of adversarial example generation.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Hdf8CFLeKpY", "reviewer": "Reviewer_KVr6", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this work, the authors study in detail the effect of a diverse set of data-augmentation techniques on the transferability of adversarial attacks in the black-box setting. The paper presents a large systematized empirical study of the effect of adversarial transferability with the use of augmentations such as Color Jitter, Fancy Principle Component Analysis, Channel shuffle, Greyscale, random erasing, sharpening, CutMix, neural transfer and AutoAugment. In particular, the paper finds that the composition of several such augmentations generally yields increased transferability.", "summary_of_the_review": "The paper presents a systematic study regarding the composition of augmentations to boost adversarial transferability. Indeed, it finds a performative subset such that enhanced transferability is observed over prior state-of-the-art approaches, which are largely subsumed by the proposed composition. However, as mentioned in the weaknesses section, more details could be provided to clarify key aspects of the proposed approach, alongside complexity comparisons to prior approaches. I would be willing to raise my score further if these points could be addressed in detail.\n\n\nPost-Rebuttal Update:\nA note has been added titled \"Post-Rebuttal Comments and Modification to Recommendation\" in the thread below. I would like to update my score to be “3: reject, not good enough\".\n", "strength_and_weaknesses": "Strengths:\n1) The paper clearly motivates the study of augmentation to boost adversarial transferability, and covers contributions of past works in a clear, concise manner which helps place the findings uncovered in a systematic framework.\n2) The paper presents a fairly detailed set of empirical evaluations on a wide range of deep networks with different architectures, on a 430-class subset of ImageNet with 1000 images that is commonly used to assess adversarial transferability. The inclusion of both normally trained and adversarially trained models in the evaluations helps establish the results in a detailed manner.\n3) The proposed combination of augmentations is largely seen to outperform the current state of the art, which largely serves as a baseline, especially considering that DST-MI-FGSM is a subset of the final augmentations utilized.\n\n\n\nWeaknesses:\n1) With the large number of possible augmentations available, the paper primarily focuses on the parallel composition of augmentations, as opposed to the serial case as done in prior works (eg. as in DST-MI-FGSM). However, given that this is a key aspect of the paper, more details could have been provided in Section-3.3 to better understand the associated pros and cons. \n2) For example, it appears that parallel augmentations apriori require additional forward and backward passes as opposed to serial composition. Could the authors kindly clarify if the parameter of interest “m” in Algorithm1 is kept constant for the proposed approach as compared to prior works? Or is the sampling over the distribution D done independently for different augmentations in the parallel-composition setting?\n3) Furthermore, given that the prior methods such as DST-MI-FGSM are subsumed by the proposed approach, a more thorough complexity analysis needs to be included. Thus, the running time required for generating the attacks using UltimateCombo could be reported for a subset of the models considered.\n4) While the empirical results presented are indeed interesting, the paper could have possibly devoted more attention to analyzing why such compositions are more effective. For example, a slightly more detailed discussion as currently provided in Section-5.4 could have been included with respect to cosine similarity of gradients post parallel-composition, and the direction towards mitigating the effect of overfitting during the use of a single surrogate model as compared to an ensemble.\n5) While the majority of the results are presented on the ImageNet subset, additional evaluations over other standard datasets such as CIFAR-10 or CIFAR-100 could be included, given that it has served as a benchmark for adversarial defense research for the past few years (though some prior works on adversarial transferability exclude these as well). This could also potentially help weed out results that are overly-specific to the case of ImageNet based images, and help generate a better understanding towards the compositional nature of augmentations.\n\n\nMinor Typos-\n Page3: “ set of these defense.” -> ” set of these defenses.”\nPage5: “The main difference from e is that” -> “The main difference from RE is that”\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "5eobjCNl6Fv", "reviewer": "Reviewer_T7Qz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to enhance the transferability of adversarial examples by doing adversarial attacks using multiple data augmentation parallelly. ", "summary_of_the_review": "Based on the pool clarity, I am afraid that the current submission does not reach the bar of acceptance. If the clarity is improved in the rebuttal, I will consider re-evaluating this manuscript. ", "strength_and_weaknesses": "Pros:\n\n1. The paper study a novel problem of how data augmentation affects adversarial attacks' effectiveness. \n\n2. The work considers multiple model architectures to test its proposed method. \n\nCons:\n\n1. Lack of clarity\n- 1(a) Section 3.3. The author says “In parallel composition, augmentation methods are applied independently on the input, and their outputs are aggregated (i.e., taking their union)”. I cannot understand, what means by \"their outputs are aggregated (i.e., taking their union)\". Do you feed the model with the union of augmented images? Or randomly choose one augmentation to apply?\n- 1(b) Section 5.1. The section title says \"COLOR-SPACE AUGMENTATIONS OUTPERFORM THE STATE OF THE A RT\". This title does not match the argument in this paragraph. What this paragraph says is actually, composing COLOR-SPACE AUGMENTATIONS with the SOTA method (DST) is better than the SOTA (DST-MI-FGSM)\n- 1(c) Section 5.2. The author says \"The results reflected a mostly monotonic relationship between transferability and augmentations.\" But the results seem to be not displayed in this paper. \n- 1(d) Section 5.4 The author says \"As can be seen from Table 5, the methods least conducive for transferability (NeuTrans and Sharp) ...\" I do not find any evidence saying that NeuTrans and Sharp have the least transferability. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "-60ul_EBFcg", "reviewer": "Reviewer_fNhP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper studies the use of various data augmentation in boosting the attack transferability of adversarial examples. Overall, the paper is clear and the experimental results and analysis support the claim.", "summary_of_the_review": "Overall, this paper proposes a simple way to combine different data augmentation methods to improve the attack transferability of adversarial samples. Due to the lack of novelty and comparison with related works, I tend to vote for rejecting the submission.", "strength_and_weaknesses": "__Strength__\n\n__[S1]__ The paper is easy to follow.\n\n__[S2]__ The method is simple, practical and effective.\n\n__Weakness__\n\n__[W1] Lack of novelty.__ The paper studies the use of data augmentation in improving the attack transferability of adversarial examples. Apparently, all the data augmentation schemes are based on established works. There are no new theories or methods about data augmentation and attack transferability are introduced.\n\n__[W2] Lack of comparison with related works.__ The paper should discuss and compare the method with existing works, e.g. [Huang et al., 2019], [Wang et al., 2021], [Wu et al., 2021].\n\n__[W2] Insufficient experiments.__ The study is only conducted on the ImageNet-compatible dataset using MI-FGSM on Inception- and ResNet-based models. It is unknown if the findings will be the same on other datasets (e.g. CIFAR-100), attack methods (e.g. I-FGSM) and architectures (e.g. VGG).\n\n__Ref.__\n\n[Huang et al., 2019]: Enhancing Adversarial Example Transferability with an Intermediate Level Attack.\n\n[Wang et al., 2021]: Enhancing the Transferability of Adversarial Attacks through Variance Tuning.\n\n[Wu et al., 2021]: Improving the Transferability of Adversarial Samples with Adversarial Transformations.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject