Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains
Authors: Qilong Zhang
Xiaodan Li
YueFeng Chen
Jingkuan Song
Lianli Gao
Yuan He
Hui Xue'
Venue: ICLR 2022 Poster (2022)
Keywords: practice black-box attack
cross-domain transferability

Abstract: Adversarial examples have posed a severe threat to deep neural networks due to their transferable nature. Currently, various works have paid great efforts to enhance the cross-model transferability, which mostly assume the substitute model is trained in the same domain as the target model.
However, in reality, the relevant information of the deployed model is unlikely to leak.
Hence, it is vital to build a more practical black-box threat model to overcome this limitation and evaluate the vulnerability of deployed models.
In this paper, with only the knowledge of the ImageNet domain, we propose a Beyond ImageNet Attack (BIA) to investigate the transferability towards black-box domains (unknown classification tasks). Specifically, we leverage a generative model to learn the adversarial function for disrupting low-level features of input images. 
Based on this framework, we further propose two variants to narrow the gap between the source and target domains from the data and model perspectives, respectively. Extensive experiments on coarse-grained and fine-grained domains demonstrate the effectiveness of our proposed methods. Notably,
our methods outperform state-of-the-art approaches by up to 7.71\% (towards coarse-grained domains) and 25.91\% (towards fine-grained domains) on average. Our code is available at \url{https://github.com/Alibaba-AAIG/Beyond-ImageNet-Attack}.

Published as a conference paper at ICLR 2022
BEYOND IMAGENET ATTACK: TOWARDS CRAFTING ADVERSARIAL EXAMPLES FOR BLACK-BOX DOMAINS
Qilong Zhang', Xiaodan Li?, Yuefeng Chen", Jingkuan Song " Lianli Gao', Yuan He", and Hui Xue²
'University Electronic Science and Technology of China, China gilong-Zhang @ std. www.uc.du.ccn jingkuan song @ gmail.com, lianli. gao @uestc.edu.cn
Allibaba Group, China
(fiona.Ixd, loona.Ixd.yeefeee yuefen. whaprfenggceeeyyyyyyunnnn.aa xu.h. @alibaba-inc.com
ABSTRACT
Adversarial examples have posed severe threat deep neural networks due their transferable nature. Currently, various works have paid great efforts to en- hance the cross-model transferability, which mostly assume the substitute model is trained in the same domain as the target model. However, in reality, the rel- evant information of the deployed model unlikely to leak. Hence, it is vital to build more practical black-box threat model to overcome this limitation and evaluate the vulnerability of deployed models. In this paper, with only the knowl- edge of the ImageNet domain, we propose Beyond ImageNet Attack (BIA) investigate the transferability towards black-box domains (unknown classification tasks). Specifically, we leverage generative model to learn the adversarial fune- tion for disrupting low-level features of input images. Based on this framework, we further propose two variants narrow the gap between the source and tar- get domains from the data and model perspectives, respectively. Extensive ex- periments on coarse grained and fine-grained domains demonstrate the effective- ness of our proposed methods. Notably, our methods outperform state-of state-of-the- the- art approaches by up to 7.71% (towards coarse garsse-rraned domains) and 25.91% (towards fine-grained domains) on average. Our code is available at https: / //gittuts com A baaiabaabbaaaaaa AAIG/ Beeee//eeoondiee Imagenettttaaeet--t-anaae-e- Atttckk
INTRODUCTION
Deep neural networks (DNNs) have achieved remarkable success in the image classification task in recent years. Nonetheless, advances in the field of adversarial machine learning (Szegedy al., 2014; Goodfellow et al., 2015; Zhang et al., 2022) make DNNs no longer reliable. By adding well-designed perturbation on benign image (a.k.u adversarial attack), the resulting adversarial examples can easily fool state-of-the-art DNNs. To make the matter worse, the adversarial attack technique can even be applied in the physical world (Sharif et al., 2016; Kurakin et al., 2017a; Xu et al., 2020; Duan et al., 2021), which inevitably raises concerns about the stability of deployed models. Therefore, exposing as many "blind spots" of DNNs as possible is a top priority.
Generally, deployed models are mainly challenged with two threat models: white-box and black- box. For white-box threat model (Kurakin et al., 2017b; Moosavi-Dezfooe et al., 2016; Carlini & Wagner, 2017; Shi al., 2019; Liu et al., 2022), the attacker can obtain complete knowledge of the target model, such as the gradient for any input. However, deployed models are usually opaque unauthorized users. In this scenario, prior black-box works (Poursaced et al., 2018: Dong al., 2018: Xie et al., 2019; Inkawhich et al., 2020; Gao et al., 2020b; Wang et al., 2021) mostly assume that the source data for training the target model is available and mainly explore the cross-model transferability among models trained in the same data distribution. Specifically, perturbations are crafted via accessible white-box model (a.k.a substitute model), and resulting adversarial examples sometimes can fool other black-box models as well. Yet, these works still ignore pivotal issue: A
Work done when Qilong Zhang interns at Alibaba Group, China 1 "orresponning author

---

Published as a conference paper at ICLR 2022
Table I: A comparison of datasets from different domains.
Dataset
Resolution
Type
Test size
Classes
ImageNet (Russakovsky et al., 2015)
224x224
50,000
1,000
CIFAR-10 (Krizhevsky, 2009)
32x
coarse-graing
10,000
10
CIFAR-100 (Krizhevsky, 2009)
32x32
course-graines
10,000
100
STL-10 (Coates al., 2011)
96x96
coarse-grainge
8,000
SVHN (Netzer et al., 2011)
32x32
course-graings
26,032
10
CUB-200-2011 (Wah et al.. 201 l)
448
fine-gruined
5,740
200
Stanford Cars (Krause et ul., 2013) FGVC Aircraft (Maji et al., 2013)
448x448
fine-gruined
8,041
196
448x448
fine-grained
3,333
100
model owner is unlikely to leak the relevant information of the deployed model. To overcome this limitation, query-based black-box attacks (Papernot et al., 2016; Brendel et al., 2018; Chen et al., 2020; Li al., 2021) are proposed, which adjust adversarial examples just according to the output of the target model. However, the essource-intessive query budget is extremely costly and inevitably alerts the model owner.
Therefore, we need a more "practical" black-box threat model to address this concern, i.e., without any clue about the training data distribution as well as the pre-trained model based on it, and even querying is forbidden. Intuitively, this threat model is more challenging to build for attackers and more threatening to model owners. To the best of our knowledge, a recent work called CDA (Naseer et al., 2019) is the first to attempt such attack. Specifically, it learns transferable adversarial function via a generator network against a different domain (training data and pre-trained model are all from ChestX-ray (Wang et al., 2017) domain). During inference, it directly crafts adversarial examples for benign ImageNet images to fool target ImageNet pre-traine pre-trained models. However, its cross-domain transfer strength is still moderate. Besides, relying small-scale datasets to train generator may limit the generalization of the threat model.
Considering that ImageNet is a large-scale dataset containing most of common categories in real life and there are various off-the-shelf pre-trained models, one can easily dig out much useful in- formation build strong threat model. Therefore, in this paper, solely relying on the knowledge of the ImageNet domain, we introduce an effective Beyond ImageNet Attack (BIA) framework to enhance the cross- domain transferability of adversarial examples. To reflect the applicability of our approach, we consider eight different image classification tasks (listed in Table 1). Figure illustrates an overview our method. Particularly, we learn a flexible generator network Go against ImageNet domain. Instead of optimizing the domain-specifie loss function like CDA, our method focuses on disrupting low-level features following previous literature to ensure the good transferabil- ity of our BIA. Furthermore, we propose two variants based on the vanilla BIA narrow the gap between source and target domains. Specifically, from the data perspective, we propose a random normalization (R.N) module simulate different data distributions; from the model perspective, propose a domain-agnostic attention (DA) module to capture essential features for perturbing. In the inference phase, our Go accepts images of any domain as the input and crafts adversarial examples with one forward propagation. Extensive experiments demonstrate the effectiveness of our proposed methods. Towards the gaaseegrained and fine- -raigea domains, we can outperform state-of-thed -ttther-tt approaches by up to 7.71% % and 25.91% % on average, respectively. Besides, our methods can also enhance the cross- model transferability in the source domain.
2 RELATED WORKS
Iterative Optimization Approaches. Under the black-box threat model, iterative attack methods are a popular branch, which usually adopt domain-specific loss or intermediate feature loss to craft adversarial examples. For the former, Madry et al. (2018) extend Goodfellow et al. (2015) to per- form projected gradient descent from randomly chosen starting points inside e-ball. Dong et al. (2018) introduce momentum term to stable the update direction. Xie et al. (2019) apply random transformations of the input each iteration, thus mitigating overfitting, Gao et al. (2020a) pro- pose patch-wis perturbation to better cover the discriminative region. Wu et al. (2020a) explore the security weakness of skip connections (He et al., 2016; Huang al., 2017) to boost attacks.
2

---

Published as a conference paper at ICLR 2022
X5
Hadamard Product
Optional module
Clip
cos
x
Substitute model
Generator
RV
i
Our proposed generator framework aims to decrease the cosine similarity of feature be- tween benign image Fg and adversarial example x'g during the training phase. Training data and substitute model are all from the ImageNet domain. module applied to constrain z' in the -bal of vg. RN and DA are optional, which can further improve the transferability.
Figure 1:
Different from the methods mentioned above, intermediate feature-based methods focus on disrupt- ing low-level features. For example, Zhou et (2018) maximize the Euclidean distance between the source image and target image in feature space and introduce regularization to reduce variations. Inkawhich et al. (2019) make the source image close to the target image in feature space. Lu al. (2020) propose dispersion reduction attack to make the low-level fea- tures featureless. Naseer et al. (2020) design a self-supervised perturbation mechanism for enabling transferable defense approach. Wu al. (2020b) compute model attention over extracted features to regularize the search of adversarial examples.
perturbations
Generator-orienter Approaches. Compared with iterative optimization approaches, generator- oriented attacks are more efficient (i.e., only need one inference) to generate adversarial examples. In this branch, Baluja & Fischer (2017) propose an adversarial transformation network to modify the output of the classifier given the original input. Poursaeed et al. (2018) present trainable deep neural networks for producing both image-agnostn and image-dependent perturbations. Naseer al. (2019) leverage datasets from other domain instead of ImageNet to train generator networks against pre-trained ImageNet models, and inference performed on ImageNet domain with aim of fooling black-box ImageNet model. They also attempt practical black-box threat model (from ChestX-ray to ImageNet), and the attack success rate can outperform the result of Gaussian noise.
3 TRANSFERABLE ADVERSARIAL EXAMPLES BEYOND IMAGENET
3.1 PROBLEM FORMULATION
Given target deep learning classifier fi(-) trained in specific data distribution Xt. aim to craft a human-imperceptible perturbation for the benign image 2t ~ Xt from the target domain with the only available knowledge of source ImageNet domain (including pre-trained model fs(-) and data distribution X.). Formally, suppose we have a threat model Ma- whose parameter 6* is solely derived from the source domain, our goal is craft adversarial examples for 2, from target domain so that they can fool the fz(-) successfully:
f+ (x+)) # fe(e)) s.l. [l ) xxlloo S,
where is the maximum perturbation sarial examples for the black-box domain is very challenging. As shown in Table and Figure 6 of Appendix, images from different domain vary greatly.
a minimally changed. Intuitively, crafting adver-
3.2 PRELIMINARY
Iterative/Single-ster optimization methods (Goodfellow et al., 2015; Madry et al., 2018; Zhao et al., 2020; Gao et al., 2021; Mao et al., 2021: Li et al., 2021) and generator-orientee methods (Baluja & Fischer, 2017; Poursaeed et al., 2018; Naseer et al., 2019) are two popular branches for building the threat model. Since the attacker has the large-scale ImageNet training set at hand, there is no reason not to take full advantage of them. Therefore, in this paper, adopt the gemerator-oriented framework which learns a transferable adversarial function via a generative model Gg. Given that the threat model aims at crafting transferable adversarial examples for black-box domains, relying
3

---

Published as a conference paper at ICLR 2022
Figure 2: Left:
The data distribution (i.e, mean and standard deviation) for datasets from different domains. The result is the average over the three channels. Right: Two intermediate feature maps (Maxpool.3) of VGG- 16 (Simonyan & Zisserman, 2015) (trained in ImageNet domain) for the input image from CUB-200-208 200-2011 (Wah et al., 2011).
on the last layer with domain-specific loss functions (e.g., relativistic cross-entropy loss adopted by Naseer et al. (2019)) is less effective since this might lead to overfitting to source domain. In contrast, the intermediate layers of the DNN presumably extract general features (Yosinski et al., 2014) which may share across different models. Hence, a baseline for the new black-domain attack problem, our Beyond ImageNet Attack (BIA) turns to destroy the low-level features of the substitute model existing literature (Yosinski et al., 2014; Zhou et al., 2018; Inkawhich et al., 2019).
specific layer L generate transferable adversarial examples according to
As illustrated in Figure Go is learned decrease the cosine similarity between adversarial example x', and benign image x, E R. H, (sampled from Xx) to make the feature featureless:
0" = arg min Leoe (f! (a')). f/ (x+)).
(2)
In the inference phase, our generator Go. can directly craft adversarial examples for input images Ie E IR NxNx, from the target domain:
a min(x; + max (x1), St E).
The resulting adversarial examples at are depicted in Figure 8 of Appendix. Compared with CDA, our BIA is more effective in both source (white-box) and target (black-box) domains. Yet. as shown in Figure 2, crafting more transferable adversarial examples still has some challenges:
The distribution (i.e., mean and standard deviation) of source domain is largely different from the target domain. For example, the standard deviation of ImageNet is about twice that of SVHN.
Data Perspective:
Model Perspective: Although some feature map of f (-) can capture the object (E Xt) for feature representation (e.g. the first feature map in Figure 2), there are also some feature maps that are significantly biased (e.g., the second feature map in Figure 2).
To alleviate the concern of generating poor transferable adversarial examples that may arise from the above limitations, we propose two variants, equipped with random normalization (RA) module or domain-agnosien attention (D.A) module, respectively.
3.3 RANDOM NORMALIZATION MODULE
Generally, DNNs (Simonyan & Zisserman, 2015; He et al., 2016; Huang et al., 2017) are usually equipped with normalization for input images' so that they can be modeled as samples from the standard normal distribution. However, as illustrated in Figure 2 (left), the distribution of dataset from the different domain can vary dramatically. Thus, training against specific domain may limit the generalization of the resulting Go-. Besides, the commonly used strategy of label- preserving data augmentation (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015) is less effective because has little effect on changing the distribution of the inputs (more details are shown in Appendix A.3).
To overcome this limitation, fusing knowledge from different domains might be helpful. As shown in Table 3 of Naseer et al. (2019), using training data from other domains against the pre-trained
For Pytorch pre-trained ImageNet model, input images should be normalized using mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]
4

---

Published as a conference paper at ICLR 2022
model in the souree domain usually enhances the transferability of adversarial examples towards the target domain. However, this setup is not feasible because the available knowledge for a more practical threat model may be limited, i.e., restricted to domain. Therefore, instead propose a random normalization (RN) module to simulate different data distribution in the training phase:
RN(xx)) =
+ n.
(4)
where a and are default standard deviation and mean vectors for ImageNet, and μ' ~ My ( and o' NN Tara) are two random scales² sampled from Gaussian dis- tribution. Combined with R., and the object function can be expressed as:
0 = arg min Coss (RV( (RV((z)))))))))))))))))
(5)
3.4 DOMAIN-AGNOSTIC ATTENTION MODULE
Unlike the random normalization module, the domain-agnostic attention module aims narrow the domain gap from the model perspective. Our inspiration is from prior works (Hansen & Salamon, 1990: Caruana et al., 2004; Dong et al., 2018), which demonstrate that the ensemble strategy can avoid getting trapped in the local optimum and improve performance. Since there are many feature maps at layer L and each of them can model the input, i.e., extracts the features, we can also integrate them to produce a more robust feature representation A4, thus mitigating the impact of several biased feature maps, e-g., the second feature map of Figure 2. Specifically, we apply cross-channel average pooling to the feature maps at layer L:
EE (
AL
(6)
where C' is channel number of f (xx).
As depicted in Figure 3, even if our source model not trained in the tar- get domain, the robust feature represen- tation is still able to capture the essen- tial feature of object very well. Sur- prisingly, it is even similar to the one that derived from a completely differ- ent target model. Therefore, this ro- bust feature representation can serve a domain-agnostic attention (D.A) to en- hance the cross-domain transfcrability of adversarial examples. Specifically, in the training phase, we leverage A to assign weights for each pixel of feature maps the same layer. The resulting object function can be written
Figure 3: Left: A benign image from Stanford Cars (Krause al., 2013). Middle Right: We apply cross-channel average pooling to the intermediate fea- ture maps (Mazpool.3) VGG-16 and (Comv3_8) of DCL (Chen et al., 2019) with backbone Res-50.
0" = arggin (A C f (x') (. AL c f (x.)),
where is Hadamard product. After the training, even if the inputs are not from the source domain, our generator Gg- is supposed to own an ability to capture the essential feature to disrupt.
EXPERIMENTS
Source (white-box) Domain. Our training data is the large-scale ImageNet (Russakovsky et al., 2015) training set which includes about 1.2 million 224 x 224 x 3 images. Generators are trained against four ImageNet pre-trained models including VGG-16, VGG-19 (Simonyan & Zisserman, 2015), ResNet152 (Res-152) (He al., 2016) and 66 (Dense- 169) (Huang et al., 2017). In this domain, we also consider three other models, i.e., 12 (Dense-121) (Huang et al.,
Pricans Wetar O'eann and d'ee all hyper-paramcters.
5

---

Published as a conference paper at ICLR 2022
2017), ResNet50 (Res-50) (He et al., 2016) and Inception-v3 (Inc-v3)² (Szegedy et al., 2016), to analyze the cross-model transferability. All models are available in the Torchvision library*.
Target (black-box) Domain. Generally, the image classification tasks can be divided into coarse- grained and fine-gres seee-graied tasks in terms of label granularity (Touvron et al., 2021). Thus, in addition to ImageNet (white-box domain), consider seven other black-box domains (shown in Table 1) including four coarse -eanieed (CIFAR-10, CIFAR-100 (Krizhevsky, 2009), STL-10 (Coates et al., 2011) and SVHN (Netzer et al., 2011)) and three fine-grained (CUB-200-2011 (Wah et al., 2011), Stanford Cars (Krause et al., 2013) and FGVC Aircraft (Maji et al., 2013)) classification tasks. For the fine-grained classification, we use DCL framework (Chen et al., 2019) with three different backbones: ResNet50 (Res-50) (He et al., 2016), SENet154 154 and SE-ResNet101 (SE-Res (Hu et al., 2018). The pre-trained models for the coarse-grained classification are from Github".
Implementation Details. Our generator Go adopts the same architecture as (Naseer et al., 2019), which composite of downsampling, residual (He et al., 2016) and upsampling blocks (more details can be found in Figure 7 of Appendix). The output size of Go is equal to the input size. For CDA and our methods, we use Adam optimizer (Kingma & Ba, 2015) with a learning rate of 2e-4 and the exponential decay rate for first and second moments is set to 0.5 and 0.999, respectively. All generators are trained for one epoch with the batch size 16. For the layer L. we attack the output of Marpool.3 for VGG-16 and VGG-19, the output of Cone3 8 for Res- and the output of DenceBllokk. 2 for Dense- 169 (ablation study can be found in Appendix A. For brevity, we only refer the output of specific layer/block using its layer/block name. For our RN module, we set u' ~ (0.500, 0.08) and of ~ (0.75, 0.08), and the ablation study is shown in Appendix A.2.
Competitors. We compare our proposed methods with projected gradient descent (PGD) (Madry et al., 2018), diverse inputs method (DIM) (Dong et al., 2018; Xie et al., 2019), dispersion reduction (DR) (Lu et al., 2020), self-supervised perturbation (SSP) (Naseer et al., 2020) and cross-domain attack (CDA) (Naseer et al.. 2019). The maximum perturbation e is set to 10. Follow Lu et al. (2020), we set the step size a and the number of iterations T 100 for all iterative methods. For DIM, we set the default decay factor μ 1.0 and the transformation probability p = 0.7. The Gaussian smoothing (gs) for CDA is applied by x 3 Gaussian kernel.
Evaluation Metrics. We use the t accuracy on the whole test set (test size is shown in Table 1) after attacking to evaluate the performance of different methods. We also report standard deviation across multiple random runs in Appendix A.7.
4.1 TRANSFERABILIT COMPARISONS
In this section, we first conduct experiments for the black-box domain in Section 4.1.1 (coarse- grain) and Section 4.1.2 (fine-grain), then report the results for the white-box (source) domain in Section 4.3. For the discussion of generator mechanism, changing source domain and ensemble- model attacks, we leave them in Appendix A.4, Appendix A.5 and Appendix A.6, respectively.
4.1.1 RESULTS ON COARSE-GRAINED DOMAIN
In this section, we craft transferable adversarial examples for coarse grariegrained classification tasks. The results are shown in Table 2, where we leverage four ImageNet pre-trained models, including VGG-16, VGG-19, 155 and Dense-169, to train generators, respectively.
From Table 2, a first glance shows that our proposed methods consistently surpass state-of-the-art approaches. For example, if the substitute model is VGG-16, the most effective CDA remains 1 accuracy of 66.41% % for CIFAR- 10 after attacking, while our vanilla BIA can effectively bring down it to 57.38% % and DA variant can further drop the top-1 accuracy to 55.16%. Among all tasks, the STL-10 and SVHN domains are the most difficult to attack, and the performance gap among existing attacks is moderate. Nonetheless, can significantly enhance the transferability with the help of our R. variant in these domains. Notably, if the substitute model is Res-152, RN
'Inc-v3 expects inputs with size of 299 299 3. Therefore, the adversarial examples are also craft on the same size inputs. Note the size of training data for our generator is still 224x224x 3. *https: //tp:::://ptto/hhc org/vil vttpvviiiiii wwwiio//sstttoooooooo ab Shttps://g http://gggtt.utttt/ thub. com ron- http://rrroocccccco//ooorrrobba
a e /modeel a html
6

---

Published as a conference paper at ICLR 2022
Table 2: Transferability comparisons four goarse-grained classification tasks. Here report the top-1 accuracy after attacking (the lower, the better). The generator Ge is trained in the ImageNet domain, and adversarial examples are within the perturbation budget of Loo S
10.
Madel
Artacks Clan
CIFAR-10
93.78
CIFAR-IDO
14:27
STI-10
77.59
SVHN
96.83
85.42
AVCI
Model
Artacks Clesn
CIFAR-IO
91.78
CIFAR-100
74.27
STL-10 27:59
SVHN
95.03
85.42
AVC
PGD
79.63
03.02
74.32
96.65
74.16
PGD
86.17
56.38
74.51
93.94
77.25
DIM
77.16
64.75
72:74
91.53
71.35
DIM
90.50
63.03
73.2
90.87
72.65
DR
72.49
30:04
72.56
95.27
69:34
DR
78.86
65.02
71 66
93.26
73.
VGC-16
SSP
68.54
33.63
72.77
95.08
67.23
SSP
75.54
42.38
72.66
92.63
70,80
CDA
56.41
32.37
72.91
92.17
65.97
CDA
66.47
39.30
69.81
88.09
65.92
$6,70
29:43
73.35
91.01
77:78
CDA
$3.61
57.27
73.06
90.34
76.57
BTA (Chm)
22.47
69:45
BIA (Ches)
69.91
64.
BIA-DA (Oumn)
21.71
70.00
91.76
59.66
BIA+DD (Chre)
32.68
69:65
91:38
BLAYFVV (Ouns)
$2.81
20.82
67.55
$8.03
57.30
BLA+RV (Ouns)
61.23
32.84
68.04
85.79
61.98
PGD
74.5
76.81
01:68
DIM
49.06
73.64
VOG--1
35.28
03.67
03.75
Donse 1055
SSP
DR
72.55
70.75
70U87
72:72
CDA
$1.00
51.53
71:43
92.64
74.30
CDA
67.75
35.03
69
88.76
65.
CDD.gg
$8.55
61.90
73.64
92.18
79.07
CDAAgg
73:61
75.26
(Chus)
(Ches)
BIA-17.4 (Ouss)
70.16
(Oues)
70.60
BLAYEVV (Ques)
BIA+RV (Ouns)
68.79
Table 3: Transfcrability comparisons on three fine-grained classification tasks. Here we report the top-1 accuracy after attacking (the lower, the better). The generator Go trained in ImageNet domain and adversarial examples are within the perturbation budget of foo S 10.
CUB-200-2011
Stanford Cars
FOVC Aircralt
Model
Attacks
Res-50
SENei154
SE-Res101
Res-50
SENer154
SE: SE-Res101
Res-50
SENet154
SE-Rest01
AVG.
86.81
93.36
90.85
87.45
DIM
70.02
62.86
70.57
74.72
78.10
84.33
73.54
66.88
62.38
71.49
DR
81.08
82.05
82.52
90.82
90.59
91.12
84.97
87.55
85.54
86.25
VGG-16
62.27
60.44
63,79
CDA
69.69
62.51
71.00
75.94
72.15
81.64
71.53
58.33
63.39
69.94
CDAggs
70.19
63.19
68.92
85.03
79.52
83.52
78.55
65.62
68.38
73.66
BIA (Ours)
32.74
52.99
58.04
39.51
69.90
70.17
28.92
60.31
46.92
51.07
BIA+DA (Ours)
25.00
40.27
53.24
22.
15.36
47.91
40.83
40.65
BIA+RA (Ours)
26.13
46.15
55.07
20.61
62.64
63.38
16.50
$2.54
45:48
43.17
PGD
80.98
79.00
80,60
87.5 54
88.87
90.56
84,70
84.01
83.35
84.40
DIM
61.
70.90
75.02
78:55
84.63
74.59
67.69
65,26
72.02
DR
80.83
81.57
81.95
91.00
90.23
91.15
84.43
85.96
$4.57
85.74
VOG-19
SSP
62.94
58.34
70.45
61.90
76.27
83.77
58.78
69.52
66.88
65.65
59.48
70.70
80.70
59.26
$2.24
62.26
CDA+gs
67.88
59.42
67.57
82.83
78.61
82.64
79.36
65.88
68.35
72.51
BIA (Ours)
48,90
52.35
56.47
66.34
72.45
75.08
50.95
54.04
51.79
58.71
BIA+D.A (Ours)
BIA+RA (Ours)
31.77
43.41
$1.09
42.81
68.90
66.27
27.75
52.48
45.57
47.78
PGD
73.21
75.89
76.11
83,99
$6.89
88.24
79.00
79.30
75.64
79.81
DR
77.58
82.07
80.60
87.96
90.11
90.67
77.89
82:27
80.08
83.25
Res-152
SSP
47.64
66.17
67.90
53.29
79.09
B3.45
58.35
73.36
69.34
66.51
53.69
59.11
CDA
59.32
64.24
60.08
81.15
$2.91
82.56
75.52
73.72
66.58
71.79
BIA (Oors)
43.55
49.
55.54
31.65
54,12
67.21
38.49
32.46
$1.19
47 08
BIA+DA
47.28
BIA-RA (Ours)
23.54
40.13
$1.36
12.39
47.92
60.59
32.34
32.85
46.89
38.67
PGD
79.29
81.14
79.74
87. 66
90..1
89.30
83,23
84.31
81.24
84.06
Denoe- 169
DR
74.84
78.89
77.93
88.82
89.52
77.50
78.773
74.47
80.84
SSP
41,30
49.00
59.72
26.65
68.21
74.36
16.80
55.78
44.07
48 65
$2.92
60.96
$7.04
$3.64
73.66
62.23
99.83
CDA+gs
60.36
61.34
60.10
74.95
76.35
78.86
72.94
68.68
64.66
68.75
BIA (Ours)
21.79
29.29
39.13
9.58
44.46
49.06
8.04
27.84
33.87
29.23
HIA+D. (Ours)
12.36
29:31
35.78
4.56
40.85
3.90
13.59
14.55
20.75
BIA+RA (Ours)
10.67
32.50
46.62
7.76
44.14
42.76
3.81
34.20
27.00
27.72
variant can further decrease the top-1 accuracy from 89.46% (vanilla BIA) to 85.79% on SVHN.
Compared with state-of-the-art CDA all domains. RA variant significantly outperforms it by 7.71% % on average. This demonstrates that our proposed RN module is effective coping with different distributions of inputs, thus improving the generalization of the resulting generator.

---

Published as a conference paper at ICLR 2022
4.1.2 RESULTS ON FINE-GRAINED DOMAIN
We also analyze the transferability of adversarial examples towards fine-grained classification tasks. For each domain, three black-box models with different backbones trained via the DCL framework are the target. The results are summarized in Table 3, where the leftmost column is the substitute model and the top row shows the target model.
In this scenario, the performance gap between the existing state-of-the-at algorithms and our pro- posed methods is further enlarged. Remarkably, when attacking against Dense-169 169, even the vanilla BIA can drop the average top-1 accuracy to 29.23%, while CDA+gs, CDA, SSP, DR, DIM and PGD are still with the high average top-1 accuracy of 68.75%, 61.91%, %. 48.65%, 80.84%, 68.06% and 84.06% after attacking, respectively. Furthermore, by adding DA or R.N' modules the train- ing phase, the generator Gø- is capable of crafting more transferable adversarial examples. On average, our R.N variant can reduce the top-1 accuracy from 46.52% (vanilla BIA) to 39.33%, % and DAA variant can further drop it to 36.42%. which remarkably outperforms SSP by 25.91 %. This demonstrates our proposed DAA module can effectively alleviate the bias caused by several feature maps, thus focusing disrupting essential features.
Table 4: Transferability comparisons ImageNet (source domain). Here report the top-1 curacy after attacking (the lower, the better). The generator Go trained in ImageNet domain (*** denotes white-box model) and adversarial examples are within the perturbation budget of &o S 10.
Model
Atlack
VGG-16
Dems-169
VGG-19
Res-50
Res-152
Dense-121 121
Inc-v3
AVG.
Clean
70.14
75.75
70.95
74.61
77.34
74.22
76. 19
74,17
PGD
2.49
28
61. 08.08
DIM
3.324
3.39
19.
18.59
DR
20.95*
67.91
43.59
64.90
70.00
64.73
69 22
$7.33
SSP
0.95
39.56
3.42
26 22
41.68
34.45
47.46
27.68
VGG-16
CDA
0
0.77
$1.05
CDA+gs
12.10
57.09
20-48
$1.87
60.83
52.21
55. 53.12
44.24
BIA (Ours)
1.55
32.35
3.61
25.36
42.98
26.97
41,
24.86
B (Ours)
1,04
24.52
2,017
18.63
36.43
19:97
34,
19.60
RIA+RW
28.54
17.87
PGDD
38:51
40.53
33.91
DIM
12:31
5.25*
13.20
8.98
12.93
5.91
21
11.43
DR
38.45
23.99*
41.59
50.19
58.70
49.95
63. 70
46.65
SSP
12:54
255
Dense-169
CDA
7.36
5.13
CDAggs
27.98
22.95
28.56
31.52
43.67
31.66
49.18
33.65
BLA (Ours)
4.76
6.45*
7.15
6.97
13.83
6.60
38.58
12.05
BIA+D (Ours)
3,17
3,32
4.09
4.44
5.85
3.98
26.51
7.34
BIA+RV (Ours)
3.66
4.05*
5.23
6.91
7.36
4.1.3 RESULTS ON SOURCE DOMAIN
Although proposed methods are mainly designed to improve the threat of adversarial examples towards black-box box domains, they are also effective for enhancing the cross-model black-box trans- ferability in the white-box domain. For example, by training against Dense-169, CDA still remains a top-1 accuracy of 43.78% on Inc-v3, while our BIA can achieve a relatively low top-1 accuracy of 38.58% on it. Besides, R.V variant can further decrease the top-1 accuracy box model) and Inc-v3 (black-box model) by 2.4% and 24.34%. %, respectively. This demonstrates that our proposed R.V module is also able avoid getting stuck in the local optimum of a specific model when training on a large-scale dataset. For DAA variant, since the target domain and source domain are identical, it is naturally able improve the transferability as well. As shown in Table 4, compared with vanilla BIA, D.A variant can further degrade the top-1 accuracy from 18.45% (vanilla BIA) to 13.47% on average.
Dense-169 (white-
4.2 COMBINATION OF DOMAIN- ATTENTION AND RANDOM NORMALIZATION
In this section, we report the results for BIA equipped with both RN and DA. The following is the update rule:
arg min Loos (At @ (RN((()) A o (.
(8)
As illustrated in Figure 4, RN module and D.A module are not always mutually reinforcing, For example, when training against Res-152 and transferring adversarial examples fine-grained or source domains, combining RN module and D.A module can further decrease the average top-1 accuracy to 35.75% and 13. 1..11 %, respectively. However, if the black-box domain coarse-grain,
8

---

Published as a conference paper at ICLR 2022
only applying R.V module is better than
applying both R.V and DA modules. We speculate that it may be because the
R. module affects the low-level fea-
tures extract by the substitute model,
and thus be incompatible with the DA module sometimes.
VOS CHERSOTT
V30-00 CM00-99t
Nex350 CMPT
0m06-16 CM
199 1000000 20
Since Figure 4 shows that using DA and RV in tandem less effective when training against Dense-169 while they reinforce each other in VGG-16 most cases, we visualize the cross-channel average pooling of intermediate features for V and Dense-166 169 to better ex- plain this phenomenon. As illustrated in Figure 5, it can be observed that the RN module reinforces the discrimina- tive features in VGG-16. However, it in- hibits the response of objects' essential features extracted by Dense- 166. Consequently, DA module may cause the resulting generator reduce the ability to attack essential features, thereby making challenging to use these two techniques in tandem.
V00 with
101
Figure 4: The average top-1 accuracy of DA, RN and
DA 'N variants after attacking on coarse-graiie goarse-grained, fine-grained and source domains.
Row
VGG-
VOG-
festurn
Figure 5: Visualization (cross-channel average pooling) of intermediate features for VGG- 16 and Dense-169. It can be observed that the RN module inhibits the response of objects' essential feature extracted by Dense-169, while VGG-16 16 enhances the response.
CONCLUSION
In this paper, present a practical black-box threat model for the cross-domain attack. Specifically, we train a generator network in the large-scale ImageNet domain to disrupt low-level features better, thus generating transferable adversarial examples for the black-box domain. Based this frame- work, we further propose two variants to narrow the gap between the source and target domains from the data and model perspectives, respectively. Extensive experiments demonstrate the effectiveness of our proposed methods. This also reminds the model owner that "Your deployed model is not safe even you do not leak any information the public". We hope our proposed approaches can serve as a benchmark for evaluating the stability of various deployed models.
6 ACKNOWLEDGE
This work was supported by the National Natural Science Foundation of China (Grant No. 62020106008, No. 61772116 and No. 61872064) and Alibaba Group.

---

Published as a conference paper at ICLR 2022
REFERENCES
Shumeet Baluja and lan Fischer. Adversarial transformation networks: Learning to generate adver- sarial examples. CoRR, abs/1703.09387, 2017.
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In ICI.R, 2018.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. In Symposium on Security and Privacy, 2017.
Rich Caruana, Alexandru Niculescu-Mizii, Geoff Crew, and Alex Ksikes, Ensemble selection from libraries of models. In ICML, 2004.
Jianbo Chen, Michael Jordan, and Martin Wainwright. Hopskipjumpattack: query-efficient decision-based attack. In SP, 2020.
Yue Chen, Yalong Bai, Wei Zhang, and Tao Mei. Destruction and construction learning for fine- grained image recognition. In CVPR, 2019.
Adam Coates, Andrew Y. Ng, and Honglak Lee. An analysis of single-layer networks in unsu- pervised feature learning. Geoffrey J. Gordon, David B. Dunson, and Miroslav Dudík (eds.), AISTATS, 2011.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. CVPR, 2018.
Ranjie Duan, Xiaofeng Mao, A. Kai Qin, Yuefeng Chen, Shaokai Ye, Yuan He, and Yun Yang. Adversarial laser beam: Effective physical-world attack dans blink. In CVPR, 2021.
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Hengtao Shen. Patch-wise attack for fooling deep neural network. In ECCV, 2020a.
Lianli Gao, Qilong Zhang, Jingkuan Song, and Heng Tao Shen. Patch-wise++ perturbation for adversarial targeted attacks. CoRR, abs/2012.15503, 2020b.
Lianli Gao, Yaya Cheng, Qilong Zhang. Xing Xu, and Jingkuan Song. Feature space targeted attacks by statistic alignment. In IJCAI, 2021.
lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Yoshua Bengio and Yann LeCun (eds.), ICLR, 2015.
Lars Kai Hansen and Peter Salamon. Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell., 1990.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In CVPR, 2016.
Jie Hu, Li Shen, and Gang Sun. Squeeze- cuuueze-and-exxeee and-excitation networks. In CVPR. 2018.
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In CVPR, 2017.
Nathan Inkawhich, Wei Wen, Hai (Helen) Li, and Yiran Chen. Feature space perturbations yield more transferable adversarial examples. In CVPR, 2019.
Nathan Inkawhich, Kevin J. Liang. Binghui Wang, Matthew Inkawhich, Lawrence Carin, and Yiran Chen. Perturbing across the feature hierarchy improve standard and strict blackbox attack transferability. In Hugo Larochelle, Mare" Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), NeurlPS, 2020.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Jonathan Krause, Jia Deng, Michael Stark, and Li Fei-Fei. Collecting a large-scale dataset of fine- grained cars. 2013.
10

---

network for Published as a conference paper at ICLR 2022
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. Imagenet classification with deep convo- lutional neural networks. In NeurPIS, 2012.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. In ICLR, 2017a.
Alexey Kurakin, lan J. Goodfellow, and Samy Bengio. Adversarial machine learning scale. In ICLR, 2017b.
Xiaodan Li, Jinfeng Li, Yuefeng Chen, Shaokai Ye, Yuan He, Shuhui Wang, Hang Su, and Hui Xue. QAIR: practical query-efficient black-box attacks for image retrieval. In CVPR, 2021.
Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song. Delving into transferable adversarial exam- ples and black-box attacks, In ICLR, 2017.
Ye Liu, Yaya Cheng, Lianli Gao, Xianglong Liu, Qilong Zhang, and Jingkuan Song. Practical evaluation of adversarial robustness via adaptive auto attack. In CVPR, 2022.
Yantao Lu, Yunhan Jia, Jianyu Wang, Bai Li, Weiheng Chai, Lawrence Carin, and Senem Veli- pasalar. Enhancing cross-task black-box transferability of adversarial examples with dispersion reduction. In CVPR, 2020.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. In ICLR, 2018.
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew B. Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. volume abs/1306.5151, 2013.
Xiaofeng Mao, Yuefeng Chen, Shuhui Wang, Hang Su, Yuan He, and Hui Xue. Composite adver- sarial attacks. In AAAI, 2021.
Seyed-Mohsen Moosavi-Dezzool Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. In CVPR, 2016.
Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and Venkatesh Babu. NAG: adversary generation. In CVPR, 2018.
Muzammal Naseer, Salman H. Khan, Muhammad Haris Khan, Fahad Shahbaz Khan, and Fatih Porikli. Cross-domain transferability of adversarial perturbations. In NeurPIS, 2019.
Muzammal Naseer, Salman H. Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli, A self-supervised approach for adversarial robustness. In CVPR, 2020.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Nicolas Papernot, Patrick D. McDaniel, and Ian Goodfellow. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples. CoRR, abs/ 1101160077777 2016.
Omid Poursaeed, Isay Katsman, Bicheng Gao, and Serge J. Belongie. Generative adversarial per- turbations. In CVPR, 2018.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei-Fei Li. Imagenet large scale visual recognition challenge. IJCV, 2015.
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. Accessorize to crime: Real and stealthy attacks on state-of-the-art face recognition. In SIGSAC, 2016.
Yucheng Shi, Siyu Wang, and Yahong Han, Curls & whey: Boosting black-box adversarial attacks. In CVPR, 2019.
11

---

Published as a conference paper at ICLR 2022
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Yoshua Bengio and Yann LeCun (eds.), ICLR, 2015.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, lan J. Good- fellow, and Rob Fergus. Intriguing properties of neural networks. In Yoshua Bengio and Yann LeCun (eds.), ICLR, 2014.
Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jonathon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In CVPR, 2016.
Hugo Touvron, Alexandre Sablayrolles, Matthijs Douze, Matthieu Cord, and Hervé Jégou. Grafit: Learning fine-grained image representations with coarse labels. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 874-884, 2021.
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical report, California Institute Technology, 2011.
Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He. Admix: Enhancing the transferability of adversarial attacks. In ICCV, 2021.
Xiaosong Wang. Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M. Sum- mers. Chestx-ray8: Hospital-scar chest x-ray database and benchmarks on classification and localization of common thorax diseases. In CVPR, 2017.
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma. Skip connections matter: On the transferability of adversarial examples generated with resnets. In ICLR, 2020a.
Weibin Wu, Yuxin Su, Xixian Chen, Shenglin Zhao, Irwin King, Michael R. Lyu, and Yu-Wing Tai. Boosting the transferability of adversarial samples via attention. CVPR, 2020b.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity. In CVPR, 2019.
Kaidi Xu, Gaoyuan Zhang, Sijia Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin-Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial t-shirt! evading person detectors in a physical world. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Man-Michael Frahm (eds.), ECCV, 2020.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q Weinberger (eds.), NeurIPS, 2014.
Qilong Zhang, Chaoning Zhang, Chaoqun Li, Jingkuan Song, Lianli Gao, and Heng Tao Shen. Practical no-box adversarial attacks with training-fre free hybrid image transformation. CoRR, abs/2203.04607,, 2022.
Zhengyu Zhao, Zhuoran Liu, and Martha A. Larson. On success and simplicity: A second look at transferable targeted attacks. CoRR, abs/2012. 12207, 2020.
Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang. Xiangqi Huang. Xiang Gan, and Yong Yang. Transferable adversarial perturbations. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu. and Yair Weiss (eds.), ECCV, 2018.
12

---

Published as a conference paper at ICLR 2022
A
APPENDIX
Imagehiet
GFAR-10
CIFAR-100
SVHN
STL-10
CUB-200-2011
FOVC Airoraft
Stanford Cars
5
9
Figure 6: Benign images sampled from each domain. From the top to the bottom rows are the first category, the middle category and the last category of their label space, respectively.
Downsampling
Upsampling
Conv BatchNorm
BatchNorm
BatchNorm
Drogout
Conv
Batch
Rell
Figure 7: The structure of the generator.
Imagehett
CIFAR-10
CIFAR-10C 100
SVHN
STL-10
CUB- LUB-200-2011
FGVC Aircoftt
Stanford Cars
Benign
Hedgehoo
Horse
Tulp
Monbey
Block footod albatross
Bowing 727
Aston Martin Coupe
CDA
Hedgehog
Tulp
Monbay
albatrow.
Boeing 727
Astan Martin Coupa
BIA
Guenon
Deer
Rose
Brnd
Polagic commorant
Yak-42
Ferran FR Coupe
Figure 8: Adversarial examples crafted by CDA and our vanilla BIA (e = 10). Both generator networks are trained against ImageNet pre-trained VGG-16 (Simonyan & Zisserman, 2015). Red highlighted labels represent misclassification.
13

---

Published as a conference paper at ICLR 2022
A.1 SELECT LAYER FOR ATTACKING
In this section, we analyze the impact of different intermediate layers of the substitute model on the transferability of resulting adversarial examples. The results are illustrated in Figure 10.
In general, training against the shallow and middle layers yields more cross-domain transferable but less cross-model transferable adversarial examples than against the deep layer. For example, if the substitute model is Res-152, disrupting shallow layer Conu2_3 is more effective for transferring towards coarse- -araine domain, and perturbing middle layer Conv3 8 is more effective in reducing the average top-1 accuracy of fine-grained models. In contrast, attacking deep layers like Conv5.3 3 can yield more transferable adversarial examples in the source domain. This demonstrates that low- level features are more similar across domains and high-level features are more domain- -speciii..
A.2 SELECT GAUSSIAN DISTRIBUTION FOR RANDOM NORMALIZATION
For DA module, it is parameter-free. Therefore, we only conduct the ex- periment select an optimal distri-
57.27
57.30
57.40
44.95
43.16
43.17
18.65
17.87
18.02
bution for R. module. ie., the p' and a' in Equation 4. Here we tune the mean of jí' and o' from 0.25 to 0.75 with a gramularity of 0.25, For the standard deviation of them, we fix it to 0.08 so that the sampled μ' and o' can basically take values rang- ing from 0.0 to 1.0 (according to the three-sigma rule).
5:55 deeas
57.92
0.30
44.86
44.
8
20.44
59.26
59,
47:13
45
21.
22.55
0.25
0.50
6.75
0.25
0.50
0.75
0.55
0.50
6.75
Figure
The average top-1 accuracy after attacking on coarse- granse- (left), fine-grained (middle) and source (right) domains with different j'mean and a mean for RN.
The results are shown in Figure 9, where we craft adversarial examples via VGG-16 and report the average top-1 accuracy in both source and black-box domains. From the results, observe that a bigger o' mean can effectively improve the transferability. For example, if we increase o'mean from 0.25 0.75, the top-1 accuracy on fine-grained models can be further decreased by 3.16% on average. Although the influence of I'maa is relatively moderate when d'mean = 0.75, setting Umean 0.5 usually better. Therefore, we set pz (0.50, 0.08) and o' ~ (0.75. 0.08) in our paper.
A.3 DATA AUGMENTATION VS. RANDOM NORMALIZATION
Data augmentation (Krizhevsky et al., 2012: Simonyan Zisserman, 2015) a widely used strategy transformations
for improving the generalization of the model. Nonetheless, these are less effective for training a generator craft more transferable adversarial examples.
In Figure 11, we report the results for our vanilla BIA, BIA with data augmentation (BIA+AUG) and BIA with R. RBIA+RA). posed BIA+RNN and might even degrade the performance of our vanilla BIA. For example, when training against VGG-19, BIA gets an average top-1 accuracy of 58.71 58.71% 71% on the fine-grained do- main, yet BIA+AUG degrades it to 62.29%. In contrast, our proposed BIA+RN can significantly decrease the result to 47.78% This is mainly because that the common data augmentation cannot effectively change the distribution of the training dataset, thus decreasing the generalization towards the black-box domain.
shown Figure 11, BUA less effective than our pro-
A.4 INSIGHT INTO THE GENERATOR
Although many prior works (Poursaced et al., 2018; Naseer et al., 2019) have leveraged the generator to craft adversarial examples, they hardly analyze the role of each block of the generator. This section will give an insight into the generator and understand how it processes an input. In Figure 12, we feed an image into the generator trained with the vanilla BIA (against Res-152) and visualize the
"Specifically, use three common data augmentation techniques including "RandomResizedCnpp" "Ran- domHorizoonalFiinn and "Colorlitter..
14

---

Published as a conference paper at ICLR 2022
53
40
43
20
carsegran
10
Norgtofog
Ausporino Mososcingg3
Marpcolen Raspeotin Horpscing
Denceffock DensatMace
Gencableck
VGG-16
VGG-29
Res-152
Figure 10: The average top-1 accuracy after attacking (the lower, the better) coarse maase-grained fine- grained and source domains. Our generator 9g is trained against different layers (from shallow to deep) of ImageNet pre-trained VGG-16, VGG-19, Res-152 and Dense-169, respectively.
50-
50-
Tie-4 accurary
50-
40
20
cou sorsegg
BA+AU
9/M
HA+AUG
RIM
WA+6U0
9UM
BA+AUR
VGG-16
VGG-19
Ras-152
Dense-169
Figure 11: The average top-1 accuracy after attacking on coarse-grasd muase-grained, fine-graine one-grained and source domains. Here we compare the results of vanilla BIA, BIA with data augmentation (AUG) and BIA with RNV Our generator Go is trained against ImageNet pre-trained VGG-16, VGG-19, Res-152 and Dense- 169, respectively.
output of each block. As we can observe, these blocks play different roles in crafting adversarial examples:
Downsampling block: mainly extracts discriminative features of the input image. As the size of the down-sampled images gets smaller, there is significant noise overall.
Residual block: Unlike the downsampling block, this block is responsible for adding noise and its behavior looks very similar the iterative algorithm.
Upsampling block: This block gradually reconstructs the adversarial example from abstract features.
Since the residual block mainly works suppressing or reversing the features ex- tracted by the downsampling module, the downsampling module has an essential im- pact on generating transferable adversarial examples.
Benigr: 247
variant: L-1011
variant: A340
To better understand the effectiveness of our proposed modules, here we investigate them from the perspective of the generator mechanism. Specifically, we apply cross- channel average pooling to the output of the downsampling block and calculate the difference map between vanilla BIA and proposed variants. Without loss of generality, here we only show the defi- nition of (Rf variant, BIA), and (DA variant, BIA) can be easily de- duced. Specifically, first apply cross-channel average pooling the output of the downsampling
BIA: Boeing 747
DIf variant, BIA)
(if variant, BIA)
Figure 13: A benign image (label is "Boeing 747") from FGVC Aircraft (Maji et al., 2013) and its cor- responding adversarial examples and difference map.
15

---

Published as a conference paper at ICLR 2022
Berign mage
Oownampinng
Advernsnial esampte
Upummiing
2200021
20056
Figure 12: We apply cross-channel average pooling to visualize each block our generator Gø-
block:
A EimolskNo (x))al
(9)
C
Ei=olShrA (x))d
ABIA
(10)
where GRN.. and GBZA, denote the output of the downsampling block for R.V variant and vanilla BIA, respectively. Then our difference map can be expressed by:
{1, 0,
A else.
BIA
Dif voriant, BIA)
(11)
From the result of Figure 13, we can observe that the generators derived from our proposed variants concentrate more on the body of the object (especially for DDA variant) than that of vanilla BIA (which pays more attention to the background, i.e., the black region in the difference map). This demonstrates that our proposed R.V and D variants do narrow the domain gap, and thus capable of yielding more transferable adversarial examples.
A.5 DISCUSSION ON CHANGING SOURCE DOMAIN
Since all the experiments only regarding from ImageNet domains target domains, it is un- clear how well the method will perform if the source dataset is different (cspecially when the source dataset small). Therefore, in the following Table 5, we report the results for transferability from CUB-200-201 to other domains. Generators are learned against CUB 1-B-200-2011 domain (the sub- stitute model DCL (backbone: Res-50) and training data is CUB-200-23 images)). For results of fine-graine domains, average top-1 accuracy of backbone SENet- and S0.- For result of the ImageNet domain, we average top-1 accuracy introduced in our manuscript. We can observe that our method consistently outperforms our main competitor CDA by a large margin. Besides, we also notice that transferring from CUB-200-201 to CIFAR is very challenging (compared with our reported results in Table 2). Therefore, we high- light the necessity of using a large-scale dataset such as ImageNet to train the adversarial examples generator.
testing data (5794
models
Table 5: Transferability comparisons CDA and our methods. Here we report top-1 accuracy after attacking (the lower, the better). The generator Gp is trained in CUB-200-2011 domain and adversarial examples are within the perturbation budget of &go S 10.
Atlacks
C0B-200-2011
CIFAR-10
CIFAR 100
STI-10
SVHN Stanford Cars
FGVC Aireraft
ImageNet
CDA
64.34
83.61
54.83
70.49
91.81
74.37
71.17
50.99
40.40
82.62
54.57
71.43
91.67
68.25
57.90
44.15
BIA+DAA
29,55
82.94
54.15
71.70
87.28
54.65
55.15
37.15
BIA+RN
42.88
83.84
53.63
69.79
87.95
57.01
50.85
39.43
16

---

Published as a conference paper at ICLR 2022
A.6 DISCUSSION ON ANSEMBLE-MOODE ATTACKS
As demonstrated in prior works (Liu et al., 2017; Dong et al., 2018; Mopuri al., 2018), attacking against an ensemble models can yield more transferable adversarial examples. However, is not clear whether the ensemble-moded attack is also effective in our cross-domain attack scenario. To investigate this, we conduct an experiment in Table 6, which shows the results for training against VGG-16 and an ensemble of VGG-16, 16, Vgg- 19, Res- 152 and Dense-169 respectively.
From the result, We can observe that ensemble- based training can also improve the transferability of adversarial examples towards black-box domains significantly.
Table 6: Transferability comparisons of singe-model (i.e. VGG-16) attacks and ensemble-model (i.e. an ensemble of VGG-16, VGG-19, Res-152 and Dense-169) attacks. Here report the top-1 accuracy after attacking (the lower, the better) and adversarial examples are within the perturbation budget of foo 10.
200-2000-2111 (SENet-154)
Stanford Cars (SENct-154)
FGVC Aireraft (SENet-154)
CIFAR-10
CIFAR-ION
STL-10
SVHN
BIA BIA (ensemble)
57.38
22.47
69.45
90.44
$2,99
69.90
60.31
54.96
21.73
68.94
85.85
29.15
46.47
36.87
BIA+DAA
55.16
21.71
70.00
91.76
40.27
59.48
47.91
BIA+DA A (ensemble)
52.97
20.51
69.51
89.15
18.47
38.99
25.41
BI++RN
52.81
20.82
67.55
88.03
46.15
62.64
52.54
BI++RN (ensemble)
50.99
22.35
66.06
81.66
35.40
41.87
27.81
A.7 DISCUSSION ON STANDARD DEVIATION ACROSS MULTIPLE RANDOM RUNS
To ensure the stability and credibility of the evaluations, experiments repeated several times for our methods. In Table we report the results for each random seed and standard deviation across these random runs.
Table We show the results random seeds for methods. Here we report the top-1 accuracy after attacking (the lower, the better). The generator Gg is trained against VGG-16 and adversarial examples are within the perturbation budget of &o S 10.
CIFAR-10
CIFAR-100
STL-10
SVHN
CUB 200 2011
(SENo-154)
Stanoed Cars (SENds155g 154)
PGVC Aircraft (SENcs-155 154)
ImageNet
(Rca-152)
(Table 2 & &
Puper report
$7.38
22.47
69.45
90.44
$2.99
69.90
61.31
42.95
56.75
21,86
69.61
90.48
$2.47
68:13
61.45
44.04
57.31
22.32
69.33
90.35
$2.04
69:48
$7.57
41.67
RIA
Random runs
57.03
57.13
22.58
21.82
70.06
70.05
90.01
90.43
53.04
53.27
69.54
71.91
62.00
58.54
41.02
43.09
57.36
22.50
69.55
$0.75
$1.48
68.15
$8.36
42,00
Resal:
57.36 17.36.8111 11.24
10 0.81
63.76 0.26
50.25 0.20
32.55 0.32
60.50 1.39
58.08 1.96
42.47 1.10
Clible &
Puper report
52.81
20.83
67.55
$8.01
46.
62.68
$2.54
$1.80
32.56
21.12
66.75
$8.63
44.84
63.18
$1.55
30.21
BI+
32.63
21,63
48.37
63.4
Random runs
21,42
45.41
63.2
$2.59
$2,60
21.68
67.21
$7.09
48.58
63.98
52.69
29.56
33.41
21.66
67.00
$8,00
49.01
64.16
53.96
30.96
Resuh
52,70 -: 0.42
21.46 0.23
07.10
58.20 0.63
03.58 0.40
53 1.81
$1.39
(Tuble
Pupes repert
59
BIA+DA
Resolt
0.38
0.57
(Tbblc
Prper reportt
57.26
43.56
36.11
90:95
43.03
58.4
38,07
Random
30.54
20.27
67.64
90.56
41.18
56. 46
37.00
26.07
39.31
55.
38.77
55.81
Result
50.74 0.36
20.24 0.115
61.38 L
90,85 0.331
11.13...2.19
56:90 1.33
36, 92 0.87
24.35
17

---

Published as a conference paper at ICLR 2022
A.8 EFFECTS OF RN AND DA ON COARSE-GRAINED AND FINE-GRAINED TASKS
From Table 2 and Table 3, we observe that R.V module is more effective than DA module for coarse-grained models, but well DA module for fine-grained models. There may be two reasons:
On the one hand, the default normalization of coarse-grained classification models different from ImageNet and finegrained classification models. Specifically, coarse-grained classification models use mean [0.5,0.5.0.5] and std = [0.5,0.5,0.5,, but ImageNet and and fine-grained classification models use mean = [0.485,0.456,,0 0.4660.4066 and std = [0.229,0. 0.w.0,0.222,,,,, Therefore, using R.N module can narrow normalization gap between ImageNet and coarse- -rannned domains.
On the other hand, the resolution of coarse- gaanse-rraneddddee domains such as CIFAR-10 and CIFAR-100 much lower than the ImageNet domain and fine-grained domains. Therefore, the intermediate features of images from coarse-grained domains are more coarse than those from ImageNet and fine-grained domains, which may enlarge the gap between ImageNet and coarse-grained domains.
18


Reviews:
[{"review_id": "Xdb_C0nEEf", "reviewer": "Reviewer_9vb9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. BIA focuses on disrupting low-level features to improve transferability.\n2. This work proposes random normalization (RN) module to handle the various distributions between source domains and target domains.\n3. This work proposes domain-agnostic attention (DA) module to produce a more robust feature representation.\n\nWeaknesses:\n1. RN module and DA module are not always mutually reinforcing. The reason behind this has not been analyzed.\n2. In competitors, diverse inputs method (DIM) is not new. From this perspective, why not use the more powerful MI-DI-TI-FGSM or a newer transfer attack？\n3. Table 2 and Table 3 show the transferability comparisons on classification tasks. However, the effects of DA and RN seem to depend on different models. To understand more deeply, it is necessary to analyze why different modules have different effects in different models.\n\nMinor questions:\n1. What are the experiments on the fine-grained and coarse-grained classification to prove? Why distinguish between fine-grained and coarse-grained? There is no clear explanation in this work.\n2. Does the comparison method differ greatly in training costs?\n", "summary_of_the_paper": "This paper focuses on the transferability of black-box domains. In real life, we do not know the relevant information of the deployed model and transfer attacks on black-box domains can better evaluate the vulnerability of deployed models. Therefore, Beyond ImageNet Attack (BIA) is proposed to investigate the transferability towards black-box domains (unknown classification tasks) with the only knowledge of the ImageNet domain. From the perspective of data and model, the authors propose random normalization (RN) module and domain-agnostic attention (DA) module to narrow the gap between the source and target domains. Finally, BIA achieves state-of-the-art performance in black-box domains settings.", "summary_of_the_review": "I tend to accept this paper because it focuses on more realistic black box attack settings and proposes two modules to improve performance. The design of the module is insightful and effective, but the module proposed under some models is not always effective, which limits its application and requires more adequate analysis.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Ejl5jHv80xQ", "reviewer": "Reviewer_YKsk", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Strengths:\n\n1) The problem setting (no access to target data) is of importance - in practice, access to data is as hard, if not harder, than access to model.\n2) The experiments are extensive, and clearly show a significant improvement in black-box attack capability.\n3) The code provided with the paper, along with the appendix, help gaining a clearer understanding of the method (conversely, they further emphasize readability issues of the manuscript).\n\n### Weaknesses:\n\n1) The manuscript is poorly written - grammatical mistakes and semantic mistakes are aplenty. Some phrases are the opposite of what the method actually does. Section 3.4 states \"Specifically, we apply a channel-wise average pooling to the feature maps at layer L\" where as the actually operation is cross-channel average pooling (Refer to [1]). Other mistakes are highlighted below.\n\n2) The novelty of the method is limited. In Wen Zhou et al., 2018 [2], Intermediate feature disruption is used to increasing black-box transferability. In Weibin Wu et al., 2020 [3], attention is used for increasing transferability. The paper does not mention these works.\n\n3) The claims of section 4.2 are only weakly supported. Statement: \"The downsampling module has an essential impact on the resulting adversarial examples\" I fail to see how this can be inferred from visualizing the cross-channel attention outputs.\n \n### Other weaknesses:\n\n1) Since all the experiments are only regarding ImageNet -> target datasets, it is unclear how well the method will perform if the source dataset is different (especially if the source dataset is small). \n2) The metrics do not include standard deviation across multiple random runs. Evaluating the standard deviation in at least one setting will elucidate the significance of the results.\n3) Results with combining the two proposed techniques (RN and DA) *should* be present in the main draft. This is an important question that the manuscript only deals with in the appendix. The manuscript with benefit from a discussion on the fact that using these two techniques in tandem is challenging, and fails to consistently out-perform using just a single module. \n\n### Text Errors: \n\n#### Abstract:\n1) transferability nature ==> transferable nature.\n2) the only knowledge ==> only the knowledge\n3) the coarse-grained domain ==> coarse-grained domains\n\n#### Introduction:\n\n1) possible to the spotlight ==> possible\n2) transparent ==> opaque \n3) the query ==> querying\n4) but more threatening ==> and more threatening\n5) the generator ==> a generator\n\n#### Method:\n1) they can subject to the ==> they can be modeled as samples from the standard normal distribution.\n2) even the inputs are not ==> even if the inputs are not\n\n#### Experiments:\n1) in the Torchvision. ==> in the Torchvision library.\n2) another seven ==> seven other\n\n### Questions for the authors: \n\n1) How will the generator network perform with its trained with all source models at once? (See experiments - Table 3 in Konda Reddy Mopuri et al., 2017 [4]) I suspect that it should further increase the transferability.\n\n2) Have the authors tried to increase the RGB jittering when comparing to existing methods? I suspect that with significant jittering, augmentation may perform similar to random normalization.\n\n### References: \n\n[1] Network In Network, Min Lin, Qiang Chen, Shuicheng Yan, arXiv 2013.\n\n[2] Transferable Adversarial Perturbations, Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang, Xiangqi Huang, Xiang Gan, Yong Yang, ECCV 2018.\n\n[3] Boosting the Transferability of Adversarial Samples via Attention, Weibin Wu, Yuxin Su, Xixian Chen, Shenglin Zhao, Irwin King, Michael R. Lyu, Yu-Wing Tai, CVPR 2020.\n\n[4] NAG: Network for Adversary Generation, Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, R. Venkatesh Babu, CVPR 2018", "summary_of_the_paper": "This paper tackles the challenge of generating adversarial perturbation for a target model - with no access to the model, or the model's training data (i.e. target domain). Using a trained model and data from a source domain (ImageNet), the authors train a generator to craft perturbations which maximize the cosine distance between the intermediate features of clean and adversarial images. This generator is then assisted by two techniques - random normalization of the input image, and spatial attention on Intermediate-layer features (used for cosine distance). Experiments show that this method outperforms prior methods in black-box setting (no access to target domain or model) as well as white-box setting. ", "summary_of_the_review": "The method proposed in this paper out-performs existing methods, and targets an important setting (no access to target domain or model). However, the writing is error-ridden, and the proposed method is only marginally novel w.r.t. existing works. Therefore, I rate the paper as marginally above accept threshold, conditional on the authors correcting the mistakes highlighted above.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "A35f1jfdAyT", "reviewer": "Reviewer_bvNx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "See the pros/cons below.\n\n### Pros\n1. Considering more practical threat model is certainly helpful and important for the transfer attack research.\n2. The results indeed demonstrate large improvement of BIA in terms of error rate.\n\n### Cons\n1. I'm a little concerned about the \"cross-domain\" statement made in this work. To me, the target datasets considered in this work (CIFAR, STL, CUB, Stanford Cars) are still coming from the same natural imagery \"domain\" as ImageNet, despite they have different label spaces. In particular, CUB is known to have overlap with ImageNet [1], where the \"cross-domain\" claim certainly does not hold. An example case that is more \"cross-domain\" would be to transfer from ImageNet model to a ChestX-ray model (in a similar sense to Naseer et al.).\n\n2. The specific methodology of BIA seems not new. It is known that perturbing intermediate layer features can yield more transferable adversarial examples (e.g., [2]). In fact, the formulation of BIA appears very similar to the one proposed in [2] (while BIA minimizes the cosine similarity between intermediated layer features of the clean and adversarial examples, [2] maximizes the euclidean distance, which essentially is the same). Feature space attacks are also shown to be more powerful than decision space attacks in more strict black-box transfer scenarios [3], but Sec. 3.2 fails to recognize these existing works. Clearly identifying the difference between BIA and [2] might help address this concern.\n\n3. If my above judgement of BIA not being new is correct, then my further concern comes from the result side. In Table 2, it seems that the performance gain can be largely attributed to the BIA (or essentially feature space attack) itself rather than the DA and RN module. This hurts the empirical novelty to some extent, as previous works have shown the superiority of feature space attacks in either standard or more strict transfer settings ([2,3]).\n\n\n[1] http://www.vision.caltech.edu/visipedia/CUB-200-2011.html\n\n[2] Feature Space Perturbations Yield More Transferable Adversarial Examples\n\n[3] Perturbing Across the Feature Hierarchy to Improve Standard and Strict Blackbox Attack Transferability ", "summary_of_the_paper": "This work first identifies a more practical threat model for black-box transfer adversarial attack, where the target model's domain remains unknown, and the attacker's surrogate model may be trained in another domain. Then, the BIA attack is proposed to enhance transferability, whose key idea is to distort low-level features captured by DNN's intermediate layers instead of perturbing the domain-specific features in the output layer. Two modules, DA and RN, are further proposed to improve attack success rate. Experimental results demonstrate that BIA is more effective than existing methods.", "summary_of_the_review": "This paper indeed identifies a more practical threat model, but the experiments do not closely match the proposed \"cross-domain\" scenario, and the performance gain seems to largely come from existing technique (perturbing feature space instead of decision space). These issues prevent me from recommending for acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper considers that the model's training data may be not accessible when learning the attacking model, and thus a more practical blackbox attack scheme, Beyond ImageNet Attack (BIA) framework, is designed. All the reviewers agreed that the setting in this paper is important and helpful when designing attack methods. However, the method is not totally new. Nevertheless, considering the importance of the problem investigated in this paper, the nice design of the overall framework, and the extensive experiments, the AC recommends accept for this paper.
Decision: Accept (Poster)