Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved remarkable success in the image classification task in recent years . Nonetheless , advances in the field of adversarial machine learning ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) make DNNs no longer reliable . By adding a well-designed perturbation on a benign image ( a.k.a adversarial attack ) , the resulting adversarial examples can easily fool state-of-the-art DNNs . To make the matter worse , the adversarial attack technique can even be applied in the physical world ( Sharif et al. , 2016 ; Kurakin et al. , 2017a ; Xu et al. , 2020 ; Duan et al. , 2021 ) , which inevitably raises concerns about the stability of deployed models . Therefore , exposing as many “ blind spots ” of DNNs as possible is a top priority . Generally , deployed models are mainly challenged with two threat models : white-box and blackbox . For white-box threat model ( Kurakin et al. , 2017b ; Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017 ; Shi et al. , 2019 ) , the attacker can obtain complete knowledge of the target model , such as the gradient for any input . However , deployed models are usually opaque to unauthorized users . In this scenario , prior black-box works ( Poursaeed et al. , 2018 ; Dong et al. , 2018 ; Xie et al. , 2019 ; Inkawhich et al. , 2020 ; Gao et al. , 2020b ; Wang et al. , 2021 ) mostly assume that the source data for training the target model is available and mainly explore the cross-model transferability among models trained in the same data distribution . Specifically , perturbations are crafted via accessible white-box model ( a.k.a substitute model ) , and resulting adversarial examples sometimes can fool other black-box models as well . Yet , these works still ignore a pivotal issue : A model owner is ∗Work done when Qilong Zhang interns at Alibaba Group , China †Corresponding author unlikely to leak the relevant information of the deployed model . To overcome this limitation , querybased black-box attacks ( Papernot et al. , 2016 ; Brendel et al. , 2018 ; Chen et al. , 2020 ; Li et al. , 2021 ) are proposed , which adjust adversarial examples just according to the output of the target model . However , the resource-intensive query budget is extremely costly and inevitably alerts the model owner . Therefore , we need a more “ practical ” black-box threat model to address this concern , i.e. , without any clue about the training data distribution as well as the pre-trained model based on it , and even querying is forbidden . Intuitively , this threat model is more challenging to build for attackers and more threatening to model owners . To the best of our knowledge , a recent work called CDA ( Naseer et al. , 2019 ) is the first to attempt such an attack . Specifically , it learns a transferable adversarial function via a generator network against a different domain ( training data and pre-trained model are all from ChestX-ray ( Wang et al. , 2017 ) domain ) . During inference , it directly crafts adversarial examples for benign ImageNet images to fool target ImageNet pre-trained models . However , its cross-domain transfer strength is still moderate . Besides , relying on small-scale datasets to train a generator may limit the generalization of the threat model . Considering that ImageNet is a large-scale dataset containing most of common categories in real life and there are various off-the-shelf pre-trained models , one can easily dig out much useful information to build a strong threat model . Therefore , in this paper , solely relying on the knowledge of the ImageNet domain , we introduce an effective Beyond ImageNet Attack ( BIA ) framework to enhance the cross-domain transferability of adversarial examples . To reflect the applicability of our approach , we consider eight different image classification tasks ( listed in Table 1 ) . Figure 1 illustrates an overview of our method . Particularly , we learn a flexible generator network Gθ against ImageNet domain . Instead of optimizing the domain-specific loss function like CDA , our method focuses on disrupting low-level features following previous literature to ensure the good transferability of our BIA . Furthermore , we propose two variants based on the vanilla BIA to narrow the gap between source and target domains . Specifically , from the data perspective , we propose a random normalization ( RN ) module to simulate different data distributions ; from the model perspective , we propose a domain-agnostic attention ( DA ) module to capture essential features for perturbing . In the inference phase , our Gθ accepts images of any domain as the input and crafts adversarial examples with one forward propagation . Extensive experiments demonstrate the effectiveness of our proposed methods . Towards the coarse-grained and fine-grained domains , we can outperform state-of-the-art approaches by up to 7.71 % and 25.91 % on average , respectively . Besides , our methods can also enhance the cross-model transferability in the source domain . 2 RELATED WORKS . Iterative Optimization Approaches . Under the black-box threat model , iterative attack methods are a popular branch , which usually adopt domain-specific loss or intermediate feature loss to craft adversarial examples . For the former , Madry et al . ( 2018 ) extend Goodfellow et al . ( 2015 ) to perform projected gradient descent from randomly chosen starting points inside -ball . Dong et al . ( 2018 ) introduce momentum term to stable the update direction . Xie et al . ( 2019 ) apply random transformations of the input at each iteration , thus mitigating overfitting . Gao et al . ( 2020a ) propose patch-wise perturbation to better cover the discriminative region . Wu et al . ( 2020a ) explore the security weakness of skip connections ( He et al. , 2016 ; Huang et al. , 2017 ) to boost attacks . Different from the methods mentioned above , intermediate feature-based methods focus on disrupting low-level features . For example , Zhou et al . ( 2018 ) maximize the Euclidean distance between the source image and target image in feature space and introduce regularization on perturbations to reduce variations . Inkawhich et al . ( 2019 ) make the source image close to the target image in feature space . Lu et al . ( 2020 ) propose a dispersion reduction attack to make the low-level features featureless . Naseer et al . ( 2020 ) design a self-supervised perturbation mechanism for enabling a transferable defense approach . Wu et al . ( 2020b ) compute model attention over extracted features to regularize the search of adversarial examples . Generator-oriented Approaches . Compared with iterative optimization approaches , generatororiented attacks are more efficient ( i.e. , only need one inference ) to generate adversarial examples . In this branch , Baluja & Fischer ( 2017 ) propose an adversarial transformation network to modify the output of the classifier given the original input . Poursaeed et al . ( 2018 ) present trainable deep neural networks for producing both image-agnostic and image-dependent perturbations . Naseer et al . ( 2019 ) leverage datasets from other domain instead of ImageNet to train generator networks against pre-trained ImageNet models , and inference is performed on ImageNet domain with the aim of fooling black-box ImageNet model . They also attempt a practical black-box threat model ( from ChestX-ray to ImageNet ) , and the attack success rate can outperform the result of Gaussian noise . 3 TRANSFERABLE ADVERSARIAL EXAMPLES BEYOND IMAGENET . 3.1 PROBLEM FORMULATION . Given a target deep learning classifier ft ( · ) trained in a specific data distribution χt , we aim to craft a human-imperceptible perturbation for the benign image xt ∼ χt from the target domain with the only available knowledge of source ImageNet domain ( including pre-trained model fs ( · ) and data distribution χs ) . Formally , suppose we have a threat modelMθ∗ whose parameter θ∗ is solely derived from the source domain , our goal is to craft adversarial examples for xt from target domain so that they can fool the ft ( · ) successfully : ft ( Mθ∗ ( xt ) ) 6= ft ( xt ) s.t . ||Mθ∗ ( xt ) − xt||∞ ≤ , ( 1 ) where is the maximum perturbation to ensure xt is minimally changed . Intuitively , crafting adversarial examples for the black-box domain is very challenging . As shown in Table 1 and Figure 6 of Appendix , images from different domain vary greatly . 3.2 PRELIMINARY . Iterative/Single-step optimization methods ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Zhao et al. , 2020 ; Gao et al. , 2021 ; Mao et al. , 2021 ; Li et al. , 2021 ) and generator-oriented methods ( Baluja & Fischer , 2017 ; Poursaeed et al. , 2018 ; Naseer et al. , 2019 ) are two popular branches for building the threat model . Since the attacker has the large-scale ImageNet training set at hand , there is no reason not to take full advantage of them . Therefore , in this paper , we adopt the generator-oriented framework which learns a transferable adversarial function via a generative model Gθ . Given that the threat model aims at crafting transferable adversarial examples for black-box domains , relying on the last layer with domain-specific loss functions ( e.g. , relativistic cross-entropy loss adopted by Naseer et al . ( 2019 ) ) is less effective since this might lead to overfitting to source domain . In contrast , the intermediate layers of the DNN presumably extract general features ( Yosinski et al. , 2014 ) which may share across different models . Hence , as a baseline for the new black-domain attack problem , our Beyond ImageNet Attack ( BIA ) turns to destroy the low-level features of the substitute model at a specific layer L to generate transferable adversarial examples according to existing literature ( Yosinski et al. , 2014 ; Zhou et al. , 2018 ; Inkawhich et al. , 2019 ) . As illustrated in Figure 1 , Gθ is learned to decrease the cosine similarity between adversarial example x′s and benign image xs ∈ RN×Hs×Ws ( sampled from χs ) to make the feature featureless : θ∗ = argmin θ Lcos ( fLs ( x′s ) , fLs ( xs ) ) . ( 2 ) In the inference phase , our generator Gθ∗ can directly craft adversarial examples for input images xt ∈ RN×Ht×Wt from the target domain : x′t = min ( xt + , max ( Gθ∗ ( xt ) , xt − ) . ( 3 ) The resulting adversarial examples x′t are depicted in Figure 8 of Appendix . Compared with CDA , our BIA is more effective in both source ( white-box ) and target ( black-box ) domains . Yet , as shown in Figure 2 , crafting more transferable adversarial examples still has some challenges : • Data Perspective : The distribution ( i.e. , mean and standard deviation ) of source domain is largely different from the target domain . For example , the standard deviation of ImageNet is about twice that of SVHN . • Model Perspective : Although some feature map of fLs ( · ) can capture the object ( ∈ χt ) for feature representation ( e.g. , the first feature map in Figure 2 ) , there are also some feature maps that are significantly biased ( e.g. , the second feature map in Figure 2 ) . To alleviate the concern of generating poor transferable adversarial examples that may arise from the above limitations , we propose two variants , equipped with random normalization ( RN ) module or domain-agnostic attention ( DA ) module , respectively .

Summary: This paper focuses on the transferability of black-box domains. In real life, we do not know the relevant information of the deployed model and transfer attacks on black-box domains can better evaluate the vulnerability of deployed models. Therefore, Beyond ImageNet Attack (BIA) is proposed to investigate the transferability towards black-box domains (unknown classification tasks) with the only knowledge of the ImageNet domain. From the perspective of data and model, the authors propose random normalization (RN) module and domain-agnostic attention (DA) module to narrow the gap between the source and target domains. Finally, BIA achieves state-of-the-art performance in black-box domains settings.