Beyond ImageNet Attack: Towards Crafting Adversarial Examples for Black-box Domains

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved remarkable success in the image classification task in recent years . Nonetheless , advances in the field of adversarial machine learning ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) make DNNs no longer reliable . By adding a well-designed perturbation on a benign image ( a.k.a adversarial attack ) , the resulting adversarial examples can easily fool state-of-the-art DNNs . To make the matter worse , the adversarial attack technique can even be applied in the physical world ( Sharif et al. , 2016 ; Kurakin et al. , 2017a ; Xu et al. , 2020 ; Duan et al. , 2021 ) , which inevitably raises concerns about the stability of deployed models . Therefore , exposing as many “ blind spots ” of DNNs as possible is a top priority . Generally , deployed models are mainly challenged with two threat models : white-box and blackbox . For white-box threat model ( Kurakin et al. , 2017b ; Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017 ; Shi et al. , 2019 ) , the attacker can obtain complete knowledge of the target model , such as the gradient for any input . However , deployed models are usually opaque to unauthorized users . In this scenario , prior black-box works ( Poursaeed et al. , 2018 ; Dong et al. , 2018 ; Xie et al. , 2019 ; Inkawhich et al. , 2020 ; Gao et al. , 2020b ; Wang et al. , 2021 ) mostly assume that the source data for training the target model is available and mainly explore the cross-model transferability among models trained in the same data distribution . Specifically , perturbations are crafted via accessible white-box model ( a.k.a substitute model ) , and resulting adversarial examples sometimes can fool other black-box models as well . Yet , these works still ignore a pivotal issue : A model owner is ∗Work done when Qilong Zhang interns at Alibaba Group , China †Corresponding author unlikely to leak the relevant information of the deployed model . To overcome this limitation , querybased black-box attacks ( Papernot et al. , 2016 ; Brendel et al. , 2018 ; Chen et al. , 2020 ; Li et al. , 2021 ) are proposed , which adjust adversarial examples just according to the output of the target model . However , the resource-intensive query budget is extremely costly and inevitably alerts the model owner . Therefore , we need a more “ practical ” black-box threat model to address this concern , i.e. , without any clue about the training data distribution as well as the pre-trained model based on it , and even querying is forbidden . Intuitively , this threat model is more challenging to build for attackers and more threatening to model owners . To the best of our knowledge , a recent work called CDA ( Naseer et al. , 2019 ) is the first to attempt such an attack . Specifically , it learns a transferable adversarial function via a generator network against a different domain ( training data and pre-trained model are all from ChestX-ray ( Wang et al. , 2017 ) domain ) . During inference , it directly crafts adversarial examples for benign ImageNet images to fool target ImageNet pre-trained models . However , its cross-domain transfer strength is still moderate . Besides , relying on small-scale datasets to train a generator may limit the generalization of the threat model . Considering that ImageNet is a large-scale dataset containing most of common categories in real life and there are various off-the-shelf pre-trained models , one can easily dig out much useful information to build a strong threat model . Therefore , in this paper , solely relying on the knowledge of the ImageNet domain , we introduce an effective Beyond ImageNet Attack ( BIA ) framework to enhance the cross-domain transferability of adversarial examples . To reflect the applicability of our approach , we consider eight different image classification tasks ( listed in Table 1 ) . Figure 1 illustrates an overview of our method . Particularly , we learn a flexible generator network Gθ against ImageNet domain . Instead of optimizing the domain-specific loss function like CDA , our method focuses on disrupting low-level features following previous literature to ensure the good transferability of our BIA . Furthermore , we propose two variants based on the vanilla BIA to narrow the gap between source and target domains . Specifically , from the data perspective , we propose a random normalization ( RN ) module to simulate different data distributions ; from the model perspective , we propose a domain-agnostic attention ( DA ) module to capture essential features for perturbing . In the inference phase , our Gθ accepts images of any domain as the input and crafts adversarial examples with one forward propagation . Extensive experiments demonstrate the effectiveness of our proposed methods . Towards the coarse-grained and fine-grained domains , we can outperform state-of-the-art approaches by up to 7.71 % and 25.91 % on average , respectively . Besides , our methods can also enhance the cross-model transferability in the source domain . 2 RELATED WORKS . Iterative Optimization Approaches . Under the black-box threat model , iterative attack methods are a popular branch , which usually adopt domain-specific loss or intermediate feature loss to craft adversarial examples . For the former , Madry et al . ( 2018 ) extend Goodfellow et al . ( 2015 ) to perform projected gradient descent from randomly chosen starting points inside -ball . Dong et al . ( 2018 ) introduce momentum term to stable the update direction . Xie et al . ( 2019 ) apply random transformations of the input at each iteration , thus mitigating overfitting . Gao et al . ( 2020a ) propose patch-wise perturbation to better cover the discriminative region . Wu et al . ( 2020a ) explore the security weakness of skip connections ( He et al. , 2016 ; Huang et al. , 2017 ) to boost attacks . Different from the methods mentioned above , intermediate feature-based methods focus on disrupting low-level features . For example , Zhou et al . ( 2018 ) maximize the Euclidean distance between the source image and target image in feature space and introduce regularization on perturbations to reduce variations . Inkawhich et al . ( 2019 ) make the source image close to the target image in feature space . Lu et al . ( 2020 ) propose a dispersion reduction attack to make the low-level features featureless . Naseer et al . ( 2020 ) design a self-supervised perturbation mechanism for enabling a transferable defense approach . Wu et al . ( 2020b ) compute model attention over extracted features to regularize the search of adversarial examples . Generator-oriented Approaches . Compared with iterative optimization approaches , generatororiented attacks are more efficient ( i.e. , only need one inference ) to generate adversarial examples . In this branch , Baluja & Fischer ( 2017 ) propose an adversarial transformation network to modify the output of the classifier given the original input . Poursaeed et al . ( 2018 ) present trainable deep neural networks for producing both image-agnostic and image-dependent perturbations . Naseer et al . ( 2019 ) leverage datasets from other domain instead of ImageNet to train generator networks against pre-trained ImageNet models , and inference is performed on ImageNet domain with the aim of fooling black-box ImageNet model . They also attempt a practical black-box threat model ( from ChestX-ray to ImageNet ) , and the attack success rate can outperform the result of Gaussian noise . 3 TRANSFERABLE ADVERSARIAL EXAMPLES BEYOND IMAGENET . 3.1 PROBLEM FORMULATION . Given a target deep learning classifier ft ( · ) trained in a specific data distribution χt , we aim to craft a human-imperceptible perturbation for the benign image xt ∼ χt from the target domain with the only available knowledge of source ImageNet domain ( including pre-trained model fs ( · ) and data distribution χs ) . Formally , suppose we have a threat modelMθ∗ whose parameter θ∗ is solely derived from the source domain , our goal is to craft adversarial examples for xt from target domain so that they can fool the ft ( · ) successfully : ft ( Mθ∗ ( xt ) ) 6= ft ( xt ) s.t . ||Mθ∗ ( xt ) − xt||∞ ≤ , ( 1 ) where is the maximum perturbation to ensure xt is minimally changed . Intuitively , crafting adversarial examples for the black-box domain is very challenging . As shown in Table 1 and Figure 6 of Appendix , images from different domain vary greatly . 3.2 PRELIMINARY . Iterative/Single-step optimization methods ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Zhao et al. , 2020 ; Gao et al. , 2021 ; Mao et al. , 2021 ; Li et al. , 2021 ) and generator-oriented methods ( Baluja & Fischer , 2017 ; Poursaeed et al. , 2018 ; Naseer et al. , 2019 ) are two popular branches for building the threat model . Since the attacker has the large-scale ImageNet training set at hand , there is no reason not to take full advantage of them . Therefore , in this paper , we adopt the generator-oriented framework which learns a transferable adversarial function via a generative model Gθ . Given that the threat model aims at crafting transferable adversarial examples for black-box domains , relying on the last layer with domain-specific loss functions ( e.g. , relativistic cross-entropy loss adopted by Naseer et al . ( 2019 ) ) is less effective since this might lead to overfitting to source domain . In contrast , the intermediate layers of the DNN presumably extract general features ( Yosinski et al. , 2014 ) which may share across different models . Hence , as a baseline for the new black-domain attack problem , our Beyond ImageNet Attack ( BIA ) turns to destroy the low-level features of the substitute model at a specific layer L to generate transferable adversarial examples according to existing literature ( Yosinski et al. , 2014 ; Zhou et al. , 2018 ; Inkawhich et al. , 2019 ) . As illustrated in Figure 1 , Gθ is learned to decrease the cosine similarity between adversarial example x′s and benign image xs ∈ RN×Hs×Ws ( sampled from χs ) to make the feature featureless : θ∗ = argmin θ Lcos ( fLs ( x′s ) , fLs ( xs ) ) . ( 2 ) In the inference phase , our generator Gθ∗ can directly craft adversarial examples for input images xt ∈ RN×Ht×Wt from the target domain : x′t = min ( xt + , max ( Gθ∗ ( xt ) , xt − ) . ( 3 ) The resulting adversarial examples x′t are depicted in Figure 8 of Appendix . Compared with CDA , our BIA is more effective in both source ( white-box ) and target ( black-box ) domains . Yet , as shown in Figure 2 , crafting more transferable adversarial examples still has some challenges : • Data Perspective : The distribution ( i.e. , mean and standard deviation ) of source domain is largely different from the target domain . For example , the standard deviation of ImageNet is about twice that of SVHN . • Model Perspective : Although some feature map of fLs ( · ) can capture the object ( ∈ χt ) for feature representation ( e.g. , the first feature map in Figure 2 ) , there are also some feature maps that are significantly biased ( e.g. , the second feature map in Figure 2 ) . To alleviate the concern of generating poor transferable adversarial examples that may arise from the above limitations , we propose two variants , equipped with random normalization ( RN ) module or domain-agnostic attention ( DA ) module , respectively .

Summary: This work first identifies a more practical threat model for black-box transfer adversarial attack, where the target model's domain remains unknown, and the attacker's surrogate model may be trained in another domain. Then, the BIA attack is proposed to enhance transferability, whose key idea is to distort low-level features captured by DNN's intermediate layers instead of perturbing the domain-specific features in the output layer. Two modules, DA and RN, are further proposed to improve attack success rate. Experimental results demonstrate that BIA is more effective than existing methods.