Adversarial Imitation Attack

1 INTRODUCTION . Deep neural networks are often vulnerable to imperceptible perturbations of their inputs , causing incorrect predictions ( Szegedy et al. , 2014 ) . Studies on adversarial examples developed attacks and defenses to assess and increase the robustness of models , respectively . Adversarial attacks include white-box attacks , where the attack method has full access to models , and black-box attacks , where the attacks do not need knowledge of models structures and weights . White-box attacks need training data and the gradient information of models , such as FGSM ( Fast Gradient Sign Method ) ( Goodfellow et al. , 2015 ) , BIM ( Basic Iterative Method ) ( Kurakin et al. , 2017a ) and JSMA ( Jacobian-based Saliency Map Attack ) ( Papernot et al. , 2016b ) . However , the gradient information of attacked models is hard to access , the white-box attack is not practical in real-world tasks . Literature shows adversarial examples have transferability property and they can affect different models , even the models have different architectures ( Szegedy et al. , 2014 ; Papernot et al. , 2016a ; Liu et al. , 2017 ) . Such a phenomenon is closely related to linearity and over-fitting of models ( Szegedy et al. , 2014 ; Hendrycks & Gimpel , 2017 ; Goodfellow et al. , 2015 ; Tramèr et al. , 2018 ) . Therefore , substitute attacks are proposed to attack models without the gradient information . Substitute black-box attacks utilize pre-trained models to generate adversarial examples and apply these examples to attacked models . Their attack success rates rely on the transferability of adversarial examples and are often lower than that of white-box attacks . Black-box score-based attacks ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; b ) do not need pre-trained models , they access the output probabilities of the attacked model to generate adversarial examples iteratively . Black-box decisionbased attacks ( Brendel et al. , 2017 ; Cheng et al. , 2018 ; Chen et al. , 2019 ) require less information than the score-based attacks . They utilize hard labels of the attacked model to generate adversarial examples . Adversarial attacks need knowledge of models . However , a practical attack method should require as little as possible knowledge of attacked models , which include training data and procedure , models weights and architectures , output probabilities and hard labels ( Athalye et al. , 2018 ) . The disadvantage of current substitute black-box attacks is that they need pre-trained substitute models trained by the same dataset with attacked model T ( Hendrycks & Gimpel , 2017 ; Goodfellow et al. , 2015 ; Kurakin et al. , 2017a ) or a number of images to imitate the outputs of T to produce substitute networks ( Papernot et al. , 2017 ) . Actually , the prerequisites of these attacks are hard to obtain in real-world tasks . The substitute models trained by limited images hardly generate adversarial examples with well transferability . The disadvantage of current decision-based and score-based black-box attacks is that every adversarial example is synthesized by numerous queries . Hence , developing a practical attack mechanism is necessary . In this paper , we propose an adversarial imitation training , which is a special two-player game . The game has a generative model G and a imitation model D. The G is designed to produce examples to make the predicted label of the attacked model T and D different , while the imitation model D fights for outputting the same label with T . The proposed imitation training needs much less training data than the T and does not need the labels of these data , and the data do not need to coincide with the training data . Then , the adversarial examples generated by D are utilized to fool the T like substitute attacks . We call this new attack mechanism as adversarial imitation attack . Compared with current substitute attacks , our adversarial imitation attack requires less training data . Score-based and decision-based attacks need a lot of queries to generate each adversarial attack . The similarity between the proposed method and current score-based and decision-based attacks is that adversarial imitation attack also needs to obtain a lot of queries in the training stage . The difference between these two kinds of attack is our method do not need any additional queries in the test stage like other substitute attacks . Experiments show that our proposed method achieves state-of-the-art performance compared with current substitute attacks and decision-based attack . We summarize our main contributions as follows : • The proposed new attack mechanism needs less training data of attacked models than current substitute attacks , but achieves an attack success rate close to the white-box attacks . • The proposed new attack mechanism requires the same information of attacked models with decision attacks on the training stage , but is query-independent on the testing stage . 2 RELATED WORK . Adversarial Scenes Adversarial attacks happen in two scenes , namely the white-box and the black-box settings . In the white-box settings , the attack method has complete access to attacked models , such as models internal , training strategy and data . While in the black-box settings , the attack method has little knowledge of attacked models . The black-box attack utilizes the transferability property of adversarial examples , only needs the labeled training data , but its attack success rate is often lower than that of the white-box attack method if attacked models have no defense . Actually , attack methods requiring lots of prior knowledge of attacked models are difficult to apply in practical applications ( Athalye et al. , 2018 ) . Adversarial Attacks Several methods for generating adversarial examples were proposed . Goodfellow et al . ( 2015 ) proposed a one-step attack called FGSM . On the basis of the FGSM , Kurakin et al . ( 2017a ) came up with BIM , an iterative optimization-based attack . Another iterative attack called DeepFool ( Moosavi-Dezfooli et al. , 2016 ) aims to find an adversarial example that would cross the decision boundary . Carlini & Wagner ( 2017b ) provided a stronger attack by simultaneously minimizing the perturbation and the LF norm of the perturbation . Rony et al . ( 2018 ) generated adversarial examples through decoupling the direction and the norm of the perturbation , which is also constrained by the LF norm . Liu et al . ( 2017 ) showed that targeted adversarial examples hardly have transferability , they proposed ensemble-based methods to generate adversarial examples having stronger transferability . Papernot et al . ( 2017 ) proposed a practical black-box attack which accesses the hard label to train substitute models . For score-based attacks , Chen et al . ( 2017 ) proposed the zeroth-order based attack ( ZOO ) which uses gradient estimates to attack a black-box model . Ilyas et al . ( 2018b ) improves the way to estimate gradients . Guo et al . ( 2019 ) proposed a simple black-box score-based attack on DCT space . For decision-based attacks , Brendel et al . ( 2017 ) first proposed decision-based attacks which do not rely on gradients . Cheng et al . ( 2018 ) and Chen et al . ( 2019 ) improve the query efficiency of the decision-based attack . Adversarial Defenses To increase the robustness of models , methods for defending against adversarial attacks are being proposed . Adversarial training ( Szegedy et al. , 2014 ; Madry et al. , 2018 ; Kurakin et al. , 2017b ; Tramèr et al. , 2018 ) can be considered as a kind of data augmentation . It applies adversarial examples to the training data , resulting in a robust model against adversarial attacks . Defenses based on gradient masking ( Tramèr et al. , 2018 ; Dhillon et al. , 2018 ) provide robustness against optimization-based attacks . Random transformation ( Kurakin et al. , 2017a ; Meng & Chen , 2017 ; Xie et al. , 2018 ) on inputs of models hide the gradient information , eliminate the perturbation . Buckman et al . ( 2018 ) proposed thermometer encoding based on one-hot encoding , it applied a nonlinear transformation to inputs of models , aiming to reduce the linearity of the model . However , most defenses above are still unsafe against some attacks ( Carlini & Wagner , 2017a ; He et al. , 2017 ) . Especially Athalye et al . ( 2018 ) showed that defenses based on gradient masking actually are unsafe against attacks . Instead , some researchers focus on detecting adversarial examples . Some use a neural network ( Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) to distinguish between adversarial examples and clean examples . Some achieve statistical properties ( Bhagoji et al. , 2017 ; Hendrycks & Gimpel , 2017 ; Feinman et al. , 2017 ; Ma et al. , 2018 ; Papernot & McDaniel , 2018 ) to detect adversarial examples . 3 IMITATION ATTACK . In this section , we introduce the definition of adversarial examples and then propose a new attack mechanism based on adversarial imitation training . 3.1 ADVERSARIAL EXAMPLES . X refers to the samples from the input space of the attacked model T , ytrue refers to the true labels of the samples X. T ( y|X , θ ) is the attacked model parameterized by θ . For a non-targeted attack , the objective of the adversarial attack can be formulated as : min ‖ ‖ subject to argmax yi T ( yi|X = X+ , θ ) 6= ytrue and ‖ ‖ ≤ r , ( 1 ) where the and r are perturbation of the sample and upper bound of the perturbation , respectively . To guarantee that is imperceptible , r is set to a small value in applications . X = X + are the adversarial examples which can fool the attacked model T . θ refers to the parameters of the model T . For white-box attacks , they obtain gradient information of T to generate adversarial examples , and attack the T directly . For substitute attacks , they generate adversarial examples from a substitute model T̂ , and transfer the examples to attack the T . The key point of a successful attack is the transferability of the adversarial examples . To improve the transferability of adversarial examples and avoid output query , we utilize an imitation network to imitate the characteristics of the T by accessing its output labels to improve the transferability of adversarial examples , which are generated by the imitation network . After the adversarial imitation training , adversarial examples generated by imitation network do not need additional query . In the next subsection , we introduce the proposed adversarial imitation training and imitation attack . 3.2 ADVERSARIAL IMITATION TRAINING . Inspired by the generative adversarial network ( GAN ) ( Goodfellow et al. , 2014 ) , we use the adversarial framework to copy the attacked model . We propose a two-player game based adversarial imitation training to replicate the information of attacked model T , which is shown in Figure 1 . To learn the characteristics of T , we define an imitation network D , and train it using disturbed input X̂ = G ( X ) +X and the corresponding output label yT ( X̂ ) of attacked model . X denotes training samples here . The role of G is to create new samples that yT ( X̂ ) 6= yD ( X̂ ) . Thus , D , G and T play a special two-player game . To simplify the expression but without loss of generality , we just analyze the case of binary classification . The value function of players can be presented as : max G min D VG , D = −yT ( X̂ ) logD ( X̂ ) − ( 1− yT ( X̂ ) ) log ( 1−D ( X̂ ) ) subject to X̂ = G ( X ) +X . ( 2 ) Note that the T is equivalent to the referee of this game . The two players G and D optimize their parameters based on the output yT ( X̂ ) . We suppose that adversarial perturbation ≤ r1 , and ∀ X̂ = G ( X ) +X , ‖G ( X ) ‖ ≤ r2 , r1 ≤ r2 . If the D can achieve yD ( X̂ ) = yT ( X̂ ) , our imitation attack will have the same success rate as the white-box attack without the gradient information of T . Therefore , for a well-trained imitation network , adversarial examples generated by D have strong transferability for T . A proper upper bound ( r2 ≥ r1 ) of G ( X ) is the key points for training an efficient imitation network D. Especially in targeted attacks ( T outputs the specified wrong label ) , if the characteristics of D is more similar to that of the attacked model , the transferability of adversarial examples will be stronger . In the training stage , the loss function of D is JD = VG , D . Because G is more hard to train than D , sometimes the ability of D is much stronger than G , so the loss of G fluctuates during the training stage . In order to maintain the stability of training , the loss function of G is designed as JG = e−VG , D . Therefore , the global optimal imitation network D is obtained if and only if ∀ X̂ , D ( X̂ ) = yT ( X̂ ) . At this point , JD = 0 and JG = e0 = 1 . The loss of G is always in a controllable value in training stage . As we discussed before , if r1 ≤ r2 , the adversarial examples generated by a well-trained D have strong transferability for T . Because the attack perturbation of adversarial examples is set to be a small value , we constrain the ‖G ( X ) ‖ in training stage to limit the search space of G , which can reduce the number of queries efficiently . For training methodology , we alternately train the G and D in every mini-batch , and use L2 penalty to constrain the search space of G. The procedure is shown in algorithm 1 . Algorithm 1 Mini-batch stochastic gradient descent training of imitation network . 1 : for number of training iterations do 2 : Sample mini-batch of m examples { X ( 1 ) , . . . , X ( m ) } from training set . 3 : Update the imitation model by descending its loss function : 4 : −yT ( X̂ ) logD ( X̂ ) − ( 1− yT ( X̂ ) ) log ( 1−D ( X̂ ) ) . 5 : Update the generative model by descending its loss function : 6 : e−VG , D + α‖G ( X̂ ) ‖ . 7 : end for In adversarial attacks , when the optimal D is obtained , the adversarial examples generated by D are utilized to attack T .

Summary: The authors propose to use a generative adversarial network to train a substitute that replicates (imitates) a learned model under attack. They then show that the adversarial examples for the substitute can be effectively used to attack the learned model. The idea is straightforward. The proposed approach leads to better success rates of attacking than other substitute-training approaches that require more training examples. Promising experimental results against decision and score-based attach schemes also demonstrate the effectiveness of the proposed approach.