Black-Box Adversarial Attack with Transferable Model-based Embedding

1 INTRODUCTION . The wide adoption of neural network models in modern applications has caused major security concerns , as such models are known to be vulnerable to adversarial examples that can fool neural networks to make wrong predictions ( Szegedy et al. , 2014 ) . Methods to attack neural networks can be divided into two categories based on whether the parameters of the neural network are assumed to be known to the attacker : white-box attack and black-box attack . There are several approaches to find adversarial examples for black-box neural networks . The transfer-based attack methods first pretrain a source model and then generate adversarial examples using a standard white-box attack method on the source model to attack an unknown target network ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ; Carlini & Wagner , 2017 ; Papernot et al. , 2016a ) . The score-based attack requires a loss-oracle , which enables the attacker to query the target network at multiple points to approximate its gradient . The attacker can then apply the white-box attack techniques with the approximated gradient ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; Tu et al. , 2018 ) . A major problem of the transfer-based attack is that it can not achieve very high success rate . And transfer-based attack is weak in targeted attack . On the contrary , the success rate of score-based attack has only small gap to the white-box attack but it requires many queries . Thus , it is natural to combine the two black-box attack approaches , so that we can take advantage of a pretrained white-box source neural network to perform more efficient search to attack an unknown target black-box model . In fact , in the recent NeurIPS 2018 Adversarial Vision Challenge ( Brendel et al. , 2018 ) , many teams transferred adversarial examples from a source network as the starting point to carry out black-box boundary attack ( Brendel et al. , 2017 ) . NAttack also used a regression network as initialization in the score-based attack ( Li et al. , 2019a ) . The transferred adversarial example could be a good starting point that lies close to the decision boundary for the target network and accelerate further optimization . P-RGF ( Cheng et al. , 2019 ) used the gradient information from the source model to accelerate searching process . However , gradient information is localized and sometimes it is misleading . In this paper , we push the idea of using a pretrained white-box source network to guide black-box attack significantly further , by proposing a method called TRansferable EMbedding based Black-box Attack ( TREMBA ) . TREMBA contains two stages : ( 1 ) train an encoder-decoder that can effectively generate adversarial perturbations for the source network with a low-dimensional embedding space ; ( 2 ) apply NES ( Natural Evolution Strategy ) of ( Wierstra et al. , 2014 ) to the low-dimensional embedding space of the pretrained generator to search adversarial examples for the target network . TREMBA uses global information of the source model , capturing high level semantic adversarial features that are insensitive to different models . Unlike noise-like perturbations , such perturbations would have much higher transferablity across different models . Therefore we could gain query efficiency by performing queries in the embedding space . We note that there have been a number of earlier works on using generators to produce adversarial perturbations in the white-box setting ( Baluja & Fischer , 2018 ; Xiao et al. , 2018 ; Wang & Yu , 2019 ) . While black-box attacks were also considered there , they focused on training generators with dynamic distillation . These early approaches required many queries to fine-tune the classifier for different target networks , which may not be practical for real applications . While our approach also relies on a generator , we train it as an encoder-decoder that produces a low-dimensional embedding space . By applying a standard black-box attack method such as NES on the embedding space , adversarial perturbations can be found efficiently for a target model . It is worth noting that the embedding approach has also been used in AutoZOOM ( Tu et al. , 2018 ) . However , it only trained the autoencoder to reconstruct the input , and it did not take advantage of the information of a pretrained network . Although it also produces structural perturbations , these perturbations are usually not suitable for attacking regular networks and sometimes its performance is even worse than directly applying NES to the images ( Cheng et al. , 2019 ; Guo et al. , 2019 ) . TREMBA , on the other hand , tries to learn an embedding space that can efficiently generate adversarial perturbations for a pretrained source network . Compared to AutoZOOM , our new method produces adversarial perturbation with high level semantic features that could hugely affect arbitrary target networks , resulting in significantly lower number of queries . We summarize our contributions as follows : 1 . We propose TREMBA , an attack method that explores a novel way to utilize the information of a pretrained source network to improve the query efficiency of black-box attack on a target network . 2 . We show that TREMBA can produce adversarial perturbations with high level semantic patterns , which are effective across different networks , resulting in much lower queries on MNIST and ImageNet especially for the targeted attack that has low transferablity . 3 . We demonstrate that TREMBA can be applied to SOTA defended models ( Madry et al. , 2018 ; Xie et al. , 2018 ) . Compared with other black-box attacks , TREMBA increases success rate by approximately 10 % while reduces the number of queries by more than 50 % . 2 RELATED WORKS . There have been a vast literature on adversarial examples . We will cover the most relevant topics including white-box attack , black-box attack and defense methods . White-Box Attack White-box attack requires the full knowledge of the target model . It was first discovered by ( Szegedy et al. , 2014 ) that adversarial examples could be found by solving an optimization problem with L-BFGS ( Nocedal , 1980 ) . Later on , other methods were proposed to find adversarial examples with improved success rate and efficiency ( Goodfellow et al. , 2015 ; Kurakin et al. , 2016 ; Papernot et al. , 2016b ; Moosavi-Dezfooli et al. , 2016 ) . More recently , it was shown that generators can also construct adversarial noises with high success rate ( Xiao et al. , 2018 ; Baluja & Fischer , 2018 ) . Black-Box Attack Black-box attack can be divided into three categories : transfer-based , score-based and decision-based . It is well known that adversaries have high transferablity across different networks ( Papernot et al. , 2016a ) . Transfer-based methods generate adversarial noises on a source model and then transfer it to an unknown target network . It is known that targeted attack is harder than untargeted attack for transfer-based methods , and using an ensemble of source models can improve the success rate ( Liu et al. , 2016 ) . Score-based attack assumes that the attacker can query the output scores of the target network . The attacker usually uses sampling methods to approximate the true gradient ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; Li et al. , 2019a ; Chen et al. , 2018 ) . AutoZOOM tried to improve the query efficiency by reducing the sampling space with a bilinear transformation or an autoencoder ( Tu et al. , 2018 ) . ( Ilyas et al. , 2018b ) incorporated data and time prior to accelerate attacking . In contrast to the gradient based method , ( Moon et al. , 2019 ) used combinatorial optimization to achieve good efficiency . In decision-based attack , the attacker only knows the output label of the classifier . Boundary attack and its variants are very powerful in this setting ( Brendel et al. , 2017 ; Dong et al. , 2019 ) . In NeutIPS 2018 Adversarial Vision Challenge ( Brendel et al. , 2018 ) , some teams combined transfer-based attack and decision-based attack in their attacking methods ( Brunner et al. , 2018 ) . And in a similar spirit , NAttack also used a regression network as initialization in score-based attack ( Li et al. , 2019a ) . Gradient information from the surrogate model could also be used to accelerate the scored-based attack ( Cheng et al. , 2019 ) . Defense Methods Several methods have been proposed to overcome the vulnerability of neural networks . Gradient masking based methods add non-differential operations in the model , interrupting the backward pass of gradients . However , they are vulnerable to adversarial attacks with the approximated gradient ( Athalye et al. , 2018 ; Li et al. , 2019a ) . Adversarial training is the SOTA method that can be used to improve the robustness of neural networks . Adversarial training is a minimax game . The outside minimizer performs regular training of the neural network , and the inner maximizer finds a perturbation of the input to attack the network . The inner maximization process can be approximated with FGSM ( Goodfellow et al. , 2015 ) , PGD ( Madry et al. , 2018 ) , adversarial generator ( Wang & Yu , 2019 ) etc . Moreover , feature denoising can improve the robustness of neural networks on ImageNet ( Xie et al. , 2018 ) . 3 BLACK-BOX ADVERSARIAL ATTACK WITH GENERATOR . Consider a DNN classifier F ( x ) . Let x ∈ [ 0 , 1 ] dim ( x ) be an input , and let F ( x ) be the output vector obtained before the softmax layer . We denote F ( x ) i as the i-th component for the output vector and y as the label for the input . For un-targeted attack , our goal is to find a small perturbation δ such that the classifier predicts the wrong label , i.e . argmax F ( x+ δ ) 6= y . And for targeted attack , we want the classifier to predicts the target label t , i.e . argmax F ( x+ δ ) = t. The perturbation δ is usually bounded by ` p norm : ‖δ‖p ≤ ε , with a small ε > 0 . Adversarial perturbations often have high transferablity across different DNNs . Given a white-box source DNN Fs with known architecture and parameters , we can transfer its white-box adversarial perturbation δs to a black-box target DNN Ft with reasonably good success rate . It is known that even if x+ δs fails to be an adversarial example , δs can still act as a good starting point for searching adversarial examples using a score-based attack method . This paper shows that the information of Fs can be further utilized to train a generator , and performing search on its embedding space leads to more efficient black-box attacks of an unknown target network Ft . 3.1 GENERATING ADVERSARIAL PERTURBATIONS WITH GENERATOR . Adversarial perturbations can be generated by a generator network G. We explicitly divide the generator into two parts : an encoder E and a decoder D. The encoder takes the origin input x and output a latent vector z = E ( x ) , where dim ( z ) dim ( x ) . The decoder takes z as the input and outputs an adversarial perturbation δ = ε tanh ( D ( z ) ) with dim ( δ ) = dim ( x ) . In our new method , we will train the generator G so that δ = ε tanh ( G ( x ) ) can fool the source network Fs . Suppose we have a training set { ( x1 , y1 ) , . . . , ( xn , yn ) } , where xi denotes the input and yi denotes its label . For un-targeted attack , we train the desired generator by minimizing the hinge loss used in the C & W attack ( Carlini & Wagner , 2017 ) : Luntarget ( xi , yi ) = max ( Fs ( ε tanh ( G ( xi ) ) + xi ) yi −max j 6=yi Fs ( ε tanh ( G ( xi ) ) + xi ) j , −κ ) , ( 1 ) And for targeted , we use Ltarget ( xi , t ) = max ( max j 6=t Fs ( ε tanh ( G ( xi ) ) + xi ) j − Fs ( ε tanh ( G ( xi ) ) + xi ) t , −κ ) , ( 2 ) where t denotes the targeted class and κ is the margin parameter that can be used to adjust transferability of the generator . A higher value of κ leads to higher transferability to other models ( Carlini & Wagner , 2017 ) . We focus on ` ∞ norm in this work . By adding point-wise tanh function to an unnormalized output D ( z ) , and scaling it with ε , δ = ε tanh ( D ( z ) ) is already bounded as ‖δ‖∞ < ε . Therefore we employ this transformation , so that we do not need to impose the infinity norm constraint explicitly . While hinge loss is employed in this paper , we believe other loss functions such the cross entropy loss will also work .

Summary: Review: The paper proposes a new framework (TREMBA) for black-box adversarial attack. The method utilizes a pretrained source network to learn a low dimensional embedding, it then searches efficiently within the embedding space (using NES) and produces an adversarial perturbation that can attack an unknown target network. A generator model first encodes an input to a latent vector and then decodes it to give an adversarial perturbation as an output. This generator is trained so that it can fool the source network and is then used to find the adversarial pattern when searching in the latent space. TREMBA produces perturbations with high level semantic patterns, and is easily transferable to different target architectures. The paper demonstrates its performance in terms of number of queries vs success rate on different datasets, Google cloud vision API and adversarially defended networks.