Yet another but more efficient black-box adversarial attack: tiling and evolution strategies

1 INTRODUCTION . Despite their success , deep learning algorithms have shown vulnerability to adversarial attacks ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) , i.e . small imperceptible perturbations of the inputs , that lead the networks to misclassify the generated adversarial examples . Since their discovery , adversarial attacks and defenses have become one of the hottest research topics in the machine learning community as serious security issues are raised in many critical fields . They also question our understanding of deep learning behaviors . Although some advances have been made to explain theoretically ( Fawzi et al. , 2016 ; Sinha et al. , 2017 ; Cohen et al. , 2019 ; Pinot et al. , 2019 ) and experimentally ( Goodfellow et al. , 2015 ; Xie et al. , 2018 ; Meng & Chen , 2017 ; Samangouei et al. , 2018 ; Araujo et al. , 2019 ) adversarial attacks , the phenomenon remains misunderstood and there is still a gap to come up with principled guarantees on the robustness of neural networks against maliciously crafted attacks . Designing new and stronger attacks helps building better defenses , hence the motivation of our work . First attacks were generated in a setting where the attacker knows all the information of the network ( architecture and parameters ) . In this white box setting , the main idea is to perturb the input in the direction of the gradient of the loss w.r.t . the input ( Goodfellow et al. , 2015 ; Kurakin et al. , 2016 ; Carlini & Wagner , 2017 ; Moosavi-Dezfooli et al. , 2016 ) . This case is unrealistic because the attacker has only limited access to the network in practice . For instance , web services that propose commercial recognition systems such as Amazon or Google are backed by pretrained neural networks . A user can query this system by sending an image to classify . For such a query , the user only has access to the inference results of the classifier which might be either the label , probabilities or logits . Such a setting is coined in the literature as the black box setting . It is more realistic but also more challenging from the attacker ’ s standpoint . As a consequence , several works proposed black box attacks by just querying the inference results of a given classifier . A natural way consists in exploiting the transferability of an adversarial attack , based on the idea that if an example fools a classifier , it is more likely that it fools another one ( Papernot et al. , 2016a ) . In this case , a white box attack is crafted on a fully known classifier . Papernot et al . ( 2017 ) exploited this property to derive practical black box attacks . Another approach within the black box setting consists in estimating the gradient of the loss by querying the classifier ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; b ) . For these attacks , the PGD attack ( Kurakin et al. , 2016 ; Madry et al. , 2018a ) algorithm is used and the gradient is replaced by its estimation . In this paper , we propose efficient black box adversarial attacks using stochastic derivative free optimization ( DFO ) methods with only access to the logits of the classifier . By efficient , we mean that our model requires a limited number of queries while outperforming the state of the art in terms of attack success rate . At the very core of our approach is a new objective function particularly designed to suit classical derivative free optimization . We also highlight a new intriguing property that deep neural networks are not robust to single shot tiled attacks . It leverages results and ideas from ` ∞-attacks . We also explore a large spectrum of evolution strategies and other derivative-free optimization methods thanks to the Nevergrad framework ( Rapin & Teytaud , 2018 ) . Outline of the paper . We present in Section 2 the related work on adversarial attacks . Section 3 presents the core of our approach . We introduce a new generic objective function and discuss two practical instantiations leading to a discrete and a continuous optimization problems . We then give more details on the best performing derivative-free optimization methods , and provide some insights on our models and optimization strategies . Section 4 is dedicated to a thorough experimental analysis , where we show we reach state of the art performances by comparing our models with the most powerful black-box approaches on both targeted and untargeted attacks . We also assess our models against the most efficient so far defense strategy based on adversarial training . We finally conclude our paper in Section 5 . 2 RELATED WORK . Adversarial attacks have a long standing history in the machine learning community . Early works appeared in the mid 2000 ’ s where the authors were concerned about Spam classification ( Biggio et al. , 2009 ) . Szegedy et al . ( 2014 ) revives this research topic by highlighting that deep convolutional networks can be easily fooled . Many adversarial attacks against deep neural networks have been proposed since then . One can distinguish two classes of attacks : white box and black box attacks . In the white box setting , the adversary is supposed to have full knowledge of the network ( architecture and parameters ) , while in the black box one , the adversary only has limited access to the network : she does not know the architecture , and can only query the network and gets labels , logits or probabilities from her queries . An attack is said to have suceeded ( we also talk about Attack Success Rate ) , if the input was originally well classified and the generated example is classified to the targeted label . The white box setting attracted more attention even if it is the more unrealistic between the two . The attacks are crafted by by back-propagating the gradient of the loss function w.r.t . the input . The problem writes as a non-convex optimization procedure that either constraints the perturbation or aims at minimizing its norm . Among the most popular ones , one can cite FGSM ( Goodfellow et al. , 2015 ) , PGD ( Kurakin et al. , 2016 ; Madry et al. , 2018a ) , Deepfool ( Moosavi-Dezfooli et al. , 2016 ) , JSMA ( Papernot et al. , 2016b ) , Carlini & Wagner attack ( Carlini & Wagner , 2017 ) and EAD ( Chen et al. , 2018 ) . The black box setting is more realistic , but also more challenging . Two strategies emerged in the literature to craft attacks within this setting : transferability from a substitute network , and gradient estimation algorithms . Transferability has been pointed out by Papernot et al . ( 2017 ) . It consists in generating a white-box adversarial example on a fully known substitute neural network , i.e . a network trained on the same classification task . This crafted adversarial example can be transferred to the targeted unknown network . Leveraging this property , Moosavi-Dezfooli et al . ( 2017 ) proposed an algorithm to craft a single adversarial attack that is the same for all examples and all networks . Despite the popularity of these methods , gradient estimation algorithms outperform transferability methods . Chen et al . ( 2017 ) proposed a variant of the powerful white-box attack introduced in ( Carlini & Wagner , 2017 ) , based on gradient estimation with finite differences . This method achieves good results in practice but requires a high number of queries to the network . To reduce the number of queries , Ilyas et al . ( 2018a ) proposed to rely rather on Natural Evolution Strategies ( NES ) . These derivative-free optimization approaches consist in estimating the parametric distribution of the min- ima of a given objective function . This amounts for most of NES algorithms to perform a natural gradient descent in the space of distributions ( Ollivier et al. , 2017 ) . In ( Al-Dujaili & O ’ Reilly , 2019 ) , the authors propose to rather estimate the sign of the gradient instead of estimating the its magnitude suing zeroth-order optimization techniques . They show further how to reduce the search space from exponential to linear . The achieved results were state of the art at the publication date . In Liu et al . ( 2019 ) , the authors introduced a zeroth-order version of the signSGD algorithm , studied its convergence properties and showed its efficiency in crafting adversarial black-box attacks . The results are promising but fail to beat the state of the art . In Tu et al . ( 2019 ) , the authors introduce the AutoZOOM framework combining gradient estimation and an auto-encoder trained offline with unlabeled data . The idea is appealing but requires training an auto-encoder with an available dataset , which an additional effort for the attacker . Besides , this may be unrealistic for several use cases . More recently , Moon et al . ( 2019 ) proposed a method based on discrete and combinatorial optimization where the perturbations are pushed towards the corners of the ` ∞ ball . This method is to the best of our knowledge the state of the art in the black box setting in terms of queries budget and success rate . We will focus in our experiments on this method and show how our approaches achieve better results . Several defense strategies have been proposed to diminish the impact of adversarial attacks on networks accuracies . A basic workaround , introduced in ( Goodfellow et al. , 2015 ) , is to augment the learning set with adversarial attacks examples . Such an approach is called adversarial training in the literature . It helps recovering some accuracy but fails to fully defend the network , and lacks theoretical guarantees , in particular principled certificates . Defenses based on randomization at inference time were also proposed ( Lecuyer et al. , 2018 ; Cohen et al. , 2019 ; Pinot et al. , 2019 ) . These methods are grounded theoretically , but the guarantees can not ensure full protection against adversarial examples . The question of defenses and attacks is still widely open since our understanding of this phenomenon is still in its infancy . We evaluate our approach against adversarial training , the most powerful defense method so far . 3 METHODS . 3.1 GENERAL FRAMEWORK . Let us consider a classification task X 7→ [ K ] where X ⊆ Rd is the input space and [ K ] = { 1 , ... , K } is the corresponding label set . Let f : Rd → RK be a classifier ( a feed forward neural network in our paper ) from an input space X returning the logits of each label in [ K ] such that the predicted label for a given input is argmaxi∈ [ K ] fi ( x ) . The aim of ||.||∞-bounded untargeted adversarial attacks is , for some input x with label y , to find a perturbation τ such that argmaxi∈ [ K ] fi ( x ) 6= y. Classically , ||.||∞-bounded untargeted adversarial attacks aims at optimizing the following objective : max τ : ||τ ||∞≤ L ( f ( x+ τ ) , y ) ( 1 ) where L is a loss function ( typically the cross entropy ) and y the true label . For targeted attacks , the attacker targets a label yt by maximizing −L ( f ( x+ τ ) , yt ) . With access to the gradients of the network , gradient descent methods have proved their efficiency ( Kurakin et al. , 2016 ; Madry et al. , 2018a ) . So far , the outline of most black box attacks was to estimate the gradient using either finite differences or natural evolution strategies . Here using evolutionary strategies heuristics , we do not want to take care of the gradient estimation problem .

Summary: This paper proposes a black box adversarial attacks to deep neural networks. The proposed approaches consist of tiling technique proposed by Ilyas et al (2018) and derivative free approaches. The proposed approaches have been applied to targeted and untargeted adversarial attacks against modern neural network architectures such as VGG16, ResNet50, and InceptionV3 trained on ImageNet and CIFAR10 datasets. Experimental results show higher attack success rate with a smaller number of queries.