Black-box Adversarial Attacks with Bayesian Optimization

1 INTRODUCTION . Neural networks are now well-known to be vulnerable to adversarial examples : additive perturbations that , when applied to the input , change the network ’ s output classification ( Goodfellow et al. , 2014 ) . Work investigating this lack of robustness to adversarial examples often takes the form of a back-and-forth between newly proposed adversarial attacks , methods for quickly and efficiently crafting adversarial examples , and corresponding defenses that modify the classifier at either training or test time to improve robustness . The most successful adversarial attacks use gradient-based optimization methods ( Goodfellow et al. , 2014 ; Madry et al. , 2017 ) , which require complete knowledge of the architecture and parameters of the target network ; this assumption is referred to as the white-box attack setting . Conversely , the more realistic black-box setting requires an attacker to find an adversarial perturbation without such knowledge : information about the network can be obtained only through querying the target network , i.e. , supplying an input to the network and receiving the corresponding output . In real-world scenarios , it is extremely improbable for an attacker to have unlimited bandwidth to query a target classifier . In evaluation of black box attacks , this constraint is usually formalized via the introduction of a query budget : a maximum number of queries allowed to the model per input , after which an attack is considered to be unsuccessful . Several recent papers have proposed attacks specifically to operate in this query-limited context ( Ilyas et al. , 2019 ; 2018 ; Chen et al. , 2017 ; Tu et al. , 2019 ; Moon et al. , 2019 ) ; nevertheless , these papers typically consider query budgets on the order of 10,000 or 100,000 . This leaves open questions as to whether black-box attacks can successfully attack a deep network based classifier in severely query limited settings , e.g. , with a query budget of 100-200 . In such a query limited regime , it is natural for an attacker to use the entire query budget , so we ask the pertinent question : In a constrained query limited setting , can one design query efficient yet successful black box adversarial attacks ? This work proposes a black-box attack method grounded in Bayesian optimization ( Jones et al. , 1998 ; Frazier , 2018 ) , which has recently emerged as a state of the art black-box optimization technique in settings where minimizing the number of queries is of paramount importance . Straightforward application of Bayesian optimization to the problem of finding adversarial examples is not feasible : the input dimension of even a small neural network-based image classifier is orders of magnitude larger than the standard use case for Bayesian optimization . Rather , we show that we can bridge this gap by performing Bayesian optimization in a reduced-dimension setting and upsampling to obtain our final perturbation . We explore several upsampling techniques and find that a relatively simple nearest-neighbor upsampling method allows us to sufficiently reduce the optimization prob- lem dimension such that Bayesian optimization can find adversarial perturbations with more success than existing black-box attacks in query-constrained settings . We compare the efficacy of our adversarial attack with a set of experiments attacking three of the most commonly used pretrained ImageNet ( Deng et al. , 2009 ) classifiers : ResNet50 ( He et al. , 2015 ) , Inception-v3 ( Szegedy et al. , 2015 ) , and VGG16-bn ( Simonyan & Zisserman , 2014 ) . Results from these experiments show that with very small query budgets ( under 200 queries ) , the proposed method BAYES-ATTACK achieves success rates comparable to or exceeding existing methods , and does so with far smaller average and median query counts . Further experiments are performed on the MNIST dataset to compare how various upsampling techniques affect the attack accuracy of our method . Given these results we argue that , despite being a simple approach ( indeed , largely because it is such a simple and standard approach for black-box optimization ) , Bayesian Optimization should be a standard baseline for any black-box adversarial attack task in the future , especially in the small query budget regime . 2 RELATED WORK . Within the black-box setting , adversarial attacks can be further categorized by the exact nature of the information received from a query . The most closely related work to our approach are score-based attacks , where queries to the network return the entire output layer of the network , either as logits or probabilities . Within this category , existing approaches draw from a variety of optimization fields and techniques . One popular approach in this area is to attack with zeroth-order methods via some method of derivative-free gradient estimation , as in methods proposed in Ilyas et al . ( 2019 ) , which uses time-dependent and data-dependent priors to improve the estimate , as well as Ilyas et al . ( 2018 ) , which replaces the gradient direction found using natural evolution strategies ( NES ) . Other methods search for the best perturbation outside of this paradigm ; Moon et al . ( 2019 ) cast the problem of finding an adversarial perturbation as a discrete optimization problem and use local search methods to solve . These works all search for adversarial perturbations within a search space with a hard constraint on perturbation size ; other work ( Chen et al. , 2017 ; Tu et al. , 2019 ) incorporates a soft version of this constraint and performs coordinate descent to decrease the perturbation size while keeping the perturbed image misclassified . The latter of these methods incorporates an autoencoderbased upsampling method with which we compare in Section 5.3.1 . One may instead assume that only part of the information from the network ’ s output layer is received as the result of a query . This can take the form of only receiving the output of the top k predicted classes ( Ilyas et al. , 2018 ) , but more often the restrictive decision-based setting is considered . Here , queries yield only the predicted class , with no probability information . The most successful work in this area is in Cheng et al . ( 2018 ) , which reformulates the problem as a search for the direction of the nearest decision boundary and solves using a random gradient-free method , and in Brendel et al . ( 2017 ) and Chen et al . ( 2019 ) , both of which use random walks along the decision boundary to perform an attack . The latter work significantly improves over the former with respect to query efficiency , but the number of queries required to produce adversarial examples with small perturbations in this setting remains in the tens of thousands . A separate class of transfer-based attacks train a second , fully-observable substitute network , attack this network with white-box methods , and transfer these attacks to the original target network . These may fall into one of the preceding categories or exist outside of the distinction : in Papernot et al . ( 2016 ) , the substitute model is built with score-based queries to the target network , whereas Liu et al . ( 2016 ) trains an ensemble of models without directly querying the network at all . These methods come with their own drawbacks : they require training a substitute model , which may be costly or time-consuming , and overall attack success tends to be lower than that of gradient-based methods . Finally , there has been some recent interest in leveraging Bayesian optimization for constructing adversarial perturbations . Bayesian optimization ( BO ) has played a supporting role in several methods . For example , Zhao et al . ( 2019 ) use BO to solve the δ-step of an alternating direction of method multipliers ( ADMM ) approach , Co et al . ( 2018 ) search within a set of procedural noise perturbations using BO and Gopakumar et al . ( 2018 ) use BO to find maximal distortion error by optimizing perturbations defined using 3 parameters . On the other hand , prior work in which Bayesian optimization plays a central role performs experiments only in relatively low-dimensional problems , highlighting the main challenge of its application : Suya et al . ( 2017 ) examines an attack on a spam email classifier with 57 input features , and in Co ( 2017 ) image classifiers are attacked but notably do not scale beyond MNIST classifiers . In contrast to these past works , the main contribution of this paper is to show that Bayesian Optimization presents a scalable , query-efficient approach for large-scale black-box adversarial attacks , when combined with upsampling procedures . 3 PROBLEM FORMULATION . The following notation and definitions will be used throughout the remainder of the paper . Let F be the target neural network . We assume that F : Rd → [ 0 , 1 ] K is a K-class image classifier that takes normalized inputs : each dimension of an input x ∈ Rd represents a single pixel and is bounded between 0 and 1 , y ∈ { 1 , · · ·K } denotes the original label , and the corresponding output F ( x ) is a K-dimensional vector representing a probability distribution over classes . Rigorous evaluation of an adversarial attack requires careful definition of a threat model : a set of formal assumptions about the goals , knowledge , and capabilities of an attacker ( Carlini & Wagner , 2017 ) . We assume that , given a correctly classified input image x , the goal of the attacker is to find a perturbation δ such that x + δ is misclassified , i.e. , arg maxk F ( x + δ ) 6= arg maxk F ( x ) . We operate in the score-based black-box setting , where we have no knowledge of the internal workings of the network , and a query to the network F yields the entire corresponding K-dimensional output . To enforce the notion that the adversarial perturbation should be small , we take the common approach of requiring that ‖δ‖p be smaller than a given threshold in some ` p norm , where varies depending on the classifier . This work considers the ` ∞ norm , but our attack can easily be adapted to other norms . Finally , we denote the query budget with t ; if an adversarial example is not found after t queries to the target network , the attack fails . As in most work , we pose the attack as a constrained optimization problem . We use an objective function suggested by Carlini & Wagner ( 2017 ) and used in Tu et al . ( 2019 ) ; Chen et al . ( 2017 ) : max δ f ( x , y , δ ) subject to ‖δ‖p ≤ and ( x + δ ) ∈ [ 0 , 1 ] d , ( 1 ) where f ( x , y , δ ) = { max k 6=y log [ F ( x + δ ) ] k − log [ F ( x + δ ) ] y } . Most importantly , the input x+ δ to f is an adversarial example for F if and only if f ( x , y , δ ) > 0 . We briefly note that the above threat model and objective function were chosen for simplicity and for ease of directly comparing with other black box attacks , but the attack method we propose is compatible with many other threat models . For example , we may change the goals of the attacker or measure δ in ` 1 or ` 2 norms instead of ` ∞ with appropriate modifications to the objective function and constraints in equation 1 .

Summary: This paper applies Bayesian optimisation (BO), a sample efficient global optimisation technique, to the problem of finding adversarial perturbation. First, the application is starightforward application of BO to the well-known problem of adversarial perturbation. Nothing innovative. Second, the paper addresses the high dimensional optimisation with simple upsampling technique like nearest-neighbour, without even trying their hands dirty by using one of many many high-dimensional Bayesian optimisation algorithm (a quick Google search will reveal them), The work  thus fail in thoroughness also. Third, adversarial perturbation are known to exist even around the image such that even a simple gradient descet optimisation starting from the target image would be able to provide perceptually small perturbation (it does not have to the smallest to be perceptually small). Hence, the impact is also missing. Thus accroding to me this paper is not good enough for acceptance.