Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach

We study the problem of attacking machine learning models in the hard-label black-box setting , where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions . This is a very challenging problem since the direct extension of state-of-the-art white-box attacks ( e.g. , C & W or PGD ) to the hard-label black-box setting will require minimizing a non-continuous step function , which is combinatorial and can not be solved by a gradient-based optimizer . The only two current approaches are based on random walk on the boundary ( Brendel et al. , 2017 ) and random trials to evaluate the loss function ( Ilyas et al. , 2018 ) , which require lots of queries and lacks convergence guarantees . We propose a novel way to formulate the hard-label black-box attack as a real-valued optimization problem which is usually continuous and can be solved by any zeroth order optimization algorithm , such as randomized gradientfree method ( Nesterov & Spokoiny , 2017 ) . We demonstrate that our proposed method outperforms the previous stochastic approaches to attacking convolutional neural networks on MNIST , CIFAR , and ImageNet datasets . More interestingly , the proposed algorithm can also be used to attack other discrete and non-continuous machine learning models , such as Gradient Boosting Decision Trees . 1 INTRODUCTION . It has been observed recently that machine learning algorithms , especially deep neural networks , are vulnerable to adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2013 ; Moosavi-Dezfooli et al . ; Moosavi Dezfooli et al. , 2016 ; Chen et al. , 2018a ; Cheng et al. , 2018 ) . For example , in image classification problems , attack algorithms ( Carlini & Wagner , 2017 ; Goodfellow et al. , 2014 ; Chen et al. , 2017 ) can find adversarial examples for almost every image with very small humanimperceptible perturbation . The problem of finding an adversarial example can be posed as solving an optimization problem—within a small neighbourhood around the original example , find a point to optimize the cost function measuring the “ successfulness ” of an attack . Solving this objective function with gradient-based optimizer leads to state-of-the-art attacks ( Carlini & Wagner , 2017 ; Goodfellow et al. , 2014 ; Chen et al. , 2017 ; Szegedy et al. , 2013 ; Madry et al. , 2018 ) . Most current attacks ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2017 ; Szegedy et al. , 2013 ; Chen et al. , 2018b ) consider the “ white-box ” setting , where the machine learning model is fully exposed to the attacker . In this setting , the gradient of an attacking objective function can be computed by back-propagation , so attacks can be done very easily . This white-box setting is clearly unrealistic when the model parameters are unknown to an attacker . Instead , several recent works consider the “ score-based black-box ” setting , where the machine learning model is unknown to the attacker , but it is possible to make queries to obtain the corresponding probability outputs of the model ( Chen et al. , 2017 ; Ilyas et al. , 2018 ) . However , in many cases real-world models will not provide probability outputs to users . Instead , only the final decision ( e.g. , top-1 predicted class ) can be observed . It is therefore interesting to show whether machine learning model is vulnerable in this setting . Furthermore , existing gradient-based attacks can not be applied to some non-continuous machine learning models which involve discrete decisions . For example , the robustness of decision-tree based models ( random forest and gradient boosting decision trees ( GBDT ) ) can not be evaluated using gradient-based approaches , since the gradient of these functions may not exist . In this paper , we develop an optimization-based framework for attacking machine learning models in a more realistic and general “ hard-label black-box ” setting . We assume that the model is not revealed and the attacker can only make queries to acquire the corresponding hard-label decision instead of the probability outputs ( also known as soft labels ) . We make hard-label black-box attacks query-efficient by reformulating the attack as a novel realvalued optimization problem , which is usually continuous and much easier to solve . Although the objective function of this reformulation can not be written in an analytical form , we show how to use ( hard-label ) model queries to evaluate its function value and thus any zeroth order optimization algorithm can be applied to solve it . In the experiments , we show our algorithm can be successfully used to attack hard-label black-box CNN models on MNIST , CIFAR , and ImageNet with far less number of queries compared to the state-of-art algorithm both in L2 and L∞ metric . Moreover , since our algorithm does not depend on the gradient of the classifier , we can apply it to attack other non-differentiable classifiers besides neural networks . We show an interesting application in attacking Gradient Boosting Decision Tree , which can not be attacked by all the existing gradientbased methods even in the white-box setting . Our method can successfully find adversarial examples with imperceptible perturbations for a GBDT within 30,000 queries . 2 BACKGROUND AND RELATED WORK . We will first introduce our problem setting and give a brief literature review to highlight the difficulty of attacking hard-label black-box models . Problem Setting For simplicity , we consider attacking a K-way multi-class classification model in this paper . Given the classification model f : Rd → { 1 , . . . , K } and an original example x0 , the goal is to generate an adversarial example x such that x is close to x0 and f ( x ) 6= f ( x0 ) ( i.e. , x is misclassified by model f . ) ( 1 ) White-box attacks Most attack algorithms in the literature consider the white-box setting , where the classifier f is exposed to the attacker . For neural networks , under this assumption , backpropagation can be conducted on the target model because both network structure and weights are known by the attacker . For classification models in neural networks , it is usually assumed that f ( x ) = argmaxi ( Z ( x ) i ) , where Z ( x ) ∈ RK is the final ( logit ) layer output , and Z ( x ) i is the prediction score for the i-th class . The objectives in ( 1 ) can then be naturally formulated as the following optimization problem : argmin x { Dis ( x , x0 ) + cL ( Z ( x ) ) } : = h ( x ) , ( 2 ) where Dis ( · , · ) is some distance measurement ( e.g. , L2 , L1 or L∞ norm in Euclidean space ) , L ( · ) is the loss function corresponding to the goal of the attack , and c is a balancing parameter . For untargeted attack , where the goal is to make the target classifier misclassify , the loss function can be defined as L ( Z ( x ) ) = max { [ Z ( x ) ] y0 −max i 6=y0 [ Z ( x ) ] i , −κ } , ( 3 ) where y0 is the original label predicted by the classifier , κ is the margin ( usually set to be 1 or 0 ) of the hinge loss . For targeted attack , where the goal is to turn it into a specific target class t , the loss function can also be defined accordingly . Therefore , attacking a machine learning model can be posed as solving this optimization problem ( Carlini & Wagner , 2017 ; Chen et al. , 2018b ) , which is also known as the C & W attack or the EAD attack depending on the choice of the distance measurement . To solve ( 2 ) , one can apply any gradient-based optimization algorithm such as SGD or Adam , since the gradient of L ( Z ( x ) ) can be computed via back-propagation . The ability of computing gradient also enables many different attacks in the white-box setting . For example , eq ( 2 ) can also be turned into a constrained optimization problem , which can then be solved by projected gradient descent ( PGD ) ( Madry et al. , 2018 ) . FGSM ( Goodfellow et al. , 2014 ) is the special case of one step PGD with L∞ norm distance . Other algorithms such as Deepfool ( Moosavi Dezfooli et al. , 2016 ) also solve similar optimization problems to construct adversarial examples . Previous work on black-box attack In real-world systems , usually the underlying machine learning model will not be revealed and thus white-box attacks can not be applied . This motivates the study of attacking machine learning models in the black-box setting , where attackers do not have any information about the function f . And the only valid operation is to make queries to the model and acquire the corresponding output f ( x ) . The first approach for black-box attack is using transfer attack ( Papernot et al. , 2017 ) – instead of attacking the original model f , attackers try to construct a substitute model f̂ to mimic f and then attack f̂ using white-box attack methods . This approach has been well studied and analyzed in ( Liu et al. , 2017 ; Bhagoji et al. , 2017 ) . However , recent papers have shown that attacking the substitute model usually leads to much larger distortion and low success rate ( Chen et al. , 2017 ) . Therefore , instead , ( Chen et al. , 2017 ) considers the score-based black-box setting , where attackers can use x to query the softmax layer output in addition to the final classification result . In this case , they can reconstruct the loss function ( 3 ) and evaluate it as long as the objective function h ( x ) exists for any x . Thus a zeroth order optimization approach can be directly applied to minimize h ( x ) . ( Tu et al. , 2018 ) further improves the query complexity of ( Chen et al. , 2017 ) by introducing an autoencoder-based approach to reduce query counts and an adaptive random gradient estimation to balance query counts and distortion . Difficulty of hard-label black-box attacks Throughout this paper , the hard-label black-box setting refers to cases where real-world ML systems only provide limited prediction results of an input query . Specifically , only the final decision ( top-1 predicted label ) instead of probability outputs is known to an attacker . Attacking in this setting is indeed very challenging . In Figure 1a , we show a simple 3-layer neural network ’ s decision boundary . Note that the L ( Z ( x ) ) term is continuous as in Figure 1b because the logit layer output is real-valued functions . However , in the hard-label black-box setting , only f ( · ) is available instead of Z ( · ) . Since f ( · ) can only be a one-hot vector , if we plug-in f into the loss function , L ( f ( x ) ) ( as shown in Figure 1c ) will be discontinuous and with discrete outputs . Optimizing this function will require combinatorial optimization or search algorithms , which is challenging given the high dimensionality of the problem . The only two current approaches ( Brendel et al. , 2017 ; Ilyas et al. , 2018 ) are based on random-walk on the boundary and random trails on the loss function . Although these “ Boundary attack ” and “ Limited attack ” can find adversarial examples with comparable distortion with white-box attacks , they need lots of queries to explore the high-dimensional space and lack convergence guarantees . We show that our optimization-based algorithm can significantly reduce the number of queries , and has guaranteed convergence in the number of iterations ( queries ) when the objective function is lipschitz smooth .

Summary: In this paper the authors propose optimizing for adversarial examples against black box models by considering minimizing the distance to the decision boundary.  They show that because this gives real valued feedback, the optimizer is able to find closer adversarial examples with fewer queries.  This would be heavily dependent on the model structure (with more complex decision boundaries likely being harder to optimize) but they show empirically in 4 models that this method works well.