Query-Efficient Hard-label Black-box Attack: An Optimization-based Approach

We study the problem of attacking machine learning models in the hard-label black-box setting , where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions . This is a very challenging problem since the direct extension of state-of-the-art white-box attacks ( e.g. , C & W or PGD ) to the hard-label black-box setting will require minimizing a non-continuous step function , which is combinatorial and can not be solved by a gradient-based optimizer . The only two current approaches are based on random walk on the boundary ( Brendel et al. , 2017 ) and random trials to evaluate the loss function ( Ilyas et al. , 2018 ) , which require lots of queries and lacks convergence guarantees . We propose a novel way to formulate the hard-label black-box attack as a real-valued optimization problem which is usually continuous and can be solved by any zeroth order optimization algorithm , such as randomized gradientfree method ( Nesterov & Spokoiny , 2017 ) . We demonstrate that our proposed method outperforms the previous stochastic approaches to attacking convolutional neural networks on MNIST , CIFAR , and ImageNet datasets . More interestingly , the proposed algorithm can also be used to attack other discrete and non-continuous machine learning models , such as Gradient Boosting Decision Trees . 1 INTRODUCTION . It has been observed recently that machine learning algorithms , especially deep neural networks , are vulnerable to adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2013 ; Moosavi-Dezfooli et al . ; Moosavi Dezfooli et al. , 2016 ; Chen et al. , 2018a ; Cheng et al. , 2018 ) . For example , in image classification problems , attack algorithms ( Carlini & Wagner , 2017 ; Goodfellow et al. , 2014 ; Chen et al. , 2017 ) can find adversarial examples for almost every image with very small humanimperceptible perturbation . The problem of finding an adversarial example can be posed as solving an optimization problem—within a small neighbourhood around the original example , find a point to optimize the cost function measuring the “ successfulness ” of an attack . Solving this objective function with gradient-based optimizer leads to state-of-the-art attacks ( Carlini & Wagner , 2017 ; Goodfellow et al. , 2014 ; Chen et al. , 2017 ; Szegedy et al. , 2013 ; Madry et al. , 2018 ) . Most current attacks ( Goodfellow et al. , 2014 ; Carlini & Wagner , 2017 ; Szegedy et al. , 2013 ; Chen et al. , 2018b ) consider the “ white-box ” setting , where the machine learning model is fully exposed to the attacker . In this setting , the gradient of an attacking objective function can be computed by back-propagation , so attacks can be done very easily . This white-box setting is clearly unrealistic when the model parameters are unknown to an attacker . Instead , several recent works consider the “ score-based black-box ” setting , where the machine learning model is unknown to the attacker , but it is possible to make queries to obtain the corresponding probability outputs of the model ( Chen et al. , 2017 ; Ilyas et al. , 2018 ) . However , in many cases real-world models will not provide probability outputs to users . Instead , only the final decision ( e.g. , top-1 predicted class ) can be observed . It is therefore interesting to show whether machine learning model is vulnerable in this setting . Furthermore , existing gradient-based attacks can not be applied to some non-continuous machine learning models which involve discrete decisions . For example , the robustness of decision-tree based models ( random forest and gradient boosting decision trees ( GBDT ) ) can not be evaluated using gradient-based approaches , since the gradient of these functions may not exist . In this paper , we develop an optimization-based framework for attacking machine learning models in a more realistic and general “ hard-label black-box ” setting . We assume that the model is not revealed and the attacker can only make queries to acquire the corresponding hard-label decision instead of the probability outputs ( also known as soft labels ) . We make hard-label black-box attacks query-efficient by reformulating the attack as a novel realvalued optimization problem , which is usually continuous and much easier to solve . Although the objective function of this reformulation can not be written in an analytical form , we show how to use ( hard-label ) model queries to evaluate its function value and thus any zeroth order optimization algorithm can be applied to solve it . In the experiments , we show our algorithm can be successfully used to attack hard-label black-box CNN models on MNIST , CIFAR , and ImageNet with far less number of queries compared to the state-of-art algorithm both in L2 and L∞ metric . Moreover , since our algorithm does not depend on the gradient of the classifier , we can apply it to attack other non-differentiable classifiers besides neural networks . We show an interesting application in attacking Gradient Boosting Decision Tree , which can not be attacked by all the existing gradientbased methods even in the white-box setting . Our method can successfully find adversarial examples with imperceptible perturbations for a GBDT within 30,000 queries . 2 BACKGROUND AND RELATED WORK . We will first introduce our problem setting and give a brief literature review to highlight the difficulty of attacking hard-label black-box models . Problem Setting For simplicity , we consider attacking a K-way multi-class classification model in this paper . Given the classification model f : Rd → { 1 , . . . , K } and an original example x0 , the goal is to generate an adversarial example x such that x is close to x0 and f ( x ) 6= f ( x0 ) ( i.e. , x is misclassified by model f . ) ( 1 ) White-box attacks Most attack algorithms in the literature consider the white-box setting , where the classifier f is exposed to the attacker . For neural networks , under this assumption , backpropagation can be conducted on the target model because both network structure and weights are known by the attacker . For classification models in neural networks , it is usually assumed that f ( x ) = argmaxi ( Z ( x ) i ) , where Z ( x ) ∈ RK is the final ( logit ) layer output , and Z ( x ) i is the prediction score for the i-th class . The objectives in ( 1 ) can then be naturally formulated as the following optimization problem : argmin x { Dis ( x , x0 ) + cL ( Z ( x ) ) } : = h ( x ) , ( 2 ) where Dis ( · , · ) is some distance measurement ( e.g. , L2 , L1 or L∞ norm in Euclidean space ) , L ( · ) is the loss function corresponding to the goal of the attack , and c is a balancing parameter . For untargeted attack , where the goal is to make the target classifier misclassify , the loss function can be defined as L ( Z ( x ) ) = max { [ Z ( x ) ] y0 −max i 6=y0 [ Z ( x ) ] i , −κ } , ( 3 ) where y0 is the original label predicted by the classifier , κ is the margin ( usually set to be 1 or 0 ) of the hinge loss . For targeted attack , where the goal is to turn it into a specific target class t , the loss function can also be defined accordingly . Therefore , attacking a machine learning model can be posed as solving this optimization problem ( Carlini & Wagner , 2017 ; Chen et al. , 2018b ) , which is also known as the C & W attack or the EAD attack depending on the choice of the distance measurement . To solve ( 2 ) , one can apply any gradient-based optimization algorithm such as SGD or Adam , since the gradient of L ( Z ( x ) ) can be computed via back-propagation . The ability of computing gradient also enables many different attacks in the white-box setting . For example , eq ( 2 ) can also be turned into a constrained optimization problem , which can then be solved by projected gradient descent ( PGD ) ( Madry et al. , 2018 ) . FGSM ( Goodfellow et al. , 2014 ) is the special case of one step PGD with L∞ norm distance . Other algorithms such as Deepfool ( Moosavi Dezfooli et al. , 2016 ) also solve similar optimization problems to construct adversarial examples . Previous work on black-box attack In real-world systems , usually the underlying machine learning model will not be revealed and thus white-box attacks can not be applied . This motivates the study of attacking machine learning models in the black-box setting , where attackers do not have any information about the function f . And the only valid operation is to make queries to the model and acquire the corresponding output f ( x ) . The first approach for black-box attack is using transfer attack ( Papernot et al. , 2017 ) – instead of attacking the original model f , attackers try to construct a substitute model f̂ to mimic f and then attack f̂ using white-box attack methods . This approach has been well studied and analyzed in ( Liu et al. , 2017 ; Bhagoji et al. , 2017 ) . However , recent papers have shown that attacking the substitute model usually leads to much larger distortion and low success rate ( Chen et al. , 2017 ) . Therefore , instead , ( Chen et al. , 2017 ) considers the score-based black-box setting , where attackers can use x to query the softmax layer output in addition to the final classification result . In this case , they can reconstruct the loss function ( 3 ) and evaluate it as long as the objective function h ( x ) exists for any x . Thus a zeroth order optimization approach can be directly applied to minimize h ( x ) . ( Tu et al. , 2018 ) further improves the query complexity of ( Chen et al. , 2017 ) by introducing an autoencoder-based approach to reduce query counts and an adaptive random gradient estimation to balance query counts and distortion . Difficulty of hard-label black-box attacks Throughout this paper , the hard-label black-box setting refers to cases where real-world ML systems only provide limited prediction results of an input query . Specifically , only the final decision ( top-1 predicted label ) instead of probability outputs is known to an attacker . Attacking in this setting is indeed very challenging . In Figure 1a , we show a simple 3-layer neural network ’ s decision boundary . Note that the L ( Z ( x ) ) term is continuous as in Figure 1b because the logit layer output is real-valued functions . However , in the hard-label black-box setting , only f ( · ) is available instead of Z ( · ) . Since f ( · ) can only be a one-hot vector , if we plug-in f into the loss function , L ( f ( x ) ) ( as shown in Figure 1c ) will be discontinuous and with discrete outputs . Optimizing this function will require combinatorial optimization or search algorithms , which is challenging given the high dimensionality of the problem . The only two current approaches ( Brendel et al. , 2017 ; Ilyas et al. , 2018 ) are based on random-walk on the boundary and random trails on the loss function . Although these “ Boundary attack ” and “ Limited attack ” can find adversarial examples with comparable distortion with white-box attacks , they need lots of queries to explore the high-dimensional space and lack convergence guarantees . We show that our optimization-based algorithm can significantly reduce the number of queries , and has guaranteed convergence in the number of iterations ( queries ) when the objective function is lipschitz smooth .

Summary: This paper addresses black-box classifier attacks in the “hard-label” setting, meaning that the only information the attacker has access to is single top-1 label predictions. Relative to even the standard black-box setting where the attacker has access to the per-class logits or probabilities, this setting is difficult as it makes the optimization landscape non-smooth. The proposed approach reformulates the optimization problem such that the outer-loop optimizes the direction using approximate gradients, and the inner-loop estimates the distance to the nearest attack in a given direction. The results show that the proposed approach successfully finds both untargeted and targeted adversarial examples for classifiers of various image datasets (including ImageNet), usually with substantially better query-efficiency and better final results (lower distance and/or higher success rate) than competing methods.