BOSH: An Efficient Meta Algorithm for Decision-based Attacks

1 INTRODUCTION . It has been shown that machine learning models , including deep neural networks , are vulnerable to adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2013 ; Chen et al. , 2017a ) . Therefore , evaluating the robustness of a given model becomes crucial for security sensitive applications . In order to evaluate the robustness of deep neural networks , researchers have developed “ attack algorithms ” to generate adversarial examples that can mislead a given neural network while being as close as possible to the original example ( Goodfellow et al. , 2014 ; Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017b ; Chen et al. , 2017b ) . Most of these attack methods are based on maximizing a loss function with a gradient-based optimizer , where the gradient is either computed by back-propagation ( in the white-box setting ) or finite-difference estimation ( in the soft-label blackbox setting ) . Although these methods work well on standard neural networks , when it comes to complex or even discontinuous models , such as decision trees and detection-based defense models , they can not be directly applied because the gradient is not available . Hard-label black-box attacks , also known as decision-based attacks , consider the most difficult but realistic setting where the attacker has no information about the model structure and parameters , and the only valid operation is to query the model to get the corresponding decision-based ( hard-label ) output ( Brendel et al. , 2017 ) . This type of attacks can be used as a “ universal ” way to evaluate robustness of any given models , no matter continuous or discrete . For instance , Cheng et al . ( 2018 ) ; Chen et al . ( 2019a ) have applied decision-based attacks for evaluating robustness of Gradient Boosting Decision Trees ( GBDT ) and random forest . Current decision-based attacks , including Brendel et al . ( 2017 ) ; Cheng et al . ( 2018 ) ; Chen et al . ( 2019b ) ; Cheng et al . ( 2019 ) , are based on iterative local updates – starting from an initial point on the decision surface , they iteratively move the points along the surface until reaching a local minimum ( in terms of distance to the original example ) . The update is often based on gradient estimation or some other heuristics . However , the local update nature makes these methods sensitive to the starting point . As we demonstrate in Figure 1 ( a ) , the perturbation of converged adversarial examples for a neural network are quite different for different initialization configurations , and this phenomenon becomes more severe when it comes to discrete models such as GBDTs ( see Figure 1 ( b ) ) . This makes decision-based attacks converge to a sub- optimal perturbation . As a result , the solution can not really reflect the robustness of the targeted model . To overcome these difficulties and make decision-based attacks better reflect the robustness of models , we propose a meta algorithm called BOSH-attack that consistently boosts the solution quality of existing iterative local update based attacks . Our main idea is to combine Bayesian optimization , which finds solution closer to global optimum but suffers from high computation cost , with iterative local updates , which converges fast but often get stuck in local minimum . Specifically , given a decision based attack A , our algorithm maintains a pool of solutions and at each iteration we run A for m steps on each solution . The proposed Bayesian Optimization resampling ( BO ) and Successive Halving ( SH ) are then used to explore important solution space based on current information and cut out unnecessary solution paths . Our contributions are summarized below : 1 . We conduct thorough experiments to show that current decision-based attacks often converge to a local optimum , thus further improvements are required . 2 . Based on the idea of Bayesian optimization and successive halving , we design a meta algorithm to boost the performance of current decision-based attack algorithms and encourage them to find a much smaller adversarial perturbation efficiently . 3 . Comprehensive experiments demonstrate that BOSH-attack can consistently boost existing decision-based attacks to find better examples with much smaller perturbation . In addition to the standard neural network models , we also test our algorithms on attacking discrete GBDT models and detector-based defense models . Moreover , our algorithm can reduce the computation cost by 10x compared to the naive approach . 2 BACKGROUND AND RELATED WORK . Given a classification model F : Rd → { 1 , . . . , C } and an example x0 , adversarial attacks aim to find the adversarial example that is closest to x0 . For example , an untargeted attack aims to find the minimum perturbation to change the predicted class , which corresponds to the following optimization problem : min δ ‖δ‖ s.t . F ( x0 + δ ) 6= F ( x0 ) . ( 1 ) Exactly minimizing ( 1 ) is usually intractable ; therefore , we can only expect to get a feasible solution of ( 1 ) while hoping ‖δ‖ to be as small as possible . White-box Attack . For neural networks , we can replace the constraint in ( 1 ) by a loss function defined on the logit layer output , leading to a continuous optimization problem which can be solved by gradient-based optimizers . This approach has been used in popular methods such as FGSM ( Goodfellow et al. , 2014 ) , C & W attack ( Carlini & Wagner , 2017b ) and PGD attack ( Madry et al. , 2017 ) . All these white-box attacks developed for neural networks assume the existence of the gradient . However , for models with discrete components such as GBDT , the objective can not be easily defined and gradient-based white-box attacks are not applicable . There are few white-box attacks developed for specific discrete models , such as Mixed Integer Linear Programming ( MILP ) approach for attacking tree ensemble ( Kantchelian et al. , 2016 ) . However , those algorithms are time consuming and require significant efforts for developing each model . Soft-label Black-box Attack Black box setting considers the cases when an attacker has no direct access to the model ’ s parameter and architecture , and the only valid operation is to query input examples and get the corresponding model output . In the soft-label black box setting , it is assumed that the model outputs the probability of each label for an input query . Chen et al . ( 2017b ) showed that the attack can still be formulated as an optimization problem where the objective function value can be computed while the gradient is unavailable . Based on this , various zeroth order optimization algorithms have been proposed , including NES-attack ( Ilyas et al. , 2018a ) , EAD attack ( Chen et al. , 2018 ) , bandit-attack ( Ilyas et al. , 2018b ) , Autozoom ( Tu et al. , 2019 ) , Genetic algorithm ( Alzantot et al. , 2018 ) . Hard-label Black-box attack ( Decision-based attack ) In this paper , we focus on the hard-label black box attack ( also known as decision-based attack ) . In contrast to the soft-label setting , the attacker can only query the model and get the top-1 predicted label without any probability information . To minimize the perturbation in the decision-based setting , Brendel et al . ( 2017 ) first proposed a Boundary Attack based on random walk on the decision surface . Later on , Cheng et al . ( 2018 ) showed that hard-label attack can be reformulated as another continuous optimization problem , and zeroth order optimization algorithms such as NES can be used to solve this problem . Cheng et al . ( 2019 ) further reduces the queries by only calculating the sign of ZOO updating function , and Chen et al . ( 2019b ) proposed another algorithm to improve over boundary attack . Such methods can converge quickly but suffer from local optimum problems , and thus require more careful and thorough search in the solution space . Probability based black-box optimization There are two commonly used methods to solve a black-box or non-differentiable optimization problem : gradient-based and probabilistic-based algorithms . Gradient-based methods are based on iterative local updates until convergence , while probabilistic-based algorithms such as Bayesian optimization ( BO ) ( Pelikan et al. , 1999 ; Snoek et al. , 2012 ) approximate the objective function by a probabilistic model . Generally speaking , gradient-based methods are commonly used in black-box attack because it converges fast . However , these methods often stuck in some local optimal directions , especially when the searching space is high dimensional and non-convex . Probabilistic-based algorithms are frequently used in low-dimensional problems such as hyperparameter tuning and can have a better chance to find more global optimal values ( Snoek et al. , 2012 ; Bergstra et al. , 2011 ) . However , the computation cost grows exponentially while the dimension increases and quickly become unacceptable . Therefore , they can not be directly applied to generate adversarial examples . In this paper , we attempt to combine Bayesian Optimization and iterative local updates to improve the solution quality of current attack algorithms while being able to scale to high dimensional problems . Combinatorial heuristic and genetic algorithms There exist various heuristic algorithms commonly applied to combinatorial optimization problems . In these algorithms , they try to leverage the effects between greedy and random . Commonly , they will search for different directions and drop the bad ones , and then put more attention on the relatively good candidates . Greedy randomized adaptive search ( Feo & Resende , 1995 ) finds good solutions in an iterative way . It first generates a set of solutions and use a greedy function to rank these solutions . Later , good candidates are placed in a restricted candidate list , and randomly chosen when forming the solution . Tabu search ( Glover & Laguna , 1999 ) selects a candidate and checks its immediate neighbors , trying to find an improved solution . In order to avoid stucking in local optimal areas , it maintains a list called tabu list to store past good solutions ( often local optimal solutions ) . In further searches , it will prevent from looking for this areas . Other approaches like Genetic Algorithms ( GA ) and Simulated Annealing ( SA ) also try to adopt random in the searching process . In this paper , we simply use Successive Halving ( SH ) ( Jamieson & Talwalkar , 2016 ) to remove unimportant candidate configurations iteratively . The details can be found in Section 3.1 . 3 THE PROPOSED ALGORITHM . Observation : Decision based attacks are easily stuck in local optimum . Most existing adversarial attacks adopt iterative local updates to find adversarial examples – starting from an initial point , they iteratively update the solution until convergence . For example , in white-box attacks such as C & W and PGD methods , they aim at optimizing a non-convex loss function by iterative gradient updates . In Figure 1 ( a ) , we plot the 2-dimensional projection of the decision surface of a neural network . We can observe that there are two local minimums1 . Results show that there are two local minimums and the attack algorithm converges to one of them according the the initialization region . Similarly , in decision-based attacks , existing methods start from some point on the decision surface and then iteratively update the point locally on the surface either by gradient update ( Cheng et al. , 2018 ; Chen et al. , 2019b ) or random walk ( Brendel et al. , 2017 ) . In Figure 1 ( b ) we plot the decision surface of a GBDT . We observe a similar issue that there are many local minima in the GBDT decision boundary . 1Here local minimum indicates a point on the decision boundary that has shortest distance to the original example , compared to other nearby points on the decision boundary . Those local minimums are the points where a decision-based attack can converge to . We further quantify how serious the problem is . On an MNIST network , Figure 2 ( a ) shows the distribution of converged adversarial perturbations of C & W attack ( white-box attack ) and SignOPT attack ( decision-based attack ) under approximately 400 random initial points . We observe that the converged solutions of C & W attack are quite concentrated between [ 1.41 , 1.47 ] . However , when considering decision-based attack such as Sign-OPT , the converged solutions are widely spread from 1.36 to 1.55 . In general , our experiments suggested that decision based attacks are much more sensitive to initialization . This is because they only update solutions on the decision boundary while C & W and PGD attack can update solution inside/outside the boundary . Furthermore , such phenomenon is obvious when the victim model is GBDT . For example , in Figure 2 ( b ) we can see the converged solution spread from 0.5 to 1.5 when applying Sign-OPT attack . Therefore , the solution of any single run of Sign-OPT on GBDT can not really reflect the minimum adversarial perturbation of the given model , and thus it is crucial to design an algorithm that converges to a better solution . Since the phenomenon is more severe for decision based attacks , we will mainly focus on improving the quality of decision based attacks in this paper , while in general our method can also be used to boost the performance of white-box attacks marginally , as illustrated in Appendix A .

Summary: This paper proposes a meta-algorithm for the so-called "decision-based attack" problem, where a model that can be accessed only via label queries for a given input is attacked by a minimal perturbation to the input that changes the predicted label. The algorithm, BOSH, augments any iterative algorithm for this problem with a diversification strategy based on bayesian optimization and throwing away bad solutions. Empirically, it is shown that BOSH can improve the performance of recently developed algorithms for this problem, by exploring more solutions and refining them intelligently.