BOSH: An Efficient Meta Algorithm for Decision-based Attacks

1 INTRODUCTION . It has been shown that machine learning models , including deep neural networks , are vulnerable to adversarial examples ( Goodfellow et al. , 2014 ; Szegedy et al. , 2013 ; Chen et al. , 2017a ) . Therefore , evaluating the robustness of a given model becomes crucial for security sensitive applications . In order to evaluate the robustness of deep neural networks , researchers have developed “ attack algorithms ” to generate adversarial examples that can mislead a given neural network while being as close as possible to the original example ( Goodfellow et al. , 2014 ; Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017b ; Chen et al. , 2017b ) . Most of these attack methods are based on maximizing a loss function with a gradient-based optimizer , where the gradient is either computed by back-propagation ( in the white-box setting ) or finite-difference estimation ( in the soft-label blackbox setting ) . Although these methods work well on standard neural networks , when it comes to complex or even discontinuous models , such as decision trees and detection-based defense models , they can not be directly applied because the gradient is not available . Hard-label black-box attacks , also known as decision-based attacks , consider the most difficult but realistic setting where the attacker has no information about the model structure and parameters , and the only valid operation is to query the model to get the corresponding decision-based ( hard-label ) output ( Brendel et al. , 2017 ) . This type of attacks can be used as a “ universal ” way to evaluate robustness of any given models , no matter continuous or discrete . For instance , Cheng et al . ( 2018 ) ; Chen et al . ( 2019a ) have applied decision-based attacks for evaluating robustness of Gradient Boosting Decision Trees ( GBDT ) and random forest . Current decision-based attacks , including Brendel et al . ( 2017 ) ; Cheng et al . ( 2018 ) ; Chen et al . ( 2019b ) ; Cheng et al . ( 2019 ) , are based on iterative local updates – starting from an initial point on the decision surface , they iteratively move the points along the surface until reaching a local minimum ( in terms of distance to the original example ) . The update is often based on gradient estimation or some other heuristics . However , the local update nature makes these methods sensitive to the starting point . As we demonstrate in Figure 1 ( a ) , the perturbation of converged adversarial examples for a neural network are quite different for different initialization configurations , and this phenomenon becomes more severe when it comes to discrete models such as GBDTs ( see Figure 1 ( b ) ) . This makes decision-based attacks converge to a sub- optimal perturbation . As a result , the solution can not really reflect the robustness of the targeted model . To overcome these difficulties and make decision-based attacks better reflect the robustness of models , we propose a meta algorithm called BOSH-attack that consistently boosts the solution quality of existing iterative local update based attacks . Our main idea is to combine Bayesian optimization , which finds solution closer to global optimum but suffers from high computation cost , with iterative local updates , which converges fast but often get stuck in local minimum . Specifically , given a decision based attack A , our algorithm maintains a pool of solutions and at each iteration we run A for m steps on each solution . The proposed Bayesian Optimization resampling ( BO ) and Successive Halving ( SH ) are then used to explore important solution space based on current information and cut out unnecessary solution paths . Our contributions are summarized below : 1 . We conduct thorough experiments to show that current decision-based attacks often converge to a local optimum , thus further improvements are required . 2 . Based on the idea of Bayesian optimization and successive halving , we design a meta algorithm to boost the performance of current decision-based attack algorithms and encourage them to find a much smaller adversarial perturbation efficiently . 3 . Comprehensive experiments demonstrate that BOSH-attack can consistently boost existing decision-based attacks to find better examples with much smaller perturbation . In addition to the standard neural network models , we also test our algorithms on attacking discrete GBDT models and detector-based defense models . Moreover , our algorithm can reduce the computation cost by 10x compared to the naive approach . 2 BACKGROUND AND RELATED WORK . Given a classification model F : Rd → { 1 , . . . , C } and an example x0 , adversarial attacks aim to find the adversarial example that is closest to x0 . For example , an untargeted attack aims to find the minimum perturbation to change the predicted class , which corresponds to the following optimization problem : min δ ‖δ‖ s.t . F ( x0 + δ ) 6= F ( x0 ) . ( 1 ) Exactly minimizing ( 1 ) is usually intractable ; therefore , we can only expect to get a feasible solution of ( 1 ) while hoping ‖δ‖ to be as small as possible . White-box Attack . For neural networks , we can replace the constraint in ( 1 ) by a loss function defined on the logit layer output , leading to a continuous optimization problem which can be solved by gradient-based optimizers . This approach has been used in popular methods such as FGSM ( Goodfellow et al. , 2014 ) , C & W attack ( Carlini & Wagner , 2017b ) and PGD attack ( Madry et al. , 2017 ) . All these white-box attacks developed for neural networks assume the existence of the gradient . However , for models with discrete components such as GBDT , the objective can not be easily defined and gradient-based white-box attacks are not applicable . There are few white-box attacks developed for specific discrete models , such as Mixed Integer Linear Programming ( MILP ) approach for attacking tree ensemble ( Kantchelian et al. , 2016 ) . However , those algorithms are time consuming and require significant efforts for developing each model . Soft-label Black-box Attack Black box setting considers the cases when an attacker has no direct access to the model ’ s parameter and architecture , and the only valid operation is to query input examples and get the corresponding model output . In the soft-label black box setting , it is assumed that the model outputs the probability of each label for an input query . Chen et al . ( 2017b ) showed that the attack can still be formulated as an optimization problem where the objective function value can be computed while the gradient is unavailable . Based on this , various zeroth order optimization algorithms have been proposed , including NES-attack ( Ilyas et al. , 2018a ) , EAD attack ( Chen et al. , 2018 ) , bandit-attack ( Ilyas et al. , 2018b ) , Autozoom ( Tu et al. , 2019 ) , Genetic algorithm ( Alzantot et al. , 2018 ) . Hard-label Black-box attack ( Decision-based attack ) In this paper , we focus on the hard-label black box attack ( also known as decision-based attack ) . In contrast to the soft-label setting , the attacker can only query the model and get the top-1 predicted label without any probability information . To minimize the perturbation in the decision-based setting , Brendel et al . ( 2017 ) first proposed a Boundary Attack based on random walk on the decision surface . Later on , Cheng et al . ( 2018 ) showed that hard-label attack can be reformulated as another continuous optimization problem , and zeroth order optimization algorithms such as NES can be used to solve this problem . Cheng et al . ( 2019 ) further reduces the queries by only calculating the sign of ZOO updating function , and Chen et al . ( 2019b ) proposed another algorithm to improve over boundary attack . Such methods can converge quickly but suffer from local optimum problems , and thus require more careful and thorough search in the solution space . Probability based black-box optimization There are two commonly used methods to solve a black-box or non-differentiable optimization problem : gradient-based and probabilistic-based algorithms . Gradient-based methods are based on iterative local updates until convergence , while probabilistic-based algorithms such as Bayesian optimization ( BO ) ( Pelikan et al. , 1999 ; Snoek et al. , 2012 ) approximate the objective function by a probabilistic model . Generally speaking , gradient-based methods are commonly used in black-box attack because it converges fast . However , these methods often stuck in some local optimal directions , especially when the searching space is high dimensional and non-convex . Probabilistic-based algorithms are frequently used in low-dimensional problems such as hyperparameter tuning and can have a better chance to find more global optimal values ( Snoek et al. , 2012 ; Bergstra et al. , 2011 ) . However , the computation cost grows exponentially while the dimension increases and quickly become unacceptable . Therefore , they can not be directly applied to generate adversarial examples . In this paper , we attempt to combine Bayesian Optimization and iterative local updates to improve the solution quality of current attack algorithms while being able to scale to high dimensional problems . Combinatorial heuristic and genetic algorithms There exist various heuristic algorithms commonly applied to combinatorial optimization problems . In these algorithms , they try to leverage the effects between greedy and random . Commonly , they will search for different directions and drop the bad ones , and then put more attention on the relatively good candidates . Greedy randomized adaptive search ( Feo & Resende , 1995 ) finds good solutions in an iterative way . It first generates a set of solutions and use a greedy function to rank these solutions . Later , good candidates are placed in a restricted candidate list , and randomly chosen when forming the solution . Tabu search ( Glover & Laguna , 1999 ) selects a candidate and checks its immediate neighbors , trying to find an improved solution . In order to avoid stucking in local optimal areas , it maintains a list called tabu list to store past good solutions ( often local optimal solutions ) . In further searches , it will prevent from looking for this areas . Other approaches like Genetic Algorithms ( GA ) and Simulated Annealing ( SA ) also try to adopt random in the searching process . In this paper , we simply use Successive Halving ( SH ) ( Jamieson & Talwalkar , 2016 ) to remove unimportant candidate configurations iteratively . The details can be found in Section 3.1 . 3 THE PROPOSED ALGORITHM . Observation : Decision based attacks are easily stuck in local optimum . Most existing adversarial attacks adopt iterative local updates to find adversarial examples – starting from an initial point , they iteratively update the solution until convergence . For example , in white-box attacks such as C & W and PGD methods , they aim at optimizing a non-convex loss function by iterative gradient updates . In Figure 1 ( a ) , we plot the 2-dimensional projection of the decision surface of a neural network . We can observe that there are two local minimums1 . Results show that there are two local minimums and the attack algorithm converges to one of them according the the initialization region . Similarly , in decision-based attacks , existing methods start from some point on the decision surface and then iteratively update the point locally on the surface either by gradient update ( Cheng et al. , 2018 ; Chen et al. , 2019b ) or random walk ( Brendel et al. , 2017 ) . In Figure 1 ( b ) we plot the decision surface of a GBDT . We observe a similar issue that there are many local minima in the GBDT decision boundary . 1Here local minimum indicates a point on the decision boundary that has shortest distance to the original example , compared to other nearby points on the decision boundary . Those local minimums are the points where a decision-based attack can converge to . We further quantify how serious the problem is . On an MNIST network , Figure 2 ( a ) shows the distribution of converged adversarial perturbations of C & W attack ( white-box attack ) and SignOPT attack ( decision-based attack ) under approximately 400 random initial points . We observe that the converged solutions of C & W attack are quite concentrated between [ 1.41 , 1.47 ] . However , when considering decision-based attack such as Sign-OPT , the converged solutions are widely spread from 1.36 to 1.55 . In general , our experiments suggested that decision based attacks are much more sensitive to initialization . This is because they only update solutions on the decision boundary while C & W and PGD attack can update solution inside/outside the boundary . Furthermore , such phenomenon is obvious when the victim model is GBDT . For example , in Figure 2 ( b ) we can see the converged solution spread from 0.5 to 1.5 when applying Sign-OPT attack . Therefore , the solution of any single run of Sign-OPT on GBDT can not really reflect the minimum adversarial perturbation of the given model , and thus it is crucial to design an algorithm that converges to a better solution . Since the phenomenon is more severe for decision based attacks , we will mainly focus on improving the quality of decision based attacks in this paper , while in general our method can also be used to boost the performance of white-box attacks marginally , as illustrated in Appendix A .

Summary: In this paper, the authors study the adversarial example generation problem, in the difficult case where the attacked model is a black box. Since the model is unknown, the approaches based on the minimization of a loss function with a gradient based optimizer do not apply. The current alternatives, known as decision-based attack, use iterative local updates from a starting point to a local minimum, where the class of the adversarial example is different from the initial example while its distance stays close to the initial one.