Sign-OPT: A Query-Efficient Hard-label Adversarial Attack

1 INTRODUCTION . It has been shown that neural networks are vulnerable to adversarial examples ( Szegedy et al. , 2016 ; Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ; Athalye et al. , 2018 ) . Given a victim neural network model and a correctly classified example , an adversarial attack aims to compute a small perturbation such that with this perturbation added , the original example will be misclassified . Many adversarial attacks have been proposed in the literature . Most of them consider the white-box setting , where the attacker has full knowledge about the victim model , and thus gradient based optimization can be used for attack . Popular Examples include C & W ( Carlini & Wagner , 2017 ) and PGD ( Madry et al. , 2017 ) attacks . On the other hand , some more recent attacks have considered the probability black-box setting where the attacker does not know the victim model ’ s structure and weights , but can iteratively query the model and get the corresponding probability output . In this setting , although gradient ( of output probability to the input layer ) is not computable , it can still be estimated using finite differences , and algorithms many attacks are based on this ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; Tu et al. , 2019 ; Jun et al. , 2018 ) . In this paper , we consider the most challenging and practical attack setting – hard-label black-box setting – where the model is hidden to the attacker and the attacker can only make queries and get the corresponding hard-label decisions ( e.g. , predicted labels ) of the model . A commonly used algorithm proposed in this setting , also called Boundary attack ( Brendel et al. , 2017 ) , is based on random walks on the decision surface , but it does not have any convergence guarantee . More recently , Cheng et al . ( 2019 ) showed that finding the minimum adversarial perturbation in the hard-label setting can be reformulated as another optimization problem ( we call this Cheng ’ s formulation in this paper ) . This ∗Equal Contribution . new formulation enjoys the benefit of having a smooth boundary in most tasks and the function value is computable using hard-label queries . Therefore , the authors of ( Cheng et al. , 2019 ) are able to use standard zeroth order optimization to solve the new formulation . Although their algorithm converges quickly , it still requires large number of queries ( e.g. , 20,000 ) for attacking a single image since every function evaluation of Cheng ’ s formulation has to be computed using binary search requiring tens of queries . In this paper , we follow the same optimization formulation of ( Cheng et al. , 2019 ) which has the advantage of smoothness , but instead of using finite differences to estimate the magnitude of directional derivative , we propose to evaluate its sign using only a single query . With this single-query sign oracle , we design novel algorithms for solving the Cheng ’ s formulation , and we theoretically prove and empirically demonstrate the significant reduction in the number of queries required for hard-label black box attack . Our contribution are summarized below : • Novelty in terms of adversarial attack . We elucidate an efficient approach to compute the sign of directional derivative of Cheng ’ s formulation using a single query , and based on this technique we develop a novel optimization algorithm called Sign-OPT for hard-label black-box attack . • Novelty in terms of optimization . Our method can be viewed as a new zeroth order optimization algorithm that features fast convergence of signSGD . Instead of directly taking the sign of gradient estimation , our algorithm utilizes the scale of random direction . This make existing analysis inappropriate to our case , and we provide a new recipe to prove the convergence of this new optimizer . • We conduct comprehensive experiments on several datasets and models . We show that the proposed algorithm consistently reduces the query count by 5–10 times across different models and datasets , suggesting a practical and query-efficient robustness evaluation tool . Furthermore , on most datasets our algorithm can find an adversarial example with smaller distortion compared with previous approaches . 2 RELATED WORK . White-box attack Since it was firstly found that neural networks are easy to be fooled by adversarial examples ( Goodfellow et al. , 2015 ) , a lot of work has been proposed in the white-box attack setting , where the classifier f is completely exposed to the attacker . For neural networks , under this assumption , back-propagation can be conducted on the target model because both network structure and weights are known by the attacker . Algorithms including ( Goodfellow et al. , 2015 ; Kurakin et al. , 2016 ; Carlini & Wagner , 2017 ; Chen et al. , 2018 ; Madry et al. , 2017 ) are then proposed based on gradient computation . Recently , the BPDA attack introduced by Athalye et al . ( 2018 ) bypasses some models with obfuscated gradients and is shown to successfully circumvent many defenses . In addition to typical attacks based on small ` p norm perturbation , non- ` p norm perturbations such as scaling or shifting have also been considered ( Zhang et al. , 2019 ) . Black-box attack Recently , black-box setting is drawing rapidly increasing attention . In black-box setting , the attacker can query the model but has no ( direct ) access to any internal information inside the model . Although there are some works based on transfer attack ( Papernot et al. , 2017 ) , we consider the query-based attack in the paper . Depending on the model ’ s feedback for a given query , an attack can be classified as a soft-label or hard-label attack . In the soft-label setting , the model outputs a probability score for each decision . Chen et al . ( 2017 ) uses a finite difference in a coordinate-wise manner to approximately estimate the output probability changes and does a coordinate descent to conduct the attack . Ilyas et al . ( 2018a ) uses Neural evolution strategy ( NES ) to approximately estimate the gradient directly . Later , some variants ( Ilyas et al. , 2018b ; Tu et al. , 2019 ) were proposed to utilize the side information to further speed up the attack procedure . Alzantot et al . ( 2019 ) uses a evolutionary algorithm as a black-box optimizer for the soft-label setting . Recently , Al-Dujaili & O ’ Reilly ( 2019 ) proposes SignHunter algorithm based on signSGD ( Bernstein et al. , 2018 ) to achieve faster convergence in the soft-label setting . The recent work ( Al-Dujaili & O ’ Reilly , 2019 ) proposes SignHunter algorithm to achieve a more query-efficent sign estimate when crafting black-box adversarial examples through soft-label information . In the hard-label case , only the final decision , i.e . the top-1 predicted class , is observed . As a result , the attacker can only make queries to acquire the corresponding hard-label decision instead of the probability outputs . Brendel et al . ( 2017 ) first studied this problem and proposed an algorithm based on random walks near the decision boundary . By selecting a random direction and projecting it onto a boundary sphere in each iteration , it aims to generate a high-quality adversarial example . Query-Limited attack ( Ilyas et al. , 2018a ) tries to estimate the output probability scores with model query and turn the hard-label into a soft-label problem . Cheng et al . ( 2019 ) instead re-formalizes the hard-label attack into an optimization problem that finds a direction which could produce the shortest distance to decision boundary . The recent arXiv paper ( Chen et al. , 2019 ) applied the zeroth-order sign oracle to improve Boundary attack , and also demonstrated significant improvement . The major differences to our algorithm are that we propose a new zeroth-order gradient descent algorithm , provide its algorithmic convergence guarantees , and aim to improve the query complexity of the attack formulation proposed in ( Cheng et al. , 2019 ) . For completeness , we also compare with this method in Section A.1 . Moreover , ( Chen et al. , 2019 ) uses one-point gradient estimate , which is unbiased but may encounter larger variance compared with the gradient estimate in our paper . Thus , we can observe in Section A.1 that although they are slightly faster in the initial stage , Sign-OPT will catch up and eventually lead to a slightly better solution . 3 PROPOSED METHOD . We follow the same formulation in ( Cheng et al. , 2019 ) and consider the hard-label attack as the problem of finding the direction with shortest distance to the decision boundary . Specifically , for a given example x0 , true label y0 and the hard-label black-box function f : Rd → { 1 , . . . , K } , the objective function g : Rd → R ( for the untargeted attack ) can be written as : min θ g ( θ ) where g ( θ ) = arg min λ > 0 ( f ( x0 + λ θ ‖θ‖ ) 6= y0 ) . ( 1 ) It has been shown that this objective function is usually smooth and the objective function g can be evaluated by a binary search procedure locally . At each binary search step , we query the function f ( x0 + λ θ ‖θ‖ ) and determine whether the distance to decision boundary in the direction θ is greater or smaller than λ based on the hard-label prediction1 . As the objective function is computable , the directional derivative of g can be estimated by finite differences : ∇̂g ( θ ; u ) : = g ( θ + u ) − g ( θ ) u ( 2 ) where u is a random Gaussian vector and > 0 is a very small smoothing parameter . This is a standard zeroth order oracle for estimating directional derivative and based on this we can apply many different zeroth order optimization algorithms to minimize g. For example , Cheng et al . ( 2019 ) used the Random Derivative Free algorithm Nesterov & Spokoiny ( 2017 ) to solve problem ( 1 ) . However , each computation of ( 2 ) requires many hard-label queries due to binary search , so Cheng et al . ( 2019 ) still requires a huge number of queries despite having fast convergence . In this work , we introduce an algorithm that hugely improves the query complexity over Cheng et al . ( 2019 ) . Our algorithm is based on the following key ideas : ( i ) one does not need very accurate values of directional derivative in order to make the algorithm converge , and ( ii ) there exists an imperfect but informative estimation of directional derivative of g that can be computed by a single query . 1Note that binary search only works in a small local region ; in more general case g ( θ ) has to be computed by a fine-grained search plus binary search , as discussed in Cheng et al . ( 2019 ) . Algorithm 1 : Sign-OPT attack Input : Hard-label model f , original image x0 , initial θ0 ; for t = 1 , 2 , . . . , T do Randomly sample u1 , . . . , uQ from a Gaussian or Uniform distribution ; Compute ĝ ← 1Q ∑Q q=1 sign ( g ( θt + uq ) − g ( θt ) ) · uq ; Update θt+1 ← θt − ηĝ ; Evaluate g ( θt ) using the same search algorithm in Cheng et al . ( 2019 ) ; end

Summary: The paper develops a query efficient algorithm for computing black box adversarial examples given only hard labels in the context of deep neural networks. Intuitively, the only information of the function provided to the algorithm is the label for a given sample. Technically, the authors use the formulation proposed by Cheng et al, and derive a zeroth optimization algorithm that uses less queries with nice convergence properties. Experimentally, the proposed algorithm is very effective on three different standard datasets in vision.