Original: S IGN -OPT: A Q UERY -E FFICIENT H ARD - LABEL A DVERSARIAL A TTACK
Revised: S IGN -OPT: A Q UERY -E FFICIENT H ARD - LABEL A DVERSARIAL A TTACK

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input.
Revised: We study the most practical problem setup for evaluating adversarial robustness of a machine learning system with limited access: the hard-label black-box attack setting for generating adversarial examples, where limited model queries are allowed and only the decision is provided to a queried data input.

Original: Several algorithms have been proposed for this problem but they typically require huge amount ( > 20,000) of queries for attacking one example.
Revised: Several algorithms have been proposed for this problem but they typically require huge amount ( > 20,000) of queries for attacking one example.

Original: Among them, one of the state-of-the-art approaches (Cheng et al., 2019) showed that hard-label attack can be modeled as an optimization problem where the objective function can be evaluated by binary search with additional model queries, thereby a zeroth order optimization algorithm can be applied.
Revised: Among them, one of the state-of-the-art approaches (Cheng et al., 2019) showed that hard-label attack can be modeled as an optimization problem where the objective function can be evaluated by binary search with additional model queries, thereby a zeroth order optimization algorithm can be applied.

Original: In this paper, we adopt the same optimization formulation but propose to directly estimate the sign of gradient at any direction instead of the gradient itself, which enjoys the beneﬁt of single query.
Revised: In this paper, we adopt the same optimization formulation but propose to directly estimate the sign of gradient at any direction instead of the gradient itself, which enjoys the beneﬁt of single query.

Original: Using this single query oracle for retrieving sign of directional derivative, we develop a novel query-efﬁcient Sign-OPT approach for hard-label black-box attack.
Revised: Using this single query oracle for retrieving sign of directional derivative, we develop a novel query-efﬁcient Sign-OPT approach for hard-label black-box attack.

Original: We provide a convergence analysis of the new algorithm and conduct experiments on several models on MNIST, CIFAR-10 and ImageNet.
Revised: We provide a convergence analysis of the new algorithm and conduct experiments on several models on MNIST, CIFAR-10 and ImageNet.

Original: We ﬁnd that Sign-OPT attack consistently requires 5 × to 10 × fewer queries when compared to the current state-of-the-art approaches, and usually converges to an adversarial example with smaller perturbation.
Revised: We ﬁnd that Sign-OPT attack consistently requires 5 × to 10 × fewer queries when compared to the current state-of-the-art approaches, and usually converges to an adversarial example with smaller perturbation.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: It has been shown that neural networks are vulnerable to adversarial examples (Szegedy et al., 2016;
Revised: It has been shown that neural networks are vulnerable to adversarial examples (Szegedy et al., 2016;

Original: Goodfellow et al., 2015; Carlini & Wagner, 2017; Athalye et al., 2018).
Revised: Goodfellow et al., 2015; Carlini & Wagner, 2017; Athalye et al., 2018).

Original: Given a victim neural network model and a correctly classiﬁed example, an adversarial attack aims to compute a small perturbation such that with this perturbation added, the original example will be misclassiﬁed.
Revised: Given a victim neural network model and a correctly classiﬁed example, an adversarial attack aims to compute a small perturbation such that with this perturbation added, the original example will be misclassiﬁed.

Original: Many adversarial attacks have been proposed in the literature.
Revised: Many adversarial attacks have been proposed in the literature.

Original: Most of them consider the white-box setting, where the attacker has full knowledge about the victim model, and thus gradient based optimization can be used for attack.
Revised: Most of them consider the white-box setting, where the attacker has full knowledge about the victim model, and thus gradient based optimization can be used for attack.

Original: Popular Examples include C&W (Carlini & Wagner, 2017) and PGD (Madry et al., 2017) attacks.
Revised: Popular Examples include C&W (Carlini & Wagner, 2017) and PGD (Madry et al., 2017) attacks.

Original: On the other hand, some more recent attacks have considered the probability black-box setting where the attacker does not know the victim model’s structure and weights, but can iteratively query the model and get the corresponding probability output.
Revised: On the other hand, some more recent attacks have considered the probability black-box setting where the attacker does not know the victim model’s structure and weights, but can iteratively query the model and get the corresponding probability output.

Original: In this setting, although gradient (of output probability to the input layer) is not computable, it can still be estimated using ﬁnite differences, and algorithms many attacks are based on this (Chen et al., 2017; Ilyas et al., 2018a;
Revised: In this setting, although gradient (of output probability to the input layer) is not computable, it can still be estimated using ﬁnite differences, and algorithms many attacks are based on this (Chen et al., 2017; Ilyas et al., 2018a;

Original: Tu et al., 2019; Jun et al., 2018).
Revised: Tu et al., 2019; Jun et al., 2018).

Original: In this paper, we consider the most challenging and practical attack setting – hard-label black-box setting – where the model is hidden to the attacker and the attacker can only make queries and get the corresponding hard-label decisions (e.g., predicted labels) of the model.
Revised: In this paper, we consider the most challenging and practical attack setting – hard-label black-box setting – where the model is hidden to the attacker and the attacker can only make queries and get the corresponding hard-label decisions (e.g., predicted labels) of the model.

Original: A commonly used algorithm proposed in this setting, also called Boundary attack (Brendel et al., 2017), is based on random walks on the decision surface, but it does not have any convergence guarantee.
Revised: A commonly used algorithm proposed in this setting, also called Boundary attack (Brendel et al., 2017), is based on random walks on the decision surface, but it does not have any convergence guarantee.

Original: More recently, Cheng et al.
Revised: More recently, Cheng et al.

Original: (2019) showed that ﬁnding the minimum adversarial perturbation in the hard-label setting can be reformulated as another optimization problem (we call this Cheng’s formulation in this paper).
Revised: (2019) showed that ﬁnding the minimum adversarial perturbation in the hard-label setting can be reformulated as another optimization problem (we call this Cheng’s formulation in this paper).

Revised: This ∗ Equal Contribution.

Original: This new formulation enjoys the beneﬁt of having a smooth boundary in most tasks and the function value is computable using hard-label queries.
Revised: new formulation enjoys the beneﬁt of having a smooth boundary in most tasks and the function value is computable using hard-label queries.

Original: Therefore, the authors of (Cheng et al., 2019) are able to use standard zeroth order optimization to solve the new formulation.
Revised: Therefore, the authors of (Cheng et al., 2019) are able to use standard zeroth order optimization to solve the new formulation.

Original: Although their algorithm converges quickly, it still requires large number of queries (e.g., 20,000) for attacking a single image since every function evaluation of Cheng’s formulation has to be computed using binary search requiring tens of queries.
Revised: Although their algorithm converges quickly, it still requires large number of queries (e.g., 20,000) for attacking a single image since every function evaluation of Cheng’s formulation has to be computed using binary search requiring tens of queries.

Original: In this paper, we follow the same optimization formulation of (Cheng et al., 2019) which has the advantage of smoothness, but instead of using ﬁnite differences to estimate the magnitude of directional derivative, we propose to evaluate its sign using only a single query .
Revised: In this paper, we follow the same optimization formulation of (Cheng et al., 2019) which has the advantage of smoothness, but instead of using ﬁnite differences to estimate the magnitude of directional derivative, we propose to evaluate its sign using only a single query .

Original: With this single-query sign oracle, we design novel algorithms for solving the Cheng’s formulation, and we theoretically prove and empirically demonstrate the signiﬁcant reduction in the number of queries required for hard-label black box attack.
Revised: With this single-query sign oracle, we design novel algorithms for solving the Cheng’s formulation, and we theoretically prove and empirically demonstrate the signiﬁcant reduction in the number of queries required for hard-label black box attack.

Original: Our contribution are summarized below:
Revised: Our contribution are summarized below:

Original: • Novelty in terms of adversarial attack.
Revised: • Novelty in terms of adversarial attack.

Original: We elucidate an efﬁcient approach to compute the sign of directional derivative of Cheng’s formulation using a single query, and based on this technique we develop a novel optimization algorithm called Sign-OPT for hard-label black-box attack.
Revised: We elucidate an efﬁcient approach to compute the sign of directional derivative of Cheng’s formulation using a single query, and based on this technique we develop a novel optimization algorithm called Sign-OPT for hard-label black-box attack.

Original: • Novelty in terms of optimization.
Revised: • Novelty in terms of optimization.

Original: Our method can be viewed as a new zeroth order optimization algorithm that features fast convergence of signSGD.
Revised: Our method can be viewed as a new zeroth order optimization algorithm that features fast convergence of signSGD.

Original: Instead of directly taking the sign of gradient estimation, our algorithm utilizes the scale of random direction.
Revised: Instead of directly taking the sign of gradient estimation, our algorithm utilizes the scale of random direction.

Original: This make existing analysis inappropriate to our case, and we provide a new recipe to prove the convergence of this new optimizer.
Revised: This make existing analysis inappropriate to our case, and we provide a new recipe to prove the convergence of this new optimizer.

Original: • We conduct comprehensive experiments on several datasets and models.
Revised: • We conduct comprehensive experiments on several datasets and models.

Original: We show that the proposed algorithm consistently reduces the query count by 5–10 times across different models and datasets, suggesting a practical and query-efﬁcient robustness evaluation tool.
Revised: We show that the proposed algorithm consistently reduces the query count by 5–10 times across different models and datasets, suggesting a practical and query-efﬁcient robustness evaluation tool.

Original: Furthermore, on most datasets our algorithm can ﬁnd an adversarial example with smaller distortion compared with previous approaches.
Revised: Furthermore, on most datasets our algorithm can ﬁnd an adversarial example with smaller distortion compared with previous approaches.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: White-box attack Since it was ﬁrstly found that neural networks are easy to be fooled by adversarial examples (Goodfellow et al., 2015)
Revised: White-box attack Since it was ﬁrstly found that neural networks are easy to be fooled by adversarial examples (Goodfellow et al., 2015)

Original: , a lot of work has been proposed in the white-box attack setting, where the classiﬁer f is completely exposed to the attacker.
Revised: , a lot of work has been proposed in the white-box attack setting, where the classiﬁer f is completely exposed to the attacker.

Original: For neural networks, under this assumption, back-propagation can be conducted on the target model because both network structure and weights are known by the attacker.
Revised: For neural networks, under this assumption, back-propagation can be conducted on the target model because both network structure and weights are known by the attacker.

Original: Algorithms including (Goodfellow et al., 2015; Kurakin et al., 2016; Carlini & Wagner, 2017; Chen et al., 2018; Madry et al., 2017) are then proposed based on gradient computation.
Revised: Algorithms including (Goodfellow et al., 2015; Kurakin et al., 2016; Carlini & Wagner, 2017; Chen et al., 2018; Madry et al., 2017) are then proposed based on gradient computation.

Original: Recently, the BPDA attack introduced by Athalye et al.
Revised: Recently, the BPDA attack introduced by Athalye et al.

Original: bypasses some models with obfuscated gradients and is shown to successfully circumvent many defenses.
Revised: bypasses some models with obfuscated gradients and is shown to successfully circumvent many defenses.

Revised: In addition to typical attacks based on small (cid:96) p norm perturbation, non- (cid:96) p norm perturbations such as scaling or shifting have also been considered (Zhang et al., 2019).

Original: Black-box attack
Revised: Black-box attack

Original: Recently, black-box setting is drawing rapidly increasing attention.
Revised: Recently, black-box setting is drawing rapidly increasing attention.

Original: In black-box setting, the attacker can query the model but has no (direct) access to any internal information inside the model.
Revised: In black-box setting, the attacker can query the model but has no (direct) access to any internal information inside the model.

Original: Although there are some works based on transfer attack (Papernot et al., 2017), we consider the query-based attack in the paper.
Revised: Although there are some works based on transfer attack (Papernot et al., 2017), we consider the query-based attack in the paper.

Original: Depending on the model’s feedback for a given query, an attack can be classiﬁed as a soft-label or hard-label attack.
Revised: Depending on the model’s feedback for a given query, an attack can be classiﬁed as a soft-label or hard-label attack.

Original: In the soft-label setting, the model outputs a probability score for each decision.
Revised: In the soft-label setting, the model outputs a probability score for each decision.

Original: Chen et al.
Revised: Chen et al.

Original: (2017) uses a ﬁnite difference in a coordinate-wise manner to approximately estimate the output probability changes and does a coordinate descent to conduct the attack.
Revised: (2017) uses a ﬁnite difference in a coordinate-wise manner to approximately estimate the output probability changes and does a coordinate descent to conduct the attack.

Original: Ilyas et al.
Revised: Ilyas et al.

Original: (2018a) uses Neural evolution strategy (NES) to approximately estimate the gradient directly.
Revised: (2018a) uses Neural evolution strategy (NES) to approximately estimate the gradient directly.

Original: Later, some variants (Ilyas et al., 2018b; Tu et al., 2019) were proposed to utilize the side information to further speed up the attack procedure. 
Revised: Later, some variants (Ilyas et al., 2018b; Tu et al., 2019) were proposed to utilize the side information to further speed up the attack procedure. Alzantot et al.

Revised: (2019) uses a evolutionary algorithm as a black-box optimizer for the soft-label setting.

Original: Recently,
Revised: Recently,

Original: Al-Dujaili & O’Reilly (2019) proposes SignHunter algorithm based on signSGD
Revised: Al-Dujaili & O’Reilly (2019) proposes SignHunter algorithm based on signSGD

Original: (Bernstein et al., 2018) to achieve faster convergence in the soft-label setting.
Revised: (Bernstein et al., 2018) to achieve faster convergence in the soft-label setting.

Original: The recent work (Al-Dujaili & O’Reilly, 2019) proposes SignHunter algorithm to achieve a more query-efﬁcent sign estimate when crafting black-box adversarial examples through soft-label information.
Revised: The recent work (Al-Dujaili & O’Reilly, 2019) proposes SignHunter algorithm to achieve a more query-efﬁcent sign estimate when crafting black-box adversarial examples through soft-label information.

Original: In the hard-label case, only the ﬁnal decision, i.e. the top-1 predicted class, is observed.
Revised: In the hard-label case, only the ﬁnal decision, i.e. the top-1 predicted class, is observed.

Original: As a result, the attacker can only make queries to acquire the corresponding hard-label decision instead of the probability outputs.
Revised: As a result, the attacker can only make queries to acquire the corresponding hard-label decision instead of the probability outputs.

Original: Brendel et al.
Revised: Brendel et al.

Original: ﬁrst studied this problem and proposed an algorithm based on random walks near the decision boundary.
Revised: ﬁrst studied this problem and proposed an algorithm based on random walks near the decision boundary.

Original: By selecting a random direction and projecting it onto a boundary sphere in each iteration, it aims to generate a high-quality adversarial example.
Revised: By selecting a random direction and projecting it onto a boundary sphere in each iteration, it aims to generate a high-quality adversarial example.

Original: Query-Limited attack (Ilyas et al., 2018a) tries to estimate the output probability scores with model query and turn the hard-label into a soft-label problem.
Revised: Query-Limited attack (Ilyas et al., 2018a) tries to estimate the output probability scores with model query and turn the hard-label into a soft-label problem.

Original: Cheng et al.
Revised: Cheng et al.

Original: (2019) instead re-formalizes the hard-label attack into an optimization problem that ﬁnds a direction which could produce the shortest distance to decision boundary.
Revised: (2019) instead re-formalizes the hard-label attack into an optimization problem that ﬁnds a direction which could produce the shortest distance to decision boundary.

Original: The recent arXiv paper (Chen et al., 2019) applied the zeroth-order sign oracle to improve Boundary attack, and also demonstrated signiﬁcant improvement.
Revised: The recent arXiv paper (Chen et al., 2019) applied the zeroth-order sign oracle to improve Boundary attack, and also demonstrated signiﬁcant improvement.

Original: The major differences to our algorithm are that we propose a new zeroth-order gradient descent algorithm, provide its algorithmic convergence guarantees, and aim to improve the query complexity of the attack formulation proposed in (Cheng et al., 2019).
Revised: The major differences to our algorithm are that we propose a new zeroth-order gradient descent algorithm, provide its algorithmic convergence guarantees, and aim to improve the query complexity of the attack formulation proposed in (Cheng et al., 2019).

Original: For completeness, we also compare with this method in Section A.1.
Revised: For completeness, we also compare with this method in Section A.1.

Original: Moreover, (Chen et al., 2019) uses one-point gradient estimate, which is unbiased but may encounter larger variance compared with the gradient estimate in our paper.
Revised: Moreover, (Chen et al., 2019) uses one-point gradient estimate, which is unbiased but may encounter larger variance compared with the gradient estimate in our paper.

Original: Thus, we can observe in Section A.1 that although they are slightly faster in the initial stage, Sign-OPT will catch up and eventually lead to a slightly better solution.
Revised: Thus, we can observe in Section A.1 that although they are slightly faster in the initial stage, Sign-OPT will catch up and eventually lead to a slightly better solution.

Original:  P ROPOSED M ETHOD
Revised:  P ROPOSED M ETHOD

Original: We follow the same formulation in (Cheng et al., 2019) and consider the hard-label attack as the problem of ﬁnding the direction with shortest distance to the decision boundary.
Revised: We follow the same formulation in (Cheng et al., 2019) and consider the hard-label attack as the problem of ﬁnding the direction with shortest distance to the decision boundary.

Original: Speciﬁcally, for a given example x 0 , true label y 0 and the hard-label black-box function f : R d → { 1 , . .
Revised: Speciﬁcally, for a given example x 0 , true label y 0 and the hard-label black-box function f : R d → { 1 , . .

Original: , K } , the objective function g : R d → R (for the untargeted attack) can be written as: [Equation] g 
Revised: , K } , the objective function g : R d → R (for the untargeted attack) can be written as: min θ g ( θ ) where g ( θ )

Original: [Equation]
Revised: [Equation]

Original: θ ) where g ( θ ) =

Original: arg min

Original: [Equation]

Original: [Equation]

Original: It has been shown that this objective function is usually smooth and the objective function g can be evaluated by a binary search procedure locally.
Revised: It has been shown that this objective function is usually smooth and the objective function g can be evaluated by a binary search procedure locally.

Original: At each binary search step, we query the function f
Revised: At each binary search step, we query the function f

Original: ( x 0 + λ θ (cid:107) θ (cid:107) ) and determine whether the distance to decision boundary in the direction θ is greater or smaller than λ based on the hard-label prediction 1 .
Revised: ( x 0 + λ θ (cid:107) θ (cid:107) ) and determine whether the distance to decision boundary in the direction θ is greater or smaller than λ based on the hard-label prediction 1 .

Original: As the objective function is computable, the directional derivative of g can be estimated by ﬁnite differences:
Revised: As the objective function is computable, the directional derivative of g can be estimated by ﬁnite differences:

Original: [Equation]
Revised: [Equation]

Original: where u is a random Gaussian vector and (cid:15) > 0 is a very small smoothing parameter.
Revised: where u is a random Gaussian vector and (cid:15) > 0 is a very small smoothing parameter.

Original: This is a standard zeroth order oracle for estimating directional derivative and based on this we can apply many different zeroth order optimization algorithms to minimize g . 
Revised: This is a standard zeroth order oracle for estimating directional derivative and based on this we can apply many different zeroth order optimization algorithms to minimize g . [Figure]

Original: For example, Cheng et al.
Revised: For example, Cheng et al.

Original: (2019) used the Random Derivative Free algorithm Nesterov & Spokoiny (2017) to solve problem (1).
Revised: (2019) used the Random Derivative Free algorithm Nesterov & Spokoiny (2017) to solve problem (1).

Original: However, each computation of (2) requires many hard-label queries due to binary search, so Cheng et al.
Revised: However, each computation of (2) requires many hard-label queries due to binary search, so Cheng et al.

Original: (2019) still requires a huge number of queries despite having fast convergence.
Revised: (2019) still requires a huge number of queries despite having fast convergence.

Original: [Figure]

Original: In this work, we introduce an algorithm that hugely improves the query complexity over Cheng et al.
Revised: In this work, we introduce an algorithm that hugely improves the query complexity over Cheng et al.

Original: Our algorithm is based on the following key ideas: (i) one does not need very accurate values of directional derivative in order to make the algorithm converge, and (ii) there exists an imperfect but informative estimation of directional derivative of g that can be computed by a single query.
Revised: Our algorithm is based on the following key ideas: (i) one does not need very accurate values of directional derivative in order to make the algorithm converge, and (ii) there exists an imperfect but informative estimation of directional derivative of g that can be computed by a single query.

Original: 3. A SINGLE QUERY ORACLE
Revised: 3. A SINGLE QUERY ORACLE

Original: As mentioned before, the previous approach requires computing g ( θ + (cid:15) u ) − g
Revised: As mentioned before, the previous approach requires computing g ( θ + (cid:15) u ) − g

Original: ( θ ) which consumes a lot of queries.
Revised: ( θ ) which consumes a lot of queries.

Original: However, based on the deﬁnition of g ( · ) , we can compute the sign of this value sign ( g ( θ + (cid:15) u ) − g
Revised: However, based on the deﬁnition of g ( · ) , we can compute the sign of this value sign ( g ( θ + (cid:15) u ) − g

Original: ( θ )) using a single query.
Revised: ( θ )) using a single query.

Original: Considering the untargeted attack case, the sign can be computed by
Revised: Considering the untargeted attack case, the sign can be computed by

Original: [Equation]
Revised: [Equation]

Original: This is illustrated in Figure 1.
Revised: This is illustrated in Figure 1.

Original: Essentially, for a new direction θ + (cid:15) u , we test whether a point at the original distance g ( θ ) from x 0 in this direction lies inside or outside the decision boundary, i.e. if the produced perturbation will result in a wrong prediction by classiﬁer.
Revised: Essentially, for a new direction θ + (cid:15) u , we test whether a point at the original distance g ( θ ) from x 0 in this direction lies inside or outside the decision boundary, i.e. if the produced perturbation will result in a wrong prediction by classiﬁer.

Original: If the produced perturbation is outside the boundary i.e. f ( x 0 + g ( θ ) ( θ + (cid:15) u ) (cid:107) θ + (cid:15) u (cid:107) )
Revised: If the produced perturbation is outside the boundary i.e. f ( x 0 + g ( θ ) ( θ + (cid:15) u ) (cid:107) θ + (cid:15) u (cid:107) )

Original: (cid:54) =
Revised: (cid:54) =

Original: y 0 , the new direction has a smaller distance to decision boundary, and thus giving a smaller value of g .
Revised: y 0 , the new direction has a smaller distance to decision boundary, and thus giving a smaller value of g .

Original: It indicates that u is a descent direction to minimize g .
Revised: It indicates that u is a descent direction to minimize g .

Original: 3. S IGN -OPT ATTACK
Revised: 3. S IGN -OPT ATTACK

Original: By sampling random Gaussian vector Q times, we can estimate the imperfect gradient by
Revised: By sampling random Gaussian vector Q times, we can estimate the imperfect gradient by

Original: [Equation]
Revised: [Equation]

Original: (4) which only requires Q queries.
Revised: which only requires Q queries.

Original: We then use this imperfect gradient estimate to update our search direction θ as θ ← θ − η ˆ g with a step size η and use the same search procedure to compute g ( θ ) up to a certain accuracy.
Revised: We then use this imperfect gradient estimate to update our search direction θ as θ ← θ − η ˆ g with a step size η and use the same search procedure to compute g ( θ ) up to a certain accuracy.

Original: The detailed procedure is shown in Algorithm 1.
Revised: The detailed procedure is shown in Algorithm 1.

Original: We note that Liu et al.
Revised: We note that Liu et al.

Original: (2019) designed a Zeroth Order SignSGD algorithm for soft-label black box attack (not hard-label setting).
Revised: (2019) designed a Zeroth Order SignSGD algorithm for soft-label black box attack (not hard-label setting).

Original: They use ˆ ∇ g ( θ )
Revised: They use ˆ ∇ g ( θ )

Original: ≈
Revised: ≈

Original: ˆ g := (cid:80)
Revised: ˆ g := (cid:80)

Original: Qq =1 sign ( g ( θ + (cid:15) u q ) − g
Revised: Qq =1 sign ( g ( θ + (cid:15) u q ) − g

Original: ( θ ) u q ) and shows that it could achieve a comparable or even better convergence rate than zeroth order stochastic gradient descent by using only sign information of gradient estimation.
Revised: ( θ ) u q ) and shows that it could achieve a comparable or even better convergence rate than zeroth order stochastic gradient descent by using only sign information of gradient estimation.

Original: Although it is possible to combine ZO-SignSGD with our proposed single query oracle for solving hard-label attack, their estimator will take sign of the whole vector and thus ignore the direction of u q , which leads to slower convergence in practice (please refer to Section 4.4 and Figure 5(b) for more details).
Revised: Although it is possible to combine ZO-SignSGD with our proposed single query oracle for solving hard-label attack, their estimator will take sign of the whole vector and thus ignore the direction of u q , which leads to slower convergence in practice (please refer to Section 4.4 and Figure 5(b) for more details).

Original: To the best of our knowledge, no previous analysis can be used to prove convergence of Algorithm 1.
Revised: To the best of our knowledge, no previous analysis can be used to prove convergence of Algorithm 1.

Original: In the following, we show that Algorithm 1 can in fact converge and furthermore, with similar convergence rate compared with (Liu et al., 2019) despite using a different gradient estimator.
Revised: In the following, we show that Algorithm 1 can in fact converge and furthermore, with similar convergence rate compared with (Liu et al., 2019) despite using a different gradient estimator.

Original: Assumption 1.
Revised: Assumption 1.

Original: Function g ( θ ) is L-smooth with a ﬁnite value of L.
Revised: Function g ( θ ) is L-smooth with a ﬁnite value of L.

Original: Assumption 2.
Revised: Assumption 2.

Original: At any iteration step t, the gradient of the function g is upper bounded by (cid:107)∇ g [Equation] ≤ σ .
Revised: At any iteration step t, the gradient of the function g is upper bounded by (cid:107)∇ g ( θ t ) (cid:107) 2 ≤ σ .

Original: Theorem 3.1.
Revised: Theorem 3.1.

Original: Suppose that the conditions in the assumptions hold, and the distribution of gradient noise is unimodal and symmetric.
Revised: Suppose that the conditions in the assumptions hold, and the distribution of gradient noise is unimodal and symmetric.

Original: Then, Sign-OPT attack with learning rate η t =
Revised: Then, Sign-OPT attack with learning rate η t =

Original: O ( 1 Q √ dT ) and (cid:15)
Revised: O ( 1 Q √ dT ) and (cid:15)

Original: =
Revised: =

Original: O ( 1 dT ) will give following bound on E [ (cid:107)∇ g ( θ ) (cid:107) 2 ] :
Revised: O ( 1 dT ) will give following bound on E [ (cid:107)∇ g ( θ ) (cid:107) 2 ] :

Original: [Equation]
Revised: [Equation]

Original: The proof can be found in subsection A.2.
Revised: The proof can be found in subsection A.2.

Original: The main difference with the original analysis provided by Liu et al.
Revised: The main difference with the original analysis provided by Liu et al.

Original: (2019) is that they only only deal with sign of each element, while our analysis also takes the magnitudes of each element of u q into account.
Revised: (2019) is that they only only deal with sign of each element, while our analysis also takes the magnitudes of each element of u q into account.

Original: 3. O THER GRADIENT ESTIMATIONS
Revised: 3. O THER GRADIENT ESTIMATIONS

Original: Note that the value sign ( g ( θ + (cid:15) u ) − g ( θ )) computed by our single query oracle is actually the sign of the directional derivative:
Revised: Note that the value sign ( g ( θ + (cid:15) u ) − g ( θ )) computed by our single query oracle is actually the sign of the directional derivative:

Original: [Equation]
Revised: [Equation]

Original: = sign ( lim (cid:15) →∞ g 
Revised: Therefore, we can use this information to estimate the original gradient. The Sign-OPT approach in the previous section uses (cid:80) q sign ( (cid:104)∇ g ( θ ) , u q (cid:105) ) u q as an estimation of gradient. Let y q := sign ( (cid:104)∇ g ( θ ) , u q (cid:105) ) , a more accurate gradient estimation can be cast as the following constraint optimization problem:

Original:  ( θ + (cid:15) u  ) − g
Revised: Find a vector z such that sign ( (cid:104) z , u q (cid:105) ) = y q ∀ q = 1 , . . . , Q. Therefore, this is equivalent to a hard constraint SVM problem where each u q is a training sample and y q is the corresponding label.

Original: ( θ ) (cid:15) )
Revised: The gradient can then be recovered by solving the following quadratic programming problem: [Equation] [Equation] By solving this problem, we can get a good estimation of the gradient.

Original: =
Revised: As explained earlier, each y q can be determined with a single query. Therefore, we propose a variant of Sign-OPT, which is called SVM-OPT attack. The detailed procedure is shown in Algorithm 2.

Original: sign ( g ( θ + (cid:15) u ) − g
Revised: We will present an empirical comparison of our two algorithms in subsection 4.1. Algorithm 2:  E XPERIMENTAL R ESULTS We evaluate the SIGN-OPT algorithm for attacking black-box models in a hard-label setting on three different standard datasets - MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky et al.) and ImageNet-1000 (Deng et al., 2009) and compare it with existing methods.

Revised: For fair and easy comparison, we use the CNN networks provided by (Carlini & Wagner, 2017), which have also been used by other previous hard-label attacks as well.

Revised: Speciﬁcally, for both MNIST and CIFAR-10, the model consists of nine layers in total - four convolutional layers, two max-pooling layers and two fully-connected layers.

Original: θ ))
Revised: Further details about implementation, training and parameters are available on (Carlini & Wagner, 2017). As reported in (Carlini & Wagner, 2017) and (Cheng et al., 2019), we were able to achieve an accuracy of 99.5% on MNIST and 82.5% on CIFAR-10. We use the pretrained Resnet-50 (He et al., 2016) network provided by torchvision (Marcel & Rodriguez, 2010) for ImageNet-1000, which achieves a Top-1 accuracy of 76.15%. In our experiments, we found that Sign-OPT and SVM-OPT perform quite similarly in terms of query efﬁciency.

Original: for a small (cid:15) .
Revised: Hence we compare only Sign-OPT attack with previous approaches and provide a comparison between Sign-OPT and SVM-OPT in subsection 4.1. We compare the following attacks: • Sign-OPT attack (black box): The approach presented in this paper. • Opt-based attack (black box): The method proposed in Cheng et al.

Revised: (2019) where they use Randomized

Original: d =7. n = d =3.
Revised: d =7. n = d =3.

Original: n = d =2.
Revised: n = d =2.

Original: n = d =1. n = d =1.
Revised: n = d =1. n = d =1.

Original: n =
Revised: n =

Original: d =0.
Revised: d =0.

Original: n =
Revised: n =

Original: d =0.
Revised: d =0.

Original: n =
Revised: n =

Original: Original d =7.
Revised: Original d =7.

Original: n = d =3. n = d =3.
Revised: n = d =3. n = d =3.

Original: n = d =3.
Revised: n = d =3.

Original: n = d =2.
Revised: n = d =2.

Original: n = d =1. n = d =1.
Revised: n = d =1. n = d =1.

Original: n =
Revised: n =

Original: Original d =15.
Revised: Original d =15.

Original: n = d =9.
Revised: n = d =9.

Original: n = d =6.
Revised: n = d =6.

Original: n = d =2.
Revised: n = d =2.

Original: n = d =1.
Revised: n = d =1.

Original: n =
Revised: n =

Original: d =0.
Revised: d =0.

Original: n =
Revised: n =

Original: d =0.
Revised: d =0.

Original: n =
Revised: n =

Original: Original d =805.
Revised: Original d =805.

Original: n = d =436.
Revised: n = d =436.

Original: n = d =368.
Revised: n = d =368.

Original: n = d =255.
Revised: n = d =255.

Original: n = d =88.
Revised: n = d =88.

Original: n = d =39.
Revised: n = d =39.

Original: n = d =7. n =
Revised: n = d =7. n =

Original: Original
Revised: Original

Original: Therefore, we can use this information to estimate the original gradient.

Original: The Sign-OPT approach in the previous section uses (cid:80) q sign ( (cid:104)∇ g ( θ ) , u q (cid:105) )

Original: u q as an estimation of gradient.

Original: Let y q := sign ( (cid:104)∇ g ( θ ) , u q (cid:105) ) , a more accurate gradient estimation can be cast as the following constraint optimization problem:

Original: Find

Original: a vector z such that sign ( (cid:104)

Original: z

Original: [Equation]

Original: ) =

Original: y

Original: [Equation]

Original: = 1 ,

Original: [Equation]

Original: [Equation]

Original: Therefore, this is equivalent to a hard constraint SVM problem where each u q is a training sample and y q is the corresponding label.

Original: The gradient can then be recovered by solving the following quadratic programming problem:

Original: [Equation]

Original: By solving this problem, we can get a good estimation of the gradient.

Original: As explained earlier, each y q can be determined with a single query.

Original: Therefore, we propose a variant of Sign-OPT, which is called

Original: SVM-OPT attack.

Original: The detailed procedure is shown in Algorithm 2.

Original: We will present an empirical comparison of our two algorithms in subsection 4.1.

Original: E XPERIMENTAL R ESULTS

Original: We evaluate the SIGN-OPT algorithm for attacking black-box models in a hard-label setting on three different standard datasets - MNIST (LeCun et al., 1998), CIFAR-10 (Krizhevsky et al.) and ImageNet-1000 (Deng et al., 2009) and compare it with existing methods.

Original: For fair and easy comparison, we use the CNN networks provided by (Carlini & Wagner, 2017), which have also been used by other previous hard-label attacks as well.

Original: Speciﬁcally, for both MNIST and CIFAR-10, the model consists of nine layers in total - four convolutional layers, two max-pooling layers and two fully-connected layers.

Original: Further details about implementation, training and parameters are available on (Carlini & Wagner, 2017).

Original: As reported in (Carlini & Wagner, 2017) and (Cheng et al., 2019), we were able to achieve an accuracy of 99.5% on MNIST and 82.5% on CIFAR-10.

Original: We use the pretrained Resnet-50 (He et al., 2016) network provided by torchvision (Marcel & Rodriguez, 2010) for ImageNet-1000, which achieves a Top-1 accuracy of 76.15%.

Original: In our experiments, we found that Sign-OPT and SVM-OPT perform quite similarly in terms of query efﬁciency.

Original: Hence we compare only Sign-OPT attack with previous approaches and provide a comparison between Sign-OPT and SVM-OPT in subsection 4.1.

Original: We compare the following attacks:

Original: • Sign-OPT attack (black box): The approach presented in this paper.

Original: • Opt-based attack (black box): The method proposed in Cheng et al.

Original: (2019) where they use Randomized Gradient-Free method to optimize the same objective function.

Original: We use the implementation provided at https://github.com/LeMinhThong/blackbox-attack .

Original: • Boundary attack (black box): The method proposed in Brendel et al. (2017).
Revised: • Boundary attack (black box): The method proposed in Brendel et al. (2017).

Original: This is compared only in L 2 setting as it is designed for the same.
Revised: This is compared only in L 2 setting as it is designed for the same.

Original: We use the implementation provided in Foolbox ( https://github.com/bethgelab/foolbox ).
Revised: We use the implementation provided in Foolbox ( https://github.com/bethgelab/foolbox ).

Original: • Guessing Smart Attack (black box): The method proposed in (Brunner et al., 2018).
Revised: • Guessing Smart Attack (black box): The method proposed in (Brunner et al., 2018).

Original: This attack enhances boundary attack by biasing sampling towards three priors.
Revised: This attack enhances boundary attack by biasing sampling towards three priors.

Original: Note that one of the priors assumes access to a similar model as the target model and for a fair comparison we do not incorporate this bias in our experiments.
Revised: Note that one of the priors assumes access to a similar model as the target model and for a fair comparison we do not incorporate this bias in our experiments.

Original: We use the implementation provided at https://github.com/ttbrunner/biased_boundary_attack .
Revised: We use the implementation provided at https://github.com/ttbrunner/biased_boundary_attack .

Original: • C&W attack (white box): One of the most popular methods in the white-box setting proposed in Carlini & Wagner (2017).
Revised: • C&W attack (white box): One of the most popular methods in the white-box setting proposed in Carlini & Wagner (2017).

Original: We use C&W L 2 norm attack as a baseline for the white-box attack performance.
Revised: We use C&W L 2 norm attack as a baseline for the white-box attack performance.

Original: For each attack, we randomly sample 100 examples from validation set and generate adversarial perturbations for them.
Revised: For each attack, we randomly sample 100 examples from validation set and generate adversarial perturbations for them.

Original: For untargeted attack, we only consider examples that are correctly predicted by model and for targeted attack, we consider examples that are already not predicted as target label by the model.
Revised: For untargeted attack, we only consider examples that are correctly predicted by model and for targeted attack, we consider examples that are already not predicted as target label by the model.

Original: To compare different methods, we mainly use median distortion as the metric.
Revised: To compare different methods, we mainly use median distortion as the metric.

Original: Median distortion for x queries is the median adversarial perturbation of all examples achieved by a method using less than x queries.
Revised: Median distortion for x queries is the median adversarial perturbation of all examples achieved by a method using less than x queries.

Original: Since all the hard-label attack algorithms will start from an adversarial exmample and keep reduce the distortion, if we stop at any time they will always give an adversarial example and medium distortion will be the most suitable metric to compare their performance.
Revised: Since all the hard-label attack algorithms will start from an adversarial exmample and keep reduce the distortion, if we stop at any time they will always give an adversarial example and medium distortion will be the most suitable metric to compare their performance.

Original: Besides, we also show success rate (SR) for x queries for a given threshold ( (cid:15) ), which is the percentage of number of examples that have achieved an adversarial perturbation below (cid:15) with less than x queries.
Revised: Besides, we also show success rate (SR) for x queries for a given threshold ( (cid:15) ), which is the percentage of number of examples that have achieved an adversarial perturbation below (cid:15) with less than x queries.

Original: We evaluate success rate on different thresholds which depend on the dataset being used.
Revised: We evaluate success rate on different thresholds which depend on the dataset being used.

Original: For comparison of different algorithms in each setting, we chose the same set of examples across all attacks.
Revised: For comparison of different algorithms in each setting, we chose the same set of examples across all attacks.

Original: Implementation details : To optimize algorithm 1, we estimate the step size η using the same line search procedure implemented in Cheng et al.
Revised: Implementation details : To optimize algorithm 1, we estimate the step size η using the same line search procedure implemented in Cheng et al.

Original: At the cost of a relatively small number of queries, this provides signiﬁcant speedup in the optimization.
Revised: At the cost of a relatively small number of queries, this provides signiﬁcant speedup in the optimization.

Original: Similar to Cheng et al.
Revised: Similar to Cheng et al.

Original: (2019), g ( θ ) in last step of algorithm 1 is approximated via binary search.
Revised: (2019), g ( θ ) in last step of algorithm 1 is approximated via binary search.

Revised: The initial θ 0 in algorithm 1 is calculated

Original: The initial θin algorithm 1 is calculated by evaluating g ( θ ) on 100 random directions and taking the best one.
Revised: [Figure] r [Figure] by evaluating g ( θ ) on 100 random directions and taking the best one.

Original: We provide our implementation publicly at https://drive.google.com/file/d/ 1ObXfAjAHm9FJfLy41TJfTlgQ4fSiP8Eq .
Revised: We provide our implementation publicly 2 .

Original: 4. C OMPARISON BETWEEN S IGN -OPT AND SVM-OPT
Revised: 4. C OMPARISON BETWEEN S IGN -OPT AND SVM-OPT

Original: In our experiments, we found that the performance in terms of queries of both these attacks is remarkably similar in all settings (both L 2 / L ∞ & Targeted/Untargeted) and datasets.
Revised: In our experiments, we found that the performance in terms of queries of both these attacks is remarkably similar in all settings (both L 2 / L ∞ & Targeted/Untargeted) and datasets.

Original: We present a comparison for MNIST and CIFAR-10 ( L 2 norm-based) for both targeted and untargeted attacks in
Revised: We present a comparison for MNIST and CIFAR-10 ( L 2 norm-based) for both targeted and untargeted attacks in

Original: Figure 3.
Revised: Figure 3.

Original: We see that the median distortion achieved for a given number of queries is quite on part for both Sign-OPT and SVM-OPT.
Revised: We see that the median distortion achieved for a given number of queries is quite on part for both Sign-OPT and SVM-OPT.

Original: Number of queries per gradient estimate :
Revised: Number of queries per gradient estimate :

Original: In Figure 3, we show the comparison of Sign-OPT attack with different values of Q .
Revised: In Figure 3, we show the comparison of Sign-OPT attack with different values of Q .

Original: Our experiments suggest that Q does not have an impact on the[Figure] [Figure] convergence point reached by the algorithm.
Revised: Our experiments suggest that Q does not have an impact on the convergence point reached by the algorithm.

Original: Although, small values of Q provide a noisy gradient estimate and hence delayed convergence to an adversarial perturbation.
Revised: Although, small values of Q provide a noisy gradient estimate and hence delayed convergence to an adversarial perturbation.

Original: Large values of Q , on the other hand, require large amount of time per gradient estimate.
Revised: Large values of Q , on the other hand, require large amount of time per gradient estimate.

Original: After ﬁne tuning on a small set of examples [Figure] we found that Q = 200 provides a good balance between the two.
Revised: After ﬁne tuning on a small set of examples, we found that Q = 200 provides a good balance between the two.

Original: Hence, we set the value of Q = 200 for all our experiments in this section [Figure]
Revised: Hence, we set the value of Q = 200 for all our experiments in this section.

Original: 4. U NTARGETED ATTACK
Revised: 4. U NTARGETED ATTACK

Original: In this attack [Figure] the objective is to generate an adversary from an original image for which the prediction by model is different from that of original image.
Revised: In this attack, the objective is to generate an adversary from an original image for which the prediction by model is different from that of original image.

Original: Figure 4 provides an elaborate comparison of different attacks for L 2 case for the three datasets.
Revised: Figure 4 provides an elaborate comparison of different attacks for L 2 case for the three datasets.

Original: Sign-OPT attack consistently outperforms the current approaches in terms of queries.
Revised: Sign-OPT attack consistently outperforms the current approaches in terms of queries.

Original: Not only is Sign-OPT more efﬁcient in terms of queries, in most cases it converges to a lower distortion than what is possible by other hard-label attacks.
Revised: Not only is Sign-OPT more efﬁcient in terms of queries, in most cases it converges to a lower distortion than what is possible by other hard-label attacks.

Original: Furthermore, we observe Sign-OPT converges to a solution comparable with C&W white-box attack (better on CIFAR-10, worse on MNIST, comparable on ImageNet).
Revised: Furthermore, we observe Sign-OPT converges to a solution comparable with C&W white-box attack (better on CIFAR-10, worse on MNIST, comparable on ImageNet).

Original: This is signiﬁcant for a hard-label attack algorithm since we are given very limited information.
Revised: This is signiﬁcant for a hard-label attack algorithm since we are given very limited information.

Original: We highlight some of the comparisons of Boundary attack, OPT-based attack and Sign-OPT attack ( L 2 norm-based) in Table 1.
Revised: We highlight some of the comparisons of Boundary attack, OPT-based attack and Sign-OPT attack ( L 2 norm-based) in Table 1.

Original: Particularly for ImageNet dataset on ResNet-50 model, Sign-OPT attack reaches a median distortion below 3.0 in less than 30 k queries while other attacks need more than 200 k queries for the same.
Revised: Particularly for ImageNet dataset on ResNet-50 model, Sign-OPT attack reaches a median distortion below 3.0 in less than 30 k queries while other attacks need more than 200 k queries for the same.

Original: 4. T ARGETED ATTACK
Revised: 4. T ARGETED ATTACK

Original: In targeted attack, the goal is to generate an adversarial perturbation for an image so that the prediction of resulting image is the same as a speciﬁed target.
Revised: In targeted attack, the goal is to generate an adversarial perturbation for an image so that the prediction of resulting image is the same as a speciﬁed target.

Original: For each example, we randomly specify the target label, keeping it consistent across different attacks.
Revised: For each example, we randomly specify the target label, keeping it consistent across different attacks.

Original: We calculate the initial θ 0 in algorithmusing 100 samples in target label class from training dataset and this θ 0 is the same across different attacks.
Revised: We calculate the initial θ 0 in algorithmusing 100 samples in target label class from training dataset and this θ 0 is the same across different attacks.

Original: Figure 2 shows some examples of adversarial examples generated by Sign-OPT attack and the Opt-based attack.
Revised: Figure 2 shows some examples of adversarial examples generated by Sign-OPT attack and the Opt-based attack.

Original: The ﬁrst two rows show comparison of Sign-OPT and Opt attack respectively on an example from MNIST dataset.
Revised: The ﬁrst two rows show comparison of Sign-OPT and Opt attack respectively on an example from MNIST dataset.

Original: The ﬁgures show adversarial examples generated at almost  same number of queries for both attacks.
Revised: The ﬁgures show adversarial examples generated at almost [Figure] [Figure] same number of queries for both attacks.

Original: Sign-OPT method generates an L 2 adversarial perturbation of 0.94 in ∼ 6 k queries for this particular example while Opt-based attack requires ∼ 35 k for the [Figure] [Figure]
Revised: Sign-OPT method generates an L 2 adversarial perturbation of 0.94 in ∼ 6 k queries for this particular example while Opt-based attack requires ∼ 35 k for the same.

Original: Figure 5 displays a comparison among different attacks in targeted setting.
Revised: Figure 5 displays a comparison among different attacks in targeted setting.

Original: In our experiments, average distortion achieved by white box attack C&W for MNIST dataset is 1.51, for which Sign-OPT requires ∼ 12 k queries while others need > 120 k queries.
Revised: In our experiments, average distortion achieved by white box attack C&W for MNIST dataset is 1.51, for which Sign-OPT requires ∼ 12 k queries while others need > 120 k queries.

Original: We present a comparison of success rate of different attacks for CIFAR-10 dataset in Figure 6 for both targeted and untargeted cases.
Revised: We present a comparison of success rate of different attacks for CIFAR-10 dataset in Figure 6 for both targeted and untargeted cases.

Original: 4. T HE POWER OF SINGLE QUERY ORACLE
Revised: 4. T HE POWER OF SINGLE QUERY ORACLE

Original: In this subsection, we conduct several experiments to prove the effectiveness of our proposed single query oracle in hard-label adversarial attack setting.
Revised: In this subsection [Figure] we conduct several experiments to prove the effectiveness of our proposed single query oracle in hard-label adversarial attack setting.

Original: ZO-SignSGD algorithm
Revised: ZO-SignSGD algorithm

Original: (Liu et al., 2019) is proposed for soft-label black box attack and we extend it into hard-label setting.
Revised: (Liu et al., 2019) is proposed for soft-label black box attack and we extend it into hard-label setting.

Original: A straightforward way is simply applying ZO-SignSGD to solve the hard-label objective proposed in Cheng et al.
Revised: A straightforward way is simply applying ZO-SignSGD to solve the hard-label objective proposed in Cheng et al.

Original: (2019), estimate the gradient using binary search as (Cheng et al., 2019) and take its sign.
Revised: (2019), estimate the gradient using binary search as (Cheng et al., 2019) and take its sign.

Original: In Figure 5(b), we clearly observe that simply combining ZO-SignSGD and Cheng et al.
Revised: In Figure 5(b) [Figure] we clearly observe that simply combining ZO-SignSGD and Cheng et al.

Original: (2019) is not efﬁcient.
Revised: (2019) is not efﬁcient.

Original: With the proposed single query sign oracle, we can also reduce the query count of this method, as demonstrated in Figure 5(b).
Revised: With the proposed single query sign oracle [Figure] we can also reduce the query count of this method, as demonstrated in Figure 5(b).

Original: This veriﬁes the effectiveness of single query oracle, which can universally improve many different optimization methods in the hard-label attack setting.
Revised: This veriﬁes the effectiveness of single query oracle, which can universally improve many different optimization methods in the hard-label attack setting.

Original: To be noted, there is still improvement on Sign-OPT over ZO-SignSGD with single query oracle because instead  of directly taking the sign of gradient estimation, our algorithm utilizes the scale of random direction u as well.
Revised: To be noted, there is still improvement on Sign-OPT over ZO-SignSGD with single query oracle because instead [Figure] [Table] of directly taking the sign of gradient estimation, our algorithm utilizes the scale of random direction u as well.

Original: In other words, signSGD’s gradient norm is always 1 while our gradient norm takes into account the magnitude of u .
Revised: In other words, signSGD’s gradient norm is always 1 while our gradient norm takes into account the magnitude of u .

Original: Therefore, our signOPT optimization algorithm is fundamentally different (Liu et al., 2019) or any other proposed signSGD varieties.
Revised: Therefore, our signOPT optimization algorithm is fundamentally different (Liu et al., 2019) or any other proposed signSGD varieties.

Original: Our method can be viewed as a new zeroth order optimization algorithm that features fast convergence in signSGD.
Revised: Our method can be viewed as a new zeroth order optimization algorithm that features fast convergence in signSGD.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We developed a new and ultra query-efﬁcient algorithm for adversarial attack in the hard-label black-box setting.
Revised: We developed a new and ultra query-efﬁcient algorithm for adversarial attack in the hard-label black-box setting.

Original: Using the same smooth reformulation in Cheng et al.
Revised: Using the same smooth reformulation in Cheng et al.

Original: (2019), we design a novel zeroth order oracle that can compute the sign of directional derivative of the attack objective using single query.
Revised: (2019), we design a novel zeroth order oracle that can compute the sign of directional derivative of the attack objective using single query.

Original: Equipped with this single-query oracle, we design a new optimization algorithm that can dramatically reduce number of queries compared with Cheng et al.
Revised: Equipped with this single-query oracle, we design a new optimization algorithm that can dramatically reduce number of queries compared with Cheng et al.

Original: We prove the convergence of the proposed algorithm and show our new algorithm is overwhelmingly better than current hard-label black-box attacks.
Revised: We prove the convergence of the proposed algorithm and show our new algorithm is overwhelmingly better than current hard-label black-box attacks.

Original: [Table]
Revised: This work is based upon work supported by the Department of Energy National Energy Technology Laboratory under Award Number DE-OE0000911 and by NSF under IIS1719097.

Original: A A PPENDIX A. C OMPARISON WITH H OP S KIP J UMP A TTACK
Revised: A A PPENDIX A. C OMPARISON WITH H OP S KIP J UMP A TTACK

Original: There is a recent paper (Chen et al., 2019) that applied the zeroth-order sign oracle to improve
Revised: There is a recent paper (Chen et al., 2019) that applied the zeroth-order sign oracle to improve

Original: Boundary attack, and also demonstrated signiﬁcant improvement.
Revised: Boundary attack, and also demonstrated signiﬁcant improvement.

Original: The major differences to our algorithm are that we propose a new zeroth-order gradient descent algorithm, provide its algorithmic convergence guarantees, and aim to improve the query complexity of the attack formulation proposed in (Cheng et al., 2019).
Revised: The major differences to our algorithm are that we propose a new zeroth-order gradient descent algorithm, provide its algorithmic convergence guarantees, and aim to improve the query complexity of the attack formulation proposed in (Cheng et al., 2019).

Original: To be noted, HopSkipJumpAttack only provides the bias and variance analysis (Theorem 2 and 3) without convergence rate analysis.
Revised: To be noted, HopSkipJumpAttack only provides the bias and variance analysis (Theorem 2 and 3) without convergence rate analysis.

Original: Also, HopSkipJumpAttack uses one-point gradient estimate compared to the 2-point gradient estimate used by SignOPT.
Revised: Also, HopSkipJumpAttack uses one-point gradient estimate compared to the 2-point gradient estimate used by SignOPT.

Original: Therefore, although the estimation is unbiased, it has large variance, which achieves successful attack faster but generates a worse adversarial example with larger distortion than ours.
Revised: Therefore, although the estimation is unbiased, it has large variance, which achieves successful attack faster but generates a worse adversarial example with larger distortion than ours.

Original: For completeness, we also compare with this method (and mention the results) as follows.
Revised: For completeness, we also compare with this method (and mention the results) as follows.

Original: Figure 7 shows a comparison of Sign-OPT and HopSkipJumpAttack for CIFAR-10 and MNIST datasets for the case of L 2 norm based attack.
Revised: Figure 7 shows a comparison of Sign-OPT and HopSkipJumpAttack for CIFAR-10 and MNIST datasets for the case of L 2 norm based attack.

Original: We ﬁnd in our experiments that performance of both attacks is comparable in terms of queries consumed.
Revised: We ﬁnd in our experiments that performance of both attacks is comparable in terms of queries consumed.

Original: In some cases, Sign-OPT converges to a better solution.
Revised: In some cases, Sign-OPT converges to a better solution.

Original: [Figure]
Revised: [Figure]

Original: Median distortion vs
Revised: distortion vs

Original: Queries.
Revised: Queries.

Original: (U) represents untargeted attack and (T) represents targeted attack.
Revised: (U) represents untargeted attack and (T) represents targeted attack.

Original: A. P ROOF
Revised: A. P ROOF

Original: Deﬁne following notations:
Revised: Deﬁne following notations:

Original: [Equation]
Revised: [Equation]

Original: Thus we could write the corresponding estimate of gradients as follow:
Revised: Thus we could write the corresponding estimate of gradients as follow:

Original: [Equation]
Revised: [Equation]

Original: Clearly, we have ¯ ∇ g ( θ t ; u q ) =
Revised: Clearly, we have ¯ ∇ g ( θ t ; u q ) =

Original: sign ( ˙ ∇ g ( θ t ; u q )) and we could relate ¯ ∇ g ( θ t ; u q ) and ˆ ∇ g ( θ t ; u q ) by writing ˆ ∇ g ( θ t ; u q ) =
Revised: sign ( ˙ ∇ g ( θ t ; u q )) and we could relate ¯ ∇ g ( θ t ; u q ) and ˆ ∇ g ( θ t ; u q ) by writing ˆ ∇ g ( θ t ; u q ) =

Original: G q (cid:12) ¯ ∇ g ( θ t ; u q ) where where G q ∈ R d is absolute value of vector u q (i.e.
Revised: G q (cid:12) ¯ ∇ g ( θ t ; u q ) where where G q ∈ R d is absolute value of vector u q (i.e.

Original: G q = ( | u q, 1 | 
Revised: G q = ( | u q, 1 | ,

Original: Note that Zeroth-order gradient estimate ˙ ∇ g ( θ t ; u q ) is a biased approximation to the true gradient of g. Instead, it becomes unbiased to the gradient of the randomized smoothing function g (cid:15) ( θ ) =
Revised: Note that Zeroth-order gradient estimate ˙ ∇ g ( θ t ; u q ) is a biased approximation to the true gradient of g. Instead, it becomes unbiased to the gradient of the randomized smoothing function g (cid:15) ( θ ) =

Original: E u
Revised: E u

Original: [ g ( θ + (cid:15) u )] Duchi et al.
Revised: [ g ( θ + (cid:15) u )] Duchi et al.

Original: Our analysis is based on the following two assumptions:
Revised: Our analysis is based on the following two assumptions:

Original: Assumption 1 function g is L-smooth with a ﬁnite value of L.
Revised: Assumption 1 function g is L-smooth with a ﬁnite value of L.

Original: Assumption 2 At any iteration step t, the gradient of the function g is upper bounded by
Revised: Assumption 2 At any iteration step t, the gradient of the function g is upper bounded by

Original: [Equation]
Revised: [Equation]

Original: To prove the convergence of proposed method, we need the information on variance of the update ˙ ∇ g ( θ t ; u q ) .
Revised: To prove the convergence of proposed method, we need the information on variance of the update ˙ ∇ g ( θ t ; u q ) .

Original: Here, we introduce a lemma from previous works.
Revised: Here, we introduce a lemma from previous works.

Original: Lemma 1 The variance of Zeroth-Order gradient estimate ˙ ∇ g ( θ t ; u q ) is upper bounded by
Revised: Lemma 1 The variance of Zeroth-Order gradient estimate ˙ ∇ g ( θ t ; u q ) is upper bounded by

Original: [Equation]
Revised: [Equation]

Original: where C ( d, (cid:15) ) :
Revised: where C ( d, (cid:15) ) :

Original: = 2 dσ 2 + (cid:15) 2 L 2 d 2 /
Revised: = 2 dσ 2 + (cid:15) 2 L 2 d 2 /

Original: Proof of Lemma 

Revised: Proof of Lemma

Original: This lemma could be proved by using proposition 2 in Liu et al.
Revised: This lemma could be proved by using proposition 2 in Liu et al.

Original: with b = 1 and q = Q.
Revised: with b = 1 and q = Q.

Original: When b = 1 there is no difference between with/without replacement, and we opt for with replacement case to obtain above bound.
Revised: When b = 1 there is no difference between with/without replacement, and we opt for with replacement case to obtain above bound.

Original: By talking Q = 1, we know that E (cid:2) (cid:107) ˙
Revised: By talking Q = 1, we know that E (cid:2) (cid:107) ˙

Original: ∇ g ( θ t ; u q ) − ∇ g (cid:15) ( θ t ) (cid:13)(cid:13) 2 2 ] is upper bounded.
Revised: ∇ g ( θ t ; u q ) − ∇ g (cid:15) ( θ t ) (cid:13)(cid:13) 2 2 ] is upper bounded.

Original: And by Jensen’s inequality, we also know that the
Revised: And by Jensen’s inequality, we also know that the

Original: [Equation]
Revised: [Equation]

Original: where δ l denotes the upper bound of lth coordinate of E (cid:2)
Revised: where δ l denotes the upper bound of lth coordinate of E (cid:2)

Original: | ˙ ∇ g ( θ t ; u q ) − ∇ g (cid:15) ( θ t ) | (cid:3) , and δ l is ﬁnite since E (cid:2) (cid:107) ˙
Revised: | ˙ ∇ g ( θ t ; u q ) − ∇ g (cid:15) ( θ t ) | (cid:3) , and δ l is ﬁnite since E (cid:2) (cid:107) ˙

Original: ∇ g ( θ t ; u q ) − ∇ g (cid:15) ( θ t ) (cid:13)(cid:13) 2 2 ] is upper bounded.
Revised: ∇ g ( θ t ; u q ) − ∇ g (cid:15) ( θ t ) (cid:13)(cid:13) 2 2 ] is upper bounded.

Original: Next, we want to show the Prob [ sign ((¯ g t ) l ) (cid:54)
Revised: Next, we want to show the Prob [ sign ((¯ g t ) l ) (cid:54)

Original: =
Revised: =

Original: sign (( ∇ g (cid:15) ( θ t )) l )] by following lemma.
Revised: sign (( ∇ g (cid:15) ( θ t )) l )] by following lemma.

Original: Lemma
Revised: Lemma

Original: [Equation]
Revised: [Equation]

Original: Proof of Lemma 2 Similar to Bernstein et al.
Revised: Proof of Lemma 2 Similar to Bernstein et al.

Original: (2018), we ﬁrst relax Prob [ sign (( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) =
Revised: (2018), we ﬁrst relax Prob [ sign (( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) =

Original: sign ( ∇ g (cid:15) ( θ t )) l ] by Markov inequality:
Revised: sign ( ∇ g (cid:15) ( θ t )) l ] by Markov inequality:

Revised: Prob [

Original: [Equation]
Revised: [Equation]

Original: where the last inequality comes from eq (6).
Revised: where the last inequality comes from eq (6).

Original: Recall that ( ˙ ∇ g ( θ t ; u q )) l ) is an unbiased estimation to ( ∇ g (cid:15) ( θ t )) l .
Revised: Recall that ( ˙ ∇ g ( θ t ; u q )) l ) is an unbiased estimation to ( ∇ g (cid:15) ( θ t )) l .

Original: Under the assumption that the noise distribution is unimodal and symmetric, from
Revised: Under the assumption that the noise distribution is unimodal and symmetric, from

Original: Bernstein et al.
Revised: Bernstein et al.

Original: Lemma D1, we will have
Revised: Lemma D1, we will have

Original: [Equation]
Revised: [Equation]

Original: where S :
Revised: where S :

Original: =
Revised: =

Original: |∇ g (cid:15) ( θ t ) l | /δ l .
Revised: |∇ g (cid:15) ( θ t ) l | /δ l .

Original: Note that this probability bound applies uniformly to all q ∈ Q regardless of the magnitude | ( u q ) l | .
Revised: Note that this probability bound applies uniformly to all q ∈ Q regardless of the magnitude | ( u q ) l | .

Original: That is,
Revised: That is,

Original: [Equation]
Revised: [Equation]

Original: This is true as when all | ( u q ) l | = 1, Prob [ sign (( (cid:80) Qq =1 sign ( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) =
Revised: This is true as when all | ( u q ) l | = 1, Prob [ sign (( (cid:80) Qq =1 sign ( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) =

Original: sign ( ∇ g (cid:15) ( θ t )) l ] is equivalent to majority voting of each estimate q yielding correct sign.
Revised: sign ( ∇ g (cid:15) ( θ t )) l ] is equivalent to majority voting of each estimate q yielding correct sign.

Original: This is the same as sum of Q bernoulli trials (i.e. binomial distribution) with error rate M.
Revised: This is the same as sum of Q bernoulli trials (i.e. binomial distribution) with error rate M.

Original: And since error probability M is independent of sampling of | ( u q ) l | , calculating Prob [ sign ( (cid:80) Qq = [Equation] | ( u q ) [Equation]{+=
+}
Revised: And since error probability M is independent of sampling of | ( u q ) l | , calculating Prob [ sign ( (cid:80) Qq {+=
+}

Original:  sign (( [Equation] g ( [Equation]
Revised: | ( u q ) l | sign (( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) =

Original: u

Original: [Equation] )) l ) (cid:54) = sign ( ∇ g (cid:15) ( [Equation] )) l ] could be thought as taking Q bernoulli experiments and then independently draw a weight from unit length for each of Q experiment.
Revised: sign ( ∇ g (cid:15) ( θ t )) l ] could be thought as taking Q bernoulli experiments and then independently draw a weight from unit length for each of Q experiment.

Original: Since the weight is uniform, we will have expectation of weights on correct counts and incorrect counts are the same and equal to 1/2.
Revised: Since the weight is uniform, we will have expectation of weights on correct counts and incorrect counts are the same and equal to 1/2.

Original: Therefore, the probability of Prob [ sign ( (cid:80) Q q =1 | ( u q ) l | sign (( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) = sign ( ∇ g (cid:15) ( θ t )) l ] is still the same as original non-weighted binomial distribution.
Revised: Therefore, the probability of Prob [ sign ( (cid:80) Q q =1 | ( u q ) l | sign (( ˙ ∇ g ( θ t ; u q )) l ) (cid:54) = sign ( ∇ g (cid:15) ( θ t )) l ] is still the same as original non-weighted binomial distribution.

Original: Notice that by our notation, we will have sign ( ˙ ∇ g ( θ t ; u q ) l ) =
Revised: Notice that by our notation, we will have sign ( ˙ ∇ g ( θ t ; u q ) l ) =

Original: ¯ ∇ g ( θ t ; u q ) l thus 1 Q (cid:80)
Revised: ¯ ∇ g ( θ t ; u q ) l thus 1 Q (cid:80)

Original: Qq =1 sign ( ˙ ∇ g ( θ t ; u q ))
Revised: Qq =1 sign ( ˙ ∇ g ( θ t ; u q ))

Original: l = (¯ g t ) l .
Revised: l = (¯ g t ) l .

Original: Let Z counts the number of estimates ˙
Revised: Let Z counts the number of estimates ˙

Original: ∇ g ( θ t ; u q ) l yielding correct sign of ∇ g (cid:15) ( θ t ) l .
Revised: ∇ g ( θ t ; u q ) l yielding correct sign of ∇ g (cid:15) ( θ t ) l .

Original: Probability in eq (7) could be written as:
Revised: Probability in eq (7) could be written as:

Original: [Equation]
Revised: [Equation]

Original: Following the derivation of theorem 2b in Bernstein et al.
Revised: Following the derivation of theorem 2b in Bernstein et al.

Original: , we could get
Revised: , we could get

Original: [Equation]
Revised: [Equation]

Original: We also need few more lemmas on properties of function g.
Revised: We also need few more lemmas on properties of function g.

Original: Lemma 3 g (cid:15) ( θ 1 ) − g (cid:15) ( θ T ) ≤ g (cid:15) ( θ 1 ) − g ∗ + (cid:15) 2 L
Revised: Lemma 3 g [Equation]

Original: Proof of Lemma 

Revised: Proof of Lemma

Original: The proof can be found in Liu et al.
Revised: The proof can be found in Liu et al.

Original: Lemma C.
Revised: Lemma C.

Original: Lemma 4 E
Revised: Lemma 4 E

Original: [ (cid:107)∇ g ( θ ) (cid:107) 2 ] ≤ √ 2 E [ (cid:107)∇ g (cid:15)
Revised: [ (cid:107)∇ g ( θ ) (cid:107) 2 ] ≤ √ 2 E [ (cid:107)∇ g (cid:15)

Original: ( θ ) (cid:107) 2 ] + (cid:15)Ld √ 2 , where g ∗ =
Revised: ( θ ) (cid:107) 2 ] + (cid:15)Ld √ 2 , where g ∗ =

Original: min θ g ( θ ) .
Revised: min θ g ( θ ) .

Original: Proof of Lemma 4 The proof can be found in Liu et al.
Revised: Proof of Lemma 4 The proof can be found in Liu et al.

Original: Theorem 1 Suppose that the conditions in the assumptions hold, and the distribution of gradient noise is unimodal and symmetric.
Revised: Theorem 1 Suppose that the conditions in the assumptions hold, and the distribution of gradient noise is unimodal and symmetric.

Original: Then, Sign-OPT attack with learning rate η t =
Revised: Then, Sign-OPT attack with learning rate η t =

Original: O ( 1 Q √ dT ) and (cid:15)
Revised: O ( 1 Q √ dT ) and (cid:15)

Original: =
Revised: =

Original: O ( 1 dT ) will give following bound on E [ (cid:107)∇ g ( θ ) 
Revised: O ( 1 dT ) will give following bound on E [ (cid:107)∇ g ( θ ) (cid:107)

Original: [Equation]
Revised: [Equation]

Original: Proof of Theorem 1 From L-smoothness assumption we could have
Revised: Proof of Theorem 1 From L-smoothness assumption we could have

Original: [Equation]
Revised: [Equation]

Original: where ¯ G t is deﬁned as ( ¯ G t ) l = (cid:80) Qq =
Revised: where ¯ G t is deﬁned as ( ¯ G t ) l = (cid:80) Qq =1 ( G q )

Original: [Equation]
Revised: [Equation]

Original: G q )
Revised: ¯

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: g

Original: [Equation]
Revised: [Equation]

Original: =
Revised: ; u

Original: [Equation]
Revised: [Equation]

Original: Qq =
Revised: ) l =

Original: [Equation]
Revised: [Equation]

Original: |
Revised: Qq =

Original: [Equation]
Revised: [Equation]

Original:  u q ) l | ¯ ∇ g ( θ t ; u q ) l .
Revised: | ( u q ) l | ¯ ∇ g ( θ t ; u q ) l .

Original: Continue the inequality,
Revised: Continue the inequality,

Original: [Equation]
Revised: [Equation]

Original: √

Original: Q by eq (8)
Revised: Q by eq (8)

Original: [Equation]
Revised: [Equation]

Original: Q
Revised: Q

Original: [Equation]
Revised: [Equation]

Original: Q by eq (6) .
Revised: Q by eq (6) .

Original: Thus we will have,
Revised: Thus we will have,

Original: [Equation]
Revised: [Equation]

Original: Q
Revised: Q

Original: [Equation]
Revised: [Equation]

Original: Q ⇒
Revised: Q ⇒

Original: [Equation]
Revised: [Equation]

Original: √ Q , where we deﬁne ˆ η t :=
Revised: Q , where we deﬁne ˆ η t :=

Original: η t (cid:12) ¯ G t .
Revised: η t (cid:12) ¯ G t .

Original: Sum up all inequalities for all ts and take expectation on both side, we will have
Revised: Sum up all inequalities for all ts and take expectation on both side, we will have

Original: T(cid:88) t =
Revised: T

Original: [Equation]
Revised: [Equation]

Original: by Lemma 1 .
Revised: by Lemma 1 .

Original: Substitute Lemma 3 into above inequality, we get
Revised: Substitute Lemma 3 into above inequality, we get

Revised: T (cid:88)

Original: [Equation]
Revised: [Equation]

Original: Since (cid:107) · (cid:107) 2 ≤ (cid:107) · (cid:107) 

Revised: Since (cid:107) · (cid:107) 2 ≤ (cid:107) · (cid:107)

Original: and we could divide (cid:80)
Revised: and we could divide (cid:80)

Original: Tt =
Revised: Tt =

Original: 1 ˆ η t on both side to get
Revised: 1 ˆ η t on both side to get

Original: T (cid:88)
Revised: T (cid:88)

Original: [Equation]
Revised: [Equation]

Original: Deﬁne a new random variable R with probability P
Revised: Deﬁne a new random variable R with probability P

Original: ( R = t ) =
Revised: ( R = t ) =

Revised: [Equation]

Revised: Tt

Original: [Equation]
Revised: [Equation]

Original: t , we will have
Revised: t , we will have

Original: [Equation]
Revised: [Equation]

Original: Substitute all the quantities into Lemma 4, we will get
Revised: Substitute all the quantities into Lemma 4, we will get

Original: E [ (cid:107)∇ g ( θ ) [Equation] ≤
Revised: E [ (cid:107)∇ g ( θ ) (cid:107) 2 ] ≤

Original: [Equation]
Revised: [Equation]

Original: By choosing (cid:15) =
Revised: By choosing (cid:15) =

Original: O ( 1 dT ) and η t =
Revised: O ( 1 dT ) and η t =

Original: O ( 1 Q √ dT ) , then the convergence rate as shown in above
Revised: O ( 1 Q √ dT ) , then the convergence rate as shown in above

Original: is
Revised: is

Original: O ( dT + d √ Q ) .
Revised: O ( dT + d √ Q ) .