QUERY EFFICIENT DECISION BASED SPARSE ATTACKS AGAINST BLACK-BOX DEEP LEARNING MODELS
Authors: Viet Vo
Ehsan M Abbasnejad
Damith Ranasinghe
Venue: ICLR 2022 Poster (2022)
Keywords: decision-based attacks
sparse attacks
evolution algorithms
vision transformer
convolutional neural network

Abstract: Despite our best efforts, deep learning models remain highly vulnerable to even tiny adversarial perturbations applied to the inputs. The ability to extract information from solely the output of a machine learning model to craft adversarial perturbations to black-box models is a practical threat against real-world systems, such as Machine Learning as a Service (MLaaS), particularly $sparse~attacks$. The realization of sparse attacks in black-box settings demonstrates that machine learning models are more vulnerable than we believe. Because, these attacks aim to $minimize~the~number~of~perturbed~pixels$—measured by $l_0$ norm—required to mislead a model by $solely$ observing the decision ($the~predicted~label$) returned to a model query; the so-called $decision-based~setting$. But, such an attack leads to an NP-hard optimization problem. We develop an evolution-based algorithm—$SparseEvo$—for the problem and evaluate it against both convolutional deep neural networks and $vision~transformers$. Notably, vision transformers are yet to be investigated under a decision-based attack setting. SparseEvo requires significantly fewer queries than the state-of-the-art sparse attack $Pointwise$ for both untargeted and targeted attacks. The attack algorithm, although conceptually simple, is competitive with only a limited query budget against the state-of-the-art gradient-based $white-box$ attacks in standard computer vision tasks such as $ImageNet$. Importantly, the query efficient SparseEvo, along with decision-based attacks, in general, raises new questions regarding the safety of deployed systems and poses new directions to study and understand the robustness of machine learning models.

Published as a conference paper at ICLR 2022
QUERY AGAINST BLACK-BOX DEEP LEARNING MODELS
BASED SPARSE ATTACKS
Viet Quoe Vo, Ehsan Abbasnejad, Damith C. Ranasinghe The University Of Adelaide (viet. vo, ehsan. abbasne jad, damith th. ranahrnnnsnngnneeeeeee
edu.
ABSTRACT
Despite our best efforts, deep learning models remain highly vulnerable even tiny adversarial perturbations applied to the inputs. The ability to extract infor- mation from solely the output of a machine learning model to craft adversarial perturbations to black-box models is a practical threat against real-world systems, such as Machine Learning as a Service (MLaaS), particularly sparse attacks. The realisation of sparse attacks in black-box settings demonstrates that machine learn- ing models are more vulnerable than we believe. Because, these attacks aim to minimize number of perturbed pixels- by lo norm required to mis- lead a model by solely observing the decision (the predicted label) returned to a model query; the so-called decision-bused setting. But, such attack leads an NP-hard optimization problem. We develop an evolution- basdd algorithm- SparseEvo-for the problem and evaluate it against both convolutional deep neu- ral networks and vision transformers. Notably, vision transformers are yer to be in- vestigated under a decision- based setting. SparseEvo requires significantly fewer queries than the state-of-the-art sparse attack Pointwise for both untargeted and targeted attacks. The attack algorithm, although conceptually simple, is compet- itive with only a limited query budget against the Nate-of-thee-rt gradient- whitebox attacks in standard computer vision tasks such as ImageNet. Impor- tantly, the query efficient SparseEvo, along with decision-based attacks, in gen- eral, raises new questions regarding the safety of deployed systems and poses new directions to study and understand the robustness of machine learning models.
INTRODUCTION
In spite of the impressive performance achieved from deep neural network (DNN) models va- riety of vision tasks, a flurry of research on adversarial attacks over the last few years have demon- strated the vulnerability of deep learning models to tiny, maliciously crafted perturbations applied to their inputs (Szegedy et al. 2014). These malicious perturbations, although imperceptible to humans, able to evade and mislead DNNs. Thus, embedding DNNs in systems creates new attack surface as well as the incentive for malevolent actors to strike systems such as autonomous cars or machine learning models service (MLaaS) employed in real-world applications such as self-driving cars (Chen et al., 2013), Google Cloud Vision or Amazon Rekognition.
In a black-box setting, an adversary may access all or only the top-/ predicted label and score-a score-based setting (Chen et al. (017)-00000 simply the predicted label for given input-a decision- based (Brendel et 2018) setting. Importantly, the similarity measure, used to quantify the im- perceptibility of the perturbation, can describe an attack as a dense attack-l2, loo norm constrained adversarial attack-or or sparse attack norm constrained adversarial attack.
Significantly, score-based and decision-based settings present practical threat model for deployed systems; the latter being particularly more threatening to model owners and applications. Because, an adversary is still capable of exploiting the very minimal information exposed- top-1 pre- dicted label for constructing an perturbation. Importantly, while dense attacks (Athalye et al.) 2018 Shukla et al. 2021; Ilas al. 2018) are widely explored, sparse attacks have not drawn much attention. This potentially leads to a lack of knowledge on model vulnerabilities to this per- turbation regime. From security standpoint, sparse attacks particularly as threatening as dense

---

Published as a conference paper at ICLR 2022
Vision Transformer
with sparso perturbations
Terget class
STOP SO
ResNe150
Misclasstlad es Target dlsss
Query Budget
200D
4003
GIDO
BOOD
10000
Figure 1: Targeted Attack. Malicious instances generated for a sparse attack with different query budgets using our SparseEvo attack algorithm employed on black-bor models built for the ImageNes task. With extremely sparse perturbation (78 perturbed pixels over total 50, 176 pixels), an image with ground-truth label traffic light is misclassified as a street sign.
attacks. Therefore, investigating sparse perturbation regimes is pivotal and necessary dense perturbation counterparts; in this study, we spend our efforts to extensively investigate the robust- ness of DNNs against sparse attacks.
Scope of the Study-Vision Transformers and Convolutional Networks. Attention-based archi- tectures introduced by Cordonnier al. (2020); Ramachandran et (2019); Touvron et (2021), particularly the Vision Transformer (ViT) model proposed by Dosovitskiy et al. (2021). l), can be com- petitive or even outperform convolution- based architectures (Bhojanapalli et al. 2021 Carion al. 2020). Existing studies have not considered adversarial attacks in lo norm constraint based pertur- bation regimes against ViT, although a few studies have explored robustness against l2 and lo norm constraints (Shao al. 2021). This raises critical security concern for reliable deployment of real- world applications based on vision transformers. Therefore, our efforts will focus on investigating a method capable of evaluating the robustness of convolutional DNNs well as transformer networks to understand the fragility of ViT in relation to CNNs under to norm adversarial attacks.
An NP-Hard Problem. Yielding sparse perturbations is incredibly difficult minimizing t0 norm leads to an NP-hard problem (Modas & Moosavi- Dezfooli 2019 Dong et al.) 2020). Existing sparse attacks in black-bos box settings, particularly in diccssion-baser scenarios, have key shortcoming the algorithms require large number of model queries to achieve sparsity and invisibility, Conse- quently, we propose a novel evolutionary algorithm based sparse attack method in the decision- based setting, we refer to as SparseEvo. The method is significantly more query efficient than the ofte-oof-the-att counteppart-Piintiisss Pointwise (Schott et al. 2019). We illustrate an example of targeted attack with our proposed algorithm in Fig. [ on the standard computer vision task, ImageNet.
need for Query Efficiency. decision-based or black-box settings, achieving query efficiency with high attack success rate is crucial to adversarial objective. Because: i) adversaries are able carry out attacks at scale; ii) the cost of mounting the attack is reduced; and iii) adversaries are capable of bypassing system that can employ methods to recognize malicious activities fraud based on pragmatically large number of successive queries with analogous inputs and thwart their attacks. Further, from defense perspective, the lower number of queries significantly reduces the evaluation time of both trained models and defense mechanisms. Therefore, query efficient attack algorithms facilitate research in designing new defenses, model architectures well as benefit MLaaS providers by enabling the evaluation of their models prior to deployment at scale.
We summarize contributions and results below:
We formulate a novel sparse attack-Spaatt SparsSevo----aaaee an evolution algorithm is capable of exploiting access to solely the top-/ predicted label from model to search for ad- versarial example in the model's input space whilst minimizing the number of perturbed pixels required mislead the model.
Our attack algorithm can significantly reduce the number of model queries compared with the sate-of-the-aa the-art counterpart, Pointwise. Further, SparseEvo achieves comparable suc- cess to PGDp- the state-of-the-as white-box attack in terms of attack success rate with a limited query budget.
2

---

Published as a conference paper at ICLR 2022
We conduct the first vulnerability evaluation of Vision Transformer (ViT) the standard computer vision task ImageNet in a decision-based and ly norm constrained setting. We compare results with ResNet to assess the relative robustness of the ViT model.
2 BACKGROUND AND RELATED WORK ON SPARSE ATTACKS
Adversarial Attack Primer. Three different criteria can be used to categorize adversarial at- tacks (Chen et al. 2020); the attack goal, similarity measure used to quantify the imperceptibility of the perturbation and the level of access to information (threat model). In terms of the attack goal, an adversarial attack is either untargeted (an input is simply misclassified) or targeted. Mcanwhile, ad- be classified into two sub-categories: white-box black-box according the
versarial attacks threat model. In the white-box setting, an adversary has full knowledge and access to the machine learning model (Goodfellow et al.) 2014] Maddy al.] 2018:) [u et al. 2019 Carlini & Wagner) 2017) whereas in the black-box setting, solely the outputs of a model are exposed or accessible to the adversary. In the black-box context, an adversary can access all top-/ predicted score-a score-based setting (Suya et al. 2020 Chen et al. 2017; Guo et al. 2019) or simply the predicted labels of given imput a decision-basee perceptibility, based on a similarity measure, can describe an attack as dense attack loo norm constrained adversarial attacks or sparse attack lo norm constrained adversarial attacks.
(Brendel al., 2018 Cheng et al., 2020) setting. Im-
Sparse Attacks. The main aim of sparse attacks is to minimize the number of perturbed pixels required to mislead a target machine learning model. Only a handful of works have investigated sparse attacks and these works can be broadly categorised based on various degrees of adversarial access to a model.
White-box methods. To realize sparse attacks in a white-box setting, SparseFool attack intro- duced by Modas & Moosavi- Moosavi-Dezfoolij (2019) employed the idea of 4 relaxation from (Andrei & Ion| 20155) and exploited low mean curvature of decision boundaries for to minimization. JSMA (Papernot et al. 2017) constructed a saliency map for an input to search for high impact pixels on the model's 'ss decision. Recently, CCroce& & Hein (2019) introduced PGD, that projects the adversarial perturbation yielded by PGD (Madry et al. (018)) to the to ball. This attack method is capable of generating significantly lower lo perturbation and was shown outperform other white-box algo- rithms. Therefore, we use the PGDs algorithm as an ideal case baseline to compare the success achievable in a black- setting.
Score-based methods. (Su et al. 2019) proposed the One-Pixel attack based on a differential evo- lutionary algorithm. Although the One- method is capable of searching and obtaining the most sparse perturbation, its attack success rate (ASR) on large neural networks and high resolution im- ages is relatively low. Importantly, the method requires significant number of queries because modifies one pixel at a time while the input search space, dependent image resolution, can be enormous. Score-based methods exploit information exposed from a change in confident score to alter pixel-subset in an input image; a model owner may prevent this leakage by only exposing the top-1 predicted label to a model query.
Decision-based methods. In the decision-based setting, only the top-1 predicted label of DNN model is exposed to adversaries. Now, perturbing an input image slightly will not expose subtle changes in the output corresponding to the perturbation; since only the predicted class label is re- vealed. Therefore, a decision-based attack is the most restrictive and challenging scenario. Most existing decision-based attack algorithms are dense altacks (the objective is minimise L2 or Loe distortion). Interestingly, these methods, including BA (Brendel el al.) (018)) HSJA (Chen et al.) 2020 QEBA (Li et al. (220). NLBA LL et al. [021), PSBA (Zhng et al. (2227). Sign-OPT et al. 2020) or the covariance matrix adaptation evolution strategy (CMA-ES) based method for face recognition tasks in DDng et al. 2019), can be adapted to a sparse attack setting by a projection to Lo-ball; however this is not effective, as we show later in Appendix Although CMA-ES (Dong let al.) (019)) is an evolutionary algorithm, albeit for a dense attack, the formulation requires indi- viduals of population to be real number vectors that can be sampled from Gaussian distribution.
Thus, CMA-ES is well suited to the problem of dense attacks. In contrast, the optimization problem in sparse attack (Lo constrained) aims minimize the number of perturbed pixels. Importantly, the discrete search space encountered in a sparse attack hinders the adoption of these dense attack algorithms search for a sparse adversarial example, efficiently.
3

---

applying greedy Published as a conference paper at ICLR 2022
To the best of our knowledge, the recent attack Pointwise (Schott al. 2019 search method to find sparse adversarial perturbations is the first decision beccsiion-based sparse method. This method is effective in untargeted settings and on low resolution datasets, but it is seen to require prohibitively large number of queries to achieve low sparse adversarial perturbations on large scale datasets and in a targeted attack setting (as seen in Section 4. In summary, the current black- box, sparse adversarial attack approaches still have shortcomings on sparsity and query efficiency. Developing brasionn-based sparse attacks poses a challenging optimization problem because of: i) limited access to only the decision of target model; and ii) the NP-hard problem of lo norm constrained optimization.
PROPOSED METHOD
3.1 PROBLEM FORMULATION
In our sparse attack setting, we are given a normalized source image x E [0. [J,1] and its corresponding ground truth label y from the label set Y = {1,2, K} where K denotes the number of classes, C, W and H denotes the number of channels, width and height of an image, respectively. The classifier that we aim to attack is f RC w - W; our access is limited to its output label. In targeted setting, a perturhed such that the instance E RCX x obtained is misclassified to a desired class label ÿ E Y selected by the adversary. We refer to the desired class of the input æ as the target class and its ground-truth class as the source class. In an untargeted setting, the adversary manipulates input æ to change the decision of the classifier any class label other than its ground-truth, i.e. ÿ E Y where ÿ + y. Formally, a sparse adversarial attack (either targetted or untargetted) to find the best adversarial instance x can be formulated as a constrained optimization problem:
xx = arg min &
|| allo
S.L. f(x)) = 7.
where ||||o is the to norm denoting the number of perturbed pixels. The optimization problem in equation [ aiming to minimize the number of perturbed pixels leads to an NP-hard problem (Modas) & Moosavi-Dezfooli 2019; Dong et al. (2020). Thus, the solution to the optimisation problem is non-trivial given the constraint and the fact that f is not differentiable in this setting.
3.2 SPARSEEVO ATTACK ALGORITHM
We devise an efficient parametric search method Soaddd-Soa-----o-aaea based on an evolutionary algorithm approach to search for desirable solution through an iterative process of improving upon poten- tial solutions. Through a process of recombination, mutation, fitness evaluation and selection, the quality of population improves over time to yield desirable solution. Importantly, our evolution- based search method does not require prior knowledge about the underlying target model, such as model architecture model parameters to construct fitness function for assessing potential solu- tions. Consequently, this method detailed in Algorithm | and Fig. 2 is well-suited for solving the non-trivial optimization problem in equation i black-box setting and provides a possible remedy for the NP-hard problem. We detail formulation of the algorithm in the following.
Defining a Dimensionality Reduced Search Space. In applying a parametric search method to the problem, each candidate solution can be defined as a parameter set consisting of coordinates and RGB values defining all perturbed pixels of an adversarial input in the search space CR Naively applying a generic parametric search method to seek potential solutions parameter sets as observed One-pixel algorithm (Su et al., 2019), is not effective because the number of queries to the model grows rapidly with respect to the input image size and the number of perturbed pixels.
We propose two techniques to reduce the search space. To facilitate a parametric search method, in- stead of searching for parameters defining coordinates and RGB values of each perturbed pixel, we propose to solely search for parameters defining coordinates of pixels in the source image to perturb-i.e. image we aim to craft adversarial perturbations for. Constructing all candi- date solutions which are parameter sets in a form of coordinate values is dependent on the num- ber of perturbed pixels and hinders the method implementation. Therefore.
vectorize each candidate solution in a population as a binary vector v E {0, 1 where 0-bits and 1-bits de- notes non-perturbed and perturbed pixels respectively and N is the total number of pixels of an
4

---

Published as a conference paper at ICLR 2022
image. Each element of v corresponds pixel and the position of each element is identified by a mapping function (n, m). Here, we employ a simple flattening technique defined by mapping function o(n, m) + W (m 1) where of an image to reduce the search space further. For the color values of these perturbed pixels, we select RGB values from their corresponding pixels in starting image from the target class (we aim to misclasify the source image to the target class in a targeted attack), We illustrate source image and starting image in the context of the algorithm in Figure 2 All candidate solutions- rectors-can be changed and evolved over iterations until desirable solution is reached. Thus our parametric search method essentially transforms to one that will discover the minimum set of most effective pixels to inject into the source image to construct an adversarial example. Surprisingly, this method shown to be an extremely effective strategy for a decision-baat based sparse attack.
coordinates of pixel, and W is the width
The original search space C is now transformed to the new search space {0, 1].8 where N W H the total number of pixels. In other words, a search space on RGB values and n, m coordinates is transformed into a search space on i ó(n,, m) without exploring RGB values, As result, these techniques lead to a reduction in the size of the search space when compared with the original search space.
Fitness Evaluation
STOP
Binary Differential Recombination
Mutation
[m)
Adversanial
Exampla
Construction,
Source class (source image)
Target class (starting image)
(1a)
Computation
Filness
(o/Re) [(JRk
{ulm) m(
Population Initialization
Population
Selection
(Algorthm 2)
Update
Figure 2: An illustration of SparseEvo algorithm. Population Initialization creates the first popu- lation generation. This population is evolved over iterations through Binary Differential Recombi- nation, Mutation, Finness Evaluation (Adversarial Example Construction and Fitness Computation) and Selection stages. The source and starting images (used in a targeted attack) are employed to cre- ate the initial candidate solutions -binary vector representations--u at Population Initialisation and to construct an adversarial example based candidate solution (mt)at Fitness Evaluation stage.
Algorithm 1: SparseEvo
Input: source image æ, starting image æ', source label y, target label y", model f population size p. initialization rate mutation rate μ. query limit T
t + 0: V, G' +- Initialise xx, x'' f.p..p.)
ke < arg max(G') Ko + arg min // Find best and worst individuals
for f 1. do
Uniformly select 00) ( V. Uk. at random
Yield w using equation and (kx),, n0) ()))
// Recombinatiion
Yield (m)) by uniformly altering fraction μ all 1-bits nd at random // Mutation Construct z using equation 2 with æ, x' and (m)
Calculate g( using equation and ff fā)
// Fitness computation // Se Selection
g(ā.)) < Ga, then
GO, g())
Uk. (m))
Ker arg max hb arg (G)
13 end for
Construct & using equation 2 with æ, a' and (k) // Build adversarial example
15 return &
Fitness Evaluation. Prior describing the other phases of the algorithm, describe the Fitness Evaluation employed for determining the goodness of a candidate solution necessary for the Popu- lation Initialization and the Fitness Evaluation stages, first
5

---

Published as a conference paper at ICLR 2022
Adversarial Example Construction. Since candidate solution-4 binary vector v--is used struct an adversarial example, its fitmess is measured by computing an optimization objective for its corresponding adversarial example. Therefore, first yield an adversarial example corresponding to o based on the following with c, n, representing channel and two coordinates of a pixel.
Fenm (1 D.mmm + 0,2
The Fitness Function Formulation. A fitness function should reflect the optimization objective. In the ccore-based setting, the objective is to optimize loss such that a given input can be misclassified, reasonable choice for the fitness function is based output scores in (Alzantot et al.) 2019]
Qiu et al. 202T). However, in our problem, the objective to minimize lo distortion directly results an NP-hard problem. To alleviate this computational burden, Modas Moosavi-Dezfooli (2019) Murden,,Modas relaxed lo to ( norm to construct the white-box attack, SparseFool and had access to the output decision-based setting. Nonetheless, in the decision- setting, we find that optimizing l2 norm provides a better alternative than I1. Therefore, in this paper, we formulate our fitness function g (for the targeted attack) as:
scores, unlike
x E112
i =
g (
(3)
DO,
otherwise
Where H is an image constructed using equation and y is a target class. A similar fitness function for the untargeted attack can be formulated as equation but the constraint now f(Ø) + y.
Population Initialization. Recall, our search objective is to discover a minimum perturbation rep- resented by binary vector candidate solution. Hence, we initialize population of different candidate solutions from an initialized vector e formulated as following with C number of chan- nels.
f0, 1
if Ten.m = Te ='mam VC E [I,--.,,C
(0)
(4)
otherwise
Every candidate is generated by only randomly altering d 1-bits of (o)) where d = [aW HJ, a an initialization rate. A candidate solution is successfully added to the population, if its fitness score is not oo; we explain our fitness function in equation Otherwise, another d 1-bits is randomly flipped generate another candidate solution. This process is repeated until all p successful candidates found and stored in a population set V. The corresponding fitness score of each candidate solution stored fitness score matrix G. The pseudocode of Population Initialization phase is detailed in Algorithm 2 in Appendix A.
Binary Differential Recombination. In some recombination methods used in ge- netic algorithms (GA) e.g. k-point or uni- form crossover, couple of parents are mated to produce an offspring for the next generation. However, after Population Ini- tialization stage, all first-generation par- ents are slightly different from each other since all them are generated from an ini- tialized vector (()) Consequently, these crossover variants lead to sub-par solutions and low query efficiency. To address this problem, we increase diversity in a popu- lation. Inspired from the differential evo- lutionary (DE) algorithms (Storn & Price 1997). we create the next generation by mu- tating and combining multiple existing par- ents. Nonetheless, applying DE naively is impractical since the mutation operation of DE algorithm adds the weighted difference of multiple selected parents to another parent to yield an offspring. These individuals are vectors in real coordinate space so the offspring can benefit from weighted real-valued difference but it cannot be gained in proposed search space in which all candidate solutions are binary vectors. Therefore, we propose Binary Differential Recombination scheme- a hybrid method based on the uniform crossover in GA and the notion of mutation in DE.
Two randomly selecied candidate solutonn (1 10)
uniform crossover
O The best candidate solution
2 The offspring after recombination step
Figure The Binary Differential Recombination shown in Algorithm [ (line 6) and equation [ C is an element-wise product, ()(k), n,0) v(4)) are the best and two randomly selected candidate solutions from population respectively.

---

Published as a conference paper at ICLR 2022
There different mutation schemes which can influence the overall performance (Manolis & Vagelis| 2020). In the problem of recision-based attacks, through our empirical results shown in Appendix A.3 we observe that the approach of recombining the best and two selected candidate solutions outperforms others. Hence, we first select two candidate solutions v0, v' uniformly at random from the population. We then employ uniform crossover for selecting each bit from either selected candidate solutions with equal probability to yield new candidate solution. Subsequently, the best individual (0%) in the population is recombined with the new candidate solution by altering all 1-bits of (kee) whose corresponding bits in the new candidate solution are O-bits. Formally, the Binary Differential Recombination can be formulated as:
v (k)) @ un formCrossover m/41)
(5)
As a consequence of gaining from the difference between individuals, our method is capable of boosting evolutionary progress as shown in Section 4
where () is an element- wise product. This operation is visualized Fig. 3
Diversity in the population is key factor that enables exploration in the search space to obtain better individuals. As a result, mutation operation aiming to promote this population diversity is crucial component of our method and every offspring after recombination step can be subject to mutation. In practice, we uniformly select a fraction μ of all 1-bits of the offspring v% at random and set these bits to zero. We do not select O-bits for altering because it hinders the optimization progress and requires more iteration to search for the optimum.
Mutation.
Selection. Our simple intuition is that individuals with better fitness values should lead to survival over future generations. In problem | a smaller fitness value is better and represents a more imper- ceptible adversarial example. To this end, if the worst individual in the population has higher fitness value than the offspring's s. it will be discarded and the new offspring is then chosen to take its place.
4 EXPERIMENTS AND EVALUATIONS
4.1 EXPERIMENT SETTINGS
For comprehensive evaluation of the effectiveness of SparseEvo, we em- ploy two standard computer vision tasks with different dimensions: CIFARIC (Krizhevsyy et al.) and ImageNet. (Deng et al. 2009). We compare with the state-of-the-arr sparse attack algorithm in Pointwise (Schott et al., 2019) and use the white-box sparse attack PGD, (Croce & Hein 2019) to benchmark against the black-box decision-based counterparts. For the evaluation sets, we se- lect a balanced sample set. We randomly draw 1,000 and 200 correctly classified test images from CIFAR10 and ImageNet, respectively. These selected images are evenly distributed among the 10 (CIFAR10)) and 200 randomly selected (ImageNet) classes. In the targeted setting, while each image from CIFARIC C is attacked to flip its ground-truth label to 9 target classes, set of five target classes are randomly selected for each image from ImageNet to reduce the computational burden of the evaluation tasks. All of the parameter settings summarized in Appendix A.2
Attacks and Datasets.
(He et al., Models. For convolution-based models, we use a state-of-the-ar architecture-RRete
2016), particularly, ResNet18 for CIFAR10 0, achieving 95, 95.28% test accuracy, and pre-trained ResNet-50 provided by torchvision (Marcel & Rodriguez] 2010) for ImageNet with a 15% Top-1 label test accuracy. For attention-based models, we selected pre-trained ViT-B/16 model obtaining 77.91% % Top-1 label test accuracy (Dosovitskiy et al. 2021). Notably, this model was trained by Google on the large scale and high resolution ImageNet dataset.
Evaluation Measures. To evaluate the performance of methods, we define a normalised sparsity measure as norm distortion divided by the total number of pixels of an image and then compute the median of sparsity over an evaluation set-since it is not sensitive to outliers. A measure used to evaluate the robustness of a model is Attack Success Rate (ASR). generated perturbation is successful if it can yield an adversarial example with a sparsity below given sparsity threshold, then ASR is defined as the number of successful attacks over the entire evaluation set. In black-box settings, ASR can be calculated at different sparsity thresholds after the assessment of the evaluation set with a given query budget. Notably, there is no query constraint for PGD,- We run PGD, with different perturbation budgets and ASR is calculated based the best achieved results.
7

---

Published as a conference paper at ICLR 2022
Attack initialization (targeted and untargeted). We need starting image x' to initialize attack. For targeted attacks, we consider a randomly chosen correctly classified image from the dataset. For untargeted attacks, we may perturb the source image by adding uniform, Gaussian (Cheng et al.) 2020 Chen et al. 2020) or sals and pepper noise (Schott et al.) 2019) until it is misclassified. In practice, we observe that employing salt and pepper noise for our untargeted attack is more effective.
Experimental Regime Summary. We conduct: 1) Attacks against conventional CNNs on CIFAR10 and ImageNet tasks but we defer the results of CIFAR10 to Appendix [A.5 2) Attacks Against a Vision Transformer on the ImageNet task; 3) Compare the robustness of the ViT model with the CNN model; and we defer the following experiments to the Appendix: 4) Attacks against defended models (Appendix 5) Comparing with an improved WointWise algorithm as a base- line (Appendix A.7 6) Comparing with dense attacks adapted to sparse setting (Appendix A.8 Further, in addition to Figure we provide more visual comparisons in Figure T0
4.2
ATTACKS AGAINST CONVOLUTIONAL DEEP NEURAL NETWORKS
Untargeted ImageNet
Targeted ImageNet
Untargeted ImageNet
Targeted InageNet
1.20
100
Searsity
0.02
0.75
(6)
75
75
0.50
SparseEvo Pointwise
10
SparseEvo
50
0.01
Pointwise
0.25
25
PGDIO
25
(Wilieooo base ine)
20K
0.00
1000
2000
3000 Query
4035
5003
5035
10003 Query
15030
20000
0.000 0.001
0,002 n.coo 0.004 0.006
0:00
0.01
0.52
0.00
Sparsity
Sparsity
Figure 4: Evaluation set from ImageNet using the ResNet50 model with image size (Wx H)): 224x224, a) Median sparsity with the first and third quartiles used as lower and upper error bars versus the number of model queries; and b) attack success rate versus sparsity thresholds.
Query Efficiency Evaluation. Fig. 4a shows the median sparsity against model query budgets on the TmageNet task, Our attack consistently outperforms the the Pointwise method in terms of queries and sparsity. In the untargeted setting, SparseEvo achieves a lower sparsity than the Pointwise attack under various query budgets. In the targeted setting, our attack is able to craft adversarial images with extremely sparse perturbation within 20,000 queries for most images from ImageNet but Pointwise does not perform well in this task.
Attack Success Rate. Fig. 4b illustrates ASR against different sparsity threshold at different query budgets for SparseEvo on the TmageNet task and we compare with the best achievement of PGD, (ideal, whitebox attack) and Pointwise observe that SparseEvo achieves a higher ASR than Pointwise employing a 5,000 query budget with the small budget of 1,000 queries. In the targeted setting, our attack with 10,000 query budget demonstrates significantly better ASR than Pointwise employing 20,000 queries. Interestingly, a small query budget of 5,000 queries is adequate to achieve the same ASR as the white-box setting in the PGD, attack in the untargeted setting, while around 20,000 queries achieves comparable performance to the ideal woxte-box setting for a targeted attack. This is significant for decision- based attacks since adversaries are given very limited access to a the model.
pnacision-based sparse attack). In the untargeted setting, we
The evaluation results with CIFAR10 in Appendix A.5 confirm the the observations on the ImageNet task and demonstrate the generalizability of the SparseEvo algorithm. We also sum- marise results at query budgets and attack settings on the two vision tasks in Table in the Appendix.
4.3 ATTACKS AGAINST VISION TRANSFORMER
Query Efficiency Evaluation. Fig. 5 shows the median sparsity against the queries. With a limited number of queries, SporseEvo is able to achieve significantly lower sparsity than Pointwise in both targeted and untargeted setting. While our attack is able to converge to a extremely high sparsity after 3,000 and 15,000 queries for untargeted and targeted setting, respectively. Pointwise fails to converge to lower values in both settings.
Attack Success Rate. Fig. 5b illustrates that with only 1000 queries, SparseEvo outperforms Point- wise with 5,000 query budget across all different sparsity thresholds. Notably, in the untargeted
8

---

Published as a conference paper at ICLR 2022
Untargeted ImageNet
Targeted- ImageNet
Untargeted
ImageNet
Targeted ImageNet
100
0.20
Sparsity
0.15
5.75
ASR (99)
78
0.10
1.50
SparseEvo
50
SparseEvo
Pointwise
0.08
1.55
Pointivise
25
20K
(Mtridea leod
PGDID
0.00
5,00
0.02
6,04 6:055 Sparsity
Query
Query
Sparsity
Figure 5: Evaluation set from ImageNet using the ViT model with image size (W (W) 224 x224 a) Median sparsity with the first and third quartiles used as lower and upper error bars versus the number of model queries; and b) attack success rate versus sparsity thresholds.
setting, SparseEvo with a query budget of 5,000 able to achieve slightly higher ASR than the ideal white-box PGDn from sparsity threshold of 0.002. In the harder, targeted Betting-SpaaseFFv with only 15,000 queries able to obtain marginally lower ASR than PGD, whereas with a 20,000 query budget, our attack is robust as PGD, when sparsity threshold is larger than 0.01.
4.4 COMPARE THE ROBUSTNESS OF THE TRANSFORMER AND THE CNN
In this section, we compare the robustness of ViT and ResNet50 models to sparse perturbation in untargeted and targeted settings. Fig. 6 reports the accuracy of these models over adversarial examples of an evaluation set of 100 images from ImageNet. We summarise results at query budgets and attack settings in Table the Appendix. Overall, we find that the performance of ViT degrades as expected but it appears to be less susceptible than the ResNet50 model. Particularly, in the untargeted setting, the accuracy of ViT across different sparsity thresholds is higher than the ResNe150 model under both SparseEvo and PGD, Interestingly, SparseEvo only needs small query budget of 2,000 to degrade the accuracy of ResNet50 that is similar to white-box PGD,, while up to 5,000 queries are needed to make SparseEvo attack ViT worse than PGDu- In the targeted scenario, we observe that at a low query budget e.g. 10,000, ResNet50 is much more robust than VIT under SparseEvo whereas 20,000 queries, the accuracy of both ResNet50 and VIT models is almost analogous and drops to approximately zero when sparse perturbation is larger than 0.02. Notably, SparseEvo with a sufficient query limit e.g. 20,000 is able maintain its attack effectiveness against both ViT and ResNet50 while attack effectiveness of PGDo is reduced- demonstrated by lower accuracy scores when attacking VIT.
a) Untargeted Query Budget 2K Untargeted Query Budget 5K h)
Targeted Query Budget 10K
Targeted Query Budget 20K
100
SparseiEvv (Rea)aa)
100
100
100
Accuracy (96)
75
SparseEvo (VIT)
PGDIO (Reshvet)
25
Accuracy (9)
x
PGDID (ym
5.03: 0032
0:54 0:06 aste 0010 Sparsity
0.000 0.00
0.004
0,006
3,008
2010
0.00
0.01
0.02
5.03
0.00
001
0.22
0.08
Sparsity
Sparsity
Sparsily
Figure 6: Attack success rate versus sparsity thresholds different query budgets for the evaluation set from ImageNet with VIT vs ResNet. PGD, is a white-box box attack (ideal).
CONCLUSION
In this work, we propose a new algorithm for sparse attack Srtack-SSprseEvvooorr SparseEvo--under a decision-based scenario. Our comprehensive results demonstrate SparseEvo outperforms the stale-of-the-arr black- box attack in terms of sparsity and ASR within given query budget. More importantly, in a high resolution and large scale dataset, SparseEvo illustrates significant query-efficiency and remark- ably lower sparsity when compared with the existing sparse attacks in the black-box setting. Most notably, our black-box attack, under small query budgets, achieves comparable success to the state- of-the-art white-box Brack-PGD, (for further insights we refer the reader to Appendix A.9..

---

Published as a conference paper at ICLR 2022
REFERENCES
M. Alzantot, Y. Sharma, S. Chakraborty, H. Zhang, C. Hsieh, and M. Srivastava. Genattack: Practi- cal black-box attacks with gradient-free optimization. The Genetic and Evolutionary Computation Conference (GECCO), 2019.
P. Andrei and N. lon. Random coordinate descent methods for lo regularized convex optimiza- tion. IEEE Transactions on Automatic Control, 60(7):181-1824, 2015. doi: 10.1109/TAC.2015. 2390551.
A. Athalye, N. Carlini, and D. Wagner. Obfuscated gradients give a false sense of security: Cir- cumventing defenses adversarial examples. International Conference on Machine Learning (ICML). 2018
S. Bhojanapalli, A. Chakrabarti, D. Glasner, D. Li, T. Unterthiner, and A. Veit. Understanding robustness of transformers for image classification. 2021. URL /tKL:tttp:://rxdv.rr////
2103. 1.33.1488 14586
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision- based adversarial attacks: Reli- able attacks against black-box machine learning models. International Conference on Learning (eccognition(ICLR), 2018
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-toend object detection with transformers. European Conference on Computer Vision (ECCV), 2020.
N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. IEEE Symposium on Security and Privacy, 2017.
C. Chen, A. Seff, A. Kornhauser, and J. Xiao. DeepDriving: Learning Affordance for Direct Percep- tion in Autonomous Driving. The IEEE International Conference on Computer Vision (ICCV), 2015.
J. Chen, M. I. Jordan, and M. J. Wainwright. Hopskiijjumpattakk: A query-efficient decision-based attack. IEEE Symposium on Security and Privacy (SSP), 2020.
Jinghui Chen and Quanquan Gu. RayS: Ray Searching Method for Hard-label Adversarial Attack. Knowledge Discovery and Data Mining (KDD), 2020.
P.Y. Chen, H. Zhang, Y Sharma, J Yi, and C.J. Hsieh. Zoo: Zeroth order optimization based black- box attacks to deep neural networks without training substitute models. ACM Workshop on Arti- ficial Intelligence and Security (AISec), pp. 15-26, 2017.
M. Cheng, S Singh, P Chen, P.-Y. Chen, H. Yi, J. Zhang, and C.-J. Hsich. Sign-opt: A query-efficient hard-label adversarial attack. International Conference on Learning Recognition(ICLR) 2020.
J.B. Cordonnier, A. Loukas, and M. Jaggi. On the relationship between self-attention and convolu- tional layers. International Conference Learning Recognition(ICLRR), 2020.
F. Croce and M. Hein. Sparse and imperceivable adversarial attacks. International Conference Computer Vision (ICCV), 2019.
Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database. Computer Vision and Pattern Aecoonittin(CCPPR 2009.
X. Dong, D. Chen, J. Bao, C. Qin, L. Yuan, W. Zhang, N. Yu, and D. Chen. GreedyFool: Distortion- Aware Sparse Adversarial Attack. Neural Information Processing Systems (NeurIPS), 2020.
Y. Dong, H. Su, B. Wu, Z. Li, W. Liu, T. Zhang, and Zhu. Efficient Decision-based Black-box Adversarial Attacks Face Recognition. Computer Vision and Pattern Recognition( (Ccconn 2019.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. International Conference on Learning Recognition(/IIR), 2021.
10

---

Published as a conference paper at ICLR 2022
Yanbo Fan, Baoyuan Wu, Tuanhui Li, Zhang Yong, Mingyang Li, Zhifeng Li, and Yujiu Yang- Sparse Adversarial Attack via Perturbation Factorization. European Conference on Computer Vision (ECCV), 2020.
I. J. Goodfellow, J. Shlens, and Szegedy. Explaining and harnessing adversarial examples. Inter- national Conference Learning Recognitiio((/II.R) 2014.
C. Guo, J.R. Gardner, Y. You, A. G. Wilson, and K.Q. Weinberger. Simple Black-box Adversarial Attacks. International Conference Machine Learning (ICML), 2019.
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. Computer Vision and Paltern Recognition (CVPR), pp. 770-778, 2016.
A. Ilyas, L. Engstrom, A. Athalye, and J. Lin. Black-box adversarial attacks with limited queries and information. International Conference on Machine Learning (ICML), 2018.
A. Krizhevsky, V. Nair, and G. Hinton. Cifar- 10 (canadian institute for advanced research). URL http://wer /www. cs toronto. edu/ kriz/ cifar. html
Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and Bo Li. QEBA: Query-Efficient Boundary-Band Based Blackbox Attack. Computer Vision and Puttern Recognition(CVPR) 2020.
Huichen Li, Linyi Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and Bo Li. Nonlinear Projec- tion Based Gradient Estimation for Query Efficient Blackbox Attacks. Artificial Intelligence and Statistics (AISTATS), 2021.
A. Madry, A. Makelov, Schmidt, Tsipras, and Vladu. Towards deep learning models resistant adversarial attacks. International Conference Learning Recognition(ICLR), 2018.
G. Manolis and P. Vagelis. A comparative study of differential evolution variants in constrained structural optimization. Frontiers in Built Environment, 602, 2020. ISSN 2297-3362. doi: 10. 338%
URL httph:://///m wwtp:::wwwooo..rriiieeoooo http://tteeeei.oootattooooeeee/e
Fbuil 2020 00102
S. Marcel and Y. Rodriguez. Torchvision the machine-vision package torch. Proceedings of the 18th ACM International Conference Multimedia, pp. 1485-1488, 2010. URL https:
//doi. 11tgg/000111//13315111 1874255
A. Modas and P. Moosavi-Dezfooli: S. Frossard. Sparsefool: few pixels make big difference. Computer Vision and Pattern Recognition (CVPR), 2019.
N. Papernot, McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami. Practical black- box attacks against machine learning. ACM Asia Conference on Computer and Communications Security(ASIA CCS), 2017.
Hao Qiu, Leonardo Lucio Custode, and Giovanni Iacca. Black-box adversarial attacks using evolu- tion strategies. The Genetic and Evolutionary Computation Conference (GECCO). 2021.
Prajit Ramachandran, Niki Parmar, Ashish Vaswani, Irwan Bello Bello, Anselm Levskaya, and Jonathon Shlens. Stand-alone self-attention vision model. Neural Information Processing Systems (NeurIPS), 2019.
Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the first adversarially robust neural network model mnist. International Conference on Learning Recognition(ICLR) 2019.
R. Shao, Z. Shi, J. Yi, P.Y. Chen, and C.J. Hsich. On the adversarial robustness of visual transform- ers. 2021. URL
S. N. Shukla, A. K. Sahu, D. Willmott, and Z. Kolter. Simple and Efficient Hard Label Black-box Adversarial Attacks in Low Query Budget Regimes. Knowledge Discovery and Data Mining (KDD), 2021.
11

---

Published as a conference paper at ICLR 2022
Rainer Storn and Kenneth Price. Differential Evolution A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization, pp. 341-359, 1997. URL
Atttps://ooi wttp://.....11111111112220002200
J. Su, D. V. Vargas, and K. Sakurai. One pixel attack for fooling deep neural networks. IEEE Trans- actions on Evolutionary Computation, 23:828-841 2019, doi: 10.1 1109/TEVC2201..28900000
F. Suya. J. Chi, D. Evans, and Y. Tian. Hybrid Batch Attacks: Finding Black-box Adversarial Examples with Limited Queries. USENIX Security Symposium, 2020.
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. International Conference on Learning Recognition(/CLRR) 2014. URL
wwtp:viivrrr.aoooooo abs /ttp///1.... 6199
H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jegou. Training data-efficient image transformers distillation through attention. International Conference Machine Learn- ing (ICML), 2021.
K. Xu, S. Liu, Zhao, P.Y. Chen, H. Zhang, Q. Fan, D. Erdogmus, Wang, and X. Lin. Structured Adversarial Attack: Towards General Implementation and Better Interpretability. International Conference on Learning Recognition(/LRR) 2019.
Jiawei Zhang, Huichen Li, Xiaolu Zhang, Shuang Yang, and Bo Li. Progressive-Scale Bound- ary Blackbox Attack via Projective Gradient Estimation. International Conference on Machine Learning (ICML), 2021.
A APPENDIX
A.1 POPULATION INITIALIZATION
Algorithm presents pseudo-code for our Population Initialization approach as presented Sec- tion 3.2
Algorithm 2: InitialisePopulation
Input:
source image æ, starting image x', source label y, target label y", model f population size p, initialization rate
+ Ø, DC
+- [aWH]
11 W, are image width and height
Generate binary vector v using equation for 1,2,....... .p do
while True do
Generate (()) by uniformly altering of all 1-bits of e at random
Construct using equation 2 with æ, æ' and v(o)
Calculate g( using equation and / (())
// Calculate Fitness Score
10
if (ā)
G, then
12
G. - (())
V + U (2())
break
15
end while end for
17 return V,G G
12

---

Published as a conference paper at ICLR 2022
A.2 HYPER-PARAMEIIE PARAMETERS
We list in in Table[tth the key Sypeer-arameterr used for SparseEvo on the two different evaluation sets across CIFAR10 0 and Image ImageNet. This hyperparameter set can applicable for attacking against VIT-B/16 on large scale and high resolution dataset- Tataset-TmageNet.. Notably, we only needed to adjust the mutation rate for when moving from the high resolution the low resolution CIFAR1 task; thus, our method provides a robust algorithm that can be easily adopted for different vision tasks.
The image size used in all our ImageNet experimental tasks (including experiments on ResNet50 and ViT models) is (3 channels) x 224 (W) x 224 (H). This is the standard input size for the pre-trained model (Py (Prch) the ImageNet dataset used.
Table Hyper-paramette setting in our experiments
CIFARTO
ImageNet
Parameters
Untargeted
Targeted
Untargeted
Targeted
Population size (p)
10
10
10
Initialization rate (a)
0.004
0.004
0.004
0.004
Mutation rate (14)
0.04
0.01
0.004
0.001
A.3 ROBUSTNESS TO MUTATION SCHEMES
AND INVESTIGATING RECOMBINATION AND
In this section, we conduct comprehensive experiments to study the impacts of hyper-parameters used in our algorithm and different recombination and mutation schemes we considered. These experiments are carried on 1,000 randomly selected images from CIFAR10 in untargeted setting. For the aperr-aaraetter study, we tune population size or mutation rate at a time while using the scheme of recombining the best and two randomly selected candidates from the population well as the scheme of mutating only 1-bit binary values.
Fig. 77 shows that with different population sizes and mutation rate of 0.04, even small popula- tion size of 10 is adequate for SparseFyoo to converges rapidly. Our method with a larger population size almost converges to low sparsity as the population size of 10 after 200 queries. So population size has a small impact on the overall performance of SparseEvo. A mutation rate at 0.04 and fixed population size of 10, the algorithm performs well and converges fastest to a low sparsity compared to others mutation rates as shown in Fig. Pb. Consequently, our attack method is more influenced by the mutation rate but this not unexpected.
Population Size
Mutation rate
Recombination schemes
Mutation schemes
1.0
1.0
10
1.0
Spam//
6.6
8.4
844
2.4
0.4
100
200
300
400
150
200
450
930
806
1000
0
253
400
665
300
1000
280
453
600
852
1080
a)
Query
b)
Quarr
0)
Quary
d)
Quarry
Population Size
10
Mutation rate 00.0
beet candidates
1-bits mutation
Population Size
- random condidates
mulation 0.8
Population
rate
2 random candicatos
0811-iit mutation 0.9
Population Size = 50
Mutation rate
0.01
Figure 7: Sparsity versus number of model queries on CIFAR10 0 with ResNet18 8 to show the impacts of different hyper-parameters SparseEivo.
To evaluate how different schemes of recombination and mutation steps affect our method, we use the population size of 10 and mutation rate of 0.01 and change the recombination mutation scheme, one at a time. Figure 77 illustrates that recombining three randomly selected individu- als does not achieve high query-efficiency as the scheme of recombining the best and other two
13

---

Published as a conference paper at ICLR 2022
randomly selected from the population. For mutation schemes, intend mutate merely 1-bits- 1-bits mutation or both O-bits and 1-bits & I-bits mutation of a binary vector at time. For 1-bits mutation scheme, randomly alter factor μ of all 1-bits of selected binary vector. For schemes mutating both O-bits and 1-bits, we randomly flipped n 1-bits and n(1-9) O-bits where 8 n = -488. We find that the scheme of mutating only 1-bits performs marginally better than other schemes with / 0.8 and 3 0.9 because mutating both 0 and 1-bits possible slows down the convergent speed as illustrated in Figure 71.
A.4 ROBUSTNESS OF SPARSE ATTACKS AGAINST AN ADVERSARIALES TRAINED MODEL
In this section, we study the robustness of different sparse attacks against adversarially trained ResNet-18 network on the CIFAR10 task using Loc perturbations -oe the most effective de- fense mechanisms against adversarial attacks (Athalye et al. 2018). The accuracy of this adversari- ally trained network is 83.87%. We choose PGD, (Croce & Hein 2019). state-of-the-art white-bon attack baseline for comparison. The adversarial training based models used in this experiment is trained with Projected Gradient Descent (PGD) adversarial training proposed by Madry et al. (0188)
The experiment is conducted on balance evaluation set withdrawn from CIFAR1 randomly (we describe the dataset in Section 4.1 Median sparsity against the number of queries is shown in Figure 8 The results indicate that SparseEvoo converges faster than the Pointwise attack. Figure g also shows the attack success rate (ASR) different distortion levels and query limits for different attack methods against the adversarially trained model. We observe that our attacks are able to obtain a comparable performance with the ideal white-box PGD, baseline attacks with very limited query budget of merely 500 queries. Meanwhile SparseEvo is comparable with Pointwise with given query budget of 200, and outperforms Pointwise with a query budget of 500.
Distortion vs Query
Query Budget 200
Query Budget 500
0.100
100
Sparsity
0.075
ASR (%)
75
75
0.050
50
50
0.000
0
100
200
300
400
500
0.00
0.02
0.04
0.08
0.08
0.10
0.00
0.02
0.04
0.06
0.08
0.10
Query
Sparsity
Sparsity
SparseEvo
Pointwise
White-box attack (ideal): PGDIO
Figure& 8: Different sparse attacks against an adversarially trained model on the CIFAR10 0 task. We show sparsity versus queries and ASR versus sparsity two different query budgets: 200 and 500.
A.5 ATTACKS AGAINST A CNN MODEL FOR THE CIFAR10 TASK
Fig. Dh shows the median sparsity against the queries as well as the first and third quartiles used as lower and upper error bars. The figure provides comprehensive comparison for different attacks on the evaluation set from CIFAR10 0 in both untargeted and targeted settings. Our attack consistently outperforms the the Pointwise attack in terms of queries and sparsity. Particularly, in the untargeted setting, our attack is able to craft adversarial images by perturbing extremely low number of pixels, on average within 2,000 queries for most images on CIFAR10: 0; while Pointwise only obtains sparsity of 0.75 for this evaluation set. In the targeted setting, SparseEvo converges to lower sparsity than the Pointwise attack with given query budget.
Attack Success Rate. Figure 9b illustrates ASR against different sparsity threshold at different query budgets for SvarseEvo on the evaluation set from CIFAR10 0 and also compare with the best achievement of PGD, (ideal. white-box baseline) and Pointwise of black-box sparse attack). In the untargeted setting, we observe that SparseEvo using 200 queries or more achieve higher success rates than Pointwise using 500 queries. Notably, the our black-hox sparse attack can
14

---

Published as a conference paper at ICLR 2022
achieve comparable ASR to PGDg with small query budget of 500 queries. In the targeted setting, with only 500 queries our attack demonstrates significantly better ASR than Pointwise across all sparsity thresholds, while SparseEvo achieves marginally lower ASR than PGD, (ideal, white-box baseline) with a query budget of 2,000.
Untargeted C1FAR10
Targeted CTRAR10
Untargeted CTFARR0
Targeted - PARR
102
100
Sparsity
004
0.70
ASR (%)
030
0.0K
6.2
0.2
26
0.00
0.00
300
400
Quary
Query
Sparsity
Sparsity
8)
SparseEvo Pointwise
b)
SparseEvo Pointwise White-box attack (ideal):: PGDIO
Figure 9: Evaluation set from CIFAR10 using ResNet18 model. Median sparsity with the first and third quartiles used as lower and upper error bars versus number of model queries; and b) attack success rate (ASR) versus sparsity thresholds.
Table 2: Median sparsity and ASR at different query budgets. A comprehensive comparison among different attacks (PGDo, Pointwise and SparseEvo) on small and large scale balance datasets.
Query budget
CIFAR10 0
Query budget
ImageNet
Setting
Methods
Median
ASR
Median
ASR
PGD,
0.0059
99.8%
0.0005
100%
Pointwise
0.0078
88.0%
0.0016
68.0%
200
2000
Untargeted
SparseEvo
0.0049
96.5%
0.0008
96.5%
Pointwise
0.0078
96.2%
0.0012
77.0%
500
5000
SparseEvo
0.0049
99.2%
0.0008
99.0%
PGDg
0.0703
99.8%
0.0061
99.0%
Pointwise
0.9612
0.0%
0.9997
0.0%
Targeted
1000
10000
SparseEvo
0.0311
96.5%
0.0511
48.5%
Pointwise 0.7863
0.0%
0.9975
0.0%
2000
20000
SparseEvo
0.0251
99.6%
0.0076
99.1%
Table 3: Accuracy of ResNet50 and VIT under attacks at different query budgets and sparsity thresh- olds. comprehensive comparison among different attacks (PGD, and SparseEvo) on small and large scale balanced evaluation sets from ImageNet
Setting
Methods
Query Budget
ResNer50
VIT
Sparsity
0.002
0.004
0.002
0.004
PGDg
na
5%
0.0%
31%
14%
Untargeted
2000
20%
5%
45%
25%
SparseEvo
5000
17%
0.0%
35%
7%
Sparsity
0.02
0.03
0.02
0.03
PGDg
2.0%
1.2%
4.4%
0.2%
Targeted
10000
66.8%
52.8%
20%
9.0%
SparseEvo
20000
2.2%
0.6%
2.4%
0.2%
A.6 ALGORITHMIC COMPARISON WITH POINTWISE
In this section, we discuss why SparseEvo is capable of searching for desirable solution (an adver- sarial example with smaller number of perturbed pixels) with much fewer queries.
15

---

Published as a conference paper at ICLR 2022
1. Greedy vs. Evolutionary approach. Pointwise chooses to greedily minimize the number of perturbed pixels by randomly selecting and altering one dimension (i.e. single colour chamnel) of a randomly selected pixel position i.j of the starting image x' RCx x at time (i.e per query). If the alternation successfully fools the model, it will be retained; oth- erwise, the change will be discarded. In contrast, SparseEvo evaluates candidate proposals to alter several pixels at a time and all dimensions of a pixel simultancously to yield new candidates solutions or offssring for the next evolution; so it is able converge faster and with less queries.
2. Smaller search space. Pointwise formulation leads to search space with a size of C' x W H where is the three RGB channels, W is image width and H image height. We reduce this search space to W x H because SparseEvo solely searches for pixel positions but does not try to search for different colors for each pixel (see "Defining a Dimensionality Reduced Search Space" in Section 3.2 and Appendix (A.7).
3. Better scalability large image sizes. Given that PointWise only changes dimension at a time (i.e a pixel), to reduce the number of starting image (target class) pixel values different from the source image (to minimize 40), the random selection method needs to select: i) the same pixel position i, j; and ii) different colour channel for the same pixel position j in subsequent iterations to move a given pixel value i, j in a starting image (target class image) to be the same as the source image. While this is more likely in a small image task (with smaller W and values) like CIFAR10, is far less likely, even within the 20,000 query budget used with large input images in the ImageNet task where mean sparsity values for the 1000 test image pairs remain nearly
4. Iterative improvements to "good" solutions. Importantly, our approach formulates a search for solution with the minimum number of perturbed pixels through an iterative process of improving upon good solutions from previous iterations informed by our objective function. In contrast, Pointwise employs purely random method to select the pixel dimension and position i, j to alter.
A.7 COMPARISON WITH AN IMPROVED POINTWISE ALGORITIIM AS BASELINE
Table 4: Mean sparsity measure at different queries (lower is better) for targeted attack setting. A comparison between SparseEvo and improved Pointwise on set of 100 image pairs on Tma geNet (here PW-n, denotes PointWise with number of selections set to np and italicised fonts indicate the best results for PW.)
Query Budgets
500
1000
2000
4000
8000
12000
16000
20000
PW(published version)
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
PW-4
1.00
1.00
1.00
1.00
1.00
0.99
0.97
0.93
0.88
PW-8
1.00
1.00
1.00
1.00
0.99
0.94
0.81
0.58
0.35
PW-16
1.00
1.00
1.00
0.99
0.94
0.64
0.45
0.42
0.40
PW-32
1.00
1.00
0.99
0.95
0.71
0.54
0.50
0.46
0.42
PW-64
1.00
1.00
0.95
0.78
0.67
0.62
0.56
0.51
0.46
PW-128
1.00
0.96
0.84
0.77
0.74
0.67
0.61
0.56
0.52
SparseEvo
1.00
0.76
0.63
0.46
0.26
0.08
0.03
0.01
0.01
PointWise randomly selects and alters one dimension (a colour channel) of randomly selected pixel position i, j of an image x' E RCXWXB xH at a time (i.e per query). Therefore, the Pointwise formulation leads to a search space with size of C x W H where C the three RGB channels, W is image width and H is image height. Consequently, it is not scalable to large image sizes, for example ImageNet with size of 224 224; this be observed in Fig. 4 Haand]
In this section, we attempted to make PointWise more query efficient on ImageNet by modifying PointWise to perform multiple selections time (i.e per query) and perform series of experiments using different selection parameters n, Table 4 shows the mean sparsity obtained by our improved Pointwise method with different selection parameter values; np 4, 8. 16, 32, 64, 128. The results
16

---

Published as a conference paper at ICLR 2022
show that the best performance of the modified Pointwise algorithm PW-8- is much better than the original implementation but it is still far behind our method. SparseEvo still outperforms our improved Pointwise algorithms across various query budgets.
A.8 COMPARISON WITH DENSE ATTACKS ADAPTED TO CONSTRUCT SPARSE ATTACKS
We are motivated to investigate if beceeion-baased dense attacks (22 and Lo constrained) such as BA (Brendel et al., 2 HSJA (Chen et al.] 2020), QEBA (Li et al. 2020), NLBA (Li et al. 2021). PSBA (Zhang et al. 2021), SignOPT (Cheng et al. 2020) or RayS (Chen & Gu 2020 can be adapted sparse setting by projection to lo-ball. This idea is promising because PGD can be successfully adapted to a sparse setting to provide a sparse attack algorithm in white-box setting. In this section, we conduct a study evaluate this idea by modifying the HSJA method because is shown to be query-efficient becision-basee dense attack (l2 and lo constraint), to an lg constraint algorithm called HS-ASJ. Notably, the same could be done for other methods e.g. QEBA, NLBA, PSBA, SignOPT, or RayS.
Importantly, the authors HSJA proposed two different ways of gradient estimation purposely for- mulated for 12 and Ixx scenarios. However, the lo distance metric is non-differentiable and therefore ill-suited for standard gradient descent (Carlini & Wagner 2017 Fan et al., 2020) so we leverage to estimate the gradient. The difference between the 4o- HSJA algorithm and published HSJA is the projection step. Instead of performing (2 and (x projection steps as in HSJA, 40 lo-HSJA performs an lo projection as in the PGDIO method. To search for the minimum number of pixels to perturb, we adopt a binary search to minimise lo- At each iteration (with the discovered adversarial sample from HSJA), we perform the following projection procedure:
1. 1o-HSJA sorts pixel differences between the sample adversarial crafted by HSJA and the source image.
2. 1o- (o-HSJA then performs a binary search for k denoting the minimum number of (perturbed) pixels to retain from the sample adversarial crafted by HSJA. Here, k= k=antir where ir and ur are lower and upper ranges, initialized with 0 and N, respectively. N is the total number of pixels in an image.
3. Subsequently, we create a candidate sparse adversarial example by keeping only the top-k pixels of the HSJA crafted adversarial sample and replacing the rest of the pixels of the crafted sample with their corresponding pixel in the source image we plan to fool. These top-k pixels have the least difference to their corresponding pixels. This yields the projected image Tn for evaluation. If the projected sample can mislead victim model successfully, ur is updated with & (to search for a lower number of perturbed pixels). Otherwise, Ir is updated with k.
4. This step is repeated until the ur and Ir difference is less than or equal to the threshold 1.
For the following iteration of (S-A, we use the projected image Zp to craft example oip to attempt to improve upon the projected adversarial example from the current iteration.
adversarial
The results we obtained, shown in Table 5 illustrate the average sparsity for set of 100 image pairs on CIFAR1O. Our evaluations show that applying lo projection to dense attacks (formulated for l2 and Lo0 methods) does not yield a query efficient sparse attack aiming to minimize the number of perturbed pixels. We can understand this result, because, at each projection step, the modified lo-HSJA algorithm still requires a large number of queries determine a projection that minimises lo (in other words, to determine the minimum number of pixels to retain where the crafted sample is still adversarial).
To the best of our knowledge, there is no efficient method in a black- decision- setting to determine how many pixels and which pixels can be selected to be projected such that the perturbed image does not cross the unknown decision boundary of the DNN model. Additionally, the problem of minimizing the number of selected pixels to be projected leads to an NP-hard problem (Modas) & -oosavi-Dezfoolli 2019 Dong et al. 2020). Although we use the projected image with the minimum number of perturbed pixels, lz and T Eecision-bases attacks require perturbing a whole image in the following iteration, thus the next iteration does not necessarily move the input towards the objective of minimizing the number of perturbed pixels. Thus, 4 -SJA and other dense methods do not provide efficient algorithm for sparse attacks.
17

---

Published as a conference paper at ICLR 2022
Table 5: Mean sparsity measure at different queries (lower is better) for targeted setting. A com- parison between lo- (S-J and SparseEvo on set of 100 image pairs on CIFAR10
Queries
500
1000
2000
4000
8000
12000
16000
20000
IS-HSIA 1.00
0.82
0.95
0.92
0.92
0.95
0.95
0.94
0.94
SparseEvo
1.00
0.36
0.027
0.025
0.025
0.025
0.025
0.025
0.025
A.9 A DISCUSSION ON RESULTS WITH THE WHITEBOX BASELINE
Notably, PGD, is an adapted-tto-lyversio of the PGD attack with a projection. PGDp simply projects the adversarial example generated by PGD attack onto the lo-ball (we described the process in Appendix eareerr regarding adopting non-sparse decision based attacks). This projection does not guarantee that projected solution yields the best gradient descent direction for the following iteration of PGD to find an adversarial example that minimises lo- Hence, even with full access to the model, p may not always yield the optimal solution but rather an approximation. So PGDg may not always be an upper bound for the attack performance, particularly in the untargeted setting on ImageNet as shown Figure 5(b) and the second plot of Figure
18

---

Published as a conference paper at ICLR 2022
A.10
VISUALIZATION OF SPARSE ADVERSARIAL EXAMPLES
Target model Vision Transformer
Target class
Source class
Target model
ResNet50
Target model
Vision Transformer
Target class
Source class
Target model
ResNet50
Query Budget
2000
4000
6000
8000
10000
Figure 10: Visualisations from a targeted attack Settings, Malicious instances generated for a sparse attack with different query budgets using SparseEvo attack algorithm employed on black-box models built for the ImageNet task.
19


Reviews:
[{"review_id": "hR_r3FLAuvS", "reviewer": "Reviewer_t1fV", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "***Strengths***\n- The paper is largely clear and well-written.\n- The experimental results are solid, and experiments are carried out on vision datasets and models of interest.\n- The attack is both sparse and effective.\n***Weaknesses***\n- The main issue with this paper is the lack of an intuitive explanation as to why this attack is better at finding sparse and effective adversarial examples than previous work. I would have liked to see a more detailed algorithmic comparison with previous work.\n", "summary_of_the_paper": "This paper proposes the use of an evolutionary algorithm to construct decision-based black-box adversarial examples with L0 or sparsity constraints against image classifiers such as CNNs and Image Transformers. The algorithm uses an L2 distance constraint to check the fitness of a solution, and employs several tricks such as differential recombination and mutation to improve the quality of the solution. The experimental results demonstrate that the attack is more effective than the current SOTA sparse attacks, and is almost as effective as white-box attacks given enough queries.", "summary_of_the_review": "Overall this is a solid paper that makes a reasonable contribution to a problem of some interest to the community.\n\n+++++++++++++++++++++++++++++++++\nHaving read the rebuttal, I retain my score. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JepwZTIYn3u", "reviewer": "Reviewer_DgSu", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- The proposed methods are well-motivated and novel. The paper is easy to follow for an adequately prepared reader. Prior work is sufficiently discussed.\n- The experiments are convincing and the experiment results show the effectiveness of the proposed attack.\n- The amount of detail is good, it seems sufficient to reproduce results.", "summary_of_the_paper": "This work proposes a novel sparse attack method called SparseEvo. Based on evolution algorithm, the SparseEvo searchs a sparse adversarial perturbation in limited query budget. It can significantly reduce the queries compared with the SOTA method, i.e., Pointwise. The paper also conduct the first vulnerability evaluation of a ViT on ImageNet in a decision-based and $l_0$ norm constrained setting.", "summary_of_the_review": "Overall, I think this paper is a good one.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "ArsMA-4ovX", "reviewer": "Reviewer_HSPi", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe paper shows promising experimental results for the method. As a paper proposing a blackbox method, it also shows comparison with a white-box attack to showcase its superiority (my concerns are described in the weakness part below).\n\nWeaknesses:\n1. The experiment section can be more comprehensive. The submission only compares with one paper on decision-based sparse attack, and that work only shows experiments on MNIST dataset, not ImageNet. (a) The comparison on ImageNet shown in this submission is not fair (the PointWise method sparsity is always 1, which means it basically fails to create any sparsity): if the comparison were to be made, the submission can instead make some minimal adjustments to the baseline method to make it not completely useless. (b) There are many other decision-based attacks on ImageNet. Although most of them are showing results in L-2 metrics (e.g., BA, HSJA, QEBA, NLBA, PSBA, SignOPT, etc.) and some of them show L-\\infty metrics (e.g., RayS), many of them can be easily adapted to L-0 case with projections based on my experience. The submission can try to compare with these stronger baselines on ImageNet to showcase its method performance.\n2. The paper can discuss its relationship/difference with the existing literature more clearly. For example, using evolutionary methods for decision-based attacks is not an invention by this submission: the paper “Efficient Decision-based Black-box Adversarial Attacks on Face Recognition” has proposed one in 2019. Also, as mentioned above, though many existing decision-based attack papers did not show results on L-0 metrics, they can be adapted easily, thus very related with this paper. The paper should consider a more detailed discussion on its related works and justify its novelty in terms of the proposed method.\n\nQuestions: \nOn the second plot in Figure 6, for the two solid curves, the red curve (SpaEvo (ViT)) even goes lower than the black curve (PGDl0 (ViT)) both at the beginning and the end. Is there an explanation for this observation? Are you using different images for different curves so that the white-box attack PGD is not the upper bound of the attack performance in this plot? Or is the PGD not optimized properly? The “Untargeted-ImageNet” plot in Figure 5(b) is also weird in a similar sense.\n", "summary_of_the_paper": "The paper proposes an evolution-based algorithm to conduct a sparse attack against convolutional deep neural networks and vision transformers. The evaluation results show that the proposed method requires fewer model queries than the state-of-the-art sparse attack Pointwise for both untargeted and targeted attacks.", "summary_of_the_review": "The paper shows good experimental results, but there are some concerns about the experimental part (whether it's fair and valid). Also, the novelty of the method and the relationship with the literature are not discussed in detail.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9K1_kiItlyL", "reviewer": "Reviewer_8jjY", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n1. The experimental performance is really good in terms of it being a decision-based sparse attack. \n2. Using the L2/L1 distance as the fitness function and using an evolution algorithm instead of some estimated gradients to generate adversarial examples is novel.\n\nCons:\n\n1. The comparison with the pointwise attack in the targeted attack experiments is somehow unfair. SparseEvo relies on a random target image to generate the adversary while the pointwise attack doesn't. It would be better to find a way to let the pointwise attack leverage the target image or adapt another black-box attack for doing the sparse attacks.\n2. I am wondering what's the image size used in the ImageNet experiments? Since you only reduce the search space by a factor of the # channels (typically 3). So I am wondering how the scalability of SparseEvo is against the big images.", "summary_of_the_paper": "This paper proposes a black-box decision-based spare attack based on the evolution algorithm (called SparseEvo).  The authors test their method on two types of classification models and two popular vision datasets: ResNet (CIFAR10 and ImageNet) and Vision Transformer (ImageNet).  Through the comparison with Pointwise attack (for efficiency and sparsity) and PGD0 (for success rate), SparseEvo achieves good performance in both success rate and efficiency.", "summary_of_the_review": "This paper proposes a novel black-box decision-based space adversarial attack method based on the evolution algorithm. The basic idea is to use the L2/L1 distance with the original image as the fitness function to adjust the current images towards the target images. The experimental results are good. I am only concerned a little bit about the comparison in the targeted attack since it is somehow unfair (see in the Main Review).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a technique to generate L0 adversarial examples in
a black-box manner. The reviews are largely positive, with the reviewers
especially commenting on the paper being well written and clearly explaining
the method. The main drawbacks raised by the reviewers is that the method
is not clearly compared to some prior work, but in the rebuttal the authors
provide many of these numbers. On the whole this is a useful and interesting
attack that would be worth accepting.
Decision: Accept (Poster)