I MPERCEPTIBLE BLACK - BOX ATTACK VIA REFINING IN SALIENT REGION
Anonymous authors
Paper under double-blind review
A BSTRACT
Deep neural networks are vulnerable to adversarial examples, even in the blackbox setting where the attacker only has query access to the model output.
Recent studies have devised successful black-box attacks with high query efﬁciency.
However, such performance often comes at the cost of the imperceptibility of adversarial attacks, which is essential for attackers.
To address this issue, in this paper we propose to use segmentation priors for black-box attacks such that the perturbations are limited in the salient region.
We ﬁnd that state-of-the-art blackbox attacks equipped with segmentation priors can achieve much better imperceptibility performance with little reduction in query efﬁciency and success rate.
We further propose the Saliency Attack, a new gradient-free black-box attack that can further improve the imperceptibility by reﬁning perturbations in the salient region.
Experimental results show that the perturbations generated by our approach are much more imperceptible than the ones generated by other attacks, and are interpretable to some extent.
Furthermore, our approach is found to be more robust to detection-based defense, which demonstrates its efﬁcacy as well.
I NTRODUCTION
Deep neural networks (DNNs) have achieved signiﬁcant progress in wide applications, such as image classiﬁcation (Deng et al., 2009), face recognition (Parkhi et al., 2015), object detection (Redmon et al., 2016), speech recognition (4, 2012) and machine translation (Bahdanau et al., 2015).
Despite their success, deep learning models have revealed vulnerability to adversarial attacks (Szegedy et al., 2014).
Crafted by adding some small perturbations to benign inputs, adversarial examples (AEs) can fool DNNs into making wrong predictions, which is a critical threat especially for some security-sensitive scenarios such as autonomous driving (Sun et al., 2020).
Based on the knowledge of target model, the adversarial attacks could be divided into white-box attack and black-box attack.
White-box attacks (Szegedy et al., 2014; Goodfellow et al., 2015; Papernot et al., 2016; Carlini & Wagner, 2017; Moosavi-Dezfooli et al., 2016) have full access to the architecture and parameters of the target model, and can easily generate a successful adversarial example via back propagation.
By contrast, black-box attacks (Narodytska & Kasiviswanathan, 2016;
Ilyas et al., 2018; 2019; Moon et al., 2019; Andriushchenko et al., 2020; Li et al., 2021; Papernot et al., 2017; Liu et al., 2017) can only query the target model to obtain the output prediction, which is a more realistic and challenging setting.
Since many real-world online application programming interfaces (APIs) have time or monetary limit for user query (Ilyas et al., 2018), most current research efforts focus on how to design query-efﬁcient attacks (Ilyas et al., 2019; Moon et al., 2019;
Andriushchenko et al., 2020), which indeed achieve a huge improvement.
For example, so far the state-of-the-art black-box attack (Andriushchenko et al., 2020) could succeed in untargeted attack on ImageNet dataset (Deng et al., 2009) with only tens of queries on average.
However, this success has sacriﬁced the imperceptibility of AEs, whose global perturbations are generated with various tricks like random vertical stripes (Andriushchenko et al., 2020), and thus very obvious and perceptible to human eyes (see Figure 1 (e) and (f) for examples).
In effect, imperceptibility is essential for attackers.
Currently online APIs usually integrate detectors into their services to detect anomalous inputs (Li et al., 2021).
The images with too visible perturbations are difﬁcult to pass these detectors, not to mention human judgement.
[Figure]
To alleviate or even address this problem, we could utilize some priors to lead black-box attacks to search in small but efﬁcient space, instead of global perturbations.
Previous DNNs visualization work (Simonyan et al., 2014) proposed a type of saliency map to indicate which features should inﬂuence the output signiﬁcantly.
This BP saliency map 1 is generated by calculating the derivatives of the model output with respect to input.
Just as show in Figure 1.(a), the brighter pixels in BP saliency maps denote they have a greater impact on the model output.
The classical white-box attack JSMA (Papernot et al., 2016) just constructs such an adversarial saliency map and iteratively selects pixels from this map to perturb.
But due to the black-box setting, we cannot derive this saliency map directly.
Nevertheless, we could ﬁnd that the region of bright pixels roughly represents the position of the main object in an image, which also accords with our intuition.
Inspired by this, we propose to ﬁrst generate a saliency mask to extract the regions of the main object in an image via image segmentation, and then limit the perturbation search in this smaller but more important regions, as presented in Figure 1.
Several recent studies also propose to generate similar local perturbations but applied in different attack settings with different methods (Dong et al., 2020; Xiang et al., 2021).
In particular, Dong et al.
(2020) produce saliency maps with the class activation mapping (CAM)
(Zhou et al., 2016), which requires the internal information of the model and needs to change the model architecture.
This is infeasible in black-box scenarios.
Xiang et al.
(2021) replace CAM with Grad-CAM (Selvaraju et al., 2017), which does not need to change the architecture of the model.
However, both CAM and
Grad-CAM still rely on the knowledge of the model, but in black-box setting the internal information of the target model is inaccessible.
Thus, we choose salient object segmentation (Zhao & Wu, 2019), which can automatically extract salient object(s) in an image (see Figure 1.(b)).
Compared with CAM and Grad-CAM, it doesn’t need any other information except for the input image.
On the other hand, these studies (Dong et al., 2020; Xiang et al., 2021) are applied to transfer-based attacks (actually a grey-box setting), which ﬁrst build a substitute model to approximate the target model, then use gradient-based white-box attacks to generate adversarial examples, and ﬁnally transfer them to attack the target model.
For transfer-based attack, we must either train a substitute model from scratch with huge number of queries (Papernot et al., 2017) or assume a pretrained substitute model is trained on the similar data distribution with the target model’s
(Liu et al., 2017).
And for the latter, the transferability of both adversarial examples and saliency maps highly depends on the selection of the pair of substitute model(s) and target model.
Hence, in this paper we focus on gradient-free black-box attacks (Narodytska & Kasiviswanathan, 2016;
Moon et al., 2019; Andriushchenko et al., 2020) that take no account of gradients and search for successful perturbations only according to the model output.
We add these segmentation priors to SOTA black-box attacks and ﬁnd the imperceptibility of AEs is indeed enhanced but still limited due to ”global perturbation” in local region (See Figure 4 and 5).
To further improve the imperceptibility, our another novelty is that even in the salient region, “saliency” could still be reﬁned.
Our inspiration also comes from some DNNs visualization works (Zhou et al., 2016; Zeiler & Fergus, 2014; Olah et al., 2018).
For instance, given a dog image, CAM produces a localization heatmap and shows the dog face region is most highly lighted (Zhou et al., 2016).
Zeiler & Fergus (2014) systematically cover up different portions of a dog image, and ﬁnds when the dog face is obscured, the activity in the feature map and classiﬁer output changes dramatically.
In further, by combining internal feature visualization with output prediction, Olah et al.
(2018) reveal even in dog’s face region, its ears and eyes seem to be more important when distinguishing dogs.
Therefore, we are inspired to assume the salient region in an image is progressive with respect to its impact on model output.
If we could ﬁnd smaller but more salient region, the perturbation will be more efﬁcient and meanwhile the imperceptibility of AEs could also be enhanced.
Thus, we propose our
Saliency Attack, a new gradient-free black-box attack via reﬁning perturbations in the salient region according to their saliency.
Our main contributions can be summarized as follows:
• To our best knowledge, we are the ﬁrst to use salient object segmentation to extract binary salient masks in black-box settings.
Experiments show that the SOTA black-box attacks limited in such regions can achieve much better imperceptibility performance with little reduction in query efﬁciency and success rate.
• We propose a new gradient-free black-box attack via reﬁning in salient regions.
Compared with the search methods used in other gradient-free attacks, our method is able to generate smaller but effective perturbations, which is interpretable to some extent and can further improve the imperceptibility.
• We demonstrate that the perturbations generated by our Saliency Attack is more robust against some detection-based defense like Feature Squeezing.
R ELATED W ORK 2. B LACK - BOX ATTACKS
Gradient estimation attacks.
Gradient estimation attacks ﬁrst estimate the gradients by querying the target model and then apply them to run white-box attacks (Goodfellow et al., 2015; Carlini & Wagner, 2017; Madry et al., 2018).
ZOO attack (Chen et al., 2017)
ﬁrst adopts the symmetric difference quotient to approximate the gradients and then perform Carlini-Wagner (CW) white-box attack (Carlini & Wagner, 2017).
AutoZOOM (Tu et al., 2019) uses a random vector based gradient estimation to estimate gradients, which reduces the query number per iteration from 2 D in ZOO to
N +
( D is the dimensionality and N is the sample size).
To further enhance query efﬁciency, Ilyas et al.
(2019) propose the “tiling trick” that updates a square of pixels simultaneously instead of a single pixel, which dramatically decreases the dimensionality by a factor of k 2 ( k is tile length).
Gradient-free attacks.
Gradient-free attacks take no account of gradients and directly generate AEs with random search or heuristic methods according to the query result.
Su et al.
(2019) propose one pixel attack that adopts differential evolution algorithm to perturb the most important pixel in the image.
Alzantot et al.
(2019) propose GenAttack, which use genetic algorithm to generate AEs.
To improve the query efﬁciency, Moon et al.
(2019) consider a discrete surrogate optimization problem that transforms the original constraint of a continuous range [ − (cid:15), + (cid:15) ] to a discrete set {− (cid:15), + (cid:15) } , achieving a huge reduction in the search space.
This is motivated by linear program (LP) where the optimal solution is attained at an extreme point of the feasible set (Schrijver, 1998).
Combing tiling trick (Ilyas et al., 2019) and discrete optimization (Moon et al., 2019), Square Attack (Andriushchenko et al., 2020) has obtained the best result on success rate and query performance so far with a randomized search scheme.
Hybrid attacks.
Hybrid attacks try to combine multiple types of methods, using candidate adversarial examples generated on substitute model as starting points for further optimization with gradient estimation attacks or gradient-free attacks.
Suya et al.
ﬁrst proposed to generate candidate
AEs by white-box projected gradient descent (PGD) attack (Madry et al., 2018), and further optimize them with gradient estimation methods.
Wang et al.
(2020) combines a white-box attack with a gradient-free method using microbial genetic algorithm (Harvey, 2009).
Similar with transfer-based attacks, hybrid attacks also suffer from the similarity between substitute model(s) and target model.
2. R ELATED WORK ON IMPERCEPTIBILITY
The imperceptibility of AEs is vital for attackers and some attacks consider it in different manners.
Guo et al.
(2019) diﬁne the imperceptibility as the smoothness in the sense of low frequency and thus search for AEs in frequency domain.
While Zhang et al.
(2020) regard the imperceptibility as the smoothness of visual content in an image and integrate Laplacian smoothing into optimization.
Boundary Attack (Brendel et al., 2018) starts from an example that is already adversarial and approaches the original image gradually to reduce the distortion.
Instead of global perturbations, some studies (Dong et al., 2020; Xiang et al., 2021) consider local perturbations to reduce the square of perturbations.
On the other hand, how to estimate the imperceptibility of AEs is still a problem.
Most of the adversarial attacks use L p norms ( L 0 , L 2 and L ∞ ) to measure the human perceptual distance between the perturbed image and the original one.
Nonetheless, L p norms are found not suitable enough for human vision system (Sharif et al., 2018).
To ﬁnd a proper metric, Fezza et al.
(2019) design subjective experiments to obtain the subjective scores on different AEs, and then test various image ﬁdelity assessment (IFA) metrics including L p norms.
Among them, most apparent distortion (MAD) (Larson & Chandler, 2010) metric is found closest to subjective scores.
Hence, in this paper we adopt MAD as our main imperceptibility metric to estimate the imperceptibility of AEs.
P ROPOSED M ETHOD
In this section, we ﬁrst introduce the problem formulation of crafting adversarial examples for image classiﬁcation models, and then detail our approach.
The overall ﬂowchart of our proposed Saliency
Attack is depicted in Figure 2.
[Figure]
3. P RELIMINARY
Given a well-trained DNN classiﬁer F : [0 , 1] d → R K , where d is the dimension of the input x , and
K is the number of classes.
We denote F k ( x ) as the predicted score that x belongs to class k .
So the classiﬁer assigns the class that maximizes F k ( x ) to the input x .
The goal of an untargeted attack is to ﬁnd an adversarial example x adv , that results in the model misclassiﬁcation from the ground-truth class y , and meanwhile keeps the distance between the adversarial and benign input smaller than a threshold (cid:15) :
[Equation]
where the other constraint indicates that the generated x adv must be an image in valid range.
In this paper we focus on L ∞ as our distance norm following (Moon et al., 2019; Andriushchenko et al., 2020).
Traditionally, this task of ﬁnding x adv can be rephrased as solving a constrained continuous problem:
[Equation]
(2) where L is a loss function.
To improve the query efﬁciency, Moon et al.
(2019) transform the continuous problem into a discrete surrogate problem, where the perturbation δ = x
adv − x is generated at the corner of the L ∞ ball.
Besides, we propose to limit δ only in the salient region.
So our complete optimization problem is:
[Equation]
where S is the set of the pixels in salient region.
3. S ALIENT OBJECT SEGMENTATION
Salient object segmentation, as one changing task in image segmentation domain, aims to automatically and accurately extract salient object in an image.
Speciﬁcally, given an input image, salient object segmentation model can generate a saliency map where the brighter pixels denote they have higher saliency scores, just as the saliency map shows in Figure 2.
Compared with other model visualization methods like CAM (Zhou et al., 2016) and Grad-CAM (Selvaraju et al., 2017), this type of model does not require any information other than the input image, which is very suitable for the black-box setting.
We adopt the Pyramid Feature Attention (PFA) network (Zhao & Wu, 2019) that achieves the state-of-the-art performance in multiple datasets.
After obtaining the saliency map where all pixel values are between 0 and 1, we use a binarization threshold φ to transform the saliency map to a binary salient mask.
The binarization can be expressed as
[Equation]
where s is the saliency map and s ∗ is the binary salient mask, s ij and s ∗
i,j are the corresponding value at the position i, j .
With this saliency mask s ∗ , we could limit the perturbation search in the salient region.
3. R EFINING PERTURBATION IN SALIENT REGION
[Figure]
block is a root node of a tree, and its child nodes are the ﬁner blocks in the current block.
In this part, we design a search algorithm to reﬁne perturbations based on a tree structure for an image by using the idea of depth-ﬁrst search.
As shown in Figure 3, an input image merged with its saliency mask is split into four initial blocks (a coarse grid).
Then we try to add a + (cid:15) or − (cid:15) perturbation on each initial block individually and ﬁnd the best block that maximizes the loss function for further perturbation.
After ﬁnding the best one among all initial blocks, we can further split this block into smaller blocks (a ﬁner grid), and again try to add a perturbation on each block individually (at this time we just ﬂip the perturbation for convenience e.g. + (cid:15) to − (cid:15) ) to ﬁnd the best smaller block.
We iterate this process until the minimal block (e.g. 1 pixel) or no smaller block has a better loss.
Then we go back to the last level of split blocks and use the second best block for further perturbation.
We use the loss function from CW attack (Carlini &
Wagner, 2017) for untargted attack:
[Equation]
where Z ( x adv ) o is the logit with respect to the true class of the original image.
In this way, the loss function is imposed to leave a margin between the true class and other classes.
The reﬁning process is presented in Algorithm 1, and the overall saliency attack is given in Algorithm 2.
E XPERIMENTS
In this paper, we evaluate the performance of the Saliency Attack comparing against Boundary Attack (Brendel et al., 2018), TVDBA (Li & Chen, 2021) Parsimonious attack (Moon et al., 2019), Square attack (Andriushchenko et al., 2020) and their modiﬁed version equipped with segmentation priors.
Among them, Boundary Attack is able to constantly reduce the distortion to a very small magnitude, TVDBA tries to minimize the distortion via integrating Structural SIMilarity (SSIM)
(Wang et al., 2004) into the loss, and Square Attack can achieve the current state-of-the-art query efﬁciency and success rate in the black-box setting.
We consider L ∞ threat model on ImageNet dataset (Deng et al., 2009), set (cid:15) to 0.05 in [0,1] scale, and test 1000 randomly selected exmaples in all experiments.
The threshold φ to produce binary salient masks is chosen to be 0.1.
All parameters of the compared attacks remain consistent with those recommended in their papers.
We use Inception v3 (Szegedy et al., 2016) as the target model, and different query budgets { 3000 , 10000 , 30000 } for untargeted attack.
For performance metrics, we employ commonly used success rate (SR) and average number of queries (Avg. queries).
To evaluate the imperceptibility of AEs, we consider L 0 , L 2 and MAD (see the appendix for details), which is validated as closest to human vision system among existing IFA metrics (Fezza et al., 2019).
All these three imperceptibility metrics are the smaller the better.
4. SOTA BLACK - BOX ATTACKS WITH SEGMENTATION PRIORS
To verify the feasibility of our segmentation priors, we modify Parsimonious Attack and Square
Attack so that the perturbations are limited in salient regions.
We show the results in Table 1 and some examples in Figure 4.
From the examples, we can easily ﬁnd that the perturbations of original attacks are very obvious in the entire image due to global perturbations, while their segmentation versions are relatively more imperceptible since no perturbation exists in background regions.
Besides, through quantitative analysis, the attacks equipped with segmentation priors can indeed achieve much better imperceptibility performance with around one third improvement in L 2 and
MAD metric.
L 0 is also restricted to less than 50% , which implies the average square of salient regions among all examples only occupies less than half of the whole image.
And this improvement only comes at little cost in SR and Avg.
queries, which is still in the same order of magnitude.
We also present the change of MAD scores in Figure 9 in appendix.
Thus, the results demonstrate that the segmentation prior is indeed able to enhance the imperceptibility of other black-box attacks.
[Table]
[Figure]
4. C OMPARISON OF S ALIENCY A TTACK WITH OTHER ATTACKS
Although the segmentation prior is helpful to the imperceptibility, the perturbations generated by other attacks are still global, taking up almost all salient regions.
Hence, we design our Saliency
Attack via further reﬁning perturbations in salient regions.
Original image are resized to 256 ×and we set the initial block size to 16.
For an imperceptible AE, a low MAD value is important.
So besides SR , we use a new metric SR true .
It denotes the rate of successful AEs whose MAD values are below some threshold.
Here we choose 30 as the threshold because we ﬁnd the AEs with
MAD ≤ 30 are basically imperceptible to human eyes (See Figure 10 in appendix for details).
The comparison results are given in Table 2.
We can ﬁnd in all query budgets, our Saliency Attack achieves outstanding performance with a huge gap in terms of SR true and three imperceptibility metrics.
Although TVDBA, Parsimonious and Square Attack can obtain a better SR , their successful AEs are not true imperceptible, and will be easily detected by some defenses or humans.
For Boundary Attack, it can gradually make progress on SR true , L 2 and MAD as the query budget increases, but they are still much worse than Saliency Attack.
This is because it takes at least hundreds of thousands of queries for Boundary Attack to converge (Brendel et al., 2018), which is infeasible in practical.
We further test more eamples to verify the statistical signiﬁcance in Table 5, and present a comparison of SR true versus query number in Figure 11 in appendix.
From the exhabited examples in Figure 5, we can ﬁnd the AEs of Boundary Attack contain obvious coarse textures because of inadequate query budget.
In addition, even restricted in the same salient regions, the perturbations of Parsimonious-seg and Square seg attack are complex and irregular, while our perturbations are smaller and more important, roughtly corresponding to the bright pixels in BP saliency map.
They further represent the postions of dogs’ noses or ears, which accord with our inspiration before.
[Figure]
[Table]
4. H YPERPARAMETER TUNING
Our Saliency Attack contains only one hyperparameter namely the initial block size k which determines the ﬁrst level of split blocks.
We test the effect of different k on SR, imperceptibility and query number respectively in Figure 6.
As k decreases, SR and imperceptibility can be improved and SR reaches the peak when k equals 16.
This is because with smaller initial blocks, Saliency Attack can search for perturbations more ﬁnely and accurately leading to higher SR and better imperceptibility performance.
Meanwhile inevitably more queries are needed, especially for sorting initial blocks.
That is why under a limited query budget like 10000, a turning point occurs in SR.
We also show some examples in Figure 7.
It can be observed that as k decreases, the generated perturbations also become smaller but more salient.
For instance, in the ﬁrst row the perturbation with k = 128 roughly covers the region of the dog face, which is also the brightest region in the
BP saliency map.
While the perturbation with k = 32 or k = 16 focuses on smaller region of the dog ear.
This indicates that our Saliency Attack could ﬁnd smaller and more important perturbations progressively as the initial block size decreases.
And the perturbations are interpretable to some extent like perturbing the speciﬁc regions of dog’s ears, eyes or nose.
4. A BLATION S TUDY
We do ablation study of Saliency Attack, including reﬁning in salient region, in non-salient region and without saliency (reﬁning in the whole image).
We also design a greedy search as baseline to demonstrate our Reﬁne search.
We test multiple block sizes for greedy search and use 32 as the best choice (see Table 6 in appendix).
The results and some examples are given in Table 3 and Figure 8.
Notice that reﬁning in salient region and reﬁning without saliency generate the same or almost the same perturbations, which means the salient regions indeed contain useful parts and enhance the query efﬁciency by limiting the search space.
But for reﬁning in non-salient region, its perturbation is more complex and visible with worse query efﬁciency and SR.
Compared with greedy search, our
Reﬁne search has much better query efﬁciency and SR true , which proves its superiority.
Hence, we can conclude that both salient region and Reﬁne search facilitate our Saliency Attack.
[Figure]
S R    8 Block size
I m pe r c ep t i b ili t y
MADL     8 Block size
A v g .
que r i e s
[Figure]
Method
[Table]
SR
[Table]
true
[Table]
reﬁne in salient region
[Table]
[Table]
[Table]
[Table]
reﬁne in non-salient region 78.2%
[Table]
[Table]
[Table] reﬁne without saliency 95.5% 79.6% [Table] [Table]
[Table]
greedy search in salient region 56.0% 50.7% 4.
[Figure]
4. A TTACKING DETECTION - BASED DEFENSE
To further validate the superiority of our method, we attack against one detection-based defense Feature Squeezing (Xu et al., 2018), which applies transformations to input image and judge whether it is adversarial through the stability of model output.
We consider the detection rate of both successful adversarial examples (SAEs) and failure adversarial examples (FAEs).
This is because if we could detect FAEs, we can raise the alarm that some potential attacker is attacking the model.
From Table 4, we can ﬁnd our Saliency attack is able to achieve lower detection rate in both SAEs and FAEs than other attacks, which proves the imperceptibility of our AEs from the defensive perspective.
[Table]
C ONCLUSION
In this paper we propose to provide segmentation priors for black-box attacks to improve the imperceptibility of adversarial examples, whose perturbations are limited in salient regions.
We utilize a salient object segmentation model to produce such saliency maps with no need for any information other than the input image.
Experiments show that SOTA black-box attacks equipped with segmentation priors can indeed achieve better imperceptibility performance.
Furthermore, we devise a new gradient-free black-box Saliency attack that further enhances the imperceptibility via reﬁning in salient regions.
The perturbations generated are much smaller, imperceptible and interpretable to some extent.
Finally, we also attack against some detection-based defense and the results show that the adversarial examples generated by our attack are harder to be detected
A A PPENDIX A. MAD METRIC
Most Apparent distortion (MAD) metric (Larson & Chandler, 2010) is one of the state-of-the-art full-reference image quality assessment methods.
MAD attempts to merge two separate strategies for two kinds of distorted images respectively.
For high-quality images with near-threshold distortion (just visible), MAD focuses on detection-based strategy to look for distortions in the presence of the image.
While for low-quality images with suprathreshold distortion (clearly visible), MAD focuses on appearance-based strategy to look for image content in the presence of the distortions.
MAD will control the weight of two strategies according to the type of distorted images.
The calculation process of MAD can be summarized as following steps and we recommend interested readers to read the original literature.
Compute locations of visible distortions based on luminance images.
Combine the visibility map with local error image.
Decompose both the distorted and original images into log-Gabor subbands.
Calculate different statistics of each subband.
Calculate the adaptive blending score.
A. S CATTER PLOT OF MAD SCORES
Equipped with segmentation priors, 68.1% and 94.3% examples of Parsimonious-seg attack and
Square-seg attack have better MAD scores compared with their original version respectively.
Since
Parsimonious Attack uses a greedy local search while Square attack adopts a random search
[Figure]
it is obvious that limiting the search in salient regions is more helpful to Square Attack.
[Figure]
A. T HRESHOLD FOR MAD METRIC
[Figure]
[Figure] [Figure]
Here we further present a comparison of SR true versus query number under different MAD thresholds.
It shows that Square Attack and Parsimonious Attack can generate some successful AEs with very few queries due to global perturbations and some tricks like initializing perturbations with random vertical stripes (Andriushchenko et al., 2020), but they lack the ability to further improve the imperceptibility.
Instead, Saliency Attack conservatively select small regions to perturb, hence the query efﬁciency is a little lower than Square Attack and Parsimonious Attack at the beginning but soon exceeds them.
[Figure]
A. C OMPARISON OF S ALIENCY A TTACK WITH OTHER ATTACKS
We randomly choose 10,000 examples from the ImageNet validation set and divide them intogroups.
Then we test different attacks under 3000 query budget and report their mean and standard deviation (SD).
The best results are recorded in bold based on Wilcoxon signed-rank test with signiﬁcance level at 0.05.
We can ﬁnd that our Saliency Attack is signiﬁcantly better than all the baselines.
[Table]
G REEDY SEARCH IN SALIENT REGION WITH DIFFERENT BLOCK SIZES
We test different block sizes for greedy search in salient region, and choose the best one to be compared in ablation study.
[Table]
[Table]