Question: Why only consider two gradient-free black-box attacks?
Evidence: In this paper, we evaluate the performance of the Saliency Attack comparing against Boundary Attack, TVDBA) Parsimonious attack, k int ← k int /2; end Square attack and their modified version equipped with segmentation priors. Among them, Boundary Attack is able to constantly reduce the distortion to a very small magnitude, TVDBA tries to minimize the distortion via integrating Structural SIMilarity (SSIM) into the loss, and Square Attack can achieve the current state-of-the-art query efficiency and success rate in the black-box setting. We consider L ∞ threat model on ImageNet dataset, set to 0.05 in [0,1] scale, and test 1000 randomly selected exmaples in all experiments. The threshold φ to produce binary salient masks is chosen to be 0.1. All parameters of the compared attacks remain consistent with those recommended in their papers. We use Inception v3 as the target model, and different query budgets {3000, 10000, 30000} for untargeted attack. For performance metrics, we employ commonly used success rate (SR) and average number of queries (Avg. queries). To evaluate the imperceptibility of AEs, we consider L 0 , L 2 and MAD (see the appendix for details), which is validated as closest to human vision system among existing IFA metrics. All these three imperceptibility metrics are the smaller the better.
In this paper, we evaluate the performance of the Saliency Attack comparing against Boundary Attack (Brendel et al., 2018), TVDBA (Li & Chen, 2021) Parsimonious attack (Moon et al., 2019),Square attack (Andriushchenko et al., 2020) and their modified version equipped with segmentation priors. Among them, Boundary Attack is able to constantly reduce the distortion to a very small magnitude, TVDBA tries to minimize the distortion via integrating Structural SIMilarity (SSIM) (Wang et al., 2004) into the loss, and Square Attack can achieve the current state-of-the-art query efficiency and success rate in the black-box setting.
Answer: In this paper we actually consider three gradient-free black-box attacks, including Parsimonious Attack (Moon et al., 2019), Square Attack (Andriushchenko et al., 2020) and Boundary Attack (Brendel et al., 2018). Parsimonious Attack first proposed discrete optimization which dramatically reduces the search space and save lots of queries, which is essential in black-box attack setting. We also follow this work to use discrete optimization. Square Attack achieves the SOTA success rate (SR) and query efficiency performance in our score-based black-box attack setting. Boundary Attack can constantly reduce the distortion to a very small magnitude. Since our paper focus on imperceptible black-box attack, we also consider this one. For black-box attacks with Bayesian optimization such as (Shukla et al., 2019; Ru et al., 2019), we find they underperform Square Attack in terms of SR and query efficiency, and their distortion are also moderate. Besides, as suggested by Reviewer 2cEz, we tested (Li & Chen, 2021) which also aims to generate imperceptible perturbations. We find our method can outperform this work in all aspects (Table 2, page 8, revised paper).
Shukla, S. N., Sahu, A. K., Willmott, D., & Kolter, J. Z. (2019). Black-box adversarial attacks with bayesian optimization. arXiv preprint arXiv:1909.13857.
Ru, B., Cobb, A., Blaas, A., & Gal, Y. (2019, September). Bayesopt adversarial attack. In International Conference on Learning Representations.
Li, N., & Chen, Z. (2021). Toward Visual Distortion in Black-Box Attacks. IEEE Transactions on Image Processing, 30, 6156-6167.