Practical No-box Adversarial Attacks with Training-free Hybrid Image Transformation

Abstract: 	    In recent years, the adversarial vulnerability of deep neural networks (DNNs) has raised increasing attention. 
		Among all the threat models, no-box attacks are the most practical but extremely challenging since they neither rely on any knowledge of the target model or similar substitute model, nor access the dataset for training a new substitute model. Although a recent method has attempted such an attack in a loose sense, its performance is not good enough and the computational overhead of training is expensive.
		In this paper, we move a step forward and show the existence of a \textbf{training-free} adversarial perturbation under the no-box threat model, which can be successfully used to attack different DNNs in real-time.
		Motivated by our observation that high-frequency component (HFC) domains in low-level features and plays a crucial role in classification, we attack an image mainly by manipulating its frequency components. Specifically, the perturbation is combined by the suppression of the original HFC and the adding of noisy HFC.
		We empirically and experimentally analyze the requirements of effective noisy HFC and show that it should be regionally homogeneous, repeating and dense.
		Extensive experiments on the ImageNet dataset demonstrate the effectiveness of our proposed no-box method. It attacks ten well-known models with a success rate of \textbf{98.13\%} on average, which outperforms state-of-the-art no-box attacks by \textbf{29.39\%}. Furthermore, our method is even competitive to mainstream transfer-based black-box attacks. Our code is available in our appendix. 

Reviews and author responses:

The authors have proposed an attack based on the modification of high-frequency components of an image through different types of masks such as circular, square, and rhombus. The experiments are performed using multiple networks on a subset of the ImageNet database.  It is well known that the high-frequency components of an image are critical for both classification and robustness. Therefore, the authors need to revisit the text accordingly regarding the claim and highlight the text both in the abstract and the paper and refer to the following but not limited to the following papers. 

[1] High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks, Haohan Wang, Xindi Wu, Zeyi Huang, Eric P. Xing, CVPR2020
[2] D. Anshumaan, A. Agarwal, M. Vatsa, and R. Singh,  WaveTransform: Crafting Adversarial Examples via Input Decomposition,  ECCV Workshop on Adversarial Robustness in the Real World, 2020, pp. 152-168,
[3] Steganographic universal adversarial perturbations, PRL 2020

The comparison shown in the paper is extremely weak and does not use the existing works that utilize the same concept [2-6].

[4] Guo, C., Frank, J.S. and Weinberger, K.Q., 2018. Low-frequency adversarial perturbation. arXiv preprint arXiv:1809.08758. 
[5] Sharma, Y., Ding, G.W. and Brubaker, M., 2019. On the effectiveness of low-frequency perturbations. arXiv preprint arXiv:1903.00073.
[6] Hashemi et al., Transferable Universal Adversarial Perturbations Using Generative Models, 2020

The technical novelty of the paper is not clear as the authors use the concept of Hybrid image which is well known.

The attack is highly dependent on the proto-patterns and why only the circle pattern is more effective is not well studied. 

The attack norms is also a concern, the authors have used slightly higher norm. The ablation study corresponding to different epsilon values such as 2, 4, 8, 10 needs to be added and a comparison with existing algorithms on the same norm strength can be added.

Are the models trained using the data augmentation which usually performs corruption-based augmentation as well? Are those models are also vulnerable? What about ensemble-based models?

Another limitation of the attack is the dependency on the tile size. The paper lacks several such analyses to explain why this dependency exists and why hyperparameter is more successful than others.

Please add the comparison with the following transferable perturbations and defense as well.

Learning Transferable Adversarial Examples via Ghost Networks, AAAI 2020.
Adversarial Robustness Across Representation Spaces. CVPR

What is the computation cost of PI-MI-DI2-FGSM?


 The technical contribution of the papers needs to be highlighted better, the comparison needs to be updated, and the analysis concerning several parameters used in the attack needs significant attention.



 Thank you a lot for your response! We admit that other reviewers raise some reasonable concerns, but we also believe that our rebuttal has addressed most of their concerns, although reviewer cY6Z and h2W3 have not replied to us yet (hope they could give feedback to our rebuttal)

Nevertheless, we still appreciate your recognition and approval of our work. We also hope that our work will have the opportunity to be published at ICLR 2022. Thanks again!

 I appreciate the authors' feedback. In particular you have addressed the questions I had posed. However, the other authors have raised some reasonable concerns as well. Thus at the moment I have decided not to change my score.

 Thank you a lot for your response and suggestion! We will spend time on testing the performace of our HIT on adversarial detector and include this experiment in the final version.

 In this paper, we come up with a **more practical** no-box attack (compared with the first no-box attack of Li et al.): whether there exists an attack method that can generate adversarial examples without any training and any knowledge of the target model? To this, we propose our Training-free Hybrid Image Transformation method which is inspired by the classification logic of DNNs.  **The main contribution of our approach is to elegantly design effective noisy HFC**. With it, we can easily fool various models and outperform the previous state-of-the-art methods by a large margin. 
Besides, in the latest revision, we also add experiments about attacking other classification tasks (in Appendix Sec. A.5) and a real-world recognition system (in Appendix Sec. A.6), which further demonstrates the effectiveness and practicality of our method. 

Given all that, we would suggest that this work has its unique novelties and practical meaning in both conception and technique.

 In this paper, we come up with a **more practical** no-box attack (compared with the first no-box attack of Li et al.): whether there exists an attack method that can generate adversarial examples without any training and any knowledge of the target model? To this, we propose our Training-free Hybrid Image Transformation method which is inspired by the classification logic of DNNs.  **The main contribution of our approach is to elegantly design effective noisy HFC**. With it, we can easily fool various models and outperform the previous state-of-the-art methods by a large margin. 
Besides, in the latest revision, we also add experiments about attacking other classification tasks (in Appendix Sec. A.5) and a real-world recognition system (in Appendix Sec. A.6), which further demonstrates the effectiveness and practicality of our method. 

Given all that, we would suggest that this work has its unique novelties and practical meaning in both conception and technique.

 _**Discussion 1: Discussion on "Adversarial Robustness Across Representation Spaces"**_

**Discussion:** For "Adversarial Robustness Across Representation Spaces", the authors of this paper have not released their pre-trained models, and their paper mainly discusses small-scale datasets (e.g. CIFAR-10). However, our manuscript mainly evaluates the performance on the large-scale ImageNet dataset and the defense models in our manuscript are all for ImageNet and publicly available. To ensure that our manuscript is coherent, we may need to train the model you mentioned for ImageNet from scratch. It will raise two concerns:
1. It is not sure that a defense technique that works for CIFAR will necessarily work for ImageNet, due to the huge difference in scale between the two datasets.
2. Adversarially training an ImageNet defense is not easy, which needs a lot of GPUs and time.

Since our HIT is a no-box attack and we have evaluated our attack performance on ten normally trained models and six adversarially trained models (our main no-box competitor Li et al. tested only 8 normally trained models), we do not think we need to train an uncertain-performance defense model from scratch for testing. Nevertheless, we thank the reviewers for introducing us to this interesting defense paper and we have cited it in our manuscript.



_**Experiment 1: Results for smaller $\epsilon$ and Ghost Networks**_

**Details:** In this experiment, we report the average success rate (%) between state-of-the-art black-box attacks (including Ghost Networks you mentioned) and our proposed no-box attack under $\epsilon=8$. As demonstrated in Table a, our proposed methods are still competitive to mainstream transfer-based black-box attacks, even though they combine many effective techniques. Remarkably, our no-box attack can significantly outperform Ghost Networks (+MI-FGSM). Although Ghost Networks (+PI-MI-DI-FGSM) is much more powerful, our no-box attack can surpass it in some cases. For example, when fooling Shuffle, our HIT (w/ $6\times6$ Circle) can outperform Ghost Networks (+PI-MI-DI-FGSM) by about 8%. 



Table a. The average success rate (%) between state-of-the-art black-box attacks and our proposed no-box attack. For black-box attacks, the substitute model is Inc-v3. The maximum perturbation $\epsilon=8$. 

|         |   VGG19   |  ResNet   | DenseNet  |    WRN    |   SENet   |    PNA    |  Shuffle  |  Squeeze  |  Mobile   |   AVG.    |
|:---:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|
|          MI-FGSM          |   21.26   |   15.63   |   20.47   |   14.45   |   11.58   |   23.36   |   24.13   |   32.61   |   25.64   |   21.01   |
| Ghost Networks + MI-FGSM  |   27.12   |   17.45   |   22.31   |   14.92   |   13.43   |   28.63   |   30.08   |   40.80   |   33.67   |   25.38   |
|          DI-FGSM          |   18.41   |   11.24   |   16.44   |   10.19   |   8.28    |   17.59   |   15.03   |   18.40   |   18.20   |   14.86   |
|          PI-FGSM          |   24.12   |   14.98   |   22.91   |   15.38   |   12.21   |   27.32   |   25.93   |   39.20   |   27.29   |   23.26   |
|       PI-MI-DI-FGSM       |   40.88   |   31.02   |   41.34   |   29.70   |   25.56   |   38.46  |  36.79   |   46.70   |   42.38   |   36.98   |
|Ghost Networks + PI-MI-DI-FGSM | **63.56** |	**43.59** |	**55.21** |	**40.91** |	**40.33** |	**54.73** |	63.48 |	**81.98** |	**73.46** 	|**57.47** |
| HIT w/ 6$\times$6 Circle  | 37.64 | 36.21 | 40.80 | 30.82 | 21.36 | 37.63 | **71.45** | 79.34 | 69.90 | 47.24 |
| HIT w/ 6$\times$6 Square  |   13.40   |   17.50   |   25.85   |   21.99   |   13.79   |   18.31   |   53.03   |   61.61   |   50.88   |   30.70   |
| HIT w/ 6$\times$6 Rhombus |   17.24   |   19.95   |   27.65   |   22.00   |   12.85   |   18.91   |   58.84   |   63.49   |   57.90   |   33.20   |


_**-------------------------------------------------------------Revision update-------------------------------------------------------------**_

We also conduct more additional experiments to demonstrate the effectiveness of our method.
1. We add experiment (in Appendix Sec. A.5) for attacking other classification tasks (e.g. CUB-200-2011) and discussing hyperparameter settings in this case.
2. We add experiment (in Appendix Sec. A.6) for attacking a real-world recognition system, i.e., Google Cloud Vision API.
3. We add experiment (in Appendix Sec. A.7) for using a smaller epsilon setting.
4. We add discussion (in Appendix Sec. A.9) on the targeted attack potential of our HIT.

_**-------------------------------------------------------------Revision update-------------------------------------------------------------**_

The proposed approach involves creating adversarial examples in a training-free manner by manipulating the frequency components of the image. High -frequency information is used from simple geometric patterns and combined with low-frequency component from the input to create a hybrid image. The resultant image is shown to fool classifiers under multiple settings which presents an easy way to construct adversarial examples. ### **Strengths**:

— The creation of adversarial example is simple and well analyzed showing improved performances compared to previous approaches across multiple settings. The construction of an adversarial example in a training-free manner without using gradients is an interesting phenomenon and can be used to improve robustness of Neural networks.

— The paper is well written and easy to follow. 

### **Concerns**:

— Looking at the qualitative examples in Figure 13, the resultant adversarial example consists of artifacts based on the type of pattern used. This can make it easier to identify based on manual visual inspection. More qualitative examples of the proposed approach compared with the examples from Li et al. , can improve be helpful.

— The quantitative experiments are mainly presented for ImageNet dataset. It would be interesting to see the effectiveness of the proposed approach for other datasets such as Places365 or even other vision tasks. This would also provide an opportunity to discuss choosing different hyperparameters associated with the approach for custom datasets. 

— Attacking a real world recognition system as done in Li et al., would show the practicality of the algorithm.

— In Table 3, the proposed attack is compared against gradient based attacks on different adversarially trained models. One question that arises is that why the proposed approach is better than even gradient based attacks for the defense models. Specifically, ResNet152_D which involves denoising operation specifically designed to suppress the effect of high frequency components. So it would be expected that the proposed approach that relies on the effect of HFC would not be better than gradient based attacks. More analysis or insight into this experiment would be useful. The authors are also encouraged to discuss the work by Yin et al. (arXiv:1906.08988) which is also along similar lines. 

— Metzen et al. ( arXiv:1702.04267 ) showed that auxiliary CNNs can be used to detect adversarial perturbations. It was shown later that gradient based adversarial examples can be constructed to fool both the classifier and adversarial detector (arXiv:1705.07263 ). However, since the proposed approach does not rely on substitute models or gradient based information, it would be interesting to see if such an auxiliary classifier can be an effective defense for the proposed approach which relies on  feature statistics from HFC.

### **Minor comments**:

— Figure 1 suggests that High-frequency component has more confidence than the original image itself for all data. Although the authors mention in the text that this is not the case for all input images, Figure 1 can mislead readers into thinking this is true for all examples. The authors are suggested to either include the text in the figure caption or alter the position of the image.

— The related work section discusses some important works in the adversarial attack literature. Authors are encouraged to also discuss works related to adversarial patches which have been used to fool multiple vision tasks and how the proposed work (which does not rely on learning the perturbation ) is different from such works. Since both approaches rely on using high frequency information from the perturbation, it would improve understanding among readers. The proposed work is an interesting approach towards constructing adversarial examples in a training-free manner. Such attacks can provide more insight into understanding the architectural aspects of CNNs that make models vulnerable and hence can be useful to the community. I believe that evaluations against real world recognition systems and other defense algorithms can improve impact of the work to the community. 

 Thanks to the authors for the detailed experiments and response. I believe most of my concerns have been addressed. Authors show that the proposed attack can be used for multiple datasets and also fool real world recognition systems. But I believe that it is important to discuss defense algorithms that can circumvent new attacks which was the motivation behind auxiliary classifier experiment and authors are encouraged to pursue this direction.

Nonetheless, I believe this is an interesting paper which presents a simpler attack using HFCs whose effectiveness is shown across multiple classifiers and datasets. For these reasons, I will raise my score. 

 Thanks for reviewing our paper. The following is our rebuttal:

Q1: The comparison shown in the paper is extremely weak.
- For the issue, please allow us to disagree with you. Since our HIT is training-free, we do not need to provide experiments about optimization iteration and queries (e.g., [4]). Nonetheless, our paper still illustrates 12 Figures and 6 Tables to discuss our proposed method, while our main competitor Li et al. [A] only show 7 Figures and 5 Tables. Specifically, we validate the effectiveness of our method on 10 normally trained and 6 defense models, and we also compare our HIT with well-known black-box attack algorithms. Besides, **the works introduced in our experiments are all from the top conference, e.g., ICLR, NeurIPS, CVPR and ECCV.**
So we are confused by the comment "The comparison shown in the paper is **extremely** weak."


Q2: The technical novelty of the paper is not clear as the authors use the concept of Hybrid image which is well known.
- Indeed, **the most significant contribution of our paper is the design of effective adversarial HFC for practical no-box threat model**. As demonstrated in Figure 11 and Table 4, under $\ell_\infty$ constraint, simply utilizing the HFC of raw images like [3] (even if they are very richly textured) to manipulate adversarial examples is much less effective. In contrast, adversarial examples crafted with our proposed HFC can significantly reduce the top-1 accuracy of different models.


Q3: why only the circle pattern is more effective is not well studied.
- Our paper **focuses on exploring the properties of generic high-frequency perturbations and using them to implement our training-free no-box approach**. Our purpose of designing three simple geometric patterns is to avoid cherry-picking and verify our conjecture, i.e., the requirements of effective noisy HFC should be regionally homogeneous, repeating and dense.  Hence, why the circle always performs better is not the focus of our paper and we just briefly analyze it via *cosine similarity* in A.10 of Appendix. 


Q4: The attack norms is also a concern, the authors have used slightly higher norm.
- In fact, **the $\epsilon$ is set according to the default settings of comparison methods** [A,B,C]. We kindly remind the reviewer that our main competitor [A] adopts a larger norm (i.e. $\epsilon=25.5$) in their paper. Therefore, we disagree with your comment that our norm setting is a concern.


Q5: The ablation study corresponding to different epsilon values such as 2, 4, 8, 10 needs to be added and a comparison with existing algorithms on the same norm strength can be added.
- Thanks for your suggestion. **We have already reported the ablation study corresponding to different epsilon values such as 2, 4, 8, 10 in Figure 6.** Since we want to report the results for 10 models in each subplot, adding results from other methods may overlap the curves and thus reduce readability. Therefore, we did not compare it with other methods, but we will consider your suggestion and redraw this Figure.


Q6. Another limitation of the attack is the dependency on the tile size. The paper lacks several such analyses to explain why this dependency exists and why hyperparameter is more successful than others.
- In Sec 3.2, we conjecture that noisy HFC with **repeating property** will effectively fool DNNs. Therefore, we repeat and tile the proto-patterns to verify it, and results support our conjecture. We disagree with your comment that the tile size is the limitation of our attack. On the contrary, **the tile size can make our no-box HIT (since it is training-free) more flexible when attacking different models**. As demonstrated in Figure 12 and Figure 5 (right), defense models are more vulnerable to adversarial patches with a big tile size, while normally trained ones are the opposite. This is because that defense models are less sensitive to high-frequency noise. Therefore, a big tile size is more effective for fooling defense model because it can enlarge the area of each regionally homogeneous pattern in the resulting noise.


Q7. What is the computation cost of PI-MI-DI2-FGSM?
- Crafting adversarial samples by PI-MI-DI2-FGSM requires multiple forward and backward propagation on the substitute model. This is very time-consuming, even when using the GPU.
However, our HIT does not suffer from this expensive computation cost because it **only involves two convolution calculations**.

Q8. Revisit the text according to [2-6] and add the comparison with transferable perturbations and defense you provide.
- We are very grateful for your suggestions. Thanks for bringing these papers into our attention. We have cited and discussed them in the final version.


### Reference 
[A] Li et al. Practical No-box Adversarial Attacks against DNNs (NeurIPS 2020)

[B] Dong et al. Boosting Adversarial Attacks with Momentum (CVPR 2018)

[C] Gao et al. Patch-wise Attack for Fooling Deep Neural Network (ECCV 2020)

 Thank you for your comments on our paper. We will address your concerns in the following rebuttal:

Q1. It is a little weak about the discussion of why the circle always performs better.
- In fact, our paper focuses on **exploring the properties of effective noisy HFC** and using them to implement our training-free approach. Our purpose of designing three simple geometric patterns is to avoid cherry-picking and verify our conjecture, i.e., the requirements of effective noisy HFC should be regionally homogeneous, repeating and dense.  Hence, why the circle always performs better is not the focus of our paper. Nonetheless, we still try to give an insight into this phenomenon via  *cosine similarity* about intermediate features in A.10 of Appendix.

Q2. Do the circle/square/rhombus perturbations always lead the victim models to predict certain classes?
- Indeed, our experimental results show that our proposed perturbations **do lead the victim models to predict several certain classes**. For example, about 70% of misclassified adversarial examples (crafted by HIT w/ 6x6 Circle) can mislead ResNet to predict "shower curtain (class 794)". Interestingly, we find that the HFC of the images from class 794 are very similar to our perturbations. For this phenomenon, we suspect it might be because that our proposed perturbation dominates the overall features of the image, and instead the original features of the image become noise. _**More details can be found in Appendix Sec. 9 of our latest revision.**_

Q3. Can we optimize the perturbation to find an optimal proto-pattern?
- If our setting is white-box or black-box, we think it is feasible to optimize a better proto-pattern (e.g. like [1]).
However, we highlight that our main focus is to build a more practical no-box threat model, i.e., **model-free** and **data-free**. Besides, we hope our method is easy to follow and can efficiently evaluate the robustness of the model (i.e. **training-free**).
Therefore, we do not optimize the perturbation to find an optimal proto-pattern in our paper. Nonetheless, we thank the reviewer for suggesting an interesting direction for extending our work. We agree that this is a very promising direction especially for other less challenging threat models, i.e., black-box. We hope our work can inspire more future works in this direction.

[1] Li et al.  Regional homogeneity: Towards learning transferable universal adversarial perturbations against defenses.

 To all reviewers,

We have spent tremendous effort on addressing the concerns in the rebuttal. Could you take the time to check our reply and confirm whether your concerns are well addressed? Your further reply would be very appreciated. Thank you for your reply in advance.



 We are grateful to all reviewers for their comments on our work!

To dispel the confusion of the reviewers, we spent a great deal of effort on the Revision (**the latest Revision is updated today**).  Considering that the first discussion period is about to be closed and there are no responses to our work, we hope that the reviewers could give us some responses to our rebuttals. If the reviewers have more concerns, we would like to have an opportunity to further address them.

Thanks!

 _**Experiment 1: Discussion on $\lambda$ for different datasets**_

**Details:** Here we conduct the experiment about $\lambda$ for different datasets.  Specifically, the target model is DCL (backbone: Res-50) [a] and the maximum perturbation $\epsilon=16.0$. We consider three well-known dataset: CUB-200-2011 [b], Stanford Cars [c] and FGVC aircraft [d]. The resolution of inputs is $448\times448$. We report the results of our HIT (w/ 6$\times$6 Circle) on fine-grained classification models in the following Table a. 

Although set $\lambda=1.0$ is not optimal, the gap between the best results and the results of $\lambda=1.0$ is very small. Therefore, **our default setting for HIT is still applicable.**


Table a. The average attack success rate (%) of our HIT w/ 6$\times$6 Circle w.r.t $\lambda$. Our attack performs on the testing datasets. 

| $\lambda$ 	| CUB-200-2011 	| Stanford Cars 	| FGVC aircraft 	|
|:---:	|:---:	|:---:	|:---:	|
| 0.2 	| 51.90  	| 63.72  	| 37.92  	|
| 0.4 	| 67.18  	| 83.92  	| 64.03  	|
| 0.6 	| **68.31**  	| 87.55  	| 71.96  	|
| 0.8 	| 67.97  	| **88.36**  	| 75.34  	|
| 1.0 	| 67.26  	| 88.25  	| 76.73  	|
| 1.2 	| 66.54  	| 87.99  	| **76.77**  	|
| 1.4 	| 66.13  	| 87.66  	| 76.41  	|
| 1.6 	| 65.88  	| 87.28  	| 76.28  	|
| 1.8 	| 65.46  	| 87.06  	| 76.08  	|
| 2.0 	| 65.22  	| 86.93  	| 75.76  	|

_**Experiment 2: Discussion on smaller $\epsilon=8$**_

**Details:** In this experiment, we report the average success rate (%) between state-of-the-art black-box attacks and our proposed no-box attack under $\epsilon=8$. Notably, our approach is comparable to state-of-the-art black-box algorithms and even surpasses them in some cases. As demonstrated in Table b, when the victim model is Shuffle, **our "HIT w/ 6$\times$6" can achieve a 71.45% success rate.** However, state-of-the-art PI-MI-DI-FGSM only obtains a 36.79% success rate, which is significantly lower than ours.

Table b. The average success rate (%) between state-of-the-art black-box attacks and our proposed no-box attack. For black-box attacks, the substitute model is Inc-v3. The maximum perturbation $\epsilon=8$.

| Attacks 	| VGG19 	| ResNet 	| DenseNet 	| WRN 	| SENet 	| PNA 	| Shuffle 	| Squeeze 	| Mobile 	| AVG. 	|
|:---:	|:---:	|:---:	|:---:	|:---:	|:---:	|:---:	|:---:	|:---:	|:---:	|:---:	|
| MI-FGSM 	| 21.26  	| 15.63  	| 20.47  	| 14.45  	| 11.58  	| 23.36  	| 24.13  	| 32.61  	| 25.64  	| 21.01  	|
| DI-FGSM 	| 18.41  	| 11.24  	| 16.44  	| 10.19  	| 8.28  	| 17.59  	| 15.03  	| 18.40  	| 18.20  	|  14.86	|
| PI-FGSM 	| 24.12  	| 14.98  	| 22.91  	| 15.38  	| 12.21  	| 27.32  	| 25.93  	| 39.20  	| 27.29  	| 23.26  	|
| PI-MI-DI-FGSM 	| **40.88**  |	31.02 |	**41.34** |	29.70 |	**25.56** 	|**38.46** |	36.79 |	46.70	|42.38|	36.98|
| HIT w/ 6$\times$6 Circle 	| 37.64 	| **36.21**  	| 40.80 	| **30.82**  	| 21.36  	| 37.63 	| **71.45**  	| **79.34**  	| **69.90**  	| **47.24**  	|
| HIT w/ 6$\times$6 Square 	| 13.40  	| 17.50  	| 25.85  	| 21.99  	| 13.79  	| 18.31  	| 53.03  	| 61.61  	| 50.88  	| 30.71 	|
| HIT w/ 6$\times$6 Rhombus 	| 17.24  	| 19.95  	| 27.65  	| 22.00  	| 12.85  	| 18.91  	| 58.84  	| 63.49  	| 57.90  	| 33.20  	| 


### Reference:
[a] Chen et al. Destruction and Construction Learning for Fine-grained Image Recognition. (CVPR 2019)

[b] http://www.vision.caltech.edu/visipedia/CUB-200-2011.html

[c] http://ai.stanford.edu/~jkrause/cars/car_dataset.html

[d] https://www.robots.ox.ac.uk/~vgg/data/fgvc-aircraft/

 We thank all the reviewers for their constructive comments and insightful suggestions. We carefully revise the manuscript according to the comments of all the reviewers. For convenience, we highlight the revised text in color except for minor modifications. Here we briefly summarize the updates for the revision:
- We alter the position of Figure 1 to better explain it
- We add additional discussion on several related works
- We illustrated adversarial examples (in Appendix A.4) crafted via our competitors and our HIT.  Notably, our adversarial examples are less perceptible.
- We add experiment (in Appendix Sec. A.5) for attacking other classification tasks (e.g. CUB-200-2011) and discussing hyperparameter settings in this case.
- We add experiment (in Appendix Sec. A.6) for attacking a real-world recognition system, i.e., Google Cloud Vision API.
- We add experiment (in Appendix Sec. A.7) for Ghost Networks (Li et al.)
- We add experiment (in Appendix Sec. A.7) for using a smaller epsilon setting.

------------------------------------------------------21 Nov Update------------------------------------------------------
- We add discussion (in Appendix Sec. A.9) on the targeted attack potential of our HIT.


 _**Q4:** So it would be expected that the proposed approach that relies on the effect of HFC would not be better than gradient based attacks._

**A4:** Feature denoising models [e] do have stronger robustness to high-frequency noise. Our HIT only reduces the top-1 accuracy by 5%, which is not significant despite surpassing the gradient-based attacks. For this question, there are two main reasons that might explain it:
1. In our manuscript, gradient-based attacks craft adversarial examples via a substitute model. Therefore, if the gap between the substitute and target models is very large, the transferability of adversarial examples will be very low. However, our HIT is based on the observation that DNNs are vulnerable to regionally homogeneous, repeating and dense noise. **Therefore, our method does not overfit to the substitute model and may sometimes yield more transferable adversarial examples.**
2. For our HIT, HFC=image-LFC. **Therefore, if the kernel size of the Gaussian kernel in Eq. 2 and tile size are big, the resulting HFC will contain more relatively low-frequency components (we also illustrate the resulting HFC in Appendix Sec.9). Besides, by adding our HFC to an image, the original LFC of an image can also be destroyed.** Consequently, our HIT can circumvent the defense of [e] to some extent.

_**Q5:** If an auxiliary classifier (e.g. Metzen et al. arXiv:1702.04267) can be an effective defense for the proposed approach which relies on feature statistics from HFC._

**A5:** We believe that it would not be challenging to train an auxiliary classifier to detect our adversarial examples. Similarly, such a classifier can also detect the gradient-based methods and the gradient-based method can be trained to reattack the classifier. However, we do not believe that our approach is inferior to gradient-based methods in this sense. The main reason is that we focus on the no-box scenario, which is more practical yet challenging.



_**Q6:** Fig 1 and related work section._

**A6:** Thanks very much for your valuable suggestions on this manuscript. We have altered the position of Fig 1 to be near the description of this image. For the related work section, we have added an introduction about how adversarial patches work.




### Reference:

[e] Xie et al. Feature denoising for improving adversarial robustness. (CVPR 2019)

 Thanks for your positive feedback and insightful comments. We apologize for not being able to respond to you in a timely manner because some of the experiments are very time-consuming (e.g. train several target models from scratch to test attacks).


_**Q1:** More qualitative examples of the proposed approach compared with the examples from Li et al. can improve be helpful._

**A1:** Thanks for your suggestion. We provide more qualitative examples (maximum perturbation $\epsilon=16.0$) in the following anonymous link:
https://imgur.com/oRTLSJj (also can be found in Figure 9 of our revision). 
Compared with state-of-the-art black-box PI-FGSM and no-box attack (Li et al.), **the adversarial examples crafted by our no-box HIT are less perceptible.**

_**Q2:** It would be interesting to see the effectiveness of the proposed approach for other datasets such as Places365 or even other vision tasks._

**A2:** Thanks for your interest in our work. Given that we are more familiar with fine-grained classification, here we report the results of our HIT (w/ Circle) on fine-grained classification models. Specifically, the target model is DCL (backbone: Res-50) [a] and the maximum perturbation $\epsilon=16.0$. We consider three well-known dataset: CUB-200-2011 [b], Stanford Cars [c] and FGVC aircraft [d]. The resolution of inputs is $448\times448$. Therefore, we set the "tile size = 448 / tile scheme". For example, if the tile scheme is $4\times4$, and then the tile size is 112. 

We first report the average attack success rate (%) of our HIT w/ Circle w.r.t tile-scheme in Table a. From the result, we can observe that our HIT is also effective for attacking other datasets. **Notably, our HIT can fool DCL with about 90% success rate on Stanford Cars dataset.** Besides, a relatively smaller tile-size is also helpful in improving the success rate of the attack, which is consistent with the conjecture (i.e. Sec. 3.2) we presented in our manuscript. We also conduct the experiment about $\lambda$. Although set $\lambda=1$ is not optimal, the gap between the best results and the results of $\lambda=1$ is very small. Therefore, our default setting for HIT is still applicable.

Table a. The average attack success rate (%) of our HIT w/ Circle w.r.t tile-scheme. Our attack performs on the testing datasets. 

| Attacks 	| CUB-200-2011 	| Stanford Cars 	| FGVC aircraft 	|
|:---:	|:---:	|:---:	|:---:	|
| 1$\times$1 	| 30.07  	| 28.46  	| 43.81  	|
| 2$\times$2 	| 38.87  	| 54.11  	| 55.47  	|
| 3$\times$3 	| 51.22  	| 81.08  	| 73.16  	|
| 4$\times$4 	| 55.19  	| 80.18  	| 74.04  	|
| 5$\times$5 	| 59.23  	| 84.94  	| 70.95  	|
| 6$\times$6 	| 67.30  	| 88.25  	| **76.71**  	|
| 7$\times$7 	| **70.19**  	| **88.46**  	| 74.69  	|


Table b. The average attack success rate (%) of our HIT w/ 6$\times$6 Circle w.r.t $\lambda$. Our attack performs on the testing datasets. 

| $\lambda$ 	| CUB-200-2011 	| Stanford Cars 	| FGVC aircraft 	|
|:---:	|:---:	|:---:	|:---:	|
| 0.2 	| 51.90  	| 63.72  	| 37.92  	|
| 0.4 	| 67.18  	| 83.92  	| 64.03  	|
| 0.6 	| **68.31**  	| 87.55  	| 71.96  	|
| 0.8 	| 67.97  	| **88.36**  	| 75.34  	|
| 1.0 	| 67.26  	| 88.25  	| 76.73  	|
| 1.2 	| 66.54  	| 87.99  	| **76.77**  	|
| 1.4 	| 66.13  	| 87.66  	| 76.41  	|
| 1.6 	| 65.88  	| 87.28  	| 76.28  	|
| 1.8 	| 65.46  	| 87.06  	| 76.08  	|
| 2.0 	| 65.22  	| 86.93  	| 75.76  	|


_**Q3:** Attack a real world recognition system as done in Li et al._

**A3:** We provide several examples for attacking Google Cloud Vision API in the following anonymous link: https://imgur.com/RsAIgbY (also can be found in Figure 10 of our revision).  Notably, our no-box HIT can effectively change top-k labels.




### Reference:
[a] Chen et al. Destruction and Construction Learning for Fine-grained Image Recognition. (CVPR 2019)

[b] http://www.vision.caltech.edu/visipedia/CUB-200-2011.html

[c] http://ai.stanford.edu/~jkrause/cars/car_dataset.html

[d] https://www.robots.ox.ac.uk/~vgg/data/fgvc-aircraft/

 Thanks for your encouraging remark! We will answer your questions in the following:

Q1. Why is epsilon set to 16?
- In our paper, **the $\epsilon$ is set according to the default settings of comparison methods**. For example, Li et al. set $\epsilon$ to 25.5 for their no-box attack, hence our Table 1 also adopts this setting. For most transfer-based attacks (e.g., MI-FGSM and PI-FGSM), they usually set $\epsilon$ to 16. Therefore, Table 2 and Table 3 set $\epsilon$ to 16. Besides, in Figure 6, we also provide the results of our HIT under different $\epsilon$ settings (from 2 to 26). 

Q2. Would an adversary need to tune $\lambda$ for different datasets or not?
- In Sec.3 of our Appendix, we can observe a phenomenon that the growth trend of average attack success rate gradually tends to level off when $\lambda$ exceeds 0.5. This suggests that our proposed methods are not sensitive to $\lambda$ (under $\ell_\infty$ constraint).  Therefore, **in practice, our recommendation is that an adversary can simply adopt this default setting (i.e. $\lambda=1.0$)**. If the adversary is not satisfied with the results, he/she can consider fine-tuning it to see if it can bring some improvement.



Following the scenario of No-box Attacks, they propose a simple but efficient method, that crafts adversarial perturbations by three types of handmade images, i.e., concentric circles, concentric squares, and concentric rhombus, and add them to the high-frequency component of benign samples. Experimental results show that their handmade perturbations significantly outperform previous No-box attack methods, even competitively to some black-box methods.

 The authors separate a benign image into two components: LFC(low-frequency component) and HFC(high-frequency component). They demonstrate that the high-frequency component is an essential part of the DNN model's performance. So they add some handmade images in the high-frequency part of benign images. The method seems novel, and the paper is well written and easy to follow. I think it is an interesting paper.
Here I have some concerns:
1. It is a little weak about the discussion of why the circle always performs better. Do the circle/square/rhombus perturbations always lead the victim models to predict certain classes?
2. Can we optimize the perturbation to find an optimal proto-pattern? I think the contribution about HPC is not novel, since some works have pointed it out. The contributions of this paper are only marginally significant or novel.

The authors propose a new method for generating adversarial images for image classifier. This is a No-Box attack named Hybrid Image Transformation (HIT) or hit attack which is both model free and data-free (no training required). In the experiment section the authors show the efficacy of the proposed attack on the ImageNet dataset for different DNNs. The authors start the discussion by revisiting the properties of both white-box and black-box adversarial image classification attack. They highlight some important characteristics of such attacks and argue why most proposed attacks may not be useful in practice. Based on this discussion they argue in favor of previously proposed No-Box attack. A No-Box attack must be both model-free and data-free so that it is not dependent on the training process of one particular Neural Network. 

The authors then propose an attack named HIT which is training free and uses the LFC and HFC components of an image to craft the adversarial image. As can be inferred from the previous statement, the main contributors of this attack are the LFC and HFC components of an image. To better understand the role of these components, the authors first split the raw images into these two pieces using Gaussian low-pass filter. This analysis shows that the DNNs rely on HFC components to classify images where as the LFC parts are more visible to humans. This finding helped the authors come to the conjectures that effective adversarial HFCs must be regional homegeneous, repeating and dense. They then craft the HIT attacks by adding such adversarial HFCs to the LFC of an image and then clipping the image to fall in an predefined inf-norm ball.

The authors then perform experiments on the ImageNet dataset with different DNNs. Their attack outperforms the previous state-of-the-art     by a large margin. Furthermore, their attack performs well when compared to Black-Box attacks as well.  I like this paper overall. It is easy to read and follow the argument. Furthermore, it sheds new light on the classification logic of DNNs with respect to adversarial examples. The attack methods they is also very strong and widely applicable as shown by the experimental results. 

Questions for Authors
1. Why is epsilon set to 16? Usually this is set set to 8 to the best of my knowledge.
2. I have read the authors discussion regarding lambda and why a value of 1.0 is chosen. I would like to know if this lambda depends on some property of the dataset i.e., would an adversary need to tune lambda for different datasets or not?

Meta-review: This paper proposed a no-box attack method, with mixed reviews. While I appreciate the authors efforts which attempted to address the questions raised by the reviewers, they were not sufficient to remove the concerns. Based on  the reviewers comments and author responses, I won't be able to recommend the paper to be accepted in its current form.

Decision: Reject