TRAINING-FREE HYBRID IMAGE TRANSFORMATION
Year: 2021

Abstract:
In recent years, the adversarial vulnerability of deep neural networks (DNNs) has raised increasing attention. Among all the threat models, no-box attacks are the most practical but extremely challenging since they neither rely on any knowledge of the target model or similar substitute model, nor access the dataset for training a new substitute model. Although a recent method has attempted such an attack in a loose sense, its performance is not good enough and the computational overhead of training is expensive. In this paper, we move a step forward and show the existence of a training-free adversarial perturbation under the nobox threat model, which can be successfully used to attack different DNNs in real-time. Motivated by our observation that high-frequency component (HFC) domains in low-level features and plays a crucial role in classification, we attack an image mainly by manipulating its frequency components. Specifically, the perturbation is combined by the suppression of the original HFC and the adding of noisy HFC. We empirically and experimentally analyze the requirements of effective noisy HFC and show that it should be regionally homogeneous, repeating and dense. Extensive experiments on the ImageNet dataset demonstrate the effectiveness of our proposed no-box method. It attacks ten well-known models with a success rate of 98.13% on average, which outperforms state-of-the-art no-box attacks by 29.39%. Furthermore, our method is even competitive to mainstream transfer-based black-box attacks. Our code is available in our appendix.

heading: 1 INTRODUCTION
text: Deep neural networks (DNNs) are widely known to be vulnerable to adversarial examples (Szegedy et al., 2013; Goodfellow et al., 2015), i.e., a human-imperceptible perturbation can lead to misclassification. In adversarial machine learning, the term threat model defines the rules of the attack, such as the resources the attacker can access. Based on the threat model, the attacks are often divided into white-box attacks and black-box attacks. In the white-box threat model (Szegedy et al., 2013; Goodfellow et al., 2015; Madry et al., 2018a), the attacker has full knowledge of a target model, such as the model weights and the whole training dataset. Recognizing the threat of these adversarial attacks, a model owner is unlikely to leak a model’s information to the public. Thus, the white-box attack is often used to evaluate the model robustness for revealing its weakest point (Madry et al., 2018a), but often not considered as a practical attack method (Chen et al., 2017). To this end, numerous works have investigated a more realistic threat model, where the attacker does not require full knowledge of the target model, i.e., the backpropagation on the target model is prohibited. This threat model is called black-box attack (Papernot et al., 2016; Tramèr et al., 2016; Papernot et al., 2017; Narodytska & Kasiviswanathan, 2017; Chen et al., 2017; Brendel et al., 2017; Dong et al., 2019b; Yan et al., 2019; Chen et al., 2020; Zhou et al., 2020). However, such a black-box threat model usually involves a major concern of being resource-intensive in terms of query cost and time. In real-world attack scenarios, even if we ignore such concerns, query-based black-box attack can still be infeasible, e.g., the model API is inaccessible to the attacker. Moreover, it might cause suspicion due to repeated queries to the model with almost the same adversarial image. To alleviate this issue, another line of black-box threat model (Dong et al., 2018; Xie et al., 2019b; Dong et al., 2019a; Wu et al., 2020; Lin et al., 2020; Gao et al., 2020a; 2021) called transfer-based attack is proposed. In this threat model, adversarial examples are crafted via the local available pre-trained substitute model, which usually trains on the same training dataset as the target model. The resultant adversarial examples
are expected to attack the target model. However, without the feedback from the target model, the transferability heavily depends on how large the gap between the substitute model and target model. In practice, this gap is large because the structure and the training technique of the target model are usually not publicly available due to security and privacy concerns.
From the analysis above, we argue that both white-box and black-box attacks can hardly be considered as practical attacks. A practical attack should satisfy two criteria: (a) model-free, i.e., no dependence on the pre-trained substitute model or the target model for either backward propagation or only forward query; (b) data-free, i.e., no dependence on the dataset for training a substitute model. We term it no-box attack. A recent work (Li et al., 2020a) is the first (to our knowledge) as well as the only work to have attempted such an attack in a loose sense. Their threat model still requires a small number of auxiliary samples, such as 20 images. Admittedly, collecting a small number of samples might not be difficult in most cases, but might be still infeasible in some security-sensitive applications. Specifically, their approach (Li et al., 2020a) attempts to train a substitute model by adopting the classical auto-encoder model instead of the supervised classification model due to the constraint of a small-scale dataset. Overall, to attack a certain sample, their approach consists of three steps: (1) collecting a small number of images; (2) training a substitute model; (3) white-box attack on the substitute model. If a new sample, especially from a different class, needs to be attacked, the above process needs to be repeated. Thus, their approach is very resource-intensive. Besides, their attack success rate is still significantly lower than existing black-box attacks.
By contrast, our approach does not require any of the above three steps and is even training-free. With the help of visualization technique proposed by (Zeiler & Fergus, 2014), we observe that the high-frequency component (HFC), e.g., the edge and texture features, is dominant in shallow layers and the low-frequency component (LFC), e.g., the plain areas in the image, is paid less attention to be extracted. Combined with the insight
into the classification logic of DNNs in Sec. 3.1, we observe that HFC plays a crucial role in recognition. As shown in Fig. 1, without LFC, the confidence of HFC is even higher than the raw image. Although it does not hold true for all samples, it does demonstrate the importance of HFC.
Motivated by this, we take the idea of hybrid image (Oliva, 2013) and propose a novel Hybrid Image Transformation (HIT) attack method to craft adversarial examples. Formally, it only needs three steps but can effectively fool various DNNs without any training: First, due to the trainingfree setting and inspired by the analysis from Sec. 3.2, we simply utilize matplotlib1 tool to draw several geometric patterns which serve as the proto-patterns, and the resultant synthesized adversarial patches are thus richer in regionally homogeneous, repeating and dense HFC. Second, we extract the LFC of the raw image and HFC of the adversarial patch. Finally, we combine these two pieces of components and clip them to the ε-ball of the raw image to get the resultant adversarial hybrid example. Extensive experiments on ImageNet demonstrate the effectiveness of our method. By attacking ten state-of-the-art models in the no-box manner, our HIT significantly increases the average success rate from 68.74% to 98.13%. Notably, our HIT is even competitive to mainstream transfer-based black-box attacks.

heading: 2 RELATED WORK
text: Adversarial Attack. Let x denote raw image without any perturbation, xadv and y denote the corresponding adversarial example and true label respectively. In generally, we use l∞-norm to measure the perceptibility of adversarial perturbations, i.e., ||xadv − x||∞ ≤ ε. In this paper, we focus on non-targeted attacks (Dong et al., 2018; Xie et al., 2019b; Wu et al., 2020; Lin et al., 2020; Gao et al., 2020a) which aim to cause misclassification of DNNs f(·), i.e., f(xadv) 6= y.
1https://matplotlib.org/
Competitors. Transferability is an important property for adversarial examples. With it, the resultant adversarial example crafted via one model may fool others. For the black-box threat model, Goodfellow et al. (2015) argue that the vulnerability of DNNs is their linear nature, and generate adversarial examples efficiently by performing FGSM which is a single-step attack. Papernot et al. (2017) train a local model with many queries to substitute for the target model. Dong et al. (2018) integrate a momentum term into I-FGSM Kurakin et al. (2017) to stabilize the update direction during the attack iterations. Xie et al. (2019b) apply diverse input patterns to improve the transferability of adversarial examples. Dong et al. (2019a) propose a translation-invariant attack to mitigate the effect of different discriminative regions between models. Gao et al. (2020a) introduce patch-wise perturbation by amplifying the step size and reuse the cut noise to perturb more information in discriminative regions. For the no-box threat model, Li et al. (2020a) attempt to attack the target model without any model query or the accessible pre-trained substitute model. In their work, with a limited amount of data, they try different mechanisms (with or without supervised technique) to train the substitute model, and then utilize this substitute model to craft transferable adversarial examples. Different from these approaches, our method does not depend on transferability since we do not need any substitute model. In this paper, we craft the adversarial examples from the perspective of the classification logic of DNNs.
Frequency Perspective on DNNs. Our approach is highly inspired by existing works which explain the generalization and adversarial vulnerability of DNNs from the frequency perspective. The fact that DNNs have good generalization while being vulnerable to small adversarial perturbations has motivated (Jo & Bengio, 2017; Wang et al., 2020) to investigate the underlying mechanism, suggesting that surface-statistical content with high-frequency property is essential for the classification task. From the perspective of texture vs. shape, Geirhos et al. (2019); Wang et al. (2020) reveal that DNNs are biased towards texture instead of shape. Since the texture content is considered to have high-frequency property, their finding can be interpreted as the DNN being biased towards HFC. On the other hand, adversarial perturbations are also known to have the high-frequency property and various defense methods have also been motivated from this insight (Aydemir et al., 2018; Das et al., 2018; Liu & JaJa, 2019; Xie et al., 2019a). Nonetheless, it remains unknown whether manually designed high-frequency patterns are sufficient for attacking the network.

heading: 3 METHODOLOGY
text: Although many adversarial attack methods (Papernot et al., 2016; Dong et al., 2018; Gao et al., 2020a; Li et al., 2020a) have achieved pretty high success rates in both black-box and no-box cases, they all need training, especially for query-based (Papernot et al., 2016; Zhou et al., 2020) and nobox adversarial perturbations (Li et al., 2020a) whose training is usually time-consuming. Then a natural question arises: Is it possible to generate robust adversarial perturbations without any training? In the following subsections, we will give our answer and introduce our design.

heading: 3.1 MOTIVATION
text: To better understand the role of HFC and LFC for the classification results of DNNs, we split the information of raw images into these two pieces via Gaussian low-pass filter (defined in Eq. 1).
As illustrated in Fig. 3, when the kernel size is small, i.e., the cutoff frequency is high, the average accuracy of LFC on ten state-of-the-art models is close to 100%. However, if we continue to increase the kernel size, the average accuracy of HFC begins to exceed LFC one. To our surprise, for several specific raw images, e.g., left image of Fig. 1, the true label’s confidence of HFC which is mostly black is even higher than the raw image.
To explain the above phenomenon, we turn to the perspective of feature space. Inspired by recent intermediate feature-based attacks (Zhou et al., 2018; Ganeshan & Babu, 2019; Inkawhich et al., 2019), we argue low-level features are critical to the classification. Interestingly, as shown in Fig. 2, most2 feature maps in
the shallow layers generally extract the edge and texture features (typical ones are highlighted by red boxes), i.e., HFC, and pay less attention to plain areas in images, i.e., LFC. Therefore, if a perturbation can effectively manipulate the HFC of an image, totally different low-level features will be extracted and may lead to misclassification.

heading: 3.2 EFFECTIVE ADVERSARIAL HFC
text: However, what kind of training-free noisy HFC can effectively fool DNNs is still unknown because the performance of any other raw image’s HFC is unsatisfactory (see Appendix Sec. A.8). Zhang et al. (2020) have demonstrated that the effectiveness of adversarial perturbation lies in the fact that it contains irrelevant features. The features of perturbation dominate over the features in the raw image, thus leading to misclassification. Inspired by their finding, we intend to design adversarial HFC with strong irrelevant features, and we conjecture that the following properties are essential.
Regionally Homogeneous. Several recent works (Li et al., 2020b; Gao et al., 2020a; Dong et al., 2019a; Gao et al., 2020b) have demonstrated that adversarial perturbations with regionally homogeneous (or patch-wise (Gao et al., 2020a)) property can enhance the transferability of adversarial examples. Inspired by that the raw image is a composite of homogeneous patterns, the reason might be attributed to that this perturbation tend to form irrelevant features recognizable by the DNNs.
Repeating. Nguyen et al. (2015) observe that extra copies of the repeating element do improve the confidence of DNNs. From the perspective of strengthening the irrelevant features, it is expected that repeating the content is beneficial.
Dense. Analogous to the above repeating property that performs global repeating, i.e., increases the amount of irrelevant features globally, we can also perform local repeating to strengthen its adversarial effect further. For term distinction, we term this property dense.
To verify the effect of the above properties, we conduct the ablation study in Sec. 4.1, and results support our conjecture. Besides, the analysis in Appendix Sec. A.9 also show that our HIT has potential to become a targeted attack.

heading: 3.3 HYBRID IMAGE TRANSFORMATION
text: Motivated by the above discussion, we take the idea of hybrid image (Oliva, 2013) to apply our no-box attacks. Specifically, Oliva (2013) replaces the HFC of one image with the HFC of another carefully picked image and craft hybrid images with two different interpretations: one that appears when the image is viewed up-close, and the other that appears from afar (see Fig.5 of Oliva (2013)). However, confusing human’s vision system (without ε constrain) cannot guarantee the misclassification of DNNs since adversarial examples are constrained by the maximum perturbation. Therefore, we propose a novel Hybrid Image Transformation (HIT) attack method which reduces3 original
2see quantitative analysis in Appendix Sec. A.2. 3Due to the ε constraint, we can not completely replace HFC with others
HFC, and meanwhile, adds well-designed noisy ones to attack DNNs. Our method only needs three steps but can generate robust training-free adversarial perturbations in real time:
First, we provide an adversarial patch xp to generate noisy HFC. Unlike the traditional way that needs training, here we use the matplotlib tool to draw it. Inspired by the observation in Sec. 3.2, we consider three simple regionally homogeneous proto-patterns (to avoid cherry-picking) as our basic adversarial patches: concentric circles, concentric squares, and concentric rhombus in Fig. 4. The effect of concentric pattern is to make the resultant HFC dense. Then we repeat these adversarial patches.
Second, we extract the LFC of the raw image and the HFC of the adversarial patch. Note that several methods can be utilized to extract the HFC and LFC of an image, e.g., Fourier transformation. In this paper, we use an approximated yet simple Gaussian low-pass filterGwhose size is (4k+1)×(4k+1) to get LFC, which can be written as:
Gi,j = 1 2πσ2 e(− i2+j2 2σ2 ), (1)
where σ = k determines the width of our G. In general, the larger σ is, the more HFC is filtered out. We are not going to introduce a new high-pass filter here for simplicity and just get HFC byG. More specifically, we obtain HFC by subtracting the LFC of the adversarial patch.
Finally, we can synthesize these two part components to generate our adversarial hybrid image xadv: xadv = clipx ,ε(x ∗G+ λ · (xp − xp ∗G)), (2) where “*” denotes convolution operation, λ is a weight factor to balance the LFC and HFC, and clipx,ε(·) restricts the resultant adversarial examples within the ε-ball of the raw image in l∞ space. Therefore, our method is different from adversarial patch attacks (Brown et al., 2017; Liu et al., 2020) which replace a subregion of the image with a well-design patch.
As illustrated in Fig. 2(c), our HIT can effectively reduce relevant HFC and add many other irrelevant noisy ones, e.g., highlighted yellow boxes in (c) cannot find any obvious HFC associated with “cat” at all. As a result, the target model can not extract correct features to make a reasonable prediction, thus leading to misclassification. Besides, our adversarial examples are less perceptible than those of our competitors (See Appendix Sec. A.4).

heading: 4 EXPERIMENTS
text: Networks. Here we consider ten well-known classification models: VGG19 (Simonyan & Zisserman, 2015), Inception-v3 (Inc-v3) (Szegedy et al., 2016), ResNet-152 (ResNet) (He et al., 2016), DenseNet-121 (Dense) (Huang et al., 2017), WideResNet (WRN) (Zagoruyko & Komodakis, 2016), SENet (Hu et al., 2018), PNASNet (PNA) (Liu et al., 2018), ShuffleNet-v2 (Shuffle) (Ma et al., 2018), SqueezeNet (Squeeze) (Iandola et al., 2017) and MobileNet-v2 (Mobile) (Sandler et al., 2018) as our target models. All the models are available in the Torchvision4, except for PNA and SENet which are obtained from Github5. We also perform our attack on a real-world recognition system in Appendix Sec. A.6.
4https://github.com/pytorch/vision/tree/master/torchvision/models 5https://github.com/Cadene/pretrained-models.pytorch
Dataset. To make our method more convincing and avoid cherry-picking, we choose 10,000 images (each category contains about 10 images which are resized to 299 × 299 × 3 beforehand) from the ImageNet validation set (Russakovsky et al., 2015) which are classified correctly by all ten networks we consider. We also discuss our methods on other classification tasks in Appendix Sec. A.5.
Parameters. In our experiments, we use l∞-norm to measure the perceptibility of adversarial noises, unless specified, the maximum perturbation ε is set to 16 (results with a smaller ε can be found in Appendix Sec. A.7). For our HIT, the size of Gaussian kernel G is 17 × 17 (i.e. k = 4), weight factor λ is set to 1.0 (the discussion about λ is shown in Appendix Sec. A.3), and density of proto-pattern is set to 12. For tile-size, unless specified, we set to 50 × 50, i.e., tile-scheme is 6× 6. For no-box methods, we follow the same setting as (Li et al., 2020a). For black-box methods, the iteration T is set to 10 and the step size α is 1.6. For MI-FGSM (Dong et al., 2018), we adopt the default decay factor µ = 1.0. For DI2-FGSM (Xie et al., 2019b), we set the transformation probability to 0.7. For TI-FGSM (Dong et al., 2019a), the length of Gaussian kernel is 15. For PI-FGSM (Gao et al., 2020a), the length of project kernel is 3, the amplification factor β and project factor γ are 10.0 and 16.0, respectively. Different from PI-FGSM, β and γ for PI-MI-DI2-FGSM and PI-TI-DI2-FGSM (Gao et al., 2020a) is 2.5 and 2.0.

heading: 4.1 ABLATION STUDY
text: In this section, we conduct a series of ablation study for our HIT. Specifically, we investigate the effectiveness of regionally homogeneous pattern, repeating pattern and dense pattern in Sec. 4.1.1, Sec. 4.1.2 and Sec. 4.1.3, respectively. Besides, we also analyze the effect of perturbation size on the performance in Sec. 4.1.4. For the result of HIT without reducing HFC beforehand is shown in Appendix Tab. A.12.

heading: 4.1.1 THE EFFECT OF REGIONALLY HOMOGENEOUS PATTERN
text: To the best of our knowledge, regionally homogeneous perturbations (Dong et al., 2019a; Gao et al., 2020a;b; Li et al., 2020b) are mostly based on the gradient to craft, thereby training is necessary. However, whether arbitrary noise can benefit from the homogeneous property remains unclear. Therefore, we compare random noises with semi-random ones to check it:
Random noise: For a given random location pair set L, we callNr ∈ RH×W×C random noise if it meets the following formula:
Nr[i, j, c] = { ε · random(−1, 1), (i, j, c) ∈ L 0, else
(3)
Semi-random noise: Different from the random noise, semi-random noise has some regularity. Let S denotes a semi-random location pair set, and here we take H-dimension random noise as an example. Nsr can be written as:
Nsr[i, :, :] = { ε · random(−1, 1), i ∈ S 0, else
(4)
where random(-1, 1) returns 1 or -1 randomly. As depicted in Fig. 5, the success rates ofNsr are consistently higher than those of Nr. As the number of perturbed pixels increases, the margin between them also increases. This demonstrates that training-free noise can also benefit from regionally homogeneous property. To exploit this conclusion further, in Fig. 4, we extend semi-random noise to other more complex “continuous” patterns, e.g., circle.

heading: 4.1.2 THE EFFECT OF REPEATING PATTERN
text: In this section, we show the experimental results of our proposed HIT w.r.t different tile-sizes. Here we consider seven different tile-schemes including 1 × 1, 2 × 2, 3 × 3, 4 × 4, 5 × 5, 6 × 6 and 7× 7, and the tile-sizes thereby are 300× 300, 150× 150, 100× 100, 75× 75, 60× 60, 50× 50, 42 × 42, respectively. We will resize back to 299 × 299 × 3 to match the size of raw images. The visualizations of these patches can be found in Appendix Sec. A.11.
In Fig. 5, we report the average attack success rates of ten models. The success rates increase very quickly at first and then keep stable after 4× 4 tile-scheme. If we continue to increase the tile-size, the attack success rates may go down. The main reason might be that the distortion caused by the resizing operation. It indirectly blurs resultant tiled adversarial patches, thus reducing the available HFC. Compared to the other two geometric patterns, we find that circle patches always perform the best. For example, the success rate is up to 88.67% when tile-size is 6× 6. This result demonstrates that the attack ability of training-free perturbations can benefit from repeating property.

heading: 4.1.3 THE EFFECT OF DENSE PATTERN
text: To validate the effect of dense pattern, we analyze the average attack success rates w.r.t densities. Since the trends of different patterns are similar, we only discuss the results of circle patch whose tile-scheme is 6 × 6. Here we control the density from 1 to 12. For example, “2” denotes only two circles in the proto-pattern, and more visualizations can be found in Appendix Sec. A.11.
As shown in Fig. 5, the success rates increase rapidly at the beginning, then remain stable after the density exceeds 8, and reach the peak at 12. This experiment demonstrates the effectiveness of dense pattern. Therefore, we set the default density of each proto-pattern to 12 in our paper.

heading: 4.1.4 THE SIZE OF PERTURBATION
text: In this section, we study the influence of the maximum perturbation ε on the performance of our HIT. The result of Fig. 6 depicts the growth trends of each model under different adversarial patches. No matter what the adversarial patch is, the performance proliferates at first, then remains stable after ε exceeds 16 for most models. Besides, the circle patch (curve-like) always performs best while the performance of the other two adversarial patches (straight-like) is similar. For example, when ε = 16 and the target model is VGG19, the attack success rate of circles patch is 94.75% while the square patch and rhombuses patch ones are 81.52% and 83.63%, respectively. This demonstrates that DNNs are more vulnerable to curve-like perturbations than straight-like ones (we also analyze the reasons for this in Appendix Sec. A.10).
Another observation from this result is that our HIT can serve as a universal attack, although not in a strict sense. As demonstrated in Fig. 6, when ε = 10 which is the common constraint for universal adversarial perturbations (Mopuri et al., 2017; Moosavi-Dezfooli et al., 2017; Mopuri et al., 2018; Reddy Mopuri et al., 2018; Liu et al., 2019; Hashemi et al., 2020), our HIT with circle patch can achieve a success rate of 63.23% on average. Notably, it can be up to 89.74% on Squeeze.

heading: 4.2 COMPARISON OF HIT WITH NO-BOX ATTACKS
text: In this section, we compare the performance of our no-box HIT with state-of-the-art no-box attacks (Li et al., 2020a). Note that Li et al. (2020a) need to pay 15,000 iterations at most to train a substitute model, and then runs extra 200 iterations baseline attacks and 100 iterations ILA (Huang et al., 2019), which is extremely time-consuming. Significantly different from Li et al. (2020a), our HIT is training-free which does not require any auxiliary images to train a substitute model, thus achieving real-time attack.
The experimental results are reported in Tab. 1. A first glance shows that our HIT outperforms Li et al. (2020a) by a large margin. No matter what the adversarial patches are, our HIT can consistently achieve a success rate of over 92% on average. By contrast, the best performance of Li et al. (2020a), i.e., Prototypical∗ w/ Sup, is only 68.74% on average. Notably, our HIT with circle patch remarkably outperforms Li et al. (2020a) by 29.39% on average and 42.04% at most when attacking PNA.

heading: 4.3 COMPARISON OF HIT WITH BLACK-BOX ATTACKS
text: In this section, we compare our no-box HIT with mainstream transfer-based attacks. For MI-FGSM, DI2-FGSM, PI-FGSM and their extensions PI-MI-DI2-FGSM, we utilize VGG19, Inc-v3, ResNet
and Dense to iteratively (ten forward & backward propagation) craft adversarial examples and use them to attack the rest of black-box models. As for our proposed HIT, we do not need any substitute model or training process. The results are summarized in Tab. 2, where the models in the leftmost column are substitute models, and the bottom block shows the results of our HIT.
As demonstrated in Tab. 2, our HIT is even on par with state-of-the-art PI-MI-DI2-FGSM. Specifically, on average, the best performance of PI-MI-DI2-FGSM is 88.84%, and our HIT based on circle patch can get up to 88.67%. However, the transferability of adversarial examples largely depends on the substitute model. For example, when adversarial examples are crafted via Inc-v3, the performance of PI-MI-DI2-FGSM is limited and our HIT can remarkably outperform it by 23.34% on average. Besides, when the target model is in lightweight models, e.g., Shuffle, our method consistently outperforms these mainstream transfer-based attacks by a large margin.
Since adversarial training technique (Madry et al., 2018b; Tramèr et al., 2018; Awasthi et al., 2021) can effectively defend against adversarial examples, we conduct an extra experiment on several defense models to demonstrate the effectiveness of our method. The additional target models including three ensemble adversarial training models (EAT) (Tramèr et al., 2018): Inc-v3ens3, Inc-v3ens4 and IncRes-v2ens, and three feature denoising models (FD) (Xie et al., 2019a): ResNet152 Baseline (Res152B), ResNet152 Denoise (Res152D) and ResNeXt101 DenoiseAll (ResNeXtDA). As demonstrated in previous works(Guo et al., 2019; Sharma et al., 2019), low-frequency perturbations are more effective for attacking defense models. Motivated by it, we change the tile-schemes to smaller ones (i.e., 2 × 2 for EAT and 1 × 1 for FD) and other parameters stay the same (see more details in Appendix Sec. A.12). As observed in Tab. 3, our HIT is effective even for defense models. Notably, HIT based on circle patch can successfully attack Inc-v3ens4 by 61.86%. Besides, for more robust FD, even crafting adversarial examples via an ensemble of VGG19, Inc-v3, ResNet and Dense, transfer-based PI-TI-DI2-FGSM is still inferior to our HIT. This experimental result reveals that current defenses have not achieved real security, which is even vulnerable to training-free adversarial examples.

heading: 5 CONCLUSION
text: In this paper, we rethink the classification logic of deep neural networks with respect to adversarial examples. We observe that HFC domains in low-level features and plays a crucial role in classification. Besides, we demonstrate that DNNs are vulnerable to training-free perturbations with regionally homogeneous, repeating, dense property through empirically and experimentally analysis. Motivated by these observations, we propose a novel Hybrid Image Transformation (HIT) attack method by combining the LFC of raw images with the HFC of our well-designed adversarial patches to destroy the useful features and add strong irrelevant noisy ones. Extensive experiments on the ImageNet dataset demonstrate the effectiveness of the proposed method. Surprisingly, our simple method outperforms existing no-box attacks by a significant margin and is even on par with transfer-based black-box attacks that require the substitute model to craft adversarial examples.
In another aspect, since most models are vulnerable to our method, it implies that our adversarial examples may capture the common “blind spots” of them. Therefore, a defense can improve the robustness and stability by covering these “blind spots”, i.e., applying data augmentation technique using our adversarial examples.

heading: A APPENDIX

heading: A.1 SETUP
text: Networks. Here we consider ten well-known classification models: VGG19 Simonyan & Zisserman (2015), Inception-v3 (Inc-v3) Szegedy et al. (2016), ResNet-152 (ResNet) He et al. (2016), DenseNet-121 (Dense) Huang et al. (2017), WideResNet (WRN) Zagoruyko & Komodakis (2016), SENet Hu et al. (2018), PNASNet (PNA) Liu et al. (2018), ShuffleNet-v2 (Shuffle) Ma et al. (2018), SqueezeNet (Squeeze) Iandola et al. (2017) and MobileNet-v2 (Mobile) Sandler et al. (2018) as our target models.
Dataset. To make our method more convincing and avoid cherry-picking, we choose 10,000 images (each category contains about 10 images) from the ImageNet validation set Russakovsky et al. (2015) which are classified correctly by all ten networks we consider. Besides, all images are resized to 299× 299× 3 beforehand. Parameters. In our experiments, we use l∞-norm to measure the perceptibility of adversarial noises, the maximum perturbation ε is set to 16. For our HIT, the size of Gaussian kernel G is 17× 17 (i.e. k = 4), weight factor λ is set to 1.0.

heading: A.2 QUANTITATIVE ANALYSIS ABOUT HFC AND LFC
text: To quantitatively analyze whether HFC or LFC is dominant in the feature map of shallow layer, we conducted this experiment. Considering that the size of each shallow-layer feature map in Fig. 2(b) is 147, we first resize the Fig. 2(a) (299× 299) to 147× 147, and denote the resultant image by xr. Then we get the xHr (HFC of xr) by:
xHr = xr − xr ∗G. (5)
To quantitatively compare the response of HFC and LFC, we calculate the average response of each feature map φ(x) in low-frequency regions versus that in the other HFC regions. To that end, we generate two masks to distinguish the two regions. More specific, the mask of high-frequency regionsMH can be written as:
MHi,j = { 1, |xHr(i,j)| > τ 0, else , (6)
where τ = 20 is the pre-set threshold which applied to filter out low response. After getting the MH , the mask of LFCML can be easy derivated:
ML = 1−MH . (7)
Therefore, the average response of HFC aH and the average response of LFC aL can be expressed as:
aH =
∑ i,jM
H φ(x)∑ i,jM H , (8)
aL =
∑ i,jM
L φ(x)∑ i,jM L . (9)
In this paper, if a feature map meets aH > aL, we call it “HFC dominant”, otherwise we call it “LFC dominant”. As demonstrate in Fig. 7, most feature maps are focus on HFC, and the “HFC dominant” to “LFC dominant” ratio is 3:1.
A.3 THE EFFECT OF WEIGHT FACTOR λ
In this section, we discuss the effect of different weight factors λ on the experimental results. We tune λ from 0.1 to 10, and the results are shown in Fig. 8. When λ ≤ 1, the attack success rate increases rapidly at the beginning and then remains stable. However, further increasing λ from 1 to 10 does not improve the performance. Actually, the success rates keep stable with a slight drop.
Apparently, a larger λ leads to more perturbations (i.e., increase the average perturbation of all pixels), and our reported results are a little inconsistent with linear assumption (Goodfellow et al., 2015). It probably because our HIT is completely independent of any prior information (e.g. the gradient of any model or data distribution). So it is not the larger the noise is, the farther the deviation from the true label will be. Besides, we notice that the activation functions of these victim’s models are all Relu, which may be another reason for this phenomenon. More specifically, Relu is defined as
Relu(z) =
{ 0, z < 0.
z, else. (10)
where z is the intermediate output before activation layer. If the intermediate adversarial perturbation is large enough, i.e., δ′ ≤ −z, then Relu(z+ δ′) will return 0. But for a misclassification label yadv 6= y, positive activation which is different from the original z may be more helpful than 0.

heading: A.4 QUALITATIVE COMPARISON FOR ADVERSARIAL EXAMPLES
text: To better reflect the advantages of our approach, in this section we compare the visual quality of the generated adversarial examples. Specifically, we consider state-of-the-art black-box PI-FGSM (Gao et al., 2020a) and no-box attack (Li et al., 2020a) as our competitor. As depicted in Fig. 9, both PI-FGSM (Gao et al., 2020a) and no-box attack (Li et al., 2020a) will cause more perceptible distortions. In contrast, the adversarial perturbation crafted by our HIT is much more imperceptible.

heading: A.5 ATTACK OTHER CLASSIFICATION TASKS
text: To highlight the practical property of our HIT, in this section we apply our HIT for other classification tasks. Specifically, we consider three well-known fine-grained classification including

heading: Raw images
text: CUB-200-2011 (Wah et al., 2011), Stanford Cars (Krause et al., 2013) and FGVC Aircraft (Maji et al., 2013) and the victim model is trained via DCL (backbone: Res-50) (Chen et al., 2019). The resolution of inputs is 448× 448. Therefore, we set the ”tile size = 448 / tile scheme”. For example, if the tile-scheme is 4 × 4, then the tile-size is 112. To ensure our defaulting setting (i.e., λ and tile-scheme) for HIT is applicable, we conduct two experiments in the following.
Discussion on tile scheme. We first report the average attack success rates (%) of our HIT w/ Circle w.r.t tile-scheme in Tab. 4. From the result, we can observe that our HIT is also effective for attacking other datasets. Notably, our HIT can fool DCL with about 90% success rate on Stanford Cars dataset. Besides, a relatively smaller tile-size is also helpful in improving the success rate of the attack, which is consistent with the conjecture given in Sec. 3.2.
Discussion on λ. We then report the average attack success rate (%) of our HIT w/ Circle w.r.t λ in Tab. 5. Although set λ = 1.0 is not optimal, the gap between the best results and the results of is very small. Therefore, our default setting for HIT is still applicable.
A.6 ATTACK REAL-WORLD RECOGNITION SYSTEM
To further demonstrate the practical property of our HIT, in this section we apply our HIT (w/ Circle) to attack a real-world recognition system, i.e., Google Cloud Vision API6. Different from existing works (Chen et al., 2017; Brendel et al., 2017) which need a large number of queries for optimization, we directly apply our HIT with the default setting (i.e., tilescheme is 6×6 and λ = 1.0). As illustrated in Fig. 10, our no-box HIT with ε = 16 can effectively change top-k labels. For example, the top-5 label of “fish” is “Fish”, “Fin”, “Seafood”, “Ray-finned fish” and “Marine biology”, while our adversarial example is
“Reptile”,“Turtle”, “Terrestrial Animal”, “Pattern” and “Art”. Notably, there is no overlap on top-k labels between clean image and our adversarial example, which also demonstrate the effectiveness of our no-box HIT.

heading: A.7 RESULTS FOR SMALLER PERTURBATION
text: In this experiment, we report the average success rates (%) between state-of-the-art black-box attacks (further add Ghost Network algorithm (Li et al., 2020c) as our competitor) and our proposed no-box attack with a smaller perturbation ε = 8.
As demonstrated in Tab. 6, our proposed methods are still competitive to mainstream transfer-based black-box attacks, even though they combine many effective techniques. Remarkably, our no-box attack can significantly outperform Ghost Networks (+MI-FGSM). Although Ghost Networks (+PIMI-DI-FGSM) is much more powerful, our no-box attack can surpass it in some cases. For example, when fooling Shuffle, our HIT (w/ Circle) can outperform Ghost Networks (+PI-MI-DI-FGSM) by about 8%.

heading: A.8 RAW IMAGE FOR ATTACK
text: To highlight the effectiveness of our design for adversarial patches, here we conduct the experiment where raw images (shown in Fig. 11) serve as “adversarial patch”. More specially, we utilize the HFC of these raw images (like Din et al.) to manipulate adversarial examples. However, even the HFC of texture-rich raw images (e.g., “Grifola frondosa” and “Capitulum”) do not achieve a good result. As demonstrated in Tab. 8, the average attack success rates are all less than 40%. By contrast, our well-designed adversarial patches can significantly achieve a success rate of nearly 90%, which demonstrates the effectiveness of our design.
6https://cloud.google.com/vision/docs/drag-and-drop
Table 6: The comparison of attack success rates (%) on normally trained models between black-box attacks and our no-box attacks with maximum perturbation ε = 8. For black-box attacks, adversarial examples are crafted via Inc-v3.
Attacks VGG19 ResNet DenseNet WRN SENet PNA Shuffle Squeeze Mobile AVG.
MI-FGSM 21.26 15.63 20.47 14.45 11.58 23.36 24.13 32.61 25.64 21.01 Ghost Networks (+MI-FGSM) 27.12 17.45 22.31 14.92 13.43 28.63 30.08 40.80 33.67 25.38
DI-FGSM 18.41 11.24 16.44 10.19 8.28 17.59 15.03 18.40 18.20 14.86 PI-FGSM 24.12 14.98 22.91 15.38 12.21 27.32 25.93 39.20 27.29 23.26 PI-MI-DI2-FGSM 40.88 31.02 41.34 29.70 25.56 38.46 36.79 46.70 42.38 36.98 Ghost Networks (+PI-MI-DI2-FGSM) 63.56 43.59 55.21 40.91 40.33 54.73 63.48 81.98 73.46 57.47
HIT w/ 6× 6 Circle 37.64 36.21 40.80 30.82 21.36 37.63 71.45 79.34 69.90 47.24 HIT w/ 6× 6 Square 13.40 17.50 25.85 21.99 13.79 18.31 53.03 61.61 50.88 30.71 HIT w/ 6× 6 Rhombus 17.24 19.95 27.65 22.00 12.85 18.91 58.84 63.49 57.90 33.20

heading: A.9 DISCUSSION ON TARGETED ATTACK
text: (a) Adversarial Patch (b) Label ID: 794 (“shower curtain”)
Figure 12: We show (a) our adversarial patch and (b) some images which classified as shower curtain from ImageNet dataset. The bottom row is their HFC extracted by Eq. 1.
Although we do not explicitly force the resultant adversarial examples to be misclassified as a specific targeted label, we observe that our HIT tends to implement a targeted attack due to the frequency domain operation and classification logic of DNNs. In Tab. 7, we report the top-5 prediction labels of our adversarial examples, which are crafted by 6× 6 concentric circle pattern. A first glance shows that almost all models tend to misclassify adversarial examples generated by our HIT as several specific labels, e.g., 794 (“shower curtain”). Furthermore, this phenomenon is more obvious for Mobile and ResNet whose ratio is up to 47.69% and 75.75% respectively.
To better understand this phenomenon, we show several clean images whose labels are “shower curtain” from the ImageNet dataset and our adversarial patch in Fig. 12. We observe that the HFC of “shower curtain” is somehow aligned with our adversarial patch, i.e., they all show similar certain repetitive circles. We suspect this phenomenon might be because our proposed perturbation dominates the overall features of the image, and instead, the original features of the image become noise. Since existing algorithms are not effective yet and simply replacing our adversarial patch with a clean targeted image does not achieve an effective targeted attack (as demonstrated in Sec. A.8), we will further study the selection and generation of adversarial patches, e.g., fusing the shallow texture information of targeted distribution to guide the resultant adversarial examples towards the targeted category.

heading: A.10 WHY CIRCLE PATTERN IS USUALLY BETTER?
text: Here we attempt to provide an insight into the performance gap between Circle and the other two patterns by analyzing the intermediate feature response. Without loss of generality, we set the layer index to “depth of each DNN” / 2 and report the average cosine similarity of the features between 10,000 raw images and their adversarial examples. The result from Tab. 9 shows that Circle consistently leads to lower cosine similarity than other patterns. Consequently, the features that feed to the deep layer are more featureless, thus leading to misclassification.
A.11 VISUALIZATION OF OUR ADVERSARIAL PATCHES
In this section, we first visualize the concentric circle with respect to densities in Fig. 13. Here we control the density from 1 to 12, e.g., “2” denotes only two circles in the proto-pattern. With the increase of density, the distance between any two circles will also be reduced.
Then we list our adversarial patches with respect to tile-schemes in Fig. 14. More specifically, we first crop the 600 × 600 × 3 proto-patterns to 300 × 300 × 3 adversarial patches, then resize them into different tile-sizes (e.g., 150 × 150 × 3) and tile them to 300 × 300 × 3, finally resize back to 299×299×3 to match the size of raw images. As we can see, if we decrease the tile-size, distortion is inevitable.

heading: A.12 THE EFFECT OF REPEATING PATTERN FOR DEFENSES
text: In this section, we further consider six additional well-known defense models, which including three ensemble adversarial training models (EAT) (Tramèr et al., 2018): Inc-v3ens3, Inc-v3ens4 and IncRes-v2ens,7 and three feature denoising models (FD) (Xie et al., 2019a): ResNet152 Baseline (Res152B), ResNet152 Denoise (Res152D), ResNeXt101 DenoiseAll (ResNeXtDA),8 to discuss the effect of repeating pattern.
7https://github.com/tensorflow/models/tree/archive/research/adv_ imagenet_models
8https://github.com/facebookresearch/ImageNet-Adversarial-Training
Table 9: The cosine similarity comparison for different patterns.
Model Attack VGG19 Inc-v3 ResNet Dense WRN SENet PNA Squeeze Shuffle Mobile Avg.
- HIT w/ Square (Ours) 0.6215 0.7419 0.7638 0.8090 0.7599 0.5838 0.7437 0.6940 0.6704 0.4838 0.6872
HIT w/ Rhombus (Ours) 0.6218 0.7458 0.7448 0.7853 0.7280 0.6258 0.7672 0.6738 0.6461 0.4005 0.6746 HIT w/ Circle (Ours) 0.5472 0.6685 0.7306 0.7779 0.7223 0.5613 0.6747 0.6643 0.6062 0.3617 0.6314
Density:7 Density:8 Density:9 Density:10 Density:11 Density:12

heading: Density:1 Density:2 Density:3 Density:4 Density:5 Density:6
text: Figure 13: We visualize our proto-patterns w.r.t densities. Here we take concentric circles as an example.
Generally, a smaller tile-scheme can generate a more perceptible perturbation. As shown in Fig. 15, the area of each regionally homogeneous (i.e. continues) line in adversarial examples crafted by 1× 1 patches is bigger than 6× 6 ones. Different from the trends on NTs, smaller tile-schemes are more effective for attacking defense models. As demonstrated in Tab. 16, when attacking EAT, 2×2 adversarial patches perform best, and further increasing the tile-scheme will significantly degrade performance, e.g., 7 × 7 rhombuses only successfully attack EAT by 6.61% on average. The trend of FD is similar to that of EAT, except that 1×1 adversarial patches work best. The reason might be that thin regionally homogeneous lines are more easily to filter out by the denoising block of (Xie et al., 2019a). Therefore, in our paper, we use 2× 2 and 1× 1 adversarial patches to attack EAT and FD, respectively.

heading: 1x1 2x2 3x3 4x4 5x5 6x6 7x7

Review summaries:
The proposed approach involves creating adversarial examples in a training-free manner by manipulating the frequency components of the image. High -frequency information is used from simple geometric patterns and combined with low-frequency component from the input to create a hybrid image. The resultant image is shown to fool classifiers under multiple settings which presents an easy way to construct adversarial examples.

The authors propose a new method for generating adversarial images for image classifier. This is a No-Box attack named Hybrid Image Transformation (HIT) or hit attack which is both model free and data-free (no training required). In the experiment section the authors show the efficacy of the proposed attack on the ImageNet dataset for different DNNs.