Title: MITIGATING ADVERSARIAL EFFECTS THROUGH RAN-DOMIZATION

Abstract: Convolutional neural networks have demonstrated high accuracy on various tasks in recent years. However, they are extremely vulnerable to adversarial examples. For example, imperceptible perturbations added to clean images can cause convolutional neural networks to fail. In this paper, we propose to utilize randomization at inference time to mitigate adversarial effects. Specifically, we use two randomization operations: random resizing, which resizes the input images to a random size, and random padding, which pads zeros around the input images in a random manner. Extensive experiments demonstrate that the proposed randomization method is very effective at defending against both single-step and iterative attacks. Our method provides the following advantages: 1) no additional training or fine-tuning, 2) very few additional computations, 3) compatible with other adversarial defense methods. By combining the proposed randomization method with an adversarially trained model, it achieves a normalized score of 0.924 (ranked No.2 among 107 defense teams) in the NIPS 2017 adversarial examples defense challenge, which is far better than using adversarial training alone with a normalized score of 0.773 (ranked No.56).

1. INTRODUCTION:

Convolutional Neural Networks (CNNs) have been successfully applied to a wide range of vision tasks, including image classification (Krizhevsky et al., 2012; Simonyan & Zisserman, 2015; He et al., 2016a) , object detection (Girshick, 2015; Ren et al., 2015; Zhang et al., 2017) , semantic segmentation (Long et al., 2015; Chen et al., 2017) , visual concept discovery (Wang et al., 2017) etc. However, recent works show that CNNs are extremely vulnerable to small perturbations to the input image. For example, adding visually imperceptible perturbations to the original image can result in failures for image classification (Szegedy et al., 2014; Goodfellow et al., 2015) , object detection (Xie et al., 2017) and semantic segmentation (Xie et al., 2017; Fischer et al., 2017; Cisse et al., 2017) . These perturbed images are called adversarial examples and Figure 1 gives an example. Adversarial examples pose a great security danger to the deployment of commercial machine learning systems. Thus, making CNNs more robust to adversarial examples is a very important yet challenging problem. Recent works (Papernot et al., 2016b; Kurakin et al., 2017; Tramèr et al., 2017; Cao & Gong, king penguin adversarial perturbation chihuahua

Figure 1 : This is an adversarial example crafted for VGG (Simonyan & Zisserman, 2015) . The left image is classified correctly as king penguin, the center image is the adversarial perturbation (magnified by 10 and enlarged by 128 for better visualization), and the right image is the adversarial example misclassfied as chihuahua.

2017; Metzen et al., 2017; Feinman et al., 2017; Meng & Chen, 2017 ) are making progress on this line of research.

Adversarial attacks can be divided into two categories: single-step attacks, which perform only one step of gradient computation, and iterative attacks, which perform multiple steps. Intuitively, the perturbation generated by iterative methods may easily get over-fitted to the specific network parameters, and thus be less transferable. On the other hand, single-step methods may not be strong enough to fool the network. For examples, it has been demonstrated that single-step attacks, like Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2015) , have better transferability but weaker attack rate than iterative attacks, like DeepFool (Moosavi-Dezfooli et al., 2016) .

Due to the weak generalization of iterative attacks, low-level image transformations, e.g., resizing, padding, compression, etc, may probably destroy the specific structure of adversarial perturbations, thus making it a good defense. It can even defend against white-box iterative attacks if random transformations are applied. This is because each test image goes through random transformations and the attacker does not know the specific transformation when generating adversarial noise. Recently, adversarial training (Kurakin et al., 2017; Tramèr et al., 2017) was developed to defend against single-step attacks. Thus by adding the proposed random transformations as additional layers to an adversarially trained model (Tramèr et al., 2017) , it is expected that the method is able to effectively defend against both single-step and iterative attacks, including both black-box and white-box settings.

Based on the above reasoning, in this paper, we propose a defense method by randomization at inference time, i.e., random resizing and random padding, to mitigate adversarial effects. To the best of our knowledge, this is the first work that demonstrates the effectiveness of randomization at inference time on mitigating adversarial effects on large-scale dataset, e.g., ImageNet (Deng et al., 2009) . The proposed method provides the following advantages:

• Randomization at inference time makes the network much more robust to adversarial images, especially for iterative attacks (both white-box and black box), but hardly hurts the performance on clean (non-adversarial) images. Experiments on section 4.2 support this argument.

• There is no additional training or fine-tuning required which is easy for implementation.

• Very few computations are required by adding the two randomization layers, thus there is nearly no run time increase.

• Randomization layers are compatible to different network structures and adversarial defense methods, which can serve as a basic network module for adversarial defense.

We conduct comprehensive experiments to test the effectiveness of our defense method, using different network structures, against different attack methods, and under different attack scenarios.

The results in Section 4 demonstrate that the proposed randomization layers can significantly mitigate adversarial effects, especially for iterative attack methods. Moreover, we submitted the model, which combines the proposed randomization layers and an adversarially trained model (Tramèr et al., 2017) , to the NIPS 2017 adversarial examples defense challenge. It reaches a normalized score of 0.924 (ranked No.2 among 107 defense teams), which is far better than just using adversarial training (Tramèr et al., 2017) alone with a normalized score of 0.773 (ranked No.56).

2. RELATED WORK:



2.1. GENERATING ADVERSARIAL EXAMPLES:

Generating adversarial examples has been extensively studied recently. (Szegedy et al., 2014) first showed that adversarial examples, computed by adding visually imperceptible perturbations to the original images, make CNNs predict wrong labels with high confidence. (Goodfellow et al., 2015) proposed the fast gradient sign method to generate adversarial examples based on the linear nature of CNNs, and also proposed adversarial training for defense. (Moosavi-Dezfooli et al., 2016) generated adversarial examples by assuming that the loss function can be linearized around the current data point at each iteration. (Carlini & Wagner, 2017) developed a stronger attack to find adversarial perturbations by introducing auxiliary variables which incooperate the pixel value constrain, e.g., pixel intensity must be within the range [0,255], naturally into the loss function and make the optimization process easier. (Liu et al., 2017) proposed an ensemble-based approaches to generate adversarial examples with stronger transferability. Unlike the works above, (Biggio & Laskov, 2012; Koh & Liang, 2017) showed that manipulating only a small fraction of the training data can significantly increase the number of misclassified samples at test time for learning algorithms, and such attacks are called poisoning attacks.

2.2. DEFENDING AGAINST ADVERSARIAL EXAMPLES:

Opposite to generating adversarial examples, there is also progress on reducing the effects of adversarial examples. (Papernot et al., 2016b) showed networks trained using defensive distillation can effectively defend against adversarial examples. (Kurakin et al., 2017) proposed to replace the original clean images with a mixture of clean images and corresponding adversarial images in each training batch to improve the network robustness. (Tramèr et al., 2017) 

3. APPROACH:



3.1. AN OVERVIEW OF GENERATING ADVERSARIAL EXAMPLES:

Before introducing the proposed adversarial defense method, we give an overview of generating adversarial examples. Let X n denote the n-th image in a dataset containing N images, and let y true n denote the corresponding ground-truth label. We use θ to denote the network parameters, and L(X n , y true n ; θ) to denote the loss. For the adversarial example generation, the goal is to maximize the loss L(X n + r n , y true n ; θ) for each image X n , under the constraint that the generated adversarial example X adv n = X n + r n should look visually similar to the original image X n , i.e., ||r n || ≤ , and the corresponding predicted label y adv n = y true n . In our experiment, we consider three different attack methods, including one single-step attack method and two iterative attack methods. We use the cleverhans library (Papernot et al., 2016a) to generate adversarial examples, where all these attacks have been implemented via TensorFlow.

• Fast Gradient Sign Method (FGSM): FGSM (Goodfellow et al., 2015) is a single-step attack method. It finds the adversarial perturbation that yields the highest increase of the linear cost function under l ∞ -norm. The update equation is

X adv n = X n + • sign( Xn L(X n , y true n ; θ)),

(1) where controls the magnitude of adversarial perturbation. In the experiment, we choose = {2, 5, 10}, which corresponds to small, medium and high magnitude of adversarial perturbations, respectively.

• DeepFool: DeepFool (Moosavi-Dezfooli et al., 2016 ) is an iterative attack method which finds the minimal perturbation to cross the decision boundary based on the linearization of the classifier at each iteration. Any l p -norm can be used with DeepFool, and we choose l 2 -norm for the study in this paper. • Carlini & Wagner (C&W): C&W (Carlini & Wagner, 2017 ) is a stronger iterative attack method proposed recently. It finds the adversarial perturbation r n by using an auxiliary variable ω n as

EQUATION

Then the loss function optimizes the auxiliary variable ω n

EQUATION

The function f (•) is defined as

EQUATION

) where Z(x) i is the logits output for class i, and k controls the confidence gap between the adversarial class and true class. C&W can also work with various l p -norm, and we choose l 2 -norm in the experiments.

The goal of defense is to build a network that is robust to adversarial examples, i.e., it can classify adversarial images correctly with little performance loss on non-adversarial (clean) images. Towards this goal, we propose a randomization-based method, as shown in Figure 2 , which adds a random resizing layer and a random padding layer to the beginning of the classification networks. There is no re-training or fine-tuning needed which makes the proposed method very easy to implement.

3.2.1. RANDOMIZATION LAYERS:

The first randomization layer is a random resizing layer, which resizes the original image X n with the size W × H × 3 to a new image X n with random size W × H × 3. Note that, |W -W | and |H -H| should be within a reasonablely small range, otherwise the network performance on non-adversarial images would significantly drop. Taking Inception-ResNet network (Szegedy et al., 2017) as an example, the original data input size is 299 × 299 × 3. Empirically we found that the network performance hardly drops if we control the height and width of the resized image X n to be within the range [299, 331) .

The second randomization layer is the random padding layer, which pads zeros around the resized image in a random manner. Specifically, by padding the resized image X n into a new image X n with the size W × H × 3, we can choose to pad w zero pixels on the left, W -Ww zero pixels on the right, h zero pixels on the top and H -Hh zero pixels on the bottom. This results in a total number of (W -W + 1) × (H -H + 1) different possible padding patterns.

During implementation, the original image first goes through two randomization layers, and then we pass the transformed image to the original CNN for classification. The pipeline is illustrated in Figure 2 .

3.2.2. RANDOMIZATION LAYERS + ADVERSARIAL TRAINING:

Note that our randomization-based method is good at defending against iterative attacks, and adversarial training (Kurakin et al., 2017; Tramèr et al., 2017) can effectively increase the robustness of neural networks to single-step attacks. Thus, to make the best of both worlds, we can combine the proposed randomization layers and an adversarially trained model (Tramèr et al., 2017) together to defend against both single-step and iterative attacks. 

4. EXPERIMENTS:



4.1. EXPERIMENT SETUP:

Dataset: It is less meaningful to attack the images that are already classified wrongly. Therefore, we randomly choose 5000 images from the ImageNet validation set that are classified correctly by all the considered networks to form our test dataset. All these images are of the size 299 × 299 × 3.

Networks::

We test with four publicly available networks1 ,2 , including Inception-v3 (Szegedy et al., 2016) , ResNet-v2 (He et al., 2016b) of 101 layers, Inception-ResNet-v2 (Szegedy et al., 2017) , and ens-adv-Inception-ResNet-v2 which applies the ensemble adversarial training (Tramèr et al., 2017) on Inception-ResNet-v2. These networks have been trained on ImageNet, and we do not perform any re-training or fine-tuning on them for the whole experiments.

Defense Models: The defense models consist of the original networks (i.e., four above-mentioned networks) and two additional randomization layers. For the random resizing layer, it changes the input shape from 299 × 299 × 3 to rnd × rnd × 3, where rnd is a integer randomly sampled from the range [299, 331) . For the random padding layer, it pads the resized image to the shape of 331 × 331 × 3 in a random manner. By applying these two randomization layers, we can create 330 rnd=299 (331 -rnd + 1) 2 = 12528 different patterns for a single image. Since there exists small variance on model performance w.r.t. different random patterns, we run the defense model three times independently and report the average accuracy.

Target Models under Different Attack Scenarios: The strongest attack would be that the attackers consider ALL possible patterns of the defense models when generating the adversarial examples. However, this is computationally impossible, because failing a large number of patterns (e.g., 12528 here) at the same time takes extremely long time, and may not even converge. Thus, we let attackers use the target models to generate adversarial examples instead, and consider the following three different attack scenarios.

• Vanilla Attack: The attackers do not know the existence of the randomization layers and the target model is just the original network. • Single-Pattern Attack: The attackers know the existence of the randomization layers. In order to mimic the structures of defense models, the target model is chosen as the original network + randomization layers with only one predefined pattern.

• Ensemble-Pattern Attack: The attackers know the existence of the randomization layers.

In order to mimic the structures of defense models in a more representative way, the target model is chosen as the original network + randomization layers with an ensemble of predefined patterns.

Target Models and Defense Models: The target models and the defense models are exactly the same except for the parameter settings of the randomization layers, i.e., the randomization parameters at the target models are predefined while randomization parameters at the defense models are randomly generated at test time. The original networks (e.g., Inception-v3) utilized by the target models and the defense models are the same. The attackers first generate adversarial examples using the target models, and then evaluate the classification accuracy of these generated adversarial examples on both the target and defense models. A low accuracy of the target model indicates that the attack is successful, and a high accuracy of the defense model indicates that the defense is effective.

4.2. CLEAN IMAGES:

Table 1 shows the top-1 accuracy of networks with and without randomization layers on the clean images. We can see that randomization layers introduce negligible performance degradation on clean images. Specifically, we can observe that: (1) models with more advanced architectures tend to have less performance degradation, e.g., Inception-ResNet-v2 only has 0.7% degradation while Inception-v3 has 2.7% degradation;

(2) ensemble adversarial training brings nearly no performance degradation to the models, e.g., Inception-ResNet-v2 and ens-adv-Inception-ResNet-v2 have nearly the same performance degradation.

4.3. VANILLA ATTACK SCENARIO:

For the vanilla attack scenario, the attackers are not aware of randomization layers, and directly use the original networks as the target model to generate adversarial examples. The attack ability on the defense models mostly rely on the transferability of adversarial examples to different resizing and padding. From the top-1 accuracy presented in Table 2 , we observe that randomization layers can mitigate the adversarial effects for both single-step and iterative attacks significantly. As for singlestep attacks FGSM-, larger indicates stronger transferability, thus making it harder to defend. However, we can still get satisfactory accuracy of the defense model on single-step attacks (even with large ) using ens-adv-Inception-ResNet-v2 (94.3% top-1 accuracy). As for iterative attacks, attackers always reach a very high attack rate on target model, but have almost no impact on models after randomization layers are applied. This is because iterative attack methods are over-fitted to the target models thus have weak transferability.

4.4. SINGLE-PATTERN ATTACK SCENARIO:

For the single-pattern attack scenario, the attackers are aware of the existence of randomization layers and also the parameters of the random resizing and random padding (i.e., from 299 × 299 to 331 × 331), but they do not know the specific randomization patterns utilized by the defense models (even the defense models themselves do not know these specific randomization patterns since they are randomly instantiated at test time). In order to generate adversarial examples, the attackers choose the target models as the original networks + randomization layers but with only one specific pattern to compute the gradient. In this experiment, the specific pattern that we use is to place the original input X n at the center of the padded image X n , i.e., no resizing is applied, and 16 zeros pixels are padded on the left, right, top and bottom on the input images, respectively. Table 3 shows the top-1 accuracy of both target models and defense models, and similar results to vanilla attack scenario are observed: (1) for single-step attacks, randomization layers are less effective on mitigating adversarial effects for a larger , while the adversarially trained models are able to defend against such attacks; (2) for iterative attacks, they reach high attack rates on target models, while have nearly no impact on defense models.

4.5. ENSEMBLE-PATTERN ATTACK SCENARIO:

For the ensemble-pattern attack scenario, similar to single-pattern attack scenario, the attackers are aware of the randomization layers and the parameters of the random resizing and random padding (i.e., starting from 299 × 299 to 331 × 331), but they do not know the specific patterns utilized by the defense models at test time. The target models thus are constructed in a more representative way: let randomization layers choose an ensemble of predefined patterns, and the goal of the attackers is to let all chosen patterns fail on classification. In this experiment, the specific ensemble patterns that we choose are: (1) first resize the input image to five different scales {299, 307, 315, 323, 331};

(2) then pad each resized image to five different patterns, where the resized image is placed at the top left, top right, bottom left, bottom right, and center of the padded image, respectively. Since there is only one padding pattern for the resized image with size 331, we can obtain 4 * 5+1 = 21 patterns in total. Due to the large computation amounts introduced by the ensemble-pattern attack scenario, we randomly choose 500 images out of the entire test dataset for this experiment. The top-1 accuracy for the target model here is calculated by summing up the number of correctly classified patterns of each 

4.6. DIAGNOSTIC EXPERIMENT:

Due to the large amount of possible patterns introduced by randomization layers, it is hard to analyze the effectiveness of random resizing and random padding precisely. In this section, we limit the freedom of randomization to be a small number (i.e., 4 in random padding and 1 in random resizing) and analyze the effectiveness of these two operations separately. The same 500 images in section 4.5 are used in this experiment. In addition, the input images for target models and defense models are resized to the shape 330 × 330 × 3 beforehand.

4.6.1. ONE PIXEL PADDING:

For the random padding, there are only 4 patterns when padding the input images from 330×330×3 to 331×331×3. In order to construct a stronger attack, we follow the experiment setup in section 4.5 where 3 chosen patterns are ensembled. Specifically, the target model takes an ensemble of patterns where the original images are at the top left, top right and bottom left (3 patterns) of the padded images, and the defense model takes the last pattern where the original images are at the bottom right of the padded images. Note that, since there is no randomization in the defense model, we only run the defense model once. 

5. NIPS 2017 ADVERSARIAL EXAMPLES DEFENSE CHALLENGE:

We submitted our model to the NIPS 2017 adversarial examples defense challenge3 for a more comprehensive performance evaluation. The test dataset contains 5000 images which are all of the size 299 × 299 × 3, and their corresponding labels are the same as the ImageNet 1000-class labels.

Each defense method are run on all 5000 adversarial images generated against all adversarial attacks. For each correctly classified image, the defense method gets one point. The normalized score for each defense method is computed using the following formula:

EQUATION

where A is the set of all attacks, M is the total number of generated adversarial examples by all attacks, and the function [•] is the indicator function which equals to 1 when the prediction is true.

5.1. CHALLENGE RESULTS:

The best defense model in our experiments, i.e., randomization layers + ens-adv-Inception-Resnet-v2, was submitted to the challenge. To increase the classification accuracy, we (1) changed the resizing range from [299, 331) to [310, 331); (2) averaged the prediction results over 30 randomization patterns for each image; (3) flipped the input image with probability 0.5 for each randomization pattern.

By evaluating our model against 156 different attacks, it reaches a normalized score of 0.924 (ranked No.2 among 107 defense models), which is far better than using ensemble adversarial training (Tramèr et al., 2017) alone with a normalized score of 0.773 (ranked No.56). This result further demonstrates that the proposed randomization method effectively make deep networks much more robust to adversarial attacks.

6. CONCLUSION:

In this paper, we propose a randomization-based mechanism to mitigate adversarial effects. We conduct comprehensive experiments to validate the effectiveness of our defense method, using different network structures, against different attack methods, and under different attack scenarios.

The experimental results show that adversarial examples rarely transfer between different randomization patterns, especially for iterative attacks. In addition, the proposed randomization layers are compatible to different network structures and adversarial defense methods, which can serve as a basic module for defense against adversarial examples. By adding the proposed randomization layers to an adversarially trained model (Tramèr et al., 2017) We then evaluate the effectiveness of these randomization methods against the adversarial examples generated under the vanilla attack scenario. The results are shown in the Tables 8 -11. Compared to the results in Table 2 , all these four methods are much less effective than the proposed randomization layers. By combining the proposed randomization layers with each of these four randomization methods (denoted as "++"), the performance can be slightly improved than using the proposed randomization layers alone.

Since each of these four randomization methods alone are not as effective as our proposed randomization methods against adversarial examples generated under the vanilla attack scenario, we do not further investigate their effectiveness under single-pattern attack and ensemble-pattern attack scenarios. However, combining these randomization methods with our proposed randomization layers together provides a way to build a slightly stronger defense mechanism. Instead of resizing the input image to a larger size, we here resize the input image to a smaller size, i.e., the resizing parameter is randomly sampled from the range [267, 299). The random padding layer then pads the resized image to the shape of 299 × 299 × 3 in a random manner. Note that, the random resizing layer and the random padding layer here have the same freedom as the ones used in the paper, i.e., they create the same number, 12528, of different patterns for a single image. We evaluate the effectiveness of this parameter setting on both the 5000 clean images and the adversarial examples generated under the vanilla attack scenario. The results are shown in the Table 12 . We see that randomization layers still work well with smaller size images, but is slightly worse than using larger size images as in the paper. This is because resizing to a smaller size loses certain information of the original image.

Table 12 : Top-1 classification accuracy on the clean images and the adversarial examples generated under the vanilla attack scenario. Compared to the results in Tables 1 and 2 , randomization parameters applied here (i.e., resize between [267, 299) , and pad to 299 × 299 × 3) is slightly worse than the randomization parameters applied in the paper (i.e., resize between [299, 331) , and pad to 331 × 331 × 3). 

Models:



APPENDIX C RANDOMIZATION LAYERS WITH MULTIPLE ITERATIONS:

In this section, we show the relationship between the top-1 accuracy of the defense model and the iteration number performed on each image. Specifically, we choose ens-adv-Inception-ResNet-v2 + randomization layers as the defense model for the experiment. The same trend can be observed for other defense models.

For the defense model, the iteration number is chosen to be {1, 5, 10, 20, 30}, and it is evaluated 

:

https://github.com/tensorflow/models/tree/master/research/slim

http://download.tensorflow.org/models/ens adv inception resnet v2 2017 08 18.tar.gz

https://www.kaggle.com/c/nips-2017-defense-against-adversarial-attack

https://github.com/tensorflow/models/blob/master/research/inception/inception/image processing.py#L182

Figures and Tables:

Figure 2: The pipeline of our randomization-based defense mechanism. The input image X n first goes through the random resizing layer with a random scale applied. Then the random padding layer pads the resized image X n in a random manner. The resulting padded image X n is used for classification.

Figure 3: Top-1 classification accuracy on the clean images and the adversarial examples generated under the vanilla attack scenrio.

Top-1 classification accuracy on the clean images. We see that adding random resizing and random padding cause very little accuracy drop on clean (non-adversarial) images.

Table content: <table><tr><td>Models</td><td>Inception-v3</td><td>ResNet-v2-101</td><td>Inception-ResNet-v2</td><td>ens-adv-Inception-ResNet-v2</td></tr><tr><td>w/o randomization layers</td><td>100%</td><td>100%</td><td>100%</td><td>100%</td></tr><tr><td>w randomization layers</td><td>97.3%</td><td>98.3%</td><td>99.3%</td><td>99.2%</td></tr></table>

Top-1 classification accuracy under the vanilla attack scenario. We see that randomization layers effectively mitigate adversarial effects for all attacks and all networks. Particularly, combining randomization layers with ensemble adversarial training (ens-adv-Inception-ResNet-v2) performs very well on all attacks.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td></tr><tr><td/><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td></tr><tr><td>FGSM-2</td><td>33.2%</td><td>65.1%</td><td>26.3%</td><td>71.8%</td><td>65.3%</td><td>81.0%</td><td>84.4%</td><td>95.7%</td></tr><tr><td>FGSM-5</td><td>31.1%</td><td>54.5%</td><td>20.4%</td><td>54.3%</td><td>61.7%</td><td>74.1%</td><td>87.4%</td><td>94.5%</td></tr><tr><td colspan="2">FGSM-10 33.0%</td><td>52.4%</td><td>20.4%</td><td>46.1%</td><td>61.2%</td><td>71.3%</td><td>90.2%</td><td>94.3%</td></tr><tr><td>DeepFool</td><td>0%</td><td>98.3%</td><td>0%</td><td>97.7%</td><td>0%</td><td>98.2%</td><td>0.2%</td><td>99.1%</td></tr><tr><td>C&amp;W</td><td>0%</td><td>96.9%</td><td>0%</td><td>97.1%</td><td>0.3%</td><td>97.7%</td><td>0.9%</td><td>98.8%</td></tr></table>

Top-1 classification accuracy under the single-pattern attack scenario. We see that randomization layers effectively mitigate adversarial effects for all attacks and all networks. Particularly, combining randomization layers with ensemble adversarial training (ens-adv-Inception-ResNet-v2) performs very well on all attacks.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td></tr><tr><td/><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td></tr><tr><td>FGSM-2</td><td>35.1%</td><td>63.8%</td><td>29.5%</td><td>70.1%</td><td>71.6%</td><td>83.4%</td><td>86.3%</td><td>96.4%</td></tr><tr><td>FGSM-5</td><td>32.4%</td><td>53.9%</td><td>23.2%</td><td>52.3%</td><td>68.3%</td><td>78.2%</td><td>88.4%</td><td>95.4%</td></tr><tr><td colspan="2">FGSM-10 34.7%</td><td>51.8%</td><td>22.4%</td><td>43.8%</td><td>66.8%</td><td>75.6%</td><td>90.7%</td><td>95.2%</td></tr><tr><td>DeepFool</td><td>1.1%</td><td>98.2%</td><td>1.7%</td><td>97.8%</td><td>0.6%</td><td>98.4%</td><td>1.0%</td><td>99.2%</td></tr><tr><td>C&amp;W</td><td>1.1%</td><td>97.4%</td><td>1.7%</td><td>97.0%</td><td>0.8%</td><td>97.9%</td><td>1.6%</td><td>99.1%</td></tr></table>

Top-1 classification accuracy under the ensemble-pattern attack scenario. Similar to vanilla attack and single-pattern attack scenarios, we see that randomization layers increase the accuracy under all attacks and networks. This clearly demonstrates the effectiveness of the proposed randomization method on defending against adversarial examples, even under this very strong attack scenario. entire pattern number of all images. For the results presented in Table4, we can see that the adversarial examples generated under ensemble-pattern attack scenario are much stronger. For single-step attacks, the generated adversarial examples can let the performance of the defense model with an adversarially trained network drop around 8% compared to the performance under vanilla attack and single-pattern attack scenarios, and drop much more on other defense models.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td></tr><tr><td/><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td></tr><tr><td>FGSM-2</td><td>37.3%</td><td>41.2%</td><td>39.2%</td><td>44.9%</td><td>71.5%</td><td>74.3%</td><td>86.2%</td><td>88.9%</td></tr><tr><td>FGSM-5</td><td>31.7%</td><td>34.0%</td><td>24.6%</td><td>29.7%</td><td>65.2%</td><td>67.3%</td><td>85.8%</td><td>87.5%</td></tr><tr><td colspan="2">FGSM-10 30.4%</td><td>32.8%</td><td>18.6%</td><td>21.7%</td><td>62.9%</td><td>64.5%</td><td>86.6%</td><td>87.9%</td></tr><tr><td>DeepFool</td><td>0.6%</td><td>81.3%</td><td>0.9%</td><td>80.5%</td><td>0.9%</td><td>69.4%</td><td>1.6%</td><td>93.5%</td></tr><tr><td>C&amp;W</td><td>0.6%</td><td>62.9%</td><td>1.0%</td><td>74.3%</td><td>1.6%</td><td>68.3%</td><td>5.8%</td><td>86.1%</td></tr><tr><td>image over the</td><td/><td/><td/><td/><td/><td/><td/><td/></tr></table>

Top-1 classification accuracy under one pixel padding scenario. This table shows that creating different padding patterns (even 1-pixel padding) can effectively mitigate adversarial effects.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td></tr><tr><td/><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td></tr><tr><td>FGSM-2</td><td>36.4%</td><td>39.6%</td><td>29.8%</td><td>34.4%</td><td>71.3%</td><td>74.0%</td><td>88.2%</td><td>94.8%</td></tr><tr><td>FGSM-5</td><td>33.5%</td><td>36.2%</td><td>22.2%</td><td>26.2%</td><td>68.4%</td><td>71.0%</td><td>92.1%</td><td>94.4%</td></tr><tr><td colspan="2">FGSM-10 34.5%</td><td>38.8%</td><td>21.3%</td><td>23.6%</td><td>67.4%</td><td>70.4%</td><td>93.7%</td><td>94.0%</td></tr><tr><td>DeepFool</td><td>0.9%</td><td>97.2%</td><td>0.9%</td><td>95.2%</td><td>0.9%</td><td>87.6%</td><td>1.5%</td><td>99.2%</td></tr><tr><td>C&amp;W</td><td>0.8%</td><td>70.2%</td><td>0.9%</td><td>76.8%</td><td>1.0%</td><td>79.4%</td><td>2.4%</td><td>98.2%</td></tr></table>

Top-1 classification accuracy under one pixel resizing scenario. This table shows that resizing image to a different scale (even 1-pixel scale) can effectively mitigate adversarial effects.For the random resizing, there is only 1 pattern that exists when the input images are resized from 330 × 330 × 3 to 331 × 331 × 3. The results in Table6indicate that resizing the images by only 1 pixel can effectively destroy the transferability of adversarial examples by both single-step and iterative attacks.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td><td>target</td><td>defense</td></tr><tr><td/><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td><td>model</td></tr><tr><td>FGSM-2</td><td>30.8%</td><td>56.2%</td><td>31.6%</td><td>44.6%</td><td>66.2%</td><td>75.0%</td><td>87.6%</td><td>97.2%</td></tr><tr><td>FGSM-5</td><td>31.2%</td><td>48.8%</td><td>25.6%</td><td>35.8%</td><td>61.4%</td><td>70.2%</td><td>91.2%</td><td>96.6%</td></tr><tr><td colspan="2">FGSM-10 36.4%</td><td>51.0%</td><td>23.8%</td><td>32.6%</td><td>62.8%</td><td>68.2%</td><td>94.8%</td><td>95.2%</td></tr><tr><td>DeepFool</td><td>2.6%</td><td>99.4%</td><td>1.0%</td><td>98.6%</td><td>1.2%</td><td>97.4%</td><td>1.2%</td><td>99.4%</td></tr><tr><td>C&amp;W</td><td>2.6%</td><td>97.8%</td><td>1.0%</td><td>94.8%</td><td>2.0%</td><td>94.8%</td><td>1.8%</td><td>99.6%</td></tr><tr><td colspan="3">4.6.2 ONE-PIXEL RESIZING</td><td/><td/><td/><td/><td/><td/></tr></table>

, it achieves a normalized score of 0.924 (ranked No.2 among 107 defense models) in the NIPS 2017 adversarial examples defense challenge, which is far better than using adversarial training alone with a normalized score of 0.773 (ranked No.56). The code is public available at https://github.com/cihangxie/NIPS2017_ adv_challenge_defense. Top-1 classification accuracy on clean images. We see that these four randomization methods hardly hurt the performance on clean images. We use "++" to denote the addition of the proposed randomization layers, i.e., random resizing and random padding, and the results indicate that combined models still performs pretty good on clean images.

Table content: <table><tr><td>Models</td><td>Inception-v3</td><td>ResNet-v2-101</td><td>Inception-ResNet-v2</td><td>ens-adv-Inception-ResNet-v2</td></tr><tr><td>random brightness</td><td>99.6%</td><td>99.7%</td><td>99.8%</td><td>99.8%</td></tr><tr><td>random brightness ++</td><td>98.6%</td><td>98.1%</td><td>99.1%</td><td>99.2%</td></tr><tr><td>random saturation</td><td>99.6%</td><td>99.7%</td><td>99.9%</td><td>99.9%</td></tr><tr><td>random saturation ++</td><td>98.6%</td><td>98.3%</td><td>99.3%</td><td>99.3%</td></tr><tr><td>random hue</td><td>99.4%</td><td>99.6%</td><td>99.7%</td><td>99.4%</td></tr><tr><td>random hue ++</td><td>98.6%</td><td>98.3%</td><td>99.2%</td><td>99.1%</td></tr><tr><td>random contrast</td><td>99.5%</td><td>99.6%</td><td>99.7%</td><td>99.6%</td></tr><tr><td>random contrast ++</td><td>98.6%</td><td>98.2%</td><td>99.3%</td><td>99.1%</td></tr></table>

Top-1 classification accuracy by using random brightness under the vanilla attack scenario. Compared to the results in Table2, random brightness is much less effective than the proposed randomization layers. By combing random brightness and the proposed randomization layers (denoted as random brightness++), it reaches slightly better performance than using the proposed randomization layers alone.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td></tr><tr><td/><td>bright-</td><td>bright-</td><td>bright-</td><td>bright-</td><td>bright-</td><td>bright-</td><td>bright-</td><td>bright-</td></tr><tr><td/><td>ness</td><td>ness++</td><td>ness</td><td>ness++</td><td>ness</td><td>ness++</td><td>ness</td><td>ness++</td></tr><tr><td>FGSM-2</td><td>34.9%</td><td>67.0%</td><td>28.5%</td><td>73.5%</td><td>66.4%</td><td>81.3%</td><td>85.0%</td><td>95.8%</td></tr><tr><td>FGSM-5</td><td>31.9%</td><td>55.5%</td><td>21.6%</td><td>55.7%</td><td>62.4%</td><td>74.7%</td><td>87.6%</td><td>95.0%</td></tr><tr><td colspan="2">FGSM-10 33.2%</td><td>52.9%</td><td>20.9%</td><td>47.2%</td><td>61.8%</td><td>71.5%</td><td>90.4%</td><td>94.5%</td></tr><tr><td>DeepFool</td><td>79.4%</td><td>98.1%</td><td>82.3%</td><td>97.5%</td><td>62.8%</td><td>98.3%</td><td>79.0%</td><td>99.1%</td></tr><tr><td>C&amp;W</td><td>34.5%</td><td>96.9%</td><td>47.7%</td><td>97.2%</td><td>51.3%</td><td>98.0%</td><td>42.3%</td><td>98.6%</td></tr></table>

Top-1 classification accuracy by using random saturation under the vanilla attack scenario. Compared to the results in Table2, random saturation is much less effective than the proposed randomization layers. By combing random saturation and the proposed randomization layers (denoted as random saturation++), it reaches slightly better performance than using the proposed randomization layers alone.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td></tr><tr><td/><td>satura-</td><td>satura-</td><td>satura-</td><td>satura-</td><td>satura-</td><td>satura-</td><td>satura-</td><td>satura-</td></tr><tr><td/><td>tion</td><td>tion++</td><td>tion</td><td>tion++</td><td>tion</td><td>tion++</td><td>tion</td><td>tion++</td></tr><tr><td>FGSM-2</td><td>34.2%</td><td>66.5%</td><td>27.9%</td><td>73.6%</td><td>66.3%</td><td>81.5%</td><td>85.2%</td><td>95.7%</td></tr><tr><td>FGSM-5</td><td>31.8%</td><td>55.2%</td><td>21.1%</td><td>55.7%</td><td>62.1%</td><td>74.6%</td><td>87.0%</td><td>95.0%</td></tr><tr><td colspan="2">FGSM-10 33.5%</td><td>52.2%</td><td>20.7%</td><td>46.5%</td><td>61.7%</td><td>71.4%</td><td>90.1%</td><td>93.9%</td></tr><tr><td>DeepFool</td><td>82.6%</td><td>98.1%</td><td>79.6%</td><td>97.6%</td><td>64.7%</td><td>98.2%</td><td>78.5%</td><td>99.0%</td></tr><tr><td>C&amp;W</td><td>39.2%</td><td>97.2%</td><td>47.5%</td><td>96.9%</td><td>51.7%</td><td>97.7%</td><td>50.9%</td><td>99.1%</td></tr></table>

Top-1 classification accuracy by using random hue under the vanilla attack scenario. Compared to the results in Table2, random hue is much less effective than the proposed randomization layers. By combing random hue and the proposed randomization layers (denoted as random hue++), it reaches slightly better performance than using the proposed randomization layers alone.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td></tr><tr><td/><td>hue</td><td>hue++</td><td>hue</td><td>hue++</td><td>hue</td><td>hue++</td><td>hue</td><td>hue++</td></tr><tr><td>FGSM-2</td><td>38.1%</td><td>69.0%</td><td>32.0%</td><td>74.9%</td><td>68.6%</td><td>83.0%</td><td>87.4%</td><td>95.8%</td></tr><tr><td>FGSM-5</td><td>33.9%</td><td>57.2%</td><td>23.0%</td><td>57.6%</td><td>64.0%</td><td>76.1%</td><td>86.7%</td><td>93.9%</td></tr><tr><td colspan="2">FGSM-10 36.4%</td><td>54.2%</td><td>22.1%</td><td>48.4%</td><td>63.0%</td><td>72.5%</td><td>88.0%</td><td>91.3%</td></tr><tr><td>DeepFool</td><td>95.0%</td><td>97.9%</td><td>91.2%</td><td>97.6%</td><td>86.5%</td><td>98.4%</td><td>96.8%</td><td>99.1%</td></tr><tr><td>C&amp;W</td><td>72.1%</td><td>97.3%</td><td>74.0%</td><td>97.0%</td><td>77.4%</td><td>98.2%</td><td>81.1%</td><td>98.8%</td></tr></table>

Top-1 classification accuracy by using random contrast under the vanilla attack scenario. Compared to the results in Table2, random contrast is much less effective than the proposed randomization layers. By combing random contrast and the proposed randomization layers (denoted as random contrast++), it reaches slightly better performance than using the proposed randomization layers alone.

Table content: <table><tr><td>Models</td><td colspan="2">Inception-v3</td><td colspan="2">ResNet-v2-101</td><td colspan="2">Inception-ResNet-v2</td><td colspan="2">ens-adv-Inception-ResNet-v2</td></tr><tr><td/><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td><td>random</td></tr><tr><td/><td>con-</td><td>con-</td><td>con-</td><td>con-</td><td>con-</td><td>con-</td><td>con-</td><td>con-</td></tr><tr><td/><td>trast</td><td>trast++</td><td>trast</td><td>trast++</td><td>trast</td><td>trast++</td><td>trast</td><td>trast++</td></tr><tr><td>FGSM-2</td><td>37.0%</td><td>68.0%</td><td>29.4%</td><td>74.1%</td><td>67.2%</td><td>82.5%</td><td>85.9%</td><td>96.0%</td></tr><tr><td>FGSM-5</td><td>32.7%</td><td>56.3%</td><td>22.7%</td><td>57.0%</td><td>63.1%</td><td>74.9%</td><td>88.1%</td><td>94.9%</td></tr><tr><td colspan="2">FGSM-10 34.4%</td><td>53.5%</td><td>21.2%</td><td>47.0%</td><td>62.1%</td><td>72.0%</td><td>90.6%</td><td>94.3%</td></tr><tr><td>DeepFool</td><td>90.4%</td><td>98.1%</td><td>87.5%</td><td>97.5%</td><td>73.8%</td><td>98.1%</td><td>90.8%</td><td>99.0%</td></tr><tr><td>C&amp;W</td><td>56.8%</td><td>97.0%</td><td>57.1%</td><td>96.7%</td><td>63.7%</td><td>97.9%</td><td>67.6%</td><td>98.8%</td></tr><tr><td colspan="8">APPENDIX B RANDOMIZATION LAYERS WITH SMALLER SIZE</td><td/></tr></table>

Peer review (iclr2018):
 The authors propose a simple defense against adversarial attacks, which is to add randomization in the input of the CNNs. They experiment with different CNNs and published adversarial training techniques and show that randomized inputs mitigate adversarial attacks. Pros: (+) The idea introduced is simple and flexible to be used for any CNN architecture (+) Experiments on ImageNet1k prove demonstrate its effectiveness Cons: (-) Experiments are not thorougly explained (-) Novelty is extremely limited (-) Some baselines missing The experimental section of the paper was rather confusing. The authors should explain the experiments and the settings in the table, as those are not very clear. In particular, it was not clear whether the defense model was trained with the input randomization layers? Also, in Tables 1-6, how was the target model trained? How do the training procedures of target vs. defense model differ? In those tables, what is the testing procedure for the target model and how does it compare to the defense model? The gap between the target and defense model in Table 4 (ensemble pattern attack scenario) shrinks for single step attack methods. This means that when the attacker is aware of the randomization parameters, the effect of randomization might diminish. A baseline that reports the performance when the attacker is fully aware of the randomization of the defender (parameters, patterns etc.) is missing but is very useful. While the experiments show that the randomization layers mitigate the effect of randomization attacks, it*s not clear whether the effectiveness of this very simple approach is heavily biased towards the published ways of generating adversarial attacks and the particular problem (i.e. classification). The form of attacks studied in the paper is that of additive noise. But there is many types of attacks that could be closely related to the randomization procedure of the input and that could lead to very different results.