Title: DEFENSE-GAN: PROTECTING CLASSIFIERS AGAINST ADVERSARIAL ATTACKS USING GENERATIVE MODELS

Abstract: In recent years, deep neural network approaches have been widely adopted for machine learning tasks, including classification. However, they were shown to be vulnerable to adversarial perturbations: carefully crafted small perturbations can cause misclassification of legitimate images. We propose Defense-GAN, a new framework leveraging the expressive capability of generative models to defend deep neural networks against such attacks. Defense-GAN is trained to model the distribution of unperturbed images. At inference time, it finds a close output to a given image which does not contain the adversarial changes. This output is then fed to the classifier. Our proposed method can be used with any classification model and does not modify the classifier structure or training procedure. It can also be used as a defense against any attack as it does not assume knowledge of the process for generating the adversarial examples. We empirically show that Defense-GAN is consistently effective against different attack methods and improves on existing defense strategies.

1. INTRODUCTION:

Despite their outstanding performance on several machine learning tasks, deep neural networks have been shown to be susceptible to adversarial attacks (Szegedy et al., 2014; Goodfellow et al., 2015) . These attacks come in the form of adversarial examples: carefully crafted perturbations added to a legitimate input sample. In the context of classification, these perturbations cause the legitimate sample to be misclassified at inference time (Szegedy et al., 2014; Goodfellow et al., 2015; Papernot et al., 2016b; Liu et al., 2017) . Such perturbations are often small in magnitude and do not affect human recognition but can drastically change the output of the classifier.

Recent literature has considered two types of threat models: black-box and white-box attacks. Under the black-box attack model, the attacker does not have access to the classification model parameters; whereas in the white-box attack model, the attacker has complete access to the model architecture and parameters, including potential defense mechanisms (Papernot et al., 2017; Tramèr et al., 2017; Carlini & Wagner, 2017) .

Various defenses have been proposed to mitigate the effect of adversarial attacks. These defenses can be grouped under three different approaches: (1) modifying the training data to make the classifier more robust against attacks, e.g., adversarial training which augments the training data of the classifier with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015) , (2) modifying the training procedure of the classifier to reduce the magnitude of gradients, e.g., defensive distillation (Papernot et al., 2016d) , and (3) attempting to remove the adversarial noise from the input samples (Hendrycks & Gimpel, 2017; Meng & Chen, 2017 ). All of these approaches have limitations in the sense that they are effective against either white-box attacks or black-box attacks, but not both (Tramèr et al., 2017; Meng & Chen, 2017) . Furthermore, some of these defenses are devised with specific attack models in mind and are not effective against new attacks.

In this paper, we propose a novel defense mechanism which is effective against both white-box and black-box attacks. We propose to leverage the representative power of Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) to diminish the effect of the adversarial perturbation, by "projecting" input images onto the range of the GAN's generator prior to feeding them to the classifier. In the GAN framework, two models are trained simultaneously in an adversarial setting: a generative model that emulates the data distribution, and a discriminative model that predicts whether a certain input came from real data or was artificially created. The generative model learns a mapping G from a low-dimensional vector z ∈ R k to the high-dimensional input sample space R n . During training of the GAN, G is encouraged to generate samples which resemble the training data. It is, therefore, expected that legitimate samples will be close to some point in the range of G, whereas adversarial samples will be further away from the range of G. Furthermore, "projecting" the adversarial examples onto the range of the generator G can have the desirable effect of reducing the adversarial perturbation. The projected output, computed using Gradient Descent (GD), is fed into the classifier instead of the original (potentially adversarially modified) image. We empirically demonstrate that this is an effective defense against both black-box and white-box attacks on two benchmark image datasets.

The rest of the paper is organized as follows. We introduce the necessary background regarding known attack models, defense mechanisms, and GANs in Section 2. Our defense mechanism, which we call Defense-GAN, is formally motivated and introduced in Section 3. Finally, experimental results, under different threat models, as well as comparisons to other defenses are presented in Section 4.

2. RELATED WORK AND BACKGROUND INFORMATION:

In this work, we propose to use GANs for the purpose of defending against adversarial attacks in classification problems. Before detailing our approach in the next section, we explain related work in three parts. First, we discuss different attack models employed in the literature. We, then, go over related defense mechanisms against these attacks and discuss their strengths and shortcomings. Lastly, we explain necessary background information regarding GANs.

2.1. ATTACK MODELS AND ALGORITHMS:

Various attack models and algorithms have been used to target classifiers. All attack models we consider aim to find a perturbation δ to be added to a (legitimate) input x ∈ R n , resulting in the adversarial example x = x + δ. The ∞ -norm of the perturbation is denoted by (Goodfellow et al., 2015) and is chosen to be small enough so as to remain undetectable. We consider two threat levels: black-and white-box attacks.

2.1.1. WHITE-BOX ATTACK MODELS:

White-box models assume that the attacker has complete knowledge of all the classifier parameters, i.e., network architecture and weights, as well as the details of any defense mechanism. Given an input image x and its associated ground-truth label y, the attacker thus has access to the loss function J(x, y) used to train the network, and uses it to compute the adversarial perturbation δ. Attacks can be targeted, in that they attempt to cause the perturbed image to be misclassified to a specific target class, or untargeted when no target class is specified.

In this work, we focus on untargeted white-box attacks computed using the Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2015) , the Randomized Fast Gradient Sign Method (RAND+FGSM) (Tramèr et al., 2017) , and the Carlini-Wagner (CW) attack (Carlini & Wagner, 2017) . Although other attack models exist, such as the Iterative FGSM (Kurakin et al., 2017) , the Jacobian-based Saliency Map Attack (JSMA) (Papernot et al., 2016b), and Deepfool (Moosavi-Dezfooli et al., 2016) , we focus on these three models as they cover a good breadth of attack algorthims. FGSM is a very simple and fast attack algorithm which makes it extremely amenable to real-time attack deployment. On the other hand, RAND+FGSM, an equally simple attack, increases the power of FGSM for white-box attacks (Tramèr et al., 2017) , and finally, the CW attack is one of the most powerful white-box attacks to-date (Carlini & Wagner, 2017) .

Fast Gradient Sign Method (FGSM) Given an image x and its corresponding true label y, the FGSM attack sets the perturbation δ to:

δ = • sign(∇ x J(x, y)).

(1) FGSM (Goodfellow et al., 2015) was designed to be extremely fast rather than optimal. It simply uses the sign of the gradient at every pixel to determine the direction with which to change the corresponding pixel value.

Randomized Fast Gradient Sign Method (RAND+FGSM) The RAND+FGSM (Tramèr et al., 2017) attack is a simple yet effective method to increase the power of FGSM against models which were adversarially trained. The idea is to first apply a small random perturbation before using FGSM. More explicitly, for α < , random noise is first added to the legitimate image x:

EQUATION

Then, the FGSM attack is computed on x , resulting in

EQUATION

The Carlini-Wagner (CW) attack The CW attack is an effective optimization-based attack model (Carlini & Wagner, 2017) . In many cases, it can reduce the classifier accuracy to almost 0% (Carlini & Wagner, 2017; Meng & Chen, 2017) . The perturbation δ is found by solving an optimization problem of the form:

EQUATION

where f is an objective function that drives the example x to be misclassified, and c > 0 is a suitably chosen constant. The 2 , 0 , and ∞ norms are considered. We refer the reader to (Carlini & Wagner, 2017) for details regarding the approach to solving (4) and setting the constant c.

2.1.2. BLACK-BOX ATTACK MODELS:

For black-box attacks we consider untargeted FGSM attacks computed on a substitute model (Papernot et al., 2017) . As previously mentioned, black-box adversaries have no access to the classifier or defense parameters. It is further assumed that they do not have access to a large training dataset but can query the targeted DNN as a black-box, i.e., access labels produced by the classifier for specific query images. The adversary trains a model, called substitute, which has a (potentially) different architecture than the targeted classifier, using a very small dataset augmented by synthetic images labeled by querying the classifier. Adversarial examples are then found by applying any attack method on the substitute network. It was found that such examples designed to fool the substitute often end up being misclassified by the targeted classifier (Szegedy et al., 2014; Papernot et al., 2017) . In other words, black-box attacks are easily transferrable from one model to the other.

2.2. DEFENSE MECHANISMS:

Various defense mechanisms have been employed to combat the threat from adversarial attacks. In what follows, we describe one representative defense strategy from each of the three general groups of defenses.

2.2.1. ADVERSARIAL TRAINING:

A popular approach to defend against adversarial noise is to augment the training dataset with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015; Moosavi-Dezfooli et al., 2016) . Adversarial examples are generated using one or more chosen attack models and added to the training set. This often results in increased robustness when the attack model used to generate the augmented training set is the same as that used by the attacker. However, adversarial training does not perform as well when a different attack strategy is used by the attacker. Additionally, it tends to make the model more robust to white-box attacks than to black-box attacks due to gradient masking (Papernot et al., 2016c; 2017; Tramèr et al., 2017) .

2.2.2. DEFENSIVE DISTILLATION:

Defensive distillation (Papernot et al., 2016d) trains the classifier in two rounds using a variant of the distillation (Hinton et al., 2014) method. This has the desirable effect of learning a smoother network and reducing the amplitude of gradients around input points, making it difficult for attackers to generate adversarial examples (Papernot et al., 2016d) . It was, however, shown that, while defensive distillation is effective against white-box attacks, it fails to adequately protect against black-box attacks transferred from other networks (Carlini & Wagner, 2017) .

2.2.3. MAGNET:

Recently, Meng & Chen (2017) introduced MagNet as an effective defense strategy. It trains a reformer network (which is an auto-encoder or a collection of auto-encoders) to move adversarial examples closer to the manifold of legitimate, or natural, examples. When using a collection of auto-encoders, one reformer network is chosen at random at test time, thus strengthening the defense. It was shown to be an effective defense against gray-box attacks where the attacker knows everything about the network and defense, except the parameters. MagNet is the closest defense to our approach, as it attempts to reform an adversarial sample using a learnt auto-encoder. The main differences between MagNet and our approach are: (1) we use GANs instead of auto-encoders, and, most importantly, (2) we use GD minimization to find latent codes as opposed to a feedforward encoder network. This makes Defense-GAN more robust, especially against white-box attacks.

2.3. GENERATIVE ADVERSARIAL NETWORKS (GANS):

GANs, originally introduced by Goodfellow et al. ( 2014), consist of two neural networks, G and D. G : R k → R n maps a low-dimensional latent space to the high dimensional sample space of x. D is a binary neural network classifier. In the training phase, G and D are typically learned in an adversarial fashion using actual input data samples x and random vectors z. An isotropic Gaussian prior is usually assumed on z. While G learns to generate outputs G(z) that have a distribution similar to that of x, D learns to discriminate between "real" samples x and "fake" samples G(z). D and G are trained in an alternating fashion to minimize the following min-max loss (Goodfellow et al., 2014) :

EQUATION

It was shown that the optimal GAN is obtained when the resulting generator distribution p g = p data (Goodfellow et al., 2014) .

However, GANs turned out to be difficult to train in practice (Gulrajani et al., 2017) , and alternative formulations have been proposed. Arjovsky et al. (2017) introduced Wasserstein GANs (WGANs) which are a variant of GANs that use the Wasserstein distance, resulting in a loss function with more desirable properties:

EQUATION

In this work, we use WGANs as our generative model due to the stability of their training methods, especially using the approach in (Gulrajani et al., 2017) .

Figure 1 : Overview of the Defense-GAN algorithm.

3.1. MOTIVATION:

As mentioned in Section 2.3, the GAN min-max loss in (5) admits a global optimum when p g = p data (Goodfellow et al., 2014) . It can be similarly shown that WGAN admits an optimum to its own minmax loss in (6), when the set {x | p g (x) = p data (x)} has zero Lebesgue-measure. Formally, Lemma 1 A generator distribution p g is a global optimum for the WGAN min-max game defined in ( 6), if and only if p g (x) = p data (x) for all x ∈ R n , potentially except on a set of zero Lebesguemeasure.

A sketch of the proof can be found in Appendix A.

Additionally, it was shown that, if G and D have enough capacity to represent the data, and if the training algorithm is such that p g converges to p data , then

E x∼pdata min z ||G t (z) -x|| 2 -→ 0 (7)

where G t is the generator of a GAN or WGAN1 after t steps of its training algorithm (Kabkab et al., 2018) .

This serves to show that, under ideal conditions, the addition of the GAN reconstruction loss minimization step should not affect the performance of the classifier on natural, legitimate samples, as such samples should be almost exactly recovered. Furthermore, we hypothesize that this step will help reduce the adversarial noise which follows a different distribution than that of the GAN training examples.

3.2. DEFENSE-GAN ALGORITHM:

Defense-GAN is a defense strategy to combat both white-box and black-box adversarial attacks against classification networks. At inference time, given a trained GAN generator G and an image x to be classified, z * is first found so as to minimize

EQUATION

G(z * ) is then given as the input to the classifier. The algorithm is illustrated in Figure 1 . As ( 8) is a highly non-convex minimization problem, we approximate it by doing a fixed number L of GD steps using R different random initializations of z (which we call random restarts), as shown in Figures 1 and 2 .

The GAN is trained on the available classifier training dataset in an unsupervised manner. The classifier can be trained on the original training images, their reconstructions using the generator G, or a combination of the two. As was discussed in Section 3.1, as long as the GAN is appropriately trained and has enough capacity to represent the data, original clean images and their reconstructions should not defer much. Therefore, these two classifier training strategies should, at least theoretically, not differ in performance.

Compared to existing defense mechanisms, our approach is different in the following aspects: 1. Defense-GAN can be used in conjunction with any classifier and does not modify the classifier structure itself. It can be seen as an add-on or pre-processing step prior to classification.

2. If the GAN is representative enough, re-training the classifier should not be necessary and any drop in performance due to the addition of Defense-GAN should not be significant.

3. Defense-GAN can be used as a defense to any attack: it does not assume an attack model, but simply leverages the generative power of GANs to reconstruct adversarial examples.

4. Defense-GAN is highly non-linear and white-box gradient-based attacks will be difficult to perform due to the GD loop. A detailed discussion about this can be found in Appendix B.

4. EXPERIMENTS:

We assume three different attack threat levels:

1. Black-box attacks: the attacker does not have access to the details of the classifier and defense strategy. It therefore trains a substitute network to find adversarial examples.

2. White-box attacks: the attacker knows all the details of the classifier and defense strategy.

It can compute gradients on the classifier and defense networks in order to find adversarial examples.

3. White-box attacks, revisited: in addition to the details of the architectures and parameters of the classifier and defense, the attacker has access to the random seed and random number generator. In the case of Defense-GAN, this means that the attacker knows all the random initializations {z

(i) 0 } R i=1 .

We compare our method to adversarial training (Goodfellow et al., 2015) and MagNet (Meng & Chen, 2017 ) under the FGSM, RAND+FGSM, and CW (with 2 norm) white-box attacks, as well as the FGSM black-box attack. Details of all network architectures used in this paper can be found in Appendix C. When the classifier is trained using the reconstructed images (G(z * )), we refer to our method as Defense-GAN-Rec, and we use Defense-GAN-Orig when the original images (x) are used to train the classifier. Our GAN follows the WGAN training procedure in (Gulrajani et al., 2017) , and details of the generator and discriminator network architectures are given in Table 6 . The reformer network (encoder) for the MagNet baseline is provided in Table 7 . Our implementation is based on TensorFlow (Abadi et al., 2015) and builds on open-source software: CleverHans by Papernot et al. (2016a) and improved WGAN training by Gulrajani et al. (2017) . We use machines equipped with NVIDIA GeForce GTX TITAN X GPUs.

In our experiments, we use two different image datasets: the MNIST handwritten digits dataset (LeCun et al., 1998) and the Fashion-MNIST (F-MNIST) clothing articles dataset (Xiao et al., 2017) .

Both datasets consist of 60, 000 training images and 10, 000 testing images. We split the training images into a training set of 50, 000 images and hold-out a validation set containing 10, 000 images. For white-box attacks, the testing set is kept the same (10, 000 samples). For black-box attacks, the testing set is divided into a small hold-out set of 150 samples reserved for adversary substitute training, as was done in (Papernot et al., 2017) , and the remaining 9, 850 samples are used for testing the different methods.

4.1. RESULTS ON BLACK-BOX ATTACKS:

In this section, we present experimental results on FGSM black-box attacks. As previously mentioned, the attacker trains a substitute model, which could differ in architecture from the targeted model, using a limited dataset consisting of 150 legitimate images augmented with synthetic images labeled using the target classifier. The classifier and substitute model architectures used and referred to throughout this section are described in Table 5 in the Appendix.

In Tables 1 and 2 , we present our classification accuracy results and compare to other defense methods. As can be seen, FGSM black-box attacks were successful at reducing the classifier accuracy by up to 70%. All considered defense mechanisms are relatively successful at diminishing the effect of the attacks. We note that, as expected, the performance of Defense-GAN-Rec and that of Defense-GAN-Orig are very close. In addition, they both perform consistently well across different classifier and substitute model combinations. MagNet also performs in a consistent manner, but achieves lower accuracy than Defense-GAN. In addition, the Defense-GAN parameters used in this experiment were kept the same for both Tables, in order to study the effect of dataset complexity, and can be further optimized as investigated in the next section. 

4.1.2. EFFECT OF ADVERSARIAL NOISE NORM:

We now investigate the effect of changing the attack in Table 3 . As expected, with higher , the FGSM attack is more successful, especially on the F-MNIST dataset where the noise norm seems to have a more pronounced effect with nearly 37% drop in performance between = 0.1 and 0.3. Figure 7 in Appendix D shows adversarial samples as well as their reconstructions with Defense-GAN at different values of . We can see that for large , the class is difficult to discern, even for the human eye.

Even though it seems that increasing is a desirable strategy for the attacker, this increases the likelihood that the adversarial noise is discernible and therefore the attack is detected. It is trivial for the attacker to provide adversarial images at very high , and a good measure of an attack's strength is its ability to affect performance at low . In fact, in the next section, we discuss how Defense-GAN can be used to not only diminish the effect of attacks, but to also detect them.

4.1.3. ATTACK DETECTION:

We intuitively expect that clean, unperturbed images will lie closer to the range of the Defense-GAN generator G than adversarial examples. This is due to the fact that G was trained to produce images which resemble the legitimate data. In light of this observation, we propose to use the MSE of an image with it is reconstruction from (8) as a "metric" to decide whether or not the image was adversarially manipulated. In order words, for a given threshold θ > 0, the hypothesis test is:

EQUATION

)

We compute the reconstruction MSEs for every image from the test dataset, and its adversarially manipulated version using FGSM. We show the Receiver Operating Characteristic (ROC) curves as well as the Area Under the Curve (AUC) metric for different Defense-GAN parameters and values in Figures 4 and 5 . The results show that this attack detection strategy is effective especially when the number of GD iterations L and random restarts R are large. From the left and middle Figures, we can conclude that the number of random restarts plays a very important role in the detection false positive and true positive rates as was discussed in Section 4.1.1. Furthermore, when is very small, it becomes difficult to detect attacks at low false positive rates.

4.1.4. RESULTS ON WHITE-BOX ATTACKS:

We now present results on white-box attacks using three different strategies: FGSM, RAND+FGSM, and CW. We perform the CW attack for 100 iterations of projected GD, with learning rate 10.0, and use c = 100 in equation ( 4). Table 4 shows the classification performance of different classifier models across different attack and defense strategies. We note that Defense-GAN significantly outperforms the two other baseline defenses. We even give the adversarial attacker access to the random initializations of z. However, we noticed that the performance does not change much when the attacker does not know the initialization. Adversarial training was done using FGSM to generate the adversarial samples. It is interesting to mention that when CW attack is used, adversarial training performs extremely poorly. As previously discussed, adversarial training does not generalize well against different attack methods.

Due to the loop of L steps of GD, Defense-GAN is resilient to GD-based white-box attacks, since the attacker needs to "un-roll" the GD loop and propagate the gradient of the loss all the way across L steps. In fact, from L = 25, the performance of the same network drops to 0.947 (more than 5% drop). This shows that using a larger L significantly increases the robustness of Defense-GAN against GD-based whitebox attacks. This comes at the expense of increased inference time complexity. We present a more detailed discussion about the difficulty of GD-based white-box attacks in Appendix B and time complexity in Appendix G. Additional white-box experimental results on higher-dimensional images are reported in Appendix F. 

5. CONCLUSION:

In this paper, we proposed Defense-GAN, a novel defense strategy utilizing GANs to enhance the robustness of classification models against black-box and white-box adversarial attacks. Our method does not assume a particular attack model and was shown to be effective against most commonly considered attack strategies. We empirically show that Defense-GAN consistently provides adequate defense on two benchmark computer vision datasets, whereas other methods had many shortcomings on at least one type of attack.

It is worth mentioning that, although Defense-GAN was shown to be a feasible defense mechanism against adversarial attacks, one might come across practical difficulties while implementing and deploying this method. The success of Defense-GAN relies on the expressiveness and generative power of the GAN. However, training GANs is still a challenging task and an active area of research, and if the GAN is not properly trained and tuned, the performance of Defense-GAN will suffer on both original and adversarial examples. Moreover, the choice of hyper-parameters L and R is also critical to the effectiveness of the defense and it may be challenging to tune them without knowledge of the attack. 

ACKNOWLEDGMENT:



Appendices:

A OPTIMALITY OF p g = p DATA FOR WGANS Sketch of proof of Lemma 1: The WGAN min-max loss is given by:

EQUATION

For a fixed G, the optimal discriminator D which maximizes V W (D, G) is such that:

D * G (x) = 1 if p data (x) ≥ p g (x) 0 otherwise (13)

Plugging D * G back into (12), we get:

V W (D * G , G) = x (p data (x) -p g (x)) D * G (x)dx (14) = {x | pdata(x)≥pg(x)} (p data (x) -p g (x)) dx (15) Let X = {x | p data (x) ≥ p g (x)}.

Clearly, to minimize (15), we need to set p data (x) = p g (x) for x ∈ X . Then, since both pdfs should integrate to 1,

EQUATION

However, this is a contradiction since p g (x) < p data (x) for x ∈ X c , unless µ(X c ) = 0 where µ is the Lebesgue measure. This concludes the proof.

B DIFFICULTY OF GD-BASED WHITE-BOX ATTACKS ON DEFENSE-GAN:

In order to perform a GD-based white-box attack on models using Defense-GAN, an attacker needs to compute the gradient of the output of the classifier with respect to the input. From Figure 1 , the generator and the classifier can be seen as one, combined, feedforward network, through which it is easy to propagate gradients. The difficulty lies in the orange box of the GD optimization detailed in Figure 2 .

For the sake of simplicity, let's assume that R = 1. Define L(x, z) = ||G(z) -x|| 2 2 . Then z * = z L , which is computed recursively as follows:

EQUATION

z 2 = z 1 + η 1 ∇ z L(x, z)| z=z1 (18) = z 0 + η 0 ∇ z L(x, z)| z=z0 + η 1 ∇ z L(x, z)| z=z0+η0 ∇zL(x,z)| z=z 0 (19)

and so on. Therefore, computing the gradient of z * with respect to x involves a large number (L) of recursive chain rules and high-dimensional Jacobian tensors. This computation gets increasingly prohibitive for large L.

C NEURAL NETWORK ARCHITECTURES:

We describe the neural network architectures used throughout the paper. The detail of models A through F used for classifier and substitute networks can be found in Table 5 . In Table 6 , the GAN architectures are described, and in 

F ADDITIONAL RESULTS ON WHITE-BOX ATTACKS:

We report results on white-box attacks on the CelebFaces Attributes dataset (CelebA) (Liu et al., 2015) in Table 12 . The CelebA dataset is a large-scale face dataset consisting of more than 200, 000 face images, split into training, validation, and testing sets. The RGB images were center-cropped and resized to 64 × 64. We performed the task of gender classification on this dataset. The GAN architecture is the same as that in Table 6 , except for an additional ConvT(128, 5 × 5, 1) layer in the generator network. 

G TIME COMPLEXITY:

The computational complexity of reconstructing an image using Defense-GAN is on the order of the number of GD iterations performed to estimate z * , multiplied by the time to compute gradients. The number of random restarts R has less effect on the running time, since random restarts are independent and can run in parallel if enough resources are available. Table 13 shows the average running time, in seconds, to find the reconstructions of MNIST and F-MNIST images on one NVIDIA GeForce GTX TITAN X GPU. For most applications, these running times are not prohibitive. We can see a tradeoff between running time and defense robustness as well as accuracy.

Table 13: 

:

PROPOSED DEFENSE-GANWe propose a new defense strategy which uses a WGAN trained on legitimate (un-perturbed) training samples to "denoise" adversarial examples. At test time, prior to feeding an image x to the classifier, we project it onto the range of the generator by minimizing the reconstruction error ||G(z) -x|| 2 2 , using L steps of GD. The resulting reconstruction G(z) is then given to the classifier. Since the generator was trained to model the unperturbed training data distribution, we expect this added step to result in a substantial reduction of any potential adversarial noise. We formally motivate this approach in the following section.

For simplicity, we will use GAN and WGAN interchangeably in the rest of this manuscript, with the understanding that our implementation follows the WGAN loss.

Figures and Tables:

Figure 2: L steps of Gradient Descent are used to estimate the projection of the image onto the range of the generator.

Figure3shows the effect of varying the number of GD iterations L as well as the random restarts R used to compute the GAN reconstructions of input images. Across different L and R values, Defense-GAN-Rec and Defense-GAN-Orig have comparable performance. Increasing L has the expected effect of improving performance when no attack is present. Interestingly, with an FGSM attack, the classification performance decreases after a certain L value. With too many GD iterations on the mean squared error (MSE) ||G(z) -(x + δ)|| 2 2 , some of the adversarial noise components are retained. In the right Figure, the effect of varying R is shown to be extremely pronounced. This is due to the non-convex nature of the MSE, and increasing R enables us to sample different local minima.

Figure 3: Classification accuracy of Model F using Defense-GAN on the MNIST dataset, under FGSM black-box attacks with = 0.3 and substitute Model E. Left: various number of iterations L are used (R = 10). Right: various number of random restarts R are used (L = 100).

Figure 4: ROC Curves when using Defense-GAN MSE for FGSM attack detections on the MNIST dataset (Classifier Model F, Substitute Model E). Left: Results for various number of GD iterations are shown with R = 10, = 0.30. Middle: Results for various number of random restarts R are shown with L = 100, = 0.30. Right: Results for various are shown with L = 400, R = 10.

Figure 5: ROC Curves when using Defense-GAN MSE for FGSM attack detections on the F-MNIST dataset (Classifier Model F, Substitute Model E). Left: Results for various number of GD iterations are shown with R = 10, = 0.30. Middle: Results for various number of random restarts R are shown with L = 100, = 0.30. Right: Results for various are shown with L = 200, R = 10.

Figure 6: Examples from MNIST and F-MNIST. Left: Original, FGSM adversarial = 0.3, and reconstruction images for R = 1 and various L are shown. Right: Original, FGSM adversarial = 0.3, and reconstruction images for L = 25 and various R are shown.

Figure 8: Classification accuracy of different models using Defense-GAN on the MNIST dataset, under FGSM white-box attack with = 0.3, for various number of iterations L and R = 10.

Average time, in seconds, to compute reconstructions of MNIST/F-MNIST images for various values of L and R. ± 0.027 0.070 ± 0.003 0.137 ± 0.004 0.273 ± 0.006 L = 0.543 ± 0.017 R = 2 0.042 ± 0.026 0.067 ± 0.002 0.131 ± 0.003 0.261 ± 0.006 L = 0.510 ± 0.006 R = 5 0.043 ± 0.029 0.070 ± 0.002 0.136 ± 0.004 0.270 ± 0.004 L = 0.535 ± 0.008 R = 10 0.051 ± 0.032 0.086 ± 0.001 0.170 ± 0.002 0.338 ± 0.008 L = 0.675 ± 0.016 R = 20 0.060 ± 0.035 0.105 ± 0.003 0.209 ± 0.006 0.414 ± 0.012 L = 0.825 ± 0.022

Two adversarial training defenses are presented: the first one obtains the adversarial examples assuming the same attack = 0.3, and the second assumes a different = 0.15. With incorrect knowledge of , the performance of adversarial training generally decreases. In addition, the classification performance of this defense method has very large variance across the different architectures. It is worth noting that adversarial training defense is only fit against FGSM attacks, because the adversarially augmented data, even with a different , is generated using the same method as the black-box attack (FGSM). In contrast, Defense-GAN and MagNet are general defense mechanisms which do not assume a specific attack model.The performances of defenses on the F-MNIST dataset, shown in Table2, are noticeably lower than on MNIST. This is due to the large = 0.3 in the FGSM attack. Please see Appendix D for qualitative examples showing that = 0.3 represents very high noise, which makes F-MNIST images difficult to classify, even by a human.

Table content: <table/>

Classification accuracies of different classifier and substitute model combinations using various defense strategies on the MNIST dataset, under FGSM black-box attacks with = 0.3. Defense-GAN has L = 200 and R = 10.

Table content: <table><tr><td>Classifier/ Substitute</td><td>No Attack</td><td>No Defense</td><td>Defense-GAN-Rec</td><td>Defense-GAN-Orig</td><td>MagNet</td><td>Adv. Tr. = 0.3</td><td>Adv. Tr. = 0.15</td></tr><tr><td>A/B</td><td colspan="2">0.9970 0.6343</td><td>0.9312</td><td>0.9282</td><td>0.6937</td><td>0.9654</td><td>0.6223</td></tr><tr><td>A/E</td><td colspan="2">0.9970 0.5432</td><td>0.9139</td><td>0.9221</td><td>0.6710</td><td>0.9668</td><td>0.9327</td></tr><tr><td>B/B</td><td colspan="2">0.9618 0.2816</td><td>0.9057</td><td>0.9105</td><td>0.5687</td><td>0.2092</td><td>0.3441</td></tr><tr><td>B/E</td><td colspan="2">0.9618 0.2128</td><td>0.8841</td><td>0.8892</td><td>0.4627</td><td>0.1120</td><td>0.3354</td></tr><tr><td>C/B</td><td colspan="2">0.9959 0.6648</td><td>0.9357</td><td>0.9322</td><td>0.7571</td><td>0.9834</td><td>0.9208</td></tr><tr><td>C/E</td><td colspan="2">0.9959 0.8050</td><td>0.9223</td><td>0.9182</td><td>0.6760</td><td>0.9843</td><td>0.9755</td></tr><tr><td>D/B</td><td colspan="2">0.9920 0.4641</td><td>0.9272</td><td>0.9323</td><td>0.6817</td><td>0.7667</td><td>0.8514</td></tr><tr><td>D/E</td><td colspan="2">0.9920 0.3931</td><td>0.9164</td><td>0.9155</td><td>0.6073</td><td>0.7676</td><td>0.7129</td></tr><tr><td colspan="7">4.1.1 EFFECT OF NUMBER OF GD ITERATIONS L AND RANDOM RESTARTS R</td><td/></tr></table>

Classification accuracies of different classifier and substitute model combinations using various defense strategies on the F-MNIST dataset, under FGSM black-box attacks with = 0.3. Defense-GAN has L = 200 and R = 10.

Table content: <table><tr><td>Classifier/ Substitute</td><td>No Attack</td><td>No Defense</td><td>Defense-GAN-Rec</td><td>Defense-GAN-Orig</td><td>MagNet</td><td>Adv. Tr. = 0.3</td><td>Adv. Tr. = 0.15</td></tr><tr><td>A/B</td><td colspan="2">0.9346 0.5131</td><td>0.586</td><td>0.5803</td><td>0.5404</td><td>0.7393</td><td>0.6600</td></tr><tr><td>A/E</td><td colspan="2">0.9346 0.3653</td><td>0.4790</td><td>0.4616</td><td>0.3311</td><td>0.6945</td><td>0.5638</td></tr><tr><td>B/B</td><td colspan="2">0.7470 0.4017</td><td>0.4940</td><td>0.5530</td><td>0.3812</td><td>0.3177</td><td>0.3560</td></tr><tr><td>B/E</td><td colspan="2">0.7470 0.3123</td><td>0.3720</td><td>0.4187</td><td>0.3119</td><td>0.2617</td><td>0.2453</td></tr><tr><td>C/B</td><td colspan="2">0.9334 0.2635</td><td>0.5289</td><td>0.6079</td><td>0.4664</td><td>0.7791</td><td>0.6838</td></tr><tr><td>C/E</td><td colspan="2">0.9334 0.2066</td><td>0.4871</td><td>0.4625</td><td>0.3016</td><td>0.7504</td><td>0.6655</td></tr><tr><td>D/B</td><td colspan="2">0.8923 0.4541</td><td>0.5779</td><td>0.5853</td><td>0.5478</td><td>0.6172</td><td>0.6395</td></tr><tr><td>D/E</td><td colspan="2">0.8923 0.2543</td><td>0.4007</td><td>0.4730</td><td>0.3396</td><td>0.5093</td><td>0.4962</td></tr></table>

Classification accuracy of Model F using Defense-GAN (L = 400, R = 10), under FGSM black-box attacks for various noise norms and substitute Model E.

Table content: <table><tr><td/><td/><td/><td/><td/><td/><td/><td/><td>Defense-GAN-Rec</td><td>Defense-GAN-Rec</td></tr><tr><td/><td/><td/><td/><td/><td/><td/><td/><td>MNIST</td><td>F-MNIST</td></tr><tr><td/><td/><td/><td/><td/><td/><td colspan="2">0.10</td><td>0.9864 ± 0.0011</td><td>0.8844 ± 0.0017</td></tr><tr><td/><td/><td/><td/><td/><td/><td colspan="2">0.15</td><td>0.9836 ± 0.0026</td><td>0.8267 ± 0.0065</td></tr><tr><td/><td/><td/><td/><td/><td/><td colspan="2">0.20</td><td>0.9772 ± 0.0019</td><td>0.7492 ± 0.0170</td></tr><tr><td/><td/><td/><td/><td/><td/><td colspan="2">0.25</td><td>0.9641 ± 0.0001</td><td>0.6384 ± 0.0159</td></tr><tr><td/><td/><td/><td/><td/><td/><td colspan="2">0.30</td><td>0.9307 ± 0.0034</td><td>0.5126 ± 0.0096</td></tr><tr><td>0.0</td><td>0.1</td><td>0.2</td><td>0.3</td><td>0.4 False Positive Rate 0.5 0.6</td><td>0.7</td><td>0.8</td><td>0.9</td><td>1.0</td></tr></table>



Table content: <table><tr><td colspan="9">Published as a conference paper at ICLR 2018</td></tr><tr><td>0.0</td><td>0.1</td><td>0.2</td><td>0.3</td><td>0.4 False Positive Rate 0.5 0.6</td><td>0.7</td><td>0.8</td><td>0.9</td><td>1.0</td></tr></table>

Classification accuracies of different classifier models using various defense strategies on the MNIST (top) and F-MNIST (bottom) datasets, under FGSM, RAND+FGSM, and CW white-box attacks. Defense-GAN has L = 200 and R = 10.

Table content: <table><tr><td>Attack</td><td>Classifier Model</td><td>No Attack</td><td>No Defense</td><td>Defense-GAN-Rec</td><td>MagNet</td><td>Adv. Tr. = 0.3</td></tr><tr><td/><td>A</td><td>0.997</td><td>0.217</td><td>0.988</td><td>0.191</td><td>0.651</td></tr><tr><td>FGSM</td><td>B</td><td>0.962</td><td>0.022</td><td>0.956</td><td>0.082</td><td>0.060</td></tr><tr><td>= 0.3</td><td>C</td><td>0.996</td><td>0.331</td><td>0.989</td><td>0.163</td><td>0.786</td></tr><tr><td/><td>D</td><td>0.992</td><td>0.038</td><td>0.980</td><td>0.094</td><td>0.732</td></tr><tr><td/><td>A</td><td>0.997</td><td>0.179</td><td>0.988</td><td>0.171</td><td>0.774</td></tr><tr><td>RAND+FGSM</td><td>B</td><td>0.962</td><td>0.017</td><td>0.944</td><td>0.091</td><td>0.138</td></tr><tr><td>= 0.3, α = 0.05</td><td>C</td><td>0.996</td><td>0.103</td><td>0.985</td><td>0.151</td><td>0.907</td></tr><tr><td/><td>D</td><td>0.992</td><td>0.050</td><td>0.980</td><td>0.115</td><td>0.539</td></tr><tr><td/><td>A</td><td>0.997</td><td>0.141</td><td>0.989</td><td>0.038</td><td>0.077</td></tr><tr><td>CW</td><td>B</td><td>0.962</td><td>0.032</td><td>0.916</td><td>0.034</td><td>0.280</td></tr><tr><td>2 norm</td><td>C</td><td>0.996</td><td>0.126</td><td>0.989</td><td>0.025</td><td>0.031</td></tr><tr><td/><td>D</td><td>0.992</td><td>0.032</td><td>0.983</td><td>0.021</td><td>0.010</td></tr><tr><td>Attack</td><td>Classifier Model</td><td>No Attack</td><td>No Defense</td><td>Defense-GAN-Rec</td><td>MagNet</td><td>Adv. Tr. = 0.3</td></tr><tr><td/><td>A</td><td>0.934</td><td>0.102</td><td>0.879</td><td>0.089</td><td>0.797</td></tr><tr><td>FGSM</td><td>B</td><td>0.747</td><td>0.102</td><td>0.629</td><td>0.168</td><td>0.136</td></tr><tr><td>= 0.3</td><td>C</td><td>0.933</td><td>0.139</td><td>0.896</td><td>0.110</td><td>0.804</td></tr><tr><td/><td>D</td><td>0.892</td><td>0.082</td><td>0.875</td><td>0.099</td><td>0.698</td></tr><tr><td/><td>A</td><td>0.934</td><td>0.102</td><td>0.888</td><td>0.096</td><td>0.447</td></tr><tr><td>RAND+FGSM</td><td>B</td><td>0.747</td><td>0.131</td><td>0.661</td><td>0.161</td><td>0.119</td></tr><tr><td>= 0.3, α = 0.05</td><td>C</td><td>0.933</td><td>0.105</td><td>0.893</td><td>0.112</td><td>0.699</td></tr><tr><td/><td>D</td><td>0.892</td><td>0.091</td><td>0.862</td><td>0.104</td><td>0.626</td></tr><tr><td/><td>A</td><td>0.934</td><td>0.076</td><td>0.896</td><td>0.060</td><td>0.157</td></tr><tr><td>CW</td><td>B</td><td>0.747</td><td>0.172</td><td>0.656</td><td>0.131</td><td>0.118</td></tr><tr><td>2 norm</td><td>C</td><td>0.933</td><td>0.063</td><td>0.896</td><td>0.084</td><td>0.107</td></tr><tr><td/><td>D</td><td>0.892</td><td>0.090</td><td>0.875</td><td>0.069</td><td>0.149</td></tr></table>

This research is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA R&D Contract No. 2014-14071600012. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.

Table content: <table/>

Table 7, the encoder architecture for the MagNet baseline is given. In what follows: • Conv(m, k × k, s) refers to a convolutional layer with m feature maps, filter size k × k, and stride s • ConvT(m, k × k) refers to the transpose (gradient) of Conv (sometimes referred to as "deconvolution") with m feature maps, filter size k × k, and stride s Neural network architectures used for classifiers and substitute models.

Table content: <table><tr><td colspan="3">• FC(m) refers to a fully-connected layer with m outputs</td><td/></tr><tr><td colspan="3">• Dropout(p) refers to a dropout layer with probability p</td><td/></tr><tr><td colspan="3">• ReLU refers to the Rectified Linear Unit activation</td><td/></tr><tr><td colspan="4">• LeakyReLU(α) is the leaky version of the Rectified Linear Unit with parameter α</td></tr><tr><td>A</td><td>B, F*</td><td>C</td><td>D, E*</td></tr><tr><td>Conv(64, 5 × 5, 1)</td><td>Dropout(0.2)</td><td>Conv(128, 3 × 3, 1)</td><td>FC(200)</td></tr><tr><td>ReLU</td><td>Conv(64, 8 × 8, 2)</td><td>ReLU</td><td>ReLU</td></tr><tr><td>Conv(64, 5 × 5, 2)</td><td>ReLU</td><td>Conv(64, 3 × 3, 2)</td><td>Dropout(0.5)</td></tr><tr><td>ReLU</td><td>Conv(128, 6 × 6, 2)</td><td>ReLU</td><td>FC(200)</td></tr><tr><td>Dropout(0.25)</td><td>ReLU</td><td>Dropout(0.25)</td><td>ReLU</td></tr><tr><td>FC(128)</td><td>Conv(128, 5 × 5, 1)</td><td>FC(128)</td><td>Dropout(0.5)</td></tr><tr><td>ReLU</td><td>ReLU</td><td>ReLU</td><td>FC(10) + Softmax</td></tr><tr><td>Dropout(0.5)</td><td>Dropout(0.5)</td><td>Dropout(0.5)</td><td/></tr><tr><td>FC(10) + Softmax</td><td>FC(10) + Softmax</td><td>FC(10) + Softmax</td><td/></tr><tr><td>[</td><td/><td/><td/></tr></table>

Neural network architectures used for GANs.

Table content: <table><tr><td>Generator</td><td>Discriminator</td></tr><tr><td>FC(4096)</td><td>Conv(64, 5 × 5, 2)</td></tr><tr><td>ReLU</td><td>LeakyReLU(0.2)</td></tr><tr><td colspan="2">ConvT(256, 5 × 5, 1) Conv(128, 5 × 5, 2)</td></tr><tr><td>ReLU</td><td>LeakyReLU(0.2)</td></tr><tr><td colspan="2">ConvT(128, 5 × 5, 1) Conv(256, 5 × 5, 2)</td></tr><tr><td>ReLU</td><td>LeakyReLU(0.2)</td></tr><tr><td>ConvT(1, 5 × 5, 1)</td><td>FC(1)</td></tr><tr><td>Sigmoid</td><td>Sigmoid</td></tr></table>

Neural network architecture used for the MagNet encoder.

Table content: <table><tr><td>Encoder</td></tr><tr><td>Conv(64, 5 × 5, 2)</td></tr><tr><td>LeakyReLU(0.2)</td></tr><tr><td>Conv(128, 5 × 5, 2)</td></tr><tr><td>LeakyReLU(0.2)</td></tr><tr><td>Conv(256, 5 × 5, 2)</td></tr><tr><td>LeakyReLU(0.2)</td></tr><tr><td>FC(128) + tanh</td></tr></table>

Classification accuracy of Model F using Defense-GAN with various number of iterations L (R = 10), on the MNIST dataset, under FGSM black-box attack with = 0.3.

Table content: <table><tr><td>L</td><td>Defense-GAN-Rec No attack</td><td>Defense-GAN-Orig No attack</td><td>Defense-GAN-Rec Adversarial</td><td>Defense-GAN-Orig Adversarial</td></tr><tr><td>25</td><td>0.9273 ± 0.0215</td><td>0.9141 ± 0.0033</td><td>0.7955 ± 0.0045</td><td>0.7998 ± 0.0063</td></tr><tr><td>50</td><td>0.9567 ± 0.0203</td><td>0.9371 ± 0.0048</td><td>0.8516 ± 0.0078</td><td>0.8472 ± 0.0026</td></tr><tr><td>100</td><td>0.9728 ± 0.0164</td><td>0.9560 ± 0.0051</td><td>0.8953 ± 0.0027</td><td>0.8911 ± 0.0024</td></tr><tr><td>200</td><td>0.9860 ± 0.0010</td><td>0.9712 ± 0.0028</td><td>0.9210 ± 0.0023</td><td>0.9155 ± 0.0032</td></tr><tr><td>400</td><td>0.9869 ± 0.0082</td><td>0.9808 ± 0.0044</td><td>0.9332 ± 0.0027</td><td>0.9307 ± 0.0034</td></tr><tr><td>800</td><td>0.9934 ± 0.0009</td><td>0.9938 ± 0.0004</td><td>0.9319 ± 0.0038</td><td>0.9216 ± 0.0005</td></tr><tr><td>1600</td><td>0.9963 ± 0.0013</td><td>0.9967 ± 0.0005</td><td>0.9081 ± 0.0062</td><td>0.9008 ± 0.0095</td></tr></table>

Classification accuracy of Model F using Defense-GAN with various number of iterations L (R = 10), on the F-MNIST dataset, under FGSM black-box attack with = 0.3.

Table content: <table><tr><td>L</td><td>Defense-GAN-Rec No attack</td><td>Defense-GAN-Orig No attack</td><td>Defense-GAN-Rec Adversarial</td><td>Defense-GAN-Orig Adversarial</td></tr><tr><td>25</td><td>0.8037 ± 0.0050</td><td>0.7595 ± 0.0009</td><td>0.4040 ± 0.0149</td><td>0.3910 ± 0.0119</td></tr><tr><td>50</td><td>0.8676 ± 0.0018</td><td>0.7898 ± 0.0016</td><td>0.4412 ± 0.0023</td><td>0.3980 ± 0.0114</td></tr><tr><td>100</td><td>0.9101 ± 0.0032</td><td>0.8190 ± 0.0043</td><td>0.4808 ± 0.0088</td><td>0.4221 ± 0.0255</td></tr><tr><td>200</td><td>0.9145 ± 0.0014</td><td>0.8373 ± 0.0054</td><td>0.5119 ± 0.0038</td><td>0.4594 ± 0.0056</td></tr><tr><td>400</td><td>0.9490 ± 0.0013</td><td>0.8557 ± 0.0049</td><td>0.5126 ± 0.0096</td><td>0.4754 ± 0.0102</td></tr><tr><td>800</td><td>0.9588 ± 0.0065</td><td>0.8832 ± 0.0042</td><td>0.5520 ± 0.0098</td><td>0.4644 ± 0.0092</td></tr><tr><td>1600</td><td>0.9640 ± 0.0010</td><td>0.9125 ± 0.0040</td><td>0.5335 ± 0.0226</td><td>0.4952 ± 0.0155</td></tr></table>

Classification accuracy of Model F using Defense-GAN with various number of random restarts R (L = 100), on the MNIST dataset, under FGSM black-box attack with = 0.3.

Table content: <table><tr><td>R</td><td>Defense-GAN-Rec No attack</td><td>Defense-GAN-Orig No attack</td><td>Defense-GAN-Rec Adversarial</td><td>Defense-GAN-Orig Adversarial</td></tr><tr><td>1</td><td>0.7035 ± 0.0035</td><td>0.6436 ± 0.0017</td><td>0.5329 ± 0.0094</td><td>0.5011 ± 0.0085</td></tr><tr><td>2</td><td>0.8619 ± 0.0010</td><td>0.8080 ± 0.0029</td><td>0.6722 ± 0.0041</td><td>0.6605 ± 0.0050</td></tr><tr><td>5</td><td>0.9523 ± 0.0006</td><td>0.9213 ± 0.0024</td><td>0.8199 ± 0.0097</td><td>0.8228 ± 0.0038</td></tr><tr><td>10</td><td>0.9810 ± 0.0015</td><td>0.9560 ± 0.0051</td><td>0.8956 ± 0.0032</td><td>0.8911 ± 0.0024</td></tr><tr><td>20</td><td>0.9966 ± 0.0009</td><td>0.9753 ± 0.0010</td><td>0.9456 ± 0.0031</td><td>0.9310 ± 0.0023</td></tr></table>

Classification accuracy of Model F using Defense-GAN with various number of random restarts R (L = 100), on the F-MNIST dataset, under FGSM black-box attack with = 0.3.

Table content: <table><tr><td>R</td><td>Defense-GAN-Rec No attack</td><td colspan="3">Defense-GAN-Orig No attack</td><td>Defense-GAN-Rec Adversarial</td><td>Defense-GAN-Orig Adversarial</td></tr><tr><td>1</td><td>0.8425 ± 0.0008</td><td/><td colspan="2">0.5597 ± 0.0015</td><td>0.3504 ± 0.0102</td><td>0.3380 ± 0.0043</td></tr><tr><td>2</td><td>0.8994 ± 0.0051</td><td/><td colspan="2">0.7793 ± 0.0023</td><td>0.4050 ± 0.0148</td><td>0.3508 ± 0.0167</td></tr><tr><td>5</td><td>0.9260 ± 0.0028</td><td/><td colspan="2">0.6726 ± 0.0006</td><td>0.4521 ± 0.0177</td><td>0.4024 ± 0.0085</td></tr><tr><td>10</td><td>0.9101 ± 0.0032</td><td/><td colspan="2">0.8190 ± 0.0043</td><td>0.4808 ± 0.0088</td><td>0.4221 ± 0.0255</td></tr><tr><td/><td/><td>1.000</td><td/><td/></tr><tr><td/><td/><td>0.975</td><td/><td/></tr><tr><td/><td/><td>0.950</td><td/><td/></tr><tr><td/><td/><td>0.925</td><td/><td/></tr><tr><td/><td/><td>0.900</td><td/><td/></tr><tr><td/><td/><td>0.875</td><td/><td/></tr><tr><td/><td/><td>0.850</td><td>0</td><td colspan="2">Number of GD iterations L 50 100 150 200 250 300 350 400</td></tr></table>

Classification accuracies of different classifier models using various defense strategies on the CelebA gender classification task, under FGSM, RAND+FGSM, and CW white-box attacks. Defense-GAN has L = 200 and R = 2.

Table content: <table><tr><td>Attack</td><td>Classifier Model</td><td>No Attack</td><td>No Defense</td><td>Defense-GAN-Rec</td><td>MagNet</td><td>Adv. Tr. = 0.3</td></tr><tr><td/><td>A</td><td colspan="2">0.9652 0.0870</td><td>0.9255</td><td>0.0985</td><td>0.1225</td></tr><tr><td>FGSM</td><td>B</td><td colspan="2">0.9468 0.0995</td><td>0.9140</td><td>0.0920</td><td>0.2345</td></tr><tr><td>= 0.3</td><td>C</td><td colspan="2">0.9459 0.0460</td><td>0.9255</td><td>0.1085</td><td>0.1130</td></tr><tr><td/><td>D</td><td colspan="2">0.9476 0.0605</td><td>0.9205</td><td>0.0975</td><td>0.7755</td></tr><tr><td/><td>A</td><td colspan="2">0.9652 0.0560</td><td>0.9280</td><td>0.1105</td><td>0.0700</td></tr><tr><td>RAND+FGSM</td><td>B</td><td colspan="2">0.9468 0.1785</td><td>0.9030</td><td>0.1015</td><td>0.4515</td></tr><tr><td>= 0.3, α = 0.05</td><td>C</td><td colspan="2">0.9459 0.0470</td><td>0.9200</td><td>0.1045</td><td>0.1055</td></tr><tr><td/><td>D</td><td colspan="2">0.9476 0.0665</td><td>0.9165</td><td>0.1105</td><td>0.696</td></tr><tr><td/><td>A</td><td colspan="2">0.9652 0.0460</td><td>0.8210</td><td>0.0985</td><td>0.5690</td></tr><tr><td>CW</td><td>B</td><td colspan="2">0.9468 0.0575</td><td>0.7465</td><td>0.0955</td><td>0.0725</td></tr><tr><td>2 norm</td><td>C</td><td colspan="2">0.9459 0.0435</td><td>0.7985</td><td>0.0985</td><td>0.2635</td></tr><tr><td/><td>D</td><td colspan="2">0.9476 0.0660</td><td>0.7740</td><td>0.1040</td><td>0.5010</td></tr></table>

Peer review (iclr2018):
 The authors describe a new defense mechanism against adversarial attacks on classifiers (e.g., FGSM). They propose utilizing Generative Adversarial Networks (GAN), which are usually used for training generative models for an unknown distribution, but have a natural adversarial interpretation. In particular, a GAN consists of a generator NN G which maps a random vector z to an example x, and a discriminator NN D which seeks to discriminate between an examples produced by G and examples drawn from the true distribution. The GAN is trained to minimize the max min loss of D on this discrimination task, thereby producing a G (in the limit) whose outputs are indistinguishable from the true distribution by the best discriminator. Utilizing a trained GAN, the authors propose the following defense at inference time. Given a sample x (which has been adversarially perturbed), first project x onto the range of G by solving the minimization problem z* = argmin_z ||G(z) - x||_2. This is done by SGD. Then apply any classifier trained on the true distribution on the resulting x* = G(z*). In the case of existing black-box attacks, the authors argue (convincingly) that the method is both flexible and empirically effective. In particular, the defense can be applied in conjunction with any classifier (including already hardened classifiers), and does not assume any specific attack model. Nevertheless, it appears to be effective against FGSM attacks, and competitive with adversarial training specifically to defend against FGSM. The authors provide less-convincing evidence that the defense is effective against white-box attacks. In particular, the method is shown to be robust against FGSM, RAND+FGSM, and CW white-box attacks. However, it is not clear to me that the method is invulnerable to novel white-box attacks. In particular, it seems that the attacker can design an x which projects onto some desired x* (using some other method entirely), which then fools the classifier downstream. Nevertheless, the method is shown to be an effective tool for hardening any classifier against existing black-box attacks (which is arguably of great practical value). It is novel and should generate further research with respect to understanding its vulnerabilities more completely. Minor Comments: The sentence starting “Unless otherwise specified…” at the top of page 7 is confusing given the actual contents of Tables 1 and 2, which are clarified only by looking at Table 5 in the appendix. This should be fixed.