Original: D EFENSE -GAN: P ROTECTING C LASSIFIERS A GAINST A DVERSARIAL A TTACKS U SING G ENERATIVE M ODELS
Revised: D EFENSE -GAN: P ROTECTING C LASSIFIERS A GAINST A DVERSARIAL A TTACKS U SING G ENERATIVE M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: In recent years,  neural network approaches have been widely adopted for machine learning tasks, including classiﬁcation.
Revised: In recent years, deep neural network approaches have been widely adopted for machine learning tasks, including classiﬁcation.

Original: However, they were shown to be vulnerable to adversarial perturbations: carefully crafted small perturbations can cause misclassiﬁcation of legitimate images.
Revised: However, they were shown to be vulnerable to adversarial perturbations: carefully crafted small perturbations can cause misclassiﬁcation of legitimate images.

Original: We propose Defense-GAN, a new framework leveraging the expressive capability of generative models to defend deep neural networks against such attacks.
Revised: We propose Defense-GAN, a new framework leveraging the expressive capability of generative models to defend deep neural networks against such attacks.

Original: Defense-GAN is trained to model the distribution of unperturbed images and, at inference time, ﬁnds a close output to a given image.
Revised: Defense-GAN is trained to model the distribution of unperturbed images and, at inference time, ﬁnds a close output to a given image.

Original: This output will not contain the adversarial changes and is fed to the classiﬁer.
Revised: This output will not contain the adversarial changes and is fed to the classiﬁer.

Original: Our proposed method can be used with any classiﬁcation model and does not modify the classiﬁer structure or training procedure.
Revised: Our proposed method can be used with any classiﬁcation model and does not modify the classiﬁer structure or training procedure.

Original: It can also be used as a defense against any attack as it does not assume knowledge of the process for generating the adversarial examples.
Revised: It can also be used as a defense against any attack as it does not assume knowledge of the process for generating the adversarial examples.

Original: We empirically show that Defense-GAN is consistently effective against different attack methods and improves on existing defense strategies.
Revised: We empirically show that Defense-GAN is consistently effective against different attack methods and improves on existing defense strategies.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Despite their outstanding performance on several machine learning tasks (Krizhevsky et al., 2012;
Revised: Despite their outstanding performance on several machine learning tasks (Krizhevsky et al., 2012;

Original: Sermanet et al., 2014), deep neural networks have been shown to be susceptible to adversarial attacks (Szegedy et al., 2014; Goodfellow et al., 2015).
Revised: Sermanet et al., 2014), deep neural networks have been shown to be susceptible to adversarial attacks (Szegedy et al., 2014; Goodfellow et al., 2015).

Original: These attacks come in the form of adversarial examples : carefully crafted perturbations added to a legitimate input sample.
Revised: These attacks come in the form of adversarial examples : carefully crafted perturbations added to a legitimate input sample.

Original: In the context of classiﬁcation, these perturbations cause the legitimate sample to be misclassiﬁed at inference time (Szegedy et al., 2014; Goodfellow et al., 2015; Papernot et al., 2016b; Liu et al., 2017).
Revised: In the context of classiﬁcation, these perturbations cause the legitimate sample to be misclassiﬁed at inference time (Szegedy et al., 2014; Goodfellow et al., 2015; Papernot et al., 2016b; Liu et al., 2017).

Original: Such perturbations are often small in magnitude and do not affect human recognition but can drastically change the output of the classiﬁer.
Revised: Such perturbations are often small in magnitude and do not affect human recognition but can drastically change the output of the classiﬁer.

Original: Recent literature has considered two types of threat models: black-box and white-box attacks.
Revised: Recent literature has considered two types of threat models: black-box and white-box attacks.

Original: Under the black-box attack model, the attacker does not have access to the classiﬁcation model parameters; whereas in the white-box attack model, the attacker has complete access to the model architecture and parameters, including potential defense mechanisms (Papernot et al., 2017; Tram`er et al., 2017;
Revised: Under the black-box attack model, the attacker does not have access to the classiﬁcation model parameters; whereas in the white-box attack model, the attacker has complete access to the model architecture and parameters, including potential defense mechanisms (Papernot et al., 2017; Tram`er et al., 2017;

Original: Carlini & Wagner, 2017).
Revised: Carlini & Wagner, 2017).

Original: Various defenses have been proposed to mitigate the effect of adversarial attacks.
Revised: Various defenses have been proposed to mitigate the effect of adversarial attacks.

Original: These defenses can be grouped under three different approaches: (1) modifying the training data to make the classiﬁer more robust against attacks, e.g., adversarial training which augments the training data of the classiﬁer with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015), (2) modifying the training procedure of the classiﬁer to reduce the magnitude of gradients, e.g., defensive distillation (Papernot et al., 2016d), and (3) attempting to remove the adversarial noise from the input samples (Hendrycks & Gimpel, 2017; Meng & Chen, 2017).
Revised: These defenses can be grouped under three different approaches: (1) modifying the training data to make the classiﬁer more robust against attacks, e.g., adversarial training which augments the training data of the classiﬁer with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015), (2) modifying the training procedure of the classiﬁer to reduce the magnitude of gradients, e.g., defensive distillation (Papernot et al., 2016d), and (3) attempting to remove the adversarial noise from the input samples (Hendrycks & Gimpel, 2017; Meng & Chen, 2017).

Original: All of these approaches have limitations in the sense that they are effective against either white-box attacks or black-box attacks, but not both (Tram`er et al., 2017; Meng & Chen, 2017).
Revised: All of these approaches have limitations in the sense that they are effective against either white-box attacks or black-box attacks, but not both (Tram`er et al., 2017; Meng & Chen, 2017).

Original: Furthermore, some of these defenses are devised with speciﬁc attack models in mind and fail to generalize against new attacks.
Revised: Furthermore, some of these defenses are devised with speciﬁc attack models in mind and are not effective against new attacks.

Original: In this paper, we propose a novel defense mechanism which is effective against both white-box and black-box attacks.
Revised: In this paper, we propose a novel defense mechanism which is effective against both white-box and black-box attacks.

Original: We propose to leverage the representative power of Generative Adversarial
Revised: We propose to leverage the representative power of Generative Adversarial

Original: Networks (GAN) (Goodfellow et al., 2014) to diminish the effect of the adversarial perturbation, by projecting input images onto the range of the GAN’s generator prior to feeding them to the classiﬁer.
Revised: Networks (GAN) (Goodfellow et al., 2014) to diminish the effect of the adversarial perturbation, by projecting input images onto the range of the GAN’s generator prior to feeding them to the classiﬁer.

Original: In the GAN framework, two models are trained simultaneously in an adversarial setting: a generative model that emulates the data distribution, and a discriminative model that predicts whether a certain input came from real data or was artiﬁcially created.
Revised: In the GAN framework, two models are trained simultaneously in an adversarial setting: a generative model that emulates the data distribution, and a discriminative model that predicts whether a certain input came from real data or was artiﬁcially created.

Original: The generative model learns a mapping G from a low-dimensional vector z ∈ R k to the high-dimensional input sample space R n .
Revised: The generative model learns a mapping G from a low-dimensional vector z ∈ R k to the high-dimensional input sample space R n .

Original: During training of the GAN, G is encouraged to generate samples which resemble the training data.
Revised: During training of the GAN, G is encouraged to generate samples which resemble the training data.

Original: It is, therefore, expected that legitimate samples will be close to some point in the range of G , whereas adversarial samples will be further away from the range of G .
Revised: It is, therefore, expected that legitimate samples will be close to some point in the range of G , whereas adversarial samples will be further away from the range of G .

Original: Furthermore, “projecting” the adversarial examples onto the range of the generator G can have the desirable effect of reducing the adversarial perturbation.
Revised: Furthermore, “projecting” the adversarial examples onto the range of the generator G can have the desirable effect of reducing the adversarial perturbation.

Original: The projected output, computed using Gradient Descent (GD), is fed into the classiﬁer instead of the original (potentially adversarially modiﬁed) image.
Revised: The projected output, computed using Gradient Descent (GD), is fed into the classiﬁer instead of the original (potentially adversarially modiﬁed) image.

Original: We empirically demonstrate that this is an effective defense against both black-box and white-box attacks on two benchmark image datasets.
Revised: We empirically demonstrate that this is an effective defense against both black-box and white-box attacks on two benchmark image datasets.

Original: The rest of the paper is organized as follows.
Revised: The rest of the paper is organized as follows.

Original: We introduce necessary background regarding known attack models, defense mechanisms, and GANs in Section 2.
Revised: We introduce necessary background regarding known attack models, defense mechanisms, and GANs in Section 2.

Original: Our defense mechanism, which we call Defense-GAN , is formally motivated and deﬁned in Section 3.
Revised: Our defense mechanism, which we call Defense-GAN , is formally motivated and introduced in Section 3.

Original: Finally, experimental results, under different threat models, as well as comparisons to other defenses are presented in Section 4.
Revised: Finally, experimental results, under different threat models, as well as comparisons to other defenses are presented in Section 4.

Original:  R ELATED WORK AND BACKGROUND INFORMATION
Revised:  R ELATED WORK AND BACKGROUND INFORMATION

Original: In this work, we propose to use GANs for the purpose of defending against adversarial attacks in classiﬁcation problems.
Revised: In this work, we propose to use GANs for the purpose of defending against adversarial attacks in classiﬁcation problems.

Original: Before detailing our approach in the next section, we explain related work in three parts.
Revised: Before detailing our approach in the next section, we explain related work in three parts.

Original: First, we discuss different attack models employed in the literature.
Revised: First, we discuss different attack models employed in the literature.

Original: We, then, go over main defense mechanisms against these attacks and discuss their strengths and shortcomings.
Revised: We, then, go over main defense mechanisms against these attacks and discuss their strengths and shortcomings.

Original: Lastly, we explain necessary background information regarding GANs.
Revised: Lastly, we explain necessary background information regarding GANs.

Original: 2. A TTACK MODELS AND ALGORITHMS
Revised: 2. A TTACK MODELS AND ALGORITHMS

Original: Various attack models and algorithms have been used to target classiﬁers.
Revised: Various attack models and algorithms have been used to target classiﬁers.

Original: All attack models we consider aim to ﬁnd a perturbation δ to be added to a (legitimate) input x ∈ R n , resulting in the adversarial example ˜ x
Revised: All attack models we consider aim to ﬁnd a perturbation δ to be added to a (legitimate) input x ∈ R n , resulting in the adversarial example ˜ x

Original: = x + δ .
Revised: = x + δ .

Original: The (cid:96) ∞ -norm of the perturbation is denoted by (cid:15)
Revised: The (cid:96) ∞ -norm of the perturbation is denoted by (cid:15)

Original: (Goodfellow et al., 2015) and is chosen to be small enough so as to remain undetectable.
Revised: (Goodfellow et al., 2015) and is chosen to be small enough so as to remain undetectable.

Original: We consider two threat levels:black- and white-box attacks.
Revised: We consider two threat levels:

Original: 2.1. W HITE - BOX ATTACK MODELS
Revised: 2.1. W HITE - BOX ATTACK MODELS

Original: White-box models assume that the attacker has complete access to all the classiﬁer parameters, i.e., network architecture and weights, as well as the details of any defense mechanism (and potentially even the randomness).
Revised: White-box models assume that the attacker has complete access to all the classiﬁer parameters, i.e., network architecture and weights, as well as the details of any defense mechanism (and potentially even the randomness).

Original: Given an input image x and its associated ground-truth label y , the attacker thus has access to the loss function J ( x , y ) used to train the network, and uses it to compute the adversarial perturbation δ .
Revised: Given an input image x and its associated ground-truth label y , the attacker thus has access to the loss function J ( x , y ) used to train the network, and uses it to compute the adversarial perturbation δ .

Original: Attacks can be targeted , in that they attempt to cause the perturbed image to be misclassiﬁed to a speciﬁc target class, or untargeted when no target class is speciﬁed.
Revised: Attacks can be targeted , in that they attempt to cause the perturbed image to be misclassiﬁed to a speciﬁc target class, or untargeted when no target class is speciﬁed.

Original: In this work, we focus on untargeted white-box attacks computed using the Fast Gradient
Revised: In this work, we focus on untargeted white-box attacks computed using the Fast Gradient

Original: Sign Method (FGSM) (Goodfellow et al., 2015), the Randomized Fast Gradient Sign Method (RAND+FGSM) (Tram`er et al., 2017), and the Carlini-Wagner (CW) attack (Carlini & Wagner, 2017).
Revised: Sign Method (FGSM) (Goodfellow et al., 2015), the Randomized Fast Gradient Sign Method (RAND+FGSM) (Tram`er et al., 2017), and the Carlini-Wagner (CW) attack (Carlini & Wagner, 2017).

Original: Although other attack models exist, such as the Iterative FGSM (Kurakin et al., 2017), the
Revised: Although other attack models exist, such as the Iterative FGSM (Kurakin et al., 2017), the

Original: Jacobian-based Saliency Map Attack (JSMA) (Papernot et al., 2016b), and Deepfool (MoosaviDezfooli et al., 2016)
Revised: Jacobian-based Saliency Map Attack (JSMA) (Papernot et al., 2016b), and Deepfool (MoosaviDezfooli et al., 2016)

Original: , we focus on these three models as they cover a good breadth of attack algorthims.
Revised: , we focus on these three models as they cover a good breadth of attack algorthims.

Original: FGSM is a very simple and fast attack algorithm which makes it extremely amenable to real-time attack deployment.
Revised: FGSM is a very simple and fast attack algorithm which makes it extremely amenable to real-time attack deployment.

Original: On the other hand, RAND+FGSM, an equally simple attack, increases the power of FGSM for white-box attacks (Tram`er et al., 2017), and ﬁnally, the CW attack is one of the most powerful white-box attacks to-date (Carlini & Wagner, 2017).
Revised: On the other hand, RAND+FGSM, an equally simple attack, increases the power of FGSM for white-box attacks (Tram`er et al., 2017), and ﬁnally, the CW attack is one of the most powerful white-box attacks to-date (Carlini & Wagner, 2017).

Original: Fast Gradient Sign Method (FGSM)
Revised: Fast Gradient Sign Method (FGSM)

Original: Given an image x and its corresponding true label y , the
Revised: Given an image x and its corresponding true label y , the

Original: FGSM attack sets the perturbation δ to:
Revised: FGSM attack sets the perturbation δ to:

Original: [Equation]
Revised: [Equation]

Original: FGSM (Goodfellow et al., 2015) was designed to be extremely fast rather than optimal.
Revised: FGSM (Goodfellow et al., 2015) was designed to be extremely fast rather than optimal.

Original: It simply uses the sign of the gradient at every pixel to determine the direction with which to change the corresponding pixel value.
Revised: It simply uses the sign of the gradient at every pixel to determine the direction with which to change the corresponding pixel value.

Original: Randomized Fast Gradient Sign Method (RAND+FGSM)
Revised: Randomized Fast Gradient Sign Method (RAND+FGSM)

Original: The RAND+FGSM (Tram`er et al., 2017) attack is a simple yet effective method to increase the power of the FGSM against models which were adversarially trained.
Revised: The RAND+FGSM (Tram`er et al., 2017) attack is a simple yet effective method to increase the power of the FGSM against models which were adversarially trained.

Original: The idea is to ﬁrst apply a small random perturbation before using the FGSM.
Revised: The idea is to ﬁrst apply a small random perturbation before using the FGSM.

Original: More explicitly, for α < (cid:15) , random noise is ﬁrst added to the legitimate image x :
Revised: More explicitly, for α < (cid:15) , random noise is ﬁrst added to the legitimate image x :

Original: [Equation]
Revised: [Equation]

Original: Then, the FGSM attack is computed on x (cid:48) , resulting in
Revised: Then, the FGSM attack is computed on x (cid:48) , resulting in

Original: [Equation]
Revised: [Equation]

Original: The Carlini-Wagner (CW) attack The CW attack is an elaborate but extremely effective attack model (Carlini & Wagner, 2017).
Revised: The Carlini-Wagner (CW) attack The CW attack is an elaborate and extremely effective attack model (Carlini & Wagner, 2017).

Original: In many cases, it can reduce the classiﬁer accuracy to almost 0% (Carlini & Wagner, 2017; Meng & Chen, 2017).
Revised: In many cases, it can reduce the classiﬁer accuracy to almost 0% (Carlini & Wagner, 2017; Meng & Chen, 2017).

Original: The perturbation δ is found by solving an optimization problem of the form:
Revised: The perturbation δ is found by solving an optimization problem of the form:

Original: [Equation]
Revised: [Equation]

Original: where f is an objective function that drives the example x to be misclassiﬁed, and c > 0 is a suitably chosen constant.
Revised: where f is an objective function that drives the example x to be misclassiﬁed, and c > 0 is a suitably chosen constant.

Original: The (cid:96) 2 , (cid:96) 0 , and (cid:96) ∞
Revised: The (cid:96) 2 , (cid:96) 0 , and (cid:96) ∞

Original: norms are considered.
Revised: norms are considered.

Original: We refer the reader to (Carlini & Wagner, 2017) for details regarding the approach to solving (4) and setting the constant c .
Revised: We refer the reader to (Carlini & Wagner, 2017) for details regarding the approach to solving (4) and setting the constant c .

Original: 2.1. B LACK - BOX ATTACK MODELS
Revised: 2.1. B LACK - BOX ATTACK MODELS

Original: For black-box attacks we consider untargeted FGSM attacks computed on a substitute model (Papernot et al., 2017).
Revised: For black-box attacks we consider untargeted FGSM attacks computed on a substitute model (Papernot et al., 2017).

Original: As previously mentioned, black-box adversaries have no access to the classiﬁer or defense parameters.
Revised: As previously mentioned, black-box adversaries have no access to the classiﬁer or defense parameters.

Original: It is further assumed that they don’t have access to a large training dataset but can query the targeted DNN as a black-box, i.e., access labels produced by the classiﬁer to speciﬁc query images.
Revised: It is further assumed that they don’t have access to a large training dataset but can query the targeted DNN as a black-box, i.e., access labels produced by the classiﬁer to speciﬁc query images.

Original: The adversary trains a model, called substitute , which has a (potentially) different architecture than the targeted classiﬁer, using a very small dataset augmented by synthetic images labeled by querying the classiﬁer.
Revised: The adversary trains a model, called substitute , which has a (potentially) different architecture than the targeted classiﬁer, using a very small dataset augmented by synthetic images labeled by querying the classiﬁer.

Original: Adversarial examples are then found by applying any attack method on the substitute network.
Revised: Adversarial examples are then found by applying any attack method on the substitute network.

Original: It was found that such examples designed to fool the substitute often end up being misclassiﬁed by the targeted classiﬁer (Szegedy et al., 2014; Papernot et al., 2017).
Revised: It was found that such examples designed to fool the substitute often end up being misclassiﬁed by the targeted classiﬁer (Szegedy et al., 2014; Papernot et al., 2017).

Original: In other words, black-box attacks are easily transferrable from one model to the other.
Revised: In other words, black-box attacks are easily transferrable from one model to the other.

Original: D EFENSE MECHANISMS
Revised: D EFENSE MECHANISMS

Original: Various defense mechanisms have been employed to combat the threat from adversarial attacks.
Revised: Various defense mechanisms have been employed to combat the threat from adversarial attacks.

Original: In what follows, we describe one representative defense strategy from each of the three general groups of defenses.
Revised: In what follows, we describe one representative defense strategy from each of the three general groups of defenses.

Original: 2.2. A DVERSARIAL TRAINING
Revised: 2.2. A DVERSARIAL TRAINING

Original: A popular approach to defend against adversarial noise is to augment the training dataset with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015; Moosavi-Dezfooli et al., 2016).
Revised: A popular approach to defend against adversarial noise is to augment the training dataset with adversarial examples (Szegedy et al., 2014; Goodfellow et al., 2015; Moosavi-Dezfooli et al., 2016).

Original: Adversarial examples are generated using one or more chosen attack models and added to the training set.
Revised: Adversarial examples are generated using one or more chosen attack models and added to the training set.

Original: This often results in increased robustness when the attack model used to generate the augmented training set is the same as that used by the attacker.
Revised: This often results in increased robustness when the attack model used to generate the augmented training set is the same as that used by the attacker.

Original: However, adversarial training does not perform as well when a different attack strategy is used by the attacker.
Revised: However, adversarial training does not perform as well when a different attack strategy is used by the attacker.

Original: Additionally, it tends to make the model more robust to white-box attacks than to black-box attacks due to gradient masking (Papernot et al., 2016c; 2017; Tram`er et al., 2017).
Revised: Additionally, it tends to make the model more robust to white-box attacks than to black-box attacks due to gradient masking (Papernot et al., 2016c; 2017; Tram`er et al., 2017).

Original: 2.2. D EFENSIVE DISTILLATION
Revised: 2.2. D EFENSIVE DISTILLATION

Original: Defensive distillation (Papernot et al., 2016d) trains the classiﬁer in two rounds using a variant of the distillation (Hinton et al., 2014) method.
Revised: Defensive distillation (Papernot et al., 2016d) trains the classiﬁer in two rounds using a variant of the distillation (Hinton et al., 2014) method.

Original: This has the desirable effect of learning a smoother network and reducing the amplitude of gradients around input points, making it difﬁcult for attackers to generate adversarial examples (Papernot et al., 2016d).
Revised: This has the desirable effect of learning a smoother network and reducing the amplitude of gradients around input points, making it difﬁcult for attackers to generate adversarial examples (Papernot et al., 2016d).

Original: It was however shown (Carlini & Wagner, 2017) that, while defensive distillation is effective against white-box attacks, it fails to adequately protect against black-box attacks transferred from other networks.
Revised: It was however shown (Carlini & Wagner, 2017) that, while defensive distillation is effective against white-box attacks, it fails to adequately protect against black-box attacks transferred from other networks.

Original: 2.2. M AG N ET
Revised: 2.2. M AG N ET

Original: Recently, Meng & Chen (2017) introduced MagNet as an effective defense strategy.
Revised: Recently, Meng & Chen (2017) introduced MagNet as an effective defense strategy.

Original: It trains a reformer network (which is an auto-encoder or a collection of auto-encoders) to move adversarial examples closer to the manifold of legitimate, or natural, examples.
Revised: It trains a reformer network (which is an auto-encoder or a collection of auto-encoders) to move adversarial examples closer to the manifold of legitimate, or natural, examples.

Original: When using a collection of auto-encoders, one reformer network is chosen at random at test time, thus strengthening the defense.
Revised: When using a collection of auto-encoders, one reformer network is chosen at random at test time, thus strengthening the defense.

Original: It was shown to be an effective defense against gray-box attacks where the attacker knows everything about the network and defense, except the parameters.
Revised: It was shown to be an effective defense against gray-box attacks where the attacker knows everything about the network and defense, except the parameters.

Original: MagNet is the closest defense to our approach, as it attempts to reform an adversarial sample using a learnt auto-encoder.
Revised: MagNet is the closest defense to our approach, as it attempts to reform an adversarial sample using a learnt auto-encoder.

Original: The main differences between MagNet and our approach are: (1) we use GANs instead of auto-encoders, and, most importantly, (2) we use of GD minimization to ﬁnd latent codes as opposed to a feedforward encoder network.
Revised: The main differences between MagNet and our approach are: (1) we use GANs instead of auto-encoders, and, most importantly, (2) we use of GD minimization to ﬁnd latent codes as opposed to a feedforward encoder network.

Original: This makes Defense-GAN a more robust defense, especially against white-box attacks.
Revised: This makes Defense-GAN a more robust defense, especially against white-box attacks.

Original: 2. G ENERATIVE A DVERSARIAL N ETWORKS (GAN S )
Revised: 2. G ENERATIVE A DVERSARIAL N ETWORKS (GAN S )

Original: GANs, originally introduced by Goodfellow et al.
Revised: GANs, originally introduced by Goodfellow et al.

Original: (2014), consist of two neural networks, G and
Revised: (2014), consist of two neural networks, G and

Original: D .
Revised: D .

Original: G : R k
Revised: G : R k

Original: → R n maps a low-dimensional latent space to the high dimensional sample space of x .
Revised: → R n maps a low-dimensional latent space to the high dimensional sample space of x .

Original: D is a binary neural network classiﬁer.
Revised: D is a binary neural network classiﬁer.

Original: In the training phase, G and D are typically learned in an adversarial fashion using actual input data samples x and random vectors z .
Revised: In the training phase, G and D are typically learned in an adversarial fashion using actual input data samples x and random vectors z .

Original: An isotropic Gaussian prior is usually assumed on z .
Revised: An isotropic Gaussian prior is usually assumed on z .

Original: While G learns to generate outputs G ( z ) that have a distribution similar to that of x , D learns to discriminate between “real” samples x and “fake” samples G ( z ) .
Revised: While G learns to generate outputs G ( z ) that have a distribution similar to that of x , D learns to discriminate between “real” samples x and “fake” samples G ( z ) .

Original: D and G are trained in an alternating fashion to minimize the following min-max loss (Goodfellow et al., 2014):
Revised: D and G are trained in an alternating fashion to minimize the following min-max loss (Goodfellow et al., 2014):

Original: [Equation]
Revised: [Equation]

Original: It was shown that the optimal GAN is obtained when the resulting generator distribution p g = p data (Goodfellow et al., 2014).
Revised: It was shown that the optimal GAN is obtained when the resulting generator distribution p g = p data (Goodfellow et al., 2014).

Original: However, GANs turned out to be difﬁcult to train in practice (Gulrajani et al., 2017), and alternative formulations were proposed.
Revised: However, GANs turned out to be difﬁcult to train in practice (Gulrajani et al., 2017), and alternative formulations have been proposed.

Original: Arjovsky et al.
Revised: Arjovsky et al.

Original: (2017) introduced Wasserstein GANs (WGANs) which are a variant of GANs using the Wasserstein distance, resulting in a loss function with more desirable properties:
Revised: (2017) introduced Wasserstein GANs (WGANs) which are a variant of GANs that use the Wasserstein distance, resulting in a loss function with more desirable properties:

Original: [Equation]
Revised: [Equation]

Original: In this work, we use WGANs as our generative model due to the stability of their training methods, especially using the approach in (Gulrajani et al., 2017).
Revised: In this work, we use WGANs as our generative model due to the stability of their training methods, especially using the approach in (Gulrajani et al., 2017).

Original:  P ROPOSED D EFENSE -GAN
Revised:  P ROPOSED D EFENSE -GAN

Original: We propose a new defense strategy which uses a WGAN trained on legitimate (un-perturbed) training samples to “denoise” adversarial examples.
Revised: We propose a new defense strategy which uses a WGAN trained on legitimate (un-perturbed) training samples to “denoise” adversarial examples.

Original: At test time, prior to feeding an image x to the classiﬁer, we project it onto the range of the generator by minimizing the reconstruction error
Revised: At test time, prior to feeding an image x to the classiﬁer, we project it onto the range of the generator by minimizing the reconstruction error

Original: || G ( z )
Revised: || G ( z )

Original: − x || 22 , using L steps of GD.
Revised: − x || 22 , using L steps of GD.

Original: The resulting reconstruction G ( z ) is then given to the classiﬁer.
Revised: The resulting reconstruction G ( z ) is then given to the classiﬁer.

Original: Since the generator was trained to model the unperturbed training data distribution, we expect this added step to result in a substantial reduction of any potential adversarial noise.
Revised: Since the generator was trained to model the unperturbed training data distribution, we expect this added step to result in a substantial reduction of any potential adversarial noise.

Original: We formally motivate this approach in the following section.
Revised: We formally motivate this approach in the following section.

Original: 3. M OTIVATION
Revised: 3. M OTIVATION

Original: As mentioned in Section 2.3, the GAN min-max loss in (5) admits a global optimum when p g = p data (Goodfellow et al., 2014).
Revised: As mentioned in Section 2.3, the GAN min-max loss in (5) admits a global optimum when p g = p data (Goodfellow et al., 2014).

Original: It can be similarly shown that WGAN admits an optimum to its own minmax loss in (6), when the set { x | p g ( x ) (cid:54) =
Revised: It can be similarly shown that WGAN admits an optimum to its own minmax loss in (6), when the set { x | p g ( x ) (cid:54) =

Original: p data ( x ) } has zero Lebesgue-measure.
Revised: p data ( x ) } has zero Lebesgue-measure.

Original: Formally,
Revised: Formally,

Original: Lemma 1 A generator distribution p g is a global optimum for the WGAN min-max game deﬁned in (6), if and only if p g ( x ) = p data ( x ) for all x ∈ R n , potentially except on a set of zero Lebesguemeasure.
Revised: Lemma 1 A generator distribution p g is a global optimum for the WGAN min-max game deﬁned in (6), if and only if p g ( x ) = p data ( x ) for all x ∈ R n , potentially except on a set of zero Lebesguemeasure.

Original: A sketch of the proof can be found in Appendix A.
Revised: A sketch of the proof can be found in Appendix A.

Original: Additionally, it was shown that, if G and D have enough capacity to represent the data, and if the training algorithm is such that p g converges to p data , then
Revised: Additionally, it was shown that, if G and D have enough capacity to represent the data, and if the training algorithm is such that p g converges to p data , then

Original: [Equation]
Revised: [Equation]

Original: where G t is the generator of a GAN or WGAN 1 after t steps of its training algorithm (Anonymous
Revised: where G t is the generator of a GAN or WGAN 1 after t steps of its training algorithm (Anonymous

Original: Authors, 2017).
Revised: Authors, 2017).

Original: This serves to show that, under ideal conditions, the addition of the GAN reconstruction loss minimization step should not affect the performance of the classiﬁer on natural, legitimate samples, as such samples should be almost exactly recovered.
Revised: This serves to show that, under ideal conditions, the addition of the GAN reconstruction loss minimization step should not affect the performance of the classiﬁer on natural, legitimate samples, as such samples should be almost exactly recovered.

Original: Furthermore, we hypothesize that this step will help reduce the adversarial noise which follows a different distribution than that of the GAN training examples.
Revised: Furthermore, we hypothesize that this step will help reduce the adversarial noise which follows a different distribution than that of the GAN training examples.

Original: 3. D EFENSE -GAN ALGORITHM
Revised: 3. D EFENSE -GAN ALGORITHM

Original: Defense-GAN is a defense strategy to combat both white-box and black-box adversarial attacks against classiﬁcation networks.
Revised: Defense-GAN is a defense strategy to combat both white-box and black-box adversarial attacks against classiﬁcation networks.

Original: At inference time, given a trained GAN generator G and an image x to be classiﬁed, z ∗ is ﬁrst found so as to minimize
Revised: At inference time, given a trained GAN generator G and an image x to be classiﬁed, z ∗ is ﬁrst found so as to minimize

Original: [Equation]
Revised: [Equation]

Original: G ( z ∗ ) is then given as the input to the classiﬁer.
Revised: G ( z ∗ ) is then given as the input to the classiﬁer.

Original: The algorithm is illustrated in Figure 1.
Revised: The algorithm is illustrated in Figure 1.

Original: As (8) is a highly non-convex minimization problem, we approximate it by doing a ﬁxed number L of GD steps using R different random initializations of z (which we call random restarts), as shown in Figuresand 2.
Revised: As (8) is a highly non-convex minimization problem, we approximate it by doing a ﬁxed number L of GD steps using R different random initializations of z (which we call random restarts), as shown in Figuresand 2.

Original: The GAN is trained on the available classiﬁer training dataset in an unsupervised manner.
Revised: The GAN is trained on the available classiﬁer training dataset in an unsupervised manner.

Original: The classiﬁer can be trained on the original training images ( x ), their reconstructions using the generator ( G ( z ∗ ) ), or a combination of the two.
Revised: The classiﬁer can be trained on the original training images ( x ), their reconstructions using the generator ( G ( z ∗ ) ), or a combination of the two.

Original: As was discussed in Section 3.1, as long as the GAN was appropriately trained and has enough capacity to represent the data, original clean images and their reconstructions should not defer much.
Revised: As was discussed in Section 3.1, as long as the GAN was appropriately trained and has enough capacity to represent the data, original clean images and their reconstructions should not defer much.

Original: Therefore, these two classiﬁer training strategies should, at least theoretically, not affect performance.
Revised: Therefore, these two classiﬁer training strategies should, at least theoretically, not affect performance.

Original: Compared to existing defense mechanisms, our approach distinguishes itself in the following aspects:
Revised: Compared to existing defense mechanisms, our approach distinguishes itself in the following aspects:

Original: Defense-GAN can be used in conjunction with any classiﬁer and does not modify the classiﬁer structure itself.
Revised: Defense-GAN can be used in conjunction with any classiﬁer and does not modify the classiﬁer structure itself.

Original: It can be seen as an add-on or pre-processing step prior to classiﬁcation.
Revised: It can be seen as an add-on or pre-processing step prior to classiﬁcation.

Original: If the GAN is representative enough, re-training the classiﬁer should not be necessary and any drop in performance due to the addition of Defense-GAN should not be signiﬁcant.
Revised: If the GAN is representative enough, re-training the classiﬁer should not be necessary and any drop in performance due to the addition of Defense-GAN should not be signiﬁcant.

Original: Defense-GAN can be used as a defense to any attack: it does not assume an attack model, but simply leverages the generative power of GANs to reconstruct adversarial examples.
Revised: Defense-GAN can be used as a defense to any attack: it does not assume an attack model, but simply leverages the generative power of GANs to reconstruct adversarial examples.

Original: Defense-GAN is highly non-linear and white-box gradient-based attacks will be difﬁcult to perform due to the GD loop.
Revised: Defense-GAN is highly non-linear and white-box gradient-based attacks will be difﬁcult to perform due to the GD loop.

Original: A detailed discussion about this can be found in Appendix B.
Revised: A detailed discussion about this can be found in Appendix B.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We assume three different attack threat levels:
Revised: We assume three different attack threat levels:

Original: Black-box attacks: the attacker does not have access to the details of the classiﬁer and defense strategy.
Revised: Black-box attacks: the attacker does not have access to the details of the classiﬁer and defense strategy.

Original: It therefore trains a substitute network to ﬁnd adversarial examples.
Revised: It therefore trains a substitute network to ﬁnd adversarial examples.

Original: White-box attacks
Revised: White-box attacks

Original: : the attacker knows all the details of the classiﬁer and defense strategy.
Revised: : the attacker knows all the details of the classiﬁer and defense strategy.

Original: It can compute gradients on the classiﬁer and defense networks in order to ﬁnd adversarial examples.
Revised: It can compute gradients on the classiﬁer and defense networks in order to ﬁnd adversarial examples.

Original: White-box attacks, revisited: in addition to the details of the architectures and parameters of the classiﬁer and defense, the attacker has access to the random seed and random number generator.
Revised: White-box attacks, revisited: in addition to the details of the architectures and parameters of the classiﬁer and defense, the attacker has access to the random seed and random number generator.

Original: In the case of Defense-GAN, this means that the attacker knows all the random initializations { z ( i ) 

Revised: In the case of Defense-GAN, this means that the attacker knows all the random initializations { z ( i )

Original: } Ri =1 .
Revised: } Ri =1 .

Original: We compare our method to adversarial training (Goodfellow et al., 2015) and MagNet (Meng &
Revised: We compare our method to adversarial training (Goodfellow et al., 2015) and MagNet (Meng &

Original: Chen, 2017) under the FGSM, RAND+FGSM, and CW (with (cid:96) 2 norm) white-box attacks, as well as the FGSM black-box attack.
Revised: Chen, 2017) under the FGSM, RAND+FGSM, and CW (with (cid:96) 2 norm) white-box attacks, as well as the FGSM black-box attack.

Original: Details of all network architectures used in this paper can be found in Appendix C. When the classiﬁer is trained using the reconstructed images ( G ( z ∗ ) ), we refer to our method as Defense-GAN-Rec, and we use Defense-GAN-Orig when the original images ( x ) are used to train the classiﬁer.
Revised: Details of all network architectures used in this paper can be found in Appendix C. When the classiﬁer is trained using the reconstructed images ( G ( z ∗ ) ), we refer to our method as Defense-GAN-Rec, and we use Defense-GAN-Orig when the original images ( x ) are used to train the classiﬁer.

Original: Our GAN follows the WGAN training procedure in (Gulrajani et al., 2017), and details of the generator and discriminator network architectures are given in Table 6.
Revised: Our GAN follows the WGAN training procedure in (Gulrajani et al., 2017), and details of the generator and discriminator network architectures are given in Table 6.

Original: The reformer network (encoder) for the MagNet baseline is provided in Table 7.
Revised: The reformer network (encoder) for the MagNet baseline is provided in Table 7.

Original: Our implementation is based on TensorFlow (Abadi et al., 2015) and builds on open-source software CleverHans by
Revised: Our implementation is based on TensorFlow (Abadi et al., 2015) and builds on open-source software CleverHans by

Original: Papernot et al.
Revised: Papernot et al.

Original: (2016a)
Revised: (2016a)

Original: and improved WGAN training by Gulrajani et al.
Revised: and improved WGAN training by Gulrajani et al.

Original: We use machines equipped with NVIDIA GeForce GTX TITAN X GPUs.
Revised: We use machines equipped with NVIDIA GeForce GTX TITAN X GPUs.

Original: In our experiments, we use two different image datasets: the MNIST handwritten digits dataset (LeCun et al., 1998) and the Fashion-MNIST (F-MNIST) clothing articles dataset (Xiao et al., 2017).
Revised: In our experiments, we use two different image datasets: the MNIST handwritten digits dataset (LeCun et al., 1998) and the Fashion-MNIST (F-MNIST) clothing articles dataset (Xiao et al., 2017).

Original: Both datasets consist of 60 , 000 training images and 10 , 000 testing images.
Revised: Both datasets consist of 60 , 000 training images and 10 , 000 testing images.

Original: We split the training images into a training set of 50 , 000 images and hold-out a validation set containing 10 , 000 images.
Revised: We split the training images into a training set of 50 , 000 images and hold-out a validation set containing 10 , 000 images.

Original: For white-box attacks, the testing set is kept the same ( 10 , 000 samples).
Revised: For white-box attacks, the testing set is kept the same ( 10 , 000 samples).

Original: For black-box attacks, the testing set is divided into a small hold-out set of 150 samples reserved for adversary substitute training, as was done in (Papernot et al., 2017), and the remaining 9 , 850 samples are used for testing the different methods.
Revised: For black-box attacks, the testing set is divided into a small hold-out set of 150 samples reserved for adversary substitute training, as was done in (Papernot et al., 2017), and the remaining 9 , 850 samples are used for testing the different methods.

Original: 4. R ESULTS ON BLACK - BOX ATTACKS
Revised: 4. R ESULTS ON BLACK - BOX ATTACKS

Original: In this section, we present experimental results on FGSM black-box attacks.
Revised: In this section, we present experimental results on FGSM black-box attacks.

Original: As previously mentioned, the attacker trains a substitute model, which could differ in architecture from the targeted model, using a limited dataset consisting of 150 legitimate images augmented with synthetic images labeled using the target classiﬁer.
Revised: As previously mentioned, the attacker trains a substitute model, which could differ in architecture from the targeted model, using a limited dataset consisting of 150 legitimate images augmented with synthetic images labeled using the target classiﬁer.

Original: Unless otherwise speciﬁed, we use Model F as the classiﬁer andModel E as the black-box substitute network, as described in Table 5.
Revised: The classiﬁer and substitute model architectures used and referred to throughout this section are described in Table 5 in the Appendix.

Original: In Tables 1 and 2, we present our classiﬁcation accuracy results and compare to other defense methods.
Revised: In Tables 1 and 2, we present our classiﬁcation accuracy results and compare to other defense methods.

Original: As can be seen, the FGSM black-box attacks were successful at reducing the classiﬁer accuracy by up to 70% .
Revised: As can be seen, the FGSM black-box attacks were successful at reducing the classiﬁer accuracy by up to 70% .

Original: All considered defense mechanisms are relatively successful at diminishing the effect of the attacks.
Revised: All considered defense mechanisms are relatively successful at diminishing the effect of the attacks.

Original: We note that, as expected, the performance of Defense-GAN-Rec and that of
Revised: We note that, as expected, the performance of Defense-GAN-Rec and that of

Original: Defense-GAN-Orig are very close.
Revised: Defense-GAN-Orig are very close.

Original: In addition, they both perform consistently well across different classiﬁer and substitute model combinations.
Revised: In addition, they both perform consistently well across different classiﬁer and substitute model combinations.

Original: MagNet also performs in a consistent manner, but achieves lower accuracy than Defense-GAN.
Revised: MagNet also performs in a consistent manner, but achieves lower accuracy than Defense-GAN.

Original: Two adversarial training defenses are presented: the ﬁrst one obtains the adversarial examples assuming the same attack (cid:15) = 0 . 3 , and the second assumes a different (cid:15)
Revised: Two adversarial training defenses are presented: the ﬁrst one obtains the adversarial examples assuming the same attack (cid:15) = 0 . 3 , and the second assumes a different (cid:15)

Original: = 0 .
Revised: = 0 .

Original: With incorrect knowledge of (cid:15) , the performance of adversarial training generally decreases.
Revised: With incorrect knowledge of (cid:15) , the performance of adversarial training generally decreases.

Original: In addition, the classiﬁcation performance of this defense method has very large variance across the different architectures.
Revised: In addition, the classiﬁcation performance of this defense method has very large variance across the different architectures.

Original: It is worth nothing that adversarial training defense is only ﬁt against FGSM attacks, because the adversarially augmented data, even with a different (cid:15) , is generated using the same method as the black-box attack (FGSM).
Revised: It is worth nothing that adversarial training defense is only ﬁt against FGSM attacks, because the adversarially augmented data, even with a different (cid:15) , is generated using the same method as the black-box attack (FGSM).

Original: In contrast, Defense-GAN and
Revised: In contrast, Defense-GAN and

Original: MagNet are general defense mechanisms and do not assume a speciﬁc attack model.
Revised: MagNet are general defense mechanisms and do not assume a speciﬁc attack model.

Original: The performances of defenses on the F-MNIST dataset, shown in Table 2, are noticeably lower than on MNIST.
Revised: The performances of defenses on the F-MNIST dataset, shown in Table 2, are noticeably lower than on MNIST.

Original: This is due to the large (cid:15) = 0 .
Revised: This is due to the large (cid:15) = 0 .

Original: 3 in the FGSM attack.
Revised: 3 in the FGSM attack.

Original: Please see Appendix D for qualitative examples showing that (cid:15) = 0 .
Revised: Please see Appendix D for qualitative examples showing that (cid:15) = 0 .

Original: 3 represents very high noise, which makes F-MNIST images difﬁcult to classify, even by a human.
Revised: 3 represents very high noise, which makes F-MNIST images difﬁcult to classify, even by a human.

Original: In addition, the Defense-GAN parameters used in this experiment were kept the same for both
Revised: In addition, the Defense-GAN parameters used in this experiment were kept the same for both

Original: Tables, in order to study the effect of dataset complexity, and can be further optimized as investigated in the next section. 
Revised: Tables, in order to study the effect of dataset complexity, and can be further optimized as investigated in the next section. [Table]

Original: [Table]
Revised: [Table]

Revised: black-box attacks with

Revised: Defense-GAN has L = 200 and R = 10 .

Original: Classiﬁer/
Revised: Classiﬁer/

Original: Substitute
Revised: Substitute

Original: NoAttack
Revised: NoAttack

Original: No
Revised: No

Original: Defense
Revised: Defense

Original: DefenseGAN-Rec DefenseGAN-Orig
Revised: DefenseGAN-Rec DefenseGAN-Orig

Original: MagNet
Revised: MagNet

Original: Adv.
Revised: Adv.

Original: Tr.
Revised: Tr.

Original: (cid:15)
Revised: (cid:15)

Original: = 0 .
Revised: = 0 .

Original: Adv.
Revised: Adv.

Original: Tr.
Revised: Tr.

Original: (cid:15)
Revised: (cid:15)

Original: = 0 .
Revised: = 0 .

Original: A/B 0 .0 .0 .0 .0 .0 .0 .
Revised: A/B 0 .0 .0 .0 .0 .0 .0 .

Original: A/E 0 .0 .0 .0 .0 .0 .0 .
Revised: A/E 0 .0 .0 .0 .0 .0 .0 .

Original: B/B 0 .0 .0 .0 .0 .

Revised: B/B 0 .0 .0 .0 .0 .

Original: B/E 0 .0 .

Revised: B/E 0 .0 .

Original: C/B 0 .0 .0 .0 .0 .

Revised: C/B 0 .0 .0 .0 .0 .

Original: C/E 0 .0 .0 .

Revised: C/E 0 .0 .0 .

Original: D/B 0 .0 .0 .0 .0 .0 .0 .
Revised: D/B 0 .0 .0 .0 .0 .0 .0 .

Original: D/E 0 .0 .0 .0 .0 .0 .0 .
Revised: D/E 0 .0 .0 .0 .0 .0 .0 .

Original: 4.1. E FFECT OF NUMBER OF GD ITERATIONS L AND RANDOM RESTARTS R
Revised: 4.1. E FFECT OF NUMBER OF GD ITERATIONS L AND RANDOM RESTARTS R

Original: Figure 3 shows the effect of varying the number of GD iterations L as well as the random restarts
Revised: Figure 3 shows the effect of varying the number of GD iterations L as well as the random restarts

Original: R used to compute the GAN reconstructions of input images.
Revised: R used to compute the GAN reconstructions of input images.

Original: Across different L and R values,
Revised: Across different L and R values,

Original: Defense-GAN-Rec and Defense-GAN-Orig have comparable performance.
Revised: Defense-GAN-Rec and Defense-GAN-Orig have comparable performance.

Original: Increasing L has the expected effect of improving performance when no attack is present.
Revised: Increasing L has the expected effect of improving performance when no attack is present.

Original: Interestingly, with an FGSM attack, the classiﬁcation performance decreases after a certain L value.
Revised: Interestingly, with an FGSM attack, the classiﬁcation performance decreases after a certain L value.

Original: With too many GD iterations on the mean squared error (MSE)
Revised: With too many GD iterations on the mean squared error (MSE)

Original: ||
Revised: ||

Original: G ( z ) − ( x + δ ) || 22 , some of the adversarial noise components are retained.
Revised: G ( z ) − ( x + δ ) || 22 , some of the adversarial noise components are retained.

Original: In the right Figure, the effect of varying R is shown to be extremely pronounced.
Revised: In the right Figure, the effect of varying R is shown to be extremely pronounced.

Original: This is due to the non-convex nature of the MSE
Revised: This is due to the non-convex nature of the MSE

Original: [Figure]
Revised: [Figure]

Original: and increasing R enables us to sample different local minima
Revised: and increasing R enables us to sample different local minima

Original: [Figure]
Revised: [Figure]

Original:  ( L = 100 ).
Revised: R are used ( L = 100 ).

Original: 4.1. E FFECT OF ADVERSARIAL NOISE NORM (cid:15)
Revised: 4.1. E FFECT OF ADVERSARIAL NOISE NORM (cid:15)

Original: We now investigate the effect of changing the attack (cid:15) in Table 3.
Revised: We now investigate the effect of changing the attack (cid:15) in Table 3.

Original: As expected, with higher (cid:15) , the
Revised: As expected, with higher (cid:15) , the

Original: FGSM attack is more successful, especially on the F-MNIST dataset where the noise norm seems to have a more pronounced effect with nearly 37% drop in performance between (cid:15) = 0 .
Revised: FGSM attack is more successful, especially on the F-MNIST dataset where the noise norm seems to have a more pronounced effect with nearly 37% drop in performance between (cid:15) = 0 .

Original: 1 and 0 . 3 .
Revised: 1 and 0 . 3 .

Original: Figure 7 in Appendix D shows adversarial samples as well as their reconstructions with DefenseGAN at different values of (cid:15) .
Revised: Figure 7 in Appendix D shows adversarial samples as well as their reconstructions with DefenseGAN at different values of (cid:15) .

Original: We can see that for large (cid:15) , the class is difﬁcult to discern, even for the human eye.
Revised: We can see that for large (cid:15) , the class is difﬁcult to discern, even for the human eye.

Original: Even though it seems that increasing (cid:15) is a desirable strategy for the attacker, this increases the likelihood that the adversarial noise is discernible and therefore the attack is detected.
Revised: Even though it seems that increasing (cid:15) is a desirable strategy for the attacker, this increases the likelihood that the adversarial noise is discernible and therefore the attack is detected.

Original: It is trivial for the attacker to provide adversarial images at very high (cid:15) [Table] and a good measure of an attack’s strength is its ability to affect performance at low (cid:15) .
Revised: It is trivial for the attacker to provide adversarial images at very high (cid:15) , and a good measure of an attack’s strength is its ability to affect performance at low (cid:15) .

Original: In fact, in the next section [Table] we discuss how Defense-GAN can be used to not only diminish the effect of attacks, but also to detect them.
Revised: In fact, in the next section, we discuss how Defense-GAN can be used to not only diminish the effect of attacks, but also to detect them.

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 4.1. A TTACK DETECTION
Revised: . A TTACK DETECTION

Original: We intuitively expect that clean, unperturbed images will lie closer to the range of the Defense-GAN generator G than adversarial examples.
Revised: We intuitively expect that clean, unperturbed images will lie closer to the range of the Defense-GAN generator G than adversarial examples.

Original: This is due to the fact that G was trained to produce images which resemble the legitimate data.
Revised: This is due to the fact that G was trained to produce images which resemble the legitimate data.

Original: In light of this observation, we propose to use the MSE of an image with it is reconstruction from (8) as a “metric” to decide whether or not the image was adversarially manipulated.
Revised: In light of this observation, we propose to use the MSE of an image with it is reconstruction from (8) as a “metric” to decide whether or not the image was adversarially manipulated.

Original: In order words, for a given threshold θ > 0 , the hypothesis test is:
Revised: In order words, for a given threshold θ > 0 , the hypothesis test is:

Original: [Equation]
Revised: [Equation]

Original: We compute the reconstruction MSEs for every image from the test dataset, and its adversarially manipulated version using FGSM.
Revised: We compute the reconstruction MSEs for every image from the test dataset, and its adversarially manipulated version using FGSM.

Original: We show the Receiver Operating Characteristic (ROC) curves as well as the Area Under the Curve (AUC) metric for different Defense-GAN parameters and (cid:15) values in Figures 4 and 5.
Revised: We show the Receiver Operating Characteristic (ROC) curves as well as the Area Under the Curve (AUC) metric for different Defense-GAN parameters and (cid:15) values in Figures 4 and 5.

Original: The results show that this attack detection strategy is effective especially when the number of GD iterations L and random restarts R are large.
Revised: The results show that this attack detection strategy is effective especially when the number of GD iterations L and random restarts R are large.

Original: From the left and middle Figures, we can conclude that the number of random restarts plays a very important role in the detection false positive and true positive rates as was discussed in Section 4.1.1.
Revised: From the left and middle Figures, we can conclude that the number of random restarts plays a very important role in the detection false positive and true positive rates as was discussed in Section 4.1.1.

Original: Furthermore, when (cid:15) is very small, it becomes difﬁcult to detect attacks at low false positive rates.
Revised: Furthermore, when (cid:15) is very small, it becomes difﬁcult to detect attacks at low false positive rates.

Original: 4.1. R ESULTS ON WHITE - BOX ATTACKS
Revised: 4.1. R ESULTS ON WHITE - BOX ATTACKS

Original: We now present results on white-box attacks using three different strategies: FGSM, RAND+FGSM, and CW.
Revised: We now present results on white-box attacks using three different strategies: FGSM, RAND+FGSM, and CW.

Original: Table 4 shows the classiﬁcation performance of different classiﬁer models across different attack and defense strategies.
Revised: Table 4 shows the classiﬁcation performance of different classiﬁer models across different attack and defense strategies.

Original: We note that Defense-GAN signiﬁcantly outperforms the two other baseline defenses.
Revised: We note that Defense-GAN signiﬁcantly outperforms the two other baseline defenses.

Original: We even give the adversarial attacker access to the random initializations of z .
Revised: We even give the adversarial attacker access to the random initializations of z .

Original: However, we noticed that the performance does not change much when the attacker does not know the initialization.
Revised: However, we noticed that the performance does not change much when the attacker does not know the initialization.

Original: Adversarial training was done using FGSM to generate the adversarial samples.
Revised: Adversarial training was done using FGSM to generate the adversarial samples.

Original: It is interesting to mention that when CW attack is used, adversarial training performs extremely poorly.
Revised: It is interesting to mention that when CW attack is used, adversarial training performs extremely poorly.

Original: As previously discussed, adversarial training does not generalize well against different attack methods.
Revised: As previously discussed, adversarial training does not generalize well against different attack methods.

Original: Due to the loop of L steps of GD, Defense-GAN is resilient to  white-box attacks, since the attacker needs to “un-roll” the GD loop and propagate the gradient of the loss all the way across L steps.
Revised: Due to the loop of L steps of GD, Defense-GAN is resilient to GD-based white-box attacks, since the attacker needs to “un-roll” the GD loop and propagate the gradient of the loss all the way across L steps.

Original: In fact, from Table 4, the performance of classiﬁer A with Defense-GAN on the MNIST dataset drops less than 1% from 0 .
Revised: In fact, from Table 4, the performance of classiﬁer A with Defense-GAN on the MNIST dataset drops less than 1% from 0 .

Original: 997 to 0 .
Revised: 997 to 0 .

Original: 988 under FGSM.
Revised: 988 under FGSM.

Original: In comparison, from Figure 8, when L = 25 , the performance of the same network drops to 0 .
Revised: In comparison, from Figure 8, when L = 25 , the performance of the same network drops to 0 .

Original: 947 (more than 5% drop).
Revised: 947 (more than 5% drop).

Original: This shows that using a larger L signiﬁcantly increases the robustness of Defense-GAN against white-box attacks.
Revised: This shows that using a larger L signiﬁcantly increases the robustness of Defense-GAN against GD-based whitebox attacks.

Original: This comes at an expense of increased inference time complexity.
Revised: This comes at the expense of increased inference time complexity.

Original: We present a more detailed discussion about the difﬁculty of  white-box attacks in Appendix B and time complexity in Appendix  F.
Revised: We present a more detailed discussion about the difﬁculty of GD-based white-box attacks in Appendix B and time complexity in Appendix G. Additional white-box experimental results are reported in Appendix F.

Original: [Table]
Revised: [Table]

Revised: D 0 .0 .0 .0 .0 .

Original: A 0 .0 .0 .0 .0 .
Revised: A 0 .0 .0 .0 .0 .

Original: RAND+FGSM
Revised: RAND+FGSM

Original: B 0 .0 .0 .0 .0 .(cid:15)
Revised: B 0 .0 .0 .0 .0 .(cid:15)

Original: = 0 .
Revised: = 0 .

Original: 3 , α = 0 .
Revised: 3 , α = 0 .

Original: C 0 .0 .0 .0 .0 .
Revised: C 0 .0 .0 .0 .0 .

Original: D 0 .0 .0 .0 .0 .
Revised: D 0 .0 .0 .0 .0 .

Original: A 0 .0 .0 .0 .0 .
Revised: A 0 .0 .0 .0 .0 .

Original: CW
Revised: CW

Original: B 0 .0 .0 .0 .

Revised: B 0 .0 .0 .0 .

Original: .(cid:96)
Revised: .(cid:96)

Revised: [Table]

Original: 2 norm
Revised: norm

Original: C 0 .0 .0 .0 .0 .
Revised: C 0 .0 .0 .0 .0 .

Original: D 0 .0 .0 .0 .0 .
Revised: D 0 .0 .0 .0 .0 .

Original: Attack
Revised: Attack

Original: Classiﬁer
Revised: Classiﬁer

Original: Model
Revised: Model

Original: No
Revised: No

Original: Attack
Revised: Attack

Original: No
Revised: No

Original: Defense
Revised: Defense

Original: DefenseGAN-Rec
Revised: DefenseGAN-Rec

Original: MagNet
Revised: MagNet

Original: Adv.
Revised: Adv.

Original: Tr.
Revised: Tr.

Original: (cid:15)
Revised: (cid:15)

Original: = 0 .
Revised: = 0 .

Original: A 0 .0 .0 .0 .0 .
Revised: A 0 .0 .0 .0 .0 .

Original: FGSM
Revised: FGSM

Original: B 0 .0 .0 .0 .0 .(cid:15)
Revised: B 0 .0 .0 .0 .0 .(cid:15)

Original: = 0 .
Revised: = 0 .

Original: C 0 .0 .0 .0 .0 .
Revised: C 0 .0 .0 .0 .0 .

Original: D 0 .0 .0 .0 .0 .
Revised: D 0 .0 .0 .0 .0 .

Original: A 0 .0 .0 .0 .0 .
Revised: A 0 .0 .0 .0 .0 .

Original: RAND+FGSM
Revised: RAND+ [Table]

Original: B 0 .0 .0 .0 .0 .(cid:15)
Revised: B 0 .0 .0 .0 .0 .(cid:15)

Original: = 0 .
Revised: [Table]

Original: 3 , α = 0 . C 0 .0 .0 .0 .0 .
Revised: [Table]

Original: D 0 .0 .0 .
Revised: C 0 .0 .0 .

Original: [Table]
Revised: [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we proposed Defense-GAN, a novel defense strategy utilizing GANs to enhance the robustness of classiﬁcation models against black-box and white-box adversarial attacks.
Revised: In this paper, we proposed Defense-GAN, a novel defense strategy utilizing GANs to enhance the robustness of classiﬁcation models against black-box and white-box adversarial attacks.

Original: Our method does not assume a particular attack model and was shown to be effective against most commonly considered attack strategies.
Revised: Our method does not assume a particular attack model and was shown to be effective against most commonly considered attack strategies.

Original: We empirically show that Defense-GAN consistently provides adequate defense on two benchmark computer vision datasets, whereas other methods had many shortcomings on at least one type of attack.
Revised: We empirically show that Defense-GAN consistently provides adequate defense on two benchmark computer vision datasets, whereas other methods had many shortcomings on at least one type of attack.

Original: Appendices A O PTIMALITY OF p g = p DATA FOR WGAN S
Revised: Appendices A O PTIMALITY OF p g = p DATA FOR WGAN S

Original: Sketch of proof of Lemma 1: The WGAN min-max loss is given by:
Revised: Sketch of proof of Lemma 1: The WGAN min-max loss is given by:

Original: [Equation]
Revised: [Equation]

Original: For a ﬁxed G , the optimal discriminator D which maximizes V W ( D, G ) is such that:
Revised: For a ﬁxed G , the optimal discriminator D which maximizes V W ( D, G ) is such that:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Plugging D ∗ G back into (12), we get:
Revised: Plugging D ∗ G back into (12), we get:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Let X =
Revised: Let X =

Original: { x | p data ( x ) ≥ p g ( x ) } .
Revised: { x | p data ( x ) ≥ p g ( x ) } .

Original: Clearly, to minimize (15), we need to set p data ( x ) =
Revised: Clearly, to minimize (15), we need to set p data ( x ) =

Original: p g ( x ) for x ∈ X .
Revised: p g ( x ) for x ∈ X .

Original: Then, since both pdfs should integrate to 1,
Revised: Then, since both pdfs should integrate to 1,

Original: [Equation]
Revised: [Equation]

Original: However, this is a contradiction since p g ( x ) <
Revised: However, this is a contradiction since p g ( x ) <

Original: p data ( x ) for x ∈ X c , unless µ ( X c ) = 0 where µ is the Lebesgue measure.
Revised: p data ( x ) for x ∈ X c , unless µ ( X c ) = 0 where µ is the Lebesgue measure.

Original: This concludes the proof.
Revised: This concludes the proof.

Original: B D IFFICULTY OF  WHITE - BOX ATTACKS ON D EFENSE -GAN
Revised: B D IFFICULTY OF GD- BASED WHITE - BOX ATTACKS ON D EFENSE -GAN

Original: In order to perform a  white-box attack on models using Defense-GAN, an attacker needs to compute the gradient of the output of the classiﬁer with respect to the input.
Revised: In order to perform a GD-based white-box attack on models using Defense-GAN, an attacker needs to compute the gradient of the output of the classiﬁer with respect to the input.

Original: From Figure 1, the generator and the classiﬁer can be seen as one, combined, feedforward network,  which  is easy to take gradients of.
Revised: From Figure 1, the generator and the classiﬁer can be seen as one, combined, feedforward network, through which it is easy to propagate gradients.

Original: The difﬁculty lies in the orange box of the GD optimization detailed in Figure 2.
Revised: The difﬁculty lies in the orange box of the GD optimization detailed in Figure 2.

Original: For the sake of simplicity, let’s assume that R = 1 .
Revised: For the sake of simplicity, let’s assume that R = 1 .

Original: Deﬁne L ( x , z ) =
Revised: Deﬁne L ( x , z ) =

Original: || G ( z ) − x || 22 .
Revised: || G ( z ) − x || 2 2 .

Original: Then z ∗ =
Revised: Then z ∗ =

Original: z L , which is computed recursively as follows:
Revised: z L , which is computed recursively as follows:

Original: [Equation]
Revised: [Equation]

Original: and so on.
Revised: and so on.

Original: Therefore, computing the gradient of z ∗ with respect to x involves a large number ( L ) of recursive chain rules and high-dimensional Jacobian tensors.
Revised: Therefore, computing the gradient of z ∗ with respect to x involves a large number ( L ) of recursive chain rules and high-dimensional Jacobian tensors.

Original: This computation gets increasingly prohibitive for large L .
Revised: This computation gets increasingly prohibitive for large L .

Original: C N EURAL NETWORK ARCHITECTURES
Revised: C N EURAL NETWORK ARCHITECTURES

Original: We describe the neural network architectures used throughout the paper.
Revised: We describe the neural network architectures used throughout the paper.

Original: The detail ofthe classiﬁer models A through F used for the MNIST and F-MNIST datasets can be found in Table 5.
Revised: The detail of models A through F used for classiﬁer and substitute networks can be found in Table 5.

Original: In Table 6, the GAN architectures are described, and in Table 7, the encoder architecture for the MagNet baseline is given.
Revised: In Table 6, the GAN architectures are described, and in Table 7, the encoder architecture for the MagNet baseline is given.

Original: In what follows:
Revised: In what follows:

Original: • Conv( m , k × k , s ) refers to a convolutional layer with m feature maps, ﬁlter size k × k , and stride s • ConvT( m , k × k ) refers to the transpose (gradient) of Conv (sometimes referred to as “deconvolution”) with m feature maps, ﬁlter size k × k , and stride s • FC( m ) refers to a fully-connected layer with m outputs • Dropout( p ) refers to a dropout layer with probability p • ReLU refers to the Rectiﬁed Linear Unit activation • LeakyReLU
Revised: • Conv( m , k × k , s ) refers to a convolutional layer with m feature maps, ﬁlter size k × k , and stride s • ConvT( m , k × k ) refers to the transpose (gradient) of Conv (sometimes referred to as “deconvolution”) with m feature maps, ﬁlter size k × k , and stride s • FC( m ) refers to a fully-connected layer with m outputs • Dropout( p ) refers to a dropout layer with probability p • ReLU refers to the Rectiﬁed Linear Unit activation • LeakyReLU

Original: ( α ) is the leaky version of the Rectiﬁed Linear Unit with parameter α
Revised: ( α ) is the leaky version of the Rectiﬁed Linear Unit with parameter α

Original: [Table]

Original: [Table]

Original: × 5 , 2 )

Original: ReLU

Original: LeakyReLU( 0 . 2 )

Original: ConvT( 128 ,

Original: [Table]
Revised: [Table]

Original: Conv( 128 ,

Original: [Table]
Revised: [Table]

Original: ReLU

Original: LeakyReLU( 0 . 2 )

Original: ConvT( 64 , 5 × 5 , 1 ) Conv( 256 , 5 × 5 , 2 ) ReLU LeakyReLU( 0 . 2 )
Revised: [Table]

Original: ConvT( 1 , 5 × 5 , 1 )

Original: FC( 1 )

Original: [Table]

Original: [Table]
Revised: [Table]

Original: D Q UALITATIVE EXAMPLES
Revised: D Q UALITATIVE EXAMPLES

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: E A DDITIONAL RESULTS ON THE EFFECT OF VARYING THE NUMBER OF GD ITERATIONS L AND RANDOM RESTARTS R
Revised: E A DDITIONAL RESULTS ON THE EFFECT OF VARYING THE NUMBER OF GD ITERATIONS L AND RANDOM RESTARTS R

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: F T IME COMPLEXITY
Revised: F A DDITIONAL RESULTS ON WHITE - BOX ATTACKS

Revised: We report results on white-box attacks on the CelebFaces Attributes dataset (CelebA) (Liu et al., 2015) in Table 12. The CelebA dataset is a large-scale face dataset consisting of more than 200 ,face images, split into training, validation, and testing sets. The RGB images were center-cropped and resized to 64 × 64 .

Revised: We performed the task of gender classiﬁcation on this dataset.

Revised: The GAN architecture is the same as that in Table 6, except for an additional ConvT layer in the generator network.

Revised: [Table]

Original: The computational complexity of reconstructing an image using Defense-GAN is on the order of the number of GD iterations performed to estimate z ∗ , multiplied by the time to compute gradients.
Revised: The computational complexity of reconstructing an image using Defense-GAN is on the order of the number of GD iterations performed to estimate z ∗ , multiplied by the time to compute gradients.

Original: The number of random restarts R has less effect on the running time, since they are independent and can run in parallel, given enough resources.
Revised: The number of random restarts R has less effect on the running time, since random restarts are independent and can run in parallel if enough resources are available.

Original: Table 12 shows the average running time, in seconds, to ﬁnd the reconstructions  on one NVIDIA GeForce GTX TITAN X GPU.
Revised: Table 13 shows the average running time, in seconds, to ﬁnd the reconstructions of MNIST and F-MNIST images on one NVIDIA GeForce GTX TITAN X GPU.

Original: For most applications, these running times are not prohibitive.
Revised: For most applications, these running times are not prohibitive.

Original: We can see a tradeoff between running time and defense robustness as well as accuracy.
Revised: We can see a tradeoff between running time and defense robustness as well as accuracy.

Original: [Table]
Revised: [Table]

Revised: L =

Revised: [Table]