Title: DEFENSE AGAINST ADVERSARIAL EXAMPLES BY ENCODER-ASSISTED SEARCH IN THE LATENT COD-ING SPACE

Abstract: Deep neural networks were shown to be vulnerable to crafted adversarial perturbations, and thus bring serious safety problems. To solve this problem, we proposed AE-GAN +rs , a framework for purifying input images by searching the closest natural reconstruction with little computation. We first build a reconstruction network AE-GAN, which adapted auto-encoder by introducing adversarial loss to the objective function. In this way, we can enhance the generative ability of decoder and preserve the abstraction ability of encoder to form a self-organized latent space. In the inference time, when given an input, we will start a search process in the latent space which aims to find the closest reconstruction to the given image on the distribution of natural data. The encoder can provide a good start point for the searching process, which saves much computation cost. Experiments show that our method is robust against various attacks and can reach comparable even better performance to similar methods with much fewer computations.

1. INTRODUCTION:

Deep neural networks (DNNs) have achieved state-of-the-art performances among various challenging computer vision tasks. However, they are shown to be vulnerable to adversarial attacks (Szegedy et al., 2013; Goodfellow et al., 2014b; Madry et al., 2017b) , which add carefully crafted perturbations to a legitimate input sample. The perturbations are small and not perceptible to a human, but they can significantly mislead the target model. Moreover, adversarial examples have considerable transferability between different models, which entertains the feasibility of black-box attacks in the real world. Therefore, it is important to find effective strategies to defend against adversarial attacks.

There have been many efforts to defend against adversarial attacks by diminishing perturbations of samples before feeding them into classifiers. Auto-encoder based methods have achieved prevalence though (Gu & Rigazio, 2014; Meng & Chen, 2017) , they are shown to be vulnerable to white-box attacks (Samangouei et al., 2018) because the model stacked by auto-encoder and classifier can be attacked again. Other generative models (Samangouei et al., 2018; Song et al., 2017) have been trained to model the distribution of normal data and project back the adversarial samples to the manifold of normal samples. In the process of purifying input images, they often include complicated optimization. Therefore, they can easily resist the white-box attacks but at the cost of time.

Compared to natural images, adversarial examples have much lower probability densities under the image distribution (Song et al., 2017) , which is regarded as an unknown input pattern for an autoencoder trained with legitimate examples. As a result, the output of the encoder can be influenced by the crafted perturbations and lead to a top-down reconstruction far away from the input. When humans handle unknown input patterns, we will first find a conception in our mind and correct our conception step by step by checking the consistency between the input and the reconstruction. We can apply this mechanism to neural networks, where the latent code for reconstruction is calculated by a search process guided by feedback from the visual space. The same idea has been included in Defense-GAN (Samangouei et al., 2018) , where a generator is trained to model the distribution of unperturbed images. When given an image, it uses GD minimization to find the optimum latent code, which corresponds to the closest output without perturbations to the given image. However, Defense-GAN requires multiple random restart points and considerable iterations to guarantee the Figure 1 : Framework of the encoder assisted search process in the inference stage. The encoder and decoder are pretrained. The search process is implemented in the latent space formed by the encoder and starts at the point given by the encoder. The discrepancy between the input and the reconstruction can act as the feedback from the visual space to guide the search. performance, which brings a high computation cost. That's because the GD minimization is sensitive to the initial points, which is given randomly in Defense-GAN.

Our approach takes the advantages of both auto-encoder and Defense-GAN. We follow the encoderdecoder framework to build the reconstruction model but also include a search process to find the best reconstruction. The encoder-decoder framework can help to form a self-organized latent space and preserves the neighborhood relations (Xu, 2019) , which can provide a fair start point and a more conducive environment for the search process. However, the traditional auto-encoder is trained to form an identical mapping from the input to output instead of minimizing the distance between the distribution of natural and generated images. Consequently, auto-encoder may find the closest image to a given perturbed image but out of the distribution of real data, which is still hard for a classifier to give a right prediction. To solve this problem, we add a discriminator to distinguish between natural images and generated images and force the decoder to model the distribution of natural data. The training for the reformer network is simple and requires no adversarial samples. Experimental results show that the decoder can always generate natural samples when the encoder can provide a good start point and a self-organized latent space for the search process. Besides, the encoder-decoder framework can also act as an effective detector (Meng & Chen, 2017 ). In the inference stage, when there comes a test sample, we can first detect if the sample is a malicious image or not. If not, we simply reconstruct the input image to eliminate missing noise. If true, we start the search process in the latent space, which is implemented by GD minimization, to find the closest natural reconstruction to the given image. Compared with Defense-GAN, we have two advantages: (1) The detecting enables us to discriminate natural samples and adversarial samples effectively, thus we can bypass the searching stage on natural samples, which saves much time; (2) For malicious samples, the search process starts at a good initial, thus we need no random restarts and only a few iterations. Compared with auto-encoder, we have a strong defense against white-box attacks due to the searching process. Experiments show that our method is robust against different attacks and can reach comparable even better performance with Defense-GAN while using much fewer computations. (Szegedy et al., 2013) and generated adversarial perturbations using an L-BFGS method to solve an optimization problem. For better efficiency, Goodfellow et al. proposed Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2014b) , an efficient single-step attack where the perturbation is updated along with the sign of the gradient of the loss w.r.t the input images. Later, Basic Iterative Method (BIM) (Kurakin et al., 2016b) is proposed as an extension of FGSM by running a finer optimization for multiple iterations and clipped pixel values in each iteration. After that, Moosavi-Dexfooli et al. proposed DeepFool (Moosavi-Dezfooli et al., 2016) to find the closest distance from the normal input to the decision boundary of adversar-ial samples. More recently, Carli and Wagner (Carlini & Wagner, 2016) launched a powerful attack to defeat defensive distillation (Papernot et al., 2015) , which generates adversarial examples with small perturbations. It is also possible to use generative models to generate universal adversarial perturbations (Reddy Mopuri et al., 2018) .

2. RELATED WORK:



DEFENSE METHODS:

In recent years, a variety of techniques have been proposed to defend against adversarial attacks. Adversarial training (Kurakin et al., 2016a; Goodfellow et al., 2014b; He et al., 2017; Madry et al., 2017b) is one of the most popular ways which trains the classifier on the augmented training set with adversarial samples. It improves the accuracy on adversarial samples but can not generalize well to new attacks (He et al., 2017) . Meanwhile, it is time-consuming to generate adversarial examples and retrain the classifier. Gradient masking method (Papernot et al., 2015) has also been considered to mask or reduce the gradients in magnitude when training the classifier, but it has been shown to give a false sense of security in defending against attacks (Athalye et al., 2018) . The two popular methods include retraining the classifier, while the classifier may not be allowed to be modified in the physical world.

Another way is to pre-process adversarial samples before feeding them into classifiers. Some previous work resort to traditional denoising methods (Das et al., 2017; Dziugaite et al., 2016; Osadchy et al., 2017) to purify the adversarial examples. The main drawback of these methods is that they can only fix small perturbations and may cause information loss. Denoising auto-encoder (Gu & Rigazio, 2014) is firstly proposed to defend against adversarial attacks by training an auto-encoder to map the corrupted images to clean ones. As discussed in (Samangouei et al., 2018) , the stacked network is still easy to be attacked. MagNet ensembles many auto-encoders and applies a two-step defense mechanism where the hard adversarial attacks can be detected and soft adversarial attacks can be reformed. However, MagNet still suffers from white-box attacks. Different from autoencoder, Defense-GAN (Samangouei et al., 2018) is proposed to leverage the representative power of GAN to diminish adversarial perturbations. At the inference stage, they implement a gradient descent minimization to find the best code corresponding to the closest reconstruction. Since the gradient can not pass the searching process, Defense-Gan can strongly defense against white-box attacks while the searching process is time-consuming.

3. APPROACH:



3.1. THREAT MODEL:

We consider the following types of adversarial attacks, according to the level of how much information is available to the attackers:

• Black-box attacks: no details about the classifier and defense mechanism are accessible. A substitute network is trained to mimic the classifier and then is used to generate adversarial examples. • White-box attacks: complete access to all the information about the classifier and the defense strategy. • Gray-box attacks: complete access to the classifier but no access to defense strategy.

3.2. METHOD OVERVIEW:

Let f θ (x) : R d → N be a classifier parameterized by θ, where R d is the image space and N is the set of natural numbers denoting class labels. Given a clean image x, an adversary can perturb the clean image with small perturbation η but confuse f :

f θ (x + η) = f θ (x), η < attack (1)

where • is a measurement and attack is the perturbation scale which sets the maximum perturbation allowed for each pixel and often set to a small number to get almost imperceptible difference between x adv = x + η and x.

We want to improve robustness by learning a transformation T (.), such that the predicted label of the transformed image does not change when the image corrupted with perturbations, i.e., 

f θ (T (x adv )) = f θ (T (x)).

The basic idea of our method is to purify input images by searching the closest reconstruction without perturbations within little computation cost. To achieve this, we build a reconstruction network based on the encoder-decoder framework, which is denoted as AE-GAN +r and will be described in detail in Section 3.3. After training, the encoder of AE-GAN +r can provide a good representation for most inputs, and the decoder can always generate an image on the manifold of natural samples during the search.

In the inference time, when given an input, we first detect if the sample is adversarial or not. If not, we simply reconstruct the input image by AE-GAN +r to diminish missing noises. If true, we start a search process in the latent coding space to find the best code, which corresponds to the closest reconstruction without perturbations. The search process starts at the output of the encoder and implemented by GD minimization, which will be described in Section 3.4. Formally, we denote this encoder assisted search process as AE-GAN +rs .

3.3. ARCHITECTURE OF AE-GAN:

An auto-encoder (AE) is composed of two components: the encoder F and the decoder G. Let X be the visual space and Z be the latent space. The mapping F : X → Z performs image abstraction, which encode the image x ∈ X into a code z ∈ Z. The mapping Z → X performs image generation, which generates image x ∈ X from the output of encoder z ∈ Z. Formally, we denote the distribution of training images as p data . The loss function of AE is defined as:

E x∼p data G(F (x)) -x 2 2

(2)

As shown in Equation 2, traditional AE minimizes the euclidean distance between input and output. Thus, it is trained to form an identical mapping from the input to output instead of minimizing the distance between the distribution of natural images p data and generated images p g . Consequently, when we search for the best code z in the latent space, AE may find a close image G(z) but out of P data , as shown in Figure 5 in Appendix E, which is a meaningless purification because the accuracy of classifier on these images is not good. To solve this problem, we adapt the auto-encoder by introducing a discriminator and the min-max loss in GANs (Goodfellow et al., 2014a) , as shown in Figure 2 . We denote the adapted AE as AE-GAN. GANs consists of two neural networks, a generator G and a discriminator D. G maps a low-dimensional latent space Z to the high dimensional sample space X . D is a binary neural network classifier. While G learns to generate samples that similar to the real data x, D learns to distinguish between "real" samples x and "fake" samples G(z). D and G are trained in an alternating fashion to minimize the following min-max loss (Goodfellow et al., 2014a) :

EQUATION

To preserve the advantages of AE, we still keep the mean square error to drive the abstraction mapping and self-organizing in the latent space. Meanwhile, we add a discriminator learning to distinguish between the samples generated by the decoder and real samples. Thus, the objective of AE-GAN is to minimize the following min-max loss:

min F,G max D V (D, F, G) =E x∼p data [log(D(x))] + E z∼pz [log(1 -D(G(z)))]+ E x∼p data G(F (x)) -x 2 2 (4)

As the start point of the search process, the output of the encoder is expected to be close to the optimum code and thus help the search to converge quickly. Since AE-GAN is trained with legitimate images, the encoder F may suffer from the covariate shift when the input is corrupted with large perturbations. To solve this, we perturb the clean images with random Gaussian noise before feeding them into the encoder when calculating the reconstruction error. Adding noise in the input layer acts as a regularization (Bishop, 1995) , thus we denote the method as noise regularization (+r) in this paper. The final objective function of AE-GAN +r can be written as Equation 5.

EQUATION

where η ∼ N (0, σ), σ follows the uniform distribution on a given region.

Bidirectional Generative Adversarial Networks (BiGANs) (Donahue et al., 2016 ) also includes encoder-decoder mapping. In addition to the generator from the standard GAN framework, BiGAN includes an encoder to map the input data into the latent representation. The discriminator in BiGAN discriminates jointly data and latent code, i.e., (x, E(x)) versus (G(z), z)). However, experimental results show that BiGAN has poor reconstruction performance. When BiGAN equipped with the searching process, the encoder in BiGAN can not provide a good initial point for the decoder.

3.4. ENCODER ASSISTED SEARCH PROCESS:

Although AE-GAN +r has been trained with noise regularization, the added noise is simple while the adversarial noise is carefully crafted, the distribution between them is different. Moreover, when the input x * is seriously corrupted, e.g., a large region of the image is masked, the code derived from encoder F (x * ) can hardly represent the original image x. Therefore, a correction for F (x * ) is required when the input pattern is significantly different from the training data. Meanwhile, the endto-end encoder-decoder framework can not perform well against white-box attacks (Samangouei et al., 2018) . Therefore, we include a search (denoted as +s) process in the latent space in the inference stage.

As shown in Figure 1 , the core idea is that for every input sample x, AE-GAN +s will process a search process in the latent space to find the best code for an input, which can lead the decoder to generate the closest reconstruction. For a given input x, we define the objective function J of the search process as:

EQUATION

where G(.) denotes the decoder, L(.) is a measurement of the discrepancy between x and G(z). We aims to find the optimum code z * such that

EQUATION

Since our latent space is designed to be continuous, thus the direction and step size for the search process can be calculated by Gradient Decent (GD) method. Therefore, the search process is converted to a GD minimization to minimize the discrepancy between the input and the reconstruction.

Here we simply use euclidean distance on the pixel level to measure the discrepancy. A more detailed description of the searching process is stated in Appendix C.

In our framework, the encoder forms a self-organized latent space that preserves neighborhood relations and provides a good start point. Both of them can assist the search process in convergence.

In Defense-GAN (Samangouei et al., 2018) , the latent space follows a single Gaussian distribution while the gradient descent method is sensitive to initial values. Therefore, it requires multiple random restart points and considerable iterations to guarantee the performance, which brings a high computation cost. (Papernot et al., 2017) . We split the testing set into a small hold-out set of 150 samples for training the substitute model and the remaining 9850 samples for testing different defense strategies. For all the attack methods, we use the implementation in Cleverhans (Papernot et al., 2018) , a python library to benchmark machine learning systems' vulnerability to adversarial examples, and enforce the image to remain within [0, 1] H×W by clipping.

4. EXPERIMENTS:



4.2. RESULTS ON ADVERSARIAL ATTACKS:

We compare our method to PGD adversarial training (Madry et al., 2017b) , MagNet (Meng & Chen, 2017) , Defense-Gan (Samangouei et al., 2018) , BiGAN (Donahue et al., 2016) under the FGSM, PGD and CW (with l 2 norm) white-box and gray-box attacks, as well as the FGSM black-box attacks. Our reported method is AE-GAN +rs , i.e., AE-GAN with noise regularization and search process, where the number of iterations L = 15. We also assist BiGAN with the search process for fair comparisons, which is denoted as BiGAN +s . Reported Defense-GAN follows the original set-tings where iterations L = 200 and restart points R = 10. All experimental results are summarized in Table 1 .

From Table 1 , we can observe that adversarial training successfully defends against the FGSM and PGD attack, but can not generalize to CW attacks. Because it needs adversarial examples to retrain the classifier. As discussed in Section 3.3, on small scale perturbations like CW the performance of MagNet and AE-GAN +rs is close, because the code derived from the encoder is robust to small perturbations. However, on large scale perturbations like FGSM ( = 0.3) and PGD ( = 0.3), the reported AE-GAN +rs can achieve about 20% ∼ 30% improvements than MagNet.

As shown in Table 1 , adversarial training and MagNet can only perform well under specific situations, while Defense-GAN and our method can defense against various attacks. Attack-agnostic is important for a defense strategy because we have no information about the attacker in the physical world. While our method reaches comparable performance with Defense-GAN under white-box and black-box attacks, we significantly reduce the computation complexity to 15/2000 of Defense-GAN. Another observation is that when the performance both decreases on the F-MNIST dataset due to the large perturbations, our method performs much better than Defense-GAN on gray-box attacks.

The possible reason is that images in F-MNIST are more complex and require more details, which raises the difficulty of searching the best code in the latent space from random points. For white-box attacks, the performance of our method has a slight drop from Defense-GAN. However, it has been studied in (Samangouei et al., 2018 ) that more iterations can increase the robustness for Defense-GAN against GD-based white-box attacks. When we decline L from 200 to 15 for Defense-GAN, the performance drops by 8% decline. We can also observe that with fewer iterations, AE-GAN +rs still performs much better than BiGAN +s under various attacks. Though BiGAN +s also includes an encoder-decoder mapping, the encoder of BiGAN +s can not provide a good start point for the searching process. This is due to the poor reconstruction problem of BiGAN, which will be further studied in Section 4.5.

Table 1 also shows that the performance of defenses against gray-box and black-box attacks on the F-MNIST dataset is noticeably lower than on MNIST. This is due to the large perturbations generated by the FGSM attack with = 0.3. It is more difficult to find an accurate reconstruction for the seriously perturbed image, as shown in Figure 10 in Appendix H.

4.3. TRADE-OFF BETWEEN COMPUTATION AND PERFORMANCE:

We further investigate the trade-off between computation cost and performance in Defense-GAN and our method. Table 2 shows the robustness of Defense-GAN and AE-GAN +rs when changing the number of restart points R and iterations L. It can be seen that with fewer restart points, the performance of Defense-GAN decreases a lot after a certain L value, which means that the effect of varying R is extremely pronounced. This is due to the non-convex nature of MSE and increasing R enables Defense-GAN to sample different local minimum (Samangouei et al., 2018) . However, in our framework, taking the output of the encoder as the initial start point can enable the search process to reach a reasonable local minimum in a few iterations. When we decrease L from 60 to 10, there is 6% -10% drop for the accuracy of Defense-GAN, while there is little change in the performance of AE-GAN +rs . Therefore, compared with Defense-GAN, our method requires less computation than Defense-GAN when they reach comparable performance. The performance of our is robust to the computation.

4.4. DETECTION:

A trained AE-GAN +r can well reconstruct natural images without searching. From this observation, we propose to introduce the detection mechanism to bypass the searching process for clean images.

In AE-GAN +r , the decoder is trained to produce images that resemble the legitimate data. Thus, the adversarial examples usually have a high reconstruction error. Therefore, we can choose the mean square error (MSE) between input and reconstruction as our indicator to decide whether or not the image is adversarial. The setting for the threshold η is important for the detection. A larger threshold can tolerant more perturbations and enhance efficiency but reduce security.

To investigate the performance of detection, we compute the reconstruction error for every image in the testing dataset, and their corresponding adversarially perturbed images under different strengths The results show that a smaller threshold corresponds to lower FNR but higher FPR, which makes the detection safer but may cause false alarms. Meanwhile, when the perturbation is small, it is hard to detect attacks at both low FNR and FPR, but AE-GAN +r is good at purifying small perturbations, which can make up this shortcuts, as shown in Figure 3 (c ). We also test the detection performance when using L 1 loss as the indicator, which is shown to be more effective. It can reach both small FNR and FPR even under small perturbations. For more details, please refer to Appendix F.

In summary, for small perturbations which pass the detection, AE-GAN +r can successfully eliminate them. For larger perturbations that are detected as adversarial, they will trigger the searching stage and be purified by AE-GAN +rs for achieving higher accuracy.

4.5. ABLATION STUDY:

To investigate the effect of each module in our method, we compare the performance of different models under the FGSM gray-box attack and half-masked attack (half of the input image is blocked, as shown in Figure 5 (c) in Appendix E. Table 3 shows the defense performance of the different models on different image corruptions visualized in Figure 5 , where the number of iterations L is set to be 20 for models with search process.

Firstly, by comparing the performance of AE-GAN and AE-GAN +r , we can observe that noise regularization has an expected effect of forming a more accurate latent code for the corrupted input.

Secondly, the accuracies of AE +s and AE-GAN +s as well as the qualitative results in Figure 5 (see

A IMPLEMENTATIONS:

For adversarial training, we follow (Madry et al., 2017b) and augment the training dataset with adversarial samples generated by PGD. We use the same for training and testing and set = 0.3 for MNIST and F-MNSIT, = 0.1 for CelebA dataset.

For Defense-GAN, we adopt the implementation of original work (Samangouei et al., 2018) .

For BiGAN, the original implementation (Donahue et al., 2016) is based on the fully-connected network. For fair comparisons, we adopt the convolutional architecture in another similar work (Dumoulin et al., 2016) . We slightly adapt the architecture on CIFAR-10 ( Krizhevsky et al., 2009) by changing the input channel from 3 to 1. As shown in Figure 4 , the model we trained reach a fair performance on generating samples. For MagNet, we follow the implementation in Defense-GAN (Samangouei et al., 2018) and restate the details in Table 6 , which shares the same architecture with AE-GAN. However, we find that the latent space dimension 128 is too large for MagNet to eliminate strong perturbations. Because the details of noise are also encoded and can be reconstructed. Thus, we also take the latent dimension as 20 for comparisons. The results in Table 8 show that the change can enhance the performance on the gray-box attack.

B NEURAL NETWORK ARCHITECTURES AND TRAINING SETTINGS:

For fair comparisons, we compare different strategies through this paper on the same classifier A, and we take two substitute classifiers for black-box attacks, B for convolutional structure and C for fully connected structure. The training settings and architectures almost follow the settings in Defense-GAN (Samangouei et al., 2018) . For clarification, here we restate their architectures and training settings in Table 5 and Table 7 respectively.

The architecture of our model and MagNet is described in Table 6 , which make a slight modification on the MagNet structure given in (Samangouei et al., 2018) 

∇ θ d 1 m m i=1 log D x (i) + log 1 -D G z (i) Sample σ from uniform distribution U (0, η)

Sample minibatch of m noise samples { (1) , (2) , . . . , (m) } from N (0, η) Update the decoder by descending its stochastic gradient

∇ θg 1 m m i=1 [log 1 -D G z (i) + (F (z (i) ) -(x (i) + (i) )

) 2 ] Update the encoder by descending its stochastic gradient.

∇ θe

1 m m i=1 (F (z (i) ) -(x (i) + (i) )) 2 end while

Algorithm 2 Search process of AE-GAN +rs in inference stage Input: input image x, trained encoder F , trained decoder G, number of steps L, initial step size η F ADDITIONAL RESULTS ON DETECTION When using L 1 loss as the indicator to decide whether a image is adversarial or not, we can obtain better detection performance on MNIST dataset. As shown in Figure 6 , when the threshold is set to 0.06, the detection can reach both small False Negative Rate and small False Positive Rate on small perturbations, while it is hard to achieve this with MSE, as shown in Figure 3 in the main text. By comparing Figure 7 and Figure 8 , we can get similar observations on F-MNIST dataset. 

Output: Optimal reconstruction G(z k ) of x initiate z 0 = F (x) while k < L do z k = z k-1 + η∇ z (L 2 (G(z k ) -x)) if k mod 5 == 0 then η = η × 1 2 end if end while return G(z k ) D ADDITIONAL RESULTS ON VARIOUS LATENT DIMENSIONS

G ADDITIONAL RESULTS ON CELEBA:

To investigate the scalability of our method, we conduct experiments on CelebFaces Attributes dataset (CelebA) (Liu et al., 2015) and report the results in Table 9 . CelebA contains more than 200,000 face images and we extract 162770, 10000, 10000 images for training, validation, and testing. Each image is center-cropped and resized to 64 × 64. Following the settings in Defense-GAN (Samangouei et al., 2018) , we perform the classification of gender on the dataset and the architecture of AE-GAN also adopts the implementation in Defense-GAN for a fair comparison. The architecture of the encoder is the same as the discriminator except for modifying the dimension of the top layer. The accuracy of the trained classifier A on the training dataset of CelebA is 95.96%. 

:



Figures and Tables:

Figure 2: Architecture of AE-GAN

Figure 3: Detections on MNIST. Left: ROC curves when using AE-GAN +r to detect FGSM attacks with various . Middle: The False Negetive Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various . Right: Detection performance on different strength of FGSM attacks on MNIST dataset and the purification performance of AE-GAN +r .

Figure 4: Generated samples by BiGAN trained on MNIST (left) and F-MNIST (right).

Figure5shows the purification performance of AE +s and AE-GAN +s on corrupted images. We can easily see that though AE +s incorporates a searching process, it can not find a natural reconstruction to the given corrupted image. The comparison can show the effectiveness of discriminator and the adversarial loss we introduced in AE-GAN +s .

Figure 6: Detections on MNIST when using L 1 distance as the indicator. Left: ROC curves when using AE-GAN +r to detect FGSM attacks with various . Right: The False Negetive Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various .

Figure 7: Detections on F-MNIST when using L 1 distance as the indicator. Left: ROC curves when using AE-GAN +r to detect FGSM attacks with various . Right: The False Negative Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various .

Figure 8: Detections on F-MNIST when using MSE as the indicator. Left: ROC curves when using AE-GAN +r to detect FGSM attacks with various . Right: The False Negative Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various .

Figure 9: Example results of our method on MNIST. The left part shows adversarial examples generated by different attacks: FGSM attack with = 0.3 (a), PGD attack with = 3 (b) and CW with l 2 norm (c), while the right part shows purified images by AE-GAN +rs . The number of iterations for guided search is set to 15.

Classification accuracies of classifier A using various defense strategies on the MNIST (top) and F-MNIST (bottom) dataset, under FGSM ( = 0.3), PGD ( = 0.3) and CW (with L 2 norm) white-box attacks, gray-box attacks, and FGSM black-box attacks. Here the corresponding iterations of GD minimization are: 15 for our method, 60 for BiGAN, and 200 for Defense-GAN. (Adv.Tr.: adversarial training with PGD ( = 0.3))

Table content: <table><tr><td colspan="3">Attack Method None</td><td>Our</td><td colspan="4">Adv. Tr. MagNet BiGAN +s Defense-Gan</td></tr><tr><td/><td colspan="3">FGSM 0.144 0.984</td><td>0.949</td><td>0.293</td><td>0.912</td><td>0.981</td></tr><tr><td>White</td><td>PGD</td><td colspan="2">0.007 0.982</td><td>0.920</td><td>0.012</td><td>0.924</td><td>0.989</td></tr><tr><td/><td>CW</td><td colspan="2">0.008 0.979</td><td>0.772</td><td>0.013</td><td>0.905</td><td>0.981</td></tr><tr><td/><td colspan="3">FGSM 0.144 0.882</td><td>-</td><td>0.561</td><td>0.731</td><td>0.880</td></tr><tr><td>Gray</td><td>PGD</td><td colspan="2">0.007 0.910</td><td>-</td><td>0.673</td><td>0.739</td><td>0.913</td></tr><tr><td/><td>CW</td><td colspan="2">0.008 0.915</td><td>-</td><td>0.881</td><td>0.805</td><td>0.918</td></tr><tr><td>Black</td><td>A/B A/C</td><td colspan="2">0.701 0.935 0.813 0.933</td><td>0.971 0.970</td><td>0.464 0.590</td><td>0.701 0.632</td><td>0.928 0.922</td></tr><tr><td colspan="3">Attack Method None</td><td>Our</td><td colspan="4">Adv. Tr. MagNet BiGAN +s Defense-Gan</td></tr><tr><td/><td colspan="3">FGSM 0.073 0.804</td><td>0.739</td><td>0.199</td><td>0.613</td><td>0.814</td></tr><tr><td>White</td><td>PGD</td><td colspan="2">0.028 0.816</td><td>0.717</td><td>0.071</td><td>0.653</td><td>0.852</td></tr><tr><td/><td>CW</td><td colspan="2">0.062 0.767</td><td>0.308</td><td>0.084</td><td>0.601</td><td>0.810</td></tr><tr><td/><td colspan="3">FGSM 0.073 0.607</td><td>-</td><td>0.304</td><td>0.570</td><td>0.389</td></tr><tr><td>Gray</td><td>PGD</td><td colspan="2">0.028 0.736</td><td>-</td><td>0.427</td><td>0.605</td><td>0.491</td></tr><tr><td/><td>CW</td><td colspan="2">0.062 0.719</td><td>-</td><td>0.646</td><td>0.608</td><td>0.435</td></tr><tr><td>Black</td><td>A/B A/C</td><td colspan="2">0.227 0.603 0.295 0.507</td><td>0.784 0.777</td><td>0.308 0.267</td><td>0.398 0.348</td><td>0.602 0.461</td></tr><tr><td colspan="8">Specifically, we consider the white-box and gray-box attacks by FGSM (Goodfellow et al., 2014b),</td></tr><tr><td colspan="8">PGD (Madry et al., 2017a) and Carlini-Wagner (CW) attack with 2 norm (Carlini &amp; Wagner, 2016),</td></tr><tr><td colspan="8">and black-box attacks by FGSM. The training procedure of the substitute model in black-box attacks</td></tr><tr><td colspan="3">is the same as the setting in</td><td/><td/><td/><td/></tr></table>

the bound of random noise while epoch < iteration do Sample minibatch of m noise samples {z (1) , z (2) , . . . , z (m) } from noise prior p(z) Sample minibatch of m samples {x (1) , x (2) , . . . , x (m) } from training set Update the discriminator by ascending its stochastic gradient

Table content: <table><tr><td colspan="3">: Training parameters for AE-GAN and MagNet</td></tr><tr><td>Parameters</td><td colspan="2">MNIST, F-MNIST CelebA</td></tr><tr><td>Epochs</td><td>60</td><td>100</td></tr><tr><td>Learning Rate</td><td>0.0002</td><td>0.0002</td></tr><tr><td>Optimization Method</td><td>Adam</td><td>Adam</td></tr><tr><td>Batch Size</td><td>100</td><td>64</td></tr></table>

Classification accuracy of A using MagNet and AE-GAN on MNIST with various latent space dimensions. D: the dimension of latent space (the number of iterations is set to 15).

Table content: <table><tr><td colspan="3">Attack Attack None</td><td colspan="4">D=20 MagNet AE-GAN +rs MagNet AE-GAN +rs D=128</td></tr><tr><td/><td colspan="2">FGSM 0.144</td><td>0.298</td><td>0.972</td><td>0.293</td><td>0.984</td></tr><tr><td>White</td><td>PGD</td><td>0.007</td><td>0.012</td><td>0.975</td><td>0.006</td><td>0.982</td></tr><tr><td/><td>CW</td><td>0.008</td><td>0.013</td><td>0.970</td><td>0.008</td><td>0.979</td></tr><tr><td/><td colspan="2">FGSM 0.144</td><td>0.561</td><td>0.875</td><td>0.316</td><td>0.882</td></tr><tr><td>Gray</td><td>PGD</td><td>0.007</td><td>0.673</td><td>0.911</td><td>0.145</td><td>0.910</td></tr><tr><td/><td>CW</td><td>0.008</td><td>0.881</td><td>0.907</td><td>0.345</td><td>0.915</td></tr></table>

Peer review (iclr2020):
Summary: This paper proposes AE-GAN+sr, an auto-encoder based GAN for equipping neural networks with better defenses against adversarial attacks. The authors evaluate their method on black-box attacks, white-box attacks, and gray-box attacks on MNIST and Fashion-MNIST, and show decent empirical results when compared to baselines. Decision: Reject. The writing was hard to follow and the experimental evaluations could have been stronger. Supporting Arguments/Feedback: - There are a lot of phrases which should be reworded in the text (e.g. “it requires attention to” in paragraph 1, “detect if sample is from a normal distribution” at the end of Section 1, etc.) for better clarity. The writing was pretty hard to follow and the paper would really benefit overall after it has been improved. - It was hard for me to tell from the text how exactly the models were implemented and trained (see Question #1). Making this point more clear, in either the main text or the Appendix, would be really helpful. - The performance of AE-GAN+sr as evaluated on MNIST and FashionMNIST did not provide convincing evidence that it outperformed baselines such as Defense-GAN or adversarial training, though I did appreciate the authors’ additional analysis on the tradeoffs between computational cost and performance for the Defense-GAN. The results would also have been more compelling had the authors evaluated their method on more complex datasets such as CIFAR-10. Questions: - It wasn’t clear to me how the encoder-assisted search process (Section 3.4) works in practice. When you use gradient descent to find the best encoding for a corrupted input, how many steps do you need to take? Does this step happen while the autoencoder and GAN are being trained, or are those pre-trained and you just take additional gradient steps in the latent space? - Given that the BiGAN also incorporates an encoder (and they were also benchmarked against in the experiments), where do the advantages of the AE-GAN+sr come from? I think making this point in the text would also be helpful as well.