Defense against Adversarial Examples by Encoder-Assisted Search in the Latent Coding Space
Authors: Wenjing Huang
Shikui Tu
Lei Xu
Venue:  (2020)
Keywords: Adversarial Defense
Auto-encoder
Adversarial Attack
GAN

Abstract: Deep neural networks were shown to be vulnerable to crafted adversarial perturbations, and thus bring serious safety problems. To solve this problem, we proposed $\text{AE-GAN}_\text{+sr}$, a framework for purifying input images by searching a closest natural reconstruction with little computation. We first build a reconstruction network AE-GAN, which adapted auto-encoder by introducing adversarial loss to the objective function. In this way, we can enhance the generative ability of decoder and preserve the abstraction ability of encoder to form a self-organized latent space. In the inference time, when given an input, we will start a search process in the latent space which aims to find the closest reconstruction to the given image on the distribution of normal data. The encoder can provide a good start point for the searching process, which saves much computation cost. Experiments show that our method is robust against various attacks and can reach comparable even better performance to similar methods with much fewer computations.

Under review as a conference paper at ICLR 2020
DEFENSE AGAINST ADVERSARIAL EXAMPLES BY ENCODER-ASSISTED SEARCH IN THE LATENT COD- ING SPACE
Anonymous authors Paper under double-blind review
ABSTRACT
Deep neural networks were shown to be vulnerable to crafted adversarial perturba- tions, and thus bring serious safety problems. To solve this problem, we proposed AE-GAN+ts a framework for purifying input images by searching the closest nat- ural reconstruction with little computation. We first build reconstruction network AE-GAN, which adapted auto-encoder by introducing adversarial loss to the ob- jective function. In this way, we can enhance the generative ability of decoder and preserve the abstraction ability of encoder to form a self-organized latent space. In the inference time, when given an input, we will start search process the latent space which aims to find the closest reconstruction to the given image on the distribution of natural data. The encoder can provide a good start point for the searching process, which saves much computation cost. Experiments show that our method robust against various attacks and can reach comparable even better performance to similar methods with much fewer computations.
INTRODUCTION
Deep neural networks (DNNs) have achieved state-of-the-art performances among various challeng- ing computer vision tasks. However, they are shown to be vulnerable to adversarial attacks (Szegedy) al.] [2013] Goodfellow et al.] 2014b] Madry et al.] 2017b), which add carefully crafted perturba- tions to a legitimate input sample. The perturbations are small and not perceptible to human, but they can significantly mislead the target model. Moreover, adversarial examples have considerable transferability between different models, which entertains the feasibility of black-box attacks in the real world. Therefore, it is important to find effective strategies to defend against adversarial attacks.
There have been many efforts to defend against adversarial attacks by diminishing perturbations of samples before feeding them into classifiers. Auto-encoder based methods have achieved prevalence though (Gu & Rigazio| 2014 Meng & Chen, (017). they are shown to be vulnerable to white-box attacks (Samangouei et al. 2018) because the model stacked by auto-encoder and classifier can be attacked again. Other generative models (Samangouei
al.] 2018 Song al. 2017) have been trained to model the distribution of normal data and project back the adversarial samples to the man- ifold of normal samples. In the process of purifying input images, they often include complicated optimization. Therefore, they can easily resist the white-box attacks but at the cost of time.
Compared to natural images, adversarial examples have much lower probability densities under the image distribution (Song et al., 2017). which is regarded as an unknown input pattern for an auto- encoder trained with legitimate examples. As result, the output of the encoder can be influenced by the crafted perturbations and lead to top-down reconstruction far away from the input. When humans handle unknown input patterns, we will first find a conception in our mind and correct our conception step by step by checking the consistency between the input and the reconstruction. We can apply this mechanism to neural networks, where the latent code for reconstruction calculated by search process guided by feedback from the visual space. The same idea has been included in Defense-GAN GAN (Samangouei et al., 2018), where generator is trained to model the distribution of unperturbed images. When given image, it uses GD minimization to find the optimum latent code, which corresponds to the closest output without perturbations to the given image. However, Defense-Gal GAN requires multiple random restart points and considerable iterations to guarantee the

---

Under review as a conference paper at ICLR 2020
Foodback
2
8 = ((a))
Decoder
L(x,5)
Encodor
Latent Space
R G(zz)
Decoder
L(x.8g)
Serrch
2
min Mxx (x=))
$ = G(2')
Decoder
Figure 1: Framework of the encoder assisted search process in the inference stage. The encoder and decoder pretrained. The search process is implemented in the latent space formed by the encoder and starts at the point given by the encoder. The discrepancy between the input and the reconstruction can act as the feedback from the visual space guide the search.
performance, which brings a high computation cost. That's because the GD minimization sensitive to the initial points, which is given randomly in Defense-Gaad GAN.
Our approach takes the advantages of both auto-encoder and Defense-GAN, We follow the encoder- decoder framework to build the reconstruction model but also include search process to find the best reconstruction. The encoder-decoder framework can help to form self-organized latent space and preserves the neighborhood relations (Xu, 2019), which can provide a fair start point and more conducive environment for the search process. However, the traditional auto-encoder is trained to form identical mapping from the input to output instead of minimizing the distance between the distribution natural and generated images. Consequently, auto-encoder may find the closest image given perturbed image but out of the distribution of real data, which is still hard for classifier to give a right prediction. To solve this problem, we add a discriminator to distinguish between natural images and generated images and force the decoder model the distribution of natural data. The training for the reformer network is simple and requires no adversarial samples. Experimental results show that the decoder can always generate natural samples when the encoder can provide good start point and a self-organized latent space for the search process. Besides, the encoder-decoder framework can also act as an effective detector (Meng Chen 2017). In the inference stage, when there comes a test sample, we can first detect if the sample is a malicious image or not. not, we simply reconstruct the input image to eliminate missing noise. true, we start the search process in the latent space, which is implemented by GD minimization, to find the closest natural reconstruction to the given image. Compared with Defense- Sefennee-GAN, we have two advantages: (I) The detecting enables us discriminate natural samples and adversarial samples effectively, thus we can bypass the searching stage on natural samples, which saves much time; (2) For malicious samples, the search process starts at good initial, thus need random restarts and only a few iterations. Compared with aucoeen,oder, we have a strong defense against white-bol box attacks due to the searching process. Experiments show that our method robust against different attacks and can reach comparable even better performance with Defense-GAN while using much fewer computations.
RELATED WORK
ADVERSARIAN
ADVERSARIAL ATTACKS
Various attack models have been studied to generate adversarial examples. Szegedy et al. first intro- duced adversarial examples against neural networks (Szegedy al,, 2013) and generated adversar- ial perturbations using an L-BFGS method to solve an optimization problem. For better efficiency, Goodfellow et al. proposed Fast Gradient Sign Method (FGSM) (Goodfellow et al. 2014b), an effi- cient single-step attack where the perturbation is updated along with the sign of the gradient of the loss w.r.t the input images. Later, Basic Iterative Method (BIM) (Kurakin et al., (016b) is proposed extension of FGSM by rumning finer optimization for multiple iterations and clipped pixel et al. proposed DeepFool (Moosavi Dezfooli et al. 2016) to find the closest distance from the normal input to the decision boundary of adversar-
values in each iteration. After that,
Moosavi-Dezfoolll
2

---

complete access all the information about the classifier and the defense Under review as a conference paper at ICLR 2020
ial samples. More recently, Carli and Wagner (Carlini & Wagner 2016) launched powerful attack to defeat defensive distillation (Papernot et al. (015), which generates adversarial examples with small perturbations. It is also possible use generative models to generate universal adversarial perturbations (Reddy Mopuri et al. 2018).
DEFENSE METHODS
In recent years, variety of techniques have been proposed defend against adversarial attacks. Adversarial training (Kurakin et al. 20166a Goodfellow et al. 2014b) He et al. [0117] Madry et al. 2017b) is one of the most popular ways which trains the classifier on the augmented training set with adversarial samples. It improves the accuracy on adversarial samples but can not generalize well to new attacks (He et al. 2017). Meanwhile, and retrain the classifier. Gradient masking method (Papernot et al. 2015) has also been considered to mask or reduce the gradients magnitude when training the classifier, but it has been shown to give false sense of security in defending against attacks (Athalye et al., 2018). The two popular methods include retraining the classifier, while the classifier may not be allowed to be modified in the physical world.
time-consuming to generate adversarial examples
Another way is to pre-process adversarial samples before feeding them into classifiers. Some previ- work resort to traditional denoising methods (Das al., 2017 Dziugaite et al. 2016 Osadchy jet al. (2017) to purify the adversarial examples. The main drawbac back of these methods is that they can only fix small perturbations and may cause information loss. Denoising auto-encoder (Gu & Rigazio 2014) is firstly proposed to defend against adversarial attacks by training an auto-encoder map the corrupted images clean ones. As discussed in (Samangouei al.) 2018), the stacked network is still easy to be attacked. MagNet ensembles many auto-encoders and applies a two-step defense mechanism where the hard adversarial attacks can be detected and soft adversarial attacks can be reformed. However, MagNet still suffers from white-box attacks. Different from auto- encoder, Defense-GAN (Samangouei et al.) 2018) is proposed to leverage the representative power of GAN to diminish adversarial perturbations. At the inference stage, they implement gradient descent minimization to find the best code corresponding to the closest reconstruction. Since the gradient can not pass the searching process, Defense-Gan can strongly defense against white-box attacks while the searching process come-consuming.
APPROACH
3.1
THREAT MODEL
We consider the following types of adversarial attacks, according to the level of how much informa- tion available to the attackers:
Black-box attacks: details about the classifier and defense mechanism accessible. A sub- stitute network trained to mimic the classifier and then used to generate adversarial examples.
White-box attacks:
strategy.
Gray-box attacks: complete access the classifier but no access to defense strategy.
3.2 METHOD OVERVIEW
Let fo(x: d + N be a classifier parameterized by 0, where Rdd is the image space and N is the set of natural numbers denoting class labels. Given a clean image x, an adversary can perturb the clean image with small perturbation 7 but confuse f:
fo(1 + 11) + fo(x), ||r]|| < Eattack
(I)
where ||||| is measurement and Cattack is the perturbation scale which sets the maximum perturba- tion allowed for each pixel and often set to a small number to get almost imperceptible difference between Fads
+ 77 and æ.
We want to improve robustness by learning a transformation T(.). such that the predicted la- bel of the transformed image does not change when the image corrupted with perturbations, i.e.,
3

---

Under review as a conference paper at ICLR 2020
== -x))
x' G(z)
X
Encoder
Decoder
=1
Loss fi f
y N(0,T)
x'= G(y)
Decoder
Discriminator
-x,,x(xx,"))
X
Figure 2: Architecture of AE-GAN
(o(((siad))) fo((xx)) The basic idea of our method is to purify input images by searching the closest reconstruction without perturbations within little computation cost. To achieve this, we build a reconstruction network based on the encoder-decoder framework, which denoted as AE-GAN, and will be described in detail in Section 3.3] After training. the encoder of AE-GAN, can provide good representation for most inputs, and the decoder can always generate an image on the manifold of natural samples during the search.
In the inference time, when given an input, we first detect if the sample is adversarial or not. not, we simply reconstruct the input image by AE-GAN,, to diminish missing noises. If true, we start a search process in the latent coding space find the best code, which corresponds the closest reconstruction without perturbations. The search process starts the output of the encoder and implemented by GD minimization, which will be described Section Formally, we denote this encoder assisted search process as AE-GAN..-
3.3 ARCHITECTURE OF AE-GAN
An auto-encoder (AE) is composed of two components: the encoder F and the decoder G. Let X' be the visual space and Z be the latent space. The mapping F X Z performs image abstraction, which encode the image r E X' into code 2 E 2. The mapping 2 + X performs image generation, which generates image x E X from the output of encoder
Z. Formally, we denote the distribution of training images as Pdtta The loss function of AE is defined as:
Eawpsaao [[((F(z)) - =
(2)
As shown in Equation traditional AE minimizes the euclidean distance between input and output. Thus, it is trained to form an identical mapping from the input to output instead of minimizing the distance between the distribution of natural images Pdata and generated images Pg- Consequently, when we search for the best code in the latent space, AE may find close image G(z) but out of Paota: as shown in Figure in Appendix which a meaningless purification because accuracy of classifier on these images is not good. To solve this problem, we adapt the auto-encoder by introducing a discriminator and the min-max loss in GANs (Goodfellow et al. 2014a). as shown in 2 We denote the adapted AE AE-GAN. GANs consists of two neural networks, a generator G and a discriminator D. G maps a nww-dimensional latent space to the high dimensional sample space X. D is binary neural network classifier. While G learns generate samples that similar to the real data x, D learns to distinguish between "real" samples x and "fake" samples G(z). D and G are trained in alternating fashion minimize the following min-max loss (Goodfellow al.
2014
min max VDD.) =Eg--patt Dlog D(x)) + [lo(( D(G(=)))
To preserve the advantages of AE, still keep the mean square error to drive the abstraction mapping and self-organizing in the latent space. Meanwhile, we add a discriminator learning to distinguish between the samples generated by the decoder and real samples. Thus, the objective of AE-GAN is to minimize the following min-max loss:
min max V(D,F,G) Exeppace [log(D(x)) + Ee ( P.G.
(4)
Exnpaste [[(((z)) =112 -
As the start point of the search process, the output of the encoder is expected to be close to the op- timum code and thus help the search converge quickly. Since AE-GAN is trained with legitimate
4

---

Under review as a conference paper at ICLR 2020
images, the encoder F may suffer from the covariate shift when the input is corrupted with large perturbations. To solve this, we perturb the clean images with random Gaussian noise before feeding them into the encoder when calculating the reconstruction error Adding noise in the input layer acts as a regularization (Bishop, 1995), thus we denote the method as noise regularization (+r) this paper. The final objective function of AF-GAN,r be written Equation S
min max V.D,F.G) Exppaace (oaeiiaeeoogggeeo [log( E Es-p, [log D(G( F.G.
(5)
Exwpdata [[ +n) ==12:
where 77 ~ N(0, o), , follows the uniform distribution on given region.
Bidirectional Generative Adversarial Networks (BiGANs) (Donahue et al. 2016) also includes encoder-decoder mapping. In addition to the generator from standard GAN framework, BiGAN includes an encoder to map the input data into the latent representation. The discriminator in BiGAN discriminates jointly data and latent code, i.c., (z, E(x)) versus (G(z), 2)). However, experimental results show that BiGAN has poor reconstruction performance. When BiGAN equipped with the searching process, the encoder in BiGAN can not provide a good initial point for the decoder.
3.4 ENCODER ASSISTED SEARCH PROCESS
Although ,E-GAN,, has been trained with noise regularization, the added noise is simple while the adversarial noise is carefully crafted, the distribution between them is different. Moreover, when the input x² is seriously corrupted, e.g., a large region of the image is masked, the code derived from encoder F(x*) can hardly represent the original image æ. Therefore, correction for F(x*) is required when the input pattern is significantly different from the training data. Meanwhile, the end- to-end encoder-decoder framework can perform well against white-box attacks (Samangouei et al., 2018). Therefore, we include a search (denoted as +*) process the latent space in the inference stage.
As shown in Figure [ the core idea is that for every input sample AE-GAN,s will process search process the latent space to find the best code for input, which can lead the decoder to generate the closest reconstruction. For a given input we define the objective function J of the search process as:
((:) L(x, G(2)), for
(6)
where G(.) denotes the decoder, L(.) is measurement of the discrepancy between and G(z). We aims to find the optimum code 2 such that
arg min (() ZE2
Since our latent space is designed to be continuous, thus the direction and step size for the search process can be calculated by Gradient Decent (GD) method. Therefore, the search process con- verted to GD minimization minimize the discrepancy between the input and the reconstruction. Here we simply use euclidean distance on the pixel level to measure the discrepancy. more de- tailed description of the searching process is stated in Appendix C
In our framework, the encoder forms a self-organized latent space that preserves neighborhood re- lations and provides good start point. Both of them assist the search process in convergence. In Defense-GAN (Samangouei et al. 2018), the latent space follows a single Gaussian distribution while the gradient descent method is sensitive initial values. Therefore, requires multiple ran- dom restart points and considerable iterations to guarantee the performance, which brings a high computation cost.
EXPERIMENTS
4.1 SETTINGS
In this section, carry out experiments two image datasets: the MNIST LeCCnn et al. 1998) and F-MNIST (Xiao et al. (2017) dataset. Both them contain 60, 000 training images and 10. 000 testing images. We split the original training by 20:1 for training and validation, and
5

---

Under review as a conference paper at ICLR 2020
Classification accuracies of classifier A using various defense strategies the MNIST (top) and F-MNIST (bottom) dataset, under FGSM (e = 0.3), PGD (e = 0.3) and CW (with L2 norm) white-box attacks, gray-box attacks, and FGSM black-box attacks. Here the corresponding iterations of GD minimization are: 15 for our method, 60 for BiGAN, and 200 for Defense-GAN. (Adv.Tr.: adversarial training with PGD (c 0.3))
Table 1:
Attack
Method
None
Our
Adv. Tr.
MagNet
B
Defense-Gan
FGSM
0.144
0.984
0.949
0.293
0.912
0,981
White
PGD
0.007
0.982
0.920
0.012
0.924
0.989
CW
0.008
0.979
0.772
0.013
0.905
0.981
FGSM
0.144
0.882
0.561
0.731
0.880
Gray
PGD
0.007
0.910
0.673
0.739
0.913
CW
0.008
0.915
0.881
0.805
0.918
A/B
0.701
0.935
0.971
0,464
0.701
0,928
Black
A/C
0.813
0.933
0.970
0.590
0.632
0.922
Attack
Method
None
Our
Adv. Tr.
MagNet
BiGAN
Defense-Gan
FGSM
0.073
0.804
0.739
0.199
0.613
0.814
White
PGD
0.028
0.816
0.717
0.071
0.653
0.852
CW
0.062
0.767
0.308
0.084
0.601
0.810
FGSM
0.073
0.607
0.304
0.570
0.389
Gray
PGD
0.028
0.736
0.427
0.605
0,491
CW
0.062
0.719
0.646
0.608
0.435
A/B
0.227
0.603
0.784
0.308
0.398
0.602
Black
A/C
0.295
0.507
0.777
0.267
0.348
0,461
keep the original testing set as our testing set. Besides, we randomly choose 2000 samples from the testing set as development dataset, shortly dev-set, on which we conduct some exploratory experiments. For the architecture and training of the target classifier A. we follow the settings in et al. 2018) and the details are described in Appendix After training, we get an accuracy of 99.0% and 91.8% on MNIST and F-MNIST respectively, which near state of the art. For the architecture and training details of AE-GAN, adopt the settings in Defense-GAN Samangouei et al. 2018) and restate them in Appendix B with a detailed description.
Specifically, we consider the white-box and gray-box attacks by FGSM (Goodfellow et al. 201 4b)b) PGD (Madry et al. (017 and Carlini- WWgner (CW) attack with & norm (Carlini & Wagner 2016), and black-box attacks FGSM. The training procedure of the substitute model in black- box attacks is the same as the setting in (Papernot et al.) 2017). We split the testing set into a small hold-out set of 1,50 samples for training the substitute model and the remaining 9850 samples for testing different Cleverhans (Papernot
defense strategies. For all the attack methods, we use the implementation al.) 2018). python library benchmark machine learning systems' vulnerability to adversarial examples, and enforce the image to remain within [0, 1| by clipping.
4.2 RESULTS ON ADVERSARIAL ATTACKS
We compare our method to PGD adversarial training (Madry et al. 2017b), MagNet (Meng & Chen 2017), Defense-Gan (Samangouei et al. 2018). BiGAN (Donahue et al. 2016) under the FGSM, PGD and CW (with l2 norm) white-bo box and gray-box attacks, as well as the FGSM black-box attacks. Our reported method is AE-GAN i... AE-GAN with noise regularization and search process, where the number of iterations = 15. We also assist BiGAN with the search process for fair comparisons, which is denoted BiGAN... Reported Defense-GAN follows the original set-

---

Under review as a conference paper at ICLR 2020
tings where iterations L 200 and restart points R = 10. All experimental results are summarized in Table
observe that adversarial training successfully defends against the FGSM and PGD attack, but can not generalize to CW attacks. Because it needs adversarial examples to retrain the classifier. As discussed in Section 3.3 on small scale perturbations like CW the performance of MagNet and AE-GAN,,s is close, because the code derived from the encoder is robust to small perturbations. However, on large scale perturbations like FGSM (E 0.3) and PGD (e 0.3), the reported AE-GAN,.. can achieve about 20% ~ 30% improvements than MagNet.
From Table 1
As shown in Table D adversarial training and MagNet can only perform well under specific situa- tions, while Defense-GAN and our method can defense against various attacks. Attack -atack-ggnostic is important for defense strategy because we have no information about the attacker in the physical world. While our method reaches comparable performance with Defense-GAN under white-box and black-box attacks, we significantly reduce the computation complexity to 15 15/2000 of Defense- GAN. Another observation is that when the performance both decreases on the F-MNIST dataset due to the large perturbations, our method performs much better than Defense-GAN on gray-box attacks. The possible reason is that images in F-MNIST are more complex and require more details, which raises the difficulty of searching the best code in the latent space from random points. For white- attacks, the performance of our method has a slight drop from Defense-GAt studied in (Samangouei et al. 2018) that more iterations can increase the robustness for Defense- GAN against GD-based woite-box attacks. When we decline L from 200 to 15 for Defense- GAN, the performance drops by 8% decline. We can also observe that with fewer iterations, AE-GAN.rr still performs much better than BiGAN.s under various attacks. Though BiGAN., also includes an encoder-decoder mapping, the encoder of BiGAN,, can not provide a good start point for the searching process. This is due the poor reconstruction problem of BiGAN, which will be further studied in Section 4.5
However, it has been
Table also shows that the performance of defenses against gray-box and black-box attacks on the F-MNIST dataset is noticeably lower than on MNIST. This is due to the large perturbations generated by the FGSM attack with = 0.3. It is more difficult to find an accurate reconstruction for the seriously perturbed image, as shown in Figure 10 in Appendix
4.3 TRADE-OFF BETWEEN COMPUTATION AND PERFORMANCE
We further investigate the trade-off between computation cost and performance in Defense-GAN and our method. Table 2 shows the robustness of Defense-GAN and AE-GAN+s when changing the number of restart points R and iterations L. It can be seen that with fewer restart points, the performance of Defense-GAN decreases a lot after a certain value, which means that the effect of varying R is extremely pronounced. This is due to the non-convex nature of MSE and increasing R enables Defense-GAN to sample different local minimum (Samangouei et al.] 2018). However, in our framework, taking the output of the encoder as the initial start point can enable the search process to reach a reasonable local minimum in a few iterations. When we decrease L from 60 to 10, there is 6% 10% drop for the accuracy of Defense- GAN, while there little change in the performance of AE-GAN... Therefore, compared with Defense-GAN, our method requires less computation than Defense-Gan GAN when they reach comparable performance. The performance of our is robust to the computation.
4.4 DETECTION
A trained AE-GAN,, can well reconstruct natural images without searching. From this observation, we propose to introduce the detection mechanism to bypass the searching process for clean images. In AE-GAN,,, the decoder is trained to produce images that resemble the legitimate data. Thus, the adversarial examples usually have a high reconstruction error. Therefore, we can choose the mean square error (MSE) between input and reconstruction as indicator to decide whether or not the image is adversarial. The setting for the threshold 77 is important for the detection. A larger threshold can tolerant more perturbations and enhance efficiency but reduce security.
To investigate the performance of detection, we compute the reconstruction error for every image in the testing dataset, and their corresponding adversarially perturbed images under different strengths
7

---

Under review as a conference paper at ICLR 2020
Table 2: Classification accuracy of classifier A using different defense strategy test set of M- NIST (left) and F-MNIST (right), under FGSM (e 0.3) attack, with different GD iterations 10, 60, 200 and various numbers of restart points R = 10.
R = 10
Attack
Method
L
10
L 60
L 10
L= 60
Our
0.982 0.837
0.985 0.841 0.841
Clean
Defense-Gan
0.545%,0.45
0.693 \0.530
0.909 \ 0.696
0.967 0.788
Our
0.984 0.779
0.985 0.985%,.804
White
Defense-Gar
0.537) 0.403
0.688 0.688)0.527
00.527
0.905\0.667
0.964)0.787
Our 0.880
0..604
0.884 0.595
Gray
Defense-Gan
0.398 002296
0.570
0..225
0.769\ 0.441
0.863)
3.4
220
605
15
(b)
(C)
Figure 3: Detections MNIST. Left: ROC curves when using AE-GAN,r to detect FGSM attacks with various e. Middle: The False Negetive Rate and False Positive Rate when using different thresh- olds to detect FGSM attacks with various e. Right: Detection performance on different strength of FGSM attacks on MNIST dataset and the purification performance of AE-GAN,
of FGSM attacks. Figure (a) shows the Receiver Operating Characteristic (ROC) curves well as the Area Under the Curve (AUC) metric, which indicates that the detection is effective especially for large perturbations. When the threshold ry increases, we calculate the False Negative Rate (FNR, percentage of missed diagnosis) and False Positive Rate (FPR, the percentage of false alarms) and visualize how FNR and FPR changes with 7) under different strengths attacks in Figure The results show that a smaller threshold corresponds to lower FNR but higher FPR, which makes the detection safer but may cause false alarms. Mcanwhile, when the perturbation is small, is hard to detect attacks at both low FNR and FPR, but AE-GAN. is good purifying small perturbations, which can make up this shortcuts, as shown in Figure 3 (c). We also test the detection performance when using La loss the indicator, which is shown to be more effective. It reach both small FNR and FPR even under small perturbations. For more details, please refer to Appendix F
In summary, for small perturbations which pass the detection, AE-GAN, can successfully eliminate them. For larger perturbations that are detected as adversarial, they will trigger the searching stage and be purified by AE-GAN for achieving higher accuracy.
4.5 ABLATION STUDY
To investigate the effect of each module our method, we compare the performance of different models under the FGSM gray-box attack and half- attack (half of the input image is blocked, as shown in Figure 5 (c) in Appendix E Table 3 shows the defense performance of the different models different image corruptions visualized in Figure 5 where the number of iterations L is set to be 20 for models with search process.
observe that noise regularization has an expected effect forming more accurate latent code for the corrupted input. Secondly, the accuracies of AE,, and AE-GAN,, well as the qualitative results in Figure (see
Firstly, by comparing the performance of AE-ANN and AE-GAN,,,
8

---

Under review as a conference paper at ICLR 2020
Table 3: Classification accuracies of classifier A using various defense strategies on the MNIST dev- set, under FGSM (e = 0.3), CW (with 2y norm) gray-box attack and Half-Masked (H.M.) attack. (A.G.: AE-GAN; B.G.: BiGAN; +s: includes search process; +r. with noise regularization.
D
AE
AE,,
B.G.
B.G
A.G.
A.G.4s
A.G.
A.G.
FGSM
0.515
0.678
0,428
0.688
0.502
0.781
0.806
0.832
CW
0.861
0.852
0.477
0.749
0.868
0.887
0.870
0.889
H.M
0.552
0.672
0.408
0.590
0.494
0.807
0,570
0.776
Appendix F reveals that the introduced discriminator and adversarial loss improve the generative ability of the decoder. Thirdly, also look the consistency between input and output of BiGAN. As illustrated in Table 3 the defense performance of BiGAN is not good which indicates that the output of the encoder sometimes inconsistent with the original images, thus the start point can not assist the searching process BiGAN,, to reach a local minimum within a certain number of steps.
CONCLUSION
We proposed AE-GAN.s. a framework to effectively defend against various adversarial attacks. Our method takes the advantages of both auto-encoder and GANs and avoids their drawbacks. We build a reconstruction network following the encoder-decoder framework, as well as introduce the search process in the latent space to find the closet reconstruction without perturbations to the giv- en image. Compared with auto-encoder based methods, we can strongly defend against white-box attacks. Compared with GANs method, which requires multiple random restart points and con- siderable iterations to purify a sample, the encoder in our framework can help the search process converge quickly. We empirically show that Defense-GAN and our method can effectively defend against various attacks, while other methods have many shortcomings on at least one type of at- tack. Meanwhile, we can achieve comparable even better performance than Defense-GAt with significant reduction of computations.
The success of AE-GAN relies on both the representative power of the encoder and the generative power of the generator. However, as stated in Defense-GAN (Samangouei et al. 2018). training GANs is still challenging task and an active area of research. Meanwhile, obtaining a powerful auto-encoder on large dataset is also challenging. Besides this, we hope the proposed method provides a way to reducing the computations in searching based defense methods.
REFERENCES
Anish Athalye, Nicholas Carlini, and David A. Wagner. Obfuscated gradients give false sense of security: Circumventing defenses adversarial examples. CoRR, abs/1802.00420,, 2018. URL
http:/ /ttp://aavvvvveaa/ org.o/gg/a/...... abs 1802 00420
Chris M Bishop. Training with noise is equivalent tikhonov regularization. Neural computation, 7(1):188-116, 1995.
Nicholas Carlini and David A. Wagner. Towards evaluating the robustness of neural networks. CoRR, hbs/1608.04644 2016. http::////axxx org/ taabs/600 1608. 04644
Nilaksh Das, Madhuri Shanbhogue, Shang-Tse Chen, Fred Hohman, Li Chen, Michael E Kounavis, and Duen Horng Chau. Keeping the bad guys out: Protecting and vaccinating deep learning with jpeg compression. arXiv preprint arXiv:1705.02900 2017.
Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell. Adversarial feature learning. arXiv preprint 1rXiv:1605.09788 2016.
Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Ar- jovsky, and Aaron Courville. Adversarially learned inference, 2016.

---

Under review as a conference paper at ICLR 2020
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M Roy. A study of the effect of jpg compression on adversarial images. arXiv preprint arXiv: 100::000800,,, 2016.
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014a.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014b.
Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068, 2014.
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defense: Ensembles of weak defenses are not strong. In //th {USENIX) Workshop on Offensive Technologies ( (WWOOT) 17), 2017.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning at scale. arXiv preprint arXiv: 1.01236, 2016a.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial exampess in the physical world. CoRR, abs/1607.025533 2016b. HtRLhtp::/// http://wwww.rr//////1....00
Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324 Nov 1998.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. CoRR, abs//706.06008 2017a.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017b.
Dongyu Meng and Hao Chen. Magnet: proppodd defense against adversarial examples. CoRR, abs/ 705.09064, 2017.
/ttp://aww.....r//////15...... 09064
abs/1705.
Seyed-Mohsen Moosavi-Dezfoole Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks. In Proceedings of the IEEE conference com- puter vision and pattern recognition, pp. 2574-2582, 2016.
Margarita Osadchy, Julio Hermandez-Castrr Stuart Gibson, Orr Dunkelman, and Daniel Pérez- Cabo. No bot expects the deepcaptcha! introducing immutable adversarial examples, with appli- cations to captcha generation. IEEE Transactions on Information Forensics and Security, 12(11): 2640-2653, 2017.
Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversarial perturbations against deep neural networks. CoRR, abs/1511.00 1.0508, 2015. URL /tB.l//ttpp//a.x...cr//////
C4508
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM Asia Conference on Computer and Communications Security, ASIA CCS '17, pp. 506-519, New York, NY, USA, 2017. ACM. ISBN 1777177--533344444444 doi: 10.1 10.11457205277333330009 URL http: //:: acm. htt..oog/1..111/33522222..
10

---

Under review as a conference paper at ICLR 2020
Nicolas Papernot, Fartash Faghri, Nicholas Carlini, Ian Goodfellow, Reuben Feinman, Alexey Ku- rakin, Cihang Xie, Yash Sharma, Tom Brown, Aurko Roy. Alexander Matyasko, Vahid Behzadan, Karen Hambardzumyan, Zhishuai Zhang. Yi-Lin Juang, Zhi Li, Ryan Sheatsley, Abhibhav Garg, Jonathan Uesato, Willi Gierke, Yinpeng Dong, David Berthelot, Paul Hendricks, Jonas Rauber, and Rujun Long. Technical report the cleverhans v2.1.0 adversarial examples library. arXiv preprint arXiv:1610.000 000668, 2018.
Konda Reddy Mopuri, Utkarsh Ojha, Utsav Garg, and R Venkatesh Babu. Nag: Network for ad- versary generation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 742-751, 2018.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense- gan:: Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Represen- fations, 2018.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint
2017.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013, URL htttp//arxiv.co org/
6199
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mniss a novel image dataset for benchmark- ing machine learning algorithms, 2017.
Lei Xu. An overview and perspectives bidirectional intelligence: Lmser duality, double ia har- mony, and causal computation. IEEE/CAA Journal of Automatica Sinica, 6(4):865-8)) 8833, 2019.

---

Under review as a conference paper at ICLR 2020
A IMPLEMENTATIONS
For adversarial training, we follow (Madry et al., 2017b) and augment the training dataset with adversarial samples generated by PGD. We use the same for training and testing and set 0.3
for MNIST and F-MNSIT, c = 0.1 for CelebA dataset.
For GAN, adopt the implementation of original work (Samangouei et al. 2018).
For BiGAN, the original implementation (Donahue et al. 2016) is based on the fully-connected network. For fair comparisons, adopt the convolutional architecture in another similar work
(Dumoulin et al. (016). We slightly adapt the architecture on CIFAR-10 (Krizhevsky et al. 2009) by changing the input channel from 3 to 1. As shown in Figure the model trained reach fair performance on generating samples.
9
7
4
6
6
3
4
2
1
8
8
S
7
3
7
6 o o 3 9 5
|
6
1 2 0 0 4 5
9 2
9
11725
2
f
8
7
0
0
0
8
7
3
9
5
8
4
4
2 3
9 7 5
2
0
8
5
3
6
3
/
0
4 4
0
3
0
6
0 7
3
7
6
4
8
l
2
3 3
7 0
5
4
I
3
7
2
(
(b)
Figure 4: Generated samples by BiGAN trained MNIST (left) and F-MNIST (right).
For MagNet, follow the implementation in Defense-GAN (Samangouei al. 2018) and restate the details in Table which shares the same architecture with AE-GAN. However, we find that the latent space dimension 128 is too large for MagNet eliminate strong perturbations. Because the details of noise are also encoded and can be reconstructed. Thus, we also take the latent dimension as 20 for comparisons. The results in Table 8 that the change can enhance the performance on the gray-box attack.
B NEURAL NETWORK ARCHITECTURES AND TRAINING SETTINGS
For fair comparisons, we compare different strategies through this paper on the same classifier A, and we take two substitute classifiers for black-box attacks, B for convolutional structure and C for fully connected structure. The training settings and architectures almost follow the settings in Defense- DefeenseGGANSaamaaeeee (Smmangougo
al., 2018). For clarification, here restate their architectures and training settings in Table 5 and Table respectively.
The architecture of model and MagNet is described in Table which make slight modification on the MagNet structure given in (Samangouei et al. 2018
Table Training parameters for LE-GAN and MagNet
Parameters
MNIST, F-MNIST
CelebA
Epochs Learning Rate
60
100
0,0002
0.0002
Optimization Method
Adam
Adam
Batch Size
100
64
12

---

Under review as a conference paper at ICLR 2020
Table 5: Neural architectures used for classifiers and substitute models
A
B
C
Conv(64, 5, 1)
Conv(64,
2)
ReLU
ReLU
EC(200)
Conv(64, 5, 2)
Conv( C1nv(128, x 6, 2)
ReLU
ReLU
ReLU
Dropout(O.5)
Dropout(0.25)
Conv (128, x 5, 1)
FC(200)
FC(128)
ReLU
ReLU
ReL.U Drpupout(O.S)
Drpupout(0.5) FC(10)+Softmax
Dropout(0.5)
FC(10)+
FC(10)+Softmax
Table 6: Neural architectures used for AE-GAN and MagNet
Encoder
Decoder
Discriminator
Conv(1, 64, x 5, 2) LeakyReLU(O.2)
FC(1024) & ReLU
Conv(1, 64, x 5, 2) Lea.)))LU(O.2)
ConvT(256, 128, 2)
Conv(64, 128, 5 x 5, 2)
BatchNorm & ReLU
Conv(64, 128, 5 x 5, 2)
BatchNorm & LeakyReLU Conv(128, 256, 5 x 5, 2)
ConvT(128, 64, 2)
BatchNorm & LeakyReLU
BatchNorm & ReLU
Cony(128, 256, 5 x 5, 2)
BatchNorm & LeakyReLU FC(h28)ttanh
ConvT(64, 5 5, 2)
BatchNorm & LeakyReLU
Sigmoid
FC(1)+Sigmoid
Table 7: Training parameters for classifiers
Parameters
MNIST, F-MNIST
Epochs
10
Learning Rate
0.001
Optimization Method
Adam
Batch Size
100
C DETAILED DESCRIPTION OF THE TRAINING AND INFERENCE STAGE.
Algorithm describes the training process of AB-GAN,, Algorithm describes the search process of AE-GAN +ss in the inference stage.
Algorithm Training process of AE-GAN
Notations: F: encoder, G: decoder; D: discriminator; 77: the bound of random noise while epoch < iteration do
(m)) from noise prior p(2) Sample minibatch of samples (2x11,, (2) .()),.((m) ((m)) from training set
Sample minibatch of noise samples {2(1), 2(2) ....,,(m))
Update the discriminator by ascending its stochastic gradient
V m D [log D (z()) log D1-D(G) (2( (2(1
Sample a from uniform distribution U(0. 17)
Sample minibatch of m noise samples {e( e(2) ..... (?2),..,,((()) from NO,
Update the decoder by descending its stochastic gradient
VA, Exe [log (1 D (G (z())) + (F(2(1) - (x(() + ++(0)+++)))))) (+))
Update the encoder by descending its stochastic gradient.
Vo_ 1 Σm (F(20)) - (x + 10)
end while
13

---

Under review as a conference paper at ICLR 2020
Algorithm 2 Search process of AE-GAN.rs in inference stage
Input: input image æ; trained encoder F, trained decoder G, number of steps I, initial step size 1) Optimal reconstruction G(zz) of
Output:
initiate 20
(x)
while &- < L do
2k 2k-1 + v/V (L2((())((z)) z))
if k mod 5 = then
" x 2 end if
end while
return G(*+)
D ADDITIONAL RESULTS ON VARIOUS LATENT DIMENSIONS
Table 8: Classification accuracy of A using MagNet and AE-GAN on MNIST with various latent
space dimensions. the dimension of latent space (the number of iterations is set 15).
D=20
D=128
Attack
Attack
None
MagNet
AE-GAN- -rs
MagNet
AE-GAN+ +5
FGSM
0.144
0.298
0.972
0,293
0.984
White
PGD
0.007
0.012
0.975
0.006
0.982
CW
0.008
0.013
0.970
0.008
0.979
FGSM
0.144
0.561
0.875
0.316
0.882
Gray
PGD
0.007
0.673
0,911
0,145
0.910
CW
0.008
0.881
0.907
0.345
0.915
E QUALITATIVE COMPARISONS BETWEEN AE,s AND AE-GAN+s
Figure shows the purification performance of AE., and AE-GAN., on corrupted images. We can easily that though AF,,, incorporates searching process, can find natural reconstruction to the given corrupted image. The comparison can show the effectiveness of discriminator and the adversarial loss we introduced AE-GAN.s.
4
3
5
4
7
4
4
9
(a)
(b)
(c)
Figure 5: Purified images on (a) MNIST and (b) F-MNIST dataset, under FGSM (c = 0.3) gray-box attack, well as (c) Hale-Masked attack F-MNIST. First row: the corrupted images; Second row: purified image by AE,s; Third row: purified image by AE-GAN,s; Last row: ground truth.
14

---

Under review as a conference paper at ICLR 2020
F ADDITIONAL RESULTS ON DETECTION
When using L1 loss as the indicator to decide whether image is adversarial or not, we can obtain better detection performance on MNIST dataset. As shown Figure 6 when the threshold set to 0.06, the detection can reach both small False Negative Rate and small False Positive Rate on small perturbations, while it is hard to achieve this with MSE, as shown in Figure 3 in the main text. By comparing Figure and Figure g we can get similar observations on F-MNIST dataset.
FAB), Epeilt
FMn1, Epnilcom0.70
Trual Phe Ta8e
21-
epsico UL ALC-- 301
Coutsrmo ALC-0.90
buskinnd AUC-1
00
3.5
02
0.1
5.0
10
0.00
0.22
0.24
0.50
0.05
2.30
0.32
0.54
Pasc
(a)
(b)
Figure 6: Detections on MNIST when using L1 distance the indicator. Left: ROC curves when using AE-GAN,, to detect FGSM attacks with various Right: The False Negetive Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various
Trual Pee Cadk
PA0h Aps 10-207
1pe.or-0.70
Fpulon-o 0.10 ALC-0.807
Eppatoned 13, AL 563
todico--p Aut-0 2077
Coutsrmm AUC-0
0.0
bastunad AUC-D
60
3.5
02
0.1
5.0
10
0.00
592
0.11
6:01
0.28
0.5
Pasc
(a)
(b)
Figure 7: Detections on F-MNIST when using L, distance as the indicator. Left: ROC curves when using AE-GAN,, to detect FGSM attacks with various e. Right: The False Negative Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various
G ADDITIONAL RESULTS ON CELEBA
To investigate the scalability of our method, we conduct experiments on CelebFaces Attributes dataset (CelebA) (Liu et al. 2015) and report the results in Table CelebA contains more than images for training, validation, and test- ing. Each image is center-cropped and resized to 64 x 64. Following the settings in Defense-GAN Samangouei et al. 2018). we perform the classification of gender on the dataset and the architecture of AE-GAN also adopts the implementation in Defense-Gar GAN for a fair comparison. The architec- ture of the encoder is the same as the discriminator except for modifying the dimension of the top layer. The accuracy of the trained classifier A the training dataset of CelebA is 95.96%.
200,000 face images and we extract 162770, 10000, 10000
15

---

Under review as a conference paper at ICLR 2020
30-
Than Prun--o Bale
FMn cp
PMR. tpsicr-o.2
PMR Rad:cmm.20
PAR. todi1-00.0
Bo ALC--0.0
Epeinn.n o.11 A.C 8.81
Oonicree 20, AUDeg 542
eadinnaazs, Auc-g 199
Courcned ALC
00
2.0
02
0.1
24
10
0.00
0.0
061
602
0.33
6.5
H838 Foobiie
iPrsshodd
(b)
Figure 8: Detections on F-MNIST when using MSE the indicator. Left: ROC curves when using AE-GAN,, to detect FGSM attacks with various e. Right: The False Negative Rate and False Positive Rate when using different thresholds to detect FGSM attacks with various
Table 9: Classification accuracies of classifier A using various defense strategies CelebA dataset, under FGSM (c = 0.1), PGD (c = 0.1) and CW (with L2 norm) white-box, gray-box, and FGSM black-box. Here the corresponding iterations of GD minimization are: 20 for method, 200 for Defense-GAN. (Adv.Tr. Tr.: adversarial training with PGD (e 0.1))
Attack
Method
None
Our
Adv. Tr.
MagNet
Defense-Gan
FGSM
0.0416
0.9364
0.6119
0.0750
0.8879
White
PGD
0.0404
0.9327
0.6119
0.0510
0.8942
CW
0.0404
0.9433
0.6119
0.0493
0.861
FGSM
0.0416
0.8872
0.8897
0.8559
Gray
PGD
0.0404
0.9005
0.9065
0.8621
CW
0.0404
0.9233
0.9273
0.867
Black
A/B
0.8325
0.8839
0.6121
0.8743
0.8517
H QUALITATIVE EXAMPLES
16

---

Under review as a conference paper at ICLR 2020
5 7 8
7
4
5
6
7
8 9 0
8
7
7 5
6 7
8 0 7 3
4 1 8
9 7
8 /
2
3
4
8 8 4 4 3 6
7
9
2 6 7 / 8 777744
2 8 6 9 /
9
3
O / 2 8783444
0 2
2
8
7
8
S 3 6 5
7 /
4060
3 3
3
9
/ 4 666
777
0 0
6
2
/
7 7 9
4
0
7
37
4
O
7
0
3
6 7
3 2 1
5
2
9
4
4 /? 3
3
3
3
0
|
1143
3
6
6
0
4 56
8
8
2
4 86
a
5
7
8
9
0
/
2
3
4
5 7 7
8 O
4 4 7 8 9 7
8
0
/
2
3
4
7 8 9 9
8 6 7 4
7 7 4 4 7
8
6
9 / 9
3
4 4
0 9 8 3
7 8 3 6 0
0
/ 8
8 2 260
G S 5 5
3 4 / 4 0400003000000
3 8 9 / 4 66
/ 0 0 6
2 / / 7 0 0 6 a / 0
7 8
4 0 7 0 3 8
4 6 0 7 0
3 6 8
3 3 9
/
5
2 4 9
4
3 4 /
3 / S 2
3
6
3 0
4
7 8 9 1 2 3
4 5
7
8
9
23
5
(b)
$ 9 0
2
7
4 5' 6
7
3
9
0
8 7
4/
5 6 7
8' O / 7
3 4 ? 8 9 7 8 0 / 2
3
4
7
8 9
B l 9 / 9 7 8 4 4 7 8 6 9
9
3
8 4 4
21945758600148875
O 1 9 9 5 060001011 9 8
7
8 2 6 0
" 3 4
4
8
/ d 406 6
/ 0 0 6 2 / / 7 7 3
7 7 9 3 6 7 4
0 6 2
4 4 2 0
3 4 8 7 / 4
0 7
0
5 1 4 9
4/ 7 4 4 / 7 5 2
3
/ 66
2
4 3 6
5
1
3
7 8
9
0
1
2
3
4 5 6 7 8 9
2
Figure 9: Example results of our method on MNIST. The left part shows adversarial examples
generated by different attacks: FGSM attack with = 0.3 (a), PGD attack with e = (b) and CW with l2 norm (c), while the right part shows purified images by AE-GAN The number of iterations for guided search set to 15.
17

---

Under review as a conference paper at ICLR 2020
Figure 10: Example results of our method on F-MNIST. The left part shows adversarial examples generated by different attacks: FGSM attack with = 0.3 (a), PGD attack with e = 3 (b) and CW with norm (c), while the right part shows purified images by AE-GAN The number of iterations for guided search set to 15.
18


Reviews:
[{"review_id": "rJxMJaDpcr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors propose to use a modified autoencoder at the input of a network to ward off adversarial samples by purifying them. The encoder-decoder is trained with a discriminator which can identify whether the input is malicious or normal. A malicious input image is purified with the help of a search procedure in the latent space of the auto-encoder. The gradient based search obtains a latent code that corresponds to the reconstructed image that is closest to the input. The algorithm uses output of the decoder as a starting point in the search. Experiments show a reasonable performance against adversarial attacks on MNIST and F-MNIST images. The proposal seems to be an alternative to the Defence-GAN method of (Samangouei et. al. 2018) where search is computationally expensive - due to multiple starting points and many iterations. The ease of optimization in the proposed method comes at the cost of accuracy (Table 1). However, for a single starting point during search and fewer iterations, it outperforms Defense-GAN by a significant margin (Table 2). ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "H1lFTg4w9B", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper combines an auto-encoder (AE) based approach to correct the adversarially perturbed samples with a GAN based approach for the same task. More specifically, the encoder of the AE is used to provide a better initialization for the latent space optimization employed in the Defense-GAN approach. \n\nThe authors propose a two-stage inference process. First, the autoencoder is used to detect if an input sample is from the natural image distribution or adversarial distribution. In the latter case, Defense-GAN is employed that uses gradient descent in the latent space, with encoder output as the initialization, to find a natural or non-adversarial counterpart of the input sample. Results are reported for MNIST and F-MNIST that show that the proposed method is computationally cheaper than Defence-GAN.\n\nQuestions / Concerns:\n\n - The methods seem to heavily rely on the autoencoder's ability to detect adversarial samples. The detection performance can be still susceptible to white-box attacks. The methods also relies on the capacity of the auto-encoder to learn good representations/reconstructions. It will be useful to show the performance on more complicated datasets was learning a good AE model is more challenging.   \n - The autoencoder is also equipped with an adversarial loss such that the decoder produces realistic images. Is there any assurance that the latent distribution F(x) from the data and the prior p(z) will be similar. It might be useful to explicitly ensure this. The discriminator currently never sees images that re reconstructions of the true data samples. So the decoder behavior could be different for different regions of the input space. \n- It is not clear why the proposed method performs better on stringer white-box attacks compared to much weaker black-box attacks. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Bkxa-E0hcr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper aims to refine DefenseGAN (ICLR 18), where an autoencoder is used to initialize the search for projecting an adversarial examples to the manifold of real examples. \n\nThe main contribution is to reduce the computational cost of DefenseGAN, the claim is \"by an order of magnitude\". \n\nThough the idea is good, I found the contribution to be too incremental for the paper to be accepted:\n* The comparison should include the state of the art adversarial training defenses, PGD Adversarial Training (Madry et al; you might want to cite the ICLR 18 paper) and TRADES (Zhang et al., Interpreting adversarially trained convolutional NN, ICML 2019);\n* I would consider the Szegedy baseline as obsolete;\n* Fig. 4 is unclear: the percentage of adversarial examples detected by the detector, but quid of the false alarms; \n\nThere are quite some typos (nosie; iterarion; tabel; unsafety) and missing words. The paper has been hastily written; and it includes one page more than allowed.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "B1lfFS_qFr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary: This paper proposes AE-GAN+sr, an auto-encoder based GAN for equipping neural networks with better defenses against adversarial attacks. The authors evaluate their method on black-box attacks, white-box attacks, and gray-box attacks on MNIST and Fashion-MNIST, and show decent empirical results when compared to baselines.\n\nDecision: Reject. The writing was hard to follow and the experimental evaluations could have been stronger.\n\nSupporting Arguments/Feedback: \n- There are a lot of phrases which should be reworded in the text (e.g. “it requires attention to” in paragraph 1, “detect if sample is from a normal distribution” at the end of Section 1, etc.) for better clarity. The writing was pretty hard to follow and the paper would really benefit overall after it has been improved.\n- It was hard for me to tell from the text how exactly the models were implemented and trained (see Question #1). Making this point more clear, in either the main text or the Appendix, would be really helpful.\n- The performance of AE-GAN+sr as evaluated on MNIST and FashionMNIST did not provide convincing evidence that it outperformed baselines such as Defense-GAN or adversarial training, though I did appreciate the authors’ additional analysis on the tradeoffs between computational cost and performance for the Defense-GAN. The results would also have been more compelling had the authors evaluated their method on more complex datasets such as CIFAR-10.\n\nQuestions:\n- It wasn’t clear to me how the encoder-assisted search process (Section 3.4) works in practice. When you use gradient descent to find the best encoding for a corrupted input, how many steps do you need to take? Does this step happen while the autoencoder and GAN are being trained, or are those pre-trained and you just take additional gradient steps in the latent space?\n- Given that the BiGAN also incorporates an encoder (and they were also benchmarked against in the experiments), where do the advantages of the AE-GAN+sr come from? I think making this point in the text would also be helpful as well.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a defense for adversarial attacks based on autoencoders that tries to find the closest point to the natural image in the output span of the decoder and "purify" the adversarial example. There were concerns about the work being too incremental over DefenseGAN and about empirical evaluation of the defense. It is crucial to test the defense methods against best available attacks to establish the effectiveness. Authors should also discuss and consider evaluating their method against the attack proposed in https://arxiv.org/pdf/1712.09196.pdf that claims to greatly reduce the defense accuracy of DefenseGAN. 
Decision: Reject