Generative Cleaning Networks with Quantized Nonlinear Transform  for  Deep Neural Network Defense
Authors: Jianhe Yuan
Zhihai He
Venue:  (2020)
Keywords: Adversarial Defense
Adversarial Attack

Abstract: Effective defense of deep neural networks against adversarial attacks remains a challenging problem, especially under white-box attacks.
In this paper, we develop a new generative cleaning network  with quantized nonlinear transform  for effective defense of deep neural networks.  The generative cleaning network, equipped with a trainable quantized nonlinear  transform block, is able to destroy the sophisticated noise pattern of adversarial attacks and recover the original image content. The generative cleaning network and attack detector network are jointly trained using adversarial learning  to minimize both perceptual loss and adversarial loss. Our extensive experimental results demonstrate that our approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. For example, it improves the classification accuracy for white-box attacks upon the second best method by more than 40\% on the SVHN dataset and more than 20\% on the challenging CIFAR-10 dataset. 

Under review as a conference paper at ICLR 2020
GENERATIVE CLEANING NETWORKS WITH QUAN- TIZED NONLINEAR TRANSFORM FOR DEEP NEURAL NETWORK DEFENSE
Anonymous authors Paper under double-bling blind review
ABSTRACT
Effective defense of deep neural networks against adversarial attacks remains a challenging problem, especially under white-box attacks. In this paper, we de- velop a new generative cleaning network with quantized nonlinear transform for effective defense of deep neural networks. The generative cleaning network. equipped with a trainable quantized nonlinear transform block, is able to destroy the sophisticated noise pattern of adversarial attacks and recover the original im- age content. The generative cleaning network and attack detector network are jointly trained using adversarial learning minimize both perceptual loss and adversarial loss. Our extensive experimental results demonstrate that our ap- proach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. For example, it improves the classification accuracy for white-box attacks upon the second best method by more than 40% on the SVHN dataset and more than 20% on the challenging CIFAR-10 dataset.
INTRODUCTION
Recent research has shown that deep neural networks are sensitive adversarial attacks (Szegedy et al.) 2013). Very small changes of the input image can fool the of-te-ofartt classifier with very high success probabilities. During the past few years, number of methods have been proposed to construct adversarial samples to attack the deep neural networks, including fast gradient sign (FGS) method (Goodfellow et al., 2014b). Jacobian-based saliency map attack (J-BSMA) (Papernot al.) 2016a), and projected gradient descent (PGD) attack (Kurakin et al., 2016 Madry et al.18) It has also been demonstrated that different classifiers can be attacked by the same adversarial perturbation (Szegedy et al. 2013). The fragility of deep neural networks and the availability of these powerful attacking methods present an urgent need for effective defense methods. During the past few years, a number of deep neural network defense methods have been developed, including adversarial training
(Kurakin et al. |0166] Szegedy et al. 2013), defensive distillation (Papernot et al. 201 Carlini & Wagner 2016 Papernot & McDaniel 2016), Magnet (Meng & Chen 2017) and featuring squeezing (He et al. 20177 Xu et al. (017)) It has been recognized that these methods suffer from significant performance degradation under strong attacks. especially white-box attacks with large magnitude and iterations (Samangouei et al. 2018).
In this work, we explore a new approach to defend various attacks by developing a generative clean- ing network with quantized nonlinear transform. We recognize that the attack noise is not random and has sophisticated patterns. The attackers often generate noise patterns by exploring the spe- cific network architecture or classification behavior of the target deep neural network so that the small noise at the input layer can accumulate along the network inference layers, finally exceed the decision threshold at the output layer, and result in false decision. On the other hand, we know well-trained deep neural networks are robust to random noise (Arjovsky et al., (017). such as Gaus- sian noise. Therefore, the key issue in network defense is randomize destroy the sophisticated pattern of the attack noise while preserving the original image content.
Motivated by this observation, design new generative cleaning network with quantized nonlin- ear transform to first destroy the sophisticated noise patterns of adversarial attacks and then recover the original image content damaged during this nonlinear transform. We also construct detector

---

Under review as a conference paper at ICLR 2020
network which serves the dual network for the target classifier to be defended. The generative cleaning network and detector network are jointly trained using adversarial learning so that the de- tector network cannot detect the existence of attack noise pattern in the images recovered by the generative cleaning network. Our extensive experimental results demonstrate that our approach out- performs the state-of-art methods by large margins in both white-hox and black-box attacks. It significantly improves the classification accuracy for white-box PGD attacks upon the second best method by more than 40% on the SVHN dataset from 46.90% 93.80% and more than 20% on the challenging CIFAR-10 10 dataset from 60.15% to 86.05%.
The major contributions of this work can be summarized as follows. (1) We have proposed a new approach for deep neural network defense by developing a unique generative cleaning network with quantized nonlinear transform. (2) We have formulated the problem of destroying the noise patterns of adversarial attacks and reconstructing original image content into generative adversarial network design and training which considers both perceptual loss and adversarial loss. (3) Our new method has significantly improved the performance of the state-of-the-art methods in the literature under wide variety of attacks.
The rest of this paper is organized as follows. Section 2 reviews related work. The proposed method is presented in Section 3. Experimental results and performance comparison with existing methods are provided in Section 4. Section 5 concludes the paper.
2 RELATED WORK
In this section, we review related work on adversarial attack and network defense methods.
(A) Attack methods. Attack methods can be divided into two threat models: white-box attacks and black-box attacks. The white-box attacker has full access to the classifier network parameters, net- work architecture, and weights. The black-box attacker has no knowledge of or access to the target network. For white-box attack, simple and fast approach called Fast Gradient Sign (FGS) method has been developed by Goodffellow et al. (2014b) using error back propagation to directly modify the original image. Basic Iterative Method (BIM) is an improved version of the FGS method. Car- lini & Wagner (0166) designed an optimization based attack method, called Carlini- Carlini-Wagner (C& W attack, which is able to fool the target network with the smallest perturbation. X al. (2018)
trained a generative adversarial network (GAN) (Goodfellow et al., 2014a) to generate perturba-
tions. Kamnan al. (2018) found that the Projected Gradient Descent (PGD) the strongest among all attack methods. It can be viewed as a multi-step variant of FGS* (Madry et al. 2018). Athalyy al.) (2018) introduced a method, called Backward Pass Differentiable Approximation (BPDA), to attack networks where gradients are not available. It is able to successfully attack all existing ofate-of-the--atss defense methods. For black-box attacks, the attacker has no knowledge about target classifier. Papernot et al. (2017) introduced the first approach for black-box attack using a substi- tute model. Dong et al. 2018) proposed a momentum-based iterative algorithms to improve the transferability of adversarial examples. Xie et al. (2018c) boosted the transferability of adversarial examples by creating diverse input patterns.
(B) Defense methods Several approaches have recently been proposed for defending both white- box attacks and black-box attacks. Adversarial training defends various attacks by training the target model with adversarial examples (Szegedy al. 2013 Goodfellow et al., 2014b). Madry et al. (2018) suggested that training with adversarial examples generated by PGD improves the ro- bustness. Meng & Chen (2017) proposed a method, called MagNet, which detects the perturbations and then reshape them according to the difference between clean and adversarial examples. Re- cently, there are several defense methods based on GANs have been developed. [Samangouei et al.) (2018) projected the adversarial examples into a trained generative adversarial network (GAN) to approximate the input using generated clean image with multiple iterations. Recently, some defense methods have been developed based on input transformations, Guo et al. (2018) proposed several input transformations to defend the adversarial examples, including image cropping and re-scaling, bit-depth reduction, and JPEG compression. Xie et al. (2018a) proposed to defend against adversar- ial attacks by adding a randomization layer, which randomly re-scales the image and then randomly zero-pads the image. Jia et al. (2019) proposed an image compression framework to defend adver- sarial examples, called ComDefend. Xie et al. (2018b) introduced a feature denoising method for defending PGD white-box attacks.
2

---

Under review as a conference paper at ICLR 2020
Quantized Nonlinear Transform
Detector Network
Clean / Attacked
Discriminative
X" Attacked Image
GeneratiweCleaning Network Go
X
Loss LD
Original Clean Image.J x
Target Classifier
Generative Clean Image X",
Fp
Lg Norm
FB
Clean Images
Ca
Perceptual Loss Lp
Figure 1: Overview of our generative cleaning network method for deep neural network defense.
Our proposed defense method is also related to GANs and image transformations, But, compared to existing methods, our method is unique in the following aspects: (1) We introduce a special layer called quantized nonlinear transform, into the generative cleaning network destroy the sophisti- cated noise pattern of adversarial attacks. (2) Unlike the GAN-based methods in (Wang & Yu 2019 Xiao al. 2018) which aim to approximate input noise image using images generated by the GAN over multiple iterations, our generative cleaning network aims to reconstruct the image content dam- aged by quantized nonlinear transform. (3) Our method does not need to modify the target network to be protected.
THE PROPOSED DEFENSE METHOD
In this section, present proposed generative cleaning network method for effective deep neural network defense. For convenience, we refer to our proposed method by GCLN.
3.1 METHOD OVERVIEW
Figure T provides an overview the proposed method. The attacked image x" is fed into the gener- ative cleaning network Gg. The network has special layer, called quantized nonlinear transform, to destroy the noise pattern of the adversarial attack in the input image. The generative cleaning network aims recover the original image content and produce recovered image T. This recov- recognition. To
ered image E will be passed to the target classifier C. for image classification successfully learn the generative cleaning network Ge. we construct detector network D which serves as the dual network for the target classifier network C,,, The task of D is to determine if the input image is clean or being attacked. our proposed method, the generative cleaning network Go and the detector network D are jointly trained through adversarial learning: the Go network trying to recover the image so that D e cannot detect any attack noise it. the following sections, we will explain the proposed method in more detail.
3.2 QUANTIZED NONLINEAR TRANSFORM LAYER THE GENERATIVE CLEANING NETWORK
During the generative cleaning network design, We incorporate one special layer into the network, called quantized nonlinear transform. This transform aims disturb and partially destroy the phisticated pattern of the attack noise. In this work, we propose to construct such a transform using linear transform T. followed by quantizer Q and inverse transform T-1 For the linear transform, we can use the discrete cosine transform (DCT) (Ahmed et al., 1974) which has been in JPEG image compression (Wallace) 1992). Specifically, of M x M. The original image block is denoted by X; B = [E The output block X [MylıseJıM after DCT transform is given by
partition the input image into blocks
M-1M-1
B: = C.C, Σ
Σ
Enk cos(in + cos(jn
2k
(1)
2M
2M
==0
0=0

---

Under review as a conference paper at ICLR 2020
with C, 1/v2 for 0, and C; 1 for i + 0. After transform, will quantize the transform coefficient Pay as follows
RQ( = Round
x q
(2)
where g is the quantization parameter. Certainly, this DCT transform can be replaced with other in- vertible transform, such as discrete wavelet transform (Daubechies) 1990). During network training, this special quantized nonlinear transform layer is implemented in the same way as the pooling lay- ers in existing deep neural networks and included into the training process of the whole generative cleaning network.
3.3 ADVERSARIAL TRAINING FOR GENERATIVE CLEANING NETWORKS
In our defense method design, the generative cleaning network Gg and the detector network D trained against each other, just like the existing generative adversarial networks (GAN). D is binary classifier to detect if the input image is clean or not. During the initial phase of training, D trained with the clean images and their attacked versions generated by existing attack methods. should be noted that, when training D we do not need to know the model inside the target network C..
The goal of the generative cleaning network Ga is two-fold: (I) first, it needs to successfully remove the residual attack noise so that the noise cannot be detected by the detector network D g. (2) Second, it needs to make sure that the original image content is largely recovered. To achieve the above two goals, we formulate the following generative loss function for training the generative cleaning network G
LG = ALp + (1-A)LA:
where Lp is perceptual loss and LA is the adversarial loss. À is weighting parameter. In our experiments, we set it to be 0.5. To define the perceptual loss, the L2-norm between the recovered image E and the original image x" is often used (Johnson et al. (016)). In this work, observe that the small adversarial perturbation often leads to very substantial noise in the feature map of the network (Xie et al. 2018b). Motivated by this, Fg to generate visual features for the recovered image 7 and the original image x", and use their feature difference the perceptual loss Lp. Specifically,
a pre-trained VGG-19 network, denoted by
Lp (xx) F (G)())))
(4)
The adversarial loss LA aims train G. that the recovered images will be detected clean by the detector network Dg. It is formulated as
LA- Egg en + (Go((((") Lelea]]
(5)
Here, + represents the cross-entropy between the output generated by the generative network and the target label Ielan for clean images. We train our discriminative network DØ- along with the generative cleaning network Gg, to optimize the following min- max loss function:
min max EEr [log Dø(x") + Ex+en- [log(1 D& ())]- D
(6)
Here, Ω2 and Ω2 represent the clean and attacked images of the training dataset. The goal of gener- ative model Gg is to fool the discriminator D& that is trained to distinguish adversarial images from clean images. With this framework, our generator learns recover images that highly similar to clean images and difficult to be detected by D The detector network D acts as a dual network for the original classifier C., Cascaded with the generative cleaning network Gg, it will guide the training of G& using back propagation of gradients from its own network, aiming to minimize the above loss function. In our design, during the adversarial learning process, the target classifier C. is called to determine if the recovered image E is clean or not, as illustrated in Figure I If it is clean, it is added back into the clean training sample set I2" on the fly enhance the learning process.
EXPERIMENTAL RESULTS
In this section, implement and evaluate our GCLN defense method and compare its performance with state-off oh-the-art defense methods under a wide variety of attacks, including white-box box and black- box attacks.
4

---

Under review as a conference paper at ICLR 2020
Table 1: Performance of our method against white-box attacks on CIFAR-10 dataset (c 8/256).
Defense Methods
Clean
FGS
PGD
BIM
C&W
No Defense
94.38%
31.89%
0.00%
0.00%
0.99%
Adversarial BIM
87.00%
52.00%
32.00%
42.00%
Label Smoothing (Warde-Farde 2016
92.00%
54.00%
8.00%
2.00%
Feature Squeezing (Xu al. 2017)
84.00%
20.00%
0.00%
78.00%
PixelDefend (Song el al., 2018)
85.00%
46.00%
70.00%
80.00%
Adversarial- PGD (Trmmr et al. 2018)
83.50%
67.92%
60.15%
Adversarial network [Wang & Yu 2019
91.32%
73.77%
49.55%
Our method (Qs 5)
91.65%
75.03% 75.22%
85.84%
86.39% 86.43 %
85.47%
Our method (Qs 10)
91.65%
86.05%
86.42%
Table
BPDA attack results on CIFAR-10 dataset. adversarial training.
Results with are achieved with additional
Defense Methods
Distance
Accuracy
TE (Buckman et al. 2018
0.031 (Ixa)
0%"
SAP SAP (Dhilon et al. 2018
0.031 (Loo)
0%
LID (Ma al. 2018)
0.031 (Loo)
5%
PixelDefend (Song et al. 2018
0,031 (Loo)
9% *
Cascade Adversarial Training (Na a..
2018
0.015 (I.))
15%
PGD Adversarial Training (Madry et al.
2018
0.031 (Ios)
47%
STL Sun et al. 2019
0.031 (Loe)
42%
Our method (Qs 5)
0.031 (Ixa)
51%
Our method (Qs = 10)
0.031 (Loo)
53%
4.1 EXPERIMENT SETUP
Following existing methods in the literature, we use CIFAR-10 and SVHN (Street View House Number) datasets. The CIFAR-10 dataset consists of 60,000 images in 10 classes, with 32x32 image size. The Street View House Numbers (SVHN) dataset (Netzer et al.) 2011) has about 200K images of street numbers. The attack methods be considered in this work include FGS (Goodfellow al. 2014b), PGD (adry et al. 2018), BIM attack (Kurakin et al.) 2016), and C&W W attack (Carlini & Wagner 2017
4.2 RESULTS ON THE CIFAR- 10 DATASET
We compare the performance of our defense method with 6 -tate-of-the-art methods developed in the literature under four different white-box attacks: FGS attack (Goodfellow et al. 20145). PGD (Madry et al.) 2018) attack, BIM attack (Kurakin al. (016) and C&W attack (Carlini & Wagner, 2017). Following (Kannan et al. 2018) and (Wang & Yu 2019). the white-box attackers generate adversarial perturbations within a range of 8/255. In addition, set the step size of attackers to be 1/255 with 10 attack iterations as the baseline settings. Table T image classification
Table 3: Performance of our method against black-bor attacks on CIFAR- (c = 8/256..
Defense Methods
No Attack
FGS
PGD
No Defense
94.38%
63.21%
38.71%
Adversarial-PGD Tramr al. 2018)
83.50%
57.73%
55.72%
Adversarial network (aan & Yu 2019)
91.32%
77.23%
74.04%
Our method (Qs = 5)
91.65%
77.01%
77.90%
Our method (Qs 10)
91.65%
77.56%
79.81
5

---

Under review as a conference paper at ICLR 2020
Table 4: Performance of our method against attacks SVHN dataset(e 0.05).
White-box Attack
Black-box Attack
Defense Methods
No Attack
PGD
PGD
No Defense
96.21% %
0.15%
67.66%
M-PGD (Madry al. 2018
96.21 %
44.40%
55.40%
ALP (Kannan et al.
2018
96.20%
46.90%
56.20%
Adversarial-GGD Tramr et
2018
87.45%
42.96%
83.23%
Adversarial network (Wang & Yu 2019
96.21% %
37.97%
81.68%
Our method (Qs 5)
96.00%
93.39%
87.15%
Our method (Qs 10)
96.00%
93.80%
88.69 %
accuracy with different defense methods on the CIFAR-10 dataset. The second column shows the classification accuracy when the input images are all clean. We can see that some methods, such as the Adversarial BIM, Feature Squeezing, and Adversarial PGD degrade the classification accuracy of clean images. This implies that their defense methods have caused significant damages to the original images, or they cannot accurately tell if the input image is clean or being attacked. The rest four columns list the final image classification accuracy with different defense methods. Some methods did not provide results on specific attack methods, which were left blank (marked with '-') in the table. For all of these four attacks, our methods significantly outperforms existing methods. For example, for the powerful PGD attack, our method outperforms the Adversarial-PGu method by more than 28%. We can also see that the GCLN with quantization step size Qs 10 achieves better efficiency than that with Qs = 5. This is because the quantized nonlinear transform layers with larger quantization parameters are relatively more efficient in removing the noise in feature maps.
Defending against the BPDA attack. The Backward Pass Differentiable Approximation (BPDA) (Athalye et al.) (2018) attack is very challenging to defend since it can iteratively strengthen the adversarial examples using gradient approximation according to the defense mechanism. Table 2 summarizes the defense results of our algorithm in comparison with other seven methods. We can see that our GCLN is able to improve the classification accuracy up the second best method by 6%.
Defending against black-box attacks. We generate the black-box adversarial examples using FGS and PGD attacks with substitute model (Papernot et al., 2017). The substitute model is trained in the same way the target classifier with ResNet-34 network (He et al.) 2016) structure. Table shows the performance of our defense mechanism under back-box attacks on the CIFAR-10 dataset. constructed with 8/256 under the substitute model. We observe
The adversarial examples that the target classifier is much less sensitive to adversarial examples generated by FGS and PGD black-box attacks than the white-box ones. But the powerful PGD attack is still able to decrease the overall classification accuracy to a very low level, 38.71%. We compare our method with the Adersarial-Pee ReDD (Madry et al., 2018) and Adversarial Network (Wang & Yu 2019) methods. We include these two because they are the only ones that provide performance results on CIFAR- 10 with black-br box attacks. From the Table 3] we can see our method improves the accuracy by 5.8% over the state-of-the-art Adversarial Network method for the PGD attack.
4.3 RESULTS ON THE SVHN DATASET.
We evaluate our GCLN method on the SVHN dataset with comparison with four sfate-of-the-ar defense methods: M-PGD (Madry et al. 2018), 8), ALP (Kannan et al. (2088). adversarial PGD (Tramr) jtt al. 2018) and Adversaral network (Wang& Yu 2019). For SVHN dataset, as in the existing methods (Kannan al. 2018: Wang & Yu 2019). we used the Resnet-18 (He et al. 2016) for the target classifier. The average classification accuracy is 96.21%. We use the same parameters as in (Kannan al., 2018) for the PGD attack with total magnitude of 0.05 (12/255). Within each single step, the perturbation magnitude is set to be c = 0.01 (3/255) and 10 iterative steps are used.
Defending against white-box attack. Table 4 summarizes the experimental results and perfor- mance comparisons with those four existing defense methods. We can see that on this dataset the PGD attack is able to decrease the overall classification accuracy extremely low level, 0.15%

---

Under review as a conference paper at ICLR 2020
Figure 2: Visualization of the generative cleaning process. The adversarial perturbation pro-
duced using PGD attack with maximum perturbation 16/255.
60
60
50
50
Accuracy (%)
Accuracy (%)
40
20
30
10
L.rs 031, Oue10
LiciDofnnd,
L,00,031
Method
20
4
6
12 14
16
18
20
22 24
26
28
10
20
40
60
70
80
90
100
Quantization Parameter
Attack Iterations
Figure 3: Left: Classification accuracy our method defense on BPDA attack with different quan- tization parameter. Right: results against white-box BPDA attack with 10 to 100 attack iterations.
Our algorithm outperforms existing methods by a very large margin. For example, for the PGD at- tack, our algorithm outperforms the second best ALP (Kannan et al. 2018) algorithm by more than 46%.
Defending against black-box attack. We also perform experiments of defending black-box attacks on the SVHN dataset. Table summarizes our experimental results with the powerful PGD attack and provides the comparison with those four methods. We see that our approach outperforms other methods by 2.39% for the FGS attacks and 7.01% for the PGD attacks. From the above results, that our proposed method is particularly effective for defense against the strong attacks, for example, the PGD attacks with large iteration steps and noise magnitude.
Visualizing the defense process. Network defense is essentially denosing process of the feature maps. To further understand how the the proposed GCLN method works, we visualize the feature maps original, attacked, and GCLN-cleaned images. We use the feature map from the activation layer, the third from the last layer in the network. Figure 2 shows three examples. In the first example, the first row is the original image (classified into flamigo), its feature map, its gradient- weighted class activation heatmap, and the heatmap overlaied on the original image. The heatmap shows which parts of the original image the classification network is paying attention to. The second row shows the attacked image (being classified into hoopskirt), its feature map, heatmap, and the heatmap overlaid on the attacked image. We can see that the feature map very noisy and heatmap is distorted. The third row shows the GCLN-cleaned images. We can that both the feature map and heatmaps have been largely restored.
4.4 ABLATION STUDIES AND ALGORITHM ANALYSIS
In this section, we provide in-depth ablation study results of our algorithm to further understand its capability.

---

Under review as a conference paper at ICLR 2020
Vanilla Adv. Training
Vanilla Adv. Treining (PGD)
80
80
(PGD)
Our Method
Our Method (PGT)
Our Method (BPDA)
Accuracy (%6
70
Accuracy (%)
60
60
40
50
20
40
0-
10
20
30
40
50
60
70
80
90
100
0.0
0.1
0.2 0.3 0.4 0.5
0.6 0.7 0.8 0.9
Attack Iterations
Perturbation Magnitude (c)
Figure 4: Left: Classification accuracy of our method defense PGD attack with 10 to 100 attack iterations. Right: results against white-box PGD attack with 0.01 to 1.0 adversarial epsilon.
(A) Analyze the impact of the quantization parameter. We notice that the quantization parameter plays important role in the defense. Figure (3lfft) shows the defense performance (classification accuracy after defense) of our method on the CiFAR-10 dataset with white-box BPDA attacks. We can see that the quantization step size within the range of 8 to 12 yields the best performance. Small quantization parameters do not provide efficient defense since the quantized nonlinear transform is able to disturb and destroy the attack noise pattern. However, when the quantization parameter becomes too large, it will damage the original image content too much which cannot be recovered by the subsequent generative cleaning network.
(B) Defense against large-iteration BPDA attacks. The impact of the white-box BPDA attacks increases with its number of iterations since accesses the network and performs gradient back- propagation with more iterations to force the network towards wrong classification output. Fol- lowing the protocol of ALP (Kannan et al. 2018), we evaluate the capacity of our defense method against different numbers of BPDA white-box attack iterations. Figure rright) shows the perfor- mance of our method with an increasing number of attack iteration. We can see that our method is able to withstand large number of BPDA attack iterations. The impact of attack becomes relatively stable after 50 iterations.
(C) Defense against large-iteration and large-epsilon PGD attacks. As shown in Fig. 4 gradually increasing the PGD attack iterations will raise the attack strength. This significantly degrades the accuracy of the vanilla adversary training method (Madry et al.] 2018) and the PNI (Parametric Noise Injection) method (He et al.) 2019). as well as our method. But, our method signficantly outperforms the other two. In both cases, the perturbed-data accuracy starts saturating and do not degrade any further when Nstep 2 40. Our method still outperforms the vanilla adversarial training and PNI defense methods even when the magnitude of adversarial noise is increased up to 0.3.
CONCLUSION
We have developed a new method for defending deep neural networks against adversarial attacks based on generative cleaning networks with quantized nonlinear transform. This network is able to recover the original image while cleaning up the residual attack noise. We developed detector network, which serves as the dual network the target classifier network to be defended, to detect the image is clean or being attacked. This detector network and the generative cleaning network are jointly trained with adversarial learning so that the detector network cannot find any attack noise in the output image of generative cleaning network. Our extensive experimental results demonstrated that our approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. For example, it dramatically improves the classification accuracy upon the second best method more than 30% on the SVHN dataset and more than 14% on the challenging CIFAR-10 10 dataset.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Nasir Ahmed, T. Natarajan, and Kamisetty R Rao. Discrete cosine transform. IEEE transactions on Computers, 100(1):90-95 1974.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. arXiv:1701.07875. 2017.
Wasserstein
gan. arXiv preprint
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense se- curity: Circumventing defenses to adversarial examples. In Proceedings of the 35th International Conference on Machine Learning, pp. 274-283, 2018.
Jacob Buckman, Aurko Roy, Colin Raffel, and lan Goodfellow: Thermometer encoding: One hot way to resist adversarial examples. In International Conference on Learning Representations, 2018.
CW
Nicholas Carlini and David Wagner. Defensive distillation is not robust to adversarial examples. arXiv preprint arXiv: 1v07.03600.04111 2016.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP), May 2017. doi: 20.11.99 URL
http:/ dx. doi. wwt./oogg/......... http://......gg1...o.ooo
2017.
Ingrid Daubechies. The wavelet transform, time-frequency localization and signal analysis. IEEE
fransactions on information theory, 36(5):661-1005, 1990.
Guneet S. Dhillon, Kamyar Azizzadenesheli, Jeremy D. Bernstein, Jean Kossaifi, Aran Khanna, Zachary C. Lipton, and Animashree Anandkumar. Stochastic activation pruning for robust adversarial defense. In International Conference on Learning Representations, 2018. URL
htttps://epenreviiee new. http//ooommmmmm-Ho4AGG.
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. 2018 IEEE/CVF Conference Computer Vision and Pattern Recognition, Jun 2018. doi: 10.1109//vp:22188.0007 URL http://///// dx doi. org
1.0.1109//00 cvpr. 2018 00957
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-2680, 2014a.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint 1116v:1412.6572. 2014b.
Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens van der Maaten. Countering adversarial images using input transformations. In International Conference Learning Representations, 2018. httt:::://ppereeeie...tteerorrre /ttp://oroouaa--yyyyy/CCCCCC
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. Adversarial example defense: Ensembles of weak defenses are not strong. In 11th {USENIX) Workshop on Offensive Technologies (1 ((WOOT} 17), 2017.
Zhezhi He. Adnan Siraj Rakin, and Deliang Fan. Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 988-597, 2019.
Xiaojun Jia, Xingxing Wei. Xiaochun Cao, and Hassan Foroosh. Comdefend: An efficient image compression model to defend adversarial examples. CoRR, abs/1811.12673. 2019.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694-711. Springer, 2016.

---

Under review as a conference paper at ICLR 2020
Harini Kannan, Alexey Kurakin, and Ian Goodfellow. Adversarial logit pairing, 2018.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial machine learning scale. arXiv preprint 1rXiv:661.002250 2016.
Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani. Sudanthi Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality, 2018.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, To- wards deep learning models resistant adversarial attacks. In International Conference on Learn- ing Representations, 2018. [RRtpptts////peerreett
net/ forum wewoorummidr---liii rid
Dongyu Meng and Hao Chen. Magnet: two-pronged defense against adversarial examples. In
Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 135-147. ACM, 2017.
Taesik Na, Jong Hwan Ko, and Saibal Mukhopadhyay. Cascade adversarial machine learning reg- ularized with a unified embedding. In International Conference on Learning Representations, 2018. /RRLattts////peeeeeiiewe htt.:///w.....t..ooooooo.a/aa/aa
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Nicolas Papernot and Patrick McDaniel. On the effectiveness of defensive distillation. arXiv preprint 1rX.v:1607.051113 2016.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, 7. Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. pp. 372-387, 2016a.
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 582-597. IEFE, 2016b.
Nicolas Papernot, Patrick McDaniel, Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506-519, ACM, 2017.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense Defense-gan: Protecting classifiers against adversarial attacks using generative models. arXiv preprint arXiv: 1805. 060000060555..0 2018
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In Interna- tional Conference on Learning Representationss 2018. URL httppsss//peeeeeeeee...
Forum?ld=rJJYGxxbCy
Bo Sun, Nian-Hsuan Tsai, Fangchen Liu, Ronald Yu, and Hao Su. Adversarial defense by strati- fied convolutional sparse coding. CVPR, Jan 2019. URL http::ttt///aat nsf httt::://par.nff.ooo/bbbbbb gov/biblio/
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv: 13/2.61312..199// 2013.
Florian Tramr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Bonch, and Patrick McDaniel. Ensemble adversarial training: Attacks and defenses. In International Confer- ence on Learning Representations, 2018. URL httpt:://opeeeeeeeeee nwt.:/rumt//dddd
rkZvSe-RZ
Gregory Wallace. The jpeg still picture compression standard. IEEE transactions on consumer electronics, 38()):xviiixxxiv, 1992.
Huaxia Wang and Chun-Nam Yu. A direct approach robust deep learning using adversar- ial networks. In International Conference on Learning Representations, 2019. URL hetps:
//openreviee ew. nwt. d=s IMm05F7
10

---

Under review as a conference paper at ICLR 2020
David Warde-Farley. 11 adversarial perturbations of deep neural networks. Perturbations, Opti- mization, and Statistics, 311, 2016.
Chaowei Xiao, Bo Li, Jun- Zhu, Warren He. Mingyan Liu, and Dawn Song. Generating adver- sarial examples with adversarial networks. arXiv preprint arXiv: 1801. 0000002600, 2018.
Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille. Mitigating adversarial effects through randomization. In International Conference on Learning Representations, 201 8a. URL BKKlhttpps://peeeeviee net/ www.://wwwweeeeeeeeeeeeam 1isiimiiisskkk00000
Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Yuille, and Kaiming He. Feature denoising for improving adversarial robustmess. arXiv preprint arXiv: ftff::////333// 2018b.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan Yuille. Improving transferability of adversarial examples with input diversity, 2018c.
Weilin Xu, David Evans, and Yanjun Qi. Feature squeezing: Detecting adversarial examples in deep neural networks. arXiv preprint vrXiv:17044 155, 2017.


Reviews:
[{"review_id": "SygPDfaXcr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper developed a method for defending deep neural networks against adversarial attacks based on generative cleaning networks with quantized nonlinear transform. The network is claimed to recover the original image while cleaning up the residual attack noise. The authors developed a detector network, which serves as the dual network of the target classifier network to be defended, to detect if the image is clean or being attacked. This detector network and the generative cleaning network are jointly trained with adversarial learning so that the detector network cannot find any attack noise in the output image of generative cleaning network. The experimental results demonstrated that the proposed approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. \n\nA few comments: \n\n1. It does not provide theoretical reasons why the prosed method can defend against those attacks. \n\n2. The experiments are a bit messy and the attacks' setup need to improve. \n\n3. The proposed defense showed only empirical results against the target attack. It seems to provide no theoretical / provable guarantees. \n\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SkxWHoIRYH", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new method to defend a neural network agains adversarial attacks (both white-box and black-box attacks). By jointly training a Generative Cleaning Network with quantized nonlinear transform, and a Detector Network, the proposed cleans the incoming attacked image and correctly classifies its true label. The authors use state-of-the-art attack methods on various models, and the proposed model consistently outperforms all baseline models, even dramatically outperforming them for some specific attack methods.\n\nComment:\nIs there a reason the authors did not test the same set of attack methods for SVHN as they did for CIFAR-10?", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "BJgs00VTKS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method for adversarial defense based on generative cleaning.\n\nThe paper does not follow any of the best practices for evaluating adversarial robustness, e.g. in these two papers:\n\"On Evaluating Adversarial Robustness\" https://arxiv.org/abs/1902.06705\n\"Obfuscated Gradients Give a False Sense of Security\" https://arxiv.org/abs/1802.00420\n\nFor instance the paper does not use a large number of PGD iterations (10 is too small) and does not check that accuracies go to zero for large epsilon (an important sanity check to reveal gradient masking). In the one place where a larger number of attack iterations is used (100 for BPDA) the gap with adversarial training mostly vanishes.\n\nIn the absence of these best practices it is impossible to assess the validity of the results, so the paper should be rejected.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method to defend neural networks from adversarial attack. The proposed generative cleaning network has a trainable quantization module which is claimed to be able to eliminate adversarial noise and recover the original image. 
After the intensive interaction with authors and discussion, one expert reviewer (R3) admitted that the experimental procedure basically makes sense and increased the score to Weak Reject. Yet, R3 is still not satisfied with some details such as the number of BPDA iterations, and more importantly, concludes that the meaningful numbers reported in the paper show only small gains, making the claim of the paper less convincing. As authors seem to have less interest in providing theoretical analysis and support, this issue is critical for decision, and there was no objection from other reviewers. After carefully reading the paper myself, I decided to support the opinion and therefore would like to recommend rejection. 

Decision: Reject