Year: 2019

Abstract:
Effective defense of deep neural networks against adversarial attacks remains a challenging problem, especially under white-box attacks. In this paper, we develop a new generative cleaning network with quantized nonlinear transform for effective defense of deep neural networks. The generative cleaning network, equipped with a trainable quantized nonlinear transform block, is able to destroy the sophisticated noise pattern of adversarial attacks and recover the original image content. The generative cleaning network and attack detector network are jointly trained using adversarial learning to minimize both perceptual loss and adversarial loss. Our extensive experimental results demonstrate that our approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. For example, it improves the classification accuracy for white-box attacks upon the second best method by more than 40% on the SVHN dataset and more than 20% on the challenging CIFAR-10 dataset.

heading: 1 INTRODUCTION
text: Recent research has shown that deep neural networks are sensitive to adversarial attacks (Szegedy et al., 2013). Very small changes of the input image can fool the state-of-art classifier with very high success probabilities. During the past few years, a number of methods have been proposed to construct adversarial samples to attack the deep neural networks, including fast gradient sign (FGS) method (Goodfellow et al., 2014b), Jacobian-based saliency map attack (J-BSMA) (Papernot et al., 2016a), and projected gradient descent (PGD) attack (Kurakin et al., 2016; Madry et al., 2018). It has also been demonstrated that different classifiers can be attacked by the same adversarial perturbation (Szegedy et al., 2013). The fragility of deep neural networks and the availability of these powerful attacking methods present an urgent need for effective defense methods. During the past few years, a number of deep neural network defense methods have been developed, including adversarial training (Kurakin et al., 2016; Szegedy et al., 2013), defensive distillation (Papernot et al., 2016b; Carlini & Wagner, 2016; Papernot & McDaniel, 2016), Magnet (Meng & Chen, 2017) and featuring squeezing (He et al., 2017; Xu et al., 2017). It has been recognized that these methods suffer from significant performance degradation under strong attacks, especially white-box attacks with large magnitude and iterations (Samangouei et al., 2018).
In this work, we explore a new approach to defend various attacks by developing a generative cleaning network with quantized nonlinear transform. We recognize that the attack noise is not random and has sophisticated patterns. The attackers often generate noise patterns by exploring the specific network architecture or classification behavior of the target deep neural network so that the small noise at the input layer can accumulate along the network inference layers, finally exceed the decision threshold at the output layer, and result in false decision. On the other hand, we know a well-trained deep neural networks are robust to random noise (Arjovsky et al., 2017), such as Gaussian noise. Therefore, the key issue in network defense is to randomize or destroy the sophisticated pattern of the attack noise while preserving the original image content.
Motivated by this observation, we design a new generative cleaning network with quantized nonlinear transform to first destroy the sophisticated noise patterns of adversarial attacks and then recover the original image content damaged during this nonlinear transform. We also construct a detector
network which serves as the dual network for the target classifier to be defended. The generative cleaning network and detector network are jointly trained using adversarial learning so that the detector network cannot detect the existence of attack noise pattern in the images recovered by the generative cleaning network. Our extensive experimental results demonstrate that our approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. It significantly improves the classification accuracy for white-box PGD attacks upon the second best method by more than 40% on the SVHN dataset from 46.90% to 93.80%, and more than 20% on the challenging CIFAR-10 dataset from 60.15% to 86.05%.
The major contributions of this work can be summarized as follows. (1) We have proposed a new approach for deep neural network defense by developing a unique generative cleaning network with quantized nonlinear transform. (2) We have formulated the problem of destroying the noise patterns of adversarial attacks and reconstructing original image content into generative adversarial network design and training which considers both perceptual loss and adversarial loss. (3) Our new method has significantly improved the performance of the state-of-the-art methods in the literature under a wide variety of attacks.
The rest of this paper is organized as follows. Section 2 reviews related work. The proposed method is presented in Section 3. Experimental results and performance comparison with existing methods are provided in Section 4. Section 5 concludes the paper.

heading: 2 RELATED WORK
text: In this section, we review related work on adversarial attack and network defense methods.
(A) Attack methods. Attack methods can be divided into two threat models: white-box attacks and black-box attacks. The white-box attacker has full access to the classifier network parameters, network architecture, and weights. The black-box attacker has no knowledge of or access to the target network. For white-box attack, a simple and fast approach called Fast Gradient Sign (FGS) method has been developed by Goodfellow et al. (2014b) using error back propagation to directly modify the original image. Basic Iterative Method (BIM) is an improved version of the FGS method. Carlini & Wagner (2016) designed an optimization-based attack method, called Carlini-Wagner (C&W) attack, which is able to fool the target network with the smallest perturbation. Xiao et al. (2018) trained a generative adversarial network (GAN) (Goodfellow et al., 2014a) to generate perturbations. Kannan et al. (2018) found that the Projected Gradient Descent (PGD) is the strongest among all attack methods. It can be viewed as a multi-step variant of FGSk (Madry et al., 2018). Athalye et al. (2018) introduced a method, called Backward Pass Differentiable Approximation (BPDA), to attack networks where gradients are not available. It is able to successfully attack all existing state-of-the-arts defense methods. For black-box attacks, the attacker has no knowledge about target classifier. Papernot et al. (2017) introduced the first approach for black-box attack using a substitute model. Dong et al. (2018) proposed a momentum-based iterative algorithms to improve the transferability of adversarial examples. Xie et al. (2018c) boosted the transferability of adversarial examples by creating diverse input patterns.
(B) Defense methods Several approaches have recently been proposed for defending both whitebox attacks and black-box attacks. Adversarial training defends various attacks by training the target model with adversarial examples (Szegedy et al., 2013; Goodfellow et al., 2014b). Madry et al. (2018) suggested that training with adversarial examples generated by PGD improves the robustness. Meng & Chen (2017) proposed a method, called MagNet, which detects the perturbations and then reshape them according to the difference between clean and adversarial examples. Recently, there are several defense methods based on GANs have been developed. Samangouei et al. (2018) projected the adversarial examples into a trained generative adversarial network (GAN) to approximate the input using generated clean image with multiple iterations. Recently, some defense methods have been developed based on input transformations. Guo et al. (2018) proposed several input transformations to defend the adversarial examples, including image cropping and re-scaling, bit-depth reduction, and JPEG compression. Xie et al. (2018a) proposed to defend against adversarial attacks by adding a randomization layer, which randomly re-scales the image and then randomly zero-pads the image. Jia et al. (2019) proposed an image compression framework to defend adversarial examples, called ComDefend. Xie et al. (2018b) introduced a feature denoising method for defending PGD white-box attacks.
Our proposed defense method is also related to GANs and image transformations. But, compared to existing methods, our method is unique in the following aspects: (1) We introduce a special layer called quantized nonlinear transform, into the generative cleaning network to destroy the sophisticated noise pattern of adversarial attacks. (2) Unlike the GAN-based methods in (Wang & Yu, 2019; Xiao et al., 2018) which aim to approximate input noise image using images generated by the GAN over multiple iterations, our generative cleaning network aims to reconstruct the image content damaged by quantized nonlinear transform. (3) Our method does not need to modify the target network to be protected.

heading: 3 THE PROPOSED DEFENSE METHOD
text: In this section, we present our proposed generative cleaning network method for effective deep neural network defense. For convenience, we refer to our proposed method by GCLN.

heading: 3.1 METHOD OVERVIEW
text: Figure 1 provides an overview of the proposed method. The attacked image x∗ is fed into the generative cleaning network Gθ. The network has a special layer, called quantized nonlinear transform, to destroy the noise pattern of the adversarial attack in the input image. The generative cleaning network aims to recover the original image content and produce a recovered image x̄. This recovered image x̄ will be passed to the target classifier Cα for image classification or recognition. To successfully learn the generative cleaning network Gθ, we construct a detector network Dφ, which serves as the dual network for the target classifier network Cα. The task of Dφ is to determine if the input image is clean or being attacked. In our proposed method, the generative cleaning network Gθ and the detector network Dφ are jointly trained through adversarial learning: the Gθ network is trying to recover the image x̂ so that Dφ cannot detect any attack noise in it. In the following sections, we will explain the proposed method in more detail.

heading: 3.2 QUANTIZED NONLINEAR TRANSFORM LAYER IN THE GENERATIVE CLEANING NETWORK
text: During the generative cleaning network design, We incorporate one special layer into the network, called quantized nonlinear transform. This transform aims to disturb and partially destroy the sophisticated pattern of the attack noise. In this work, we propose to construct such a transform using a linear transform T , followed by a quantizer Q and an inverse transform T−1. For the linear transform, we can use the discrete cosine transform (DCT) (Ahmed et al., 1974) which has been in JPEG image compression (Wallace, 1992). Specifically, we partition the input image into blocks of M × M . The original image block is denoted by X∗B = [x∗nk]1≤n,k≤M . The output block X̂B = [x̂ij ]1≤i,j≤M after DCT transform is given by
x̂ij = 1
4 CiCj M−1∑ n=0 M−1∑ k=0 xnk cos(iπ 2n+ 1 2M ) cos(jπ 2k + 1 2M ), (1)
with Ci = 1/ √
2 for i = 0, and Ci = 1 for i 6= 0. After transform, we will quantize the transform coefficient x̂ij as follows
RQ(x̂ij) = Round ( x̂ij q ) × q, (2)
where q is the quantization parameter. Certainly, this DCT transform can be replaced with other invertible transform, such as discrete wavelet transform (Daubechies, 1990). During network training, this special quantized nonlinear transform layer is implemented in the same way as the pooling layers in existing deep neural networks and included into the training process of the whole generative cleaning network.

heading: 3.3 ADVERSARIAL TRAINING FOR GENERATIVE CLEANING NETWORKS
text: In our defense method design, the generative cleaning network Gθ and the detector network Dφ are trained against each other, just like the existing generative adversarial networks (GAN). Dφ is a binary classifier to detect if the input image is clean or not. During the initial phase of training, Dφ is trained with the clean images and their attacked versions generated by existing attack methods. It should be noted that, when training Dφ, we do not need to know the model inside the target network Cα.
The goal of the generative cleaning network Gθ is two-fold: (1) first, it needs to successfully remove the residual attack noise so that the noise cannot be detected by the detector network Dφ. (2) Second, it needs to make sure that the original image content is largely recovered. To achieve the above two goals, we formulate the following generative loss function for training the generative cleaning network Gθ LG = λLP + (1− λ)LA, (3) where LP is perceptual loss and LA is the adversarial loss. λ is a weighting parameter. In our experiments, we set it to be 0.5. To define the perceptual loss, the L2-norm between the recovered image x̄ and the original image xo is often used (Johnson et al., 2016). In this work, we observe that the small adversarial perturbation often leads to very substantial noise in the feature map of the network (Xie et al., 2018b). Motivated by this, we use a pre-trained VGG-19 network, denoted by Fβ to generate visual features for the recovered image x̄ and the original image xo, and use their feature difference as the perceptual loss LP . Specifically,
LP = ||Fβ(xo)− Fβ(Gθ(x̂))||2, (4) The adversarial loss LA aims to train Gθ so that the recovered images will be detected as clean by the detector network Dφ. It is formulated as LA = Ex∗∈Ω∗Φ[Dφ(Gθ(x∗)), Iclean]. (5) Here, Φ[·, ·] represents the cross-entropy between the output generated by the generative network and the target label Iclean for clean images. We train our discriminative network Dφ, along with the generative cleaning network Gθ, to optimize the following min-max loss function:
min Gθ max Dφ {Exo∈Ωo [logDφ(xo)] + Ex∗∈Ω∗ [log(1−Dφ(Gθ(x∗)))]}. (6)
Here, Ωo and Ω∗ represent the clean and attacked images of the training dataset. The goal of generative model Gθ is to fool the discriminator Dφ that is trained to distinguish adversarial images from clean images. With this framework, our generator learns to recover images that are highly similar to clean images and difficult to be detected by Dφ. The detector network Dφ acts as a dual network for the original classifier Cα. Cascaded with the generative cleaning network Gφ, it will guide the training of Gφ using back propagation of gradients from its own network, aiming to minimize the above loss function. In our design, during the adversarial learning process, the target classifier Cα is called to determine if the recovered image x̄ is clean or not, as illustrated in Figure 1. If it is clean, it is added back into the clean training sample set Ωo on the fly to enhance the learning process.

heading: 4 EXPERIMENTAL RESULTS
text: In this section, we implement and evaluate our GCLN defense method and compare its performance with state-of-the-art defense methods under a wide variety of attacks, including white-box and blackbox attacks.

heading: 4.1 EXPERIMENT SETUP
text: Following existing methods in the literature, we use CIFAR-10 and SVHN (Street View House Number) datasets. The CIFAR-10 dataset consists of 60,000 images in 10 classes, with 32×32 image size. The Street View House Numbers (SVHN) dataset (Netzer et al., 2011) has about 200K images of street numbers. The attack methods to be considered in this work include FGS (Goodfellow et al., 2014b), PGD (Madry et al., 2018), BIM attack (Kurakin et al., 2016), and C&W attack (Carlini & Wagner, 2017).

heading: 4.2 RESULTS ON THE CIFAR-10 DATASET
text: We compare the performance of our defense method with 6 state-of-the-art methods developed in the literature under four different white-box attacks: FGS attack (Goodfellow et al., 2014b), PGD (Madry et al., 2018) attack, BIM attack (Kurakin et al., 2016) and C&W attack (Carlini & Wagner, 2017). Following (Kannan et al., 2018) and (Wang & Yu, 2019), the white-box attackers generate adversarial perturbations within a range of = 8/255. In addition, we set the step size of attackers to be = 1/255 with 10 attack iterations as the baseline settings. Table 1 shows image classification
accuracy with different defense methods on the CIFAR-10 dataset. The second column shows the classification accuracy when the input images are all clean. We can see that some methods, such as the Adversarial BIM, Feature Squeezing, and Adversarial PGD degrade the classification accuracy of clean images. This implies that their defense methods have caused significant damages to the original images, or they cannot accurately tell if the input image is clean or being attacked. The rest four columns list the final image classification accuracy with different defense methods. Some methods did not provide results on specific attack methods, which were left blank (marked with ’-’) in the table. For all of these four attacks, our methods significantly outperforms existing methods. For example, for the powerful PGD attack, our method outperforms the Adversarial-PGD method by more than 28%. We can also see that the GCLN with quantization step size Qs = 10 achieves better efficiency than that with Qs = 5. This is because the quantized nonlinear transform layers with larger quantization parameters are relatively more efficient in removing the noise in feature maps.
Defending against the BPDA attack. The Backward Pass Differentiable Approximation (BPDA) (Athalye et al., 2018) attack is very challenging to defend since it can iteratively strengthen the adversarial examples using gradient approximation according to the defense mechanism. Table 2 summarizes the defense results of our algorithm in comparison with other seven methods. We can see that our GCLN is able to improve the classification accuracy up the second best method by 6%.
Defending against black-box attacks. We generate the black-box adversarial examples using FGS and PGD attacks with a substitute model (Papernot et al., 2017). The substitute model is trained in the same way as the target classifier with ResNet-34 network (He et al., 2016) structure. Table 3 shows the performance of our defense mechanism under back-box attacks on the CIFAR-10 dataset. The adversarial examples are constructed with = 8/256 under the substitute model. We observe that the target classifier is much less sensitive to adversarial examples generated by FGS and PGD black-box attacks than the white-box ones. But the powerful PGD attack is still able to decrease the overall classification accuracy to a very low level, 38.71%. We compare our method with the Adersarial-PGD (Madry et al., 2018) and Adversarial Network (Wang & Yu, 2019) methods. We include these two because they are the only ones that provide performance results on CIFAR-10 with black-box attacks. From the Table 3, we can see our method improves the accuracy by 5.8% over the state-of-the-art Adversarial Network method for the PGD attack.

heading: 4.3 RESULTS ON THE SVHN DATASET.
text: We evaluate our GCLN method on the SVHN dataset with comparison with four state-of-the-art defense methods: M-PGD (Madry et al., 2018), ALP (Kannan et al., 2018), adversarial PGD (Tramr et al., 2018) and Adversaral network (Wang & Yu, 2019). For the SVHN dataset, as in the existing methods (Kannan et al., 2018; Wang & Yu, 2019), we used the Resnet-18 (He et al., 2016) for the target classifier. The average classification accuracy is 96.21%. We use the same parameters as in (Kannan et al., 2018) for the PGD attack with a total magnitude of = 0.05 (12/255). Within each single step, the perturbation magnitude is set to be = 0.01 (3/255) and 10 iterative steps are used.
Defending against white-box attack. Table 4 summarizes the experimental results and performance comparisons with those four existing defense methods. We can see that on this dataset the PGD attack is able to decrease the overall classification accuracy to an extremely low level, 0.15%.
Our algorithm outperforms existing methods by a very large margin. For example, for the PGD attack, our algorithm outperforms the second best ALP (Kannan et al., 2018) algorithm by more than 46%.
Defending against black-box attack. We also perform experiments of defending black-box attacks on the SVHN dataset. Table 4 summarizes our experimental results with the powerful PGD attack and provides the comparison with those four methods. We can see that our approach outperforms other methods by 2.39% for the FGS attacks and 7.01% for the PGD attacks. From the above results, we can see that our proposed method is particularly effective for defense against the strong attacks, for example, the PGD attacks with large iteration steps and noise magnitude.
Visualizing the defense process. Network defense is essentially a denosing process of the feature maps. To further understand how the the proposed GCLN method works, we visualize the feature maps of original, attacked, and GCLN-cleaned images. We use the feature map from the activation layer, the third from the last layer in the network. Figure 2 shows three examples. In the first example, the first row is the original image (classified into flamigo), its feature map, its gradientweighted class activation heatmap, and the heatmap overlaied on the original image. The heatmap shows which parts of the original image the classification network is paying attention to. The second row shows the attacked image (being classified into hoopskirt), its feature map, heatmap, and the heatmap overlaid on the attacked image. We can see that the feature map is very noisy and the heatmap is distorted. The third row shows the GCLN-cleaned images. We can see that both the feature map and heatmaps have been largely restored.

heading: 4.4 ABLATION STUDIES AND ALGORITHM ANALYSIS
text: In this section, we provide in-depth ablation study results of our algorithm to further understand its capability.
(A) Analyze the impact of the quantization parameter. We notice that the quantization parameter plays an important role in the defense. Figure 3(left) shows the defense performance (classification accuracy after defense) of our method on the CIFAR-10 dataset with white-box BPDA attacks. We can see that the quantization step size within the range of 8 to 12 yields the best performance. Small quantization parameters do not provide efficient defense since the quantized nonlinear transform is not able to disturb and destroy the attack noise pattern. However, when the quantization parameter becomes too large, it will damage the original image content too much which cannot be recovered by the subsequent generative cleaning network.
(B) Defense against large-iteration BPDA attacks. The impact of the white-box BPDA attacks increases with its number of iterations since it accesses the network and performs gradient backpropagation with more iterations to force the network towards wrong classification output. Following the protocol of ALP (Kannan et al., 2018), we evaluate the capacity of our defense method against different numbers of BPDA white-box attack iterations. Figure 3(right) shows the performance of our method with an increasing number of attack iteration. We can see that our method is able to withstand large number of BPDA attack iterations. The impact of attack becomes relatively stable after 50 iterations.
(C) Defense against large-iteration and large-epsilon PGD attacks. As shown in Fig. 4, gradually increasing the PGD attack iterations will raise the attack strength. This significantly degrades the accuracy of the vanilla adversary training method (Madry et al., 2018) and the PNI (Parametric Noise Injection) method (He et al., 2019), as well as our method. But, our method signficantly outperforms the other two. In both cases, the perturbed-data accuracy starts saturating and do not degrade any further when Nstep ≥ 40. Our method still outperforms the vanilla adversarial training and PNI defense methods even when the magnitude of adversarial noise is increased up to = 0.3.

heading: 5 CONCLUSION
text: We have developed a new method for defending deep neural networks against adversarial attacks based on generative cleaning networks with quantized nonlinear transform. This network is able to recover the original image while cleaning up the residual attack noise. We developed a detector network, which serves as the dual network of the target classifier network to be defended, to detect if the image is clean or being attacked. This detector network and the generative cleaning network are jointly trained with adversarial learning so that the detector network cannot find any attack noise in the output image of generative cleaning network. Our extensive experimental results demonstrated that our approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks. For example, it dramatically improves the classification accuracy upon the second best method more than 30% on the SVHN dataset and more than 14% on the challenging CIFAR-10 dataset.

Review summaries:
This paper proposes a new method to defend a neural network agains adversarial attacks (both white-box and black-box attacks). By jointly training a Generative Cleaning Network with quantized nonlinear transform, and a Detector Network, the proposed cleans the incoming attacked image and correctly classifies its true label. The authors use state-of-the-art attack methods on various models, and the proposed model consistently outperforms all baseline models, even dramatically outperforming them for some specific attack methods.

This paper developed a method for defending deep neural networks against adversarial attacks based on generative cleaning networks with quantized nonlinear transform. The network is claimed to recover the original image while cleaning up the residual attack noise. The authors developed a detector network, which serves as the dual network of the target classifier network to be defended, to detect if the image is clean or being attacked. This detector network and the generative cleaning network are jointly trained with adversarial learning so that the detector network cannot find any attack noise in the output image of generative cleaning network. The experimental results demonstrated that the proposed approach outperforms the state-of-art methods by large margins in both white-box and black-box attacks.