A DIRECT APPROACH TO ROBUST DEEP LEARNING USING ADVERSARIAL NETWORKS
Year: 2019

Abstract:
Deep neural networks have been shown to perform well in many classical machine learning problems, especially in image classification tasks. However, researchers have found that neural networks can be easily fooled, and they are surprisingly sensitive to small perturbations imperceptible to humans. Carefully crafted input images (adversarial examples) can force a well-trained neural network to provide arbitrary outputs. Including adversarial examples during training is a popular defense mechanism against adversarial attacks. In this paper we propose a new defensive mechanism under the generative adversarial network (GAN) framework. We model the adversarial noise using a generative network, trained jointly with a classification discriminative network as a minimax game. We show empirically that our adversarial network approach works well against black box attacks, with performance on par with state-of-art methods such as ensemble adversarial training and adversarial training with projected gradient descent.

heading: 1 INTRODUCTION
text: Deep neural networks have been successfully applied to a variety of tasks, including image classification (Krizhevsky et al., 2012), speech recognition (Graves et al., 2013), and human-level playing of video games through deep reinforcement learning (Mnih et al., 2015). However, Szegedy et al. (2014) showed that convolutional neural networks (CNN) are extremely sensitive to carefully crafted small perturbations added to the input images. Since then, many adversarial examples generating methods have been proposed, including Jacobian based saliency map attack (JSMA) (Papernot et al., 2016a), projected gradient descent (PGD) attack (Madry et al., 2018), and C&W’s attack (Carlini & Wagner, 2017). In general, there are two types of attack models: white box attack and black box attack. Attackers in white box attack model have complete knowledge of the target network, including network’s architecture and parameters. Whereas in black box attacks, attackers only have partial or no information on the target network (Papernot et al., 2017).
Various defensive methods have been proposed to mitigate the effect of the adversarial examples. Adversarial training which augments the training set with adversarial examples shows good defensive performance in terms of white box attacks (Kurakin et al., 2017; Madry et al., 2018). Apart from adversarial training, there are many other defensive approaches including defensive distillation (Papernot et al., 2016b), using randomization at inference time (Xie et al., 2018), and thermometer encoding (Buckman et al., 2018), etc.
In this paper, we propose a defensive method based on generative adversarial network (GAN) (Goodfellow et al., 2014). Instead of using the generative network to generate samples that can fool the discriminative network as real data, we train the generative network to generate (additive) adversarial noise that can fool the discriminative network into misclassifying the input image. This allows flexible modeling of the adversarial noise by the generative network, which can take in the original image or a random vector or even the class label to create different types of noise. The discriminative networks used in our approach are just the usual neural networks designed for their specific classification tasks. The purpose of the discriminative network is to classify both clean and adversarial example with correct label, while the generative network aims to generate powerful perturbations
to fool the discriminative network. This approach is simple and it directly uses the minimax game concept employed by GAN. Our main contributions include:
• We show that our adversarial network approach can produce neural networks that are robust towards black box attacks. In the experiments they show similar, and in some cases better, performance when compared to state-of-art defense methods such as ensemble adversarial training (Tramèr et al., 2018) and adversarial training with projected gradient descent (Madry et al., 2018). To our best knowledge we are also the first to study the joint training of a generative attack network and a discriminative network.
• We study the effectiveness of different generative networks in attacking a trained discriminative network, and show that a variety of generative networks, including those taking in random noise or labels as inputs, can be effective in attacks. We also show that training against these generative networks can provide robustness against different attacks.
The rest of the paper is organized as follows. In Section 2, related works including multiple attack and defense methods are discussed. Section 3 presents our defensive method in details. Experimental results are shown in Section 4, with conclusions of the paper in Section 5.

heading: 2 RELATED WORKS
text: In this section, we briefly review the attack and defense methods in neural network training.

heading: 2.1 ATTACK MODEL
text: Given a neural network model Dθ parameterized by θ trained for classification, an input image x ∈ Rd and its label y, we want to find a small adversarial perturbation ∆x such that x+ ∆x is not classified as y. The minimum norm solution ∆x can be described as:
arg min ∆x
‖∆x‖ s.t. arg maxDθ(x+ ∆x) 6= y, (1)
where arg maxDθ(x) gives the predicted class for input x. Szegedy et al. (2014) introduced the first method to generate adversarial examples by considering the following optimization problem,
∆x = arg min z
λ‖z‖+ L(Dθ(x+ z), ŷ), x+ z ∈ [0, 1]d (2)
where L is a distance function measuring the closeness of the outputDθ(x+z) with some target ŷ 6= y. The objective is minimized using box-constrained L-BFGS. Goodfellow et al. (2015) introduced the fast gradient sign method (FGS) to generate adversarial examples in one step, which can be represented as ∆x = · sign (∇xl(Dθ(x), y)), where l is the cross-entropy loss used in neural networks training. Madry et al. (2018) argues with strong evidence that projected gradient descent (PGD), which can be viewed as an iterative version of the fast gradient sign method, is the strongest attack using only first-order gradient information. Papernot et al. (2017) presented a Jacobian-based saliency-map attack (J-BSMA) model to generate adversarial examples by changing a small number of pixels. Moosavi-Dezfooli et al. (2017) showed that there exist a single/universal small image perturbation that fools all natural images. Papernot et al. (2017) introduced the first demonstration of black-box attacks against neural network classifiers. The adversary has no information about the architecture and parameters of the neural networks, and does not have access to large training dataset.

heading: 2.2 DEFENSE MODEL
text: In order to mitigate the effect of the generated adversarial examples, various defensive methods have been proposed. Papernot et al. (2016b) introduced distillation as a defense to adversarial examples. Lou et al. (2016) introduced a foveation-based mechanism to alleviate adversarial examples.
The idea of adversarial training was first proposed by Szegedy et al. (2014). The effect of adversarial examples can be reduced through explicitly training the model with both original and perturbed adversarial images. Adversarial training can be viewed as a minimax game,
θ∗ = arg min θ Ex,y max ∆x l(Dθ(x+ ∆x), y). (3)
The inner maximization requires a separate oracle for generating the perturbations ∆x. FGS is a common method for generating the adversarial perturbations ∆x due to its speed. Madry et al. (2018) advocates the use of PGD in generating adversarial examples. Moreover, a cascade adversarial training is presented in Na et al. (2018), which injects adversarial examples from an already defended network added with adversarial images from the network being trained.
There are a few recent works on using GANs for generating and defending against adversarial examples. Samangouei et al. (2018) and Ilyas et al. (2017) use GAN for defense by learning the manifold of input distribution with GAN, and then project any input examples onto this learned manifold before classification to filter out any potential adversarial noise. Our approach is more direct because we do not learn the input distribution and no input denoising is involved. Both Baluja & Fischer (2018) and Xiao et al. (2018) train neural networks to generate adversarial examples by maximizing the loss over a fixed pre-trained discriminative network. They show that they can train neural networks to effectively attack undefended discriminative networks while ensuring the generated adversarial examples look similar to the original samples. Our work is different from these because instead of having a fixed discriminative network, we co-train the discriminative network together with the adversarial generative network in a minimax game. Xiao et al. (2018) also train a second discriminative network as in typical GANs, but their discriminative network is used for ensuring the generated images look like the original samples, and not for classification. Lee et al. (2017) also considered the use of GAN to train robust discriminative networks. However, the inputs to their generative network is the gradient of the discriminative network with respect to the input image x, not just the image x as in our current work. This causes complex dependence of the gradient of the generative network parameters to the discriminative network parameters, and makes the parameter updates for the generative network more complicated. Also there is no single minimax objective that they are solving for in their work; the update rules for the discriminative and generative networks optimize related but different objectives.

heading: 3 METHOD
text: In generative adversarial networks (GAN) (Goodfellow et al., 2014), the goal is to learn a generative neural network that can model a distribution of unlabeled training examples. The generative network transforms a random input vector into an output that is similar to the training examples, and there is a separate discriminative network that tries to distinguish the real training examples against samples generated by the generative network. The generative and discriminative networks are trained jointly with gradient descent, and at equilibrium we want the samples from the generative network to be indistinguishable from the real training data by the discriminative network, i.e., the discriminative network does no better than doing a random coin flip.
We adopt the GAN approach in generating adversarial noise for a discriminative model to train against. This approach has already been hinted at in Tramèr et al. (2018), but they decided to train against a static set of adversarial models instead of training against a generative noise network that can dynamically adapt in a truly GAN fashion. In this work we show that this idea can be carried out fruitfully to train robust discriminative neural networks.
Given an input x with correct label y, from the viewpoint of the adversary we want to find additive noise ∆x such that x + ∆x will be incorrectly classified by the discriminative neural network to some other labels ŷ 6= y. We model this additive noise as G(x), where G is a generative neural
network that generates instance specific noise based on the input x and is the scaling factor that controls the size of the noise. Notice that unlike white box attack methods such as FGS or PGD, once trained G does not need to know the parameters of the discriminative network that it is attacking. G can also take in other inputs to generate adversarial noise, e.g., Gaussian random vector z ∈ Rd as in typical GAN, or even the class label y. For simplicity we assume G takes in x as input in the descriptions below.
Suppose we have a training set {(x1, y1), . . . , (xn, yn)} of image-label pairs. Let Dθ be the discriminator network (for classification) parameterized by θ, and Gφ be the generator network parameterized by φ. We want to solve the following minimax game between Dθ and Gφ:
min θ max φ n∑ i=1 l(Dθ(xi), yi) + λ n∑ i=1 l(Dθ(xi + Gφ(xi)), yi), (4)
where l is the cross-entropy loss, λ is the trade-off parameter between minimizing the loss on normal examples versus minimizing the loss on the adversarial examples, and is the magnitude of the noise. See Figure 1 for an illustration of the model.
In this work we focus on perturbations based on `∞ norm. This can be achieved easily by adding a tanh layer as the final layer of the generator network Gφ, which normalizes the output to the range of [−1, 1]. Perturbations based on `1 or `2 norms can be accommodated by having the appropriate normalization layers in the final layer of Gφ.
We now explain the intuition of our approach. Ideally, we would like to find a solution θ that has small risk on clean examples
R(θ) = n∑ i=1 l(Dθ(xi), yi), (5)
and also small risk on the adversarial examples under maximum perturbation of size
Radv(θ) = n∑ i=1 max ∆x,‖∆x‖≤ l(Dθ(xi + ∆x), yi). (6)
However, except for simple datasets like MNIST, there are usually fairly large differences between the solutions ofR(θ) and solutions ofRadv(θ) under the same model classDθ (Tsipras et al., 2018). Optimizing for the risk under white box attacks Radv(θ) involves tradeoff on the risk on clean data R(θ). Note that Radv(θ) represent the risk under white box attacks, since we are free to choose the perturbation ∆x with knowledge of θ. This can be approximated using the powerful PGD attack.
Instead of allowing the perturbations ∆x to be completely free, we model the adversary as a neural network Gφ with finite capacity
RG(θ) = max φ n∑ i=1 l(Dθ(xi + Gφ(xi)), yi). (7)
Here the adversarial noise Gφ(xi) is not allowed to directly depend on the discriminative network parameters θ. Also, the generative network parameter φ is shared across all examples, not computed per example like ∆x. We believe this is closer to the situation of defending against black box attacks, when the adversary does not know the discriminator network parameters. However, we still want Gφ to be expressive enough to represent powerful attacks, so that Dθ has a good adversary to train against. Previous work (Xiao et al., 2018; Baluja & Fischer, 2018) show that there are powerful classes of Gφ that can attack trained classifiers Dθ effectively.
In traditional GANs we are most interested in the distributions learned by the generative network. The discriminative network is a helper that drives the training, but can be discarded afterwards. In our setting we are interested in both the discriminative network and the generative network. The generative network in our formulation can give us a powerful adversary for attacking, while the discriminative network can give us a robust classifier that can defend against adversarial noise.

heading: 3.1 STABILIZING THE GAN TRAINING
text: The stability and convergence of GAN training is still an area of active research (Mescheder et al., 2018). In this paper we adopt gradient regularization (Mescheder et al., 2017) to stabilize the gradient descent/ascent training. Denote the minimax objective in Eq. 4 as F (θ, φ). With the generative
network parameter fixed at φk, instead of minimizing the usual objective F (θ, φk) to update θ for the discriminator network, we instead try to minimize the regularized objective
F (θ, φk) + γ
2 ‖∇φF (θ, φk)‖2, (8)
where γ is the regularization parameter for gradient regularization. Minimizing the gradient norm ‖∇φF (θ, φk)‖2 jointly makes sure that the norm of the gradient for φ at φk does not grow when we update θ to reduce the objective F (θ, φk). This is important because if the gradient norm ‖∇φF (θ, φk)‖2 becomes large after an update of θ, it is easy to update φ to make the objective large again, leading to zigzagging behaviour and slow convergence. Note that the gradient norm term is zero at a saddle point according to the first-order optimality conditions, so the regularizer does not change the set of solutions. With these we update θ using SGD with step size ηD:
θl+1 = θl − ηD∇θ[F (θl, φk) + γ
2 ‖∇φF (θl, φk)‖2]
= θl − ηD[∇θF (θl, φk) + γ∇2θφF (θl, φk)∇φF (θl, φk)] The Hessian-vector product term ∇2θφF (θl, φk)∇φF (θl, φk) can be computed with double backpropagation provided by packages like Tensorflow/PyTorch, but we find it faster to compute it with finite difference approximation. Recall that for a function f(x) with gradient g(x) and Hessian H(x), the Hessian-vector product H(x)v can be approximated by (g(x+ hv)− g(x))/h for small h (Pearlmutter, 1994). Therefore we approximate:
∇2θφF (θl, φk)∇φF (θl, φk) ≈ ∇θ[ F (θl, φk + hv)− F (θl, φk)
h ], (9)
where v = ∇φF (θl, φk). Note that φk + hv is exactly a gradient step for generative network Gφ. Setting h to be too small can lead to numerical instability. We therefore correlate h with the gradient step size and set h = ηG/10 to capture the curvature at the scale of the gradient ascent algorithm.
We update the generative network parameters φ with using (stochastic) gradient ascent. With the discriminative network parameters fixed at θl and step size ηG, we update:
φk+1 = φk + ηG∇φF (θl, φk). We do not add a gradient regularization term for φ, since empirically we find that adding gradient regularization to θ is sufficient to stabilize the training.

heading: 3.2 GENERATIVE AND DISCRIMINATIVE NETWORK PARAMETER UPDATES
text: In the experiments we train both the discriminative network and generative network from scratch with random weight initializations. We do not need to pre-train the discriminative network with clean examples, or the generative network against some fixed discriminative networks, to arrive at good saddle point solutions.
In our experiments we find that the discriminative networks Dθ we use tend to overpower the generative networkGφ if we just perform simultaneous parameter updates to both networks. This can lead to saddle point solutions where it seems Gφ cannot be improved locally against Dθ, but in reality can be made more powerful by just running more gradient steps on φ. In other words we want the region around the saddle point solution to be relatively flat for Gφ. To make the generative network more powerful so that the discriminative network has a good adversary to train against, we adopt the following strategy. For each update of θ for Dθ, we perform multiple gradient steps on φ using the same mini-batch. This allows the generative network to learn to map the inputs in the mini-batch to adversarial noises with high loss directly, compared to running multiple gradient steps on different mini-batches. In the experiments we run 5 gradient steps on each mini-batch. We fix the tradeoff parameter λ (Eq. 4) over loss on clean examples and adversarial loss at 1. We also fix the gradient regularization parameter γ (Eq. 8) at 0.01, which works well for different datasets.

heading: 4 EXPERIMENTS
text: We implemented our adversarial network approach using Tensorflow(Abadi et al., 2016), with the experiments run on several machines each with 4 GTX1080 Ti GPUs. In addition to our adversarial
networks, we also train standard undefended models and models trained with adversarial training using PGD for comparison. For attacks we focus on the commonly used fast gradient sign (FGS) method, and the more powerful projected gradient descent (PGD) method.
For the fast gradient sign (FGS) attack, we compute the adversarial image by
x̂i = ProjX (xi + sign∇xl(Dθ(xi), yi)) , (10) where ProjX projects onto the feasible range of rescaled pixel values X (e.g., [-1,1]).
For the projected gradient descent (PGD) attack, we iterate the fast gradient sign attack multiple times with projection, with random initialization near the starting point neighbourhood.
x̂0i = ProjX (xi + u) x̂k+1i = ProjB∞ (xi)∩X ( x̂ki + δ sign∇xl(Dθ(x̂ki ), yi) ) ,
(11)
where u ∈ Rd is a uniform random vector in [−1, 1]d, δ is the step size, and B∞ (xi) is an `∞ ball centered around the input xi with radius . In the experiments we set δ to be a quarter of the perturbation , i.e., /4, and the number of PGD steps k to be 10. We adopt exactly the same PGD attack procedure when generating adversarial examples in PGD adversarial training. Our implementation is available at https://github.com/whxbergkamp/RobustDL_GAN.

heading: 4.1 MNIST
text: For MNIST the inputs are black and white images of digits of size 28x28 with pixel values scaled between 0 and 1. We rescale the inputs to the range of [-1,1]. Following previous work (Kannan et al., 2018), we study perturbations of = 0.3 (in the original scale of [0,1]). We use a simple convolutional neural network similar to LeNet5 as our discriminator networks for all training methods. For our adversarial approach we use an encoder-decoder network for the generator. See Model D1 and Model G0 in the Appendix for the details of these networks. We use SGD with learning rate of ηD = 0.01 and momentum 0.9, batch size of 64, and run for 200k iterations for all the discriminative networks. The learning rates are decreased by a factor of 10 after 100k iterations. We use SGD with a fixed learning rate ηG = 0.01 with momentum 0.9 for the generative network. We use weight decay of 1E-4 for standard and adversarial PGD training, and 1E-5 for our adversarial network approach (for bothDθ andGφ). For this dataset we find that we can improve the robustness of Dθ by running more updates on Gφ, so we run 5 updates on Gφ (each update contains 5 gradient steps described in Section 3.2 ) for each update on Dθ.
Table 1(left) shows the white box attack accuracies of different models, under perturbations of = 0.3 for input pixel values between 0 and 1. Adversarial training with PGD performs best under white box attacks. Its accuracies stay above 90% under FGS and PGD attacks. Our adversarial network model performs much better than the undefended standard training model, but there is still a gap in accuracies compared to the PGD model. However, the PGD model has a small but noticeable drop in accuracy on clean examples compared to the standard model and adversarial network model.
Table 1(right) shows the black box attack accuracies of different models. We generate the black box attack images by running the FGS and PGD attacks on surrogate models A’, B’ and C’. These surrogate models are trained in the same way as their counterparts (standard - A, PGD - B, adversarial network - C) with the same network architecture, but using a different random seed. We notice that the black box attacks tend to be the most effective on models trained with the same method (A’ on A, B’ on B, and C’ on C). Although adversarial PGD beats our adversarial network approach on white box attacks, they have comparable performance on these black box attacks. Interestingly, the adversarial examples from adversarial PGD (B’) and adversarial networks (C’) do not transfer well to the undefended standard model. The undefended model still have accuracies between 85-95%.

heading: 4.2 SVHN
text: For the Street View House Number(SVHN) data, we use the original training set, augmented with 80k randomly sampled images from the extra set as our training data. The test set remains the same and we do not perform any preprocessing on the images apart from scaling it to the range of [-1,1]. We study perturbations of size = 0.05 (in the range of [0,1]). We use a version of ResNet-18(He
et al., 2016) adapted to 32x32 images as our discriminative networks. For the generator in our adversarial network we use an encoder-decoder network based on residual blocks from ResNet. See Model D2 and Model G1 in the Appendix for details. For the discriminative networks we use SGD with learning rate of ηD = 0.01 and momentum 0.9, batch size of 64, weight decay of 1E-4 and run for 100k iterations, and then decrease the learning rate to 0.001 and run for another 100k iterations. For the generative network we use SGD with a fixed learning rate of ηG = 0.01 and momentum 0.9, and use weight decay of 1E-4.
Table 2(left) shows the white box attack accuracies of the models. Adversarial PGD performs best against PGD attacks, but has lower accuracies on clean data and against FGS attacks, since it is difficult to optimize over all three objectives with finite network capacity. Our adversarial network approach has the best accuracies on clean data and against FGS attacks, and also improved accuracies against PGD over standard training.
Table 2(right) shows the black box attack accuracies of the models. As before A’, B’, C’ are networks trained in the same ways as their counterparts, but with a different random seed. We can see that the adversarial network approach performs best across most attacks, except the PGD attack from its own copy C’. It is also interesting to note that for this dataset, adversarial examples generated from the adversarial PGD model B’ have the strongest attack power across all models. In the other two datasets, adversarial examples generated from a model are usually most effective against their counterparts that are trained in the same way.

heading: 4.3 CIFAR10
text: For CIFAR10 we scale the 32x32 inputs to the range of [-1,1]. We also perform data augmentation by randomly padding and cropping the images by at most 4 pixels, and randomly flipping the images left to right. In this experiment we use the same discriminative and generative networks as in SVHN. We study perturbations of size = 8/256. We train the discriminative networks with batch size of 64, and learning rate of ηD = 0.1 for 100k iterations, and decrease learning rate to 0.01 for another 100k iterations. We use Adam with learning rate ηG = 0.002, β1 = 0.5, β2 = 0.999 for the generative network. We use weight decay 1E-4 for standard training, and 1E-5 for adversarial PGD and our adversarial networks.
Table 3(left) shows the white box accuracies of different models under attack with = 8/256. The PGD model has the best accuracy under PGD attack, but suffer a considerably lower accuracy on clean data and FGS attack. One reason for this is that it is difficult to balance between the objective of getting good accuracies on clean examples and good accuracies on very hard PGD attack adversarial examples with a discriminative network of limited capacity. Our adversarial model is able to keep up with the standard model in terms of accuracies on clean examples, and improve upon it on accuracies against FGS and PGD attacks.
Table 3(right) shows the black box attack accuracies of the models. Our adversarial network method works better than the other approaches in general, except for the PGD attack from the most similar model C’. The adversarial PGD model also works quite well except against its own closest model
B’, and offers the smallest drop in accuracies in general. But its overall results are not the best since it suffers from the disadvantage of having a lower baseline accuracy on clean examples.
We have also performed experiments on CIFAR10 using a wider version of ResNet (Zagoruyko & Komodakis, 2016) by multiplying the number of filters by 10 in each of the convolutional layers. These wider version of ResNets have higher accuracies, but the relative strengths of the methods are similar to those presented here. In addition we have experiments on CIFAR100, and the results are qualitatively similar to CIFAR10. All these results are presented in the Appendix due to space restrictions.

heading: 4.4 COMPARING AGAINST ENSEMBLE ADVERSARIAL TRAINING
text: We also compare against a version of ensemble adversarial training (Tramèr et al., 2018) on the above 3 datasets. Ensemble adversarial training works by including adversarial examples generated from static pre-trained models to enlarge the training set, and then train a new model on top of it. The quality of solutions depends on the type of adversarial examples included. Here we construct adversarial examples by running FGS (Eq. 10) and PGD (Eq. 11) on an undefended model, i.e., FGS(A) and PGD(A) in the previous tables. Here for FGS we substitute the target label y with the most likely class arg maxDθ(x) to avoid the problem of label leakage. Following Tramèr et al. (2018) we also include another attack using the least likely class:
x̂i = ProjX (xi − sign∇xl(Dθ(xi), yLL)) , (12)
where yLL = arg minDθ(xi) is the least likely class. We include all these adversarial examples together with the original clean data for training. We use the same perturbations as in the respective experiments above.
Table 4 shows the results comparing ensemble adversarial training (EAT) with our adversarial networks approach. On MNIST, adversarial networks is better on white box attacks and also better on all black box attacks using models trained with standard training(A’), adversarial PGD(B’), and our adversarial networks approach(C’) with different random seeds. On SVHN and CIFAR10 adversarial networks is better on white box attacks, and both methods have wins and losses on the black box attacks, depending on the attacks used. In general adversarial networks seem to have better white box attack accuracies since they are trained dynamically with a varying adversary. The black box accuracies depend a lot on the dataset and the type of attacks used. There is no definitive conclusion on whether training against a static set of adversaries as in EAT or training against a dynamically adjusting adversary as in adversarial networks is a better approach against black box attacks. This is an interesting question requiring further research.

heading: 4.5 EXAMINING THE GENERATIVE NETWORKS
text: We also did a more in-depth study on the generative network with CIFAR10. We want to understand how the capacity of the generative network affects the quality of saddle point solution, and also the power of the generative networks themselves as adversarial attack methods. First we study the ability of the generative networks to learn to attack a fixed undefended discriminative network. The architectures of the generative networks (G1, G2, G3) are described in the Appendix. Here we study a narrow (G1, k = 8) and a wide version (G1, k = 64) of autoencoder networks using the input images as inputs, and also decoder networks G(z) using random Gaussian vectors z ∈ Rd (G2) or networks G(y) using the labels y (G3) as inputs. We run SGD for 200k iterations with step size 0.01 and momentum of 0.9, and use weight decay of 1E-5. We report test accuracies on the original discriminator after attacks.
From Table 5 the wide autoencoder is more powerful than the narrow autoencoder in attacking the undefended discriminator network across different models. As a white-box attack method, the wide autoencoder is close to PGD in terms of attack power (6.08% vs 1.32% in Table 3(left)) on the undefended model. As a black-box attack method on the undefended model A’, it works even better than PGD (14.74% vs 22.88% in Table 3(right)). However, on defended models trained with PGD and our adversarial network approach the trained generator networks do not have much effect. PGD is especially robust with very small drops in accuracies against these attacks.
It is interesting that generator network G(z) with random Gaussian z as inputs and G(y) with label as input works well against undefended models A and A’, reducing the accuracies by more than 30%, even though they are not as effective as using the image as input. G(z) is essentially a distribution of random adversarial noise that we add to the image without knowing the image or label. G(y) is a generator network with many parameters, but after training it is essentially a set of 10 class conditional 32x32x3 filters. We have also performed similar experiments on attacking models trained with adversarial PGD and our adversarial networks using the above generative networks. The results are included in the Appendix due to space restrictions.
We also co-train these different generative networks with our discriminative network (D2) on CIFAR10. The results are shown in Table 6. It is slightly surprising that they all produce very similar performance in terms of white box and black box attacks, even as they have different attack powers against undefended networks. The generative networks do have very similar decoder portions, and this could be a reason why they all converge to saddle points of similar quality.

heading: 4.6 DISCUSSIONS
text: In the experiments above we see that adversarial PGD training usually works best on white box attacks, but there is a tradeoff between accuracies on clean data against accuracies on adversarial examples due to finite model capacity. We can try to use models with larger capacity, but there is always a tradeoff between the two, especially for larger perturbations . There are some recent works that indicate training for standard accuracy and training for adversarial accuracy (e.g., with PGD) are two fairly different problems (Schmidt et al., 2018; Tsipras et al., 2018). Examples generated from PGD are particularly difficult to train against. This makes adversarial PGD training disadvantaged in many black box attack situations, when compared with models trained with weaker adversaries, e.g., ensemble adversarial training and our adversarial networks method.
We have also observed in the experiments that for black box attacks, the most effective adversarial examples are usually those constructed from models trained using the same method but with different random seed. This suggests hiding the knowledge of the training method from the attacker could be an important factor in defending against black box attacks. Defending against black box attacks is closely related to the question of the transferability of adversarial examples. Although there are some previous works exploring this question (Liu et al., 2017), the underlying factors affecting transferability are still not well understood.
In our experimentation with the architectures of the discriminative and generative networks, the choice of architectures of Gφ does not seem to have a big effect on the quality of solution. The dynamics of training, such as the step size used and the number of iterations to run for each network during gradient descent/ascent, seem to have a bigger effect on the saddle point solution quality than the network architecture. It would be interesting to find classes of generative network architectures that lead to substantially different saddle points when trained against a particular discriminative network architecture. Also, recent works have shown that there are connected flat regions in the minima of neural network loss landscapes (Garipov et al., 2018; Draxler et al., 2018). We believe that the same might hold true for GANs, and it would be interesting to explore how the training dynamics can lead to different GAN solutions that might have different robustness properties.
Our approach can be extended with multiple discriminative networks playing against multiple generative networks. It can also be combined with ensemble adversarial training, where some adversarial examples come from static pre-trained models, while some other come from dynamically adjusting generative networks.

heading: 5 CONCLUSIONS
text: We have proposed an adversarial network approach to learning discriminative neural networks that are robust to adversarial noise, especially under black box attacks. For future work we are interested in extending the experiments to ImageNet, and exploring the choice of architectures of the discriminative and generative networks and their interaction.

Review summaries:
The paper "A Direct Approach to Robust Deep Learning Using Adversarial Networks" proposes a GAN solution for deep models of classification, faced to white and black box attacks. It defines an architecture where a generator network seeks to produce slight pertubations that succeed in fooling the discriminator. The discriminator is the targetted classification model. 

The paper proposed a defensive mechanism against adversarial attacks using GANs. The general network structure is very much similar to a standard GANs -- generated perturbations are used as adversarial examples, and a discriminator is used to distinguish between them. The performance on MNIST, SVHN, and CIFAR10 demonstrate the effectiveness of the approach, and in general, the performance is on par with carefully crafted algorithms for such task.