BRIDGING ADVERSARIAL SAMPLES AND ADVERSARIAL NETWORKS

1 INTRODUCTION . Generative adversarial networks ( GANs ) have been applied successfully in various research fields such as natural image modeling ( Radford et al. , 2015 ) , image translation ( Isola et al. , 2016 ; Zhu et al. , 2017 ) , cross-modal image generation ( Dash et al. , 2017 ) , image super-resolution ( Ledig et al. , 2016 ) , semi-supervised learning ( Odena , 2016 ) and sequential data modeling ( Mogren , 2016 ; Yu et al. , 2016 ) . Different from explicit density estimation based models ( Kingma et al. , 2014 ; Oord et al. , 2016 ; Hinton , 2012 ) , GANs are implicit generative models with two neural networks playing min-max game to find a map from random noise to target distribution , in which the generator tries to generate fake samples to fool discriminator and the discriminator tries to distinguish them from real samples ( Goodfellow et al. , 2014 ) . In original GANs formula , optimal discriminator measures the Jensen-Shannon divergence between real data distribution and generated distribution . The discrepancy measure can be generalized to f-divergence ( Nowozin et al. , 2016 ) or replaced by earth-mover distance ( Arjovsky et al. , 2017 ) . Despite the success , GANs are notoriously difficult to train ( Kodali et al. , 2018 ; Arjovsky & Bottou , 2017 ) , which are very sensitive to hyper-parameters . When the support of these two distributions are approximately disjoint , gradient given by discriminator with standard objective may vanish , and training becomes unstable ( Arjovsky et al. , 2017 ) . More seriously , generated distribution can fail to cover the whole data distribution and collapse to a single mode in some cases ( Dumoulin et al. , 2017 ; Che et al. , 2016 ) . The condition of discriminator determines the training stability and performance to a great extent . On the one hand , representation capacity of discriminator realized by a neural network is not infinite . Meanwhile , the discriminator is usually not optimal to measure true discrepancy when trained in an alternative manner practically . On the other hand , discriminator as a classifier is also vulnerable to adversarial samples ( Appendix D ) : benign samples added by imperceptible perturbation can mislead classifier to give wrong prediction ( Szegedy et al. , 2014 ) ( Goodfellow et al. , 2015 ) . Adversarial samples can be easily crafted by gradient-based method such as Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2015 ) or Basic Iterative Method ( BIM ) ( Kurakin et al. , 2017 ) . It should be noted that the gradient given by discriminator that guides update of the generator is exactly the same as gradient used to craft adversarial samples of the discriminator . In other words , the gradient contains uninformative adversarial noise which is imperceptible but can mislead the generator . However , generator can still generate meaningful samples in classic GANs training procedure . This is because discriminator is adversarially trained with diverse generated fake samples . Nevertheless , adversarial training on real samples does not exist in classic training framework . As a consequence , training will become unstable when generated distribution approximates target distribution because the gradient given by non-robust discriminator around real samples contains more adversarial noise . To this end , we introduce adversarial training on real samples into classic GANs training framework to further improve the robustness of discriminator , which can reduce adversarial noise contained in gradient . It can be proved by results shown in Figure 1 empirically that the noise in gradient of adversarially trained discriminator is partly eliminated . Meanwhile , our proposed method can regularize the capability discriminator by performing adversarial training both on real samples and fake samples to alleviate training collapse . We validate the proposed method on image generation tasks with widely adopted DCGAN ( Radford et al. , 2015 ) and ResNet ( He et al. , 2015 ; Gulrajani et al. , 2017 ) architecture , which shows consistent improvement of training stability and acceleration of convergence . To our best knowledge , this is the first work to consider GANs from the perspective of adversarial samples , besides which we make GANs training scheme symmetric and improve performance efficiently with acceptable computation cost . We term the proposed method as adversarial symmetric GAN ( AS-GAN ) . 2 RELATED WORK . There is a large body of work on how to stabilize GANs training and alleviate mode collapse . ( Arjovsky & Bottou , 2017 ) proved that the widely adopted non-saturating loss function for the generator can be decomposed into Kullback–Leibler divergence minus two Jensen-Shannon divergence when discriminator trained to be optimal , which accounts for training instability and mode dropping during GANs training . ( Metz et al. , 2016 ) proposed to unroll the optimization of discriminator as surrogate objective to guide update of the generator , which shows improvement of training stability with relatively large computation overhead . ( Kodali et al. , 2018 ) claimed that the existence of undesirable local equilibria is responsible for mode collapse and proposed to regularize the discriminator around real data samples with gradient penalty . Integral probability metric ( IPM ) based GANs such as Wasserstein GAN ( Arjovsky et al. , 2017 ) and its variants ( Gulrajani et al. , 2017 ; Wu et al. , 2018 ) can solve gradient vanishing in GANs training theoretically but it is not simple to make discriminator 1-Lipschitz required by the duality conversion practically . Wasserstein GAN ( Arjovsky et al. , 2017 ) suggests earth-mover ( EM ) distance as a measure of discrepancy between two distributions and adopts weight clip to make the discriminator 1-Lipschitz constrained . WGAN-GP ( Gulrajani et al. , 2017 ) adopts gradient penalty to regularize discriminator in a less rigorous way , but it requires calculation of the second order derivative with remarkable computation overhead . Spectral normalization on the weight of discriminator proposed by ( Miyato et al. , 2018 ) can make discriminator 1-Lipschitz constrained efficiently , but capacity of discriminator is significantly constrained . Adversarial vulnerability is an intriguing property of neural network-based classifier ( Szegedy et al. , 2014 ) . A well-trained neural network can give totally wrong prediction to adversarial samples that human can recognize accurately . Small-magnitude adversarial perturbation added to benign data can be easily calculated based on gradient ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ; Dong et al. , 2018 ) . ( Goodfellow et al. , 2015 ) proposed to augment training data with adversarial samples to improve the robustness of neural networks , which can smooth the decision boundary of classifier around training samples . Gradient of adversarially trained classifier contains more semantic information and less adversarial noise ( Tsipras et al. , 2018 ; Kim et al. , 2019 ) . Some work tried to craft or defense against adversarial samples using GANs . ( Xiao et al. , 2018 ) proposed to generate adversarial samples efficiently with GANs , in which a generator is used to generate adversarial perturbation for target classifier given original samples . ( Shen et al. , 2017 ) proposed AE-GAN to eliminate adversarial perturbation in an adversarial training manner , which can output images with better perceptual quality . Different from their motivations , our work aims at improving the robustness of discriminator by introducing adversarial training on real samples , which does not exist in classic GANs training framework . Similar to our motivation , ( Liu & Hsieh ) proposed to perform projected gradient attack in supervised GAN training to robustify discriminator . In addition to that , our work clarifies that standard GAN training is approximately equivalent to adversarial training on fake samples . Moreover , we incorporate the proposed adversarial training scheme into unsupervised GAN and achieve improvement of FID score by a large margin . 3 METHOD . 3.1 CLASSIC GAN TRAINING FRAMEWORK . In GAN training framework proposed by ( Goodfellow et al. , 2014 ) , the generator Gφ ( z ) parameterized by φ tries to generate fake samples to fool discriminator and the discriminator Dθ ( x ) parameterized by θ tries to distinguish generated samples between real samples . The formulation using min-max optimization is as follows : min φ max θ V ( θ , φ ) ( 1 ) where V ( θ , φ ) is the objective function . Equation 1 can be formulated as a binary classification problem with cross entropy loss : V ( θ , φ ) = Ex∼Pdata [ logDθ ( x ) ] + Ez∼N ( 0 , I ) [ logDθ ( 1−Gφ ( z ) ) ] ( 2 ) where Pdata is real data distribution and noise z obeys standard Gaussian distribution . When discriminator is trained to be optimal , the training objective for the generator can be reformulated as Jensen-Shannon divergence , which can measure dissimilarity between two distributions . In practice , we use mini-batch gradient descent to optimize generator and discriminator alternatively . At each iteration , update rule can be derived as follows : θ′ = θ + ηθ∇θVm ( θ , φ , x , z ) ( 3 ) φ′ = φ− ηφ∇φVm ( θ , φ , x , z ) ( 4 ) where ηθ and ηφ are the learning rate of discriminator and generator , respectively . Vm ( θ , φ , x , z ) denotes the objective function of mini-batch with m real samples and m fake samples , which is : Vm ( θ , φ , x , z ) = 1 m m∑ i=1 [ logDθ ( x i ) + logDθ ( 1−Gφ ( zi ) ) ] ( 5 ) After updating parameters of networks , fake samples generated by Gφ′ ( z ) are adjusted as following equation according to chain rule : Gφ′ ( z ) ≈ Gφ ( z ) − ηφ ∂Gφ ( z ) ∂φ ∇φVm ( θ , φ , x , z ) ( 6 ) = Gφ ( z ) − ηφ ∂Gφ ( z ) ∂φ ( ∂Gφ ( z ) ∂φ ) T∇Gφ ( z ) Vm ( θ , φ , x , z ) ( 7 ) where ∂Gφ ( z ) ∂φ is a Jacobian matrix . The updated Gφ′ ( z ) can be seen as adversarial samples of the discriminator at this iteration because ηφ is usually small . These samples will be fed into the discriminator at future iteration to perform adversarial training . From this point of view , the classic training framework mainly includes adversarial training on fake samples , which is illustrated as the first pass in Figure 2 . Nevertheless , adversarial training of discriminator on real samples does not exist in this framework , which makes adversarial training unsymmetric and unbalanced . Adversarial noise contained in the gradient of non-robust discriminator can make training unstable because of the unsmoothed decision boundary of discriminator around real data . 3.2 ROUBUST OPTIMIZATION . In order to make discriminator more robust , we propose the following objective for robust optimization of discriminator : V ( θ , φ ) = Ex∼Pdata [ min ‖δ‖p≤ε logDθ ( x− δ ) ] + Ez∼N ( 0 , I ) [ logDθ ( 1−Gφ ( z ) ) ] ( 8 ) In fact , when training data of real samples is infinite , the above objective is approximately equivalent to the original . However , in practice , training data is always limited , which partly accounts for the existence of adversarial samples . With the proposed objective , discriminator is not only required to classify real data correctly but also should not be vulnerable to small perturbation . This robust optimization training scheme can smooth the decision boundary of discriminator and prevent it from being stuck in a local minimum .

Summary: This paper tries to propose a new method to stabilize the training procedure of GAN. To this end, they use adversarial samples of real data to train the discriminator, and claim that it is helpful to reduce the adversarial noise contained in the gradient. Although training GAN with adversarial samples of discriminator is somewhat novel, the method and experiments are not convincing.