Jacobian Adversarially Regularized Networks for Robustness

1 INTRODUCTION . Deep learning models have shown impressive performance in a myriad of classification tasks ( LeCun et al. , 2015 ) . Despite their success , deep neural image classifiers are found to be easily fooled by visually imperceptible adversarial perturbations ( Szegedy et al. , 2013 ) . These perturbations can be crafted to reduce accuracy during test time or veer predictions towards a target class . This vulnerability not only poses a security risk in using neural networks in critical applications like autonomous driving ( Bojarski et al. , 2016 ) but also presents an interesting research problem about how these models work . Many adversarial attacks have come into the scene ( Carlini & Wagner , 2017 ; Papernot et al. , 2018 ; Croce & Hein , 2019 ) , not without defenses proposed to counter them ( Gowal et al. , 2018 ; Zhang et al. , 2019 ) . Among them , the best defenses are based on adversarial training ( AT ) where models are trained on adversarial examples to better classify adversarial examples during test time ( Madry et al. , 2017 ) . While several effective defenses that employ adversarial examples have emerged ( Qin et al. , 2019 ; Shafahi et al. , 2019 ) , generating strong adversarial training examples adds non-trivial computational burden on the training process ( Kannan et al. , 2018 ; Xie et al. , 2019 ) . Adversarially trained models gain robustness and are also observed to produce more salient Jacobian matrices ( Jacobians ) at the input layer as a side effect ( Tsipras et al. , 2018 ) . These Jacobians visually resemble their corresponding images for robust models but look much noisier for standard non-robust models . It is shown in theory that the saliency in Jacobian is a result of robustness ( Etmann et al. , 2019 ) . A natural question to ask is this : can an improvement in Jacobian saliency induce robustness in models ? In other words , could this side effect be a new avenue to boost model robustness ? To the best of our knowledge , this paper is the first to show affirmative findings for this question . To enhance the saliency of Jacobians , we draw inspirations from neural generative networks ( Choi et al. , 2018 ; Dai & Wipf , 2019 ) . More specifically , in generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ) , a generator network learns to generate natural-looking images with a training objective to fool a discriminator network . In our proposed approach , Jacobian Adversarially ∗Corresponding author : guoweial001 @ ntu.edu.sg 1Source code available at https : //github.com/alvinchangw/JARN ICLR2020 Regularized Networks ( JARN ) , the classifier learns to produce salient Jacobians with a regularization objective to fool a discriminator network into classifying them as input images . This method offers a new way to look at improving robustness without relying on adversarial examples during training . With JARN , we show that directly training for salient Jacobians can advance model robustness against adversarial examples in the MNIST , SVHN and CIFAR-10 image dataset . When augmented with adversarial training , JARN can provide additive robustness to models thus attaining competitive results . All in all , the prime contributions of this paper are as follows : • We show that directly improving the saliency of classifiers ’ input Jacobian matrices can increase its adversarial robustness . • To achieve this , we propose Jacobian adversarially regularized networks ( JARN ) as a method to train classifiers to produce salient Jacobians that resemble input images . • Through experiments in MNIST , SVHN and CIFAR-10 , we find that JARN boosts adversarial robustness in image classifiers and provides additive robustness to adversarial training . 2 BACKGROUND AND RELATED WORK . Given an input x , a classifier f ( x ; θ ) : x 7→ Rk maps it to output probabilities for k classes in set C , where θ is the classifier ’ s parameters and y ∈ Rk is the one-hot label for the input . With a training dataset D , the standard method to train a classifier f is empirical risk minimization ( ERM ) , through minθ E ( x , y ) ∼DL ( x , y ) , where L ( x , y ) is the standard cross-entropy loss function defined as L ( x , y ) = E ( x , y ) ∼D [ −y > log f ( x ) ] ( 1 ) While ERM trains neural networks that perform well on holdout test data , their accuracy drops drastically in the face of adversarial test examples . With an adversarial perturbation of magnitude ε at input x , a model is robust against this attack if arg max i∈C fi ( x ; θ ) = arg max i∈C fi ( x+δ ; θ ) , ∀δ ∈ Bp ( ε ) = δ : ‖δ‖p ≤ ε ( 2 ) We focus on p =∞ in this paper . Adversarial Training To improve models ’ robustness , adversarial training ( AT ) ( Goodfellow et al. , 2016 ) seek to match the training data distribution with the adversarial test distribution by training classifiers on adversarial examples . Specifically , AT minimizes the loss function : L ( x , y ) = E ( x , y ) ∼D [ max δ∈B ( ε ) L ( x+δ , y ) ] ( 3 ) where the inner maximization , maxδ∈B ( ε ) L ( x+δ , y ) , is usually performed with an iterative gradient-based optimization . Projected gradient descent ( PGD ) is one such strong defense which performs the following gradient step iteratively : δ ← Proj [ δ − η sign ( ∇δL ( x+δ , y ) ) ] ( 4 ) where Proj ( x ) = arg minζ∈B ( ε ) ‖x−ζ‖ . The computational cost of solving Equation ( 3 ) is dominated by the inner maximization problem of generating adversarial training examples . A naive way to mitigate the computational cost involved is to reduce the number gradient descent iterations but that would result in weaker adversarial training examples . A consequence of this is that the models are unable to resist stronger adversarial examples that are generated with more gradient steps , due to a phenomenon called obfuscated gradients ( Carlini & Wagner , 2017 ; Uesato et al. , 2018 ) . Since the introduction of AT , a line of work has emerged that also boosts robustness with adversarial training examples . Capturing the trade-off between natural and adversarial errors , TRADES ( Zhang et al. , 2019 ) encourages the decision boundary to be smooth by adding a regularization term to reduce the difference between the prediction of natural and adversarial examples . Qin et al . ( 2019 ) seeks to smoothen the loss landscape through local linearization by minimizing the difference between the real and linearly estimated loss value of adversarial examples . To improve adversarial training , Zhang & Wang ( 2019 ) generates adversarial examples by feature scattering , i.e. , maximizing feature matching distance between the examples and clean samples . Tsipras et al . ( 2018 ) observes that adversarially trained models display an interesting phenomenon : they produce salient Jacobian matrices ( ∇xL ) that loosely resemble input images while less robust standard models have noisier Jacobian . Etmann et al . ( 2019 ) explains that linearized robustness ( distance from samples to decision boundary ) increases as the alignment between the Jacobian and input image grows . They show that this connection is strictly true for linear models but weakens for non-linear neural networks . While these two papers show that robustly trained models result in salient Jacobian matrices , our paper aims to investigate whether directly training to generate salient Jacobian matrices can result in robust models . Non-Adversarial Training Regularization Provable defenses are first proposed to bound minimum adversarial perturbation for certain types of neural networks ( Hein & Andriushchenko , 2017 ; Weng et al. , 2018 ; Raghunathan et al. , 2018 ) . One of the most advanced defense from this class of work ( Wong et al. , 2018 ) uses a dual network to bound the adversarial perturbation with linear programming . The authors then optimize this bound during training to boost adversarial robustness . Apart from this category , closer to our work , several works have studied a regularization term on top of the standard training objective to reduce the Jacobian ’ s Frobenius norm . This term aims to reduce the effect input perturbations have on model predictions . Drucker & Le Cun ( 1991 ) first proposed this to improve model generalization on natural test samples and called it ‘ double backpropagation ’ . Two subsequent studies found this to also increases robustness against adversarial examples Ross & Doshi-Velez ( 2018 ) ; Jakubovitz & Giryes ( 2018 ) . Recently , Hoffman et al . ( 2019 ) proposed an efficient method to approximate the input-class probability output Jacobians of a classifier to minimize the norms of these Jacobians with a much lower computational cost . Simon-Gabriel et al . ( 2019 ) proved that double backpropagation is equivalent to adversarial training with l2 examples . Etmann et al . ( 2019 ) trained robust models using double backpropagation to study the link between robustness and alignment in non-linear models but did not propose a new defense in their paper . While the double backpropagation term improves robustness by reducing the effect that perturbations in individual pixel have on the classifiers prediction through the Jacobians norm , it does not have the aim to optimize Jacobians to explicitly resemble their corresponding images semantically . Different from these prior work , we train the classifier with an adversarial loss term with the aim to make the Jacobian resemble input images more closely and show in our experiments that this approach confers more robustness . 3 JACOBIAN ADVERSARIALLY REGULARIZED NETWORKS ( JARN ) . Motivation Robustly trained models are observed to produce salient Jacobian matrices that resemble the input images . This begs a question in the reverse direction : will an objective function that encourages Jacobian to more closely resemble input images , will standard networks become robust ? To study this , we look at neural generative networks where models are trained to produce natural-looking images . We draw inspiration from generative adversarial networks ( GANs ) where a generator network is trained to progressively generate more natural images that fool a discriminator model , in a min-max optimization scenario ( Goodfellow et al. , 2014 ) . More specifically , we frame a classifier as the generator model in the GAN framework so that its Jacobians can progressively fool a discriminator model to interpret them as input images . Another motivation lies in the high computational cost of the strongest defense to date , adversarial training . The cost on top of standard training is proportional to the number of steps its adversarial examples take to be crafted , requiring an additional backpropagation for each iteration . Especially with larger datasets , there is a need for less resource-intensive defense . In our proposed method ( JARN ) , there is only one additional backpropagation through the classifier and the discriminator model on top of standard training . We share JARN in the following paragraphs and offer some theoretical analysis in § 3.1 . Jacobian Adversarially Regularized Networks Denoting input as x ∈ Rhwc for h × w-size images with c channels , one-hot label vector of k classes as y ∈ Rk , we express fcls ( x ) ∈ Rk as the prediction of the classifier ( fcls ) , parameterized by θ . The standard cross-entropy loss is Lcls = E ( x , y ) [ −y > log fcls ( x ) ] ( 5 ) With gradient backpropagation to the input layer , through fcls with respect to Lcls , we can get the Jacobian matrix J ∈ Rhwc as : J ( x ) : = ∇xLcls = [ ∂Lcls ∂x1 · · · ∂Lcls ∂xd ] ( 6 ) where d = hwc . The next part of JARN entails adversarial regularization of Jacobian matrices to induce resemblance with input images . Though the Jacobians of robust models are empirically observed to be similar to images , their distributions of pixel values do not visually match ( Etmann et al. , 2019 ) . The discriminator model may easily distinguish between the Jacobian and natural images through this difference , resulting in the vanishing gradient ( Arjovsky et al. , 2017 ) for the classifier train on . To address this , an adaptor network ( fapt ) is introduced to map the Jacobian into the domain of input images . In our experiments , we use a single 1x1 convolutional layer with tanh activation function to model fapt , expressing its model parameters as ψ . With the J as the input of fapt , we get the adapted Jacobian matrix J ′ ∈ Rhwc , J ′ = fapt ( J ) ( 7 ) We can frame the classifier and adaptor networks as a generator G ( x , y ) Gθ , ψ ( x , y ) = fapt ( ∇xLcls ( x , y ) ) ( 8 ) learning to model distribution of pJ′ that resembles px . We now denote a discriminator network , parameterized by φ , as fdisc that outputs a single scalar . fdisc ( x ) represents the probability that x came from training images px rather than pJ′ . To train Gθ , ψ to produce J ′ that fdisc perceive as natural images , we employ the following adversarial loss : Ladv = Ex [ log fdisc ( x ) ] + EJ′ [ log ( 1− fdisc ( J ′ ) ) ] = Ex [ log fdisc ( x ) ] + E ( x , y ) [ log ( 1− fdisc ( Gθ , ψ ( x ) ) ) ] = Ex [ log fdisc ( x ) ] + E ( x , y ) [ log ( 1− fdisc ( fapt ( ∇xLcls ( x , y ) ) ) ) ] ( 9 ) Combining this regularization loss with the classification loss function Lcls in Equation ( 5 ) , we can optimize through stochastic gradient descent to approximate the optimal parameters for the classifier fcls as follows , θ∗ = arg min θ ( Lcls + λadvLadv ) ( 10 ) where λadv control how much Jacobian adversarial regularization term dominates the training . Since the adaptor network ( fapt ) is part of the generator G , its optimal parameters ψ∗ can be found with minimization of the adversarial loss , ψ∗ = arg min ψ Ladv ( 11 ) On the other hand , the discriminator ( fdisc ) is optimized to maximize the adversarial loss term to distinguish Jacobian from input images correctly , φ∗ = arg max φ Ladv ( 12 ) Analogous to how generator from GANs learn to generate images from noise , we add [ −ε , −ε ] uniformly distributed noise to input image pixels during JARN training phase . Figure 1 shows a summary of JARN training phase while Algorithm 1 details the corresponding pseudo-codes . In our experiments , we find that using JARN framework only on the last few epoch ( 25 % ) to train the classifier confers similar adversarial robustness compared to training with JARN for the whole duration . This practice saves compute time and is used for the results reported in this paper . Algorithm 1 : Jacobian Adversarially Regularized Network 1 Input : Training data Dtrain , Learning rates for classifier fcls , adaptor fapt and discriminator fdisc : ( α , β , γ ) 2 for each training iteration do 3 Sample ( x , y ) ∼ Dtrain 4 x← x+ξ , ξi ∼ unif [ −ε , ε ] 5 Lcls ← −y > log fcls ( x ) . ( 1 ) Compute classification cross-entropy loss 6 J ← ∇xLcls . ( 2 ) Compute Jacobian matrix 7 J ′ ← fapt ( J ) . ( 3 ) Adapt Jacobian to image domain 8 Ladv ← log fdisc ( x ) + log ( 1− fdisc ( J ′ ) ) . ( 4 ) Compute adversarial loss 9 θ ← θ − α∇θ ( Lcls + λadvLadv ) . ( 5a ) Update the classifier fcls to minimize Lcls and Ladv 10 ψ ← ψ − β ∇ψLadv . ( 5b ) Update the adaptor fapt to minimize Ladv 11 φ← φ+ γ ∇φLadv . ( 5c ) Update the discriminator fdisc to maximize Ladv

Summary: This paper proposes a novel regularization strategy for improving the robustness of networks to adversarial noise. A term is added to the standard supervised cross-entropy loss that encourages the Jacobian of the network to itself be interpreted as a valid image. This "regularization" term is constructed by running the input-output Jacobian of the classification network through an "adapter network" and then in turn interpreting its output as a "generator" in a GAN setup. A separate discriminator network is training to distinguish real input images from these adapter-processed input-output Jacobians. The overall regularization is the standard minimax GAN loss applied to this generator/discriminator setup. The impetus for this stems from a previous observation that salient or interpretable input-output Jacobians naturally arise for networks that have undergone adversarial training to increase robustness.