Adversarial Feature Desensitization

1 Introduction . When training a classifier , it is common to assume that the training and test samples are drawn from the same underlying distribution . In adversarial machine learning , however , this assumption is intentionally violated by using the classifier itself to perturb the samples from the original ( natural ) data distribution towards a new distribution over which the classifier ’ s error rate is increased [ 52 ] . As expected , when tested on such adversarially generated input distribution , the classifier severely underperforms . To date , various methods have been proposed to defend the neural networks against adversarial attacks [ 34 , 2 ] , additive noise patterns and corruptions [ 24 , 25 , 45 ] , and transformations [ 17 ] . Among these methods , two of the most successful adversarial defense methods to date are adversarial training [ 34 ] , which trains the neural network with examples that are perturbed to maximize the loss on the target model , and TRADES [ 57 ] , which regularizes the classifier to push the decision boundary away from the data . While past adversarial defence methods have successfully improved the neural network robustness against adversarial examples , it has also been shown that these robust networks remain susceptible to even slightly larger adversarial perturbations or other forms of attacks [ 19 , 46 , 48 ] . In this paper , we propose to view the problem of adversarial robustness through the lens of domain adaptation , and to consider distributions of natural and adversarial images as distinct input domains 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . that a classifier is expected to perform well on . We then focus our attention on learning features that are invariant under such domain shifts . Building upon domain adaptation literature [ 4 ] , we use the classification-basedH∆H-divergence to quantify the distance between the natural and adversarial domains . The theory of domain adaptation allows us to formulate a bound on the adversarial classification error ( i.e . the error under the distribution of adversarial examples ) in terms of the classification error on natural images and the divergence between the natural and adversarial features . We further propose an algorithm for minimizing the adversarial error using this bound . For this , we train a classifier and a domain discriminator to respectively minimize their losses on the label classification and domain discrimination tasks . The feature extractor is trained to minimize the label classifier ’ s loss and maximise the discriminator ’ s loss . In this way , the feature extractor network is encouraged to learn features that are both predictive for the classification task and insensitive to the adversarial attacks . The proposed setup is conceptually similar to prior work in adversarial domain adaptation [ 18 , 53 ] , where domain-invariant features are learned through an adversarial game between the domain discriminator and a feature extractor network . This setup is similar to the adversarial learning paradigm widely used in image generation and transformation [ 20 , 28 , 60 ] , unsupervised and semi-supervised learning [ 39 ] , video prediction [ 35 , 31 ] , active learning [ 47 ] , and continual learning [ 16 ] . Some prior work have also considered adversarial learning to tackle the problem of adversarial examples [ 54 , 36 , 9 , 8 ] . These methods used generative models to learn the distribution of the adversarial images [ 54 , 36 ] , or to learn the distribution of input gradients [ 9 , 8 ] . Unlike our method which learns a discriminator function between distributions of adversarial and natural features and updates the feature extractor to reduce the discriminability of those distributions . The main contributions of this work are as follows : • We apply domain-adaptation theory to the problem of adversarial robustness ; this allows to bound the adversarial error in terms of the error on the natural inputs and the divergence between the feature ( representation ) distributions of adversarial and natural domains . • Aiming to minimize this bound , we propose a method which learns adversarially robust features that are both predictive and insensitive to adversarial attacks , i.e . can not be used to discriminate between natural and adversarial data . • We empirically demonstrate the effectiveness of the proposed method in learning robust models against a wide range of attack types and attack strengths , and show that our proposed approach often significantly outperforms most previous defense methods . 2 Related Work . There is an extensive literature on mitigating susceptibility to adversarial perturbations [ 34 , 57 , 13 , 59 , 3 , 22 , 7 ] . Adversarial training [ 34 ] is one of the earliest successful attempts to improve robustness of the learned representations to potential perturbations to the input pattern by solving a min-max optimization problem . TRADES [ 57 ] adds a regularization term to the cross-entropy loss which penalizes the network for assigning different labels to natural images and their corresponding perturbed images . [ 41 ] proposed an additional regularization term ( local linearity regularizer ) that encourages the classification loss to behave linearly around the training examples . [ 55 , 51 ] proposed to regularize the flatness of the loss to improve adversarial robustness . Our work is closely related to the domain adaptation literature in which adversarial optimization has recently gained much attention [ 18 , 32 , 53 ] . From this viewpoint one could consider the clean and perturbed inputs as two distinct domains for which a network aims to learn an invariant feature set . Although in our setting , i ) the perturbed domain continuously evolves while the parameters of the feature network are tuned ; ii ) unlike the usual setting in domain-adaptation problems , here we have access to the labels associated with some samples from the perturbed ( target ) domain . Recent work [ 49 ] regularized the network to have similar logit values in response to clean and perturbed inputs and showed that this additional term leads to better robust generalization to unseen perturbations . Related to this , Adversarial Logit Pairing [ 27 ] increases robustness by directly matching the logits for clean and adversarial inputs . JARN [ 9 ] Another line of work is on developing certified defenses which consist of methods with provable bounds over which the network is certified to operate robustly [ 58 , 56 , 10 ] . While these approaches provide a sense of guarantee about the proposed defenses , they are usually prohibitively expensive to train , drastically reduce the performance of the network on natural images , and the empirical robustness gained against standard attacks is low . 3 Our approach . We will now make a connection between the domain adaptation and adversarial robustness , and build upon this connection to develop an approach for improving the network ’ s robustness against adversarial attacks . 3.1 Preliminaries . Let Fθ ( x ) : X → Z , where X ⊆ Rn , Z ⊆ Rm , be a feature extractor ( e.g . a neural network with parameters θ ) mapping the input x ∈ X into the feature vector ( representation ) z ∈ Z , and let Cφ : Z → Y , where Y = { 1 , . . . , K } are the class labels , be a classifier , with parameters φ ( e.g. , the last linear layer of a neural network plus the softmax function , on top of the extracted features ) . Adversarial attack : Let π ( x , ) denote a perturbation function ( an adversarial attack ) which , for a given ( x , y ) ∈ X × Y , generates a perturbed sample x′ ∈ B ( x , ) within the -neighborhood of x , B ( x , ) = { x′ ∈ X : ‖x′ − x‖ < } , by solving the following maximization problem max t∈B ( x , ) L ( Cφ ( Fθ ( t ) ) , y ) , ( 1 ) where L is the task classification loss function . In practice , however , the perturbed sample x′ found by an attacker is typically an approximate rather than the exact solution to this maximization problem . In order to characterize the distance between the natural and adversarial data distributions , the following notion of distance between two probability distributions , defined in [ 4 , 18 ] , will be used later to make a connection with domain adaptation theory . H∆H-distance : Let H be a set of binary classifiers ( hypotheses ) , called a hypothesis space ; then the symmetric difference hypothesis space H∆H defines the set of hypotheses that capture the disagreements between two hypotheses inH , as in [ 4 ] : g ∈ H∆H ⇐⇒ g ( x ) = h ( x ) ⊕ h′ ( x ) for some h , h′ ∈ H , ( 2 ) where ⊕ denotes the XOR function . Then theH∆H-distance [ 4 , 18 ] between two data distributions ( domains ) S and T , with respect to the hypothesis spaceH , is defined as : dH∆H ( S , T ) = 2 sup h∈H∆H |Px∼S [ h ( x ) = 1 ] − Px∼T [ h ( x ) = 1 ] | . ( 3 ) This equation turns into an inequation when the supremum is taken over the hypothesis space H instead ofH∆H [ 18 ] . 3.2 A Domain Adaptation View of Adversarial Robustness . A domain is defined as a data distribution D on the set of inputs X [ 5 ] . In the adversarial robustness setting , we consider two domains – the natural and the adversarial domains , corresponding respectively to the source and target domains in domain adaptation . We denote by DX and D′X the natural and adversarial distributions of input instances respectively and by DZ and D′Z their corresponding induced distributions over the feature space Z . As in domain adaptation , we assume that f : X → Y is a labeling function common to both domains . The expected classification error Z of the classifier Cφ over DZ is defined as the probability that the classifier Cφ disagrees with the function f̃ : Z ( Cφ ) = Ez∼DZ [ y 6= Cφ ( z ) ] , ( 4 ) where f̃ : Z → Y is a mapping from the features to the class label such that f ( x ) = f̃ ( Fθ ( x ) ) . We similarly define ′Z as the expected error of Cφ over DZ′ . Using theorem 2 from [ 4 ] that relates the source and the target domain errors , we get an upper bound on the expected adversarial error ′Z as : ′Z ( h ) ≤ Z ( h ) + 1 2 dH∆H ( DZ , D′Z ) + c , ( 5 ) where c is a constant term w.r.t . h. Eq . 5 essentially gives a bound on the adversarial error ′Z in terms of the natural error Z and a divergence dH∆H between the natural and adversarial domains with respect to their induced representation distributions DZ and D′Z . In the next section , we will describe an algorithm for improving adversarial robustness of a model by iteratively estimating and minimizing these two components of the error bound . 3.3 Adversarial Feature Desensitization . Based on Eq . 5 , the expected adversarial error could be reduced by jointly minimizing the natural error and the divergence between the distributions of natural and adversarial representations dH∆H ( DZ , D′Z ) . While minimizing the natural error X is straightforward , minimizing the crossdomain divergence requires us to estimate dH∆H ( DZ , D′Z ) . As was shown before [ 18 ] , training a domain discriminator Dψ is closely related to estimating the dH∆H ( DZ , D′Z ) . The domain discriminator is a classifier trained to assign a label of 1 to samples from DZ , and -1 to samples from D′Z . Namely , it is shown [ 18 ] that dH∆H ( DZ , D′Z ) ≤ 2 sup h∈H |αDZ , D′Z ( h ) − 1| , ( 6 ) where αDZ , D′Z ( h ) = Pz∼DZ [ h ( z ) = 1 ] +Pz∼D′Z [ h ( z ) = −1 ] combines the true positives and true negatives , and is thus maximized by the optimal domain discriminator h = Dψ . Note that , if the domain distributions DZ and D′Z are the same , then even the best choice of domain discriminator Dψ will achieve chance-level accuracy , corresponding to αDZ , D′Z ( Dψ ) = 1 . Our approach will aim at minimizing this estimated distance dH∆H ( DZ , D′Z ) by tuning the feature extractor network parameters θ in the direction that pushes the distributions DZ and D′Z closer together . In parallel , we train the domain discriminator to estimate and guide the progress of the feature extractor ’ s tuning . We now describe the proposed approach ( see Algorithm 1 ) which essentially involves simultaneous training of the feature extractor Fθ , the task classifier Cφ and the domain discriminator Dψ ( see Figure 1a ) 1 . One iteration of the training procedure consists of the following three steps . First , parameters of the feature extractor Fθ and classifier Cφ are updated aiming to minimize the natural error X using the cross-entropy loss on natural inputs : LC = − 1 m m∑ i=1 ỹi · log ( softmax ( Cφ ( Fθ ( xi ) ) ) ) , ( 7 ) where ỹi is a one-hot encoding of the true label of the i-th sample xi . Next , steps two and three essentially implement a two-player minimax game similar to that in Generative Adversarial Networks ( GAN ) [ 20 ] , carried out between the feature extractor network Fθ and the domain discriminator Dψ , with a value function V ( Fθ , Dψ ) = Ep ( y ) [ Ep ( x|y ) [ S ( −Dψ ( Fθ ( x ) , y ) ) ] ] + Eq ( y ) [ Eq ( x|y ) [ S ( Dψ ( Fθ ( x ) , y ) ) ] ] , ( 8 ) 1Note that we will somewhat abuse the notation , assuming that Cφ and Dψ below correspond to the logits ( last-layer output ) of the corresponding networks . Also , we will use class-conditional discriminators , Dψ ( Fθ ( x , y ) ) , i.e . train different domain discriminator for different label values y. Algorithm 1 : AFD training procedure Input : Adversarial perturbation function ( attack ) π , feature extractor Fθ , task classifier Cφ , domain discriminator Dψ , learning rates α , β , and γ. repeat input next mini-batch { ( xi , yi ) , ... , ( xm , ym ) } for i=1 to m : x′i ← π ( xi , ) Compute LC according to Eq . 7 Compute LD according to Eq . 9 Compute LF according to Eq . 10 ( θ , φ ) ← ( θ , φ ) − α∇θ , φLC % update feature extractor and task classifier ψ ← ψ − β∇ψLD % update domain discriminator θ ← θ − γ∇θLF % update feature extractor until convergence ; where S is the softplus function . In particular , parameters of the domain discriminator Dψ are updated to minimize the cross-entropy loss associated with discriminating natural and adversarial inputs , maximizing α ( h ) in Eq . 6 . LD = 1 m m∑ i=1 [ S ( −Dψ ( Fθ ( xi ) , yi ) ) + S ( Dψ ( Fθ ( x′i ) , yi ) ) ] , ( 9 ) while the parameters of the feature extractor function Fθ are adversarially updated to maximize the domain discriminator ’ s loss from Eq . 9 LF = 1 m m∑ i=1 S ( −Dψ ( Fθ ( x′i ) , yi ) ) . ( 10 ) In Figure 1b , we visually compare the learning dynamics in adversarial training , TRADES and AFD . Essentially , the adversarial training solves the classification problem by pushing the representation of adversarial examples from different classes away . TRADES regularizes the normal classification loss on the natural inputs with an additional term that encourages the representation of adversarial and natural images to match . Similar to TRADES , in AFD , the regular classification loss on natural inputs is augmented but with an adversarial game which consists of training the domain discriminator that distinguishes between the adversarial and natural inputs for each class followed by updates to the feature extractor to make the representations for natural and adversarial examples to become indistinguishable from each other . Notably , because the parameter update for the feature extractor network is done to maximize the domain discriminator loss and not to decrease the loss for particular adversarial examples ( as is done in adversarial training or TRADES ) , it potentially increases the network robustness against any perturbation that could be correctly classified using the same domain discriminator . This could potentially lead to a broader form of generalization learned by the network . Discussion : Relation to Adversarial Training . Adversarial training minimizes the expected error on adversarial examples ( the perturbed versions of the natural samples ) , generated by an attacker in order to maximize the classification loss . The adversarial training procedure involves a minimax optimization problem consisting of an inner maximization to find adversarial examples that maximize the classification loss and an outer minimization to find model parameters that minimize the adversarial loss . From the domain adaptation point of view , the inner optimization of adversarial training is equal to a sampling procedure that generates samples from the target domain . Intuitively , direct training of the classifier on samples from the target domain would be the best way to improve the accuracy in that domain ( i.e . adversarial classification accuracy ) . However , it ’ s important to note that the adversarial examples found through the inner optimization only approximately maximize the classification loss , and therefore the adversarial error associated with these samples only act as a lower bound on the true adversarial error and therefore the outer loop of the adversarial training method essentially minimizes a lower bound on the adversarial classification error . In contrast to this setup , our proposed method minimizes a conservative upper bound on the adversarial error and therefore is more likely to generalize to a larger set of unseen attacks , and to stronger versions of previously seen attacks ( i.e . ones that generate higher-loss samples in the inner optimization loop ) .

Summary: The paper proposes Adversarial Feature Desensitization (AFD) to train classifiers robust to adversarial attacks. In particular, AFD trains jointly a feature extractor, a classifier for the original task and a discriminator which distinguishes between natural and adversarial inputs, in order to find features which are effective for classification but not sensitive to adversarial attacks. In the experimental evaluation on four datasets, AFD outperforms standard methods in most of the cases.