Towards Counteracting Adversarial Perturbations to Resist Adversarial Examples

1 INTRODUCTION . Deep neural networks ( DNNs ) have become the dominant approach for various tasks including image understanding , natural language processing and speech recognition ( He et al. , 2016 ; Devlin et al. , 2018 ; Park et al. , 2018 ) . However , recent studies demonstrate that neural networks are vulnerable to adversarial examples ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) . That is , these network models make an incorrect prediction with high confidence for inputs that are only slightly different from correctly predicted examples . This reveals a potential threat to neural network-based artificial intelligence systems , many of which have been widely deployed in real-world applications . The adversarial vulnerability of neural networks reveals fundamental blind spots in the learning algorithms . Even with advanced learning and regularization techniques , neural networks are not learning the true underlying distribution of the training data , although they can obtain extraordinary performance on test sets . This phenomenon is now attracting much research attention . There have been increasing studies attempting to explain neural networks ’ adversarial vulnerability and develop methods to resist adversarial examples ( Madry et al. , 2018 ; Zhang et al. , 2020 ; Pang et al. , 2020 ) . While much progress has been made , most existing studies remain preliminary . Because it is difficult to construct a theoretical model to explain the adversarial perturbation generating process , defending against adversarial attacks is still a challenging task . Existing methods of resisting adversarial perturbations perform defense either at training time or inference time . Training time defense methods attempt to increase model capacity to improve adversarial robustness . One of the commonly used methods is adversarial training ( Szegedy et al. , 2014 ) , in which a mixture of adversarial and clean examples are used to train the neural network . The adversarial training method can be seen as minimizing the worst case loss when the training example is perturbed by an adversary ( Goodfellow et al. , 2015 ) . Adversarial training requires an adversary to generate adversarial examples in the training procedure . This can significantly increase the training time . Adversarial training also results in reduced performance on clean examples . Lamb et al . ( 2019 ) recently introduced interpolated adversarial training ( IAT ) that incorporates interpolation-based training into the adversarial training framework . The IAT method helps to improve performance on clean examples while maintaining adversarial robustness . As to inference time defense methods , the main idea is to transfer adversarial perturbations such that the obtained inputs are no longer adversarial . Tabacof & Valle ( 2016 ) studied the use of random noise such as Gaussian noise and heavy-tail noise to resist adversarial perturbations . Xie et al . ( 2018 ) introduced to apply two randomization operations , i.e. , random resizing and random zero padding , to inputs to improve adversarial robustness . Guo et al . ( 2018 ) investigated the use of random cropping and rescaling to transfer adversarial perturbations . More recently , Pang et al . ( 2020 ) proposed the mixup inference method that uses the interpolation between the input and a randomly selected clean image for inference . This method can shrink adversarial perturbations somewhat by the interpolation operation . Inference time defense methods can be directly applied to off-the-shelf network models without retraining or finetuning them . This can be much efficient as compared to training time defense methods . Though adversarial perturbations are not readily perceivable by a human observer , it is suggested that adversarial examples are outside the natural image manifold ( Hu et al. , 2019 ) . Previous studies have suggested that adversarial vulnerability is caused by the locally unstable behavior of classifiers on data manifolds ( Fawzi et al. , 2016 ; Pang et al. , 2018 ) . Pang et al . ( 2020 ) also suggested that adversarial perturbations have the locality property and could be resisted by breaking the locality . Existing inference time defense methods mainly use stochastic transformations such as mixup and random cropping and rescaling to break the locality . In this research , we observe that applying small perturbations generated for non-predicted class labels to the adversarial example helps to counteract the adversarial effect . Motivated by this observation , we propose a method that employs the use of small perturbations to counteract adversarial perturbations . In the proposed method , we generate small perturbation using local first-order gradient information for a number of randomly selected class lables . These small perturbations are added together and projected onto a specified space before finally applying to the adversarial example . Our method can be used as a preliminary step before applying existing inference time defense methods . To the best of our knowledge , this is the first research on using local first-order gradient information to resist adversarial perturbations . Successful attack methods such as projected gradient descent ( PGD ) ( Madry et al. , 2018 ) usually use local gradient to obtain adversarial perturbations . Compared to random transformations , it would be more effective to use local gradient to resist adversarial perturbations . We show through experiments that our method is effective and complementary to random transformation-based methods to improve defense performance . The contributions of this paper can be summarized as follows : • We propose a method that uses small first-order perturbations to defend against adversarial attacks . We show that our method is effective in counteracting adversarial perturbations and improving adversarial robustness . • We evaluate our method on CIFAR-10 and CIFAR-100 against PGD attacks in different settings . The experimental results demonstrate that our method significantly improves the defense performance of the baseline methods against both untargeted and targeted attacks and that it performs well in resisting strong adversarial examples generated using more iterations . 2 PRELIMINARY . 2.1 ADVERSARIAL EXAMPLES . We consider a neural network f ( · ) with parameters θ that outputs a vector of probabilities for L = { 1 , 2 , ... , l } categories . In supervised learning , empirical risk minimization ( ERM ) ( Vapnik , 1998 ) has been commonly used as the principle to optimize the parameters on a training set . Given an input x , the neural network makes a prediction c ( x ) = argmaxj∈L fj ( x ) . The prediction is correct if c ( x ) is the same as the actual target c∗ ( x ) . Unfortunately , ERM trained neural networks are vulnerable to adversarial examples , inputs formed by applying small but intentionally crafted perturbations ( Szegedy et al. , 2014 ; Madry et al. , 2018 ) . That is , an adversarial example x′ is close to a clean example x under a distance metric , e.g. , ℓ∞ distance , but the neural network outputs an incorrect result for the adversarial example x′ with high confidence . In most cases , the difference between the adversarial example and clean example is not readily recognizable to humans . 2.2 ATTACK METHODS . Existing attack methods can be categorized into white-box attacks and black-box attacks . We focus on defending against white-box attacks , wherein the adversary has full access to the network model including the architecture and weights . The fast gradient sign ( FGSM ) method ( Goodfellow et al. , 2015 ) and PGD are two successful optimization-based attack methods . The FGSM method is a one-step attack method . It generates adversarial perturbations that yield the highest loss increase in the gradient sign direction . Let x be the input to a network model , y the label associate with x and L ( θ , x , y ) be the loss function for training the neural network . The FGSM method generates a max-norm constrained perturbation as follows : η = εsign ( ∇xL ( θ , x , y ) ) , ( 1 ) where ε denotes the max-norm . This method was developed based on the view that the primary cause of neural networks ’ adversarial vulnerability is their linear nature . The required gradient can be computed efficiently using backpropagation . The PGD method is a multistep attack method that iteratively applies projected gradient descent on the negative loss function ( Kurakin et al. , 2016 ) as follows : xt+1 = Πx+S ( x t + αsign ( ∇xtL ( θ , xt , y ) ) ) , ( 2 ) where α denotes the step size and Π denotes the projection operator that projects the perturbed input onto x+ S. We consider projecting the perturbed input onto a predefined ℓ∞ ball from the original input . The PGD attack method can be seen as a multistep FGSM method . It is a much strong adversary that reliably causes a variety of neural networks to misclassify their input . 3 METHODOLOGY While many studies have been conducted on defending against adversarial attacks at inference time , these studies have not considered using local gradient information to resist adversarial perturbations . Previous work has suggested that the primary cause of neural networks ’ adversarial vulnerability is their linear nature ( Goodfellow et al. , 2015 ) . It would be more effective to use first-order gradient information to counteract adversarial perturbations such that the resulted perturbations no longer result in the model making an incorrect prediction . Adversarial perturbations are small crafted perturbations that slightly affect the visual quality of inputs but cause the neural network to misclassify the inputs in favor of an incorrect answer with high probability . We show that this effect can be counteracted by applying small perturbations generated using local first-order gradient information for class labels other than the predicted one . An illustration of this phenomenon is shown in Figure 1 . We see that by adding perturbations generated for non-predicted labels to the input , the prediction probability for the correct category increases and that for the incorrect label is suppressed . Algorithm 1 Counteracting adversarial perturbations using local first-order gradient . Input : Neural network f ; input x ; step size α used in PGD to generate perturbations to counteract the adver- sarial perturbation . Output : Prediction result for x . 1 : Randomly select N class labels { l1 , l2 , ... , lN } ; 2 : for i = 1 to N do 3 : ηi = PGD ( li , α , step=1 ) // generate perturbation ηi for li using the one-step PGD method . 4 : end for 5 : x = x+ΠC ( ∑N i=1 ηi ( x ) ) // C is a ℓ∞ bounded space . 6 : return f ( x ) . Based on this phenomenon , we propose a method of counteracting adversarial perturbations to improve adversarial robustness . In the proposed method , we generate small perturbations for a number of randomly selected class labels and apply these perturbations to the input to resist the adversarial perturbation . Let x be the input to a model , which can be an adversarial or clean example . We randomly select N class labels and generate small first-order perturbations for the N selected labels . These N small perturbations are added together and then projected onto a ℓ∞-bounded space before applying to the input . This procedure can be formulated as follows : x̃ = x+ΠC ( N∑ i=1 ηi ( x ) ) , ( 3 ) where ηi ( x ) denotes the small perturbation generated for the i-th selected class label , C = { t| ∥t− x∥∞ ≤ µ } is a µ bounded ℓ∞ space . The one-step PGD method is used to generate small perturbations . This is the same as using the FGSM method and empirically achieves better performance than using multiple steps . The perturbations can be generated in an untargeted or targeted manner . The combined perturbation is projected onto the space C. This ensures that the obtained example is visually similar to the original one . We detail the procedure for counteracting adversarial perturbations in Algorithm 1 . Discussion and Analysis Adversarial examples exposes underlying flaws in the training algorithms . While much progress has been made in defending against adversarial attacks , it is difficult to theoretically understand neural networks ’ vulnerability to adversarial examples . Previous work ( Athalye et al. , 2018 ) has suggested that the adversarial perturbation δ can be obtained by solving the following optimization problem : min ∥δ∥p , s.t . c ( x+ δ ) ̸= c∗ ( x ) , ∥δ∥p ≤ ξ , ( 4 ) where ξ is a hyperparameter constraining the size of the perturbation . This problem can be effectively solved by gradient descent-based attack methods such as PGD and FGSM that reliably cause neural networks to output an incorrect result . These attack methods typically use local first-order gradient to find the optimal solution . Because state-of-the-art neural networks usually have many parameters , perturbations obtained with these attack methods may overfit to the inputs . Therefore , perturbing and transferring these adversarial perturbations could be an effective way to resist the adversarial effect . Unlike previous random transformation-based methods , we employ the use of local first-order gradient information to counteract the adversarial effect . We show that the proposed method is effective in improving defense performance , especially against strong adversarial examples generated using more iterations . Let x0 be a clean example and δ be the adversarial perturbation . In our method , the following input is fed to the neural network : x0 + δ · 1z ( x0 ) +ΠC ( N∑ i=1 ηi ( x0 ) ) , where 1z ( x0 ) = { 0 , x0 is not subject to adversarial attack , 1 , x0 is subject to adversarial attack . ( 5 ) The perturbation ηi generated to counteract the adversarial perturbation should be small , otherwise it would be a new adversarial perturbation . This would essentially have no effect in counteracting the adversarial perturbation . Adversarial training that has been shown to be effective to improve adversarial robustness usually employs a first-order adversarial like PGD to provide adversarial examples for training . These adversarial examples help to regularize the model to be resistant to adversarial perturbations . We show through experiments that our method is complementary to adversarial training to improve overall defense performance against both untargeted and targeted attacks . The proposed method is applied at inference time . It can be directly applied to off-the-shelf models without retraining or finetuning them . The required gradient for generating small perturbations can be computed efficiently in parallel using backpropagation . This would not increase too much time for inference .

Summary: The paper proposes a defense that works by adding multiple targeted adversarial perturbations (with random classes) on the input sample before classifying it. There is little theoretical reasoning for why this is a sensible defense. More importantly though, the defense is only evaluated in an oblivious threat model where the attacker is unaware of the defense mechanism. As has been argued again and again in the literature and in community guidelines such as [1, 2], the oblivious threat model is trivial and yields absolutely no insights into the effectiveness of a defense (e.g. you can just manipulate the backpropagated gradient in random ways to prevent any gradient-based attack from finding adversarial perturbations). The problem with oblivious attacks is clearly visible in the results section where more PGD iterations are less effective than fewer iterations - a clear red flag that the evaluation is ineffective. The paper also fails to point out that Pang et al. 2020, one of the methods they combine their method with, has been shown to be ineffective [2].