Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attacks

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved state-of-the-art performance on various tasks ( Goodfellow et al. , 2016 ) . However , counter-intuitive adversarial examples generally exist in different domains , including computer vision ( Szegedy et al. , 2014 ) , natural language processing ( Jin et al. , 2019 ) , reinforcement learning ( Huang et al. , 2017 ) , speech ( Carlini & Wagner , 2018 ) and graph data ( Dai et al. , 2018 ) . As DNNs are being widely deployed , it is imperative to improve model robustness and defend adversarial attacks , especially in safety-critical cases . Previous work shows that adversarial examples mainly root from the locally unstable behavior of classifiers on the data manifolds ( Goodfellow et al. , 2015 ; Fawzi et al. , 2016 ; 2018 ; Pang et al. , 2018b ) , where a small adversarial perturbation in the input space can lead to an unreasonable shift in the feature space . On the one hand , many previous methods try to solve this problem in the inference phase , by introducing transformations on the input images . These attempts include performing local linear transformation like adding Gaussian noise ( Tabacof & Valle , 2016 ) , where the processed inputs are kept nearby the original ones , such that the classifiers can maintain high performance on the clean inputs . However , as shown in Fig . 1 ( a ) , the equivalent perturbation , i.e. , the crafted adversarial perturbation , is still δ and this strategy is easy to be adaptively evaded since the randomness of x0 w.r.t x0 is local ( Athalye et al. , 2018 ) . Another category of these attempts is to apply various non-linear transformations , e.g. , different operations of image processing ( Guo et al. , 2018 ; Xie et al. , 2018 ; Raff et al. , 2019 ) . They are usually off-the-shelf for different classifiers , and generally aim to disturb the adversarial perturbations , as shown in Fig . 1 ( b ) . Yet these methods are not quite reliable since there is no illustration or guarantee on to what extent they can work . On the other hand , many efforts have been devoted to improving adversarial robustness in the training phase . For examples , the adversarial training ( AT ) methods ( Madry et al. , 2018 ; Zhang et al. , 2019 ; Shafahi et al. , 2019 ) induce locally stable behavior via data augmentation on adversarial examples . However , AT methods are usually computationally expensive , and will often degenerate model ∗Equal contribution . †Corresponding author . performance on the clean inputs or under general-purpose transformations like rotation ( Engstrom et al. , 2019 ) . In contrast , the mixup training method ( Zhang et al. , 2018 ) introduces globally linear behavior in-between the data manifolds , which can also improve adversarial robustness ( Zhang et al. , 2018 ; Verma et al. , 2019a ) . Although this improvement is usually less significant than it resulted by AT methods , mixup-trained models can keep state-of-the-art performance on the clean inputs ; meanwhile , the mixup training is computationally more efficient than AT . The interpolated AT method ( Lamb et al. , 2019 ) also shows that the mixup mechanism can further benefit the AT methods . However , most of the previous work only focuses on embedding the mixup mechanism in the training phase , while the induced global linearity of the model predictions is not well exploited in the inference phase . Compared to passive defense by directly classifying the inputs ( Zhang et al. , 2018 ; Lamb et al. , 2019 ) , it would be more effective to actively defend adversarial attacks by breaking their locality via the globally linear behavior of the mixup-trained models . In this paper , we develop an inference principle for mixup-trained models , named mixup inference ( MI ) . In each execution , MI performs a global linear transformation on the inputs , which mixups the input x with a sampled clean example xs , i.e. , x̃ = λx+ ( 1− λ ) xs ( detailed in Alg . 1 ) , and feed x̃ into the classifier as the processed input . There are two basic mechanisms for robustness improving under the MI operation ( detailed in Sec . 3.2.1 ) , which can be illustrated by simple geometric intuition in Fig . 1 ( c ) . One is perturbation shrinkage : if the input is adversarial , i.e. , x = x0 + δ , the perturbation δ will shrink by a factor λ after performing MI , which is exactly the mixup ratio of MI according to the similarity between triangles . Another one is input transfer : after the MI operation , the reduced perturbation λδ acts on random x̃0 . Comparing to the spatially or semantically local randomness introduced by Gaussian noise or image processing , x̃0 introduces spatially global and semantically diverse randomness w.r.t x0 . This makes it less effective to perform adaptive attacks against MI ( Athalye et al. , 2018 ) . Furthermore , the global linearity of the mixup-trained models ensures that the information of x0 remained in x̃0 is proportional to λ , such that the identity of x0 can be recovered from the statistics of x̃0 . In experiments , we evaluate MI on CIFAR-10 and CIFAR-100 ( Krizhevsky & Hinton , 2009 ) under the oblivious attacks ( Carlini & Wagner , 2017 ) and the adaptive attacks ( Athalye et al. , 2018 ) . The results demonstrate that our MI method is efficient in defending adversarial attacks in inference , and is also compatible with other variants of mixup , e.g. , the interpolated AT method ( Lamb et al. , 2019 ) . Note that Shimada et al . ( 2019 ) also propose to mixup the input points in the test phase , but they do not consider their method from the aspect of adversarial robustness . 2 PRELIMINARIES . In this section , we first introduce the notations applied in this paper , then we provide the formula of mixup in training . We introduce the adversarial attacks and threat models in Appendix A.1 . 2.1 NOTATIONS . Given an input-label pair ( x , y ) , a classifier F returns the softmax prediction vector F ( x ) and the predicted label ŷ = arg maxj∈ [ L ] Fj ( x ) , where L is the number of classes and [ L ] = { 1 , · · · , L } . The classifier F makes a correct prediction on x if y = ŷ . In the adversarial setting , we augment the data pair ( x , y ) to a triplet ( x , y , z ) with an extra binary variable z , i.e. , z = { 1 , if x is adversarial , 0 , if x is clean . ( 1 ) The variable z is usually considered as hidden in the inference phase , so an input x ( either clean or adversarially corrupted ) can be generally denoted as x = x0 + δ · 1z=1 . Here x0 is a clean sample from the data manifold p ( x ) with label y0 , 1z=1 is the indicator function , and δ is a potential perturbation crafted by adversaries . It is worthy to note that the perturbation δ should not change the true label of the input , i.e. , y = y0 . For ` p-norm adversarial attacks ( Kurakin et al. , 2017 ; Madry et al. , 2018 ) , we have ‖δ‖p ≤ , where is a preset threshold . Based on the assumption that adversarial examples are off the data manifolds , we formally have x0 + δ /∈ supp ( p ( x ) ) ( Pang et al. , 2018a ) . 2.2 MIXUP IN TRAINING . In supervised learning , the most commonly used training mechanism is the empirical risk minimization ( ERM ) principle ( Vapnik , 2013 ) , which minimizes 1n ∑n i=1 L ( F ( xi ) , yi ) on the training dataset D = { ( xi , yi ) } ni=1 with the loss function L. While computationally efficient , ERM could lead to memorization of data ( Zhang et al. , 2017 ) and weak adversarial robustness ( Szegedy et al. , 2014 ) . As an alternative , Zhang et al . ( 2018 ) introduce the mixup training mechanism , which minimizes 1 m ∑m j=1 L ( F ( x̃j ) , ỹj ) . Here x̃j = λxj0 + ( 1−λ ) xj1 ; ỹj = λyj0 + ( 1−λ ) yj1 , the input-label pairs ( xj0 , yj0 ) and ( xj1 , yj1 ) are randomly sampled from the training dataset , λ ∼ Beta ( α , α ) and α is a hyperparameter . Training by mixup will induce globally linear behavior of models in-between data manifolds , which can empirically improve generalization performance and adversarial robustness ( Zhang et al. , 2018 ; Tokozume et al. , 2018a ; b ; Verma et al. , 2019a ; b ) . Compared to the adversarial training ( AT ) methods ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) , trained by mixup requires much less computation and can keep state-of-the-art performance on the clean inputs . 3 METHODOLOGY . Although the mixup mechanism has been widely shown to be effective in different domains ( Berthelot et al. , 2019 ; Beckham et al. , 2019 ; Verma et al. , 2019a ; b ) , most of the previous work only focuses on embedding the mixup mechanism in the training phase , while in the inference phase the global linearity of the trained model is not well exploited . Compared to passively defending adversarial examples by directly classifying them , it would be more effective to actively utilize the globality of mixup-trained models in the inference phase to break the locality of adversarial perturbations . 3.1 MIXUP INFERENCE . The above insight inspires us to propose the mixup inference ( MI ) method , which is a specialized inference principle for the mixup-trained models . In the following , we apply colored y , ŷ and ys to visually distinguish different notations . Consider an input triplet ( x , y , z ) , where z is unknown in advance . When directly feeding x into the classifier F , we can obtain the predicted label ŷ . In the adversarial setting , we are only interested in the cases where x is correctly classified by F if it is clean , or wrongly classified if it is adversarial ( Kurakin et al. , 2018 ) . This can be formally denoted as 1y 6=ŷ = 1z=1 . ( 2 ) The general mechanism of MI works as follows . Every time we execute MI , we first sample a label ys ∼ ps ( y ) , then we sample xs from ps ( x|ys ) and mixup it with x as x̃ = λx+ ( 1− λ ) xs . ps ( x , y ) denotes the sample distribution , which is constrained to be on the data manifold , i.e. , supp ( ps ( x ) ) ⊂ supp ( p ( x ) ) . In practice , we execute MI for N times and average the output predictions to obtain FMI ( x ) , as described in Alg . 1 . Here we fix the mixup ratio λ in MI as a hyperparameter , while similar properties hold if λ comes from certain distribution .

Summary: This paper introduces a novel method for an adversarial attack named mixup inference (MI).  Most of the work focuses on embedding mixup mechanism in the training phase, but MI uses the mixup in the inference phase. MI method has two main effects for the adversarial attack: one is perturbation shrinkage, and the other one is input transfer because MI can exploit