Learning Contextual Perturbation Budgets for Training Robust Neural Networks

1 INTRODUCTION . It has been demonstrated that deep neural networks , although achieving impressive performance on various tasks , are vulnerable to adverarial perturbations ( Szegedy et al. , 2013 ) . Models with high accuracy on clean and unperturbed data can be fooled to have extremely poor performance when input data are adversarially perturbed . The existence of adversarial perturbations causes concerns in safety-critical applications such as self-driving cars , face recognition and medical diagnosis . A number of methods have been proposed for training robust neural networks that can resist to adversarial perturbations to some extent . Among them , adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) and certified defenses ( Wong et al. , 2018 ; Gowal et al. , 2018 ; Zhang et al. , 2020 ) are of the most reliable ones so far , and most of them are trying to make the network robust to any perturbation within an ` p norm ball . Taking the commonly used ` ∞-ball defense as an example , robust training methods aim to make model robust to perturbation on any pixel , which means the model is uniformly robust on all the input dimensions . But is this a valid assumption we should make ? As we know , human perception is non-uniform ( humans focus on important features even though these features can be sensitive to small noise ) and context dependent ( what part of image is important heavily depends on what is on the image ) . We expect a robust model to be close to human perception , rather than learn to defend against a particular fixed threat model , e.g. , the traditional ` ∞-norm one . Intuitively , we expect a good model to be more sensitive to important features and less sensitive to unimportant features , and the importance of features should be context-dependent . Taking the MNIST hand-written digit classification problem as an example , the digit 9 can be transformed to 4 simply by modifying just a few pixels on its head , so those pixels should be considered more important , and enforcing them to be robust to a large perturbation may not be correct . On the other hand , the pixels on the frame of such an input image can be safely modified without changing the ground-truth label of the image . Therefore , a uniform budget in robust training can greatly hamper the performance of neural networks on certain tasks , and will force network to ignore some important features that are important for classification . Robustness certification with non-uniform perturbation budgets has been discussed in a prior work ( Liu et al. , 2019 ) , but training robust models and learning context-dependent perturbation budgets has not been addressed in prior works , which is more challenging and important for obtaining robust models . A detailed discussion on our difference with Liu et al . ( 2019 ) is in Sec . 2.2 . In this paper , we propose the first method that can learn context-dependent non-uniform perturbation budgets in certified robust training , based on prior certified defense algorithms on ` p-norm threat models ( Zhang et al. , 2020 ; Xu et al. , 2020 ) . To learn a context-dependent budget without introducing too many learnable parameters , we introduce a perturbation budget generator with an auxiliary neural network , to generate the context-dependent budgets based on the input image . We also impose constraints on the generator to make generated budgets satisfy target robustness volumes and ranges of budgets , where robustness volume is defined as the multiplication of budgets on all input dimensions . We then train the classifier with a linear-relaxation-based certified defense algorithm , auto LiRPA ( Xu et al. , 2020 ) generalized from CROWN-IBP ( Zhang et al. , 2020 ) , to minimize the verified error under given budget constraints . The gradients of the loss function can be back-propagated to the perturbation budgets , allowing training the classification network and budget generator jointly in robust training . Our contribution can be summarized below : • We propose a novel algorithm to train robust networks with contextual perturbation budgets rather than uniform ones . We show that it can be incorporated into certified defense methods with linear relaxation-based robustness verification . • We demonstrate that our method can effectively train both the classifier and the perturbation generator jointly , and we able to train models on relatively larger robustness volumes and outperform those trained with uniform budgets . • We also show that the learned perturbation budgets are semantically meaningful and align well with the importance of different pixels in the input image . We further confirm this with two synthetic tasks and datasets constructed from MNIST and CIFAR-10 respectively . 2 BACKGROUND AND RELATED WORK . 2.1 TRAINING ROBUST NEURAL NETWORKS . Since the discovery of adversarial examples ( Szegedy et al . ( 2013 ) , Biggio et al . ( 2013 ) ) , a great number of works has been devoted to improving the robustness of neural networks from both attack and defense perspectives ( Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017 ; Papernot et al. , 2016 ; Moosavi-Dezfooli et al. , 2017 ; Gowal et al. , 2019 ) . On a K-way classification task , training an adversarially robust neural network fw with weight w can generally be formulated as solving the following min-max optimization problem : min w E ( x , y ) ∼D max δ∈∆ L ( fw ( x + δ ) , y ) , ( 1 ) where D is the data distribution , and ∆ is a threat model defining the space of the perturbations , and L is a loss function . Adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) applies adversarial attacks to solve the inner maximization problem and train the neural network on generated adversarial examples , with efficiency advanced in some recent works ( Shafahi et al. , 2019 ; Wong et al. , 2020 ) . However , robustness improvements from adversarial training do not have provable guarantees . Some other recent works seek to train networks that have provable robustness , namely certified defense methods . Such methods solves the inner maximization by computing certified upper bounds that provably hold true for any perturbation within the threat model , including abstract interpretation ( Singh et al. , 2018 ) , interval bound propagation ( Gowal et al. , 2018 ; Mirman et al. , 2018 ) , randomized smoothing ( Cohen et al. , 2019 ; Salman et al. , 2019 ; Zhai et al. , 2020 ) , and linear-relaxation-based methods ( Wong & Kolter , 2018 ; Mirman et al. , 2018 ; Wong et al. , 2018 ; Zhang et al. , 2020 ; Xu et al. , 2020 ) . However , nearly all existing certified defense methods treat all input features equally in the threat model , such as an ` p-ball threat models , ∆= { δ : ‖δ ‖p≤ } , especially the ` ∞ threat model commonly used in many previous works . But different input pixels are not uniformly important to the prediction , and thus we propose to learn contextual perturbation budgets for different pixels . 2.2 ROBUSTNESS CERTIFICATION WITH NON-UNIFORM BUDGETS . There has been a recent work Liu et al . ( 2019 ) studying robustness certification with non-uniform budgets . Their work aims to maximize the robustness volume while ensuring the network prediction is certifiably correct , and this optimization problem is solved using augmented Lagrangian method . To highlight , their method has several limitations : 1 ) The perturbation budgets are obtained by solving a constrained optimization problem for each input example , which is very time-consuming . This is not only too inefficient for training as it can bring a large overhead in each training step , but also incapable for learning perturbation budgets jointly with the classifier . In contrast , we have a significantly different scheme – we aim to maximize the accuracy under given target robustness volumes . And our perturbation budgets is obtained with a lightweight neural-network-based generator , and it can be jointly trained with the classifier in an end-to-end way . 2 ) Their work only focuses on certifying trained models due to the inherent limitation of the method , while we address of the problem of training robust neural networks and learning contextual perturbation budgets simultaneously . Consequently , we are able to empirically demonstrate that we can effectively train robust models with much larger robustness volumes and achieve lower errors under given target robustness volumes . 3 ) We further have experiments with synthetic tasks on MNIST and CIFAR-10 in Sec . 4.2 and Sec . 4.3 respectively to demonstrate that the learned perturbation budgets are semantically meaningful and can capture contextual information . 3 PROPOSED METHOD . 3.1 PROBLEM SETTING . Variable threat model Unlike many previous works that use an ` ∞ threat model with a uniform on all input dimensions , we allow different pixels to have different perturbation budgets , but they need to meet some constraints as we will define below . For an n-dimensional input x , when the perturbation budget of pixel xi is i , the threat model is ∆ ( 1 , 2 , · · · , n ) = { δ : |δi| ≤ i , 1 ≤ i ≤ n } . And thereby the ` ∞ threat model is a special case with i = 0 ( ∀1 ≤ i ≤ n ) . We define the robustness volume of a threat model ∆ as the multiplication of all i ( 1 ≤ i ≤ n ) : V ( ∆ ) = n∏ i=1 i , i.e. , log V ( ∆ ) = n∑ i=1 log i . ( 2 ) In principal , we have a target robustness volume , V0 = n0 , and fw is considered to be provably robust under this target robustness volume on instance ( x , y ) if and only if : ∃∆ ∈ D , ∀k 6= y , min δ∈∆ ( [ fw ( x + δ ) ] y − [ fw ( x + δ ) ] k ) > 0 . ( 3 ) This means that the predicted score of the ground-truth class y is certifiably larger than any other class k 6= y under some threat model ∆ . And instead of using a fixed ∆ , in our framework ∆ can be taken from a threat model space D , which consists of threat models ∆ ( 1 , 2 , · · · , n ) satisfying the following two constraints : Volume constraint : ∑ i log i = n log 0 , ( 4 ) Range constraint : l ≤ i ≤ u , l = α 0 , u = min ( α 0 , 1 ) . ( 5 ) The volume constraint states that the robustness volume of ∆ is equal to the target robustness volume n0 ( written in log domain above ) . For the range constraint , there are relative factors α and α controlling the perturbation budget range of each pixel , namely [ l , u ] . This constraint can be set to guarantee a minimum robustness and also prevent the model from over-invariant on each pixel . Perturbation budget generation A classifier can be accompanied by a perturbation budget generator ( x ) which tries to find perturbation budgets 1 , 2 , · · · , n and the corresponding threat model ∆ ( x ) that can minimize the verified loss of the classifier while satisfying constraints ( 4 ) and ( 5 ) , so that ( 3 ) is more likely to hold true with ∆ generated by ( x ) . We will state the optimization problem in the next paragraph . Robust classification Accompanied by a perturbation budget generator , we aim to learn a robust classifier fw with the following min-max optimization problem : min w E ( x , y ) ∼D max δ∈∆ ( x ) L ( fw ( x + δ ) , y ) . ( 6 ) Note that a key difference between this problem and the traditional one in ( 1 ) is that now the threat model ∆ ( x ) is variable and dependent on the input x . This ∆ ( x ) is generated by ( x ) , under the given volume and range constraints . We evaluate the robustness of a classifier fw by computing an average verified error on all the test instances , where the verified correctness on each instance is evaluated similarly as ( 3 ) , where ∆ is taken as the generated ∆ ( x ) .

Summary: This paper proposes to change the perturbation budget for adversarial attacks to a non-uniform setting where differet input pixels have different perturbation budgets. To achieve this, an additional network is trained to learn the perturbation budget for each part of the input. The approach seems to perform better than a uniform perturbation budget and also learns semantically meaningful budgets for the input.