Training for Faster Adversarial Robustness Verification via Inducing ReLU Stability

1 INTRODUCTION . Deep neural networks ( DNNs ) have recently achieved widespread success in image classification ( Krizhevsky et al. , 2012 ) , face and speech recognition ( Taigman et al. , 2014 ; Hinton et al. , 2012 ) , and game playing ( Silver et al. , 2016 ; 2017 ) . This success motivates their application in a broader set of domains , including more safety-critical environments . This thrust makes understanding the reliability and robustness of the underlying models , let alone their resilience to manipulation by malicious actors , a central question . However , predictions made by machine learning models are often brittle . A prominent example is the existence of adversarial examples ( Szegedy et al. , 2014 ) : imperceptibly modified inputs that cause state-of-the-art models to misclassify with high confidence . There has been a long line of work on both generating adversarial examples , called attacks ( Carlini and Wagner , 2017a ; b ; Athalye et al. , 2018a ; b ; Uesato et al. , 2018 ; Evtimov et al. , 2017 ) , and training models robust to adversarial examples , called defenses ( Goodfellow et al. , 2015 ; Papernot et al. , 2016 ; Madry et al. , 2018 ; Kannan et al. , 2018 ) . However , recent research has shown that most defenses are ineffective ( Carlini and Wagner , 2017a ; Athalye et al. , 2018a ; Uesato et al. , 2018 ) . Furthermore , even for defenses such as that of Madry et al . ( 2018 ) that have seen empirical success against many attacks , we are unable to conclude yet with certainty that they are robust to all attacks that we want these models to be resilient to . This state of affairs gives rise to the need for verification of networks , i.e. , the task of formally proving that no small perturbations of a given input can cause it to be misclassified by the network model . Although many exact verifiers1 have been designed to solve this problem , the verification process is often intractably slow . For example , when using the Reluplex verifier of Katz et al . ( 2017 ) , even verifying a small MNIST network turns out to be computationally infeasible . Thus , addressing this intractability of exact verification is the primary goal of this work . Our Contributions Our starting point is the observation that , typically , model training and verification are decoupled and seen as two distinct steps . Even though this separation is natural , it misses a key opportunity : the ability to align these two stages . Specifically , applying the principle of co-design during model 1Also sometimes referred to as combinatorial verifiers . training is possible : training models in a way to encourage them to be simultaneously robust and easy-to-exactly-verify . This insight is the cornerstone of the techniques we develop in this paper . In this work , we use the principle of co-design to develop training techniques that give models that are both robust and easy-to-verify . Our techniques rely on improving two key properties of networks : weight sparsity and ReLU stability . Specifically , we first show that natural methods for improving weight sparsity during training , such as ` 1-regularization , give models that can already be verified much faster than current methods . This speedup happens because in general , exact verifiers benefit from having fewer variables in their formulations of the verification task . For instance , for exact verifiers that rely on linear programming ( LP ) solvers , sparser weight matrices means fewer variables in those constraints . We then focus on the major speed bottleneck of current approaches to exact verification of ReLU networks : the need of exact verification methods to “ branch , ” i.e. , consider two possible cases for each ReLU ( ReLU being active or inactive ) . Branching drastically increases the complexity of verification . Thus , well-optimized verifiers will not need to branch on a ReLU if it can determine that the ReLU is stable , i.e . that the ReLU will always be active or always be inactive for any perturbation of an input . This motivates the key goal of the techniques presented in this paper : we aim to minimize branching by maximizing the number of stable ReLUs . We call this goal ReLU stability and introduce a regularization technique to induce it . Our techniques enable us to train weight-sparse and ReLU stable networks for MNIST and CIFAR10 that can be verified significantly faster . Specifically , by combining natural methods for inducing weight sparsity with a robust adversarial training procedure ( cf . Goodfellow et al . ( 2015 ) ) , we are able to train networks for which almost 90 % of inputs can be verified in an amount of time that is small2 compared to previous verification techniques . Then , by also adding our regularization technique for inducing ReLU stability , we are able to train models that can be verified an additional 4–13x times as fast while maintaining state-of-the-art accuracy on MNIST . Our techniques show similar improvements for exact verification of CIFAR models . In particular , we achieve the following verification speed and provable robustness results for ` ∞ norm-bound adversaries : Dataset Epsilon Provable Adversarial Accuracy Average Solve Time ( s ) MNIST = 0.1 94.33 % 0.49 = 0.2 89.79 % 1.13 = 0.3 80.68 % 2.78 CIFAR = 2/255 45.93 % 13.50 = 8/255 20.27 % 22.33 Our network for = 0.1 on MNIST achieves provable adversarial accuracies comparable with the current best results of Wong et al . ( 2018 ) and Dvijotham et al . ( 2018 ) , and our results for = 0.2 and = 0.3 achieve the best provable adversarial accuracies yet . To the best of our knowledge , we also achieve the first nontrivial provable adversarial accuracy results using exact verifiers for CIFAR-10 . Finally , we design our training techniques with universality as a goal . We focus on improving the input to the verification process , regardless of the verifier we end up using . This is particularly important because research into network verification methods is still in its early stages , and our codesign methods are compatible with the best current verifiers ( LP/MILP-based methods ) and should be compatible with any future improvements in verification . Our code is available at https : //github.com/MadryLab/relu_stable . 2 BACKGROUND AND RELATED WORK . Exact verification of networks has been the subject of many recent works ( Katz et al. , 2017 ; Ehlers , 2017 ; Carlini et al. , 2017 ; Tjeng et al. , 2017 ; Lomuscio and Maganti , 2017 ; Cheng et al. , 2017a ) . To understand the context of these works , observe that for linear networks , the task of exact verification is relatively simple and can be done by solving a LP . For more complex models , the presence of nonlinear ReLUs makes verification over all perturbations of an input much more challenging . This 2We chose our time budget for verification to be 120 seconds per input image . is so as ReLUs can be active or inactive depending on the input , which can cause exact verifiers to “ branch '' and consider these two cases separately . The number of such cases that verifiers have to consider might grow exponentially with the number of ReLUs , so verification speed will also grow exponentially in the worst case . Katz et al . ( 2017 ) further illustrated the difficulty of exact verification by proving that it is NP-complete . In recent years , formal verification methods were developed to verify networks . Most of these methods use satisfiability modulo theory ( SMT ) solvers ( Katz et al. , 2017 ; Ehlers , 2017 ; Carlini et al. , 2017 ) or LP and Mixed-Integer Linear Programming ( MILP ) solvers ( Tjeng et al. , 2017 ; Lomuscio and Maganti , 2017 ; Cheng et al. , 2017a ) . However , all of them are limited by the same issue of scaling poorly with the number of ReLUs in a network , making them prohibitively slow in practice for even medium-sized models . One recent approach for dealing with the inefficiency of exact verifiers is to focus on certification methods3 ( Wong and Kolter , 2018 ; Wong et al. , 2018 ; Dvijotham et al. , 2018 ; Raghunathan et al. , 2018 ; Mirman et al. , 2018 ; Sinha et al. , 2018 ) . In contrast to exact verification , these methods do not solve the verification task directly ; instead , they rely on solving a relaxation of the verification problem . This relaxation is usually derived by overapproximating the adversarial polytope , or the space of outputs of a network for a region of possible inputs . These approaches rely on training models in a specific manner that makes certification of those models easier . As a result , they can often obtain provable adversarial accuracy results faster . However , certification is fundamentally different from verification in two primary ways . First , it solves a relaxation of the original verification problem . As a result , certification methods can fail to certify many inputs that are actually robust to perturbations – only exact verifiers , given enough time , can give conclusive answers on robustness for every single input . Second , certification approaches fall under the paradigm of co-training , where a certification method only works well on models specifically trained for that certification step . When used as a black box on arbitrary models , the certification step can yield a high rate of false negatives . For example , Raghunathan et al . ( 2018 ) found that their certification step was significantly less effective when used on a model trained using Wong and Kolter ( 2018 ) ’ s training method , and vice versa . In contrast , we design our methods to be universal . They can be combined with any standard training procedure for networks and will improve exact verification speed for any LP/MILP-based exact verifier . Our methods can also decrease the amount of overapproximation incurred by certification methods like Wong and Kolter ( 2018 ) ; Dvijotham et al . ( 2018 ) . Similar to most of the certification methods , our technique can be made to have very little training time overhead . Finally , subsequent work of Gowal et al . ( 2018 ) shows how applying interval bound propagation during training , combined with MILP-based exact verification , can lead to provably robust networks . 3 TRAINING VERIFIABLE NETWORK MODELS . We begin by discussing the task of verifying a network and identify two key properties of networks that lead to improved verification speed : weight sparsity and so-called ReLU stability . We then use natural regularization methods for inducing weight sparsity as well as a new regularization method for inducing ReLU stability . Finally , we demonstrate that these methods significantly speed up verification while maintaining state-of-the-art accuracy . 3.1 VERIFYING ADVERSARIAL ROBUSTNESS OF NETWORK MODELS . Deep neural network models . Our focus will be on one of the most common architectures for state-of-the-art models : k-layer fully-connected feed-forward DNN classifiers with ReLU nonlinearities4 . Such models can be viewed as a function f ( · , W , b ) , where W and b represent the weight matrices and biases of each layer . For an input x , the output f ( x , W , b ) of the DNN is 3These works use both “ verification ” and “ certification ” to describe their methods . For clarity , we use “ certification ” to describe their approaches , while we use “ verification ” to describe exact verification approaches . For a more detailed discussion of the differences , see Appendix F. 4Note that this encompasses common convolutional network architectures because every convolutional layer can be replaced by an equivalent fully-connected layer . defined as : z0 = x ( 1 ) ẑi = zi−1Wi + bi for i = 1 , 2 , . . . , k − 1 ( 2 ) zi = max ( ẑi , 0 ) for i = 1 , 2 , . . . , k − 2 ( 3 ) f ( x , W , b ) = ẑk−1 ( 4 ) Here , for each layer i , we let ẑij denote the jth ReLU pre-activation and let ẑij ( x ) denote the value of ẑij on an input x. ẑk−1 is the final , output layer with an output unit for each possible label ( the logits ) . The network will make predictions by selecting the label with the largest logit . Adversarial robustness . For a network to be reliable , it should make predictions that are robust – that is , it should predict the same output for inputs that are very similar . Specifically , we want the DNN classifier ’ s predictions to be robust to a set Adv ( x ) of possible adversarial perturbations of an input x . We focus on ` ∞ norm-bound adversarial perturbations , where Adv ( x ) = { x′ : ||x′ − x||∞ ≤ } for some constant , since it is the most common one considered in adversarial robustness and verification literature ( thus , it currently constitutes a canonical benchmark ) . Even so , our methods can be applied to other ` p norms and broader sets of perturbations . Verification of network models . For an input x with correct label y , a perturbed input x′ can cause a misclassification if it makes the logit of some incorrect label ŷ larger than the logit of y on x′ . We can thus express the task of finding an adversarial perturbation as the optimization problem : min x′ , ŷ f ( x′ , W ) y − f ( x′ , W ) ŷ subject to x′ ∈ Adv ( x ) An adversarial perturbation exists if and only if the objective of the optimization problem is negative . Adversarial accuracies . We define the true adversarial accuracy of a model to be the fraction of test set inputs for which the model is robust to all allowed perturbations . By definition , evaluations against specific adversarial attacks like PGD or FGSM provide an upper bound to this accuracy , while certification methods provide lower bounds . Given sufficient time for each input , an exact verifier can prove robustness to perturbations , or find a perturbation where the network makes a misclassification on the input , and thus exactly determine the true adversarial accuracy . This is in contrast to certification methods , which solve a relaxation of the verification problem and can not exactly determine the true adversarial accuracy no matter how much time they have . However , such exact verification may take impractically long for certain inputs , so we instead compute the provable adversarial accuracy , which we define as the fraction of test set inputs for which the verifier can prove robustness to perturbations within an allocated time budget ( timeout ) . Similarly to certifiable accuracy , this accuracy constitutes a lower bound on the true adversarial accuracy . A model can thus , e.g. , have high true adversarial accuracy and low provable adversarial accuracy if verification of the model is too slow and often fails to complete before timeout . Also , in our evaluations , we chose to use the MILP exact verifier of Tjeng et al . ( 2017 ) when performing experiments , as it is both open source and the fastest verifier we are aware of .

Summary: This paper proposes methods to train robust neural networks that can also be verified faster. Specifically, it uses pruning methods to encourage weight sparsity and uses regularization to encourage ReLU stability. Both weight sparsity and ReLU stability reduces time needed for verification. The verified robust accuracy reported in this paper is close to previous SOTA certified robust accuracy, although not beating SOTA.