Evaluating Robustness of Neural Networks with Mixed Integer Programming

1 INTRODUCTION . Neural networks trained only to optimize for training accuracy have been shown to be vulnerable to adversarial examples : perturbed inputs that are very similar to some regular input but for which the output is radically different ( Szegedy et al. , 2014 ) . There is now a large body of work proposing defense methods to produce classifiers that are more robust to adversarial examples . However , as long as a defense is evaluated only via heuristic attacks ( such as the Fast Gradient Sign Method ( FGSM ) ( Goodfellow et al. , 2015 ) or Carlini & Wagner ( 2017b ) ’ s attack ( CW ) ) , we have no guarantee that the defense actually increases the robustness of the classifier produced . Defense methods thought to be successful when published have often later been found to be vulnerable to a new class of attacks . For instance , multiple defense methods are defeated in Carlini & Wagner ( 2017a ) by constructing defense-specific loss functions and in Athalye et al . ( 2018 ) by overcoming obfuscated gradients . Fortunately , we can evaluate robustness to adversarial examples in a principled fashion . One option is to determine ( for each test input ) the minimum distance to the closest adversarial example , which we call the minimum adversarial distortion ( Carlini et al. , 2017 ) . Alternatively , we can determine the adversarial test accuracy ( Bastani et al. , 2016 ) , which is the proportion of the test set for which no perturbation in some bounded class causes a misclassification . An increase in the mean minimum adversarial distortion or in the adversarial test accuracy indicates an improvement in robustness.1 We present an efficient implementation of a mixed-integer linear programming ( MILP ) verifier for properties of piecewise-linear feed-forward neural networks . Our tight formulation for nonlinearities and our novel presolve algorithm combine to minimize the number of binary variables in the MILP problem and dramatically improve its numerical conditioning . Optimizations in our MILP 1The two measures are related : a solver that can find certificates for bounded perturbations can be used iteratively ( in a binary search process ) to find minimum distortions . implementation improve performance by several orders of magnitude when compared to a naı̈ve MILP implementation , and we are two to three orders of magnitude faster than the state-of-the-art Satisfiability Modulo Theories ( SMT ) based verifier , Reluplex ( Katz et al. , 2017 ) We make the following key contributions : • We demonstrate that , despite considering the full combinatorial nature of the network , our verifier can succeed at evaluating the robustness of larger neural networks , including those with convolutional and residual layers . • We identify why we can succeed on larger neural networks with hundreds of thousands of units . First , a large fraction of the ReLUs can be shown to be either always active or always inactive over the bounded input domain . Second , since the predicted label is determined by the unit in the final layer with the maximum activation , proving that a unit never has the maximum activation over all bounded perturbations eliminates it from consideration . We exploit both phenomena , reducing the overall number of non-linearities considered . • We determine for the first time the exact adversarial accuracy for MNIST classifiers to perturbations with bounded l∞ norm . We are also able to certify more samples than the state-of-the-art and find more adversarial examples across MNIST and CIFAR-10 classifiers with different architectures trained with a variety of robust training procedures . Our code is available at https : //github.com/vtjeng/MIPVerify.jl . 2 RELATED WORK . Our work relates most closely to other work on verification of piecewise-linear neural networks ; Bunel et al . ( 2018 ) provides a good overview of the field . We categorize verification procedures as complete or incomplete . To understand the difference between these two types of procedures , we consider the example of evaluating adversarial accuracy . As in Kolter & Wong ( 2017 ) , we call the exact set of all final-layer activations that can be achieved by applying a bounded perturbation to the input the adversarial polytope . Incomplete verifiers reason over an outer approximation of the adversarial polytope . As a result , when using incomplete verifiers , the answer to some queries about the adversarial polytope may not be decidable . In particular , incomplete verifiers can only certify robustness for a fraction of robust input ; the status for the remaining input is undetermined . In contrast , complete verifiers reason over the exact adversarial polytope . Given sufficient time , a complete verifier can provide a definite answer to any query about the adversarial polytope . In the context of adversarial accuracy , complete verifiers will obtain a valid adversarial example or a certificate of robustness for every input . When a time limit is set , complete verifiers behave like incomplete verifiers , and resolve only a fraction of queries . However , complete verifiers do allow users to answer a larger fraction of queries by extending the set time limit . Incomplete verifiers for evaluating network robustness employ a range of techniques , including duality ( Dvijotham et al. , 2018 ; Kolter & Wong , 2017 ; Raghunathan et al. , 2018 ) , layer-by-layer approximations of the adversarial polytope ( Xiang et al. , 2018 ) , discretizing the search space ( Huang et al. , 2017 ) , abstract interpretation ( Gehr et al. , 2018 ) , bounding the local Lipschitz constant ( Weng et al. , 2018 ) , or bounding the activation of the ReLU with linear functions ( Weng et al. , 2018 ) . Complete verifiers typically employ either MILP solvers as we do ( Cheng et al. , 2017 ; Dutta et al. , 2018 ; Fischetti & Jo , 2018 ; Lomuscio & Maganti , 2017 ) or SMT solvers ( Carlini et al. , 2017 ; Ehlers , 2017 ; Katz et al. , 2017 ; Scheibler et al. , 2015 ) . Our approach improves upon existing MILP-based approaches with a tighter formulation for non-linearities and a novel presolve algorithm that makes full use of all information available , leading to solve times several orders of magnitude faster than a naı̈vely implemented MILP-based approach . When comparing our approach to the state-of-the-art SMT-based approach ( Reluplex ) on the task of finding minimum adversarial distortions , we find that our verifier is two to three orders of magnitude faster . Crucially , these improvements in performance allow our verifier to verify a network with over 100,000 units — several orders of magnitude larger than the largest MNIST classifier previously verified with a complete verifier . A complementary line of research to verification is in robust training procedures that train networks designed to be robust to bounded perturbations . Robust training aims to minimize the “ worst-case loss ” for each example — that is , the maximum loss over all bounded perturbations of that example ( Kolter & Wong , 2017 ) . Since calculating the exact worst-case loss can be computationally costly , robust training procedures typically minimize an estimate of the worst-case loss : either a lower bound as is the case for adversarial training ( Goodfellow et al. , 2015 ) , or an upper bound as is the case for certified training approaches ( Hein & Andriushchenko , 2017 ; Kolter & Wong , 2017 ; Raghunathan et al. , 2018 ) . Complete verifiers such as ours can augment robust training procedures by resolving the status of input for which heuristic attacks can not find an adversarial example and incomplete verifiers can not guarantee robustness , enabling more accurate comparisons between different training procedures . 3 BACKGROUND AND NOTATION . We denote a neural network by a function f ( · ; θ ) : Rm → Rn parameterized by a ( fixed ) vector of weights θ . For a classifier , the output layer has a neuron for each target class . Verification as solving an MILP . The general problem of verification is to determine whether some property P on the output of a neural network holds for all input in a bounded input domain C ⊆ Rm . For the verification problem to be expressible as solving an MILP , P must be expressible as the conjunction or disjunction of linear properties Pi , j over some set of polyhedra Ci , where C = ∪Ci . In addition , f ( · ) must be composed of piecewise-linear layers . This is not a particularly restrictive requirement : piecewise-linear layers include linear transformations ( such as fully-connected , convolution , and average-pooling layers ) and layers that use piecewise-linear functions ( such as ReLU or maximum-pooling layers ) . We provide details on how to express these piecewise-linear functions in the MILP framework in Section 4.1 . The “ shortcut connections ” used in architectures such as ResNet ( He et al. , 2016 ) are also linear , and batch normalization ( Ioffe & Szegedy , 2015 ) or dropout ( Srivastava et al. , 2014 ) are linear transformations at evaluation time ( Bunel et al. , 2018 ) . 4 FORMULATING ROBUSTNESS EVALUATION OF CLASSIFIERS AS AN MILP . Evaluating Adversarial Accuracy . Let G ( x ) denote the region in the input domain corresponding to all allowable perturbations of a particular input x . In general , perturbed inputs must also remain in the domain of valid inputs Xvalid . For example , for normalized images with pixel values ranging from 0 to 1 , Xvalid = [ 0 , 1 ] m. As in Madry et al . ( 2018 ) , we say that a neural network is robust to perturbations on x if the predicted probability of the true label λ ( x ) exceeds that of every other label for all perturbations : ∀x′ ∈ ( G ( x ) ∩ Xvalid ) : argmaxi ( fi ( x′ ) ) = λ ( x ) ( 1 ) Equivalently , the network is robust to perturbations on x if and only if Equation 2 is infeasible for x′ . ( x′ ∈ ( G ( x ) ∩ Xvalid ) ) ∧ ( fλ ( x ) ( x ′ ) < max µ∈ [ 1 , n ] \ { λ ( x ) } fµ ( x ′ ) ) ( 2 ) where fi ( · ) is the ith output of the network . For conciseness , we call x robust with respect to the network if f ( · ) is robust to perturbations on x . If x is not robust , we call any x′ satisfying the constraints a valid adversarial example to x . The adversarial accuracy of a network is the fraction of the test set that is robust ; the adversarial error is the complement of the adversarial accuracy . As long as G ( x ) ∩ Xvalid can be expressed as the union of a set of polyhedra , the feasibility problem can be expressed as an MILP . The four robust training procedures we consider ( Kolter & Wong , 2017 ; Wong et al. , 2018 ; Madry et al. , 2018 ; Raghunathan et al. , 2018 ) are designed to be robust to perturbations with bounded l∞ norm , and the l∞-ball of radius around each input x can be succinctly represented by the set of linear constraints G ( x ) = { x′ | ∀i : − ≤ ( x− x′ ) i ≤ } . Evaluating Mean Minimum Adversarial Distortion . Let d ( · , · ) denote a distance metric that measures the perceptual similarity between two input images . The minimum adversarial distortion under d for input x with true label λ ( x ) corresponds to the solution to the optimization : minx′ d ( x ′ , x ) ( 3 ) subject to argmaxi ( fi ( x ′ ) ) 6= λ ( x ) ( 4 ) x′ ∈ Xvalid ( 5 ) We can target the attack to generate an adversarial example that is classified in one of a set of target labels T by replacing Equation 4 with argmaxi ( fi ( x ′ ) ) ∈ T . The most prevalent distance metrics in the literature for generating adversarial examples are the l1 ( Carlini & Wagner , 2017b ; Chen et al. , 2018 ) , l2 ( Szegedy et al. , 2014 ) , and l∞ ( Goodfellow et al. , 2015 ; Papernot et al. , 2016 ) norms . All three can be expressed in the objective without adding any additional integer variables to the model ( Boyd & Vandenberghe , 2004 ) ; details are in Appendix A.3 .

Summary: The authors perform a careful study of mixed integer linear programming approaches for verifying robustness of neural networks to adversarial perturbations. They propose three enhancements to MILP formulations of neural network verification: Asymmetric bounds, restricted domain and progressive bound tightening, which lead to significantly more scalable verification algorithms vis-a-vis prior work. They study the effectiveness of MILP solvers both in terms of verifying robustness (compared to other complete/incomplete verifiers) and generating adversarial attacks (compared to PGD attacks) and show that their approach compares favorable across a number of architectures on MNIST and CIFAR-10. They perform careful ablation studies to validate the importance of the