Provably robust classification of adversarial examples with detection

1 INTRODUCTION . Despite popularity and success of deep neural networks in many applications , their performance declines sharply in adversarial settings . Small adversarial perturbations are shown to greatly deteriorate the performance of neural network classifiers , which creates a growing concern for utilizing them in safety critical application where robust performance is key . In adversarial training , different methods with varying levels of computational complexity aim at robustifying the network by finding such adversarial examples at each training steps and adding them to the training dataset . While such methods exhibit empirical robustness , they lack verifiable guarantees as it is not provable that a more rigorous adversary , e.g. , one that does brute-force enumeration to compute adversarial perturbations , will not be able to cause the classifier to misclassify . It is thus desirable to provably verify the performance of robust classifiers without restricting the adversarial perturbations by inexact solvers , while restraining perturbations to a class of admissible set , e.g. , within an ` ∞ norm-bounded ball . Progress has been made by ‘ complete methods ’ that use Satisfiability Modulo Theory ( SMT ) or Mixed-Integer Programming ( MIP ) to provide exact robustness bounds , however , such approaches are expensive , and difficult to scale to large networks as exhaustive enumeration in the worst case is required ( Tjeng et al. , 2017 ; Ehlers , 2017 ; Xiao et al. , 2018 ) . ∗Work was done when the author was an intern at Bosch Center for Artificial Intelligence , Pittsburgh , PA. ‘ Incomplete methods ’ on the other hand , proceed by computing a differential upper bound on the worst-case adversarial loss , and similarly for the verification violations , with lower computational complexity and improved scalability . Such upper bounds , if easy to compute , can be utilized during the training , and yield provably robust networks with tight bounds . In particular , bound propagation via various methods such as differentiable geometric abstractions ( Mirman et al. , 2018 ) , convex polytope relaxation ( Wong & Kolter , 2018 ) , and more recently in ( Salman et al. , 2019 ; Balunovic & Vechev , 2020 ; Gowal et al. , 2018 ; Zhang et al. , 2020 ) , together with other techniques such as semidefinite relaxation , ( Fazlyab et al. , 2019 ; Raghunathan et al. , 2018 ) , and dual solutions via additional verifier networks ( Dvijotham et al. , 2018 ) fall within this category . In particular , recent successful use of Interval Bound Propagation ( IBP ) as a simple layer-by-layer bound propagation mechanism was shown to be very effective in Gowal et al . ( 2018 ) , which despite its light computational complexity exhibits SOTA robustness verification . Additionally , combining IBP in a forward bounding pass with linear relaxation based backward bounding pass ( CROWN ) Zhang et al . ( 2020 ) leads to improved robustness , although it can be up to 3-10 times slower . Alternative to robust classification , detection of adversarial examples can also provide robustness against adversarial attacks , where suspicious inputs will be flagged and the classifier “ rejects/abstains ” from assigning a label . There has been some work on detection of out-of-distribution examples Bitterwolf et al . ( 2020 ) , however the situation in the literature on the detection of adversarial examples is quite different from above . Most techniques that attempt to detect adversarial examples , either by training explicit classifiers to do so or by simply formulating “ hand-tuned ” detectors , still largely look to identify and exploit statistical properties of adversarial examples that appear in practice Smith & Gal ( 2018 ) ; Roth et al . ( 2019 ) . However , to provide a fair evaluation , a defense must be evaluated under attackers that attempt to fool both the classifier and the detector , while addressing particular characteristics of a given defense , e.g. , gradient obfuscation , non-differentability , randomization , and simplifying the attacker ’ s objective for increased efficiency . A non-exhaustive list of recent detection methods entails randomization and sparsity-based defenses ( Xiao et al. , 2019 ; Roth et al. , 2019 ; Pang et al. , 2019b ) , confidence and uncertainty-based detection ( Smith & Gal , 2018 ; Stutz et al. , 2020 ; Sheikholeslami et al. , 2020 ) , transformation-based defenses ( Bafna et al. , 2018 ; Yang et al. , 2019 ) , ensemble methods ( Verma & Swami , 2019 ; Pang et al. , 2019a ) , generative adversarial training Yin et al . ( 2020 ) , and many more . Unfortunately , existing defenses have largely proven to have poor performance against adaptive attacks ( Athalye et al. , 2018 ; Tramer et al. , 2020 ) , necessitating provable guarantees on detectors as well . Recently Laidlaw & Feizi ( 2019 ) have proposed joint training of classifier and detector , however it also does not provided any provable guarantees . Our contribution . In this work , we propose a new method for jointly training a provably robust classifier and detector . Specifically , by introducing an additional “ abstain/detection ” into a classifier , we show that the existing certified defense mechanisms can be modified , and by building on the detection capability of the network , classifier can effectively choose to either robustly classify or detect adversarial attacks . We extend the light-weight Interval Bound Propagation ( IBP ) method to account for our new robust objective , enabling verification of the network for provable performance guarantees . Our proposed robust training objective is also effectively upper bounded , enabling its incorporation into the training procedure leading to tight provably robust performance . While tightening of the bound propagation may be additionally possible for tighter verification , to the best of our knowledge , our approach is the first method to extend certification techniques by considering detection while providing provable verification . By stabilizing the training , as also used in similar IBP-based methods , experiments on MNIST and CIFAR-10 empirically show that the proposed method can successfully leverage its detection capability , and improves traditional IBP used in isolation , especially for large perturbation sizes . 2 BACKGROUND AND RELATED WORK . Let us consider an L-layer feed-forward neural network , trained for a K-class classification task . Given input x , it will pass through a sequential model , with hl denoting the mapping at layer l , recursively parameterized by zl = hl ( zl−1 ) = σl ( W > l zl−1 + bl ) , l = 1 , · · · , L Wl ∈ Rnl−1×nl , bl ∈ Rnl ( 1 ) where σl ( . ) is a monotonic activation function , z0 denotes the input , and zL ∈ RK is the preactivation unnormalized K-dimensional output vector ( nL = K and σL ( . ) as identity operator ) , referred to as the logits . Robust classifiers can be obtained by minimizing the worst-case ( adversarial ) classification loss , formally trained by the following min-max optimization Madry et al . ( 2017 ) minimize θ E ( x , y ) ∼D [ max δ∈∆ ` ( fθ ( x + δ ) , y ) ] . ( 2 ) where θ denotes network parameters , vector fθ ( x ) = zL is the logit output for input x , ` ( . ) is the misclassfication loss , e.g. , ` xent ( . ) defined as the cross-entropy loss , and ∆ denotes the set of permissible perturbations , e.g. , for ` ∞-norm ball of radius giving ∆ : = { δ | ‖δ‖∞ ≤ } . Although augmenting the training set with adversarial inputs , obtained by approximately solving the inner maximization in Eq . 2 , empirically leads to improved adversarial robustness Madry et al . ( 2017 ) ; Shafahi et al . ( 2019 ) ; Wong et al . ( 2019 ) ; Zhang et al . ( 2019 ) , inexact solution of the inner maximization prevents such methods from providing provable guarantees . In critical applications however , provable verification of classification accuracy against a given threat model is crucial . 2.1 PERFORMANCE VERIFICATION AND NETWORK RELAXATION . Given input ( x , y ) , a classification network is considered verifiably robust if all of its perturbed variations , that is x + δ for ∀δ ∈ ∆ , are correctly classified as class y . Such verification can be effectively obtained by p∗i = min zL∈ZL c > y , izL where , ZL : = { zL|zl = hl ( zl−1 ) , l = 1 , ... , L , z0 = x + δ , ∀δ ∈ ∆ } where cy , i = ey − ei for i = 1 , 2 , .. , K , i 6= y , and ei is the standard ith canonical basis vector . If p∗i > 0 ∀i 6= y , then the classifier is verifiably robust at point ( x , y ) as this guarantees that zi ≤ zy ∀i 6= y for all admissible perturbations δ ∈ ∆ . The feasible set ZL is generally nonconvex , rendering obtaining p∗i intractable . Any convex relaxation ofZL however , will provide a lower bound on p∗i , and can be alternatively used for verification . As outlined in Section 1 , various relaxation techniques have been proposed in the literature . Specifically , IBP in ( Mirman et al. , 2018 ; Gowal et al. , 2018 ) proceeds by bounding the activation zl of each layer by propagating an element-wise bounding box using interval arithmetic for networks with monotonic activation functions . Despite its simplicity and relatively small computational complexity ( computational requirements for bound propagation for a given input using IBP is equal to two forward passes of the input ) , it can provide tight bounds once the network is trained accordingly . Specifically , starting from the input layer z0 , it can be bounded for the perturbation class δ ∈ ∆ as z0 = x− 1 and z0 = x + 1 , and zl for the following layers can be bounded as zl = σl ( W > l zl−1 + zl−1 2 − |W > l | zl−1 − zl−1 2 ) , zl = σl ( W > l zl−1 + zl−1 2 + |W > l | zl−1 − zl−1 2 ) , ( 3 ) where | · | is the element-wise absolute-value operator . The verification problem over the relaxed feasible set ẐL : = { zL | zL , i ≤ zL , i ≤ z̄L , i } , where ZL ⊆ ẐL is then easily solved as p∗i = min zL∈ZL c > y , izL ≥ min zL∈ẐL c > y , izL = zL , y − z̄L , i . ( 4 ) 2.2 ROBUST TRAINING OF VERIFIABLE NETWORKS . It has been shown that convex relaxation ofZL can also provide a tractable upper bound on the inner maximization in Eq . 2 . While this holds for various relaxation techniques , focusing on the IBP let us define JIBP , θ ( x , y ) : = [ J IBP 1 , J IBP 2 , ... , J IBP K ] where J IBP i : = min zL∈ẐL c > y , izL ( 5 ) with ( θ , ) implicitly influencing ẐL ( dropped for brevity ) , and upperbound the inner-max in Eq . 2 max δ∈∆ ` xent ( fθ ( x + δ ) , y ) ≤ ` xent ( −JIBP , θ ( x , y ) , y ) , ` xent ( z , c ) : = − log ( exp ( zc ) ∑ i exp ( zi ) ) ( 6 ) By using this tractable upper bound of the robust optimization , network can now be trained by minimize θ ∑ ( x , y ) ∈D ( 1− κ ) ` xent ( −JIBP , θ ( x , y ) , y ) + κγ ` xent ( fθ ( x ) , y ) , ( 7 ) where γ trades natural versus robust accuracy , and κ is scheduled through a ramp-down process to stabilize the training and tightening of IBP Gowal et al . ( 2018 ) ( where γ = 1 is selected therein ) .

Summary: This paper deals with the problem of bounding the amount of errors that a model can make when attacked by an adversary limited to small perturbation of an input image. Similarly to previous paper, it proposes to extend the classifier with a detector to identify adversarial examples. As opposed to previous papers, the authors suggest to perform formal verification to prove that the samples are either robustly classified or abstained on (as opposed to testing the classifier + detector adversarially which would have left open the question of whether or not the attack against the pair was done in an adequate way). The method used for this is based on the simple IBP algorithm but the authors propose a proper encoding of the max(abstain_logit, gt_logit). They also follow the IBP strategy of using the bound resulting from verification as part of the training objective in order to encourage the network to behave properly.