Degradation Attacks on Certifiably Robust Neural Networks

1 INTRODUCTION . An adversarial example for a neural classifier is the result of applying small modifications to a correctly classified valid input such that the modified input is classified incorrectly . For neural classifiers trained in a standard manner , it has been shown that adversarial examples are rampant ( Szegedy et al. , 2014 ; Carlini & Wagner , 2017 ) . While training models in an adversarially-aware manner helps mitigate the issue to an extent , it does not solve the problem , as evidenced by the large gaps between the clean accuracy and verified robust accuracy1 of models . Certifiably robust classifiers ( Weng et al. , 2018 ; Wong et al. , 2018 ; Cohen et al. , 2019 ; Leino et al. , 2021 ) offer the most rigorous solution to the problem and provably protect models against adversarial attacks . These classifiers are constructed by composing a standard classifier with a certified run-time defense . The defenses aim to detect adversarial examples during model evaluation , by checking if the model is -locally robust at the evaluated input . If the check fails , the input is flagged as ( potentially ) adversarial and the model rejects the input . However , a rejection is not free of cost . Every time the model rejects an input , a user of the model has to resort to some other process other than their presumed first choice ( the defended model ) to make a prediction , reducing model utility . We show that existing certified run-time defenses are overly cautious and susceptible to erroneously flagging non-adversarial inputs as adversarial . This over-cautiousness is inherent in the design of such defenses , and it is manifested even when the -local robustness check is exact ( i.e. , the defense is complete ) . Not only does this lead to a degradation in model utility because of unnecessary rejections , but it also exposes models to a new line of attack that we refer to as degradation attacks . We develop new attacks that are aimed at causing certifiably robust classifiers to reject inputs frequently , and we show that this is a significant problem in practice . For state-of-the-art certifiably robust models , such as GloRo Nets ( Leino et al. , 2021 ) and randomized smoothed models ( Cohen et al. , 2019 ) , our attacks succeed on as many as 56 % of the inputs where the model is already known to be robust . This is particularly distressing given the already considerable computational costs of training and defending certifiably robust models . As a concrete scenario , consider an autonomous driving system that uses a neural classifier for labeling road signs . It is unsafe for the classifier to misclassify adversarially perturbed road signs , and we want to prevent this at all costs . One simple strategy is for the classifier to always reject inputs and hand over decision-making to the human driver . This is perfectly safe behavior , but this model has zero utility . Ideally , we want the model to hand over control ( i.e. , the run-time defense should raise a flag ) only when the perturbed example is actually going to cause misclassification . Our result implies that an adversary can cause the model to hand over control to the human driver even when a perturbed input would not have 1 % of test inputs where the model is accurate and also locally robust been misclassified . Although this does not happen for all the inputs ( as in our simple example ) , it happens often enough that the adversary can cause a drastic and unnecessary reduction in model utility . Degradation attacks are successful because certified run-time defenses do not account for the data manifold . An adversarial example is obtained by applying modifications to valid inputs . We formalize valid inputs by means of a set M which is the support set of the underlying input distribution that characterizes the learning problem . Although the exact description ofM is unknown , we know that both the training and test data used are inM . In order for a model to be free of adversarial examples , it only needs to be locally robust at every point in M . Certified defenses , however , try to enforce local robustness at all evaluated inputs , irrespective of whether the input belongs to M . An incorrectly flagged non-adversarial input is then an input at which the local robustness check fails but if the model were not to reject the input , it would be classified consistently with all valid inputs within an distance . While a norm-bounded adversary can not change the classification of an input in the presence of a certified run-time defense , it can apply norm-bounded modifications to existing valid inputs ( i.e. , inputs in M ) and force the model to reject otherwise correctly classifiable inputs , thereby degrading the model ’ s utility . As we discuss in Section 4 , defenders against degradation attacks have two options : they can use existing methods for constructing certifiably robust models but trained and validated with double the radius that the adversary is allowed for perturbations , or they can develop new defense techniques that account for the fact that models only need to be locally robust at points in M . We evaluate the ramifications of the former option and leave the latter for future work . To summarize , the main contributions of our work are as follows : ( 1 ) we describe new attacks , referred to as degradation attacks , that can force certifiably robust models ( with either deterministic or stochastic defense mechanisms ) to frequently and unnecessarily reject inputs ; ( 2 ) we empirically demonstrate the severity of the problem for models using state-of-the-art certified defense mechanisms like GloRo Nets and Randomized Smoothing ; ( 3 ) we make explicit the set , M , from which valid inputs are drawn , and this helps us explain that certified run-time defenses , based on checking -local robustness , are susceptible to degradation attacks , as they enforce local robustness at all inputs and not just inputs inM ; and ( 4 ) we discuss two possible defenses against degradation attacks , and evaluate the one based on doubling the radius enforced by certifiably robust models . The rest of this paper is organized as follows . In Section 2 , we revisit definitions of adversarial robustness and sketch a general approach for certified defenses . In Section 3 , we demonstrate how a certified defense can unnecessarily reject inputs ; we present degradation attack algorithms to this end , and evaluate their efficacy . In Section 4 , we sketch two defenses against degradation attacks , and concretely evaluate one of these proposals . In Section 5 , we describe the related work . Finally , we conclude in Section 6 . 2 REVISITING DEFINITIONS OF ADVERSARIAL ROBUSTNESS . A neural classifier f∈Rd→L is a function from a d-dimensional real vector space , Rd , to a finite set of labels , L. The inputs to the classifier are drawn from some distribution with support setM⊆Rd . In other words , the training and test sets are comprised of elements fromM and their corresponding labels . Though the description of M is not available to us , it plays a key role in formalizing the notion of adversarial examples . An adversarial example for a neural classifier is the result of applying small modifications to a correctly classified valid input such that the modified input is classified incorrectly . Formally , an input is valid if it belongs toM . Definition 1 below , proposed by Szegedy et al . ( 2014 ) , formalizes the notion of a targeted adversarial example . Definition 1 ( Targeted Adversarial Example ) . Given a neural classifier f∈Rd→L and an input x∈Rd , an input x′ : =x+r is an adversarial example with respect to a target label l∈L , ` p metric , and a fixed constant ∈R if the solution r to the following optimization problem is such that ||r||p≤ : Minimize ||r||p subject to f ( x+r ) =l and x+r∈ [ 0,1 ] d Untargeted adversarial examples , or simply adversarial examples , are defined similarly except that the constraint f ( x+r ) =l is replaced with f ( x+r ) 6=f ( x ) . As stated informally in ( Carlini & Wagner , 2017 ; Szegedy et al. , 2014 ) x needs to be a valid input , i.e. , x∈M , for x′ to be an adversarial example . Making this requirement explicit and formal ( via set M ) is a key step in the design of our degradation attacks . Algorithm 2.1 : Prediction with a certified run-time defense Inputs : A model f∈Rd→L , an input x′∈Rd , an attack bound ∈R , and a distance metric ` p Output : A prediction l∈L ∪ { ⊥ } 1 PredictWithDefense ( f , x′ , , ` p ) : 2 if f is ( , ` p ) −locally robust at x′ then 3 return f ( x′ ) 4 else 5 return⊥ A classifier is protected from adversarial examples with respect to a valid input x if it is locally robust at x . As stated in Definition 2 , a classifier is locally robust at x , if its prediction does not change in an -ball centered at x . Definition 2 ( Local Robustness ) . A neural classifier f∈Rd→L is ( , ` p ) -locally robust at x∈Rd if , ∀x′∈Rd . ||x−x′||p≤ =⇒ f ( x ) =f ( x′ ) The formal notions of validity and local robustness enable us to make the following key observation . Observation 3 . A classifier is protected from all adversarial examples if it is locally robust at all points inM . Certified run-time defenses . Classifiers , when composed with certified run-time defenses , are guaranteed protection from adversarial examples . These defenses , briefly surveyed in Section 5 , detect adversarial examples during model evaluation ( or run-time ) by checking if the model is locally robust at the input being evaluated . Algorithm 2.1 describes the manner in which certified run-time defenses are deployed for protection . The common assumption is that an adversary has white-box access to f as well as to valid inputs ( denoted x ) , and that the adversary may or may not succeed in building adversarial examples ( denoted x′ ) that are -close to x under the ` p metric . Given input x′ , a certified run-time defense helps decide whether x′ is an adversarial example or not . As described by PredictWithDefense ( lines 1-5 ) , the defense mechanism is first used to check if the model f is locally robust at x′ ( line 2 ) . If the check passes , then x′ is guaranteed to not be an adversarial example and the model prediction at x′ is returned as the label ( line 3 ) . However , if the check fails , then x′ maybe an adversarial example , though it not guaranteed to be one . The model conservatively rejects the input , indicated here as returning the⊥ label ( line 5 ) . 3 DEGRADATION ATTACKS . In this section , we describe our degradation attack algorithms and demonstrate their efficacy empirically . First , in Section 3.1 , by means of an example , we demonstrate how certified defenses can lead to unnecessary rejections . Next , in Section 3.2 , we describe our degradation attack algorithms against deterministic and stochastic certified defenses . Finally , in Section 3.3 , we evaluate our attacks empirically .

Summary: This paper proposes a degradation attack on certifiable defenses GloRo and randomized smoothing aiming to find correct inputs within robustness region but hard to certify, causing rejections. The proposed attack is simple which is just a smoothed variance of PGD attack. Experimental results show that the two evaluated certification methods are susceptible to this attack.