Relaxing Local Robustness

1 Introduction . The discovery of adversarial examples [ 9 , 24 , 28 ] has led to security concerns in deep learning . A growing body of work has sought to address this problem by providing provable guarantees that a model ’ s predictions are robust to small-norm perturbations [ 3 , 5 , 8 , 13 , 16 , 21 , 22 , 25 , 29 , 31 , 32 , 33 ] . This objective is typically captured by ensuring that a model satisfies point-wise local robustness ; i.e. , given a point , x , the model ’ s predictions must remain invariant over the -ball centered at x . Local robustness is accompanied by a metric , verified-robust accuracy ( VRA ) , which corresponds to the fraction of points that are both correctly classified and locally robust . In some contexts , however , it is not always clear that VRA is the most desirable objective or the most natural metric for measuring a model ’ s success against adversaries . For example , in some contexts , not all adversarial examples are equally bad—this may reflect simply that a mistake is understandable , even if it was caused by an adversary ( e.g. , if a model mistakenly predicts an image of a leopard to be a jaguar ) ; or that the correct label may be arbitrary in certain cases ( e.g. , if an image contains two classes of subjects ) . For similar sorts of reasons , some computer vision tasks often use top-k accuracy as a benchmark metric that relaxes standard accuracy , allowing a prediction to be considered correct so long as the correct label appears among the model ’ s k highest logit outputs . In many applications , top-k accuracy is considered a more suitable metric/objective than standard top-1 accuracy [ 1 ] ; meanwhile , studied robustness properties are typically only defined with respect to a single predicted class . Thus , as part of this work , we introduce an analogous relaxation of local robustness to top-k accuracy , which we call relaxed top-K ( RTK ) robustness ( Section 3 , Definition 3 ) . Moreover , we demonstrate how 1our code is publicly available at https : //github.com/klasleino/gloro 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . a neural network can be instrumented in order to naturally incorporate certifiable RTK robustness into its learning objective , making runtime certification of this property essentially free ( Section 5 ) . In addition to applying to only top-1 predictions , standard robustness also focuses specifically on the problem of undirected adversarial examples . More concretely , adversarial examples are obtained broadly via two categories of attacks : ( 1 ) evasion ( undirected ) attacks , and ( 2 ) targeted ( directed ) attacks . Local robustness provides guarantees against the former , while the latter , which requires only a weaker guarantee , has remained largely unexplored by work on certifiable robustness . In this work we introduce affinity robustness ( Section 4 , Definition 4 ) , which captures resistance to specified sets of directed adversarial attacks . We show that certifiable affinity robustness can also be achieved in a manner similar to RTK robustness ( Section 5 ) . In recent work , Leino et al . [ 22 ] introduced a concept of -global-robustness , which can be thought of as requiring regions of the input space that are labeled as different classes to be separated by a margin of , with any interstitial space labeled as⊥ , signifying rejection . By relaxing the notion of robustness , we essentially allow certain classes to be grouped together without any intervening rejected space . This gives rise to a spatially-arranged hierarchy of classes that is conveyed through the robustness guarantee ( see Figure 2 in Section 3 for an example ) . Interestingly , we find that models trained with the objective of RTK robustness often form a logical hierarchy even without supervision . Additionally , affinity robustness provides a way to supervise the hierarchy that forms , giving finer control over the spatial relationships between the various labels in the model ’ s decision surface . In summary , the primary contributions in this work are as follows : • We introduce RTK robustness , a notion of model robustness compatible with top-k accuracy , and affinity robustness , a notion of model robustness that captures resistance to particular sets of directed adversarial attacks . • We show how to construct models that can be efficiently certified against these two properties . • We show that applying these relaxed robustness variants to suitable domains leads to certifiable models with lower rejection rates and higher certified accuracy . • We show that these relaxed robustness variants lead to interesting properties regarding how a network ’ s prediction space is laid out , and that this layout of classes can be supervised to impart a priori hierarchies . 2 Related Work . Robustness certification for deep networks has become a well-studied topic , and numerous certification approaches have been proposed [ 3 , 5 , 8 , 13 , 16 , 21 , 22 , 25 , 29 , 31 , 32 , 33 ] . However , virtually all of this work has focused specifically on certifying guarantees against ` p-norm-bounded , undirected attacks on a model ’ s top prediction . By contrast , we introduce two relaxed variants of this standard robustness property that have not formerly been studied , which we subsequently show how to certify . Although our proposed definitions are agnostic to both the chosen norm and the method used to certify them , we focus on deterministic robustness certification ( as opposed to stochastic certification , e.g. , Randomized Smoothing [ 3 , 20 ] ) for the ` 2 norm . We build our certification approach from a recently proposed method by Leino et al . [ 22 ] , which has been demonstrated to be among the most scalable state-of-the-art methods for deterministic ` 2 certification . Much of this work focuses on generalizing the most commonly-used threat model characterizing what it means to be “ robust ” , namely , ` p local robustness , which stipulates prediction invariance over perturbations within a small ` p ball . Other work has also considered generalizations of this robustness definition , though not typically in the context of certification . E.g. , previous literature has proposed invariance under unions of multiple ` p balls [ 4 , 30 ] , invariance to rotations and translations [ 6 ] , and invariance to inconspicuous , physically-realizable perturbations [ 2 , 7 , 18 , 27 ] . The first of the two robustness definitions we propose , RTK robustness , is closely related to top-k accuracy . Recently , Jia et al . [ 12 ] also proposed a definition that aims to capture certified robustness for top-k predictions . However , their work differs from ours in two key ways . First , their certification method is derived from Randomized Smoothing , which gives a stochastic guarantee and relies on hundreds of thousands of samples for conclusive certification . More importantly , Jia et al . evaluate their property with respect to the ground truth class of the point in question , making it unclear how one might certify this property in such a manner on unlabeled points , e.g. , those seen by the model in deployment . We provide more discussion on this problem and how our work addresses it in Appendix A in the supplementary material . As we will see , devising a robustness analogue to top-k accuracy in a manner that relaxes standard robustness and does not depend on the ground truth is a subtle task—we address this issue carefully in Section 3 . Our second proposed robustness definition , affinity robustness , is related to targeted adversarial attacks [ 28 ] , which have only rarely been studied in the context of certifiable defenses [ 10 ] . 3 Relaxed Top-K Robustness . In this work , we consider relaxations of the standard notion of local robustness ( Definition 1 ) that may serve as a better learning objective and evaluation metric in some contexts , e.g. , those where not all adversarial examples are equally bad . Definition 1 ( Local Robustness ) A model , F , is -locally-robust at point , x , w.r.t . norm , ||·|| , if ∀x′ . ||x−x′||≤ =⇒ F ( x ) =F ( x′ ) We begin in this section by introducing a notion of robustness that is inspired by the relaxed accuracy metric , top-k accuracy . Top-k accuracy is a common benchmark metric in vision applications such as Imagenet , where the class labels are particularly fine-grained , and may even be arbitrary on some instances . Furthermore , top-k accuracy has been studied as a learning objective in its own right [ 1 ] , and has been identified as desirable in the context of robustness certification [ 12 ] . In order to create a notion of relaxed robustness that is analogous to top-k accuracy , we will consider the network to output a set of classes rather than a single class . Let us define the following notation representing the set of the top k outputs of a model : let f be the function computing the logit values of a neural network , and let fk ( x ) be the kth-highest logit output of f on x . We then define F k ( x ) = { j : fj ( x ) ≥ fk ( x ) } , that is , F k is the set of classes corresponding to the top k outputs of f . Using this notation , we define top-k robustness ( Definition 2 ) , which requires that the set of classes with the k highest logits remains invariant over small-norm perturbations . We note that if we had a single class of interest , c , e.g. , the ground truth , we could simply require that c remain inF k under small perturbations [ 12 ] ; however , top-k accuracy is useful precisely because any of the classes in F k could be correct , meaning that all classes in F kshould be treated equally and guarded against perturbations . Definition 2 ( Top-k Robustness ) A model , F , is top-k -locally-robust at point , x , w.r.t . norm , ||·|| , if ∀x′ . ||x−x′||≤ =⇒ F k ( x ) =F k ( x′ ) While top-k robustness may appear to capture the idea of top-k accuracy , we observe that the analogy fails , as top-k robustness is not a relaxation of standard local robustness . For example , if a model is top-1 accurate , it is also top-2 accurate ; however , if a model is top-1 robust , it may not be top-2 robust . Figure 1a provides an example illustrating this point . We thus turn our attention to a modification of Definition 2 that does relax standard local robustness . Definition 3 provides a notion of what we call relaxed top-K robustness , or RTK robustness , which is properly analogous to the concept of top-k accuracy . Essentially , a point is considered RTK robust if it is top-k robust for some k in { 1 , ... , K } . Definition 3 ( Relaxed Top-K Robustness ) A model , F , is relaxed-top-K ( RTK ) -locally-robust at point , x , w.r.t . norm , ||·|| , if ∀x′ . ||x−x′||≤ =⇒ ∃ k≤K : F k ( x ) =F k ( x′ ) From the definition , it is clear that RTK robustness is a relaxation of standard local robustness : first , RT1 robustness is equivalent to top-1 robustness , which is equivalent to local robustness ; second , RT1 robustness implies RTK robustness forK > 1 . We can think of RTK robustness as allowing the model to output a set of labels ( with size at mostK ) such that the output set remains invariant under bounded-norm perturbations . In some cases there may be multiple such sets , e.g. , if the model is both top-1 robust and top-2 robust on the point ( see Figure 1b for an example ) . These can be thought of as the sets of classes that are “ safe ” to predict on a given point ; if the only such set is empty then no classes are safe , and the model abstains from predicting . This gives rise to a spatially-arranged hierarchy of classes that is conveyed through the robustness guarantee : each region on the decision surface corresponds to some set ( or sets ) of classes that can be robustly predicted . Figure 2c gives an example of how such a decision surface might look on a synthetic 2D dataset , via a coloring that indicates the smallest safe set that can be predicted .

Summary: This paper describes a relaxation of local robustness to top-k robustness and affinity robustness. In addition, the authors connect the relaxation properties with the globally robust neural networks for certification. In the experimental results, the authors found lower rejection rates and higher certified accuracy in a few domains.