Understanding Intrinsic Robustness Using Label Uncertainty

1 INTRODUCTION . Since the initial reports of adversarial examples against deep neural networks ( Szegedy et al. , 2014 ; Goodfellow et al. , 2015 ) , many defensive mechanisms have been proposed aiming to enhance the robustness of machine learning classifiers . Most have failed , however , against stronger adaptive attacks ( Athalye et al. , 2018 ; Tramer et al. , 2020 ) . PGD-based adversarial training ( Mądry et al. , 2018 ) and its variants ( Zhang et al. , 2019 ; Carmon et al. , 2019 ) are among the few heuristic defenses that have not been broken so far , but these methods still fail to produce satisfactorily robust classifiers , even for classification tasks on benchmark datasets like CIFAR-10 . Motivated by the empirical hardness of adversarially-robust learning , a line of theoretical works ( Gilmer et al. , 2018 ; Fawzi et al. , 2018 ; Mahloujifar et al. , 2019a ; Shafahi et al. , 2019 ) have argued that adversarial examples are unavoidable . In particular , these works proved that as long as the input distributions are concentrated with respect to the perturbation metric , adversarially robust classifiers do not exist . Recently , Mahloujifar et al . ( 2019b ) and Prescott et al . ( 2021 ) generalized these results by developing empirical methods for measuring the concentration of arbitrary input distributions to derive an intrinsic robustness limit . ( Appendix A provides a more thorough discussion of related work . ) We argue that the standard concentration of measure problem , which was studied in all of the aforementioned works , is not sufficient to capture a realistic intrinsic robustness limit for a classification problem . In particular , the standard concentration function is defined as an inherent property regarding the input metric probability space that does not take account of the underlying label information . We argue that such label information is essential for any supervised learning problem , including adversarially robust classification , so must be incorporated into intrinsic robustness limits . Contributions . We identify the insufficiency of the standard concentration of measure problem and demonstrate why it fails to capture a realistic intrinsic robustness limit ( Section 3 ) . Then , we introduce the notion of label uncertainty ( Definition 4.1 ) , which characterizes the average uncertainty of label assignments for an input region . We then incorporate label uncertainty in the standard concentration measure as an initial step towards a more realistic characterization of intrinsic robustness ( Section 4 ) . Experiments on the CIFAR-10 and CIFAR-10H ( Peterson et al. , 2019 ) datasets demonstrate that error regions induced by state-of-the-art classification models all have high label uncertainty ( Section 6.1 ) , which validates the proposed label uncertainty constrained concentration problem . By adapting the standard concentration estimation method in Mahloujifar et al . ( 2019b ) , we propose an empirical estimator for the label uncertainty constrained concentration function . We then theoretically study the asymptotic behavior of the proposed estimator and provide a corresponding heuristic algorithm for typical perturbation metrics ( Section 5 ) . We demonstrate that our method is able to produce a more accurate characterization of intrinsic robustness limit for benchmark datasets such as CIFAR-10 ( Section 6.2 ) than was possible using prior methods that do not consider labels . We also provide empirical evidence showing that both the clean and robust accuracies of stateof-the-art robust classification models are largely affected by the label uncertainty of the tested examples , suggesting that adding an abstain option based on label uncertainty is a promising avenue for improving adversarial robustness of deployed machine learning systems ( Section 6.3 ) . Notation . We use lowercase boldface letters to denote vectors and use [ k ] to denote { 1 , 2 , . . . , k } . We use In to denote the n × n identity matrix . For any set A , |A| denotes its cardinality , pow ( A ) is all its measurable subsets , and 1A ( · ) is the indicator function of A . Consider metric probability space ( X , µ , ∆ ) , where ∆ : X × X → R≥0 is a distance metric on X . Define the empirical measure of µ with respect to a data set S sampled from µ as µ̂S ( A ) = ∑ x∈S 1A ( x ) /|S| . Denote by B ( x , ∆ ) the ball around x with radius measured by ∆ . The -expansion of A is defined as A ( ∆ ) = { x ∈ X : ∃ x′ ∈ B ( x , ∆ ) ∩ A } . When ∆ is free of context , we simply write B ( x ) = B ( x , ∆ ) and A = A ( ∆ ) . 2 PRELIMINARIES . Adversarial Risk . Adversarial risk captures the vulnerability of a classifier against adversarial perturbations . In particular , we adopt the following adversarial risk definition , which has been studied in several previous works , such as Gilmer et al . ( 2018 ) ; Bubeck et al . ( 2019 ) ; Mahloujifar et al . ( 2019a ; b ) ; Zhang et al . ( 2020b ) ; Prescott et al . ( 2021 ) . Definition 2.1 ( Adversarial Risk ) . Let ( X , µ , ∆ ) be a metric probability space of instances and Y be the set of possible class labels . Assume c : X → Y is a concept function that gives each instance a label . For any classifier f : X → Y and ≥ 0 , the adversarial risk of f is defined as : AdvRisk ( f , c ) = Pr x∼µ [ ∃ x′ ∈ B ( x ) s.t . f ( x′ ) 6= c ( x′ ) ] . The adversarial robustness of f is defined as : AdvRob ( f , c ) = 1−AdvRisk ( f , c ) . When = 0 , adversarial risk equals to the standard risk . Namely , AdvRisk0 ( f , c ) = Risk ( f , c ) : = Prx∼µ [ f ( x ) 6= c ( x ) ] holds for any classifier f . Other definitions of adversarial risk have been proposed , such as the one used in Mądry et al . ( 2018 ) . These definitions are equivalent to the one we use , as long as small perturbations preserve the labels assigned by c ( · ) . Intrinsic Robustness . The definition of intrinsic robustness was first introduced by Mahloujifar et al . ( 2019b ) to capture the maximum adversarial robustness with respect to some set of classifiers : Definition 2.2 ( Intrinsic Robustness ) . Consider the input metric probability space ( X , µ , ∆ ) and the set of labels Y . Let c : X → Y be a concept function that gives a label to each input . For any set of classifiers F ⊆ { f : X → Y } and ≥ 0 , the intrinsic robustness with respect to F is defined as : AdvRob ( F , c ) = 1− inf f∈F { AdvRisk ( f , c ) } = sup f∈F { AdvRob ( f , c ) } . According to the definition of intrinsic robustness , there does not exist any classifier in F with adversarial robustness higher than AdvRob ( F , c ) for the considered task . Prior works , including Gilmer et al . ( 2018 ) ; Mahloujifar et al . ( 2019a ; b ) ; Zhang et al . ( 2020b ) , selected F in Definition 2.2 as the set of imperfect classifiers Fα = { f : Risk ( f , c ) ≥ α } , where α ∈ ( 0 , 1 ) is set as a small constant that reflects the best classification error rates achieved by state-of-the-art methods . Concentration of Measure . Concentration of measure captures a ‘ closeness ’ property for a metric probability space of instances . More formally , it is defined by the concentration function : Definition 2.3 ( Concentration Function ) . Let ( X , µ , ∆ ) be a metric probability space . For any α ∈ ( 0 , 1 ) and ≥ 0 , the concentration function of ( X , µ , ∆ ) is defined as : h ( µ , α , ) = inf E∈pow ( X ) { µ ( E ) : µ ( E ) ≥ α } . The standard notion of concentration function considers a special case of Definition 2.3 with α = 1/2 ( e.g. , Talagrand ( 1995 ) ) . For some special metric probability spaces , one can prove the closed-form solution of the concentration function . The Gaussian Isoperimetric Inequality ( Borell , 1975 ; Sudakov & Tsirelson , 1974 ) characterizes the concentration function for spherical Gaussian distribution and ` 2-norm distance metric , and was generalized by Prescott et al . ( 2021 ) to other ` p norms . 3 STANDARD CONCENTRATION IS INSUFFICIENT . We first explain a fundamental connection between the concentration of measure and the intrinsic robustness with respect to imperfect classifiers shown in previous work , and then argue that standard concentration fails to capture a realistic intrinsic robustness limit because it ignores data labels . Connecting Intrinsic Robustness with Concentration of Measure . Let ( X , µ , ∆ ) be the considered input metric probability space , Y be the set of possible labels , and c : X → Y be the concept function that gives each input a label . Given parameters 0 < α < 1 and ≥ 0 , the standard concentration problem can be cast into an optimization problem as follows : minimize E∈pow ( X ) µ ( E ) subject to µ ( E ) ≥ α . ( 3.1 ) For any classifier f , let Ef = { x ∈ X : f ( x ) 6= c ( x ) } be its induced error region with respect to c ( · ) . By connecting the risk of f with the measure of Ef and the adversarial risk of f with the measure of the -expansion of Ef , Mahloujifar et al . ( 2019a ) proved that the standard concentration problem ( 3.1 ) is equivalent to the following optimization problem regarding risk and adversarial risk : minimize f AdvRisk ( f , c ) subject to Risk ( f , c ) ≥ α . To be more specific , the following lemma characterizes the connection between the standard concentration function and the intrinsic robustness limit with respect to the set of imperfect classifiers : Lemma 3.1 ( Mahloujifar et al . ( 2019a ) ) . Let α ∈ ( 0 , 1 ) and Fα = { f : Risk ( f , c ) ≥ α } be the set of imperfect classifiers . For any ≥ 0 , it holds that AdvRob ( Fα , c ) = 1− h ( µ , α , ) . Lemma 3.1 suggests that the concentration function of the input metric probability space h ( µ , α , ) can be translated into an adversarial robustness upper bound that applies to any classifier with risk at least α . If this upper bound is shown to be small , then one can conclude that it is impossible to learn an adversarially robust classifier , as long as the learned classifier has risk at least α . Concentration without Labels Mischaracterizes Intrinsic Robustness . Despite the appealing relationship between concentration of measure and intrinsic robustness , we argue that solving the standard concentration problem is not enough to capture a meaningful intrinsic limit for adversarially robust classification . The standard concentration of measure problem ( 3.1 ) , which aims to find the optimal subset that has the smallest -expansion with regard to the input metric probability space ( X , µ , ∆ ) , does not involve the concept function c ( · ) that determines the underlying class label of each input . Therefore , no matter how we assign the labels to the inputs , the concentration function h ( µ , α , ) will remain the same for the considered metric probability space . In sharp contrast , learning an adversarially-robust classifier depends on the joint distribution of both the inputs and the labels . Moreover , when the standard concentration function is translated into an intrinsic limit of adversarial robustness , it is defined with respect to the set of imperfect classfiers Fα ( see Lemma 3.1 ) . The only restriction imposed by Fα is that the classifier ( or equivalently , the measure of the corresponding error region ) has risk at least α . This fails to consider whereas whether the classifier is learnable or not under the given classification problem . Therefore , the intrinsic robustness limit implied by standard concentration AdvRob ( Fα , c ) could be much higher than AdvRob ( Flearn , c ) , where Flearn denotes the set of classifiers that can be produced by some supervised learning method . Hence , it is not surprising that Mahloujifar et al . ( 2019b ) found that the adversarial robustness attained by state-of-theart robust training methods for several image benchmarks is much lower than the intrinsic robustness limit implied by standard concentration of measure . In this work , to obtain a more meaningful intrinsic robustness limit we restrict the search space of the standard concentration problem ( 3.1 ) by considering both the underlying class labels and the learnability of the given classification problem . Gaussian Mixture Model . We further illustrate the insufficiency of standard concentration under a simple Gaussian mixture model . Let X ⊆ Rn be the input space and Y = { −1 , +1 } be the label space . Assume all the inputs are first generated according to a mixture of 2-Gaussian distribution : x ∼ µ = 12N ( −θ , σ 2In ) + 1 2N ( θ , σ 2In ) , then labeled by a concept function c ( x ) = sgn ( θ > x ) , where θ ∈ Rn and σ ∈ R are given parameters ( this concept function is also the Bayes optimal classifier , which best separates the two Gaussian clusters ) . Theorem 3.2 , proven in Appendix C.1 , characterizes the optimal solution to the standard concentration problem under this assumed model . Theorem 3.2 . Consider the above Gaussian mixture model with ` 2 perturbation metric . The optimal solution to the standard concentration problem ( 3.1 ) is a halfspace , either H− = { x ∈ X : θ > x+ b · ‖θ‖2 ≤ 0 } or H+ = { x ∈ X : θ > x− b · ‖θ‖2 ≥ 0 } , where b is a parameter depending on α and θ such that µ ( H− ) = µ ( H+ ) = α . Remark 3.3 . Theorem 3.2 suggests that for the Gaussian mixture model , the optimal subset achieving the smallest -expansion under ` 2-norm distance metric is a halfspaceH , which is far away from the boundary between the two Gaussian classes for small α . When translated into the intrinsic robustness problem , the corresponding optimal classifier f has to be constructed by treatingH as the only error region , or more precisely f ( x ) = c ( x ) if x /∈ H ; f ( x ) 6= c ( x ) otherwise . This optimally constructed classifier f , however , does not match our intuition of what a predictive classifier would do under the considered Gaussian mixture model . In particular , since all the inputs inH and their neighbours share the same class label and are also far away from the boundary , examples that fall intoH should be easily classified correctly using simple decision rule , such as k-nearest neighbour or maximum margin , whereas examples that are close to the boundary should be more likely to be misclassified as errors by supervisedly-learned classifiers . This confirms our claim that standard concentration is not sufficient for capturing a meaningful intrinsic robustness limit .

Summary: The paper improves the previous results on the intrinsic robustness (i.e., an upper bound on the adversarial robustness over a set of classifiers) based on concentration of data distribution, by incorporating the constraint on the label uncertainty of the models. Specifically, the paper tightens the upper bound on the adversarial robustness given a family of models with error rates $\ge \alpha$, by additionally assuming that the average label uncertainty must be $\ge \gamma$. This requires an information of label uncertainty for each data sample, so the paper leveraged CIFAR-10H (CIFAR-10 with human-annotated uncertainty) in their experiments. Experimental results on CIFAR-10/CIFAR-10H show that the proposed upper bound can be practically computable, and improves the previous upper bound given the knowledge of CIFAR-10H.