The Many Faces of Adversarial Risk

1 Introduction . Neural networks are known to be vulnerable to adversarial attacks , which are imperceptible perturbations to input data that maximize loss [ 38 , 15 , 5 ] . Developing algorithms resistant to such attacks has received considerable attention in recent years [ 8 , 28 , 24 , 20 ] , motivated by safety-critical applications such as autonomous driving [ 18 , 27 ] , medical imaging [ 17 , 23 , 22 ] and law [ 21 , 6 ] . A classification algorithm with high accuracy ( low risk ) in the absence of an adversary may have poor accuracy ( high risk ) when an adversary is present . Thus , a modified notion known as adversarial risk is used to quantify the adversarial robustness of algorithms . Algorithms that minimize adversarial risk are deemed robust . Procedures for finding them have been effective in practice [ 24 , 41 , 28 ] , spurring numerous theoretical investigations into adversarial risk and its minimizers . There is no universally agreed upon definition of adversarial risk . Even the simplest setting of binary classification in Rd with an ` 2 adversary admits various definitions involving set expansions [ 9 , 16 ] , transport maps [ 29 ] , Markov kernels [ 31 ] , and couplings [ 26 ] . These works broadly interpret adversarial risk as a measure of robustness to small perturbations , but their definitions differ in subtle details such as the class of adversaries and algorithms considered , budget constraints placed on the adversary , assumptions on the loss function , and the geometry of decision boundaries . Optimal adversarial risk is most commonly defined as the minimax risk under adversarial contamination [ 24 , 33 ] . Other notable characterizations include an optimal transport cost between data generating distributions in [ 30 , 2 , 10 , 11 ] , the optimal value of a distributionally robust optimization problem [ 36 , 35 , 40 ] , and the value of a two-player zero-sum game [ 26 , 29 , 3 , 4 ] . The diversity of definitions for adversarial risk makes it challenging to compare approaches . Moreover , not all approaches are rigorous . For instance , the classes of adversarial strategies and classifier 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . algorithms are often unclear , and issues of measurability are ignored . Although this may be harmless for applied research , it has led to incorrect proofs and insufficient assumptions in some theoretical works ; a mathematically rigorous foundation for adversarial risk is essential for future research . In this paper , we examine various notions of adversarial risk for binary classification in a nonparametric setting , where the decision boundary ( or decision region ) of a classifier is an arbitrary subset of the input space . We present rigorous definitions of adversarial risk and identify conditions under which these definitions are equivalent . We consider the general setting of Polish spaces ( complete , separable metric spaces ) , and present stronger results for the Euclidean space ( Rd ) . Our contributions are as follows : • We examine the definition of adversarial risk based on set expansions . For Polish spaces , we observe that adversarial risk is not Borel measurable , and hence , not well-defined when the decision region is an arbitrary set . We show that the problem can be resolved by considering a Polish space equipped with the universal completion of the Borel σ-algebra and restricting the decision regions to Borel sets . For the Euclidean space with the Lebesgue σ-algebra , we show that adversarial risk is well-defined for any Lebesgue measurable decision region . Our key lemma ( Lemma 4.3 ) shows that the Lebesgue σ-algebra is preferred over the Borel σ-algebra because set expansions are Lebesgue measurable but not necessarily Borel measurable . These results are contained in Section 4 . • We show that the definition of adversarial risk using set expansions is identical to a notion of risk that appears in robust hypothesis testing with∞-Wasserstein uncertainty sets . We prove this result in Polish spaces using the theory of measurable selections [ 1 , 43 ] . In Rd , we are able to use the powerful theory of Choquet capacities [ 7 ] ( in particular , Huber and Strassen ’ s 2-alternating capacities [ 19 ] ) to establish results of a similar nature . These results are contained in Section 5 . • We consider the binary classification setup with unequal priors and show ( under suitable assumptions ) that the optimal adversarial risk from the above definitions is characterized by an unbalanced optimal transport cost between data-generating distributions . For both Polish spaces and Rd , the main tool we use is Theorem 6.1 in which we generalize a classical result of Strassen on excess-cost optimal transport [ 37 , 42 ] from probability measures to finite measures with possibly unequal mass . This generalizes results of [ 31 , 2 ] on binary classification , which were only for equal priors . These results are contained in Section 6 . • We consider the setup of a zero-sum game between the adversary and the algorithm . We show that the value of this game ( adversarial risk ) is equal to the minimum Bayes error between a pair of distributions belonging to the∞-Wasserstein uncertainty sets centered around true data-generating distributions . We prove the existence of a pure Nash equilibrium in this game for Rd and for Polish spaces with a midpoint property . This extends/strengthens the results of [ 26 , 29 , 3 ] to non-parametric classifiers . These results are contained in Section 7 . The paper is organized as follows : In Section 2 , we present preliminary definitions from optimal transport and metric space topology . In Section 3 , we discuss various definitions of adversarial risk and present more related work . Sections 4 , 5 , 6 and 7 contain our main contributions summarized above . We conclude the paper in Section 8 and discuss future research directions . We emphasize that rectifying measure theoretic issues with existing formulations of adversarial risk is one of our contributions , but not the main focus of our paper . We start our presentation with fixing measurability and well-definedness ( in Section 4 ) because otherwise we will not be able to rigorously present our main results in the subsequent sections , namely : relation to robust hypothesis testing and Choquet capacities in section 5 , generalizing the results of [ 2 , 30 ] in section , 6 proving minimax theorems and existence of Nash equilibria and extending the results of [ 26 , 3 , 29 ] in section 7 . Notation : Throughout the paper , we use X to denote a Polish space ( a complete , separable metric space ) with metric d and Borel σ-algebra B ( X ) . For x ∈ X and r ≥ 0 , let Br ( x ) denote the closed ball of radius r centered at x . We use P ( X ) andM ( X ) to denote the space of probability measures and finite measures defined on the measure space ( X , B ( X ) ) , respectively . Let B ( X ) denote the universal completion of B ( X ) . Let P ( X ) andM ( X ) denote the space of probability measures and finite measures defined on the complete measure space ( X , B ( X ) ) . For µ , ν ∈ M ( X ) , we say ν dominates µ if µ ( A ) ≤ ν ( A ) for all A ∈ B ( X ) and write µ ν . When X is Rd , we use L ( X ) to denote the Lebesgue σ-algebra and λ to denote the d-dimensional Lebesgue measure . Note that L ( X ) = B ( X ) for X = Rd . For a positive integer n , we use [ n ] to denote the finite set { 1 , . . . , n } . 2 Preliminaries . 2.1 Metric Space Topology . We introduce three different notions of set expansions . For ≥ 0 and A ∈ B ( X ) , the -Minkowski expansion of A is given by A⊕ : = ∪a∈AB ( a ) . The -closed expansion of A is defined as A : = { x ∈ X : d ( x , A ) ≤ } , where d ( x , A ) = infa∈A d ( x , a ) . The -open expansion of A is defined as A ) : = { x ∈ X : d ( x , A ) < } . We use the notation A− to denote ( ( Ac ) ) c. Similarly , A : = ( ( Ac ) ⊕ ) c. For example , consider the set A = ( 0 , 1 ] in the space ( X , d ) = ( R , | · | ) and > 0 . Then A⊕ = ( − , 1 + ] , A = [ − , 1 + ] and A ) = ( − , 1 + ) . For any A ∈ B ( X ) , A is closed and A ) is open . Hence , A , A ) ∈ B ( X ) . Moreover , A ) ⊆ A⊕ ⊆ A . However , A⊕ may not be in B ( X ) ( see Appendix for an example ) . In general , the Minkowski sum of two Borel sets need not be Borel [ 13 ] , and that of two Lebesgue measurable sets need not be Lebesgue measurable [ 34 ] . 2.2 Optimal Transport . Let µ , ν ∈ P ( X ) . A coupling between µ and ν is a joint probability measure π ∈ P ( X 2 ) with marginals µ and ν . The set Π ( µ , ν ) ⊆ P ( X 2 ) denotes the set of all couplings between µ and ν . The optimal transport cost between µ and ν under a cost function c : X × X → [ 0 , ∞ ) is defined as Tc ( µ , ν ) = infπ∈Π ( µ , ν ) ∫ X 2 c ( x , x ′ ) dπ ( x , x′ ) . For a positive integer p , the p-Wasserstein distance between µ and ν is defined as , Wp ( µ , ν ) = ( Tdp ( µ , ν ) ) 1 p . The∞-Wasserstein metric is defined as W∞ ( µ , ν ) = limp→∞Wp ( µ , ν ) . It can also be expressed in the following ways : W∞ ( µ , ν ) = inf π∈Π ( µ , ν ) ess sup ( x , x′ ) ∼π d ( x , x′ ) = inf { δ > 0 : µ ( A ) ≤ ν ( Aδ ) ∀A ∈ B ( X ) } . ( 1 ) Given a µ ∈ P ( X ) and a measurable function f : X → X , the push-forward of µ by f is defined as a probability measure f ] µ ∈ P ( X ) given by f ] µ = µ ( f−1 ( A ) ) for all A ∈ B ( X ) . 3 Adversarial Risk : Definitions and Related Work . We consider a binary classification setting on feature space X . Let p0 , p1 ∈ P ( X ) be the datagenerating distributions for labels 0 and 1 , respectively . Let the prior probabilities for labels 0 and 1 be in the ratio T : 1 where we assume T ≥ 1 without loss of generality . For a space of classifiers parametrized by w ∈ W and a loss function ` : ( X × Y ) ×W → [ 0 , ∞ ) , the adversarial risk of a classifier w ∈ W under an adversarial budget of ≥ 0 is defined as [ 24 , 33 ] , R⊕ ( ` , w ) = E ( x , y ) [ sup d ( x , x′ ) ≤ ` ( ( x′ , y ) , w ) ] . ( 2 ) If the loss function ` ( · , w ) is measurable , upper semi-continuous and bounded above for all w ∈ W , [ 26 ] show that R ( ` , w ) is well-defined . But in general , it may not be so . A case of special interest is the 0-1 loss function with non-parametric classifiers of the form fA ( x ) : = 1 { x ∈ A } where A ∈ B ( X ) . In this case , ` 0/1 ( ( x , y ) , A ) = 1 { x ∈ A , y = 0 } + 1 { x ∈ Ac , y = 1 } . Hence , R⊕ ( ` 0/1 , A ) = T T + 1 Ep0 [ sup d ( x , x′ ) ≤ 1 { x′ ∈ A } ] + 1 T + 1 Ep1 [ sup d ( x , x′ ) ≤ 1 { x′ ∈ Ac } ] = T T + 1 p0 ( A ⊕ ) + 1 T + 1 p1 ( ( A c ) ⊕ ) . ( 3 ) A problem with the formulation in equation 3 is the ambiguity over the measurability of the sets A⊕ and ( Ac ) ⊕ . Even when A ∈ B ( X ) , it is not guaranteed that A⊕ , ( Ac ) ⊕ ∈ B ( X ) ( see Appendix for an example ) . Hence , R⊕ ( ` 0/1 , A ) is not well-defined for all A ∈ B ( X ) . It is shown in [ 31 ] that R⊕ ( ` 0/1 , A ) is well-defined when A is either closed or open , but its validity beyond that is unknown . A simple fix to this measurability problem is to use closed set expansion instead of the Minkowski set expansion , as done in [ 25 ] . This leads to the following formulation . R ( ` 0/1 , A ) = T T + 1 p0 ( A ) + 1 T + 1 p1 ( ( A c ) ) . ( 4 ) The above definition is well-defined for any A ∈ B ( X ) because A and ( Ac ) are both closed and hence , measurable . However , under the above definition , a point x ∈ A may be perturbed to x′ ∈ A such that d ( x , x′ ) > . For example , when A = ( 0 , 1 ) , we have A = [ − , ] and an adversary may transport x = δ > 0 to x′ = − , violating the budget constraint at x . Remark 1 . The formulations in equations ( 2 ) , ( 3 ) and ( 4 ) can give a strictly positive adversarial risk even for a “ perfect ” ( i.e . Bayes optimal ) classifier . This is consistent with the literature on adversarial examples where even a perfect classifier is forced to make errors in the presence of evasion attacks . These formulations of adversarial risk correspond to “ constant-in-the-ball ” risk of [ 16 ] and “ corrupted-instance ” risk in [ 9 , 25 ] . Here , an adversarial risk of zero is only possible if the supports of p0 and p1 are non-overlapping and separated by at least 2 . This is not the case with other formulations of adversarial risk such as “ exact-in-the-ball ” risk [ 16 ] , “ prediction-change risk and “ error-region ” risk [ 9 , 25 ] . We focus on the “ corrupted-instance ” family of risks in this work . Another approach to defining adversarial risk is by explicitly defining the class of adversaries of budget as measurable transport maps f : X → X that push-forward the true data distribution such that no point is transported by more than a distance of ; i.e. , d ( x , f ( x ) ) ≤ . The transport map-based adversarial risk [ 29 ] is formally defined as follows : RF ( ` 0/1 , A ) = sup f0 , f1 : X→X ∀x∈X , d ( x , fi ( x ) ) ≤ T T + 1 f0 ] p0 ( A ) + 1 T + 1 f1 ] p1 ( ( A c ) ) . ( 5 ) Yet another definition uses the robust hypothesis testing framework with W∞ uncertainty sets . In this approach , an adversary perturbs the true distribution pi to a corrupted distribution p′i such that W∞ ( pi , p ′ i ) ≤ . From ( 1 ) , this is equivalent to the existence of a coupling π ∈ Π ( pi , p′i ) such that ess sup ( x , x′ ) ∼π d ( x , x ′ ) ≤ . The adversarial risk with such an adversary is given by RΓ ( ` 0/1 , A ) = sup W∞ ( p1 , p′1 ) , W∞ ( p0 , p ′ 0 ) ≤ T T + 1 p′0 ( A ) + 1 T + 1 p′1 ( ( A c ) ) . ( 6 ) Clearly , RF ( ` 0/1 , A ) ≤ RΓ ( ` 0/1 , A ) , but conditions for equality were not studied in prior work . Moreover , their relation to set expansion-based definitions in ( 3 ) and ( 4 ) was also unknown . Next we discuss some characterizations of optimal adversarial risk , defined as R∗⊕ : = infA∈B ( X ) R⊕ ( ` 0/1 , A ) . In [ 30 , 2 ] , it is shown that R∗ = 1 2 [ 1 − D ( p0 , p1 ) ] for equal priors ( T = 1 ) , where D is an optimal transport cost defined as follows . Definition 1 ( D cost ) . Let µ , ν ∈ P ( X ) . Let ≥ 0 . Let c : X 2 → { 0 , 1 } be such that c ( x , x′ ) = 1 { ( x , x′ ) ∈ X × X : d ( x , x′ ) > 2 } . Then for µ , ν ∈ P ( X ) and ≥ 0 , D ( µ , ν ) = Tc ( µ , ν ) . For = 0 , D reduces to the total variation distance . While D0 is a metric on P ( X ) , D ( for > 0 ) is neither a metric nor a pseudometric [ 31 ] . Other formulations of optimal adversarial risk are inspired from game theory [ 29 , 26 , 3 ] . Consider a game between two players : ( 1 ) The adversary whose action space is pairs of distributions p′0 , p ′ 1 ∈ P ( X ) , and ( 2 ) The algorithm whose action space is the space of decision regions of the form A ∈ B ( X ) } . For T > 0 , define r : B ( X ) × P ( X ) × P ( X ) → [ 0 , 1 ] as , r ( A , µ , ν ) = TT+1µ ( A ) + 1 T+1ν ( ( A c ) ) . The payoff when the algorithm plays first is given by infA∈B ( X ) supW∞ ( p0 , p′0 ) , W∞ ( p1 , p′1 ) ≤ r ( A , p ′ 0 , p ′ 1 ) , and this quantity is interpreted as the optimal adversarial risk in this setup .

Summary: This paper formalizes different definitions of adversarial risk for binary classification and studies the equivalence between those definitions. Moreover, the authors analyze optimal adversarial risk through the lens of optimal transport, which leads to the existence of a Nash equilibrium of the adversarial robustness game. The contribution of this paper is theoretical.