The Many Faces of Adversarial Risk

1 Introduction . Neural networks are known to be vulnerable to adversarial attacks , which are imperceptible perturbations to input data that maximize loss [ 38 , 15 , 5 ] . Developing algorithms resistant to such attacks has received considerable attention in recent years [ 8 , 28 , 24 , 20 ] , motivated by safety-critical applications such as autonomous driving [ 18 , 27 ] , medical imaging [ 17 , 23 , 22 ] and law [ 21 , 6 ] . A classification algorithm with high accuracy ( low risk ) in the absence of an adversary may have poor accuracy ( high risk ) when an adversary is present . Thus , a modified notion known as adversarial risk is used to quantify the adversarial robustness of algorithms . Algorithms that minimize adversarial risk are deemed robust . Procedures for finding them have been effective in practice [ 24 , 41 , 28 ] , spurring numerous theoretical investigations into adversarial risk and its minimizers . There is no universally agreed upon definition of adversarial risk . Even the simplest setting of binary classification in Rd with an ` 2 adversary admits various definitions involving set expansions [ 9 , 16 ] , transport maps [ 29 ] , Markov kernels [ 31 ] , and couplings [ 26 ] . These works broadly interpret adversarial risk as a measure of robustness to small perturbations , but their definitions differ in subtle details such as the class of adversaries and algorithms considered , budget constraints placed on the adversary , assumptions on the loss function , and the geometry of decision boundaries . Optimal adversarial risk is most commonly defined as the minimax risk under adversarial contamination [ 24 , 33 ] . Other notable characterizations include an optimal transport cost between data generating distributions in [ 30 , 2 , 10 , 11 ] , the optimal value of a distributionally robust optimization problem [ 36 , 35 , 40 ] , and the value of a two-player zero-sum game [ 26 , 29 , 3 , 4 ] . The diversity of definitions for adversarial risk makes it challenging to compare approaches . Moreover , not all approaches are rigorous . For instance , the classes of adversarial strategies and classifier 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . algorithms are often unclear , and issues of measurability are ignored . Although this may be harmless for applied research , it has led to incorrect proofs and insufficient assumptions in some theoretical works ; a mathematically rigorous foundation for adversarial risk is essential for future research . In this paper , we examine various notions of adversarial risk for binary classification in a nonparametric setting , where the decision boundary ( or decision region ) of a classifier is an arbitrary subset of the input space . We present rigorous definitions of adversarial risk and identify conditions under which these definitions are equivalent . We consider the general setting of Polish spaces ( complete , separable metric spaces ) , and present stronger results for the Euclidean space ( Rd ) . Our contributions are as follows : • We examine the definition of adversarial risk based on set expansions . For Polish spaces , we observe that adversarial risk is not Borel measurable , and hence , not well-defined when the decision region is an arbitrary set . We show that the problem can be resolved by considering a Polish space equipped with the universal completion of the Borel σ-algebra and restricting the decision regions to Borel sets . For the Euclidean space with the Lebesgue σ-algebra , we show that adversarial risk is well-defined for any Lebesgue measurable decision region . Our key lemma ( Lemma 4.3 ) shows that the Lebesgue σ-algebra is preferred over the Borel σ-algebra because set expansions are Lebesgue measurable but not necessarily Borel measurable . These results are contained in Section 4 . • We show that the definition of adversarial risk using set expansions is identical to a notion of risk that appears in robust hypothesis testing with∞-Wasserstein uncertainty sets . We prove this result in Polish spaces using the theory of measurable selections [ 1 , 43 ] . In Rd , we are able to use the powerful theory of Choquet capacities [ 7 ] ( in particular , Huber and Strassen ’ s 2-alternating capacities [ 19 ] ) to establish results of a similar nature . These results are contained in Section 5 . • We consider the binary classification setup with unequal priors and show ( under suitable assumptions ) that the optimal adversarial risk from the above definitions is characterized by an unbalanced optimal transport cost between data-generating distributions . For both Polish spaces and Rd , the main tool we use is Theorem 6.1 in which we generalize a classical result of Strassen on excess-cost optimal transport [ 37 , 42 ] from probability measures to finite measures with possibly unequal mass . This generalizes results of [ 31 , 2 ] on binary classification , which were only for equal priors . These results are contained in Section 6 . • We consider the setup of a zero-sum game between the adversary and the algorithm . We show that the value of this game ( adversarial risk ) is equal to the minimum Bayes error between a pair of distributions belonging to the∞-Wasserstein uncertainty sets centered around true data-generating distributions . We prove the existence of a pure Nash equilibrium in this game for Rd and for Polish spaces with a midpoint property . This extends/strengthens the results of [ 26 , 29 , 3 ] to non-parametric classifiers . These results are contained in Section 7 . The paper is organized as follows : In Section 2 , we present preliminary definitions from optimal transport and metric space topology . In Section 3 , we discuss various definitions of adversarial risk and present more related work . Sections 4 , 5 , 6 and 7 contain our main contributions summarized above . We conclude the paper in Section 8 and discuss future research directions . We emphasize that rectifying measure theoretic issues with existing formulations of adversarial risk is one of our contributions , but not the main focus of our paper . We start our presentation with fixing measurability and well-definedness ( in Section 4 ) because otherwise we will not be able to rigorously present our main results in the subsequent sections , namely : relation to robust hypothesis testing and Choquet capacities in section 5 , generalizing the results of [ 2 , 30 ] in section , 6 proving minimax theorems and existence of Nash equilibria and extending the results of [ 26 , 3 , 29 ] in section 7 . Notation : Throughout the paper , we use X to denote a Polish space ( a complete , separable metric space ) with metric d and Borel σ-algebra B ( X ) . For x ∈ X and r ≥ 0 , let Br ( x ) denote the closed ball of radius r centered at x . We use P ( X ) andM ( X ) to denote the space of probability measures and finite measures defined on the measure space ( X , B ( X ) ) , respectively . Let B ( X ) denote the universal completion of B ( X ) . Let P ( X ) andM ( X ) denote the space of probability measures and finite measures defined on the complete measure space ( X , B ( X ) ) . For µ , ν ∈ M ( X ) , we say ν dominates µ if µ ( A ) ≤ ν ( A ) for all A ∈ B ( X ) and write µ ν . When X is Rd , we use L ( X ) to denote the Lebesgue σ-algebra and λ to denote the d-dimensional Lebesgue measure . Note that L ( X ) = B ( X ) for X = Rd . For a positive integer n , we use [ n ] to denote the finite set { 1 , . . . , n } . 2 Preliminaries . 2.1 Metric Space Topology . We introduce three different notions of set expansions . For ≥ 0 and A ∈ B ( X ) , the -Minkowski expansion of A is given by A⊕ : = ∪a∈AB ( a ) . The -closed expansion of A is defined as A : = { x ∈ X : d ( x , A ) ≤ } , where d ( x , A ) = infa∈A d ( x , a ) . The -open expansion of A is defined as A ) : = { x ∈ X : d ( x , A ) < } . We use the notation A− to denote ( ( Ac ) ) c. Similarly , A : = ( ( Ac ) ⊕ ) c. For example , consider the set A = ( 0 , 1 ] in the space ( X , d ) = ( R , | · | ) and > 0 . Then A⊕ = ( − , 1 + ] , A = [ − , 1 + ] and A ) = ( − , 1 + ) . For any A ∈ B ( X ) , A is closed and A ) is open . Hence , A , A ) ∈ B ( X ) . Moreover , A ) ⊆ A⊕ ⊆ A . However , A⊕ may not be in B ( X ) ( see Appendix for an example ) . In general , the Minkowski sum of two Borel sets need not be Borel [ 13 ] , and that of two Lebesgue measurable sets need not be Lebesgue measurable [ 34 ] . 2.2 Optimal Transport . Let µ , ν ∈ P ( X ) . A coupling between µ and ν is a joint probability measure π ∈ P ( X 2 ) with marginals µ and ν . The set Π ( µ , ν ) ⊆ P ( X 2 ) denotes the set of all couplings between µ and ν . The optimal transport cost between µ and ν under a cost function c : X × X → [ 0 , ∞ ) is defined as Tc ( µ , ν ) = infπ∈Π ( µ , ν ) ∫ X 2 c ( x , x ′ ) dπ ( x , x′ ) . For a positive integer p , the p-Wasserstein distance between µ and ν is defined as , Wp ( µ , ν ) = ( Tdp ( µ , ν ) ) 1 p . The∞-Wasserstein metric is defined as W∞ ( µ , ν ) = limp→∞Wp ( µ , ν ) . It can also be expressed in the following ways : W∞ ( µ , ν ) = inf π∈Π ( µ , ν ) ess sup ( x , x′ ) ∼π d ( x , x′ ) = inf { δ > 0 : µ ( A ) ≤ ν ( Aδ ) ∀A ∈ B ( X ) } . ( 1 ) Given a µ ∈ P ( X ) and a measurable function f : X → X , the push-forward of µ by f is defined as a probability measure f ] µ ∈ P ( X ) given by f ] µ = µ ( f−1 ( A ) ) for all A ∈ B ( X ) . 3 Adversarial Risk : Definitions and Related Work . We consider a binary classification setting on feature space X . Let p0 , p1 ∈ P ( X ) be the datagenerating distributions for labels 0 and 1 , respectively . Let the prior probabilities for labels 0 and 1 be in the ratio T : 1 where we assume T ≥ 1 without loss of generality . For a space of classifiers parametrized by w ∈ W and a loss function ` : ( X × Y ) ×W → [ 0 , ∞ ) , the adversarial risk of a classifier w ∈ W under an adversarial budget of ≥ 0 is defined as [ 24 , 33 ] , R⊕ ( ` , w ) = E ( x , y ) [ sup d ( x , x′ ) ≤ ` ( ( x′ , y ) , w ) ] . ( 2 ) If the loss function ` ( · , w ) is measurable , upper semi-continuous and bounded above for all w ∈ W , [ 26 ] show that R ( ` , w ) is well-defined . But in general , it may not be so . A case of special interest is the 0-1 loss function with non-parametric classifiers of the form fA ( x ) : = 1 { x ∈ A } where A ∈ B ( X ) . In this case , ` 0/1 ( ( x , y ) , A ) = 1 { x ∈ A , y = 0 } + 1 { x ∈ Ac , y = 1 } . Hence , R⊕ ( ` 0/1 , A ) = T T + 1 Ep0 [ sup d ( x , x′ ) ≤ 1 { x′ ∈ A } ] + 1 T + 1 Ep1 [ sup d ( x , x′ ) ≤ 1 { x′ ∈ Ac } ] = T T + 1 p0 ( A ⊕ ) + 1 T + 1 p1 ( ( A c ) ⊕ ) . ( 3 ) A problem with the formulation in equation 3 is the ambiguity over the measurability of the sets A⊕ and ( Ac ) ⊕ . Even when A ∈ B ( X ) , it is not guaranteed that A⊕ , ( Ac ) ⊕ ∈ B ( X ) ( see Appendix for an example ) . Hence , R⊕ ( ` 0/1 , A ) is not well-defined for all A ∈ B ( X ) . It is shown in [ 31 ] that R⊕ ( ` 0/1 , A ) is well-defined when A is either closed or open , but its validity beyond that is unknown . A simple fix to this measurability problem is to use closed set expansion instead of the Minkowski set expansion , as done in [ 25 ] . This leads to the following formulation . R ( ` 0/1 , A ) = T T + 1 p0 ( A ) + 1 T + 1 p1 ( ( A c ) ) . ( 4 ) The above definition is well-defined for any A ∈ B ( X ) because A and ( Ac ) are both closed and hence , measurable . However , under the above definition , a point x ∈ A may be perturbed to x′ ∈ A such that d ( x , x′ ) > . For example , when A = ( 0 , 1 ) , we have A = [ − , ] and an adversary may transport x = δ > 0 to x′ = − , violating the budget constraint at x . Remark 1 . The formulations in equations ( 2 ) , ( 3 ) and ( 4 ) can give a strictly positive adversarial risk even for a “ perfect ” ( i.e . Bayes optimal ) classifier . This is consistent with the literature on adversarial examples where even a perfect classifier is forced to make errors in the presence of evasion attacks . These formulations of adversarial risk correspond to “ constant-in-the-ball ” risk of [ 16 ] and “ corrupted-instance ” risk in [ 9 , 25 ] . Here , an adversarial risk of zero is only possible if the supports of p0 and p1 are non-overlapping and separated by at least 2 . This is not the case with other formulations of adversarial risk such as “ exact-in-the-ball ” risk [ 16 ] , “ prediction-change risk and “ error-region ” risk [ 9 , 25 ] . We focus on the “ corrupted-instance ” family of risks in this work . Another approach to defining adversarial risk is by explicitly defining the class of adversaries of budget as measurable transport maps f : X → X that push-forward the true data distribution such that no point is transported by more than a distance of ; i.e. , d ( x , f ( x ) ) ≤ . The transport map-based adversarial risk [ 29 ] is formally defined as follows : RF ( ` 0/1 , A ) = sup f0 , f1 : X→X ∀x∈X , d ( x , fi ( x ) ) ≤ T T + 1 f0 ] p0 ( A ) + 1 T + 1 f1 ] p1 ( ( A c ) ) . ( 5 ) Yet another definition uses the robust hypothesis testing framework with W∞ uncertainty sets . In this approach , an adversary perturbs the true distribution pi to a corrupted distribution p′i such that W∞ ( pi , p ′ i ) ≤ . From ( 1 ) , this is equivalent to the existence of a coupling π ∈ Π ( pi , p′i ) such that ess sup ( x , x′ ) ∼π d ( x , x ′ ) ≤ . The adversarial risk with such an adversary is given by RΓ ( ` 0/1 , A ) = sup W∞ ( p1 , p′1 ) , W∞ ( p0 , p ′ 0 ) ≤ T T + 1 p′0 ( A ) + 1 T + 1 p′1 ( ( A c ) ) . ( 6 ) Clearly , RF ( ` 0/1 , A ) ≤ RΓ ( ` 0/1 , A ) , but conditions for equality were not studied in prior work . Moreover , their relation to set expansion-based definitions in ( 3 ) and ( 4 ) was also unknown . Next we discuss some characterizations of optimal adversarial risk , defined as R∗⊕ : = infA∈B ( X ) R⊕ ( ` 0/1 , A ) . In [ 30 , 2 ] , it is shown that R∗ = 1 2 [ 1 − D ( p0 , p1 ) ] for equal priors ( T = 1 ) , where D is an optimal transport cost defined as follows . Definition 1 ( D cost ) . Let µ , ν ∈ P ( X ) . Let ≥ 0 . Let c : X 2 → { 0 , 1 } be such that c ( x , x′ ) = 1 { ( x , x′ ) ∈ X × X : d ( x , x′ ) > 2 } . Then for µ , ν ∈ P ( X ) and ≥ 0 , D ( µ , ν ) = Tc ( µ , ν ) . For = 0 , D reduces to the total variation distance . While D0 is a metric on P ( X ) , D ( for > 0 ) is neither a metric nor a pseudometric [ 31 ] . Other formulations of optimal adversarial risk are inspired from game theory [ 29 , 26 , 3 ] . Consider a game between two players : ( 1 ) The adversary whose action space is pairs of distributions p′0 , p ′ 1 ∈ P ( X ) , and ( 2 ) The algorithm whose action space is the space of decision regions of the form A ∈ B ( X ) } . For T > 0 , define r : B ( X ) × P ( X ) × P ( X ) → [ 0 , 1 ] as , r ( A , µ , ν ) = TT+1µ ( A ) + 1 T+1ν ( ( A c ) ) . The payoff when the algorithm plays first is given by infA∈B ( X ) supW∞ ( p0 , p′0 ) , W∞ ( p1 , p′1 ) ≤ r ( A , p ′ 0 , p ′ 1 ) , and this quantity is interpreted as the optimal adversarial risk in this setup .

Summary: The paper extends previous works of the study of the adversarial risk. The authors interest in the well definition of the adversarial risk. They then show a minimax theorem on the game between the classifier and the adversary.