Title: It begins with a boundary: A geometric view on probabilistically robust learning
Venue: NeurIPS 2023
Accepted: no
Keywords: probabilistically robust learning

adversarial attacks

existence of solutions

nonlocal perimeters

regularization
Abstract: Although deep neural networks have achieved super-human performance on many classification tasks, they often exhibit a worrying lack of robustness towards adversarially generated examples. Thus, considerable effort has been invested into reformulating Empirical Risk Minimization (ERM) into an adversarially robust framework. Recently, attention has shifted towards approaches which interpolate between the robustness offered by adversarial training and the higher clean accuracy and faster training times of ERM. In this paper, we take a fresh and geometric view on one such method---Probabilistically Robust Learning (PRL) (Robey et al., ICML, 2022). We propose a geometric framework for understanding PRL, which allows us to identify a subtle flaw in its original formulation and to introduce a family of probabilistic nonlocal perimeter functionals to address this. We prove existence of solutions using novel relaxation methods and study properties as well as local limits of the introduced perimeters.

Abstract

Although deep neural networks have achieved super-human performance on many classification tasks, they often exhibit a worrying lack of robustness towards adversarially generated examples. Thus, considerable effort has been invested into reformulating Empirical Risk Minimization (ERM) into an adversarially robust framework. Recently, attention has shifted towards approaches which interpolate between the robustness offered by adversarial training and the higher clean accuracy and faster training times of ERM. In this paper, we take a fresh and geometric view on one such method-Probabilistically Robust Learning (PRL) [Robey et al., 2022]. We propose a geometric framework for understanding PRL, which allows us to identify a subtle flaw in its original formulation and to introduce a family of probabilistic nonlocal perimeter functionals to address this. We prove existence of solutions using novel relaxation methods and study properties as well as local limits of the introduced perimeters.

1 Introduction 14

The fragility of DNN-based classifiers in the face of adversarial examples [Goodfellow et al., 2014, 15

Chen et al., 2017, Qin et al., 2019, Cai et al., 2021] and distributional shifts [Quinoñero Candela 16

17

et al., 2008, Hendrycks et al., 2021] is by now nearly as familiar as their successes. In light of this,

a multitude of works (see Section 1.4) propose replacing standard Empirical Risk Minimization 18

(ERM) [Vapnik, 1999] with a more robust alternative (see, e.g., Madry et al. [2017]). Unfortunately 19

there is no free lunch: robust classifiers frequently exhibit degraded performance on clean data and 20

significantly longer training times [Tsipras et al., 2018]. Consequently, identifying frameworks which 21

balance performance and robustness is of pressing interest to the Machine Learning (ML) community, 22

23

and over the past several years many such frameworks have been proposed [Zhang et al., 2019, Wang

24

et al., 2020, Robey et al., 2022]. Moreover, it is crucial that the mechanism by which such frameworks

balance these competing aims be understood. 25

Beginning with the Probabilistically Robust Learning (PRL) of Robey et al. [2022] we analyze such 26

frameworks geometrically. This perspective reveals a subtle, paradoxical aspect of PRL: sometimes 27

28

the adversary modeled by this framework corrects, instead of exploits, the learner! Fortunately, the

geometric perspective we propose suggests a natural remedy which leads to an interpretation of the 29

corrected PRL as regularized ERM where a certain nonlocal notion of length (or perimeter) of the 30

decision boundary acts as a regularizer. We exemplify this correction in Figure 1. The interpretation 31

of PRL as perimeter-regularized ERM leads us to further generalizations, and we provide a novel 32

view of the Conditional Value at Risk (CVaR) relaxation of PRL proposed by Robey et al. [2022]. 33

A

≜

class green

A

c

≜

class red

A

≜

class green

A

c

≜

class red

1.1 From empirical risk minimization to robustness 34

Given an input space X , an output space Y , a probability measure µ ∈ P ( X × Y ) , a loss function 35 ℓ : Y × Y → R , and a hypothesis class H , the standard risk minimization problem is 36

For training classifiers which are robust against adversarial attacks Goodfellow et al. [2014], Madry 37 et al. [2017] suggested adversarial training: 38

Here X is assumed to have the structure of a metric space and B ε ( x ) for ε ≥ 0 denotes the (open or 39 closed) ball of radius ε around x . 40

The recent work by Robey et al. [2022] offered an alternative to adversarial training in order to 41 reduce the (in general) large trade-off between accuracy and robustness inherent in (2), see Tsipras 42 et al. [2018], Robey et al. [2022] for discussion. Instead of requiring classifiers to be robust to all 43 available attacks around a point x -as enforced through the supremum in (2)-one may consider 44 a less stringent notion of robustness, only requiring classifiers to be robust to 100 × (1 -p )% of 45 possible attacks when attacks are drawn from a certain distribution p x centered at x . For this, the 46 authors introduced the so-called p -ess sup operator for p ∈ [0 , 1) and suggested replacing (2) by 47

where { p x } x ∈X is a family of probability distributions. The prototypical example to keep in mind 48 for X = R d is the uniform distribution over the ε -ball around x , i.e., p x := Unif( B ε ( x )) , which is 49 particularly relevant when dealing with adversarial attacks on image classifiers. 50

For a probability distribution p and a function f , the quantity p -ess sup x ′ ∼ p f ( x ′ ) is defined as the 51 smallest value t ∈ R such that the probability of a randomly chosen point x ′ ∼ p satisfying f ( x ′ ) > t 52 is smaller than p , which reduces to the usual essential supremum of f with respect to p if p = 0 : 53

To better understand the model (3) we temporarily restrict our attention to binary classification (i.e., 54 Y = { 0 , 1 } ) using indicator functions of admissible sets (i.e., H := { 1 A : A ∈ A} ). Note that we 55 identify the two expressions 1 A ( x ) = 1 x ∈ A . We focus on the 0 -1 loss ℓ (˜ y, y ) = 1 ˜ y = y which equals 56 one if y = ˜ y and zero otherwise. In this scenario (1) reduces to the geometric problem 57

̸

̸

and minimizers are called Bayes classifiers. Similarly, adversarial training (2) can be rewritten as 58

59

60

61

62

63

64

65

66

67

68

69

where for a set A ∈ A its fattening by ε -balls is defined as A ⊕ ε := ⋃ x ∈ A B ε ( x ) . Hence (5) enforces that all points with distance at most ε to the decision boundary be adversarially robust.

On the other hand the PRL model (3) reduces to

where A ⊕ ε is replaced by a 'probabilistic fattening', i.e., one considers the set of all x for which the probability that a neighboring point sampled from p x lies inside A is larger than p . To the best of our knowledge, existence of solutions for (6) or even (3) has not been proved so far.

1.2 Geometric modification of probabilistically robust learning

To motivate our geometric modification of the PRL model from Robey et al. [2022], it is insightful to investigate the regularization effect that PRL has compared to standard risk minimization. We let ρ i ( · ) := µ ( · × { i } ) denote the non-normalized conditional distributions of the points with label i . Subtracting the standard risk in (4) from the one in (6) and disintegrating using ρ 0 and ρ 1 we obtain

We highlight that this expression does not constitute a non-negative functional of A . Hence the loss 70 function in (6) is not a regularized version of the standard risk (4) and in fact can be strictly smaller. 71 This observation reveals a subtle flaw in the approach of Robey et al. [2022]: Points which lie in thin 72 or spike-like regions of A penetrating the other class and that are more likely to have the label zero 73 than the label one (meaning they lie in the set { ρ 0 > ρ 1 } ) yield negative contributions in (7) and 74 are hence favored . Such a scenario is visualized on the left side of Figure 1. From an adversarial 75 perspective this means that points which are already misclassified are attacked nevertheless, which 76 can lead to the bizarre situation that the adversary helps the learner by putting these points in the 77 correct class with high probability, thereby reducing both adversarial robustness and clean accuracy. 78

79

80

81

We fix this by designing a probabilistically robust risk as non-negative regularization of the standard risk. For this we define probabilistic perimeter functionals which only penalize points which are classified correctly and admit a large portion of attacks around them, see the right side of Figure 1.

82

83

84

85

86

87

88

89

90

91

92

93

1.3 Our contributions

Our main contributions are the following:

We address the geometric limitation of the model by Robey et al. [2022] by introducing a family of perimeter regularizations.

We prove existence of soft and hard binary classifiers under weak conditions on the family of perimeters and hypothesis classes, using novel relaxation techniques.

We investigate the relationship between the introduced family of perimeters and local perimeters in Euclidean space for small adversarial budgets.

We extend our models to encompass general loss functions and hypothesis classes. Our numerical experiments demonstrate that our geometric correction can enhance the adversarial robustness of probabilistically robust classifiers without compromising clean accuracy.

1.4 Related work

Adversarial training was developed by Goodfellow et al. [2014], Madry et al. [2017] as an approach 94 to train networks that are less sensitive to adversarial attacks. Shafahi et al. [2019] reduced its 95 computational complexity by reusing gradients from the backpropagation when training neural 96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

networks. Wong et al. [2020] showed that training with noise perturbations followed by a single signed gradient ascent (FGSM) step can be on par with adversarial training while being much cheaper. This approach was picked up and improved upon by Andriushchenko and Flammarion [2020] based on gradient alignment. Different authors also investigated test-time robustification of pretrained classifiers using randomized smoothing [Cohen et al., 2019] or geometric / gradient-based approaches [Schwinn et al., 2021, 2022]. While some of the previous models use a combination of random perturbations and gradient-based adversarial attacks to robustify classifiers, Robey et al. [2022] proposed probabilistically robust learning, which is entirely based on random perturbations. PRL aims to interpolate between clean and adversarial accuracy and enjoys the favorable sample complexity of vanilla empirical risk minimization; see also Raman et al. [2023] for more insights on this issue. Connections between adversarial training and local perimeter regularization of decision boundaries were explored by García Trillos and Murray [2022] and then rigorously tied by Bungert and Stinson [2022]. Our work is in line with a series of papers [Pydi and Jog, 2021, Awasthi et al., 2021a,b, Frank and Niles-Weed, 2022, Frank, 2022, Bungert et al., 2023, García Trillos et al., 2023] that explore the existence of solutions to adversarial training problems in different settings. These existence proofs involve dealing with different kinds of measurability issues, depending on whether open or closed balls B ε ( x ) are used in the attack model. For open balls one can work with the Borel σ -algebra A = B ( X ) [Bungert et al., 2023], whereas closed balls require the use of the universal σ -algebra to make sure that A ⊕ ε is measurable [Pydi and Jog, 2021, Awasthi et al., 2021a,b]. Recently, these results were improved by García Trillos et al. [2023] who also proved for the case of multi-class classification that even for the closed ball model Borel measurable classifiers (albeit not necessarily indicator functions of measurable sets) exist and that for all but countably many values of the adversarial budget ε > 0 the open and the closed ball models have the same minimal value.

2 Geometry and existence of probabilistically robust classifiers

2.1 The binary classification setting with 0 -1 loss

In this section we shall introduce our baseline model, which is based on a suitable geometric regularization of the standard risk. Later we shall embed it into a family of models. For clarity we first discuss hard classifiers (characteristic functions of sets) and then soft classifiers (functions with values in [0 , 1] ). The generalization to general models and loss functions is postponed to Section 3.

We start by defining the probabilistic perimeter for p ∈ [0 , 1) of an admissible set A ∈ A as follows:

ProbPer( A ) penalizes correctly classified points x for which more than 100 × p % of their neighbors, sampled from p x , constitute an attack. The perimeter can be rewritten in integral form:

The first reformulation (9) should be compared to (7), while the one in (10) will be useful later 129 on. The use of the term perimeter to describe the functional ProbPer will become more apparent 130 shortly in Section 2.4, and at this point it is worth highlighting that ProbPer is always a non-negative 131 quantity. This motivates introducing the following regularized risk 132

Our first theorem states that ProbR equals the expected maximum of the sample-wise standard risk and the probabilistically robust risk from Robey et al. [2022], cf. (4) and (6).

Theorem 1. For all A ∈ A it holds that

133

134

135

̸

̸

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

The interpretation of the statement of this theorem in the light of Figure 1 is clear: Only if a point 136 x is correctly classified-meaning 1 1 A ( x ) = y = 0 -the probabilistically robust regularization kicks 137 in through the first term in the maximum. Points which are incorrectly classified will always be 138 penalized even if most attacks correct the label, i.e., if 1 P x ′ ∼ p x [ 1 A ( x ′ ) = y ] >p = 0 . Thus, minimizing 139 ProbR instead of R prob corrects the pathology identified in Section 1.2. 140

141

142

143

̸

2.2 Extensions in the binary classification setting

Given the formula of ProbPer in (10), several natural extensions suggest themselves. E.g., one may replace the indicator function 1 t>p with a different function Ψ( t ) to define other notions of perimeter

as well as their corresponding probabilistically robust losses 144

For Ψ( t ) := 1 t>p the perimeter ProbPer Ψ reduces to ProbPer and so do the associated risks. Of particular interest is Ψ p ( t ) := min { t/p, 1 } -the smallest concave function that lies above Ψ( t ) = 1 t>p -which will allow us to develop deep connections between the theoretical and computational aspects of probabilistically robust learning. Our relaxation using the function Ψ is very similar to the one by Raman et al. [2023] who proved PAC learnability if Ψ is Lipschitz, see Appendix A.6 for more details. In order to rigorously study ProbR Ψ we first make our setting precise.

Assumption 1. We let X be a set and A ⊂ 2 X be a σ -algebra. We assume that:

( X × Y , A⊗ 2 { 0 , 1 } , µ ) is a probability space;

( X , A , ρ ) is a probability space, where we define ρ ( · ) := µ ( · × { 0 , 1 } ) ;

{ p x } x ∈X is a family such that ( X , A , p x ) is a probability space for ρ -almost every x ∈ X .

The following theorem establishes existence of minimizers of the risk ProbR Ψ for concave and non-decreasing functions Ψ . This existence result is astonishing since the standard method of calculus of variations is not directly applicable, with the reason being that problem (15) does not provide enough compactness for lower semicontinuity of the perimeter functional ProbPer Ψ . Instead, the proof is based on convex relaxations to soft classifiers where we use a lower semicontinuous surrogate functional and a total variation defined through a coarea formula which-if Ψ is concave and non-decreasing-lower-bounds the surrogate.

Theorem 2. Suppose Ψ : [0 , 1] → [0 , 1] is concave and non-decreasing, and that Assumption 1 holds. Then, there exists a solution to the problem

Furthermore, ProbR Ψ can also be interpreted as a sample-wise maximum, analogous to Theorem 1.

Theorem 3. For all A ∈ A and measurable Ψ : [0 , 1] → [0 , 1] it holds

̸

̸

Note that for the non-concave function Ψ( t ) = 1 t>p an existence proof along the lines of Theorem 2 166 is not available since certain relaxation techniques therein rely on concavity of Ψ . However, in the 167 next section we shall provide an existence theorem for soft classifiers which is valid for very general 168 functions Ψ , including Ψ( t ) = 1 t>p . 169

̸

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

2.3 Extension to soft classifiers

Another natural extension features 'soft classifiers' instead of indicator functions of admissible sets. Such classifiers are particularly relevant since they include the neural network based models with Softmax activation in the last layer which are used in practice. We start by defining a suitable regularization functional for soft classifiers. Given a A -measurable function u : X → [0 , 1] we define

which satisfies J Ψ ( 1 A ) = ProbPer Ψ ( A ) for every choice of Ψ . Hence, it is a natural generalization of the perimeter to soft classifiers and one could call J Ψ a total variation. However, it is neither positively homogeneous nor convex so this name would be misleading. Instead, for the proof of Theorem 2 we shall construct a suitable total variation functional ProbTV Ψ which upper-bounds J Ψ .

The next theorem asserts existence of soft classifiers for the regularized risk minimization using J Ψ for very general functions Ψ and hypothesis classes H , requiring only that Ψ be lower semicontinuous. For example, every continuous function and also Ψ( t ) = 1 t>p for p ∈ [0 , 1] satisfies this. The existence theorem is valid for all hypotheses classes which are closed in a suitable sense.

Theorem 4. Under Assumption 1, for every lower semicontinuous function Ψ : [0 , 1] → [0 , 1] , and whenever H is a weak-* closed hypothesis class of A -measurable functions u : X → [0 , 1] in the sense of Definition 1 in the appendix, there exists a solution to the problem

Example 1. Let us consider three interesting hypothesis classes of weak-* closed classifiers for which Theorem 4 applies. More detailed explanations are given in Appendix A.8.

The simplest such class H is the class of all A -measurable soft classifiers u : X → [0 , 1] which could be referred to as agnostic classifiers since they are not parametrized.

An example with more practical relevance is the class of (feedforward or residual) neural networks defined on the unit cube X := [ -1 , 1] d with uniformly bounded parameters

where we assume that the activations σ l : R → R are continuous. Note that the boundedness of the weights cannot be relaxed. To see this, consider the (very simplistic) neural network u n ( x ) = tanh( w n x ) for x ∈ [ -1 , 1] and w n ∈ R . For w n →∞ it is easy to see that u n converges to u ( x ) := sign( x ) which does not lie in the same hypothesis class.

Finally, one can also consider the class of hard linear classifiers on R d . Letting θ ( t ) := 1 t> 0 denote the Heaviside function, this class is given by

where one interprets u ( x ) := θ ( w · x + b ) as u ≡ 1 if b = ∞ and u ≡ 0 if b = -∞ . If the distributions ρ 0 , ρ 1 , and p x are sufficiently nice, then H has the desired closedness property.

2.4 Properties and asymptotics of ProbPer Ψ

In this section we shall discuss the interpretation of the functional ProbPer Ψ defined in (13) as a perimeter . We do this in two ways.

First, we focus on the case where Ψ is concave and non-decreasing and prove that ProbPer Ψ is a submodular functional . If, in addition, Ψ is assumed to satisfy Ψ(0) = 0 , then ProbPer Ψ ( X ) = ProbPer Ψ ( ∅ ) = 0 . Following Chambolle et al. [2015], for Ψ satisfying these properties one can interpret ProbPer Ψ as a generalized perimeter, i.e., a functional that can be used to measure the 'size' of the boundary of a set. In Appendix A.3 we introduce ProbPer Ψ 's induced (generalized) total variation and use it in the proof of Theorem 2; note that, as discussed by Bungert et al. [2023], the adversarial problem (5) also induces a generalized perimeter with associated total variation.

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

Theorem 5. If Ψ(0) = 0 , then ProbPer Ψ ( X ) = ProbPer Ψ ( ∅ ) = 0 . If Ψ is concave and nondecreasing, then the functional ProbPer Ψ is submodular, meaning that

ProbPer Ψ ( A ∪ B ) + ProbPer Ψ ( A ∩ B ) ≤ ProbPer Ψ ( A ) + ProbPer Ψ ( B ) ∀ A,B ∈ A .

Example 2. For Ψ( t ) = t our perimeter reduces to the perimeter on the random walk space ( X , p ) , introduced by Mazón et al. [2020]: ProbPer Ψ ( A ) = ∫ X\ A ∫ A d p x d ρ 0 ( x ) + ∫ A ∫ X\ A d p x d ρ 1 ( x ) .

Second, we consider more general Ψ and show that ProbPer Ψ is related to a standard local perimeter when the adversarial budget approaches zero; for the case of adversarial training such a connection was proved by Bungert and Stinson [2022] where the authors utilized the notion of Gamma-convergence of functionals. We take a first step in this direction by proving that for sufficiently smooth sets the probabilistic perimeter converges to a local one if the family of probability distributions p x localizes suitably. For example, one could think of p x := Unif( B ε ( x )) , which converges to a point mass at x if ε → 0 . To make our setting precise, we pose the following general assumption:

Assumption 2. We assume that X = R d , Ψ(0) = 0 , Ψ is measurable and bounded, and ρ 1 , ρ 0 have continuous densities with respect to the Lebesgue measure which we shall also denote as ρ 1 , ρ 0 . Furthermore, we assume that there is ε > 0 and a measurable function K : X × R d → [0 , ∞ ) such that for every x ∈ R d we have the representation

We also assume that for every x ∈ X we have K ( x, · ) ∈ L 1 ( R d ) , ∫ R d K ( x, z ) d z = 1 , and K ( x, z ) = 0 if | z | > 1 , and that for every z ∈ R d the mapping x ↦→ K ( x, z ) is C 1 .

Proposition 1. Under Assumption 2, if A has a compact C 1 , 1 boundary and either Ψ is continuous or there exists a constant c > 0 such that K ( x, z ) ≥ c for all x ∈ X and | z | ≤ 1 , then

where we let n ( x ) denote the normal to ∂A at a point x ∈ ∂A , and for any vector v ∈ R d we define

Remark 1. If K is radially symmetric and independent of x ∈ X , then σ 0 Ψ = σ 1 Ψ =: σ Ψ is just a constant. E.g., for K ( x, z ) := | B 1 (0) | -1 1 | z |≤ 1 and Ψ( t ) = 1 t>p it is trivial that for p = 0 we have σ Ψ = 1 . However, for p ≥ 1 2 one easily sees σ Ψ = 0 , hence the limiting perimeter equals zero and there is no regularization effect. Using the function Ψ( t ) = min { t/p, 1 } corrects this degeneracy.

Notably, for radially symmetric K the limiting perimeter in (17) coincides, provided σ Ψ > 0 , with the one derived for adversarial training (problem (5)) by Bungert and Stinson [2022], although they considered more general (potentially discontinuous) densities ρ i . In particular, our result indicates that for very small adversarial budgets the regularization effect of both probabilistically robust learning and adversarial training is dominated by the perimeter in (17). While Proposition 1 already completes half of the proof (namely the limsup inequality) of Gamma-convergence of 1 ε ProbPer Ψ to the limiting perimeter, the remaining liminf inequality is beyond the scope of this paper. Proving that the convergence (17) does not only hold for sufficiently smooth sets as assumed in Proposition 1 but even in the sense of Gamma-convergence is an extremely important topic for future work since only Gamma-convergence allows to deduce from the convergence of the perimeters that also the solutions of probabilistically robust learning converge to certain regular Bayes classifiers as ε → 0 , see Bungert and Stinson [2022, Section 4.2].

3 General models

We now shift our attention to training general hypotheses h ∈ H using general loss functions ℓ : Y × Y → R . Motivated by Theorems 1 and 3 we propose the following probabilistically robust optimization problem:

In the mathematical finance or economics literature the p -ess sup operator is better known as the value 250 at risk (VaR) of a random variable at level p and it is notoriously hard to optimize. VaR is closely 251 related to other risk measures like, for instance, the conditional value at risk (CVaR) which is convex 252 and easier to optimize [Robey et al., 2022, Rockafellar et al., 2000]. For a function f : X → R and a 253 probability distribution p the CVaR at level p is defined as 254

It is easy to see that p -ess sup x ′ ∼ p f ( x ′ ) ≤ CVaR p ( f ; p ) . Using CVaR in place of the p -ess sup operator, a tractable version of (18) is

We emphasize that, if the loss function ℓ ( · , · ) is convex in its first argument, then (20) is a convex function of the hypothesis h . Furthermore, CVaR is positively homogeneous and hence also (20) is positively homogeneous in the loss function. So, taking the maximum of the samplewise CVaR and standard risk is meaningful as both terms scale in the same way.

In the binary classification case we can prove the following interesting result that the CVaR relaxation corresponds precisely to using the risk ProbR Ψ with a special piecewise linear and concave function Ψ for which our theory from Section 2.2 applies. In Appendix A.5 we prove a more general version of the following statement, replacing the [ · ] + operation in (19) with a Leaky ReLU .

Theorem 6. Let the function Ψ p : [0 , 1] → [0 , 1] be defined as Ψ p ( t ) := min { t/p, 1 } . Then it holds

̸

̸

and as a consequence for all A ∈ A :

̸

̸

An immediate consequence of Theorem 6 is that for binary classification (20) has a solution.

Corollary 1. Under Assumption 1 and in the setting of Theorem 6 problem (20) has a solution.

In Appendix A.5 we collect a few more observations concerning the CVaR, especially focussing on its behavior for p > 1 . These geometric properties, the homogeneity with respect to the loss function, its potentially favorable sample complexity (see the discussion in Appendix A.6), and its versatility for algorithmic implementation make (20) a notable generalization of the adversarial training problem (2). Notice that when p → 0 one formally recovers (2) from (20).

4 Numerical results

We build upon the code of Robey et al. [2022]. The algorithmic realization of (20) is a straightforward adaptation of their algorithm, which alternatingly minimizes the inner optimization problem that defines CVaR and the outer optimization to find a suitable classifier, see Algorithm 1 in Appendix B. In our experiments, we conduct a comparative analysis between their algorithm (denoted as 'Original' in Table 1) and Algorithm 1 in the appendix which is based on (20) (denoted as 'Geometric'). We report the clean, and adversarial accuracies (subject to PGD attacks), as well as accuracies on noise-augmented data and quantile accuracies for different values of p (see [Robey et al., 2022, (6.1)] for the definition) averaged over three runs; see Appendix B.2 for more training details. Our experiments are conducted on MNIST and CIFAR-10 and to ensure a fair comparison we adhere to the hyperparameter settings described by Robey et al. [2022], such that both the original and geometric algorithms utilize the same set of hyperparameters for each specified value of p . The corresponding results for several baseline algorithms including empirical risk minimization and adversarial training can be found in their paper. We perform model selection based on the best clean validation accuracy. The results in Table 1 show that for moderate values of p our geometric modification induces higher adversarial robustness than the original PRL without loss of clean accuracy (see, in particular, the results for MNIST with p = 0 . 1 and for CIFAR-10 with p = 0 . 3 ). In the noise augmented metrics as well as for extreme values of p close to 0 or equal to 0 . 5 both algorithms behave comparably. The latter can be expected from out theoretical results, in particular Proposition 1.

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

Note that the original or the geometric version of PRL should not be expected to match the adversarial robustness of classifiers trained with PGD attacks [Madry et al., 2017] or other worst-case optimization techniques. Instead, they shine with superior clean accuracies and easier training while maintaining probabilistic and a certain degree of adversarial robustness, as also observed by Robey et al. [2022].

We also remark that our sweep over different values of p confirms that increasing this parameter interpolates between low and high clean accuracies. However, it should be noted that it does not necessarily result in a direct interpolation between high and low adversarial or probabilistic accuracy, as claimed by Robey et al. [2022]. These observations hold true for both the original algorithm and our geometric modification, and despite utilizing their code and hyperparameters, we were unable to reproduce the exact results reported by Robey et al. [2022, Tables 1-4].

5 Discussion and Conclusion

In this paper we considered probabilistically robust learning (PRL), originally proposed by Robey et al. [2022]. We corrected a subtle but crucial theoretical flaw in the original formulation by introducing a regularization of the standard risk with nonlocal perimeters measuring the susceptibility of the decision boundary towards high-probability adversarial attacks. For binary classification we proved existence of optimal hard classifiers and of very general classes of soft classifiers including neural networks. We also provided an asymptotic expansion for smooth decision boundaries to show that for small adversarial budgets the probabilistic perimeters discussed in the paper induce the same regularization effect as adversarial training. For general (not necessarily binary) problems we showed that the natural loss function to choose is the sample-wise maximum of the standard loss and conditional value at risk (CVaR).

One limitation of PRL is that it does not completely solve the accuracy vs. robustness trade-off, which remains a challenging problem. Furthermore, while the formal limit of PRL as p → 0 is the worst-case adversarial problem, the algorithms for solving PRL exhibit limitations for very small values of p (in the computation of CVaR p ). Still, the results for moderately large values of p are encouraging and future work should focus on understanding of this trade-off better.

The rich mathematical theory developed in this paper opens up new avenues for research, such as the explicit design of probabilistic regularizers for algorithms and exploring the variational convergence of the probabilistic perimeter and its implications for adversarial robustness.

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

References

Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training. Advances in Neural Information Processing Systems , 33:16048-16059, 2020.

Pranjal Awasthi, Natalie S Frank, and Mehryar Mohri. On the existence of the adversarial Bayes classifier. Advances in Neural Information Processing Systems , 34:2978-2990, 2021a.

Pranjal Awasthi, Natalie S Frank, and Mehryar Mohri. On the existence of the adversarial Bayes classifier (extended version). arXiv preprint arXiv:2112.01694 , 2021b.

Leon Bungert and Kerrek Stinson. Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning. arXiv preprint arXiv:2211.15223 , 2022.

Leon Bungert, Nicolás García Trillos, and Ryan Murray. The geometry of adversarial training in binary classification. Information and Inference: A Journal of the IMA , 12(2):921-968, 06 2023. ISSN 2049-8772. doi: 10.1093/imaiai/iaac029.

HanQin Cai, Yuchen Lou, Daniel McKenzie, and Wotao Yin. A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization. In International Conference on Machine Learning , pages 1193-1203. PMLR, 2021.

Antonin Chambolle, Massimiliano Morini, and Marcello Ponsiglione. Nonlocal curvature flows. Archive for Rational Mechanics and Analysis , 218:1263-1329, 2015.

Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security , pages 15-26, 2017.

Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In international conference on machine learning , pages 1310-1320. PMLR, 2019.

Nelson Dunford and Jacob T Schwartz. Linear Operators: General theory . Linear Operators. Interscience Publishers, 1958. ISBN 9780470226056.

Natalie S Frank. Existence and minimax theorems for adversarial surrogate risks in binary classification. arXiv preprint arXiv:2206.09098 , 2022.

Natalie S Frank and Jonathan Niles-Weed. The consistency of adversarial training for binary classification. arXiv preprint arXiv:2206.09099 , 2022.

Nicolás García Trillos and Ryan Murray. Adversarial classification: Necessary conditions and geometric flows. Journal of Machine Learning Research , 23(187):1-38, 2022.

Nicolás García Trillos, Matt Jacobs, and Jakwang Kim. On the existence of solutions to adversarial training in multiclass classification. arXiv preprint arXiv:2305.00075 , 2023.

Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014.

Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 8340-8349, 2021.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 , 2017.

José M Mazón, Marcos Solera, and Julián Toledo. The total variation flow in metric random walk spaces. Calculus of Variations and Partial Differential Equations , 59:1-64, 2020.

Muni Sreenivas Pydi and Varun Jog. The many faces of adversarial risk. Advances in Neural Information Processing Systems , 34:10000-10012, 2021.

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

Yao Qin, Nicholas Carlini, Garrison Cottrell, Ian Goodfellow, and Colin Raffel. Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In International conference on machine learning , pages 5231-5240. PMLR, 2019.

Joaquin Quinoñero Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning . Mit Press, 2008.

Vinod Raman, Unique Subedi, and Ambuj Tewari. On proper learnability between average- and worst-case robustness. arXiv preprint arXiv:2211.05656 , 2023.

Alexander Robey, Luiz Chamon, George J Pappas, and Hamed Hassani. Probabilistically robust learning: Balancing average and worst-case performance. In International Conference on Machine Learning , pages 18667-18686. PMLR, 2022.

R Tyrrell Rockafellar, Stanislav Uryasev, et al. Optimization of conditional value-at-risk. Journal of risk , 2:21-42, 2000.

Leo Schwinn, An Nguyen, René Raab, Leon Bungert, Daniel Tenbrinck, Dario Zanca, Martin Burger, and Bjoern Eskofier. Identifying untrustworthy predictions in neural networks by geometric gradient analysis. In Uncertainty in Artificial Intelligence , pages 854-864. PMLR, 2021.

Leo Schwinn, Leon Bungert, An Nguyen, René Raab, Falk Pulsmeyer, Doina Precup, Björn Eskofier, and Dario Zanca. Improving robustness against real-world and worst-case distribution shifts through decision region quantification. In International Conference on Machine Learning , pages 19434-19449. PMLR, 2022.

Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Advances in Neural Information Processing Systems , 32, 2019.

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. arXiv preprint arXiv:1805.12152 , 2018.

Vladimir Vapnik. The nature of statistical learning theory . Springer science & business media, 1999.

Bao Wang, Binjie Yuan, Zuoqiang Shi, and Stanley J Osher. EnResNet: ResNets ensemble via the Feynman-Kac formalism for adversarial defense and beyond. SIAM Journal on Mathematics of Data Science , 2(3):559-582, 2020.

Eric Wong, Leslie Rice, and J Zico Kolter. Fast is better than free: Revisiting adversarial training. arXiv preprint arXiv:2001.03994 , 2020.

Matthew D Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701 , 2012.

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning , pages 7472-7482. PMLR, 2019.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper claims to identify a flaw in the formulation of probabilistically robust learning.  To correct this supposed flaw, the authors propose a regularization scheme based on a notion of probabilistic perimeter.  This leads to a set of theoretical results that generalize the losses used in probabilistic robustness, identify conditions under which solutions exist, and initiate a theory of the limiting cases of these generalizations.  Experiments on MNIST and CIFAR-10 are also provided, showing that algorithmically, this new approach performs almost exactly the same as the previously proposed CVaR approach.\n\nSTRENGTHS:\n**New generalizations of probabilistic robustness.**  This paper rigorously analyzes the framework of probabilistic robustness, and provides several avenues for future research.  The perimeter-based perspective gives a new interpretation; it feels almost, but not necessarily the same as saying that this paper is to the original PRL paper as TRADES is to vanilla adversarial training (e.g., PGD).  I think that mathematically, this paper presents a novel and deep set of results which may be of interest to others in the community.  Furthermore, the idea of introducing the probabilistic perimeter is relatively clean, in the sense that it seems natural to consider the ProbR(A) function as a decomposition of the probabilistic risk and the clean risk.\n\n**Existence of solutions.**  The existence proofs are interesting and general here.  In particular, I found the insight that the CVaR problem can be directly related to a generalized perimeter problem in Theorem 6 to be surprising.  Although it was previously known that solutions existed in special cases, this result is much more general.\n\n**Writing.**  In general, I found the writing in this paper to be relatively strong.  I would tend to soften some of the language, such as referring to various results as \"astonishing\" and calling various results \"extremely important,\" but at the end of the day, these are stylistic choices that can be made at the discretion of the authors.\n\nWEAKNESSES:\n### Contributions\n\nI'll start by evaluating the extent to which this paper lives up to the contributions it claims to make in Section 1.3.  In particular, I am going to focus on the first and last contributions; the middle two contributions are solid in my opinion.\n\n**Contribution 1.**  The first claimed contribution is as follows:\n\n> \"We address the geometric limitation of the model by Robey et al. [2022] by introducing a family of perimeter regularizations.\"\n\nThroughout the paper, the authors make several related claims regarding the validity of the approach outlined in [Robey et al. 2022], e.g.,\n\n> \"We propose a geometric framework for understanding PRL, which allows us to identify a subtle flaw in its original formulation and to introduce a family of probabilistic nonlocal perimeter functionals to address this.\"\n\nIn order to validate this contribution, it is essential that the authors \n\n1. Clearly articulate which property PRL is missing; \n2. Explain why missing this property constitute a flaw; and\n3. Show that their perimeter-based method addresses this flaw.  \n\nIn this respect, I'm not sure whether any of these three points are satisfied.  I'll go through each of these points, and then hopefully we can have a discussion about this during the rebuttal phase.  To this end, let's start by looking into the claims of this PRL paper.  Based on my reading, the goal of that work is to do the following (as quoted from the introduction of [Robey et al., 2022]):\n\n> \"The fundamental drawbacks of these learning paradigms motivate the need for a new robust learning framework that (i) avoids the conservatism of adversarial robustness without incurring the brittleness of ERM, (ii) provides an interpretable way to balance nominal performance and robustness, and (iii) admits an efficient and effective algorithm. To this end, in this paper we propose a framework called probabilistic robustness that bridges the gap between the accurate, yet brittle average-case approach of ERM and the robust, yet conservative worst-case approach of adversarial training. By enforcing robustness to most rather than to all perturbations, we show theoretically and empirically that probabilistic robustness meets the desiderata in (i)–(iii)\"\n\nIn short, the goal seems to be to design a learning framework that enforces robustness to most of the perturbations in a ball around each data point.  Now, returning to the paper under review, the supposed flaw is described as follows:\n\n> \"We highlight that [the PRL model] does not constitute a non-negative functional of A. Hence the loss function in (6) is not a regularized version of the standard risk (4) and in fact can be strictly smaller. This observation reveals a subtle flaw in the approach of Robey et al. [2022]: Points which lie in thin or spike-like regions of A penetrating the other class and that are more likely to have the label zero than the label one (meaning they lie in the set $[\\rho_0 > \\rho_1]$) yield negative contributions in (7) and are hence favored.  Such a scenario is visualized on the left side of Figure 1. From an adversarial perspective this means that points which are already misclassified are attacked nevertheless, which can lead to the bizarre situation that the adversary helps the learner by putting these points in the correct class with high probability, thereby reducing both adversarial robustness and clean accuracy.\"\n\nBelow I'll list some of my thoughts about this, but first, it seems important to agree on what constitutes a \"flaw.\"  It seems reasonable to define a flaw to be **an inherent limitation or weakness that affects the validity, reliability, or applicability** of a particular method.  Hopefully the authors agree with this definition, because I'm going to base my discussion of this on this definition (and if not, we can hash it out in the rebuttal).  Here are my thoughts vis-a-vis this definition.\n\n* *Non-negative functional.* Unless I missed something, it's not clear to me whether [Robey et al., 2022] claimed that their framework constituted a \"non-negative functional of A.\"  Or, zooming out, they did not claim that it was essential to their paradigm to ensure that the clean data was classified correctly.  While they do measure the clean accuracy in their experiments, their paradigm does not seem to be designed to explicitly minimize the standard risk or any regularized form of the standard risk.  On the contrary, the aim seems to be to enforce robustness to \"most\" of the perturbations in a ball around each data point, which more or less seems to be accomplished in the PRL paper.  Therefore, the motivation based on the non-negative functional perspective underlying the supposed flaw seems to be in doubt.  Or in other words, can a feature of paradigm constitute a flaw if that paradigm never claimed that feature as a contribution?\n\n* *Spike-like regions.* The next issue is that the authors do not define what they mean by \"spike-like\" regions.  This confusion is amplified by Figure 1.  In this figure, why is PRL being applied to a non-robust classifier?  And what do the green and yellow regions correspond to?  Are the regions $\\epsilon$-expansions of the decision boundary?  If this is the case, then why are these regions not a uniform distance away from the decision boundary?  And why are some points in red on the wrong side of the decision boundary?  Without *clearly* identifying the problem, it's difficult for the reader to accept that a flaw has truly been found.\n\n* *Label probabilities.* I don't understand how the idea that the data points which are \"most likely to have the label zero than the label one\" yield negative contributions to the standard risk.  Can the authors explain this further, or can you illustrate this set $\\{x : \\rho_1(x) > \\rho_0(x)$ in Figure 1?  And then how is this impacted by the perturbations added to the instance? The picture that I (perhaps incorrectly) have in mind is Figure 1 from the randomized smoothing paper by Cohen et al. (https://arxiv.org/pdf/1902.02918.pdf), although I'm having trouble mapping back onto the proposed approach.\n\n* *Adversarial perspective.* The next point worth discussing is the \"adversarial perspective\" mentioned above.  Based on my understanding of PRL, there is no adversary.  The learner simply receives a distribution over perturbations and attempts to be robust against most of the perturbations sampled from that distribution.  And so from this perspective, it's unclear how the adversary factors in.  Thus, I don't understand the so-called \"bizarre situation\" that the authors claim to identify, or why it would serve to reduce both clean and adversarial accuracy.  Could the authors elaborate here?\n\nTo summarize, it's unclear to me what the supposed flaw is and why this flaw constitutes an inherent limitation of the method.  And given this, the contribution involving introducing a new family of \"perimeter regularization\" is unclear, because it's not clear what flaw or problem this solution is addressing.  More generally, I think it's worthy being wary of claiming to have found a flaw in existing work unless that flaw can be clearly identified.  While scientific research should certainly scrutinize past work, it seems important to charitably acknowledge what that past work claims -- and dually, what it does not claim -- to accomplish.  And in this case, my understanding of the argument laid out by the authors is that it is somewhat parallel to the ideas laid out in PRL, in that the supposed flaw is not something that the PRL framework was ever intended to do.  However, it could be the case that I have misunderstood either the author's argument or the original PRL paper, and therefore a discussion of this would be welcome.\n\n**Contribution 4.**  The final contribution is as follows:\n\n> \"We extend our models to encompass general loss functions and hypothesis classes. Our numerical experiments demonstrate that our geometric correction can enhance the adversarial robustness of probabilistically robust classifiers without compromising clean accuracy.\"\n\nI believe that this is inaccurate.  Based on Table 1, it seems that around half of the time, the original PRL method actually achieves better adversarial robustness than the proposed geometric method.  Moreover, the clean accuracy of models trained with the original PRL method seem to be either better or within a single percentage point of the accuracies reported for the geometric variant.  Therefore, it seems at best inconclusive as to whether this approach actually results in better performance.\n\n---\n\n### **Notation**\n\n**Nested subscripts.**  The notation in this paper is at times difficult to parse.  I find it particularly hard to parse expressions like (9) and (10).  I think that it would be easier to read if the authors used expectation notation, and avoided the up and down carets for and/or.  For example, the first term in (10) could be written as\n\n$$\n\\mathbb{E} [ x\\in A \\quad\\text{and}\\quad \\text{Pr}_{x'\\sim \\mathbb{P}_x}[x'\\in A] > p ]\n$$\n\nand one could perhaps add $x\\sim\\rho_0$ as an underscore (LaTeX in openreview isn't rendering this underscore correctly).  This would be much clearer, and it would prevent the use of three levels of underscores, e.g., $1_{\\mathbb{P}_{x'\\sim p_x}}$ in (10).\n\n**Definition of the probabilistic perimeter.**  I think that the definition of the probabilistic perimeter could be clearer.  Overall, I'm not sure what is gained by reducing the problem to the setting of binary classification and considering sets $A$.  (8) defines ProbPer in this special case, but to me, it seems clearer to define it as follows:\n\n$$\n\\text{ProbPer}(f) = \\text{Pr}_{x,y} [  f(x)\\neq y \\quad\\text{and}\\quad \\text{Pr}[f(x+\\delta) = y] > p ]\n$$\n\nwhere $f$ is a classifier and the inner probability is taken WRT $\\mathfrak{p}_x$.  This avoids the somewhat confusing $\\rho_0$ notation and naturally extends to multi-class classification settings.\n\n### **Related work**\n\n**Other interpolation methods.**  It would be worth citing other recent relevant work on robust learning between the average and worst case.  Two notable papers, which are compared to in the PRL paper, are \n\n> Rice, Leslie, et al. \"Robustness between the worst and average case.\" Advances in Neural Information Processing Systems 34 (2021): 27840-27851.\n> Li, Tian, et al. \"Tilted empirical risk minimization.\" arXiv preprint arXiv:2007.01162 (2020).\n\nOne could also think of the TRADES paper as interpolating between adversarial training and ERM through the trade-off parameter $\\beta$, so it would be worth comparing to this paper as well.\n\n**Engagement with PRL.**  As discussed above, it would be worth softening the claims regarding the extent to which PRL has fundamental flaws.  In addition, the authors make several other claims in the same spirit which don't necessarily agree with past work.  For instance,\n\n> \"To the best of our knowledge, existence of solutions for (6) or even (3) has not been proved so far.\" \n\nHaving read the PRL paper, this seems to be untrue.  In [Robey et al., 2022], the authors derive closed form expressions for PRL in linear regression and mixture-of-Gaussians classifications.  This latter scenario is particularly relevant given that the authors restrict their attention for much of the paper to binary classification.  Therefore, I think it may make sense for the authors to revise this claim.\n\n### **Experiments**\n\n**Introducing upper bounds on the 0-1 loss.** I'm not sure I understand the value in introducing the upper bound $\\Psi$ in the paragraph starting on line 145.  The authors do not conduct any experiments (as far as I can tell) using this formulation.  So the question remains: Is there any practical/empirical value in using such a surrogate?\n\n### **Theory**\n\n**What constitutes a theorem?**  It's worth broaching what can at times be a sensitive topic: What constitutes a theorem?  I would argue that Theorem 1 and 3 are closer to being remarks than they are to being theorems, although I could see counterarguments.  The fact is that these results hold essentially by construction, whereas as results such as Theorem 2 are quite non-trivial.\n\n**What is weak-*?**  It would be helpful if the authors could offer some of the preliminaries necessary to understand their results.  For instance, I do not have really any understanding of what weak-* means (even after reading definition 1) and how it differences form other notions of convergence (e.g., in the L^p sense). Moreover, I expect that most of the people reading this paper will be in the same boat.  This not an issue per se, but it's worth considering that you may lose a good chunk of your audience by Section 2.3 due to the inaccessibility of some of this math.  One way to ameliorate this would be to offer an intuitive explanation of what weak-* convergence means and to point to references where one could learn about what weak-* means, or to define weak-* in the appendix somewhere.  If it only appeared in the appendix, I would probably not have noticed it, but since it features relatively prominently in Theorem 4 and Example 1, it's definitely worth discussing and adding some intuition before jumping into examples.  Another way of helping the reader to understand the classes of hypotheses that mean the weak-* closed condition would be to give an example or two that do *not* satisfy this property.\n\n**Why are the properties and asymptotics of PropPer interesting?**  The contribution of the Section 2.4 is unclear to me.  The fact that under certain conditions, $\\text{ProbPer}_\\Psi$ is submodular doesn't seem to have a punchline, i.e., it's unclear why this property is important and/or why one would want this result to hold.  Perhaps the authors could elaborate here on why submodularity is desirable, or what properties it engenders on the task of optimizing ProbPer?\n\nA similar criticism applies to Proposition 1.  The authors say that this result speaks to the fact that for adversarial training and probabilistic robustness, the perimeter dominates the regularization.  Again, I'm left wondering, why is this notable?  Does this lead to a better understanding of how to optimize ProbR(A)?  I can see why one might want it to be the case that PRL converges to the Bayes classifier as $\\epsilon\\to zero$, but again, it's not clear to me why such a result would be \"extremely important.\"  One way to make this more clear would be to elaborate more on the implications listed in the paragraph starting on line 269.\n\n### **Miscellaneous**\n\nHere are a few other points:\n\n* I think that \"Machine Learning\" doesn't need to be capitalized on page 1, nor does \"Probabilistically Robust Learning.\"\n* There is a heavy reliance of $p$ and similar looking characters here.  We have $p$ as the risk probability, $\\mathfrak{p}_x$ as the distribution over perturbed instances, $\\mathbb{P}$ as the probability function, and $\\rho$ as a measure over a particular class.  It would improve readability to use characters that look more different from one another.\n* What is meant by an *admissible* set in line 126?  It doesn't seem relevant to statistical notions of admissibility unless I'm missing something.\n* The comma after neighbors on line 127 should be removed. \n* On line 174, it should be *an* $\\mathcal{A}$-measurable function.\n\n### **Overall evaluation**\n\nOverall, I thought that there were some interesting insights in this paper.  The formulation is relatively clean and there are several interesting new insights about probabilistic robustness.  The writing is generally of high quality and some of the new results are surprising and generalize past results.\n\nHowever, there are also numerous drawbacks.  I can't help but feel that this paper -- as it is currently written -- will have a limited impact.  Many may read it and come to the conclusion that this paper does a lot of interesting math and generalizes notions of probabilistic robustness.  But at the same time, these readers may wonder: Why did we do all of this?  Or, in other words, do these generalizations offer any insights into how one could achieve better probabilistic robustness in practice.  And unfortunately, I don't see how one could answer this question in the affirmative.  The experiments do not demonstrate the benefit of the vanilla geometric or the generalized variants (e.g., soft classifiers, smoothed losses $\\Psi$, etc.) of probabilistic robustness.  This is all to say that I think that this would be a much stronger paper if there was a resounding demonstration that it's actually worth generalizing probabilistic robustness, and/or whether or not the supposed flaw actually holds back the original framework.\n\nThis is related to my feeling that there isn't an inherent flaw in the original framework.  I was not convinced by the arguments given by the authors (as explained above) and I think that it would be worth rethinking the claims to having identified this flaw, especially given that the authors are in a sense focusing on a feature that was never claimed to be a component of probabilistic robustness (based on my limited understanding).  This doesn't necessarily dilute the contribution; it would simply serve to engage with the past work in a different way.  \n\nOverall, I think that this paper is borderline, and as written I would tend to recommend that it not be accepted on the basis that all of the mathematical insights given in the main text do not lead to any notable improvements on the algorithmic end.  If the authors can demonstrate that these insights do lead to better algorithms, and if we can work through some of the topics discussed above, I would consider raising my score.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nI don't see any major limitations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nCongratulations to authors for a timely and interesting paper. The paper advances the work of Robey et al. [2022] that established computational less expensive version of robust learning, dubbed Probabilistic Robust Learning (PRT). In particular, random perturbations and gradient based attacks had been used before and aside, while the work of [Robey 2022] proposed entirely probabilistic RT, which enjoys relatively lower computational costs.\n\nThe paper casts PRT as regularised Empirical Risk Minimisation. It further 1.) identifies a regularizer - 2.) improves it by introducing new objective $ProbR$ and 3.) relax it convex $\\Psi$ instead of indicator function used in ProbR Further, the Section 2.3. 5 4.) Extends it to a soft classifiers (relevant to soft-max classification used in NNs). \nSection 3 deals with general loss functions and upper bounds probabilistically robust objective by $CVaR$ (conditional Value at Risk) that is convex if loss is convex in the first argument (e.g. most of common losses used in NNs) and easier to optimise. Theorem 6 then establishes CVaR as a special case of $ProbR_{\\Psi}$ and its corollary establishes the existence of the solution (novel result).\n\nNumerical experiments demonstrate that hereby proposed version of PRT works on par with original version [Robey 2022] supported by pseudo code algorithm provided in Appendix to help adaptation.\n\nSTRENGTHS:\n+ Clarity: Accessible presentation of the problem, solution and use. Yet high standard of rigour is present in both main pair and appendices.\n+ Theoretical framework with potential: Besides correcting the PRT and establishing existence guarantees (novel results) the paper develops clear mathematical toolkit for solving similar problems in adversarial robustness in rigour.\nOn top the Appendix A.6 PAC learnability for Lipschitz continuous Ψ - suggests future research and improvement of the statements of the paper.\n\n+ Applicability/Impact: Appendix B1 contains the pseudo code for practical usability together with drawing the connections to CVaR as a surrogate objective.\n\nWEAKNESSES:\n- (minor) Numerical experiments presented are minimalistic yet arguably sufficient since theoretical guarantees have been provided. Intriguing results are mentioned but not furhterexplored or commented: “Note that the original or the geometric version of PRL should not be expected to match the adversarial robustness of classifiers trained with PGD attacks [Madry et al., 2017] or other worst-case optimization techniques. Instead, they shine with superior clean accuracies and easier training while maintaining probabilistic and a certain degree of adversarial robustness, as also observed by Robey et al. [2022].”\n\nQUESTIONS:\nThe paper addresses the weak point of the PRT. Never the less, since an attack on such points would only lead to improved model (as also mentioned between lines 76-78), does it need to be addressed? Could you put more comments on why is the 'corrected version' of PRT needed? (Note that this is not questioning the added value of the other parts of the paper such as existence theorem and extensions)\n\nLIMITATIONS:\nLimitations have been commented on in a reasonable extend in Conclusions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe probabilistically robust learning (PRL) framework aims to trade-off clean accuracy and adversarial robustness in model training by introducing a loss that penalizes the suceptibility towards adversarially attacks in the perimeter of the data points. The authors of the paper under review note that the loss of this framework has some (possibly) unintended property: In binary classification, for misclassified data samples an adversarial attacker would attack the sample ‘correctly’, i.e. assign it the ground truth label instead of the wrongly learned one. For the PRL loss such a setting would decrease the loss, incentivising such behaviour. On noting this flaw, the authors propose an new loss which simply applies the PRL loss only to correctly classified points, and the standard ERM loss to the ones that are not (yet) correctly classified. The authors then go on to prove for various settings the existence of solutions to these minimization problems and finally show that empirically, their approach leads to a competitive clean accuracy while increasing the adversarial robustness for small datasets.\n\nSTRENGTHS:\n1. This work notes the question: What should one do with misclassified points in the context of adversarial robustness? Since they do not refer to previous work on this angle to adversarial robustness, I assume that this is a novel question. It seems like a very natural and relevant discussion point.\n2. The existence results are novel and seem to be following a line of work which has previously explored the topic. Adding this new type seems relevant.\n3. The writing is easy to follow and clear, as the authors successfully explain the differences between the different losses.\n\nWEAKNESSES:\n1. It is unclear and not discussed how detrimental considering misclassified examples in the PRL loss is in practice, possibly also depending on the quality and clean accuracy of the learned function. It would be especially interesting to discuss this as the large/small p results for MNIST and CIFAR seem to have opposite trends (improving/decreasing the adv score compared to “original” for large p).\n2.  I believe the empirical results would be stronger, if datasets with a larger number of difficult datapoints were selected. The high accuracy in MNIST leaves only few datapoints to be misclassified and it is known that these may be ambigous even for human interpretation.\n3. Weakness of the reviewer: I find myself unable to access correctness and relevance of the existence results.\n\nQUESTIONS:\n1. I think Fig. 1 could be improved. From looking at it immediately it was unclear to me if A is the set of ground truth green points, or the area in space which predicts a point to be green. I now understand it to be the latter, but I am still unsure whether the margin is actually stemming from a specific probabilistic model or if it is only illustrative (I think the latter again?). Since the penalization around the boundary stems from the training data points themselves, maybe the illustration would be more clear if one uses the [Robey 2022] way of putting the perimeter around the points and marking the penalized area, which was extremely helpful for my intuitive understanding when I checked that paper.  Note that red and green data sample points can be problematic to distinguish for the colorblind. \n2. In line 55 $\\mathcal{A}$ is not defined.\n3. It seems that Eq. (4) is missing context which says that elements in A should have ground truth y=1.\n4. The authors leave out the SVHN dataset from the previous work, which they otherwise compare with. I wonder if there is a specific reason for this.\n\nLIMITATIONS:\nI was mostly interested in the abstract due to the geometrical interpretation of decision boundaries and its relation to robustness. \nSince I am more interested in the application side, for me personally the most intersting points of the paper are the observation that an adversary might correct a model and the definition and practical evaluation of the novel and improved loss.\nI acknowledge that the relevance for the authors might weight more on the proof techniques, but I find myself not being able to assess their value and novelty with full justice.\nSince the empirical evaluation and discussion of the idea of corrective adversaries within its context seem to be lacking, as described in Weaknesses, I assign the paper a score of 4. \nHowever, I am willing to increase my score in line with the other reviewers if they are confident and convinced about the evaluation of the existing results.\n\n*** Update after rebuttal: Edited Soundness score 1->3, updated overall score 4->5", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-27"}

comments: SUMMARY:
This paper takes a fresh and geometric view on the method—Probabilistically Robust Learning (PRL) [Robey et al., 2022], an approach which interpolates between the robustness offered by adversarial training and the higher clean accuracy and faster training times of Empirical Risk Minimization. The paper proposes a geometric framework for understanding PRL, which identifying a subtle flaw in its original formulation and to introduce a family of probabilistic nonlocal perimeter functionals to address this. The paper proves existence of solutions using novel relaxation methods and study properties as well as local limits of the introduced perimeters.

STRENGTHS:
The paper is well written and organized, and is innovative and original. The structure of the paper is clear and rigorous.

WEAKNESSES:
Experiments are not sufficient, only minist and cifar-10 datasets.

QUESTIONS:
1.Does the theoretical results in this paper hold for larger datasets?e.g.ImageNet
2.The proposed method can only satisfied the clean accuracy and adversarial robust trade-off under the PGD adversarial attack, what about other attacks,e.g. C&W attack.
3.In line 290,the results for Geometric on MNIST with p=0.1 in Table 1 is not consistent, for low clean accuracy and high robust accuracy compared with original.
4.small typo errors,line 292

LIMITATIONS:
It's better to evaluate the theoretical conclusions on larger datasets,for example ImageNet.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06