Title: Wasserstein distributional robustness of neural networks
Venue: NeurIPS 2023
Accepted: yes
Keywords: adversarial attack

adversarial robustness of DNN

adversarial training

Wasserstein distance

distributionally robust optimization

sensitivity analysis

asymptotic bounds
Abstract: Deep neural networks are known to be vulnerable to adversarial attacks (AA). For an image recognition task, this means that a small perturbation of the original can result in the image being misclassified. Design of such attacks as well as methods of adversarial training against them are subject of intense research. We re-cast the problem using techniques of Wasserstein distributionally robust optimization (DRO) and obtain novel contributions leveraging recent insights from DRO sensitivity analysis. We consider a set of distributional threat models. Unlike the traditional pointwise attacks, which assume a uniform bound on perturbation of each input data point, distributional threat models allow attackers to perturb inputs in a non-uniform way. We link these more general attacks with questions of out-of-sample performance and Knightian uncertainty. To evaluate the distributional robustness of neural networks, we propose a first-order AA algorithm and its multistep version. Our attack algorithms include Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) as special cases. Furthermore, we provide a new asymptotic estimate of the adversarial accuracy against distributional threat models. The bound is fast to compute and first-order accurate, offering new insights even for the pointwise AA. It also naturally yields out-of-sample performance guarantees. We conduct numerical experiments on CIFAR-10, CIFAR-100, ImageNet datasets using DNNs on RobustBench to illustrate our theoretical results. Our code is available at https://github.com/JanObloj/W-DRO-Adversarial-Methods.

Wasserstein distributional robustness of neural networks

Abstract

Deep neural networks are known to be vulnerable to adversarial attacks (AA). For an image recognition task, this means that a small perturbation of the original can result in the image being misclassified. Design of such attacks as well as methods of adversarial training against them are subject of intense research. We re-cast the problem using techniques of Wasserstein distributionally robust optimization (DRO) and obtain novel contributions leveraging recent insights from DRO sensitivity analysis. We consider a set of distributional threat models. Unlike the traditional pointwise attacks, which assume a uniform bound on perturbation of each input data point, distributional threat models allow attackers to perturb inputs in a nonuniform way. We link these more general attacks with questions of out-of-sample performance and Knightian uncertainty. To evaluate the distributional robustness of neural networks, we propose a first-order AA algorithm and its multistep version. Our attack algorithms include Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) as special cases. Furthermore, we provide a new asymptotic estimate of the adversarial accuracy against distributional threat models. The bound is fast to compute and first-order accurate, offering new insights even for the pointwise AA. It also naturally yields out-of-sample performance guarantees. We conduct numerical experiments on CIFAR-10, CIFAR-100, ImageNet datasets using DNNs on RobustBench to illustrate our theoretical results. Our code is available at https://github.com/JanObloj/W-DRO-Adversarial-Methods .

1 Introduction

Model uncertainty is a ubiquitous phenomenon across different fields of science. In decision theory and economics, it is often referred to as the Knightian uncertainty (Knight, 1921), or the unknown unknowns , to distinguish it from the risk which stems from the randomness embedded by design in the scientific process, see Hansen and Marinacci (2016) for an overview. Transcribing to the context of data science, risk refers to the randomness embedded in a training by design, e.g., through random initialization, drop-outs etc., and uncertainty encompasses the extent to which the dataset is an adequate description of reality. Robustness , the ability to perform well under uncertainty, thus relates to several themes in ML including adversarial attacks, out-of-sample performance and

Guangyi He

Jan Obłój ˚

out-of-distribution performance. In this work, we mainly focus on the former but offer a unified perspective on robustness in all of its facets.

Vulnerability of DNNs to crafted adversarial attacks (AA), diagnosed in Biggio et al. (2013), Goodfellow et al. (2015), relates to the ability of an attacker to manipulate network's outputs by changing the input images only slightly - often in ways imperceptible to a human eye. As such, AA are of key importance for security-sensitive applications and an active field of research. Most works so far have focused on attacks under pointwise l p -bounded image distortions but a growing stream of research, pioneered by Staib and Jegelka (2017) and Sinha et al. (2018), frames the problem using Wasserstein distributionally robust optimization (DRO). We offer novel contributions to this literature.

Our key contributions can be summarized as follows. 1) We propose a unified approach to adversarial attacks and training based on sensitivity analysis for Wasserstein DRO. We believe this approach, leveraging results from Bartl et al. (2021), is better suited for gradient-based optimization methods than duality approach adopted in most of the works to date. We further link the adversarial accuracy to the adversarial loss, and investigate the out-of-sample performance. 2) We derive a general adversarial attack method. As a special case, this recovers the classical FGSM attack lending it a further theoretical underpinning. However, our method also allows one to carry out attacks under a distributional threat model which, we believe, has not been done before. We also propose a rectified DLR loss suitable for the distributional attacks. 3) We develop asymptotically certified bounds on adversarial accuracy, applicable to a general threat, including the classical pointwise perturbations. The bounds are first-order accurate and much faster to compute than, e.g., the AutoAttack (Croce and Hein, 2020) benchmark. The performance of our methods is documented using CIFAR-10 (Krizhevsky, 2009), CIFAR-100 (Krizhevsky, 2009), ImageNet (Deng et al., 2009) datasets and neural networks from RobustBench (Croce et al., 2021).

2 Related Work

Adversarial Attack (AA). Original research focused on the pointwise l p -bounded image distortion. Numerous attack methods under this threat model have been proposed in the literature, including Fast Gradient Sign Method (FGSM) (Goodfellow et al., 2015), Projected Gradient Descent (PGD) (Madry et al., 2018), CW attack (Carlini and Wagner, 2017), etc. In these white-box attacks, the attacker has full knowledge of the neural network. There are also black-box attacks, such as Zeroth Order Optimization (ZOO) (Chen et al., 2017), Boundary Attack (Brendel et al., 2018), and Query-limited Attack (Ilyas et al., 2018). AutoAttack (Croce and Hein, 2020), an ensemble of white-box and blackbox attacks, provides a useful benchmark for pointwise l p -robustness of neural networks. Notably Hua et al. (2022) considered AA with l p distance replaced by a proxy for human eye evaluation.

Adversarial Defense. Early works on data augmentation (Goodfellow et al., 2015, Madry et al., 2018, Tramèr et al., 2018) make use of strong adversarial attacks to augment the training data with adversarial examples; more recent works (Gowal et al., 2021, Xing et al., 2022, Wang et al., 2023) focus on adding randomness to training data through generative models such as GANs and diffusion models. Zhang et al. (2019) consider the trade-off between robustness and accuracy of a neural network via TRADES, a regularized loss. Analogous research includes MART (Wang et al., 2020) and SCORE (Pang et al., 2022). Other loss regularization methods such as adversarial distributional training (Dong et al., 2020) and adversarial weight perturbation (Wu et al., 2020) have been shown to smooth the loss landscape and improve the robustness. In addition, various training techniques can be overlaid to improve robustness, including droup-out layers, early stopping and parameter fine-tuning Sehwag et al. (2020). The closest to our setting are Sinha et al. (2018), García Trillos and García Trillos (2022) which employ Wasserstein penalization and constraint respectively. However, so far, even the papers which used distributional threat models to motivate DRO-based training methods actually used classical pointwise attacks to evaluate robustness of their trained DNNs. This highlights the novelty of our distributional threat attack .

Robust Performance Bounds. Each AA method gives a particular upper bound on the adversarial accuracy of the network. In contrast, research on certified robustness aims at classifying images which are robust to all possible attacks allowed in the threat model and thus providing an attack-agnostic lower bound on the classification accuracy. To verify robustness of images, deterministic methods using off-the-shelf solvers (Tjeng et al., 2019), relaxed linear programming (Wong and Kolter, 2018,

Weng et al., 2018a) or semi-definite programming (Raghunathan et al., 2018, Dathathri et al., 2020) have been applied. Hein and Andriushchenko (2017), Weng et al. (2018b) derive Lipschitz-based metrics to characterize the maximum distortion an image can uphold; Cohen et al. (2019) constructs a certifiable classifier by adding smooth noise to the original classifier; see Li et al. (2023) for a review.

Distributionally Robust Optimization (DRO). Mathematically, it is formulated as a min-max problem

where we minimize the worst-case loss over all possible distributions Q P P . In financial economics, such criteria appear in the context of multi-prior preferences, see (Gilboa and Schmeidler, 1989, Föllmer and Weber, 2015). We refer to (Rahimian and Mehrotra, 2019) for a survey of the DRO.

We focus on the Wasserstein ambiguity set P ' B δ p P q , which is a ball centered at the reference distribution P with radius δ under the Wasserstein distance. We refer to Gao and Kleywegt (2022) for a discussion of many advantages of this distance. In particular, measures close to each other can have different supports which is key in capturing data perturbations, see Sinha et al. (2018). Staib and Jegelka (2017) interpreted pointwise adversarial training as a special case of Wasserstein DRO (W-DRO). Volpi et al. (2018) utilized W-DRO to improve network performance on unseen data distributions. More recently, Bui et al. (2022) unified various classical adversarial training methods, such as PGD-AT, TRADES, and MART, under the W-DRO framework.

W-DRO, while compelling theoretically, is often numerically intractable. In the literature, two lines of research have been proposed to tackle this problem. The duality approach rewrites (1), changing the sup to a univariate inf featuring a transform of f θ . We refer to Mohajerin Esfahani and Kuhn (2018) for the data-driven case, Blanchet and Murthy (2019), Bartl et al. (2020), Gao and Kleywegt (2022) for general probability measures and Huang et al. (2022) for a further application with coresets. The second approach, which we adopt here, considers the first order approximation to the original DRO problem. This can be seen as computing the sensitivity of the value function with respect to the model uncertainty as derived in Bartl et al. (2021), see also Lam (2016), García Trillos and García Trillos (2022) for analogous results in different setups.

3 Preliminaries

Image Classification Task. An image is interpreted as a tuple p x, y q where the feature vector x P X encodes the graphic information and y P Y ' t 1 , . . . , m u denotes the class, or tag, of the image. W.l.o.g., we take X ' r 0 , 1 s n . A distribution of labelled images corresponds to a probability measure P on X ˆ Y . We are given the training set D tr and the test set D tt , subsets of X ˆ Y , i.i.d. sampled from P . We denote p P (resp. q P ) the empirical measure of points in the training set (resp. test set), i.e., p P ' 1 | D tr | ř p x,y qP D tr δ p x,y q . A neural network is a map f θ : X Ñ R m

σ is a nonlinear activation function, and θ ' t w i , b i : 1 ď i ď l u is the collection of parameters. We denote S the set of images equipped with their labels generated by f θ , i.e.,

The aim of image classification is to find a network f θ with high (clean) prediction accuracy A : ' P p S q ' E P r ✶ S s . To this end, f θ is trained solving 2 the stochastic optimization problem inf θ P Θ E P r L p f θ p x q , y qs , (2)

where Θ denotes the set of admissible parameters, and L is a (piecewise) smooth loss function, e.g., cross entropy loss 3 CE : R m ˆ Y Ñ R given by

Wasserstein Distances. Throughout, p p, q q is a pair of conjugate indices, 1 { p ` 1 { q ' 1 , with 1 ď p ď 8 . We consider a norm } ¨ } on X and denote } ¨ } ˚ its dual, } ˜ x } ˚ ' sup tx x, ˜ x y : } x } ď 1 u . Our main interest is in } ¨ } ' } ¨ } r the l r -norm for which } ¨ } ˚ ' } ¨ } s , where p r, s q are conjugate indices, 1 ď r ď 8 . We consider adversarial attacks which perturb the image feature x but not its label y . Accordingly, we define a pseudo distance 4 d on X ˆ Y as d pp x 1 , y 1 q , p x 2 , y 2 qq ' } x 1 ´ x 2 } ` 8 ✶ t y 1 ‰ y 2 u . (4) We denote Π p P, Q q the set of couplings between p x, y q and p x 1 , y 1 q whose first margin is P and second margin is Q , and T # P : ' P ˝ T ´ 1 denotes the pushforward measure of P under a map T .

The p -Wasserstein distance, 1 ď p ă 8 , between probability measures P and Q on X ˆ Y is

The 8 -Wasserstein distance W 8 is given by

We denote the p -Wasserstein ball centered at P with radius δ by B δ p P q . We mainly consider the cases where p, r P t 2 , 8u . Intuitively, we can view p as the index of image-wise flexibility and r as the index of pixel-wise flexibility. Unless p ' 1 is explicitly allowed, p ą 1 in what follows.

4 Wasserstein Distributional Robustness: adversarial attacks and training

W-DRO Formulation. The Wasserstein DRO (W-DRO) formulation of a DNN training task is given by:

where B δ p P q is the p -Wasserstein ball centered at P and δ denotes the budget of the adversarial attack. In practice, P is not accessible and is replaced with p P . When p ' 8 , the above adversarial loss coincides with the pointwise adversarial loss of Madry et al. (2018) given by

Recently, Bui et al. (2022) considered a more general criterion they called unified distributional robustness . It can be re-cast equivalently as an extended W-DRO formulation using couplings:

where Π δ p P, ¨q is the set of couplings between p x, y q and p x 1 , y 1 q whose first margin is P and the second margin is within a Wasserstein δ -ball centered at P . This formulation was motivated by the observation that for p ' 8 , taking J θ p x, y, x 1 , y 1 q ' L p f θ p x q , y q ` βL p f θ p x q , f θ p x 1 qq , it retrieves the TRADES loss of (Zhang et al., 2019) given by

W-DRO Sensitivity. In practice, training using (7), let alone (8), is computationally infeasible. To back propagate θ it is essential to understand the inner maximization problem denoted by

where we write J θ p x, y q ' L p f θ p x q , y q . One can view the adversarial loss V p δ q as a certain regularization of the vanilla loss. Though we are not able to compute the exact value of V p δ q for neural networks with sufficient expressivity, DRO sensitivity analysis results allow us to derive a numerical approximation to V p δ q and further apply gradient-based optimization methods. This is the main novelty of our approach - previous works considering a W-DRO formulation mostly relied on duality results in the spirit of Blanchet and Murthy (2019) to rewrite (7).

Assumption 4.1. We assume the map p x, y q ÞÑ J θ p x, y q is L -Lipschitz under d , i.e.,

The above assumption is weaker than the L -smoothness assumption encountered in the literature which requires Lipschitz continuity of gradients of J θ , see for example Sinha et al. (2018)[Assumption B] and Volpi et al. (2018)[Assumptions 1 & 2]. We also remark that our assumption holds for any continuously differentiable J θ as the feature space X ' r 0 , 1 s n is compact.

The following result follows readily from (Bartl et al., 2021, Theorem 2.2) and its proof.

Theorem 4.1. Under Assumption 4.1, the following first order approximations hold:

and h is uniquely determined by x h p x q , x y ' } x } ˚ .

The above holds for any probability measure, in particular with P replaced consistently by an empirical measure p P or q P . In Figure 1, we illustrate the performance of our first order approximation of the adversarial loss on CIFAR-10 (Krizhevsky, 2009) under different threat models.

0

.

00

0

.

01

0

.

02

0

.

03

0

.

04

0

.

7

0

.

8

0

.

9

1

.

0

1

.

1

1

.

2

(

W

∞

, l

∞

) Threat Model

V

(

δ

)

V

(0) +

δ

Υ

0

.

0

0

.

1

0

.

2

0

.

3

0

.

4

0

.

5

0

.

6

-

0

.

7

-

0

.

6

-

0

.

5

-

0

.

4

-

0

.

3

(

W

2

, l

2

) Threat Model

V

(

δ

)

V

(0) +

δ

Υ

WD-Adversarial Accuracy. We consider an attacker with perfect knowledge of the network f θ and the data distribution P , aiming to minimize the prediction accuracy of f θ under an admissible attack. Complementing the W-DRO training formulation, Staib and Jegelka (2017), Sinha et al. (2018) proposed Wasserstein distributional threat models under which an attack is admissible if the resulting attacked distribution Q stays in the p -Wasserstein ball B δ p P q , where δ is the attack budget, i.e., the tolerance for distributional image distortion. We define the adversarial accuracy as:

Note that A δ is decreasing in δ with A 0 ' A , the clean accuracy. For p ' 8 , the Wasserstein distance essentially degenerates to the uniform distance between images and hence the proposed threat model coincides with the popular pointwise threat model. For 1 ď p ă 8 , the distributional threat model is strictly stronger than the pointwise one, as observed in Staib and Jegelka (2017, Prop. 3.1). Intuitively, it is because the attacker has a greater flexibility and can perturb images close to the decision boundary only slightly while spending more of the attack budget on images farther away from the boundary. The threat is also closely related to out-of-distribution generalization, see Shen et al. (2021) for a survey.

WD-Adversarial Attack. We propose Wasserstein distributionally adversarial attack methods. As mentioned above, to date, even the papers which used distributional threat models to motivate DRO-based training methods then used classical pointwise attacks to evaluate robustness of their trained DNNs. Our contribution is novel and enabled by the explicit first-order expression for the distributional attack in Theorem 4.1(ii), which is not accessible using duality methods.

We recall the Difference of Logits Ratio (DLR) loss of Croce and Hein (2020). If we write z ' p z 1 , . . . , z m q ' f θ p x q for the output of a neural network, and z p 1 q ě ¨ ¨ ¨ ě z p m q are the order statistics of z , then DLR loss is given by

The combination of CE loss and DLR loss has been widely shown as an effective empirical attack for pointwise threat models. However, under distributional threat models, intuitively, an effective attack should perturb more aggressively images classified far from the decision boundary and leave the misclassified images unchanged. Consequently, neither CE loss nor DLR loss are appropriate this intuition is confirmed in our numerical experiments, see Table 1 for details. To rectify this, we propose ReDLR (Rectified DLR) loss:

Its key property is to leave unaffected those images that are already misclassified. Our experiments show it performs superior to CE or DLR.

An attack is performed using the test data set. For a given loss function, our proposed attack is:

where α is the step size and proj δ is a projection which ensures the empirical measure q P t ` 1 : ' 1 | D tt | ř p x,y qP D tt δ p x t ` 1 ,y q stays inside the Wasserstein ball B δ p q P q . In the case p ' r ' 8 , one can verify h p x q ' sgn p x q and write (11) as

This gives exactly Fast Gradient Sign Method (single step) and Projected Gradient Descent (multistep) proposed in Goodfellow et al. (2015), Madry et al. (2018) and we adopt the same labels for our more general algorithms. 5 A pseudocode for the above attack is summarized in Appendix C.

Finally, note that Theorem 4.1 offers computationally tractable approximations to the W-DRO adversarial training objectives (7) and (8). In Appendix D we propose two possible training methods but do not evaluate their performance and otherwise leave this topic to future research.

5 Performance Bounds

Understanding how a DNN classifier will perform outside the training data set is of key importance. We leverage the DRO sensitivity results now to obtain a lower bound on A δ . We then use results on convergence of empirical measures in Fournier and Guillin (2015) to translate our lower bound into guarantees on out-of-sample performance.

Bounds on Adversarial Accuracy. We propose the following metric of robustness:

Previous works mostly focus on the maximum distortion a neural network can withhold to retain certain adversarial performance, see Hein and Andriushchenko (2017), Weng et al. (2018b) for

local robustness and Bastani et al. (2016) for global robustness. However, there is no immediate connection between such a maximum distortion and the adversarial accuracy, especially in face of a distributionally adversarial attack. In contrast, since A ' A 0 is known, computing R δ is equivalent to computing A δ . We choose to focus on the relative loss of accuracy as it provides a convenient normalization: 0 ď R δ ď 1 . R δ ' 1 corresponds to a very robust architecture which performs as well under attacks as it does on clean test data, while R δ ' 0 corresponds to an architecture which loses all of its predictive power under an adversarial attack. Together the couple p A,A δ q thus summarizes the performance of a given classifier. However, computing A δ is difficult and time-consuming. Below, we develop a simple and efficient method to calculate theoretical guaranteed bounds on R and thus also on A δ .

Assumption 5.1. We assume that for any Q P B δ p P q

(i) 0 ă Q p S q ă 1 .

(ii) W p p Q p¨| S q , P p¨| S qq ` W p p Q p¨| S c q , P p¨| S c qq ' o p δ q , where S c ' p X ˆ Y qz S and the conditional distribution is given by Q p E | S q ' Q p E X S q{ Q p S q .

The first condition stipulates non-degeneracy: the classifier does not perform perfectly but retains some accuracy under attacks. The second condition says the classes are well-separated: for δ small enough an admissible attack can rarely succeed.

We write the adversarial loss condition on the correctly classified images and misclassified images as

We note that an upper bound on R δ is given by any adversarial attack. In particular,

Theorem 5.1. Under Assumptions 4.1 and 5.1, we have an asymptotic lower bound as δ Ñ 0

where the first order approximations are given by

The equality between the lower bound and the two first-order approximations r R l δ and R l δ follows from Theorem 4.1. Consequently, R l δ : ' min t r R l δ , R l δ u allows us to estimate the model robustness without performing any sophisticated adversarial attack. Our experiments, detailed below, show the bound is reliable for small δ and is orders of magnitude faster to compute than R δ even in the classical case of pointwise attacks. The proof is reported in Appendix A. Its key ingredient is the following tower-like property.

Proposition 5.2. Under Assumptions 4.1 and 5.1, we have

Bounds on Out-of-Sample Performance. Our results on distributionally adversarial robustness translate into bounds for performance of the trained DNN on unseen data. We rely on the results of Fournier and Guillin (2015) and refer to Lee and Raginsky (2018) for analogous applications to finite sample guarantees and to Gao (2022) for further results and discussion.

We fix 1 ă p ă n { 2 and let N ' | D tr | , M ' | D tt | . If sampling of data from P is described on a probability space p Ω , F , P q then p P is a random measure on this space and, by ergodic theorem, P -a.s., it converges weakly to P as N Ñ8 . In fact, W p p p P,P q converges to zero P -a.s. Crucially the rates of convergence were obtained in Dereich et al. (2013), Fournier and Guillin (2015) and yield

where K is a constant depending on p and n which can be computed explicitly, see for example Guo and Obłój (2019, Appendix). This, with triangle inequality and Theorem 5.1, gives

Corollary 5.3. Under Assumptions 4.1 and 5.1 on measure p P , with probability at least 1 ´ 2 K exp p´ Kε n min t M,N uq it holds that

Next results provide a finer statistical guarantee on the out-of-sample performance for robust (W-DRO) training. Its proof is reported in Appendix B.

Theorem 5.4. Under Assumption 4.1, with probability at least 1 ´ K exp p´ KNε n q we have

where B ‹ δ p p P q ' arg max Q P B δ p p P q E Q r J θ p x, y qs and constant K only depends on p and n .

Our lower bound estimate in Theorem 5.1 can be restated as

We now use Theorem 5.4 to bound ∆ A p δ q , the shortfall of the adversarial accuracy under P , using quantities evaluated under p P .

Corollary 5.5. Under Assumptions 4.1 and 5.1, with probability at least 1 ´ K exp p´ KNδ n q it holds that

We remark that the above results are easily extended to the out-of-sample performance on the test set, via the triangle inequality W p p p P, q P q ď W p p p P,P q ` W p p P, q P q . By using complexity measures such as entropy integral (Lee and Raginsky, 2018), Rademacher complexity (Gao, 2022, Gao et al., 2022) a further analysis can be undertaken for

In particular, a dimension-free estimate of out-of-sample performance is obtained in (Gao, 2022) under a Lipschitz framework with light-tail reference measures.

6 Numerical Experiments

Experimental Setting. We conduct experiments on a high performance computing server equipped with 49 GPU nodes. The algorithms are implemented in Python. Experiments are conducted on CIFAR-10, CIFAR-100, and ImageNet datasets. Numerical results are consistent across different datasets, and we present the results on CIFAR-10 in body paragraph only for the sake of brevity. Results on CIFAR-100 and ImageNet are reported in Appendix F.

CIFAR-10 (Krizhevsky, 2009) comprises 60,000 color images across 10 mutually exclusive classes, with 6,000 images per class. Each image contains 32 ˆ 32 pixels in 3 color channels. We normalize the input feature as a vector x P r 0 , 1 s 3 ˆ 32 ˆ 32 . The dataset is further divided into training and test sets, containing 50,000 and 10,000 images respectively. We evaluate the robustness of neural networks on the test set only.

We consider four threat models p W p , l r q with p, r P t 2 , 8u with different range of attack budget δ depending on the relative strength of the attack. E.g., roughly speaking, if an l 8 -attack modifies one third of the pixels of an image with strength 4/255, then it corresponds to an l 2 -attack with strength 1/2. When clear from the context, we drop the δ subscript.

We take top neural networks from RobustBench (Croce et al., 2021), a lively maintained repository that records benchmark robust neural networks on CIFAR-10 against pointwise attacks. For pointwise threat models p W 8 , l r q , RobustBench reports A δ obtained using AutoAttack (Croce and Hein, 2020) for l 8 , δ ' 8 { 255 and l 2 , δ ' 1 { 2 , see Appendix H. However, due to high computational

cost of AutoAttack, we apply PGD-50 based on CE and DLR losses as a substitute to obtain the reference adversarial accuracy for attacks with relatively small budgets δ ' 2 { 255 , 4 { 255 for l 8 and δ ' 1 { 8 , 1 { 4 for l 2 . For distributional threat models p W 2 , l r q , there is no existing benchmark attacking method. Therefore, W-PGD attack (11) based on ReDLR loss is implemented to obtain the reference adversarial accuracy A δ . All PGD attacks are run with 50 iteration steps and take between 1 and 12 hours to run on a single GPU environment. Bounds R l , R u compute ca. 50 times faster.

Distributionally Adversarial Attack. We report in Table 1 the average accuracy of top neural networks on RobustBench against pointwise and distributional attacks under different loss functions. The predicted drop in accuracy between a pointwise, i.e., 8 -W-DRO attack and a distributional 2-W-DRO attack is only realized using the ReDLR loss.

In Figure 2, we compare the adversarial accuracy of robust networks on RobustBench against pointwise threat models and distributional threat models. We notice a significant drop of the adversarial accuracy even for those neural networks robust against pointwise threat models.

30%

40%

50%

60%

70%

80%

90%

100%

, l

)

30%

40%

50%

60%

70%

80%

90%

100%

W

2

, l

∞

)

δ

= 2

/

255

δ

= 4

/

255

δ

= 8

/

255

y

=

x

50%

60%

70%

80%

90%

100%

, l

2

)

50%

60%

70%

80%

90%

100%

W

2

, l

2

)

δ

= 1

/

8

δ

= 1

/

4

δ

= 1

/

2

y

=

x

W

∞

∞

W

∞

Bounds on Adversarial Accuracy. We report in Table 2 the computation time of our proposed bounds R l δ ' min t r R l δ , R l δ u in (14) and R u δ in (12) with the computation time of R δ obtained from AutoAttack. Computing our proposed bounds R l , R u is orders of magnitude faster than performing an attack to estimate R . This also holds for distributional threat attacks.

To illustrate the applications of Theorem 5.1, we plot the bounds R l and R u against R for neural networks on RobustBench. The results are plotted in Figure 3 and showcase the applicability of our bounds across different architectures. 6 Note that smaller δ values are suitable for the stronger W 2 -distributional attack. For pointwise threat models (top row) we compute the bounds using CE loss. For distributional threat models (bottom row), reference adversarial accuracy is obtained from a

W-PGD-ReDLR attack and, accordingly, we use ReDLR loss to compute R u and R l . In this case, the width of the gap between our upper and lower bounds varies significantly for different DNNs. To improve the bounds, instead of R l , we could estimate V p δ q and use the lower bound in (13). This offers a trade-off between computational time and accuracy which is explored further in Appendix E.

0

.

88

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

0

.

88

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

(

W

∞

, l

∞

) Threat Model with

δ

= 2

/

255

R

u

R

l

R

0

.

750

0

.

775

0

.

800

0

.

825

0

.

850

0

.

875

0

.

900

0

.

925

0

.

950

0

.

750

0

.

775

0

.

800

0

.

825

0

.

850

0

.

875

0

.

900

0

.

925

0

.

950

(

W

∞

, l

∞

) Threat Model with

δ

= 4

/

255

R

u

R

l

R

0

.

93

0

.

94

0

.

95

0

.

96

0

.

97

0

.

98

0

.

99

1

.

00

0

.

93

0

.

94

0

.

95

0

.

96

0

.

97

0

.

98

0

.

99

1

.

00

(

W

∞

, l

2

) Threat Model with

δ

= 1

/

8

R

u

R

l

R

0

.

86

0

.

88

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

0

.

86

0

.

88

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

(

W

∞

, l

2

) Threat Model with

δ

= 1

/

4

R

u

R

l

R

0

.

75

0

.

80

0

.

85

0

.

90

0

.

95

1

.

00

(

W

2

, l

∞

) Threat Model with

δ

= 1

/

510

R

u

R

l

R

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

(

W

2

, l

∞

) Threat Model with

δ

= 1

/

255

R

u

R

l

R

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

(

W

2

, l

2

) Threat Model with

δ

= 1

/

32

R

u

R

l

R

0

.

800

0

.

825

0

.

850

0

.

875

0

.

900

0

.

925

0

.

950

0

.

975

1

.

000

(

W

2

, l

2

) Threat Model with

δ

= 1

/

16

R

u

R

l

R

0

.

75

0

.

80

0

.

85

0

.

90

0

.

95

1

.

00

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

0

.

800

0

.

825

0

.

850

0

.

875

0

.

900

0

.

925

0

.

950

0

.

975

1

.

000

7 Limitations and Future Work

Limitations. Our theoretical results are asymptotic and their validity is confined to the linear approximation regime. We believe that the empirical results we presented from all the leaderboard models on RobustBench across different attack types provide an overwhelming evidence that our results are valid and relevant for the range of attack budget δ considered in AA settings. However, as δ increases we are likely to go outside of the linear approximation regime, see Figure 1. In Appendix E we plot the results for pointwise attack with δ ' 8 { 255 where some of neural networks have a lower bound R l greater than the reference R . We do not have theoretical results to provide guarantees on the range of applicability but, in Appendix E, discuss a possible rule of thumb solution.

Future Work. We believe our research opens up many avenues for future work. These include: developing stronger attacks under distributional threat models, testing the performance of the two training algorithms derived here and investigating further sensitivity-based ones, as well as analyzing the relation between the values and optimizers in (16), verifying empirical performance of our out-ofsample results, including Corollary 5.5, and extending these to out-of-distribution performance.

Broader Impact. Our work contributes to the understanding of robustness of DNN classifiers. We believe it can help users in designing and testing DNN architectures. It also offers a wider viewpoint on the question of robustness and naturally links the questions of adversarial attacks, out-of-sample performance, out-of-distribution performance and Knightian uncertainty. We provide computationally efficient tools to evaluate robustness of DNNs. However, our results are asymptotic and hence valid for small attacks and we acknowledge the risk that some users may try to apply the methods outside of their applicable regimes. Finally, in principle, our work could also enhance understanding of malicious agents aiming to identify and attack vulnerable DNN-based classifiers.

References

Bartl, S. Drapeau, and L. Tangpi. Computational aspects of robust optimized certainty equivalents and option pricing. Mathematical Finance , 30(1):287-309, 2020.

Bartl, S. Drapeau, J. Obłój, and J. Wiesel. Sensitivity analysis of Wasserstein distributionally robust optimization problems. Proc. R. Soc. A. , 477(2256):20210176, Dec. 2021.

Bastani, Y. Ioannou, L. Lampropoulos, D. Vytiniotis, A. Nori, and A. Criminisi. Measuring neural net robustness with constraint. In Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016.

Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndi´ c, P. Laskov, G. Giacinto, and F. Roli. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases , Lecture Notes in Computer Science, pages 387-402, Berlin, Heidelberg, 2013. Springer.

Blanchet and K. Murthy. Quantifying distributional model risk via optimal transport. Mathematics of Operations Research , 44(2):565-600, May 2019.

Brendel, J. Rauber, and M. Bethge. Decision-based adversarial attacks: reliable attacks against black-box machine learning models. In International Conference on Learning Representations , 2018.

T. Bui, T. Le, Q. H. Tran, H. Zhao, and D. Phung. A unified Wasserstein distributional robustness framework for adversarial training. In International Conference on Learning Representations , Jan. 2022.

Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP) , pages 39-57. IEEE, May 2017.

P.-Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C.-J. Hsieh. ZOO: zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security , pages 15-26. ACM, Nov. 2017.

Cohen, E. Rosenfeld, and Z. Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th International Conference on Machine Learning , pages 1310-1320. PMLR, May 2019.

Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of the 37th International Conference on Machine Learning , pages 2206-2216. PMLR, Nov. 2020.

Croce, M. Andriushchenko, V. Sehwag, E. Debenedetti, N. Flammarion, M. Chiang, P. Mittal, and M. Hein. RobustBench: A standardized adversarial robustness benchmark, Oct. 2021.

Dathathri, K. Dvijotham, A. Kurakin, A. Raghunathan, J. Uesato, R. R. Bunel, S. Shankar, J. Steinhardt, I. Goodfellow, P. S. Liang, and P. Kohli. Enabling certification of verificationagnostic networks via memory-efficient semidefinite programming. In Advances in Neural Information Processing Systems , volume 33, pages 5318-5331. Curran Associates, Inc., 2020.

Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09 , 2009.

Dereich, M. Scheutzow, and R. Schottstedt. Constructive quantization: approximation by empirical measures. In Annales de l'IHP Probabilités et statistiques , volume 49, pages 1183-1203, 2013.

Dong, Z. Deng, T. Pang, J. Zhu, and H. Su. Adversarial distributional training for robust deep learning. In Advances in Neural Information Processing Systems , volume 33, pages 8270-8283. Curran Associates, Inc., 2020.

Föllmer and S. Weber. The axiomatic approach to risk measures for capital determination. Annual Review of Financial Economics , 7(1):301-337, 2015.

Fournier and A. Guillin. On the rate of convergence in wasserstein distance of the empirical measure. Probability Theory and Related Fields , 162(3-4):707-738, 2015.

Gao. Finite-sample guarantees for wasserstein distributionally robust optimization: breaking the curse of dimensionality. Operations Research , 2022.

Gao and A. Kleywegt. Distributionally robust stochastic optimization with Wasserstein distance. Mathematics of OR , Aug. 2022.

Gao, X. Chen, and A. J. Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research , 2022.

A. García Trillos and N. García Trillos. On the regularized risk of distributionally robust learning over deep neural networks. Res Math Sci , 9(3):54, Aug. 2022.

Gilboa and D. Schmeidler. Maxmin expected utility with non-unique prior. Journal of Mathematical Economics , 18(2):141-153, 1989.

Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations , 2015.

Gowal, C. Qin, J. Uesato, T. Mann, and P. Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593 , 2020.

Gowal, S.-A. Rebuffi, O. Wiles, F. Stimberg, D. A. Calian, and T. A. Mann. Improving robustness using generated data. In Advances in Neural Information Processing Systems , volume 34, pages 4218-4233. Curran Associates, Inc., 2021.

Guo and J. Obłój. Computational methods for martingale optimal transport problems. The Annals of Applied Probability , 29(6):3311 - 3347, 2019.

P. Hansen and M. Marinacci. Ambiguity aversion and model misspecification: an economic perspective. Statistical Science , 31(4):511-515, 2016.

Hein and M. Andriushchenko. Formal guarantees on the robustness of a classifier against adversarial manipulation. In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017.

Hua, H. Xu, J. Blanchet, and V. A. Nguyen. Human imperceptible attacks and applications to improve fairness. In 2022 Winter Simulation Conference (WSC) , pages 2641-2652, 2022.

Huang, J. Huang, W. Liu, and H. Ding. Coresets for Wasserstein distributionally robust optimization problems. In Advances in Neural Information Processing Systems , Oct. 2022.

Ilyas, L. Engstrom, A. Athalye, and J. Lin. Black-box adversarial attacks with limited queries and information. In Proceedings of the 35th International Conference on Machine Learning , pages 2137-2146. PMLR, July 2018.

H. Knight. Risk, Uncertainty and Profit . Boston, New York, Houghton Mifflin Company, 1921.

Krizhevsky. Learning multiple layers of features from tiny images. 2009.

Lam. Robust sensitivity analysis for stochastic systems. Mathematics of OR , 41(4):1248-1275, Nov. 2016.

Larsson, J. Park, and J. Wiesel. On concentration of the empirical measure for general transport costs, 2023.

Lee and M. Raginsky. Minimax statistical learning with Wasserstein distances. Advances in Neural Information Processing Systems , 31, 2018.

Li, T. Xie, and B. Li. SoK: Certified robustness for deep neural networks. In 44th IEEE Symposium on Security and Privacy . IEEE, 2023.

Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

Mohajerin Esfahani and D. Kuhn. Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Math. Program. , 171(1): 115-166, Sept. 2018.

L. M. Olea, C. Rush, A. Velez, and J. Wiesel. The out-of-sample prediction error of the squareroot-lasso and related estimators, 2023.

Pang, M. Lin, X. Yang, J. Zhu, and S. Yan. Robustness and accuracy could be reconcilable by (proper) definition. In Proceedings of the 39th International Conference on Machine Learning , pages 17258-17277. PMLR, June 2022.

Raghunathan, J. Steinhardt, and P. S. Liang. Semidefinite relaxations for certifying robustness to adversarial examples. In Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Rahimian and S. Mehrotra. Distributionally robust optimization: A review, Aug. 2019.

Sehwag, S. Wang, P. Mittal, and S. Jana. HYDRA: pruning adversarially robust neural networks. In Advances in Neural Information Processing Systems , volume 33, pages 19655-19666. Curran Associates, Inc., 2020.

Shen, J. Liu, Y . He, X. Zhang, R. Xu, H. Yu, and P. Cui. Towards out-of-distribution generalization: a survey, Aug. 2021.

Sinha, H. Namkoong, and J. Duchi. Certifying some distributional robustness with principled adversarial training. In International Conference on Learning Representations , 2018.

Staib and S. Jegelka. Distributionally robust deep learning as a generalization of adversarial training. In NIPS workshop on Machine Learning and Computer Security , volume 3, page 4, 2017.

Tjeng, K. Y. Xiao, and R. Tedrake. Evaluating robustness of neural networks with mixed integer programming. In International Conference on Learning Representations , 2019.

Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel. Ensemble adversarial training: attacks and defenses. In International Conference on Learning Representations , 2018.

Volpi, H. Namkoong, O. Sener, J. C. Duchi, V. Murino, and S. Savarese. Generalizing to unseen domains via adversarial data augmentation. Advances in Neural Information Processing Systems , 31, 2018.

Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu. Improving adversarial robustness requires revisiting misclassified examples. In International Conference on Learning Representations , 2020.

Wang, T. Pang, C. Du, M. Lin, W. Liu, and S. Yan. Better diffusion models further improve adversarial training. In Proceedings of the 40th International Conference on Machine Learning , 2023.

Weng, H. Zhang, H. Chen, Z. Song, C.-J. Hsieh, L. Daniel, D. Boning, and I. Dhillon. Towards fast computation of certified robustness for ReLU networks. In Proceedings of the 35th International Conference on Machine Learning , pages 5276-5285. PMLR, July 2018a.

W. Weng, H. Zhang, P.-Y. Chen, J. Yi, D. Su, Y. Gao, C.-J. Hsieh, and L. Daniel. Evaluating the robustness of neural networks: an extreme value theory approach. In International Conference on Learning Representations . International Conference on Learning Representations, ICLR, 2018b.

Wong and Z. Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. In Proceedings of the 35th International Conference on Machine Learning , pages 5286-5295. PMLR, July 2018.

Wu, S.-T. Xia, and Y. Wang. Adversarial weight perturbation helps robust generalization. In Advances in Neural Information Processing Systems , volume 33, pages 2958-2969. Curran Associates, Inc., 2020.

Xing, Q. Song, and G. Cheng. Why do artificially generated data help adversarial robustness. In Advances in Neural Information Processing Systems , Oct. 2022.

Zhang, Y. Yu, J. Jiao, E. Xing, L. E. Ghaoui, and M. Jordan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference on Machine Learning , pages 7472-7482. PMLR, May 2019.

A Bounds on Adversarial Accuracy

Recall that Proposition 5.2 states the following tower-like property where

Proof of Proposition 5.2. One direction follows directly from the usual tower property of conditional expectation:

For the other direction, note that

are well-defined for any Borel E by Assumption 5.1. Take an arbitrary ε ą 0 and Q c , Q w P B δ p P q such that

We further take Q ‹ P B δ p P q such that

and write distribution r Q given by

These give us

Recall Assumption 5.1 (ii) gives for any Q P B δ p P q

Now we take π c P Π p Q ‹ p¨| S q , Q c p¨| S qq such that E π c r d p X,Y q p s ' W p p Q ‹ p¨| S q , Q c p¨| S qq p , and similarly π w P Π p Q ‹ p¨| S c q , Q w p¨| S c qq . Then by definition of r Q , we have π ' Q ‹ p S q π c ` Q ‹ p S c q π w P Π p Q ‹ , r Q q . Moreover, we derive

which implies W p p P, r Q q ' δ ` o p δ q by triangle inequality. Hence, by L -Lipschitzness of J θ and (Bartl et al., 2021, Appendix Corollary 7.5) we obtain

Finally, by taking ε Ñ 0 , we deduce

which concludes the proof of Proposition 5.2.

Now, we present the proof of the lower bound estimate in Theorem 5.1,

Proof of Theorem 5.1. By adding and subtracting W p δ q S , Proposition 5.2 now gives

Naturally, we can rewrite loss V p 0 q using the usual tower property

Together these yield

Plugging in C p 0 q ' r V p 0 q ´ p 1 ´ A q W p 0 qs{ A completes the proof.

B Bounds on Out-of-Sample Performance

The concentration inequality in Fournier and Guillin (2015) is pivotal to derive the out-of-sample performance bounds. It characterizes how likely an empirical measure can deviate from its generating distribution. We note that recently (Larsson et al., 2023, Olea et al., 2023) obtained concentration inequalities considering other transport costs, including sliced Wasserstein distances which are of particular interest for regression tasks. Recall we denote p P as the empirical measure of training set D tr with size N and q P as the empirical measure of test set D tt with size M . We use the same p , q notations to denote quantities computed from p P and q P , respectively. We restate the concentration inequality as

where K is a constant only depending on n and p . For convenience, K might change from line to line in this section. Together with Theorem 5.1, we give an out-of-sample clean accuracy guarantee in Corollary 5.3.

Proof of Corollary 5.3. By triangle inequality and concentration inequality, we have

With Theorem 5.1, it implies that with probability at least 1 ´ 2 K exp p´ Kε n min t M,N uq , we have

q

A

ě

A

2

ε

ě

A

R

l

2

ε

`

o

p

ε

q

.

The following results provide a guarantee on the out-of-sample adversarial performance.

Proof of Theorem 5.4. Estimates in Fournier and Guillin (2015) imply that with probability at least 1 ´ K exp p´ KNε n q , we have B δ p p P q Ď B δ ` ε p P q . Hence, we derive V p δ q ď p V p δ ` ε q . On the other hand, since J θ is L -Lipschitz, from (Bartl et al., 2021, Appendix Corollary 7.5) we obtain

where B ‹ δ p p P q ' arg max Q P B δ p p P q E Q r J θ p x, y qs . Combining above results, we conclude that with probability at least 1 ´ K exp p´ KNε n q

p

p

p

Proof Corollary 5.5. By Theorem 5.1, we have

We now bound the numerator and denominator separately. By taking ε ' δ in Theorem 5.4, we have with probability at least 1 ´ K exp p´ KNδ n q , V p δ q ď p V p δ q ` L δ. Similarly, we can also show that V p 0 q ě p V p 0 q ´ L δ . Hence, we have with probability at least 1 ´ K exp p´ KNδ n q ,

For the denominator, notice that P p p P P B δ p P qq ě 1 ´ K exp p´ KN exp p δ n qq . By Assumption 5.1 (ii), we have with probability at least 1 ´ K exp p´ KNδ n q ,

This implies

Combining above results, we conclude that with probability at least 1 ´ K exp p´ KNδ n q ,

C W-PGD Algorithm

We give the details of W-PGD algorithm implemented in this paper. Recall in Section 4, we propose

where we take } ¨ } ' } ¨ } r and hence h is given by x h p x q , x y ' } x } s . In particular, h p x q ' sgn p x q for s ' 1 and h p x q ' x {} x } 2 for s ' 2 . The projection step proj δ is based on any off-the-shelf optimal transport solver S which pushes the adversarial images back into the Wasserstein ball along the geodesics. The solver S gives the optimal matching T between the original test data D tt and the perturbed test data D 1 tt . Formally, proj δ maps

where d ' ! 1 | D tt | ř p x,y qP D tt } T p x q ´ x } p r ) 1 { p the Wasserstein distance between q P and q P 1 . See Algorithm 1 for pseudocodes. In numerical experiments, due to the high computational cost of the OT solver, we always couple each image with its own perturbation.

Algorithm 1: W-PGD Attack

D Wasserstein Distributionally Adversarial Training

Theorem 4.1 offers natural computationally tractable approximations to the W-DRO training objective

First, consider a regularized optimization problem:

The extra regularization term δ Υ allows us to approximate the W-DRO objective above. A similar approach has been studied in García Trillos and García Trillos (2022) in the context of Neural ODEs, and Sinha et al. (2018) considered Wasserstein distance penalization and used duality.

Training is done by replacing P with p P . Note that p Υ is a statistics over the whole data set. In order to implement stochastic gradient descent method, an asynchronous update of the parameters is needed. We consider } ¨ } ˚ ' } ¨ } s and, by a direct calculation, obtain

We calculate the term p Υ 1 ´ q from the whole training dataset using parameter θ ˚ from previous epoch; we estimate

on a mini-batch and update current parameter θ after each batch. At the end of an epoch, we update θ ˚ to θ . See Algorithm 2 for pseudocodes.

Another classical approach consists in clean training the network but on the adversarial perturbed data. To this we employ the Wasserstein FGSM attack described in Section 4 and shift training data p x, y q by

and its extension

Similarly to the discussion above, an asynchronous update of parameters is applied, see Algorithm 3 for details. Empirical evaluation of the performance of Algorithms 2 and 3 is left for future research.

Algorithm 2: Loss Regularization

Algorithm 3: Adversarial Data Perturbation

E Robust Performance Bounds

As pointed out in Section 7, with attack budget δ ' 8 { 255 some neural networks have lower bounds R l surpassing the reference value R obtained from AutoAttack. It is because for most of neural networks δ ' 8 { 255 is outside the linear approximation region of the adversarial loss V p δ q , and we underestimate V p δ q by using first order approximations in Theorem 4.1. In Figure 4, we plot our proposed bounds R u and R l against R for p W 8 , l 8 q threat model with δ ' 8 { 255 .

In general, to compute more accurate lower bounds on the adversarial accuracy, as explained in Section 6, we can consider the first lower bound in (13). We thus introduce R l p n q given by

where V p δ, n q is the approximated adversarial loss computed by a W-PDG-( n ) attack. In Figures 5 and 6, we include plots for different bounds of R under W 2 threat models which illustrate the changing performance of the lower bound in Theorem 5.1 as V p δ q is computed to an increasing accuracy. We achieve this performing a W-PDG-( n ) attack, where n ' 5 , 50 . An n ' 50 attack takes

0

.

40

0

.

45

0

.

50

0

.

55

0

.

60

0

.

65

0

.

70

0

.

75

0

.

80

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

( W ∞ , l ∞ ) Threat Model with δ = 8 / 255

R

u

R

l

R

10 times more computational time than the n ' 5 attack and the latter is 5 times more computational costly than the one-step bound R l . The plots thus illustrate a trade-off between computational time and accuracy of the proposed lower bound. For clarity, we also point out that Figure 5 has a different scaling from the other plots.

In practice, we would compare our one-step-attack bound R l with the bound obtained by iterating the attack several steps: we propose W-PGD(5) in our paper and report R l p 5 q . If the difference between the two is small it indicates linear approximation is working very well. If the difference is significant, we would use R l p 5 q and maybe compare it against R l p 10 q . If we observe that V p δ q is convex - as for the CE loss under p W 8 , l 8 q attack - the lower bound should decrease. In this case, the one-step bound may in fact not be a lower bound for too large as shown in Figure 4. If we observe V p δ q is concave - as for the ReDLR loss and our W 2 attacks - the one-step lower bound might be too low and will increase with additional PGD steps. This is visible in Figures 5 and 6.

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

0

.

86

0

.

88

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

δ

= 1

/

510

R

u

R

l

R

l

(5)

R

l

(50)

y

=

x

0

.

800

0

.

825

0

.

850

0

.

875

0

.

900

0

.

925

0

.

950

0

.

975

1

.

000

0

.

70

0

.

75

0

.

80

0

.

85

0

.

90

0

.

95

1

.

00

δ

= 1

/

255

R

u

R

l

R

l

(5)

R

l

(50)

y

=

x

F Experiments on Other Datasets

We illustrate our theoretical results on CIFAR-100 (Krizhevsky, 2009) and ImageNet (Deng et al., 2009) datasets. We analyze all the networks included on RobustBench leaderboards. Only l 8 norm on images is considered as RobustBench does not provide any models for l 2 for these datasets. Similar to CIFAR-10, CIFAR-100 is a low resolution dataset but with 100 classes. Each class contains 500 training images and 100 testing images, and each image has 32 ˆ 32 pixels in 3 color channels. For

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

0

.

90

0

.

92

0

.

94

0

.

96

0

.

98

1

.

00

δ

= 1

/

32

R

u

R

l

R

l

(5)

R

l

(50)

y

=

x

0

.

80

0

.

85

0

.

90

0

.

95

1

.

00

0

.

80

0

.

85

0

.

90

0

.

95

1

.

00

δ

= 1

/

16

R

u

R

l

R

l

(5)

R

l

(50)

y

=

x

ImageNet dataset, we use a subset of 5000 images selected by RobustBench. The size of image may vary among different preprocessing. A typical size is 224 ˆ 224 pixels in 3 color channels. Numerical results mirror those for CIFAR-10 in the body paragraph.

Before the discussion on the numerical results, we remark that the computation time of our proposed methods do not suffer from the curse of dimensionality. Despite the fact that complexity of Wasserstein distance in higher dim can cause troubles, this does not happen for our methods due to our formulae for the first order sensitivity. This means that the approximation to the worst case Wasserstein adversarial attack is explicit and only requires us to compute the gradient ∇ x J θ p x, y q . In consequence, the size of the image only affects the first layer of the DNN and the number of classes affects the last layer of the DNN. The computational time is thus mostly determined by the (hidden) architecture of the DNN model we consider.

In Figure 7, we compare the adversarial accuracy under pointwise threat models and distributional threat models. For W 8 threat model, we apply a combination PGD-50 attack based on CE, DLR losses to obtain the reference adversarial accuracy; for W 2 threat model, we apply a 50-step W-PGDReDLR to obtain the reference WD-adversarial accuracy. Similar to CIFAR-10, we observe a drop of adversarial accuracy under distributional threat models on CIFAR-100 and ImageNet.

In Figure 8, we plot the bounds R l and R u against R on CIFAR-100 and ImageNet. We compute bounds under W 8 -attack based on CE loss across neural networks on RobustBench; W 2 -attack for the same sets of neural networks with bounds computed using ReDLR loss, where R l p 5 q denotes the lower bound (eq. (13)) computed from a 5-step W-PGD attack. For pointwise threat models, our upper and lower bounds sandwich the true (AutoAttack) accuracy ratio very well at a fraction of the computational cost. For distributional threat models, our bounds work but the tightness of the bound is model-dependent. We display also the improved bound obtained with 5 steps of W-PGD attack which leads to a marked improvement in the lower bound's performance. This is akin to Figure 5 in Appendix E.

G Comparison to Existing Attack Methods

In this section, we compare our proposed W-PGD attack with existing attack methods. We reiterate that our method focuses on distributional threats under a given budget δ . While the W-DRO formulation has been explored in numerous works, as discussed, we believe that none of them proposed an AA method specifically tailored for the distributional threat under a given attack budget.

We illustrate the main difference between our sensitivity approach and duality approach employed in Sinha et al. (2018), Volpi et al. (2018), Hua et al. (2022), etc. When using duality of W-DRO, the optimization involves a minimization over the Lagrangian multiplier λ . However, in the aforementioned works, a fixed λ is used. While their choice of λ may be optimal for some budget δ , there is no universal approach to achieve an optimal attack for a given budget δ . In contrast, our attack is an explicit first-order approximation designed to be optimal for a given budget δ . It only involves ∇ x J θ p x, y q and its norms which are fast to compute using standard methods.

10%

20%

30%

40%

50%

60%

70%

80%

, l

)

10%

20%

30%

40%

50%

60%

70%

80%

W

2

, l

∞

)

CIFAR-100

δ

= 2

/

255

δ

= 4

/

255

δ

= 8

/

255

y

=

x

0%

10%

20%

30%

40%

50%

60%

70%

, l

)

0%

10%

20%

30%

40%

50%

60%

70%

W

2

, l

∞

)

ImageNet

δ

= 2

/

255

δ

= 4

/

255

δ

= 8

/

255

y

=

x

W

∞

∞

W

∞

∞

0

.

70

0

.

75

0

.

80

0

.

85

0

.

90

0

.

95

0

.

70

0

.

75

0

.

80

0

.

85

0

.

90

0

.

95

(

W

∞

, l

∞

) Threat Model with

δ

= 2

/

255

R

u

R

l

R

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

(

W

∞

, l

∞

) Threat Model with

δ

= 4

/

255

R

u

R

l

R

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

(

W

2

, l

∞

) Threat Model with

δ

= 1

/

510

R

u

R

l

R

l

(5)

R

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

(

W

2

, l

∞

) Threat Model with

δ

= 1

/

255

R

u

R

l

R

l

(5)

R

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

0

.

90

(

W

∞

, l

∞

) Threat Model with

δ

= 2

/

255

R

u

R

l

R

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

(

W

∞

, l

∞

) Threat Model with

δ

= 4

/

255

R

u

R

l

R

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

(

W

2

, l

∞

) Threat Model with

δ

= 1

/

510

R

u

R

l

R

l

(5)

R

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

1

.

0

(

W

2

, l

∞

) Threat Model with

δ

= 1

/

255

R

u

R

l

R

l

(5)

R

0

.

65

0

.

70

0

.

75

0

.

80

0

.

85

0

.

90

0

.

3

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

4

0

.

5

0

.

6

0

.

7

0

.

8

0

.

9

1

.

0

0

.

0

0

.

2

0

.

4

0

.

6

0

.

8

Despite no existing baseline for addressing distributional threat, it is possible to reverse engineer a comparison setup ex post. We employ CW attack which gives the minimum distortion needed to successfully attack an image. Due to the computational cost of CW attack, the experiment is conducted on the first 1000 correctly classified images from the ImageNet subset used by RobustBench. By calculating the square root of the mean square of the minimum distortion over this chosen dataset, we obtain a reference p W 2 , l 8 q budget 0.000711 to perform our distributional attacks. Employing our W-PGD(50)-ReDLR attack results in 934 images being successfully attacked out of the 1000. The CW attack, which by construction was successful on all 1000 images, took ca 4.5h to run (on a Tesla V100-SXM2-32GB-LS GPU) and the time needed to be scaled linearly with the number of images as the attack has to be performed image by image. In contrast, our attack took ca 50min to run (on a NVIDIA A100-SXM4-40GB GPU). In addition, we use mini-batches, which allows for computational speed improvements based on the available GPU memory.

H Additional Numerical Results

In Tables 3 and 4, we report the clean accuracy and the adversarial accuracy under different threat models for all but 4 neural networks available on RobustBench (model zoo). All networks are named after their labels on RobustBench (model zoo). We remove Standard l 8 -network and Standard l 2 -network because they are not robust to any attacks. In addition, we also remove Kang2021Stable which contains NeuralODE blocks and Ding2020MMA which has a huge gap between adversarial accuracies obtained from PGD and AutoAttack.

In Tables 5, 6, 7 and 8, we include the complete list of R and its bounds used in Figure 3. We write ∆ R u ' R u ´ R and ∆ R l ' R l ´ R .

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel adversarial attack methodology premised on Wasserstein distributionally robust optimization. This approach serves as a generalized technique that encompasses many established adversarial attack methods such as FGSM and TRADES. To ensure computational tractability, the authors introduce a first-order approximation of the attack loss. This approximation is particularly suited for gradient-based optimization methodologies, offering a much faster computational speed compared to SOTA techniques such as AutoAttack. Theoretical evidence is provided for certified bounds on adversarial accuracy and out-of-sample performance. Experiments on the CIFAR-10 dataset are conducted to demonstrate the effectiveness of the proposed method.\n\nSTRENGTHS:\n+ The proposed ReDLR loss (10) is both simple and effective. It is built on clear intuition, that the attack should perturb images classified far from the decision boundary more aggressively while leaving misclassified images unchanged. The theoretical analysis and guarantees provided further validate the proposed approach.\n\n+ The method is presented using robust proofs and in-depth analysis. I didn't notice any substantial unjustifiable assumptions or logical inaccuracies. The section on \"Bounds on Out-of-Sample Performance\" is particularly intriguing. Despite substantial parts of the proof being left in the appendix due to page limits, the analysis still provides invaluable insights for future research on empirical distributions of test sets and broader applications of Wasserstein balls.\n\n+ The empirical results demonstrate notable improvements over robust baseline methods like DLR. It is worth noting that while the proposed ReDLR method may appear very similar to DLR at first glance, the empirical results show significant improvements, attesting to the effectiveness of the theoretical analysis.\n\n+ The paper is well-structured and clearly written. Even the first part of the method section could serve as a comprehensive introduction to DLR-based methods, making it approachable and comprehensible even for novices in the field.\n\nWEAKNESSES:\n- One weakness of the paper is that the number of experiments provided is limited (only results on CIFAR-10 with a limited number of baselines compared). However, considering the major contribution of the paper is to provide a comprehensive theoretical analysis and the existing results already demonstrate the effectiveness of the method, I would say the weakness of experiments does not affect the overall quality of the paper.\n\nQUESTIONS:\n- I personally found the intuition stated in line 171-173 is quite important to understand the method and should be further stressed, both in the introduction and the earlier subsections of the method. \n\n- S^c seems to first appeare in line 215, but no clear definition is provided. \n\n- How strong is Assumption 4.1? Though intuitively for classification problems it should be held, it would be great if the authors provide some experimental analysis (with several commonly used NNs), or pinpoint some references.\n\nLIMITATIONS:\nSee Weakness and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on the issue of distributional robustness in deep neural networks and introduces a novel approach to address it. The authors present a tractable distributional attack within a Wasserstein ball, which allows attackers to perturb inputs in a non-uniform manner. They propose a first-order approximation of the problem and introduce a new loss function called ReDLR. This framework encompasses the Fast Gradient Sign Method (FGSM) attack as a special case. The authors also provide tractable bounds for adversarial accuracy against distributional threat models, which offer insights even for pointwise attacks. To validate their approach, experiments are conducted on the CIFAR-10 dataset using state-of-the-art deep neural networks on RobustBench, demonstrating the effectiveness of the attack and the tightness of the bounds.\n\nSTRENGTHS:\nThe strengths of the paper are:\n\n-The introduction of a tractable attack for Wasserstein Distributionally Robust Optimization (WDRO) stands as a significant contribution, as does the adversarial accuracy boundary. Both offer key advancements to the current understanding and applications of deep learning models under adversarial conditions.\n\n-The paper is underpinned by robust mathematical foundations, and its place in relation to the existing body of work in the field is clearly and precisely articulated, demonstrating a deep and accurate understanding of the state-of-the-art.\n\nWEAKNESSES:\n-The guarantees provided in the paper are asymptotic in nature and depend on the quality of the first-order approximation of the problem. This introduces some limitations and potential uncertainty in the practical applicability of the results.\n\nQUESTIONS:\nThe paper demonstrates strong mathematical foundations and is very well written with a very accurate state of the art. The contributions, particularly the formation of a tractable framework for Wasserstein Distributionally Robust Optimization (WDRO) with asymptotic guarantees, are significant.\n\nHowever, certain concerns arise:\n\n-The proposed attack and adversarial accuracy boundaries heavily rely on the validity of the first-order approximation. Therefore, one query would be how the first-order estimation scales with the dimension of the input space. Specifically, does this approach retain its efficacy when applied to larger vision problems, such as those found in ImageNet (224x224x3)?\n\n-The use of the term 'certified bound' seems somewhat misleading as it actually refers to an asymptotic result. The proposed bounds, as they currently stand, cannot provide certification for a given radius (even a small one) as depicted in appendix Figure E1. Furthermore, the lower bound appears to be loose in the experiments. The approach proposed in the appendix to tighten the bound sometimes leads to overestimation, rendering it unsuitable for certification without additional guarantees. Therefore, how could this issue be addressed for a more accurate and reliable certification?\n\nLIMITATIONS:\nThe authors adequately addressed the limitations", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new method combining Wasserstein distance, DRO, and adversarial attack. The novelty is that the method computes the adversarial attack bounded by the Wasserstein distance. The paper provides a bound on the performance of out-of-domain samples. It applies the new adversarial attack on CIFAR10 dataset. The numerical results show that the attack is effective on models that are robust to pointwise attacks, and computing attacks is much faster than other methods.\n\nSTRENGTHS:\n1. The paper proposes an interesting direction by combining DRO and adversarial attacks. It is clear in the writing and presenting methods.\n2. The paper gives some theoretical results on the proposed methods, like lower bound on $R_{\\delta}$. The paper also shows numerical results comparing the lower bound from different methods.\n3. The paper provides numerical results to show that the adversarial attack is effective at attacking and fast to compute.\n\nWEAKNESSES:\n1. The paper misses some references and does not compare the method with the existing DRO adversarial attack methods, like [A].\n[B, C] also have a similar approach by combining DRO with adversarial attacks. In the area of traditional attacks, the paper can compare with more methods, like CW attack in [D].\n\n[A] Sinha, A., Namkoong, H., Volpi, R. and Duchi, J., 2017. Certifying some distributional robustness with principled adversarial training. arXiv preprint arXiv:1710.10571.\n[B] Volpi, R., Namkoong, H., Sener, O., Duchi, J.C., Murino, V. and Savarese, S., 2018. Generalizing to unseen domains via adversarial data augmentation. Advances in neural information processing systems, 31.\n[C] Hua, X., Xu, H., Blanchet, J. and Nguyen, V.A., 2022, December. Human imperceptible attacks and applications to improve fairness. In 2022 Winter Simulation Conference (WSC) (pp. 2641-2652). IEEE.\n[D] Carlini, N. and Wagner, D., 2017, May. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) (pp. 39-57). Ieee.\n\n2. The paper only does experiments on CIFAR10 dataset. It would be helpful to see results on other datasets.\n\nQUESTIONS:\n1. For the computation time in Table 2, do the authors compare with PGD? PGD is a simple gradient-based method, so the computation time should be very small. Other methods, like FGSM, are also very fast. Also, for Table 2, do all the algorithms stop when they successfully attack all the images?\n2. In line 165, the authors mention another distributional attack, why this method is not compared to in the paper?\n3. In line 39, the paper mentions the approach \"is better suited for gradient-based optimization methods than duality approach adopted in most of the works to date\". Can you clarify on this point?\n\nLIMITATIONS:\nThe authors discuss some of the limitations of the work, including only valid on small-scale attacks. More limitations could include the stability of the attack and how it compares to more state-of-the-art attack methods. The method also should be compared with more attack methods, like DRO-based attacks. From the current experiments, it is not very convincing that the method outperforms many other adversarial attack methods.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper links more general attacks with question of out-of-sample performance and Knightian uncertainty. To evaluate the distributional robustness of neural networks, this paper proposes a first-order AA algorithm and its multistep version. The proposed attack algorithms include Fast Gradient Sign Method (FGSM) and Projected Gradient Descent (PGD) as special cases. Furthermore, the authors provide a new asymptotic estimate of the adversarial accuracy against distributional threat models. The bound is fast to compute and first-order accurate, offering new insights even for the pointwise AA. It also naturally yields out-of-sample performance guarantees. This paper also conducts numerical experiments on the CIFAR-10 dataset to illustrate the theoretical results.\n\nSTRENGTHS:\n1. This paper proposes a unified approach to adversarial attacks and training based on sensitivity analysis for Wasserstein DRO. This approach leveraging results from Bartl et al. (2021) is better suited for gradient-based optimization methods than duality approach adopted in most of the works to date. This paper further links the adversarial accuracy to the adversarial loss, and investigate the out-of-sample performance.\n2. This paper derives a general adversarial attack method. As a special case, this recovers the classical FGSM attack lending it a further theoretical underpinning. However, the proposed method also allows to carry out attacks under a distributional threat model which has not been done before. This paper develops certified bounds on adversarial accuracy including the classical pointwise perturbations. The bounds are first-order accurate and much faster to compute than the AutoAttack benchmark.\n\nWEAKNESSES:\n1. The Wasserstein distance for the adversarial robustness was usually discussed in the theoretical analysis. This paper utilizes the first order approximation of the adversarial loss on the CIFAR-10 for the verification. This approach contributes on the efficiency of the Wasserstein distributional robustness for the practical use. Yet, it is still questionable on the robustness on the large-scale dataset. Due to the computational complexity of the data dimension for the Wasserstein distance, it would be interesting to see the performance and the efficiency of the proposed approach on the image data such as tiny-ImageNet with dimension 64 and ImageNet with dimension 224.\n2. The Wasserstein distance is about the overall loss rather than the single norm such as L_2 and L_\\infty. Thus, the Wasserstein robustness may contribute to other types of robustness such as common corruptions, natural adversarial examples and the perturbations beyond the adversarial robustness. It would be more sufficient to see the potential improvement of the proposed approach on more general perturbed images.\n\nQUESTIONS:\nThe proposed results are valid for small attacks and it is still questionable for the proposed approach for the practical use. Additionally, it would be more general for the proposed method to further discuss on the generalization ability of the Wasserstein-robust models.\n\nLIMITATIONS:\nThis paper uses Wasserstein distributionally robust optimization (DRO) and obtain novel contributions leveraging recent insights from DRO sensitivity analysis. Yet, it would be better to clarify with more sufficient evidences. The detailed comments can be seen in the weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}