Title: Adversarially Robust Learning with Uncertain Perturbation Sets
Venue: NeurIPS 2023
Accepted: yes
Keywords: adversarially robust learning
Abstract: In many real-world settings exact perturbation sets to be used by an adversary are not plausibly available to a learner. While prior literature has studied both scenarios with completely known and completely unknown perturbation sets, we propose an in-between setting of learning with respect to a class of perturbation sets. We show that in this setting we can improve on previous results with completely unknown perturbation sets, while still addressing the concerns of not having perfect knowledge of these sets in real life. In particular, we give the first positive results for the learnability of infinite Littlestone classes when having access to a perfect-attack oracle. We also consider a setting of learning with abstention, where predictions are considered robustness violations, only when the wrong prediction is made within the perturbation set. We show there are classes for which perturbation-set unaware learning without query access is possible, but abstention is required.

Adversarially Robust Learning with Uncertain Perturbation Sets

Abstract

In many real-world settings exact perturbation sets to be used by an adversary are not plausibly available to a learner. While prior literature has studied both scenarios with completely known and completely unknown perturbation sets, we propose an in-between setting of learning with respect to a class of perturbation sets. We show that in this setting we can improve on previous results with completely unknown perturbation sets, while still addressing the concerns of not having perfect knowledge of these sets in real life. In particular, we give the first positive results for the learnability of infinite Littlestone classes when having access to a perfect-attack oracle. We also consider a setting of learning with abstention, where predictions are considered robustness violations, only when the wrong label prediction is made within the perturbation set. We show there are classes for which perturbation-set unaware learning without query access is possible, but abstention is required.

1 Introduction

Adversarial perturbations, imperceivably small manipulations to input instances that cause unexpected failures of learned predictors, pose significant safety concerns for their employment. The phenomenon, first exposed a decade ago in image classification tasks [Szegedy et al., 2014], has since received substantial attention, both in the practical and theoretical machine learning research communities. Studies in both lines of research often skip over the question of how to suitably model 'imperceivable perturbations' and rather investigate how to defend against various (fixed) types of attacks. In theoretical studies, this is typically modeled by defining an adversarial loss that, in addition to misclassification, penalizes the existence of a perturbed instance on which a classifier assigns a different label. This type of loss definition then crucially depends on a given perturbation type , that is some function u : X → 2 X that assigns every domain instance a set of points that would be viewed as an admissible perturbation (often balls of some radius with respect to some norm are considered).

However, in most realistic settings, exact perturbation sets to be used by an adversary are not plausibly available to a learner. Moreover, it has been shown that encouraging robustness with respect to a perturbation type that is not suitable for the task easily degrades classification accuracy [Tsipras et al., 2019, Zhang et al., 2019, Yang et al., 2020]. In response, some work has then explored the question of learnability when the perturbation type is not known at all in advance [Montasser et al., 2021], but information about possible perturbations can be accessed through queries. It is not difficult to be convinced though that without any restrictions or knowledge of the perturbations to be used, robust

learning from samples only is impossible. Even for a small collection of possible perturbation types, as soon as these induce inconsistencies between optimal predictors a sample based learner cannot be simultaneously successful with respect to all options.

In this work we explore a middle ground between fixed and unknown perturbation types, which we term learning with uncertain perturbation sets . We will assume that the true perturbation type is a member of a fixed class of perturbation types. One can think of this as knowing that we should encourage robustness with respect to some type of norm induced ball, but may not be certain about which type of norm or which radius is most suitable for the task. We then study which structures (on the interplay between perturbation type class and the class of predictors to be learned) allow for PAC-type learnability guarantees.

Given a class U of perturbation types and hypothesis class H , we define an H -induced partial order on U . We show that when U is actually totally ordered in this sense, statistical (PAC) learning is possible in the realizable case as soon as H has finite VC-dimension. U being totally ordered applies to a wide variety of settings, for example the case where the various perturbation types are balls in some metric space while the radius varies over U (and actually also does not need to be constant over domain X for each u ∈ U ). However, we also show that, without the realizability assumption learning from samples alone is not feasible, even when U is totally ordered with respect to H .

We thus explore two natural remedies: (1) We allow the learner to interact with a perfect attack oracle [Montasser et al., 2021]. Given an instance x and classifier f , such an oracle certifies robustness or provides an adversarial perturbation of x for f . Previous work on learning with unknown perturbation sets has employed such an oracle and shown learnability for classes with finite Littlestone dimension [Montasser et al., 2021]. In this work, we show that provided certain structures ( U being totally ordered or a finite union of totally ordered perturbation type classes) learning with a perfect attack oracle becomes feasible for all classes of finite VC-dimension (which is a much wider space of natural hypothesis classes than those of finite Littlestone dimension). (2) We allow the learner to output a classifier that sometimes abstains from prediction . For this, we define a modification of the adversarial loss, where a hypothesis does not suffer loss when it abstains on an instance that was actually manipulated. We show that this again yields learnability for VC-classes when the perturbation type class is a finite union of totally ordered classes. We then consider the case in which H has finite disagreement-coefficient [Hanneke, 2007] and show that such H can be learned with respect to every class of perturbation types. And finally, we define a notion of ( ϵ, H ) -cover for a class of perturbation types U , introduce learners for this kind of cover and show that a class H is robustly learnable with respect to U if there is a finite-disagreement-cover learner for ( H , U ) .

Our results on learning with abstentions show that different strategies for guarantees for different kinds of robustness (for example different types of perturbation sets) can be combined into an abstention learner that only predicts in the agreement region. The guarantees for each learning strategy then yield a guarantee for the combined classifier that depends on the number of strategies used. This highlights a different aspect to the power of abstention in adversarially robust learning from what has been established in prior work [Balcan et al., 2020, 2023]. We show that abstentions can also be used to address uncertainty in perturbation types.

1.1 Related work

Providing PAC type learning guarantees under adversarial robustness requirements has received an enormous amount of research attention in the past decade [Feige et al., 2015, Cullina et al., 2018, Wang et al., 2018, Awasthi et al., 2019, Montasser et al., 2019, Attias et al., 2019, Montasser et al., 2020, Ashtiani et al., 2020, Montasser et al., 2021, Gourdeau et al., 2021, 2022, Montasser et al., 2022, Attias and Hanneke, 2022, Attias et al., 2022, Bhattacharjee et al., 2021, Awasthi et al., 2022b,a, 2023, Mao et al., 2023]. We will focus here on aspects within this literature that are most relevant to our settings.

Most prior works study the sample complexity of adversarial robustness with respect to a fixed type of adversarial perturbation (often a metric ball with a certain radius). However, recent research has developed frameworks of analysis that go beyond learning with respect to a fixed type of perturbations. It has been argued that the 'correct' type of admissible adversarial perturbations (which originally were 'imperceivable' modifications to input instances that lead to misclassification by a learned predictor) should depend on the underlying data-generating process [Bhattacharjee and Chaudhuri,

2021, Chowdhury and Urner, 2022]. These studies define a notion of 'adaptive robustness' with respect to which a predictor should be evaluated. The drawback of these notions is that the correct perturbation type is not available to the learner and thus a predictor cannot be straightforwardly evaluated with respect to this loss. A different way of relaxing the assumption of a fixed, known perturbation type is provided in the framework of tolerance [Ashtiani et al., 2023, Bhattacharjee et al., 2023]. Here, a learner is evaluated with respect to some fixed perturbation type, while being compared to the optimal achievable loss with respect to a larger perturbation type, relaxing the requirement for the learner and reflecting that the exact relevant perturbation type is typically not known (or even uniquely existing).

Most relevant to our work is a recent study on PAC learning with respect to unknown perturbation sets [Montasser et al., 2021]. This work provides learning guarantees for the case that the robustness requirement will be with respect to an entirely unknown perturbation type without any prior knowledge about the nature of these perturbations. Those learning bounds assume access to a perfect attack oracle (as we do in some parts of this work), and require the class to have finite Littlestone dimension. In this work, we show that adding some structure to the class of perturbation types (as well as the promise that the true perturbation type will be a member of this class), allows for sample and query complexity bounds that are independent of the hypothesis class's Littlestone dimension.

Finally, there is a long line of work studying the benefits of abstentions for classification tasks [Bartlett and Wegkamp, 2008, El-Yaniv and Wiener, 2010, Wiener and El-Yaniv, 2015, Yuan and Wegkamp, 2010, Zhang and Chaudhuri, 2016]. Some recent studies have also shown that the ability to abstain from prediction for manipulated instances can be beneficial for adversarial robustness requirements [Balcan et al., 2020, 2023]. However, the former studies a different setting for the generation of adversarial perturbations, and the latter considers fixed perturbation types.

1.2 Overview of results

In Section 2, we define our setup, introduce the notions of learnability and loss we investigate and provide an overview of the different kinds of prior knowledge and oracle access we employ. In particular, we there define our partial order on classes of perturbation types.

In Section 3 we analyze perturbation type classes for which this order is a total order. Theorem 1 states that every hypothesis class with finite VC dimension can be robustly learned in the realizable case with respect to any totally ordered perturtbation type class. We then prove that this result cannot be extended to the agnostic case (Observation 1). This motivates investigating the benefits of a perfect attack oracle. Theorem 2 shows that the impossibility can be overcome with access to a perfect attack oracle and provides a finite sample and oracle-query bound for the agnostic case.

In Section 4, we investigate classes of perturbation types U that are not necessarily totally ordered. We first extend our previous results on learning with access to a perfect-attack-oracle to the case where U is a union of finitely many totally ordered perturbation classes (Theorem 3). We then establish that there are classes of perturbation types and hypothesis classes which cannot be learned without abstention, even with access to a perfect attack oracle (Observation 3), but which can be learned with abstention without the need for access to a perfect-attack-oracle (Observation 4). This motivates our investigations into learning with respect to our adversarial abstention loss (Definition 5).

We show that in the realizable case, every hypothesis class with finite VC-dimension can be learned with respect to a finite union of totally ordered perturbation types in Theorem 4. We then consider the case in which H has finite disagreement-coefficient and show that such H can be learned with respect to every class of perturbation types (Theorem 5). Lastly, we define a notion of ( ϵ, H ) -cover for a class of perturbation types U , which states that for every u ∈ U there is a close-to-optimal hypothesis for u in the cover (Definition 7) and introduce the notion of learners for this kind of cover (Definition 8). We then generalize our previous results by showing that a class H is robustly learnable with respect to U if there is a finite-disagreement-cover learner for ( H , U ) (Theorem 6). While we provide some proof sketches in the main body of the paper, the detailed proofs for all results are in the supplement.

2 Setup

For sets A and B , we let 2 A denote the powerset of A and A B the set of all functions from B to A . Let X be a domain and Y = { 0 , 1 } be a label space. We model the data generation as a distribution

̸

P over X × Y , and let P X denote its marginal over X . A hypothesis or predictor is a function h : X → { 0 , 1 , ⋆ } . Thus, on input x ∈ X , a hypothesis h either predicts label 0 or 1 or abstains by outputting ⋆ . We call a hypothesis non-abstaining if h ( x ) = ⋆ for all x ∈ X . A hypothesis class H is a set of non-abstaining predictors. We will let F = { 0 , 1 , ⋆ } X denote the set of all predictors over X .

The performance of a predictor h is measured by a loss function ℓ : F × X × Y → R . For loss ℓ , we use the notation L P ( h ) = E ( x,y ) ∼ P [ ℓ ( h, x, y )] , to denote the expected loss of predictor h with respect to distribution P and L S ( h ) = 1 | S | ∑ | S | i =1 ℓ ( h, x i , y i ) the empirical loss on dataset S = (( x 1 , y 1 ) , . . . , ( x m , y m )) . Further, for a hypothesis class H we will let opt P ( H ) = inf h ∈H L P ( h ) denote the approximation error of class H with respect to P and loss ℓ .

̸

The standard loss for classification tasks is the binary loss ℓ 0 / 1 ( h, x, y ) = ✶ [ h ( x ) = y ] , where ✶ [ · ] denotes the indicator function. The standard loss for adversarial robustness additionally penalizes h on instance ( x, y ) for the existence of a perturbation on x that h does not label with y . This loss is defined for a specified perturbation type , which is a function u : X → 2 X that assigns every instance x ∈ X a set u ( x ) ⊆ X with x ∈ u ( x ) of admissible perturbations . The adversarial loss with respect to u is then defined as

̸

✶ Wewill use the notation ℓ u, ⊥ ( h, x ) to denote the adversarial component loss , which indicates whether a domain point x lies close to the decision boundary with respect to the perturbation type u :

̸

✶ For hypotheses that have the option to abstain from prediction, we propose a variation of the adversarial loss that allows for a predictor to abstain from prediction on perturbed instances (but not on unperturbed instances) without suffering loss. The following definition captures this modification:

Definition 1 (Adversarial abstention loss) . For a perturbation type u : X → 2 X , the adversarial abstention loss is defined by

̸

̸

̸

̸

The main difference with adversarial loss is that if h ( z ) = ⋆ for an adversarial point z ∈ u ( x ) with z = x , then there is no penalty and ℓ u, abst ( h, x, y ) = 0 , thus allowing the predictor to abstain on points that were perturbed. This definition models the scenario where if the predictor can correctly detect that an adversarial attack has happened and abstains, then it is not penalized. However, abstaining on non-perturbed points is still penalized. Note that in case a predictor h never abstains (that is h ( x ) = ⋆ for all x ∈ X ) the two definitions of adversarial loss coincide. We let L u P ( h ) , L u, abst P ( h ) , L u S ( h ) , L u, abst S ( h ) denote the corresponding expected and empirical losses, and opt u P ( H ) , opt u, abst P ( H ) the corresponding approximation errors of a class H and distribution P .

̸

Uncertain perturbation type In the literature, adversarial robustness is mostly treated with respect to a fixed (known) perturbation type u , while learning with respect to an entirely unknown perturbation type has also been investigated. In this work, we introduce a setting that naturally interpolates between these two extremes. We assume that the true perturbation type u ∗ is unknown to the learner, but promised to lie within a fixed class U of perturbation types . That is, the learner has prior knowledge of a class U with u ∗ ∈ U , where u ∗ is the perturbation type that the learner will be evaluated with (see Definition 3 below). We can think of u ∗ as the perturbation type that the true adversarial environment will employ to manipulate input instances and induce misclassifications. We let U all denote the class that contains all possible perturbation types.

It is easy to see that without any restrictions on the class U and any capability of the learner to (interactively) gather information about u ∗ and no option to abstain, a learner cannot succeed (see also Observations 1 and 2 below). To gain control over potentially infinite classes of perturbation types, we will now define a partial order on a class U that is induced by a hypothesis class H . This structure will prove to be conducive for learnability.

Definition 2. Let H ⊆ { 0 , 1 } X be a hypothesis class. For perturbation types u 1 , u 2 we say that u 1 is smaller than u 2 with respect to class H , and write u 1 ⪯ H u 2 , if and only if for all h ∈ H and all x ∈ X we have

We say a set of perturbation types U is totally ordered , with respect to ⪯ H , if for every u 1 , u 2 ∈ U , we have either u 1 ⪯ H u 2 or u 2 ⪯ H u 1 .

We have u 1 ⪯ H u 2 if, for every function h ∈ H the margin area (the points which receive positive adversarial component loss) with respect to u 1 is included in the margin area with respect to u 2 . This is, for example, the case if all perturbation sets with respect to u 1 are included in those induced by u 2 .

Resources to learners and learnability A learner is a function A : ( X × Y ) ∗ → F that takes in a finite sequence S = (( x 1 , y 1 ) , . . . ( x m , y m )) of labeled examples and outputs a predictor A ( S ) : X → { 0 , 1 , ⋆ } . We will study the following PAC-type [Valiant, 1984] notion of learnability of hypothesis classes H with perturbation type classes U .

Definition 3 (Learnability without abstentions) . Let H ⊆ { 0 , 1 } X be a hypothesis class and U a class of perturbation types. We say that H is robustly learnable with respect to perturbation class U if there exists a learner A and a function m : (0 , 1) 2 → N such that for every ϵ, δ > 0 , every m ≥ m ( ϵ, δ ) , every u ∗ ∈ U and every data-generating distribution P we have

We speak of learnability in the realizable case if the above requirement is relaxed to only hold for all distributions P with opt u ∗ P ( H ) = 0 . Without this assumption, we also speak of learnability in the agnostic case .

If the above definition is fulfilled for a class H , then the function m : (0 , 1) 2 → N in the above definition is an upper bound on the sample complexity of the learning task. Note that, while we don't require the learner in the above definition to output a non-abstaining hypothesis, predicting ⋆ will always cause loss with respect to ℓ u . We can thus assume all considered predictors are non-abstaining classifiers for this setting.

Note that the above definition generalizes and unifies previous notions of adversarially robust learnability. We obtain learnability with respect a known perturbation type (the setting that is mostly considered in prior works) when U = { u ∗ } is a singleton class. Learning with unknown perturbations [Montasser et al., 2021] on the other hand, is the setting where U is the set of all possible perturbation types. In this work, we are interested in studying more fine-grained conditions on the structure of (the combination of) H and U that allow for learnability. We will call the case where U is neither a singleton set nor the set of all possible perturbation types learning with uncertain perturbation sets .

We will show that, even when the class of perturbation types U in consideration is totally ordered with respect to hypothesis class H , agnostically learning H with U is impossible even in very simple cases (see Observation 1). We will thus, as has been done in earlier studies [Ashtiani et al., 2020, Montasser et al., 2021] in addition allow the learner access to a perfect attack oracle , which we model as follows:

Definition 4 (Perfect Attack Oracle) . A perfect attack oracle for perturbation type u is a function O u : F × X → X ∪ { robust } , that takes as input a non-abstaining predictor f and a domain point x and either certifies that f is robust on x with respect to u by outputting robust or returns an admissible perturbation of x for f . That is

̸

We say that a class H and perturbation class U are learnable with access to a perfect attack oracle if there exists a function n : (0 , 1) 2 → N such that the conditions in Definition 3 are satisfied for a learner that additionally makes at most n ( ϵ, δ ) queries to the perfect attack oracle. The function n ( · , · ) then is an upper bound on the attack oracle complexity of the learning problem.

In Section 4 we will consider more general pairs of hypothesis and perturbation classes H and U . We will show that if U is not necessarily totally ordered with respect to H (or a finite union of such totally

ordered classes) there are tasks on which no learner (even with access to a perfect attack oracle) can succeed (see Observation 3 below). For such cases we will explore learnability with respect to the adversarial abstention loss:

Definition 5 (Learnability with abstentions) . Let H ⊆ { 0 , 1 } X be a hypothesis class and U a class of perturbation types. We say that H is robustly learnable with abstentions with respect to perturbation class U if there exists a learner A and a function m : (0 , 1) 2 → N such that for every ϵ, δ > 0 , every m ≥ m ( ϵ, δ ) , every u ∗ ∈ U and every data-generating distribution P we have

Note that since H contains only non-abstaining predictors, we have opt u ∗ P ( H ) = opt u ∗ , abst P ( H ) .

Discussion on additional parameters and assumptions Throughout this paper, we will also work with various standard complexity measures, such as bounded VC-dimension or bounded Littlestone dimension of the hypothesis class H , bounded VC-dimension of the loss class or the adversarial component loss class. We refer the reader to the appendix for a reminder of the definitions of these notions.

We will further assume that learners have the capability to identify empirical risk minimizing hypotheses from a class. It is standard (though often implicit) to assume this as oracle access for the learner as well. We consider the standard ERM oracle, a robust ERM (RERM) oracle (for a fixed perturbation type u ) and a maximally robust ERM (MRERM) oracle (for a class U of perturbation types, and realizable samples only). We define the following oracles:

That is, we will assume that MRERM U H will return an error if the input sample S is not ℓ u -realizable for any u ∈ U . If, on the other hand, there does exist a u ∈ U for which S is ℓ u -realizable, then MRERM U H will return a maximal (with respect to ⪯ H ) such perturbation type u ∗ S and corresponding hypothesis h u ∗ S S with L u ∗ S ( h u ∗ S ) = 0 . In particular, we assume that for every non-empty sample S there exists such a maximal element u ∗ S ∈ U and corresponding ERM hypothesis from H . While these do not always exist in U and H a priori, it has recently been shown that it is possible to embed U and H into slightly larger classes so that the MRERM U H oracle is always well defined [Lechner et al., 2023]. See Appendix Section A for more details on this.

Since we focus on studying sample complexity (independently of computational considerations) and consider learners as functions throughout, assuming access to the above oracles is not restricting the validity of our bounds. It is still interesting to understand when is their existence reasonable to assume. Both ERM and RERM oracles have been widely used in the literature and they are often computationally hard to implement. Assuming their existence, however, an MRERM oracle can be easily implemented for the case where U is finite and totally ordered by running a binary search over U and calling RERM for the comparison step. If we assume U is parametrized by a k -bit number, then this requires O ( k ) queries to RERM.

3 Learning with a totally ordered perturbation class

We will start with investigating the case where the perturbation class U is totally ordered with respect to hypothesis class H (see Definition 2). Note that if two perturbation types u 1 and u 2 we satisfy u 1 ( x ) ⊆ u 2 ( x ) for all x ∈ X , then u 1 is smaller than u 2 , u 1 ⪯ H u 2 , for every hypothesis class H ⊂ { 0 , 1 } X . This is, for example, the case when u 1 and u 2 assign balls (with respect to some metric) centered at x to every domain point x and u 2 always assigns a ball of radius at least as large as the ball assigned by u 1 (while the balls assigned by u 1 and u 2 are not necessarily required to have the same radii uniformly over the space X ).

In this section, we will focus on learning with respect to the standard adversarial loss and thus only consider non-abstaining predictors. We first show that for H and totally ordered U , whenever the class H has bounded VC-dimension we get learnability in the realizable case. This result is based on adapting a compression argument for adversarially robust learning [Montasser et al., 2019] to the case of uncertain perturbation sets. A very similar adaptation has recently been made for the related problem of strategic classification [Lechner et al., 2023]. The proof of this result can be found in the appendix.

Theorem 1. Let U be a perturbation class that is totally ordered with respect to a hypothesis class H with VC( H ) = d < ∞ . Then H is learnable in the realizable case with respect to U with sample complexity O ( d · 2 d · log(1 /δ ) ϵ ) .

Recall that realizability here means opt u ∗ P ( H ) = 0 for the true perturbation type u ∗ (with respect to which the learner is evaluated). We now show that without this assumption learning guarantees are impossible without equipping the learner with additional resources (or weakening the success criteria).

Observation 1. There exists a class H with VC( H ) = 1 and a perturbation class U that is totally ordered with respect to H that is not learnable (in the sense of Definition 3).

We will now show that allowing a learner access to a perfect attack oracle yields learnability in the agnostic case.

Theorem 2. Let U be a perturbation class that is totally ordered with respect to a hypothesis class H with VC( H ) = d < ∞ and assume access to a perfect attack oracle. Then H is learnable in the agnostic case with respect to U with sample complexity m ( ϵ, δ ) = O ( d · 2 d · log(1 /δ ) ϵ 2 ) and oracle complexity O ( m ( ϵ, δ ) 2 ) .

Proof. We will employ a well-known reduction from agnostic learning to realizable compressionbased learning [Montasser et al., 2019]. Since our proof for Theorem 1 for the realizable case of learning with uncertain, totally ordered perturbation sets employs a compression-based learner, for the agnostic case it suffices to show that, given any sample S = (( x 1 , y 1 ) , . . . , ( x m , y m )) , we can identify a largest subsample S ′ of S that is realizable with respect to the underlying perturbation type u ∗ . We will now outline how the perfect attack oracle can be employed to achieve this.

Employing the MRERM U H -oracle, the learner can generate an ordered list T = (( S 1 , h 1 , u 1 ) , . . . ( S n , h n , u n ))) of subsamples of S that are realizable with respect to some u ∈ U , such that ( h i , u i ) = MRERM U H ( S i ) for all i and u i ⪯ H u j for all i ≤ j . Note that we have u ∗ ⪯ H u i if and only if the perfect attack oracle returns robust for all sample points in S i when tested on h i , that is O u ∗ ( h i , x ) = robust for all x with ( x, y ) ∈ S i for some y . Thus, using a binary search over the list T , we can identify the smallest index ι for which u ∗ ⪯ H u ι . Since we have n ≤ 2 m and every test in the search employs at most m calls to the perfect attack oracle, this search terminates after at most m 2 queries. Finally, note that all subsets S i from T for which i ≥ ι are realizable with respect to u ∗ . Thus we can choose any largest among S ι , . . . S n as a maximal realizable subset with respect to the unknown perturbation type u ∗ .

In Section B we provide additional guarantees for the agnostic case.

4 Beyond totally ordered perturbation classes

We now consider a more general setting where U is not necessarily totally ordered with respect to H .

4.1 Learning with the standard adversarial loss

We will start by showing that H , U are learnable with access to a perfect attack oracle if U is the union of k sub-classes which are each totally ordered with respect to H . This naturally contains the case that U is finite. Another natural case where this occurs is considering perturbation sets that are balls around domain points of certain radii and with respect to one of a number of possible norms. Consider X = R d and the set

where we let B p r ( x ) denote the ℓ p -norm ball around x . Then the above class U is a union of 4 totally ordered perturbation classes (one for balls of each norm) with respect to any hypothesis class H .

Theorem 3. Let H be a hypothesis class with VC( H ) = d < ∞ , and let U be a perturbation class, which is a union U = U 1 ∪U 2 ∪ . . . ∪U k of k subclasses which are each totally ordered with respect to class H . Then H is learnable with access to a perfect attack oracle O u ∗ in the realizable case with respect to U with sample complexity O ( d · 2 d · log( k/δ ) ϵ + log( k ) · log(1 /ϵ )+log(1 /δ ) ϵ 2 ) and query complexity O ( ( log( k ) · log(1 /ϵ )+log(1 /δ ) ϵ 2 ) 2 ) .

Proof. We can generate k hypotheses h 1 , h 2 , . . . , h k that are the output of an ( ϵ, δ ) -successful learner for H with respect to each of the U i . Since each of the classes U i are totally ordered with respect to H , the result of Theorem 1 tells us that O ( d · 2 d · log( k/δ ) ϵ ) sample points suffice to guarantee that, with probability at least 1 -δ , we have ℓ u ∗ ( h i ) ≤ ϵ for all i with u ∗ ∈ U i . Thus the class H k = { h 1 , h 2 , . . . , h k } of these k learning outputs has approximation error at most ϵ with respect to the data generating distribution P . Note that, since we have |H k | = k , this class has also bounded Littlestone dimension at most log( k ) . Employing a result (Montasser et al. [2021], Theorem 3) on agnostically learning finite Littlestone classes with access to a perfect attack oracle, an additional O ( log( k ) · log(1 /ϵ )+log(1 /δ ) ϵ 2 ) samples and the stated number of queries suffice to output a hypothesis h with L u ∗ P ( h ) ≤ 3 ϵ .

It is not difficult to see that, even in the realizable case, and when U is the union of just two totally ordered subclasses, a learner without access to a perfect attack oracle cannot succeed with respect to the standard adversarial loss.

Observation 2. There exists a class H with VC( H ) = 1 and a perturbation class U that is the union of two totally ordered subclasses with respect to H that are not learnable in the realizable case (in the sense of Definition 3) without access to a perfect attack oracle.

For the proof of this observation we refer the reader to the supplementary material. Now we show that if we remove even more structure from the set of perturbation types U then learning success becomes impossible even with access to a perfect attack oracle. The following observation will then motivate to investigate learning with respect to the adversarial abstention loss (rather than the standard adversarial loss).

Observation 3. There exists a class H with VC( H ) = 1 and a perturbation class U that are not learnable (in the sense of Definition 3), even with access to a perfect attack oracle.

Proof. We consider the same domain, class, and distribution as in the proof of the previous observation: X = R , P with P (( -3 , 1) = P ((3 , 0)) = 0 . 5 , and

H = { h t : x ↦→ ✶ [ x ≤ t ] | t ∈ R } . Further, we consider the class U = { u a,b | a, b ∈ R } of perturbation types u a,b , that assign all points x ≤ 0 to a ball of radius a around x and all points x > 0 to a ball of radius b . Then, even when promised realizability, a learner cannot distinguish the cases where u ∗ = u a,b for some values a, b with a + b ≤ 6 from samples and any finite number of queries to the perfect attack oracle. In particular, if u ∗ = u a,b for some a, b with a + b = 6 , predicting with respect to one such pair a, b will induce loss 0 . 5 for all other such pairs.

4.2 Learning with the adversarial abstention loss

We will now consider learning with respect to the adversarial abstention loss. We will first show that learning with abstentions can actually help overcome some of the impossibilities shown above. For this, let us revisit the example seen in Observation 3.

Observation 4. There exists a class H with VC( H ) = 1 and a perturbation class U that are not learnable (in the sense of Definition 3), even with access to a perfect attack oracle, but that is robustly learnable with abstentions (in the sense of Definition 5) in the realizable case (without access to a perfect attack oracle).

Proof sketch. We consider the example from the proof of Observation 3. Now consider the subclasses U 1 = { u a, 0 : a ∈ R } and U 2 = { u 0 ,b : b ∈ R } . Since both U 1 and U 2 are totally ordered we can learn successful hyptheses h 1 and h 2 respectively. We now define h : X → { 0 , 1 , ⋆ } by h ( x ) = y if h 1 ( x ) = h 2 ( x ) = y and h ( x ) = ⋆ otherwise. This is a successful hypothesis. For more detail, we refer the reader to the appendix.

We will now continue by showing that, similar to the case where we have access to a perfect-attackoracle, a finite union U = ⋃ k i =1 U i of totally ordered perturbation sets U i is robustly learnable with abstention in the robustly realizable case.

Theorem 4. Let U = ⋃ k i =1 U i , where every U i is totally ordered. Furthermore, let H have a finite VC dimension. Then H is robustly learnable with abstension with respect to U in the robustly realizable case with sample complexity O ( d · 2 d · k 2 log( k/δ ) ϵ ) .

4.2.1 Abstention learning with disagreement coefficient

The proof of Theorem 4 shows how bounding the mass of the region on which two learners with small 0 / 1 -loss disagree, yields a robust error guarantee for learning with abstentions. In this section, we will review the definition of the disagreement coefficient (Hanneke [2007], Hanneke et al. [2014]), and show that hypothesis classes H with finite disagreement coefficient are robustly learnable with abstention in the realizable case with respect to every class of perturbation types U .

For a distribution P X over X , we let the P X -induced difference between two hypotheses be denoted by

̸

The ball in H around a hypothesis h with radius r is then denoted by

Lastly, let the disagreement region of a set of hypotheses H be

̸

We can now state the definition of the disagreement coefficient.

Definition 6 (Hanneke et al. [2014]) . For a distribution P over X × Y and a class H ⊆ { 0 , 1 } X , let h ∗ ∈ arg min h ′ ∈H L P ( h ′ ) . The disagreement-coefficient of H with respect to P is defined by:

Furthermore, let the worst-case disagreement coefficient be θ ( H ) = sup P ∈ ∆( X×Y ) θ P ( H ) .

Theorem 5. Let H be a hypothesis class with θ ( H ) < ∞ and V C ( H ) = d < ∞ . Then the class H is robustly learnable with abstention in the realizable case with respect to the class of all perturbation types U all with sample complexity O ( d +log( 1 δ ) ( ϵ θ ( H ) ) 2 ) .

The proof can be found in Appendix D.6, but the main idea is that for any point x , we return a y ∈ { 0 , 1 } if and only if every h with L 0 / 1 S ( h ) = 0 agrees with y , otherwise we abstain. Because of the bound on the disagreement coefficient, one can show that for a large enough S , this leads to a hypothesis with low abstention loss.

4.2.2 Abstention learning via ϵ -cover

We will now generalize our previous results, by introducing the concept of an ( ϵ, H ) -cover.

Definition 7 (( ϵ, H )-cover of U ) . Let H ⊆ { 0 , 1 } X be a hypothesis class, P a distribution over X × Y and U a class of perturbation types. A set of hypotheses H ′ is a ( ϵ, H ) -cover for U with respect to P , if for every u ∈ U , there exists an h ′ ∈ H ′ such that L u P ( h ′ ) < inf h ∈H L u P ( h ) + ϵ .

We will now provide the definition of a successful cover learner.

Definition 8. Let H ⊆ { 0 , 1 } X be a hypothesis class, and U be a class of perturbation types. We say a function A cover : ( X ×Y ) ∗ → 2 { 0 , 1 } X that maps a sample S to a hypothesis class ˆ H ⊂ F is a successful finite-disagreement-cover learner for ( H , U ) , if there is a function m ( H , U ) ,cover : (0 , 1) 3 → N , such that for every ϵ, η, δ ∈ (0 , 1) , every m ≥ m ( H , U ) ,cover ( ϵ, η, δ ) and every distribution P over X × { 0 , 1 } with probability 1 -δ over S ∼ P m both of the following statements hold:

A ( S ) is an ( ϵ, H ) -cover for U with respect to P

P X (DIS( A ( S )) ≤ η .

We note that any successful cover learner, that is guaranteed to output a finite set is a successful finite-disagreement cover learner. The case, where U is a union of k totally ordered perturbation type classes U i , we have seen before, can be interpreted as first learning a cover ˆ H of size k , where each of the element of the union "covers" one of the classes. The case in which H has a finite disagreement coefficient can furthermore be seen as a case where the set of all ERM H hypotheses constitute an ( ϵ, H ) -cover H ′ for every class of perturbation types U . The mass of the disagreement region of H is then bounded by the disagreement coefficient. We now state our result generalizing these observations.

Theorem 6. A class H is robustly learnable with abstentions with respect to U in the robustly realizable case, if there is a successful finite-disagreement-cover learner for ( H , U ) . Furthermore, the sample complexity of robustly learning with abstentions is then bounded by m H , U ( ϵ, δ ) ≤ m H , U ,cover ( ϵ/ 2 , ϵ/ 2 , δ ) .

Inspired by considerations of adaptive robustness radii [Chowdhury and Urner, 2022, Bhattacharjee and Chaudhuri, 2021], in Section C of the appendix we discuss a setting of abstention learning in which we allow the size of the perturbation sets to vary locally between different regions of the domain. We provide a learning guarantee for this setting that is not covered by our previous results.

5 Conclusion

In this work we relaxed the assumption that we know exactly the perturbation sets that are to be used by an adversary. Our model provides an interpolation between knowing the perturbation sets exactly and not knowing them at all. Many of our results rely on realizability. We have also shown that without realizability, learning is not possible unless the learner receives other feedback, such as access to a perfect attack oracle. It might be interesting to consider some milder relaxations of realizability. For example, instead of saying opt u ∗ P ( H ) = 0 , what if we only know that the adversarial Bayes loss is zero for some u ∗ ∈ U ?

While our work focuses on statistical aspects, it will also be interesting to understand the kinds of computation required for our results. For example, we rely on the existence of an MRERM oracle. When are they computationally feasible? Do we require accurate MRERM oracles or can approximate (and hence potentially more tractable) versions also be used for the learning task?

Another question is to find a dimension that characterizes learning in various settings. This was only recently resolved for adversarially robust learning (for the case when u is known) Montasser et al. [2022], however, the dimension proposed in the paper is not easy to define. On the other hand, in the setting when the only information we can obtain about u is through a perfect attack oracle, the Littlestone dimension of H has been shown to characterize adversarially robust learning Montasser et al. [2021]. It will be interesting to see if such a simple dimension can be obtained for the settings considered in our paper.

References

Hassan Ashtiani, Vinayak Pathak, and Ruth Urner. Black-box certification and learning under adversarial perturbations. In Proceedings of the 37th International Conference on Machine Learning, ICML , pages 388-398, 2020.

Hassan Ashtiani, Vinayak Pathak, and Ruth Urner. Adversarially robust learning with tolerance. In International Conference on Algorithmic Learning Theory, ALT , pages 115-135, 2023.

Idan Attias and Steve Hanneke. Adversarially robust pac learnability of real-valued functions. arXiv preprint arXiv:2206.12977 , 2022.

Idan Attias, Aryeh Kontorovich, and Yishay Mansour. Improved generalization bounds for robust learning. In Algorithmic Learning Theory, ALT , pages 162-183, 2019.

Idan Attias, Steve Hanneke, and Yishay Mansour. A characterization of semi-supervised adversarially robust pac learnability. Advances in Neural Information Processing Systems , 35:23646-23659, 2022.

Pranjal Awasthi, Abhratanu Dutta, and Aravindan Vijayaraghavan. On robustness to adversarial examples and polynomial optimization. In Advances in Neural Information Processing Systems, NeurIPS , pages 13760-13770, 2019.

Pranjal Awasthi, Anqi Mao, Mehryar Mohri, and Yutao Zhong. H-consistency bounds for surrogate loss minimizers. In International Conference on Machine Learning , pages 1117-1174. PMLR, 2022a.

Pranjal Awasthi, Anqi Mao, Mehryar Mohri, and Yutao Zhong. Multi-class h -consistency bounds. Advances in neural information processing systems , 35:782-795, 2022b.

Pranjal Awasthi, Anqi Mao, Mehryar Mohri, and Yutao Zhong. Theoretically grounded loss functions and algorithms for adversarial robustness. In International Conference on Artificial Intelligence and Statistics , pages 10077-10094. PMLR, 2023.

Maria-Florina Balcan, Avrim Blum, Dravyansh Sharma, and Hongyang Zhang. On the power of abstention and data-driven decision making for adversarial robustness. CoRR , abs/2010.06154, 2020.

Maria-Florina Balcan, Steve Hanneke, Rattana Pukdee, and Dravyansh Sharma. Reliable learning for test-time attacks and distribution shift. CoRR , abs/2304.03370, 2023.

Peter L. Bartlett and Marten H. Wegkamp. Classification with a reject option using a hinge loss. Journal of Machine Learning Research , 9(59):1823-1840, 2008.

Robi Bhattacharjee and Kamalika Chaudhuri. Consistent non-parametric methods for maximizing robustness. In Advances in Neural Information Processing Systems 34 NeurIPS , pages 9036-9048, 2021.

Robi Bhattacharjee, Somesh Jha, and Kamalika Chaudhuri. Sample complexity of robust linear classification on separated data. In International Conference on Machine Learning , pages 884-893. PMLR, 2021.

Robi Bhattacharjee, Max Hopkins, Akash Kumar, Hantao Yu, and Kamalika Chaudhuri. Robust empirical risk minimization with tolerance. In International Conference on Algorithmic Learning Theory ALT , pages 182-203, 2023.

Sadia Chowdhury and Ruth Urner. Robustness should not be at odds with accuracy. In 3rd Symposium on Foundations of Responsible Computing, FORC , pages 5:1-5:20, 2022.

Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal. Pac-learning in the presence of adversaries. In Advances in Neural Information Processing Systems, NeurIPS , pages 230-241, 2018.

Ran El-Yaniv and Yair Wiener. On the foundations of noise-free selective classification. J. Mach. Learn. Res. , 11:1605-1641, 2010.

Uriel Feige, Yishay Mansour, and Robert Schapire. Learning and inference in the presence of corrupted inputs. In Conference on Learning Theory, COLT , pages 637-657, 2015.

Pascale Gourdeau, Varun Kanade, Marta Kwiatkowska, and James Worrell. On the hardness of robust classification. J. Mach. Learn. Res. , 22:273:1-273:29, 2021.

Pascale Gourdeau, Varun Kanade, Marta Kwiatkowska, and James Worrell. Sample complexity bounds for robustly learning decision lists against evasion attacks. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI , pages 3022-3028, 2022.

Steve Hanneke. A bound on the label complexity of agnostic active learning. In Machine Learning, Proceedings of the Twenty-Fourth International Conference ICML , pages 353-360, 2007.

Steve Hanneke et al. Theory of disagreement-based active learning. Foundations and Trends® in Machine Learning , 7(2-3):131-309, 2014.

Tosca Lechner, Ruth Urner, and Shai Ben-David. Strategic classification with unknown user manipulations. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volume 202 of Proceedings of Machine Learning Research , pages 18714-18732. PMLR, 2023.

Anqi Mao, Mehryar Mohri, and Yutao Zhong. Cross-entropy loss functions: Theoretical analysis and applications. arXiv preprint arXiv:2304.07288 , 2023.

Omar Montasser, Steve Hanneke, and Nathan Srebro. VC classes are adversarially robustly learnable, but only improperly. In Conference on Learning Theory, COLT , pages 2512-2530, 2019.

Omar Montasser, Surbhi Goel, Ilias Diakonikolas, and Nathan Srebro. Efficiently learning adversarially robust halfspaces with noise. arXiv preprint arXiv:2005.07652 , 2020.

Omar Montasser, Steve Hanneke, and Nathan Srebro. Adversarially robust learning with unknown perturbation sets. In Conference on Learning Theory, COLT , pages 3452-3482, 2021.

Omar Montasser, Steve Hanneke, and Nati Srebro. Adversarially robust learning: A generic minimax optimal learner and characterization. Advances in Neural Information Processing Systems , 35: 37458-37470, 2022.

Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms . Cambridge University Press, 2014.

Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR , 2014.

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In 7th International Conference on Learning Representations, ICLR , 2019.

Leslie G. Valiant. A theory of the learnable. Commun. ACM , 27(11):1134-1142, 1984.

Yizhen Wang, Somesh Jha, and Kamalika Chaudhuri. Analyzing the robustness of nearest neighbors to adversarial examples. In Proceedings of the 35th International Conference on Machine Learning, ICML , pages 5120-5129, 2018.

Yair Wiener and Ran El-Yaniv. Agnostic pointwise-competitive selective classification. Journal of Artificial Intelligence Research , 52:171-201, 2015.

Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Ruslan Salakhutdinov, and Kamalika Chaudhuri. A closer look at accuracy vs. robustness. In Advances in Neural Information Processing Systems 33, NeurIPS , 2020.

Ming Yuan and Marten H. Wegkamp. Classification methods with reject option based on convex risk minimization. J. Mach. Learn. Res. , 11:111-130, 2010.

Chicheng Zhang and Kamalika Chaudhuri. The extended littlestone's dimension for learning with mistakes and abstentions. In Proceedings of the 29th Conference on Learning Theory, COLT , pages 1584-1616, 2016.

Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th International Conference on Machine Learning, ICML , pages 7472-7482, 2019.

A Maximally Robust Empirical Risk Minimization

Recall the definition of the empirical risk minimization oracles:

While there always exists functions h S and h u S that are solutions to the queries in the first two oracles, the last one requires a bit more care.

Let S be some labeled dataset and assume that H and U are hypothesis and perturbation type classes such that S is H -realizable for the true perturbation type u ∗ ∈ U . We can then define a maximal perturbation type u ∗ S for S , H and U as follows:

This defines a perturbation type u ∗ S ( x ) , which is not necessarily in U but does satisfy u ∗ ⪯ H u ∗ S , which suffices for our purposes.

If, given u ∗ S , the set argmin h ∈H L u ∗ S S ( h ) is non-empty, the oracle can return any function in this set as h u ∗ S S . We will now argue that we can define a 'closure' ¯ H of the hypothesis class H , which has the same VC-dimension as H and such that h u ∗ S S ∈ ¯ H .

We here show how such an embedding of H into the closure class ¯ H can be constructed in the case of a countable domain X and a perturbation class U that is separable , in the sense that for every U ′ ⊂ U there exists a countable subset U ′′ ⊂ U ′ , such that for every x ∈ X : ⋃ u ∈U ′ u ( x ) = ⋃ u ∈U ′′ u ( x ) . This is the case for a wide range of perturbation types, in particular in the case for which the sets u ( x ) are always open balls with respect to some ℓ p -norm in R d . In the general case (where X may be uncountable and U not separable in the above sense) recent work has shown that the existence of a closure class with identical VC-dimension is still guaranteed [Lechner et al., 2023]. The more general construction there employs the set-theoretic concept of filters . It is shown in the context of strategic classification, but the technique is readily available to adversarial perturbation sets. We here illustrate the intuition behind that abstract construction by focusing on countable and separable case.

We now assume that X is countable and let ( x i ) i ∈ N be an enumeration of the elements of X . The separability assumption on the perturbation class U implies that for all S there is a countable subset U c ⊆ U such that for all x ∈ X we have

For a given sample S , let ( u i ) i ∈ N be an enumeration of the set

and let ( h i ) i ∈ N be an enumeration of corresponding empirical risk minimizing hypotheses from H :

We will now define the function h ∗ S by successively choosing subsequences from the sequence ( h i ) i ∈ N as follows. We set F 0 = ( h i ) i ∈ N and J 0 = N . Then, for every i ∈ N , we define a set of indices J i such that

Case 1: There exists a k ∈ J i -1 such that h j ( x i ) = h k ( x i ) for all j ≥ k , j ∈ J i -1 . Then we define:

Case 2: Both labels are assigned infinitely often to x i by the sequence ( h j ) j ∈ J i -1 . Then we define

and we define subsequence F i = ( h j ) j ∈ J i . Note that, by construction, in each resulting sequence F i the first i points x 1 , x 2 , . . . , x i receive the same label by all functions in the sequence. Now, by choosing indices to select the i -th function from each sequence F i (by choosing the first index of J 1 , the second index in J 2 , the third in J 3 and so on), and re-identifying these indices with the natural numbers, we obtain yet another subsequence F = ( f i ) i ∈ N . In this sequence we now have for every i ∈ N and every k ≥ i

That is, this sequence of functions converges pointwise and we can use the above equation define the function h ∗ S as follows:

It is not difficult to see that the so defined function h u ∗ S S satisfied L S u ∗ S ( h u ∗ S S ) = 0 . Let let ¯ H be defined by adding these function h u ∗ S S to H for all possible S .

We observe that every behavior that ¯ H exhibits on a finite subset C ⊆ X , is already witnessed by a function from H . Thus we get VC( ¯ H ) = VC( H ) , and we can always let MRERM U H return a function from ¯ H .

B Agnostic learning for finite unions of total orders

Theorem 7. Let H be a hypothesis class with VC( H ) = d < ∞ , and let U be a perturbation class, which is a union U = U 1 ∪U 2 ∪ . . . ∪U k of k subclasses which are each totally ordered with respect to class H . Then H is agnostically learnable with access to a perfect attack oracle O u ∗ with respect to U with sample complexity m 1 ( ϵ, δ ) + m 2 ( ϵ, δ ) , and query complexity n 1 ( ϵ, δ ) + n 2 ( ϵ, δ ) where:

m 1 ( ϵ, δ ) = O ( d · 2 d · log( k/δ ) ϵ 2 )

m 2 ( ϵ, δ ) is the sample complexity of agnostically learning a hypothesis class of size k wrt U all .

n 1 ( ϵ, δ ) = k · ( m 1 ( ϵ, δ )) 2

n 2 ( ϵ, δ ) is the query complexity of agnostically learning a hypothesis class of size k wrt U all

Proof. We can generate k hypotheses h 1 , h 2 , . . . , h k that are the output of an ( ϵ, δ ) -successful learner for H with respect to each of the U i . Since each of the classes U i are totally ordered with respect to H , the result of Theorem 2 (combined with a union bound) tells us that m 1 ( ϵ, δ ) sample points and n 1 ( ϵ, δ ) queries suffice to guarantee that, with probability at least 1 -δ , we have ℓ u ∗ ( h i ) ≤ opt u ∗ P ( H ) + ϵ for all i with u ∗ ∈ U i . Thus the class H k = { h 1 , h 2 , . . . , h k } of these k learning outputs has approximation error at most opt u ∗ P ( H ) + ϵ with respect to the data generating distribution P . Note that, since we have |H k | = k , this class has bounded Littlestone dimension at most log( k ) , VC-dimension at most log( k ) , and dual VC-dimension at most k . Employing a result (Montasser et al. [2021], Theorem 4) on agnostically learning finite VC classes with access to a perfect attack oracle, we get that both m 2 ( ϵ, δ ) and n 2 ( ϵ, δ ) are finite and L u ∗ P ( h ) ≤ opt u ∗ P ( H ) + 2 ϵ .

C ϵ -cover Learning

C.1 Abstention learning for regions with different radii

Let us now explore the case, where we allow our perturbation types to consist of perturbation sets whose size varies throughout the domain. While a lot of work assumes that the size of the radius

is fixed throughout the domain, it is not a prior clear that the distances in the domain correspond perfectly to what humans find perceptible. This was noted in prior works [Bhattacharjee and Chaudhuri, 2021, Chowdhury and Urner, 2022] and motivated their adaptive adversarial loss. We explore such variation of local radii for the setting of uncertain perturbation sets. We have seen examples of this in Observation 3 and Observation 4. We will now generalize these perturbation types. Let X = R . For a function f : X → [ k ] , let the perturbation type u f,a 1 ,...,a k defined by:

That is, we have k different regions in the domain, which are determined by f . Furthermore, we assume that each perturbation set within a region i is a ball with radius of a fixed size a i , while radii can vary between regions. For a fixed function f , let us now consider

Let VC( H ) ≤ VC( H×U ) ℓ ≤ d . We note that in general, that is excluding particular hypothesis sets H and function f , this set of perturbation sets cannot be expressed as a finite union of totally ordered sets.

For every i ∈ [ k ] consider the classes U f,i = { u f, 0 ,..., 0 ,a i , 0 ,..., 0 : a i ∈ R } . Let P be a realizable distribution with respect to H and u ∗ ∈ U . Let m ( ϵ, δ ) = C d +log( 1 δ ) ϵ 2 , where C is the constant we require to get a uniform-convergence guarantee (which exists due to the corresponding VCdimensions being finite, see Shalev-Shwartz and Ben-David [2014]), and let m ≥ m ( ϵ k (2 k 2 +1) , δ k ) . Now for a sample S and any i , let ( h i , u i ) = MRERM H U f , i ( S ) . We denote by a max i the maximum realizable radius for component i , i.e. h 0 ,..., 0 ,a max i , 0 ,..., 0 = h i . Now let u ∗ = u f,a ∗ 1 ,...,a ∗ k be the ground true perturbation set. Then for every i , a ∗ i ≤ a max i . Let u max = u f,a max 1 ,...,a max k . We note that for every x ∈ X , y ∈ Y and h ∈ F , we have ℓ u max ( h, x, y ) ≤ max i ∈ [ k ] ℓ u i ( h, x, y ) . Now let

We note that by the PAC-learning guarantee with respect to 0 / 1 -loss, with probability 1 -δ , for every i, j ∈ [ k ] we have a pairwise disagreement between h i and h j of at most 2 ϵ (2 k 2 +1) k . Thus the disagreement-region of { h i : i ∈ [ k ] } has mass at most k 2 ϵ k (2 k 2 +1) . Thus, with probability 1 -δ over the sample generation the abstention loss of h we have

Note, that we can thus view { h } as an ( H , ϵ ) -cover of U with respect to P and adversarial abstention loss.

D Proofs

D.1 Proof of Theorem 1

Proof. For this result we adapt a compression based argument for the case of fixed perturbation type [Montasser et al., 2019] to the case of a class U of perturbation types. A similar adaptation has recently been shown for the setting of strategic classification [Lechner et al., 2023]. The key difference to the original compression scheme is that we use the MRERM U H oracle for the weak learners in each round of boosting, rather than the simple RERM u H oracle as is done in the original publication [Montasser et al., 2019]. Importantly, the maximal ERM paradigm is well defined on all subsets of an original given data sample, and thus each required weak hypothesis can be encoded with a finite number of samples. Finally, since the maximal perturbation type for any subsample is always at least as large as the true perturbation type (under the realizablity assumption), using MRERM U H for each boosting step we obtain the required guarantee for the true perturbation type (without the learner/compressor requiring knowledge of the true type). We thus obtain the same compression size and implied sample complexity in the case of a totally ordered class of perturbation types as for a fixed perturbation type.

D.2 Proof of Observation 1

Proof. Consider a finite domain X = { x 1 , x 2 , x 3 } and distribution P over X × { 0 , 1 } with P (( x 1 , 1)) = 1 / 4 , P (( x 3 , 0)) = 3 / 4 and P (( x, y )) = 0 for all ( x, y ) / ∈ { ( x 1 , 1) , ( x 3 , 0) } . We define a class U = { u 1 , u 2 } containing two perturbation types u 1 and u 2 defined as follows:

and

We let h y 1 y 2 y 3 denote the function that labels x i with y i on this domain (there are only 8 different non-abstaining predictors over X ), and let H = { h 110 , h 100 } be a hypothesis class containing only two of these predictors. Clearly VC( H ) = 1 and u 1 ⪯ H = u 2 . Note that

̸

while L u 1 P ( h y 1 y 2 y 3 ) ≥ 1 / 4 for all predictors h y 1 y 2 y 3 = h 110 . We further note that

while L u 2 P ( h y 1 y 2 y 3 ) ≥ 3 / 4 for all predictors h y 1 y 2 y 3 / ∈ { h 000 , h 100 } . Samples from P will not allow distinguishing whether the true perturbation type is u 1 or u 2 (since we are not imposing a realizability assumption here). Thus, no learner will be able to output predictors that are (close to) optimal for all possible perturbation types in U .

D.3 Proof of Observation 2

Proof. We consider X = R and H to be the class of threshold classifiers:

H = { h t : x ↦→ ✶ [ x ≤ t ] | t ∈ R } . Further consider U = U -∪ U + , where U -consists of perturbation types u -r that assign each x to a ball (interval) of radius r around x if x ≤ 0 and a ball of radius r/ 2 around x if x > 0 , and U + similarly consists of perturbation types u + r that assign each x to a ball (interval) of radius r around x if x > 0 and a ball of radius r/ 2 around x if x ≤ 0 . Both U -and U + are totally ordered (with respect to any hypothesis class) but their union U is not totally ordered with respect to H . Consider a distribution P with P (( -3 , 1) = P ((3 , 0)) = 0 . 5 . Then, even when promised realizability, a learner cannot distinguish the cases u ∗ = u -4 and u ∗ = u + 4 from samples; and performing well with respect to one of these perturbation types will induce loss 0 . 5 with respect to the other.

D.4 Proof of Observation 4

Proof. We consider the example from Observation 3 Now consider the classes U 1 = { u a, 0 : a ∈ R } and U 2 = { u 0 ,b ∈ R } . Since both U 1 and U 2 are totally ordered there are successful robust learners A 1 for H with respect to U 1 and A 2 with respect to U 2 . We run both learners on a sample with a size to guarantee ( ϵ/ 2 , δ/ 2) -success in both cases. We denote the resulting hypotheses with h 1 and h 2 respectively. We now define h : X → { 0 , 1 , ⋆ } by h ( x ) = y if h 1 ( x ) = h 2 ( x ) = y and h ( x ) = ⋆ otherwise.

We now argue that with probability 1 -δ , h has adversarial abstention loss less than ϵ . Let u ∗ = u a ∗ ,b ∗ ∈ U be the ground-truth perturbation type. Now let P be any ℓ u a ∗ ,b ∗ -realizable distribution. Let ( · , u a 1 , 0 ) = MRERM U 1 H ( S ) and ( · , u 0 ,b 2 ) = MRERM U 2 H ( S ) . With probability 1 , we have both a 1 > a ∗ and b 2 > b ∗ . Further, we note every x ∈ X and every u a,b ∈ U , ℓ u a,b ( h, x, y ) is either determined by a, h and y or b, h and y , not both, since either x ≤ 0 or x > 0 . This means, that for every x ∈ X , ℓ u a 1 ,b 2 ( h, x, y ) ≤ max { ℓ u a 1 , 0 ( h, x, y ) , ℓ u 0 ,b 2 ( h, x, y ) } .

Combining these observations, we get

D.5 Proof of Theorem 4

Proof. We assume robust realizability with respect to U . In particular this means that we are in the non-robustly realizable case. Thus for any U i , we know that the set U ′ i = U i ∪ {{{ x } : x ∈ X}} is totally ordered and that it contains some u i ∈ U ′ i such that H is robustly realizable with respect to the ground truth distribution P and u i . We know from Theorem 1, that for every i , there is a successful maximally robust learner A i for H with respect to U ′ i in the realizable case, since U ′ i is totally ordered.

Now let A be defined by

Furthermore, let ( h, u i ( S )) = MRERM U i H ( S ) . Now let u ∗ ∈ U be the ground-truth perturbation type. We know that there is i ∗ ∈ [ k ] , such that u ∗ ⪯ u i ∗ . Let P be a distribution with opt u ∗ P ( H ) = 0 . Now if S ∼ P m with m ≥ max i ∈ [ k ] m ( ϵ 2 k 2 +1 , δ k ) , then with probability 1 -δ , for every i ∈ [ k ] , L u i P ( A i ( S )) ≤ ϵ 2 k 2 +1 and therefore, L 0 / 1 P ( A i ( S )) ≤ ϵ 2 k 2 +1 . Thus, for any two i, j ∈ [ k ] , we have

̸

Thus,

D.6 Proof of Theorem 5

Proof. Let u ∗ ∈ U all be any perturbation set and P any distribution with opt u ∗ P ( H ) = 0 . Furthermore, let m ≥ m H ( ϵ, δ ) where m H is the sample complexity function in O ( d +log( 1 δ ) ( ϵ θ ( H ) ) 2 . Let S ∼ P m . Let

ˆ H = { h ∈ H : L 0 / 1 S ( h ) = 0 } . Now we define our output hypothesis ˆ h by

We note that there is some h ∗ ∈ ˆ H with L u ∗ P ( h ∗ ) = L 0 / 1 P ( h ∗ ) = 0 . Since every h ∈ ˆ H is an ERM H hypothesis and H has VC dimension d , the sample of ERMs gives us

̸

D.7 Proof of Theorem 6

Proof. Let m ≥ m U , H ,cover ( ϵ/ 2 , ϵ/ 2 , δ ) and S ∼ P m . And let u ∗ ∈ U the ground-truth perturbation set. We define the abstention learner as:

̸

Then with probability 1 -δ , P X (DIS( A cover ( S ))) ≤ ϵ 2 and ∃ h ∈ A cover ( S ) with L u ∗ P ( h ) ≤ min h ′ ∈H L u ∗ P ( h ′ ) + ϵ/ 2 . Since for every x ∈ X either A ( S )( x ) = h ( x ) or x ∈ A cover ( S ) . Thus, L u ∗ , abst P ( A ( S )) ≤ L u ∗ , abst P ( h ) + P X (DIS( A cover ( S ))) ≤ min h ′ ∈H L u ∗ , abst P ( h ′ ) + ϵ.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies adversarially robust learning with uncertain perturbation sets, where the set of points to which an adversary can perturb any test input is random, in contrast to previously studied settings of fixed known or unknown perturbation sets and unknown perturbation sets. The perturbation set is assumed to be coming from some fixed perturbation class.\n\nGiven a perfect attack oracle which certifies robustness or gives a successful attack, learnability is shown for concept classes with finite VC dimension with finite bounds on sample complexity and query complexity to the oracle, with some assumptions on U which are satisfied by commonly studied perturbation classes e.g. finite union of totally ordered perturbation sets which includes $L_p$ balls for a finite collection of $p$ values. \n\nWhen the classifier is equipped with abstention, learnability is possible under weaker assumptions e.g. finite disagreement coefficient of H wrt the data distribution. The authors further propose new notion of finite disagreement cover that depends on the perturbation class U, and reduce robust learning under abstentions to successful learning of a finite disagreement cover in the sense of their proposed notion.\n\nSTRENGTHS:\n- The work considers removing a common and unrealistic restriction in prior theoretical work that the adversary uses a fixed perturbation set.\n- The authors give examples demonstrating power of abstention and the difficulty of agnostic learning under their definitions.\n- Proofs or proof sketches of several major results are included and discussed.\n- Comparison to prior works and techniques e.g. Montasser et al. 2021 helps clarify the novelty and technical contributions of the present work.\n\nWEAKNESSES:\n- Realizability: Realizability of perturbation set seems like a strong assumption on the adversary and should be discussed. No positive results are given in the main body in the agnostic case, which could be a limitation of the model under study.\n- Some results are not adequately discussed, e.g. Theorem 5 and last paragraph on Pg 9.\n\nQUESTIONS:\nQuestions:\n- Are there any practical examples or approximations of the oracle considered?\n- What is the sample complexity in Theorem 4?\n- Are the results applicable when the adversary uses a perturbation set outside of U?\n\nSuggestions:\n- It is not clear initially (e.g. lines 42-48) that the realizability being talked about is wrt U and not H. \n- Typos: lines 113, 358, 364.\n- Consider stating positive results in the agnostic case when learning with abstention in the main body (lines 408-413).\n\nLIMITATIONS:\nI do not anticipate potential negative impacts as the work is primarily theoretical, but encourage the authors to summarize limitations and further questions in a conclusion section e.g. lack of positive results in the agnostic case, tightness of sample complexity bounds, etc.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- This paper bridges the gap of PAC learning theory for adversarially robust learning between completely known and completely unknown perturbation across various settings.\n- The authors introduce a notion of hypothesis class-induced partial ordering on the class of perturbation type which they use throughout the paper (setting 1).\n- They show that in a realizable setting, hypothesis class with finite VC dimension and setting 1 are robustly learnable. They show that the same is not valid for agnostic setting without extra assumptions such as access to perfect attack oracle.\n- They further show the existence of perturbation type and hypothesis class which cannot be robustly learned without abstention even with access to perfect attack oracle.\n- They also investigate hypothesis classes with finite disagreement coefficients and present results for this setting.\n- The define an $(\\epsilon, \\mathcal{H})$ cover for a class of perturbation types and use this definition to provide a generalization of their previous results.\n\nSTRENGTHS:\nThe paper is well-written for the most part and provides a good explanation for the results. Furthermore, the claims seem mathematically sound and the authors have made good use of statistical learning theory to present their results. The results are interesting and close the gap between two known settings. They have also introduced some new notions such as the partial ordering of perturbation types with respect to hypothesis class and  $(\\epsilon, \\mathcal{H})$ cover for a class of perturbation types which are interesting concepts.\n\nWEAKNESSES:\nI liked the results and the general writing style of the paper. At times, the novelty seems to be a derivative of existing works with proofs being a clever extension of the proofs of existing settings. I believe this is bound to happen as the problem setting itself is an intermediary between two known settings. \n\nThe following is not a criticism but a remark - It would have been better to include some more mathematical proofs in a theoretical paper like this. However, I understand that space limitations sometimes prohibit this.\n\nQUESTIONS:\nAre there any perturbation models (other than norm balls) which satisfy the setting of this paper?\n\nLIMITATIONS:\nThe authors have addressed the limitations adequately. In that, this paper doesn't propose any (efficient) optimization algorithms, it rather presents sample complexity guarantees with Oracle access to the solution of the optimization problems.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies robust classification when the perturbation region an adversary can access is unknown to the learner, but where it is guaranteed it comes from a certain class of perturbations, which is known to the learner. The authors show various results when the learner has access to a perfect attack oracle (PAO) (which either returns a robust loss of 0, or an admissible perturbation on which the hypothesis is not robust). \n\nFor perturbation classes with total order, without access to the PAO, finite VC dimension is sufficient in the realizable setting, but not in the agnostic one. With access to the PAO, this becomes possible, by a reduction from agnostic to realizable robust learning. \n\nWhen the perturbation class is a finite union of totally ordered perturbations, robust learning is also possible with the PAO (but in general, not without it). Removing structure on the perturbation class renders robust learning impossible (even with PAO). \n\nThe authors then consider robust learning with abstentions, where the learner cannot abstain on an unperturbed point, but can on a perturbed instance. They show that, in this set up,  robust learning is possible. They then relate robust learnability with the disagreement coefficient of a hypothesis class.\n\nSTRENGTHS:\n- Important question, nice set up that lies between unknown perturbation sets and models where the perturbation function is known a priori \n- The results are interesting and draw on multiples different concepts (learning with abstentions, disagreement coefficients, previous robust learning results, etc.)\n- The paper is clear and well-written\n- Meets the technical standards of NeurIPS\n\nWEAKNESSES:\n- No conclusion section - please add one for the final version!\n- MRERM: it seems to be quite a strong assumption to have access to this oracle? Especially since it returns the perturbation $u^*$! The existence of $u^*$ is fine, but that an algorithm can find it for any/most $\\mathcal{H}$ seems quite strong. Could you expand on this, and give examples?\n\nQUESTIONS:\n- Proof of theorem 2: the quantity $n$ is not defined. Do you mean $n=2^m$, the set of all potential subsamples of $S$ ?\n\nComment: \n- It would be worth adding on lines 124-125 and in the statement of Theorem 4 that the result holds for learning with abstentions.\n\nTypos/etc:\n- l.239 \"implicite\"\n- \"can not\" vs \"cannot\" (multiple times throughout the text): e.g., \"it can not be evaluated\" means \"it is possible that it is not evaluated\", while \"it cannot be evaluated\" means \"it is impossible to evaluate it\" - quite different semantics!\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present theoretical results on classification in a version of the PAC model where an adversary can perturb input instances.  Previous work assumed either a fixed perturbation type, known to the learner, or an unknown perturbation type.  This paper explores a middle ground where the perturbation type is a member of a fixed class of perturbation types.  The learner knows the class, but does not know which perturbation type within that class is being used.  For example, the perturbation class might correspond to balls with different radii.\n\nThe paper presents results on learning for a class U of perturbation types and a hypothesis class H.  It defines a partial order on perturbation types in U, defined with respect to H.  It considers variants of the learning model where the learner can abstain from predicting the label of a given example (which is counted as a misclassification error unless the example has been perturbed).  It also considers a variant of the learning model where the learner can has query access to a \"perfect attack oracle,\" a type of oracle studied in previous work on learning with unknown perturbation sets.  \n\nThe results in the paper are all with regard to sample size.  (Computational complexity is not considered.). The first two main results are as follows.  (1) When the class U is totally ordered with respect to H, and H has finite VC dimension, then learning in the realizable case is possible with a polynomial-sized sample (treating VC-dimension as a constant). (2) For the unrealizable case, when U is totally ordered wrt H, a polynomial-sized sample also suffices if the learner can also make polynomially many calls to a perfect attack oracle.  \n\nThe next result considers a similar situation to that in (1) above, except in a more general setting where where U is a union of totally ordered subclasses.  In this setting, even with finite VC dimension and realizability, learning from (finitely many) examples is not possible, but can be achieved with a combination of a polynomial-sized sample and the ability to make polynomially many calls to a perfect attack oracle.  \n\nThe remaining results are concerned with learning with abstentions.  The main results are (1) Learnability in the realizable case, with abstentions, when U is totally ordered and H has finite VC dimension (2) Learnability in the realizable case, with abstentions, when U is the class of all perturbation, when the class has finite VC dimension and finite disagreement coefficient.  A result is also given on learning with abstentions in the realizable case if there is a successful finite-disagreement-cover learner for (H,U).\n\nThe paper also contains simple negative results demonstrating the need for abstentions, total order, etc.\n\nSTRENGTHS:\nA comprehensive exploration of uncertain perturbation sets, bridging work on unknown perturbation sets on the one hand, and a single known perturbation set on the other hand.  The paper is well-written.\n\nDiscussion phase:  I raised my score from 5 to 6.\n\nWEAKNESSES:\nFrom the presentation in the main paper, it does not seem that the paper introduces new techniques or that the presented results are surprising.  It seems as if the techniques have already been used in previous work on perturbations and that the results are similar.  Overall, the work seems like a solid contribution, but lacking in real novelty.\n\nQUESTIONS:\nNo specific questions.  It would be helpful to the reader to justify Definition 1, relating it to the possible behavior of an adversary.\n\nLIMITATIONS:\nI have no concerns in this area.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn adversarially robust PAC learning to test time attacks, usually we assume that the learner knows the perturbation function. \nMontasser et al. ['21] studied a setting were the learner doesn't know the perturbation function, but can interact with it through some oracles. In this paper, the setting is in between, assuming that there is a class of possible perturbations $U$ known to the learner, and the performance of the learner is measured on the worst-case perturbation function in this set. For example, the perturbation can lie in a ball centered in the original input, but the norm isn't known. \n\nThe main contributions are as follows. \n\n- Considering an order on $U$ w.r.t. $H$, finite VC is sufficient for learning in the realizable case, but not in the agnostic case. \n\n- When the learner can interact with $U$ through a perfect attack oracle, and assuming some structure on $U$, finite VC is sufficient for learning. This is an improved result compared to the setting of an unknown perturbation function, where finite Littlestone dimension is sufficient, and $VC(H)\\leq Lit(H)$ and the gap can be arbitrarily large.\n\n- Introducing a setting where the learner can abstain, and showing that finite VC is sufficient for learning (assuming some structure on $U$).\n\n- Assuming that $H$  has a finite disagreement coefficient (a parameter that is related to active learning), then $H$ can be learned with respect to every class of perturbations.\n\nSTRENGTHS:\nThe in-between setting of robust learning suggested in this paper is natural and complements the picture of the other two settings (known $U$ or completely unknown $U$).\nThis paper studies thoroughly various scenarios, showing some limitations of this model, and when we can guarantee robust learning. \nThe results look correct to me. \nI think that this paper could be of interest to the community of theoretical robust learning.\n\nWEAKNESSES:\nIf I'm not missing anything, the technical contribution of the paper is moderate and mostly relies on standard ideas from PAC learning. \n\nMany references on theoretical robust learning are missing. For example, H-consistency bounds for surrogate loss minimizers (ICML 2022), Multi-class H-consistency bounds (NeurIPS 2022), Theoretically grounded loss functions and algorithms for adversarial robustness (AISTATS 2023), Cross-Entropy Loss Functions: Theoretical Analysis and Applications (ICML 2023), A Characterization of Semi-Supervised Adversarially Robust PAC Learnability (NeurIPS 2022), Adversarially Robust PAC Learnability of Real-Valued Functions (ICML 2023), Sample complexity of robust linear classification separated data (ICML 2021)...and many more!\n\nQUESTIONS:\n-\n\nLIMITATIONS:\nI don't see any.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}