Original: On Optimal Learning Under Targeted Data Poisoning
Revised: On Optimal Learning Under Targeted Data Poisoning

Original: Abstract
Revised: Abstract

Original: Consider the task of learning a hypothesis class H in the presence of an adversary that can replace up to an η fraction of the examples in the training set with arbitrary adversarial examples.
Revised: Consider the task of learning a hypothesis class H in the presence of an adversary that can replace up to an η fraction of the examples in the training set with arbitrary adversarial examples.

Original: The adversary aims to fail the learner on a particular target test point x which is known to the adversary but not to the learner.
Revised: The adversary aims to fail the learner on a particular target test point x which is known to the adversary but not to the learner.

Original: In this work we aim to characterize the smallest achievable error ε = ε ( η ) by the learner in the presence of such an adversary in both realizable and agnostic settings.
Revised: In this work we aim to characterize the smallest achievable error ε = ε ( η ) by the learner in the presence of such an adversary in both realizable and agnostic settings.

Original: We fully achieve this in the realizable setting, proving that ε =
Revised: We fully achieve this in the realizable setting, proving that ε =

Original: Θ( VC ( H ) · η ) , where
Revised: Θ( VC ( H ) · η ) , where

Original: VC ( H ) is the VC dimension of H .
Revised: VC ( H ) is the VC dimension of H .

Original: Remarkably, we show that the upper bound can be attained by a deterministic learner.
Revised: Remarkably, we show that the upper bound can be attained by a deterministic learner.

Original: In the agnostic setting we reveal a more elaborate landscape: we devise a deterministic learner with a multiplicative regret guarantee of ε ≤ C · OPT + O ( VC ( H ) · η ) , where C > 1 is a universal numerical constant.
Revised: In the agnostic setting we reveal a more elaborate landscape: we devise a deterministic learner with a multiplicative regret guarantee of ε ≤ C · OPT + O ( VC ( H ) · η ) , where C > 1 is a universal numerical constant.

Original: We complement this by showing that for any deterministic learner there is an attack which worsens its error to at least 2 · OPT .
Revised: We complement this by showing that for any deterministic learner there is an attack which worsens its error to at least 2 · OPT .

Original: This implies that a multiplicative deterioration in the regret is unavoidable in this case.
Revised: This implies that a multiplicative deterioration in the regret is unavoidable in this case.

Original: Finally, the algorithms we develop for achieving the optimal rates are inherently improper.
Revised: Finally, the algorithms we develop for achieving the optimal rates are inherently improper.

Original: Nevertheless, we show that for a variety of natural concept classes, such as linear classiﬁers, it is possible to retain the dependence ε
Revised: Nevertheless, we show that for a variety of natural concept classes, such as linear classiﬁers, it is possible to retain the dependence ε

Original: =
Revised: =

Original: Θ H ( η ) by a proper algorithm in the realizable setting.
Revised: Θ H ( η ) by a proper algorithm in the realizable setting.

Original: Here Θ H conceals a polynomial dependence on VC ( H ) .
Revised: Here Θ H conceals a polynomial dependence on VC ( H ) .

Original:  Introduction
Revised:  Introduction

Original: A basic goal in machine learning is to develop a predicting model from labeled examples (i e., training data) that can reliably generalize to unseen examples (i.e., test data).
Revised: A basic goal in machine learning is to develop a predicting model from labeled examples (i e., training data) that can reliably generalize to unseen examples (i.e., test data).

Original: In its simplest form, namely, binary classiﬁcation, a learner Lrn is given a training set S =
Revised: In its simplest form, namely, binary classiﬁcation, a learner Lrn is given a training set S =

Original: { ( x 1 , y 1 ) , . . .
Revised: { ( x 1 , y 1 ) , . . .

Original: , ( x n , y n ) } , usually assumed to be i.i.d.
Revised: , ( x n , y n ) } , usually assumed to be i.i.d.

Original: samples drawn from an unknown distribution D of labeled examples where x i ’s are the domain instances (or data points) and y i ∈ { 0 , 1 } are the labels.
Revised: samples drawn from an unknown distribution D of labeled examples where x i ’s are the domain instances (or data points) and y i ∈ { 0 , 1 } are the labels.

Original: The aim is to produce a mapping h
Revised: The aim is to produce a mapping h

Original: = Lrn ( S ) that predicts the labels of fresh examples ( x, y )
Revised: = Lrn ( S ) that predicts the labels of fresh examples ( x, y )

Original: ∼ D as accurately as possible, i.e., to minimize the population loss L D ( h ) =
Revised: ∼ D as accurately as possible, i.e., to minimize the population loss L D ( h ) =

Original: Pr ( x,y ) ∼ D
Revised: Pr ( x,y ) ∼ D

Original: [ h ( x ) (cid:54) =
Revised: [ h ( x ) (cid:54) =

Original: y ] .
Revised: y ] .

Original: This classical setting has been extensively studied in the last half a century.
Revised: This classical setting has been extensively studied in the last half a century.

Original: This accumulated work resulted in fundamental mathematical characterizations regarding the nature of learnability when the training samples are truly i.i.d without any tampering by an adversary [Shalev-Shwartz and Ben-David, 2014].
Revised: This accumulated work resulted in fundamental mathematical characterizations regarding the nature of learnability when the training samples are truly i.i.d without any tampering by an adversary [Shalev-Shwartz and Ben-David, 2014].

Original: The goal of this paper is to offer a similar characterization in the presence of an adversary who can tamper with a subset of the training data.
Revised: The goal of this paper is to offer a similar characterization in the presence of an adversary who can tamper with a subset of the training data.

Original: With the emergence of sensitive machine learning applications, it is critical to ensure the trustworthy of such predictive models in the non-ideal scenarios.
Revised: With the emergence of sensitive machine learning applications, it is critical to ensure the trustworthy of such predictive models in the non-ideal scenarios.

Original: In this paper, we consider robust learnability when the training examples can be altered by an adversary whose goal is to make sure that a target test point will be predicted incorrectly.
Revised: In this paper, we consider robust learnability when the training examples can be altered by an adversary whose goal is to make sure that a target test point will be predicted incorrectly.

Original: For instance, consider an adversary who aims to fool a self-driving car to speed up once it observes a stop sign.
Revised: For instance, consider an adversary who aims to fool a self-driving car to speed up once it observes a stop sign.

Original: If such an adversary can somehow inﬂuence the training sets used for training the decision rules, she has all the reasons to strategically change them with the speciﬁc goal of misleading the self-driving car.
Revised: If such an adversary can somehow inﬂuence the training sets used for training the decision rules, she has all the reasons to strategically change them with the speciﬁc goal of misleading the self-driving car.

Original: As another example, consider a loan applicant who wants to make sure that his loan will be granted.
Revised: As another example, consider a loan applicant who wants to make sure that his loan will be granted.

Original: If he can somehow change the training set used by the bank, he might be able to make his application approved.
Revised: If he can somehow change the training set used by the bank, he might be able to make his application approved.

Original: Note that the training set is the lens through which a learning algorithm obtains information about the underlying learning process.
Revised: Note that the training set is the lens through which a learning algorithm obtains information about the underlying learning process.

Original: Therefore, once we allow the training examples to be tampered with by an adversary, even slightly, unexpected outcomes may take place.
Revised: Therefore, once we allow the training examples to be tampered with by an adversary, even slightly, unexpected outcomes may take place.

Original: To quantify the robustness of learning algorithms, in this paper, we show how much the outcome of a learning algorithm for a particular target test can be trusted once the training set is being altered.
Revised: To quantify the robustness of learning algorithms, in this paper, we show how much the outcome of a learning algorithm for a particular target test can be trusted once the training set is being altered.

Original: (PAC) learning under instance-targeted poisoning.
Revised: (PAC) learning under instance-targeted poisoning.

Original: More formally, we consider an adversary Adv that is allowed to replace an η -fraction of the training sample S , resulting to a tampered training sample S (cid:48) given to the learning algorithm Lrn .
Revised: More formally, we consider an adversary Adv that is allowed to replace an η -fraction of the training sample S , resulting to a tampered training sample S (cid:48) given to the learning algorithm Lrn .

Original: Note that even though the training sample S is drawn i.i.d.
Revised: Note that even though the training sample S is drawn i.i.d.

Original: from a distribution D , the tampered training sample S (cid:48) does not enjoy this property anymore.
Revised: from a distribution D , the tampered training sample S (cid:48) does not enjoy this property anymore.

Original: Such attackers are also called poisoning adversaries [Barreno, Nelson, Sears, Joseph, and Tygar, 2006], and variants of them are previously studied under the name of malicious noise
Revised: Such attackers are also called poisoning adversaries [Barreno, Nelson, Sears, Joseph, and Tygar, 2006], and variants of them are previously studied under the name of malicious noise

Original: [Valiant, 1985,
Revised: [Valiant, 1985,

Original: Kearns and Li, 1993] or nasty noise [Bshouty, Eiron, and Kushilevitz, 2002].
Revised: Kearns and Li, 1993] or nasty noise [Bshouty, Eiron, and Kushilevitz, 2002].

Original: More speciﬁcally, we study poisoning settings in which the adversarial perturbation of the original sample S can also depend on the ﬁnal test instance x .
Revised: More speciﬁcally, we study poisoning settings in which the adversarial perturbation of the original sample S can also depend on the ﬁnal test instance x .

Original: Due to the adversary’s knowledge of the target test point x , such poisoning attacks are sometimes referred to as instance-targeted poisoning attacks [Barreno,
Revised: Due to the adversary’s knowledge of the target test point x , such poisoning attacks are sometimes referred to as instance-targeted poisoning attacks [Barreno,

Original: Nelson, Sears, Joseph, and Tygar, 2006].
Revised: Nelson, Sears, Joseph, and Tygar, 2006].

Original: Even without any manipulation to the training set, it is too much to ask the learning algorithm to predict correctly all the time while given only a ﬁnite number of examples to learn from.
Revised: Even without any manipulation to the training set, it is too much to ask the learning algorithm to predict correctly all the time while given only a ﬁnite number of examples to learn from.

Original: In the same vein, we can only hope to design a robust learning algorithm that is correct with high probability over the selection of ( x, y ) ∼ D , especially if the adversary knows the test instance ( x, y ) before manipulating the training set S to S (cid:48) .
Revised: In the same vein, we can only hope to design a robust learning algorithm that is correct with high probability over the selection of ( x, y ) ∼ D , especially if the adversary knows the test instance ( x, y ) before manipulating the training set S to S (cid:48) .

Original: Gao, Karbasi, and Mahmoody [2021], building on ideas from [Levine and Feizi, 2020], proved that PAC learnability under instance-targeted poisoning attacks is achievable only when η = o (1) .
Revised: Gao, Karbasi, and Mahmoody [2021], building on ideas from [Levine and Feizi, 2020], proved that PAC learnability under instance-targeted poisoning attacks is achievable only when η = o (1) .

Original: In other words, when the adversary can only change a sublinear o ( n ) number of n examples, then the optimal learner can achieve error o (1) that goes to zero when the number of examples n goes to inﬁnity.
Revised: In other words, when the adversary can only change a sublinear o ( n ) number of n examples, then the optimal learner can achieve error o (1) that goes to zero when the number of examples n goes to inﬁnity.

Original: 1.1 Our Results
Revised: 1.1 Our Results

Original: The prior work leaves several key questions open on the exact parameters of learnability under instance-targeted poisoning.
Revised: The prior work leaves several key questions open on the exact parameters of learnability under instance-targeted poisoning.

Original: Most importantly, the work of Gao, Karbasi, and Mahmoody [2021] does not quantify the error rate when the adversary’s budget is η = Ω(1) (e.g., if the adversary can corrupt n/ 100 of the examples).
Revised: Most importantly, the work of Gao, Karbasi, and Mahmoody [2021] does not quantify the error rate when the adversary’s budget is η = Ω(1) (e.g., if the adversary can corrupt n/ 100 of the examples).

Original: Secondly, Gao, Karbasi, and Mahmoody [2021] only assume the realizable setting as it is crucial for their results that all the “sub-models” trained using the bagging technique will have error that goes to zero .
Revised: Secondly, Gao, Karbasi, and Mahmoody [2021] only assume the realizable setting as it is crucial for their results that all the “sub-models” trained using the bagging technique will have error that goes to zero .

Original: Hence, the question of ﬁnding optimal learning rates is left open for both realizable and agnostic settings.
Revised: Hence, the question of ﬁnding optimal learning rates is left open for both realizable and agnostic settings.

Original: Finally, as the developed robust algorithms are all based on “bagging” they are inherently improper learning technique.
Revised: Finally, as the developed robust algorithms are all based on “bagging” they are inherently improper learning technique.

Original: In this work, we make progress on all the directions above and achieve optimal error rates (up to constant factors) for general η , both for the realizable and agnostic settings.
Revised: In this work, we make progress on all the directions above and achieve optimal error rates (up to constant factors) for general η , both for the realizable and agnostic settings.

Original: We further study the proper nature of the obtained algorithms and give the ﬁrst proper learning methods that are robust against instance-targeted poisoning attacks for natural hypothesis classes such as linear classiﬁers.
Revised: We further study the proper nature of the obtained algorithms and give the ﬁrst proper learning methods that are robust against instance-targeted poisoning attacks for natural hypothesis classes such as linear classiﬁers.

Original: More precisely, we give a characterization of the optimal error rate of learning under instance-targeted poisoning attacks with budget η · n as follows.
Revised: More precisely, we give a characterization of the optimal error rate of learning under instance-targeted poisoning attacks with budget η · n as follows.

Original: Realizable setting.
Revised: Realizable setting.

Original: We show that the optimal error is Θ( η · d ) where d is the VC dimension of the hypothesis set H .
Revised: We show that the optimal error is Θ( η · d ) where d is the VC dimension of the hypothesis set H .

Original: To prove this, we ﬁrst present an upper bound, showing that a (deterministic) learner can guarantee the error to be at most O ( ηd ) under any instance-targeted poisoning attacks of budget ηn .
Revised: To prove this, we ﬁrst present an upper bound, showing that a (deterministic) learner can guarantee the error to be at most O ( ηd ) under any instance-targeted poisoning attacks of budget ηn .

Original: We then also show a matching lower bound (up to a constant factor) as follows.
Revised: We then also show a matching lower bound (up to a constant factor) as follows.

Original: For any nontrivial 1 hypothesis class of VC dimension d , we show how to design a distribution D over the examples such that
Revised: For any nontrivial 1 hypothesis class of VC dimension d , we show how to design a distribution D over the examples such that

Original: no matter how the learning proceeds, there always exists an adversary of budget ηn that can increase the error (under the instance-targeted attack) to Ω( ηd ) .
Revised: no matter how the learning proceeds, there always exists an adversary of budget ηn that can increase the error (under the instance-targeted attack) to Ω( ηd ) .

Original: Our lower bound above holds even if the learning algorithm uses private randomness that is not known to the adversary 2 .
Revised: Our lower bound above holds even if the learning algorithm uses private randomness that is not known to the adversary 2 .

Original: Our positive result, however, is deterministic, and so can be seen as satisfying the stronger guarantee, in which the adversary’s perturbations to the training set is allowed to depend on learner’s randomness.
Revised: Our positive result, however, is deterministic, and so can be seen as satisfying the stronger guarantee, in which the adversary’s perturbations to the training set is allowed to depend on learner’s randomness.

Original: Agnostic setting.
Revised: Agnostic setting.

Original: We also extend our result above to the agnostic setting in which all hypotheses h ∈ H have population loss bounded away from zero (even before the attack).
Revised: We also extend our result above to the agnostic setting in which all hypotheses h ∈ H have population loss bounded away from zero (even before the attack).

Original: In this setting, we devise a deterministic algorithm whose expected error on the test point is O ( OPT + η · d ) , where
Revised: In this setting, we devise a deterministic algorithm whose expected error on the test point is O ( OPT + η · d ) , where

Original: OPT is the population loss of the best hypothesis h ∈ H .
Revised: OPT is the population loss of the best hypothesis h ∈ H .

Original: A natural question that arises is whether one can achieve an additive regret guarantee of OPT +
Revised: A natural question that arises is whether one can achieve an additive regret guarantee of OPT +

Original: O ( η · d )?
Revised: O ( η · d )?

Original: (Note that agnostic learning is usually deﬁned with respect to additive regret).
Revised: (Note that agnostic learning is usually deﬁned with respect to additive regret).

Original: We show that this is in fact not possible , at least for deterministic learners, by presenting a negative result.
Revised: We show that this is in fact not possible , at least for deterministic learners, by presenting a negative result.

Original: In particular we show that for any deterministic learner Lrn , there is an extremely simple hypothesis class (just consisting of two functions) and an input distribution such that the learner is forced to have adversarial error ≥ 2 OPT .
Revised: In particular we show that for any deterministic learner Lrn , there is an extremely simple hypothesis class (just consisting of two functions) and an input distribution such that the learner is forced to have adversarial error ≥ 2 OPT .

Original: This negative result uses tools from the computational concentration of products [Talagrand, 1995] and a continuity intermediate-value argument.
Revised: This negative result uses tools from the computational concentration of products [Talagrand, 1995] and a continuity intermediate-value argument.

Original: Proper learning.
Revised: Proper learning.

Original: The deterministic algorithm witnessing the above upper bound is inherently improper which might be a disadvantage in terms of interpretability or test-time computational complexity.
Revised: The deterministic algorithm witnessing the above upper bound is inherently improper which might be a disadvantage in terms of interpretability or test-time computational complexity.

Original: In contrast, in (the non-adversarial) PAC setting proper algorithms are known to achieve near optimal learning rates (up to log factors).
Revised: In contrast, in (the non-adversarial) PAC setting proper algorithms are known to achieve near optimal learning rates (up to log factors).

Original: We therefore explore the cost of proper learning under instance-targeted poisoning attacks.
Revised: We therefore explore the cost of proper learning under instance-targeted poisoning attacks.

Original: We show that in many natural classes, such as half spaces, it is indeed possible to obtain proper learning rules that are robust to instance-targeted poisoning attacks, with guarantees which are only polynomially worse than optimal.
Revised: We show that in many natural classes, such as half spaces, it is indeed possible to obtain proper learning rules that are robust to instance-targeted poisoning attacks, with guarantees which are only polynomially worse than optimal.

Original: For example, for the class of half-spaces in R d we derive a deterministic proper learning rule whose error rate is at most O ( d 3 η ) .
Revised: For example, for the class of half-spaces in R d we derive a deterministic proper learning rule whose error rate is at most O ( d 3 η ) .

Original: At a technical level, we achieve this result by relying on the projection number of the class [Bousquet,
Revised: At a technical level, we achieve this result by relying on the projection number of the class [Bousquet,

Original: Hanneke, Moran, and Zhivotovskiy, 2020, Kane, Livni, Moran, and Yehudayoff, 2019, Braverman,
Revised: Hanneke, Moran, and Zhivotovskiy, 2020, Kane, Livni, Moran, and Yehudayoff, 2019, Braverman,

Original: Kol, Moran, and Saxena, 2019].
Revised: Kol, Moran, and Saxena, 2019].

Original: 1.2 Relation to Certiﬁcation and Stability
Revised: 1.2 Relation to Certiﬁcation and Stability

Original: Certiﬁcation.
Revised: Certiﬁcation.

Original: Robustness to instance-targeted poisoning boils down to the following type of stability: on most of the test instances x , the prediction of the learner y =
Revised: Robustness to instance-targeted poisoning boils down to the following type of stability: on most of the test instances x , the prediction of the learner y =

Original: y ( x ) remains the same even if at most η fraction of the examples in the training-set S are replaced.
Revised: y ( x ) remains the same even if at most η fraction of the examples in the training-set S are replaced.

Original: It is natural to require the learning rule to certify this stability.
Revised: It is natural to require the learning rule to certify this stability.

Original: That is, a certifying learning rule provides a bound k
Revised: That is, a certifying learning rule provides a bound k

Original: = k
Revised: = k

Original: ( x ) along with the prediction label y
Revised: ( x ) along with the prediction label y

Original: = y ( x ) , where the meaning of k is that the prediction y
Revised: = y ( x ) , where the meaning of k is that the prediction y

Original: = y ( x ) remains the same even if at most k examples in the input sample are replaced.
Revised: = y ( x ) remains the same even if at most k examples in the input sample are replaced.

Original: Note that it is always possible to provide the trivial guarantee of k = 0 , and therefore the goal is to design robust learners that provide non-trivial certiﬁcates.
Revised: Note that it is always possible to provide the trivial guarantee of k = 0 , and therefore the goal is to design robust learners that provide non-trivial certiﬁcates.

Original: Our algorithm naturally achieves that: for ≈ 1 − ε of the test instances x it provides a guarantee of k ≈ ηn .
Revised: Our algorithm naturally achieves that: for ≈ 1 − ε of the test instances x it provides a guarantee of k ≈ ηn .

Original: Connection to stability.
Revised: Connection to stability.

Original: We also present a new perspective on instance-targeted poisoning attacks by showing how they can be seen as natural forms of algorithmic stability [Bousquet and Elisseeff, 2002,
Revised: We also present a new perspective on instance-targeted poisoning attacks by showing how they can be seen as natural forms of algorithmic stability [Bousquet and Elisseeff, 2002,

Original: Rakhlin, Mukherjee, and Poggio, 2005].
Revised: Rakhlin, Mukherjee, and Poggio, 2005].

Original: In particular, we show that one can study the adversarial robustness (around the true label) to instance-targeted poisoning by decoupling the (pure) stability aspect (which does not depend on the true labels) from the (non-adversarial) risk.
Revised: In particular, we show that one can study the adversarial robustness (around the true label) to instance-targeted poisoning by decoupling the (pure) stability aspect (which does not depend on the true labels) from the (non-adversarial) risk.

Original: We refer to the former as the prediction stability .
Revised: We refer to the former as the prediction stability .

Original: Roughly speaking, prediction stability requires that the model’s prediction on x does not change even if the adversary changes the training set withing its budget ηn .
Revised: Roughly speaking, prediction stability requires that the model’s prediction on x does not change even if the adversary changes the training set withing its budget ηn .

Original: Note that here we do not care whether the model’s output on x is the correct label or not, and hence is a pure measure of stability of the predictions.
Revised: Note that here we do not care whether the model’s output on x is the correct label or not, and hence is a pure measure of stability of the predictions.

Original: It might be helpful to compare prediction stability with the algorithmic stability of [Bousquet and
Revised: It might be helpful to compare prediction stability with the algorithmic stability of [Bousquet and

Original: Elisseeff, 2002, Rakhlin, Mukherjee, and Poggio, 2005].
Revised: Elisseeff, 2002, Rakhlin, Mukherjee, and Poggio, 2005].

Original: The later requires that for a typical sample S of size n , and for every ﬁxed i ∈ [ n ] , the prediction of the model trained on S and tested on a random test-point x is likely not changed if one substitutes the i -th example in S with a fresh random example.
Revised: The later requires that for a typical sample S of size n , and for every ﬁxed i ∈ [ n ] , the prediction of the model trained on S and tested on a random test-point x is likely not changed if one substitutes the i -th example in S with a fresh random example.

Original: Prediction stability strengthens this condition in two ways: (1) the choice of what coordinate in S to change can adversarially depend on the test instance x , (2) the adversary is allowed to change more than one examples (i.e., up to η · n ).
Revised: Prediction stability strengthens this condition in two ways: (1) the choice of what coordinate in S to change can adversarially depend on the test instance x , (2) the adversary is allowed to change more than one examples (i.e., up to η · n ).

Original: 1.3 Related Work
Revised: 1.3 Related Work

Original: Poisoning attacks are studied in theoretical learning under various noise models [Valiant, 1985,
Revised: Poisoning attacks are studied in theoretical learning under various noise models [Valiant, 1985,

Original: Kearns and Li, 1993, Sloan, 1995, Bshouty, Eiron, and Kushilevitz, 2002].
Revised: Kearns and Li, 1993, Sloan, 1995, Bshouty, Eiron, and Kushilevitz, 2002].

Original: However, these works focus on the non-targeted setting in which the adversary does not know the target instance.
Revised: However, these works focus on the non-targeted setting in which the adversary does not know the target instance.

Original: The computational aspects of efﬁcient learning under (non-targeted) poisoning have been studied in various works, including that of Kalai, Klivans, Mansour, and Servedio [2008], Klivans, Long, and
Revised: The computational aspects of efﬁcient learning under (non-targeted) poisoning have been studied in various works, including that of Kalai, Klivans, Mansour, and Servedio [2008], Klivans, Long, and

Original: Servedio [2009], Awasthi, Balcan, and Long [2014], with this last work obtaining nearly optimal (up to constants) learning guarantees among polynomial-time algorithms for learning homogeneous linear separators with malicious noise under distribution restrictions.
Revised: Servedio [2009], Awasthi, Balcan, and Long [2014], with this last work obtaining nearly optimal (up to constants) learning guarantees among polynomial-time algorithms for learning homogeneous linear separators with malicious noise under distribution restrictions.

Original: That result was subsequently extended to the nasty noise model by Diakonikolas, Kane, and Stewart [2018], via techniques that also enable them to study other geometric concept classes.
Revised: That result was subsequently extended to the nasty noise model by Diakonikolas, Kane, and Stewart [2018], via techniques that also enable them to study other geometric concept classes.

Original: In the unsupervised setting, [Diakonikolas, Kamath,
Revised: In the unsupervised setting, Diakonikolas, Kamath,

Original: Kane, Li, Moitra, and Stewart, 2016, Lai, Rao, and Vempala, 2016] studied the computational aspect of learning under poisoning.
Revised: Kane, Li, Moitra, and Stewart [2016], Lai, Rao, and Vempala [2016] studied the computational aspect of learning under poisoning.

Original: In contrast, our work focuses on (supervised) instance-targeted poisoning, and we study the learning rates information theoretically regardless of learner’s computing power.
Revised: In contrast, our work focuses on (supervised) instance-targeted poisoning, and we study the learning rates information theoretically regardless of learner’s computing power.

Original: The work of Steinhardt, Koh, and Liang [2017] further studied the certiﬁcation of the overall (non-targeted) error.
Revised: The work of Steinhardt, Koh, and Liang [2017] further studied the certiﬁcation of the overall (non-targeted) error.

Original: More recently, such (non-targeted) poisoning attacks are combined with test-time attacks and are studied under the name of backdoor attacks [Gu, Dolan-Gavitt, and Garg, 2017, Ji,
Revised: More recently, such (non-targeted) poisoning attacks are combined with test-time attacks and are studied under the name of backdoor attacks [Gu, Dolan-Gavitt, and Garg, 2017, Ji,

Original: Zhang, and Wang, 2017].
Revised: Zhang, and Wang, 2017].

Original: Besides instance-targeted attacks (which are the focus of this paper), other notions of targeted attacks were studied in the literature: for example, in model-targeted attacks, the adversary’s goal is to make the learner predict according to a speciﬁc model.
Revised: Besides instance-targeted attacks (which are the focus of this paper), other notions of targeted attacks were studied in the literature: for example, in model-targeted attacks, the adversary’s goal is to make the learner predict according to a speciﬁc model.

Original: Recent works on this model include [Farhadkhani,
Revised: Recent works on this model include [Farhadkhani,

Original: Guerraoui, Hoang, and Villemaud, 2022, Suya, Mahloujifar, Suri, Evans, and Tian, 2021].
Revised: Guerraoui, Hoang, and Villemaud, 2022, Suya, Mahloujifar, Suri, Evans, and Tian, 2021].

Original: Some other works study label-targeted attacks, in which the adversary’s goal is to ﬂip the decision on the test instance to a speciﬁc label (e.g., see targeted misclassiﬁcation attacks in [Chakraborty, Alam,
Revised: Some other works study label-targeted attacks, in which the adversary’s goal is to ﬂip the decision on the test instance to a speciﬁc label (e.g., see targeted misclassiﬁcation attacks in [Chakraborty, Alam,

Original: Dey, Chattopadhyay, and Mukhopadhyay, 2018]).
Revised: Dey, Chattopadhyay, and Mukhopadhyay, 2018]).

Original: The work of [Jagielski, Severi, Pousette Harger, and Oprea, 2021] studies a generalization of instance-targeted attacks, called subpopulation attacks, in which the adversary knows the subset of the inputs, from which the test instance will be drawn.
Revised: The work of [Jagielski, Severi, Pousette Harger, and Oprea, 2021] studies a generalization of instance-targeted attacks, called subpopulation attacks, in which the adversary knows the subset of the inputs, from which the test instance will be drawn.

Original: Most relevant to our setting are the recent works of Gao, Karbasi, and Mahmoody [2021], Blum,
Revised: Most relevant to our setting are the recent works of Gao, Karbasi, and Mahmoody [2021], Blum,

Original: Hanneke, Qian, and Shao [2021] where the general problem of learning (and more quantitative variant of learning error rate) under instance-targeted poisoning was formally deﬁned and studied.
Revised: Hanneke, Qian, and Shao [2021] where the general problem of learning (and more quantitative variant of learning error rate) under instance-targeted poisoning was formally deﬁned and studied.

Original: In particular, Blum, Hanneke, Qian, and Shao [2021] studied learnability under instance-targeted poisoning where the adversary can add an unbounded number of so-called clean-label examples to the training set.
Revised: In particular, Blum, Hanneke, Qian, and Shao [2021] studied learnability under instance-targeted poisoning where the adversary can add an unbounded number of so-called clean-label examples to the training set.

Original: A clean-label example ( x, y ) has the property that y is the correct label of x , while x could be an arbitrary instance that is not sampled from the same distribution that generates other instances in the training set.
Revised: A clean-label example ( x, y ) has the property that y is the correct label of x , while x could be an arbitrary instance that is not sampled from the same distribution that generates other instances in the training set.

Original: Gao, Karbasi, and Mahmoody [2021] also showed that when the adversary’s corruption is only an o (1) fraction of the training set, PAC learning is possible (if it is possible without the attack).
Revised: Gao, Karbasi, and Mahmoody [2021] also showed that when the adversary’s corruption is only an o (1) fraction of the training set, PAC learning is possible (if it is possible without the attack).

Original: In a concurrent work, Balcan, Blum, Hanneke, and Sharma [2022] study the problem of certifying the correct prediction even under instance-targeted data poisoning.
Revised: In a concurrent work, Balcan, Blum, Hanneke, and Sharma [2022] study the problem of certifying the correct prediction even under instance-targeted data poisoning.

Original: Our methods, however, can be used to obtain certiﬁcation of the stability of the model around their prediction (even though the prediction might not be true always), while controlling the overall error to be provably small (again under the instance-targeted attack).
Revised: Our methods, however, can be used to obtain certiﬁcation of the stability of the model around their prediction (even though the prediction might not be true always), while controlling the overall error to be provably small (again under the instance-targeted attack).

Original: Rosenfeld, Winston, Ravikumar, and Kolter [2020] empirically demonstrated that randomized smoothing [Cohen, Rosenfeld, and Kolter, 2019] can provide robustness against label-ﬂipping attacks, in which the adversary is limited to merely ﬂipping the label of a subset of the training set.
Revised: Rosenfeld, Winston, Ravikumar, and Kolter [2020] empirically demonstrated that randomized smoothing [Cohen, Rosenfeld, and Kolter, 2019] can provide robustness against label-ﬂipping attacks, in which the adversary is limited to merely ﬂipping the label of a subset of the training set.

Original: They also showed that randomized smoothing can be used to handle replacing attacks (the model also studied in this paper), in which the adversary substitutes a part of the training set with a new set of same size.
Revised: They also showed that randomized smoothing can be used to handle replacing attacks (the model also studied in this paper), in which the adversary substitutes a part of the training set with a new set of same size.

Original: Subsequently, Levine and Feizi [2020] used deterministic methods that further allowed attacks that can add examples to or remove them from the training set.
Revised: Subsequently, Levine and Feizi [2020] used deterministic methods that further allowed attacks that can add examples to or remove them from the training set.

Original: Chen, Li, Wu, Sheng, and Li [2020],
Revised: Chen, Li, Wu, Sheng, and Li [2020],

Original: Weber, Xu, Karlas, Zhang, and Li [2020], Jia, Cao, and Gong [2020] further developed the technique of randomized bagging/sub-sampling for the goal of resisting instance-targeted poisoning attacks.
Revised: Weber, Xu, Karlas, Zhang, and Li [2020], Jia, Cao, and Gong [2020] further developed the technique of randomized bagging/sub-sampling for the goal of resisting instance-targeted poisoning attacks.

Original: Finally, we comment that other theoretical works have also studied instance-targeted poisoning attacks [Mahloujifar and Mahmoody, 2017, Etesami, Mahloujifar, and Mahmoody, 2020].
Revised: Finally, we comment that other theoretical works have also studied instance-targeted poisoning attacks [Mahloujifar and Mahmoody, 2017, Etesami, Mahloujifar, and Mahmoody, 2020].

Original: These works show how to amplify error for speciﬁc test instances, say from 0 .
Revised: These works show how to amplify error for speciﬁc test instances, say from 0 .

Original: 01 error to 0 .
Revised: 01 error to 0 .

Original: 5 , through instance-targeted poisoning.
Revised: 5 , through instance-targeted poisoning.

Original: In particular, these works do not talk about the fraction of the test population that is vulnerable to targeted poisoning.
Revised: In particular, these works do not talk about the fraction of the test population that is vulnerable to targeted poisoning.

Original: The work of Shafahi, Huang, Najibi, Suciu,
Revised: The work of Shafahi, Huang, Najibi, Suciu,

Original: Studer, Dumitras, and Goldstein [2018] studied the power of such attacks empirically.
Revised: Studer, Dumitras, and Goldstein [2018] studied the power of such attacks empirically.

Original:  Preliminaries
Revised:  Preliminaries

Original: Notation and basic learning theory deﬁnitions.
Revised: Notation and basic learning theory deﬁnitions.

Original: We consider the setting of binary classiﬁcation.
Revised: We consider the setting of binary classiﬁcation.

Original: Let X denote the input domain and Y = { 0 , 1 } denote the label-set.
Revised: Let X denote the input domain and Y = { 0 , 1 } denote the label-set.

Original: A pair ( x, y ) ∈
Revised: A pair ( x, y ) ∈

Original: X × Y is called an example .
Revised: X × Y is called an example .

Original: A sequence S =
Revised: A sequence S =

Original: ( x 1 , y 1 ) , . . .
Revised: ( x 1 , y 1 ) , . . .

Original: , ( x n , y n ) ∈
Revised: , ( x n , y n ) ∈

Original: ( X × Y ) n of n examples is a sample of size n .
Revised: ( X × Y ) n of n examples is a sample of size n .

Original: The i ’th example in S is denoted by S i .
Revised: The i ’th example in S is denoted by S i .

Original: A function h : X → Y is called an hypothesis or a concept.
Revised: A function h : X → Y is called an hypothesis or a concept.

Original: A set of hypotheses H ⊂ Y X is called an hypothesis class , or a concept class .
Revised: A set of hypotheses H ⊂ Y X is called an hypothesis class , or a concept class .

Original: We denote the VC-dimension of a concept class H by d = d ( H ) .
Revised: We denote the VC-dimension of a concept class H by d = d ( H ) .

Original: For a set Z , let Z ∗ = ∪ n Z n denote the set of all ﬁnite sequences with elements from Z .
Revised: For a set Z , let Z ∗ = ∪ n Z n denote the set of all ﬁnite sequences with elements from Z .

Original: A learning rule or learning algorithm or learner Lrn : ( X × Y ) ∗ → X Y is a deterministic 3 mapping which takes an input sample S ∈ ( X × Y ) ∗ and maps it to a hypothesis Lrn ( S ) =
Revised: A learning rule or learning algorithm or learner Lrn : ( X × Y ) ∗ → X Y is a deterministic 3 mapping which takes an input sample S ∈ ( X × Y ) ∗ and maps it to a hypothesis Lrn ( S ) =

Original: h ∈ X Y .
Revised: h ∈ X Y .

Original: If it is guaranteed that Lrn ( S ) ∈ H for all input samples S then Lrn is said to be proper ; otherwise, it is improper .
Revised: If it is guaranteed that Lrn ( S ) ∈ H for all input samples S then Lrn is said to be proper ; otherwise, it is improper .

Original: Let D be a distribution over examples, and let h be an hypothesis.
Revised: Let D be a distribution over examples, and let h be an hypothesis.

Original: The population loss of h with respect to D is deﬁned by L D ( h ) =
Revised: The population loss of h with respect to D is deﬁned by L D ( h ) =

Original: Pr ( x,y ) ∼ D
Revised: Pr ( x,y ) ∼ D

Original: [ h ( x ) (cid:54) =
Revised: [ h ( x ) (cid:54) =

Original: y ]
Revised: y ]

Original: = E ( x,y ) ∼ D [1[ h ( x ) (cid:54) =
Revised: = E ( x,y ) ∼ D [1[ h ( x ) (cid:54) =

Original: y ]] .
Revised: y ]] .

Original: A distribution D is said to be realizable by H if inf h ∈H L D ( h ) = 0 .
Revised: A distribution D is said to be realizable by H if inf h ∈H L D ( h ) = 0 .

Original: Similarly, for a sample S , let
Revised: Similarly, for a sample S , let

Original: L S ( h ) = 1 | S | (cid:80) ni
Revised: L S ( h ) = 1 | S | (cid:80) ni

Original: =1 1[ h ( x i ) (cid:54) =
Revised: =1 1[ h ( x i ) (cid:54) =

Original: y i ] denote the empirical error of h with respect to S , and call a sample realizable by a class H if there exists h ∈ H such that L S ( h ) = 0 .
Revised: y i ] denote the empirical error of h with respect to S , and call a sample realizable by a class H if there exists h ∈ H such that L S ( h ) = 0 .

Original: The expected loss (also called risk) of a learning algorithm Lrn w.r.t a distribution D and sample size n is deﬁned by
Revised: The expected loss (also called risk) of a learning algorithm Lrn w.r.t a distribution D and sample size n is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: The function n (cid:55)→ ε n ( Lrn | D ) is called the learning curve , or learning rate of Lrn w.r.t D .
Revised: The function n (cid:55)→ ε n ( Lrn | D ) is called the learning curve , or learning rate of Lrn w.r.t D .

Original: For a real number r , let (cid:98) r (cid:101) denote the nearest integer to r .
Revised: For a real number r , let (cid:98) r (cid:101) denote the nearest integer to r .

Original: In case of ties, when r = k + 1 / 2 for some k ∈ Z , then deﬁne (cid:98) r (cid:101) =
Revised: In case of ties, when r = k + 1 / 2 for some k ∈ Z , then deﬁne (cid:98) r (cid:101) =

Original: k + 1 .
Revised: k + 1 .

Original: For any ﬁnite multiset H (cid:48)
Revised: For any ﬁnite multiset H (cid:48)

Original: ⊂ H , denote by Maj ( H (cid:48) )
Revised: ⊂ H , denote by Maj ( H (cid:48) )

Original: the function deﬁned for all x ∈ X by
Revised: the function deﬁned for all x ∈ X by

Original: [Equation]
Revised: [Equation]

Original: Adversarial risk and prediction stability.
Revised: Adversarial risk and prediction stability.

Original: Before we introduce the deﬁnition of Adversarial risk, we deﬁne Hamming distance between samples, which is a natural way to quantify distance between samples of equal size.
Revised: Before we introduce the deﬁnition of Adversarial risk, we deﬁne Hamming distance between samples, which is a natural way to quantify distance between samples of equal size.

Original: Deﬁnition 2.1 (Hamming distance between samples) .
Revised: Deﬁnition 2.1 (Hamming distance between samples) .

Original: Fix n ∈ N and let S, S (cid:

Revised: Fix n ∈ N and let S, S (cid:

Original: ) ∈ ( X × Y ) n .
Revised: ) ∈ ( X × Y ) n .

Original: We deﬁne the Hamming distance between S and S (cid:48) , denoted by d H ( S, S (cid:48) ) as follows:
Revised: We deﬁne the Hamming distance between S and S (cid:48) , denoted by d H ( S, S (cid:48) ) as follows:

Original: [Equation]
Revised: [Equation]

Original: i = 1[ S i (cid:54) = S (cid:48) i ] .
Revised: i = 1[ S i (cid:54) = S (cid:48) i ] .

Original: Note that the Hamming distance is deﬁned only for samples of equal sizes.
Revised: Note that the Hamming distance is deﬁned only for samples of equal sizes.

Original: If d H ( S, S (cid:48) ) ≤ η · n , we say that S, S (cid:48) are η -close.
Revised: If d H ( S, S (cid:48) ) ≤ η · n , we say that S, S (cid:48) are η -close.

Original: For any sample S , let
Revised: For any sample S , let

Original: B η ( S ) :
Revised: B η ( S ) :

Original: = (cid:8) S (cid:48) :
Revised: = (cid:8) S (cid:48) :

Original: d H ( S, S (cid:48) ) ≤ η · n (cid:9) .
Revised: d H ( S, S (cid:48) ) ≤ η · n (cid:9) .

Original: Deﬁnition 2.2 ( η -adversarial risk) .
Revised: Deﬁnition 2.2 ( η -adversarial risk) .

Original: Let η ∈ (0 , 1) be the adversary’s budget, let Lrn be a learning rule, and let D be a distribution over examples.
Revised: Let η ∈ (0 , 1) be the adversary’s budget, let Lrn be a learning rule, and let D be a distribution over examples.

Original: The η -adversarial risk of Lrn w.r.t D and sample size n is deﬁned by
Revised: The η -adversarial risk of Lrn w.r.t D and sample size n is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: Thus, robust learning with respect to instance-targeted poisoning with budget η boils down to minimizing the adversarial risk.
Revised: Thus, robust learning with respect to instance-targeted poisoning with budget η boils down to minimizing the adversarial risk.

Original: Indeed, given an input sample S and a test example ( x, y ) , an adversary with budget γ can force a mistake on x if and only if Lrn ( S (cid:48) )
Revised: Indeed, given an input sample S and a test example ( x, y ) , an adversary with budget γ can force a mistake on x if and only if Lrn ( S (cid:48) )

Original: ( x ) (cid:54) =
Revised: ( x ) (cid:54) =

Original: y for some S (cid:48)
Revised: y for some S (cid:48)

Original: ∈
Revised: ∈

Original: B η ( S ) .
Revised: B η ( S ) .

Original: Randomness.
Revised: Randomness.

Original: In Deﬁnition 2.2 above we deﬁne adversarial risk for the setting in which both the learner Lrn and the model h
Revised: In Deﬁnition 2.2 above we deﬁne adversarial risk for the setting in which both the learner Lrn and the model h

Original: = Lrn ( S ) are deterministic .
Revised: = Lrn ( S ) are deterministic .

Original: When either Lrn or h is allowed to use randomness, then the notion of adversarial risk as deﬁned in Deﬁnition 2.2 can be extended in several ways, depending on whether the adversary can see the randomness of the learner or not.
Revised: When either Lrn or h is allowed to use randomness, then the notion of adversarial risk as deﬁned in Deﬁnition 2.2 can be extended in several ways, depending on whether the adversary can see the randomness of the learner or not.

Original: Some of these variations are discussed in the work of Gao, Karbasi, and Mahmoody [2021].
Revised: Some of these variations are discussed in the work of Gao, Karbasi, and Mahmoody [2021].

Original: We remark however that our results in the realizable setting apply to all variations.
Revised: We remark however that our results in the realizable setting apply to all variations.

Original: This is simply because our upper bounds are achieved by deterministic learners, whereas our lower bound uses the weakest type of an adversary (which does not depend on the randomness of the learner).
Revised: This is simply because our upper bounds are achieved by deterministic learners, whereas our lower bound uses the weakest type of an adversary (which does not depend on the randomness of the learner).

Original: In contrast, our lower bound in the agnostic setting applies only to deterministic learners.
Revised: In contrast, our lower bound in the agnostic setting applies only to deterministic learners.

Original: Explicit bounds.
Revised: Explicit bounds.

Original: We do not try to optimize the constants hidden in the O ( · ) , Ω( · ) notation in the derived bounds.
Revised: We do not try to optimize the constants hidden in the O ( · ) , Ω( · ) notation in the derived bounds.

Original: The reason is because on the one hand, this way the proofs are simpler and more accessible, and on the other hand, we do not know how to get tight (or nearly tight) lower and upper bounds on the constants.
Revised: The reason is because on the one hand, this way the proofs are simpler and more accessible, and on the other hand, we do not know how to get tight (or nearly tight) lower and upper bounds on the constants.

Original: Obtaining tight bounds is a natural direction for future research; we elaborate on this in Section 5.
Revised: Obtaining tight bounds is a natural direction for future research; we elaborate on this in Section 5.

Original: Nevertheless, the complete proofs (which are given in the appendix) include explicit numerical bounds on the constants.
Revised: Nevertheless, the complete proofs (which are given in the appendix) include explicit numerical bounds on the constants.

Original: Decoupling adversarial risk into stability and risk.
Revised: Decoupling adversarial risk into stability and risk.

Original: It is convenient and illustrative to decouple robust learnability to two properties: small expected loss and prediction stability.
Revised: It is convenient and illustrative to decouple robust learnability to two properties: small expected loss and prediction stability.

Original: The latter means that the prediction of the learning algorithm on a random test point is stable under replacing a bounded amount of examples from the training set:
Revised: The latter means that the prediction of the learning algorithm on a random test point is stable under replacing a bounded amount of examples from the training set:

Original: Deﬁnition 2.3 (Prediction stability) .
Revised: Deﬁnition 2.3 (Prediction stability) .

Original: Let n ∈ N , σ, η ∈ (0 , 1) .
Revised: Let n ∈ N , σ, η ∈ (0 , 1) .

Original: Let Lrn be a learning rule and D be a distribution over examples.
Revised: Let Lrn be a learning rule and D be a distribution over examples.

Original: We say that the learning rule Lrn is ( n, σ, η ) -prediction stable with respect to D if the following holds
Revised: We say that the learning rule Lrn is ( n, σ, η ) -prediction stable with respect to D if the following holds

Original: [Equation]
Revised: [Equation]

Original: where D x is the marginal distribution induced by D on the domain X .
Revised: where D x is the marginal distribution induced by D on the domain X .

Original: Of course, prediction stability alone does not guarantee robust learning.
Revised: Of course, prediction stability alone does not guarantee robust learning.

Original: Indeed, useless learning rule that always outputs the all 0 ’s classiﬁer has maximal stability.
Revised: Indeed, useless learning rule that always outputs the all 0 ’s classiﬁer has maximal stability.

Original: At the very least, the learning rule should learn the class in the classical sense (in the absence of an adversary).
Revised: At the very least, the learning rule should learn the class in the classical sense (in the absence of an adversary).

Original: The following observation asserts that prediction-stable learning rules with small loss are robust learners:
Revised: The following observation asserts that prediction-stable learning rules with small loss are robust learners:

Original: Observation 2.4 (Prediction stability + small error = robust learning) .
Revised: Observation 2.4 (Prediction stability + small error = robust learning) .

Original: Let Lrn be a learner and D a distribution over examples.
Revised: Let Lrn be a learner and D a distribution over examples.

Original: Then,
Revised: Then,

Original: [Equation]
Revised: [Equation]

Original: In other words, if Lrn is ( n, σ, η ) -prediction stable with respect to D whose expected population loss is ε n ( Lrn | D ) ≤ ε .
Revised: In other words, if Lrn is ( n, σ, η ) -prediction stable with respect to D whose expected population loss is ε n ( Lrn | D ) ≤ ε .

Original: Then Lrn learns D with an adversarial expected loss σ + (cid:15) .
Revised: Then Lrn learns D with an adversarial expected loss σ + (cid:15) .

Original: Conversely, if ε n ( Lrn | D, η ) ≤ ε then Lrn is ( n, ε, η ) -prediction stable with respect to D and its expected population loss is also ε n ( Lrn | D ) ≤ ε .
Revised: Conversely, if ε n ( Lrn | D, η ) ≤ ε then Lrn is ( n, ε, η ) -prediction stable with respect to D and its expected population loss is also ε n ( Lrn | D ) ≤ ε .

Original: We leave the (simple) proof of Observation 2.4 to the reader.
Revised: We leave the (simple) proof of Observation 2.4 to the reader.

Original:  Realizable Setting
Revised:  Realizable Setting

Original: Theorems 3.1 and 3.3 below characterize the optimal adversarial risk in the realizable setting.
Revised: Theorems 3.1 and 3.3 below characterize the optimal adversarial risk in the realizable setting.

Original: Theorem 3.1 (Realizable case – positive result) .
Revised: Theorem 3.1 (Realizable case – positive result) .

Original: There exists a constant c 1 > 0 so that the following holds.
Revised: There exists a constant c 1 > 0 so that the following holds.

Original: Let H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .
Revised: Let H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .

Original: Then there exists a learner Lrn having η -adversarial risk
Revised: Then there exists a learner Lrn having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: for any distribution D realizable by H and for any sample size n ≥ 1 /η .
Revised: for any distribution D realizable by H and for any sample size n ≥ 1 /η .

Original: We prove Theorem 3.1 in Appendix A.
Revised: We prove Theorem 3.1 in Appendix A.

Original: Note that the requirement that the sample size is n ≥ 1 /η is necessary since otherwise η · n < 1 , which means that the adversary cannot modify the input sample, and so this case reduces to classical learning without an adversary.
Revised: Note that the requirement that the sample size is n ≥ 1 /η is necessary since otherwise η · n < 1 , which means that the adversary cannot modify the input sample, and so this case reduces to classical learning without an adversary.

Original: where n ≥ 1 /η .
Revised: where n ≥ 1 /η .

Original: Partition S into (cid:100) 7 ηn (cid:101) consecutive subsamples such that all ﬁrst t = (cid:98) 7 ηn (cid:99) subsamples are of size at least 17 η .
Revised: Partition S into (cid:100) 7 ηn (cid:101) consecutive subsamples such that all ﬁrst t = (cid:98) 7 ηn (cid:99) subsamples are of size at least 17 η .

Original: Denote the i ’th subsample by S ( i ) .
Revised: Denote the i ’th subsample by S ( i ) .

Original: For all i ∈ [ t ] , run the learning algorithm Lrn on S ( i ) to obtain a hypothesis h i =
Revised: For all i ∈ [ t ] , run the learning algorithm Lrn on S ( i ) to obtain a hypothesis h i =

Original: Lrn ( S ( i ) ) .
Revised: Lrn ( S ( i ) ) .

Original: Return the hypothesis h deﬁned as follows for all x ∈ X :
Revised: Return the hypothesis h deﬁned as follows for all x ∈ X :

Original: [Equation]
Revised: [Equation]

Original: Theorem 3.1 is proven using the S TABLE P ARTITION AND V OTE (or SPV , for short) meta-algorithm, described in Figure 1.
Revised: Theorem 3.1 is proven using the S TABLE P ARTITION AND V OTE (or SPV , for short) meta-algorithm, described in Figure 1.

Original: The meta-algorithm is based on the idea of partitioning and then voting used in [Gao, Karbasi, and Mahmoody, 2021], but with a more reﬁned and precise analysis.
Revised: The meta-algorithm is based on the idea of partitioning and then voting used in [Gao, Karbasi, and Mahmoody, 2021], but with a more reﬁned and precise analysis.

Original: The partition and vote technique works as follows.
Revised: The partition and vote technique works as follows.

Original: First, partition the input sample to subsamples of a carefully chosen size.
Revised: First, partition the input sample to subsamples of a carefully chosen size.

Original: Then, train a given learner (which is called the input learner of SPV ) on each subsample, and ﬁnally let the trained learners vote to determine the output label.
Revised: Then, train a given learner (which is called the input learner of SPV ) on each subsample, and ﬁnally let the trained learners vote to determine the output label.

Original: The size of each subsample trades-off, in a way, expected loss and prediction-stability: if it is too small, the given learner will perform poorly on each subsample.
Revised: The size of each subsample trades-off, in a way, expected loss and prediction-stability: if it is too small, the given learner will perform poorly on each subsample.

Original: On the other hand, if it is relatively large then the number of learners that participate in the majority vote is small and the adversary can poison a large fraction of these learners and ﬂip the overall majority vote.
Revised: On the other hand, if it is relatively large then the number of learners that participate in the majority vote is small and the adversary can poison a large fraction of these learners and ﬂip the overall majority vote.

Original: We elaborate on this when proving Theorem 3.1.
Revised: We elaborate on this when proving Theorem 3.1.

Original: Notice that the time complexity of SPV is proportional to the time complexity of the learner Lrn .
Revised: Notice that the time complexity of SPV is proportional to the time complexity of the learner Lrn .

Original: To state the complementing impossibility result, we need the following deﬁnition of non-trivial concept classes [Bshouty, Eiron, and Kushilevitz, 2002].
Revised: To state the complementing impossibility result, we need the following deﬁnition of non-trivial concept classes [Bshouty, Eiron, and Kushilevitz, 2002].

Original: Deﬁnition 3.2 (Non-trivial concept classes) .
Revised: Deﬁnition 3.2 (Non-trivial concept classes) .

Original: We say that a concept class H over a domain X is non-trivial , if there are x 1 , x 2 ∈ X and h 1 , h 2 ∈ H
Revised: We say that a concept class H over a domain X is non-trivial , if there are x 1 , x 2 ∈ X and h 1 , h 2 ∈ H

Original: so that h 1 ( x 1 ) =
Revised: so that h 1 ( x 1 ) =

Original: h 2 ( x 1 ) and h 1 ( x 2 ) (cid:54)
Revised: h 2 ( x 1 ) and h 1 ( x 2 ) (cid:54)

Original: =
Revised: =

Original: h 2 ( x 2 ) .
Revised: h 2 ( x 2 ) .

Original: Theorem 3.3 (Realizable case – impossibility result) .
Revised: Theorem 3.3 (Realizable case – impossibility result) .

Original: There exists a constant c 2 > 0 so that the following holds.
Revised: There exists a constant c 2 > 0 so that the following holds.

Original: Let H be a non-trivial hypothesis class with VC dimension d and let η ∈ (0 , 1) .
Revised: Let H be a non-trivial hypothesis class with VC dimension d and let η ∈ (0 , 1) .

Original: Then, there exists a distribution D realizable by H , so that every learner Lrn has η -adversarial risk
Revised: Then, there exists a distribution D realizable by H , so that every learner Lrn has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: for any sample size n ≥ 1 /η .
Revised: for any sample size n ≥ 1 /η .

Original: We note that this impossibility result applies also to a variety of randomized learners; we elaborate on this in Appendix B, where we also prove Theorem 3.3.
Revised: We note that this impossibility result applies also to a variety of randomized learners; we elaborate on this in Appendix B, where we also prove Theorem 3.3.

Original: The above lower bound demonstrates how vulnerability to instance-targeted attacks depends greatly on the hypothesis class we want to learn, and speciﬁcally on its VC-dimension.
Revised: The above lower bound demonstrates how vulnerability to instance-targeted attacks depends greatly on the hypothesis class we want to learn, and speciﬁcally on its VC-dimension.

Original: 3.1 Certiﬁcation
Revised: 3.1 Certiﬁcation

Original: Besides prediction-stability, another useful property our SPV meta-algorithm has is the ability to efﬁciently calculate and output a certiﬁcate for the stability of its predictions.
Revised: Besides prediction-stability, another useful property our SPV meta-algorithm has is the ability to efﬁciently calculate and output a certiﬁcate for the stability of its predictions.

Original: Formally, given an input sample S , a certiﬁcate is a function η S : X → [0 , 1] , outputted by a learner in addition to its output hypothesis h S such that the following is satisﬁed: h S ( x ) = h S (cid:48) ( x ) for every point x and for every input sample S (cid:48) which is η S ( x ) -close to S .
Revised: Formally, given an input sample S , a certiﬁcate is a function η S : X → [0 , 1] , outputted by a learner in addition to its output hypothesis h S such that the following is satisﬁed: h S ( x ) = h S (cid:48) ( x ) for every point x and for every input sample S (cid:48) which is η S ( x ) -close to S .

Original: If one ignores computational considerations, outputting optimal certiﬁcates is always possible:
Revised: If one ignores computational considerations, outputting optimal certiﬁcates is always possible:

Original: Deﬁnition 3.4 (Optimal Certiﬁcate) .
Revised: Deﬁnition 3.4 (Optimal Certiﬁcate) .

Original: Let Lrn be any learning rule, and let S be an input sample.
Revised: Let Lrn be any learning rule, and let S be an input sample.

Original: Deﬁne the optimal certiﬁcate η (cid:63) ( · ) =
Revised: Deﬁne the optimal certiﬁcate η (cid:63) ( · ) =

Original: η (cid:63) ( ·| S ) of Lrn on input sample S as follows.
Revised: η (cid:63) ( ·| S ) of Lrn on input sample S as follows.

Original: The optimal certiﬁcate η (cid:63) ( x | S ) is equal to kn where k is the largest integer for which Lrn ( S (cid:48) )
Revised: The optimal certiﬁcate η (cid:63) ( x | S ) is equal to kn where k is the largest integer for which Lrn ( S (cid:48) )

Original: ( x ) =
Revised: ( x ) =

Original: Lrn ( S )( x ) for every sample S (cid:48) with hamming distance at most k from S .
Revised: Lrn ( S )( x ) for every sample S (cid:48) with hamming distance at most k from S .

Original: PSPV
Revised: PSPV

Original: Input: Stability parameter η ∈ (0 , 1) , a proper learning algorithm Lrn p and an input sample
Revised: Input: Stability parameter η ∈ (0 , 1) , a proper learning algorithm Lrn p and an input sample

Original: S ∼ D n where n ≥ 1 /η .
Revised: S ∼ D n where n ≥ 1 /η .

Original: Partition S into (cid:100) 5 k p ηn (cid:101) consecutive subsamples such that all ﬁrst t = (cid:98) 5 k p ηn (cid:99) subsamples are of size at least
Revised: Partition S into (cid:100) 5 k p ηn (cid:101) consecutive subsamples such that all ﬁrst t = (cid:98) 5 k p ηn (cid:99) subsamples are of size at least

Original: k
Revised: k

Original: Denote the i ’th subsample by S ( i ) .
Revised: Denote the i ’th subsample by S ( i ) .

Original: For all i ∈ [ t ] , train Lrn p on S ( i ) to obtain a hypothesis h
Revised: For all i ∈ [ t ] , train Lrn p on S ( i ) to obtain a hypothesis h

Original: i
Revised: i

Original: =
Revised: =

Original: Lrn p ( S ( i ) ) .
Revised: Lrn p ( S ( i ) ) .

Original: Return h ∈ H such that
Revised: Return h ∈ H such that

Original: In other words, if S is a sample that was corrupted by an adversary with budget η such that η ≤ η (cid:63) ( x | S ) then the output label Lrn ( S )( x ) is equal to the label that would have been outputted if the learner was trained with the uncorrupted sample.
Revised: In other words, if S is a sample that was corrupted by an adversary with budget η such that η ≤ η (cid:63) ( x | S ) then the output label Lrn ( S )( x ) is equal to the label that would have been outputted if the learner was trained with the uncorrupted sample.

Original: The issue with the optimal certiﬁcate η (cid:63) ( x ) is that it can be impossible to compute as it requires to iterate over the potentially inﬁnite space of all samples S (cid:48) of hamming distance at most n · η ( x ) from the input sample S .
Revised: The issue with the optimal certiﬁcate η (cid:63) ( x ) is that it can be impossible to compute as it requires to iterate over the potentially inﬁnite space of all samples S (cid:48) of hamming distance at most n · η ( x ) from the input sample S .

Original: In contrast, our SPV learner can efﬁciently calculate a non-trivial lower bound on η (cid:63) which therefore also serves as a certiﬁcate.
Revised: In contrast, our SPV learner can efﬁciently calculate a non-trivial lower bound on η (cid:63) which therefore also serves as a certiﬁcate.

Original: The key property which enables this is the fact that its output hypothesis is the majority vote of base learners, each trained on a disjoint subsample.
Revised: The key property which enables this is the fact that its output hypothesis is the majority vote of base learners, each trained on a disjoint subsample.

Original: This is summarized in the following proposition:
Revised: This is summarized in the following proposition:

Original: Proposition 3.5.
Revised: Proposition 3.5.

Original: Consider a learner whose output hypothesis is given by a majority vote of t learners
Revised: Consider a learner whose output hypothesis is given by a majority vote of t learners

Original: L 1 , . . .
Revised: L 1 , . . .

Original: , L t that are trained on t disjoint subsamples S 1 , . . .
Revised: , L t that are trained on t disjoint subsamples S 1 , . . .

Original: , S t of the input sample S .
Revised: , S t of the input sample S .

Original: Deﬁne
Revised: Deﬁne

Original: [Equation]
Revised: [Equation]

Original: where h i is the output hypothesis of L i , y is the output label of the majority vote of the L i ’s, and n is the size of the input sample S .
Revised: where h i is the output hypothesis of L i , y is the output label of the majority vote of the L i ’s, and n is the size of the input sample S .

Original: Then, η ( x | S ) ≤ η (cid:63) ( x | S ) .
Revised: Then, η ( x | S ) ≤ η (cid:63) ( x | S ) .

Original: Proof.
Revised: Proof.

Original: Notice that n · η ( x | S ) + 1 is equal to the minimal number of h i ’s whose prediction on x must be ﬂipped in order to enforce that
Revised: Notice that n · η ( x | S ) + 1 is equal to the minimal number of h i ’s whose prediction on x must be ﬂipped in order to enforce that

Original: [Equation]
Revised: [Equation]

Original: Therefore, at least one example in each S i such that h i ( x ) =
Revised: Therefore, at least one example in each S i such that h i ( x ) =

Original: y must be replaced in order to change the prediction of Lrn ( S ) on x .
Revised: y must be replaced in order to change the prediction of Lrn ( S ) on x .

Original: In particular, if only n · η ( x | S ) examples are replaced than the prediction of Lrn ( S ) on x remains the same.
Revised: In particular, if only n · η ( x | S ) examples are replaced than the prediction of Lrn ( S ) on x remains the same.

Original: This implies that η (cid:63) ( x | S ) ≥ η ( x | S ) as stated.
Revised: This implies that η (cid:63) ( x | S ) ≥ η ( x | S ) as stated.

Original: In light of Proposition 3.5, our SPV learner can efﬁciently compute and output a certiﬁcate η ( x ) which is proportional to η (where η is the stability parameter given to SPV ), with probability proportional to the expected loss of the input learner given to SPV when executed on a sample of size (cid:108) 17 η (cid:109) .
Revised: In light of Proposition 3.5, our SPV learner can efﬁciently compute and output a certiﬁcate η ( x ) which is proportional to η (where η is the stability parameter given to SPV ), with probability proportional to the expected loss of the input learner given to SPV when executed on a sample of size (cid:108) 17 η (cid:109) .

Original: 3.2 A Proper Variant of SPV
Revised: 3.2 A Proper Variant of SPV

Original: We now present a proper version of SPV for classes H with a ﬁnite projection number , described in Figure 2.
Revised: We now present a proper version of SPV for classes H with a ﬁnite projection number , described in Figure 2.

Original: The projection number of a concept class H is denoted by k p = k p
Revised: The projection number of a concept class H is denoted by k p = k p

Original: ( H ) (we present its deﬁnition after the statement of Theorem 3.6 below).
Revised: ( H ) (we present its deﬁnition after the statement of Theorem 3.6 below).

Original: In particular, for the class of halfspaces it yields a robust learner with the following guarantee:
Revised: In particular, for the class of halfspaces it yields a robust learner with the following guarantee:

Original: Theorem 3.6.
Revised: Theorem 3.6.

Original: There exists a constant c > 0 so that the following holds.
Revised: There exists a constant c > 0 so that the following holds.

Original: Let H be the class of halfspaces over R d for some d ≥ 1 , and let η ∈ (0 , 1) .
Revised: Let H be the class of halfspaces over R d for some d ≥ 1 , and let η ∈ (0 , 1) .

Original: Then, there exists a proper learner Lrn having η -adversarial risk
Revised: Then, there exists a proper learner Lrn having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: for any distribution D realizable by H and for any sample size n ≥ 1 /η .
Revised: for any distribution D realizable by H and for any sample size n ≥ 1 /η .

Original: The proof of Theorem 3.6 is deferred to Appendix C.
Revised: The proof of Theorem 3.6 is deferred to Appendix C.

Original: To derive Theorem 3.6, we reinforce the SPV algorithm with a technique introduced by Kane,
Revised: To derive Theorem 3.6, we reinforce the SPV algorithm with a technique introduced by Kane,

Original: Livni, Moran, and Yehudayoff [2019] and further developed by Bousquet, Hanneke, Moran, and
Revised: Livni, Moran, and Yehudayoff [2019] and further developed by Bousquet, Hanneke, Moran, and

Original: Zhivotovskiy [2020].
Revised: Zhivotovskiy [2020].

Original: This technique allows in certain cases to project a majority vote of hypotheses from the class H back to H .
Revised: This technique allows in certain cases to project a majority vote of hypotheses from the class H back to H .

Original: Its applicability hinges on a combinatorial parameter called the projection number .
Revised: Its applicability hinges on a combinatorial parameter called the projection number .

Original: The PSPV learner explicitly uses the projection number, so for completeness we give its deﬁnition below.
Revised: The PSPV learner explicitly uses the projection number, so for completeness we give its deﬁnition below.

Original: The interested may see the work of Bousquet, Hanneke, Moran, and Zhivotovskiy [2020] for an insightful discussion on the role of the projection number in proper learning.
Revised: The interested may see the work of Bousquet, Hanneke, Moran, and Zhivotovskiy [2020] for an insightful discussion on the role of the projection number in proper learning.

Original: Deﬁnition 3.7 (Projection Number) .
Revised: Deﬁnition 3.7 (Projection Number) .

Original: Let H be a concept class.
Revised: Let H be a concept class.

Original: For any (cid:96) ≥ 2 and for any multiset
Revised: For any (cid:96) ≥ 2 and for any multiset

Original: H (cid:48)
Revised: H (cid:48)

Original: ⊂
Revised: ⊂

Original: H deﬁne the set X H (cid:48) ,(cid:96) to be the set of all x ∈ X , for which the number of hypotheses in H (cid:48) that disagree with Maj ( H (cid:48) )
Revised: H deﬁne the set X H (cid:48) ,(cid:96) to be the set of all x ∈ X , for which the number of hypotheses in H (cid:48) that disagree with Maj ( H (cid:48) )

Original: ( x ) is less than |H (cid:48) | /(cid:96) .
Revised: ( x ) is less than |H (cid:48) | /(cid:96) .

Original: The Projection Number of the class H , denoted k p = k p ( H ) , is deﬁned to be the smallest (cid:96) so that for any ﬁnite multiset H (cid:48)
Revised: The Projection Number of the class H , denoted k p = k p ( H ) , is deﬁned to be the smallest (cid:96) so that for any ﬁnite multiset H (cid:48)

Original: ⊂ H , there exist h ∈ H such that h ( x ) =
Revised: ⊂ H , there exist h ∈ H such that h ( x ) =

Original: Maj ( H (cid:48) )
Revised: Maj ( H (cid:48) )

Original: ( x ) for all x ∈ X H (cid:48) ,(cid:96) .
Revised: ( x ) for all x ∈ X H (cid:48) ,(cid:96) .

Original: If no such (cid:96) exists then k p = ∞ .
Revised: If no such (cid:96) exists then k p = ∞ .

Original:  Agnostic Setting
Revised:  Agnostic Setting

Original: In this section, we extend the results on robust learnability to the agnostic case.
Revised: In this section, we extend the results on robust learnability to the agnostic case.

Original: First, by a simple generalization of the positive result for the realizable case, we provide a robust semi-agnostic learner.
Revised: First, by a simple generalization of the positive result for the realizable case, we provide a robust semi-agnostic learner.

Original: That is, our learner has adversarial risk depending linearly on OPT =
Revised: That is, our learner has adversarial risk depending linearly on OPT =

Original: OPT ( H , D ) :
Revised: OPT ( H , D ) :

Original: = min h ∈H L D ( h ) .
Revised: = min h ∈H L D ( h ) .

Original: While semi-agnostic learning is considered not ideal in many cases, we complement our positive result by showing that semi-agnostic learning is unavoidable when the goal is to design a robust and deterministic (as ours) learner for the agnostic setting.
Revised: While semi-agnostic learning is considered not ideal in many cases, we complement our positive result by showing that semi-agnostic learning is unavoidable when the goal is to design a robust and deterministic (as ours) learner for the agnostic setting.

Original: 4.1 A Semi-agnostic Learner
Revised: 4.1 A Semi-agnostic Learner

Original: Formally, a semi agnostic learner is deﬁned as follows.
Revised: Formally, a semi agnostic learner is deﬁned as follows.

Original: Let c ∈ R .
Revised: Let c ∈ R .

Original: A learning rule Lrn is a csemi agnostic learner if the following holds.
Revised: A learning rule Lrn is a csemi agnostic learner if the following holds.

Original: Let H be a concept class and let D be a distribution over examples.
Revised: Let H be a concept class and let D be a distribution over examples.

Original: Then there exists an excess error rate ε Agn : N → [0 , 1] such that ε n ( Lrn | D ) ≤ c OPT + ε Agn ( n ) where
Revised: Then there exists an excess error rate ε Agn : N → [0 , 1] such that ε n ( Lrn | D ) ≤ c OPT + ε Agn ( n ) where

Original: OPT = inf h ∈H L D ( h ) .
Revised: OPT = inf h ∈H L D ( h ) .

Original: Before stating our positive result in this setting, we ﬁrst discuss how achieving adversarial risk
Revised: Before stating our positive result in this setting, we ﬁrst discuss how achieving adversarial risk

Original: O ( d ( OPT + η ) is possible by reduction to the realizable setting.
Revised: O ( d ( OPT + η ) is possible by reduction to the realizable setting.

Original: Reduction to the realizable setting.
Revised: Reduction to the realizable setting.

Original: Suppose a learner is given a training set S (cid:48) of size n that comes with ηn replacements made by the adversary on the original set S .
Revised: Suppose a learner is given a training set S (cid:48) of size n that comes with ηn replacements made by the adversary on the original set S .

Original: Moreover, suppose that S is sampled from a distribution D such that the best h ∈ H has OPT error on D .
Revised: Moreover, suppose that S is sampled from a distribution D such that the best h ∈ H has OPT error on D .

Original: This means that, roughly OPT fraction of S does not match to h .
Revised: This means that, roughly OPT fraction of S does not match to h .

Original: Therefore, one can see S (cid:48) as ﬁrst sampled from D (without noise) followed by ≈ ( η + OPT ) · n replacement corruptions.
Revised: Therefore, one can see S (cid:48) as ﬁrst sampled from D (without noise) followed by ≈ ( η + OPT ) · n replacement corruptions.

Original: This way, one can employ a learner that can tolerate η (cid:48)
Revised: This way, one can employ a learner that can tolerate η (cid:48)

Original: =
Revised: =

Original: η + OPT fraction of adversarial corruptions in the realizable setting and obtain total adversarial risk
Revised: η + OPT fraction of adversarial corruptions in the realizable setting and obtain total adversarial risk

Original: O ( d ( OPT + η ) .
Revised: O ( d ( OPT + η ) .

Original: The above discussion raises a natural question: can a learner achieve adversarial risk O ( OPT + dη ) or even (ideally) OPT + O ( dη ) ?
Revised: The above discussion raises a natural question: can a learner achieve adversarial risk O ( OPT + dη ) or even (ideally) OPT + O ( dη ) ?

Original: The latter is the typical type of risk bound in agnostic settings, where there is no multiplicative dependence on OPT in the risk.
Revised: The latter is the typical type of risk bound in agnostic settings, where there is no multiplicative dependence on OPT in the risk.

Original: The following theorem, which we prove in Appendix D states the positive result.
Revised: The following theorem, which we prove in Appendix D states the positive result.

Original: Theorem 4.1 (Positive result for the agnostic case) .
Revised: Theorem 4.1 (Positive result for the agnostic case) .

Original: There exist constants c 1 , c 2 so that the following holds.
Revised: There exist constants c 1 , c 2 so that the following holds.

Original: Let H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .
Revised: Let H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .

Original: Then, there exists a learner Lrn having η -adversarial risk
Revised: Then, there exists a learner Lrn having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: for any distribution D over examples and for any sample size n ≥ 1 /η .
Revised: for any distribution D over examples and for any sample size n ≥ 1 /η .

Original: As in the realizable case upper bound, the above upper bound is proved by using the SPV meta-learner.
Revised: As in the realizable case upper bound, the above upper bound is proved by using the SPV meta-learner.

Original: The main difference is that to prove this result we use a different input learner Lrn given to SPV than the one we use in the realizable case.
Revised: The main difference is that to prove this result we use a different input learner Lrn given to SPV than the one we use in the realizable case.

Original: 4.2 Ruling Out Agnostic Learning
Revised: 4.2 Ruling Out Agnostic Learning

Original: Note that Theorem 4.1 only proves the existence of a semi -agnostic learner under instance-targeted poisoning.
Revised: Note that Theorem 4.1 only proves the existence of a semi -agnostic learner under instance-targeted poisoning.

Original: A more desirable goal would be to obtain (standard) agnostic learners whose error under the attack is OPT + ψ where ψ is a vanishing (additive) error term when η → 0 .
Revised: A more desirable goal would be to obtain (standard) agnostic learners whose error under the attack is OPT + ψ where ψ is a vanishing (additive) error term when η → 0 .

Original: Here we will prove that at least when it comes to deterministic learners, such a goal is out of reach, and the best we can hope for is 2 OPT plus additive terms that depend on η and the VC dimension.
Revised: Here we will prove that at least when it comes to deterministic learners, such a goal is out of reach, and the best we can hope for is 2 OPT plus additive terms that depend on η and the VC dimension.

Original: This explains why we can only achieve a semi-agnostic learner.
Revised: This explains why we can only achieve a semi-agnostic learner.

Original: The following theorem, which we prove in Appendix E shows that in Theorem 4.1, the constant cneeds to be at least 2 , and so the standard way agnostic learners bound their regret is not possible for instance-targeted poisoning.
Revised: The following theorem, which we prove in Appendix E shows that in Theorem 4.1, the constant cneeds to be at least 2 , and so the standard way agnostic learners bound their regret is not possible for instance-targeted poisoning.

Original: Theorem 4.2 (Impossibility of agnostic learning) .
Revised: Theorem 4.2 (Impossibility of agnostic learning) .

Original: Let η (cid:48) ∈ (0 , 1) , n ∈ N .
Revised: Let η (cid:48) ∈ (0 , 1) , n ∈ N .

Original: For any hypothesis class
Revised: For any hypothesis class

Original: H that has at least two hypotheses and for any deterministic learner, there is a distribution D over (two) examples and η = η (cid:48)
Revised: H that has at least two hypotheses and for any deterministic learner, there is a distribution D over (two) examples and η = η (cid:48)

Original: + (cid:101) O
Revised: + (cid:101) O

Original: (1 / √ n ) such that Lrn has η -adversarial risk
Revised: (1 / √ n ) such that Lrn has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original:  Conclusion and Open Questions
Revised:  Conclusion and Open Questions

Original: In this work, we studied the optimal rate of learning for binary classiﬁcation problems under instancetargeted poisoning.
Revised: In this work, we studied the optimal rate of learning for binary classiﬁcation problems under instancetargeted poisoning.

Original: We showed that in the realizable setting the error rate can be characterized up to a constant factor and is proportional both to adversary’s budget and the VC dimension of the class.
Revised: We showed that in the realizable setting the error rate can be characterized up to a constant factor and is proportional both to adversary’s budget and the VC dimension of the class.

Original: In the agnostic setting, we proved a perhaps surprising lower bound that standard agnostic learning (with additive regret compared to the optimal error in the no-attack setting) is impossible for deterministic learners, and also complemented this with a positive result using a semi-agnostic learner.
Revised: In the agnostic setting, we proved a perhaps surprising lower bound that standard agnostic learning (with additive regret compared to the optimal error in the no-attack setting) is impossible for deterministic learners, and also complemented this with a positive result using a semi-agnostic learner.

Original: We also showed how to make our learners proper in a variety of interesting settings.
Revised: We also showed how to make our learners proper in a variety of interesting settings.

Original: Our work leaves a few interesting directions for future research.
Revised: Our work leaves a few interesting directions for future research.

Original: • Finding the exact constant in the realizable case.
Revised: • Finding the exact constant in the realizable case.

Original: Our results in the realizable case characterize the optimal adversarial risk up to a constant multiplicative factor in the sense that there exist constants c 1 , c 2 so that achieving η -adversarial risk of c 1 ηd is possible for any hypothesis class with VC-dimension d , whereas obtaining η -adversarial risk of c 2 ηd can’t be achieved for any hypothesis class with VC-dimension d .
Revised: Our results in the realizable case characterize the optimal adversarial risk up to a constant multiplicative factor in the sense that there exist constants c 1 , c 2 so that achieving η -adversarial risk of c 1 ηd is possible for any hypothesis class with VC-dimension d , whereas obtaining η -adversarial risk of c 2 ηd can’t be achieved for any hypothesis class with VC-dimension d .

Original: However, there is a large gap between c 1 , c 2 .
Revised: However, there is a large gap between c 1 , c 2 .

Original: Can we close or shrink this gap?
Revised: Can we close or shrink this gap?

Original: • Finding the correct multiplicative factor in the agnostic case.
Revised: • Finding the correct multiplicative factor in the agnostic case.

Original: Our results show that in the agnostic case, there must be a constant C ≥ 2 so that the best adversarial risk attainable is C · OPT .
Revised: Our results show that in the agnostic case, there must be a constant C ≥ 2 so that the best adversarial risk attainable is C · OPT .

Original: What is the value of C ?
Revised: What is the value of C ?

Original: Characterizing proper robust learning.
Revised: Characterizing proper robust learning.

Original: In the proper and realizable case, our stable learner for linear classiﬁers depends on d 3 , while our lower bound depends linearly on d , as in the general improper case.
Revised: In the proper and realizable case, our stable learner for linear classiﬁers depends on d 3 , while our lower bound depends linearly on d , as in the general improper case.

Original: It remains open to identify the correct dependence on d . • Characterizing the role of randomness.
Revised: It remains open to identify the correct dependence on d . • Characterizing the role of randomness.

Original: Our impossibility result for the agnostic learning (Theorem 4.2) only applies to deterministic learners.
Revised: Our impossibility result for the agnostic learning (Theorem 4.2) only applies to deterministic learners.

Original: It remains open to either effectively use randomness during the learning (known or unknown to the adversary) and obtain an agnostic learner, or to extend the negative result to cover such randomized learners as well.
Revised: It remains open to either effectively use randomness during the learning (known or unknown to the adversary) and obtain an agnostic learner, or to extend the negative result to cover such randomized learners as well.

Original: Acknowledgments
Revised: Acknowledgments

Original: Amin Karbasi acknowledges funding in direct support of this work from NSF (IIS-1845032), ONR (N00014- 19-1-2406), and the AI Institute for Learning-Enabled Optimization at Scale (TILOS).
Revised: Amin Karbasi acknowledges funding in direct support of this work from NSF (IIS-1845032), ONR (N00014- 19-1-2406), and the AI Institute for Learning-Enabled Optimization at Scale (TILOS).

Original: Mohammad Mahmoody is supported by NSF grants CCF-1910681 and CNS1936799.
Revised: Mohammad Mahmoody is supported by NSF grants CCF-1910681 and CNS1936799.

Original: Shay Moran is a Robert J. Shillman Fellow; he acknowledges support by ERC grant 802599, by ISF grant 1225/20, by BSF grant 2018385,
Revised: Shay Moran is a Robert J. Shillman Fellow; he acknowledges support by ERC grant 802599, by ISF grant 1225/20, by BSF grant 2018385,

Original: A Proof of Theorem 3.1 (Realizable Case – Positive Result)
Revised: A Proof of Theorem 3.1 (Realizable Case – Positive Result)

Original: Theorem (Restatement of Theorem 3.1) .
Revised: Theorem (Restatement of Theorem 3.1) .

Original: There exists a constant c 1 > 0 so that the following holds.
Revised: There exists a constant c 1 > 0 so that the following holds.

Original: Let H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .
Revised: Let H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .

Original: Then there exists a learner Lrn having η -adversarial risk
Revised: Then there exists a learner Lrn having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: for any distribution D realizable by H and for any sample size n ≥ 1 /η .
Revised: for any distribution D realizable by H and for any sample size n ≥ 1 /η .

Original: To prove Theorem 3.1, we will use the S TABLE PARTITION AND VOTE (or SPV for short) meta learner described in Figure 1 with the One-inclusion graph algorithm of Haussler, Littlestone, and
Revised: To prove Theorem 3.1, we will use the S TABLE P V (or SPV for short) meta learner described in Figure 1 with the One-inclusion graph algorithm of Haussler, Littlestone, and

Original: Warmuth [1994] as the input learner.
Revised: Warmuth [1994] as the input learner.

Original: First, we prove a more general result on the performance of our SPV meta learner.
Revised: First, we prove a more general result on the performance of our SPV meta learner.

Original: We denote the algorithm obtained by executing SPV with a learner Lrn as the input algorithm by SPV ( Lrn ) .
Revised: We denote the algorithm obtained by executing SPV with a learner Lrn as the input algorithm by SPV ( Lrn ) .

Original: Lemma A.1 (General performance of SPV ) .
Revised: Lemma A.1 (General performance of SPV ) .

Original: Let H be a concept class, D be a distribution over examples, and Lrn be a learning rule.
Revised: Let H be a concept class, D be a distribution over examples, and Lrn be a learning rule.

Original: Let also η ∈ (0 , 1) be the stability parameter given to SPV and let n ≥ 1 /η be the sample size.
Revised: Let also η ∈ (0 , 1) be the stability parameter given to SPV and let n ≥ 1 /η be the sample size.

Original: Then SPV ( Lrn ) has η -adversarial risk
Revised: Then SPV ( Lrn ) has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: Recall that ε (cid:100) 1 / (7 η ) (cid:101) ( Lrn | D ) is the expected population loss of Lrn when trained on a sample of size (cid:100) 1 / (7 η ) (cid:101) from D (in the standard, non adversarial, setting).
Revised: Recall that ε (cid:100) 1 / (7 η ) (cid:101) ( Lrn | D ) is the expected population loss of Lrn when trained on a sample of size (cid:100) 1 / (7 η ) (cid:101) from D (in the standard, non adversarial, setting).

Original: Proof.
Revised: Proof.

Original: Let S ∼ D n be the input sample, and ( x, y ) ∼ D be the test example.
Revised: Let S ∼ D n be the input sample, and ( x, y ) ∼ D be the test example.

Original: Note that for all i ∈ [ t ] (where t = (cid:98) 7 ηn (cid:99) is the number of subsamples of size at least 17 η in the partition made by SPV ) it holds that E (cid:2) 1[ h i ( x ) (cid:54) =y ] (cid:3) ≤ ε (cid:100) 1 /
Revised: Note that for all i ∈ [ t ] (where t = (cid:98) 7 ηn (cid:99) is the number of subsamples of size at least 17 η in the partition made by SPV ) it holds that E (cid:2) 1[ h i ( x ) (cid:54) =

Original: [Equation] 7 η ) [Equation] ( Lrn | D ) .
Revised: y ] (cid:3) ≤ ε (cid:100) 1 / (7 η ) (cid:101) ( Lrn | D ) .

Original: By applying linearity of expectation we get
Revised: By applying linearity of expectation we get

Original: E
Revised: E

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: ( x

Original: [Equation]
Revised: [Equation]

Revised: y

Revised: [Equation]

Revised: (cid:100)

Revised: [Equation]

Revised: 7 η

Revised: [Equation]

Revised: (cid:101) ( Lrn | D ) .

Original: By Markov’s inequality:
Revised: By Markov’s inequality:

Original: [Equation]
Revised: [Equation]

Original: Let
Revised: Let

Original: S (cid:48) ∈
Revised: S (cid:48) ∈

Original: B η ( S ) .
Revised: B η ( S ) .

Original: Let h (cid:48)
Revised: Let h (cid:48)

Original: =
Revised: =

Original: SPV ( Lrn )( S (cid:48) ) , and for all i ∈ [ t ] let h (cid:48)
Revised: SPV ( Lrn )( S (cid:48) ) , and for all i ∈ [ t ] let h (cid:48)

Original: i be the hypothesis obtained by training Lrn on S (cid:48)
Revised: i be the hypothesis obtained by training Lrn on S (cid:48)

Original: ( i ) .
Revised: ( i ) .

Original: Note that, since S and S (cid:48) are η -close by, and since n ≥ 1 /η it holds that
Revised: Note that, since S and S (cid:48) are η -close by, and since n ≥ 1 /η it holds that

Original: [Equation]
Revised: [Equation]

Original: Hence it is implied that 1 t (cid:80) t i =1 1{+
+} [ h i ( x ) (cid:54) = h (cid:48) i ( x )] ≤ 1 / 6 .
Revised: Hence it is implied that 1 t (cid:80) ti =1 {+
+} [ h i ( x ) (cid:54) = h (cid:48) i ( x )] ≤ 1 / 6 .

Original: Thus, the event that 1 t (cid:80) t i =1 1[ h (cid:48) i ( x ) (cid:54) =
Revised: Thus, the event that 1 t (cid:80) ti =1 1[ h (cid:48) i ( x ) (cid:54) =

Original: y ] ≥ 1 / 3 implies (or, is contained in) the event that 1 t (cid:80) ti =1 

Revised: y ] ≥ 1 / 3 implies (or, is contained in) the event that 1 t (cid:80) ti =1

Original: [ h i ( x ) (cid:54) =
Revised: [ h i ( x ) (cid:54) =

Original: y ] ≥ 1 / 6 , hence,
Revised: y ] ≥ 1 / 6 , hence,

Original: [Equation]
Revised: [Equation]

Original: Since h (cid:48) ( x ) is a majority vote of { h (cid:48) 1 ( x ) , . .
Revised: Since h (cid:48) ( x ) is a majority vote of { h (cid:48) 1 ( x ) , . .

Original: , h (cid:48)
Revised: , h (cid:48)

Original: t ( x ) } , the above implies that
Revised: t ( x ) } , the above implies that

Original: [Equation]
Revised: [Equation]

Original: Since S (cid:48) is an arbitrary sample in B η ( S ) , the above implies that SPV ( Lrn ) has the stated η -adversarial risk.
Revised: Since S (cid:48) is an arbitrary sample in B η ( S ) , the above implies that SPV ( Lrn ) has the stated η -adversarial risk.

Original: To prove Theorem 3.1, we will need an optimal learner as an input learner for SPV .
Revised: To prove Theorem 3.1, we will need an optimal learner as an input learner for SPV .

Original: Theorem A.2 (Haussler, Littlestone, and Warmuth [1994]) .
Revised: Theorem A.2 (Haussler, Littlestone, and Warmuth [1994]) .

Original: Let H be a concept class with VCdimension d , and let D be a distribution realizable by H .
Revised: Let H be a concept class with VCdimension d , and let D be a distribution realizable by H .

Original: Let also n ∈ N , and let Lrn be the
Revised: Let also n ∈ N , and let Lrn be the

Original: One-inclusion graph algorithm.
Revised: One-inclusion graph algorithm.

Original: Then ε n ( Lrn | D ) ≤ dn +1 .
Revised: Then ε n ( Lrn | D ) ≤ dn +1 .

Original: Theorem 3.1 can now be immediately inferred as a direct application of Lemma A.1 and Theorem A.2.
Revised: Theorem 3.1 can now be immediately inferred as a direct application of Lemma A.1 and Theorem A.2.

Original: Corollary A.3 (Realizable case – positive result) .
Revised: Corollary A.3 (Realizable case – positive result) .

Original: Let H be a concept class with VC-dimension d , let D be a distribution realizable by H , and let Lrn be the One-inclusion graph algorithm.
Revised: Let H be a concept class with VC-dimension d , let D be a distribution realizable by H , and let Lrn be the One-inclusion graph algorithm.

Original: Let also η ∈ (0 , 1) be the stability parameter given to SPV and let n ≥ 1 /η be the sample size.
Revised: Let also η ∈ (0 , 1) be the stability parameter given to SPV and let n ≥ 1 /η be the sample size.

Original: Then
Revised: Then

Original: SPV ( Lrn ) has η -adversarial risk
Revised: SPV ( Lrn ) has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Theorem A.2, plug in ε (cid:100) 1 / (7 η ) (cid:101) ( Lrn | D ) ≤ d (cid:100) 1 / (7 η ) (cid:101) +1 ≤ 7 ηd to Lemma A.1 and the result follows.
Revised: By Theorem A.2, plug in ε (cid:100) 1 / (7 η ) (cid:101) ( Lrn | D ) ≤ d (cid:100) 1 / (7 η ) (cid:101) +1 ≤ 7 ηd to Lemma A.1 and the result follows.

Original: B Proof of Theorem 3.3 (Realizable Case – Impossibility Result)
Revised: B Proof of Theorem 3.3 (Realizable Case – Impossibility Result)

Original: Randomized Learning Rules.
Revised: Randomized Learning Rules.

Original: The impossibility result in Theorem 3.3 extends to randomized learning rules.
Revised: The impossibility result in Theorem 3.3 extends to randomized learning rules.

Original: But in order for the statement in Theorem 3.3 to be meaningful, we need to deﬁne adversarial risk with respect to randomized learners.
Revised: But in order for the statement in Theorem 3.3 to be meaningful, we need to deﬁne adversarial risk with respect to randomized learners.

Original: As common in the literature on learning theory (see, e.g. the book of Shalev-Shwartz and Ben-David [2014]) we model randomized learners as deterministic learning rules with continuous predictions p ∈
Revised: As common in the literature on learning theory (see, e.g. the book of Shalev-Shwartz and Ben-David [2014]) we model randomized learners as deterministic learning rules with continuous predictions p ∈

Original: [0 , 1] , and loss function (cid:96) ( p, y ) =
Revised: [0 , 1] , and loss function (cid:96) ( p, y ) =

Original: | p − y | .
Revised: | p − y | .

Original: Indeed, the loss of a deterministic learner predicting a value p ∈ [0 , 1] under the loss function | y − p | is equal to the expected 0 / 1 -loss of a randomized learner predicting 1 with probability p .
Revised: Indeed, the loss of a deterministic learner predicting a value p ∈ [0 , 1] under the loss function | y − p | is equal to the expected 0 / 1 -loss of a randomized learner predicting 1 with probability p .

Original: In the course of discussing the impossibility result, a learning algorithm Lrn :
Revised: In the course of discussing the impossibility result, a learning algorithm Lrn :

Original: ( X × { 0 , 1 } ) ∗ → [0 , 1]
Revised: ( X × { 0 , 1 } ) ∗ → [0 , 1]

Original: X is a deterministic mapping which takes an input sample S ∈ ( X × { 0 , 1 } ) ∗ and maps it to a hypothesis f ∈ [0 , 1] X .
Revised: X is a deterministic mapping which takes an input sample S ∈ ( X × { 0 , 1 } ) ∗ and maps it to a hypothesis f ∈ [0 , 1] X .

Original: We re-deﬁne η -adversarial risk with this view of randomized learners as randomized η -adversarial risk .
Revised: We re-deﬁne η -adversarial risk with this view of randomized learners as randomized η -adversarial risk .

Original: Deﬁnition B.1 (Randomized η -Adversarial Risk) .
Revised: Deﬁnition B.1 (Randomized η -Adversarial Risk) .

Original: Let η ∈ (0 , 1) be the adversaries’ budget, let Lrn be a learning rule, and let D be a distribution over examples.
Revised: Let η ∈ (0 , 1) be the adversaries’ budget, let Lrn be a learning rule, and let D be a distribution over examples.

Original: The randomized η -adversarial risk of
Revised: The randomized η -adversarial risk of

Original: Lrn w.r.t D and sample size n is deﬁned by
Revised: Lrn w.r.t D and sample size n is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: The above deﬁnition of adversarial risk captures the case of an adversary that knows the expected prediction of the learner (that is, its test-time randomness), but not the learner’s "internal" randomness (computation-time randomness).
Revised: The above deﬁnition of adversarial risk captures the case of an adversary that knows the expected prediction of the learner (that is, its test-time randomness), but not the learner’s "internal" randomness (computation-time randomness).

Original: Indeed, the supremum is taken only with respect to the expected prediction, and not with respect to a speciﬁc execution of the algorithm determined by its internal randomness.
Revised: Indeed, the supremum is taken only with respect to the expected prediction, and not with respect to a speciﬁc execution of the algorithm determined by its internal randomness.

Original: Note that deterministic learners are a special case ( { 0 , 1 } -valued outputs), in which case this deﬁnition collapses to the previous Deﬁnition 2.2.
Revised: Note that deterministic learners are a special case ( { 0 , 1 } -valued outputs), in which case this deﬁnition collapses to the previous Deﬁnition 2.2.

Original: To avoid further notation, note that we overloaded the notation ε Adv n from Deﬁnition 2.2 in the above more general deﬁnition.
Revised: To avoid further notation, note that we overloaded the notation ε Adv n from Deﬁnition 2.2 in the above more general deﬁnition.

Original: We are now ready to prove the impossibility result.
Revised: We are now ready to prove the impossibility result.

Original: Theorem (Restatement of Theorem 3.3) .
Revised: Theorem (Restatement of Theorem 3.3) .

Original: There exists a constant c 2 > 0 so that the following holds.
Revised: There exists a constant c 2 > 0 so that the following holds.

Original: Let H be a non-trivial hypothesis class with VC dimension d and let η ∈ (0 , 1) .
Revised: Let H be a non-trivial hypothesis class with VC dimension d and let η ∈ (0 , 1) .

Original: Then, there exists a distribution D realizable by H , so that every learner Lrn has η -adversarial risk ε Adv n ( Lrn | D, η ) ≥ min { c 2 ηd, 1 / 100 } for any sample size n ≥ 1 /η .
Revised: Then, there exists a distribution D realizable by H , so that every learner Lrn has η -adversarial risk [Equation] [Equation] } for any sample size n ≥ 1 /η .

Original: Proof.
Revised: Proof.

Original: Let H be a non-trivial concept class; in particular this means that its VC-dimension d satisﬁes d ≥ 1 .
Revised: Let H be a non-trivial concept class; in particular this means that its VC-dimension d satisﬁes d ≥ 1 .

Original: Let η ∈ (0 , 1) be the adversaries’ budget and let Lrn be an arbitrary learner.
Revised: Let η ∈ (0 , 1) be the adversaries’ budget and let Lrn be an arbitrary learner.

Original: We need to show that there exists a distribution D realizable by H so that ε Adv n ( Lrn | D, η ) ≥ min { ηd/ 32 , 1 / 100 } .
Revised: We need to show that there exists a distribution D realizable by H so that ε Adv n ( Lrn | D, η ) ≥ min { ηd/ 32 , 1 / 100 } .

Original: It sufﬁces to consider the case when ηd/ 32 ≤ 1 / 100 and prove that ε Adv n ( Lrn | D, η ) ≥ ηd/ 32 .
Revised: It sufﬁces to consider the case when ηd/ 32 ≤ 1 / 100 and prove that ε Adv n ( Lrn | D, η ) ≥ ηd/ 32 .

Original: Indeed, in the complementing case we have ηd/ 32 > 1 / 100 and we need to show that ε Adv n ( Lrn | D, η ) ≥1 / 100 .
Revised: Indeed, in the complementing case we have ηd/ 32 > 1 / 100 and we need to show that ε Adv n ( Lrn | D, η ) ≥

Original: Notice that ηd/ 32 > 1 / 100 is equivalent to η > 32100 d , and thus it sufﬁces to show that even if the adversary’s budget η is reduced to η = 32100 d then ε Adv n ( Lrn | D, η ) ≥ 1 / 100 .
Revised: Notice that ηd/ 32 > 1 / 100 is equivalent to η > 32100 d , and thus it sufﬁces to show that even if the adversary’s budget η is reduced to η = 32100 d then ε Adv n ( Lrn | D, η ) ≥ 1 / 100 .

Original: The latter indeed follows from the case when ηd/ 32 ≤ 1 / 100 , because 32100 d · d/ 32 = 1 / 100 .
Revised: The latter indeed follows from the case when ηd/ 32 ≤ 1 / 100 , because 32100 d · d/ 32 = 1 / 100 .

Original: We thus assume that ηd/ 32 ≤ 1 / 100 and set out to prove that ε Adv n ( Lrn | D, η ) ≥ ηd/ 32 .
Revised: We thus assume that ηd/ 32 ≤ 1 / 100 and set out to prove that ε Adv n ( Lrn | D, η ) ≥ ηd/ 32 .

Original: We ﬁrst consider the case when the VC-dimension of H is d ≥ 2 and later handle the case when d = 1 .
Revised: We ﬁrst consider the case when the VC-dimension of H is d ≥ 2 and later handle the case when d = 1 .

Original: The VC dimensions is d ≥ 2 .
Revised: The VC dimensions is d ≥ 2 .

Original: Let V = { v 1 , . .
Revised: Let V = { v 1 , . .

Original: . , v d } ⊂ X be shattered by H .
Revised: . , v d } ⊂ X be shattered by H .

Original: Deﬁne a distribution D X over V as follows.
Revised: Deﬁne a distribution D X over V as follows.

Original: Set D X ( v i ) =
Revised: Set D X ( v i ) =

Original: η/ 2 for all 2 ≤ i ≤ d , and set D X ( v 1 ) = 1 − η ( d − 1) / 2 .
Revised: η/ 2 for all 2 ≤ i ≤ d , and set D X ( v 1 ) = 1 − η ( d − 1) / 2 .

Original: Notice that D X is well deﬁned since d ≥ 2 and η ≤ 2 / ( d − 1) (the latter is implied by the assumption that ηd/ 32 ≤ 1 / 100 ).
Revised: Notice that D X is well deﬁned since d ≥ 2 and η ≤ 2 / ( d − 1) (the latter is implied by the assumption that ηd/ 32 ≤ 1 / 100 ).

Original: For any labeling function (cid:96) ∈ Y V , let D (cid:96) denote the distribution over examples deﬁned by D (cid:96) ( v i , (cid:96) ( v i ))
Revised: For any labeling function (cid:96) ∈ Y V , let D (cid:96) denote the distribution over examples deﬁned by D (cid:96) ( v i , (cid:96) ( v i ))

Original: =
Revised: =

Original: D X ( v i ) for all
Revised: D X ( v i ) for all

Original: i ∈
Revised: i ∈

Original: [ d ] .
Revised: [ d ] .

Original: Note that D (cid:96) is realizable, since V is shattered.
Revised: Note that D (cid:96) is realizable, since V is shattered.

Original: It sufﬁces to show that if the label vector (cid:96) ∼ Y V is drawn uniformly at random then
Revised: It sufﬁces to show that if the label vector (cid:96) ∼ Y V is drawn uniformly at random then

Original: [Equation]
Revised: [Equation]

Original: Indeed, the above implies that there exists (cid:96)
Revised: Indeed, the above implies that there exists (cid:96)

Original: ∈ Y V such that
Revised: ∈ Y V such that

Original: [Equation]
Revised: [Equation]

Original: We establish Equation 1 in two steps:
Revised: We establish Equation 1 in two steps:

Original: For a sample S let S u be the unlabeled input sample underlying it.
Revised: For a sample S let S u be the unlabeled input sample underlying it.

Original: We say that an unlabeled sample S u and an instance x are hard if x (cid:54) = v 1 and x appears at most ηn times in S u .
Revised: We say that an unlabeled sample S u and an instance x are hard if x (cid:54) = v 1 and x appears at most ηn times in S u .

Original: In the ﬁrst step we show that Pr (cid:96),S, ( x,y ) [ S u , x are hard ] ≥ η ( d − 1) / 4 . 2.
Revised: In the ﬁrst step we show that Pr (cid:96),S, ( x,y ) [ S u , x are hard ] ≥ η ( d − 1) / 4 . 2.

Original: Let E 2 denote the event of all label vectors (cid:96) , input samples S , and test examples ( x, y ) such that sup S (cid:48)
Revised: Let E 2 denote the event of all label vectors (cid:96) , input samples S , and test examples ( x, y ) such that sup S (cid:48)

Original: ∈ B η ( S )
Revised: ∈ B η ( S )

Original: | Lrn ( S (cid:48) )
Revised: | Lrn ( S (cid:48) )

Original: ( x ) − y | ≥ 1 / 2 .
Revised: ( x ) − y | ≥ 1 / 2 .

Original: In the second step we show that Pr[ E 2 | S u , x are hard ] ≥ 1 / 2 .
Revised: In the second step we show that Pr[ E 2 | S u , x are hard ] ≥ 1 / 2 .

Original: Indeed, once we prove both steps we have:
Revised: Indeed [Equation] once we prove both steps we have:

Original: [Equation]
Revised: [Equation]

Original: as desired.
Revised: as desired.

Original: Let us prove step 1.
Revised: Let us prove step 1.

Original: Notice that S u and x are distributed according to the marginal distribution
Revised: Notice that S u and x are distributed according to the marginal distribution

Original: D n +1 X .
Revised: D n +1 X .

Original: Thus, x (cid:54) = v 1 with probability η
Revised: Thus, x (cid:54) = v 1 with probability η

Original: ( d − 1) / 2 , and given that x (cid:54)
Revised: ( d − 1) / 2 , and given that x (cid:54)

Original: = v 1 the expected number of appearances of x in S u is ηn/ 2 .
Revised: = v 1 the expected number of appearances of x in S u is ηn/ 2 .

Original: Therefore, by Markov’s inequality, the probability that S u and x are hard given that x (cid:54) = v 1 is at least ηn/ 2 ηn = 1 / 2 .
Revised: Therefore, by Markov’s inequality, the probability that S u and x are hard given that x (cid:54) = v 1 is at least ηn/ 2 ηn = 1 / 2 .

Original: Thus, the overall probability that S u , x are hard is at least η ( d − 1) / 4 .
Revised: Thus, the overall probability that S u , x are hard is at least η ( d − 1) / 4 .

Original: We now prove step 2.
Revised: We now prove step 2.

Original: Let S u , x be hard.
Revised: Let S u , x be hard.

Original: It sufﬁces to show that
Revised: It sufﬁces to show that

Original: [Equation]
Revised: [Equation]

Original: where (cid:96) ( x i ) is the label of the i ’th instance in S u and y is the test label.
Revised: where (cid:96) ( x i ) is the label of the i ’th instance in S u and y is the test label.

Original: Crucially, notice that the test-label y is independent of S u , x , and all other labels (cid:96) ( x i ) for x i ∈ S u such that x
Revised: Crucially, notice that the test-label y is independent of S u , x , and all other labels (cid:96) ( x i ) for x i ∈ S u such that x

Original: i (cid:54) = x .
Revised: i (cid:54) = x .

Original: Thus, even conditioned on S u , x and all labels of x i (cid:54) = x , the test-label y is distributed uniformly in
Revised: Thus, even conditioned on S u , x and all labels of x i (cid:54) = x , the test-label y is distributed uniformly in

Original: Y = { 0 , 1 } .
Revised: Y = { 0 , 1 } .

Original: Deﬁne samples S (cid:48) 0 , S (cid:48) 1 to be the same as S (cid:48) with the exception that every appearance of x in S (cid:48) 0 is labeled with 0 in S (cid:48) 

Revised: Deﬁne samples S (cid:48) 0 , S (cid:48) 1 to be the same as S (cid:48) with the exception that every appearance of x in S (cid:48) 0 is labeled with 0 in S (cid:48)

Original: and with 1 in S (cid:48) 1 .
Revised: and with 1 in S (cid:48) 1 .

Original: Note that both S (cid:48) 0 , S (cid:48) 1 ∈ B η ( S ) , because S u , x are hard.
Revised: Note that both S (cid:48) 0 , S (cid:48) 1 ∈ B η ( S ) , because S u , x are hard.

Original: We claim that, with probability at least half over the drawing of the (cid:96) ( x i ) ’s
Revised: We claim that, with probability at least half over the drawing of the (cid:96) ( x i ) ’s

Original: and y we have
Revised: and y we have

Original: [Equation]
Revised: [Equation]

Original: or|
Revised: or

Original: [Equation]
Revised: [Equation]

Original: Having this in hand, and given that ˆ S is hard, we are done: both S (cid:48) 0 , S (cid:48) 1 ∈ B η ( S ) , and Item 2 follows.
Revised: Having this in hand, and given that ˆ S is hard, we are done: both S (cid:48) 0 , S (cid:48) 1 ∈ B η ( S ) , and Item 2 follows.

Original: It thus remains to show that indeed | Lrn ( S (cid:48) 0 )
Revised: It thus remains to show that indeed | Lrn ( S (cid:48) 0 )

Original: ( x ) − y | ≥ 1 / 2 or | Lrn ( S (cid:48) 1 )( x ) − (cid:96) ( y ) | ≥ 1 / 2 with probability at least 1 / 2 over the drawing of the (cid:96) ( x i ) ’s and y .
Revised: ( x ) − y | ≥ 1 / 2 or | Lrn ( S (cid:48) 1 )( x ) − (cid:96) ( y ) | ≥ 1 / 2 with probability at least 1 / 2 over the drawing of the (cid:96) ( x i ) ’s and y .

Original: This is achieved by a simple case analysis:
Revised: This is achieved by a simple case analysis:

Original: • if both Lrn ( S (cid:48) 0 )( x ) , Lrn ( S (cid:48) 1 )( x ) ≤ 1 / 2 then with probability 1 / 2 we have y = 

Revised: • if both Lrn ( S (cid:48) 0 )( x ) , Lrn ( S (cid:48) 1 )( x ) ≤ 1 / 2 then with probability 1 / 2 we have y =

Original: and the claim follows.
Revised: and the claim follows.

Original: The case Lrn ( S (cid:48) 0 )( x ) , Lrn ( S (cid:48) 1 )( x ) > 1 / 2 is treated similarly.
Revised: The case Lrn ( S (cid:48) 0 )( x ) , Lrn ( S (cid:48) 1 )( x ) > 1 / 2 is treated similarly.

Original: • If Lrn ( S (cid:48) 0 )( x ) ≤ 1 / 2 , Lrn ( S (cid:48) 1 )( x ) ≥ 1 / 2 then | Lrn ( S (cid:48) 0 )( x ) − y | ≥ 1 / 2 or | Lrn ( S (cid:48) 1 )( x ) − y | ≥ 1 / 2 with probability 1 and the claim follows.
Revised: • If Lrn ( S (cid:48) 0 )( x ) ≤ 1 / 2 , Lrn ( S (cid:48) 1 )( x ) ≥ 1 / 2 then | Lrn ( S (cid:48) 0 )( x ) − y | ≥ 1 / 2 or | Lrn ( S (cid:48) 1 )( x ) − y | ≥ 1 / 2 with probability 1 and the claim follows.

Original: The case Lrn ( S (cid:48) 
Revised: The case Lrn ( S (cid:48) 0 )( x ) >

Original: )( x ) > 1 / 2 , Lrn ( S (cid:48) 1 )( x ) < 1 / 2 is treated similarly.
Revised: 2 , Lrn ( S (cid:48) 1 )( x ) < 1 / 2 is treated similarly.

Original: This ﬁnishes the proof of Theorem 3.3 when the VC-dimension d is at least 2 .
Revised: This ﬁnishes the proof of Theorem 3.3 when the VC-dimension d is at least 2 .

Original: The VC-dimension is d = 1 .
Revised: The VC-dimension is d = 1 .

Original: In this case, we can not deﬁne the distribution D X as before because d < 2 .
Revised: In this case, we can not deﬁne the distribution D X as before because d < 2 .

Original: However, the fact that H is non-trivial allows to modify the deﬁnition as follows.
Revised: However, the fact that H is non-trivial allows to modify the deﬁnition as follows.

Original: Let x 1 , x 2 ∈
Revised: Let x 1 , x 2 ∈

Original: X and h 1 , h 2 ∈ H
Revised: X and h 1 , h 2 ∈ H

Original: so that h 1 ( x 1 ) =
Revised: so that h 1 ( x 1 ) =

Original: h 2 ( x 1 ) and h 1 ( x 2 ) (cid:54)
Revised: h 2 ( x 1 ) and h 1 ( x 2 ) (cid:54)

Original: =
Revised: =

Original: h 2 ( x 2 ) , guaranteed by the fact that H is non-trivial.
Revised: h 2 ( x 2 ) , guaranteed by the fact that H is non-trivial.

Original: Set V = { x 1 , x 2 } , and deﬁne the distribution D X by D X ( x 1 ) = 

Revised: Set V = { x 1 , x 2 } , and deﬁne the distribution D X by D X ( x 1 ) =

Original: − η/ 2 , D X ( x 2 ) =
Revised: − η/ 2 , D X ( x 2 ) =

Original: η/as in the case d ≥ 2 .
Revised: η/as in the case d ≥ 2 .

Original: Also, deﬁne the random labeling function (cid:96) to agree with h 1 on with probability half and with h 2 with probability half.
Revised: Also, deﬁne the random labeling function (cid:96) to agree with h 1 on with probability half and with h 2 with probability half.

Original: The rest of the proof is the same.
Revised: The rest of the proof is the same.

Original: C Proof of Theorem 3.6 (Realizable and Proper Case – Positive Result)
Revised: C Proof of Theorem 3.6 (Realizable and Proper Case – Positive Result)

Original: Theorem (Restatement of Theorem 3.6) .
Revised: Theorem (Restatement of Theorem 3.6) .

Original: There exists a constant c > 0 so that the following holds.
Revised: There exists a constant c > 0 so that the following holds.

Original: Let H be the class of halfspaces over R d for some d ≥ 1 , and let η ∈ (0 , 1) .
Revised: Let H be the class of halfspaces over R d for some d ≥ 1 , and let η ∈ (0 , 1) .

Original: Then, there exists a proper learner Lrn having η -adversarial risk
Revised: Then, there exists a proper learner Lrn having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original:  for any distribution D realizable by H and for any sample size n ≥ 1 /η .
Revised: Adv [Equation] [Equation] for any distribution D realizable by H and for any sample size n ≥ 1 /η .

Original: To derive Theorem 3.6, we reinforce the SPV algorithm with a technique introduced by Kane,
Revised: To derive Theorem 3.6, we reinforce the SPV algorithm with a technique introduced by Kane,

Original: Livni, Moran, and Yehudayoff [2019] and further developed by Bousquet, Hanneke, Moran, and
Revised: Livni, Moran, and Yehudayoff [2019] and further developed by Bousquet, Hanneke, Moran, and

Original: Zhivotovskiy [2020].
Revised: Zhivotovskiy [2020].

Original: This technique allows in certain cases to project a majority vote of hypotheses from the class H back to H .
Revised: This technique allows in certain cases to project a majority vote of hypotheses from the class H back to H .

Original: Its applicability hinges on a combinatorial parameter called the projection number :
Revised: Its applicability hinges on a combinatorial parameter called the projection number :

Original: Deﬁnition C.1 (Projection Number) .
Revised: Deﬁnition C.1 (Projection Number) .

Original: Let H be a concept class.
Revised: Let H be a concept class.

Original: For any (cid:96) ≥ 2 and for any multiset
Revised: For any (cid:96) ≥ 2 and for any multiset

Original: H (cid:48)
Revised: H (cid:48)

Original: ⊂
Revised: ⊂

Original: H deﬁne the set X H (cid:48) ,(cid:96) to be the set of all x ∈ X , for which the number of hypotheses in H (cid:48) that disagree with Maj ( H (cid:48) )
Revised: H deﬁne the set X H (cid:48) ,(cid:96) to be the set of all x ∈ X , for which the number of hypotheses in H (cid:48) that disagree with Maj ( H (cid:48) )

Original: ( x ) is less than |H (cid:48) | /(cid:96) .
Revised: ( x ) is less than |H (cid:48) | /(cid:96) .

Original: The Projection Number of the class H , denoted k p = k p ( H ) , is deﬁned to be the smallest (cid:96) so that for any ﬁnite multiset H (cid:48)
Revised: The Projection Number of the class H , denoted k p = k p ( H ) , is deﬁned to be the smallest (cid:96) so that for any ﬁnite multiset H (cid:48)

Original: ⊂ H , there exist h ∈ H such that h ( x ) =
Revised: ⊂ H , there exist h ∈ H such that h ( x ) =

Original: Maj ( H (cid:48) )
Revised: Maj ( H (cid:48) )

Original: ( x ) for all x ∈ X H (cid:48) ,(cid:96) .
Revised: ( x ) for all x ∈ X H (cid:48) ,(cid:96) .

Original: If no such (cid:96) exists then k p = ∞ .
Revised: If no such (cid:96) exists then k p = ∞ .

Original: First, let us analyze the general performance of PSPV .
Revised: First, let us analyze the general performance of PSPV .

Original: Lemma C.2 (General performance of PSPV ) .
Revised: Lemma C.2 (General performance of PSPV ) .

Original: Let H be a concept class with a ﬁnite projection number k p < ∞ .
Revised: Let H be a concept class with a ﬁnite projection number k p < ∞ .

Original: Let D be a distribution over examples, and let Lrn p be a proper learning rule.
Revised: Let D be a distribution over examples, and let Lrn p be a proper learning rule.

Original: Let also η ∈ (0 , 1) be the stability parameter given to PSPV and let n ≥ 1 /η be the sample size.
Revised: Let also η ∈ (0 , 1) be the stability parameter given to PSPV and let n ≥ 1 /η be the sample size.

Original: Then
Revised: Then

Original: PSPV ( Lrn p ) is a proper learning rule having η -adversarial risk
Revised: PSPV ( Lrn p ) is a proper learning rule having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The proof follows the same lines as the proof of Lemma A.1.
Revised: The proof follows the same lines as the proof of Lemma A.1.

Original: Let S ∼ D n be the input sample, and ( x, y ) ∼ D be the test example.
Revised: Let S ∼ D n be the input sample, and ( x, y ) ∼ D be the test example.

Original: Note that for all i ∈ [ t ] (where t = (cid:98) 5 k p ηn (cid:99) is the number of subsamples of size at least 1 5 k p η in the partition made by PSPV ) it holds that
Revised: Note that for all i ∈ [ t ] (where t = (cid:98) 5 k p ηn (cid:99) is the number of subsamples of size at least 1 5 k p η in the partition made by PSPV ) it holds that

Original: E
Revised: E

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Lrn p | D ) .
Revised: Lrn [Equation] D ) .

Original: By applying linearity of expectation we get
Revised: By applying linearity of expectation we get

Original: [Equation]
Revised: [Equation]

Original: By Markov’s inequality:
Revised: By Markov’s inequality:

Original: [Equation]
Revised: [Equation]

Original: Let
Revised: Let

Original: S (cid:48) ∈
Revised: S (cid:48) ∈

Original: B η ( S ) .
Revised: B η ( S ) .

Original: Let h (cid:48) =
Revised: Let h (cid:48) =

Original: PSPV ( Lrn p )( S (cid:48) ) , and for all i ∈ [ t ] let h (cid:48)
Revised: PSPV ( Lrn p )( S (cid:48) ) , and for all i ∈ [ t ] let h (cid:48)

Original: i be the hypothesis obtained by training Lrn p on S (cid:48)
Revised: i be the hypothesis obtained by training Lrn p on S (cid:48)

Original: ( i ) .
Revised: ( i ) .

Original: Note that, since S and S (cid:48) are η -close by, and since n ≥ 1 /η it holds that
Revised: Note that, since S and S (cid:48) are η -close by, and since n ≥ 1 /η it holds that

Original: [Equation]
Revised: [Equation]

Original: Hence it is implied that 1 t (cid:80) ti =1 

Revised: Hence it is implied that 1 t (cid:80) ti =1

Original: [ h i ( x ) (cid:54) = h (cid:48)
Revised: [ h i ( x ) (cid:54) = h (cid:48)

Original: i ( x )] ≤ 14 k p .
Revised: i ( x )] ≤ 14 k p .

Original: Thus, the event that 1 t (cid:80) ti =1 1[ h (cid:48)
Revised: Thus, the event that 1 t (cid:80) ti =1 1[ h (cid:48)

Original: i ( x ) (cid:54) =
Revised: i ( x ) (cid:54) =

Original: y ] ≥ 12 k p implies (or, is contained in)
Revised: y ] ≥ 12 k p implies (or, is contained in)

Original: the event that (cid:80) ti =1 

Revised: the event that (cid:80) ti =1

Original: [ h i ( x ) (cid:54) =
Revised: [ h i ( x ) (cid:54) =

Original: y ] ≥ 14 k p , hence:
Revised: y ] ≥ 14 k p , hence:

Original: [Equation]
Revised: [Equation]

Original: Note that by deﬁnition of projection number it holds that the hypothesis h (cid:48) ∈ H returned by the algorithm exists.
Revised: Note that by deﬁnition of projection number it holds that the hypothesis h (cid:48) ∈ H returned by the algorithm exists.

Original: Hence, by deﬁnition of X { h (cid:48) 1 ,...,h
Revised: Hence, by deﬁnition of X { h (cid:48) 1 ,...,h

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: p the above implies that
Revised: p the above implies that

Original: [Equation]
Revised: [Equation]

Original: Since S (cid:48) is an arbitrary sample in B η ( S ) , the above implies that PSPV ( Lrn p ) has the stated ηadversarial risk.
Revised: Since S (cid:48) is an arbitrary sample in B η ( S ) , the above implies that PSPV ( Lrn p ) has the stated ηadversarial risk.

Original: To prove Theorem 3.6 we will use the following result regarding the projection number of halfspaces.
Revised: To prove Theorem 3.6 we will use the following result regarding the projection number of halfspaces.

Original: Theorem C.3 (Kane, Livni, Moran, and Yehudayoff [2019], Braverman, Kol, Moran, and Saxena [2019], Bousquet, Hanneke, Moran, and Zhivotovskiy [2020]) .
Revised: Theorem C.3 (Kane, Livni, Moran, and Yehudayoff [2019], Braverman, Kol, Moran, and Saxena [2019], Bousquet, Hanneke, Moran, and Zhivotovskiy [2020]) .

Original: Let H be the class of halfspaces over R m .
Revised: Let H be the class of halfspaces over R m .

Original: Then k p ( H ) =
Revised: Then k p ( H ) =

Original: d ( H )
Revised: d ( H )

Original: = m
Revised: = m

Original: + 1 .
Revised: + 1 .

Original: We will use the SVM learner as an input learner for PSPV .
Revised: We will use the SVM learner as an input learner for PSPV .

Original: Theorem C.4 (Vapnik and Chervonenkis [1974]) .
Revised: Theorem C.4 (Vapnik and Chervonenkis [1974]) .

Original: Let m ≥ 1 and let H be the class of halfspaces over R m .
Revised: Let m ≥ 1 and let H be the class of halfspaces over R m .

Original: Let D be a distribution realizable by H .
Revised: Let D be a distribution realizable by H .

Original: Let also n ∈ N , and let Lrn p be the SVM algorithm.
Revised: Let also n ∈ N , and let Lrn p be the SVM algorithm.

Original: Then ε n ( Lrn p | D ) ≤ m +1 n +1 .
Revised: Then ε n ( Lrn p | D ) ≤ m +1 n +1 .

Original: Theorem 3.6 now follows as an immediate application of Theorem C.3, Theorem C.4 and Lemma C.2.
Revised: Theorem 3.6 now follows as an immediate application of Theorem C.3, Theorem C.4 and Lemma C.2.

Original: Corollary C.5 (Realizable and proper case – positive result) .
Revised: Corollary C.5 (Realizable and proper case – positive result) .

Original: Let m ≥ 1 , let H be the class of halfspaces over R m , and let d = m + 1 be the VC-dimension of H .
Revised: Let m ≥ 1 , let H be the class of halfspaces over R m , and let d = m + 1 be the VC-dimension of H .

Original: Let D be a distribution realizable by H , and let Lrn p be the SVM learner.
Revised: Let D be a distribution realizable by H , and let Lrn p be the SVM learner.

Original: Let also η ∈ (0 , 1) be the stability parameter given to PSPV and let n ≥ 1 /η be the sample size.
Revised: Let also η ∈ (0 , 1) be the stability parameter given to PSPV and let n ≥ 1 /η be the sample size.

Original: Then PSPV ( Lrn p ) has η -adversarial risk
Revised: Then PSPV ( Lrn p ) has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Theorem C.3, if H is the class of halfspaces over R m then its projection number is k p =
Revised: By Theorem C.3, if H is the class of halfspaces over R m then its projection number is k p =

Original: d = m + 1 .
Revised: d = m + 1 .

Original: Also, by Theorem C.4, we have that ε (cid:100) 1 / (5 dη ) (cid:101) ( Lrn p | D ) ≤ 5 ηd 2 .
Revised: Also, by Theorem C.4, we have that ε (cid:100) 1 / (5 dη ) (cid:101) ( Lrn p | D ) ≤ 5 ηd 2 .

Original: Plug both results to Lemma C.2, and the result follows.
Revised: Plug both results to Lemma C.2, and the result follows.

Original: D Proof of Theorem 4.1 (Agnostic Case – Positive Result)
Revised: D Proof of Theorem 4.1 (Agnostic Case – Positive Result)

Original: Theorem (Restatement of Theorem 4.1) .
Revised: Theorem (Restatement of Theorem 4.1) .

Original: There exist constants c 1 , c 2 so that the following holds.
Revised: There exist constants c 1 , c 2 so that the following holds.

Original: Let
Revised: Let

Original: H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .
Revised: H be a hypothesis class with VC dimension d and let η ∈ (0 , 1) .

Original: Then, there exists a learner Lrn having η -adversarial risk
Revised: Then, there exists a learner Lrn having η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: for any distribution D over examples and for any sample size n ≥ 1 /η .
Revised: for any distribution D over examples and for any sample size n ≥ 1 /η .

Original: To derive Theorem 4.1, we use an agnostic variation of the One-inclusion graph learner.
Revised: To derive Theorem 4.1, we use an agnostic variation of the One-inclusion graph learner.

Original: Theorem D.1 (Corollary of Lemma 16 in [Long, 1999]) .
Revised: Theorem D.1 (Corollary of Lemma 16 in [Long, 1999]) .

Original: There exists a constant C such that the following holds.
Revised: There exists a constant C such that the following holds.

Original: Let H be a concept class with VC-dimension d and let Lrn be the agnostic variation of the One-inclusion graph algorithm implied by Lemma 16 in [Long, 1999].
Revised: Let H be a concept class with VC-dimension d and let Lrn be the agnostic variation of the One-inclusion graph algorithm implied by Lemma 16 in [Long, 1999].

Original: Let also n be the sample size.
Revised: Let also n be the sample size.

Original: Then, for any distribution D over examples (not necessarily such that is realizable by H ), it holds that ε n ( Lrn | D ) ≤ C ( OPT + d/n ) .
Revised: Then, for any distribution D over examples (not necessarily such that is realizable by H ), it holds that ε n ( Lrn | D ) ≤ C ( OPT + d/n ) .

Original: Theorem 4.1 is implied by the following immediate corollary of Theorem 4.1 and Lemma A.1.
Revised: Theorem 4.1 is implied by the following immediate corollary of Theorem 4.1 and Lemma A.1.

Original: Corollary D.2 (Agnostic case – positive result) .
Revised: Corollary D.2 (Agnostic case – positive result) .

Original: There exists a constant C such that the following holds.
Revised: There exists a constant C such that the following holds.

Original: Let H be a concept class with V C dimension d , let η ∈ (0 , 1) be the stability parameter given to SPV , and let D be a (not necessarily realizable) distribution over examples.
Revised: Let H be a concept class with V C dimension d , let η ∈ (0 , 1) be the stability parameter given to SPV , and let D be a (not necessarily realizable) distribution over examples.

Original: Let also n ≥ 1 /η be the sample size.
Revised: Let also n ≥ 1 /η be the sample size.

Original: Then SPV ( Lrn ) has η -adversarial risk
Revised: Then SPV ( Lrn ) has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: where Lrn is the agnostic variant of the One-inclusion graph algorithm mentioned in Theorem D.1.
Revised: where Lrn is the agnostic variant of the One-inclusion graph algorithm mentioned in Theorem D.1.

Original: Proof.
Revised: Proof.

Original: By Theorem D.1, there exists a constant C such that ε (cid:100) [Equation]
Revised: By Theorem D.1, there exists a constant C such that ε (cid:100) 1 / (7 η ) (cid:101)

Original:  Lrn | [Equation] ≤ C OPT + [Equation]
Revised: ( Lrn | D ) ≤ C OPT + 7 Cηd .

Original: Plug this into Lemma A.1 and the result follows.
Revised: Plug this into Lemma A.1 and the result follows.

Original: E Proof of Theorem 4.2 (Agnostic Case – Impossibility Result)
Revised: E Proof of Theorem 4.2 (Agnostic Case – Impossibility Result)

Original: Theorem (Restatement of Theorem 4.2) .
Revised: Theorem (Restatement of Theorem 4.2) .

Original: Let η (cid:48) ∈ (0 , 1) , n ∈ N .
Revised: Let η (cid:48) ∈ (0 , 1) , n ∈ N .

Original: For any hypothesis class H that has at least two hypotheses and.
Revised: For any hypothesis class H that has at least two hypotheses and.

Original: for any deterministic learner, there is a distribution D over (two) examples and η = η (cid:48)
Revised: for any deterministic learner, there is a distribution D over (two) examples and η = η (cid:48)

Original: + (cid:101) O
Revised: + (cid:101) O

Original: (1 / √ n ) such that Lrn has η -adversarial risk
Revised: (1 / √ n ) such that Lrn has η -adversarial risk

Original: [Equation]
Revised: [Equation]

Original: Let h 1 , h 2 ∈ H be two distinct hypotheses and let x ∈
Revised: Let h 1 , h 2 ∈ H be two distinct hypotheses and let x ∈

Original: X such that
Revised: X such that

Original: h 1 ( x ) (cid:54) =
Revised: h 1 ( x ) (cid:54) =

Original: h 2 ( x ) .
Revised: h 2 ( x ) .

Original: In this proof we consider distributions D supported only on { ( x, 0) , ( x, 1) } .
Revised: In this proof we consider distributions D supported only on { ( x, 0) , ( x, 1) } .

Original: Notice that such a distribution is determined by the probability p =
Revised: Notice that such a distribution is determined by the probability p =

Original: Pr ( x,y ) ∼ D
Revised: Pr ( x,y ) ∼ D

Original: [ y = 1] and hence can be thought of as a coin with bias p .
Revised: [ y = 1] and hence can be thought of as a coin with bias p .

Original: Thus, the task of agnostic learning such distributions with respect to instance-targeted data poisoning boils down to predicting a random p -coin toss given an input sample of n p -coin tosses out of which at most η · n tosses are ﬂipped by an adversary who knows the result of the coin toss that needs to be predicted.
Revised: Thus, the task of agnostic learning such distributions with respect to instance-targeted data poisoning boils down to predicting a random p -coin toss given an input sample of n p -coin tosses out of which at most η · n tosses are ﬂipped by an adversary who knows the result of the coin toss that needs to be predicted.

Original: We summarize this in the following game:
Revised: We summarize this in the following game:

Original: Deﬁnition E.1 (The coin game) .
Revised: Deﬁnition E.1 (The coin game) .

Original: The coin game is parameterized by ( n, η ) where n ∈ N , η ∈ (0 , 1) , and the game is played between an adversary Adv and a learner Lrn as follows.
Revised: The coin game is parameterized by ( n, η ) where n ∈ N , η ∈ (0 , 1) , and the game is played between an adversary Adv and a learner Lrn as follows.

Original: Adv picks p ∈ [0 , 1] .
Revised: Adv picks p ∈ [0 , 1] .

Original: c 1 , . .
Revised: c 1 , . .

Original: c n +1 ∼ X n +1 p , where X p is a binary random variable satisfying Pr[ X p = 1] = p .
Revised: c n +1 ∼ X n +1 p , where X p is a binary random variable satisfying Pr[ X p = 1] = p .

Original: Adv changes c
Revised: Adv changes c

Original: = ( c 1 , . . .
Revised: = ( c 1 , . . .

Original: , c n ) into c (cid:48)
Revised: , c n ) into c (cid:48)

Original: =
Revised: =

Original: ( c (cid:48) 1 , . .
Revised: ( c (cid:48) 1 , . .

Original: c (cid:48) n )
Revised: c (cid:48) n )

Original: where
Revised: where

Original: d H ( c, c (cid:48) ) ≤ η · n . 4.
Revised: d H ( c, c (cid:48) ) ≤ η · n . 4.

Original: Lrn gets to see c (cid:48)
Revised: Lrn gets to see c (cid:48)

Original: =
Revised: =

Original: ( c (cid:48) 1 , . .
Revised: ( c (cid:48) 1 , . .

Original: c (cid:48) n ) and outputs a bit c ∈ { 0 , 1 } .
Revised: c (cid:48) n ) and outputs a bit c ∈ { 0 , 1 } .

Original: Lrn wins if c = c n +1 , and Adv wins otherwise.
Revised: Lrn wins if c = c n +1 , and Adv wins otherwise.

Original: In this game, we deﬁne OPT p = min { p, 1 − p } to be the optimal error of the learner if it had known p , and we deﬁne ERR =
Revised: In this game, we deﬁne OPT p = min { p, 1 − p } to be the optimal error of the learner if it had known p , and we deﬁne ERR =

Original: Pr[ c (cid:54) =
Revised: Pr[ c (cid:54) =

Original: c n +1 ] (over all the randomness involved) to be the error of the game (i.e., when the learner does not win).
Revised: c n +1 ] (over all the randomness involved) to be the error of the game (i.e., when the learner does not win).

Original: We also refer to ERR − OPT p as the regret.
Revised: We also refer to ERR − OPT p as the regret.

Original: Theorem E.2.
Revised: Theorem E.2.

Original: For any η (cid:48) ∈ [0 , 1 / 2] and any deterministic learner Lrn that participates in the coin game of Deﬁnition E.1, there is an adversary Adv with a ﬁxed choice of p (determining OPT = OPT p ) and η = η (cid:48)
Revised: For any η (cid:48) ∈ [0 , 1 / 2] and any deterministic learner Lrn that participates in the coin game of Deﬁnition E.1, there is an adversary Adv with a ﬁxed choice of p (determining OPT = OPT p ) and η = η (cid:48)

Original: + (cid:101) O
Revised: + (cid:101) O

Original: (1 / √ n ) such that when we run the game of Deﬁnition E.1 with parameters ( n, η ) , it holds that ERR − OPT ≥ 1 / 2 + η (cid:48) − O (1 /n ) .
Revised: (1 / √ n ) such that when we run the game of Deﬁnition E.1 with parameters ( n, η ) , it holds that ERR − OPT ≥ 1 / 2 + η (cid:48) − O (1 /n ) .

Original: Remark 1 (On deterministic adversaries) .
Revised: Remark 1 (On deterministic adversaries) .

Original: In Theorem E.2 we show the existence of an adversary with a ﬁxed choice of p .
Revised: In Theorem E.2 we show the existence of an adversary with a ﬁxed choice of p .

Original: This adversary is in fact randomized.
Revised: This adversary is in fact randomized.

Original: Here we remark that, for every ﬁxed (even randomized) learner Lrn and a ﬁxed choice of p , there is always a deterministic adversary that achieves the maximum regret (for such Lrn , p ).
Revised: Here we remark that, for every ﬁxed (even randomized) learner Lrn and a ﬁxed choice of p , there is always a deterministic adversary that achieves the maximum regret (for such Lrn , p ).

Original: The reason is that if by using randomness r Adv the adversary achieves expected regret R ( r Adv ) over the randomness of the learner, then its overall regret will be E r Adv
Revised: The reason is that if by using randomness r Adv the adversary achieves expected regret R ( r Adv ) over the randomness of the learner, then its overall regret will be E r Adv

Original: [ R ( r Adv )] .
Revised: [ R ( r Adv )] .

Original: Therefore, if r ( p ) Adv is the randomness (for ﬁxed p ) that maximizes R ( r Adv ) , the adversary can simply ﬁx its randomness to r ( p ) Adv without decreasing its gain.
Revised: Therefore, if r ( p ) Adv is the randomness (for ﬁxed p ) that maximizes R ( r Adv ) , the adversary can simply ﬁx its randomness to r ( p ) Adv without decreasing its gain.

Original: This means that without loss of generality, the adversary of Theorem E.2 is deterministic.
Revised: This means that without loss of generality, the adversary of Theorem E.2 is deterministic.

Original: In addition, since the adversary sends the ﬁrst message p , the overall optimal strategy Adv (who picks p potentially in a randomized way) can also ﬁx p to what maximizes R ( r ( p ) Adv ) , which makes Adv fully deterministic.
Revised: In addition, since the adversary sends the ﬁrst message p , the overall optimal strategy Adv (who picks p potentially in a randomized way) can also ﬁx p to what maximizes R ( r ( p ) Adv ) , which makes Adv fully deterministic.

Original: Deriving Theorem 4.2.
Revised: Deriving Theorem 4.2.

Original: We ﬁrst show how to derive Theorem 4.2 from Theorem E.2.
Revised: We ﬁrst show how to derive Theorem 4.2 from Theorem E.2.

Original: Proof of Theorem 4.2.
Revised: Proof of Theorem 4.2.

Original: First assume η (cid:48) ≤ 1 / 2 , and at the end we explain how to deal with η (cid:48) > 1 / 2 .
Revised: First assume η (cid:48) ≤ 1 / 2 , and at the end we explain how to deal with η (cid:48) > 1 / 2 .

Original: By Theorem E.2, there is an adversary (with a ﬁxed choice of p ) in the coin game of Deﬁnition E. such that ERR − OPT ≥ 1 / 2+ η (cid:48) − O (1 /n ) when we use ( n, η ) as game parameters.
Revised: By Theorem E.2, there is an adversary (with a ﬁxed choice of p ) in the coin game of Deﬁnition E. such that ERR − OPT ≥ 1 / 2+ η (cid:48) − O (1 /n ) when we use ( n, η ) as game parameters.

Original: Since ERR ≤ 1 , we have OPT ≤ 1 / 2 − η (cid:48)
Revised: Since ERR ≤ 1 , we have OPT ≤ 1 / 2 − η (cid:48)

Original: + O (1 /n ) , and so
Revised: + O (1 /n ) , and so

Original: [Equation]
Revised: [Equation]

Original: This implies that ERR ≥ 2 OPT + 2 η (cid:48) − O (1 /n ) .
Revised: This implies that ERR ≥ 2 OPT + 2 η (cid:48) − O (1 /n ) .

Original: Note that OPT is indeed the minimal error that the learner can achieve by outputting any of the constant coins 0 , 1 , which in turn refers to outputting either of h 0 , h 1 from the hypothesis class.
Revised: Note that OPT is indeed the minimal error that the learner can achieve by outputting any of the constant coins 0 , 1 , which in turn refers to outputting either of h 0 , h 1 from the hypothesis class.

Original: In addition, ERR is equal to the adversarial risk for parameters n, η and the distribution D p for this particular attack.
Revised: In addition, ERR is equal to the adversarial risk for parameters n, η and the distribution D p for this particular attack.

Original: This means that
Revised: This means that

Original: [Equation]
Revised: [Equation]

Original: which implies Theorem 4.2.
Revised: which implies Theorem 4.2.

Original: Now, if η (cid:48) > 1 / 2 , we ﬁrst artiﬁcially decrease adversary’s budget η (cid:48) to η (cid:48)(cid:48)
Revised: Now, if η (cid:48) > 1 / 2 , we ﬁrst artiﬁcially decrease adversary’s budget η (cid:48) to η (cid:48)(cid:48)

Original: = 1 / 2 , which leads to
Revised: = 1 / 2 , which leads to

Original: [Equation]
Revised: [Equation]

Original: but we also know that η (cid:48)(cid:48)
Revised: but we also know that η (cid:48)(cid:48)

Original: =
Revised: =

Original: Ω( η (cid:48) ) , which again proves Theorem 4.2.
Revised: Ω( η (cid:48) ) , which again proves Theorem 4.2.

Original: Before proving Theorem E.2 we recall two useful tools.
Revised: Before proving Theorem E.2 we recall two useful tools.

Original: Lemma E.3 (Proposition 2.1.1 in Talagrand [1995]) .
Revised: Lemma E.3 (Proposition 2.1.1 in Talagrand [1995]) .

Original: Let µ = µ 1 × . . .
Revised: Let µ = µ 1 × . . .

Original: µ n be a product measure and f : µ (cid:55)→ { 0 , 1 } a boolean function where Pr[ f ( µ ) = 1] = 1 / 2 .
Revised: µ n be a product measure and f : µ (cid:55)→ { 0 , 1 } a boolean function where Pr[ f ( µ ) = 1] = 1 / 2 .

Original: Then, for all b ∈ [ n ] ,
Revised: Then, for all b ∈ [ n ] ,

Original: [Equation]
Revised: [Equation]

Original: In other words, with probability at least 1 − 2 e − b 2 /n over the sampling of x ∼ µ , one can change up to b of the coordinates of x and obtain x (cid:48)
Revised: In other words, with probability at least 1 − 2 e − b 2 /n over the sampling of x ∼ µ , one can change up to b of the coordinates of x and obtain x (cid:48)

Original: (i.e., d H ( x, x (cid:48) ) ≤ b ) such that f ( x (cid:48) )
Revised: (i.e., d H ( x, x (cid:48) ) ≤ b ) such that f ( x (cid:48) )

Original: = 1 .
Revised: = 1 .

Original: Lemma E.4 (Modifying coins) .
Revised: Lemma E.4 (Modifying coins) .

Original: Suppose 

Revised: Suppose

Original: ≤
Revised: ≤

Original: p, p (cid:48) ≤ 1 , and let q = | p − p (cid:48) | .
Revised: p, p (cid:48) ≤ 1 , and let q = | p − p (cid:48) | .

Original: Then there is an adversary who can change q · n coins, in expectation, of a sample c ∼ X np into c (cid:48)
Revised: Then there is an adversary who can change q · n coins, in expectation, of a sample c ∼ X np into c (cid:48)

Original: (i.e., E [ d H ( c (cid:48) , c )]
Revised: (i.e., E [ d H ( c (cid:48) , c )]

Original: = q · n ) such that c (cid:48) ∼ X np (cid:48)
Revised: = q · n ) such that c (cid:48) ∼ X np (cid:48)

Original: (Namely, the tampered sequence looks exactly like it is sampled from X np (cid:48) , while in reality it is being ﬁrst sampled from X np and then modiﬁed by the adversary in q · n points in expectation).
Revised: (Namely, the tampered sequence looks exactly like it is sampled from X np (cid:48) , while in reality it is being ﬁrst sampled from X np and then modiﬁed by the adversary in q · n points in expectation).

Original: Moreover, the probability that the adversary changes more than qn + (cid:112) ( n ln n ) / 2 of the coordinates is at most 1 /n .
Revised: Moreover, the probability that the adversary changes more than qn + (cid:112) ( n ln n ) / 2 of the coordinates is at most 1 /n .

Original: Proof.
Revised: Proof.

Original: Without loss of generality, let p (cid:48) − p = q ≥ 0 .
Revised: Without loss of generality, let p (cid:48) − p = q ≥ 0 .

Original: Then the adversary will change each of the coins with independent probability q as follows.
Revised: Then the adversary will change each of the coins with independent probability q as follows.

Original: If a coin c i = 1 , the adversary will not change it, which will happen with probability p .
Revised: If a coin c i = 1 , the adversary will not change it, which will happen with probability p .

Original: If c i = 0 , which will happen with probability 1 − p , the adversary will change this to 1 with probability q/ (1 − p ) over its own randomness.
Revised: If c i = 0 , which will happen with probability 1 − p , the adversary will change this to 1 with probability q/ (1 − p ) over its own randomness.

Original: Note that q = p (cid:48) − p ≤ (1 − p ) , and so q/ (1 − p ) ∈
Revised: Note that q = p (cid:48) − p ≤ (1 − p ) , and so q/ (1 − p ) ∈

Original: [0 , 1] can be interpreted as a probability.
Revised: [0 , 1] can be interpreted as a probability.

Original: The probability that c (cid:48)
Revised: The probability that c (cid:48)

Original: i = 1 is now exactly p + q = p (cid:48) , while the expected number of changed coins is q · n .
Revised: i = 1 is now exactly p + q = p (cid:48) , while the expected number of changed coins is q · n .

Original: Finally, since the adversary’s changes of the coin outcomes are doneindependently for each coin, the bound on the number of changes made by the adversary is implied by the Hoeffding-Chernoff bound.
Revised: Finally, since the adversary’s changes of the coin outcomes are done

Original: We now prove Theorem E.2 using the two tools above.
Revised: We now prove Theorem E.2 using the two tools above.

Original: Proof of Theorem E.2.
Revised: Proof of Theorem E.2.

Original: Fix the deterministic learning algorithm Lrn .
Revised: Fix the deterministic learning algorithm Lrn .

Original: This means that for every given input vector c
Revised: This means that for every given input vector c

Original: = ( c 1 , . . .
Revised: = ( c 1 , . . .

Original: , c n ) , we have Lrn ( c ) ∈ { 0 , 1 } .
Revised: , c n ) , we have Lrn ( c ) ∈ { 0 , 1 } .

Original: Now deﬁne α ( p ) =
Revised: Now deﬁne α ( p ) =

Original: Pr c ∼ X np
Revised: Pr c ∼ X np

Original: [ Lrn ( c ) = 1] .
Revised: [ Lrn ( c ) = 1] .

Original: We do a case study as follows.
Revised: We do a case study as follows.

Original: • If α (0) (cid:54) = 0 , it means that α (0) =
Revised: • If α (0) (cid:54) = 0 , it means that α (0) =

Original: 1 (i.e., the deterministic learner outputs 1 over the all zero vector).
Revised: 1 (i.e., the deterministic learner outputs 1 over the all zero vector).

Original: In this case, OPT = 0 and ERR = 1 , which implies ERR − OPT ≥ 1 / 2 + η (cid:48) .
Revised: In this case, OPT = 0 and ERR = 1 , which implies ERR − OPT ≥ 1 / 2 + η (cid:48) .

Original: • If α (1) (cid:54) = 1 , it implies ERR − OPT ≥ 1 / 2 + η (cid:48) similarly.
Revised: • If α (1) (cid:54) = 1 , it implies ERR − OPT ≥ 1 / 2 + η (cid:48) similarly.

Original: • If none of the above cases happens, we can assume α ( b ) =
Revised: • If none of the above cases happens, we can assume α ( b ) =

Original: b for both b ∈ { 0 , 1 } .
Revised: b for both b ∈ { 0 , 1 } .

Original: Because the learner is deterministic, Lrn ( c ) = 1 if c ∈ S for a ﬁxed set S ⊆ { 0 , 1 } n .
Revised: Because the learner is deterministic, Lrn ( c ) = 1 if c ∈ S for a ﬁxed set S ⊆ { 0 , 1 } n .

Original: Moreover, for all c ∈ { 0 , 1 } n , it holds that Pr[ X np = c ] =
Revised: Moreover, for all c ∈ { 0 , 1 } n , it holds that Pr[ X np = c ] =

Original: p d (1 − p ) n − d , where d is the number of non-zero coordinates of c .
Revised: p d (1 − p ) n − d , where d is the number of non-zero coordinates of c .

Original: This implies that α ( p ) is a polynomial of degree at most n over p , which is a continuous function.
Revised: This implies that α ( p ) is a polynomial of degree at most n over p , which is a continuous function.

Original: Therefore, there exists q ∈ (0 , 1) such that α ( q ) = 1 / 2 .
Revised: Therefore, there exists q ∈ (0 , 1) such that α ( q ) = 1 / 2 .

Original: Without loss of generality, assume that q ≤ 1 / 2 .
Revised: Without loss of generality, assume that q ≤ 1 / 2 .

Original: Then, the adversary picks p = max { 0 , q − η (cid:48) } , which guarantees OPT = p ≤ 1 / 2 − η (cid:48)
Revised: Then, the adversary picks p = max { 0 , q − η (cid:48) } , which guarantees OPT = p ≤ 1 / 2 − η (cid:48)

Original: (due to the assumptions η (cid:48) , q ≤ 1 / 2 ).
Revised: (due to the assumptions η (cid:48) , q ≤ 1 / 2 ).

Original: Then, the adversary uses Lemma E.4 to shift the coin’s distribution back to q .
Revised: Then, the adversary uses Lemma E.4 to shift the coin’s distribution back to q .

Original: For this change, the adversary makes at most η (cid:48) · n + (cid:112) ( n ln n ) / 2 changes with probability 1 − 1 /n .
Revised: For this change, the adversary makes at most η (cid:48) · n + (cid:112) ( n ln n ) / 2 changes with probability 1 − 1 /n .

Original: We then apply the algorithm of Lemma E.3 to make further (cid:112) n ln(2 n ) changes to the coins to make sure that the output of the learner is the wrong outcome (different from c n +1 ) with probability 1 − 1 /n .
Revised: We then apply the algorithm of Lemma E.3 to make further (cid:112) n ln(2 n ) changes to the coins to make sure that the output of the learner is the wrong outcome (different from c n +1 ) with probability 1 − 1 /n .

Original: In total, the adversary can make at most η (cid:48) · n + (cid:112) ( n ln n ) / 2+ (cid:112) n ln(2 n ) ∈ η (cid:48) · n + (cid:101) O ( √ n ) changes to the coin ﬂips outcomes, while the learner’s output bit is wrong with probability 1 − 1 /n − 1 /n
Revised: In total, the adversary can make at most η (cid:48) · n + (cid:112) ( n ln n ) / 2+ (cid:112) n ln(2 n ) ∈ η (cid:48) · n + (cid:101) O ( √ n ) changes to the coin ﬂips outcomes, while the learner’s output bit is wrong with probability 1 − 1 /n − 1 /n

Original: = 1 − O (1 /n ) .
Revised: = 1 − O (1 /n ) .

Original: Since OPT ≤ 1 / 2 − η (cid:48) and ERR ≥ 1 − O (1 /n ) , we getERR
Revised: Since OPT ≤ 1 / 2 − η (cid:48) and ERR ≥ 1 − O (1 /n ) , we get

Original: [Equation] + [Equation] ) , which ﬁnishes the proof.
Revised: ERR − OPT ≥ 1 / 2 + η (cid:48) − O (1 /n ) , which ﬁnishes the proof.