Exponential Separation between Two Learning Models and Adversarial Robustness

1 Introduction . We analyze the connections between adversarial training , generalization and robustness . Our key technical contribution is a novel result in the field of learning theory , showing an exponential separation for the sample/query complexity of two learning models . This result provides a theoretical confirmation for the experimental observations in Stutz et al . [ 2019 ] and points to a fresh research direction in the field of adversarial robustness . Let us start with a discussion of our main technical contribution . The first model we consider is the PAC-learning introduced in Valiant [ 1984 ] . In this model the learner is given i.i.d . samples from an unknown distribution and is tasked with finding a hypothesis with low generalization error . The second model is a version of Equivalence-Query-learning ( EQ-learning ) introduced in [ Angluin , 1988 ] . In this setting samples are acquired through an interaction between a teacher and a learner . In each round of interaction the learner presents a hypothesis and the teacher replies with a counterexample for this hypothesis . Very close models to the model we study were considered previously in Angluin and Dohrn [ 2017 ] and Bshouty and Gavinsky [ 2002 ] . In this setting the counterexamples produced by the teacher come from the data distribution constrained to being in the error set of the presented hypothesis . We use ideas similar to those used in boosting techniques [ Schapire , 1990 ] to prove that in the EQ-learning model O ( d · polylog ( 1/✏ ) ) queries suffices to achieve a standard error of ✏ , where d is the VC-dimension of the hypothesis class . This improves on the previously known 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , Sydney , Australia . bounds of O ( d/✏ ) . We refer the reader to Section 3 for an in depth review of previous literature on the subject . Standard results for the sample complexity in the PAC model yields an upper bound of O ( d/✏ ) . Comparing the two results we see the advertised , exponential ( in ✏ ) , separation . Let us discuss an “ application ” of our theorem to the relationship between adversarial robustness and generalization . In [ Tsipras et al. , 2019 , Su et al. , 2018 ] it is argued that there is an inherent trade-off between robustness and accuracy whereas in Gilmer et al . [ 2018 ] , Rozsa et al . [ 2016 ] the authors make the case that robustness improves with accuracy . This seeming contradiction is partially resolved in Stutz et al . [ 2019 ] where it is pointed out that it crucially depends on whether adversarial examples are on or off the data manifold . It is worth mentioning that at the beginning of the field of adversarial robustness it was argued that the reason for the existence of adversarial examples is exactly that they are off-manifold [ Tanay and Griffin , 2016 ] . However later on it was demonstrated that adversarial examples do also exist on-manifold [ Zhao et al. , 2018 , Song et al. , 2018 ] . The authors of Stutz et al . [ 2019 ] argue that it is important to understand this distinction . They introduce a version of adversarial training [ Goodfellow et al. , 2014 , Madry et al. , 2018 , Tramèr et al. , 2018 , Xiao et al. , 2019 ] , which they call on-manifold adversarial training , where the adversarial examples are constrained to lie on the data manifold . They then show empirically that using on-manifold adversarial training one achieves better generalization in comparison to standard training ( with the same number of samples ) . Our theorem can be seen as providing a theoretical support for this claim . We also explore connections of our results to adversarial robustness . Before we describe these connections in detail , let us position this work in a broader context . Most of the literature on adversarial robustness considers adversaries whose perturbations are in some way restricted . A very common form of restriction is to bound the perturbation in the ` 0 , ` 2 , or ` 1 norm [ Raghunathan et al. , 2018 , Wong and Kolter , 2018 ] , but other perturbations ( rotations , shifts , etc . ) were also considered [ Engstrom et al. , 2019 ] . A recent paper [ Goldwasser et al. , 2020 ] argues that in order to obtain security for real-world systems we need to consider models beyond these `` restricted perturbations . '' Unfortunately this change comes at a price : e.g. , the results in Goldwasser et al . [ 2020 ] are based on the assumption that the learner can decide not to give an answer for some inputs ( selective learning [ Chu , 1965 ] ) and that they see the test set upfront ( transductive learning ) , thus the capabilities of the learner are enhanced . We follow a third path . Instead of restricting the type of perturbations that can be applied by the adversary or enhancing the learner we require the adversary to return counterexamples according to the EQ-model . We then show that the exponential separation theorem leads to robustness in our model in the following way : Th main idea of cryptography is to base security on computational hardness of specific problems . We follows this idea and base the security of our scheme on the hardness of learning . More specifically , our adversarial learning scheme computes a classifier that is robust assuming that the underlying learning task is hard in the low error regime . 2 Models of learning and attack . We start by defining the models of learning considered in this paper . Notation . For i 2 N we define [ i ] : = [ 1 , 2 , . . . , i ] . We use log to denote the log to the base 2 . For a function f ! { 1 , +1 } and A ✓ X we define f A as a function that is equal to f at X \A and flips the prediction for points in A . For a distribution D on X and A ✓ X we denote by D|A the conditional distribution that is equal ( up to scaling ) to D on A and 0 on the complement . Throughout the paper we consider only the realizable version of learning . That is , we assume that there is a feature space X , a distribution D on X , a hypothesis class H and a function h 2 H , h : X ! { 1 , +1 } , that defines the ground truth on D. The learner knows X and H but neither knows h nor D. The goal of the learner is to find an f : X ! { 1 , +1 } that has small risk . The risk is defined as RD , h ( f ) : = Px⇠D [ f ( x ) 6= h ( x ) ] . We will consider algorithms that have access to one of the following two oracles : Example Query Oracle according to D ( EXD ) . When queried , EXD returns ( x , h ( x ) ) , where x ⇠ D. Equivalence Query Oracle according to D ( EQD ) [ Angluin , 1988 ] . For every f : X ! { 1 , +1 } ( not necessarily from H ) the result of querying EQD ( f ) is a counterexample to f dis- tributed according to D. More formally it is x ⇠ D|f 6=h . If RD , h ( f ) = 0 then EQD ( f ) returns `` YES '' indicating that f is equivalent to h. For every k 2 N we write EQD ( f , k ) to denote an oracle that returns k i.i.d . samples , each generated by EQD ( f ) . Next we define the two learning models for which we later show the advertised exponential separation . All our results are based on them . PAC-learning . We say that a learning algorithm L PAC-learns H if for every ✏ , 2 ( 0 , 1 ) there exists m ( ✏ , ) 2 N such that for every h 2 H and distribution D algorithm L ( ✏ , ) asks m ( ✏ , ) queries to the Example Query Oracle EXD and with probability 1 returns a function f 2 H such that RD , h ( f ) ✏ . EQ-learning We say that a learning algorithm L EQ-learns H if for every ✏ , 2 ( 0 , 1 ) there exists m ( ✏ , ) 2 N such that for every h 2 H and distribution D algorithm L ( ✏ , ) asks m ( ✏ , ) queries to the Equivalence Query Oracle EQD and with probability 1 returns a function f : X ! { 1 , +1 } such that RD , h ( f ) ✏ . 3 Exponential Separation . We start this section with a short literature review . It is worth mentioning that there is a slight difference between EQ-learning model and the model from Angluin and Dohrn [ 2017 ] . In Angluin and Dohrn [ 2017 ] X and H are finite , the learner has to return the exact hypothesis h that was chosen by the teacher , and the distribution on counterexamples is known , For us , on the other hand , X and H may be infinite , our goal is to return a function of small risk , and the distribution on counterexamples is not known . The model from Bshouty and Gavinsky [ 2002 ] , which is called Probably Almost Exact ( PAE ) is almost equal to the EQ-learning we consider . The difference is that in the PAE the goal of the learner is to return a hypothesis with negligible error ( 1/ ! ( poly ( n ) ) , where n measures measures the “ size ” of the input . In the EQ-learning on the other hand we want to explore the dependence on ✏ in detail . In Angluin and Dohrn [ 2017 ] it is shown that for a concept class |H| = n it is enough to ask log ( n ) queries . As we mentioned the setting is different and it ’ s not clear how to adapt this result to the EQ-learning , where we don ’ t need to find the hypothesis exactly ( apart from other differences mentioned above ) . In Bshouty and Gavinsky [ 2002 ] , where the model considered is closer to ours , it is shown that PAC-learnability = ) PAE-learnability . A direct application of their technique in our setting yields an upper bound on the query complexity , which is no better than O ( d/✏ ) . No simple modification of this technique would yield the desired O ( d · polylog ( 1/✏ ) ) query complexity . We significantly extend the ideas from Bshouty and Gavinsky [ 2002 ] to obtain our result . 3.1 Main result . We are now ready to state the main result of the paper , which is an exponential separation for the sample complexity between PAC-learning and EQ-learning . Theorem 1 . There exists a learning algorithm ( Algorithm 1 ) such that for every ✏ 2 0 , 132 , 2 ( 0 , 1 ) , every hypothesis class H of VC-dimension d , for every distribution D the algorithm invoked with parameters ✏ , , H EQ-learns H asking O ( ( d+ log ( 1/ ) ) log9 ( 1/✏ ) ) queries . To compare this result to the PAC model we recall that a slightly improved version of the PAC bound in the realizable case [ Hanneke , 2016 ] states that for a hypothesis class H of VC-dimension d in order to learn ( with constant probability ) a classifier of risk ✏ it suffices to use O d ✏ samples . This result is tight in a sense that there exist hypothesis classes and distributions for which that many samples are necessary . Theorem 1 guarantees that in the EQ-learning model O ( d · polylog ( 1/✏ ) ) many queries suffice . This is an exponential improvement . Why is this possible ? Imagine that there is a ground truth g 2 H that you try to learn and that you already found h 2 H such that RD , g ( h ) ⌘ . Then the counterexample oracle EQD ( h , · ) provides you with samples from a distribution D|h 6=g . Querying the oracle O ( d ) times you get a sample S EQD ( h , O ( d ) ) . You can now use the PAC bound : if you find an h0 2 H that is consistent with S then you know that it has an error of at most 1/2 on D|h 6=g . It is then natural to define Combine ( h , h0 ) ( x ) = ⇢ h ( x ) , if x 2 h = g h 0 ( x ) , if x 2 h 6= g . Note that Combine ( h , h0 ) has an error of at most ⌘/2 . If we repeated this procedure log ( 1/✏ ) times , thus asking O ( d log ( 1/✏ ) ) queries , you would find a classifier with error ✏ . Unfortunately it is not possible to compute Combine ( h , h0 ) . Afterall , if you knew the region of the input space where h 6= g then you could just flip the prediction of h in that region and the resulting classifier would have zero error . But it turns out that the general intuition of decreasing the error by a multiplicative factor after every O ( d ) queries can indeed be achieved . The key to this result is to find a computable version of Combine ( h , h0 , ... ) that guarantees an exponentially fast decay of the error . Our algorithm is mainly inspired by boosting techniques , most notably by an approach from Bshouty and Gavinsky [ 2002 ] . In this work the authors consider capabilities of polynomially bounded learners . If we assume that the `` complexity '' of the hypothesis class H is measured by its VC-dimension then the relevant result from Bshouty and Gavinsky [ 2002 ] can be summarized as follows . If we have an algorithm LPAC that learns H to a constant error in time ( the authors focus on time but the time is of course an upper-bound for the sample complexity ) time ( d ) then this algorithm can be boosted to an algorithm LEQ that learns H in the EQ-model to an error 1 ! ( poly ( d ) ) in time poly ( time ( d ) ) . Let us apply this boosting technique to our setting . Assume that LPAC runs in time O ( d ) ( as this is the number of samples that are required by the standard PAC bound to learn to constant error ) . Then the boosting algorithm can be used to produce LEQ that learns H to error ✏ . What bound on the run time of LEQ do we get ? Unfortunately this bound is no better than O d ✏ . This is exactly what the standard PAC bound provides in the first place . Thus , disappointingly , a direct application of these ideas do not yield a benefit in using the EQ-model versus using the PAC-model . To get the claimed exponential separation between the two models we develop a boosting-like algorithm that differs significantly in several important aspects and hence also requires a different proof technique . The main idea is to compute h1 , . . . , hO ( polylog ( 1/✏ ) ) 2 H in a sequential manner and then to define the final hypothesis as a version of a majority vote of these functions . The simplified , high level , structure of the algorithm is as follows . Repeat the following for t = O ( polylog ( 1/✏ ) ) steps : at step i ask the oracle for Si EQD ( `` Majority '' ( h1 , . . . , hi 1 ) , O ( d ) ) and then define hi : = FindConsistent ( Si , H ) ( FindConsistent returns a function from H that agrees with all samples from Si ) . At the end return `` Majority '' ( h1 , . . . , ht ) . To make this approach work we need to ensure that the error sets of h1 , . . . , ht are sufficiently independent . The following points need particular attention . First , note that the EQD provides the algorithm with samples from the error set only . Thus , correctly classified points at one stage will not automatically remain correctly classified at later stages . To make sure this happens , at every step i we include in the training set Si samples from carefully chosen regions of the feature space that are already classified correctly ( this is done in the inner `` for '' loop of the algorithm ) . Second , a simple majority vote of the previously constructed classifiers is not sufficient to get the desired result . This is true since a non-negligible region of the feature space might become incorrectly classified with higher and higher confidence by such a majority vote . This is the reason we clip the values of votes to a fixed interval ( for details see Definition 1 ) . We start now with the formal definition of the algorithm .

Summary: This paper introduces a new abstract learning model called the EQ-learning, in which the oracle returns examples that the current classifier gets incorrect (rather than arbitrary examples drawn from the data distribution) and returns YES if no such examples exist (indicating the current classifier is correct). They then provide an algorithm which achieves a sample complexity (assuming finite VC dimension d)that is polylog in $1/\epsilon$ as opposed to $O(1/\epsilon)$. This is an exponential increase, and is a very interesting theoretical result on its own.  They then argue how their formalism plays a role in learning from on-manifold adversarial examples. In particular, each of these examples reveals a point that the classifier gets incorrect, and their result consequently suggests that learning from these results should improve the training procedure.