Exponential Separation between Two Learning Models and Adversarial Robustness

1 Introduction . We analyze the connections between adversarial training , generalization and robustness . Our key technical contribution is a novel result in the field of learning theory , showing an exponential separation for the sample/query complexity of two learning models . This result provides a theoretical confirmation for the experimental observations in Stutz et al . [ 2019 ] and points to a fresh research direction in the field of adversarial robustness . Let us start with a discussion of our main technical contribution . The first model we consider is the PAC-learning introduced in Valiant [ 1984 ] . In this model the learner is given i.i.d . samples from an unknown distribution and is tasked with finding a hypothesis with low generalization error . The second model is a version of Equivalence-Query-learning ( EQ-learning ) introduced in [ Angluin , 1988 ] . In this setting samples are acquired through an interaction between a teacher and a learner . In each round of interaction the learner presents a hypothesis and the teacher replies with a counterexample for this hypothesis . Very close models to the model we study were considered previously in Angluin and Dohrn [ 2017 ] and Bshouty and Gavinsky [ 2002 ] . In this setting the counterexamples produced by the teacher come from the data distribution constrained to being in the error set of the presented hypothesis . We use ideas similar to those used in boosting techniques [ Schapire , 1990 ] to prove that in the EQ-learning model O ( d · polylog ( 1/✏ ) ) queries suffices to achieve a standard error of ✏ , where d is the VC-dimension of the hypothesis class . This improves on the previously known 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) , Sydney , Australia . bounds of O ( d/✏ ) . We refer the reader to Section 3 for an in depth review of previous literature on the subject . Standard results for the sample complexity in the PAC model yields an upper bound of O ( d/✏ ) . Comparing the two results we see the advertised , exponential ( in ✏ ) , separation . Let us discuss an “ application ” of our theorem to the relationship between adversarial robustness and generalization . In [ Tsipras et al. , 2019 , Su et al. , 2018 ] it is argued that there is an inherent trade-off between robustness and accuracy whereas in Gilmer et al . [ 2018 ] , Rozsa et al . [ 2016 ] the authors make the case that robustness improves with accuracy . This seeming contradiction is partially resolved in Stutz et al . [ 2019 ] where it is pointed out that it crucially depends on whether adversarial examples are on or off the data manifold . It is worth mentioning that at the beginning of the field of adversarial robustness it was argued that the reason for the existence of adversarial examples is exactly that they are off-manifold [ Tanay and Griffin , 2016 ] . However later on it was demonstrated that adversarial examples do also exist on-manifold [ Zhao et al. , 2018 , Song et al. , 2018 ] . The authors of Stutz et al . [ 2019 ] argue that it is important to understand this distinction . They introduce a version of adversarial training [ Goodfellow et al. , 2014 , Madry et al. , 2018 , Tramèr et al. , 2018 , Xiao et al. , 2019 ] , which they call on-manifold adversarial training , where the adversarial examples are constrained to lie on the data manifold . They then show empirically that using on-manifold adversarial training one achieves better generalization in comparison to standard training ( with the same number of samples ) . Our theorem can be seen as providing a theoretical support for this claim . We also explore connections of our results to adversarial robustness . Before we describe these connections in detail , let us position this work in a broader context . Most of the literature on adversarial robustness considers adversaries whose perturbations are in some way restricted . A very common form of restriction is to bound the perturbation in the ` 0 , ` 2 , or ` 1 norm [ Raghunathan et al. , 2018 , Wong and Kolter , 2018 ] , but other perturbations ( rotations , shifts , etc . ) were also considered [ Engstrom et al. , 2019 ] . A recent paper [ Goldwasser et al. , 2020 ] argues that in order to obtain security for real-world systems we need to consider models beyond these `` restricted perturbations . '' Unfortunately this change comes at a price : e.g. , the results in Goldwasser et al . [ 2020 ] are based on the assumption that the learner can decide not to give an answer for some inputs ( selective learning [ Chu , 1965 ] ) and that they see the test set upfront ( transductive learning ) , thus the capabilities of the learner are enhanced . We follow a third path . Instead of restricting the type of perturbations that can be applied by the adversary or enhancing the learner we require the adversary to return counterexamples according to the EQ-model . We then show that the exponential separation theorem leads to robustness in our model in the following way : Th main idea of cryptography is to base security on computational hardness of specific problems . We follows this idea and base the security of our scheme on the hardness of learning . More specifically , our adversarial learning scheme computes a classifier that is robust assuming that the underlying learning task is hard in the low error regime . 2 Models of learning and attack . We start by defining the models of learning considered in this paper . Notation . For i 2 N we define [ i ] : = [ 1 , 2 , . . . , i ] . We use log to denote the log to the base 2 . For a function f ! { 1 , +1 } and A ✓ X we define f A as a function that is equal to f at X \A and flips the prediction for points in A . For a distribution D on X and A ✓ X we denote by D|A the conditional distribution that is equal ( up to scaling ) to D on A and 0 on the complement . Throughout the paper we consider only the realizable version of learning . That is , we assume that there is a feature space X , a distribution D on X , a hypothesis class H and a function h 2 H , h : X ! { 1 , +1 } , that defines the ground truth on D. The learner knows X and H but neither knows h nor D. The goal of the learner is to find an f : X ! { 1 , +1 } that has small risk . The risk is defined as RD , h ( f ) : = Px⇠D [ f ( x ) 6= h ( x ) ] . We will consider algorithms that have access to one of the following two oracles : Example Query Oracle according to D ( EXD ) . When queried , EXD returns ( x , h ( x ) ) , where x ⇠ D. Equivalence Query Oracle according to D ( EQD ) [ Angluin , 1988 ] . For every f : X ! { 1 , +1 } ( not necessarily from H ) the result of querying EQD ( f ) is a counterexample to f dis- tributed according to D. More formally it is x ⇠ D|f 6=h . If RD , h ( f ) = 0 then EQD ( f ) returns `` YES '' indicating that f is equivalent to h. For every k 2 N we write EQD ( f , k ) to denote an oracle that returns k i.i.d . samples , each generated by EQD ( f ) . Next we define the two learning models for which we later show the advertised exponential separation . All our results are based on them . PAC-learning . We say that a learning algorithm L PAC-learns H if for every ✏ , 2 ( 0 , 1 ) there exists m ( ✏ , ) 2 N such that for every h 2 H and distribution D algorithm L ( ✏ , ) asks m ( ✏ , ) queries to the Example Query Oracle EXD and with probability 1 returns a function f 2 H such that RD , h ( f ) ✏ . EQ-learning We say that a learning algorithm L EQ-learns H if for every ✏ , 2 ( 0 , 1 ) there exists m ( ✏ , ) 2 N such that for every h 2 H and distribution D algorithm L ( ✏ , ) asks m ( ✏ , ) queries to the Equivalence Query Oracle EQD and with probability 1 returns a function f : X ! { 1 , +1 } such that RD , h ( f ) ✏ . 3 Exponential Separation . We start this section with a short literature review . It is worth mentioning that there is a slight difference between EQ-learning model and the model from Angluin and Dohrn [ 2017 ] . In Angluin and Dohrn [ 2017 ] X and H are finite , the learner has to return the exact hypothesis h that was chosen by the teacher , and the distribution on counterexamples is known , For us , on the other hand , X and H may be infinite , our goal is to return a function of small risk , and the distribution on counterexamples is not known . The model from Bshouty and Gavinsky [ 2002 ] , which is called Probably Almost Exact ( PAE ) is almost equal to the EQ-learning we consider . The difference is that in the PAE the goal of the learner is to return a hypothesis with negligible error ( 1/ ! ( poly ( n ) ) , where n measures measures the “ size ” of the input . In the EQ-learning on the other hand we want to explore the dependence on ✏ in detail . In Angluin and Dohrn [ 2017 ] it is shown that for a concept class |H| = n it is enough to ask log ( n ) queries . As we mentioned the setting is different and it ’ s not clear how to adapt this result to the EQ-learning , where we don ’ t need to find the hypothesis exactly ( apart from other differences mentioned above ) . In Bshouty and Gavinsky [ 2002 ] , where the model considered is closer to ours , it is shown that PAC-learnability = ) PAE-learnability . A direct application of their technique in our setting yields an upper bound on the query complexity , which is no better than O ( d/✏ ) . No simple modification of this technique would yield the desired O ( d · polylog ( 1/✏ ) ) query complexity . We significantly extend the ideas from Bshouty and Gavinsky [ 2002 ] to obtain our result . 3.1 Main result . We are now ready to state the main result of the paper , which is an exponential separation for the sample complexity between PAC-learning and EQ-learning . Theorem 1 . There exists a learning algorithm ( Algorithm 1 ) such that for every ✏ 2 0 , 132 , 2 ( 0 , 1 ) , every hypothesis class H of VC-dimension d , for every distribution D the algorithm invoked with parameters ✏ , , H EQ-learns H asking O ( ( d+ log ( 1/ ) ) log9 ( 1/✏ ) ) queries . To compare this result to the PAC model we recall that a slightly improved version of the PAC bound in the realizable case [ Hanneke , 2016 ] states that for a hypothesis class H of VC-dimension d in order to learn ( with constant probability ) a classifier of risk ✏ it suffices to use O d ✏ samples . This result is tight in a sense that there exist hypothesis classes and distributions for which that many samples are necessary . Theorem 1 guarantees that in the EQ-learning model O ( d · polylog ( 1/✏ ) ) many queries suffice . This is an exponential improvement . Why is this possible ? Imagine that there is a ground truth g 2 H that you try to learn and that you already found h 2 H such that RD , g ( h ) ⌘ . Then the counterexample oracle EQD ( h , · ) provides you with samples from a distribution D|h 6=g . Querying the oracle O ( d ) times you get a sample S EQD ( h , O ( d ) ) . You can now use the PAC bound : if you find an h0 2 H that is consistent with S then you know that it has an error of at most 1/2 on D|h 6=g . It is then natural to define Combine ( h , h0 ) ( x ) = ⇢ h ( x ) , if x 2 h = g h 0 ( x ) , if x 2 h 6= g . Note that Combine ( h , h0 ) has an error of at most ⌘/2 . If we repeated this procedure log ( 1/✏ ) times , thus asking O ( d log ( 1/✏ ) ) queries , you would find a classifier with error ✏ . Unfortunately it is not possible to compute Combine ( h , h0 ) . Afterall , if you knew the region of the input space where h 6= g then you could just flip the prediction of h in that region and the resulting classifier would have zero error . But it turns out that the general intuition of decreasing the error by a multiplicative factor after every O ( d ) queries can indeed be achieved . The key to this result is to find a computable version of Combine ( h , h0 , ... ) that guarantees an exponentially fast decay of the error . Our algorithm is mainly inspired by boosting techniques , most notably by an approach from Bshouty and Gavinsky [ 2002 ] . In this work the authors consider capabilities of polynomially bounded learners . If we assume that the `` complexity '' of the hypothesis class H is measured by its VC-dimension then the relevant result from Bshouty and Gavinsky [ 2002 ] can be summarized as follows . If we have an algorithm LPAC that learns H to a constant error in time ( the authors focus on time but the time is of course an upper-bound for the sample complexity ) time ( d ) then this algorithm can be boosted to an algorithm LEQ that learns H in the EQ-model to an error 1 ! ( poly ( d ) ) in time poly ( time ( d ) ) . Let us apply this boosting technique to our setting . Assume that LPAC runs in time O ( d ) ( as this is the number of samples that are required by the standard PAC bound to learn to constant error ) . Then the boosting algorithm can be used to produce LEQ that learns H to error ✏ . What bound on the run time of LEQ do we get ? Unfortunately this bound is no better than O d ✏ . This is exactly what the standard PAC bound provides in the first place . Thus , disappointingly , a direct application of these ideas do not yield a benefit in using the EQ-model versus using the PAC-model . To get the claimed exponential separation between the two models we develop a boosting-like algorithm that differs significantly in several important aspects and hence also requires a different proof technique . The main idea is to compute h1 , . . . , hO ( polylog ( 1/✏ ) ) 2 H in a sequential manner and then to define the final hypothesis as a version of a majority vote of these functions . The simplified , high level , structure of the algorithm is as follows . Repeat the following for t = O ( polylog ( 1/✏ ) ) steps : at step i ask the oracle for Si EQD ( `` Majority '' ( h1 , . . . , hi 1 ) , O ( d ) ) and then define hi : = FindConsistent ( Si , H ) ( FindConsistent returns a function from H that agrees with all samples from Si ) . At the end return `` Majority '' ( h1 , . . . , ht ) . To make this approach work we need to ensure that the error sets of h1 , . . . , ht are sufficiently independent . The following points need particular attention . First , note that the EQD provides the algorithm with samples from the error set only . Thus , correctly classified points at one stage will not automatically remain correctly classified at later stages . To make sure this happens , at every step i we include in the training set Si samples from carefully chosen regions of the feature space that are already classified correctly ( this is done in the inner `` for '' loop of the algorithm ) . Second , a simple majority vote of the previously constructed classifiers is not sufficient to get the desired result . This is true since a non-negligible region of the feature space might become incorrectly classified with higher and higher confidence by such a majority vote . This is the reason we clip the values of votes to a fixed interval ( for details see Definition 1 ) . We start now with the formal definition of the algorithm .

Summary: The paper studies the query complexity of learning a VC class under a certain query model called the Equivalence-Query (EQ) model by Angluin'98. In this model, given a distribution $\mathcal{D}$ and the ground truth $g$, the learner interacts with the data in the following way: (1) learner selects a function f and (2) learner receives a random counter-example distributed according to $\mathcal{D}$, that is, a random draw from $\mathcal{D}|_{f \ne g}$. Under this model, the authors prove an exponentially improved upper bound of $d \cdot \mathsf{polylog}(1/\epsilon)$ compared to the $d/\epsilon$ bound for PAC learning. The paper further draws connections between this model of learning and adversarial robustness. The main argument here relies on viewing the equivalence-query sample oracle as the adversary restricted to on-manifold attacks.