Why Robust Generalization in Deep Learning is Difﬁcult: Perspective of Expressive Power
Abstract
It is well-known that modern neural networks are vulnerable to adversarial examples.
To mitigate this problem, a series of robust learning algorithms have been proposed.
However, although the robust training error can be near zero via some methods, all existing algorithms lead to a high robust generalization error.
In this paper, we provide a theoretical understanding of this puzzling phenomenon from the perspective of expressive power for deep neural networks.
Speciﬁcally, for binary classiﬁcation problems with well-separated data, we show that, for ReLU networks, while mild over-parameterization is sufﬁcient for high robust training accuracy, there exists a constant robust generalization gap unless the size of the neural network is exponential in the data dimension d .
This result holds even if the data is linear separable (which means achieving standard generalization is easy), and more generally for any parameterized function classes as long as their
VC dimension is at most polynomial in the number of parameters.
Moreover, we establish an improved upper bound of exp( O ( k )) for the network size to achieve low robust generalization error when the data lies on a manifold with intrinsic dimension k ( k (cid:28) d ).
Nonetheless, we also have a lower bound that grows exponentially with respect to k — the curse of dimensionality is inevitable.
By demonstrating an exponential separation between the network size for achieving low robust training and generalization error, our results reveal that the hardness of robust generalization may stem from the expressive power of practical models.
Introduction
Deep neural networks have achieved remarkable success in a variety of disciplines including computer vision (Voulodimos et al., 2018), natural language processing (Devlin et al., 2018) as well as scientiﬁc and engineering applications (Jumper et al., 2021).
However, it is observed that neural networks are often sensitive to small adversarial attacks (Biggio et al., 2013; Szegedy et al., 2013;
Goodfellow et al., 2014), which potentially gives rise to reliability and security problems in realworld applications.
In light of this pitfall, it is highly desirable to obtain classiﬁers that are robust to small but adversarial perturbations.
A common approach is to design adversarial training algorithms by using adversarial examples as training data (Madry et al., 2017; Tramèr et al., 2018;
Shafahi et al., 2019).
Another line of works (Cohen et al., 2019; Zhang et al., 2021a) proposes some provably robust models to tackle this problem.
However, while the state-of-the-art adversarial training methods can achieve high robust training accuracy (e.g. nearly 100% on CIFAR-10 (Raghunathan et al., 2019)), all existing methods suffer from large robust test error.
Therefore, it is natural to ask what is the cause for such a large generalization gap in the context of robust learning.
Previous works have studied the hardness of achieving adversarial robustness from different perspectives.
A well-known phenomenon called the robustness-accuracy tradeoff has been empirically observed (Raghunathan et al., 2019) and theoretically proven to occur in different settings (Tsipras et al., 2019; Zhang et al., 2019).
Dohmatob (2019) shows that adversarial robustness is impossible to achieve under certain assumptions on the data distribution, while it is shown in Nakkiran (2019) that even when an adversarial robust classiﬁer does exist, it can be exponentially more complex than its non-robust counterpart.
Hassani and Javanmard (2022) studies the role of over-parameterization on adversarial robustness by focusing on random features regression models.
At ﬁrst glance, these works seem to provide convincing evidence that robustness is hard to achieve in general.
However, this view is challenged by Yang et al.
(2020), who observes that for real data sets, different classes tend to be well-separated (as deﬁned below), while the perturbation radius is often much smaller than the separation distance.
As pointed out by Yang et al.
(2020), all aforementioned works fail to take this separability property of data into consideration.
Deﬁnition 1.1 (Separated data) .
Suppose that A, B (cid:26) R d and ϵ > 0 .
We say that A, B are ϵseparated under ℓ p norm ( 1 (cid:20) p (cid:20) + 1 ) if k
[Equation]
Indeed, this assumption is necessary to ensure the existence of a robust classiﬁer.
Without this separated condition, it is clear that there is no robust classiﬁer even if a non-robust classiﬁer always exists, as discussed above.
Recently, Bubeck and Sellke (2021) shows that for regression problems, over-parameterization may be necessary for achieving robustness.
However, they measure robustness of a model by its training error and Lipschitz constant, which has a subtle difference with robust test error (Madry et al., 2017); see the discussions in (Bubeck and Sellke, 2021, Section 1.1).
To sum up the above, although existing robust training algorithms result in low robust test accuracy, previous works do not provide a satisfactory explanation of this phenomenon, since there exists a gap between their settings and practice.
In particular, it is not known whether achieving robustness can be easier for data with additional structural properties such as separability (Yang et al., 2020) and low intrinsic dimensionality
(Gong et al., 2019).
In this paper, we make an important step towards understanding robust generalization from the viewpoint of neural network expressivity.
Focusing on binary classiﬁcation problems with separated data (cf. Deﬁnition 1.1) in R d , we make the following contributions:
• Given a data set D of size N that satisﬁes a separability condition, we show in Sectionthat it is possible for a ReLU network with ˜ O ( Nd ) weights to robustly classify D .
In other words, an over-parameterized ReLU network with reasonable size can achieve 100% robust training accuracy.
We next consider the robust test error (cf.
Deﬁnition 3.1).
As a warm-up, we show in Section 3 that, in contrast with the robust training error, mere separability of data does not imply that low robust test error can be attained by neural networks, unless their size is exponential in d .
This motivates the subsequent sections where we consider data with additional structures.
• In Section 4, we prove the main result of this paper, which states that for achieving low robust test error, an exp(Ω( d )) lower bound on the network size is inevitable, even when the underlying distributions of the two classes are linear separable.
Moreover, this lower bound holds for arbitrarily small perturbation radius and more general models as long as their VC dimension is at most polynomial in the number of parameters.
• Finally, in Section 5 we consider data that lies on a k -dimensional manifold ( k (cid:28) d ), and prove an improved upper bound exp( O ( k )) for the size of neural networks for achieving [Table] low robust test error.
Nonetheless, the curse of dimensionality is inescapable – the lower bound is also exponential in k .
The upper and lower bounds on network size are summarized in Table 1.
Overall, our theoretical analysis suggests that the hardness of achieving robust generalization may stem from the expressive power of practical models.
1.1 Implications of our results
Before moving on to technical parts, we would like to ﬁrst discuss the implications of our results by comparing them to previous works.
Our main result is the exponential lower bound on the neural network size for generalization.
First, different from previous hardness results, our result is established for data sets that have desirable structural properties, hence more closely related to practical settings.
Note that the separability condition implies that there exists a classiﬁer that can perfectly and robustly classify the data i.e. achieve zero robust test error.
However, we show that such classiﬁer is hard to approximate using neural networks with moderate size.
Second, it is a popular belief that many real-world data sets are intrinsically low dimensional, although they lie in a high dimensional space.
Our results imply that low dimensional structure makes robust generalization possible with a neural network with signiﬁcantly smaller size (when k (cid:28) d ).
However, the size must still be exponential in k .
Finally, we show that there exists an exponential separation between the required size of neural networks for achieving low robust training and test error.
Based on our results, we conjecture that the widely observed drop of robust test accuracy is not due to limitations of existing algorithms – rather, it is a more fundamental issue originating from the expressive power of neural networks.
1.2 Related works
Robust Generalization.
One surprising behavior of deep learning is that over-parameterized neural networks can generalize well despite their ability to ﬁt random data (Zhang et al., 2017; Belkin et al., 2019). However, in contrast to the standard (non-robust) generalization, for the robust setting, Rice et al. (2020) empirically investigates robust performance of models based on adversarial training methods, which are designed to improve adversarial robustness (Szegedy et al., 2013; Madry et al., 2017), and shows that robust overﬁtting can be observed on multiple datasets. From the theoretical side, Madry et al.
(2017) proposes the notion of robust test error to measure the performance of a model under adversarial attacks, and the required sample complexity is studied in various settings (Schmidt et al., 2018;
Bhagoji et al., 2019; Dan et al., 2020; Bhattacharjee et al., 2021).
In this paper, we mainly focus on this robust generalization gap and provide a theoretical understanding from the perspective of expressive power.
Robust interpolation.
Bubeck et al.
(2021) proposes a conjecture that over-parameterization is necessary for smooth interpolation.
Then Bubeck and Sellke (2021) establishes a law of robustness for isoperimetric data.
Speciﬁcally, they prove an ˜Ω( p
Nd/p ) Lipschitzness lower bound for smooth interpolation, where N, d , and p denote the sample size, the inputs’ dimension, and the number of parameters, respectively.
This result indicates that over-parameterization may be necessary for robust learning.
Zhang et al.
(2022) studies many data are needed for robust interpolation.
This line of works focuses on the training error and the worst-case robustness ( i.e. Lipschitz constant), while we measure robustness via the robust generalization error.
Memorization power of neural networks.
Our work is related to another line of works (e.g., Baum, 1988; Yun et al., 2019;
Bubeck et al., 2020; Zhang et al., 2021a; Rajput et al., 2021; Zhang et al., 2021b; Vardi et al., 2021) on the memorization power of neural networks.
Among these works, Yun et al.
(2019) shows that a neural network with O ( N ) parameters can memorize the data set with zero error, where N is the size of the data set.
Under an additional separable assumption, Vardi et al.
(2021) derives an improved upper bound of ˜ O
( p N ) , which is shown to be optimal.
In this work, we show that ˜ O ( Nd ) parameters is sufﬁcient for achieving low robust training error.
This is in contrast with our exponential lower bound for low robust test error.
Function approximation.
Our work is related to a line of works on function approximation via neural networks (e.g., Cybenko, 1989; Hornik, 1991;
Lu et al., 2017; Yarotsky, 2017; Hanin, 2019).
Yarotsky (2017) is the most related, which shows that the functions in Sobolev spaces can be uniformly approximated by deep ReLU networks.
Also related is the studies of using deep ReLU networks to approximate functions supported on low dimensional manifolds (Chui and Mhaskar, 2018; Shaham et {+al., 
+}
; Chen et al., 2019).
In particular, Chen et al.
(2019) proves that any C n function in Hölder spaces can be ϵ -approximated by the neural network with size O ( ϵ − k/n ) , where k is the intrinsic dimension of the manifold embedded in R d .
In the robust classiﬁcation scenario, we can also achieve dimensionality reduction for low-dimensional data.
1.3 Notations
Throughout this paper, we use k(cid:1)k p , p 2 [1 , + 1 ] to denote the ℓ p norm in the vector space R d .
For x 2 R d and A (cid:26) R d , we can deﬁne the distance between x and A as d p ( x , A ) =
inf fk x (cid:0) y k p : y
A g .
For r > 0 , B p ( x , r ) =
[Equation]
p
[Equation]
o is deﬁned as the ℓ p -ball with radius r centered at x .
For a function class F , we use d V C ( F ) to denote its VC-dimension.
A multilayer neural network is a function from input x 2 R d to output y 2 R m , recursively deﬁned as follows:
[Equation]
where σ is the activation function and L is the depth of the neural network.
In this paper, we mainly focus on ReLU networks i.e. σ ( x ) = max f 0 , x g .
The size of a neural network is deﬁned as its number of weights/parameters i.e. the number of its non-zero connections between layers.
Mild Over-parameterized ReLU Nets Achieve Zero Robust Training Error
With access to only ﬁnite amount of data, a common practice for learning a robust classiﬁer is to minimize the robust training error (deﬁned below).
In this section, we show that neural networks with reasonable size can achieve zero robust training error on a ﬁnite training set.
Deﬁnition 2.1 (Robust training error) .
Given a data set D = f
( x i , y i ) g 1 ≤ i ≤ N , y i 2 f(cid:0) 1 , +1 g and an adversarial perturbation radius δ (cid:21) 0 , the robust training error of a classiﬁer f is deﬁned as
[Equation]
When δ = 0 , the deﬁnition coincides with the standard training error.
In this paper, we mainly focus on the case p = 2 and p = 1 , but our results can be extended to general p as well.
The following is our main result in this section, which states that for binary classiﬁcation problems, a neural network with e O ( Nd ) weights can perfectly achieve robust classiﬁcation on a data set of size N .
The detailed proof is deferred to Appendix B.3.
Theorem 2.2.
Suppose that D (cid:26) B p (0 , 1) with p 2 f 2 , + 1g consists of N data, and the two classes in D are 2 ϵ -separated (cf.
Deﬁnition 1.1), where ϵ 2 (cid:0) 0 , 12 (cid:1) is a constant.
Let robustness radius δ <
12 ϵ , then there exists a classiﬁer f represented by a ReLU network with at most
[Equation]
parameters, such that ˆ L p,δ D ( f ) = 0 .
Theorem 2.2 implies that neural networks is quite efﬁcient for robust classiﬁcation of ﬁnite training data.
We also derive a lower bound in the same setting, which is stated below.
It is an interesting future direction to study whether this lower bound can be achieved.
Theorem 2.3.
Let p 2 f 2 , + 1g and F n be the set of functions represented by ReLU networks with at most n parameters.
For arbitrary 2 ϵ -separated data set D under ℓ p norm, if there exists a classiﬁer f 2 F n such that ˆ L p,δ D ( f ) = 0 , then it must hold that n = Ω( p Nd ) .
The detailed proof of Theorem 2.3 is in Appendix B.4.
We leave it as a future direction to study whether this lower bound can be attained.
While optimal (non-robust) memorization of N data points only needs constant width (Vardi et al., 2021), our construction in Theorem 2.2 has width e O ( Nd ) .
Therefore, if our upper bound is tight, then Theorem 2.2 can probably explain why increasing the network width can beneﬁt robust training (Madry et al., 2017).
Hardness of Robust Generalization : A Warm-up
In the previous section, we give an upper bound on the size of ReLU networks to robustly classify ﬁnite training data.
However, it says nothing about generalization , or the robust test error, which is arguably a crucial aspect of evaluating the performance of a trained model.
As a warm-up, in this section we ﬁrst consider the same setting as Section 2 where we only assume the data to be well-separated.
We show that in this setting, even achieving high standard test accuracy requires exponentially large neural networks in the worst case, which is quite different from empirical observations.
This motivates to consider data with additional structures in subsequent sections.
Deﬁnition 3.1 (Robust test error) .
Given a probability measure P on R d (cid:2) f(cid:0) 1 , +1 g and a robust radius δ (cid:21) 0 , the robust test error of a classiﬁer f : R d !
R w.r.t P and δ under ℓ p norm is deﬁned as L p,δP ( f ) =
E ( x ,y )
[Equation]
In contrast with the training set which only consists of ﬁnite data points, when studying generalization, we must consider potentially inﬁnite points in the classes that we need to classify.
As a result, we consider two disjoint sets A, B 2 [0 , 1] d , where points in A have label +1 and points in B have label (cid:0) 1 .
We are interested in the following questions:
• Does there exists a robust classiﬁer of A and B ?
• If so, can we derive upper and lower bounds on the size of a neural network to robustly classify A and B ?
It turns out that, similar to the previous section, the ϵ -separated condition (cf.
Deﬁnition 1.1) ensures the existence of such a classiﬁer.
Moreover, it can be realized by a Lipschitz function.
This fact has been observed in Yang et al.
and we provide a different version of their result below for completeness.
Proposition 3.2.
For 2 ϵ -separated A, B (cid:26) [0 , 1] d under ℓ p norm with p 2 f 2 , + 1g , the classiﬁer f ∗ ( x ) := d p ( x ,B ) − d p ( x ,A ) d p ( x ,A )+ d p ( x ,B ) is ϵ − 1 -Lipschitz continuous, and satisﬁes L p,εP ( f ∗ ) = 0 for any probability distribution P on A [ B .
Based on this observation, Yang et al.
(2020) concludes that adversarial training is not inherently hard.
Rather, they argue that current pessimistic results on robust test error is due to the limits of existing algorithms.
However, it remains unclear whether the Lipschitz function constructed in
Proposition 3.2 can actually be efﬁciently approximated by neural networks.
The following theorem shows that ReLU networks with exponential size is sufﬁcient for as robust classiﬁcation.
Theorem 3.3.
For any two 2 ϵ -separated A, B (cid:26) [0 , 1] d under ℓ p norm with p 2 f 2 , + 1g , distribution P on the supporting set S =
A [ B and robust radius c 2 (0 , 1) , there exists a ReLU network f with at most ˜ O (cid:0) ((1 (cid:0) c ) ϵ ) − d (cid:1) parameters, such that L p,cϵP ( f ) = 0 .
The detailed proof is deferred to Appendix C.1.
Indeed, it is well known that without additional assumptions, an exponentially large number of parameters is also necessary for approximating
a
Lipschitz function (DeVore et al., 1989; Shen et al., 2022).
This result motivates us to consider the second question listed above.
The following result implies that even without requiring robustness, neural networks need to be exponentially large to correctly classify A and B :
Theorem 3.4.
Let F n be the set of functions represented by ReLU networks with at most n parameters.
Suppose that for any 2 ϵ -separated sets A, B (cid:26) [0 , 1] d under ℓ p norm with p 2 f 2 , + 1g , there exists f 2 F n that can classify A, B with zero (standard) test error, then it must hold that n =
[Equation]
Theorem 3.4 implies that mere separability of data sets is insufﬁcient to guarantee that they can be classiﬁed by ReLU networks, unless the network size is exponentially large.
The detailed proof is in Appendix C.2.
However, one should be careful when interpreting the conclusion of Theorem 3.4, since real-world data sets may possess additional structural properties.
Theorem 3.4 does not take these properties into consideration, so it does not rule out the possibility that these additional properties make robust classiﬁcation possible.
Speciﬁcally, the joint distribution of data can be decomposed as
[Equation]
where P ( X, Y ) , P ( Y j X ) , and P ( X ) denote the joint, conditional and marginal distributions, respectively.
In subsequent sections, we consider two well-known properties of data sets that correspond to the labeling mapping structure (Section 4) and the input structure (Section 5), respectively, and study whether they can bring improvement to neural networks’ efﬁciency for robust classiﬁcation.
Robust Generalization for Linear Separable Data
We have seen that for separated data, if no other structural properties are taken into consideration, even standard generalization requires exponentially large neural networks.
However, in practice it is often possible to train neural networks that can achieve fairly high standard test accuracy, indicating a gap between the setting of Section 3 and practice.
This motivates us to consider the following question: assuming that there exists a simple classiﬁer that achieves zero standard test error on the data, is it guaranteed that neural networks with reasonable size can also achieve high robust test accuracy?
We give a negative answer to this question.
Namely, we show that even in the arguably simplest setting where the given data is linear separable and well-separated (cf.
Deﬁnition 1.1), ReLU networks still need to be exponentially large to achieve high robust test accuracy.
4.1 Main results under the linear separable assumption
Clearly, the robust test error (cf.
Deﬁnition 3.1) depends on the underlying distribution P .
We consider a class of data distributions which have bounded density ratio with the uniform distribution:
Deﬁnition 4.1 (Balanced distribution) .
Let S (cid:26) R n such that there exists a uniform probability measure m 0 on S .
A distribution P on S is called µ -balanced if [Equation]
Remark 4.2.
Deﬁnition 4.1 has also appeared in (Shafahi et al., 2018, Theorem 1), which gives an impossibility result on robust classiﬁcation, albeit in a completely different setting.
Intuitively, it rules out the possibility that data points in certain regions are heavily under-represented.
The following theorem is the main result of this paper, and the proof sketch is deferred to Section 4.3.
Theorem 4.3.
Let ϵ 2 (0 , 1) be a small constant, p 2 f 2 , + 1g and F n be the set of functions represented by ReLU networks with at most n parameters.
There exists a sequence N d = Ω (cid:16) (2 ϵ ) − d −(cid:17) , d (cid:21) 1 and a universal constant C 1 > 0 such that the following holds: for any c 2 (0 , 1) , there exists two linear separable sets A, B (cid:26) [0 , 1] d ϵ separated ℓ such that for any µ 0 -balanced distribution P on the supporting set S =
A [ B and robust radius cϵ we have
[Equation]
Theorem 4.3 states that the robust test error is lower-bounded by a positive constant α = C 1 µunless the ReLU network has size larger than exp(Ω( d )) .
On the contrary, if we do not require robustness, then the data can be classiﬁed by a simple linear function.
Moreover, this classiﬁer can be learned with a poly-time efﬁcient algorithm
(The detailed proof is in Appendix D.2) :
Theorem 4.4.
For any two linear-separable A, B (cid:26) [0 , 1] d , a distribution P on the supporting set
S =
A [ B , δ > 0 and β > 0 , let H be the family of d (cid:0) dimensional hyperplane classiﬁers.
Then, there exists a poly-time efﬁcient algorithm A : 2 S !
H , for N = Ω( d/β 2 ) training instances independently randomly sampled from P , with probability 1 (cid:0) δ over samples, we can use the algorithm A to learn a classiﬁer ˆ f 2 F such that
[Equation]
where L P ( f ) := P ( x ,y ) ∼ P f y 6 = f ( x ) g denotes the standard test error.
The practical implication of Theorem 4.3 is two-fold.
First, by comparing with Theorem 4.4, one can conclude that robust classiﬁcation may require exponentially more parameters than the nonrobust case, which is consistent with the common practice that larger models are used for adversarial robust training.
Second, together with our upper bound in Theorem 2.2, Theorem 4.3 implies an exponential separation of neural network size for achieving high robust training and test accuracy.
4.2 Exponential lower bound for more general models
In general, our lower bounds hold true for a variety of neural network families and other function classes as well, as long as their VC dimension is at most polynomial in the number of parameters, which is formally stated as Theorem 4.5 that can be derived by the proof of Theorem 4.3 directly. Theorem 4.5. Let ϵ 2 (0 , 1) be a small constant, p 2 f 2 , + 1g and G n be the family of parameterized models with at most n parameters, satisfying the VC-dimension of function family VC-dim ( G n ) is at most poly( n ) .
Then, there exists a sequence N d = exp(Ω( d )) , d (cid:21) 1 and a universal constant
C ′ 1 > 0 such that the following holds: for any c 2 (0 , 1) , there exists two linear separable sets
A, B (cid:26) [0 , 1] d that are 2 ϵ -separated under ℓ p norm, such that for any µ 0 -balanced distribution P on the supporting set S =
A [ B and robust radius cϵ we have
[Equation]
In other words, the robust generalization error cannot be lower that a constant α = C ′ 1 µ 0 unless the model, satisfying the property of their VC dimension polynomially bounded by the number of parameters, has exponential larger size.
Indeed, this property is satisﬁed for e.g. feedforward neural networks with sigmoid (Karpinski and Macintyre, 1995) and piecewise polynomial (Bartlett et al., 2019) activation functions.
Therefore, our results reveal that the hardness of robust generalization may stem from the expressive power of generally practical models.
4.3 Proof sketch of Theorem 4.
In this subsection, we present a proof sketch for Theorem 4.3 in the ℓ ∞ -norm case.
We only consider
P to be the uniform distribution, extending to µ 0 -balanced distributions is not difﬁcult, The case of ℓ 2 -norm is similar and can be found in the Appendix.
Proof Sketch.
Let K
=
b 12 ε c , and ϕ : f 1 , 2 , (cid:1) (cid:1) (cid:1) , K g d − 1 !
f(cid:0) 1 ,
+1 g be an arbitrary mapping, we deﬁne S ϕ = n(cid:16)
i K , i 2 K , (cid:1) (cid:1) (cid:1) , i d − 1 K , 12 + ϵ 0 (cid:1) ϕ
( i 1 , i 2 , (cid:1) (cid:1) (cid:1) , i d − 1 ) (cid:17) : 1 (cid:20) i 1 , i 2 , (cid:1) (cid:1) (cid:1) , i d − 1 (cid:20) K o , where ϵ 0 is an arbitrarily small constant.
The hyperplane x ( d ) = 12 partitions S ϕ into two subsets, which we denote by A ϕ and B ϕ .
It is not difﬁcult to check that A ϕ and B ϕ satisﬁes all the required conditions.
Our goal is to show that there exists some choice of ϕ such that robust classiﬁcation is hard.
To begin with, suppose that robust classiﬁcation with accuracy 1 (cid:0) α can be achieved with at most M parameters for all ϕ , then these networks can all be embedded into an enveloping network F θ of size
O ( M 3 ) .
Deﬁne
[Equation]
Robustness implies that for all possible label assignment to ˜ S , at least (1 (cid:0) α ) K
d − 1 points can be correctly classiﬁed by F θ .
If α = 0 i.e. perfect classiﬁcation is required, then we can see that the VC-dim ( F θ ) (cid:21) K d − 1 , which implies that its size must be exponential, by applying classical VC-dimension upper bounds of neural networks (Bartlett et al., 2019).
When α > 0 , we cannot directly use the bound on VC-dimension.
Instead, we use a double-counting argument to lower-bound the growth function of some subset of ˜ S .
Let V
= 12 K d − 1 .
Each choice of ϕ corresponds to (cid:0) (1 − α )
K d − 1 V (cid:1) labelled V -subset of ˜ S that are correctly classiﬁed.
There are a total of 2 K d − 1 choices of ϕ , while each labelled V -subset can be obtained by at most 2 K d − 1 − V different ϕ .
As a result, the total number of labelled V -subset correctly classiﬁed by F θ is at least 2 V
[Equation]
On the other hand, the total number of V -subset of e S is (cid:0) K
d
−
1 V (cid:1) , thus there must exists a V -subset
V 0 (cid:26) e S , such that at least (cid:18)
[Equation]
different labellings of V 0 are correctly classiﬁed by F θ , where C α = [Equation]
Since (1) provides a lower bound for the growth function, together with the upper bound of growth function in terms of VC-dimension, we can deduce that VC-dim ( F θ ) (cid:21) 0 .
K d − 1 .
Finally, the conclusion follows by applying the VC-dimension bounds in Bartlett et al.
Remark 4.6.
The connection between VC dimension and approximation error has been explored in a number of previous works (Yarotsky, 2017; Shen et al., 2022) to provide lower bounds on the network size for approximating a given function class.
Here we consider the problem of robust classiﬁcation which is of more practical interest then function approximation, and our main technical contribution is an exponential lower bound on the VC dimension.
Our proof formalizes the folklore that adversarial training is hard since it requires a more complicated decision boundary. We note that similar ideas have been used to show beneﬁts of depth in neural networks (Telgarsky, 2016; Liang and Srikant, 2017) but their techniques are restricted to one-dimensional functions.  Robust Generalization for Low-Dimensional-Manifold Data
In this section, we focus on reﬁned structure of data’s input distribution P ( X ) .
A common belief of real-life data such as images is that the data points lie on a low-dimensional manifold.
It promotes a series of methods that are invented to make the dimensionality reduction, including linear dimensionality reduction (e.g., PCA (Pearson, 1901)) and non-linear dimensionality reduction (e.g., t -SNE (Hinton and van der Maaten, 2008)).
Several works have also empirically veriﬁed the belief.
Roweis and Saul (2000) and Tenenbaum et al.
(2000) have demonstrated that image, speech and other variant form data can be modeled nearly on low-dimensional manifolds.
In particular, Wang et al.
(2016) studies auto-encoder based dimensionality reduction, and shows that the 28 (cid:2)
=dimensional image from MNIST can be reduced to nearly 10 dimensional representations, which corresponds to the intrinsic dimension of the handwritten digital dataset.
Motivated by these ﬁndings, in this section, we assume that data lies on a low-dimensional manifold M embedded in [0 , 1] d i.e. supp( X ) (cid:26) M (cid:26) [0 , 1] d .
We will show a improved upper bound that is exponential in the intrinsic k of the manifold M instead of the ambient data dimension d for the size of networks achieving zero robust test error, which implies the efﬁciency of robust classiﬁcation under the low-dimensional manifold assumption.
Also, we point out that the exponential dependence of k is not improvable by establishing a matching lower bound.
5.1 Preliminaries
Let M be a k (cid:0) dimensional compact Riemannian manifold embedded in R d , where k is the intrinsic dimension ( k (cid:28) d ).
Deﬁnition 5.1 (Chart, atlas and smooth manifold) .
A chart for M is a pair ( U, ϕ ) such that
U (cid:26) M is open and ϕ : U !
R k , where ϕ is a homeomorphism; An atlas for M is a collection f
( U α , ϕ α )
g
α ∈ A of pairwise C n compatible charts such that S α ∈
A U α = M ;
And we call M a smooth manifold if and only if M has a C ∞ atlas.
Deﬁnition 5.2 (Partition of unity) .
A C ∞ partition of unity on a manifold M is a collection of non-negative C ∞ functions ρ α :
M ! R
+ for α 2 A that satisfy (1) the collection of supports, f supp ( ρ α ) g α ∈ A , is locally ﬁnite; and (2)
[Equation]
Deﬁnition 5.3 (Poly-Partitionable) .
We call that M is poly-partitionable if and only if, for a tangentspace-induced atlas f ( U α , T α ) g α ∈ A of M , there exists a particular partition of unity f
ρ α g α ∈
A that satisﬁes ρ α (cid:14) T − 1 α is a simple piecewise polynomial in R k , where simple piecewise polynomial is deﬁned as the composite mapping between a polynomial and a size-bounded ReLU network.
The concept, poly-partitionable, deﬁnes a class of manifolds that have simple partition of unity, which is a generalization of some structures in the standard Euclidean space R d .
For example, an explicit construction for low-dimensional manifold [0 , 1] k is f ϕ m ( x ) g in Yarotsky (2017), where the coordinate system is identity mapping.
5.2 Main results under the low-dimensional manifold assumption
Before giving our main results, we ﬁrst extend robust classiﬁcation to the version of manifold.
Deﬁnition 5.4 (Robust classiﬁcation on a manifold) .
Given a probability measure P on M (cid:2) f(cid:0) 1 , +1 g and a robust radius δ , the robust test error of a classiﬁer f : M !
R w.r.t P and δ under ℓ p norm is deﬁned as L p,δ M ,P
[Equation]
Now, we present our main result in this section, which establishes an improved upper bound for size that is mainly exponential in the intrinsic dimension k instead of the ambient data dimension d .
Theorem 5.5.
Let M (cid:26) [0 , 1] d be a k (cid:0) dimensional compact poly-partitionable Riemannian manifold with the condition number τ > 0 .
For any two 2 ϵ (cid:0) separated A, B (cid:26) M under ℓ ∞ norm, distribution P on the supporting set S =
A [ B and robust radius c 2 (0 , 1) , there exists a ReLU network f with at most ˜
[Equation]
parameters, such that L ∞ ,cϵ M ,P ( f ) = 0 , where ˜ k
= O
( k log d ) is almost linear with respect to the intrinsic dimension k , only up to a logarithmic factor.
Proof sketch.
The proof idea of Theorem 5.5 has two steps.
First, we construct a Lipschitz robust classiﬁer f ∗ in Proposition 3.2.
Then, we regard f ∗ as the target function and use a ReLU network f to approximate it on the manifold M .
The following lemma is the key technique that shows we can approximate Lipschitz functions on a manifold by using ReLU networks efﬁciently.
Lemma 5.6.
Let M (cid:26) [0 , 1] d be a k (cid:0) dimensional compact poly-partitionable Riemannian manifold with the condition number τ > 0 .
For any small δ > 0 and a L (cid:0) lipschitz function g : M !
R , there exists a function ˜
g implemented by ReLU network with at most ˜ O (cid:16) ( p dL [Equation] k (cid:17) parameters, such that j g (cid:0) ˜
g j < δ for any x 2 M , where ˜ k is the same as Theorem 5.5.
By applying the conclusion of Lemma 5.6, we can approximate the 1 /ϵ (cid:0) Lipschitz function f ∗ in
Proposition 3.2 via a ReLU network f with at most ˜ O (cid:16) exp(˜ k ) (cid:17) parameters, such that the uniform approximation error k f (cid:0) f ∗ k ℓ ∞ ( M ) at most 1 (cid:0) c .
Next, we prove the theorem by contradiction.
Assume that there exists some perturbed input x ′ that is mis-classiﬁed and the original input x is in A .
So we know f ( x ′ ) < 0 and f ∗ ( x ) <
ϵ ′ .
This impiles d ∞ ( x ′ , A ) <
d ∞ ( x ′ , B ) < 1+ ϵ ′ 1 − ϵ ′ d ∞ ( x ′ , A ) .
Combined with d ∞ ( x ′ , A ) + d ∞ ( x ′ , B ) (cid:21) d ∞ ( A, B ) (cid:21) 2 ϵ , we have d ∞ ( x ′ , A ) >
(1 (cid:0) ϵ ′ )
ϵ = cϵ , which is a contradiction.
Remark 5.7.
Chen et al.
(2019) studies network-based approximation on smooth manifolds, and also establishes an O ( δ − k ) bound for the network’s size.
However, different from their setting where the approximation error δ goes to zero, it is reasonable that the separated distance ϵ and robust radius c are constants in our setting.
If we simply follow their proofs, we can only obtain the bound O (( δ/ C M ) − k ) where C M also grows exponentially with respect to k , which further implies that the ﬁnal result will be roughly exp( O ( k 2 )) .
This bound is too loose, especially when k (cid:25) p d .
To this end, we propose a novel approximation framework so as to improve the bound to exp( O ( k )) , which is presented as Lemma 5.6.
And the detailed proof of Lemma 5.6 is deferred to Appendix E.1.
Although we have shown that robust classiﬁcation will be more efﬁcient when data lies on a lowdimensional manifold, there is also a curse of dimensionality, i.e., the upper bound for the network’s size is almost exponential in the intrinsic dimension k .
The following result shows that the curse of dimension is also inevitable under the low-dimensional manifold assumption.
Theorem 5.8.
Let ϵ 2 (0 , 1) be a small constant.
There exists a sequence f N k g k ≥ 1 that satisﬁes
N k = Ω (cid:16) (2 ϵ p d/k ) − k(cid:17) .
and a universal constant C 1 > 0 such that the following holds: let
M (cid:26) [0 , 1] d be a complete and compact k (cid:0) dimensional Riemannian manifold with non-negative
Ricci curvature , then there exists two 2 ϵ -separated sets A, B (cid:26) M under ℓ ∞ norm, such that for any µ 0 (cid:0) balanced distribution P on the supporting set S =
A [ B and robust radius c 2 (0 , 1) , we have inf fL ∞ ,cϵ P
[Equation]
In other words, the robust test error is lower-bounded by a positive constant α = C 1 µ 0 unless the neural network has size larger than exp(Ω( k )) .
The detailed proof of Theorem 5.8 is presented in
Appendix E.4.
Conclusion
This paper provides a new theoretical understanding of the gap between the robust training and generalization error.
We show that the ReLU networks with reasonable size can robustly classify the ﬁnite training data.
On the contrary, even with the linear separable and well-separated assumptions,
ReLU networks must be exponentially large to achieve low robust generalization error.
Finally, we consider the scenario where the data lies on the low dimensional manifold and prove that
the
ReLU network, with a size exponentially in the intrinsic dimension instead of the inputs’ dimension, is sufﬁcient for obtaining low robust generalization error.
We believe our work opens up many interesting directions for future work, such as the tight bounds for the robust classiﬁcation problem, or the reasonable assumptions that permit the polynomial-size ReLU networks to achieve low robust generalization error.
Acknowledgement
We thank all the anonymous reviewers for their constructive comments.
This work is supported by National Science Foundation of China (NSFC62276005), The Major Key Project of PCL (PCL2021A12), Exploratory Research Project of Zhejiang Lab (No. 2022RC0AN02), and Project 2020BD006 supported by PKUBaidu Fund.
Binghui Li is partially supported by National Innovation Training Program of China.
Jikai Jin is partially supported by the elite undergraduate training program of School of Mathematical Sciences in Peking University.