Overparameterization from Computational Constraints
Abstract
Overparameterized models with millions of parameters have been hugely successful.
In this work, we ask: can the need for large models be, at least in part, due to the computational limitations of the learner?
Additionally, we ask, is this situation exacerbated for robust learning?
We show that this indeed could be the case.
We show learning tasks for which computationally bounded learners need significantly more model parameters than what information-theoretic learners need.
Furthermore, we show that even more model parameters could be necessary for robust learning.
In particular, for computationally bounded learners, we extend the recent result of Bubeck and Sellke [NeurIPS’2021] which shows that robust models might need more parameters, to the computational regime and show that bounded learners could provably need an even larger number of parameters.
Then, we address the following related question: can we hope to remedy the situation for robust computationally bounded learning by restricting adversaries to also be computationally bounded for sake of obtaining models with fewer parameters?
Here again, we show that this could be possible.
Specifically, building on the work of Garg, Jha, Mahloujifar, and Mahmoody [ALT’2020], we demonstrate a learning task that can be learned efficiently and robustly against a computationally bounded attacker, while to be robust against an information-theoretic attacker requires the learner to utilize significantly more parameters.
Introduction
In recent years, deep neural nets with millions or even billions of parameters
[Wortsman et al., 2022, Dai et al., 2021, Yu et al., 2022] have emerged as one of the most powerful models for very basic tasks such as image classification.
A magic of DNNs is that they generalize without falling into the classical theories mentioned above, and hence they are the subject of an active line of work aiming to understand how DNNs generalize Arora et al.
[2019], Allen-Zhu et al.
[2019b,a], Novak et al.
[2018], Neyshabur et al.
[2014], Kawaguchi et al.
[2017], Zhang et al.
[2021], Arora et al.
[2018b] and the various benefits of overparametrized regimes
Xu et al.
[2018], Chang et al.
[2020], Arora et al.
[2018a], Du et al.
[2018], Lee et al.
In fact, the number of parameters in such models is so large that it is enough to memorize (and fit) to a large number of random labels [Zhang et al.,
This leads us to our first main question, in which we investigate the potential cause for having large models:
Are there any learning tasks for which computationally bounded learners need to utilize significantly more model parameters than needed information-theoretically?
One should be cautious in how to formulate the question above.
That is because, many simple (information-theoretically learnable) tasks are believed to be computationally hard to learn (e.g., learning parity with noise [Pietrzak, 2012]).
In that case, one can interpret this as saying that the efficient learner requires infinite number of parameters, as a way of saying that the learning is not possible at all!
However, as explained above, we are interested in understanding the reason behind needing a large number of model parameters when learning is possible .
Could robustness also be a cause for overparameterization?
A highly sought-after property of machine learning models is their robustness to the so-called adversarial examples [Goodfellow et al., 2014].
Here we would like to find a model f such that f ( x ) = f
( x ′ ) holds with high probability when x ← D is an honestly sampled instance and x ′
≈
x is a close instance that is perhaps minimally (yet adversarially) perturbed.
7 A recent work of Bubeck and Sellke [2021] showed that having large model parameters could be due to the robustness of the model.
Here, we are asking whether such phenomenon can have a computational variant that perhaps leads to needing even more parameters when the robust learner is running in polynomial time.
Are there any learning tasks for which computationally bounded robust learning comes at the cost of having even more model parameters than non-robust learning?
In fact, it was shown by Bubeck et al.
[2019] and Degwaker et al.
[Degwekar et al., 2019] that computational limitations of the learner could be the reason behind the vulnerability of models to adversarial examples.
In this work, we ask whether the phenomenon investigated by the prior works is also crucial when the model size is considered.
Can computational intractability of adversary help?
We ask whether natural restrictions on the adversary can help reduce model sizes.
In particular, we consider the restriction to the class of polynomially bounded adversaries.
In fact, when it comes to robust learning, the computationally bounded aspect could be imposed both on the learner as well as the adversary .
Are there any learning tasks for which robust learning can be done with fewer model parameters when dealing with polynomial-time adversaries?
Previously, it was shown that indeed working with computationally bounded adversaries can help achieving robust models [Mahloujifar and Mahmoody, 2019, Bubeck et al., 2019, Garg et al., 2020b].
Hence, we are asking a similar question in the context of model parameters.
1.1 Our results
In summary, we prove that under the computational assumption that one-way functions exist, the answer to all three of our main questions above is positive.
Indeed, we show that the computational efficiency of the learner could be the cause of having overparametarized models.
Furthermore, the computational efficiency of the adversary could reduce the size of the models.
In particular, we prove the following theorem, in which a learning task is parameterized by λ , a hypothesis class H and a class of input distributions D (see Section 2.1 for formal definitions).
Theorem 1 (Main results, informal) .
If one-way functions exist, then for arbitrary polynomials n < α < β
(e.g., n = λ 0 .
1 , α = λ 5 , β = λ 10 ) over λ the following hold.
• Part 1: There is a learning task parameterized by λ and a robustness bound (to limit how much an adversary can perturb the inputs) such that: – The instance size is Θ( n ) .
(That is, the length of the input is Θ( n ) .)
– There is a robust learner that uses Θ( λ ) parameters.
– Any polynomial-time learner needs Θ( α ) parameters to learn the task.
– Any polynomial-time learner needs Θ( β ) parameters to robustly learn the task.
• Part 2: There is a learning task parameterized by λ and a robustness bound (to limit how much an adversary can perturb the inputs) such that: – The instance size is Θ( n ) .
– When the adversary that generates the adversarial examples runs in polynomial time, there is an (efficient) learner that outputs a model with O (1) parameters that robustly predicts the output labels with small error.
– Against information-theoretic (computationally unbounded) adversaries, no learner can produce a model with < Θ( α ) parameters that later robustly make the predictions.
In Sections 3 and 4, we explain the high-level ideas behind the proofs of the two parts of Theoremand formalize these statements.
The full proofs can be found in the supplemental material.
Takeaway.
Here we put our work in perspective.
As discussed above, prior works have considered the effect of computational efficiency (for both the learner and the attacker) on the robustness of the model.
Informally, these works have shown that requiring a learner to be efficient hinders robustness, while requiring an attacker to be efficient helps achieve robustness.
Our work studies the effect of computational efficiency as well but focuses on the number of parameters of the model.
In spirit, we have shown a similar phenomenon.
Namely, requiring a learner to be efficient increases the size of the model, while requiring an attacker to be efficient helps reduce the size of the model.
Our results can be summarized as follows.
In the non-robust case, requiring the learner to be efficient increases the size of the model.
In the robust case: (1) Requiring the learner to be efficient increases the size of the model.
This holds for robustness against both efficient and inefficient attackers.
(2) Restricting to only computational efficient attackers reduces the size of the model.
This holds for both efficient and inefficient learners.
Limitations, Implications, and Open Question.
Our work shows that the phenomenon of having larger models due to computational efficiency could provably happen in certain scenarios.
It does not imply, however, that this holds for all learning problems.
It is a fascinating open question whether similar phenomena also happen to real-world problems.
We note that this is not particular to our work, but common to most prior works in the theory of learning showing “separation” results [Bubeck et al., 2019, Degwekar et al., 2019, Mahloujifar and Mahmoody, 2019, Garg et al., 2020b].
Our results provides an explanation on why small but representative classes (e.g. 2 layers neural networks) of functions do not obtain same (robust) accuracy as larger models.
This phenomenon that cannot be solely explained based on representation power of the function class might be due to computational limitations of the learning algorithm.
Finally, we note that our theorem demonstrates the separation by using the simplest setting of binary output.
One can extend it to more sophisticated settings of any finite output, particularly real numbers with bounded precision.
However, our work does not consider real numbers with infinite precision.
In such cases, one needs to revisit computational efficiency, as the inputs are infinitely long.
Preliminaries
For a distribution D , x ← D denotes that x is sampled according to D .
We use U S for the uniform distribution over S , and we use U n to represent U { 0 , 1 } n .
For a set S , s ← S means s ← U S .
For any two distributions X and Y over a finite universe Ω , their statistical distance is defined as
[Equation]
We sometimes write X ≈ ε Y to denote SD ( X, Y )
⩽ ε .
For a vector x = ( x 1 , . . .
, x n ) ∈ F n and a subset S
= { i 1 , . .
, i ℓ } ⊆ { 1 , 2 , . . . , n } , x S denotes ( x i 1 , . . .
, x i ℓ ) .
For any two vectors x = ( x 1 , . . .
, x n ) and y =
( y 1 , . . .
, y n ) , their Hamming distance is defined as HD ( x, y ) =
|{ i ∈ { 1 , 2 , . . . , n } : x i ̸ = y i }| .
For a set S and an integer[-
-]
2.1 Definitions related to learning and attacks
In this subsection, we present notions and definitions that are related to learning.
We use X to denote the inputs and Y = { 0 , 1 } to denote the outputs or the labels.
By H we denote a hypothesis class of functions from X to Y .
We use D to denote a distribution over X × Y .
A learning algorithm, takes a set S ∈
( X × Y ) ∗ and a parameter λ and outputs a function f that is supposed to predict a fresh sample from the same distribution that has generated the set S .
The parameter λ is supposed to capture the complexity of the problem, e.g., by allowing the inputs (and the running times) to grow.
(E.g., this could be the VC dimension, but not necessarily so.)
In particular, we assume that the members of the sets X λ , Y λ can be represented with poly ( λ ) bits.
A proper learning for a hypothesis class H outputs f ∈ H , while an improper learner is allowed to output arbitrary functions.
By default, we work with improper learning as we do not particularly focus on whether the learning algorithms output a function from the hypothesis class.
For a set S ⊆ X and a function h : X → Y , by S h we denote the labeled set { ( x, h ( x )) | x ∈ S} .
For a distribution D and an oracle-aided algorithm A , by A D we denote giving A access to a D sampler .
Definition 2 (Learning problems and learners) .
We use F = { F λ =
( X λ , Y λ , D λ , H λ ) } λ ∈ N to denote a family of learning problems where each H λ is a set of hypothesis functions mapping X λ to
Y λ and D λ is a set of distributions supported on X λ .
• For function ε ( · , · ) , we say L ε -learns F if
[Equation]
where Risk( h, D λ , f ) =
Pr x ← D λ [ h ( x ) ̸ = f ( x )] .
• L outputs models with at most p ( · ) bits of parameters if ∀ λ ∈ N ; (cid:12)(cid:12) Supp [Equation] L
[Equation] [Equation] λ ) [Equation]
⩽ 2 p ( λ )
[Equation]
• L is an ε -robust learner against (all) adversaries of budget r w.r.t.
distance metric d if
[Equation]
N [Equation] where Risk
[Equation]
• L runs in polynomial time if there is a polynomial poly ( · ) such that for all λ ∈ N and S ∈
( X λ , Y λ ) ∗ , the running time of L ( S , λ ) is bounded by poly ( |S| · λ ) .
• L is an ε -robust learner against polynomial-time adversaries of budget r w.r.t distance d , if for any family of poly ( λ ) -time (oracle aided) adversaries A
=
{ A ( · ) λ } λ ∈ N we have ∀ λ ∈ N , D λ
[Equation]
D λ , h ∈ H λ , n ∈ N ;
[Equation]
where Risk ( d,r,A λ ) ( h, D λ ,
[Equation]
=
[Equation]
2.2 Cryptographic primitives
Definition 3 (Negligible function) .
A function ε ( λ ) is said to be negligible, denoted by ε ( λ )
=
negl ( λ ) , if for all polynomial poly ( λ ) , it holds that ε ( λ ) < 1 / poly ( λ ) for all sufficiently large λ .
Definition 4 (Pseudorandom generator) .
Suppose α > λ .
A function f : { 0 , 1 } λ → { 0 , 1 } α is called a pseudorandom generator (PRG) if for all polynomial-time probabilistic algorithm A , it holds that
[Equation]
The ratio α/λ is refer to as the (multiplicative) stretch of the PRG.
Lemma 5 (Håstad et al.
Pseudorandom generators (with arbitrary polynomial stretch) can be constructed from any one-way functions.
Definition 6 (Signature scheme) .
A signature scheme consists of three algorithms ( Gen , Sign , Verify ) .
• ( vk , sk ) ← Gen (1 λ ) : on input the security parameter 1 λ , the randomized algorithm
Gen outputs a (public) verification key vk and a (private) signing key sk .
• σ =
Sign ( sk , m ) : on input the signing key sk and a message m , Sign outputs a signature σ .
• b = Verify ( vk , m, σ ) : on input a verification key vk , a message m , and a signature σ , Verify outputs a bit b , and b = 1 indicates tha the signature is accepted.
We require a (secure) signature scheme to satisfy the following properties.
• Correctness.
For every message m , it holds that
[Equation]
[Equation]
• Weak unforgeability.
For any PPT algorithm A and message m , it holds that
[Equation]
[Equation] [Equation] [Equation] [Equation]
[Equation]
Lemma 7 (Naor and Yung [1989], Rompel [1990]) .
Signature schemes can be constructed from any one-way function.
2.3 Coding theory
Let F be a finite field.
A linear code H with block length n and dimension k is a k -dimensional subspace of the vector space F n .
The generator matrix G ∈ F k × n maps every message m ∈ F k to its encoding, namely, the message m is encoded as m · G .
The distance d of the code H is the minimum distance between any two codewords.
That is, d = min ( x,y ∈H ) ∧ ( x ̸ = y ) HD ( x, y ) .
The rate of the codeword is defined as R = k/n .
Reed-Solomon code.
In this work, we will mainly use the Reed-Solomon (RS) code.
For the RS code, every message m = ( m 1 , . . .
, m k ) ∈ F k is parsed as a degree k − 1 polynomial
[Equation]
and the encoding is simply ( f (1) , f (2) , . . .
, f ( n )) ∈ F n .
Definition 8 (List-decodable code) .
A code H ⊆
F n is said to be ( p, L ) -list-decodable if for any string x ∈ F n , there are ⩽ L messages whose encoding c satisfies HD ( x, c ) ⩽ p · n .
We say the code
H is efficiently ( p, L ) -list-decodable if there is an efficient algorithm that finds all such messages.
Lemma 9 (Guruswami and Sudan [1998]) .
For any constant R > 0 , the Reed-Solomon code with constant rate R and block length n is efficiently (1 − √ R, poly ( n )) -list-decodable.
2.4 Randomness extraction
Definition 10 (Min-entropy) .
The min-entropy of a distribution X over a finite set Ω is defined as
[Equation]
We need the following result about the sampler.
A sampler (for a target set of coordinates { 1 , 2 , . . . , n } ) is a deterministic mapping that only takes randomness as input and outputs a subset of { 1 , 2 , . . . , n } .
Below, we let (cid:0) [ n ] t (cid:1) =
{S | |S| = t, S ⊆
[ n ] } .
Lemma 11 (Lemma 6.2 and Lemma 8.4 of Vadhan [2004]) .
For any 0 < κ 1 , κ 2 < 1 and any n, t ∈ N , there exists a deterministic sampler samp : { 0 , 1 } r → (cid:0) [ n ] t (cid:1) such that the following hold.
• If X is a random variable over { 0 , 1 } n with min-entropy ⩾ µ · n , there exists a random variable Y over { 0 , 1 } t with min-entropy ⩾ ( µ − κ 1 ) · t and
[Equation]
where the two U r in the first joint distribution refer to the same sample.
Furthermore, r = Θ ( n κ 2 ) .
This lemma by Vadhan Vadhan [2004] states that one could use a small amount of randomness to sub-sample from a distribution X with the guarantee that X samp ( U r ) is close to another distribution
Y that preserves the same min-entropy rate as X .
Due to space limitations, additional preliminaries are included in Appendix A.
Efficient learner vs. information-theoretic learner
In this section, we explain the ideas behind the proof of Part 1 of Theorem 1.
We start with the high-level ideas behind our construction.
Then, we formally state the construction of the learning task and its properties through formal statements.
The proofs can be found in the supplemental material.
Consider the problem of learning an inner product function IP P defined as IP P ( x ) = ⟨ x, P ⟩ , where the inner product is done in GF 2 .
Our first observation is that to learn IP P with a small error where
P is uniformly random, the number of model parameters that the learner employs must be as (almost) as large as the length of P .
Intuitively, one can argue it as follows.
Let Z denote the parameters in the model that the learner outputs.
Suppose that Z is shorter than P .
Then P must still be unpredictable given Z .
By a standard result in randomness extraction, one can argue that, for a uniform x , ( Z, x, ⟨ x, P ⟩ )
≈
( Z, x, U { 0 , 1 } ) .
That is, the label ⟨ x, P ⟩ looks information-theoretically uniform to the classifier who holds Z and x .
Therefore, the classifier can only output the correct label with (the trivial) probability ≈ 1 / 2 .
The conclusion is that learning all linear functions need a learner that outputs as many parameters as the function’s description is.
However, even an efficient learner can perform the learning just as well as an information-theoretic one (e.g., using the Gaussian elimination).
We now show how to modify this task to make a big difference between an efficient learner and an information-theoretic learner in terms of the number of parameters that they output.
Computational perspective.
Now, suppose P is computationally pseudorandom [Goldwasser and
Micali, 1984] rather than being truly random.
That is, let f : { 0 , 1 } λ → { 0 , 1 } α be a pseudorandom generator (Definition 4) and P is distributed as f ( U λ ) , where U λ denotes the uniform distribution over λ bits.
Since (1) an efficient learner cannot distinguish P ← f ( U λ ) from P ← U α and (2) any learner who tries to learn IP P with P ← U α needs α parameters, it can be proved that an efficient learner who tries to learn IP P with P ← f ( U λ ) must also uses α parameters.
However, an information-theoretic learner needs only λ parameters to learn IP P with P ← f ( U λ ) as it can essentially find the seed s such that P = f
( s ) and output the seed s .
To conclude, to learn IP P for
P ← f ( U λ ) , an information-theoretic learner only needs a few (i.e., λ ) parameters and an efficient learner needs a lot of (i.e., α ) parameters.
We emphasize that for a pseudorandom generator, its output length α could have an arbitrarily large polynomial dependence on its input length λ .
For instance, it could be α = λ 10 .
Robustness.
We now explain the ideas behind Theorem 15 in which we study the role of model robustness in the size of the model parameters.
We now suppose the instance is sampled according to the distribution D Q (parametrized by a string Q ), which is defined by
[Equation]
Here, U is the uniform distribution (over the right number of bits), Enc ( U ) is an error-correcting encoding of U , and the addition is coordinate-wise field addition.
Moreover, the label for this instance is the inner product ⟨ U, P ⟩ for some vector P .
Observe that if the learner learns Q , it can always find the correct label on a perturbed instance due to the error-correcting property of Enc ( U ) .
We argue that, in order to robustly learn this task for a uniformly random Q , the number of parameters in the model that the learner outputs must be almost as large as the length of Q .
Intuitively, the argument is as follows.
Let Z be the model that the learner outputs.
Since | Z | < | Q | , then Q must still be unpredictable given Z .
In this case, we prove that
[Equation]
Here, ρ stands for the noise that the adversary adds to the instance.
In words, the classifier holding the parameter Z cannot distinguish the perturbed instance Enc ( U ) + Q + ρ from a uniformly random string U ′ .
Since U is information-theoretically hidden to the classifier, it can only output the correct label ⟨ U, P ⟩ with probability ≈ 1 / 2 .
Next, to explore the difference between an efficient and information-theoretic learner, we consider the case where Q is pseudorandomly distributed, i.e. Q ← f
( U λ ) for some f : { 0 , 1 } λ → { 0 , 1 } β .
Again, since (1) an efficient learner cannot distinguish Q ← U β from Q ← f ( U λ ) and (2) any learner needs at least ≈ | Q | parameters to learn the task with Q ← U β , an efficient learner must also need at least ≈ | Q | parameters to learn the task.
On the other hand, an information-theoretic learner could again find the seed s such that Q = f ( s ) and output the seed s as the parameter.
To conclude, an information-theoretic learner requires few (i.e., λ ) parameters to robustly learn the task and an efficient learner needs a lot of (i.e., ≈ β ) parameters to robustly learn the task.
(Again, β could have an arbitrary polynomial dependence on λ .)
Making instances small.
The learning tasks we considered above suffer from one drawback: the size of the instance is very large, or at least is related to the number of parameters of the model.
Here we ask: is it possible to have a small instance size while an efficient learner still needs to output a very large model?
We answer this question positively.
In particular, for any n = poly ( λ ) (e.g., n = λ 0 . 1 ), we construct a learning problem where the instance size is Θ( n ) and the efficient learner still needs α (resp.
β ) parameters to learn (resp.
robustly learn) the task.
Our construction uses the “sampler” by Vadhan [2003].
Informally, a sampler samp (see Lemma 11) needs a small seed u and outputs a subset of { 1 , 2 , . .
. , α } of size n .
The sampler comes with the guarantee that if P is a source with high entropy, P | samp ( u ) also has high enough entropy.
To illustrate the usage of the sampler, consider learning this new inner product function IP P defined as
[Equation]
For uniformly random P , one can similarly argue that a model must output at least | P | parameters to learn the task.
Let Z denote the parameters in the model that the learner outputs.
If | Z | < | P | , then P contains high entropy conditioned on Z .
By the property of the sampler, it must hold that
P | samp ( u ) contains high entropy conditioned on Z and u .
Consequently,
[Equation]
Namely, the classifier who sees the parameter Z and the instance ( u, x ) can only predict the label with probability ≈ 1 / 2 .
Observe that the size of the instance ( u, x ) is roughly n , 12 while P could have length α ≫ n .
The use of the sampler in the robust learning case is similar to the non-robust case above.
We refer the readers to Appendix B for more details.
To sum up, we present our formal construction and formal theorem statement below.
The formal proof is presented in the supplemental material.
Construction 12 (Parameter-heavy models under efficient learning) .
Given the parameter n < λ < α < β , we construct the following learning problem.
13 We rely on the following building blocks
• Let f 1 : { 0 , 1 } λ → { 0 , 1 } α and f 2 : { 0 , 1 } λ → { 0 , 1 } β be PRGs (Definition 4).
Let Enc be a RS encoding with dimension k , block length n , and rate R = k/n .
The rate is chosen to be any constant < 1 / 3 and k is defined by R and n .
This RS code is over the field F 2 ℓ for some ℓ = Θ(log λ ) .
• Let samp 1 : { 0 , 1 } r 1 → (cid:0) { 1 ,...,α } k · ℓ (cid:1) and samp 2 : { 0 , 1 } r 2 → (cid:0) { 1 ,...,β } n · ℓ (cid:1) be samplers.
We obtain these samplers by invoking Lemma 11 with sufficiently small κ 1 and κ 2 (e.g., κ 1 = Θ(1 / log λ ) and sufficiently small constant κ 2 ).
Note that κ 1 , κ 2 define r 1 , r 2 .
• For any binary string v , we use [ v ] for an arbitrary error-correcting encoding of v (over the field F 2 ℓ ) such that [ v ] can correct > (1 − R ) n/ 2 errors.
This can always be done by encoding v using RS code with a suitable (depending on the dimension of v ) rate.
Looking forward, we shall consider an adversary that may perturb ⩽ (1 − R ) n/ 2 symbols.
Therefore, when a string v is encoded as [ v ] and the adversary perturb it to be (cid:102) [ v ] , it will always be error-corrected and decoded back to v .
We now construct the following learning task F λ =
( X λ , Y λ , D λ , H λ ) .
• X λ implicitly defined by the distribution D λ , and Y λ
= { 0 , 1 } .
• D λ consists of distributions D s for s ∈ { 0 , 1 } λ , where D s is D s = (cid:16) [ u 1 ] , [ u 2 ] , m, Enc ( m ) + (cid:16) f 2 ( s ) (cid:12)(cid:12) samp 2 ( u 2 ) (cid:17) (cid:17) , where – u 1 and u 2 are sampled uniformly at random from { 0 , 1 } r 1 and { 0 , 1 } r 2 , respectively.
– m is sampled uniformly at random from F k 2 ℓ .
– f 2 ( s ) (cid:12)(cid:12) samp 2 ( u 2 ) is interpet as a vector over F 2 ℓ and Enc ( m ) + (cid:16) f 2 ( s ) (cid:12)(cid:12) samp 2 ( u 2 ) (cid:17) is coordinate-wise addition over F 2 ℓ .
• H λ consists of all functions h s : X → Y = { 0 , 1 } for all s ∈ { 0 , 1 } λ , where h s is: h s ( x ) =
(cid:68) m, (cid:16) f 1 ( s ) (cid:12)(cid:12) samp 1 ( u 1 ) (cid:17)(cid:69) , and the inner product is over F 2 and m is interpreted as a string ∈ F k · ℓ 2 in the natural way.
• Adversary.
The entire input x =
([ u 1 ] , [ u 2 ] , m, Enc ( m ) + (cid:0) f 2 ( s ) | samp 1 ( u 1 ) (cid:1) ) is interpreted as a vector over F 2 ℓ and we consider an adversary that may perturb ⩽ (1 − R )
n/ 2 symbols.
That is, the adversary has a budget of (1 − R ) n/ 2 in Hamming distance over F 2 ℓ .
Theorem 13.
An information-theoretic learner can (robustly) ε -learn the task of Constructionwith parameter size 2 λ and sample complexity Θ( λε ) .
Moreover, an efficient learner can (robustly) ε -learn this task with parameter size α + β and sample complexity Θ( α + βε ) .
Theorem 14.
Any efficient learner that outputs models with ⩽ α/ 2 parameters cannot ε -learn F λ of
Construction 12 for ε < 1 / 3 .
Theorem 15.
There exists some constant c such that the following holds.
In the presence of an adversary that may perturb (1 − R ) n/ 2 symbols, any efficient learner for the task of Constructionthat outputs a model with c · β/ log λ parameters cannot ε -robustly learn F λ for ε < 1 / 3 .
Efficient adversary vs. information-theoretic adversary
In this section, we explain the ideas behind the proof of Part 2 of Theorem 1.
At the end of this section, we formally state the construction of the learning task and state its properties through formal statements.
The proofs can be found in the supplemental material.
We now explore whether the computational efficiency of the adversary could affect the number of parameters required to robustly learn a task.
Garg et al.
[2020a] considered the difference between an efficient adversary and an information-theoretic one in terms of their running time.
We first recall their construction.
The learning instance is sampled from the distribution [ vk ] , b, Sign ( sk , b ) .
Here, ( vk , sk ) is the verification key and signing key pair of a signature scheme (Definition 6);[ vk ] is an error-correcting encoding of vk , which ensures that [ vk ] can always be recovered after perturbation by the adversary; b is a uniform random bit and the label of the instance is simply b .
The signature scheme ensures that an efficient adversary cannot forge a valid signature and, hence, any efficient adversary that perturbs ( b, Sign ( sk , b )) will result in an invalid message/signature pair.
A classifier could then detect such perturbations and output a special symbol ⊥ indicating that perturbation is detected.
On the other hand, an information-theoretic adversary could launch a successful attack by forging a valid signature (1 − b, Sign ( sk , 1 − b )) .Therefore
, it could perturb the instance to be [ vk ] , 1 − b, Sign ( sk , 1 − b ) and, hence, flipping the label of the output.
First idea.
We want to construct a learning problem such that the learner needs few parameters against an efficient adversary, but a lot of parameters against an information-theoretic adversary.
Our first idea is to add another way of recovering b in the above learning problem.
Consider the learning problem where the instance is sampled from distribution D P defined as
[Equation]
Here, u is uniformly distributed.
Observe that if the learner learns P , it can recover b correctly from [ b + ⟨ u, P ⟩ ] and [ u ] by error-correction decoding.
However, if the number of parameters in the model that the learner outputs have < | P | parameters, then [ b + ⟨ u, P ⟩ ] and [ u ] information-theoretically hides b .
Again, this is because P is unpredictable given the parameter Z 15 and, hence,
[Equation]
Consequently, an information-theoretic adversary could again launch the attack that replace b, Sign ( sk , b ) with 1 − b, Sign ( sk , 1 − b ) and successfully flipping the label.
Therefore, a learner must employ | P | parameters to robustly learn the task against information-theoretic adversaries.
Second idea.
In the above learning task, a learner employing | P | parameters can always recover the correct label against information-theoretic adversaries.
However, against efficient adversaries, a learner with fewer parameters may not always recover the correct label but will sometimes output the special symbol ⊥ indicating that tampering is detected.
Could we twist it to ensure that a learner with fewer parameters can also always recover the correct label against efficient adversaries?
We positively answer this question by using list-decodable code (Definition 8).
Intuitively, list-decoding ensures that given an erroneous codeword, the decoding algorithm will find the list of all messages whose encoding is close enough to the erroneous codeword.
Our new learning task has instances drawing from the distribution D P defined as
[Equation]
Here, LEnc is the encoding algorithm of the list-decoding code.
The main idea is that: after the perturbation on LEnc ( b, Sign ( sk , b )) , the original message/signature pair b, Sign ( sk , b ) will always appear in the list output by the decoding algorithm.
Now, against an efficient adversary, b, Sign ( sk , b ) must be the only valid message/signature pair in the list.
Otherwise, this adversary breaks the unforgeability of the signature scheme.
Against an information-theoretic adversary, however, it could introduce (1 − b, Sign ( sk , 1 − b )) into the list recovered by the list-decoding algorithm.
Consequently, the learner cannot tell if the correct label is b or 1 − b .
Consequently, for this learning task, against information-theoretic adversaries, one still needs | P | parameters.
Against efficient adversaries, one needs only a few parameters and can always recover the correct label.
Making instances small.
Again, we have this unsatisfying feature that the instance has the same size as the model.
We resolve this issue using the sampler in a similar way.
We refer the readers to
Appendix C for more details.
We present our formal construction and theorem statement below.
The formal proof is in Appendix C.
Construction 16 (Learning task for bounded/unbounded attackers) .
Given the parameters n < λ < α , we construct the following learning problem.
We use the following tools.
• ( Gen , Sign , Verify ) be a signature scheme (see Definition 6).
• Let LEnc be a RS encoding with dimension k , block length n , and rate R = k/n .
We pick the rate R to be any constant < 1 / 4 and k is defined by R and n .
This RS code is over the field F 2 ℓ for some ℓ = Θ(log λ ) .
• Let samp : { 0 , 1 } r → (cid:0) { 1 ,...,α } n (cid:1) be samplers.
(We obtain these samplers by invoking Lemma 11 with sufficiently small κ 1 and κ 2 .
For instance, setting κ 1 = Θ(1 / log λ ) and κto be any small constant suffices.)
• For any binary string v , we use [ v ] for an arbitrary error-correcting encoding of v (over the field F 2 ℓ ) such that [ v ] can correct > (1 − √ R ) n errors.
This can always be done by encoding v using RS code with a suitable (depending on the dimension of v ) rate.
Looking forward, we shall consider an adversary that may perturb ⩽ (1 −√ R ) n symbols.
Therefore, when a string v is encoded as [ v ] and the adversary perturbs it to be (cid:102) [ v ] , it will always be error-corrected and decoded back to v .
We now construct the following learning task F λ =
( X λ , Y λ , D λ , H λ ) .
• X λ is { 0 , 1 } N for some N that is implicitly defined by D λ , and Y λ is { 0 , 1 } .
• The distribution D λ consists of all distribution D s for s ∈ { [Equation] 1 } α
[Equation]
, such that – u are sampled uniformly from { 0 , 1 } r and v is sampled uniformly from { 0 , 1 } n .
– ( vk , sk ) ← Gen (1 λ ) are sampled from the signature scheme.
– b is sampled uniformly at random from { 0 , 1 } .
( b, Sign ( sk , b )) is intepreted as a vector in F k 2 ℓ in the natural way.
• h λ consists of one single function h .
On input x =
([ u ] , [ v ] , LEnc ( b, Sign ( sk , b ) , [ b + ⟨ v, s | samp ( u ) ⟩ ]) , h ( x ) simply decodes LEnc ( b, Sign ( sk , b )) and output b .
• Adversary.
The entire input ([ u ] , [ v ] , [ vk ] , LEnc ( b, Sign ( sk , b )) , [ b + ⟨ v, s | samp ( u ) ⟩ ]) is interpreted as a vector over F 2 ℓ and we consider an adversary that may perturb ⩽ (1 − √ R ) n symbols.
That is, the adversary has a budget of (1 − √ R ) n for Hamming distance over F 2 ℓ .
Theorem 17.
For the learning task of Construction 16, there is an efficient learner (with 0 sample complexity) that outputs a model with no parameter and negl ( λ ) -robustly learns F λ against computationally-bounded adversaries of budget (1 − √ R ) n .
Theorem 18.
For computationally unbounded adversaries, any information-theoretic learner with α/ 2 parameters cannot ε -robustly learn F λ for ε < 1 / 3 for the learning task of Construction 16.
Acknowledgement
Sanjam Garg and Mingyuan Wang are supported by DARPA under Agreement No. HR00112020026,
AFOSR Award FA9550-19-1-0200, NSF CNS Award 1936826, and research grants by the Sloan
Foundation, and Visa Inc.
Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the United States
Government or DARPA.
Mohammad Mahmoody is supported by NSF grants CCF-1910681 and
CNS1936799.
Somesh Jha is partially supported by Air Force Grant FA9550-18-1-0166, the National
Science Foundation (NSF) Grants CCF-FMitF-1836978, IIS-2008559, SaTC-Frontiers-1804648,
CCF-2046710 and CCF-1652140, and ARO grant number W911NF-17-1-0405.
Somesh Jha is also partially supported by the DARPA-GARD problem under agreement number 885000.