Original: O N THE (N ON -)R OBUSTNESS OF T WO -L AYER N EURAL N ETWORKS IN D IFFERENT L EARNING R EGIMES
Revised: O N THE (N ON -)R OBUSTNESS OF T WO -L AYER N EURAL N ETWORKS IN D IFFERENT L EARNING R EGIMES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Neural networks are known to be highly sensitive to adversarial examples.
Revised: Neural networks are known to be highly sensitive to adversarial examples.

Original: These may arise due to different factors, such as random initialization, or spurious correlations in the learning problem.
Revised: These may arise due to different factors, such as random initialization, or spurious correlations in the learning problem.

Original: To better understand these factors, we provide a precise study of the adversarial robustness in different scenarios, from initialization to the end of training in different regimes, as well as intermediate scenarios, where initialization still plays a role due to “lazy” training.
Revised: To better understand these factors, we provide a precise study of the adversarial robustness in different scenarios, from initialization to the end of training in different regimes, as well as intermediate scenarios, where initialization still plays a role due to “lazy” training.

Original: We consider overparameterized networks in high dimensions with quadratic targets and infinite samples.
Revised: We consider overparameterized networks in high dimensions with quadratic targets and infinite samples.

Original: Our analysis allows us to identify new tradeoffs between approximation (as measured via test error) and robustness, whereby robustness can only get worse when test error improves, and vice versa.
Revised: Our analysis allows us to identify new tradeoffs between approximation (as measured via test error) and robustness, whereby robustness can only get worse when test error improves, and vice versa.

Original: We also show how linearized lazy training regimes can worsen robustness, due to improperly scaled random initialization.
Revised: We also show how linearized lazy training regimes can worsen robustness, due to improperly scaled random initialization.

Original: Our theoretical results are illustrated with numerical experiments.
Revised: Our theoretical results are illustrated with numerical experiments.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep neural networks have enjoyed tremendous practical success in many applications involving highdimensional data, such as images.
Revised: Deep neural networks have enjoyed tremendous practical success in many applications involving highdimensional data, such as images.

Original: Yet, such models are highly sensitive to small perturbations known as adversarial examples (Szegedy et al., 2013), which are often imperceptible by humans.
Revised: Yet, such models are highly sensitive to small perturbations known as adversarial examples (Szegedy et al., 2013), which are often imperceptible by humans.

Original: While various strategies such as adversarial training (Madry et al., 2018) can mitigate this vulnerability empirically, the situation remains highly problematic for many safety-critical applications like autonomous vehicles and health, and motivates a better theoretical understanding of what mechanisms may be causing this.
Revised: While various strategies such as adversarial training (Madry et al., 2018) can mitigate this vulnerability empirically, the situation remains highly problematic for many safety-critical applications like autonomous vehicles and health, and motivates a better theoretical understanding of what mechanisms may be causing this.

Original: Various factors are known to contribute to adversarial examples.
Revised: Various factors are known to contribute to adversarial examples.

Original: In linear models, features that are only weakly correlated with the label, possibly in a spurious manner, may improve prediction accuracy but induce large sensitivity to adversarial perturbations (Tsipras et al., 2019; Sanyal et al., 2021).
Revised: In linear models, features that are only weakly correlated with the label, possibly in a spurious manner, may improve prediction accuracy but induce large sensitivity to adversarial perturbations (Tsipras et al., 2019; Sanyal et al., 2021).

Original: On the other hand, common neural networks may exhibit high sensitivity to adversarial perturbations at random initialization (Simon-Gabriel et al., 2019; Daniely & Shacham, 2020; Bubeck et al., 2021).
Revised: On the other hand, common neural networks may exhibit high sensitivity to adversarial perturbations at random initialization (Simon-Gabriel et al., 2019; Daniely & Shacham, 2020; Bubeck et al., 2021).

Original: While such settings already capture interesting phenomena behind adversarial examples, they are restricted to either trained linear models, or nonlinear networks at initialization.
Revised: While such settings already capture interesting phenomena behind adversarial examples, they are restricted to either trained linear models, or nonlinear networks at initialization.

Original: Trained, nonlinear networks may thus involve multiple sources of vulnerability arising from initialization, training algorithms, as well as the data distribution.
Revised: Trained, nonlinear networks may thus involve multiple sources of vulnerability arising from initialization, training algorithms, as well as the data distribution.

Original: Capturing the interaction between these different components is of crucial importance for a more complete understanding of adversarial robustness.
Revised: Capturing the interaction between these different components is of crucial importance for a more complete understanding of adversarial robustness.

Original: In this paper, we study the interplay between these different factors by analyzing approximation (i.e how well the model fits the data) and robustness properties (i.e the sensitivity of the model’s outputs w.r.t perturbations in test data) of two-layer neural networks in different learning regimes.
Revised: In this paper, we study the interplay between these different factors by analyzing approximation (i.e how well the model fits the data) and robustness properties (i.e the sensitivity of the model’s outputs w.r.t perturbations in test data) of two-layer neural networks in different learning regimes.

Original: We consider two-layer finite-width networks in high dimensions with infinite training data, in asymptotic regimes inspired by Ghorbani et al.
Revised: We consider two-layer finite-width networks in high dimensions with infinite training data, in asymptotic regimes inspired by Ghorbani et al.

Original: This allows us to focus on the effects inherent to the data distribution and the inductive bias of architecture (choice of activation function, number of hidden neurons per input dimension, etc.) and training algorithms, while side-stepping issues due to finite samples.
Revised: This allows us to focus on the effects inherent to the data distribution and the inductive bias of architecture (choice of activation function, number of hidden neurons per input dimension, etc.) and training algorithms, while side-stepping issues due to finite samples.

Original: Following Ghorbani et al.
Revised: Following Ghorbani et al.

Original: (2019), we focus on nonlinear regression settings with structured quadratic target functions, and consider commonly studied training regimes for two-layer networks, namely (i) neural networks with quadratic activations trained with stochastic gradient descent on the population risk, which exhibit a form of feature learning by finding the global optimum; (ii) random features (RF, Rahimi & Recht, 2008), (iii) neural tangent (NT, Jacot et al., 2018), as well as (iv) ”lazy” training (Chizat et al., 2019) regimes for basic RF and NT, where we consider a first-order Taylor expansion of the network around initialization, including the initialization term itself (in contrast to the standard RF and NT regimes which ignore the offset due to initialization).
Revised: (2019), we focus on nonlinear regression settings with structured quadratic target functions, and consider commonly studied training regimes for two-layer networks, namely (i) neural networks with quadratic activations trained with stochastic gradient descent on the population risk which finds the global optimum; (ii) random features (RF, Rahimi & Recht, 2008), (iii) neural tangent (NT, Jacot et al., 2018), as well as (iv) ”lazy” training (Chizat et al., 2019) regimes for basic RF and NT, where we consider a first-order Taylor expansion of the network around initialization, including the initialization term itself (in contrast to the standard RF and NT regimes which ignore the offset due to initialization).

Original: Note that, though the theoretical setting is inspired by Ghorbani et al.
Revised: Note that, though the theoretical setting is inspired by Ghorbani et al.

Original: (2019), our work differs from theirs in its focus and scope.
Revised: (2019), our work differs from theirs in its focus and scope.

Original: Indeed, we are concerned with robustness and its interplay with approximation, in different learning regimes, while they are only concerned with approximation.
Revised: Indeed, we are concerned with robustness and its interplay with approximation, in different learning regimes, while they are only concerned with approximation.

Original: We also note that the lazy/linearized regimes we study as part of this work were not considered by Ghorbani et al.
Revised: We also note that the lazy/linearized regimes we study as part of this work were not considered by Ghorbani et al.

Original: (2019), and help us highlight the impact of initialization on robustness.
Revised: (2019), and help us highlight the impact of initialization on robustness.

Revised: Note that unlike the other regimes, SGD exhibits a kind of feature learning, whereby the first layer weights are learning specific directions by approximating the matrix B .

Revised: In particular, this involves non-trivial feature selection via non-linear learning, while the other regimes (RF and

Revised: NT) are linear estimators on top of non-linear but fixed features.

Original: Main contributions.
Revised: Main contributions.

Original: Our work establishes theoretical results which uncover novel tradeoffs between approximation (as measured via test error) and robustness that are inherent to all the regimes considered.
Revised: Our work establishes theoretical results which uncover novel tradeoffs between approximation (as measured via test error) and robustness that are inherent to all the regimes considered.

Original: These tradeoffs appear to be due to misalignment between the target function and the input distribution (weight distribution) for random features (Section 4), or to the inductive bias of fully-trained networks (Section 3 and Appendix D).
Revised: These tradeoffs appear to be due to misalignment between the target function and the input distribution (weight distribution) for random features (Section 4), or to the inductive bias of fully-trained networks (Section 3 and Appendix E).

Original: We also show that improperly scaled random initialization can further degrade robustness in lazy/linearized models (Section 5), since the resulting models might inherit the nonrobustness inherent to random initialization.
Revised: We also show that improperly scaled random initialization can further degrade robustness in lazy/linearized models (Section 5), since the resulting models might inherit the nonrobustness inherent to random initialization.

Original: This raises the question of how small should the initialization be in order to enhance the robustness of the trained model.
Revised: This raises the question of how small should the initialization be in order to enhance the robustness of the trained model.

Original: Our theoretical results are empirically verified with extensive numerical experiments on simulated data.
Revised: Our theoretical results are empirically verified with extensive numerical experiments on simulated data.

Original: The setting of our work is regression in a student-teacher setup where the student is a two-layer feedforward neural network and the teacher is a quadratic form.
Revised: The setting of our work is regression in a student-teacher setup where the student is a two-layer feedforward neural network and the teacher is a quadratic form.

Original: We assume access to infinite training data.
Revised: We assume access to infinite training data.

Original: Thus, the only complexity parameters are the coefficient matrix of the teacher model, the input dimension d and the width of the neural network m , assumed to both ”large” but proportional to one another.
Revised: Thus, the only complexity parameters are the coefficient matrix of the teacher model, the input dimension d and the width of the neural network m , assumed to both ”large” but proportional to one another.

Original: Refer to Section 2 for details.
Revised: Refer to Section 2 for details.

Original: In Section 6, we also show similar but weaker trade-offs for arbitrary student and teacher models.
Revised: In Appendix I, we also show similar but weaker trade-offs for arbitrary student and teacher models.

Original: The infinite-sample setting allows us to focus on the effects inherent to the data distribution and the inductive bias of architecture (choice of activation function) and different learning regimes, while side-stepping issues due to finite samples and label noise.
Revised: The infinite-sample setting allows us to focus on the effects inherent to the data distribution and the inductive bias of architecture (choice of activation function) and different learning regimes, while side-stepping issues due to finite samples and label noise.

Original: Also note that in this infinite-data setting, label noise provably has no influence on the learned model, in all the learning regimes considered.
Revised: Also note that in this infinite-data setting, label noise provably has no influence on the learned model, in all the learning regimes considered.

Original: The observation that there is a tradeoff between robustness and approximation, even in this infinite-sample setting, is one of the surprising findings of our work.
Revised: The observation that there is a tradeoff between robustness and approximation, even in this infinite-sample setting, is one of the surprising findings of our work.

Original: This complements related works such as (Bubeck et al., 2020b; Bubeck & Sellke, 2021), which show that finite training samples with label noise is a possible source of nonrobustness in neural networks.
Revised: This complements related works such as (Bubeck et al., 2020b; Bubeck & Sellke, 2021), which show that finite training samples with label noise is a possible source of nonrobustness in neural networks.

Original: Related works.
Revised: Related works.

Original: Various works have theoretically studied adversarial examples and robustness in supervised learning, and the relationship to ordinary test error / accuracy.
Revised: Various works have theoretically studied adversarial examples and robustness in supervised learning, and the relationship to ordinary test error / accuracy.

Original: Tsipras et al.
Revised: Tsipras et al.

Original: (2019) considers a specific data distribution where good test error implies poor robustness.
Revised: (2019) considers a specific data distribution where good test error implies poor robustness.

Original: Shafahi et al.
Revised: Shafahi et al.

Original: Mahloujifar et al.
Revised: Mahloujifar et al.

Original: ; Gilmer et al.
Revised: ; Gilmer et al.

Original: ; Dohmatob (2019) show that for high-dimensional data distributions which have concentration property (e.g., multivariate Gaussians, distributions satisfying log-Sobolev inequalities, etc.), an imperfect classifier will admit adversarial examples.
Revised: ; Dohmatob (2019) show that for high-dimensional data distributions which have concentration property (e.g., multivariate Gaussians, distributions satisfying log-Sobolev inequalities, etc.), an imperfect classifier will admit adversarial examples.

Original: On the other hand, Yang et al.
Revised: On the other hand, Yang et al.

Original: (2020) observed empirically that natural images are well-separated, and so locally-lipschitz classifies shouldn’t suffer any kind of test error vs robustness tradeoff.
Revised: (2020) observed empirically that natural images are well-separated, and so locally-lipschitz classifies shouldn’t suffer any kind of test error vs robustness tradeoff.

Original: However, gradient-descent is not likely to find such models.
Revised: However, gradient-descent is not likely to find such models.

Original: Our work studies regression problems with quadratic targets, and shows that there are indeed tradeoffs between test error and robustness which are controlled by the learning algorithm / regime and model.
Revised: Our work studies regression problems with quadratic targets, and shows that there are indeed tradeoffs between test error and robustness which are controlled by the learning algorithm / regime and model.

Original: Schmidt et al.
Revised: Schmidt et al.

Original: ; Khim & Loh (2018); Yin et al.
Revised: ; Khim & Loh (2018); Yin et al.

Original: Bhattacharjee et al.
Revised: Bhattacharjee et al.

Original: ; Min et al.
Revised: ; Min et al.

Original: (2021b;a) study the sample complexity of robust learning.
Revised: (2021b;a) study the sample complexity of robust learning.

Original: In contrasts, our work focuses on the case of infinite data, so that the only complexity parameters are the input dimension d and the network with m .
Revised: In contrasts, our work focuses on the case of infinite data, so that the only complexity parameters are the input dimension d and the network width m .

Original: Gao et al.
Revised: Gao et al.

Original: ; Bubeck et al.
Revised: ; Bubeck et al.

Original: (2020b);
Revised: (2020b);

Original: Bubeck & Sellke (2021) show that over-parameterization may be necessary for robust interpolation in the presence of noise.
Revised: Bubeck & Sellke (2021) show that over-parameterization may be necessary for robust interpolation in the presence of noise.

Original: In contrast, our paper considers a structured problem with noiseless signal and infinite training data, where the network width m and the input dimension d tend to infinity proportionately.
Revised: In contrast, our paper considers a structured problem with noiseless signal and infinite training data, where the network width m and the input dimension d tend to infinity proportionately.

Original: In this under-complete asymptotic setting, our results show a systematic and precise tradeoff between approximation (test error) and robustness in different learning regimes.
Revised: In this under-complete asymptotic setting, our results show a systematic and precise tradeoff between approximation (test error) and robustness in different learning regimes.

Original: Thus, our work nuances the picture presented by previous works by exhibiting a nontrivial interplay between robustness and test error, which persists even in the case of infinite training data where the resulting model isn’t affected by label noise.
Revised: Thus, our work nuances the picture presented by previous works by exhibiting a nontrivial interplay between robustness and test error, which persists even in the case of infinite training data where the resulting model isn’t affected by label noise.

Revised: Dohmatob (2021); Hassani & Javanmard (2022) study the tradeoffs between interpolation, predictive performance (test error), and robustness for finite-width over-parameterized networks in kernel regimes with noisy linear target functions.

Revised: In contrast, we consider structured quadratic target functions and compare different learning settings, including SGD optimization in a non-kernel regime, as well as lazy/linearized models.

Revised: Robustness has also been studied from a causality perspective.

Revised: For example, Rothenhausler et al.

Revised: (2021) studies tradeoffs between test error and robustness in linear regression under distributional shifts on the marginal distribution of the covariates.

Original: We provide a more detailed discussion of the related work in Appendix A.
Revised: We provide a more detailed discussion of the related work in Appendix A.

Original: P RELIMINARIES
Revised: Remark 1.1. Note that the term ”over-parametrization” is not used in our paper in the same sense as in Bubeck et al. (2020b); Bubeck & Sellke (2021); Hassani & Javanmard (2022). In those works, the setup is finite samples ( n < ∞ ), and over-parametrization means that m is substantially larger than n/d , where d is the input-dimension, and m is the network with (i.e number of neurons in the hidden layer). In our work, we focus on the infinite-sample case ( n = ∞ ), and over-parametrization means m/d is large.

Original: Notations.
Revised: Finally, the findings of Bubeck et al. (2020b);

Original: The set of integers from 1 through d will be denoted [ d ] . We will denote the identity matrix of size d by I d . The euclidean norm of a vector x ∈ R d will be denoted ∥ x ∥ . The k th largest singular-value of a matrix A will be denoted s k ( A ) , andequals the positive square-root of the k th largest eigenvalue of the positive-semidefinite (psd) matrix AA ⊤ .
Revised: Bubeck & Sellke (2021) –namely, that over-parametrization is beneficial for robustness, and of Hassani & Javanmard (2022) –namely, that over-parametrization is detrimental to robustness, are nuanced by Zhu et al.

Original: In particular, ∥ A ∥ op :=

Original: s 1 ( A ) is the spectral norm of A . If A is itself psd, then its singular-values coincide with its eigenvalues.
Revised: (2022) which shows that as the width m of a neural network is increased, there transition from over-parametrization being detrimental, to being benificial for robustness.

Original: The Frobenius norm of A is denoted ∥ A ∥ F and defined by ∥ A ∥ F := (cid:113)(cid:80) dk =1 s k
Revised: More precisely, they derived upper-bounds for robustness error which show that there critical value m 0 such that the robustness error is an increasing function of width in the interval [1 , m 0 ] (over-parametrization hurts) and a decreasing function of width in the interval [ m 0 , ∞ ) (i.e over-parametrization becomes beneficial).

Original: ( A ) 2 . More generally, we define ∥ A ∥ F,m := (cid:113)(cid:80) m ∧ d k =1 s k ( A ) 2 , so that ∥ A ∥ F,d = ∥ A ∥ F in particular.
Revised: Overall, the exact role of over-parametrization in robustness remains partly unclear, even though progress is being made on the subject.

Original:  [-Note that m (cid:55)→ ∥ A ∥ F,m is a nondecreasing function which is upper-bounded by ∥ A ∥ F . The Hadamard / element-wise product of two matrices A 
 and A 2 of the same shape will be denoted A 1 ◦ A 2 .-]
Revised:  [-Note that m (cid:55)→ ∥ A ∥ F,m is a nondecreasing function which is upper-bounded by ∥ A ∥ F . The Hadamard / element-wise product of two matrices A 
 and A 2 of the same shape will be denoted A 1 ◦ A 2 .-]P RELIMINARIES

Original: The squared L 2 -norm of a function f : R d → R w.r.t a distribution P on R d will be denoted ∥ f ∥ 2 P , and defined by ∥ f ∥ L 2 ( P ) := E P [ f ( x ) 2 ] , whenever this integral exists. Given a psd matrix of size d , we denote by N (0 , Σ) the d -dimensional multivariate gaussian distribution with covariance matrix Σ .
Revised: Notations.

Original: The usual notation O d (1) (resp. O d, P (1) ) is used to denote a quantity which remains bounded (resp. bounded in probability) in the limit d → ∞ .
Revised: We use standard notations in our manuscript.

Original: Likewise o d (1) (resp. o d, P (1) ) denotes a quantity which goes to zero (resp. which goes to zero in probability) in the limit d → ∞ . As usual, the acronym ”a.s” stands for almost-surely , while ”w.p p ” stands for with probability at least p .
Revised: A cheat sheet is provided in supplemental / appendix.

Original: 2. T HE TEACHER MODEL : A QUADRATIC FORM
Revised: 2. T HE TEACHER MODEL : A QUADRATIC FORM

Original: We consider the following regression setup proposed by Ghorbani et al.
Revised: We consider the following regression setup proposed by Ghorbani et al.

Original: Let B be
Revised: Let B be

Original: a fixed d × d psd matrix and let b 0 ∈ R be a fixed unknown scalar.
Revised: a fixed d × d psd matrix and let b 0 ∈ R be a fixed unknown scalar.

Original: Consider the quadratic teacher model
Revised: Consider the quadratic teacher model

Original: [Equation]
Revised: [Equation]

Original: any

Original: [Equation]
Revised: [Equation]

Original: We assume the input data is distributed according to N (0 , I d ) , the standard Gaussian distribution in d dimensions.
Revised: We assume the input data is distributed according to N (0 , I d ) , the standard Gaussian distribution in d dimensions.

Original: Thus, the structure of the problem of learning the teacher model
Revised: Thus, the structure of the problem of learning the teacher model

Original: f ⋆ in (1) is completely determined by the unknown d × d matrix B .
Revised: f ⋆ in (1) is completely determined by the unknown d × d matrix B .

Original: We assume an idealized scenario where the learner has access to an infinite number of iid samples of the form ( x, f ⋆ ( x )) with x ∼ P x := N (0 , I d ) .
Revised: We assume an idealized scenario where the learner has access to an infinite number of iid samples of the form ( x, f ⋆ ( x )) width x ∼ P x := N (0 , I d ) .

Original: For simplicity of analysis, we will further assume as in (Ghorbani et al., 2019) that the teacher model f ⋆ defined in (1) is centered, i.e E x ∼ N (0 ,I d )
Revised: For simplicity of analysis, we will further assume as in (Ghorbani et al., 2019) that the teacher model f ⋆ defined in (1) is centered, i.e E x ∼ N (0 ,I d )

Original: [ f ⋆ ( x )] = 0 .
Revised: [ f ⋆ ( x )] = 0 .

Original: This forces the offset b 0 = − tr( B ) .
Revised: This forces the offset b 0 = − tr( B ) .

Original: 2. T HE STUDENT MODEL : A TWO - LAYER NEURAL NETWORK
Revised: 2. T HE STUDENT MODEL : A TWO - LAYER NEURAL NETWORK

Original: Consider a two-layer student neural network
Revised: Consider a two-layer student neural network

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ) where m is the network width, i.e., the number of hidden neurons each with parameter vector w j ∈ R d , output weights z = ( z 1 , . . .
Revised: where m is the network width, i.e., the number of hidden neurons each with parameter vector w j ∈ R d , output weights z = ( z 1 , . . .

Original: z m ) ∈
Revised: z m ) ∈

Original: R m , and activation function σ : R → R .
Revised: R m , and activation function σ : R → R .

Original: We define W as the m × d matrix with j th row w j .
Revised: We define W as the m × d matrix with j th row w j .

Original: The scalar s is an offset which we will sometimes set to 0 , in which case we will simply write f W,z,s :=
Revised: The scalar s is an offset which we will sometimes set to 0 , in which case we will simply write f W,z,s :=

Original: f W,z, 0 .
Revised: f W,z, 0 .

Original: Note that the teacher model f ⋆ is itself a two-layer neural network with output weights fixed to 1 , quadratic activation function, and m = d hidden neurons with parameters W ⋆ :=
Revised: Note that the teacher model f ⋆ is itself a two-layer neural network with output weights fixed to 1 , quadratic activation function, and m = d hidden neurons with parameters W ⋆ :=

Original: B 1 / 2 , where B 1 / 2 is the unique psd matrix
Revised: B 1 / 2 , where B 1 / 2 is the unique psd matrix

Original: such that ( B 1 / 2 ) 2 = B .
Revised: such that ( B 1 / 2 ) 2 = B .

Original: The aim of learning is to approximate the teacher model f ⋆ as closely as possible with the student model f W,z,s .
Revised: The aim of learning is to approximate the teacher model f ⋆ as closely as possible with the student model f W,z,s .

Original: We will consider the following high-dimensional setup:
Revised: We will consider the following high-dimensional setup:

Original: – Infinite training data , wherein the sample size n is equal to ∞ ,
Revised: – Infinite training data , wherein the sample size n is equal to ∞ ,

Original: i
Revised: i

Original: e., the learner has access to the entire data distribution, allowing us to step-aside issues linked with finite samples.
Revised: e , the learner has access to the entire data distribution, allowing us to step-aside issues linked with finite samples.

Original: – Proportionate scaling of input dimension d and student network width m , wherein d and m are finite , and large of the same order,
Revised: – Proportionate scaling of input dimension d and student network width m , wherein d and m are finite , and large of the same order,

Original: i
Revised: i

Original: e.,
Revised: e.,

Original: [Equation]
Revised: [Equation]

Original: The parameterization rate ρ (which corresponds to the number of hidden neurons per input dimension), will play a crucial role in our analysis.
Revised: The parameterization rate ρ (which corresponds to the number of hidden neurons per input dimension), will play a crucial role in our analysis.

Original: The case ρ < 1 corresponds to under-parametrization , while ρ > 1 corresponds to over-parametrization .
Revised: The case ρ < 1 corresponds to under-parametrization , while ρ > 1 corresponds to over-parametrization .

Original: Occasionally, we will also consider the extreme over-parametrization regime corresponding to ρ ≫ 1 , or more precisely, the limiting case ρ → ∞ .
Revised: Occasionally, we will also consider the extreme over-parametrization regime corresponding to ρ ≫ 1 , or more precisely, the limiting case ρ → ∞ .

Original: 2. M ETRICS FOR TEST ERROR AND ROBUSTNESS
Revised: 2. M ETRICS FOR TEST ERROR AND ROBUSTNESS

Original: Test error.
Revised: Test error.

Original: The test / approximation error of a student model f : R d → R is defined by
Revised: The test / approximation error of a student model f : R d → R is defined by

Original: [Equation]
Revised: [Equation]

Original: where P x is the distribution of the features.
Revised: where P x is the distribution of the features.

Original: (4) measures how well the student f approximates the teacher model f ⋆ .
Revised: (4) measures how well the student f approximates the teacher model f ⋆ .

Original: Except otherwise explicitly stated, in this article we will always consider the isotropic case where the distribution of the features is P x = N (0 , I d ) , as in Ghorbani et al.
Revised: Except otherwise explicitly stated, in this article we will always consider the isotropic case where the distribution of the features is P x = N (0 , I d ) , as in Ghorbani et al.

Original: It will be instructive to compare the test error of f to that of the null predictor which outputs 0 on every input, namely ∥ f
Revised: It will be instructive to compare the test error of f to that of the null predictor which outputs 0 on every input, namely ∥ f

Original: ⋆ ∥ 2 L 2 ( P x ) .
Revised: ⋆ ∥ 2 L 2 ( P x ) .

Original: Thus, consider the normalized test error ,
Revised: Thus, consider the normalized test error ,

Original: [Equation]
Revised: [Equation]

Original: This quantity was studied in Ghorbani et al.
Revised: This quantity was studied in Ghorbani et al.

Original: (2019) where explicit analytic formulae were obtained for two-layer networks in various regimes of interest: networks fully trained by stochastic gradient-descent (SGD) on the population risk, random features (RF), and neural tangent (NT).
Revised: (2019) where explicit analytic formulae were obtained for two-layer networks in various regimes of interest: networks fully trained by stochastic gradient-descent (SGD) on the population risk, random features (RF), and neural tangent (NT).

Original: We shall consider these same regimes and establish tradeoffs between test error and robustness of the corresponding models.
Revised: We shall consider these same regimes and establish tradeoffs between test error and robustness of the corresponding models.

Original: This will paint a picture complementary to Ghorbani et al.
Revised: This will paint a picture complementary to Ghorbani et al.

Original: Measure of robustness / sensitivity.
Revised: Measure of robustness / sensitivity.

Original: We will measure the robustness of a smooth student model f : R d → R (e.g., the two-layer neural net (2)) by what we call its robustness error , defined as the square-root of its Dirichlet energy ε rob ( f ) w.r.t.
Revised: We will measure the robustness of a smooth student model f : R d → R (e.g., the two-layer neural net (2)) by what we call its robustness error , defined as the square-root of its Dirichlet energy ε rob ( f ) w.r.t.

Original: to a random test point x ∼ P x , that isε rob ( f ) : = ∥∇ x f
Revised: to a random test point x ∼ P x , that is

Original: [Equation]
Revised: [Equation]

Original: Smoothness here is in the very general sense of Gigli & Ledoux (2013, Section 4.1) with euclidean structure.
Revised: Smoothness here is in the very general sense of Gigli & Ledoux (2013, Section 4.1) with euclidean structure.

Original: The smaller the value of ε rob ( f ) , the more robust / less sensitive f is to changes in a test data point, on average.
Revised: The smaller the value of ε rob ( f ) , the more robust / less sensitive f is to changes in a test data point, on average.

Original: We justify the choice of this quantity as a measure of robustness in
Revised: We justify the choice of this quantity as a measure of robustness in

Original: Appendix C, where we will link it to more classical notions of robustness error (Madry et al., 2018).
Revised: Appendix D, where we will link it to more classical notions of robustness error (Madry et al., 2018).

Original: In particular, the teacher model has ε rob ( f ⋆ ) = 

Revised: In particular, the teacher model has ε rob ( f ⋆ ) =

Original: ∥ B ∥ 2 F .
Revised: ∥ B ∥ 2 F .

Original: Finally, note that, measures of robustness based on notions of sensitivity have been considered in other works like Bubeck et al.
Revised: Finally, note that, measures of robustness based on notions of sensitivity have been considered in other works like Bubeck et al.

Original: (2020b);
Revised: (2020b);

Original: Bubeck & Sellke (2021) for regression, and Wu et al. (2021) for classification settings.
Revised: Bubeck & Sellke (2021) for regression, and Wu et al. (2021) for classification settings.

Original: It will be convenient to compare the robustness of a student model f to that of the baseline quadratic teacher model f ⋆ defined in (1).
Revised: It will be convenient to compare the robustness of a student model f to that of the baseline quadratic teacher model f ⋆ defined in (1).

Original: To this end, consider the normalized robustness error f defined by
Revised: To this end, consider the normalized robustness error f defined by

Original: [Equation]
Revised: [Equation]

Original:  which measures the relative robustness error of the student.
Revised: 7) which measures the relative robustness error of the student.

Original: The objective of our paper is to study the quantity (cid:101) ε rob ( f ) for neural networks (2) in various regimes in the limit (3), and put to light interesting phenomena.
Revised: The objective of our paper is to study the quantity (cid:101) ε rob ( f ) for neural networks (2) in various regimes in the limit (3), and put to light interesting phenomena.

Original: In particular, we will establish tradeoffs between test error and robustness error, in the form of a nontrivial relationship (cid:101)
Revised: In particular, we will establish tradeoffs between test error and robustness error, in the form of a nontrivial relationship [Equation]

Original: ε test ( f ) + (cid:101) ε rob ( f ) = 1 , for different learning regimes. 
Revised: for different learning regimes. This paints a picture complementary to Ghorbani et al. Remark 2.1.

Original: R ESULTS FOR TWO - LAYER NEURAL NETWORKS TRAINED VIA SGD
Revised: Note that it might be tempting to thing that [Equation] Such an implication would automatically lead to a (at least) heuristic explanation of our tradeoffs (8) . However, (9) is false in general.

Original: Consider a student neural network model with quadratic activation f SGD : R d → R , i.e f SGD (  x [Equation] m
Revised: Indeed, a smooth function (small ∥∇ f ∥ L 2 ( P x ) ) can be approximated very well in L 2 by functions by very rough functions (large ∥∇ f ∥

Original: [Equation]
Revised: L 2 ( P x ) .

Original: s
Revised: This point is elaborated in Appendix K. To establish our tradeoffs, we exploit the fine structure of two-layer neural networks in the different learning regimes considered.  R ESULTS FOR TWO - LAYER NEURAL NETWORKS TRAINED VIA SGD Consider a student neural network model with quadratic activation f SGD : R d → R , i.

Original: [Equation]
Revised: [Equation]

Original: Here, W = ( w 1 , . .
Revised: Here, W = ( w 1 , . .

Original: w m ) ∈ R m × d is a matrix of learnable parameters (one per hidden neuron), and s ∈ R is a learnable offset.
Revised: w m ) ∈ R m × d is a matrix of learnable parameters (one per hidden neuron), and s ∈ R is a learnable offset.

Original: The output weights vector is fixed to z = 1 m := (1 , . .
Revised: The output weights vector is fixed to z = 1 m := (1 , . .

Original: . , 1) , while W and s are optimized via SGD (hence the subscript), where each update is on a single new sample point.
Revised: . , 1) , while W and s are optimized via SGD (hence the subscript), where each update is on a single new sample point.

Original: It is shown in Theorem 3 of Ghorbani et al.
Revised: It is shown in Theorem 3 of Ghorbani et al.

Original: that if W t ∈ R m × d is the matrix of hidden parameters after t steps of SGD, then in the limit (3), the matrix W t W ⊤ t ∈ R m × m converges a.s to the best rank- m approximation of B .
Revised: that if W t ∈ R m × d is the matrix of hidden parameters after t steps of SGD, then in the limit (3), the matrix W t W ⊤ t ∈ R m × m converges a.s to the best rank- m approximation of B .

Original: Thus, by continuity of matrix norms, we deduce that ∥ W t W ⊤ t ∥ 2 F converges a.s to ∥ B ∥ 2 F,m , in the infinite data limit t → ∞ .
Revised: Thus, by continuity of matrix norms, we deduce that ∥ W t W ⊤ t ∥ 2 F converges a.s to ∥ B ∥ 2 F,m , in the infinite data limit t → ∞ .

Original: Combining with Lemma F.1 establishes the following asymptotic formula for the (normalized) robustness of the resulting model f SGD , in the high-dimensional limit (3).
Revised: Combining with Lemma G.1 establishes the following asymptotic formula for the (normalized) robustness of the resulting model f SGD , in the high-dimensional limit (3).

Original: Theorem 3.1.
Revised: Theorem 3.1.

Original: In the limit (3) , it holds that (cid:101)
Revised: In the limit (3) , it holds that (cid:101)

Original: [Equation]
Revised: [Equation]

Original: with equality iff rank( B ) ≤ m .
Revised: equality iff rank( B ) ≤ m .

Original: In particular, if ρ ≥ 1 , then in the limit (3) , it holds that (cid:101) ε rob ( f SGD ) a.s −→ 1 .
Revised: In particular, if ρ ≥ 1 , then in the limit (3) , it holds that (cid:101) ε rob ( f SGD ) a.s −→ 1 .

Original: Tradeoff approximation and robustness.
Revised: Tradeoff approximation and robustness.

Original: We see from the above theorem that if m ≥ rank( B ) , then the robustness error for the learned student converges to that of the true model if m ≥ d , namely ε rob ( f SGD ) p → (cid:80) dj
Revised: We see from the above theorem that if m ≥ rank( B ) , then the robustness error for the learned student converges to that of the true model if m ≥ d , namely ε rob ( f SGD ) p → (cid:80) dj

Original: =1 λ j ( B ) 2 = ε rob ( f ⋆ ) .
Revised: =1 λ j ( B ) 2 = ε rob ( f ⋆ ) .

Original: This is the case if m ≥ d , for example.
Revised: This is the case if m ≥ d , for example.

Original: Otherwise (i.e if m < rank( B ) ), then the limiting value of ε rob ( f SGD ) can be arbitrarily less than ε rob ( f ⋆ ) , i.e., the learned student will be much more robust (i.e., stable) than the ground truth model.
Revised: Otherwise (i.e if m < rank( B ) ), then the limiting value of ε rob ( f SGD ) can be arbitrarily less than ε rob ( f ⋆ ) , i.e., the learned student will be much more robust (i.e., stable) than the ground truth model.

Original: Comparing with
Revised: Comparing with

Original: Theorem 3 and Proposition 1 of Ghorbani et al.
Revised: Theorem 3 and Proposition 1 of Ghorbani et al.

Original: (2019), we can see that any decrease in robustness error of the learned student (compared to the teacher model) is at the expense of increased test error, and vise versa.
Revised: (2019), we can see that any decrease in robustness error of the learned student (compared to the teacher model) is at the expense of increased test error, and vise versa.

Original: Indeed, it was shown in that paper that the normalized test error (cid:101) ε test ( f SGD ) verifies
Revised: Indeed, it was shown in that paper that the normalized test error (cid:101) ε test ( f SGD ) verifies

Original: [Equation]
Revised: [Equation]

Original: the
Revised: the

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Combining with our Theorem 3.1 above, we deduce that
Revised: Combining with our Theorem 3.1 above, we deduce that

Original: [Equation]
Revised: [Equation]

Original: The above formula highlights a tradeoff between test error and robustness error.
Revised: The above formula highlights a tradeoff between test error and robustness error.

Original: Thus, we have identified a novel tradeoff between approximation and robustness for the neural network model (2) trained via SGD.
Revised: Thus, we have identified a novel tradeoff between approximation and robustness for the neural network model (2) trained via SGD.

Original: In the sequel, we shall establish such tradeoffs for other learning regimes.
Revised: In the sequel, we shall establish such tradeoffs for other learning regimes.

Original:  R ESULTS FOR THE RANDOM FEATURES MODEL
Revised:  R ESULTS FOR THE RANDOM FEATURES MODEL

Original: Consider the two-layer model (2) with hidden neuron parameters w 1 , . .
Revised: Consider the two-layer model (2) with hidden neuron parameters w 1 , . .

Original: . , w m sampled iid from a d -dimensional multivariate Gaussian distribution N (0 , Γ) with covariance matrix Γ .
Revised: . , w m sampled iid from a d -dimensional multivariate Gaussian distribution N (0 , Γ) with covariance matrix Γ .

Original: We denote this so-called random features (RF) student model f RF , defined by
Revised: We denote this so-called random features (RF) student model f RF , defined by

Original: [Equation]
Revised: [Equation]

Original: where z RF ∈ R m solves the following linear regression problem
Revised: where z RF ∈ R m solves the following linear regression problem

Original: [Equation]
Revised: [Equation]

Original: It is easily seen that for x ∼ N (0 , I d ) , one has z RF =
Revised: It is easily seen that for x ∼ N (0 , I d ) , one has z RF =

Original: U − 1 v , with
Revised: U − 1 v , with

Original: [Equation]
Revised: [Equation]

Original: The covariance matrix Γ encompasses the inductive bias of the neurons at initialization to different directions in feature space.
Revised: The covariance matrix Γ encompasses the inductive bias of the neurons at initialization to different directions in feature space.

Original: Define the alignment α
Revised: Define the alignment α

Original: = α ( B, Γ) of the hidden neurons to the task at hand, namely learning the teacher model f ⋆ , as follows
Revised: = α ( B, Γ) of the hidden neurons to the task at hand, namely learning the teacher model f ⋆ , as follows

Original: [Equation]
Revised: [Equation]

Original: As we shall see, the task-alignment α plays a crucial role in the dynamics of prediction performance (test error) and robustness f RF .
Revised: As we shall see, the task-alignment α plays a crucial role in the dynamics of prediction performance (test error) and robustness f RF .

Original: 4. A SSUMPTIONS AND KEY QUANTITIES
Revised: 4. A SSUMPTIONS AND KEY QUANTITIES

Original: As in Ghorbani et al.
Revised: As in Ghorbani et al.

Original: (2019), we will need the following mild technical conditions on Γ .
Revised: (2019), we will need the following mild technical conditions on Γ .

Original: Condition 4.1.
Revised: Condition 4.1.

Original: The covariance matrix Γ satisfies: (A) tr(Γ)
Revised: The covariance matrix Γ satisfies: (A) tr(Γ)

Original: =
Revised: =

Original: 1 and d · ∥ Γ ∥ op
Revised: 1 and d · ∥ Γ ∥ op

Original: =
Revised: =

Original: O
Revised: O

Original: (B) The empirical eigenvalue distribution of d · Γ converges weakly to a probability distribution D on R + .
Revised: (B) The empirical eigenvalue distribution of d · Γ converges weakly to a probability distribution D on R + .

Original: This condition is quite reasonable, and moreover, it allows us to leverage standard tools from random matrix theory (RMT) in our analysis.
Revised: This condition is quite reasonable, and moreover, it allows us to leverage standard tools from random matrix theory (RMT) in our analysis.

Original: We will also need the following technical condition on the activation function σ .
Revised: We will also need the following technical condition on the activation function σ .

Original: Condition 4.2.
Revised: Condition 4.2.

Original: σ is weakly continuously-differentiable and satisfies the growth condition σ ( t ) 2 ≤ c 0 e c 1 t 2 for some c 0 > 0 and c 1 < 1 , and for all t ∈ R .
Revised: σ is weakly continuously-differentiable and satisfies the growth condition σ ( t ) 2 ≤ c 0 e c 1 t 2 for some c 0 > 0 and c 1 < 1 , and for all t ∈ R .

Original: Moreover, σ is not a purely affine function.
Revised: Moreover, σ is not a purely affine function.

Original: The above growth condition is a classical condition usually imposed for the theoretical analysis of neural networks (see, e.g., Ghorbani et al., 2019; Mei & Montanari, 2019; Montanari & Zhong, 2020), and is satisfied by all popular activation functions used in practice.
Revised: The above growth condition is a classical condition usually imposed for the theoretical analysis of neural networks (see, e.g., Ghorbani et al., 2019; Mei & Montanari, 2019; Montanari & Zhong, 2020), and is satisfied by all popular activation functions used in practice.

Original: One of its main purposes is to ensure that all the Hermite coefficients ( λ k ) k ∈ N of the activation function exist.
Revised: One of its main purposes is to ensure that all the Hermite coefficients ( λ k ) k ∈ N of the activation function exist.

Original: Refer to Section F. for precise definition of Hermite coefficients.
Revised: Refer to Section G. for precise definition of Hermite coefficients.

Original: We will also need the following condition.
Revised: We will also need the following condition.

Original: Condition 
Revised: Condition 4.3.

Original: [Equation]
Revised: (A) λ 0 := λ 0 ( σ ) = 0 . (B) λ 2 := λ 2 ( σ ) ̸ = 0 .

Original: Part (A) of this condition was introduced by Ghorbani et al.
Revised: Part (A) of this condition was introduced by Ghorbani et al.

Original: to simplify the analysis of the test error of the random features model f RF .
Revised: to simplify the analysis of the test error of the random features model f RF .

Original: Part (B) ensures that the random features model f RF does not degenerate to the null predictor.
Revised: Part (B) ensures that the random features model f RF does not degenerate to the null predictor.

Original: Definition 4.1.
Revised: Definition 4.1.

Original: With z ∼ N (0 , 1) , define the following scalars
Revised: With z ∼ N (0 , 1) , define the following scalars

Original: [Equation]
Revised: [Equation]

Original: These coefficients will turn out to be “sufficient statistics” which will completely capture the influence of activation function σ on the robustness of the random features model f RF .
Revised: These coefficients will turn out to be “sufficient statistics” which will completely capture the influence of activation function σ on the robustness of the random features model f RF .

Original: Note that by construction, λ , κ , λ ′ , and κ ′ are nonnegative.
Revised: Note that by construction, λ , κ , λ ′ , and κ ′ are nonnegative.

Original: Now, consider the random psd matrices
Revised: Now, consider the random psd matrices

Original: A 0 and D 0 defined by
Revised: A 0 and D 0 defined by

Original: [Equation]
Revised: [Equation]

Original:  with Θ := WW ⊤ ∈ R m × m .
Revised: (19) with Θ := WW ⊤ ∈ R m × m .

Original: These matrices appear upon linearizing the expressions for the test error and the robustness error of the RF model, using RMT techniques from El Karoui (2010).
Revised: These matrices appear upon linearizing the expressions for the test error and the robustness error of the RF model, using RMT techniques from El Karoui (2010).

Original: By employing the so-called Silverstein fixed-point equation (Silverstein & Choi, 1995; Ledoit & P´ech´e, 2011; Dobriban & Wager, 2018), one can show that there exist positive constants ψ 1 and ψ 2 such that
Revised: By employing the so-called Silverstein fixed-point equation (Silverstein & Choi, 1995; Ledoit & P´ech´e, 2011; Dobriban & Wager, 2018), one can show that there exist positive constants ψ 1 and ψ 2 such that

Original: [Equation]
Revised: [Equation]

Original: Moreover, the ψ k ’s only depend on (i) the parametrization rate ρ , and (ii) the limiting eigenvalue distribution D of the rescaled covariance matrix d · Γ of the hidden neurons at initialization.
Revised: Moreover, the ψ k ’s only depend on (i) the parametrization rate ρ , and (ii) the limiting eigenvalue distribution D of the rescaled covariance matrix d · Γ of the hidden neurons at initialization.

Original: Also, since λ and λ ′ are strictly positive (thanks to Condition 4.2), so are the ψ k ’s by definition of D 0 and
Revised: Also, since λ and λ ′ are strictly positive (thanks to Condition 4.2), so are the ψ k ’s by definition of D 0 and

Original: A 0 in (17).
Revised: A 0 in (19).

Original: These scalars together with those defined in (16) will play a crucial role in our analysis.
Revised: These scalars together with those defined in (18) will play a crucial role in our analysis.

Original: 4. T EST ERROR / PREDICTION PERFORMANCE IN RANDOM FEATURES REGIME
Revised: 4. T EST ERROR / PREDICTION PERFORMANCE IN RANDOM FEATURES REGIME

Original: We recall that the (normalized) test error (cid:101) ε test ( f RF ) of the random features model f RF was completely analyzed in Ghorbani et al.
Revised: We recall that the (normalized) test error (cid:101) ε test ( f RF ) of the random features model f RF was completely analyzed in Ghorbani et al.

Original: Indeed, the following was established that
Revised: Indeed, the following was established that

Original: [Equation]
Revised: [Equation]

Original: See Theorem 1 of the said paper.
Revised: See Theorem 1 of the said paper.

Original: Thus, the normalized test error only depends on the aspect ratio ρ , the limiting spectral distribution D of d · Γ , and the scale parameters ( λ, κ, τ ) defined in (16).
Revised: Thus, the normalized test error only depends on the aspect ratio ρ , the limiting spectral distribution D of d · Γ , and the scale parameters ( λ, κ, τ ) defined in (18).

Original: It was further that, if the task-alignment α of the hidden neurons defined in (15), admits a limit α ∞ when d → ∞ , then w.p
Revised: It was further that, if the task-alignment α of the hidden neurons defined in (17), admits a limit α ∞ when d → ∞ , then w.p

Original: [Equation]
Revised: [Equation]

Original: Thus, (19) predicts that the normalized test error (cid:101) ε test ( f RF ) vanishes if (i) Γ ∝ B and (ii) the number of neurons per input dimension m/d diverges, corresponding to extreme over-parametrization.
Revised: Thus, (21) predicts that the normalized test error (cid:101) ε test ( f RF ) vanishes if (i) Γ ∝ B and (ii) the number of neurons per input dimension m/d diverges, corresponding to extreme over-parametrization.

Original: 4. A NALYSIS OF ROBUSTNESS IN RANDOM FEATURES REGIME
Revised: 4. A NALYSIS OF ROBUSTNESS IN RANDOM FEATURES REGIME

Original: The following result establishes an analytic formula for the robustness in the RF regime.
Revised: The following result establishes an analytic formula for the robustness in the RF regime.

Original: [Figure]

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: Consider the random features model f RF (11) , with covariance matrix Γ satisfying
Revised: Consider the random features model f RF (13) , with covariance matrix Γ satisfying

Original: Condition 4.1 and activation function σ satisfying Conditions 4.2 and 4.3.
Revised: Condition 4.1 and activation function σ satisfying Conditions 4.2 and 4.3.

Original: (A)
Revised: A

Original: In the limit (3) , we have the following approximation
Revised: In the limit 3) , we have the following approximation

Original: [Equation]
Revised: [Equation]

Original: (B) Moreover, if lim d →∞ α = α ∞ , then lim
Revised: (B) Moreover, if lim d →∞ α = α ∞ , then lim

Original: ρ →∞ lim
Revised: ρ →∞ lim

Original: md
Revised: m

Original:  →∞
Revised: ,d →∞

Original: [Equation]
Revised: [Equation]

Original:  α [Equation] w.p 
Revised: = α 2 ∞ w.p 1 .

Original: In particular, for the optimal choice of Γ in terms of test error, namely Γ ∝ B , one has lim
Revised: In particular, for the optimal choice of Γ in terms of test error, namely Γ ∝ B , one has lim

Original: ρ →∞ lim 
Revised: ρ →∞ lim m,

Original: [Equation]
Revised: [Equation]

Original: p 1 . Thus, the robustness only depends on the aspect ratio ρ , the limiting spectral distribution D of d · Γ (via ψ 1 and ψ 2 ), and the scale parameters defined in (16).
Revised: Thus, the robustness only depends on the aspect ratio ρ , the limiting spectral distribution D of d · Γ (via ψ 1 and ψ 2 ), and the scale parameters defined in (18).

Original: The theorem is proved in the Appendix F.4.
Revised: The theorem is proved in the Appendix G.4.

Original: Tradeoff between approximation and robustness.
Revised: Tradeoff between approximation and robustness.

Original: We deduce from the above theorem that in the limit (3), the random features model f RF is more robust (i.e., less sensitive to perturbations) than the teacher model f ⋆ .
Revised: We deduce from the above theorem that in the limit (3), the random features model f RF is more robust (i.e., less sensitive to perturbations) than the teacher model f ⋆ .

Original: Interestingly, we see that this gap in robustness between the two models closes with increasing alignment α between the covariance matrix of the random features Γ and the coefficient matrix B .
Revised: Interestingly, we see that this gap in robustness between the two models closes with increasing alignment α between the covariance matrix of the random features Γ and the coefficient matrix B .

Original: Comparing with (20), we obtain the following relationship (provided ∥ Γ ∥ 2 F ≫ 1 /m ), 
Revised: Comparing with (22), we obtain the following relationship (provided ∥ Γ ∥ 2 F ≫ 1 /m ), [Equation]

Original: [Equation]
Revised: [Equation]

Original: which trades-off between the normalized test error (cid:101) ε test ( f RF ) (defined in (19)) and the normalized robustness (cid:101) ε rob ( f RF ) of f RF .
Revised: which trades-off between the normalized test error (cid:101) ε test ( f RF ) (defined in (21)) and the normalized robustness (cid:101) ε rob ( f RF ) of f RF .

Original: Thus, we have identified another novel tradeoff between the test error and the robustness in random features models.
Revised: Thus, we have identified another novel tradeoff between the test error and the robustness in random features models.

Original: Theorem 4.1 and Corollary G.1 are empirically verified in Fig.
Revised: Theorem 4.1 and Corollary H.1 are empirically verified in Fig.

Original: Results for the ReLU activation function are also shown.
Revised: Results for the ReLU activation function are also shown.

Original: Notice the perfect match between our theoretical results and experiments.
Revised: Notice the perfect match between our theoretical results and experiments.

Original:  N EURAL TANGENT REGIME
Revised:  N EURAL TANGENT REGIME

Original: Consider a two-layer network with output weights z 

Revised: Consider a two-layer network width output weights z

Original: j fixed to 1 , and hidden weight w j ∈ R d drawn from N (0 , Γ) .
Revised: j fixed to 1 , and hidden weight w j ∈ R d drawn from N (0 , Γ) .

Original: For the quadratic activation σ ( t ) :
Revised: For the quadratic activation σ ( t ) :

Original: = t 2 , the neural tangent (NT) approximation (Jacot et al., 2018; Chizat et al., 2019) w.r.t.
Revised: = t 2 , the neural tangent (NT) approximation (Jacot et al., 2018; Chizat et al., 2019) w.r.t.

Original: the first layer parameters is given by 
Revised: the first layer parameters is given by f W + A ( x ) ≈ f [Equation] f

Original: [Equation]
Revised: [Equation]

Revised: x

Original: [Equation]
Revised: [Equation]

Revised: f

Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original:  (see Appendix D for details), and A =
Revised: where f init is the function computed by the neural network at initialization (see Appendix E for details) and A =

Original: ( a 1 , . .
Revised: ( a 1 , . .

Original: . , a m ) = (∆ w 1 , . .
Revised: . , a m ) = (∆ w 1 , . .

Original: . , ∆ w m ) ∈ R m × d is the change in W .
Revised: . , ∆ w m ) ∈ R m × d is the change in W .

Original: We will see that the initialization term f init might have drastic influence on the robustness of the resulting model.
Revised: We will see that the initialization term f init might have drastic influence on the robustness of the resulting model.

Original: 5. N EURAL TANGENT APPROXIMATION WITHOUT INITIALIZATION TERM
Revised: 5. N EURAL TANGENT APPROXIMATION WITHOUT INITIALIZATION TERM

Original: We temporarily discard the initialization term f init ( x ) from the RHS of (23), and consider the simplified approximation
Revised: We temporarily discard the initialization term f init ( x ) from the RHS of (25), and consider the simplified approximation

Original: [Equation]
Revised: [Equation]

Original:  where, WLOG, we absorb the output weights z j in the parameters a j in the first-order term.
Revised: (26) where, WLOG, we absorb the output weights z j in the parameters a j in the first-order term.

Original: In (24),
Revised: In (26),

Original: A ∈ R m × d and c ∈ R are model parameters that are optimized.
Revised: A ∈ R m × d and c ∈ R are model parameters that are optimized.

Original: In terms of test error, let A NT and c NT be optimal in f NT ( · ; A, c ) , and let f NT
Revised: In terms of test error, let A NT and c NT be optimal in f NT ( · ; A, c ) , and let f NT

Original: = f NT ( · ; A NT , c NT ) for short.
Revised: = f NT ( · ; A NT , c NT ) for short.

Original: In Thm.
Revised: In Thm.

Original: 2 of Ghorbani et al.
Revised: 2 of Ghorbani et al.

Original: (2019), it is shown that the (normalized) test error of the linearized model f NT is given by
Revised: (2019), it is shown that the (normalized) test error of the linearized model f NT is given by

Original: [Equation]
Revised: [Equation]

Original: (25) where β
Revised: where β

Original: = β ( B ) := tr( B ) 2 / ( d ∥ B ∥ 2 F ) ∈ (0 , 1] .
Revised: = β ( B ) := tr( B ) 2 / ( d ∥ B ∥ [Equation] ) ∈ (0 , 1] .

Original: We now establish an analytic formula for the robustness error of f NT .
Revised: We now establish an analytic formula for the robustness error of f NT .

Original: Theorem 5.1.
Revised: Theorem 5.1.

Original: Consider the neural tangent model f NT in (24) .
Revised: Consider the neural tangent model f NT in (26) .

Original: In the limit (3) it holds that,
Revised: In the limit (3) it holds that,

Original: [Equation]
Revised: [Equation]

Original: 1) , where ρ := min( ρ, 1) . 
Revised: ρ := min( ρ, 1) . [Equation]

Original: Further observe that because 0 ≤ β ≤ 1 , the RHS of (26) is further upper-bounded by ρ ≤ 1 with equality when β
Revised: Further observe that because 0 ≤ β ≤ 1 , the RHS of (28) is further upper-bounded by ρ ≤ 1 with equality when β

Original: = 1 (e.g., for B ∝ I d ).
Revised: = 1 (e.g., for B ∝ I d ).

Original: We deduce that in the NT regime, the student neural network is at least as robust as the teacher model f ⋆ .
Revised: We deduce that in the NT regime, the student neural network is at least as robust as the teacher model f ⋆ .

Original: Comparing with (25), we obtain the following tradeoff between test error and robustness, stated only for β = 1 for simplicity of presentation.
Revised: Comparing with (27), we obtain the following tradeoff between test error and robustness, stated only for β = 1 for simplicity of presentation.

Original: Corollary 5.1.
Revised: Corollary 5.1.

Original: If β = 1 (i.e., if B ∝ I d ) , then in the limit (3) , it holds that
Revised: If β = 1 (i.e., if B ∝ I d ) , then in the limit (3) , it holds that

Original: [Equation]
Revised: [Equation]

Original: 5. N EURAL TANGENT APPROXIMATION WITH INITIALIZATION TERM
Revised: 5. N EURAL TANGENT APPROXIMATION WITH INITIALIZATION TERM

Original: We now consider the neural tangent approximation (23) without discarding the initialization term f init from the RHS of (23).
Revised: We now consider the neural tangent approximation (25) without discarding the initialization term f init from the RHS of (25).

Original: Also, let z 0 ∈ R
Revised: Also, let z 0 ∈ R

Original: m be the output weights, drawn iid from N (0 , 1 /m ) and frozen, and let Q be the m × m diagonal matrix with z 0 as its diagonal.
Revised: m be the output weights, drawn iid from N (0 , 1 /m ) and frozen, and let Q be the m × m diagonal matrix with z 0 as its diagonal.

Original: This corresponds to what  could be referred toas lazy training  regime of the hidden layer.
Revised: This corresponds to what [Figure] could be referred to training Chizat et al. (2019) regime of the hidden layer.

Original: Let f NTL ( x ; A, c ) be RHS of (23),
Revised: Let f NTL ( x ; A, c ) be RHS of (25),

Original: [Equation]
Revised: [Equation]

Original: where f init ( x ) := (cid:80) mj
Revised: where f init ( x ) := (cid:80) mj

Original: =1 z 0 j ( x ⊤ w j ) 2 = x ⊤ W ⊤ QWx defines the neural network at initialization.
Revised: =1 z 0 j ( x ⊤ w j ) 2 = x ⊤ W ⊤ QWx defines the neural network at initialization.

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: Suppose the output weights z 0 at initialization are iid from N (0 , (1 /m ) I m ) .
Revised: Suppose the output weights z 0 at initialization are iid from N (0 , (1 /m ) I m ) .

Original: Then, in the limit (3) , the following identities hold
Revised: Then, in the limit (3) , the following identities hold

Original: [Equation]
Revised: [Equation]

Original: [Equation] Thus, on average (over initialization): (i) f NTL and f NT have the same test error, i.e., the initialization term f init in f NTL does not affect its test error.
Revised: Thus, on average (over initialization): (i) f NTL and f NT have the same test error, i.e., the initialization term f init in f NTL does not affect its test error.

Original: (ii)
Revised: (ii)

Original: On the other hand, f NTL is less robust than f NT ; the deficit in robustness, namely the term (1 + ∥ Γ ∥ 2 F ) /
Revised: On the other hand, f NTL is less robust than f NT ; the deficit in robustness, namely the term (1 + ∥ Γ ∥ 2 F ) /

Original: ∥ B ∥ 2 F , corresponds exactly to the contribution of the initialization.
Revised: ∥ B ∥ 2 F , corresponds exactly to the contribution of the initialization.

Original: The situation is empirically illustrated in Fig.
Revised: The situation is empirically illustrated in Fig.

Original: Notice the perfect match between our theoretical results and experiments.
Revised: Notice the perfect match between our theoretical results and experiments.

Original: G ENERAL NONLINEAR TEACHER AND STUDENT MODELS

Original: The results we established so far are for student-teacher models which are two-layer neural networks in certain learning regimes with Gaussian data.

Original: In this section, we consider much more general scenarios, and show lower-bounds that display similar tradeoffs between test error and robustness.

Original: Suppose the distribution P x of the features is any distribution on R d which satisfies a Poincar´e inequality with constant c 2 > 0 .

Original: This means that for any smooth function f : R d → R , one has Var P x ( f ) ≤ c 2 ∥∇ f ∥ 2 L 2 ( P x ) , where Var P x ( f ) :=

Original: ∥ f − f ∥ 2 L 2 ( P x ) is the variance of f and f :=

Original: E P x f ∈ R is its mean w.r.t P x .

Original: For example, N (0 , Σ) verifies a Poincar ´ e inequality with c 2 = ∥

Original: Σ

Original: ∥ op .

Original: Consider a teacher model f ⋆ which is now any function in L 2 ( P x ) with mean f ⋆ = 0 .

Original: Theorem 6.1.

Original: For every smooth student model f : R d → R (neural network or not!), it holds that

Original: [Equation]

Original: The nature of Theorem 6.1 is a tradeoff since it directly implies that the test error cannot be decreased without increasing the robustness error.

Original: In the particular case of isotropic features where P x =

Original: N (0 , I d ) as considered in the preceding sections, a Poincar´e inequality with constant c 2 = 1 is satisfied, and we deduce from the above theorem that, for any smooth student model f , one has

Original: [Equation]

Original: Of course, apart from being only one-sided, the inequality (31) is weaker than the tradeoffs established in the preceding sections, due to the square-roots in the former.

Original: However, (31) holds without any real restriction on the teacher model f ⋆ , student

Original: f model, or learning algorithm / regime; it is solely a consequence of the high-dimensional geometry of the distribution of the features, manifested via the Poincar´e inequality.

Original: In contrast, the tradeoffs established in the preceding sections where for student-teacher models which where two-layer neural networks in various learning regimes.

Original:  C ONCLUDING REMARKS
Revised:  C ONCLUDING REMARKS

Original: In this paper, we have studied the adversarial robustness of two-layer neural networks in different high-dimensional learning regimes, and established a number of new tradeoffs between prediction performance and robustness.
Revised: In this paper, we have studied the adversarial robustness of two-layer neural networks in different high-dimensional learning regimes, and established a number of new tradeoffs between prediction performance and robustness, in the form (8).

Original: Our analysis also shows that random initialization can further degrade the robustness in lazy training regimes: for ”large” random initialization, the trained neural network inherits additional vulnerability already present at initialization.
Revised: Our analysis also shows that random initialization can further degrade the robustness in lazy training regimes: for ”large” random initialization, the trained neural network inherits additional vulnerability already present at initialization.

Original: Our work can be seen as a first step towards a rigorous theoretical understanding of the robustness of trained neural networks, an important subject which is still understudied.
Revised: Our work can be seen as a first step towards a rigorous theoretical understanding of the robustness of trained neural networks, an important subject which is still understudied

Original: Appendix / Supplementary Materials C ONTENTS A Further related work
Revised: Appendix / Supplementary Materials C ONTENTS A Further related work

Original: B  Warm-up: An insight from linear regression
Revised: B Notations C Warm-up: An insight from linear regression

Original: C Justification of our proposed measure of robustness
Revised: D Justification of our proposed measure of robustness

Original: C.1 Why not use Lipschitz constants to measure robustness ? . . . . . . . . . . . . . .
Revised: D.1 Why not use Lipschitz constants to measure robustness ? . . . . . . . . . . . . . .

Original: C.2 Proofs for Dirichlet energy as a measure of adversarial vulnerability . . . . . . . .
Revised: D.2 Proofs for Dirichlet energy as a measure of adversarial vulnerability . . . . . . . .

Original: D Neural networks at (random) initialization
Revised: E Neural networks at (random) initialization

Original: E Miscellaneous
Revised: F Miscellaneous

Original: E.1 Lazy training of output layer in RF regime . . . . . . . . . . . . . . . . . . . . . .
Revised: F. Lazy training of output layer in RF regime . . . . . . . . . . . . . . . . . . . . . .

Revised: F.

Original: E.2 Effect of regularization in RF regime . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: Effect of regularization in RF regime . . . . . . . . . . . . . . . . . . . . . . . . .

Original: F Technical proofs
Revised: G Technical proofs

Original: F. A useful lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: G.1 A useful lemma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: F. Hermite coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .F.
Revised: G.2 Hermite coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original:  Approximation of random matrices . . . . . . . . . . . . . . . . . . . . . . . . . .
Revised: G.3 Approximation of random matrices . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: F. Proof of Theorem 4.1: Analytic formula for robustness of RF model . . . . . . . .
Revised: G.4 Proof of Theorem 4.1: Analytic formula for robustness of RF model . . . . . . . .

Original: G Proofs of main results
Revised: H Proofs of main results

Original: G.1 Proof of Theorem D.1: Robustness error of neural networks at initialization . . . .
Revised: H. Proof of Theorem E 1: Robustness error of neural networks at initialization . . . .

Original: G.2 Proof of Theorem D.2: test error of neural network at initialization. . . . . . . . .
Revised: H. 2 Proof of Theorem E 2: test error of neural network at initialization

Original: G.3 The special case of quadratic activations . . . . . . . . . . . . . . . . . . . . . . .
Revised: H. 3 The special case of quadratic activations H.

Original: G.4 Proof of Theorem E.1: Random features lazy (RFL) regime. . . . . . . . . . . . .
Revised: 4 Proof of Theorem F.1: Random features lazy (RFL) regime

Original: G.5 Proof of Theorem 5.1: Neural tangent (NT) regime . . . . . . . . . . . . . . . . . 
Revised: H.5 Proof of Theorem 5.1: Neural tangent (NT) regime . . . . . . . . . . . . . . . . . H

Original: G.6 Proof of Theorem 5.2: Neural tangent lazy (NTL) regime . . . . . . . . . . . . . .
Revised: [Table] 6 Proof of Theorem 5. Neural tangent lazy (NTL) regime . . I General nonlinear teacher and student models

Original: H Proof of Theorem 6. 
Revised: J Proof of Theorem I.

Revised: K Approximating function values doesn’t amount to approximating grandients

Original: {+K.
+}
Revised: {+K.
+}

Revised: Disproving (9) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Original: {+K.
+}
Revised: {+K.
+}

Revised: Some exceptional cases where (1) holds . . . . . . . . . . . . . . . . . . . . . . .

Original: A F URTHER RELATED WORK
Revised: A F URTHER RELATED WORK

Original: Various works have theoretically studied adversarial examples and robustness in supervised learning, and the relationship to ordinary predictive performance / test error.
Revised: Various works have theoretically studied adversarial examples and robustness in supervised learning, and the relationship to ordinary predictive performance / test error.

Original: We present a detailed list here.
Revised: We present a detailed list here.

Original: Tsipras et al.
Revised: Tsipras et al.

Original: (2019) considers a specific data distribution where good accuracy implies poor robustness.
Revised: (2019) considers a specific data distribution where good accuracy implies poor robustness.

Original: Shafahi et al.
Revised: Shafahi et al.

Original: ; Mahloujifar et al.
Revised: ; Mahloujifar et al.

Original: ; Gilmer et al.
Revised: ; Gilmer et al.

Original: ; Dohmatob (2019) show that for high-dimensional data distributions which have concentration property (e.g., multivariate Gaussians, distributions satisfying log-Sobolev inequalities, etc.), an imperfect classifier will admit adversarial examples.
Revised: ; Dohmatob (2019) show that for high-dimensional data distributions which have concentration property (e.g., multivariate Gaussians, distributions satisfying log-Sobolev inequalities, etc.), an imperfect classifier will admit adversarial examples.

Original: Dobriban et al.
Revised: Dobriban et al.

Original: (2020) studies tradeoffs in Gaussian mixture classification problems, highlighting the impact of class imbalance.
Revised: (2020) studies tradeoffs in Gaussian mixture classification problems, highlighting the impact of class imbalance.

Original: On the other hand, Yang et al.
Revised: On the other hand, Yang et al.

Original: (2020) observed empirically that natural images are well-separated, and so locally-lipschitz classifies shouldn’t suffer any kind of test error vs robustness tradeoff.
Revised: (2020) observed empirically that natural images are well-separated, and so locally-lipschitz classifies shouldn’t suffer any kind of test error vs robustness tradeoff.

Original: However, gradient-descent is not likely to find such models.
Revised: However, gradient-descent is not likely to find such models.

Original: Our work studies regression problems with quadratic targets, and shows that there are indeed tradeoffs between test error and robustness which are controlled by the learning algorithm / regime and model.
Revised: Our work studies regression problems with quadratic targets, and shows that there are indeed tradeoffs between test error and robustness which are controlled by the learning algorithm / regime and model.

Original: Simon-Gabriel et al.
Revised: Simon-Gabriel et al.

Original: ; Daniely & Shacham (2020); Bubeck et al.
Revised: ; Daniely & Shacham (2020); Bubeck et al.

Original: ; Bartlett et al.
Revised: ; Bartlett et al.

Original: (2021) study adversarial vulnerability of neural networks at initialization, but do not consider the effects of training the model, in contrast to our work.
Revised: (2021) study adversarial vulnerability of neural networks at initialization, but do not consider the effects of training the model, in contrast to our work.

Original: Schmidt et al.
Revised: Schmidt et al.

Original: ; Khim & Loh (2018); Yin et al.
Revised: ; Khim & Loh (2018); Yin et al.

Original: Bhattacharjee et al.
Revised: Bhattacharjee et al.

Original: ; Min et al.
Revised: ; Min et al.

Original: (2021b;a) study the sample complexity of robust learning.
Revised: (2021b;a) study the sample complexity of robust learning.

Original: In contrasts, our work focuses on the case of infinite data, so that the only complexity parameters are the input dimension d and the network with m .
Revised: In contrasts, our work focuses on the case of infinite data, so that the only complexity parameters are the input dimension d and the network width m .

Original: Bhattacharjee et al.
Revised: Bhattacharjee et al.

Original: (2021) studies robustness vs accuracy for data distributions which are well-separated (e.g., say the two classes are supported on disjoint balls).
Revised: (2021) studies robustness vs accuracy for data distributions which are well-separated (e.g., say the two classes are supported on disjoint balls).

Original: The main finding in that paper is that (i) the robustness vs accuracy tradeoff doesn’t exist for well-separated datasets.
Revised: The main finding in that paper is that (i) the robustness vs accuracy tradeoff doesn’t exist for well-separated datasets.

Original: The work also posits that (ii) real-world datasets are well-separated.
Revised: The work also posits that (ii) real-world datasets are well-separated.

Original: We think (i) is only an artifact of the well-separatedness assumption (an assumption which fails for Gaussians (as noted in the paper), say, due to infinite support).
Revised: We think (i) is only an artifact of the well-separatedness assumption (an assumption which fails for Gaussians (as noted in the paper), say, due to infinite support).

Original: Also, (ii) is likely due to the fact that most real datasets are limited in sample size, and so, deceptively appear to be well-separated.
Revised: Also, (ii) is likely due to the fact that most real datasets are limited in sample size, and so, deceptively appear to be well-separated.

Original: Indeed, in the real world, there are cats which look like dogs (e.g, Siamese cats), even though such data might be under-represented in ML datasets.
Revised: Indeed, in the real world, there are cats which look like dogs (e.g, Siamese cats), even though such data might be under-represented in ML datasets.

Original: Gao et al.
Revised: Gao et al.

Original: ; Bubeck et al.
Revised: ; Bubeck et al.

Original: (2020b);
Revised: (2020b);

Original: Bubeck & Sellke (2021) show that over-parameterization may be necessary for robust interpolation in the presence of noise.
Revised: Bubeck & Sellke (2021) show that over-parameterization may be necessary for robust interpolation in the presence of noise.

Original: In contrast, our paper considers a structured problem with noiseless signal and infinite-data n = ∞ , where the network width m and the input dimension d tend to infinity proportionately.
Revised: In contrast, our paper considers a structured problem with noiseless signal and infinite-data n = ∞ , where the network width m and the input dimension d tend to infinity proportionately.

Original: In this under-complete asymptotic setting, our results show a precise picture of the tradeoffs between approximation (test error) and robustness in different learning regimes.
Revised: In this under-complete asymptotic setting, our results show a precise picture of the tradeoffs between approximation (test error) and robustness in different learning regimes.

Original: Our work nuances this picture by exhibiting a nontrivial interplay between robustness and test error which persists even in the case of infinite samples, where the model isn’t affected by label noise. 
Revised: Our work nuances this picture by exhibiting a nontrivial interplay between robustness and test error which persists even in the case of infinite samples, where the model isn’t affected by label noise. B N OTATIONS Linear algebra.

Original: Dohmatob (2021); Hassani & Javanmard (2022) study the tradeoffs between interpolation, predictive performance (test error), and robustness for finite-width over-parameterized networks in kernel regimes with noisy linear target functions.
Revised: The set of integers from 1 through d will be denoted [ d ] . We will denote the identity matrix of size d by I d . The euclidean norm of a vector x ∈ R d will be denoted ∥ x ∥ . The k th largest singular-value of a matrix A will be denoted s k ( A ) , and equals the positive square-root of the k th largest eigenvalue of the positive-semidefinite (psd) matrix AA ⊤ .

Revised: In particular, ∥ A ∥ op :=

Revised: s 1 ( A ) is the spectral norm of A .

Revised: If A is itself psd, then its singular-values coincide with its eigenvalues.

Revised: The Frobenius norm of A is denoted ∥ A ∥ F and defined by ∥ A ∥ F :=

Revised: (cid:113)(cid:80) dk

Revised: =1 s k

Revised: ( A ) 2 .

Revised: More generally, we define ∥ A ∥ F,m := (cid:113)(cid:80) m ∧ d k =1 s k

Revised: ( A ) 2 , so that ∥ A ∥ F,d = ∥

Revised: A ∥ F in particular.

Revised: Note that m (cid:55)→ ∥

Revised: A ∥ F,m is a nondecreasing function which is upper-bounded by ∥ A ∥ F .

Original: In contrast, we consider structured quadratic target functions{+The Hadamard / element-wise product of two matrices A 
+} and compare different learning settings, including SGD optimization in a non-kernel regime, as well as lazy/linearized models.
Revised: {+The Hadamard / element-wise product of two matrices A 
+} and A 2 of the same shape will be denoted A 1 ◦ A 2 . The squared L 2 -norm of a function f : R d → R w.r.t a distribution P on R d will be denoted ∥ f ∥ 2 P , and defined by ∥ f ∥ L 2 ( P ) :=

Revised: E P [ f ( x ) 2 ] , whenever this integral exists.

Revised: Given a psd matrix of size d , we denote by N (0 , Σ) the d -dimensional multivariate gaussian distribution with covariance matrix Σ .

Revised: Asymptotics.

Revised: The usual notation O d (1) (resp.

Revised: O d, P (1) ) is used to denote a quantity which remains bounded (resp.

Revised: bounded in probability) in the limit d → ∞ .

Revised: Likewise o d (1) (resp.

Revised: o d, P (1) ) denotes a quantity which goes to zero (resp.

Revised: which goes to zero in probability) in the limit d → ∞ .

Revised: As usual, the acronym ”a.s” stands for almost-surely , while ”w.p p ” stands for with probability at least p .

Original: B W ARM - UP : A N INSIGHT FROM LINEAR REGRESSION
Revised: C W ARM - UP : A N INSIGHT FROM LINEAR REGRESSION

Original: Before providing complete proofs in the sequel, in this section we will use linear regression to develop an intuitive understanding for the tradeoffs established in the paper.
Revised: Before providing complete proofs in the sequel, in this section we will use linear regression to develop an intuitive understanding for the tradeoffs established in the paper.

Original: Consider n sample points ( x 1 , y 1 ) , . .
Revised: Consider n sample points ( x 1 , y 1 ) , . .

Original: , ( x n , y n ) in R d × R .
Revised: , ( x n , y n ) in R d × R .

Original: Now, let X =
Revised: Now, let X =

Original: ( x 1 , . . .
Revised: ( x 1 , . . .

Original: , x n ) ∈ R n × d be the design matrix and y :=
Revised: , x n ) ∈ R n × d be the design matrix and y :=

Original: ( y 1 , . . .
Revised: ( y 1 , . . .

Original: , y n ) ∈ R n be the response vector.
Revised: , y n ) ∈ R n be the response vector.

Original: For any nonempty subset S of [ n ] , let X S ∈ R | S |× d (resp.
Revised: For any nonempty subset S of [ n ] , let X S ∈ R | S |× d (resp.

Original: y S ∈ R | S | ) be the version of X (resp.
Revised: y S ∈ R | S | ) be the version of X (resp.

Original: y ) with all rows (resp.
Revised: y ) with all rows (resp.

Original: columns) not in S removed.
Revised: columns) not in S removed.

Original: For a (possibly random) sequence of nonempty subsets ( S t ) ∞ t =1 of [ n ] , and a sequence of sufficiently small stepsizes ( α t ) ∞
Revised: For a (possibly random) sequence of nonempty subsets ( S t ) ∞ t =1 of [ n ] , and a sequence of sufficiently small stepsizes ( α t ) ∞

Original: t =1 , the following discrete dynamics
Revised: t =1 , the following discrete dynamics

Original: [Equation]
Revised: [Equation]

Original: (32) represents GD or SGD initialized at w 0 ∈ R d .
Revised: (33) represents GD or SGD initialized at w 0 ∈ R d .

Original: In particular, GD corresponds to taking S t =
Revised: In particular, GD corresponds to taking S t =

Original: [ n ] for all t ≥ 1 .
Revised: [ n ] for all t ≥ 1 .

Original: By construction, it is clear from (32) that at any iteration t ,
Revised: By construction, it is clear from (33) that at any iteration t ,

Original: [Equation]
Revised: [Equation]

Original: On the other hand, suppose w ⋆ ∈ R d is an interpolant, i.e Xw ⋆ =
Revised: On the other hand, suppose w ⋆ ∈ R d is an interpolant, i.e Xw ⋆ =

Original: y .
Revised: y .

Original: It is well-known that w t converges to a point w ∞ which is the orthogonal projection of w 0 onto the affine space of interpolants
Revised: It is well-known that w t converges to a point w ∞ which is the orthogonal projection of w 0 onto the affine space of interpolants

Original: I := { w ∈ R d | Xw = y } = w ⋆ + kern( X ) , where kern( X ) ⊆
Revised: I := { w ∈ R d | Xw = y } = w ⋆ + kern( X ) , where kern( X ) ⊆

Original: R d is the kernel of X .
Revised: R d is the kernel of X .

Original: Thanks to (33) and the closedness of the subspace span( X ) , it is clear that w ∞ − w 0 ∈ span( X ) .
Revised: Thanks to (34) and the closedness of the subspace span( X ) , it is clear that w ∞ − w 0 ∈ span( X ) .

Original: We conclude that w ∞ − w ⋆ is orthogonal to w ∞ − w 0 , and so ∥ w ⋆ − w 

Revised: We conclude that w ∞ − w ⋆ is orthogonal to w ∞ − w 0 , and so ∥ w ⋆ − w

Original: ∥ 2 = ∥ w ⋆ − w ∞ ∥
Revised: ∥ 2 = ∥ w ⋆ − w ∞ ∥

Original: 2 + ∥ w ∞ − w 0 ∥ 2 , by
Revised: 2 + ∥ w ∞ − w 0 ∥ 2 , by

Original: Pythagoras’ Theorem .
Revised: Pythagoras’ Theorem .

Original: In particular, taking w 0 = 0 , the previous identity becomes
Revised: In particular, taking w 0 = 0 , the previous identity becomes

Original: [Equation]
Revised: [Equation]

Original: where the test error is w.r.t to test data generated according to the linear model x ∼ P x := N (0 , I d ) , y = f w ⋆ ( x ) , with f w ( x ) :
Revised: where the test error is w.r.t to test data generated according to the linear model x ∼ P x := N (0 , I d ) , y = f w ⋆ ( x ) , with f w ( x ) :

Original: = x ⊤ w for all w, x ∈ R d .
Revised: = x ⊤ w for all w, x ∈ R d .

Original: Dividing through by ∥ w ⋆ ∥ 2 = ∥
Revised: Dividing through by ∥ w ⋆ ∥ 2 = ∥

Original: f w ⋆ ∥ 2 L 2 ( P x ) , we deduce the following result which can be seen as the inductive bias of GD and SGD on linear regression.
Revised: f w ⋆ ∥ 2 L 2 ( P x ) , we deduce the following result which can be seen as the inductive bias of GD and SGD on linear regression.

Original: Proposition B.1.
Revised: Proposition C.1.

Original: For GD or SGD started at w 0 = 0 , it holds that (cid:101) ε test ( f w [Equation] + (cid:101) [Equation]
Revised: For GD or SGD started at w 0 = 0 , it holds that (cid:101) ε test ( f w ∞ ) + (cid:101) ε rob ( f w ∞ )

Original: rob

Original: [Equation]

Original: f w ∞

Original: [Equation]

Original: = 1 .
Revised: = 1 .

Original: This is a tradeoff between the test error and the robustness error for GD and SGD!
Revised: This is a tradeoff between the test error and the robustness error for GD and SGD!

Original: It is valid for all sample sizes n ≥ 1 .
Revised: It is valid for all sample sizes n ≥ 1 .

Original: In contrast, the tradeoffs established in the nonlinear settings of the previous sections persist hold for infinite samples where n = ∞ .
Revised: In contrast, the tradeoffs established in the nonlinear settings of the previous sections persist hold for infinite samples where n = ∞ .

Original: Perhaps, sufficiently large but finite n is sufficient, but this investigation is left for future work.
Revised: Perhaps, sufficiently large but finite n is sufficient, but this investigation is left for future work.

Original: C J USTIFICATION OF OUR PROPOSED MEASURE OF ROBUSTNESS
Revised: D J USTIFICATION OF OUR PROPOSED MEASURE OF ROBUSTNESS

Original: Let us begin by explaining why our proposed measure of robustness based on Dirichlet energy (6) is actually a measure of robustness.
Revised: Let us begin by explaining why our proposed measure of robustness based on Dirichlet energy (6) is actually a measure of robustness.

Original: Unless otherwise stated, in this section the feature distribution will be any distribution P x on R d .
Revised: Unless otherwise stated, in this section the feature distribution will be any distribution P x on R d .

Original: Given smooth 1 f : R d → R , consider the d × d psd matrix J ( f ) and scalar S ( f ) ≥ 0 defined by
Revised: Given smooth 1 f : R d → R , consider the d × d psd matrix J ( f ) and scalar S ( f ) ≥ 0 defined by

Original: [Equation]
Revised: [Equation]

Original: Note that ε rob ( f ) =
Revised: Note that ε rob ( f ) =

Original: S ( f ) 2 .
Revised: S ( f ) 2 .

Original: The following lemma shows that S ( f ) measures the sensitivity of f to random local fluctuations in test data, on average.
Revised: The following lemma shows that S ( f ) measures the sensitivity of f to random local fluctuations in test data, on average.

Original: Lemma C.1 (Measure of local sensitivity) .
Revised: Lemma D.1 (Measure of local sensitivity) .

Original: We have
Revised: We have

Original: [Equation]
Revised: [Equation]

Original: 36) where ∆ f ( x ; δ ) :
Revised: where ∆ f ( x ; δ ) :

Original: = sup ∥ v ∥ 2 ≤ δ | f [Equation]
Revised: = sup ∥ v ∥ 2 ≤ δ | f ( x + v ) − f ( x ) | .

Original: This lemma is a direct corollary to Lemma C.3 proved later below.
Revised: This lemma is a direct corollary to Lemma D.3 proved later below.

Original: The next lemma shows that ∥ J ( f ) ∥ op measures the (non)robustness of f to universal adversarial perturbations, in the sense of Moosavi-Dezfooli et al.
Revised: The next lemma shows that ∥ J ( f ) ∥ op measures the (non)robustness of f to universal adversarial perturbations, in the sense of Moosavi-Dezfooli et al.

Original: Lemma C.2 (Measure of robustness to universal perturbations) .
Revised: Lemma D.2 (Measure of robustness to universal perturbations) .

Original: We have the identity
Revised: We have the identity

Original: [Equation]
Revised: [Equation]

Original: where ∆ [Equation]
Revised: where ∆ f ( δ ) 2 := sup ∥ v ∥≤ δ E P x ( f ( x + v ) − f ( x )) 2 .

Original: In particular, the leading eigenvector of J ( f ) corresponds to (first-order) universal adversarial perturbations of f , in the sense of Moosavi-Dezfooli et al.
Revised: In particular, the leading eigenvector of J ( f ) corresponds to (first-order) universal adversarial perturbations of f , in the sense of Moosavi-Dezfooli et al.

Original: (2017), which can be efficiently computed using the Power Method , for example.
Revised: (2017), which can be efficiently computed using the Power Method , for example.

Original: A rough sketch of the proof of the above lemma is as follows.
Revised: A rough sketch of the proof of the above lemma is as follows.

Original: To first-order, we have f ( x + v ) − f ( x ) ≈ v ⊤ ∇ f ( x ) .
Revised: To first-order, we have f ( x + v ) − f ( x ) ≈ v ⊤ ∇ f ( x ) .

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: The first lemma is proved via a similar argument.
Revised: The first lemma is proved via a similar argument.

Original: C. W HY NOT USE L IPSCHITZ CONSTANTS TO MEASURE ROBUSTNESS ?
Revised: D. W HY NOT USE L IPSCHITZ CONSTANTS TO MEASURE ROBUSTNESS ?

Original: Note for any that smooth function, S ( f ) is always a lower-bound for the Lipschitz constant ∥ f ∥ Lip of f .
Revised: Note for any that smooth function, S ( f ) is always a lower-bound for the Lipschitz constant ∥ f ∥ Lip of f .

Original: Recall that ∥ f ∥ Lip is defined by
Revised: Recall that ∥ f ∥ Lip is defined by

Original: [Equation]
Revised: [Equation]

Original: One special case where there is equality S ( f ) =
Revised: One special case where there is equality S ( f ) =

Original: ∥
Revised: ∥

Original: f
Revised: f

Original: ∥ Lip is when f is a linear function.
Revised: ∥ Lip is when f is a linear function.

Original: However, this is far from true in general:
Revised: However, this is far from true in general:

Original: ∥
Revised: ∥

Original: f ∥ Lip is a worst-case measure, while S ( f ) is an average-case measure for each q .
Revised: f ∥ Lip is a worst-case measure, while S ( f ) is an average-case measure for each q .

Original: If ∥ f ∥ Lip is small (i.e., of order O (1) ), then a small perturbation (i.e., of size O (1) ) can only result in mild change in the output of f (i.e., of order O (1) ).
Revised: If ∥ f ∥ Lip is small (i.e., of order O (1) ), then a small perturbation (i.e., of size O (1) ) can only result in mild change in the output of f (i.e., of order O (1) ).

Original: However, a large value of ∥
Revised: However, a large value of ∥

Original: f ∥ Lip is uninformative regarding adversarial examples (for example, one can think of a function which is smooth everywhere except on a set of measure zero).
Revised: f ∥ Lip is uninformative regarding adversarial examples (for example, one can think of a function which is smooth everywhere except on a set of measure zero).

Original: In contrast, a large value for S ( f ) indicates that, on average, it is possible for an adversarial to drastically change the output of f via a small modification of its input.
Revised: In contrast, a large value for S ( f ) indicates that, on average, it is possible for an adversarial to drastically change the output of f via a small modification of its input.

Original: An illustrative example.
Revised: An illustrative example.

Original: Consider a quadratic function f ( x ) :
Revised: Consider a quadratic function f ( x ) :

Original: = (1 / 2) x
Revised: = (1 / 2) x

Original: ⊤ Bx + c with isotropic feature distribution P x = N (0 , I d ) .
Revised: ⊤ Bx + c with isotropic feature distribution P x = N (0 , I d ) .

Original: Note that the teacher model f ⋆ defined in (1) is of this form.
Revised: Note that the teacher model f ⋆ defined in (1) is of this form.

Original: A direct computation reveals that ∇ f ( x ) =
Revised: A direct computation reveals that ∇ f ( x ) =

Original: Bx and so S ( f ) 2 :=
Revised: Bx and so S ( f ) 2 :=

Original: E P x ∥∇ f
Revised: E P x ∥∇ f

Original: ( x ) ∥ 2 = E P x ∥
Revised: ( x ) ∥ 2 = E P x ∥

Original: Bx ∥ 2 = ∥
Revised: Bx ∥ 2 = ∥

Original: B ∥ 2 F .
Revised: B ∥ 2 F .

Original: However, the Lipschitz constant of f restricted to the ball of radius √ d is 2 ,
Revised: However, the Lipschitz constant of f restricted to the ball of radius √ d is 2 ,

Original: [Equation]
Revised: [Equation]

Original: ∥ op , which can be up to √ d times larger than S ( f ) =
Revised: which can be up to √ d times larger than S ( f ) =

Original: ∥ B ∥ F .
Revised: ∥ B ∥ F .

Original: For example, take B to be an ill-conditioned, e.g., rank- 1 , matrix.
Revised: For example, take B to be an ill-conditioned, e.g., rank- 1 , matrix.

Original: C. P ROOFS FOR D IRICHLET ENERGY AS A MEASURE OF ADVERSARIAL VULNERABILITY
Revised: D. P ROOFS FOR D IRICHLET ENERGY AS A MEASURE OF ADVERSARIAL VULNERABILITY

Original: Let ∥ · ∥ be any norm on R d with dual norm ∥ · ∥ ⋆ .
Revised: Let ∥ · ∥ be any norm on R d with dual norm ∥ · ∥ ⋆ .

Original: Given a function f : R d → R , a tolerance parameter δ ≥ 0 (the attack budget ), and a scalar q ≥ 1 , define R δ ( f ) by
Revised: Given a function f : R d → R , a tolerance parameter δ ≥ 0 (the attack budget ), and a scalar q ≥ 1 , define R δ ( f ) by

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where ∆ f ( x ; δ ) :
Revised: where ∆ f ( x ; δ ) :

Original: = sup
Revised: = sup

Original: ∥ x ′ − x ∥≤ δ | f ( x ′ ) − f ( x ) | is the maximal variation of f in a neighborhood of size δ around x .
Revised: ∥ x ′ − x ∥≤ δ | f ( x ′ ) − f ( x ) | is the maximal variation of f in a neighborhood of size δ around x .

Original: For q = 2 , we simply write R δ ( f, g ) for R δ, 2 ( f, g ) .
Revised: For q = 2 , we simply write R δ ( f, g ) for R δ, 2 ( f, g ) .

Original: In particular, G δ ( f ) :=
Revised: In particular, G δ ( f ) :=

Original: E [ R δ ( f, f ⋆ )] is adversarial test error and G 0 ( f ) :
Revised: E [ R δ ( f, f ⋆ )] is adversarial test error and G 0 ( f ) :

Original: = E [ R 0 ( f, f ⋆ )] is the ordinary test error of f , where the expectations are w.r.t all sources of randomness in f and f ⋆ .
Revised: = E [ R 0 ( f, f ⋆ )] is the ordinary test error of f , where the expectations are w.r.t all sources of randomness in f and f ⋆ .

Original: Of course G δ ( f ) is an increasing function of δ .
Revised: Of course G δ ( f ) is an increasing function of δ .

Original: Define R q,δ ( f ) :
Revised: Define R q,δ ( f ) :

Original: =
Revised: =

Original: R q,δ ( f, f ) and R δ ( f ) :
Revised: R q,δ ( f, f ) and R δ ( f ) :

Original: = R 2 ,δ ( f, f ) , which measure the deviation of the outputs of f w.r.t to the outputs of f , under adversarial attack.
Revised: = R 2 ,δ ( f, f ) , which measure the deviation of the outputs of f w.r.t to the outputs of f , under adversarial attack.

Original: Note R q ( f ) ≡ 0 .
Revised: Note R q ( f ) ≡ 0 .

Original: Also note that in the case where ∥ · ∥ is the euclidean L 2 -norm: if f is a near perfect model (in the classical sense), meaning that its ordinary test error G 0 ( f ) is small, then R δ ( f ) is a good approximation for G δ ( f ) .
Revised: Also note that in the case where ∥ · ∥ is the euclidean L 2 -norm: if f is a near perfect model (in the classical sense), meaning that its ordinary test error G 0 ( f ) is small, then R δ ( f ) is a good approximation for G δ ( f ) .

Original: Finally, (at least for small values of α ), we can further approximate R α ( f ) (and therefore G δ ( f ) , for near perfect f ) by δ 2 times the Dirichlet energy S ( f ) 2 .
Revised: Finally, (at least for small values of α ), we can further approximate R α ( f ) (and therefore G δ ( f ) , for near perfect f ) by δ 2 times the Dirichlet energy S ( f ) 2 .

Original: Indeed,
Revised: Indeed,

Original: Lemma C.3.
Revised: Lemma D.3.

Original: Let q ∈
Revised: Let q ∈

Original: [1 , ∞ ) , and f : R d → R be a smooth function.
Revised: [1 , ∞ ) , and f : R d → R be a smooth function.

Original: Define S q ( f ) by
Revised: Define S q ( f ) by

Original: [Equation]
Revised: [Equation]

Original: Note that, in particular, if ∥ · ∥ is the euclidean L 2 -norm and q = 2 , then S q ( f ) 2 is the Dirichlet energy defined in (6) as our measure of robustness.
Revised: Note that, in particular, if ∥ · ∥ is the euclidean L 2 -norm and q = 2 , then S q ( f ) 2 is the Dirichlet energy defined in (6) as our measure of robustness.

Original: We have the following (A) General case.
Revised: We have the following (A) General case.

Original: S q ( f ) is the right derivative of the mapping δ
Revised: S q ( f ) is the right derivative of the mapping δ

Original: (cid:55)→ R δ ( f ) 1 /q at δ = 0 .
Revised: (cid:55)→ R δ ( f ) 1 /q at δ = 0 .

Original: More precisely, we have the following
Revised: More precisely, we have the following

Original: [Equation]
Revised: [Equation]

Original: or equivalently, R q,δ  f )
Revised: or equivalently, R q,δ ( f )

Original: =
Revised: =

Original: δ q · S  f ) q + Higher order terms in δ q 
Revised: δ q · S ( f ) q + Higher order terms in δ q .

Original: (B) Case of Dirichlet energy
Revised: (B) Case of Dirichlet energy

Original: In particular, if ∥ · ∥ is the euclidean L 2 -norm, and we take q = 2 ,R ( f ) = δ S q
Revised: In particular, if ∥ · ∥ is the euclidean L 2 -norm, and we take q = 2 ,

Original: f ) 2 + Higher order terms in

Original: δ 2 .

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Remark C.1.
Revised: Remark D.1.

Original: A heuristic argument was used in Simon-Gabriel et al.
Revised: A heuristic argument was used in Simon-Gabriel et al.

Original: (2019) to justify the use of average (dual-)norm of gradient (i.e the average local Lipschitz constant) E P x
Revised: (2019) to justify the use of average (dual-)norm of gradient (i.e the average local Lipschitz constant) E P x

Original: [ ∥∇ f ( x ) ∥ ⋆ ] (corresponding to q = 1 in the above) as a proxy for the adversarial generalization.
Revised: [ ∥∇ f ( x ) ∥ ⋆ ] (corresponding to q = 1 in the above) as a proxy for the adversarial generalization.

Original: The proof of Lemma C.3 follows directly Fubini’s Theorem and the following lemma.
Revised: The proof of Lemma D.3 follows directly Fubini’s Theorem and the following lemma.

Original: Lemma C.4.
Revised: Lemma D.4.

Original: If f is differentiable at x , then the function δ (cid:55)→ ∆ f ( x ; δ ) :=
Revised: If f is differentiable at x , then the function δ (cid:55)→ ∆ f ( x ; δ ) :=

Original: [Equation]
Revised: [Equation]

Original: ∥≤ δ | f

Original: [Equation]

Original: − f
Revised: − f

Original: [Equation]
Revised: [Equation]

Original:  ) | is right-differentiable at 0 with derivative given by ∆ ′
Revised: x ) | is right-differentiable at 0 with derivative given by ∆ ′

Original: f ( x ; 0) =
Revised: f ( x ; 0) =

Original: ∥∇ f ( x ) ∥ ⋆ .
Revised: ∥∇ f ( x ) ∥ ⋆ .

Original: Proof.
Revised: Proof.

Original: As f is differentiable, f ( x ′ )
Revised: As f is differentiable, f ( x ′ )

Original: =
Revised: =

Original: f ( x ) + ∇ f ( x ) ⊤ ( x ′ − x ) + o ( ∥ x ′ − x ∥ ) around x .
Revised: f ( x ) + ∇ f ( x ) ⊤ ( x ′ − x ) + o ( ∥ x ′ − x ∥ ) around x .

Original: Therefore for sufficiently small δ , if B ( x ; δ ) is ball of radius δ around x , then 
Revised: Therefore for sufficiently small δ , if B ( x ; δ ) is ball of radius δ around x , then ∆ f ( x ; δ ) =

Original: [Equation]
Revised: [Equation]

Revised: Note that sup x

Original: [Equation]
Revised: [Equation]

Original: Note that sup x ′ ∈ B ( x ; δ ) o ( ∥ x ′ − x ∥ ) δ → 0 .
Revised: δ → 0 .

Original: This proves limsup δ
Revised: This proves lim

Original: →

Original: [Equation]
Revised: [Equation]

Original: + (

Original: [Equation]

Original: ≤ ∥∇ f ( x ) ∥ ⋆ .

Original: Similarly, one computes
Revised: Similarly, one computes

Original: [Equation]
Revised: [Equation]

Original: Hence lim inf δ → 0 + (1 /δ )
Revised: Hence lim inf δ → 0 + (1 /δ )

Original: ∆ f ( x ; δ ) ≥ ∥∇ f ( x ) ∥ ⋆ , and we conclude that δ (cid:55)→ ∆ f
Revised: ∆ f ( x ; δ ) ≥ ∥∇ f ( x ) ∥ ⋆ , and we conclude that δ (cid:55)→ ∆ f

Original: ( x ; δ ) is differentiable at δ = 0 , with derivative ∆ ′
Revised: ( x ; δ ) is differentiable at δ = 0 , with derivative ∆ ′

Original: f ( x ; 0) =
Revised: f ( x ; 0) =

Original: ∥∇ f ( x ) ∥
Revised: ∥∇ f ( x ) ∥

Original: ⋆ as claimed.
Revised: ⋆ as claimed.

Original: Proof of Lemma C.3.
Revised: Proof of Lemma D.3.

Original: By basic properties of limits, one has
Revised: By basic properties of limits, one has

Original: [Equation]
Revised: [Equation]

Original: where the 3rd line is thanks to Fubini’s Theorem , and the 5th line is thanks to lemma C.4 (and the fact that ∆ f ( x ; 0) ≡ 0 ).
Revised: where the 3rd line is thanks to Fubini’s Theorem , and the 5th line is thanks to lemma D.4 (and the fact that ∆ f ( x ; 0) ≡ 0 ).

Original: Noting that R q, 0 ( f ) ≡ 0 then concludes the proof.
Revised: Noting that R q, 0 ( f ) ≡ 0 then concludes the proof.

Original: D N EURAL NETWORKS AT ( RANDOM ) INITIALIZATION
Revised: E N EURAL NETWORKS AT ( RANDOM ) INITIALIZATION

Original: We now consider networks at initialization, wherein the hidden weights matrix W
Revised: We now consider networks at initialization, wherein the hidden weights matrix W

Original: = ( w 1 , . . .
Revised: = ( w 1 , . . .

Original: , w m ) is a random m ×
Revised: , w m ) is a random m ×

Original: d matrix with iid rows from N (0 , Γ) as in the random features regime (11), but we freeze the output weight vector
Revised: d matrix with iid rows from N (0 , Γ) as in the random features regime (13), but we freeze the output weight vector

Original: z
Revised: z

Original: = z 0 ∈ R m at random initialization, with random iid entries from
Revised: = z 0 ∈ R m at random initialization, with random iid entries from

Original: N (0 , 1 /m ) , following standard initialization procedures.
Revised: N (0 , 1 /m ) , following standard initialization procedures.

Original: Let f init denote this random network, i.e.,
Revised: Let f init denote this random network, i.e.,

Original: [Equation]
Revised: [Equation]

Original: Theorem D.1.
Revised: Theorem E.1.

Original: Under the Conditions 4.1 and 4.2, we have the identity in the limit (3) ,
Revised: Under the Conditions 4.1 and 4.2, we have the identity in the limit (3) ,

Original: [Equation]
Revised: [Equation]

Original: o d, P (1) , where λ k is the k th Hermite coefficient of the activation function σ .
Revised: o d, P (1) , where λ k is the k th Hermite coefficient of the activation function σ .

Original: In particular, for the quadratic activation function σ ( t ) =
Revised: In particular, for the quadratic activation function σ ( t ) =

Original: t 2 − 1 , we have
Revised: t 2 − 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Analogously, the test error for the NN at initialization is given by the following result.
Revised: Analogously, the test error for the NN at initialization is given by the following result.

Original: Theorem D.2.
Revised: Theorem E.2.

Original: Under the Conditions 4.1 and 4.2, we have the following identity in the limit (3) ,
Revised: Under the Conditions 4.1 and 4.2, we have the following identity in the limit (3) ,

Original: [Equation]
Revised: [Equation]

Original:  In particular, for the quadratic activation σ ( t ) :
Revised: P (1) . In particular, for the quadratic activation σ ( t ) :

Original: = t 2 − 1 , we have the following identity (cid:101) ε [Equation] f 
Revised: = t 2 − 1 , we have the following identity (cid:101) ε test ( f init )

Original: [Equation] F [Equation] B
Revised: = 1 + 1 + ∥ Γ ∥

Original: [Equation]
Revised: [Equation]

Original: F + o d
Revised: ∥

Original: [Equation]
Revised: [Equation]

Original: Combining Thm.
Revised: Combining Thm.

Original: D.2 with formula (9), we deduce that training a randomly initialized neural network always improves its test error, as one would expect.
Revised: E.2 with formula (11), we deduce that training a randomly initialized neural network always improves its test error, as one would expect.

Original: On the other hand, combining Thm. 3.1 and Thm.
Revised: On the other hand, combining Thm. 3. and Thm.

Original: D.1, we deduce that fully training the networks (8) via SGD:
Revised: E.1, we deduce that fully training the networks (10) via SGD:

Original: (1) Degrades robustness if ∥ B ∥ 2 F ≳ ∥
Revised: (1) Degrades robustness if ∥ B ∥ 2 F ≳ ∥

Original: Γ ∥ 2 F + 1 .
Revised: Γ ∥ 2 F + 1 .

Original: This is because in this case, the parameters of the model align to the signal matrix B , which has much larger energy 
Revised: This is because in this case, the parameters of the model align to the signal matrix B , which has much larger energy than the parameters at initialization.

Original: than

Original: the parameters at initialization.

Original: Indeed, SGD tends to move the covariance structure of the hidden neurons from Γ to B .
Revised: Indeed, SGD tends to move the covariance structure of the hidden neurons from Γ to B .

Original: (2) Improves robustness if ∥ B ∥ 2 F ≲ ∥ Γ ∥ 2 F + 1 .
Revised: (2) Improves robustness if ∥ B ∥ 2 F ≲ ∥ Γ ∥ 2 F + 1 .

Original: E M ISCELLANEOUS E. L AZY TRAINING OF OUTPUT LAYER IN RF REGIME
Revised: F M ISCELLANEOUS F. L AZY TRAINING OF OUTPUT LAYER IN RF REGIME

Original: We now study the influence of the initialization on the random features regime.
Revised: We now study the influence of the initialization on the random features regime.

Original: Let W = ( w 1 , . . .
Revised: Let W = ( w 1 , . . .

Original: , w m ) ∈ R m × d with random rows drawn iid from N (0 , Γ) as in the RF model (11), and let the output layer be initialized at z
Revised: , w m ) ∈ R m × d with random rows drawn iid from N (0 , Γ) as in the RF model (13), and let the output layer be initialized at z

Original: = z 0 ∼ N (0 , (1 /m )
Revised: = z 0 ∼ N (0 , (1 /m )

Original: I d ) and updated via single-pass gradient-flow on the entire data distribution (infinite data).
Revised: I d ) and updated via single-pass gradient-flow on the entire data distribution (infinite data).

Original: In this so-called random features lazy (RFL) regime, we posit the following approximation neural network (2) 
Revised: In this so-called random features lazy (RFL) regime, we posit the following approximation neural network (2) [Equation]

Original: [Equation]
Revised: [Equation]

Original: (47) where z RFL ,λ :=
Revised: where z RFL ,λ : =

Original: z 

Revised: z

Original: + δ λ and δ λ ∈ R m solves the following ridge-regression problem 
Revised: + δ λ and δ λ ∈ R m solves the following ridge-regression problem [Equation]

Original: [Equation]
Revised: [Equation]

Original: The use of the ridge parameter here can be thought of as a proxy for early-stopping at iteration t ∝ 1 /λ Ali et al.
Revised: The use of the ridge parameter here can be thought of as a proxy for early-stopping at iteration t ∝ 1 /λ Ali et al.

Original: λ = 0 corresponds to training the output layer to optimality.
Revised: λ = 0 corresponds to training the output layer to optimality.

Original: Theorem E.1.
Revised: Theorem F.1.

Original: We have the following identities
Revised: We have the following identities

Original: [Equation]
Revised: [Equation]

Original: where U =
Revised: where U =

Original: U ( W ) and C = C ( W ) are the random matrices defined in (13) and (53) respectively.
Revised: U ( W ) and C = C ( W ) are the random matrices defined in (15) and (54) respectively.

Original: Because P 2 λ , U , and C are psd matrices, the residual terms tr( P 2 λ U ) /m and tr( P 2 λ C ) /m in the above formulae are nonnegative.
Revised: Because P 2 λ , U , and C are psd matrices, the residual terms tr( P 2 λ U ) /m and tr( P 2 λ C ) /m in the above formulae are nonnegative.

Original: We deduce that random initialization of the output weights hurts both test error and robustness, as long as the RFL regime is valid.
Revised: We deduce that random initialization of the output weights hurts both test error and robustness, as long as the RFL regime is valid.

Original: Infinitely regularized case λ → ∞ .
Revised: Infinitely regularized case λ → ∞ .

Original: Note that P λ converges in spectral norm a.s to the identity matrix I m in the limit λ → ∞ .
Revised: Note that P λ converges in spectral norm a.s to the identity matrix I m in the limit λ → ∞ .

Original: Thus, in this limit, z RF ,λ converges almost-surely to the all-zero m -dimensional vector
Revised: Thus, in this limit, z RF ,λ converges almost-surely to the all-zero m -dimensional vector

Original: and so, thanks to (90), the output weights z RFL ,λ of f RFL ,λ converge to the value at initialization z 0 .
Revised: and so, thanks to (91), the output weights z RFL ,λ of f RFL ,λ converge to the value at initialization z 0 .

Original: Therefore, f RFL ,λ and all its derivatives converge a.s point-wise its state f init at initialization (46).
Revised: Therefore, f RFL ,λ and all its derivatives converge a.s point-wise its state f init at initialization (47).

Original: We deduce that in the λ → ∞ limit, the neural network in the lazy regime is equivalent to an untrained model f init , in terms of test error and robustness.
Revised: We deduce that in the λ → ∞ limit, the neural network in the lazy regime is equivalent to an untrained model f init , in terms of test error and robustness.

Original: This does not come as much of a surprise, since λ → ∞ , corresponds to early-stopping at t = 0 , i.e., no optimization.
Revised: This does not come as much of a surprise, since λ → ∞ , corresponds to early-stopping at t = 0 , i.e., no optimization.

Original: Unregularized case λ
Revised: Unregularized case λ

Original: → 0 + .
Revised: → 0 + .

Original: By an analogous argument as above, P λ converges a.s. to the all-zero m × m matrix in the limit λ
Revised: By an analogous argument as above, P λ converges a.s. to the all-zero m × m matrix in the limit λ

Original: → 0 + , and so thanks to (90), we have the almost-sure convergence ∥ z RFL ,λ − z RF ,λ ∥ → 0 .
Revised: → 0 + , and so thanks to (91), we have the almost-sure convergence ∥ z RFL ,λ − z RF ,λ ∥ → 0 .

Original: We deduce that in this limit, the unregularized lazy training regime is exactly equivalent to the unregularized vanilla RF regime.
Revised: We deduce that in this limit, the unregularized lazy training regime is exactly equivalent to the unregularized vanilla RF regime.

Original: Thus, the random features lazy (RFL) regime corresponding to the approximation f RFL is an interpolation between the random features regime (corresponding to f RF ) and the untrained regime (corresponding to f init ).
Revised: Thus, the random features lazy (RFL) regime corresponding to the approximation f RFL is an interpolation between the random features regime (corresponding to f RF ) and the untrained regime (corresponding to f init ).

Original: Although this is not useful in our infinite data regime, we remark that a non-zero amount of regularization is often crucial for good statistical performance with finite samples.
Revised: Although this is not useful in our infinite data regime, we remark that a non-zero amount of regularization is often crucial for good statistical performance with finite samples.

Original: In this, case, P λ is non-zero, and we expect both the test error and robustness to become worse in this lazy RF approximation, compared to vanilla RF.
Revised: In this, case, P λ is non-zero, and we expect both the test error and robustness to become worse in this lazy RF approximation, compared to vanilla RF.

Original: E. E FFECT OF REGULARIZATION IN RF REGIME
Revised: F. E FFECT OF REGULARIZATION IN RF REGIME

Original: Suppose the estimation of the output weights of the RF model is regularized, i.e., for a fixed λ ≥ 0 , consider instead the model f RF ,λ ( x ) :
Revised: Suppose the estimation of the output weights of the RF model is regularized, i.e., for a fixed λ ≥ 0 , consider instead the model f RF ,λ ( x ) :

Original: =
Revised: =

Original: z ⊤ RF ,λ σ ( Wx ) , where z RF ,λ is chosen to solve the following ridge-regularized problem
Revised: z ⊤ RF ,λ σ ( Wx ) , where z RF ,λ is chosen to solve the following [Table]

Original: [Equation]
Revised: [Equation]

Original: A simple computation gives the explicit form
Revised: A simple computation gives the explicit form

Original: [Equation]
Revised: [Equation]

Original: where U λ :
Revised: where U λ :

Original: =
Revised: =

Original: U + λI m , U
Revised: U + λI m , U

Original: = U ( W ) is the random matrix defined in (13), and v ∈ R m is random vector defined in (14).
Revised: = U ( W ) is the random matrix defined in (15), and v ∈ R m is random vector defined in (16).

Original: An inspection of the proof of Theorem 4.1 (see Appendix F.4) reveals that the situation in the presence of ridge regularization is equivalent to the unregularized case in which we replace λ by λ
Revised: An inspection of the proof of Theorem 4.1 (see Appendix G.4) reveals that the situation in the presence of ridge regularization is equivalent to the unregularized case in which we replace λ by λ

Original: + λ in the definition of the matrix A 0 which appears in (18).
Revised: + λ in the definition of the matrix A 0 which appears in (20).

Original: This has the effect of decreasing ψ 1 and ψ 2 , and thanks to (21), decreasing the robustness of the random features model.
Revised: This has the effect of decreasing ψ 1 and ψ 2 , and thanks to (23), decreasing the robustness of the random features model.

Original: That is, (cid:101)
Revised: That is, (cid:101)

Original: ε rob ( f RF ,λ ) is a decreasing function of the amount of regularization of λ , and in fact, 
Revised: ε rob ( f RF ,λ ) is a decreasing function of the amount of regularization of λ , and in fact, lim

Original: [Equation]
Revised: [Equation]

Original: F T ECHNICAL PROOFS
Revised: G T ECHNICAL PROOFS

Original: Before proving the main results of the manuscript, we first state and prove some auxiliary results which will be instrumental.
Revised: Before proving the main results of the manuscript, we first state and prove some auxiliary results which will be instrumental.

Original: F. A USEFUL LEMMA
Revised: G. A USEFUL LEMMA

Original: Recall the definitions of the approximation error and robustness metrics from Section 2.3.
Revised: Recall the definitions of the approximation error and robustness metrics from Section 2.3.

Original: The following lemma was used to express the measure of (non)robustness ε rob ( f W,z,s ) 2 of a two-layer neural network f W,z,s as a quadratic form in the output weights, with coefficient matrix which depends on the distribution of the hidden weights.
Revised: The following lemma was used to express the measure of (non)robustness ε rob ( f W,z,s ) 2 of a two-layer neural network f W,z,s as a quadratic form in the output weights, with coefficient matrix which depends on the distribution of the hidden weights.

Original: Let us start by deriving an analytic formula for the robustness measure for the neural network general model (2).
Revised: Let us start by deriving an analytic formula for the robustness measure for the neural network general model (2).

Original: This result will be exploited in the sequel in the analysis of the different learning regimes we will consider.
Revised: This result will be exploited in the sequel in the analysis of the different learning regimes we will consider.

Original: Lemma F.1.
Revised: Lemma G.1.

Original: For the neural net f W,z,s defined in (2) , we have the analytic formula ε rob ( f W,z,s ) =
Revised: For the neural net f W,z,s defined in (2) , we have the analytic formula ε rob ( f W,z,s ) =

Original: z ⊤ Cz , where C = C ( W ) is the m × m psd matrix with entries given by (with x ∼ N (0 , I d ) )
Revised: z ⊤ Cz , where C = C ( W ) is the m × m psd matrix with entries given by (with x ∼ N (0 , I d ) )

Original: [Equation]
Revised: [Equation]

Original: In particular, for a quadratic activation σ ( t ) ≡ t 2 + s , we have c j,k = 4( w ⊤ j w k ) 2 ∀ j, k ∈ [ m ] .
Revised: In particular [Equation] for a quadratic activation σ ( t ) ≡ t 2 + s , we have c j,k = 4( w ⊤ j w k ) 2 ∀ j, k ∈ [ m ] .

Original: Proof.
Revised: Proof.

Original: One directly computes ∇ x f W,z,s ( x ) = (cid:80) mj
Revised: One directly computes ∇ x f W [Equation] z [Equation]

Revised: s ( x ) =

Revised: (cid:80) mj

Original: =

Revised: =

Original: z j σ ′ ( x ⊤ w j ) w j , and so the Laplacian of f W,z,s at x is given by
Revised: z j σ ′ ( x ⊤ w j ) w j [Equation] and so the Laplacian of f W,z,s at x is given by

Original: [Equation]
Revised: [Equation]

Original: Thus, S ( f W,z,s ) 2 evaluates to
Revised: Thus [Equation] S ( f W,z,s ) 2 evaluates to

Original: [Equation]
Revised: [Equation]

Original: where the m × m psd matrix C
Revised: where the m × m psd matrix C

Original: ( W ) is as defined in Lemma F.1.
Revised: ( W ) is as defined in Lemma G.1.

Original: In particular, for the activation function σ ( t ) :
Revised: In particular [Equation] for the activation function σ ( t ) :

Original: =
Revised: =

Original: t 2 + s , one computes
Revised: t 2 + s , one computes

Original: [Equation]
Revised: [Equation]

Original: where the last step is due to the fact that
Revised: where the last step is due to the fact that

Original: [Equation]
Revised: [Equation]

Revised: = tr(Cov( x ) w j w ⊤ k ) =

Original:  by a standard result on the mean of a quadratic form.
Revised: w ⊤ j w k , by a standard result on the mean of a quadratic form.

Original: Corollary F.1 (Robustness error of teacher model) .
Revised: Corollary G.1 (Robustness error of teacher model) .

Original: It holds that ε rob ( f ⋆ ) = 4 ∥ B ∥ 2 F .
Revised: It holds that ε rob ( f ⋆ ) = 4 ∥ B ∥ 2 F .

Original: Proof.
Revised: Proof.

Original: For the first part follows directly from Lemma F.1 with activation function σ ( t ) :
Revised: For the first part follows directly from Lemma G.1 with activation function σ ( t ) :

Original: =
Revised: =

Original: t 2 + b 0 /d and fixed output weight vector
Revised: t 2 + b 0 /d and fixed output weight vector

Original: z
Revised: z

Original: = 1 m := (1 , . . .
Revised: = 1 m := (1 , . . .

Original: F. H ERMITE COEFFICIENTS
Revised: G. H ERMITE COEFFICIENTS

Original: For any nonnegative integer k , let He k : R
Revised: For any nonnegative integer k , let He k : R

Original: → R be the (probabilist’s)
Revised: → R be the (probabilist’s)

Original: k th Hermite polynomial.
Revised: k th Hermite polynomial.

Original: For example, note that
Revised: For example, note that

Original: He 0 ( t ) := 0 , He 1 ( t ) :
Revised: He 0 ( t ) := 0 , He 1 ( t ) :

Original: = t , He 2 ( t ) ≡ t 2 − 1 , He 3 ( t ) :
Revised: = t , He 2 ( t ) ≡ t 2 − 1 , He 3 ( t ) :

Original: = t 3 − 3 t , etc.
Revised: = t 3 − 3 t , etc.

Original: The sequence (He k ) k forms an orthonormal basis for the Hilbert space L 2 =
Revised: The sequence (He k ) k forms an orthonormal basis for the Hilbert space L 2 =

Original: L 2 ( N (0 , 1)) for functions R → R which are square-integrable w.r.t the standard normal distribution N (0 , 1) .
Revised: L 2 ( N (0 , 1)) for functions R → R which are square-integrable w.r.t the standard normal distribution N (0 , 1) .

Original: Under suitable integrability conditions (refer to Section 4.1), the coefficients of the activation function σ in this basis are called its Hermite coefficients , denoted λ k , and are given by
Revised: Under suitable integrability conditions (refer to Section 4.1), the coefficients of the activation function σ in this basis are called its Hermite coefficients , denoted λ k , and are given by

Original: [Equation]
Revised: [Equation]

Original: Finally, ∥ σ ∥ 2 L 2 ( N (0 , 1))
Revised: Finally, ∥ σ ∥ 2 L 2 ( N (0 , 1))

Original: =
Revised: =

Original: E G ∼ N (0 , 1) [ σ ( G ) 2 ] defines the squared L 2 -norm of σ w.r.t the standard
Revised: E G ∼ N (0 , 1) [ σ ( G ) 2 ] defines the squared L 2 -norm of σ w.r.t the standard

Original: Gaussian distribution N (0 , 1) .
Revised: Gaussian distribution N (0 , 1) .

Original: Note that by construction, one has ∥ σ ∥ 2 L 2 ( N (0 , 1))
Revised: Note that by construction, one has ∥ σ ∥ 2 L 2 ( N (0 , 1))

Original: = (cid:80) ∞ k =

Revised: = (cid:80) ∞ k =

Original: λ 2 k ( σ ) .
Revised: λ 2 k ( σ ) .

Original: F. A PPROXIMATION OF RANDOM MATRICES
Revised: G. A PPROXIMATION OF RANDOM MATRICES

Original: This section establishes some technical results for ”linearizing” a number of complicated random matrices which occur in our analysis.
Revised: This section establishes some technical results for ”linearizing” a number of complicated random matrices which occur in our analysis.

Original: We will make heavy use of random matrix theory (RMT) techniques developed in Silverstein & Choi (1995); El Karoui (2010); Ledoit & P´ech´e (2011);
Revised: We will make heavy use of random matrix theory (RMT) techniques developed in Silverstein & Choi (1995); El Karoui (2010); Ledoit & P´ech´e (2011);

Original: Dobriban & Wager (2018)
Revised: Dobriban & Wager (2018)

Original: We begin by recalling the following definition for future reference.
Revised: We begin by recalling the following definition for future reference.

Original: Definition 4.1.
Revised: Definition 4.1.

Original: With z ∼ N (0 , 1) , define the following scalars
Revised: With z ∼ N (0 , 1) , define the following scalars

Original: [Equation]
Revised: [Equation]

Original: Let U be the random m × m psd matrix defined in (13) and let v ∈ R m be the random vector defined in (14).
Revised: Let U be the random m × m psd matrix defined in (15) and let v ∈ R m be the random vector defined in (16).

Original: Recall that λ k = λ k
Revised: Recall that λ k = λ k

Original: ( σ ) is the k Hermite coefficient of the activation function σ .
Revised: ( σ ) is the k Hermite coefficient of the activation function σ .

Original: Also recall the definition of the scalars λ , κ , τ , λ ′ , and κ ′ from (16).
Revised: Also recall the definition of the scalars λ , κ , τ , λ ′ , and κ ′ from (18).

Original: The following result was established in
Revised: The following result was established in

Original: Ghorbani et al.
Revised: Ghorbani et al.

Original: Proposition F.1 (Lemma 2 of Ghorbani et al.
Revised: Proposition G.1 (Lemma 2 of Ghorbani et al.

Original: If λ 0 = 0 and Conditions 4.1, 4.2 are in place, then in the limit (3) , it holds that
Revised: If λ 0 = 0 and Conditions 4.1, 4.2 are in place, then in the limit (3) , it holds that

Original: [Equation]
Revised: [Equation]

Original: where the random m × m psd matrix U 0 is defined by
Revised: where the random m × m psd matrix U 0 is defined by

Original: [Equation]
Revised: [Equation]

Original: and µ = ( µ 1 , . . . , µ m ) ∈ R m with µ i :=
Revised: and µ = ( µ 1 , . . . , µ m ) ∈ R m with µ i :=

Original: λ 2 · ( ∥ w i ∥ 2 − 1) / 2 .
Revised: λ 2 · ( ∥ w i ∥ 2 − 1) / 2 .

Original: A careful inspection of the proof of the estimate (56) reveals that we can remove the condition λ 0 = 0 , at the expense of incurring rank- 1 perturbations in the matrix U 0 .
Revised: A careful inspection of the proof of the estimate (57) reveals that we can remove the condition λ 0 = 0 , at the expense of incurring rank- 1 perturbations in the matrix U 0 .

Original: Indeed, let us rewrite σ = σ + λ 0 , and with λ 0 ( G ) =
Revised: Indeed, let us rewrite σ = σ + λ 0 , and with λ 0 ( G ) =

Original: E G [ σ ( G )]
Revised: E G [ σ ( G )]

Original: = 0 with G ∼ N (0 , 1) independent of the w i ’
Revised: = 0 with G ∼ N (0 , 1) independent of the w i ’

Original: s. Let
Revised: s. Let

Original: T 0 be the m × m matrix with entries ( T 0 ) ij :=
Revised: T 0 be the m × m matrix with entries ( T 0 ) ij :=

Original: λ 0 ( σ i ) λ 0 ( σ j ) , where σ i is the function defined by σ i ( z ) :
Revised: λ 0 ( σ i ) λ 0 ( σ j ) , where σ i is the function defined by σ i ( z ) :

Original: = σ ( ∥ w i ∥ z ) =
Revised: = σ ( ∥ w i ∥ z ) =

Original: σ i ( z ) + λ 0 , with σ ( ∥ w i ∥ z ) :
Revised: σ i ( z ) + λ 0 , with σ ( ∥ w i ∥ z ) :

Original: = σ ( ∥ w i ∥ z ) .
Revised: = σ ( ∥ w i ∥ z ) .

Original: Thus, we have the decomposition
Revised: Thus, we have the decomposition

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where u = ( λ 0 ( σ i ))
Revised: where u = ( λ 0 ( σ i ))

Original: i ∈ [ m ] .
Revised: i ∈ [ m ] .

Original: Let T 0 be the m × m psd matrix with entries ( T 0 ) ij :=
Revised: Let T 0 be the m × m psd matrix with entries ( T 0 ) ij :=

Original: λ 0 ( σ i ) λ 0 ( σ j ) .
Revised: λ 0 ( σ i ) λ 0 ( σ j ) .

Original: Using the arguments from Ghorbani et al.
Revised: Using the arguments from Ghorbani et al.

Original: (since λ 0 ( σ ) = 0 ), one has
Revised: (since λ 0 ( σ ) = 0 ), one has

Original: [Equation]
Revised: [Equation]

Original: Furthermore, observe that one can write u 1 ⊤ m =
Revised: Furthermore, observe that one can write u 1 ⊤ m =

Original: Rµ 1 ⊤ m , where R is the m × m diagonal matrix with
Revised: Rµ 1 ⊤ m , where R is the m × m diagonal matrix with

Original: R ii := λ 0 ( σ i )
Revised: R ii := λ 0 ( σ i )

Original: /µ i .
Revised: /µ i .

Original: Now, for large d and any i ∈ [ m ] , one computes
Revised: Now, for large d and any i ∈ [ m ] , one computes

Original: [Equation]
Revised: [Equation]

Original: We deduce that ∥ R − I m ∥ op
Revised: We deduce that ∥ R − I m ∥ op

Original: = o d, P (1) , and so ∥ u 1 ⊤ m − µ 1 ⊤ m ∥ op
Revised: = o d, P (1) , and so ∥ u 1 ⊤ m − µ 1 ⊤ m ∥ op

Original: =
Revised: =

Original: o d, P (1) .
Revised: o d, P (1) .

Original: This proves the following extension of the above lemma which will be crucial in the sequel.
Revised: This proves the following extension of the above lemma which will be crucial in the sequel.

Original: Lemma F.2 (Linearization of U without the Condition λ 0 ( σ )̸ = 0 ) .
Revised: Lemma G.2 (Linearization of U without the Condition λ 0 ( σ ) = 0 ) .

Original: Suppose Conditions 4.1 and 4.
Revised: Suppose Conditions 4.1 and 4.

Original: are in place.
Revised: are in place.

Original: In the limit (3) , it holds that
Revised: In the limit (3) , it holds that

Original: [Equation]
Revised: [Equation]

Original: where (cid:101)
Revised: where (cid:101)

Original: U 0 is the m × m random psd matrix given by
Revised: U 0 is the m × m random psd matrix given by

Original: [Equation]
Revised: [Equation]

Original: with (cid:101) µ := µ + λ 0 1 m and (cid:101) λ := λ − λ 

Revised: with (cid:101) µ := µ + λ 0 1 m and (cid:101) λ := λ − λ

Original: =
Revised: =

Original: E G ∼ N (0 , 1) [ σ ( G ) 2 ] − λ 20 − λ 21 .
Revised: E G ∼ N (0 , 1) [ σ ( G ) 2 ] − λ 20 − λ 21 .

Original: Let C
Revised: Let C

Original: =
Revised: =

Original: C
Revised: C

Original: ( W ) be the random m × m psd matrix with entries given by
Revised: ( W ) be the random m × m psd matrix with entries given by

Original: [Equation]
Revised: [Equation]

Original: Thanks to Lemma F.1, we know that ε rob ( f RF ) =
Revised: Thanks to Lemma G.1, we know that ε rob [Equation] f RF ) =

Original: z ⊤ RF Cz RF = v ⊤ U − 1 CU − 1 v , a random quadratic form in v .
Revised: z ⊤ RF Cz RF = v ⊤ U − 1 CU − 1 v , a random quadratic form in v .

Original: We start by linearizing the nonlinear random coefficient matrix C .
Revised: We start by linearizing the nonlinear random coefficient matrix C .

Original: Lemma F.3 (Linearization of C ) .
Revised: Lemma G.3 (Linearization of C ) .

Original: Suppose Conditions 4.1 and 4.2 are in place.
Revised: Suppose Conditions 4.1 and 4.2 are in place.

Original: Then, in the limit (3) , we have the following approximation
Revised: Then, in the limit (3) , we have the following approximation

Original: [Equation]
Revised: [Equation]

Original: where C 0 is the m × m random psd matrix given by
Revised: where C 0 is the m × m random psd matrix given by

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: ) with κ ′ :
Revised: 67) with κ ′ :

Original: = d · λ 23 ∥ Γ ∥ 2 F / 2 ≥ 0 , and λ ′ :
Revised: = d · λ 23 ∥ Γ ∥ 2 F / 2 ≥ 0 , and λ ′ :

Original: =
Revised: =

Original: ∥ σ ′ ∥
Revised: ∥ σ ′ ∥

Original: 2 L 2 ( N (0 , 1)) − λ 21 .
Revised: 2 L 2 ( N (0 , 1)) − λ 21 .

Original: Proof.
Revised: Proof.

Original: Note that C
Revised: Note that C

Original: = ( WW ⊤ ) ⊙ U ′ , where U ′ is the m × m random psd matrix with entries given by U ′ ij :=
Revised: = ( WW ⊤ ) ⊙ U ′ , where U ′ is the m × m random psd matrix with entries given by U ′ ij :=

Original: E x ∼ N (0 ,I m ) [ σ ′ ( x ⊤ w i ) σ [Equation] ( x ⊤ w j [Equation]
Revised: E x ∼ N (0 ,I m ) [ σ ′ ( x ⊤ w i ) σ ′ ( x ⊤ w j )] .

Original: – Step 1: Linearization.
Revised: – Step 1: Linearization.

Original: Invoking the previous lemma with σ ′ in place of σ , we know that
Revised: Invoking the previous lemma with σ ′ in place of σ , we know that

Original: [Equation]
Revised: [Equation]

Original: where U ′ 0 is the m × m random matrix given by 
Revised: where U ′ 0 is the m × m random matrix given by U

Original: [Equation]
Revised: [Equation]

Original:  and we have used the fact that
Revised: 69) and we have used the fact that

Original: [Equation]
Revised: [Equation]

Original: Now, since ∥ WW ⊤ ∥ op = O d, P (1) by standard RMT, we deduce that from (67) that,
Revised: Now, since ∥ WW ⊤ ∥ op = O d, P (1) by standard RMT, we deduce that from (68) that,

Original: [Equation]
Revised: [Equation]

Original: – Step 2: Simplification.
Revised: – Step 2: Simplification.

Original: Let E :=
Revised: Let E = diag

Revised: ∥ w i ∥

Revised: i ∈ [

Revised: m

Original: diag(( ∥ w i ∥ 2 ) i ∈ [ m ] ) and F := ( WW ⊤) ⊙ ( WW ⊤ ) .
Revised: and F := ( WW ⊤ ⊙ ( WW ⊤ ) .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Further, because max i ∈ [ n ] |∥ w i ∥ 2 − 1 | =
Revised: Further, because max i ∈ [ n ] |∥ w i ∥ 2 − 1 | =

Original: o d, P (1) by basic concentration, we have
Revised: o d, P (1) by basic concentration, we have

Original: [Equation]
Revised: [Equation]

Original: Also, thanks to (El Karoui, 2010, Theorem 2.3), we may linearize F like so
Revised: Also, thanks to (El Karoui, 2010, Theorem 2.3), we may linearize F like so

Original: [Equation]
Revised: [Equation]

Original: Combining with (70) gives (recalling that κ := d · λ 2 ( σ ) 2 ∥ Γ ∥ 2 F / 2 )
Revised: Combining with (71) gives (recalling that κ := d · λ 2 ( σ ) 2 ∥ Γ ∥ 2 F / 2 )

Original: [Equation]
Revised: [Equation]

Original: where ∥ ∆ ∥ op =
Revised: where ∥ ∆ ∥ op =

Original: o d, P (1) .
Revised: o d, P (1) .

Original: Let us rewrite U 

Revised: Let us rewrite U

Original: =
Revised: =

Original: A 1 + µµ ⊤ and C 

Revised: A 1 + µµ ⊤ and C

Original: = D 

Revised: = D

Original: + (2 κ/d )1 m 1 ⊤ m , where
Revised: + (2 κ/d )1 m 1 ⊤ m , where

Original: [Equation]
Revised: [Equation]

Original: We will need the following lemmas.
Revised: We will need the following lemmas.

Original: Lemma F.4.
Revised: Lemma G.4.

Original: We have the following approximation
Revised: We have the following approximation

Original: [Equation]
Revised: [Equation]

Original: where u :=
Revised: where u :=

Original: U − 1 0 h , with h := ( τ/ √ d )1 m = λ 2 · tr( B Γ)1 m and U 0 is defined as in Proposition F. and C 0 is as defined in Lemma F.3.
Revised: U − 1 0 h , with h := ( τ/ √ d )1 m = λ 2 · tr( B Γ)1 m and U 0 is defined as in Proposition G. and C 0 is as defined in Lemma G.3.

Original: Proof.
Revised: Proof.

Original: Thanks to Proposition F.1, the fitted output weights vector z RF ∈ R m concentrates around u :=
Revised: Thanks to Proposition G.1, the fitted output weights vector z RF ∈ R m concentrates around u :=

Original: U − 1 0 h .
Revised: U − 1 0 h .

Original: On the other hand, we know from Lemma F.1 that ε rob ( f RF ) =
Revised: On the other hand, we know from Lemma G.1 that ε rob ( f RF ) =

Original: z ⊤ RF Cz RF .
Revised: z ⊤ RF Cz RF .

Original: The result then follows from Lemma F.3.
Revised: The result then follows from Lemma G.3.

Original: Lemma F.5.
Revised: Lemma G.5.

Original: Under Condition 4.3, the following holds in the limit (3)
Revised: Under Condition 4.3, the following holds in the limit (3)

Original: [Equation]
Revised: [Equation]

Original: where ψ 1 > 0 is as defined in (18) .
Revised: where ψ 1 > 0 is as defined in (20) .

Original: Proof.
Revised: Proof.

Original: Formula (76) was established in the proof of (Ghorbani et al., 2019, Theorem 1), whilst (77) was established in the proof of Lemma 5 of the same paper.
Revised: Formula (77) was established in the proof of (Ghorbani et al., 2019, Theorem 1), whilst (78) was established in the proof of Lemma 5 of the same paper.

Original: As for (78), we note that
Revised: As for (79), we note that

Original: [Equation]
Revised: [Equation]

Original: since λ = Ω d (1) under Condition 4.3.
Revised: since λ = Ω d (1) under Condition 4.3.

Original: Similarly, one computes ∥ D 0 ∥
Revised: Similarly, one computes ∥ D 0 ∥

Original: op =
Revised: op =

Original: O d ( WW ⊤ ) =
Revised: O d ( WW ⊤ ) =

Original: O d, P (1) , by standard RMT arguments Vershynin (2012).
Revised: O d, P (1) , by standard RMT arguments Vershynin (2012).

Original: We will need one final lemma.
Revised: We will need one final lemma.

Original: Lemma F.6.
Revised: Lemma G.6.

Original: Let A 1 , A 0 , and D 0 be the random matrices defined in (74) .
Revised: Let A 1 , A 0 , and D 0 be the random matrices defined in (75) .

Original: Then, it holds that
Revised: Then, it holds that

Original: [Equation]
Revised: [Equation]

Original: where ψ 1 and ψ 2 as defined in (18) .
Revised: where ψ 1 and ψ 2 as defined in (20) .

Original: Proof.
Revised: Proof.

Original: By Sherman-Morrison formula, we have
Revised: By Sherman-Morrison formula, we have

Original: [Equation]
Revised: [Equation]

Original: and so 
Revised: and so 1 ⊤ m A − 1 1 D [Equation] A [Equation]

Revised: 1 m

Revised: [Equation]

Revised: 2 = ac 2 , where

Original: [Equation]
Revised: [Equation]

Original: Now, one has 1 ⊤ m A − 1 0 1 m /d =
Revised: Now, one has 1 ⊤ m A − 1 0 1 m /d =

Original: tr( A − 1 0 ) /d
Revised: tr( A − 1 0 ) /d

Original: + o d, P (1) , thanks to Lemmas 5 and 6 of Ghorbani et al.
Revised: + o d, P (1) , thanks to Lemmas 5 and 6 of Ghorbani et al.

Original: By an analogous argument, one can show that 1 ⊤ m A − 1 0 D 

Revised: By an analogous argument, one can show that 1 ⊤ m A − 1 0 D

Original: A − 1 0 1 m /d =
Revised: A − 1 0 1 m /d =

Original: tr( A − 2 0 D 0 )
Revised: tr( A − 2 0 D 0 )

Original: /d
Revised: /d

Original: + o d, P (1) .
Revised: + o d, P (1) .

Original: Finally, the fact that tr( A 0 ) − 1 /d and tr( A − 2 0 D 0 ) /d converge to deterministic values ψ 1 and ψ 2 respectively, can be established via standard RMT arguments Silverstein & Choi (1995);
Revised: Finally, the fact that tr( A 0 ) − 1 /d and tr( A − 2 0 D 0 ) /d converge to deterministic values ψ 1 and ψ 2 respectively, can be established via standard RMT arguments Silverstein & Choi (1995);

Original: Ledoit &
Revised: Ledoit &

Original: P ´ ech ´ e (2011).
Revised: P ´ ech ´ e (2011).

Original: F. P ROOF OF T HEOREM 4.1: A NALYTIC FORMULA FOR ROBUSTNESS OF RF MODEL
Revised: G. P ROOF OF T HEOREM 4.1: A NALYTIC FORMULA FOR ROBUSTNESS OF RF MODEL

Original: We are now ready to prove Theorem 4.1, restated here for convenience.
Revised: We are now ready to prove Theorem 4.1, restated here for convenience.

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: Consider the random features model f RF (11) , with covariance matrix Γ satisfying
Revised: Consider the random features model f RF (13) , with covariance matrix Γ satisfying

Original: Condition 4.1 and activation function σ satisfying Conditions 4.2 and 4.3.
Revised: Condition 4.1 and activation function σ satisfying Conditions 4.2 and 4.3.

Original: (A)
Revised: (A)

Original: In the limit (3) , we have the following approximation
Revised: In the limit (3) , we have the following approximation

Original: [Equation]
Revised: [Equation]

Original: (B) Moreover, if lim d →∞ α = α ∞ , then lim
Revised: (B) Moreover, if lim d →∞ α = α ∞ , then lim

Original: ρ →∞ lim m,d →∞m
Revised: ρ →∞ lim m,d →∞

Original: [Equation]
Revised: [Equation]

Original: ρ (cid:101)

Original: [Equation]

Original: rob ( f RF ) =

Original: α
Revised: α

Original: [Equation]
Revised: [Equation]

Original:  w.p 1 .
Revised: ∞ w.p 1 .

Original: In particular, for the optimal choice of Γ in terms of test error, namely Γ ∝ B , one has lim
Revised: In particular, for the optimal choice of Γ in terms of test error, namely Γ ∝ B , one has lim

Original: ρ →∞ lim m,d →∞
Revised: ρ →∞ lim m,d →∞

Original: [Equation]
Revised: [Equation]

Original: ρ

Original: [Equation]

Original: ( f RF

Original: [Equation]

Original: =

Original: [Equation]

Original: w.p 1 .

Original: Proof.
Revised: Proof.

Original: From Lemmas F.1 and F.3, we know that
Revised: From Lemmas G.1 and G.3, we know that

Original: [Equation]
Revised: [Equation]

Original: where u :=
Revised: where u :=

Original: U − 1 0 h , with h := ( τ/ √ d )1 m = λ 2 · tr( B Γ)1 m and U 0 defined as in Lemma F.2 and C ,
Revised: U − 1 0 h , with h := ( τ/ √ d )1 m = λ 2 · tr( B Γ)1 m and U 0 defined as in Lemma G.2 and C ,

Original: C 0 are as defined in Lemma F.3.
Revised: C 0 are as defined in Lemma G.3.

Original: Let A 1 , A 0 , and D 0 be the random matrices defined in (74).
Revised: Let A 1 , A 0 , and D 0 be the random matrices defined in (75).

Original: Since,
Revised: Since,

Original: C[-
-]
Revised: C[-
-]

Original: [-= D 
-]
Revised: [-= D 
-][Equation]

Original: + (2 κ/d )1 m 1 ⊤ m , one computes
Revised: ⊤ m , one computes

Original: [Equation]
Revised: [Equation]

Original: where the last step is thanks to Lemma F.5.
Revised: where the last step is thanks to Lemma G.5.

Original: It remains to estimate the first term in the above display
Revised: It remains to estimate the first term in the above display.

Original: [Equation]

Original: Using the Sherman-Morrison formula [Equation] we have
Revised: Using the Sherman-Morrison formula, we have

Original: [Equation]
Revised: [Equation]

Original: We deduce that
Revised: We deduce that

Original: [Equation]
Revised: [Equation]

Original: where a 11 , a 12 , a 21 , and a 22 are defined by
Revised: where a 11 , a 12 , a [Equation] and a 22 are defined by

Original: [Equation]
Revised: [Equation]

Original: Now
Revised: Now

Original: [Equation]
Revised: [Equation]

Original: one easily computes
Revised: one easily computes

Original: [Equation]
Revised: [Equation]

Original: , P (1) , where we have used Lemma F.5 in the last two steps.
Revised: where we have used Lemma G.5 in the last two steps [Equation]

Original: Similarly, we have,
Revised: Similarly [Equation] we have,

Original: [Equation]
Revised: [Equation]

Original: again thanks to Lemma F.5.
Revised: again thanks to Lemma G

Revised: [Equation]

Original: We conclude from (84) that
Revised: We conclude from (85) that

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: we know from Lemma F{+G.
+}
Revised: we know from Lemma {+G.
+}

Original: [Equation]

Original: 6 that
Revised: that

Original: [Equation]
Revised: [Equation]

Original: part (A) of the theorem them follows upon dividing (84) by ε rob ( f ⋆ )
Revised: part (A) of the theorem them follows upon dividing (85) by ε rob ( f ⋆ )

Original: = 4 ∥ B ∥ 2 F .
Revised: = 4 ∥ B ∥ 2 F .

Original: For part (B), one notes that ψ 1 > 0 and so
Revised: For part (B), one notes that ψ 1 > 0 and so

Original: [Equation]
Revised: [Equation]

Original: o
Revised: which completes the proof

Original: [Equation]
Revised: [Equation]

Original: , which completes the proof.

Original: G P ROOFS OF MAIN RESULTS G. P ROOF OF T HEOREM D.1: R OBUSTNESS ERROR OF NEURAL NETWORKS AT INITIALIZATION
Revised: H P ROOFS OF MAIN RESULTS H. P ROOF OF T HEOREM E.1: R OBUSTNESS ERROR OF NEURAL NETWORKS AT INITIALIZATION

Original: We restate the result here for convenience.
Revised: We restate the result here for convenience.

Original: Let f init be the function computed by the neural network at initialization, as defined in (46).
Revised: Let f init be the function computed by the neural network at initialization, as defined in (47).

Original: Theorem D.1.
Revised: Theorem E.1.

Original: Under the Conditions 4.1 and 4.2, we have the identity in the limit (3) , (cid:101)
Revised: Under the Conditions 4.1 and 4.

Original: [Equation]
Revised: [Equation]

Original: =
Revised: we have the identity in the limit (3) , (cid:101) ε rob

Original: [Equation]
Revised: [Equation]

Revised: init

Original: [Equation]
Revised: [Equation]

Original: + o d, P (1) , where λ k is the k th Hermite coefficient of the activation function σ .
Revised: (1) , where λ k is the k th Hermite coefficient of the activation function σ .

Original: In particular, for the quadratic activation function σ ( t ) =
Revised: In particular, for the quadratic activation function σ ( t ) =

Original: t 2 − 1 , we have (cid:101) 
Revised: t 2 − 1 , we have (cid:101) ε rob ( f

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Thanks to Lemma F.1, we know that ε rob ( f init ) =
Revised: Thanks to Lemma G.1, we know that ε rob ( f init ) =

Original: z ⊤ Cz , where C is the random m × m psd matrix defined in (64).
Revised: z ⊤ Cz , where C is the random m × m psd matrix defined in (65).

Original: By standard RMT, z ⊤ Cz = tr( C ) /m
Revised: By standard RMT, z ⊤ Cz = tr( C ) /m

Original: + o d, P (1) .
Revised: + o d, P (1) .

Original: Now, let C 0 be the random matrix introduced in Lemma F.3.
Revised: Now, let C 0 be the random matrix introduced in Lemma G.3.

Original: Since ∥ C − C 

Revised: Since ∥ C − C

Original: ∥ op =
Revised: ∥ op =

Original: o d, P (1) (thanks to the aforementioned lemma), one has tr( C ) /m
Revised: o d, P (1) (thanks to the aforementioned lemma), one has tr( C ) /m

Original: =
Revised: =

Original: tr( C 0 ) /m
Revised: tr( C 0 ) /m

Original: + o d, P (1) .
Revised: + o d, P (1) .

Original: Let D 0 :=
Revised: Let D 0 :=

Original: λ ′
Revised: λ ′

Original: I m + ( κ ′ /d
Revised: I m + ( κ ′ /d

Original: + λ 21 ) WW ⊤ be the matrix defined in (74) so that C 

Revised: + λ 21 ) WW ⊤ be the matrix defined in (75) so that C

Original: = D 

Revised: = D

Original: + (2 κ/d )1 m 1 ⊤ m .
Revised: + (2 κ/d )1 m 1 ⊤ m .

Original: We deduce that in the limit (3),ε rob ( f init ) = tr( D 0 ) /m + 2 κ/d + o d, P (1) = ( κ ′ /d
Revised: We deduce that in the limit (3),

Original: + λ 21 )tr( WW

Original: [Equation]
Revised: [Equation]

Original: where the third line is because tr( WW ⊤ ) /m
Revised: where the third line is because tr( WW ⊤ ) /m

Original: = (1 /m ) (cid:80) mj
Revised: = (1 /m ) (cid:80) mj

Original: =1 ∥
Revised: =1 ∥

Original: w j ∥ 2 which converges in probability to tr(Γ)
Revised: w j ∥ 2 which converges in probability to tr(Γ)

Original: = 1 , by the weak law of large numbers.
Revised: = 1 , by the weak law of large numbers.

Original: Dividing by both sides of the above display by ε rob ( f ⋆ ) = 4 ∥ B ∥ 2 F then gives the result.
Revised: Dividing by both sides of the above display by ε rob ( f ⋆ ) = 4 ∥ B ∥ 2 F then gives the result.

Original: In particular, in the case of quadratic activation σ ( t ) :
Revised: In particular, in the case of quadratic activation σ ( t ) :

Original: = t 2 − 1 , we have λ 2 = 2 , ∥ σ ′ ∥
Revised: = t 2 − 1 , we have λ 2 = 2 , ∥ σ ′ ∥

Original: 2 L 2 ( N (0 , 1))
Revised: 2 L 2 ( N (0 , 1))

Original: = 4 , λ 3 = 0 , and so we deduce that (cid:101) ε rob ( f init ) =
Revised: = 4 , λ 3 = 0 , and so we deduce that (cid:101) ε rob ( f init ) =

Original: (4 + 4 ∥ Γ ∥ 2 F ) / (4 ∥ B ∥ 2 F ) =
Revised: (4 + 4 ∥ Γ ∥ 2 F ) / (4 ∥ B ∥ 2 F ) =

Original: (1 + ∥ Γ ∥ 2 F ) /
Revised: (1 + ∥ Γ ∥ 2 F ) /

Original: ∥ B ∥ 2 F .
Revised: ∥ B ∥ 2 F .

Original: G. P ROOF OF T HEOREM D.2: TEST ERROR OF NEURAL NETWORK AT INITIALIZATION
Revised: H.

Revised: P ROOF OF T HEOREM E.2: TEST ERROR OF NEURAL NETWORK AT INITIALIZATION

Original: Theorem D.2.
Revised: Theorem E.2.

Original: Under the Conditions 4.1 and 4.2, we have the following identity in the limit (3) , (cid:101)
Revised: Under the Conditions 4.1 and 4.2, we have the following identity in the limit (3) , (cid:101)

Original: [Equation]
Revised: [Equation]

Original:  =
Revised: test [Equation] f init ) =

Original: [Equation]
Revised: [Equation]

Original: +
Revised: o

Original: [Equation]
Revised: [Equation]

Original:  In particular, for the quadratic activation σ ( t ) :
Revised: , P (1) . In particular, for the quadratic activation σ ( t ) :

Original: = t 2 − 1 , we have the following identity (cid:101) ε 
Revised: = t 2 − 1 , we have the following identity (cid:101) ε test ( f init ) = 1 + 1 + ∥

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: For random initial output weights z 0 ∼ N (0 , (1 /m )1 m ) independent of the (random) hidden weights matrix W , one computes
Revised: For random initial output weights z 0 ∼ N (0 , (1 /m )1 m ) independent of the (random) hidden weights matrix W , one computes

Original: [Equation]
Revised: [Equation]

Original: where we have used the fact that E z = 0 .
Revised: where we have used the fact that E z = 0 .

Original: The second term in the rightmost expression equals ∥ f ⋆ ∥ 2 L 2 ( N (0 ,I d ))
Revised: The second term in the rightmost expression equals ∥ f ⋆ ∥ 2 L 2 ( N (0 ,I d ))

Original: = 2 ∥ B ∥ 2 F .
Revised: = 2 ∥ B ∥ 2 F .

Original: Let Q be the m × m diagonal matrix with the output weights z on the diagonal, and let U be the m × m matrix with entries
Revised: Let Q be the m × m diagonal matrix with the output weights z on the diagonal, and let U be the m × m matrix with entries

Original: U ij :=
Revised: U ij :=

Original: E x [ σ ( x ⊤ w j ) σ ( x ⊤ w j )] introduced in (13), and let U 0 :
Revised: E x [ σ ( x ⊤ w j ) σ ( x ⊤ w j )] introduced in (15), and let U 0 :

Original: = λI m + λ 21 WW ⊤ + ( κ/d )1 m 1 ⊤ m + µµ ⊤ with µ := ( λ 2 ( ∥ w j ∥ 2 − 1))
Revised: = λI m + λ 21 WW ⊤ + ( κ/d )1 m 1 ⊤ m + µµ ⊤ with µ := ( λ 2 ( ∥ w j ∥ 2 − 1))

Original: j ∈ [ m ] ∈ R m , be its approximation given in Proposition F.1.
Revised: j ∈ [ m ] ∈ R m , be its approximation given in Proposition G.1.

Original: Then
Revised: Then

Revised: E

Original: [Equation]
Revised: [Equation]

Original: ) σ
Revised: [ f init

Original: [Equation]
Revised: [Equation]

Original:  Wx ) ⊤ ]
Revised: x [Equation] [Equation] Wx )]

Revised: [Equation]

Original: z =
Revised: x [

Original: [Equation]
Revised: [Equation]

Original: Uz = tr( U ) /m
Revised: = tr( U ) /m

Original: + o d, P (1) , by concentration of random quadratic forms = tr( U 0 ) /m
Revised: + o d, P (1) , by concentration of random quadratic forms = [Equation] U [Equation]

Original: + o d, P (1) , thanks to Proposition F.
Revised: [Equation] o d, P (1) , thanks to Proposition G.

Original: [Equation]
Revised: [Equation]

Original: The first part of the result then follows upon dividing through by ∥
Revised: The first part of the result then follows upon dividing through by ∥

Original: f ⋆ ∥ 2 L 2 ( N (0 ,I d ))
Revised: f ⋆ ∥ 2 L 2 ( N ( [Equation] I d ))

Original: = 2 ∥ B ∥ 2 F .
Revised: = 2 ∥ B ∥ 2 F .

Original: In particular, if σ is the quadratic activation, then ∥ σ ∥ 2 L 2 ( N (0 ,I d )) =
Revised: In particular, if σ is the quadratic activation, then ∥ σ ∥ 2 L 2 ( N (0 ,I d )) =

Original: λ 2 = 2 , and the second part of the result follows.
Revised: λ 2 = 2 , and the second part of the result follows.

Original: G. T HE SPECIAL CASE OF QUADRATIC ACTIVATIONS
Revised: H. T HE SPECIAL CASE OF QUADRATIC ACTIVATIONS

Original: We now specialize Theorem 4.1 to the case of the quadratic activation function and obtain more transparent formulae.
Revised: We now specialize Theorem 4.1 to the case of the quadratic activation function and obtain more transparent formulae [Equation]

Original: Corollary G.1.
Revised: Corollary H

Original:  Consider the random features model f RF with covariance matrix Γ satisfying
Revised: [Equation] 1. Consider the random features model f RF with covariance matrix Γ satisfying

Original: Condition 4.1 and quadratic activation function σ ( t ) :
Revised: Condition 4.1 and quadratic activation function σ ( t ) :

Original: =
Revised: =

Original: t 2 − 1 .
Revised: t 2 − 1 .

Original: Then, in the limit (3) , it holds that (cid:101) ε rob( f RF ) = tr
Revised: Then, in the limit (3) , it holds that (cid:101) ε rob

Original: [Equation]
Revised: [Equation]

Original: (1 /m + ∥ Γ ∥ 2 F ) [-
-]
Revised: f RF ) [-
-]= tr [Equation] [Equation]

Original: ∥ B ∥ 2 F + o d, P (1) .
Revised: [Equation] B ∥ 2 F + o d, P (1) .

Original: Furthermore, part (B) of Theorem 4.1 holds.
Revised: Furthermore, part (B) of Theorem 4.1 holds.

Original: Proof.
Revised: Proof.

Original: For quadratic activation, one easily computes [-λ 1 = λ 0 = 0 , λ 2 = 2 , λ = 2 , λ ′ = 4 , κ = λ 22 ∥ Γ ∥ 2 F d/ 2 = 2 ∥ Γ ∥ 2 F d, τ := 2tr( B Γ) / √ d, κ ′ = 0 , and so A 0 = 
-]
Revised: For quadratic activation, one easily computes [-λ 1 = λ 0 = 0 , λ 2 = 2 , λ = 2 , λ ′ = 4 , κ = λ 22 ∥ Γ ∥ 2 F d/ 2 = 2 ∥ Γ ∥ 2 F d, τ := 2tr( B Γ) / √ d, κ ′ = 0 , and so A 0 = 
-][Equation]

Original:  I m and D 

Revised: and so A 0 = 2 I m and D

Original: = 

Revised: =

Original: I m .
Revised: I m .

Original: In this case, one deduces
Revised: In this case

Original: [Equation]
Revised: [Equation]

Original: ψ
Revised: one deduces

Original: [Equation]
Revised: [Equation]

Original: Plugging these into formula (21) of Theorem 4.1 yields
Revised: Plugging these into formula (23) of Theorem [Equation] 1 yields

Original: [Equation]
Revised: [Equation]

Original: which proves the first part of the corollary.
Revised: which proves the first part of the corollary.

Original: The second part follows directly from the second part of
Revised: The second part follows directly from the second part of

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: G. P ROOF OF T HEOREM E.1: R ANDOM FEATURES LAZY (RFL) REGIME
Revised: H. P ROOF OF T HEOREM F.1: R ANDOM FEATURES LAZY (RFL) REGIME

Original: Theorem E.1.
Revised: Theorem F.1.

Original: We have the following identities
Revised: We have the following identities

Original: [Equation]
Revised: [Equation]

Original: where U =
Revised: where U =

Original: U ( W ) and C = C ( W ) are the random matrices defined in (13) and (53) respectively.
Revised: U ( W ) and C = C ( W ) are the random matrices defined in (15) and (54) respectively.

Original: Proof.
Revised: Proof.

Original: By construction, note that the vector δ λ is equivalent to the output weights of a RF approximation with true labels (cid:101)
Revised: By construction, note that the vector δ λ is equivalent to the output weights of a RF approximation with true labels (cid:101)

Original: f
Revised: f

Original: ⋆ ( x ) :
Revised: ⋆ ( x ) :

Original: =
Revised: =

Original: f ⋆ ( x ) − f init ( x ) .
Revised: f ⋆ ( x ) − f init ( x ) .

Original: If U and v are as defined in (13) and (14) respectively, then we have the closed-form solution (with U λ :
Revised: If U and v are as defined in (15) and (16) respectively, then we have the closed-form solution (with U λ :

Original: =
Revised: =

Original: U + λI m )
Revised: U + λI m )

Original: [Equation]
Revised: [Equation]

Original: Thus, for a fixed regularization parameter λ > 0 , the output weights vector in this lazy training regime is given by
Revised: Thus, for a fixed regularization parameter λ > 0 , the output weights vector in this lazy training regime is given by

Original: [Equation]
Revised: [Equation]

Original: where P λ :
Revised: where P λ :

Original: = I m − U − 1 λ U .
Revised: = I m − U − 1 λ U .

Original: We deduce that in the presence of any amount of ridge regularization, the lazy random features (RFL) regime is equivalent to the vanilla random features (RF) regime, with an additive bias of P λ z 0 ∈ R m on the fitted output weights vector.
Revised: We deduce that in the presence of any amount of ridge regularization, the lazy random features (RFL) regime is equivalent to the vanilla random features (RF) regime, with an additive bias of P λ z 0 ∈ R m on the fitted output weights vector.

Original: In particular, note that if λ = 0 , then z RFL , 0 =
Revised: In particular, note that if λ = 0 , then z RFL , 0 =

Original: z RF , 0 , that is in the absence of regularization, the RFL and RF correspond to the same regime (i.e., the initialization has no impact on the final model).
Revised: z RF , 0 , that is in the absence of regularization, the RFL and RF correspond to the same regime (i.e., the initialization has no impact on the final model).

Original: – test error.
Revised: – test error.

Original: From formula (90), and noting that z 0 is independent of W , one computes the test error of f lazy ,λ averaged over the initial output weights vector z0 as
Revised: From formula (91), and noting that z 0 is independent of W , one computes the test error of f lazy ,λ averaged over the initial output weights vector z as

Original: [Equation]
Revised: [Equation]

Original: where U =
Revised: where U =

Original: U ( W ) is the matrix defined in(13).
Revised: U ( W ) is the matrix defined in

Original: – (Non)robustness.
Revised: – (Non)robustness.

Original: From formula (90), one computes
Revised: From formula (91), one computes

Original: [Equation]
Revised: [Equation]

Original: where C = C ( W ) is the matrix defined in (64).
Revised: where C = C ( W ) is the matrix defined in (65).

Original: Taking expectations w.r.t z 0 , and noting that z 0 is independent of P λ and C only depend on W and are therefore independent of z 0 , we have
Revised: Taking expectations w.r.t z 0 , and noting that z 0 is independent of P λ and C only depend on W and are therefore independent of z 0 , we have

Original: [Equation]
Revised: [Equation]

Original: G. P ROOF OF T HEOREM 5.1: N EURAL TANGENT (NT) REGIME
Revised: H. P ROOF OF T HEOREM 5.1: N EURAL TANGENT (NT) REGIME

Original: Theorem 5.1.
Revised: Theorem 5.1.

Original: Consider the neural tangent model f NT in (24) .
Revised: Consider the neural tangent model f NT in (26) .

Original: In the limit (3) it holds that, 
Revised: In the limit (3) it holds that, E

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original:  ρ {++ ρ 2 ) / 2 + ( ρ − ρ 2 ) β/ 
 + o d (1) , where ρ := min( ρ, 1)+}
Revised: ( f NT )] = ( ρ {++ ρ 2 ) / 2 + ( ρ − ρ 2 ) β/ 
 + o d (1) , where ρ := min( ρ, 1)+}

Original: [Equation]
Revised: [Equation]

Original: Let r ≤ min( m, d ) be the rank of W .
Revised: Let r ≤ min( m, d ) be the rank of W .

Original: It is clear that r = min( m, d ) w.p 1 .
Revised: It is clear that r = min( m, d ) w.p 1 .

Original: Let
Revised: Let

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: be the singular-value decomposition of W ⊤ , where P 1 ∈ R d × r (resp.
Revised: be the singular-value decomposition of W ⊤ , where P 1 ∈ R d × r (resp.

Original: V ∈ R m × r ) is the columnorthogonal matrix of singular-vectors of W ⊤ (resp.
Revised: V ∈ R m × r ) is the columnorthogonal matrix of singular-vectors of W ⊤ (resp.

Original: W ), and S ∈ R r × r is the diagonal matrix of nonzero singular-values.
Revised: W ), and S ∈ R r × r is the diagonal matrix of nonzero singular-values.

Original: For any A ∈ R m × d , set G ( A ) :
Revised: For any A ∈ R m × d , set G ( A ) :

Original: = SV ⊤ A ∈ R r × d .
Revised: = SV ⊤ A ∈ R r × d .

Original: In their proof of (25),
Revised: In their proof of (27),

Original: Ghorbani et al.
Revised: Ghorbani et al.

Original: (2019) showed that it is optimal (in terms of test error) to chose A NT such that
Revised: (2019) showed that it is optimal (in terms of test error) to chose A NT such that

Original: G ( A NT ) =
Revised: G ( A NT ) =

Original: P ⊤ 1 B/ 2 .
Revised: P ⊤ 1 B/ 2 .

Original: Multiplying through by the orthogonal projection matrix P 1 gives
Revised: Multiplying through by the orthogonal projection matrix P 1 gives

Original: [Equation]
Revised: [Equation]

Original: For the proof of Theorem 5.1, we will need the following lemma.
Revised: For the proof of Theorem 5.1, we will need the following lemma.

Original: Lemma G.1.
Revised: Lemma H.1.

Original: ε rob ( f NT 
Revised: ε rob ( f NT )

Original: [Equation] = [Equation] W 
Revised: = 4 ∥ W ⊤ A + A ⊤ W ∥ 2 F .

Original: [Equation]

Original: F .

Original: Proof.
Revised: Proof.

Original: Note that we can rewrite
Revised: Note that we can rewrite

Original: [Equation]
Revised: [Equation]

Original:  which is linear in xx ⊤ ∈ R d × d .
Revised: , which is linear in xx ⊤ ∈ R d × d .

Original: One then readily computes ∇ f NT ( x ) = 2( W ⊤ A + A ⊤ W ) x , from which we deduce that ∥∇ f NT ( x ) ∥ 2 = 4 x ⊤ ( W ⊤ A
Revised: One then readily computes ∇ f NT ( x ) = 2( W ⊤ A + A ⊤ W ) x , from which we deduce that ∥∇ f NT ( x ) ∥ 2 = 4 x ⊤ ( W ⊤ A

Original: + A ⊤ W ) 2 x .
Revised: + A ⊤ W ) 2 x .

Original: Averaging over x ∼ N (0 , I d ) then gives 
Revised: Averaging over x ∼ N (0 , I d ) then gives ε

Original: [Equation]
Revised: [Equation]

Original: which completes the proof.
Revised: which completes the proof.

Original: We will also need the following auxiliary lemma.
Revised: We will also need the following auxiliary lemma.

Original: Lemma G.2.
Revised: Lemma H.2.

Original: Let P 1 be as in (92) and let β := tr( B ) 2 / ( d ∥ B ∥ 2 F ) as usual.
Revised: Let P 1 be as in (93) and let β := tr( B ) 2 / ( d ∥ B ∥ 2 F ) as usual.

Original: In the limit (3) , we have the identities
Revised: In the limit (3) , we have the identities

Original: [Equation]
Revised: [Equation]

Original: where ρ := min( ρ, 1) .
Revised: where ρ := min( ρ, 1) .

Original: Proof.
Revised: Proof.

Original: WLOG, let B be a diagonal matrix, so that B 2 = (cid:80)
Revised: WLOG, let B be a diagonal matrix, so that B 2 = (cid:80)

Original: j λ 2 j e j e ⊤ j , where e j is the j th standard unit-vector in R d .
Revised: j λ 2 j e j e ⊤ j , where e j is the j th standard unit-vector in R d .

Original: Then, with P = P 1 P ⊤ 1 , we have
Revised: Then, with P = P 1 P ⊤ 1 , we have

Original: [Equation]
Revised: [Equation]

Original: Therefore, E W [ ∥ P 1 P ⊤ 1 B ∥ 2 F ] = (1 /d )
Revised: Therefore, E W [ ∥ P 1 P ⊤ 1 B ∥ [Equation] ] = [Equation]

Revised: 1 /d

Original: E W [tr( P )] · (cid:80) j λ 2 j = min
Revised: [Equation] [Equation] [Equation]

Original: ( m/d, 1)
Revised: = [Equation] B ∥ [Equation]

Original: ∥ B ∥ 2 F =  ∥
Revised: = [Equation] B ∥

Original: B ∥ 2 ( ρ + o d (1)) , where we have used the fact that E W P jj = (1 /d )
Revised: [Equation] ρ [Equation] ) , where we have used the fact that E W P jj = (1 /d )

Original: E W tr( P ) for all j , due to rotation-invariance.
Revised: E W tr( P ) for all j , due to rotation-invariance.

Original: This proves (94).
Revised: This proves (95).

Original: The proof of (95) is completely analogous to the proof of formula (69) in Ghorbani et al.
Revised: The proof of (96) is completely analogous to the proof of formula (69) in Ghorbani et al.

Original: (2019), with ρ therein replaced with 1 − ρ , and is thus omitted.
Revised: (2019), with ρ therein replaced with 1 − ρ , and is thus omitted.

Original: Proof of Theorem 5.1.
Revised: Proof of Theorem 5.1.

Original: From Lemma G.1 and formula (93)), we know that
Revised: From Lemma H.1 and formula (94)), we know that

Original: [Equation]
Revised: [Equation]

Original: The result then follows upon taking expectations w.r.t the hidden weights matrix W and applying
Revised: The result then follows upon taking expectations w.r.t the hidden weights matrix W and applying

Original: Lemma G.2.
Revised: Lemma H.2.

Original: G. P ROOF OF T HEOREM 5.2: N EURAL TANGENT LAZY (NTL) REGIME
Revised: H. P ROOF OF T HEOREM 5.2: N EURAL TANGENT LAZY (NTL) REGIME

Original: Theorem 5.2.
Revised: Theorem 5.2.

Original: Suppose the output weights z 0 at initialization are iid from N (0 , (1 /m ) I m ) .
Revised: Suppose the output weights z 0 at initialization are iid from N (0 , (1 /m ) I m ) .

Original: Then, in the limit (3) , the following identities hold
Revised: Then, in the limit (3) , the following identities hold

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: First observe that f ⋆ ( x ) − f NTL ( x
Revised: First observe that f ⋆ ( x ) − f NTL ( x

Original: [Equation]
Revised: [Equation]

Original: A, c [Equation] = (cid:101)
Revised: A, c ) = (cid:101)

Original: f ⋆ ( x ) − f NT ( x ; A, c ) , where,
Revised: f ⋆ ( x ) − f NT ( x ; A, c ) , where,

Original: [Equation]
Revised: [Equation]

Original: and the d × d matrix (cid:101) B is defined by
Revised: and the d × d matrix (cid:101) B is defined by

Original: [Equation]
Revised: [Equation]

Original: Thus, fitting the model f NTL ( · ; A, c ) to the teacher model f ⋆ with coefficient matrix B is equivalent to fitting f NT ( · ; A, c ) to the modified teacher model (cid:101)
Revised: Thus, fitting the model f NTL ( · ; A, c ) to the teacher model f ⋆ with coefficient matrix B is equivalent to fitting f NT ( · ; A, c ) to the modified teacher model (cid:101)

Original: f ⋆ with coefficient matrix (cid:101) B .
Revised: f ⋆ with coefficient matrix (cid:101) B .

Original: In terms of test error (4), let A NTL , c NTL be optimal in f NT ( · ; A, c ) , and for simplicity of notation define
Revised: In terms of test error (4), let A NTL , c NTL be optimal in f NT ( · ; A, c ) , and for simplicity of notation define

Original: [Equation]
Revised: [Equation]

Original: We split the proof into two parts.
Revised: We split the proof into two parts.

Original: In the first part, we establish (30).
Revised: In the first part, we establish (32).

Original: The second part handles (29).
Revised: The second part handles (31).

Original: – Robustness error.
Revised: – Robustness error.

Original: Proceeding in the same way as in the paragraph leading to ( , one hasε rob
Revised: Proceeding in the same way as in the paragraph leading to (94), one has

Original: [Equation]
Revised: [Equation]

Original: where P 1 ∈ R d × r is the column-orthogonal matrix in (92) and r := min( m, d ) is the rank of W (w.p 1 ).
Revised: where P 1 ∈ R d × r is the column-orthogonal matrix in (93) and r := min( m, d ) is the rank of W (w.p 1 ).

Original: Now, by definition of (cid:101) B , one has (cid:101) B 2 = ( B − W ⊤ QW )
Revised: Now, by definition of (cid:101) B , one has (cid:101) B 2 = ( B − W ⊤ QW )

Original: ( B − W ⊤ QW ) , and so
Revised: ( B − W ⊤ QW ) , and so

Original: P 1 P ⊤ 1 (cid:101)

Original: [Equation]
Revised: [Equation]

Original: We now take the expectation w.r.t ( W, z 0 ) , of each term on the RHS.
Revised: We now take the expectation w.r.t ( W, z 0 ) , of each term on the RHS.

Original: Thanks to Lemma G.2, we recognize the expectation w.r.t W of the trace of the first term in (102) as
Revised: Thanks to Lemma H.2, we recognize the expectation w.r.t W of the trace of the first term in (103) as

Original: [Equation]
Revised: [Equation]

Original: Now, since W and z 0 are independent and z 0 has zero mean, the second and third terms in (101) have zero expectation w.r.t ( W, z 0 ) because they are linear in Q = diag( z 0 ) .
Revised: Now, since W and z 0 are independent and z 0 has zero mean, the second and third terms in (102) have zero expectation w.r.t ( W, z 0 ) because they are linear in Q = diag( z 0 ) .

Original: Finally, one notes that
Revised: Finally, one notes that

Original: [Equation]
Revised: [Equation]

Original: and so taking expectation w.r.t W and D (i.e z 0 ) yields
Revised: and so taking expectation w.r.t W and D (i.e z 0 ) yields

Original: [Equation]
Revised: [Equation]

Original: where the last step is thanks to the second part of Lemma F.1.
Revised: where the last step is thanks to the second part of Lemma G.1.

Original: Putting things together, we have at this point established that
Revised: Putting things together, we have at this point established that

Original: [Equation]
Revised: [Equation]

Original: Similarly, noting that P 1 P ⊤ 1 W ⊤ =
Revised: Similarly, noting that P 1 P ⊤ 1 W ⊤ =

Original: W ⊤ by definition of P 1 , one has
Revised: W ⊤ by definition of P 1 , one has

Original: [Equation]
Revised: [Equation]

Original: Taking expectation w.r.t W and z 0 then gives
Revised: Taking expectation w.r.t W and z 0 then gives

Original: [Equation]
Revised: [Equation]

Original: Combining (101), (106), (108), and (26) then completes the proof of (30).
Revised: Combining (102), (107), (109), and (28) then completes the proof of (32).

Original: – test error.
Revised: – test error.

Original: The proof of formula (29) build on the proof of Theorem 2 in Ghorbani et al.
Revised: The proof of formula (31) build on the proof of Theorem 2 in Ghorbani et al.

Original: Let P 2 be a d × ( d − min( m, d )) matrix such that the combined columns of P 1 and P 2 form an orthonormal basis for R d .
Revised: Let P 2 be a d × ( d − min( m, d )) matrix such that the combined columns of P 1 and P 2 form an orthonormal basis for R d .

Original: Then, one computes 
Revised: Then, one computes ε test ( f NTL ) : = ∥ f [Equation]

Revised: ⋆

Revised: [Equation]

Revised: N

Original: [Equation]
Revised: [Equation]

Revised: d

Original:  where (a) and (b) are due to arguments analogous to arguments made in the beginning of proof of
Revised: [Equation] E [Equation] where (a) and (b) are due to arguments analogous to arguments made in the beginning of proof of

Original: Theorem 2 in Ghorbani et al.
Revised: Theorem 2 in Ghorbani et al.

Original: (except that our (cid:101) B plays the role of B in Ghorbani et al.
Revised: (except that our (cid:101) B plays the role of B in Ghorbani et al.

Original: and (c) is because P ⊤ 2 P 1 = 0 ∈ R ( d − min( m,d ))
Revised: and (c) is because P ⊤ 2 P 1 = 0 ∈ R ( d − min( m,d ))

Original: ×
Revised: ×

Original: d by construction of P 2 .
Revised: d by construction of P 2 .

Original: Dividing through the above display by ε rob ( f ⋆ ) = 4 ∥ B ∥ 2 F then gives (29).
Revised: Dividing through the above display by ε rob ( f ⋆ ) = 4 ∥ B ∥ 2 F then gives (31).

Original: H P ROOF OF T HEOREM 6.
Revised: I G ENERAL NONLINEAR TEACHER AND STUDENT MODELS

Revised: The results we established so far are for student-teacher models which are two-layer neural networks in certain learning regimes with Gaussian data. In this section, we consider much more general scenarios, and show lower-bounds that display similar tradeoffs between test error and robustness. Suppose the distribution P x of the features is any distribution on R d which satisfies a Poincar´e inequality with constant c 2 > 0 .

Revised: This means that for any smooth function f : R d → R , one has Var P x ( f ) ≤ c 2 ∥∇ f ∥ 2 L 2 ( P x ) , where Var P x ( f ) :=

Revised: ∥ f − f ∥ 2 L 2 ( P x ) is the variance of f and f :=

Revised: E P x f ∈ R is its mean w.r.t P x .

Revised: For example, N (0 , Σ) verifies a Poincar´e inequality with c 2 = ∥

Revised: Σ

Revised: ∥ op .

Revised: Consider a teacher model f ⋆ which is now any function in L 2 ( P x ) with mean f ⋆ = 0 .

Revised: Theorem I.1.

Revised: For every smooth student model f : R d → R (neural network or not!), it holds that (cid:112)

Revised: [Equation]

Revised: The nature of Theorem I.1 is a tradeoff since it directly implies that the test error cannot be decreased without increasing the robustness error.

Revised: In the particular case of isotropic features where P x =

Original: Theorem 6.1. For every smooth student model f : R d→ R (neural network or not!), it holds that (cid:112) ε test ( f ) + c · (cid:112) ε rob ( f ) ≥ ∥
Revised: N (0 , I d ) as considered in the preceding sections, a Poincar´e inequality with constant c 2 = 1 is satisfied, and we deduce from the above theorem that, for any smooth student model f , one has [Equation] [Equation]

Original:  f ⋆ ∥
Revised: Of course, apart from being only one-sided, the inequality (110) is weaker than the tradeoffs established in the preceding sections, due to the square-roots in the former. However, (110) holds without any real restriction on the teacher model f ⋆ , student f model, or learning algorithm / regime; it is solely a consequence of the high-dimensional geometry of the distribution of the features, manifested via the Poincar´e inequality. In contrast, the tradeoffs established in the preceding sections where for student-teacher models which where two-layer neural networks in various learning regimes.

Original: L 2 ( P x ) .
Revised: Theorem I.1. For every smooth student model f : R d → R (neural network or not!), it holds that (cid:112) [Equation]

Original: Proof.
Revised: Proof.

Original: WLOG, assume ε test ( f ) ≥ ∥
Revised: WLOG, assume ε test ( f ) ≤ ∥

Original: f ⋆ ∥
Revised: f ⋆ ∥

Original: L 2 ( P x ) , since the claimed lower-bound trivially holds otherwise.
Revised: L 2 ( P x ) , since the claimed lower-bound trivially holds otherwise.

Original: By the Poincar ´ e inequality, we have
Revised: By the Poincar´e inequality, we have

Original: [Equation]
Revised: [Equation]

Original:  where f ∈ R is mean of f
Revised: P x ( [Equation] = ∥ f − f ∥ 2 L 2 ( P x ) , where f ∈ R is mean of f

Original: [Equation]
Revised: [Equation]

Original:  by the triangle inequality.
Revised: , by the triangle inequality.

Original: In particular, we have
Revised: In particular, we have

Original: [Equation]
Revised: [Equation]

Original: where the last line follows from ∥
Revised: where the last line follows from ∥

Original: f ⋆ ∥ 2 L 2 ( P x ) =
Revised: f ⋆ ∥ 2 L 2 ( P x ) =

Original: Var P x
Revised: Var P x

Original: ( f ⋆ ) ≤ ∥ f
Revised: ( f ⋆ [Equation]

Original:  ⋆ − f ∥ 2 L 2 ( P x ) .
Revised: ≤ ∥ f ⋆ − f ∥ 2 L 2 ( P x ) .

Original: The result then follows from a simple rearrangement of the terms in the above display.
Revised: The result then follows from a simple rearrangement of the terms in the above display.

Revised: K A PPROXIMATING FUNCTION VALUES DOESN ’ T AMOUNT TO APPROXIMATING GRANDIENTS K. D ISPROVING (9)

Revised: Henceforth, for a student model f , consider the residue function h :=

Revised: f − f ⋆ .

Revised: Thus, ∇ f −∇ f ⋆ = ∇ h .

Revised: In the student-teacher setup considered in our work, the student f is in general mis-specified w.r.t to the teacher f ⋆ (for example, because the students activation is arbitrary while the teacher’s activation function is fixed to quadratic), and so the residue function h has no specific structure in general.

Revised: Mindful of the previous remark, to disprove (1), it is sufficient to construct a subspace H of the weighted Sobolev space W 1 , 2 ( N (0 , I d ))

Original: {+(consisting of functions g : R d → R which are squareintegrable w.r.t N (0 , I d ) with weak-derivatives which are square-integrable w.r.t N (0 , I d ) ), such that: for every ϵ, C > 0 , there exists h 0 ∈ H with ∥ h 
+}
Revised: {+(consisting of functions g : R d → R which are squareintegrable w.r.t N (0 , I d ) with weak-derivatives which are square-integrable w.r.t N (0 , I d ) ), such that: for every ϵ, C > 0 , there exists h 0 ∈ H with ∥ h 
+}

Revised: ∥

Original: {+L 2 ( N (0 ,I d )) ≤ ϵ and ∥∇ h 
+}
Revised: {+L 2 ( N (0 ,I d )) ≤ ϵ and ∥∇ h 
+}

Revised: ∥

Revised: L 2 ( N (0 ,I d )) > C. (111)

Revised: Indeed, take H

Revised: = W 1 , 2 ( N (0 , I d )) , and for fixed α ∈ (0 , 1) , consider the sequence of residue functions ( h n ) n in H given by h n ( x ) :

Revised: = (1 /n α ) sin( nx 1 ) , for every positive integer n , and x = ( x 1 , . . .

Revised: , x d ) ∈ R d .

Revised: Note that a constructive way for realizing such residue functions is by taking

Revised: B = 0 in the teacher model f ⋆ , and activation function σ ( t ) ≡ sin( t ) in the student model f .

Revised: Now, a simple computation gives ∇ h n ( x ) =

Revised: (1 /n β )

Revised: cos( nx 1 ) e 1 , where β := 1 − α ∈ (0 , 1) and e j is j th standard basis vector in R d .

Revised: Furthermore,

Revised: [Equation]

Revised: Thus, (2) holds and we conclude that the implication (1) claimed by the reviewer fails in general.

Revised: □

Revised: K. S OME EXCEPTIONAL CASES WHERE (1) HOLDS

Revised: Let us roundup by noting that (1) can be true in very specific circumstances.

Revised: For example, if the activation function σ of the student is quadratic, just like the teacher model f ⋆ .

Revised: Indeed, in this case the set of all residue functions f − f ⋆ is contained in the set of polynomials of degree at most 2 in d real variables; this is a finite-dimensional subspace H =

Revised: P 2 ( R d ) of L 2 ( N (0 , I d )) .

Revised: In fact, it is easy to show via a simple counting argument that dim ( H ) =

Revised: dim ( P 2 ( R d )

Revised: [Equation]

Revised: d 2 .

Revised: Thus, the gradient operator ∇ is a finite-rank, and therefore compact operator on H .

Revised: It follows that the H -restricted operator norm ∥∇| H ∥ op defined by

Revised: [Equation]

Revised: 112) is finite and thus the implication (1) holds in this case.

Revised: This argument is valid whenever the linear span H of the residue functions f − f ⋆ is a finite-dimensional subspace of L 2 ( N (0 , I d )) , for example linear models, or more generally, polynomial models of degree ≤ D (corresponding to the case where the activation function σ of the student is a polynomial of degree ≤ D ), for some fixed integer D ≥ 1 ; indeed H = P D ( R d ) in this case, and has dimension (cid:0) d + D d (cid:1) =

Revised: ( d + D )( d + D − 1) . . .

Revised: ( d +1) /D !

Revised: ≲ d D .