Title: Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign?
Venue: NeurIPS 2023
Accepted: yes
Keywords: benign overfitting

neural networks

relu

hinge loss
Abstract: We study benign overfitting in two-layer ReLU networks trained using gradient descent and hinge loss on noisy data for binary classification. In particular, we consider linearly separable data for which a relatively small proportion of labels are corrupted or flipped. We identify conditions on the margin of the clean data that give rise to three distinct training outcomes: benign overfitting, in which zero loss is achieved and with high probability test data is classified correctly; overfitting, in which zero loss is achieved but test data is misclassified with probability lower bounded by a constant; and non-overfitting, in which clean points, but not corrupt points, achieve zero loss and again with high probability test data is classified correctly. Our analysis provides a fine-grained description of the dynamics of neurons throughout training and reveals two distinct phases: in the first phase clean points achieve close to zero loss, in the second phase clean points oscillate on the boundary of zero loss while corrupt points either converge towards zero loss or are eventually zeroed by the network. We prove these results using a combinatorial approach that involves bounding the number of clean versus corrupt updates during these phases of training.

Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign?

Abstract

We study benign overfitting in two-layer ReLU networks trained using gradient descent and hinge loss on noisy data for binary classification. In particular, we consider linearly separable data for which a relatively small proportion of labels are corrupted or flipped. We identify conditions on the margin of the clean data that give rise to three distinct training outcomes: benign overfitting, in which zero loss is achieved and with high probability test data is classified correctly; non-benign overfitting, in which zero loss is achieved but test data is misclassified with probability lower bounded by a constant; and non-overfitting, in which clean points, but not corrupt points, achieve zero loss and again with high probability test data is classified correctly. Our analysis provides a fine-grained description of the dynamics of neurons throughout training and reveals two distinct phases: in the first phase clean points achieve close to zero loss, in the second phase clean points oscillate on the boundary of zero loss while corrupt points either converge towards zero loss or are eventually zeroed by the network. We prove these results using a combinatorial approach that involves bounding the number of clean versus corrupt updates across these phases of training.

1 Introduction

Conventional machine learning wisdom suggests that the generalization error of a complex model will typically be worse versus a simpler model when both are trained to interpolate data. Indeed, the bias-variance trade-off implies that although choosing a complex model is advantageous in terms of approximation error, it comes at the price of an increased risk of overfitting. The traditional solution to managing this trade-off is to use some form of regularization, allowing the optimizer to select a predictor from a rich class of functions while at the same time encouraging it to choose one that is in some sense simple. However, in recent years this perspective has been challenged by the observation that deep learning models, trained with minimal if any form of regularization, can almost perfectly interpolate noisy data with nominal cost to their generalization performance (Zhang et al., 2017; Belkin et al., 2018b, 2019). This phenomenon is referred to as benign overfitting .

Following these empirical observations, a line of research has emerged aiming to theoretically characterize the conditions under which various machine learning models, trained to zero loss on noisy data, obtain, at least asymptotically, optimal generalization error. To date, the majority of analyses in this regard have focused primarily on linear models, including linear regression (Bartlett et al., 2020; Muthukumar et al., 2020; Wu & Xu, 2020; Chatterji & Long, 2021; Zou et al., 2021; Hastie et al., 2022; Koehler et al., 2021; Wang et al., 2021a; Chatterji & Long, 2022; Cao et al., 2021; Shamir, 2022), logistic regression (Chatterji & Long, 2021; Muthukumar et al., 2021; Wang et al., 2021b) and kernel regression (Belkin et al., 2018a; Mei & Montanari, 2019; Liang & Rakhlin, 2020; Liang et al., 2019). With regards to understanding benign overfitting in neural networks, in the

Neural Tangent Kernel (NTK) regime (Jacot et al., 2018) the prediction of a neural network is well approximated via kernel regression (Adlam & Pennington, 2020). However, this regime typically requires unrealistically large network width and fails to capture feature learning. Indeed, and despite being the initial source of inspiration, an understanding of when and how neural networks benignly overfit in the rich, feature learning regime is not well understood.

1.1 Contributions and related work

In this work we study benign overfitting in the context of binary classification for two-layer ReLU networks, trained using gradient descent and hinge loss, on label corrupted, linearly separable data. There are a number of recent and or concurrent works which prove benign overfitting results in a similar setting Frei et al. (2022, 2023); Xu & Gu (2023); Cao et al. (2022); Kou et al. (2023); Kornowski et al. (2023), however, we emphasize that these exclusively study exponentially tailed losses, notably the popular logistic loss. Benign overfitting is intimately related to the notion of implicit bias , the preference of an algorithm for selecting minimizers with certain properties over others. The implicit bias of homogeneous networks trained with gradient descent on an exponentially tailed loss from a low initial loss is known to converge in direction to a Karush-Kuhn-Tucker (KKT) point of the associated max-margin problem Lyu & Li (2020); Ji & Telgarsky (2020). This implies at least intuitively a certain bias towards margin maximization. In a recent work Frei et al. (2023) it is shown that if the input data is sufficiently orthogonal then a shallow, leaky ReLU network evaluated on such a KKT point is equivalent to a particular linear classifier. Moreover, and under additional data assumptions, the authors show such networks benignly overfit. Another recent paper Kornowski et al. (2023) uses a similar approach to derive benign overfitting results for ReLU networks and also provides a description of the transition between benign and tempered overfitting in the univariate input case. To the best of our knowledge, equivalent results on the implicit bias of homogeneous networks trained with non-exponentially tailed losses are not characterized. Furthermore, training a linear classifier with an exponential versus non-exponential tailed loss is known to result in a different implicit bias, with the non-exponential tailed loss potentially inducing convergence in direction to a classifier with a poor margin Ji et al. (2020). As a result, a priori it is not clear if and how the choice of hinge loss impacts the propensity for a shallow ReLU network to overfit.

There are two main existing lines of work which study benign overfitting in neural networks outside of the kernel regime. Concerning perhaps the most relevant line of prior work to our own, Frei et al. (2022) consider a smooth, leaky ReLU activation function, train the network using the logistic instead of the hinge loss and assume the data is drawn from a mixture of well-separated sub-Gaussian distributions. The key result of this work is that given a sufficient number of iterations of GD, then the network will interpolate the noisy training data while also achieving minimax optimal generalization error up to constants in the exponents. A concurrent work Xu & Gu (2023) extends this result to more general activation functions including ReLU, relaxes the assumptions on the noise distribution to being centered with bounded logarithmic Sobolev constant, and also improves the convergence rate. As highlighted in Xu & Gu (2023), the fact that ReLU is non-smooth and non-leaky significantly complicates the analysis of both the convergence and generalization. A second line of work (Cao et al., 2022; Kou et al., 2023) studies benign overfitting in two-layer convolutional as opposed to feedforward neural networks. Whereas here and in Frei et al. (2022); Xu & Gu (2023) each data point is modeled as the sum of a signal and noise component, in Cao et al. (2022); Kou et al. (2023) the signal and noise components lie in disjoint patches. The weight vector of each neuron is applied to both patches separately and a non-linearity, such as ReLU, is applied to the resulting pre-activation. In this setting, the authors prove interpolation of the noisy training data and derive conditions on the clean margin under which the network benignly vs non-benignly overfits. We emphasize that the data model studied in this work is very different to the setting we study here, and as a result we primarily restrict our comparison to that with Frei et al. (2022) and the concurrent work Xu & Gu (2023). Finally, in regard to optimizing shallow ReLU networks using hinge loss, a line of work (Brutzkus et al., 2018; Wang et al., 2019; Yang et al., 2021) studies the convergence of gradient descent on generic, linearly separable data without label corruptions. These works also require additional assumptions, notably leaky ReLU instead of ReLU, insertion of noise into the optimization algorithm or changes to the loss function.

Before we discuss our contributions we remark that a previous work Mallinar et al. (2022) describes and experimentally explores a taxonomy of overfitting: benign overfitting, where the generalization error is optimal; catastrophic overfitting, where the generalization error is close to random chance;

and tempered overfitting, which lies in between. In this work, we do not consider the full breadth of this taxonomy, and use the terms 'non-benign overfitting' or equivalently 'harmful overfitting' to refer to overfitting that may be either tempered or catastrophic. We now summarize our contributions: in particular, under certain assumptions on the model hyperparameters, we prove conditions on the clean margin resulting in the three distinct training outcomes highlighted below. We remark also that the prior works discussed primarily focus on deriving positive benign overfitting results.

Benign overfitting: Theorem 3.1 provides conditions under which the training loss converges to zero and bounds the generalization error, showing that it is asymptotically optimal. This result is analogous to those of Frei et al. (2022) and Xu & Gu (2023) but for the hinge instead of logistic loss.

Non-benign overfitting: Theorem 3.6 provides conditions under which the network achieves zero training loss while generalization error is bounded below by a constant. Unlike Frei et al. (2022) and Xu & Gu (2023), this is not due to the non-separability of the data model but is instead a result of the neural network failing to learn the optimal classifier.

No overfitting: Theorem 3.8 provides conditions under which the network achieves zero training loss on points with uncorrupted label signs but nonzero loss on points with corrupted signs. Again the generalization error is bounded and shown to be asymptotically optimal.

To conclude this section we further remark that our proof techniques are quite different from those used in Frei et al. (2022); Xu & Gu (2023) and indeed the other works highlighted in this section. Again we emphasize this is due to the fact we study the hinge loss instead of the logistic loss and discuss the differences arising from this in detail in Section 3. In particular, we set up the problem in such a way that the convergence analysis reduces to counting the number of activations of clean versus corrupt points during various stages of training. Our analysis further provides a detailed description of the dynamics of the network's neurons, thereby allowing us to understand how the network fits both the clean and corrupted data.

2 Preliminaries

2.1 Data model

We consider a training sample of 2 n pairs of points and their labels ( x i , y i ) 2 n i =1 where ( x i , y i ) ∈ R d × {-1 , +1 } for all i ∈ [2 n ] . Furthermore, we identify two disjoint subsets S T ⊂ [2 n ] = { 1 , . . . , 2 n } and S F ⊂ [2 n ] , S T ∪ S F = [2 n ] , which correspond to the clean and corrupt points in the sample respectively. The categorization of a point as clean or corrupted is determined by its label: for all i ∈ [2 n ] we assume y i = β ( i )( -1) i where β ( i ) = -1 iff i ∈ S F and β ( i ) = 1 otherwise. In addition, we assume |S F ∩ [2 n ] e | = |S F ∩ [2 n ] o | = k and |S T ∩ [2 n ] e | = |S T ∩ [2 n ] o | = n -k , where [2 n ] e ⊂ [2 n ] and [2 n ] o ⊂ [2 n ] are the even and odd indices, respectively. We remark that this assumption simplifies the exposition of our results but is not integral to our analysis. Each data point is assumed to have the form

Here v ∈ R d satisfies ∥ v ∥ = 1 and furthermore we refer to v as the signal vector as the alignment of a clean point with v determines its sign. Indeed, sign( ⟨ x i , v ⟩ ) = ( -1) i = y i for i ∈ S T whereas sign ( ⟨ x i , v ⟩ ) = -y i for i ∈ S F . Thus we may view the labels of a corrupt point as flipped from their clean state. The vectors ( n i ) 2 n i =1 are mutually independent and identically distributed (i.i.d.) random vectors drawn from the uniform distribution over S d -1 ∩ span { v } ⊥ , which we denote U ( S d -1 ∩ span { v } ⊥ ) . Clearly this distribution is symmetric, mean zero and for any n ∼ U ( S d -1 ∩ span { v } ⊥ ) it holds that n ⊥ v and ∥ n ∥ = 1 . Werefer to these vectors as noise components due to the fact that they are independent of the labels of their respective points. The real, scalar quantity γ ∈ [0 , 1] controls the strength of the signal versus the noise and also defines the clean margin. Finally, at test time a clean label y ∼ U ( {-1 , 1 } ) is sampled and the corresponding test data point is constructed,

where again n ∼ U ( S d -1 ∩ span { v } ⊥ ) .

The key idea we use to characterize the training dynamics is to reduce the analysis of the trajectory of each neuron to that of counting the number of clean versus corrupt updates to it. This combinatorial approach relies on each point having similar sized signal and noise components. In order to make our analysis as clear as possible, we select a data model which ensures the signal and noise components are consistent in size across all points. We emphasize that these assumptions are not strictly necessary and we believe analogous analyses could be conducted when the signal and noise components are instead appropriately bounded. In addition, and as discussed in more detail in Section 3.2, the orthogonality of the signal and noise components allow us to demonstrate non-benign overfitting even when a perfect classifier exists.

2.2 Network architecture, optimization and initialization

We consider a densely connected, single layer feed-forward neural network f : R 2 m × d × R d → R with the following forward pass map,

Here ϕ := max { 0 , z } denotes the ReLU activation function and w j the j -th row of the weight matrix W ∈ R 2 m × d . The network weights are optimized using full batch gradient descent (GD) with step size η > 0 in order to minimize the hinge loss over a training sample (( x i , y i )) 2 n i =1 ⊂ ( R d ×{-1 , 1 } ) 2 n sampled as described in Section 2.1. After t ′ iterations this optimization process generates a sequence of weight matrices ( W ( t ) ) t ′ t =0 . For convenience, we overload our notation for the forward pass map of the network and let f ( t, x ) := f ( W ( t ) , x ) . Furthermore, we denote the hinge loss on the i -th point at iteration t as ℓ ( t, i ) := max { 0 , 1 -y i f ( t, x i ) } . The hinge loss over the entire training sample at iteration t is therefore L ( t ) := ∑ 2 n i =1 ℓ ( t, i ) . Let F ( t ) := { i ∈ [2 n ] : ℓ ( t, x i ) > 0 } and A ( t ) j := { i ∈ [2 n ] : ⟨ w ( t ) j , x i ⟩ > 0 } denote the sets of point indices that have nonzero loss and which activate the j th neuron at iteration t respectively. With

then the GD update rule 1 for the neuron weights at iteration t ≥ 0 may be written as

In regard to the initialization of the network parameters, for convenience we assume each neuron's weight vector is drawn mutually i.i.d. uniform from the centered sphere with radius λ w > 0 . We remark that results analogous to the ones presented hold if the weights are instead initialized mutually i.i.d. as w (0) jc ∼ N (0 , σ 2 w ) for sufficiently small σ 2 w .

2.3 Notation

For indices i, j ∈ Z ≥ 1 we say i ∼ j iff ( -1) i = ( -1) j . We often refer to a data point or neuron by its index alone, e.g. 'point i ' refers to the i -th training point ( x i , y i ) . For two iterations t 0 , t 1 with t 1 > t 0 we define the following.

G j ( t 0 , t 1 ) := ∑ i ∈S T ∑ t 1 -1 τ = t 0 ✶ ( i ∈ A ( τ ) j ∩ F ( τ ) ) is the number of clean updates applied to the j -th neuron between iterations t 0 and t 1 .

G ( t 0 , t 1 ) := ∑ j ∈ [2 m ] G j ( t 0 , t 1 ) and B ( t 0 , t 1 ) := ∑ j ∈ [2 m ] B j ( t 0 , t 1 ) are the total number of clean and corrupt updates applied to the entire network between iterations t 0 and t 1 .

B j ( t 0 , t 1 ) := ∑ i ∈S F ∑ t 1 -1 τ = t 0 ✶ ( i ∈ A ( τ ) j ∩ F ( τ ) ) is the number of corrupt updates applied to the j -th neuron between iterations t 0 and t 1 .

T ( t 0 , t 1 ) := G ( t 0 , t 1 ) + B ( t 0 , t 1 ) is the total number of updates from all points applied to the entire network between iterations t 0 and t 1 .

We extend all these definitions to the case t 0 = t 1 by letting the empty sum be 0. Finally, we use C ≥ 1 and c ≤ 1 to denote generic, positive constants.

3 Results

The main contributions of this work are Theorem 3.1, Theorem 3.6 and Theorem 3.8, which characterize how the margin of the clean data drives three different training regimes: namely benign overfitting, non-benign (or harmful) overfitting and no-overfitting respectively. We primarily distinguish between the three aforementioned training outcomes based on conditions on the signal strength γ ∈ [0 , 1] which controls the clean margin. Assuming the corrupt points are the minority in the training sample, then heuristically we might expect the following behavior as γ varies: if nγ ≫ 1 , then the signal dominates the noise during training, corrupted points are never fitted and the network generalizes well. If nγ ≪ 1 , then all points are eventually fitted based on their noise component and the network generalizes poorly. As such, we expect to observe benign overfitting when γ is small but not too small: in this regime the network learns the signal, thus ensuring it generalizes well, but corrupted points can still be fitted based on their noise component, thereby allowing training to zero loss.

With each theorem we provide here we give a sketch of its proof: full proofs are contained in the Supplementary Materials, which also contain supporting numerical simulations in Appendix F. Throughout this section, and in order to establish a common setting in which to observe a variety of different behaviors, we make the following assumptions on the network and data hyperparameters.

Assumption 1. For a sufficiently large constant C ≥ 1 , failure probability δ ∈ (0 , 1 / 2) and noise inner product bound ρ ∈ (0 , 1) , let d ≥ Cρ -2 log( n/δ ) , k ≤ cn , λ w ≤ cη and η ≤ ξ , where ξ depends on n , m , k , γ , and d .

̸

We remark that the condition d ≥ Cρ -2 log( n/δ ) ensures the noise components are nearlyorthogonal: in particular, max i = ℓ |⟨ n i , n ℓ ⟩| ≤ cρ with high probability for some positive constant c . This near orthogonality condition on the noise terms is restrictive, but is a common assumption in the related works Frei et al. (2022); Xu & Gu (2023). We note that the value of ρ required for each of our results to hold varies. Likewise, the optimal constants c and C required in each case also vary and we will not concern ourselves with finding the tightest possible constants.

While there are differences the proofs of Theorem 3.1, 3.6 and 3.8 generally fit the following outline.

Use concentration to show with high probability the training data is nearly orthogonal and a certain initialization pattern is satisfied.

Characterize the activation pattern early in training before any point achieves zero loss.

Bound the activations at an iteration just before any training point achieves zero loss.

Based on bounds on the activations at a given iteration, derive an iteration-independent upper bound on the number of subsequent updates that can occur before convergence. At convergence all points either have zero loss or activate no neurons.

We emphasize that our proof techniques are significantly different from those used in Frei et al. (2022); Xu & Gu (2023) due to the differences between the hinge and logistic loss. In particular, letting σ ( z ) denote the logistic loss, a key step in the proof of these prior works is showing at any iteration t ≥ 0 that the ratio σ ′ ( y i f ( t, x i )) /σ ′ ( y l f ( t, x l )) is upper bounded by a constant for all pairs of points i, l in the training sample. For the hinge loss this approach is not feasible: indeed, if at an iteration t some points achieve zero loss while others have not then this ratio is unbounded.

3.1 Benign overfitting

The following theorem states conditions in particular on γ under which the network simultaneously achieves asymptotically optimal test error and achieves zero loss on both the clean and corrupted data after a finite number of iterations. A detailed proof of this Theorem along with the associated lemmas is provided in Appendix C.

Theorem 3.1. Let Assumption 1 hold and further assume n ≥ C log(1 /δ ) , m ≥ C log( n/δ ) , ρ ≤ cγ and C √ log( n/δ ) /d ≤ γ ≤ cn -1 . Then there exists a sufficiently small step-size η such that with probability at least 1 -δ over the randomness of the dataset and network initialization the following hold.

The training process terminates at an iteration T end ≤ Cn η .

For all i ∈ [2 n ] then ℓ ( T end , x i ) = 0 .

The generalization error satisfies

̸

Proof sketch. Recall the parameter ρ bounds the inner products of the noise components of the training data. Specifically, the conditions on d given in Assumption 1 ensure max i = l |⟨ n i , n l ⟩| ≤ ρ 1 -γ with high probability. We also identify the following sets of neurons for p ∈ {-1 , 1 } ,

̸

̸

These sets are useful in that neurons in Γ p have predictable activation patterns during the early phase of training. Furthermore, if i is the index of a corrupted point which activates a neuron in Θ y i at initialization, then this point will continue activating this neuron throughout the early phase of training. Concentration argument shows that Γ p and Θ p are sufficiently significant subsets of [2 m ] p 2 with high probability. In summary, for benign overfitting we say we have a good initialization if i) max i = l |⟨ n i , n l ⟩| ≤ ρ 1 -γ , ii) for some small constant α ∈ (0 , 1) then | Γ p | ≥ (1 -α ) m for p ∈ {-1 , 1 } , and iii) for each i ∈ S F there exists a j ∈ [2 m ] such that ( -1) j = y i and i ∈ A (0) j .

Lemma 3.2. Under the assumptions of Theorem 3.1 and assuming we have a good initialization, suppose at some iteration t 0 the loss of every clean point is bounded above by a ∈ R ≥ 0 , while the loss of every corrupted point is bounded above by b ∈ R ≥ 0 . Then for all t ≥ t 0 the total number of clean and corrupt updates which occur after t 0 are upper bounded as follows,

Because these upper bounds are independent of t then we may conclude that training reaches a steady state after a finite number of iterations. In particular, this means every point either has zero loss or activates no neurons. To prove the network achieves zero loss we need only show that every training point activates at least one neuron after the last training update. This property is simple to prove for clean points: indeed, if i ∈ S T then i activates every neuron in Γ y i after the first iteration. An inductive argument then shows i activates a neuron in every subsequent iteration. Showing that every corrupt point activates a neuron at the end of training is not as simple, and requires a more careful consideration of the training dynamics. To this end we say a neuron is a carrier of a training point between iterations t 0 and t if i ∈ A ( τ ) j for all τ ∈ [ t 0 , t ] . In order to prove the network fits the corrupt data we need to show each corrupt point ( x i , y i ) has a carrier neuron in Θ y i throughout training. If too many clean points activate such a neuron, then it is possible it will eventually cease to carry any corrupt points and if a corrupt point loses all of its carrier neurons then it cannot be fitted. We show this event cannot occur by studying the activation patterns of neurons in Γ := Γ 1 ∪ Γ -1 .

Lemma 3.3. Let the assumptions of Theorem 3.1 hold and suppose we have a good initialization. Let j ∈ Γ and t > 0 be an iteration such that no point achieves zero loss at or before this iteration. For a point i ∈ S T , then i ∈ A ( t ) j iff i ∼ j . For a point i ∈ S F with i ̸∼ j , i ∈ A ( t ) j iff i ∈ A (1) j .

The next lemma bounds the activations just before any points achieve zero loss.

Lemma 3.4. Under the assumptions of Theorem 3.1 and assuming we have a good initialization, there is an iteration T 1 ≤ C ηm [1+( γ + ρ )( n -k )] before any point achieves zero loss where the following hold for a constant that varies from line to line.

For all p ∈ {-1 , 1 } , j ∈ Γ p , i ∼ j , and i ∈ S T , then ⟨ w ( T 1 ) j , x i ⟩ ≥ cm -1 .

For all p ∈ {-1 , 1 } , j ∈ Γ p , i ̸∼ j , and i ∈ S T , then ⟨ w ( T 1 ) j , x i ⟩ ≤ -cnγm -1 .

For all i ∈ S T , then ℓ ( T 1 , x i ) ≤ c .

Due to the fact that clean points are the majority and all of them push the network in the same signal direction, then immediately after T 1 the loss of clean points is small and clean points activate all neurons in the relevant Γ p strongly. Furthermore, once the loss of a clean point is small it stays small. In subsequent iterations, if the number of corrupt updates since T 1 is also small, approximately Cεnγ/ ( η ( γ + ρ ) , then each clean point will activate on all but an ε proportion of neurons in the relevant Γ p . As the hinge loss switches off the updates from a point once it reaches zero loss, eventually clean points do not participate in every iteration. Furthermore, when they do participate their updates are spread over a large proportion of the neurons. This ensures that most neurons in Θ p cannot receive too many clean updates in isolation, thereby ensuring carrier neurons continue to carry corrupted points throughout training.

Lastly, the generalization result follows from the near orthogonality of the noise components of both the training and test data. Indeed, using the same concentration bound, a test point satisfies the same inner product noise condition as the training data with high probability.

Lemma 3.5. Consider a test label y ∈ {-1 , 1 } and point x := y √ γ v + √ 1 -γ n , where n ∼ Uniform ( S d -1 ∩ span { v } ⊥ ) is mutually i.i.d. from the training sample. Assume the conditions of Theorem 3.1 hold and that we have a good initialization. In addition, suppose that |⟨ n , n l ⟩| < ρ 1 -γ for all l ∈ [2 n ] , then yf ( T end , x ) > 0 .

3.2 Non-benign overfitting

The next theorem states a harmful overfitting result: for sufficiently small γ the network achieves again zero loss on both the clean and corrupt data after a finite number of iterations, but the probability of misclassification is bounded from below by a constant. A detailed proof of this Theorem along with the associated lemmas is provided in Appendix D.

Theorem 3.6. Let Assumption 1 hold and further assume m ≥ C log( n/δ ) , ρ ≤ cn -1 , η < 1 / (2 mn ) and γ ≤ c √ nd . Then with probability at least 1 -δ over the randomness of the dataset and network initialization the following hold.

The training process terminates at an iteration T end ≤ Cn η .

For all i ∈ [2 n ] then ℓ ( T end , x i ) = 0 .

The generalization error satisfies

̸

We remark that the above result holds for n ≥ 1 and any k . Indeed, in this regime the noise components dominate the training dynamics and we therefore expect the performance of the network on test points to be close to random. We re-emphasize that, unlike in the data model used by Frei et al. (2022) and Xu & Gu (2023), there does exist a classifier with perfect generalization error for arbitrarily small γ . The significance of Theorem 3.6 is that under the data model considered GD results in a suboptimal classifier.

̸

Proof sketch. Similar to the proof of Theorem 3.1, in the context of non-benign overfitting we say the initialization is 'good" if max i = l |⟨ n i , n l ⟩| ≤ ρ 1 -γ and if each point in the training sample

activates a neuron of the same sign. Under the conditions of Theorem 3.6 it can be shown that a good initialization in this context happens with high probability.

Lemma 3.7. In addition to the conditions of Theorem 3.6, suppose we have a good initialization and that for some iteration t 0 then ℓ ( t 0 , x i ) ≤ a for all i ∈ [2 n ] . Then T ( t 0 , t ) ≤ Cna η .

As for the benign overfitting case, we need to show that each training point activates a neuron after the last training iteration. Under the assumptions on γ it can be shown that the loss of a point decreases during every iteration it participates in, regardless of the status and activations of other points in the training sample. All that remains is to lower bound the generalization error. To this end observe for a test point ( x , y ) that

If the right-hand-side of this equality is negative we can conclude that either x or -x is misclassified. That this event is true with probability lower bounded by a constant in turn follows by appropriately upper bounding the norm of the network weights in the signal subspace, as well as lower bounding the norm of the network weights in the noise subspace.

3.3 No-overfitting

The following theorem illustrates that for γ larger than the upper bound required for benign overfitting, then after convergence, which occurs in a finite number of iterations, only the clean points achieve zero loss. By contrast, the corrupt points cease to activate any neurons and are thus zeroed by the network. The network also achieves asymptotically optimal test error. A detailed proof of this theorem along with the associated lemmas is provided in Appendix E.

Theorem 3.8. Let Assumption 1 hold and further assume m ≥ 2 , n ≥ C log ( m δ ) , ρ ≤ cγ and cn -1 ≤ γ ≤ ck -1 . Then there exists a sufficiently small step-size η such that with probability at least 1 -δ over the randomness of the dataset and network initialization we have the following.

The training process terminates at an iteration T end ≤ Cn η .

For all i ∈ S T then ℓ ( T end , x i ) = 0 while ℓ ( T end , x i ) = 1 for all i ∈ S F .

The generalization error satisfies

̸

We remark that the upper bound on γ allows us to re-deploy the same proof technique used to prove convergence in the benign overfitting case, thereby ensuring the training process converges within a finite number of iterations. We conjecture this upper bound can be relaxed but leave such an analysis to future work.

̸

Proof sketch. In the context of no-overfitting we identify a 'good' initialization as one for which max i = l |⟨ n i , n l ⟩| ≤ ρ 1 -γ and Γ = Γ -1 ∪ Γ +1 = [2 m ] . Under the conditions of Theorem 3.8 it can be shown a good initialization in this context occurs with high probability, furthermore the resulting activation pattern early during training is simple to characterize.

Lemma 3.9. Suppose that the conditions of Theorem 3.8 hold and that we have a good initialization. Consider an arbitrary j ∈ [2 m ] and iteration 2 ≤ t ≤ T 0 occurring before a point has achieved zero loss. Then i ∈ A ( t ) j iff i ∼ j .

Next we bound the activations of the training points just before T 0 , the iteration at which any training points first achieve zero loss. In the following we use F 1 , F 2 and F 3 as placeholders for expressions depending on the data and model parameters. Here, for the sake of conveying the ideas in the proof we do not write them in full and refer the reader to Supplementary Material.

Lemma 3.10. Suppose that the conditions of Theorem 3.8 hold and that we have a good initialization, then there is an iteration T 1 before any point achieves zero loss such that

Next we seek to ensure the activation patterns remain mostly fixed: in particular, we show i ∈ A ( t ) j if i ∈ S T and i ∼ j , while i / ∈ A ( t ) j if i ̸∼ j .

Lemma 3.11. Suppose that the conditions of Theorem 3.8 hold and that we have a good initialization. In addition, for a, b ∈ R assume there is a time t 0 such that ℓ ( t 0 , x i ) ≤ a for all i ∈ S T and ϕ ( ⟨ w ( t 0 ) j , x i ⟩ ) ≤ b for all i ∈ S F and i ∼ j . If i ∈ S T , i ∼ j implies i ∈ A ( τ ) j and i ̸∼ j implies i / ∈ A ( τ ) j for all τ satisfying t 0 ≤ τ < t , then

As before, this update bound is finite and iteration-independent, therefore GD converges provided the assumptions on the activation patterns are not violated. Furthermore, if these activation patterns do hold, then every clean point activates a neuron and no corrupt point activates a neuron of the same label sign. Therefore, under the assumption on the activation pattern, at convergence clean points achieve zero loss while corrupt points have non-zero loss, i.e., they activate no neurons. It therefore suffices to prove the condition on the activation pattern, which we show holds as long as

As C does not depend on the parameters, we can ensure this condition holds by letting Ck ( γ + ρ ) be sufficiently small. With ρ ≤ cn -1 , we show it suffices that γ < ck -1 . Finally, the generalization result follows in a fashion almost identical to that used for Lemma 3.5.

3.4 Comparison of results

We compare the differing regimes of our results side-by-side with those of Frei et al. (2022); Xu & Gu (2023) in Table 1. We note that comparisons are not like-for-like as Frei et al. (2022) consider smooth, leaky ReLU and logistic loss, Xu & Gu (2023) a generalized family of activation functions, which includes ReLU, and logistic loss, and this paper ReLU and hinge loss. Furthermore, in addition to differences in the noise distribution discussed in Section 2.1, Frei et al. (2022); Xu & Gu (2023) assume a data model where the norm of each data point is approximately proportional to √ d . We therefore re-scale their results in order to make comparison with this work in which all data points have unit norm.

Taken together these results suggest, at least under the type of data model considered, that benign overfitting occurs for signal strengths proportional to between roughly 1 / √ dn and 1 /n . Furthermore, our results also suggest that above approximately 1 /n one might expect to see a transition to nooverfitting, while below approximately 1 / √ nd a transition to harmful overfitting. We provide preliminary supporting experiments in the Supplementary Material. We again remark that the latter is non-trivial in our setting as for all γ > 0 the classifier h ( x ) = sign( ⟨ v , x ⟩ ) always has perfect accuracy.

4 Conclusion

Developing a theoretical description of benign overfitting in neural networks is a highly nascent area, with mathematical results available only for very limited data models. Furthermore, the conditions describing the transitions between overfitting versus non-overfitting and benign versus non-benign even in these simplified settings are yet to be fully characterized. The goal of this work was to address this issue as well as explore the impact of using the hinge loss. In particular, and admittedly for a simple data model, we prove three different training outcomes, corresponding to non-benign overfitting, benign overfitting and no-overfitting, based on conditions on the margin of the clean data. Our analysis also differs significantly from prior works due to the fact the ratio of loss between different training points can be unbounded and the implicit bias of using hinge loss versus exponentially tailed loss is poorly understood.

Limitations and future work: the key limitation of this work is the restrictiveness of the data model. In particular, as in prior and related works we use a near-orthogonal noise model and assume a rank one signal, we also place additional conditions on the noise distribution. In addition to generalizing the signal and noise model as well as improving the bounds required for our results to hold, we believe the following themes are important areas for future research: first relaxing the near orthogonal noise condition, second exploring data models beyond those which are linearly separable, third investigating the role and impact of depth.

Acknowledgments

EG, WS and DN were partially supported by NSF DMS 2011140 and NSF DMS 2108479. EG was also partially supported by NSF DGE 2034835.

References

Ben Adlam and Jeffrey Pennington. The neural tangent kernel in high dimensions: Triple descent and a multi-scale theory of generalization. In International Conference on Machine Learning , 2020.

Hilal Asi and John C. Duchi. Stochastic (approximate) proximal point methods: Convergence, optimality, and adaptivity. SIAM Journal on Optimization , 29(3):2257-2290, 2019. doi: 10.1137/ 18M1230323. URL https://doi.org/10.1137/18M1230323 .

Keith Ball. An elementary introduction to modern convex geometry. 1997.

Rémi Bardenet and Odalric-Ambrym Maillard. Concentration inequalities for sampling without replacement. 2015.

Peter L. Bartlett, Philip M. Long, Gábor Lugosi, and Alexander Tsigler. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences , 117(48):30063-30070, 2020. doi: 10.1073/pnas.1907378117. URL https://www.pnas.org/doi/abs/10.1073/pnas. 1907378117 .

Mikhail Belkin, Daniel J Hsu, and Partha Mitra. Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018a. URL https://proceedings.neurips.cc/paper_files/paper/2018/ file/e22312179bf43e61576081a2f250f845-Paper.pdf .

Mikhail Belkin, Siyuan Ma, and Soumik Mandal. To understand deep learning we need to understand kernel learning. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pp. 541-549. PMLR, 10-15 Jul 2018b. URL https://proceedings.mlr.press/v80/ belkin18a.html .

Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. Reconciling modern machine-learning practice and the classical bias-variance trade-off. Proceedings of the National Academy of Sciences , 116(32):15849-15854, 2019.

Alon Brutzkus, Amir Globerson, Eran Malach, and Shai Shalev-Shwartz. SGD learns overparameterized networks that provably generalize on linearly separable data. In International Conference on Learning Representations , 2018. URL https://openreview.net/forum? id=rJ33wwxRb .

Yuan Cao, Quanquan Gu, and Mikhail Belkin. Risk bounds for over-parameterized maximum margin classification on sub-gaussian mixtures. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems , volume 34, pp. 8407-8418. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/ file/46e0eae7d5217c79c3ef6b4c212b8c6f-Paper.pdf .

Yuan Cao, Zixiang Chen, Misha Belkin, and Quanquan Gu. Benign overfitting in two-layer convolutional neural networks. Advances in neural information processing systems , 35:25237-25250, 2022.

Niladri S. Chatterji and Philip M. Long. Finite-sample analysis of interpolating linear classifiers in the overparameterized regime. Journal of Machine Learning Research , 22(129):1-30, 2021. URL http://jmlr.org/papers/v22/20-974.html .

Niladri S. Chatterji and Philip M. Long. Foolish crowds support benign overfitting. Journal of Machine Learning Research , 23(125):1-12, 2022. URL http://jmlr.org/papers/v23/ 21-1199.html .

Spencer Frei, Niladri S Chatterji, and Peter Bartlett. Benign overfitting without linearity: Neural network classifiers trained by gradient descent for noisy linear data. In Conference on Learning Theory , pp. 2668-2703. PMLR, 2022.

Spencer Frei, Gal Vardi, Peter Bartlett, and Nathan Srebro. Benign overfitting in linear classifiers and leaky relu networks from kkt conditions for margin maximization. In Gergely Neu and Lorenzo Rosasco (eds.), Proceedings of Thirty Sixth Conference on Learning Theory , volume 195 of Proceedings of Machine Learning Research , pp. 3173-3228. PMLR, 12-15 Jul 2023. URL https://proceedings.mlr.press/v195/frei23a.html .

Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J. Tibshirani. Surprises in highdimensional ridgeless least squares interpolation. The Annals of Statistics , 50(2):949 - 986, 2022. doi: 10.1214/21-AOS2133. URL https://doi.org/10.1214/21-AOS2133 .

Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/ paper/2018/file/5a4be1fa34e62bb8a6ec6b91d2462f5a-Paper.pdf .

Ziwei Ji and Matus Telgarsky. Directional convergence and alignment in deep learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems , NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.

Ziwei Ji, Miroslav Dudík, Robert E. Schapire, and Matus Telgarsky. Gradient descent follows the regularization path for general losses. In Jacob Abernethy and Shivani Agarwal (eds.), Proceedings of Thirty Third Conference on Learning Theory , volume 125 of Proceedings of Machine Learning Research , pp. 2109-2136. PMLR, 09-12 Jul 2020. URL https://proceedings.mlr. press/v125/ji20a.html .

Frederic Koehler, Lijia Zhou, Danica J. Sutherland, and Nathan Srebro. Uniform convergence of interpolators: Gaussian width, norm bounds and benign overfitting. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems , 2021. URL https://openreview.net/forum?id=FyOhThdDBM .

Guy Kornowski, Gilad Yehudai, and Ohad Shamir. From tempered to benign overfitting in relu neural networks, 2023.

Yiwen Kou, Zixiang Chen, Yuanzhou Chen, and Quanquan Gu. Benign overfitting for two-layer relu networks, 2023.

Tengyuan Liang and Alexander Rakhlin. Just interpolate: Kernel 'Ridgeless' regression can generalize. The Annals of Statistics , 48(3):1329 - 1347, 2020. doi: 10.1214/19-AOS1849. URL https://doi.org/10.1214/19-AOS1849 .

Tengyuan Liang, Alexander Rakhlin, and Xiyu Zhai. On the multiple descent of minimum-norm interpolants and restricted lower isometry of kernels. In Annual Conference Computational Learning Theory , 2019.

Kaifeng Lyu and Jian Li. Gradient descent maximizes the margin of homogeneous neural networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net, 2020. URL https://openreview.net/forum?id= SJeLIgBKPS .

Neil Rohit Mallinar, James B Simon, Amirhesam Abedsoltan, Parthe Pandit, Misha Belkin, and Preetum Nakkiran. Benign, tempered, or catastrophic: Toward a refined taxonomy of overfitting. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems , 2022. URL https://openreview.net/forum? id=5oS20NUCJEX .

Song Mei and Andrea Montanari. The generalization error of random features regression: Precise asymptotics and the double descent curve. Communications on Pure and Applied Mathematics , 75, 2019.

Vidya Muthukumar, Kailas Vodrahalli, Vignesh Subramanian, and Anant Sahai. Harmless interpolation of noisy data in regression. IEEE Journal on Selected Areas in Information Theory , 1(1): 67-83, 2020. doi: 10.1109/JSAIT.2020.2984716.

Vidya Muthukumar, Adhyyan Narang, Vignesh Subramanian, Mikhail Belkin, Daniel Hsu, and Anant Sahai. Classification vs regression in overparameterized regimes: Does the loss function matter? J. Mach. Learn. Res. , 22(1), Jan 2021. ISSN 1532-4435.

Li S. Concise formulas for the area and volume of a hyperspherical cap. Asian Journal of Mathematics and Statistics , 4, 01 2011. doi: 10.3923/ajms.2011.66.70.

Ohad Shamir. The implicit bias of benign overfitting. In Po-Ling Loh and Maxim Raginsky (eds.), Proceedings of Thirty Fifth Conference on Learning Theory , volume 178 of Proceedings of Machine Learning Research , pp. 448-478. PMLR, 02-05 Jul 2022. URL https://proceedings.mlr. press/v178/shamir22a.html .

Gang Wang, Georgios B. Giannakis, and Jie Chen. Learning ReLU networks on linearly separable data: Algorithm, optimality, and generalization. IEEE Transactions on Signal Processing , 67(9): 2357-2370, 2019. doi: 10.1109/TSP.2019.2904921.

Guillaume Wang, Konstantin Donhauser, and Fanny Yang. Tight bounds for minimum ℓ 1 -norm interpolation of noisy data. In International Conference on Artificial Intelligence and Statistics , 2021a.

Ke Wang, Vidya Muthukumar, and Christos Thrampoulidis. Benign overfitting in multiclass classification: All roads lead to interpolation. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems , volume 34, pp. 24164-24179. Curran Associates, Inc., 2021b. URL https://proceedings.neurips.cc/paper_files/paper/2021/ file/caaa29eab72b231b0af62fbdff89bfce-Paper.pdf .

Denny Wu and Ji Xu. On the optimal weighted ℓ 2 regularization in overparameterized linear regression. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 10112-10123. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/ 2020/file/72e6d3238361fe70f22fb0ac624a7072-Paper.pdf .

Xingyu Xu and Yuantao Gu. Benign overfitting of non-smooth neural networks beyond lazy training. In Francisco Ruiz, Jennifer Dy, and Jan-Willem van de Meent (eds.), Proceedings of The 26th International Conference on Artificial Intelligence and Statistics , volume 206 of Proceedings of Machine Learning Research , pp. 11094-11117. PMLR, 25-27 Apr 2023. URL https: //proceedings.mlr.press/v206/xu23k.html .

Qiuling Yang, Alireza Sadeghi, Gang Wang, and Jian Sun. Learning two-layer ReLU networks is nearly as easy as learning linear classifiers on separable data. IEEE Transactions on Signal Processing , 69:4416-4427, 2021. doi: 10.1109/TSP.2021.3094911.

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations , 2017.

Difan Zou, Jingfeng Wu, Vladimir Braverman, Quanquan Gu, and Sham Kakade. Benign overfitting of constant-stepsize SGD for linear regression. In Mikhail Belkin and Samory Kpotufe (eds.), Proceedings of Thirty Fourth Conference on Learning Theory , volume 134 of Proceedings of Machine Learning Research , pp. 4633-4635. PMLR, 15-19 Aug 2021. URL https://proceedings.mlr.press/v134/zou21a.html .

Appendix A Properties of the data and network at initialization

For each of our results to hold we require certain properties on both the network weights and training sample to hold at initialization. Here we bound the probabilities of these events in turn. Later, for each specific setting we combine the relevant conditions using the union bound.

First, and in order to prove convergence, we require the noise components of the training sample to be approximately orthogonal to one another.

Lemma A.1. Let ρ, δ ∈ (0 , 1) . Given a sequence ( n i ) 2 n i =1 of mutually i.i.d. random vectors with n i ∼ U ( S d -1 ∩ span ( v ) ⊥ ) for all i ∈ [2 n ] , then assuming d ≥ max { 3 , 3 ρ -2 ln ( 2 n 2 δ )}

̸

Proof. Consider two pairs of mutually i.i.d. random vectors n , n ′ ∼ U ( S d -1 ∩ span ( v ) ⊥ ) and u , u ′ ∼ U ( S d -2 ) , observe

Due to the fact that u and u ′ are independent as well as the rotational invariance of U ( S d -2 ) it follows that

where e 1 := [1 , 0 .. 0] T . Let Cap ( e 1 , ρ ) := { z ∈ S d -2 : ⟨ e 1 , z ⟩ ≥ ρ } denote the spherical cap of S d -2 centered on e 1 . As d ≥ 3 then from Ball (1997)[Lemma 2.2] it follows that

Applying the union bound

̸

̸

Setting δ ≥ 2 n 2 exp ( -dρ 2 3 ) and rearranging we arrive at the result claimed.

In addition to requiring the approximate orthogonality property on the training data, our approach also requires a large proportion of the neurons at initialization to satisfy particular conditions in regard to the number of clean versus corrupt activations. To this end, we introduce the following terms where p ∈ {-1 , 1 } .

Let Γ p := { j : ( -1) j = p, G j (0 , 1)( γ -ρ ) -B j (0 , 1)( γ + ρ ) ≥ 2 λ w η } denote the set of neurons with output weight ( -1) p which have more clean points activating them than corrupt ones at initialization. We will show that these sets of neurons have a predictable behavior early during training before any clean points achieve zero loss. We further let Γ = Γ 1 ∪ Γ -1 .

Let Θ p := { j ∼ Γ p : G j (0 , 1)( γ + ρ ) -B j (0 , 1)( γ -ρ ) < 1 -γ + ρ } ⊂ Γ p . For our benign overfitting result we will show that neurons in this subset are able to carry corrupt points throughout training, eventually, at least in the overfitting setting, enabling them to achieve zero loss. We further let Θ = Θ 1 ∪ Θ -1 .

First we show Γ accounts for a significant proportion of neurons. To this end we first provide the following result.

Lemma A.2. Define µ := 2 k n + k and assume κ ∈ (0 , 1) satisfies κ > µ . Given an arbitrary neuron w j ∼ U ( S d -1 ) , we say that a collection of training points is ( ε, κ ) -good iff both T j (0 , 1) ≥ 1 and B j (0 , 1) < κT j (0 , 1) with probability at least 1 -ϵ over the randomness of the neuron. There exist positive constants C, c such that if δ := exp( -cn ( κ -µ 2 )) and n ≥ C then with probability at least 1 -δ ϵ the training sample is ( ε, κ ) -good.

Proof. First we establish certain pieces of notation specific to what follows: we say a point x is positive iff ⟨ x , v ⟩ > 0 and is negative iff ⟨ x , v ⟩ < 0 . We use S + and S -to denote these sets of points respectively. Note by construction, see (1), clean and corrupt points of the same sign are mutually i.i.d. As here we only ever consider one neuron and the activations of the training sample on this neuron at initialization, we also drop both the subscript j as well as the argument parentheses on the counting functions. We also use ± superscripts to denote the subsets corresponding to activations from positive and negative points respectively: as examples T is used as shorthand for the total number of activations, B + is the number corrupt positive activations and G -is the number of clean negative activations.

First by the symmetry of the distribution of w , P ( ⟨ w , v ⟩ > 0) = P ( ⟨ w , v ⟩ < 0) = 1 2 . As a result

As the analysis and results derived under either condition will prove identical under reversal of the signs involved, without loss of generality we let ⟨ w , v ⟩ > 0 . Using the union bound

P (( B < κT ) ∩ ( T > 0) | ⟨ w , v ⟩ > 0) ≥ 1 -P ( T = 0 | ⟨ w , v ⟩ > 0) -P ( B ≥ κT | ⟨ w , v ⟩ > 0) , therefore it suffices to upper bound the two probabilities on the right-hand-side.

Observe if ⟨ w , v ⟩ > 0 then for x ∈ S + we have P ( ⟨ x , w ⟩ ) > 1 / 2 and for x ∈ S -we have P ( ⟨ x , w ⟩ ) < 1 / 2 . By the mutual independence of the preactivations ( ⟨ x , w j ⟩ ) 2 n i =1 then P ( T = 0 | ⟨ w , v ⟩ > 0) ≤ (1 / 2) n . Consider now a slightly different data model, in which a training sample consists of n -k clean positive points and 2 k corrupt positive points. Abusing notation, we let ζ denote the event that we are instead drawing our training sample in this manner and also that ⟨ w , v ⟩ > 0 . In this setting T + = T and furthermore the event B < κT is equivalent to B + < κT + . Again, as the preactivations are mutually independent the number of positive activations can be lower bounded using a binomial distribution with probability 1 / 2 . Applying a Chernoff bound it follows that

Furthermore, observe sampling positive points which activate w j is equivalent to uniformly sampling without replacement T + points from S + . Let Z ℓ = 1 iff the ℓ -th element sampled from S + is corrupt and is 0 otherwise. Using a variant of Hoeffding's bound for sampling without replacement (see for example Proposition 1.2 of Bardenet & Maillard (2015))

Therefore

Combining these results it follows that

Therefore, there exist constants C, c > 0 such that if n ≥ C then

Note if instead we condition on the event ⟨ x , w ⟩ < 0 then swapping the roles of the negative and positive points in the argument above gives the same outcome. As a result

For convenience let X := ( x i ) 2 n i =1 denote the training sample and X c ϵ,κ := { X : P w (( B ≥ κT ) ∪ ( T = 0)) > ϵ } the set of training samples which are not ( ϵ, κ ) -good. Note here that the subscript w indicates randomness over the neuron alone, furthermore by construction

Furthermore, as

then it follows that P ( X ∈ X c ϵ,κ ) ≤ δ ϵ . As a result we conclude that the probability of drawing a ( ϵ, κ ) -good training sample is at least 1 -δ ϵ .

Based on Lemma A.2, the following lemma bounds the probability that the cardinality of Γ p is large. We note that the result presented here on non-overfitting requires | Γ p | = m while the result on benign overfitting that | Γ p | ≥ (1 -α ) m for some small constant α ∈ (0 , 1) .

Lemma A.3. Suppose n ≥ 15 k , λ w ≤ η γ -ρ 4 γ and γ ≥ 4 ρ . Then there exist positive constants C, c > 0 such that if n ≥ C the following are true.

With α ∈ (0 , 1) then

Proof. Again as here we only ever consider the activations at initialization, we write T j (0 , 1) as T j , G j (0 , 1) as G j and B j (0 , 1) as B j . Let p ∈ {-1 , 1 } and consider an arbitrary neuron j such that ( -1) j = p , by definition if

we may conclude j ∈ Γ p . Rearranging this expression, equivalently j ∈ Γ p if

As λ w η ≤ γ -ρ 4 γ , then membership to Γ p is guaranteed as long as T j ≥ 1 and B j ≤ γ -ρ 4 γ T j . Note by the assumptions of the lemma µ := 2 k n + k ≤ 1 8 and γ -ρ 4 γ ≥ 3 16 . Conditioning on the event we draw a ( ε, 3 16 ) -good training sample then the probability that j / ∈ Γ p is at most ϵ by Lemma A.2. Furthermore, with the training sample fixed the activations of each neuron are mutually independent. Let X = ( x i ) 2 n i =1 denote the draw of the training sample and X ϵ,κ = { X : P w (( B ≥ κT ) ∪ ( T = 0)) ≤ ϵ } the set of ( ε, κ ) -good training samples. In what follows we assume κ = 3 / 16 and let ϵ = exp( -cn ) where c < 16 -2 is a sufficiently small positive constant, then by Lemma A.2 there exist constants C, c such that if n ≥ C

The first result follows by applying the union bound,

For the second result, let α ′ ∈ (0 , 1) denote the smallest scalar satisfying both α < α ′ and α ′ m ∈ [ m ] . Observe

As α is a constant, there exists positive constants C, c such that if n ≥ C then

Therefore as claimed.

We now turn our attention to establishing the conditions required at initialization on the corrupt points for the result on benign overfitting. To this end, in the following two lemmas we introduce the notion of an ϵ -fine training sample and lower bound the probability of drawing one. Then, by conditioning on drawing such a training sample, we lower bound the cardinality of a set of neurons, which we denote Λ , which satisfy a property related to Θ .

Lemma A.4. Assume γ ≤ 4 5 n and γ ≥ 5 ρ . We say a training sample is ϵ -fine if for a random neuron w j the inequality G j < 4 10 T j + 5 8 n holds with probability at least 1 -ϵ . There exist constants C, c > 0 such that if n ≥ C then with probability at least 1 -ϵ -1 exp( -cn ) the training sample is ϵ -fine.

Proof. This lemma is analogous to Lemma A.2 and to this end we reuse much of the same notation. In particular, recall a point x is positive iff ⟨ x , v ⟩ > 0 and is negative iff ⟨ x , v ⟩ < 0 . Note all points with the same sign are mutually i.i.d. by construction. We use S + and S -to denote these sets of positive and negative points respectively. As here we only consider a single random neuron w j and the activations at initialization of the training sample on it, we also drop both the subscript j as well as the argument parentheses on the counting functions. We also use ± superscripts to denote the subsets corresponding to activations from clean and corrupt points. As indicative examples of our notation going forward, we denote w j as w , T is used as shorthand for the total number of activations while B + and G -are the number of corrupt positive and clean negative activations respectively.

By the symmetry of the distribution of w , P ( ⟨ w , v ⟩ > 0) = P ( ⟨ w , v ⟩ < 0) = 1 2 . As a result

As the analysis and results derived under either condition will prove identical under reversal of the signs involved, without loss of generality we let ⟨ w , v ⟩ > 0 . Consider this problem for a slightly different data model, in which a training sample consists of 2( n -k ) clean, positive points and k negative points. Abusing notation, we let ζ denote the event that we are instead drawing our training sample in this manner and also that ⟨ w , v ⟩ > 0 . Clearly

In this setting, only positive points activate w , therefore all points which activate w are identically distributed. As a result, sampling positive points which activate w is equivalent to uniformly sampling without replacement T + points from S + . Let Z ℓ = 1 if the ℓ -th element sampled from S + is clean

and is 0 otherwise. Define µ = 2( n -k ) 2 n -k , then again using a variant of Hoeffding's bound for sampling without replacement (Bardenet & Maillard, 2015)[Proposition 1.2] and as long as 4 10 + 5 n 8 T + > µ , we have

We proceed to lower and upper bound T + , to this end we first lower and upper bound the probability that a positive point activates w conditioned on the event ⟨ w , v ⟩ > 0 . Let x = √ γ v + √ 1 -γ n be a fixed positive point, then by the symmetry of the distribution of w

For convenience, let E 1 denote the event ⟨ w , x ⟩ > 0 , E 2 the event ⟨ w , v ⟩ > 0 and E 3 the event |⟨ w , v ⟩| ≤ φ for some arbitrary φ ∈ (0 , 1) . For the upper bound observe

Let Cap ( n , φ ) := { z ∈ S d -1 : ⟨ n , z ⟩ ≥ φ } denote the spherical cap of S d -1 centered on n . As d ≥ 3 and w ∼ U ( S d -1 ) then from Ball (1997)[Lemma 2.2] it follows that

Furthermore

Therefore, and noting under the assumptions of the lemma that γ 1 -γ ≤ 4 n ,

Set φ 2 = 3 √ n d , then

Letting ω ∈ (0 , 1 / 2) be an arbitrary constant, then as long n ≥ max { ln 2 ( 1 ω ) , 16 ln -2 ( 1 1 -ω ) }

In order to take advantage of concentration the upper bound on T + must be greater than E [ T + ] . On the other hand, if the upper bound is too large then the condition 4 10 + 5 n 8 T + j > µ will be compromised.

As µ < 1 if then this condition is not compromised. Setting ω = 1 48 , if n ≥ 16 ln -2 ( 48 47 ) =: C then

and therefore, applying a Chernoff bound,

Under the same conditions, to lower bound T + we again apply a Chernoff, which gives

Therefore, there exists a small positive constant c and a large constant positive constant C such that if n ≥ C then

In combination with the bound in (4), from this result it follows that there also exists a sufficiently small constant c > 0 such that

As a result, there exist positive constants C, c such that if n ≥ C then

Note if instead ⟨ x , v ⟩ < 0 , then swapping the roles of the negative and positive points in the argument outlined above gives the same result. Therefore, under the assumptions of the lemma,

Let X = ( x i ) 2 n i =1 denote the training sample and X c ϵ = { X : P w ( G ≥ 4 10 T + 5 8 n ) > ϵ } the set of training samples which are not ϵ -fine. Note the subscript w above indicates randomness over the neuron w alone. Clearly by construction

Furthermore, as

then it follows that P ( X ∈ X c ϵ ) ≤ ϵ -1 exp( -cn ) . As a result we conclude that there exist positive constants C, c such that if n ≥ C then the probability of drawing an ϵ -fine training sample is at least 1 -ϵ -1 exp( -cn ) .

Lemma A.5. Assume γ ≤ 4 5 n , γ ≥ 5 ρ and let Λ := { j ∈ [2 m ] : G j < γ -ρ 2 γ T j + 1 2 γ } . There exists a positive constants C such that for any δ ∈ (0 , 1) if n ≥ C log(1 /δ ) then

Proof. To bound the size of Λ with high probability we follow a similar approach used to bound the size of Γ p with high probability. As γ ≤ 4 5 n and ρ ≤ γ 5 , then observe

Therefore, j ∈ Λ if G j < 4 10 T j + 5 8 n . Conditioned on the event that the training sample is ϵ -fine for ϵ := exp( -cn ) for some sufficiently small constant c , then with the data fixed the preactivations of the data on each neuron are mutually independent and identically distributed by construction. As such, in this setting the events ( G j < 4 10 T j + 5 8 n ) 2 j m =1 are also mutually independent. Let X = ( x i ) 2 n i =1 denote the training sample and X ϵ the set of ϵ -fine training samples, then

It follows that there exists a sufficiently small positive constant c such that

Therefore, using Lemma A.4 there exists a sufficiently small positive constant c such that

from which the result claimed follows.

Lemma A.6. Assume γ ≤ 4 5 n , γ ≥ 5 ρ and | Γ p | > 0 . 99 m for p ∈ {-1 , 1 } . There exists a positive constant C such that for any δ ∈ (0 , 1) , if n ≥ C log( 1 δ ) and m ≥ C log ( k δ ) , then with probability at least 1 -δ for all i ∈ S F there exists a j ∈ Θ y i for which i ∈ A (0) j .

Proof. For convenience here we use T j , G j and B j for T j (0 , 1) , G j (0 , 1) and B j (0 , 1) respectively. For a neuron to be in Θ p it must satisfy the following condition,

Adding and subtracting T j ( γ -ρ ) to the left-hand-side gives

Rearranging this inequality it follows that the conditions j ∈ Γ p and G j < γ -ρ 2 γ T j + 1 2 γ are sufficient to conclude j ∈ Θ p . Let Λ := { j ∈ [2 m ] : G j < γ -ρ 2 γ T j + 1 2 γ } . Therefore, in order to prove the desired result it suffices to lower bound the probability that for each corrupt point ( x i , y i ) , the intersection between the set of neurons which x i activates, the set of neurons Γ y i and the set of neurons Λ is nonempty.

By a Chernoff bound there exists a small constant c > 0 such that with probability at least 1 -exp( -cm ) a fixed training point is activated by at least 1 / 3 of the neurons of each sign. Therefore, using the union bound, every corrupt training point is activated by at least m/ 3 of the neurons with matching sign with probability at least 1 -2 k exp( -cm ) . Conditioning on this event, then under the assumption | Γ p | > 0 . 99 m for p ∈ {-1 , 1 } , each corrupt point ( x i , y i ) activates at least 97 300 m neurons in Γ y i . Therefore, if for instance, | Λ | > 1 . 75 m , we can conclude for each ( x i , y i ) with i ∈ S F that there exists a j ∈ Θ y i such that i ∈ A (0) j . Therefore, under the conditions of the lemma, using the union bound and Lemmas A.5 and A.3, we can upper bound the failure probability of this as

Therefore, for δ ∈ (0 , 1) there exists a positive constant C such that if n ≥ C log( 1 δ ) and m ≥ C log ( k δ ) then the probability that for all i ∈ S F there exists a j ∈ Θ y i such that i ∈ A (0) j is at least 1 -δ .

The final lemma we provide here states, under mild conditions on the network width, that with high probability every point in the training sample activates a neuron whose output weight matches its label in sign. We use this to prove the result on non-benign overfitting, detailed in Section D.

Lemma A.7. Let δ ∈ (0 , 1) , if m ≥ log 2 ( 2 n δ ) then the probability that for all i ∈ [2 n ] there exists a j ∈ [2 m ] such that ( -1) j = y i and i ∈ A (0) j is at least 1 -δ .

Proof. Observe by the rotational symmetry of the weight distribution that for any j ∈ [2 m ]

By construction, for each element in the training sample ( x i , y i ) 2 n i =1 there are m neurons whose output weight has the same sign. As the preactivations of x i with each neuron are mutually independent from one another, then using the union bound it follows that

Setting δ ≥ 2 n 2 -m and rearranging we arrive at the stated result.

Appendix B Supporting Lemmas

B.1 Bounds on activations and preactivations

For any pair of iterations t, t 0 satisfying t > t 0 , unrolling the GD update rule (3) gives

Using (1) and the fact that n i ⊥ v for any i ∈ [2 n ] , then

where we define λ iℓ := ( -1) ℓ + i β ( i ) β ( ℓ ) ⟨ x ℓ , x i ⟩ . Towards the goal of bounding the activation of a neuron with a data point we provide the following results.

Therefore

̸

̸

̸

Lemma B.1. Assume |⟨ n i , n ℓ ⟩| ≤ ρ 1 -γ for all i, ℓ ∈ [2 n ] such that i = ℓ .

̸

̸

Proof. Observe by the data model that

from which the results claimed follow.

Lemma B.2. Assume |⟨ n i , n ℓ ⟩| ≤ ρ 1 -γ for all i, ℓ ∈ [2 n ] such that i = ℓ . Then for any j ∈ [2 m ] the following are true.

̸

If i ∈ S , i ̸∼ j

If i ∈ S , i ̸∼ j

Proof. Considering (5) we can further separate the summation term as follows,

̸

̸

̸

Note, with i ∈ S T and i ∼ j , or i ∈ S F and i ̸∼ j , then ( -1) j + i β ( i ) = 1 . On the other hand, with i ∈ S T and i ̸∼ j , or i ∈ S F and i ∼ j , then ( -1) j + i β ( i ) = -1 . Substituting the relevant bounds on λ iℓ provided in Lemma B.1, and observing by definition that G ( i ) j ( t 0 , t ) = ∑ ℓ ∈S T ,ℓ = i T ℓj ( t 0 , t ) and B ( i ) j ( t 0 , t ) = ∑ ℓ ∈S T ,ℓ = i T ℓj ( t 0 , t ) , one arrives at the results claimed.

̸

We will often make use of the following similar but more pessimistic bounds on the activations. Recall that ϕ is the ReLU function: ϕ ( a ) = max { a, 0 } .

Lemma B.3. For any j ∈ [2 m ] and iterations t 0 , t with t 0 ≤ t the following hold:

If i ∈ S T , i ̸∼ j then

If i ∈ S F , i ∼ j then

The η term in the upper bound for cases 2 and 3 is only necessary if T ij ( t 0 , t ) > 0 .

We remark that we will often use this result in a setting where ρ ≤ γ . In these cases, the terms that involve ϕ ( ρ -γ ) are zero and will be dropped.

Proof. For each of these results, we make use of Lemma B.2, a ≤ ϕ ( a ) for all a ∈ R , and

for all i, j, t 0 , t 1 . We will only prove the inequalities for i ∈ S T here, as the inequalities for i ∈ S F are analogous.

For the first inequality in Statement 1 we claim it suffices to show

Indeed, if (6) is true then the result claimed follows as

In order to prove (6) we bound

This follows from Statement 1 in Lemma B.2. From here, we consider two cases: first, if ⟨ w ( τ ) j , x i ⟩ ≥ 0 then ⟨ w ( τ ) j , x i ⟩ = ϕ ( ⟨ w ( τ ) j , x i ⟩ ) and so (6) clearly holds. Alternatively, if ⟨ w ( τ ) j , x i ⟩ < 0 then T ij ( τ, τ +1) = 0 , ϕ ( ⟨ w ( τ ) j , x i ⟩ ) = 0 and as a result the right-hand-side of (6) is non-positive while the left is non-negative. As such (6) holds trivially.

For the second equality in Statement 1 we bound

Since the right-hand side is non-negative, this inequality is true even if we replace the left-hand side by ϕ ( ⟨ w ( t ) j , x i ⟩ ) .

We now proceed to Statement 2. For the first inequality, notice that if ϕ ( ⟨ w ( t 0 ) j , x i ⟩ ) = 0 then the right-hand side is non-positive and therefore the inequality trivially holds. Otherwise, it must be the case that ϕ ( ⟨ w ( t 0 ) j , x i ⟩ ) = ⟨ w ( t 0 ) j , x i ⟩ . Using Statement 2 from Lemma B.2, we obtain the bound

We now turn to the second inequality in Statement 2. The corresponding statement from Lemma B.2 yields

we remark that the reason for the addition of η to the right-hand-side will soon become apparent. The desired inequality holds as long as the right-hand-side is non-negative, we therefore proceed by induction to prove

for τ ≥ t 0 . The base case τ = t 0 is trivial, assume then that the induction hypothesis holds for some τ ≥ t 0 . For iteration τ +1 there are two cases to consider: first, if ⟨ w ( τ ) j , x i ⟩ < 0 then T ij ( t 0 , τ +1) = T ij ( t 0 , τ ) . In addition, as B j ( t 0 , τ ) ≤ B j ( t 0 , τ +1) and G ( i ) j ( t 0 , τ ) ≤ G ( i ) j ( t 0 , τ +1) then

by the induction hypothesis. Alternatively, if instead ⟨ w ( τ ) j , x i ⟩ ≥ 0 one may use the second inequality from (7) to conclude that

In addition, as T ij ( t 0 , τ +1) ≤ T ij ( t 0 , τ ) + 1 it follows that

which completes the induction.

Lastly, we consider the final remark in the statement of the lemma: if T ij ( t 0 , t ) = 0 then the right hand side of the second line in (7) is non-negative trivially, so we do not need the additional η term.

B.2 Convergence of training

We say that GD terminates if it reaches a finite iteration in which a zero update is applied to the network parameters. The following lemmas are used to show that GD terminates by in turn upper bounding the number of clean and corrupt updates. The first lemma facilitates the bounding of the hinge loss of clean and corrupt points.

Lemma B.4. For any iterations t, t 0 satisfying t ≥ t 0 ,

Proof. Both statements follow from the bounds provided in Lemma B.3. For Statement 1

For Statement 2

The following lemma bounds the number of updates of corrupt and clean points in an interval of iterations in terms of their hinge loss at the beginning of the interval as well as the number of clean and corrupt updates.

Lemma B.5. Let t ≥ t 0 . For i ∈ S T ,

For i ∈ S F ,

Proof. We will show this for i ∈ S T ; the i ∈ S F case is analogous but with the roles of corrupt and clean points reversed. We proceed by induction on t and assume ℓ ( t 0 , x i ) ≤ a . If t = t 0 this holds trivially because the left-hand side is zero and the right-hand side is positive. Otherwise, assume the inequality holds at iteration t . By Lemma B.4 and our assumption on ℓ ( t 0 , x i ) ,

We consider two cases:

If η ( T i ( t 0 , t ) -( γ + ρ ) B ( t 0 , t ) -ϕ ( ρ -γ ) G ( i ) ( t 0 , t ) -m ) ≥ a then we see that ℓ ( t, x i ) = 0 . Therefore,

Otherwise, T i ( t 0 , t ) ≤ a η +( γ + ρ ) B ( t 0 , t ) + ϕ ( ρ -γ ) G ( i ) ( t 0 , t ) + m . Since there are only 2 m neurons, we bound

Appendix C Benign overfitting

Assumption 2. With δ, ρ ∈ (0 , 1) and C a generic, positive constant, then we assume the following conditions on the data and model hyperparameters.

In addition to the assumptions detailed in Assumption 2, in our analysis we use three further conditions.

Assumption 3. Let ρ ∈ (0 , 1) satisfy γ ≥ 5 ρ . In addition to the assumptions detailed in Assumption 2, assume that the following conditions hold.

| Γ p | > 0 . 99 m for p ∈ {-1 , 1 } .

For all i ∈ S F there is j ∈ Γ y i such that i ∈ A (0) j .

̸

For all i, l ∈ [2 n ] , i = l then |⟨ n i , n l ⟩| ≤ ρ 1 -γ .

We remark that under these conditions then for sufficiently large n the inequalities ρ ≤ min { n -3 k n + k γ, 1 6( n -k ) } and γ + ρ < min {√ 1 4( n -k ) k , 1 n -k , 1 99 k , 1 100 } are satisfied. As shown in the following lemma, these three additional conditions hold with high probability over the randomness of the initialization and training sample.

Lemma C.1. There exists a positive constant C such that for any δ ∈ (0 , 1) if n ≥ C log( 3 δ ) then the extra conditions of Assumption 3 hold with probability at least 1 -δ .

Proof. Using Lemma A.3, under the Assumption 3 for sufficiently large n there exists a positive constant c such that the probability the first condition does not hold is at most exp( -cn ) . Alternatively,

setting δ ≥ 3 exp( -cn ) and rearranging, as long as n ≥ C log ( 3 δ ) then the probability the first condition does not hold is at most δ 3 . Conditioned on the first event, using Lemma A.6 then if n ≥ C log( 3 δ ) and m ≥ C log ( 6 k δ ) then the probability condition two does not hold is also at most δ 3 . Therefore the probability that the first two events hold is at least (1 -δ 3 ) 2 ≥ 1 -2 δ 3 . For the third condition, noting ρ 1 -γ > ρ and ρ ≤ γ/ 5 , then by Lemma A.1 the probability the third condition does not hold is also at most δ 3 . Therefore, we conclude that all three properties hold with probability at least 1 -δ .

C.1 Proof of Lemma 3.2

The following lemma characterizes an iteration independent upper bound on the number of clean and corrupt updates. This result will prove significant for proving the termination of GD.

Lemma C.2 (Lemma 3.2) . Assume Assumption 3 holds. Suppose further that at some epoch t 0 the loss of every clean point is bounded above by a ∈ R ≥ 0 , while the loss of every corrupted point is bounded above by b ∈ R ≥ 0 . Then the total number of updates which occurs after this epoch is upper bounded as follows,

for all t ≥ t 0 .

Proof. From Lemma B.5, ρ ≤ γ , and the assumption on a and b ,

Substituting these bounds into each other, and as γ + ρ < (4( n -k ) k ) -1 / 2 under Assumption 3, we arrive at the iteration independent bound on the number of updates as claimed in the statement of the theorem.

C.2 Early training and proof of Lemma 3.3

Lemma C.3. Under Assumption 3, then for i ∈ S T and j ∈ Γ it follows that ⟨ w (1) j , x i ⟩ > 0 iff i ∼ j . For i ∈ S F , j ∈ Θ p , and i ̸∼ j it follows that ⟨ w (1) j , x i ⟩ > 0 if i ∈ A (0) j .

Proof. Suppose j ∈ Γ , i ∼ j , i ∈ S T . Recall from definition of Γ p that G ( i ) j (0 , 1)( γ -ρ ) -B ( i ) j (0 , 1)( γ + ρ ) ≥ 2 λ w η . Using Lemma B.2

On the other hand, if i ̸∼ j then again from Lemma B.2

Now consider i ∈ S F , i ̸∼ j , and i ∈ A (0) j . By Lemma B.2 and the definition of Θ ,

Lemma C.4 (Lemma 3.3) . Suppose Assumption 3 holds. Let j ∈ Γ p . Let 0 < t < T 0 . A point i ∈ A ( t ) j if one of the following conditions hold:

Furthermore, if one of the following conditions hold, then i / ∈ A ( t ) j :

Proof. We proceed by induction. For t = 1 , the i ∈ S T case was shown in Lemma C.3 and the i ∈ S F , i ̸∼ j case is clear. Now, suppose the lemma holds for iteration t and consider iteration t +1 . First let i ∈ S F , i ̸∼ j . If i ∈ A (1) j then

Here the first line is Lemma B.2, the second line comes from the inductive hypothesis, and the third line comes from ( γ + ρ ) < 1 n -k (Assumption 3). If i / ∈ A (1) j then

Again, the first line is Lemma B.2, the second line uses the inductive hypothesis, and the fourth line uses ρ ≤ n -3 k n + k γ (Assumption 3).

Now, let i ∈ S T . We again use, in order, Lemma B.2, the inductive hypothesis, and ρ ≤ n -3 k n + k γ . If i ∼ j then

If i ̸∼ j then

Lemma C.5. Suppose Assumption 3 holds. For all t 0 ≤ t 1 < T 0 ,

Proof. First we claim that for all i ∈ S T , j ̸∼ i , and t < T 0 ,

We prove the claim by induction. The base case t = 0 follows because ⟨ w (0) j , x i ⟩ ≤ λ w . Now suppose it is true at iteration t . If ⟨ w ( t ) j , x i ⟩ > 0 then by Lemma B.2,

using γ + ρ < 1 2 k . From this, the claim follows. Otherwise,

We now turn to the statement of the lemma, again proceeding by induction. The base case t 1 = t 0 is clear. Otherwise, we consider two cases:

If G j ( t 0 , t 1 ) > 1+2 k ( t 1 -t 0 +1)( γ + ρ ) γ -ρ then for all i ∈ S T and j ̸∼ i , by Lemma B.2,

by the claim and λ w < η (Assumption 3). Therefore, G j ( t 0 , t 1 +1) ≤ G j ( t 0 , t 1 ) + ( n -k ) .

If G j ( t 0 , t 1 ) ≤ 1+2 k ( t 1 -t 0 +1)( γ + ρ ) γ -ρ , then

Lemma C.6. Suppose Assumption 3 holds. For all t < T 0 , i ∈ S F , and i ∼ j ,

Proof. Consider t < T 0 . We consider three cases

If t = 0 then

If ⟨ w ( t -1) j , x i ⟩ ≤ 0 then by Lemma B.2

If ⟨ w ( t -1) j , x i ⟩ > 0 then let t ′ < t be the smallest iteration such that ⟨ w ( τ ) j , x i ⟩ > 0 for all t ′ ≤ τ < t . By Lemma B.2, Lemma C.5, and the previous two cases above,

By γ + ρ < 1 (Assumption 3) we conclude

C.3 Proof of Lemma 3.4

Lemma C.7 (Lemma 3.4) . Suppose Assumption 3 holds. There is an iteration T 1 < T 0 during training and expressions C 1 , C 2 , and C 3 where the following hold:

For all p ∈ {-1 , 1 } , j ∈ Γ , i ∼ j , and i ∈ S

For all p ∈ {-1 , 1 } , j ∈ Γ p , i ̸∼ j , and i ∈ S T

For all i ∈ S T ,

Furthermore,

Proof. Fix i ∈ S T . At every iteration 1 ≤ t < T 0 , we bound

where we use in order: t < T 0 , the definition of f ( t, x ) , Lemma C.4, Lemma B.3, Γ -1 ∩ Γ 1 = ∅ , and Lemma C.4 again. We also use | Γ p | ≥ 0 . 99 m (Assumption 3). We further simplify this bound to conclude

Additionally, we bound

Therefore as long as

then

Notice that this does not depend on i or p . Therefore we can let T 1 be the largest integer satisfying this bound for t and bound ℓ ( T 1 , x ℓ ) > 0 for all l ∈ S T . To verify that T 1 < T 0 , consider i ∈ S F :

This is less than 1 for all t < T 1 since k ≤ n 3 (Assumption 3).

Now, fix i ∈ S T again. For i ∼ j , we then can use Lemma B.2 and Lemma C.4:

using ρ ≤ 1 6( n -k ) , η is sufficiently small, and γ + ρ < min { 1 99 k , 1 100 } (Assumption 3). Now assume i ̸∼ j . Using Lemma B.2 and Lemma C.4 we can bound

using η is sufficiently small and k ≤ n 100 and ρ ≤ γ 5 (Assumption 3). Likewise, for 1 ≤ t < T 1 ,

In the first six lines we use: t < T 0 , the definition of f ( t, x ) , Lemma C.4, Lemma B.2 and Lemma B.3, Lemma C.4 again, and | Γ p | ≥ 0 . 99 m (Assumption 3), respectively.

We also bound

Combining these two bounds we see that

at this iteration, as desired. Here we use ( γ + ρ ) ≤ min { 1 99 k , 1 100 , 1 n -k } , η is sufficiently small, and ρ ≤ 1 6( n -k ) (Assumption 3)

C.4 Late training

Lemma C.8. Suppose Assumption 3 holds. Fix ε > 0 . We will say a neuron is aligned (at iteration t ) if

than at least (1 -ε ) m neurons in each Γ p will be aligned.

Proof. Let p ∈ {-1 , 1 } and t be such that εm different neurons in Γ p are unaligned at iteration t . For any neuron index j ∈ Γ p and i ∈ S T , we can use Lemma B.2 to bound

Since nγ < 1 (Assumption 3), we see that min { 3 4 m , 3 nγ 4 m } = 3 nγ 4 m . If the lower bound above is positive then ( -1) j sgn ⟨ w ( t ) j , x i ⟩ = y i . Therefore, if a neuron j is unaligned then B j ( T 1 , t ) ≥ 3 nγ 4 ηm ( γ + ρ ) ≥ 5 n 8 ηm (using ρ ≤ γ 5 from Assumption 3). If there are εm unaligned neurons, then

Denote the first iteration after T 1 where more than εm neurons in one of the Γ p are unaligned as T ε . If no such iteration exists, let T ε = ∞ . We will eventually show that indeed T ε = ∞ , by showing that the training process reaches zero loss before such an iteration can happen.

Lemma C.9. Assume Assumption 3 holds and also γ + ρ < 0 . 99(1 -ε ) 4 k . There is an iteration T 2 ≥ T 1 so that for all iterations t satisfying T 2 ≤ t < T ε and all i ∈ S T ,

Furthermore, we can choose T 2 so that

for all i ∈ S . For t ≥ T , if

Proof. Fix i ∈ S T and t < T ε -1 . Suppose ℓ ( t, x i ) > 0 . Using Lemma B.3 and t < T ε ,

Therefore,

By Lemma C.7, the loss of each clean point at T 1 is at most 1 3 , so each clean point reaches zero loss in at most

iterations.

Now suppose ℓ ( t, x i ) = 0 . We similarly argue

This implies ℓ ( t +1 , x i ) ≤ 4 ηmk . By induction, we see that if t < T ε and ℓ ( t, x i ) ≤ 4 ηmk , then ℓ ( t +1 , x i ) ≤ 4 ηmk .

Lemma C.10. Assume Assumption 3 holds and γ + ρ < 0 . 99(1 -ε ) 4 k . For all t 1 , t 2 satisfying T 2 ≤ t 1 ≤ t 2 < T ε and i ∈ S T ,

Proof. Recall Lemma B.5 (restated in this setting, using Lemma C.9):

Using t < T ε we bound

From this, the desired inequality follows.

Lemma C.11. Assume Assumption 3 holds and γ + ρ ≤ min { 0 . 99(1 -ε ) 4 k , √ 0 . 99(1 -ε ) 8( n -k ) k } . Let i ∈ S F . Suppose there is j ̸∼ i such that

Then for all t satisfying T 2 ≤ t < T ε , ⟨ w ( t ) j ′ , x i ⟩ > 0 for some neuron j ′ depending on t .

Proof. Let τ 0 ≥ T 2 be the first iteration after T 2 where ℓ ( t, x i ) = 0 . We will show by induction that for all t satisfying T 2 ≤ t ≤ τ 0 that ⟨ w ( t ) j , x i ⟩ > 0 . The case t = T 2 follows immediately by the assumption of the lemma. Otherwise, assume ⟨ w ( t ′ ) j , x i ⟩ > 0 for all T 2 ≤ t ′ < t . By Lemma B.2 and Lemma C.10,

We now continue the induction past τ 0 . If ℓ ( t, x i ) = 0 , then point i clearly activates some neuron. Let τ 1 be the first iteration after τ 0 where ℓ ( t, x i ) > 0 . By Lemma B.3

This means

and there is some j ′ satisfying

assuming η is sufficiently small (Assumption 3). We can run the original induction argument with τ 1 replacing T 2 and τ 2 = min { t ≥ τ 2 : ℓ ( t, x i ) = 0 } replacing τ 0 to verify the conclusion for τ 1 ≤ t ≤ τ 2 . By switching back and forth between these two arguments, we can show that point i activates some neuron for all T 2 ≤ t < T ε .

Lemma C.12. If Assumption 3 holds, the training process reaches loss.

Proof. In this proof, let ε = 1 5 . The conditions of Lemma C.9, Lemma C.10, and Lemma C.11 hold because γ + ρ ≤ min { 0 . 99 / 5 k , √ 0 . 99 / 5 2 k ( n -k ) } .

By Lemma C.2, there is a finite bound on the number of updates, independent of the number of iterations spent training. If we carry out the training procedure for infinitely many iterations, there must be some iteration where we make no updates. Since the training procedure is deterministic, we will not make any updates after this point, and we will have converged. It remains to show that this convergence results in zero training loss. The only way for a point to not update any neurons is for that point's loss to be zero or for that point to activate no neurons.

Lemma C.8 and Lemma C.11 say, under certain conditions, that every clean point and every corrupted point activates some neuron for each iteration t ≤ T ε . We need only to verify that these conditions hold and that B ( T 1 , t ) remains below the limitation set in Lemma C.8.

We apply Lemma C.2 starting at t 0 = T 1 . By Lemma C.7, ℓ ( T 1 , x i ) ≤ 1 3 for all i ∈ S T . Using Lemma C.6, for i ∈ S F ,

With these bounds, Lemma C.2 shows that for all t ≥ T 1 ,

using γ + ρ ≤ min { 1 n -k , 1 99 k } , η sufficiently small, and k ≤ n 100 (Assumption 3). By Lemma C.8, T ε = ∞ if n 10 η < 5 εn 8 η , which is clearly true for ε = 1 5 .

We now show that every training point i activates at least one neuron each iteration. By Lemma C.9, this is true if i ∈ S T . By Lemma C.11, this is true for i ∈ S F if there is a neuron j ̸∼ i such that ⟨ w ( T 2 ) j , x i ⟩ > η 2( n -k )( γ + ρ )(4 k +3) 0 . 99(1 -ε ) . Fix i ∈ S F .

First, assume that ℓ ( t, x i ) > 0 for all t < T 2 . By Assumption 3 and Lemma C.3, we know there is j ∈ Γ y i such that i ∈ A (1) j . Using Lemma B.2 and Lemma C.4 we can bound

and

By induction on t we see that

for t > T 1 . The base case t = T 1 +1 is clear. Suppose the inequality holds for t . Either G j ( T 1 , t ) increases by at most n -k or there is some i ′ ∈ S T ∩ A ( t +1) j with i ′ ̸∼ j . By Lemma B.2 and Lemma C.7,

from which the inequality follows. Since ( γ + ρ ) B j ( T 1 ,t ′ ) γ -ρ ≤ 3 k ( t ′ -T 1 ) < ( n -k )( t ′ -T 1 ) (using ρ < γ 5 and k ≤ n 100 from Assumption 3), this maximum occurs at τ = T 1 . This yields

We want to show this bound is larger than a quantity that is O ( η ) . This happens when both of the following hold:

which holds when η is sufficiently small.

Now suppose ℓ ( τ, x i ) = 0 for some iteration τ ≤ T 2 . By Lemma C.7, T 1 < τ . In this case, we see that

where in the third line we use Lemma C.2 and the fourth line we use γ + ρ ≤ min { 1 n -k , 1 99 k } and η sufficiently small (Assumption 3). Sine this is positive, there is some neuron j with i ̸∼ j such that ϕ ( ⟨ w ( T 2 ) j , x i ⟩ ) is at least 1 m this bound. This is an Ω(1) lower bound. Since the required condition is ⟨ w ( T 2 ) j , x i ⟩ > O ( η ) , this can be achieved by taking η sufficiently small.

C.5 Proof of Lemma 3.5

Lemma C.13 (Lemma 3.5) . Assume Assumption 3 holds. Let y ∈ {-1 , 1 } chosen uniformly and x := y √ γ v + √ 1 -γ n , where n ∼ Uniform ( S d -1 ∩ span { v } ⊥ ) . Suppose that |⟨ n , n ℓ ⟩| < ρ 1 -γ for all l ∈ [2 n ] , then yf ( T end , x ) > 0 .

Proof. Following the same steps as in (5) for any j ∈ [2 m ]

where λ ′ ℓ := ( -1) l yβ ( ℓ ) ⟨ x ℓ , x ⟩ = β ( ℓ ) γ +(1 -γ ) ⟨ n ℓ , n ⟩ . Then as in Lemma B.1

Recall, from Lemma C.4 for any j ∈ Γ p then G j (1 , T end ) ≥ G j (1 , T 1 ) = T 1 ( n -k ) . As a consequence, for j ∈ Γ p we have

For j such that ( -1) j = y then

As a result

̸

using | Γ p | ≥ 0 . 99 (Assumption 3). From Lemma C.7

furthermore, combining the assumptions 100 k < n , η sufficiently small, and γ + ρ < 1 n -k with Lemma C.2 we see

Here we also use that ℓ (0 , x i ) ≤ 1 + mλ w = 1 + O ( η ) for all i .

Combining these inequalities it follows that

again using 100 k < n , η sufficiently small, and γ + ρ < 1 n -k .

C.6 Proof of Theorem 3.1

Theorem C.14 (Theorem 3.1) . Let Assumption 2 hold with ρ = γ/ 5 . There exists a sufficiently small step-size η such that with probability at least 1 -δ over the randomness of the dataset and network initialization the following hold.

There exists a positive constant C such that the training process terminates at an iteration T end ≤ Cn η .

For all i ∈ [2 n ] then ℓ ( T end , x i ) = 0 .

There exists a positive constant c such that the generalization error satisfies

̸

.

Proof. Under Assumption 3 Statement 1 and 2 follow from Lemma C.12, Note the bound in Statement 1 comes from Lemma C.2 applied from iteration 0 to iteration T end, using 4 k ( n -k )( γ + ρ ) 2 < 4 / 99 and ℓ (0 , x i ) = 1 + O ( η ) for all i ∈ [2 n ] . With regards to Statement 3, from Lemma C.13 if |⟨ n , n ℓ ⟩| < ρ 1 -γ for all l ∈ [2 n ] it follows that sgn( f ( T end , x )) = y . Therefore, as ρ 1 -ρ > ρ and analogous to Lemma A.1, under Assumption 3 there exists a positive constant c such that

̸

Finally, under Assumption 2 then Assumption 3 holds with probability at least 1 -δ by Lemma C.1.

Appendix D Non-benign overfitting

Assumption 4. With δ, ρ ∈ (0 , 1) we assume the following conditions on the data and model hyperparameters.

In our analysis we require two additional assumptions on the training sample and activations at initialization.

Assumption 5. Let ρ ∈ (0 , 1) satisfy ρ ≤ min { 1 -γ 4 n , 1 2 n -1 -γ } and in addition to the conditions detailed in Assumption 5, assume the following two conditions hold.

̸

Note under these assumptions that γ < 1 24 n 2 , this implies 1 2 n -1 > γ and ρ ≤ min { 1 -γ 4 n , 1 2 n -1 -γ } if ρ ≤ 1 5 n . As demonstrated in the following Lemma, these additional two conditions hold with high probability over the randomness of the initialization and training set.

Lemma D.1. The additional conditions of Assumption 5 hold with probability at least 1 -δ .

Proof. Using Lemma A.7, then as long as m ≥ log 2 ( 4 n δ ) the probability the first condition does not hold is at most δ/ 2 . Using Lemma A.1, and observing ρ 1 -γ > ρ , then as long as

the probability that the second condition does not hold is also at most δ/ 2 . Using the union bound we conclude that both properties hold with probability at least δ .

D.1 Proof of Lemma 3.7

Lemma D.2 (Lemma 3.7) . In addition to Assumption 5, assume also that ℓ ( t 0 , x i ) ≤ a for all i ∈ [2 n ] . Then

Proof. From Lemma B.5, ϕ ( ρ -γ ) ≤ ρ + γ , and the assumption on a ,

If we sum over i , we get

from which the result follows.

D.2 Supporting lemmas

Lemma D.3. If Assumption 5 holds, then the training process converges to zero loss.

Proof. By Lemma D.2, there is an upper bound on the number of updates independent of iteration. This can only happen if there is some iteration after which we make no updates. In turn, this can only happen if every point is either at zero loss or activates no neurons. We prove by induction that every point activates a neuron each iteration t = 0 . Consider an arbitrary point i ∈ [2 n ] , at t = 0 the induction hypothesis is true by Statement 1 of Assumption 5. Suppose the induction hypothesis is true at iteration t , we consider the following two cases separately in order to show the induction hypothesis also holds at iteration t +1 .

If ℓ ( t, x i ) > 0 , then by assumption we can choose a j ∈ [2 m ] such that ϕ ( ⟨ w ( t ) j , x i ⟩ ) > 0 . We bound

which follows as γ + ρ < 1 2 n -1 (Assumption 5).

If ℓ ( t, x i ) = 0 , then

is bounded below by 1. This means that there is some j such that ϕ ( ⟨ w ( t ) j , x i ⟩ ) ≥ 1 m . We bound

Lemma D.4. Suppose that at epoch τ every point is at zero loss. Then

Proof. If ℓ ( τ, x i ) = 0 for all i ∈ [2 n ] , then y i f ( τ, x i ) ≥ 1 for all i ∈ [2 n ] . We bound

ηT

i

(0

Summing over i ∈ [2 n ] we see that

from which the result claimed follows.

Lemma D.5. Let y ∼ U ( {-1 , 1 } ) and consider a clean test point x = y ( √ γ v + √ 1 -γ n ) , where n ∼ U ( S d ∩ span( v ) ⊥ ) . If Assumption 5 holds, then

, τ

) +

η

(

γ

+

ρ

)

T

(

i

)

(0

, τ

) +

O

(

η

)

.

Proof. Observe by symmetry of the distributions of both y and n that -x is identically distributed to x and furthermore that the labels of x and -x are opposite. As a result, if y ( f ( T end , x ) -f ( T end , -x )) < 0 then at least one of y ( f ( T end , x ) < 0 or -y ( f ( T end , -x ) < 0 , in turn implying at least one of them is misclassified. By construction, ⟨ w ( t ) j , x ⟩ > 0 iff ⟨ w ( t ) j , -x ⟩ < 0 , therefore

Unwinding the GD update to a neuron we have

Furthermore, as

where the final equality follows from symmetry of the noise distribution, z := ∑ 2 n i =1 T i (0 , T end ) n i and u = z ∥ z ∥ . Observe

̸

where the final inequality follows from Jensen's inequality. By assumption 4 nρ < 1 -γ , and 10 mλ w ≤ nγ ≤ √ n 6 √ d , furthermore trivially (1 -γ ) > 0 . 8 . Conditioning on the event ⟨ n , u ⟩ > 0 , which holds with probability 1 / 2 , these inequalities in combination with Lemma D.4 give

Therefore, if ⟨ n , u ⟩ > 0 then the condition

implies at least one of x or -x is misclassified. Suppose n ∼ U ( S d ∩ span ( v ) ⊥ ) is such that (8) holds. Then as y ∼ U ( {-1 , 1 } ) it follows given n that either x or -x are sampled each with equal probability and thus the chance of misclassifying is at least 1 / 2 . As a result, the probability of misclassification is at least

as claimed. We note the final inequality above follows by showing that the two spherical caps corresponding to the set of unit vectors z satisfying ⟨ n , u ⟩ ≥ 1 / (2 √ d ) account for less than half the area of S d -1 , which can be derived from formulas provided in (S, 2011). This inequality has also appeared for instance in (Asi & Duchi, 2019).

D.3 Proof of Theorem 3.6

Theorem D.6 (Theorem 3.6) . Assume Assumption 4 holds with ρ = 1 5 n . With probability at least 1 -δ over the randomness of the dataset and network initialization the following hold.

There exists a positive constant C such that the training process terminates at an iteration T end ≤ Cn η .

For all i ∈ [2 n ] ℓ ( T end , x i ) = 0 .

The generalization error satisfies

̸

Proof. Under Assumption 5 Statement 1 and 2 come from Lemma D.3. The bound on T end comes from Lemma D.2 applied between iterations 0 and T end, using ℓ (0 , x i ) = 1 + O ( η ) for all i ∈ [2 n ] and ( γ + ρ )(2 n -1) < 2 3 . Statement 3 follows from Lemma D.5. We conclude by observing under Assumption 4 that Assumption 5 holds with probability at least 1 -δ .

Appendix E No-overfitting

Assumption 6. With δ, ρ ∈ (0 , 1) and C a generic, positive constant, we assume the following conditions on the data and model hyperparameters.

For our analysis we make two additional assumptions.

Assumption 7. Let ρ ∈ (0 , 1) satisfy γ ≥ 5 ρ . In addition to the assumptions detailed in Assumption 6, suppose the following conditions hold.

For all i, l ∈ [2 n ] such that i = l then |⟨ n i , n l ⟩| ≤ ρ 1 -γ .

̸

We remark under these assumptions that for sufficiently large n then ρ satisfies the inequality ρ < min { γ ( n -3 k ) -2 n + k , γ 5 , n 11 } . As shown in the following lemma, these two additional conditions hold with high probability over the randomness of the initialization and training set.

Lemma E.1. There exists a positive constant C such that if n ≥ C log ( 2 m δ ) then the extra conditions of Assumption 7 hold with probability at least 1 -δ .

Proof. Using Lemma A.3, there exists a positive constant C such that for n ≥ C there in turn exists a constant c such that the probability the first condition does not hold is at most m exp( -cn ) . Setting δ ≥ 2 m exp( -cn ) and rearranging, as long as n ≥ C log ( 2 m δ ) , then the probability the first condition does not hold is at most δ/ 2 . Using Lemma A.1 and observing ρ 1 -γ > ρ , then under the condition on d stated in Assumption 7, the probability that the second condition does not hold is also at most δ 2 . Using the union bound, we therefore conclude that both properties hold with probability at least δ .

E.1 Proof of Lemma 3.9

Lemma E.2 (Lemma 3.9) . Suppose Assumption 7 holds. Consider an arbitrary j ∈ [2 m ] and iteration t satisfying 2 ≤ t < T 0 . Then i ∈ A ( t ) j iff i ∼ j .

Proof. First we establish at iteration t = 1 that for all i ∈ S T , i ∈ A ( t ) j iff i ∼ j . The argument here is similar to that of Lemma C.3. Suppose i ∼ j and i ∈ S T . By Assumption 7 all neurons are in Γ , therefore from the definition of Γ p for all j ∈ [2 m ] we have G ( i ) j (0 , 1)( γ -ρ ) -B ( i ) j (0 , 1)( γ + ρ ) ≥ 2 λ w η . Using Lemma B.2

On the other hand, if i ̸∼ j then again from Lemma B.2

Now we consider t = 2 . If i ∈ S T and i ∼ j then

whereas if i ∈ S T and i ̸∼ j then

By assumption γ > 2+( n + k ) ρ n -3 k > ( n + k ) ρ n -3 k and therefore for i ∈ S T then i ∈ A (2) j iff i ∼ j . Again using Lemma B.2, for i ∈ S F and i ∼ j

>

-

η, and for i ∈ S F and i ̸∼ j

Therefore, as γ > 2+( n + k ) ρ n -3 k then for i ∈ S F and i ∼ j

and for i ∈ S F and i ̸∼ j

With the base case established we proceed by induction to prove if i ∈ A ( t -1) j iff i ∼ j , then i ∈ A ( t ) j iff i ∼ j . By the assumptions on γ , the induction hypothesis and again using Lemma B.2, for i ∼ j

and for i ̸∼ j

Therefore, for an epoch t satisfying 2 ≤ t ≤ T 1 then i ∈ A ( t -1) j iff i ∼ j .

E.2 Proof of Lemma 3.10

Lemma E.3 (Lemma 3.10) . Suppose Assumption 7 holds, then there is an iteration T 1 < T 0 such that

Furthermore for i ∈ S

Finally,

Proof. Let t < T 0 . By Lemma E.2 and Lemma B.2 we can bound for i ∈ S F and i ∼ j as follows,

Similarly, for i ∈ S F , i ̸∼ j ,

For i ∈ S T , i ∼ j ,

and

Lastly, for i ∈ S T , i ̸∼ j ,

Therefore, for i ∈ S T , from which we conclude

Therefore, as long as

then ℓ ( t, x i ) > 0 . Let T 1 be the largest value of t satisfying (9) and t < T 0 . We see that

From this, the bounds claimed follow.

E.3 Proof of Lemma 3.11

Lemma E.4 (Lemma 3.11) . Let Assumption 7 hold. Suppose at iteration t 0 the following conditions are satisfied.

For all iterations τ satisfying t 0 ≤ τ ≤ t it holds that i ∈ A ( τ ) j only if i ∼ j ,

For all iterations τ satisfying t 0 ≤ τ ≤ t , i ∈ A ( τ ) j if i ∼ j and i ∈ S T .

Then for j ∈ [2 m ] and p ∈ {-1 , 1 } we have

Proof. Consider an arbitrary neuron j ∈ [2 m ] , using Lemma B.3 and assumption (b) we bound for t < τ ≤ t , i ∈ S F , and i ∼ j

As ϕ ( ⟨ w ( τ ) j , x i ⟩ ) ≥ 0 in general we may conclude that

Summing over all i ∈ S F such that i ∼ j then by assumption (c)

Combining these expressions it follows that

As the number of clean updates on a pair of neurons ℓ and j with ℓ ∼ j is the same by assumptions (c) and (d), then we may rewrite this bound as

Let s ∈ [2 m ] be arbitrary, we proceed to bound ∑ j ∼ s G j ( t 0 , t ) . Using Lemma B.2, for i ∈ S T and i ∼ j

⟨ w ( τ -1) j , x i ⟩ ≥ ⟨ w ( t ) j , x i ⟩ + η ( T ij ( t 0 , τ -1) + ( γ -ρ ) G ( i ) j ( t 0 , τ -1) -( γ + ρ ) B j ( t 0 , τ -1)) . By assumptions (c) and (d)

Note either T i ( t 0 , τ ) = T i ( t 0 , τ -1) or ℓ ( τ, x i ) > 0 . Consider the case where the latter holds, then

Furthermore, suppose τ ′ ≤ τ is the first iteration before τ such that G j ( τ ′ , τ ) = 0 and let i ∈ S T , i ∼ s be a point that makes an update at iteration τ ′ -1 . Using the above bound it follows that

This implies

and

By the construction of τ ′ it follows that

From this we get the bound

Combining (10) summed over j ∼ s with (11), then

and

Using ρ ≤ γ 5 , η sufficiently small and γ ≤ 1 36 k (Assumption 7) these bounds simplify to

E.4 Late training

Lemma E.5. Under Assumption 7 the training process terminates at an iteration T end satisfying

for all i ∈ S F and j ∈ [2 m ] .

Proof. By Lemma E.3, at iteration t = T 1 with

then the first two conditions of Lemma E.4 are satisfied. Next, using Lemma B.2, we see by induction on t ≥ T 1 that if

then for i ̸∼ j and T 1 ≤ τ ≤ t ,

for all i ∈ S T and

and for i ∈ S T , i ∼ j , and T 1 ≤ τ ≤ t ,

for η sufficiently small. Thus we have shown under an additional assumption on B j ( T 1 , t ) that with t 0 = T 1 and a and b as defined above, then all four conditions of Lemma E.4 are satisfied. As a result GD converges or terminates as long as

which is equivalent to

This is true by Assumption 7, as

where above we used γ ≤ min { 1 36 k , 1 36 } and ρ ≤ min { γ 5 , n 11 } .

Lemma E.6. Assume Assumption 7 holds. Let y ∈ {-1 , 1 } be drawn uniformly at random and x := y √ γ v + √ 1 -γ n where n ∼ Uniform ( S d -1 ∩ span { v } ⊥ ) . Suppose that |⟨ n , n ℓ ⟩| < ρ 1 -γ for all l ∈ [2 n ] , then yf ( T end , x ) > 0 .

Proof. We proceed as in the proof of Lemma C.13. Following the same steps as in (5), for any j ∈ [2 m ]

where λ ′ ℓ := ( -1) l yβ ( ℓ ) ⟨ x ℓ , x ⟩ = β ( ℓ ) γ +(1 -γ ) ⟨ n ℓ , n ⟩ . Then as in Lemma B.1

γ

γ

-

(

-

+

ρ

ρ

)

λ

λ

′

ℓ

′

ℓ

γ

≤ -

+

(

γ

ρ,

-

if

ρ

)

i

,

∈ S

if

i

T

,

∈ S

F

,

.

Recall, from Lemma E.2, for any j ∈ [2 m ] then G j (2 , T end ) ≥ G j (2 , T 1 ) = ( T 1 -2)( n -k ) . As a consequence, for j ∈ Γ p we have

For j such that ( -1) j = y then

As a result

̸

Decompose B (2 , T end ) = B (2 , T 1 ) + B ( T 1 , T end ) and observe from Lemma E.2 that

From Lemma E.3 and Lemma E.4, using the assumptions ρ ≤ n 11 , η sufficiently small and γ ≤ min { k 36 , 1 36 } then

Using the assumption that k ≤ n 100 and η is sufficiently small we see that

As

then yf ( T end , x ) is positive provided ( γ -ρ ) -( γ + ρ ) / 9 is positive and η is sufficiently small. Both these conditions are guaranteed by Assumption 7 and thus the test point is correctly classified.

E.5 Proof of Theorem 3.8

Theorem E.7 (Theorem 3.8) . Let Assumption 6 hold with ρ = γ/ 5 . There exists a sufficiently small step-size η such that with probability at least 1 -δ over the randomness of the dataset and network initialization we have the following.

There exists a positive constant C such that the training process terminates at an iteration T end ≤ Cn η .

For all i ∈ S T then ℓ ( T end , x i ) = 0 while ℓ ( T end , x i ) = 1 for all i ∈ S F .

There exists a positive constant c such that the generalization error satisfies

̸

Proof. Under Assumption 7, Statements 1 and 2 follow from Lemma E.5. The bound on T end follows from Lemma E.4 applied at t 0 = 2 , indeed the number of iterations cannot exceed the number of updates which, as γ > 3 n , is bounded as

Again under Assumption 7 using Lemma E.6 then Statement 3 follows in exactly the same manner as the proof of Statement 3 for Theorem C.14. Finally, Lemma E.1 implies that under Assumption 6 then Assumption 7 holds with probability at least 1 -δ .

Appendix F Numerical simulations

Reproducibility statement: the code used to generate the following figures can be found at https: //github.com/wswartworth/benign_overfitting .

To investigate our theory we train two-layer neural networks with ReLU activations using full-batch gradient descent and a fixed step size. We train on a synthetic binary classification dataset generated as per Section 2.1. Finally, we train using both the hinge and logistic loss.

Clean Loss

Corrupt Loss

Test Loss

100

100

10°

hinge

logistic

10-1

102

102

a6uH

104

Hinge Loss

10-3

6 × 10-1

106

104

10-5

10-8

hinge

logistic

10-6

hinge

logistic

4 × 10-1

0

200

400

600

800

100012001400

0

500

1000

1500

2000

2500

3000

D

200

400

600

800

100012001400

Iteration

Iteration

Iteration

Correctly Classified Clean Points

Correctly Classified Corrupt Points

Correctly Classified Test Points

1.0

1.0

hinge

1.0

logistic

0.9

0.9

0.9

FPoints

Fraction of Points

Fraction of I

0.8

0.7

0.8

0.7

0.6

0.7

0.5

0.6

hinge

0.4

0.6

hinge

logistic

logistic

0.5

0

200

400

600

800

100012001400

0

200

400

600

800

100012001400

0

200

400

600

800

100012001400

Iteration

Iteration

Iteration

In Figure 1 we call attention to the difference in the training dynamics of hinge loss versus logistic loss. Perhaps the key difference between the hinge loss and logistic loss is that the contributions from any given point do not get smaller as the point approaches 0 loss. Furthermore, unlike with the logistic loss, points can actually attain zero hinge loss after a finite number of epochs. While a point has zero loss it ceases to contribute to the update of the network parameters. As a result, points close to zero hinge loss periodically activate and deactivate giving rise to the chaotic behavior observed as the training loss approaches zero. We emphasize that managing this behavior required a careful analysis distinct from that of prior works analysing the logistic loss.

In Figure 2 we call particular attention to the bottom right plot. Our theory predicts a phase transition between benign overfitting and non-benign overfitting when γ ≈ c/n : the phase transition we observe empirically in the bottom-right heatmap suggests this estimate is reasonable. With regard to the hinge loss over the corrupt points, displayed in the top-right heatmap, we observe another phase transition, this time between overfitting and non-overfitting. The top and bottom heatmaps of the left-hand column display the hinge loss over the clean training set and total training set respectively, these appear very similar due to the fact that clean points make up 95% of the training set. The clean points fail to achieve zero, or close to zero, hinge loss only when γ is small and n is large. As stated in the caption, in these experiments d is fixed and thus as n increases the near-orthogonality condition we require on the noise components in order to prove convergence to zero clean loss is compromised. As a result, when γ is small and the correlations between noise vectors is potentially large it is possible for pairs of points with opposite labels to be significantly correlated.

Clean Loss

CorruptLoss

1.0

0123456789

1.0

0.8

0.8

0.6

0.6

log2V

log2Y

0.4

0.4

-10

-11

-11

-12

0.2

-12

0.2

-13

-13

-14

-14

-15

-15

0.0

0.0

20 35 50 65 80 95 110125140155170185 200

20 35 50 65 80 95 110125140155170185200

n

n

Total Loss

Test Loss

0

1.0

12345678

0

- 1.0

0.8

0.8

0.6

0.6

log2V

log2Y

0.4

-9

0.4

-10

-11

-11

-12

0.2

-12

0.2

-13

-13

-14

-14

-15

-15

0.0

0.0

20 35 50 65 80 95 110125140155170185 200

20 35 50 65 80 95 110125 140155170185 200

n

n

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies overfitting when training shallow ReLU networks with the hinge loss using noisy data (i.e., with label flipping noise). They consider a setting where the data points consist of a direction $\\mathbf{v}$ that determines the (clean) labels, and noise from a sphere orthogonal to $\\mathbf{v}$. This distribution is similar to the two-clusters setting that has been studied in previous works [Frei et al. 2022, Xu and Gu 2023, Chatterji and Long 2021], except that the assumptions on the noise in the current work is stronger. The authors identify conditions on the signal strength that result in benign overfitting, harmful overfitting, and “non overfitting” (where only the clean training examples are classified correctly, and the test error on clean data is small). The results follow by analyzing the trajectory of gradient descent.\n\nSTRENGTHS:\nUnderstanding the conditions where benign overfitting occurs is an intriguing question that has be studied extensively in recent years. While the primary motivation for studying benign overfitting is to better understand interpolation learning in neural networks, most of the known results consider linear models (or kernel regression). The current paper helps understanding benign/harmful overfitting in shallow fully-connected ReLU networks trained with the hinge loss. Given our limited understanding of benign overfitting in nonlinear networks, I believe that the contribution of the current paper is significant.\n\nI did not verify the correctness of the proofs, which are in the appendix.\n\nWEAKNESSES:\n- Assumption 1 in the main text is different from Assumption 1 in the appendix. Specifically, condition 5 of Assumption 1 in the appendix includes a parameter $\\rho$ that does not appear in the assumption in the main text. I think that the authors should discuss the relationship between these two versions of Assumption 1. \n- Related to the above: Note that in lines 221-222 in the main text, the authors claim that the condition on the correlations between $\\mathbf{n}_i,\\mathbf{n}_l$ follows from Assumption 1 by letting $\\rho \\leq \\frac{\\gamma}{C}$. This is true if we use Assumption 1 from the appendix, but it does not follow from Assumption 1 in the main text. Please explain how/whether the proof holds using the assumption from the main text. This issue appears also in the other proof sketches.\n- The authors claim that there are only two papers about benign overfitting in nonlinear fully-connected neural networks: Frei et al. 2022, and Xu \\& Gu 2023. The authors provide a nice comparison between the current results and these prior results. It is worth noting that there is an additional prior work on benign overfitting in nonlinear fully-connected networks: “Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization” by Spencer Frei, Gal Vardi, Peter Bartlett and Nati Srebro. This paper appeared in arxiv only 2.5 months before the NeurIPS deadline, so it is reasonable that the authors were not aware of it when writing the current paper, but I think that a comparison to this paper would be helpful.\n- The writing is quite sloppy. Below I specify some typos and minor comments:\n    - Line 130, in the definition of $\\mathcal{F}^{(t)}$: should be $>0$ instead of $<1$.\n    - Line 132: “j=th”\n    - Line 155 — “constant that depends on $n,m,k,\\gamma,d$”: if we think about all of these parameters as constants, then what isn’t a constant?\n    - Line 156: (5) should be (4)\n    - Line 227: is the notation $[m]_p$ defined? Also, you probably meant $[2m]_p$.\n    - Line 385 in the appendix: “delta”\n    - The word “overfitting” sometimes means “interpolating also the noisy examples” and sometimes means “harmful overfitting”. I would write explicitly “harmful overfitting” when talking about the latter.\n\nQUESTIONS:\nSee the “weaknesses” section.\n\nLIMITATIONS:\nThe authors discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies training a noisy dataset with label noise using GD on a 2-layer network, without biases, and where the second layer is fixed. The main results are, that under the data distribution defined in the paper (namely, linearly separable with orthogonal noise), different types of overfitting (or no fitting) occur when changing the margin.\n\nSTRENGTHS:\n- I think the main three results are very interesting, and shed light on the role of the margin for determining the types of overfitting which occurs. \n- AFAIK the main three results are novel, as they study benign overfitting for neural networks under hinge loss, whereas previous works either studied classification losses (e.g. log loss) or kernel methods.\n- The proof techniques also seem interesting and employ combinatorial arguments about how the iterations change the activation pattern of the network.\n- The proof intuitions are very helpful and elaborative.\n\nWEAKNESSES:\nI think overall this is a nice paper that should be accepted, however, there are some weaknesses/questions which I would be happy to see the author’s response about.\n\n- I think the main weakness of the paper is the limited and very specific setting to which it is applied. This is acknowledged by the authors, however, I still think it should be further addressed.\n- The network doesn’t contain biases and only the weights of the first layer are trained. Regarding the bias terms, I am not sure how adding them would change the results and would be happy to see the author’s comment about this.\n- Regarding the data distribution, would it be possible to achieve a similar result where the noise n_i are sampled from a Gaussian distribution? If so, I think this is a much more standard setting to study (signal plus Gaussian noise). I guess also here, the noise would be almost orthogonal to the signal v. If something breaks here (rather than just make the computations more tedious), then I think the result is very brittle since it requires the noise to be exactly orthogonal to the signal.\n- I think the paper “Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting” by Mallinar et al. should also be cited, as it gives a taxonomy of the different possible types of overfitting (which is studied also here).\n- It would be nice to see an upper bound for Theorem 3.6. Currently when the margin is very small then the overfitting is either catastrophic (i.e random guess) or tempered (something between benign and catastrophic). Can the authors provide such an upper bound, or at least remark on what they think it should be? I think that it could depend on k, which depicts the noise level in the labels.\n- Small note: Assumption 1, should it be k <= n/C ?\n\nQUESTIONS:\n- How the results would change if the network had bias terms? or the second layer would also be trained?\n- Will the result still hold if the noise n_i are drawn from a Gaussian distribution? (or other symmetric distribution such as uniform over a sphere)\n- What should the upper bound on the loss in Theorem 3.6 be?\n\nLIMITATIONS:\nI think the authors adequately addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work a simple model of linear classification is analyzed to study the phenomenon of benign overfitting and generalization of neural networks with noisy data. A theoretical analysis of the trajectory of GD shows that it can be in three regimes (1) benign overfitting (2) non-benign overfitting and (3) no overfitting.  The analysis relies on a novel proof of the nonlinear dynamics which shows a bound on the number of non-trivial updates of GD until convergence.\n\nSTRENGTHS:\n1. Strong theoretical results in a very challenging setup of learning with nonlinear NNs.\n2. Novel proof. Most works consider the logistic loss without a thorough analysis of the trajectory. Here the hinge loss is considered and a detailed analysis of the trajectory is provided. The key result is a bound on the number of non-trivial updates which is independent of the total number of iterations. This is highly non-trivial: a “worst-case” dynamics can result in oscillating behavior or non-convergence. However, a clever argument is presented which derives inequalities of clean and corrupt updates that together imply convergence.  Hopefully such reasoning can be helpful in more complex settings.\n3. Full understanding of the dynamics. While the proof is lengthy, it provides a unique understanding of NN dynamics, their inner workings and the impact on generalization.\n4. The paper is mostly clearly written.\n\nWEAKNESSES:\n1. The theoretical result holds for a bounded number of samples (i.e., $n$ cannot go to infinity).\n2. Simple experiments which show the theoretical predictions could strengthen the paper.\n\nQUESTIONS:\nIn Eq. (1) why is $\\beta(i)$ needed in the definition of $\\boldsymbol{x}_i$. Isn’t the distribution of $\\beta(i)\\boldsymbol{n}_i$ the same as the distribution of $\\boldsymbol{n}_i$?\n\nLIMITATIONS:\nLimitations are discussed.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the benign overfitting (or not) of two-layer ReLU neural networks under the hinge loss. Under a new noisy (almost orthogonal) data setting without separable data setting, \n- Theorem 3.1 shows that benign overfitting exists occurs the signal-level parameter $\\gamma$ is small in $\\Theta(1/n)$;  \n- catastrophic overfitting (the generalization error is bounded below by a constant) occurs when $\\gamma$ is small enough at the order of $n^{-2/3}$. \n- Instead, if $\\gamma$ is large, no overfitting occurs, i.e., training error cannot be zero but generalizes well.\n\nSTRENGTHS:\n- provide solid analysis of generalization guarantees of two-layer ReLU neural networks under a new data setting\n- The results on benign overfitting, catastrophic overfitting, and no overfitting heavily depend on the choice of the signal-level parameter $\\gamma$\n\nWEAKNESSES:\n- The motivation of this paper is to fix the benign overfitting problem under the hinge loss. Since the analysis of previous work on the logistic loss is invalid, the authors focus on a new data generation setting and new proof framework. However, the motivation and significance of this research appear somewhat limited, particularly when considering the hinge loss. It would be beneficial to emphasize the differentiation between the results obtained with the logistic loss and the hinge loss, highlighting the theoretical advantages of employing the hinge loss. Additionally, exploring potential implicit biases related to benign overfitting and engaging in further discussions, particularly in comparison to the work conducted by Shamir (COLT2022) on the squared hinge loss, would strengthen the paper's arguments.\n\n- The problem setting and data assumption are a bit unrealistic (also as suggested by the authors). It’s ok for me but there are some assumptions missing, e.g., in the benign overfitting part, $n - k > 2*10^6$ in Lemma B.4 is needed. I think this should be clearly mentioned in the main text.\n\nMinior: \n- The early phase in line 224 is undefined.\n\nQUESTIONS:\n- I understand the data setting with $d \\geq n^2$ for the orthogonal condition, which is widely used in the benign overfitting literature for two-layer neural networks. But currently there exists some work that has fixed this issue:\n\nFrei et al. Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization. COLT2023.\n\nSo I kindly ask the authors to think about whether their assumption can be relaxed or not.\n\n- Since I’m an emergency reviewer, I don’t have enough time to check the proof. I high level read the proof framework, and find that line 236-237 is they key proof idea:”every training point activates one neuron after the last training update”. Could you please detail this?\nThe generalization performance (whether benign overfitting occurs) only depends on the signal-level parameter $\\gamma$ in the data generation process. However, intuitively, the difference on generalization guarantees is based on the changes of model complexity. So how to understand this issue?\n\n- Continuing the above question, we can find that, if $\\gamma$ is small (enough) at the $O(1/n)$ order, we have the benign overfitting. However, it means that the signal level $v$ is quite small and the noise level $n_i$ is quite large. That means, the model can fit such large random noise? If $\\gamma$ is large in Theorem 3.8, the model cannot fix the noise to achieve zero training loss if the noise level decreases. When checking these two results, I find that there exists some contradictions here? Maybe I’m missing something.\n\n- The width $m$ is not required to be over-parameterization?  More discussions are needed.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-15"}