SlowGuess/NeurIPS_2023_OCR huggingface dataset:
It is clear that  $\| \pmb{\theta} - \pmb{\theta}_{\delta}\| \stackrel {\delta \to 0}{\longrightarrow}0$ , and we will show that for small enough  $\delta$  the network above still satisfies the margin condition, yet has smaller parameter norm. To see why the margin condition is not violated for small enough  $\delta$ , notice that  $N_{\pmb{\theta}_{\delta}}(x) = N_{\pmb{\theta}}(x)$  for all  $x\notin (x_{i - 1},x_{i + 1})$  so in particular  $N_{\pmb{\theta}_{\delta}}(x_l) = N_{\pmb{\theta}}(x_l)$  for all  $l\neq i$ . Furthermore, a direct computation yields  $N_{\pmb{\theta}_{\delta}}(x_i) < N_{\pmb{\theta}}(x_i)\leq 1$ . As to the parameter norm, by a similar computation to that leading up to Eq. (11) we get that
$$
\left\| \boldsymbol {\theta} \right\| ^ {2} = \left\| \boldsymbol {\theta} _ {\delta} \right\| ^ {2} + 2 \delta \left(v _ {j _ {1}} ^ {2} - \frac {v _ {j _ {1}} w _ {j _ {1}} w _ {j _ {2}}}{v _ {j _ {2}}} - \frac {v _ {j _ {1}} b _ {j _ {1}} b _ {j _ {2}}}{v _ {j _ {2}}}\right) + O \left(\delta^ {2}\right). \tag {13}
$$
By construction,  $v_{j_1}w_{j_1} > 0$  and  $v_{j_2}w_{j_2} < 0$  hence  $-\frac{v_{j_1}w_{j_1}w_{j_2}}{v_{j_2}} > 0$ . Moreover,  $-\frac{b_{j_1}}{w_{j_1}} > 0$  and  $-\frac{b_{j_2}}{w_{j_2}} > 0$  so we also have  $-\frac{v_{j_1}b_{j_1}b_{j_2}}{v_{j_2}} > 0$ . Hence, Eq. (13) shows that  $\| \pmb{\theta}\|^{2} > \| \pmb{\theta}_{\delta}\|^{2}$  for small enough  $\delta$ , contradicting the assumption the  $\pmb{\theta}$  is a local minimum.
Case (3). The assumption implies that there exists  $j_1 \in [n]$  such that  $-\frac{b_{j_1}}{w_{j_1}} = z \in (x_i, x_{i+1})$ ,  $v_{j_1} > 0$  and  $-\frac{b_{j_1+1}}{w_{j_1+1}} \geq x_{i+1}$ . Let  $j_2 > j_1$  be the minimal index  $j \in \{j_1 + 1, \dots, n\}$  such that  $v_j < 0$  (see Footnote 7). Consider the perturbed netwrk as in Eq. (10) and continue the proof as in Case (1) verbatim.
Proof of Lemma A.7. Recall that  $N_{\pmb{\theta}}(x_i) \leq -1$ , so assume towards contradiction that  $N_{\pmb{\theta}}(x_i) < -1$ . Note that since  $N_{\pmb{\theta}}(x_{i-1}) \geq 1$ ,  $N_{\pmb{\theta}}(\cdot)$  must decrease along  $(x_{i-1}, x_i)$ , so there must exist  $j \in [n]$  with  $v_j < 0$  and  $-\frac{b_j}{w_j} < x_i$ . Denote by  $j_1 \in [n]$  the maximal such index. Similarly, since  $N_{\pmb{\theta}}(x_{i+1}) \geq 1$  there must exist  $j_2 \in [n]$  with  $v_{j_2} > 0$  and  $x_{i-1} < -\frac{b_{j_2}}{w_{j_2}} < x_{i+1}$ . Consider the perturbed network
$$
\begin{array}{l} N _ {\boldsymbol {\theta} _ {\delta}} (x) := \sum_ {j \in [ n ] \backslash \{j _ {1}, j _ {2} \}} v _ {j} \sigma \left(w _ {j} \cdot x + b _ {j}\right) \tag {14} \\ + (1 - \delta) v _ {j _ {1}} \sigma (w _ {j _ {1}} \cdot x + b _ {j _ {1}}) + v _ {j _ {2}} \sigma (\left(w _ {j _ {2}} + \frac {\delta v _ {j _ {1}} w _ {j _ {1}}}{v _ {j _ {2}}}\right) \cdot x + \left(b _ {j _ {2}} + \frac {\delta v _ {j _ {1}} b _ {j _ {1}}}{v _ {j _ {2}}}\right)) \\ \end{array}
$$
for some small  $\delta > 0$ . It is clear that  $\| \pmb{\theta} - \pmb{\theta}_{\delta}\| \stackrel{\delta \to 0}{\longrightarrow} 0$ , and we will show that for small enough  $\delta$  the network above still satisfies the margin condition, yet has smaller parameter norm. To see why the margin condition is not violated for small enough  $\delta$ , first notice that  $N_{\pmb{\theta}_{\delta}}(x) = N_{\pmb{\theta}}(x)$  for all  $x \notin (x_{i-1}, x_{i+1})$ . Furthermore,  $N_{\pmb{\theta}_{\delta}}(x_i)$  is continuous with respect to  $\delta$  so  $N_{\pmb{\theta}}(x_i) < -1$  implies that  $N_{\pmb{\theta}_{\delta}}(x_i) < -1$  for small enough  $\delta$ . As to the parameter norm, by a similar computation to that leading up to Eq. (11) we get that
$$
\left\| \boldsymbol {\theta} \right\| ^ {2} = \left\| \boldsymbol {\theta} _ {\delta} \right\| ^ {2} + 2 \delta \left(v _ {j _ {1}} ^ {2} - \frac {v _ {j _ {1}} w _ {j _ {1}} w _ {j _ {2}}}{v _ {j _ {2}}} - \frac {v _ {j _ {1}} b _ {j _ {1}} b _ {j _ {2}}}{v _ {j _ {2}}}\right) + O \left(\delta^ {2}\right). \tag {15}
$$
By construction,  $v_{j_1} < 0$  and  $v_{j_2} > 0$  hence  $-\frac{v_{j_1}w_{j_1}w_{j_2}}{v_{j_2}} > 0$ . Moreover,  $-\frac{b_{j_1}}{w_{j_1}}$  and  $-\frac{b_{j_2}}{w_{j_2}} > 0$  so we also have  $-\frac{v_{j_1}b_{j_1}b_{j_2}}{v_{j_2}} > 0$ . Hence, Eq. (15) shows that  $\| \pmb{\theta}\|^{2} > \| \pmb{\theta}_{\delta}\|^{2}$  for small enough  $\delta$ , contradicting the assumption the  $\pmb{\theta}$  is a local minimum.
Having proved that  $N_{\theta}(x_i) = -1$ , we turn to prove that  $N_{\theta}(x_{i + 1}) = 1$ . Knowing that  $N_{\theta}(x_{i + 1}) \geq 1$  we assume towards contradiction that  $N_{\theta}(x_{i + 1}) > 1$ . Recalling that  $N_{\theta}(x_{i - 1}) \geq 1$ ,  $N_{\theta}(x_i) \leq 1$  and that  $N_{\theta}(\cdot)$  is linear along  $(x_{i}, x_{i + 1})$  due to Lemma A.6, we conclude that there must exist  $j_1 \in [n]$  such that  $v_{j_1} > 0$  and  $x_{i - 1} < -\frac{b_{j_1}}{w_{j_1}} \leq x_i$ . Denote by  $j_2 > j_1$  the minimal index such that  $v_{j_2} < 0$ . Consider once again the perturbed network in Eq. (14) (only now  $j_1, j_2$  are different, as we just described). The same argument as in the previous part of the proof shows that for small enough  $\delta$  the network above still satisfies the margin condition, while Eq. (15) once again implies that  $\| \pmb{\theta} \|^2 > \| \pmb{\theta}_\delta \|^2$  for small enough  $\delta$  contradicting the assumption the  $\pmb{\theta}$  is a local minimum.
Overall, combining Lemma A.6 and Lemma A.7 finishes the proof of Proposition A.4.
Proof of Proposition A.5. The proof is essentially the same as Case (1) in the proof of Proposition A.4.
Throughout the proof we fix  $i \in [m - 1]$  for which the conditions described in the proposition hold, and we assume without loss of generality that the neurons are ordered with respect to their activation point:  $-\frac{b_1}{w_1} \leq -\frac{b_2}{w_2} \leq \dots \leq -\frac{b_n}{w_n}$ . As in the proof of Proposition A.4, we may assume without loss of generality that  $w_1, \ldots, w_n \geq 0$ . Moreover, as explained there as a consequence of Eq. (9), we observe that  $N_{\pmb{\theta}}'(z_1) < N_{\pmb{\theta}}'(z_2)$  for some  $z_1 < z_2$  implies the existence of  $j_1 \in [n]$  with  $v_{j_1} > 0$  and  $z_1 < -\frac{b_{j_1}}{w_{j_1}} < z_2$ . Similarly,  $N_{\pmb{\theta}}'(z_1) > N_{\pmb{\theta}}'(z_2)$  implies that there exists  $j_2 \in [n]$  with  $v_{j_2} < 0$ ,  $z_1 < -\frac{b_{j_2}}{w_{j_2}} < z_2$ . With this choice of  $j_1, j_2$ , the proof continues as in Case (1) in the proof of Proposition A.4 verbatim.
Lemma A.7.  $N_{\pmb{\theta}}(x_i) = -1$  and  $N_{\pmb{\theta}}(x_{i + 1}) = 1$
B Proofs of benign overfitting
We sample a dataset  $\mathbf{x}_1, \ldots, \mathbf{x}_m \sim \mathrm{Unif}(\mathbb{S}^{d-1})$ , and labels  $y_1, \ldots, y_m \sim \mathcal{D}_y$ , for  $p \leq c_1$  for some universal constant  $c_1$ . We first prove that the following properties hold with probability  $> 1 - \delta$ :
1. For every  $i,j\in [m],|\langle \mathbf{x}_i,\mathbf{x}_j\rangle |\leq \sqrt{\frac{2\log\left(\frac{3m^2}{\delta}\right)}{d}}$
2.  $\| XX^{\top}\| \leq C$  where  $C$  is some universal constant, and  $X$  is a matrix whose rows are equal to  $\mathbf{x}_i$ .
3.  $|I_{-}| \leq \frac{3pm}{2}$ .
Lemma B.1. Let  $\delta > 0$ , assume that we sample  $\mathbf{x}_1, \ldots, \mathbf{x}_m \sim \mathrm{Unif}(\mathbb{S}^{d-1})$ , and  $y_1, \ldots, y_m \sim \mathcal{D}_y$  for  $p \leq c_1$ , and that  $m > c_2 \frac{\log\left(\frac{1}{\delta}\right)}{p}$ , for some universal constant  $c_1, c_2 > 0$ . Then, with probability  $>1 - \delta$  properties 1, 2, 3 holds.
Proof. By Lemma D.1 we have that  $\operatorname*{Pr}\left[|\mathbf{x}_i^\top \mathbf{x}_j| \geq \sqrt{\frac{2\log\left(\frac{1}{\delta'}\right)}{d}}\right] \leq \delta'$ . Take  $\delta' = \frac{\delta}{3m^2}$ , and use union bound over all  $i, j \in [m]$  with  $i \neq j$ . This shows that for every  $i \neq j$  we have:
$$
\operatorname * {P r} \left[ | \mathbf {x} _ {i} ^ {\top} \mathbf {x} _ {j} | \geq \sqrt {\frac {2 \log \left(\frac {3 m ^ {2}}{\delta}\right)}{d}} \right] \leq \frac {\delta}{3}.
$$
This proves Property 1. Next, set  $X$  to be the matrix whose rows are equal to  $\mathbf{x}_i$ . By Lemma D.2 there is a constant  $c' > 0$  such that:
$$
\Pr \left[ \| X X ^ {\top} - I \| \geq \frac {c ^ {\prime}}{d} \left(\sqrt {\frac {d + \log \left(\frac {3}{\delta}\right)}{m}} + \frac {d + \log \left(\frac {3}{\delta}\right)}{m}\right) \right] \leq \frac {\delta}{3}. \tag {16}
$$
We can also bound  $\| XX^{\top} \| \leq \| I \| + \| XX^{\top} - I \| \leq 1 + \| XX^{\top} - I \|$ . Combining both bounds and using the assumption that  $m \geq \log \left( \frac{3}{\delta} \right)$  we get that there is a universal constant  $C > 0$  such that  $\operatorname*{Pr}\left[ \| XX^{\top} \| > C \right] \leq \frac{\delta}{3}$ . This proves Property 2.
Finally, using Bernstein's inequality over the choice of the labels  $y_{i}$  have that:
$$
\operatorname * {P r} \left[ | I _ {-} | \geq \frac {3 p m}{2} \right] \leq \exp \left(- \frac {p ^ {2} m ^ {2} / 4}{m p (1 - p) + m p / 6}\right) \leq \exp \left(- \frac {p m}{5}\right),
$$
where we used that  $p \leq 1$ . Hence, if  $m \geq c_2 \frac{\log\left(\frac{1}{\delta}\right)}{p}$  for some universal constant  $c_2 \geq 0$ , then  $|I_{-}| \leq \frac{3pm}{2}$ . Applying union over those three arguments proves the lemma.
From now on we condition on the event that properties 1, 2, 3 hold, and our bounds will depend on the probability of this event.
In the following lemma we show that if  $\theta$  converges to a solution of the max margin problem and the bias terms are relatively small, then the norm of  $\theta$  is relatively large:
Lemma B.2. Assume that  $m \geq c_2 \log \left(\frac{3}{\delta}\right)$  and that  $\theta = (\mathbf{w}_j, v_j, b_j)_{j=1}^n$  is a solution to the max margin problem of Eq. (1) with  $\sum_{j=1}^{n} v_j \sigma(b_j) \leq \frac{1}{2}$ . Then,  $\sum_{j=1}^{n} \| \mathbf{w}_j \|^2 + v_j^2 + b_j^2 \geq C \sqrt{|I_+|}$  where  $C$  is some universal constant.
Proof. Take  $i \in [m]$ , then we have that  $N_{\theta}(\mathbf{x}_i) \geq 1$ . By our assumption, this implies that:
$$
\begin{array}{l} \frac {1}{2} \leq N _ {\boldsymbol {\theta}} (\mathbf {x} _ {i}) - \sum_ {j = 1} ^ {n} v _ {j} \sigma (b _ {j}) \\ = \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i} + b _ {j}\right) - \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(b _ {j}\right) \\ \leq \sum_ {j = 1} ^ {n} | v _ {j} | \cdot \left| \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i} + b _ {j}\right) - \sigma \left(b _ {j}\right) \right| \\ \leq \sum_ {j = 1} ^ {n} | v _ {j} | \cdot | \mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i} | \\ \leq \sqrt {\sum_ {j = 1} ^ {n} v _ {j} ^ {2}} \sqrt {\sum_ {j = 1} ^ {n} \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i}\right) ^ {2}}, \\ \end{array}
$$
where in the last inequality we used Cauchy-Schwartz. Denote by  $S \coloneqq \sum_{j=1}^{n} \| \mathbf{w}_j \|^2 + v_j^2 + b_j^2$ . Combining the above and that  $\sqrt{\sum_{j=1}^{n} v_j^2} \leq \sqrt{S}$  we get:
$$
\sum_ {j = 1} ^ {n} \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i}\right) ^ {2} \geq \frac {1}{4 S}.
$$
We sum the above inequality for every  $i \in I_{+}$  to get:
$$
\begin{array}{l} \frac {\left| I _ {+} \right|}{4 S} \leq \sum_ {j = 1} ^ {n} \sum_ {i = 1} ^ {m} \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i}\right) ^ {2} \\ = \sum_ {j = 1} ^ {n} \sum_ {i = 1} ^ {m} \mathbf {w} _ {j} ^ {\top} \left(\mathbf {x} _ {i} \mathbf {x} _ {i} ^ {\top}\right) \mathbf {w} _ {j} \\ \leq \sum_ {j = 1} ^ {n} \left\| \mathbf {w} _ {j} \right\| ^ {2} \cdot \left\| \sum_ {i = 1} ^ {m} \mathbf {x} _ {i} \mathbf {x} _ {i} ^ {\top} \right\| \\ \leq \sum_ {j = 1} ^ {n} \| \mathbf {w} _ {j} \| ^ {2} \cdot C \leq S \cdot C \\ \end{array}
$$
where in the second to last to last inequality we used the Property 2 for some constant  $C > 0$  and in the last inequality we used that  $\sum_{j=1}^{n} \| \mathbf{w}_j \|^2 \leq S$ . Rearranging the above terms yields:
$$
S \geq \sqrt {\frac {| I _ {+} |}{1 2 C}}.
$$
We now prove a lemma which constructs a specific solution that achieves a norm bound that depends on  $|I_{+}|$ .
Lemma B.3. Assume  $d \geq 50m^2\log\left(\frac{3m^2}{\delta}\right)$  and  $p \leq \frac{1}{4}$ . There exists weights  $\boldsymbol{\theta} = (\mathbf{w}_j, v_j, b_j)_{j=1}^n$  that attain a margin of at least 1 on every sample and have  $\sum_{j=1}^{n} \|\mathbf{w}_j\|^2 + v_j^2 + b_j^2 \leq 9\sqrt{|I_-|}$ .
Proof. Assume without loss of generality that  $n$  is even (otherwise fix the last neuron to be 0). We consider the following weight assignment: For every  $j \leq \frac{n}{2}$ , consider  $\mathbf{w}_j = -\sqrt{\frac{4}{n\sqrt{|I_-|}}}\sum_{i \in I_-} \mathbf{x}_i$ ,
$v_{j} = 2\sqrt{\frac{\sqrt{|I_{-}|}}{n}}$  and  $b_{j} = \sqrt{\frac{4}{n\sqrt{|I_{-}|}}}$ . For every  $j > \frac{n}{2}$ , consider  $\mathbf{w}_j = \sqrt{\frac{4}{n\sqrt{|I_{-}|}}}\sum_{i\in I_{-}}\mathbf{x}_i$ ,  $v_{j} = -2\sqrt{\frac{\sqrt{|I_{-}|}}{n}}$  and  $b_{j} = 0$ .
We first show that this solution attains a margin of at least 1. For every  $i \in I_{+}$  we have:
$$
\begin{array}{l} N _ {\boldsymbol {\theta}} (\mathbf {x} _ {i}) = \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i} + b _ {j}\right) \\ = \sum_ {j \leq n / 2} 2 \sqrt {\frac {\sqrt {| I _ {-} |}}{n}} \sigma \left(\sqrt {\frac {4}{n \sqrt {| I _ {-} |}}} - \sqrt {\frac {4}{n \sqrt {| I _ {-} |}}} \sum_ {r \in I _ {-}} \mathbf {x} _ {r} ^ {\top} \mathbf {x} _ {i}\right) \\ - \sum_ {j > n / 2} 2 \sqrt {\frac {\sqrt {| I _ {-} |}}{n}} \sigma \left(\sqrt {\frac {4}{n \sqrt {| I _ {-} |}}} \sum_ {r \in I _ {-}} \mathbf {x} _ {r} ^ {\top} \mathbf {x} _ {i}\right) \\ \geq \sum_ {j \leq n / 2} \frac {4}{n} \sigma \left(1 - \frac {\left| I _ {-} \right| \sqrt {2 \log \left(\frac {3 m ^ {2}}{\delta}\right)}}{\sqrt {d}}\right) - \sum_ {j > n / 2} \frac {4}{n} \sigma \left(\frac {\left| I _ {-} \right| \sqrt {2 \log \left(\frac {3 m ^ {2}}{\delta}\right)}}{\sqrt {d}}\right) \\ \geq \frac {n}{2} \cdot \frac {4}{n} \cdot \left(1 - \frac {1}{1 0}\right) - \frac {n}{2} \cdot \frac {4}{n} \cdot \frac {1}{1 0} \geq 1, \\ \end{array}
$$
where we Property 1, that  $p \leq \frac{1}{4}$  hence by Property 3  $|I_{-}| \leq \frac{m}{2}$  and our assumption on  $m$  and  $d$ . For  $i \in I_{-}$  we have:
$$
\begin{array}{l} N _ {\boldsymbol {\theta}} (\mathbf {x} _ {i}) = \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} _ {i} + b _ {j}\right) \\ = \sum_ {j \leq n / 2} 2 \sqrt {\frac {\sqrt {| I _ {-} |}}{n}} \sigma \left(\sqrt {\frac {4}{n \sqrt {| I _ {-} |}}} - \sqrt {\frac {4}{n \sqrt {| I _ {-} |}}} \sum_ {r \in I _ {-}} \mathbf {x} _ {r} ^ {\top} \mathbf {x} _ {i}\right) \\ - \sum_ {j > n / 2} 2 \sqrt {\frac {\sqrt {| I _ {-} |}}{n}} \sigma \left(\sqrt {\frac {4}{n \sqrt {| I _ {-} |}}} \sum_ {r \in I _ {-}} \mathbf {x} _ {r} ^ {\top} \mathbf {x} _ {i}\right) \\ = \sum_ {j \leq n / 2} \frac {4}{n} \sigma \left(- \sum_ {r \in I _ {-} \backslash \{i \}} \mathbf {x} _ {r} ^ {\top} \mathbf {x} _ {i}\right) - \sum_ {j > n / 2} \frac {4}{n} \sigma \left(1 + \sum_ {r \in I _ {-} \backslash \{i \}} \mathbf {x} _ {r} ^ {\top} \mathbf {x} _ {i}\right) \\ \leq \sum_ {j \leq n / 2} \frac {4}{n} \sigma \left(- \frac {\left(| I _ {-} | - 1\right) \sqrt {2 \log \left(\frac {3 m ^ {2}}{\delta}\right)}}{\sqrt {d}}\right) - \sum_ {j > n / 2} \frac {4}{n} \sigma \left(1 - \frac {\left(| I _ {-} | - 1\right) \sqrt {2 \log \left(\frac {3 m ^ {2}}{\delta}\right)}}{\sqrt {d}}\right) \\ \leq \frac {n}{2} \cdot \frac {4}{n} \cdot \frac {1}{1 0} - \frac {n}{2} \cdot \frac {4}{n} \cdot \left(1 - \frac {1}{1 0}\right) \leq - 1, \\ \end{array}
$$
where again we used properties 1 and 3, that  $p \leq \frac{1}{4}$  hence  $|I_{-}| \leq \frac{m}{2}$  and our assumption on  $m$  and  $d$ . This shows that this is indeed a feasible solution. We turn to calculate the norm of this solution. First we bound the following:
$$
\begin{array}{l} \left\| \sum_ {i \in I _ {-}} \mathbf {x} _ {i} \right\| ^ {2} = \sum_ {i \in I _ {-}} \| \mathbf {x} _ {i} \| ^ {2} + \sum_ {i \neq j, i, j \in I _ {-}} \mathbf {x} _ {i} ^ {\top} \mathbf {x} _ {j} \\ \leq | I _ {-} | + | I _ {-} | ^ {2} \cdot \frac {\sqrt {2 \log \left(\frac {3 m ^ {2}}{\delta}\right)}}{\sqrt {d}} \leq | I _ {-} | \cdot \frac {1 1}{1 0}, \\ \end{array}
$$
where we used Property 1 and the assumption on  $m$  and  $d$ . We now use the above calculation to bound the norm of our solution:
$$
\begin{array}{l} \sum_ {j = 1} ^ {n} \left\| \mathbf {w} _ {j} \right\| ^ {2} + v _ {j} ^ {2} + b _ {j} ^ {2} = n \cdot \frac {4}{n \sqrt {| I _ {-} |}} \left\| \sum_ {i \in I _ {-}} \mathbf {x} _ {i} \right\| ^ {2} + \frac {2 n \sqrt {| I _ {-} |}}{n} + \frac {n}{2} \cdot \frac {4}{n \sqrt {| I _ {-} |}} \\ \leq \frac {4 4 \sqrt {| I _ {-} |}}{1 0} + 2 \sqrt {| I _ {-} |} + \frac {2}{\sqrt {| I _ {-} |}} \leq 9 \sqrt {| I _ {-} |} \\ \end{array}
$$
We are now ready to prove the main theorem of this subsection:
Proof of Theorem 4.1. Denote by  $K \coloneqq \sum_{j=1}^{n} \| \mathbf{w}_j \|^2 + v_j^2 + b_j^2$ , and assume that  $K \leq \frac{a}{\sqrt{p}} \| \boldsymbol{\theta}^* \|^2$  where  $a$  will be chosen later and  $\boldsymbol{\theta}^*$  is a solution to the max margin solution from Eq. (1). Assume on the way of contradiction that  $\sum_{j=1}^{n} v_j \sigma(b_j) \leq \frac{1}{2}$ , then by Lemma B.2 we know that:  $K \geq C \sqrt{|I_+|} \geq C \sqrt{\left(1 - \frac{3p}{2}\right)m}$ . On the other hand, by Lemma B.3 we know that  $\| \boldsymbol{\theta}^* \| \leq 9 \sqrt{|I_-|} \leq 9 \sqrt{\frac{3p}{2}m}$ . Combining this with the assumption we have on  $K$  we get that:
$$
C \sqrt {\left(1 - \frac {3 p}{2}\right) m} \leq \frac {9 a}{\sqrt {p}} \sqrt {\frac {3 p m}{2}}.
$$
Picking  $a$  to be some constant with  $a < \frac{C\sqrt{2}}{18\sqrt{3}}$  contradicts the above inequality. Thus, there exists a constant  $c_4 := \frac{a}{2}$  such that if  $K \leq \frac{c_4}{\sqrt{p}} S$  then  $\sum_{j=1}^{n} v_j \sigma(b_j) > \frac{1}{2}$ .
Suppose we sample  $\mathbf{x} \sim \mathcal{N}\left(0, \frac{1}{d} I\right)$ , then we have:
$$
\begin{array}{l} N _ {\boldsymbol {\theta}} (\mathbf {x}) = \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} + b _ {j}\right) \\ = \sum_ {j = 1} ^ {n} v _ {j} \sigma (b _ {j}) - \left(\sum_ {j = 1} ^ {n} v _ {j} \sigma (b _ {j}) - \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} + b _ {j}\right)\right) \\ \geq \frac {1}{2} - \left(\sum_ {j = 1} ^ {n} v _ {j} \sigma (b _ {j}) - \sum_ {j = 1} ^ {n} v _ {j} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} + b _ {j}\right)\right) \\ \geq \frac {1}{2} - \sum_ {j = 1} ^ {n} | v _ {j} | \cdot | \mathbf {w} _ {j} ^ {\top} \mathbf {x} | \\ \geq \frac {1}{2} - \sqrt {\sum_ {j = 1} ^ {n} v _ {j} ^ {2}} \sqrt {\sum_ {j = 1} ^ {n} \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x}\right) ^ {2}}. \tag {17} \\ \end{array}
$$
We will now bound the terms of the above equation. Note that  $\sum_{j=1}^{n} v_{j}^{2} \leq K \leq 9\sqrt{m}$ . For the second term, we denote by  $\bar{\mathbf{w}}_{j} := \frac{\mathbf{w}_{j}}{\|\mathbf{w}_{j}\|}$ , and write:
$$
\begin{array}{l} \sum_ {j = 1} ^ {n} (\mathbf {w} _ {j} ^ {\top} \mathbf {x}) ^ {2} = \sum_ {j = 1} ^ {n} \| \mathbf {w} _ {j} \| ^ {2} (\bar {\mathbf {w}} _ {j} ^ {\top} \mathbf {x}) ^ {2} \\ \leq \max  _ {j \in [ n ]} \left(\bar {\mathbf {w}} _ {j} ^ {\top} \mathbf {x}\right) ^ {2} \cdot \sum_ {j = 1} ^ {n} \left\| \mathbf {w} _ {j} \right\| ^ {2}. \\ \end{array}
$$
Again, we have that  $\sum_{j=1}^{n} \| \mathbf{w}_j \|^2 \leq K \leq 9\sqrt{m}$ . By using the stationarity KKT condition from Eq. (2), we get that  $\bar{\mathbf{w}}_j \in \operatorname{span}\{\mathbf{x}_1, \ldots, \mathbf{x}_m\}$ , thus we can write  $\bar{\mathbf{w}}_j = \sum_{i=1}^{m} \alpha_{i,j} \mathbf{x}_i = X \boldsymbol{\alpha}_j$  where
$X$  is a matrix with rows equal to  $\mathbf{x}_i$ , and  $(\alpha_j)_i = \alpha_{i,j}$ . We will bound  $a \coloneqq \arg \max_i |\alpha_{i,j}|$ :
$$
\begin{array}{l} 1 = \left\| \bar {\mathbf {w}} _ {j} \right\| ^ {2} = \left\| \sum_ {i = 1} ^ {m} \alpha_ {i, j} \mathbf {x} _ {i} \right\| ^ {2} \\ = \| X \boldsymbol {\alpha} _ {j} \| ^ {2} = \boldsymbol {\alpha} _ {j} ^ {\top} X ^ {\top} X \boldsymbol {\alpha} _ {j} \\ = \alpha_ {j} ^ {\top} I \alpha_ {j} + \alpha_ {j} ^ {\top} \left(X ^ {\top} X - I\right) \alpha_ {j} \\ \geq \left\| \boldsymbol {\alpha} _ {j} \right\| ^ {2} - \left\| \boldsymbol {\alpha} _ {j} \right\| ^ {2} \| X ^ {\top} X - I \| \\ = \| \boldsymbol {\alpha} _ {j} \| ^ {2} - \| \boldsymbol {\alpha} _ {j} \| ^ {2} \| X X ^ {\top} - I \|, \\ \end{array}
$$
where the last equality is true by using the SVD decomposition of  $X$ . Namely, write  $X = USV^{\top}$ , then  $\|X^{\top}X - I\| = \|VS^{2}V^{\top} - I\| = \|S^{2} - I\| = \|U^{\top}S^{2}U - I\| = \|XX^{\top} - I\|$ . Note that in Lemma B.1, Eq. (16) we have shown that  $\|XX^{\top} - I\| \leq c'$  for some constant  $c'$ . Note that we can choose  $m$  large enough such that  $c' \leq \frac{1}{2}$ . In total, this shows that  $\| \alpha_{j}\|^{2} \leq c''$  for some constant  $c''$ .
We now use Lemma D.1 and the union bound to get that with probability  $>1 - \epsilon$  we have for every  $i\in I$  that  $|\mathbf{x}^{\top}\mathbf{x}_i|\leq \sqrt{\frac{2\log\left(\frac{m}{\epsilon}\right)}{d}}$ . We condition on this event from now on. Applying both bounds we get:
$$
\begin{array}{l} \max  _ {j \in [ n ]} (\bar {\mathbf {w}} _ {j} ^ {\top} \mathbf {x}) \leq \max  _ {j \in [ n ]} \left(\sum_ {i = 1} ^ {m} \alpha_ {i, j} | \mathbf {x} _ {i} ^ {\top} \mathbf {x} |\right) \\ \leq \sqrt {\frac {2 \log \left(\frac {m}{\epsilon}\right)}{d}} \max  _ {j \in [ n ]} \| \boldsymbol {\alpha} _ {j} \| \\ \leq c ^ {\prime \prime} \sqrt {\frac {2 \log \left(\frac {m}{\epsilon}\right)}{d}} , \\ \end{array}
$$
Plugging in the bounds above to Eq. (17) we get:
$$
\begin{array}{l} N _ {\boldsymbol {\theta}} (\mathbf {x}) \geq \frac {1}{2} - \sqrt {\sum_ {j = 1} ^ {n} v _ {j} ^ {2}} \sqrt {\sum_ {j = 1} ^ {n} \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x}\right) ^ {2}} \\ \geq \frac {1}{2} - 9 \sqrt {m} \cdot 9 \sqrt {m} \cdot c ^ {\prime \prime} \sqrt {\frac {2 \log \left(\frac {m}{\epsilon}\right)}{d}} \\ \geq \frac {1}{2} - \frac {8 1 m c ^ {\prime \prime} \sqrt {2 \log \left(\frac {m}{\epsilon}\right)}}{\sqrt {d}}. \\ \end{array}
$$
By choosing a constant  $c_{3} > 0$  large enough such that if  $d \geq c_{3}m^{2}\sqrt{\log\left(\frac{m}{\epsilon}\right)}$  then, then  $\frac{81mc''\sqrt{2\log\left(\frac{m}{\epsilon}\right)}}{\sqrt{d}} \leq \frac{1}{4}$  we get that  $N_{\theta}(\mathbf{x}) > 0$ .
B.2 Proof of Theorem 4.3
1. For every  $i,j\in [m],|\langle \mathbf{x}_i,\mathbf{x}_j\rangle |\leq \sqrt{\frac{2\log\left(\frac{2m^2}{\delta}\right)}{d}}$
2.  $|I_{-}| \leq \frac{c_{1}m}{n^{2}}$ .
Lemma B.4. Let  $\delta > 0$ , assume that we sample  $\mathbf{x}_1, \ldots, \mathbf{x}_m \sim \mathrm{Unif}(\mathbb{S}^{d-1})$ , and  $y_1, \ldots, y_m \sim \mathcal{D}_y$  for  $p \leq \frac{c}{n^2}$ , and that  $m > c' \log \left(\frac{2}{\delta}\right)$ , for some universal constant  $c, c' > 0$ . Then, with probability  $>1 - \delta$  properties 1 and 2 hold.
The proof is the same as in Lemma B.1 so we will not repeat it for conciseness. The only different is that here we only need  $|I_{-}|$  to be smaller than  $\frac{c_1m}{n^2}$  which is independent of  $p$ . Hence, for  $p \leq \frac{c}{n^2}$  we get this concentration where  $m$  depends only on the probability. From now on we condition on the event that properties 1 and 2 hold, and our bounds will depend on the probability that this event happens.
In this section we consider a network of the form:
$$
N _ {\boldsymbol {\theta}} (\mathbf {x}) = \sum_ {j = 1} ^ {n / 2} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} + b _ {j}\right) - \sum_ {j = n / 2 + 1} ^ {n} \sigma \left(\mathbf {w} _ {j} ^ {\top} \mathbf {x} + b _ {j}\right)
$$