SlowGuess/NeurIPS_2023_OCR huggingface dataset:
* If  $y_{i} = 1$  the oracle sets  $\tilde{y} = 0$ . Since  $S$  is pseudorandom, we have  $P_{\mathbf{x}}(\mathbf{z}^{S}) = P_{\mathbf{x}}(\mathbf{z}^{S_{i}}) = y_{i} = 1$ . Hence, in the computation  $\hat{N}(\tilde{\mathbf{z}})$  there exists a neuron in  $\mathcal{E}_{1}$  with output at least  $\frac{3}{2}$  (by Property (P1)). Since all incoming weights to the output neuron in  $\hat{N}$  are in  $[-\frac{11}{10}, -\frac{9}{10}]$ , and  $\hat{b} \in [\frac{9}{10}, \frac{11}{10}]$ , then the input to output neuron (including the bias term) is at most  $\frac{11}{10} - \frac{3}{2} \cdot \frac{9}{10} < 0$ , and thus its output is 0.
* If  $y_{i} = 0$  the oracle sets  $\tilde{y} = [\hat{b} -\hat{N}_{3}(\tilde{\mathbf{z}})]_{+}$ . Since  $S$  is pseudorandom, we have  $P_{\mathbf{x}}(\mathbf{z}^{S}) = P_{\mathbf{x}}(\mathbf{z}^{S_{i}}) = y_{i} = 0$ . Therefore, in the computation  $\hat{N} (\tilde{\mathbf{z}})$  all neurons in  $\mathcal{E}_1,\mathcal{E}_2$  have output 0 (by Properties (P1) and (P2)), and hence their contribution to the output of  $\hat{N}$  is 0. Thus, by the definition of  $\hat{N}_{3}$ , we have  $\hat{N} (\tilde{\mathbf{z}}) = [\hat{b} -\hat{N}_{3}(\tilde{\mathbf{z}})]_{+}$ .
Lemma A.9. If  $S$  is pseudorandom, then for a sufficiently large  $n$ , with probability greater than  $\frac{2}{3}$  we have
$$
\ell_ {I} (h ^ {\prime}) \leq \frac {2}{n}.
$$
Proof. By Lemma A.8, if  $S$  is pseudorandom then with probability at least  $\frac{39}{40}$  (over  $\xi \sim \mathcal{N}(0, \tau^2 I_p)$  and the i.i.d. inputs  $\tilde{\mathbf{z}}_i \sim \mathcal{D}$ ) the examples  $(\tilde{\mathbf{z}}_1, \tilde{y}_1), \ldots, (\tilde{\mathbf{z}}_{m(n)}, \tilde{y}_{m(n)})$  returned by the oracle are realized by  $\hat{N}$ . Recall that the algorithm  $\mathcal{L}$  is such that with probability at least  $\frac{3}{4}$  (over  $\xi \sim \mathcal{N}(0, \tau^2 I_p)$ , the i.i.d. inputs  $\tilde{\mathbf{z}}_i \sim \mathcal{D}$ , and possibly its internal randomness), given a size-  $m(n)$  dataset labeled by  $\hat{N}$ , it returns a hypothesis  $h$  such that  $\mathbb{E}_{\tilde{\mathbf{z}} \sim \mathcal{D}}\left[(h(\tilde{\mathbf{z}}) - \hat{N}(\tilde{\mathbf{z}}))^2\right] \leq \frac{1}{n}$ . Hence, with probability at least  $\frac{3}{4} - \frac{1}{40}$  the algorithm  $\mathcal{L}$  returns such a good hypothesis  $h$ , given  $m(n)$  examples labeled by our examples oracle. Indeed, note that  $\mathcal{L}$  can return a bad hypothesis only if the random choices are either bad for  $\mathcal{L}$  (when used with realizable examples) or bad for the realizability of the examples returned by our oracle. By the definition of  $h'$  and the construction of  $\hat{N}$ , if  $h$  has small error then  $h'$  also has small error, namely,
$$
\underset {\tilde {\mathbf {z}} \sim \mathcal {D}} {\mathbb {E}} \left[ \big (h ^ {\prime} (\tilde {\mathbf {z}}) - \hat {N} (\tilde {\mathbf {z}}) \big) ^ {2} \right] \leq \underset {\tilde {\mathbf {z}} \sim \mathcal {D}} {\mathbb {E}} \left[ \big (h (\tilde {\mathbf {z}}) - \hat {N} (\tilde {\mathbf {z}}) \big) ^ {2} \right] \leq \frac {1}{n}.
$$
Let  $\hat{\ell}_I(h') = \frac{1}{|I|}\sum_{i\in I}(h'(\tilde{\mathbf{z}}_i) - \hat{N} (\tilde{\mathbf{z}}_i))^2$ . Recall that by our choice of  $\tau$  we have  $\operatorname*{Pr}[\hat{b} >\frac{11}{10} ]\leq \frac{1}{n}$ . Since,  $(h^{\prime}(\tilde{\mathbf{z}}) - \hat{N} (\tilde{\mathbf{z}}))^{2}\in [0,\hat{b}^{2}]$  for all  $\tilde{\mathbf{z}}\in \mathbb{R}^{n^2}$ , by Hoeffding's inequality, we have for a sufficiently large  $n$  that
$$
\begin{array}{l} \Pr \left[\left| \hat {\ell} _ {I} (h ^ {\prime}) - \underset {\tilde {S} _ {I}} {\mathbb {E}} \hat {\ell} _ {I} (h ^ {\prime}) \right| \geq \frac {1}{n} \right] = \Pr \left[ \right.\left| \hat {\ell} _ {I} (h ^ {\prime}) - \underset {\tilde {S} _ {I}} {\mathbb {E}} \hat {\ell} _ {I} (h ^ {\prime}) \right| \geq \frac {1}{n} \left. \right| \hat {b} \leq \frac {1 1}{1 0} \left. \right] \cdot \Pr \left[ \hat {b} \leq \frac {1 1}{1 0} \right] \\ + \Pr \left[ \right.\left| \hat {\ell} _ {I} \left(h ^ {\prime}\right) - \frac {\mathbb {E}}{\hat {\mathcal {S}} _ {I}} \hat {\ell} _ {I} \left(h ^ {\prime}\right)\right| \geq \frac {1}{n} \left. \right\rvert   \hat {b} > \frac {1 1}{1 0} \left. \right] \cdot \Pr \left[ \hat {b} > \frac {1 1}{1 0} \right] \\ \leq 2 \exp \left(- \frac {2 n ^ {3}}{n ^ {2} (1 1 / 1 0) ^ {4}}\right) \cdot 1 + 1 \cdot \frac {1}{n} \\ \leq \frac {1}{4 0}. \\ \end{array}
$$
Moreover, by Lemma A.8,
$$
\Pr \left[ \ell_ {I} \left(h ^ {\prime}\right) \neq \hat {\ell} _ {I} \left(h ^ {\prime}\right) \right] \leq \Pr \left[ \exists i \in I \text {s . t .} \tilde {y} _ {i} \neq \hat {N} (\tilde {\mathbf {z}} _ {i}) \right] \leq \frac {1}{4 0}.
$$
Overall, by the union bound we have with probability at least  $1 - \left( \frac{1}{4} + \frac{1}{40} + \frac{1}{40} + \frac{1}{40} \right) > \frac{2}{3}$  for sufficiently large  $n$  that:
$$
\begin{array}{l} \bullet \mathbb {E} _ {\tilde {\mathcal {S}} _ {I}} \hat {\ell} _ {I} (h ^ {\prime}) = \mathbb {E} _ {\tilde {\mathbf {z}} \sim \mathcal {D}} \left[ (h ^ {\prime} (\tilde {\mathbf {z}}) - \hat {N} (\tilde {\mathbf {z}})) ^ {2} \right] \leq \frac {1}{n}. \\ \cdot \left| \hat {\ell} _ {I} \left(h ^ {\prime}\right) - \mathbb {E} _ {\bar {\mathcal {S}} _ {I}} \hat {\ell} _ {I} \left(h ^ {\prime}\right) \right| \leq \frac {1}{n}. \\ \cdot \ell_ {I} (h ^ {\prime}) - \hat {\ell} _ {I} (h ^ {\prime}) = 0. \\ \end{array}
$$
Combining the above, we get that if  $S$  is pseudorandom, then with probability greater than  $\frac{2}{3}$  we have
$$
\ell_ {I} (h ^ {\prime}) = \left(\ell_ {I} (h ^ {\prime}) - \hat {\ell} _ {I} (h ^ {\prime})\right) + \left(\hat {\ell} _ {I} (h ^ {\prime}) - \underset {\tilde {\mathcal {S}} _ {I}} {\mathbb {E}} \hat {\ell} _ {I} (h ^ {\prime})\right) + \underset {\tilde {\mathcal {S}} _ {I}} {\mathbb {E}} \hat {\ell} _ {I} (h ^ {\prime}) \leq 0 + \frac {1}{n} + \frac {1}{n} = \frac {2}{n}.
$$
Lemma A.10. Let  $\mathbf{z} \in \{0, 1\}^{kn}$  be a random vector whose components are drawn i.i.d. from a Bernoulli distribution, which takes the value 0 with probability  $\frac{1}{n}$ . Then, for a sufficiently large  $n$ , the vector  $\mathbf{z}$  is an encoding of a hyperedge with probability at least  $\frac{1}{\log(n)}$ .
Proof. The vector  $\mathbf{z}$  represents a hyperedge iff in each of the  $k$  size- $n$  slices in  $\mathbf{z}$  there is exactly one 0-bit and each two of the  $k$  slices in  $\mathbf{z}$  encode different indices. Hence,
$$
\begin{array}{l} \Pr \left[ \mathbf {z} \text {r e p r e s e n t s a h y p e r e d g e} \right] = n \cdot (n - 1) \cdot \dots \cdot (n - k + 1) \cdot \left(\frac {1}{n}\right) ^ {k} \left(\frac {n - 1}{n}\right) ^ {n k - k} \\ \geq \left(\frac {n - k}{n}\right) ^ {k} \left(\frac {n - 1}{n}\right) ^ {k (n - 1)} \\ = \left(1 - \frac {k}{n}\right) ^ {k} \left(1 - \frac {1}{n}\right) ^ {k (n - 1)}. \\ \end{array}
$$
Since for every  $x \in (0,1)$  we have  $e^{-x} < 1 - \frac{x}{2}$ , then for a sufficiently large  $n$  the above is at least
$$
\exp \left(- \frac {2 k ^ {2}}{n}\right) \cdot \exp \left(- \frac {2 k (n - 1)}{n}\right) \geq \exp (- 1) \cdot \exp (- 2 k) \geq \frac {1}{\log (n)}.
$$
Lemma A.11. Let  $\tilde{\mathbf{z}}\in \mathbb{R}^{n^2}$  be the vector returned by the oracle. We have
$$
\Pr \left[ \tilde {\mathbf {z}} \in \tilde {\mathcal {Z}} \right] \geq \frac {1}{2 \log (n)}.
$$
Proof. Let  $\mathbf{z}' = \tilde{\mathbf{z}}_{[kn]}$ . We have
$$
\begin{array}{l} \Pr \left[ \tilde {\mathbf {z}} \notin \tilde {\mathcal {Z}} \right] \leq \Pr \left[ \exists j \in [ k n ] \mathrm {s . t .} z _ {j} ^ {\prime} \in \left(c - \frac {1}{n ^ {2}}, c + \frac {2}{n ^ {2}}\right) \right] \\ + \Pr \left[ \Psi \left(\mathbf {z} ^ {\prime}\right) \text {d o e s n o t r e p r e s e n t a h y p e r e d g e} \right]. \tag {1} \\ \end{array}
$$
We now bound the terms in the above RHS. First, since  $\mathbf{z}'$  has the Gaussian distribution, then its components are drawn i.i.d. from a density function bounded by  $\frac{1}{2\pi}$ . Hence, for a sufficiently large  $n$  we have
$$
\Pr \left[ \exists j \in [ k n ] \text {s . t .} z _ {j} ^ {\prime} \in \left(c - \frac {1}{n ^ {2}}, c + \frac {2}{n ^ {2}}\right) \right] \leq k n \cdot \frac {1}{2 \pi} \cdot \frac {3}{n ^ {2}} = \frac {3 k}{2 \pi n} \leq \frac {\log (n)}{n}. \tag {2}
$$
Let  $\mathbf{z} = \Psi(\mathbf{z}')$ . Note that  $\mathbf{z}$  is a random vector whose components are drawn i.i.d. from a Bernoulli distribution, where the probability to get 0 is  $\frac{1}{n}$ . By Lemma A.10,  $\mathbf{z}$  is an encoding of a hyperedge with probability at least  $\frac{1}{\log(n)}$ . Combining it with Eq. (1) and (2), we get for a sufficiently large  $n$  that
$$
\operatorname * {P r} \left[ \tilde {\mathbf {z}} \notin \tilde {\mathcal {Z}} \right] \leq \frac {\log (n)}{n} + \left(1 - \frac {1}{\log (n)}\right) \leq 1 - \frac {1}{2 \log (n)},
$$
Lemma A.12. If  $S$  is random, then for a sufficiently large  $n$  with probability larger than  $\frac{2}{3}$  we have
$$
\ell_ {I} \left(h ^ {\prime}\right) > \frac {2}{n}.
$$
Proof. Let  $\tilde{\mathcal{Z}} \subseteq \mathbb{R}^{n^2}$  be such that  $\tilde{\mathbf{z}} \in \tilde{\mathcal{Z}}$  iff  $\tilde{\mathbf{z}}_{[kn]}$  does not have components in the interval  $(c - \frac{1}{n^2}, c + \frac{2}{n^2})$ , and  $\Psi(\tilde{\mathbf{z}}_{[kn]}) = \mathbf{z}^S$  for a hyperedge  $S$ . If  $\mathcal{S}$  is random, then by the definition of our examples oracle, for every  $i \in [m(n) + n^3]$  such that  $\tilde{\mathbf{z}}_i \in \tilde{\mathcal{Z}}$ , we have  $\tilde{y}_i = \hat{b}$  with probability  $\frac{1}{2}$  and  $\tilde{y}_i = 0$  otherwise. Also, by the definition of the oracle,  $\tilde{y}_i$  is independent of  $S_i$  and independent of the choice of the vector  $\tilde{\mathbf{z}}_i$  that corresponds to  $\mathbf{z}^{S_i}$ . If  $\hat{b} \geq \frac{9}{10}$  then for a sufficiently large  $n$  the
hypothesis  $h'$  satisfies for each random example  $(\tilde{\mathbf{z}}_i, \tilde{y}_i) \in \tilde{S}_I$  the following
$$
\begin{array}{l} \Pr_ {(\tilde {\mathbf {z}} _ {i}, \tilde {y} _ {i})} \left[ \left(h ^ {\prime} (\tilde {\mathbf {z}} _ {i}) - \tilde {y} _ {i}\right) ^ {2} \geq \frac {1}{5} \right] \\ \geq \Pr_ {(\tilde {\mathbf {z}} _ {i}, \tilde {y} _ {i})} \left[ \left(h ^ {\prime} (\tilde {\mathbf {z}} _ {i}) - \tilde {y} _ {i}\right) ^ {2} \geq \frac {1}{5} \mid \tilde {\mathbf {z}} _ {i} \in \tilde {\mathcal {Z}} \right] \cdot \Pr_ {\tilde {\mathbf {z}} _ {i}} \left[ \tilde {\mathbf {z}} _ {i} \in \tilde {\mathcal {Z}} \right] \\ \geq \Pr_ {(\tilde {\mathbf {z}} _ {i}, \tilde {y} _ {i})} \left[ \left(h ^ {\prime} (\tilde {\mathbf {z}} _ {i}) - \tilde {y} _ {i}\right) ^ {2} \geq \left(\frac {\hat {b}}{2}\right) ^ {2} \mid \tilde {\mathbf {z}} _ {i} \in \tilde {\mathcal {Z}} \right] \cdot \Pr_ {\tilde {\mathbf {z}} _ {i}} \left[ \tilde {\mathbf {z}} _ {i} \in \tilde {\mathcal {Z}} \right] \\ \geq \frac {1}{2} \cdot \Pr_ {\tilde {\mathbf {z}} _ {i}} \left[ \tilde {\mathbf {z}} _ {i} \in \tilde {\mathcal {Z}} \right]. \\ \end{array}
$$
In Lemma A.11, we show that  $\operatorname{Pr}_{\tilde{\mathbf{z}}_i}\left[\tilde{\mathbf{z}}_i \in \tilde{\mathcal{Z}}\right] \geq \frac{1}{2\log(n)}$ . Hence,
$$
\Pr_ {(\tilde {\mathbf {z}} _ {i}, \tilde {y} _ {i})} \left[ \left(h ^ {\prime} (\tilde {\mathbf {z}} _ {i}) - \tilde {y} _ {i}\right) ^ {2} \geq \frac {1}{5} \right] \geq \frac {1}{2} \cdot \frac {1}{2 \log (n)} \geq \frac {1}{4 \log (n)}.
$$
Thus, if  $\hat{b} \geq \frac{9}{10}$  then we have
$$
\underset {\tilde {\mathcal {S}} _ {I}} {\mathbb {E}} \left[ \ell_ {I} \left(h ^ {\prime}\right) \right] \geq \frac {1}{5} \cdot \frac {1}{4 \log (n)} = \frac {1}{2 0 \log (n)}.
$$
Therefore, for large  $n$  we have
$$
\Pr \left[ \mathbb {E} _ {\bar {S} _ {I}} \left[ \ell_ {I} \left(h ^ {\prime}\right) \right] \geq \frac {1}{2 0 \log (n)} \right] \geq 1 - \frac {1}{n} \geq \frac {7}{8}.
$$
Since,  $(h'(\tilde{\mathbf{z}}) - \tilde{y})^2 \in [0, \hat{b}^2]$  for all  $\tilde{\mathbf{z}}, \tilde{y}$  returned by the examples oracle, and the examples  $\tilde{\mathbf{z}}_i$  for  $i \in I$  are i.i.d., then by Hoeffding's inequality, we have for a sufficiently large  $n$  that
$$
\begin{array}{l} \Pr \left[\left| \ell_ {I} (h ^ {\prime}) - \underset {\tilde {S} _ {I}} {\mathbb {E}} \ell_ {I} (h ^ {\prime}) \right| \geq \frac {1}{n} \right] = \Pr \left[ \right.\left| \ell_ {I} (h ^ {\prime}) - \underset {\tilde {S} _ {I}} {\mathbb {E}} \ell_ {I} (h ^ {\prime}) \right| \geq \frac {1}{n} \left. \right| \hat {b} \leq \frac {1 1}{1 0} \left. \right] \cdot \Pr \left[ \hat {b} \leq \frac {1 1}{1 0} \right] \\ + \Pr \left[ \right.\left| \ell_ {I} \left(h ^ {\prime}\right) - \frac {\mathbb {E}}{\hat {s} _ {I}} \ell_ {I} \left(h ^ {\prime}\right)\right| \geq \frac {1}{n} \left. \right\rvert   \hat {b} > \frac {1 1}{1 0} \left. \right] \cdot \Pr \left[ \hat {b} > \frac {1 1}{1 0} \right] \\ \leq 2 \exp \left(- \frac {2 n ^ {3}}{n ^ {2} (1 1 / 1 0) ^ {4}}\right) \cdot 1 + 1 \cdot \frac {1}{n} \\ \leq \frac {1}{8}. \\ \end{array}
$$
Hence, for large enough  $n$ , with probability at least  $1 - \frac{1}{8} - \frac{1}{8} = \frac{3}{4} > \frac{2}{3}$  we have both  $\mathbb{E}_{\tilde{S}_I}[\ell_I(h')]\geq \frac{1}{20\log(n)}$  and  $\left|\ell_I(h^{\prime}) - \mathbb{E}_{\tilde{S}_I}\ell_I(h^{\prime})\right|\leq \frac{1}{n}$ , and thus
$$
\ell_ {I} \left(h ^ {\prime}\right) \geq \frac {1}{2 0 \log (n)} - \frac {1}{n} > \frac {2}{n}.
$$
B Proof of Corollary 3.1
By the proof of Theorem 3.1, under Assumption 2.1, there is no  $\mathrm{poly}(d)$ -time algorithm  $\mathcal{L}_s$  that satisfies the following: Let  $\pmb{\theta} \in \mathbb{R}^p$  be  $B$ -bounded parameters of a depth-3 network  $N_{\pmb{\theta}}: \mathbb{R}^d \to \mathbb{R}$  and let  $\tau, \epsilon > 0$ . Assume that  $p, B, 1/\epsilon, 1/\tau \leq \mathrm{poly}(d)$ , and that the widths of the hidden layers in  $\mathcal{N}_{\pmb{\theta}}$  are  $d$  (i.e., the weight matrices are square). Let  $\pmb{\xi} \in \mathcal{N}(\mathbf{0}, \tau^2 I_p)$  and let  $\hat{\pmb{\theta}} = \pmb{\theta} + \pmb{\xi}$ . Then, with probability at least  $\frac{3}{4} - \frac{1}{1000}$ , given access to an examples oracle for  $\mathcal{N}_{\hat{\pmb{\theta}}}$ , the algorithm  $\mathcal{L}_s$  returns a hypothesis  $h$  with  $\mathbb{E}_{\mathbf{x}}\left[(h(\mathbf{x}) - N_{\hat{\pmb{\theta}}})^2\right] \leq \epsilon$ .
Note that in the above, the requirements from  $\mathcal{L}_s$  are somewhat weaker than in our original definition of learning with smoothed parameters. Indeed, we assume that the widths of the hidden layers are  $d$  and the required success probability is only  $\frac{3}{4} -\frac{1}{1000}$  (rather than  $\frac{3}{4}$ ). We now explain why the hardness result holds already under these conditions:
- Note that if we change the assumption on the learning algorithm in proof of Theorem 3.1 such that it succeeds with probability at least  $\frac{3}{4} - \frac{1}{1000}$  (rather than  $\frac{3}{4}$ ), then in the case where  $S$  is pseudorandom we get that the algorithm  $\mathcal{A}$  returns 1 with probability at least  $1 - \left( \frac{1}{4} + \frac{1}{1000} + \frac{1}{40} + \frac{1}{40} + \frac{1}{40} \right)$  (see the proof of Lemma A.9), which is still greater than  $\frac{2}{3}$ . Also, the analysis of the case where  $S$  is random does not change, and thus in this case  $\mathcal{A}$  returns 0 with probability greater than  $\frac{2}{3}$ . Consequently, we still get distinguishing advantage greater than  $\frac{1}{3}$ .
- Regarding the requirement on the widths, we note that in the proof of Theorem 3.1 the layers satisfy the following. The input dimension is  $d = n^2$ , the width of the first hidden layer is at most  $3n\log (n)\leq d$ , and the width of the second hidden layer is at most  $\log (n) + 2n + n\log (n)\leq d$  (all bounds are for a sufficiently large  $d$ ). In order to get a network where all layers are of width  $d$ , we add new neurons to the hidden layers, with incoming weights 0, outgoing weights 0, and bias terms  $-1$ . Then, for an appropriate choice of  $\tau = 1 / \mathrm{poly}(n)$ , even in the perturbed network the outputs of these new neurons will be 0 w.h.p. for every input  $\tilde{\mathbf{z}}_1,\dots ,\tilde{\mathbf{z}}_{m(n) + n^3}$ , and thus they will not affect the network's output. Thus, using the same argument as in the proof of Theorem 3.1, we conclude that the hardness results hold already for network with square weight matrices.
Suppose that there exists an efficient algorithm  $\mathcal{L}_p$  that learns in the standard PAC framework depth-3 neural networks where the minimal singular value of each weight matrix is lower bounded by  $1 / q(d)$  for any polynomial  $q(d)$ . We will use  $\mathcal{L}_p$  to obtain an efficient algorithm  $\mathcal{L}_s$  that learns depth-3 networks with smoothed parameters as described above, and thus reach a contradiction.
Let  $\pmb{\theta} \in \mathbb{R}^p$  be  $B$ -bounded parameters of a depth-3 network  $N_{\pmb{\theta}}: \mathbb{R}^d \to \mathbb{R}$ , and let  $\tau, \epsilon > 0$ . Assume that  $p, B, 1/\epsilon, 1/\tau \leq \mathrm{poly}(d)$ , and that the widths of the hidden layers in  $\mathcal{N}_{\pmb{\theta}}$  are  $d$ . For random  $\pmb{\xi} \sim \mathcal{N}(\mathbf{0}, \tau^2 I_p)$  and  $\hat{\pmb{\theta}} = \pmb{\theta} + \pmb{\xi}$ , the algorithm  $\mathcal{L}_s$  has access to examples labeled by  $N_{\hat{\pmb{\theta}}}$ . Using Lemma B.1 below with  $t = \frac{\tau}{d}$  and the union bound over the two weight matrices in  $N_{\pmb{\theta}}$ , we get that with probability at least  $1 - \frac{2 \cdot 2.35}{\sqrt{d}} \geq 1 - \frac{1}{1000}$  (for large enough  $d$ ), the minimal singular values of all weight matrices in  $\hat{\pmb{\theta}}$  are at least  $\frac{\tau}{d} \geq \frac{1}{q(d)}$  for some sufficiently large polynomial  $q(d)$ . Our algorithm  $\mathcal{L}_s$  will simply run  $\mathcal{L}_p$ . Given that the minimal singular values of the weight matrices are at least  $\frac{1}{q(d)}$ , the algorithm  $\mathcal{L}_p$  runs in time  $\mathrm{poly}(d)$  and returns with probability at least  $\frac{3}{4}$  a hypothesis  $h$  with  $\mathbb{E}_{\mathbf{x}}[(h(\mathbf{x}) - N_{\hat{\pmb{\theta}}}(\mathbf{x}))^2] \leq \epsilon$ . Overall, the algorithm  $\mathcal{L}_s$  runs in  $\mathrm{poly}(d)$  time, and with probability at least  $\frac{3}{4} - \frac{1}{1000}$  (over both  $\pmb{\xi}$  and the internal randomness) returns a hypothesis  $h$  with loss at most  $\epsilon$ .
Lemma B.1 (Sankar et al. [37], Theorem 3.3). Let  $W$  be an arbitrary square matrix in  $\mathbb{R}^{d\times d}$ , and let  $P\in \mathbb{R}^{d\times d}$  be a random matrix, where each entry is drawn i.i.d. from  $\mathcal{N}(0,\tau^2)$  for some  $\tau >0$ . Let  $\sigma_d$  be the minimal singular value of the matrix  $W + P$ . Then, for every  $t > 0$  we have
$$
\operatorname * {P r} _ {P} \left[ \sigma_ {d} \leq t \right] \leq 2. 3 5 \cdot \frac {t \sqrt {d}}{\tau}.
$$
C Proof of Theorem 3.2
The proof follows similar ideas to the proof of Theorem 3.1. The main difference is that we need to handle here a smoothed discrete input distribution rather than the standard Gaussian distribution.
For a sufficiently large  $n$ , let  $\mathcal{D}$  be a distribution on  $\{0,1\}^{n^2}$ , where each component is drawn i.i.d. from a Bernoulli distribution which takes the value 0 with probability  $\frac{1}{n}$ . Assume that there is a poly  $(n)$ -time algorithm  $\mathcal{L}$  that learns depth-3 neural networks with at most  $n^2$  hidden neurons and parameter magnitudes bounded by  $n^3$ , with smoothed parameters and inputs, under the distribution  $\mathcal{D}$ , with  $\epsilon = \frac{1}{n}$  and  $\tau, \omega = 1 / \mathrm{poly}(n)$  that we will specify later. Let  $m(n) \leq \mathrm{poly}(n)$  be the sample complexity of  $\mathcal{L}$ , namely,  $\mathcal{L}$  uses a sample of size at most  $m(n)$  and returns with probability at least  $\frac{3}{4}$  a hypothesis  $h$  with  $\mathbb{E}_{\mathbf{z} \sim \hat{\mathcal{D}}} \left[ (h(\mathbf{z}) - N_{\hat{\boldsymbol{\theta}}}(\mathbf{z}))^2 \right] \leq \epsilon = \frac{1}{n}$ . Note that  $\hat{\mathcal{D}}$  is the distribution  $\mathcal{D}$  after smoothing with parameter  $\omega$ , and the vector  $\hat{\pmb{\theta}}$  is the parameters of the target network after smoothing with parameter  $\tau$ . Let  $s > 1$  be a constant such that  $n^s \geq m(n) + n^3$  for every sufficiently large  $n$ . By Assumption 2.1, there exists a constant  $k$  and a predicate  $P: \{0,1\}^k \to \{0,1\}$ , such that  $\mathcal{F}_{P,n,n^s}$
is  $\frac{1}{3}$ -PRG. We will show an efficient algorithm  $\mathcal{A}$  with distinguishing advantage greater than  $\frac{1}{3}$  and thus reach a contradiction.
Throughout this proof, we will use some notations from the proof of Theorem 3.1. We repeat it here for convenience. For a hyperedge  $S = (i_1, \ldots, i_k)$  we denote by  $\mathbf{z}^S \in \{0, 1\}^{kn}$  the following encoding of  $S$ : the vector  $\mathbf{z}^S$  is a concatenation of  $k$  vectors in  $\{0, 1\}^n$ , such that the  $j$ -th vector has 0 in the  $i_j$ -th coordinate and 1 elsewhere. Thus,  $\mathbf{z}^S$  consists of  $k$  size-  $n$  slices, each encoding a member of  $S$ . For  $\mathbf{z} \in \{0, 1\}^{kn}$ ,  $i \in [k]$  and  $j \in [n]$ , we denote  $z_{i,j} = z_{(i-1)n+j}$ . That is,  $z_{i,j}$  is the  $j$ -th component in the  $i$ -th slice in  $\mathbf{z}$ . For  $\mathbf{x} \in \{0, 1\}^n$ , let  $P_{\mathbf{x}}: \{0, 1\}^{kn} \to \{0, 1\}$  be such that for every hyperedge  $S$  we have  $P_{\mathbf{x}}(\mathbf{z}^S) = P(\mathbf{x}_S)$ . For  $\tilde{\mathbf{z}} \in \mathbb{R}^{n^2}$  we denote  $\tilde{\mathbf{z}}_{[kn]} = (\tilde{z}_1, \ldots, \tilde{z}_{kn})$ , namely, the first  $kn$  components of  $\tilde{\mathbf{z}}$  (assuming  $n^2 \geq kn$ ).
C.1 Defining the target network for  $\mathcal{L}$
Since our goal is to use the algorithm  $\mathcal{L}$  for breaking PRGs, in this subsection we define a neural network  $\tilde{N}:\mathbb{R}^{n^2}\to \mathbb{R}$  that we will later use as a target network for  $\mathcal{L}$ . The network  $\tilde{N}$  contains the subnetworks  $N_{1},N_{2}$  that we define below.
Let  $N_{1}$  be a depth-1 neural network (i.e., one layer, with activations in the output neurons) with input dimension  $kn$ , at most  $\log(n)$  output neurons, and parameter magnitudes bounded by  $n^3$  (all bounds are for a sufficiently large  $n$ ), which satisfies the following. We denote the set of output neurons of  $N_{1}$  by  $\mathcal{E}_{1}$ . Let  $\mathbf{z}' \in \{0,1\}^{kn}$  be an input to  $N_{1}$  such that  $\mathbf{z}' = \mathbf{z}^S$  for some hyperedge  $S$ . Thus, even though  $N_{1}$  takes inputs in  $\mathbb{R}^{kn}$ , we consider now its behavior for an input  $\mathbf{z}'$  with discrete components in  $\{0,1\}$ . Fix some  $\mathbf{x} \in \{0,1\}^n$ . Then, for  $S$  with  $P_{\mathbf{x}}(\mathbf{z}^S) = 0$  the inputs to all output neurons  $\mathcal{E}_{1}$  are at most  $-1$ , and for  $S$  with  $P_{\mathbf{x}}(\mathbf{z}^S) = 1$  there exists a neuron in  $\mathcal{E}_{1}$  with input at least 2. Recall that our definition of a neuron's input includes the addition of the bias term. The construction of the network  $N_{1}$  is given in Lemma A.2. Note that the network  $N_{1}$  depends on  $\mathbf{x}$ . Let  $N_{1}' : \mathbb{R}^{kn} \to \mathbb{R}$  be a depth-2 neural network with no activation function in the output neuron, obtained from  $N_{1}$  by summing the outputs from all neurons  $\mathcal{E}_{1}$ .