SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \Pr \left(\forall \vec {z} \in \mathbb {R} ^ {d}: \| \vec {z} \| _ {1} \leq 1, \exists S: | S | = k, \left\| \left(\sum_ {i \in S} X _ {i}\right) - \vec {z} \right\| _ {\max } \leq \varepsilon\right) \\ \geq \Pr \left(\forall \vec {z} \in \mathbb {R} ^ {d}: \| \vec {z} \| _ {1} \leq 1, \exists i \in [ s ], \exists S \subset G _ {i}: | S | = k, \left\| \left(\sum_ {i \in S} X _ {i}\right) - \vec {z} \right\| _ {\max } \leq \varepsilon\right) \\ = 1 - \Pr \left(\exists \vec {z} \in \mathbb {R} ^ {d}: \| \vec {z} \| _ {1} \leq 1, \forall i \in [ s ], \forall S \subset G _ {i}: | S | = k, \left\| \left(\sum_ {i \in S} X _ {i}\right) - \vec {z} \right\| _ {\max } > \varepsilon\right) \\ \geq 1 - \frac {1}{\varepsilon^ {d}} \Pr \left(\forall i \in [ s ], \forall S \subset G _ {i}: | S | = k, \left\| \left(\sum_ {i \in S} X _ {i}\right) - \vec {z} \right\| _ {\max } > \varepsilon\right) (4) \\ \geq 1 - \frac {1}{\varepsilon^ {d}} (1 - c) ^ {\left[ C _ {1} d \log \left(\frac {1}{\varepsilon}\right) \right]}, (5) \\ \end{array}
$$
where Eq. 4 holds by the union bound, Eq. 5 comes from Eq. 3 and the independence of the variables across different  $G_{i}$ . By choosing  $C_1$  large enough,
$$
1 - \varepsilon^ {- d} (1 - c) ^ {\left\lceil C _ {1} d \log \left(\frac {1}{\varepsilon}\right)\right\rceil} = 1 - 2 ^ {d \log \frac {1}{\varepsilon} - \left\lceil C _ {1} d \log \left(\frac {1}{\varepsilon}\right)\right\rceil \log \frac {1}{1 - c}} \geq 1 - \varepsilon .
$$
Lemma 20 (Tensor Convolution Inequality). Given real tensors  $K$  and  $X$  of respective sizes  $d \times d' \times c_0 \times c_1$  and  $D \times D' \times c_0$ , it holds
$$
\| K * X \| _ {\max } \leq \| K \| _ {1} \cdot \| X \| _ {\max }.
$$
$$
\left\| K * X \right\| _ {\max }
$$
$$
\begin{array}{l} \leq \max  _ {i, j \in [ D ], \ell \in [ c _ {1} ]} \sum_ {i ^ {\prime}, j ^ {\prime} \in [ d ], k \in [ c ]} | K _ {i ^ {\prime}, j ^ {\prime}, k, \ell} X _ {i - i ^ {\prime} + 1, j - j ^ {\prime} + 1, k} | \\ \leq \max  _ {i, j \in [ D ], \ell \in [ c _ {1} ]} \left(\sum_ {i ^ {\prime}, j ^ {\prime} \in [ d ], k \in [ c ]} | K _ {i ^ {\prime}, j ^ {\prime}, k, \ell} |\right) \| X \| _ {\max } \\ \leq \max  _ {i, j \in [ D ], \ell \in [ c _ {1} ]} \| K \| _ {1} \cdot \| X \| _ {\max } \\ = \| K \| _ {1} \cdot \| X \| _ {\max}. \\ \end{array}
$$
B Omitted proofs and results
B.1 Multidimensional Random Subset Sum for normally-scaled normal vectors
Proof of Lemma 7. By Definition 4, the  $j$ -th entry of each vector  $X_{i}$  is  $(X_{i})_{j} = Z_{i}\cdot Z_{i,j}$  where each  $Z_{i}$  and  $Z_{i,j}$  are i.i.d. random variables following a standard normal distribution. Let  $\mathcal{E}^{(\ddagger)}$  be the event that  $k\left(1 - 2\sqrt{c_d}\right)\leq \sum_{i = 1}^{k}Z_i^2\leq k\left(1 + 2\sqrt{c_d} +2c_d\right)$ , and denote  $X = \sum_{i = 1}^{k}X_{i}$ . By the law of total probability, it holds that
$$
\Pr \left(X \in I _ {\varepsilon} (\vec {z})\right) = \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \Pr \left(X \in I _ {\varepsilon} (\vec {z}) \mid Z _ {1}, \dots , Z _ {k}\right) \right].
$$
As, conditional on  $Z_{1}, \ldots, Z_{k}$ , the  $d$  entries of  $X$  are independent, it follows that
$$
\begin{array}{l} \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \Pr \left(X \in I _ {\varepsilon} (\vec {z}) \mid Z _ {1}, \dots , Z _ {k}\right) \right] \\ = \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {j = 1} ^ {d} \Pr \left(\left(X\right) _ {j} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {k}\right) \right] \\ \geq \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {j = 1} ^ {d} \Pr \left(\left(X\right) _ {j} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {k}, \mathcal {E} ^ {(\updownarrow)}\right) \right] \Pr \left(\mathcal {E} ^ {(\updownarrow)}\right), \tag {6} \\ \end{array}
$$
where the inequality in Eq. 6 holds by applying again the law of total probability.
Conditional on  $Z_{1},\ldots ,Z_{k}$  , we have that  $(X)_j\sim \mathcal{N}(0,\sum_{i = 1}^{k}Z_i^2)$  . Hence,
$$
\begin{array}{l} \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {j = 1} ^ {d} \Pr \left(\left(X\right) _ {j} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {k}, \mathcal {E} ^ {(\ddagger)}\right) \right] \Pr \left(\mathcal {E} ^ {(\ddagger)}\right) \\ \geq \mathbb {E} _ {Z _ {1}, \dots , Z _ {k}} \left[ \prod_ {j = 1} ^ {d} \left(\frac {2 \varepsilon}{\sqrt {\pi \left(\sum_ {i = 1} ^ {k} Z _ {i} ^ {2}\right)}} \exp \left(- \frac {\left(\left| z _ {i} \right| + \varepsilon\right) ^ {2}}{2 \sum_ {i = 1} ^ {k} Z _ {i} ^ {2}}\right)\right) \middle | \mathcal {E} ^ {(\ddagger)} \right] \\ \cdot \Pr \left(\mathcal {E} ^ {(\ddagger)}\right) \\ \end{array}
$$
Notice that the term  $\sum_{i}Z_{i}^{2}$  is a sum of chi-square random variables, for which there are known concentration bounds (Lemma 17). By definition of  $\mathcal{E}^{(\updownarrow)}$  and by applying Lemma 17 to estimate the term  $\operatorname*{Pr}\left(\mathcal{E}^{(\updownarrow)}\right)$ , we get that
$$
\begin{array}{l} \mathbb {E} _ {Z _ {1}, \dots , Z _ {k}} \left[ \prod_ {j = 1} ^ {d} \left(\frac {2 \varepsilon}{\sqrt {\pi \left(\sum_ {i = 1} ^ {k} Z _ {i} ^ {2}\right)}} \exp \left(- \frac {\left(| z _ {i} | + \varepsilon\right) ^ {2}}{2 \sum_ {i = 1} ^ {k} Z _ {i} ^ {2}}\right)\right) \middle | \mathcal {E} ^ {(+)} \right] \\ \cdot \Pr \left(\mathcal {E} ^ {(\ddagger)}\right) \\ \end{array}
$$
$$
\begin{array}{l} \geq \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {\sum_ {i} | z _ {i} | ^ {2} + 2 \varepsilon \sum_ {i} | z _ {i} | + d \varepsilon^ {2}}{2 (1 - 2 \sqrt {c _ {d}}) k}\right) \Pr \left(\mathcal {E} ^ {(\ddagger)}\right) \\ = \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {\| \vec {z} \| _ {2} ^ {2} + 2 \varepsilon \| \vec {z} \| _ {1} + d \varepsilon^ {2}}{2 (1 - 2 \sqrt {c _ {d}}) k}\right) \operatorname * {P r} \left(\mathcal {E} ^ {(\ddagger)}\right) \\ \geq \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {\| \vec {z} \| _ {2} ^ {2} + 2 \varepsilon \| \vec {z} \| _ {1} + d \varepsilon^ {2}}{2 (1 - 2 \sqrt {c _ {d}}) k}\right) (1 - 2 e ^ {- c _ {d} k}). \\ \end{array}
$$
As  $c_d k \geq 1$  by hypotheses,  $1 - 2e^{-c_d k} \geq 1/4$ . Then,
$$
\begin{array}{l} \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {\| \vec {z} \| _ {2} ^ {2} + 2 \varepsilon \| \vec {z} \| _ {1} + d \varepsilon^ {2}}{2 (1 - 2 \sqrt {c _ {d}}) k}\right) (1 - 2 e ^ {- c _ {d} k}) \\ \geq \frac {1}{4} \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {\| \vec {z} \| _ {2} ^ {2} + 2 \varepsilon \| \vec {z} \| _ {1} + d \varepsilon^ {2}}{2 (1 - 2 \sqrt {c _ {d}}) k}\right) \\ \geq \frac {1}{4} \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {k + 2 \varepsilon \sqrt {k} + d \varepsilon^ {2}}{2 (1 - 2 \sqrt {c _ {d}}) k}\right) (7) \\ = \frac {1}{4} \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {1 + \frac {2 \varepsilon}{\sqrt {k}} + \frac {d \varepsilon^ {2}}{k}}{2 (1 - 2 \sqrt {c _ {d}})}\right) \\ \geq \frac {1}{4} \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d} \exp \left(- \frac {1 + \frac {1}{8} + \frac {1}{1 6}}{2 (1 - 2 \sqrt {c _ {d}})}\right) (8) \\ \geq \frac {1}{1 6} \left(\frac {2 \varepsilon}{\sqrt {\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}}\right) ^ {d}, (9) \\ \end{array}
$$
where we have used that  $\| \vec{z}\| _2\leq \| \vec{z}\| _1\leq \sqrt{k}$  in Ineq. 7, that  $k\geq 16,k\geq d$  , and  $\varepsilon <  1 / 4$  in Ineq. 8, and that
$$
\exp \left(- \frac {1 + \frac {1}{8} + \frac {1}{1 6}}{2 (1 - 2 \sqrt {c _ {d}})}\right) \geq \exp \left(- \frac {1 + \frac {1}{8} + \frac {1}{1 6}}{2 (1 - 2 \sqrt {\frac {1}{1 6}})}\right) \geq \frac {1}{1 6}
$$
in Ineq. 9.
Proof of Lemma 8. Let  $n = k + j$ . Since the  $X_{i}$ s are NSN random vectors, for each  $i \in [n]$  and  $j \in [d]$  we can write the  $j$ -th entry of  $X_{i}$  as  $(X_{i})_{j} = Z_{i} \cdot Z_{i,j}$  where the variables in  $\{Z_{i}\}_{i \in [n]}$  and in  $\{Z_{i,j}\}_{i \in [n], j \in [d]}$  are i.i.d. random variables following a standard normal distribution. By the law of total probability, we have
$$
\begin{array}{l} \Pr \left(A + B \in I _ {\varepsilon} (\vec {z}), B + C \in I _ {\varepsilon} (\vec {z})\right) \\ = \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \Pr \left(A + B \in I _ {\varepsilon} (\vec {z}), B + C \in I _ {\varepsilon} (\vec {z}) \mid Z _ {1}, \dots , Z _ {n}\right) \right] \\ = \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {i = 1} ^ {d} \Pr \left(A _ {i} + B _ {i} \in I _ {\varepsilon} \left(z _ {i}\right), B _ {i} + C _ {i} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}\right) \right], \tag {10} \\ \end{array}
$$
where the latter equality holds by independence.
$$
\begin{array}{l} \Pr \left(A _ {i} + B _ {i} \in I _ {\varepsilon} \left(z _ {i}\right), B _ {i} + C _ {i} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}\right) \\ = \mathbb {E} _ {B _ {i}} \left[ \Pr \left(A _ {i} \in I _ {\varepsilon} \left(z _ {i} - B _ {i}\right), C _ {i} \in I _ {\varepsilon} \left(z _ {i} - B _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \right] \\ = \mathbb {E} _ {B _ {i}} \left[ \Pr \left(A _ {i} \in I _ {\varepsilon} \left(z _ {i} - B _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \Pr \left(C _ {i} \in I _ {\varepsilon} \left(z _ {i} - B _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \right], \\ \end{array}
$$
where the latter inequality holds by independence of  $A_{i}$  and  $C_i$ . By Lemma 14, it holds that
$$
\begin{array}{l} \mathbb {E} _ {B _ {i}} \left[ \Pr \left(A _ {i} \in I _ {\varepsilon} \left(z _ {i} - B _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \Pr \left(C _ {i} \in I _ {\varepsilon} \left(z _ {i} - B _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \right] \\ \leq \mathbb {E} _ {B _ {i}} \left[ \Pr \left(A _ {i} \in I _ {\varepsilon} (0) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \Pr \left(C _ {i} \in I _ {\varepsilon} (0) \mid Z _ {1}, \dots , Z _ {n}, B _ {i}\right) \right] \\ = \Pr \left(A _ {i} \in I _ {\varepsilon} (0) \mid Z _ {1}, \dots , Z _ {n}\right) \Pr \left(C _ {i} \in I _ {\varepsilon} (0) \mid Z _ {1}, \dots , Z _ {n}\right) \\ \leq \frac {2 \varepsilon}{\sqrt {\pi \left(\sum_ {r = 1} ^ {j} Z _ {r} ^ {2}\right)}} \cdot \frac {2 \varepsilon}{\sqrt {\pi \left(\sum_ {r = k + 1} ^ {k + j} Z _ {r} ^ {2}\right)}}, \tag {11} \\ \end{array}
$$
where the latter inequality comes from the fact that, conditional on  $Z_{1},\ldots ,Z_{n}$ , we have that  $A_{i}\sim \mathcal{N}(0,\sum_{r = 1}^{j}Z_{r}^{2}),B_{i}\sim \mathcal{N}(0,\sum_{r = j + 1}^{k}Z_{r}^{2})$ , and  $C_i\sim \mathcal{N}(0,\sum_{r = k + 1}^{k + j}Z_r^2)$  for each  $i\in [d]$ .
We now proceed similarly to the proof of Lemma 7. We denote the event that  $\left(1 - 2\sqrt{c_d}\right)j\leq \sum_{i = 1}^{j}Z_i^2,\sum_{i = k + 1}^{k + j}Z_i^2$  by  $\mathcal{E}^{(\downarrow)}$ . Then, by Eq. 10 and the law of total probability, we have that
$$
\begin{array}{l} \Pr \left(A + B \in I _ {\varepsilon} (\vec {z}), B + C \in I _ {\varepsilon} (\vec {z})\right) \\ = \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {i = 1} ^ {d} \Pr \left(A _ {i} + B _ {i} \in I _ {\varepsilon} \left(z _ {i}\right), B _ {i} + C _ {i} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}\right) \right] \\ \leq \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {i = 1} ^ {d} \Pr \left(A _ {i} + B _ {i}, B _ {i} + C _ {i} \in I _ {\varepsilon} (z _ {i}) \Big | Z _ {1}, \dots , Z _ {n}, \mathcal {E} ^ {(\downarrow)}\right) \right] + \Pr \left(\overline {{\mathcal {E} ^ {(\downarrow)}}}\right). \\ \end{array}
$$
Eq. 11 implies that
$$
\begin{array}{l} \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {i = 1} ^ {d} \Pr \left(A _ {i} + B _ {i}, B _ {i} + C _ {i} \in I _ {\varepsilon} \left(z _ {i}\right) \mid Z _ {1}, \dots , Z _ {n}, \mathcal {E} ^ {(\downarrow)}\right) \right] + \Pr \left(\overline {{\mathcal {E} ^ {(\downarrow)}}}\right) \\ \leq \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \prod_ {i = 1} ^ {d} \frac {2 \varepsilon}{\sqrt {\pi \left(\sum_ {r = 1} ^ {j} Z _ {r} ^ {2}\right)}} \cdot \frac {2 \varepsilon}{\sqrt {\pi \left(\sum_ {r = k + 1} ^ {k + j} Z _ {r} ^ {2}\right)}} \mid \mathcal {E} ^ {(\downarrow)} \right] + \Pr \left(\overline {{\mathcal {E} ^ {(\downarrow)}}}\right) \\ = \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \left. \left(\frac {4 \varepsilon^ {2}}{\pi \sqrt {\left(\sum_ {r = 1} ^ {j} Z _ {r} ^ {2}\right) \left(\sum_ {r = k + 1} ^ {k + j} Z _ {r} ^ {2}\right)}}\right) ^ {d} \right| \mathcal {E} ^ {(\downarrow)} \right] + \Pr \left(\overline {{\mathcal {E} ^ {(\downarrow)}}}\right). \\ \end{array}
$$
By independence of  $\sum_{r=1}^{j} Z_r^2$  and  $\sum_{r=k+1}^{k+j} Z_r^2$  and by Lemma 17, we obtain that
$$
\begin{array}{l} \mathbb {E} _ {Z _ {1}, \dots , Z _ {n}} \left[ \left. \left(\frac {4 \varepsilon^ {2}}{\pi \sqrt {\left(\sum_ {i = 1} ^ {j} Z _ {i} ^ {2}\right) \left(\sum_ {i = k + 1} ^ {k + j} Z _ {i} ^ {2}\right)}}\right) ^ {d} \right| \mathcal {E} ^ {(\downarrow)} \right] + \Pr \left(\overline {{\mathcal {E} ^ {(\downarrow)}}}\right) \\ \leq \left(\frac {4 \varepsilon^ {2}}{\pi j (1 - 2 \sqrt {c _ {d}})}\right) ^ {d} + \Pr \left(\overline {{\mathcal {E} ^ {(\downarrow)}}}\right) \\ \leq \left(\frac {4 \varepsilon^ {2}}{\pi j (1 - 2 \sqrt {c _ {d}})}\right) ^ {d} + 2 \exp (- c _ {d} j) \\ \end{array}
$$
$$
= \exp \left(- d \log \frac {\pi j (1 - 2 \sqrt {c _ {d}})}{4 \varepsilon^ {2}}\right) + 2 \exp (- c _ {d} k).
$$
Finally, for a large enough constant  $C$ , the hypothesis on  $k$  implies that  $k \geq 2\frac{d}{c_d}\log \frac{\pi k\left(1 - 2\sqrt{c_d}\right)}{4\varepsilon^2}$ . Hence,
$$
\exp \left(- d \log \frac {\pi j (1 - 2 \sqrt {c _ {d}})}{4 \varepsilon^ {2}}\right) + 2 \exp (- c _ {d} k) \leq 3 \left(\frac {4 \varepsilon^ {2}}{\pi (1 - 2 \sqrt {c _ {d}}) k}\right) ^ {d}.
$$
Proof of Theorem 5. We use the second moment method (Lemma 15) on the  $\varepsilon$ -subset-sum number  $T_{n,k}$  of  $X_1, \ldots, X_n$ . Thus, we aim to provide a lower bound on the right-hand side of
$$
\Pr (T > 0) \geq \frac {\mathbb {E} \left[ T _ {n , k} \right] ^ {2}}{\mathbb {E} \left[ T _ {n , k} ^ {2} \right]}. \tag {12}
$$
Equivalently, we can provide an upper bound on the inverse  $\frac{\mathbb{E}[T_{n,k}^2]}{\mathbb{E}[T_{n,k}]^2}$ . By Lemma 9
$$
\mathbb {E} \left[ T _ {n, k} ^ {2} \right] = \binom {n} {k} ^ {2} \sum_ {j = 0} ^ {k} \Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | = k - j\right) \Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right) \tag {13}
$$
where  $S, S', S_i$  and  $\mathcal{E}_S^{(\vec{z})}$  are defined as in the statement of the lemma. Observe also that
$$
\mathbb {E} \left[ T _ {n, k} \right] = \sum_ {S \in \binom {\lfloor n \rfloor} {k}} \mathbb {E} \left[ \mathbf {1} _ {\mathcal {E} _ {S} ^ {(z)}} \right] = \sum_ {S \in \binom {\lfloor n \rfloor} {k}} \Pr \left(\mathcal {E} _ {S} ^ {(z)}\right) = \binom {n} {k} \Pr \left(\mathcal {E} _ {S _ {0}} ^ {(z)}\right). \tag {14}
$$
By Eq.s 13 and 14, we have
$$
\begin{array}{l} \frac {\mathbb {E} \left[ T _ {n , k} ^ {2} \right]}{\mathbb {E} \left[ T _ {n , k} \right] ^ {2}} = \frac {\binom {n} {k} ^ {2}}{\mathbb {E} \left[ T _ {n , k} \right] ^ {2}} \sum_ {j = 0} ^ {k} \operatorname * {P r} \left(\left| \mathrm {S} \cap \mathrm {S} ^ {\prime} \right| = k - j\right) \operatorname * {P r} \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right) \\ = \sum_ {j = 0} ^ {k} \Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | = k - j\right) \frac {\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right) ^ {2}}. \tag {15} \\ \end{array}
$$
As for the denominator of the second term in Eq. 15, by Lemma 7 we have
$$
\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right) ^ {2} \geq \frac {1}{2 5 6} \left(\frac {4 \varepsilon^ {2}}{\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}\right) ^ {d}. \tag {16}
$$
As for the numerator of the second term in Eq. 15, Lemma 8 implies that we have
$$
\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right) \leq 3 \left(\frac {4 \varepsilon^ {2}}{\pi (1 - 2 \sqrt {c _ {d}}) j}\right) ^ {d}. \tag {17}
$$
By plugging Eq. 17 and Eq. 16 in Eq. 15, for  $j \geq k\left(1 - \frac{1}{d}\right)$  and  $d > 1$  we can upper bound the factor  $\frac{\operatorname*{Pr}\left(\mathcal{E}_{S_0}^{(\bar{z})} \cap \mathcal{E}_{S_j}^{(\bar{z})}\right)}{\operatorname*{Pr}\left(\mathcal{E}_{S_0}^{(\bar{z})}\right)^2}$  of the summation as follows:
$$
\frac {\operatorname * {P r} \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right)}{\operatorname * {P r} \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right) ^ {2}} \leq \frac {3 \left(\frac {4 \varepsilon^ {2}}{\pi (1 - 2 \sqrt {c _ {d}}) j}\right) ^ {d}}{\frac {1}{2 5 6} \left(\frac {4 \varepsilon^ {2}}{\pi (1 + 2 \sqrt {c _ {d}} + 2 c _ {d}) k}\right) ^ {d}}
$$
$$
= 7 6 8 \left(\frac {\left(1 + 2 \sqrt {c _ {d}} + 2 c _ {d}\right) k}{\left(1 - 2 \sqrt {c _ {d}}\right) j}\right) ^ {d}.
$$
As  $j \geq k\left(1 - \frac{1}{d}\right)$  with  $d > 1$ , then
$$
\begin{array}{l} 7 6 8 \left(\frac {\left(1 + 2 \sqrt {c _ {d}} + 2 c _ {d}\right) k}{\left(1 - 2 \sqrt {c _ {d}}\right) j}\right) ^ {d} \leq 7 6 8 \left(\frac {\left(1 + 2 \sqrt {c _ {d}} + 2 c _ {d}\right)}{\left(1 - 2 \sqrt {c _ {d}}\right) \left(1 - \frac {1}{d}\right)}\right) ^ {d} \\ \leq 7 6 8 \left(\frac {2 + 7 \sqrt {c _ {d}}}{2 - 7 \sqrt {c _ {d}}}\right) ^ {d} \\ \leq 2 7 0 8 0 1, \tag {18} \\ \end{array}
$$
because  $\left(\frac{(1 + 2\sqrt{c_d} + 2c_d)}{(1 - 2\sqrt{c_d})(1 - \frac{1}{d})}\right)^d$  is maximised for  $d = 4$ . Let  $C' = 270801$ . If  $d > 1$ , by plugging Eq. 18 in Eq. 15, we have that
$$
\begin{array}{l} \frac {\mathbb {E} \left[ T _ {n , k} ^ {2} \right]}{\mathbb {E} \left[ T _ {n , k} \right] ^ {2}} = \sum_ {j = 0} ^ {\left\lceil k - \frac {k}{d} \right\rceil - 1} \Pr \left(\left| S \cap S ^ {\prime} \right| = k - j\right) \frac {\Pr \left(\mathcal {E} _ {S _ {0}} ^ {\left(\vec {z}\right)} \cap \mathcal {E} _ {S _ {j}} ^ {\left(\vec {z}\right)}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {\left(\vec {z}\right)}\right) ^ {2}} \\ + \sum_ {j = \left\lceil k - \frac {k}{d} \right\rceil} ^ {k} \Pr \left(| S \cap S ^ {\prime} | = k - j\right) \frac {\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right) ^ {2}} \\ \leq \sum_ {j = 0} ^ {\left\lceil k - \frac {k}{d} \right\rceil - 1} \Pr \left(| S \cap S ^ {\prime} | = k - j\right) \frac {\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right) ^ {2}} + C ^ {\prime}. \\ \end{array}
$$
As  $\operatorname*{Pr}\left(\mathcal{E}_{S_0}^{(\vec{z})}\cap \mathcal{E}_{S_j}^{(\vec{z})}\right)\leq \operatorname*{Pr}\left(\mathcal{E}_{S_0}^{(\vec{z})}\right)$  , then
$$
\begin{array}{l} \sum_ {j = 0} ^ {\left[ k - \frac {k}{d} \right] - 1} \Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | = k - j\right) \frac {\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})} \cap \mathcal {E} _ {S _ {j}} ^ {(\vec {z})}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right) ^ {2}} + C ^ {\prime} \\ \leq \frac {1}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {\left(\vec {z}\right)}\right)} \sum_ {j = 0} ^ {\left[ k - \frac {k}{d} \right] - 1} \Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | = k - j\right) + C ^ {\prime} \\ \leq \frac {\Pr \left(\left| S \cap S ^ {\prime} \right| > \frac {k}{d}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\bar {z})}\right)} + C ^ {\prime}. \tag {19} \\ \end{array}
$$
Notice that, if  $d = 1$ , the same bound holds as  $\operatorname*{Pr}\left(|\mathrm{S} \cap \mathrm{S}'| > \frac{k}{d}\right) = 0$ . We now observe that, by the law of total probability
$$
\Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | \geq \frac {k}{d}\right) = \sum_ {\tilde {S} \in \binom {[ n ]} {k}} \Pr \left(\mathrm {S} = \tilde {S}\right) \Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | \geq \frac {k}{d} \mid \mathrm {S} = \tilde {S}\right). \tag {20}
$$
Conditional on  $\mathrm{S} = \tilde{S}$ ,  $|\mathrm{S} \cap \mathrm{S}'|$  is a hypergeometric random variable with
$$
\mathbb {E} \left[ | \mathrm {S} \cap \mathrm {S} ^ {\prime} | \Big | \mathrm {S} = \tilde {S} \right] = \sum_ {i \in \tilde {S}} \operatorname * {P r} \left(i \in \mathrm {S} ^ {\prime}\right) = k \operatorname * {P r} \left(1 \in \mathrm {S}\right) = \frac {k ^ {2}}{n}.
$$
Since  $n \geq k^2$ , then  $\frac{k^2}{n} \leq 1$ . Hence, since Chernoff bounds hold for the hypergeometric distribution [Doerr, 2020, Theorem 1.10.25]
$$
\begin{array}{l} \Pr \left(| S \cap S ^ {\prime} | \geq \frac {k}{d} \mid S = \tilde {S}\right) \geq \Pr \left(| S \cap S ^ {\prime} | \geq \frac {k ^ {2}}{n} + \left(\frac {k}{d} - 1\right) \mid S = \tilde {S}\right) \\ \leq \exp \left(- 2 \frac {\left(\frac {k}{d} - 1\right) ^ {2}}{k}\right) \\ \leq \exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2}\right). \tag {21} \\ \end{array}
$$
Substituting Eq. 21 in Eq. 20 we get
$$
\Pr \left(| S \cap S ^ {\prime} | \geq \frac {k}{d}\right) \leq \exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2}\right). \tag {22}
$$
We can now keep bounding from above  $\frac{\mathbb{E}\left[T_{n,k}^2\right]}{\mathbb{E}\left[T_{n,k}\right]^2}$  by plugging Eq. 22 in Eq. 19:
$$
\frac {\Pr \left(| \mathrm {S} \cap \mathrm {S} ^ {\prime} | \geq \frac {k}{d}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right)} + C ^ {\prime} \leq \frac {\exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\vec {z})}\right)} + C ^ {\prime}. \tag {23}
$$
By Lemma 7, and since  $1 + 2\sqrt{c_d} + 2c_d \leq 2$ , we have
$$
\begin{array}{l} \frac {\exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2}\right)}{\Pr \left(\mathcal {E} _ {S _ {0}} ^ {(\bar {z})}\right)} + C ^ {\prime} \\ \leq \frac {1 6 \exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2}\right)}{\left(\frac {2 \varepsilon}{\sqrt {\pi 2 k}}\right) ^ {d}} + C ^ {\prime} \\ = 1 6 \exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2} + d \log \left(\frac {\sqrt {\pi 2 k}}{2 \varepsilon}\right)\right) + C ^ {\prime}. \tag {24} \\ \end{array}
$$
By the hypothesis, since  $k \geq C d^3 \log \frac{d}{\varepsilon}$  for a large enough  $C$ , it holds that
$$
\begin{array}{l} 1 6 \exp \left(- 2 \frac {k}{d ^ {2}} \left(1 - \frac {d}{k}\right) ^ {2} + d \log \left(\frac {\sqrt {\pi 2 k}}{2 \varepsilon}\right)\right) + C ^ {\prime} \\ \leq C ^ {\prime} + 1 6 \exp \left(- d \log \frac {d}{\varepsilon}\right) \\ <   C ^ {\prime} + \frac {1 6}{e}, \tag {25} \\ \end{array}
$$
where the latter inequality holds as  $\varepsilon \leq 1 / 4$
Plugging the inverse of the expression in Eq. 25 in Eq. 12 we obtain the thesis.
B.2 Kernel Pruning
Proof of Lemma 11.  $S_{2} \in \{0,1\}^{\mathrm{size}(V)}$  is such that  $\tilde{V} = V \odot \tilde{S} = (V \odot S_{1}) \odot S_{2}$  contains only non-negative edges going from each input channel  $t$  to the output channels  $2(t - 1)n + 1, \ldots, (2t - 1)n$ , and only non-positive<sup>6</sup> edges going from each input channel  $t$  to the output channels  $(2t - 1)n + 1, \ldots, 2tn$ , while all remaining edges are set to zero. Let us define some convenient notations before proceeding
<sup>6</sup>We consider 0 to be both non-negative and non-positive.