SlowGuess/NeurIPS_2023_OCR huggingface dataset:
- If  $y_{i} = 0$  then the oracle sets  $\hat{y}_i = \hat{b}$ . Since  $S$  is pseudorandom, we have  $P_{\mathbf{x}}(\mathbf{z}^S) = P_{\mathbf{x}}(\mathbf{z}^{S_i}) = y_i = 0$ . Hence, in the computation  $\hat{N} (\hat{\mathbf{z}}_i)$  the inputs to all neurons in  $\mathcal{E}_1,\mathcal{E}_2$  are at most  $-\frac{1}{2}$  (by Properties (Q1) and (Q2)), and thus their outputs are 0. Therefore,  $\hat{N} (\hat{\mathbf{z}}_i) = \hat{b}$ .
- If  $y_{i} = 1$  then the oracle sets  $\hat{y}_i = 0$ . Since  $S$  is pseudorandom, we have  $P_{\mathbf{x}}(\mathbf{z}^S) = P_{\mathbf{x}}(\mathbf{z}^{S_i}) = y_i = 1$ . Hence, in the computation  $\hat{N} (\hat{\mathbf{z}}_i)$  there exists a neuron in  $\mathcal{E}_1$  with output at least  $\frac{3}{2}$  (by Property (Q1)). Since all incoming weights to the output neuron in  $\hat{N}$  are in  $\left[-\frac{11}{10}, -\frac{9}{10}\right]$ , and  $\hat{b} \in \left[\frac{9}{10}, \frac{11}{10}\right]$ , then the input to output neuron (including the bias term) is at most  $\frac{11}{10} - \frac{3}{2} \cdot \frac{9}{10} < 0$ , and thus its output is 0.