SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We note here that it is common also to talk about the notion of local weak convergence of a sequence of (finite) random graphs  $G_{n}$ . In this case  $G_{n}\to G$  locally weakly if  $[(G_n,u_n)]\to [(G,u)]$  locally weakly where  $u_{n}\sim \mathrm{Unif}(V(G_{n}))$ .
A.2 Preliminary Results
In this section, we state two important preliminary results and a fact that are used to establish our results in the paper.
Lemma A.1. [Dreier et al., 2018, Lemma 3]. Let  $G \sim \mathrm{CSBM}(n,d,\{\mathbb{P}_{-},\mathbb{P}_{+}\},\frac{a}{n},\frac{b}{n})$  and  $r$  be a fixed constant. Then the probability that there exists an  $r$ -neighbourhood in  $G$  with  $m$  more edges than the number of vertices is bounded as follows:
$$
\mathbf {P r} (\exists G _ {r} \subset G s. t. | E (G _ {r}) | \geq | V (G _ {r}) | + m) \leq \frac {(2 (r (m + 1)) ^ {2} (a + b)) ^ {2 r (m + 1) + m}}{n ^ {m}}.
$$
Lemma A.2. [Massoulie, 2014, Lemma 4.2]. Assume  $\ell = c\log (n)$  with  $c\log \Delta < 1 / 4$ , where  $\Delta = (a + b) / 2$ . Then with high probability, no node  $i$  has more than one cycle in its  $\ell$ -neighbourhood. Moreover, for any  $m > 0$ , with probability at least  $1 - O(1 / m^2)$  the number of nodes  $i$  whose  $\ell$ -neighbourhood contains at least one cycle is bounded by  $O(m\log^3 (n)\Delta^{2\ell})$ .
Fact A.3. For any non-negative  $u, v, x, y$  such that  $x, y \leq 1$ ,
$$
x u + y v - \frac {1}{2} (x u + y v) ^ {2} \leq 1 - (1 - x) ^ {u} (1 - y) ^ {v} \leq x u + y v.
$$
A.3 Bayes Optimal Classifier
In this section, we compute the asymptotically  $\ell$ -locally Bayes optimal classifier for the general CSBM described in Section 3.2 and establish Theorem 1, followed by a proof of Corollary 1.1. Next, we compute the generalization error for the two-class case with arbitrary node features.
A.3.1 Computing the Classifier
For the proofs, we introduce the notation  $N_{k}(u)$  for a given graph to mean the set of vertices that are at a distance of exactly  $k$  from node  $u$  in the graph. Thus,  $\eta_{k}(u) = \{u\} \cup_{j=1}^{k} N_{k}(u)$ .
Theorem (Restatement of Theorem 1). For any  $\ell \geq 1$ , the asymptotically  $\ell$ -locally Bayes optimal classifier of the root for the sequence  $(G_{n},u_{n})\sim \mathrm{CSBM}(n,d,\mathbb{P},\mathbf{Q})$  is
Furthermore, there exists a choice of parameters for Architecture 1 such that it realizes  $h_{\ell}^{*}$ .
Proof. We begin by writing the MAP estimation for this problem. Note that the features  $\mathbf{X}_v\in \mathbb{R}^d$  for every node  $v\in [n]$  follow the law  $\mathbb{P}_i$  if  $y_{v} = i\in [C]$ . In addition, recall that we have the edge-probability matrix  $\mathbf{Q} = \{q_{ij}\}_{i,j\in [C]}$  with  $\mathbf{Pr}((u,v)$  is an edge  $|y_u = i,y_v = j) = q_{ij}$ , where  $q_{ij} = b_{ij} / n$  for absolute constants  $b_{ij}$ . Then we can write the likelihood of the  $\ell$ -neighbourhood of node  $u$  as the joint function:
$$
f (u, \left\{\left(\mathbf {X} _ {v}, y _ {v}\right) \right\} _ {v \in \eta_ {\ell} (u)}, \eta_ {\ell} (u)) = \mathbf {P} _ {\left\{y _ {v} \right\} _ {v \in \eta_ {\ell} (u)}} \rho_ {y _ {u}} (\mathbf {X} _ {u}) \prod_ {k \in [ \ell ]} \prod_ {v \in N _ {k} (u)} \rho_ {y _ {v}} (\mathbf {X} _ {v}) \mathbf {Q} _ {y _ {u} y _ {v}} ^ {k}. \tag {4}
$$
In the above,  $\mathbf{Q}_{ij}^{k}$  denotes the  $i,j$ -th entry of the matrix  $\mathbf{Q}^k$  and quantifies the probability that a node in class  $j$  is at a distance  $k$  from a node in class  $i$ ; while  $\mathbf{P}_{\{y_v\}_{v\in \eta_\ell (u)}}$  denotes the prior distribution of the node labels, which by our assumption is uniform. Let us now compute the MAP estimator.
$$
\begin{array}{l} h_{\ell}^{*}(u,\{\mathbf{X}_{v}\}_{v\in \eta_{\ell}(u)}) = \operatorname *{argmax}_{i\in [C]}\max_{\substack{\{y_{v}\in [C]\} \\ v\in \eta_{\ell}(u)}}f(u,\{( \mathbf{X}_{v},y_{v})\}_{v\in \eta_{\ell}(u)},\eta_{\ell}(u)) \\ = \operatorname * {a r g m a x} _ {i \in [ C ]} \rho_ {i} (\mathbf {X} _ {u}) \prod_ {k \in [ \ell ]} \prod_ {v \in N _ {k} (u)} \max  _ {j \in [ C ]} \rho_ {j} (\mathbf {X} _ {v}) \mathbf {Q} _ {i j} ^ {k} \\ = \operatorname * {a r g m a x} _ {i \in [ C ]} \log \rho_ {i} (\mathbf {X} _ {u}) + \sum_ {k \in [ \ell ]} \sum_ {v \in N _ {k} (u)} \max  _ {j \in [ C ]} \left\{\log \left(\rho_ {j} (\mathbf {X} _ {v})\right) + \log \left(\mathbf {Q} _ {i j} ^ {k}\right) \right\} \\ = \operatorname * {a r g m a x} _ {i \in [ C ]} \log \rho_ {i} (\mathbf {X} _ {u}) + \sum_ {k \in [ \ell ]} \sum_ {v \in N _ {k} (u)} \max  _ {j \in [ C ]} \mathcal {M} _ {i k} (\mathbf {X} _ {v}), \\ \end{array}
$$
where  $\mathcal{M}_{ik}(\mathbf{X}_v) = \log (\rho_j(\mathbf{X}_v)) + \log (\mathbf{Q}_{ij}^k)$ . Furthermore, note that an instance of Architecture 1 with  $L = 1$ ,  $\sigma_{1} = \{\rho_{i}\}_{i\in [C]}$ , and  $\mathbf{Q}$  for the edge-probabilities realizes the function  $h_\ell^*$  for a given root node  $u$  and its  $\ell$ -neighbourhood  $\eta_{\ell}(u)$ , in the sense that  $\hat{y}_u = h_\ell^* (u,\{\mathbf{X}_v\}_{v\in \eta_\ell (u)})$ .
Next, we obtain a simpler version of the classifier for the two-class symmetric CSBM with an arbitrary distribution of node features. Recall that  $\psi$  is the log of the likelihood ratio.
Corollary (Restatement of Corollary 1.1). For any  $\ell \geq 1$ , the asymptotically  $\ell$ -locally Bayes optimal classifier of the root for the sequence  $(G_{n},u_{n}) \sim \mathrm{CSBM}(n,d,\mathbb{P},\frac{a}{n},\frac{b}{n})$  is
where  $\mathcal{M}_k(\mathbf{x}) = \mathrm{sgn}(a - b)\cdot \varphi (\psi (\mathbf{x}),c_k)$  with  $c_{k} = \log \left(\frac{1 + \Gamma^{k}}{1 - \Gamma^{k}}\right)$ , and  $\psi (\mathbf{x}) = \log \frac{\rho_{+}(\mathbf{x})}{\rho_{-}(\mathbf{x})}$ .
Proof. The proof follows directly from Theorem 1, by taking  $C = 2$ . In this two-class case, the features  $\mathbf{X}_v \in \mathbb{R}^d$  for every node follow the law  $\mathbb{P}_{y_v}$  where the class labels  $y_v \in \{\mp 1\}$  are Unif  $\{-1, +1\}$ . In addition, we have
$$
\mathbf {P r} ((u, v) \text {i s a n e d g e}) = \left\{ \begin{array}{l l} p & y _ {u} = y _ {v} \\ q & y _ {u} \neq y _ {v} \end{array} \right.,
$$
where  $p = a / n$  and  $q = b / n$  for absolute constants  $a > 1, b \geq 0$ . Define the quantities  $p_k, q_k$  as follows for  $k \in [\ell]$ :
$$
p _ {k} = \sum_ {j = 0} ^ {\lfloor k / 2 \rfloor} \binom {k} {2 j} p ^ {k - 2 j} q ^ {2 j}, \quad q _ {k} = \sum_ {j = 1} ^ {\lceil k / 2 \rceil} \binom {k} {2 j - 1} p ^ {k - 2 j + 1} q ^ {2 j - 1}. \tag {5}
$$
Then we can simplify the likelihood of the  $\ell$ -neighbourhood of node  $u$  from (4) as follows:
$$
f(\{(\mathbf{X}_{v},y_{v})\}_{v\in \eta_{\ell}(u)}) = \rho_{y_{u}}(\mathbf{X}_{u})\prod_{k\in [\ell ]}\prod_{v\in N_{k}(u)}\left(\rho_{y_{v}}(\mathbf{X}_{v})p_{k}^{\frac{1 + y_{u}y_{v}}{2}}q_{k}^{\frac{1 - y_{u}y_{v}}{2}}\right).
$$
Then maximizing the likelihood over possible class labels, we have
$$
\begin{array}{l} h_{\ell}^{*}(u,\{\mathbf{X}_{v}\}_{v\in \eta_{\ell}(u)}) = \operatorname *{argmax}_{y_{u}\in \{\pm 1\}}\max_{\substack{\{y_{v}\in \{\pm 1\} \} \\ v\in \eta_{\ell}(u)}}f(\{( \mathbf{X}_{v},y_{v})\}_{v\in \eta_{\ell}(u)}) \\ = \operatorname *{argmax}_{y_{u}\in \{\pm 1\}}\max_{\substack{\{y_{v}\in \{\pm 1\} \} \\ v\in \eta_{\ell}(u)}}\log f(\{(X_{v},y_{v})\}_{v\in \eta_{\ell}(u)}) \\ = \operatorname *{argmax}_{y_{u}\in \{\pm 1\}}\left\{\log \rho_{y_{u}}(\mathbf{X}_{u}) + \max_{\substack{\{y_{v}\in \{\pm 1\} \} \\ v\in \eta_{\ell}(u)}}\sum_{k\in [\ell ]}\sum_{v\in N_{k}(u)}\log \left(\rho_{y_{v}}(\mathbf{X}_{v})p_{k}^{\frac{1 + y_{u}y_{v}}{2}}q_{k}^{\frac{1 - y_{u}y_{v}}{2}}\right)\right\} \\ = \operatorname {s g n} \left(\log \frac {\rho_ {+} (\mathbf {X} _ {u})}{\rho_ {-} (\mathbf {X} _ {u})} + \sum_ {k \in [ \ell ]} \sum_ {v \in N _ {k} (u)} \mathcal {M} _ {k} (\mathbf {X} _ {v})\right), \\ \end{array}
$$
where  $\mathcal{M}_k(\mathbf{x}) = \log (\max (p_k\rho_+(\mathbf{x}),q_k\rho_-(\mathbf{x}))) - \log (\max (p_k\rho_-(\mathbf{x}),q_k\rho_+(\mathbf{x})))$ . Next, we observe that for any  $w,x,y,z\in \mathbb{R}$
$$
\log (\max (w y, x z)) - \log (\max (w z, x y)) = \operatorname {s g n} (w - x) \cdot \min \left(\max \left(\log \frac {y}{z}, - \left| \log \frac {w}{x} \right|\right), \left| \log \frac {w}{x} \right|\right).
$$
Hence,  $\mathcal{M}_k(\mathbf{X}_v)$  is simply  $\mathrm{sgn}(a - b)\cdot \varphi (\log \frac{\rho_+(\mathbf{X}_v)}{\rho_-(\mathbf{X}_v)},c_k)$ , i.e., the signed likelihood ratio clipped between  $-c_{k}$  and  $c_{k}$ .
A.3.2 Generalization Error
Let us now compute the generalization error of  $h_\ell^*$ . Formally, given a data instance  $(u, \{\mathbf{X}_v\}_{v \in \eta_\ell(u)})$  along with the neighbourhood  $\eta_\ell(u)$ ,  $h_\ell^*$  outputs a label  $\hat{y}_u \in \{\pm 1\}$ , and the generalization error is defined as the probability  $\mathbf{Pr}(y_u \hat{y}_u < 1)$ . For a simple calculation, let us assume that the latent labels  $y_i$  are uniformly distributed, i.e.,  $\mathbf{Pr}(y_i = -1) = \mathbf{Pr}(y_i = 1) = \frac{1}{2}$ . It is straightforward to generalize to unbalanced settings. Recall that the features in classes  $\pm 1$  follow the law  $\mathbb{P}_{\pm}$ , and denote by the log likelihood ratio by  $\psi(\mathbf{x}) = \log (\rho_+(\mathbf{x}) / \rho_-(\mathbf{x}))$ . Then we have that for a fixed  $u$ ,
$$
\begin{array}{l} \mathcal {E} \left(h _ {\ell} ^ {*}\right) = \Pr \left(y _ {u} \left(\log \frac {\rho_ {+} \left(\mathbf {X} _ {u}\right)}{\rho_ {-} \left(\mathbf {X} _ {u}\right)} + \sum_ {k \in [ \ell ]} \sum_ {v \in N _ {k} (u)} \mathcal {M} _ {k} \left(\mathbf {X} _ {v}\right)\right) <   0\right) \\ = \frac {1}{2} \left[ \mathbf {P r} \left(\psi \left(\mathbf {Y} ^ {(1)}\right) + \sum_ {k \in [ \ell ]} \mathbf {Z} _ {k} ^ {(1)} > 0\right) + \mathbf {P r} \left(\psi \left(\mathbf {Y} ^ {(2)}\right) + \sum_ {k \in [ \ell ]} \mathbf {Z} _ {k} ^ {(2)} <   0\right) \right], \\ \end{array}
$$
where  $\mathbf{Z}_k^{(1)} = \sum_{j\in [\alpha_k]}\mathcal{M}_k(\mathbf{Y}_{k,j}^{(1)}) + \sum_{j\in [\beta_k]}\mathcal{M}_k(\mathbf{Y}_{k,j}^{(2)})$  and  $\mathbf{Z}_k^{(2)} = \sum_{j\in [\alpha_k]}\mathcal{M}_k(\mathbf{Y}_{k,j}^{(2)}) + \sum_{j\in [\beta_k]}\mathcal{M}_k(\mathbf{Y}_{k,j}^{(1)})$  are independent random variables with  $\mathbf{Y}_{k,j}^{(1)}\sim \mathbb{P}_{-}$  and  $\mathbf{Y}_{k,j}^{(2)}\sim \mathbb{P}_{+}$ .
The above expression is not particularly insightful. For this reason, we now specialize to the case of Gaussian features and interpret the error in terms of the natural SNRs associated with the Gaussian mixture and the graph, i.e., the quantities  $\gamma$  and  $\Gamma$ .
A.4 Specialization to Gaussian Features
In this section, we look at the specialized setting where the node features are sampled from a symmetric binary Gaussian mixture model. Let us begin with the generalization error in this case.
A.4.1 Generalization Error
Theorem (Restatement of Theorem 2). For any  $\ell \geq 1$ , the generalization error of the asymptotically  $\ell$ -locally Bayes optimal classifier of the root for the sequence  $(G_{n},u_{n})\sim \mathrm{CSBM}(n,d,\mathbb{P},\mathbf{Q})$  with Gaussian features is given by
Proof. For the Gaussian mixture, the log of the likelihood ratio for a node  $u$  is given by  $\frac{2}{\sigma^2}\langle \mathbf{X}_u,\pmb {\mu}\rangle \stackrel {\mathcal{D}}{=} 2y_u\gamma^2 +2\gamma g$ , where  $g\sim \mathcal{N}(0,1)$ . Replacing every  $\{\mathbf{X}_i\}_{i\in [n]}$  as  $\mathbf{X}_i = \mathbb{E}\mathbf{X}_i + \sigma \mathbf{g}_i = y_i\pmb {\mu} + \sigma \mathbf{g}_i$ , we obtain the expression in Theorem 2.
A.4.2 Extreme Graph SNRs
Let us now turn to the next result, where we analyze the generalization error in the cases where the graph SNR  $\Gamma$  takes extreme values.
Theorem (Restatement of Theorem 3). Let  $h_{\ell}^{*}$  be the classifier from Corollary 1.1,  $h_0^* (u) = \mathrm{sgn}(\langle \mathbf{X}_u,\pmb {\mu}\rangle)$  be the Bayes optimal classifier given only the feature information of the root node  $u$ , and  $h_\mathrm{gcn}$  be the one-layer vanilla GCN classifier. Then we have that for any fixed  $\ell$ :
Proof. Note that when  $\Gamma = 0$ , i.e., when  $a = b$ , then  $a_{k} = b_{k}$  for all  $k \in [\ell]$ , hence,  $c_{k} = \log (a_{k} / b_{k}) = 0$ . This implies that all information from the  $k$ -hop neighbours is truncated to 0 for all  $k \in [\ell]$ . Thus, the classifier reduces to  $h_{u}^{*} = h_{\ell}^{*}(u, \{\mathbf{X}_{v}\}_{v \in \eta_{\ell}(u)}) = \operatorname{sgn}(\psi(\mathbf{X}_{u})) = \operatorname{sgn}(g + y_{u}\gamma) = h_{0}^{*}(u, \{\mathbf{X}_{u}\})$ . Thus, the probability that  $h_{u}^{*}y_{u} < 0$  is
$$
\Pr \left(h _ {u} ^ {*} y _ {u} <   0\right) = \Pr \left(y _ {u} g + \gamma <   0\right) = \Phi (- \gamma),
$$
where  $\Phi (\cdot)$  is the standard Gaussian CDF.
For the other case where  $\Gamma \to 1$ , we have two sub-cases: Either  $a \to 0$  with  $b \neq 0$ , or  $a \neq 0$  with  $b \to 0$ . In this case,  $c_k \to \infty$  for all  $k$ , so the classifier takes the form
$$
h _ {\ell} ^ {*} (u, \left\{\mathbf {X} _ {v} \right\} _ {v \in \eta_ {\ell} (u)}) = \operatorname {s g n} \left(g + y _ {u} \gamma + \sum_ {k \in [ \ell ]} \sum_ {v \in N _ {k} (u)} \left(g _ {k, v} + y _ {v} \gamma\right)\right).
$$
Hence, the probability of making a mistake is
$$
\begin{array}{l} \Pr \left(h _ {u} ^ {*} y _ {u} <   0\right) = \Pr \left(y _ {u} g + \gamma + \sum_ {k \in [ \ell ]} \sum_ {v \in N _ {k} (u)} \left(y _ {u} g _ {k, v} + y _ {u} y _ {v} \gamma\right) <   0\right) \\ = \mathbf {P r} \left(g > \gamma \frac {| \eta_ {\ell} ^ {(a)} - \eta_ {\ell} ^ {(b)} |}{\sqrt {\eta_ {\ell} ^ {(a)} + \eta_ {\ell} ^ {(b)}}}\right), \\ \end{array}
$$
where  $\eta_{\ell}^{(a)}, \eta_{\ell}^{(b)}$  denote the total number of nodes in the  $\ell$ -neighbourhood  $\eta_{\ell}(u)$  that are in the same class as  $u$  and different class as  $u$ , respectively. The last equation is obtained by using the fact that  $(g, \{g_{k,v}\})$  are i.i.d. standard Gaussians. Note that in this case since either  $b \to 0$  or  $a \to 0$  (but not both), we have  $\eta_{\ell}^{(b)} \to 0$  or  $\eta_{\ell}^{(a)} \to 0$  using (2) for any fixed  $\ell$ . Thus,  $\xi_{\ell}(a,b) > 1$  a.s. Following a similar analysis, one can find that  $\mathcal{E}(h_{\mathrm{gcn}}) = \mathbf{Pr}(g > \gamma \cdot \xi_1(a,b))$ .
It is interesting to note that we may not have  $\xi_1(a,b) > 1$  in general, meaning that a GCN is better than methods that do not use a graph only in the case where  $\xi_1(a,b) > 1$ .
A.5 Non-asymptotic Analysis
First, consider the case where  $\ell$ , the total depth of the neighbourhood is a constant independent of  $n$ , the number of nodes.
Putting  $m = 1$  in Lemma A.1, we see that the probability is bounded by  $(8\ell^2(a + b))^{4\ell + 1} / n = O(1 / n)$ . Hence, we conclude that in the limit  $n \to \infty$ , there are no cycles in any constant-depth neighbourhoods in the graph. In particular, we obtain that the local weak limit  $(G, u)$  is a tree.
We now turn to the case where the depth of the neighbourhood is logarithmic in  $n$ .
Proposition (Restatement of Proposition 3.1). Let  $G = (V, E) \sim \mathrm{CSBM}(n, d, \mathbb{P}, \frac{a}{n}, \frac{b}{n})$ . Then for any  $\ell = c\log n$  such that  $c\log \left(\frac{a + b}{2}\right) < 1/4$ , with probability  $1 - O(1/\log^2 n)$ , the number of nodes  $u \in V$  whose  $\ell$ -neighbourhood is cycle-free is  $n\left(1 - o\left(\frac{\log^4(n)}{\sqrt{n}}\right)\right)$ .
Proof. In Lemma A.2, observe that since  $c\log \Delta < 1 / 4$ , we have  $\ell < \frac{\log n}{4\log\Delta} = \frac{1}{4}\log_{\Delta}n$ . Thus, putting  $m = \log n$ , we find that with probability at least  $1 - O(1 / \log^2 n)$ , the number of nodes whose  $\ell$ -neighbourhood contains at least one cycle is bounded by  $O(\log^4 (n)\Delta^{2\ell}) = o(\log^4 (n)\sqrt{n})$ . Hence, the fraction of nodes whose  $\ell$ -neighbourhood is cycle-free is  $1 - o(\frac{\log^4 n}{\sqrt{n}})$ .
For a fixed node  $u \in [n]$ , let us denote the number of nodes at distance  $k$  (respectively  $\leq k$ ) from  $u$  with class label  $\pm y_u$  by  $U_k^{\pm}(u)$  (respectively,  $U_{\leq k}^{\pm}(u)$ ). Also let  $n_{\pm}$  denote the number of nodes with class label  $\pm y_u$ , so that  $n = n_+ + n_-$ . Note that  $U_0^+(u) = 1$ ,  $U_0^-(u) = 0$ , and conditionally on the sigma-field  $\mathcal{F}_{k-1} = \sigma(U_t^{\pm}(u), t \leq k-1)$ , we have
$$
U _ {k} ^ {+} (u) \sim \operatorname {B i n} \left(n _ {+} - U _ {\leq k - 1} ^ {+}, 1 - (1 - a / n) ^ {U _ {k - 1} ^ {+}} (1 - b / n) ^ {U _ {k - 1} ^ {-}}\right), \tag {6}
$$
$$
U _ {k} ^ {-} (u) \sim \operatorname {B i n} \left(n _ {-} - U _ {\leq k - 1} ^ {-}, 1 - (1 - a / n) ^ {U _ {k - 1} ^ {-}} (1 - b / n) ^ {U _ {k - 1} ^ {+}}\right). \tag {7}
$$
Define  $S_{k}(u) = U_{k}^{+}(u) + U_{k}^{-}(u)$  to be the number of nodes at distance exactly  $k$  from  $u$ , and denote  $\Delta = \frac{a + b}{2}$  to be the expected degree of a node. Correspondingly, recall from (2) that we have
$$
\alpha_ {k} \sim \operatorname {P o i} \left(\frac {a \alpha_ {k - 1} + b \beta_ {k - 1}}{2}\right), \beta_ {k} \sim \operatorname {P o i} \left(\frac {a \beta_ {k - 1} + b \alpha_ {k - 1}}{2}\right) \text {f o r} k \in [ \ell ]. \tag {8}
$$
Let us now state a useful high-probability bound on  $S_{k}(u) = U_{k}^{+}(u) + U_{k}^{-}(u)$ .
Lemma A.4. [Massoulie, 2014, Theorem 2.3]. For any  $\ell = c\log n$  with  $c\log \Delta < 1/4$ , there exist constants  $C, \epsilon > 0$  such that with probability at least  $1 - O(n^{-\epsilon})$ ,  $S_k(u) \leq C\Delta^k\log (n)$  for all  $u \in [n]$  and all  $k \in [\ell]$ .
We now obtain a total variation bound between the sequences  $\{U_k^{\pm}\}_{k\geq 0}$  and  $\{\alpha_{k},\beta_{k}\}_{k\geq 0}$ .
Lemma A.5. Let  $u \in [n]$  be fixed with label  $y_{u} \in \{\pm 1\}$ . Let  $\ell = c\log n$  with  $c\log \Delta < 1/4$ . Then the total variation distance between the collections of variables  $\{U_k^+(u), U_k^-(u)\}_{k \leq \ell}$  and  $\{\alpha_k(u), \beta_k(u)\}_{k \leq \ell}$  is bounded by  $O(\log^3 n / n^{1/4})$ .
Proof. Define the following events for  $C$  as in Lemma A.4:
$$
\Omega_ {k} = \left\{S _ {k} \leq C \Delta^ {k} \log n \right\}, 1 \leq k \leq \ell . \tag {9}
$$
Conditionally on the sigma-field  $\mathcal{F}_{k - 1} = \sigma (U_t^\pm (u),t\leq k - 1)$  and the event  $\Omega_{k - 1}$ , we compute the total variation distance between the variables  $(U_k^+ (u),U_k^- (u))$  and  $(\alpha_{k}(u),\beta_{k}(u))$ . Since  $u$  is fixed, we omit it from the notation for brevity. Define the following random variables:
$$
W _ {k} ^ {+} \sim \operatorname {P o i} \left(\frac {a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}}{2}\right), \qquad W _ {k} ^ {-} \sim \operatorname {P o i} \left(\frac {a U _ {k - 1} ^ {-} + b U _ {k - 1} ^ {+}}{2}\right).
$$
We now apply the Stein-Chen method to bound  $d_{\mathrm{TV}}(U_k^{\pm},W_k^{\pm})$ . For more details on this technique, we refer to Stein [1972], Chen [1975], Barbour and Chen [2005]. In particular, we use the fact that for  $X_{1}\sim \operatorname {Bin}(n,\lambda /n)$ ,  $X_{2}\sim \operatorname {Poi}(\lambda)$  and  $X_{3}\sim \operatorname {Poi}(\lambda^{\prime})$ ,  $d_{\mathrm{TV}}(X_1,X_2)\leq \lambda /n$  and  $d_{\mathrm{TV}}(X_2,X_3)\leq |\lambda -\lambda '|$ . Let us focus on  $d_{\mathrm{TV}}(U_k^+,W_k^+)$  as the other case for  $d_{\mathrm{TV}}(U_k^-,W_k^-)$  is similar. Construct an intermediate random variable based on the distributions of  $U_{k}^{\pm}$  as in Eqs. (6) and (7),
$$
V _ {k} \sim \mathrm {P o i} \left(\left(n _ {+} - U _ {\leq k - 1} ^ {+}\right) \left(1 - (1 - a / n) ^ {U _ {k - 1} ^ {+}} (1 - b / n) ^ {U _ {k - 1} ^ {-}}\right)\right).
$$
Denote  $T_{t} = 1 - \left(1 - \frac{a}{n}\right)^{U_{t}^{+}}\left(1 - \frac{b}{n}\right)^{U_{t}^{-}}$  for brevity. Note that using triangle inequality,
$$
\begin{array}{l} d _ {\mathrm {T V}} \left(V _ {k}, W _ {k} ^ {+}\right) \leq \left| \left(n _ {+} - U _ {\leq k - 1} ^ {+}\right) T _ {k - 1} - \frac {a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}}{2} \right| \\ \leq \left| \left(n _ {+} - U _ {\leq k - 1} ^ {+} - \frac {n}{2}\right) T _ {k - 1} \right| + \left| \frac {a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-} - n T _ {k - 1}}{2} \right| \\ = \left| \left(n _ {+} - U _ {\leq k - 1} ^ {+} - \frac {n}{2}\right) T _ {k - 1} \right| + \frac {n}{2} \left| \frac {a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}}{n} - T _ {k - 1} \right| \\ \leq \left| \left(n _ {+} - U _ {\leq k - 1} ^ {+} - \frac {n}{2}\right) T _ {k - 1} \right| + \frac {1}{4 n} \left(a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}\right) ^ {2}, \\ \end{array}
$$
where in the last inequality we used Fact A.3. Then we obtain the variation distance:
$$
\begin{array}{l} d _ {\mathrm {T V}} \left(U _ {k} ^ {+}, W _ {k} ^ {+}\right) \leq d _ {\mathrm {T V}} \left(U _ {k} ^ {+}, V _ {k}\right) + d _ {\mathrm {T V}} \left(V _ {k}, W _ {k} ^ {+}\right) \\ \leq T _ {k - 1} + \left| \left(n _ {+} - U _ {\leq k - 1} ^ {+} - \frac {n}{2}\right) T _ {k - 1} \right| + \frac {1}{4 n} \left(a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}\right) ^ {2} \\ = T _ {k - 1} \left(1 + \left| n _ {+} - U _ {\leq k - 1} ^ {+} - \frac {n}{2} \right|\right) + \frac {1}{4 n} \left(a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}\right) ^ {2}. \\ \end{array}
$$
Consider now a choice of  $c$  such that  $c\log \Delta < 1 / 4$ . We have  $\ell = c\log n < \frac{1}{4}\log_{\Delta}n$ , implying that  $\Delta^{\ell}\leq n^{1 / 4}$ . Recalling (9) corresponding to Lemma A.4, we have that under the event  $\Omega_{k - 1}$  for  $k\leq \ell$ , the number of nodes at distance  $k - 1$  is
$$
S _ {k - 1} = U _ {k - 1} ^ {+} + U _ {k - 1} ^ {-} \leq C \Delta^ {k - 1} \log n \leq C \Delta^ {\ell} \log n \leq C n ^ {1 / 4} \log n. \tag {10}
$$
Observe now that from Fact A.3,  $T_{k - 1} \leq \frac{aU_{k - 1}^{+} + bU_{k - 1}^{-}}{n}$ . Recalling that  $y_{u}$  have a uniform prior, by the Chernoff bound [Vershynin, 2018, Theorem 2.3.1] on  $n_+$ , we have  $|n_{+} - \frac{n}{2}| = O(\sqrt{n}\log n)$  with probability at least  $1 - 1 / \mathrm{poly}(n)$ . Thus, we obtain that under this event,
$$
\begin{array}{l} d _ {\mathrm {T V}} (U _ {k} ^ {+}, W _ {k} ^ {+}) \leq O \left(\frac {| U _ {\leq k - 1} ^ {+} |}{n} + \frac {\log n}{\sqrt {n}}\right) \cdot \left(a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}\right) + \frac {1}{4 n} \left(a U _ {k - 1} ^ {+} + b U _ {k - 1} ^ {-}\right) ^ {2} \\ \leq O \left(\frac {\log n}{\sqrt {n}}\right) \cdot \max (a, b) S _ {k - 1} + \frac {\max (a , b) ^ {2}}{4 n} S _ {k - 1} ^ {2} = O \left(\frac {\log^ {2} n}{n ^ {1 / 4}}\right), \\ \end{array}
$$
where in the last step we used the bound from (10). Now recall that the variables  $\{U_k^{\pm},\alpha_k,\beta_k\}_{k\in [\ell ]}$  are defined as in Eqs. (6) to (8) for all  $k\leq \ell$  . For a fixed  $u\in [n]$  , we have the base cases  $U_0^+ = \alpha_0 = 1$
and  $U_0^- = \beta_0 = 0$ . Then following an induction argument with a union bound over all  $k \in \{1, \dots, \ell\}$ , we have that the variation distance between the sequences  $\{U_k^+, U_k^-\}_{k \leq \ell}$  and  $\{\alpha_k, \beta_k\}_{k \leq \ell}$  is upper bounded by  $O\left(\frac{\log^3 n}{n^{1/4}}\right)$ .
We now obtain a relationship between the misclassification error on the data model with finite  $n$ , i.e.,  $\mathcal{E}_n$  and the error on the limit of the model with  $n \to \infty$ , i.e.,  $\mathcal{E}$ .
Theorem (Restatement of Theorem 4). For any  $1 \leq \ell \leq c \log n$  such that the positive constant  $c$  satisfies  $c \log \left(\frac{a + b}{2}\right) < 1/4$ , we have that