SlowGuess/NeurIPS_2023_OCR huggingface dataset:
If the two walkers are independent, Eq. 32 evaluates to
$$
p (\text {b o t h} i) = p ^ {2} \left(\sum_ {\alpha = 1} ^ {N} \frac {1}{1 - (1 - p) \lambda_ {\alpha}} \langle j | k _ {\alpha} \rangle \langle k _ {\alpha} | i \rangle\right) ^ {2} \tag {33}
$$
where  $\lambda_{\alpha}$  and  $|k_{\alpha}\rangle$  denote the  $\alpha$ -th eigenvalue and eigenvector of the transition matrix  $\mathbf{T}$  respectively. Meanwhile, for walkers with antithetic termination, it is straightforward to derive that
$$
p \left(\operatorname {l e n} \left(\Omega_ {2}\right) = l _ {2} \mid \operatorname {l e n} \left(\Omega_ {1}\right) = l _ {1}\right) = \left\{ \begin{array}{l l} {\left(\frac {1 - 2 p}{1 - p}\right) ^ {l _ {2}} \frac {p}{1 - p}} & {\text {i f} l _ {2} <   l _ {1},} \\ {0} & {\text {i f} l _ {2} = l _ {1},} \\ {\left(\frac {1 - 2 p}{1 - p}\right) ^ {l _ {1}} (1 - p) ^ {l _ {2} - l _ {1} - 1} p} & {\text {i f} l _ {2} > l _ {1},} \end{array} \right. \tag {34}
$$
where the walk lengths are no longer independent. It follows that
$$
\begin{array}{l} p \left(\Omega_ {2} \text {t e r m i n a t e s a t} i \mid \operatorname {l e n} \left(\Omega_ {1}\right) = l _ {1}\right) = \sum_ {l _ {2} = 0} ^ {l _ {1} - 1} \left(\frac {1 - 2 p}{1 - p}\right) ^ {l _ {2}} \frac {p}{1 - p} \mathbf {T} _ {j i} ^ {l _ {2}} + \tag {35} \\ \sum_ {l _ {2} = l _ {1} + 1} ^ {\infty} \left(\frac {1 - 2 p}{1 - p}\right) ^ {l _ {1}} (1 - p) ^ {l _ {2} - l _ {1} - 1} p \mathbf {T} _ {j i} ^ {l _ {2}}. \\ \end{array}
$$
We then sum over all the possible walks  $\Omega_1$  at each length  $l_{1}$ . After tedious but straightforward algebra,
$$
\begin{array}{l} p (\text {b o t h t e r m i n a t} i) = \frac {p ^ {2}}{1 - p} \sum_ {\alpha , \beta = 1} ^ {N} \langle j | k _ {\alpha} \rangle \langle k _ {\alpha} | i \rangle \langle j | k _ {\beta} \rangle \langle k _ {\beta} | i \rangle \left(\frac {(1 - p) \lambda_ {\alpha}}{1 - (1 - p) \lambda_ {\alpha}} \right. \tag {36} \\ \left. + \frac {(1 - p) \lambda_ {\beta}}{1 - (1 - p) \lambda_ {\beta}}\right) \frac {1}{1 - (1 - 2 p) \lambda_ {\alpha} \lambda_ {\beta}}. \\ \end{array}
$$
To assess whether antithetic termination suppresses the variance, we need to consider the difference between Eqs 33 and 36. This evaluates to
$$
\begin{array}{l} \Delta p (\text {b o t h t e r m i n a t} i) = \sum_ {\alpha , \beta = 1} ^ {N} - p ^ {2} \frac {1}{1 - (1 - p) \lambda_ {\alpha}} \frac {1}{1 - (1 - p) \lambda_ {\beta}} \frac {(\lambda_ {\alpha} - 1) (\lambda_ {\beta} - 1)}{1 - (1 - 2 p) \lambda_ {\alpha} \lambda_ {\beta}} \tag {37} \\ \cdot \langle j | k _ {\alpha} \rangle \langle k _ {\alpha} | i \rangle \langle j | k _ {\beta} \rangle \langle k _ {\beta} | i \rangle . \\ \end{array}
$$
For any pair of nodes  $i,j$ , the products  $\langle j|k_{\alpha}\rangle \langle k_{\alpha}|i\rangle$  and  $\langle j|k_{\beta}\rangle \langle k_{\beta}|i\rangle$  can be treated as the  $\alpha$ - and  $\beta$ -coordinates of an  $N$ -dimensional vector. Then we simply require that the matrix whose  $\alpha, \beta$ -th element is given by
$$
- p ^ {2} \frac {1}{1 - (1 - p) \lambda_ {\alpha}} \frac {1}{1 - (1 - p) \lambda_ {\beta}} \frac {(\lambda_ {\alpha} - 1) (\lambda_ {\beta} - 1)}{1 - (1 - 2 p) \lambda_ {\alpha} \lambda_ {\beta}} \tag {38}
$$
is negative definite, which is manifestly the case provided  $p \leq \frac{1}{2}$  (e.g. by Taylor expanding the denominator and observing that it is a sum of terms symmetric in  $\lambda_{\alpha,\beta}$  with negative coefficients). It follows that  $\Delta p$  (both terminate at  $i$ )  $\leq 0$ , and the claim in Eq. 31 follows.
We run a simple numerical test to check Thm 8.1 for a number of graphs (see Sec. 4.1 of the main text for details), and find that using antithetic termination does indeed improve the quality of estimation of the PageRank vector. Fig. 4 reports the results, showing the error on the estimator  $\| \pmb{\pi} - \widehat{\pmb{\pi}} \|_2^2$  for both schemes with 2, 4, 8 and 16 walkers. We choose a termination probability  $p = 0.3$ . As per the theoretical guarantees, antithetic random walks consistently yield more accurate estimators  $\widehat{\pi}$ .
Figure 4: Error of the estimator of the PageRank vector  $\|\pi - \widehat{\pi}\|_2^2$  when approximated using i) i.i.d. and ii) antithetic random walks. Lower is better. Antithetic termination yields a more accurate estimate for every graph considered.  $N$  is the number of nodes and, for the Erdős-Rényi (ER) graphs,  $p$  is the edge-generation probability. One standard deviation is shaded but it is too small to easily see.
8.2 On the approximation of the  $d$ -regularised Laplacian using GRFs
In this appendix, we demonstrate how to approximate the  $d$ -regularised Laplacian  $\mathbf{K}_{\mathrm{lap}}^{(d)}$  with GRFs.
Recall that GRFs provide an estimator to the quantity  $(\mathbf{I}_N - \mathbf{U})^{-2}$  where  $\mathbf{U}$  is a weighted adjacency matrix. Recall also that the matrix elements of the symmetrically normalised Laplacian  $\widetilde{\mathbf{L}}$  are given by
$$
\widetilde {\mathbf {L}} _ {i j} = \left\{ \begin{array}{l l} 1 & \text {i f} i = j, \\ - \frac {\mathbf {W} _ {\mathrm {i j}}}{\sqrt {\deg_ {\mathbf {W}} (i) \deg_ {\mathbf {W}} (j)}} & \text {i f} i \in \mathcal {N} (j). \end{array} \right. \tag {39}
$$
where  $\deg_{\mathbf{W}}(i) = \sum_{j\in \mathcal{N}}\mathbf{W}_{ij}$  is the weighted degree of the node  $i$ . We are often interested in situations where  $\mathbf{W} = \mathbf{A}$ , an unweighted adjacency matrix. Now note that
$$
\mathbf {K} _ {\text {l a p} i j} ^ {(2)} = \left(\mathbf {I} _ {N} + \sigma^ {2} \widetilde {\mathbf {L}}\right) _ {i j} ^ {- 2} = \left(1 + \sigma^ {2}\right) ^ {- 2} \left(\mathbf {I} _ {N} - \mathbf {U}\right) _ {i j} ^ {- 2} \tag {40}
$$
where we defined the matrix  $\mathbf{U}$  with matrix elements
$$
\mathbf {U} _ {i j} = \frac {\sigma^ {2}}{1 + \sigma^ {2}} \frac {\mathbf {W} _ {i j}}{\sqrt {\deg_ {\mathbf {W}} (i) \deg_ {\mathbf {W}} (j)}}. \tag {41}
$$
This is itself a weighted adjacency matrix. It follows that, by estimating  $(\mathbf{I}_N - \mathbf{U})^{-2}$  with GRFs, we can estimate  $\mathbf{K}_{\mathrm{lap}}^{(2)}$ . This was reported by Choromanski [2023].
Supposing that we have constructed a GRF estimator
$$
\mathbf {K} _ {\mathrm {l a p}} ^ {(2)} = \mathbb {E} \left[ \mathbf {C C} ^ {\top} \right] \tag {42}
$$
where the matrix  $\mathbf{C} \in \mathbb{R}^{N \times N}$  has rows  $\mathbf{C}_i := \frac{1}{1 + \sigma^2} \phi(i)^\top$ ,<sup>3</sup> we note that it is straightforward to construct the 1-regularised Laplacian kernel estimator
$$
\mathbf {K} _ {\text {l a p}} ^ {(1)} = \mathbb {E} \left[ \mathbf {C D} ^ {\top} \right] \tag {43}
$$
by taking  $\mathbf{D} \coloneqq \left(\mathbf{I}_N + \sigma^2\widetilde{\mathbf{L}}\right)^\top \mathbf{C}$ . By multiplying independent copies of the estimators  $\widehat{\mathbf{K}}_{\mathrm{lap}}^{(2)}$  and  $\widehat{\mathbf{K}}_{\mathrm{lap}}^{(1)}$ , it is then trivial to obtain an unbiased estimator  $\mathbf{K}_{\mathrm{lap}}^{(d)}$  for arbitrary  $d \in \mathbb{N}$ . For instance,
$$
\widehat {\mathbf {K}} _ {\text {l a p}} ^ {(5)} = \mathbf {C} _ {1} \mathbf {C} _ {1} ^ {\top} \mathbf {C} _ {2} \mathbf {C} _ {2} ^ {\top} \mathbf {C} _ {3} \mathbf {D} _ {3} ^ {\top}. \tag {44}
$$
3Technically, for unbiasedness the random walks used for  $\mathbf{C}$  and  $\mathbf{C}^{\top}$  should be different, though this rarely makes any observable difference. See App. 8.7 for further discussion.
$$
\mathbf {K} _ {\text {d i f f}} := \exp (- \sigma^ {2} \widetilde {\mathbf {L}} t) = \lim  _ {n \rightarrow \infty} \left(1 - \frac {\sigma^ {2} \widetilde {\mathbf {L}} t}{n}\right) ^ {n} \simeq \left(1 - \frac {\sigma^ {2} \widetilde {\mathbf {L}} t}{d}\right) ^ {d} \tag {45}
$$
with  $d \gg 1$  the number of discretisation timesteps. We again included a regulariser  $\sigma$ . This shows that the ability to approximate the  $d$ -regularised Laplacian kernel for arbitrary  $d$  permits us approximate the diffusion kernel, with the corresponding estimator asymptotically unbiased as  $d \to \infty$ . In Sec. 4.2 we see that the quality of approximation (in this case measured by the accuracy of simulated diffusion) is good even with modest  $d$ .
We also note that the variance of estimates  $\widehat{\mathbf{K}}_{\mathrm{lap}}^{(d)}$  and  $\widehat{\mathbf{K}}_{\mathrm{diff}}$  constructed in this manner will vary monotonically with the variance of the approximation of the constituent estimators  $\widehat{\mathbf{K}}_{\mathrm{lap}}^{(2)}$ , so improvements provided by our QMC scheme (and any theoretical guarantees) will follow for this more general class of kernels.
8.3 Walk lengths with antithetic termination are anticorrelated (derivation of Eq. 15)
In this appendix we derive Eq. 15, which gives the expected length of some walk  $\omega_{2}$  given that its antithetic partner  $\omega_{1}$  is of length  $m$ : that is,  $\mathbb{E}(\mathrm{len}(\omega_2)|\mathrm{len}(\omega_1) = m)$ .
As a warm-up, consider the simpler marginal expected lengths. Note that
$$
p (\operatorname {l e n} (\omega) = m) = (1 - p) ^ {m} p. \tag {46}
$$
$$
\mathbb {E} (\operatorname {l e n} (\omega)) = \sum_ {m = 0} ^ {\infty} m (1 - p) ^ {m} p = \frac {1 - p}{p} \tag {47}
$$
where we evaluated the arithmetic-geometric series. We reported this result in Eq. 14. Meanwhile, the probability of a walk being of length  $i$  given that its antithetic partner is of length  $m$  is
$$
p \left(\operatorname {l e n} \left(\omega_ {2}\right) = i \mid \operatorname {l e n} \left(\omega_ {1}\right) = m\right) = \left\{ \begin{array}{l l} {\left(\frac {1 - 2 p}{1 - p}\right) ^ {i} \frac {p}{1 - p}} & {\text {i f} i <   m,} \\ {0} & {\text {i f} i = m,} \\ {\left(\frac {1 - 2 p}{1 - p}\right) ^ {m} (1 - p) ^ {i - m - 1} p} & {\text {i f} i > m.} \end{array} \right. \tag {48}
$$
The analogous sum then becomes
$$
\mathbb {E} \left(\ln \left(\omega_ {2}\right) \mid \ln \left(\omega_ {1}\right) = m\right) = \sum_ {i = 0} ^ {m - 1} \left(\frac {1 - 2 p}{1 - p}\right) ^ {i} \frac {p}{1 - p} i + \sum_ {i = m + 1} ^ {\infty} \left(\frac {1 - 2 p}{1 - p}\right) ^ {m} (1 - p) ^ {i - m - 1} p i. \tag {49}
$$
After straightforward but tedious algebra, this evaluates to
$$
\mathbb {E} \left(\operatorname {l e n} \left(\omega_ {2}\right) \mid \operatorname {l e n} \left(\omega_ {1}\right) = m\right) = \frac {1 - 2 p}{p} + 2 \left(\frac {1 - 2 p}{1 - p}\right) ^ {m}, \tag {50}
$$
as stated in Eq. 15. Note that this is greater than  $\mathbb{E}(\mathrm{len}(\omega))$  when  $m$  is small and smaller than  $\mathbb{E}(\mathrm{len}(\omega))$  when  $m$  is large; the two walk lengths are negatively correlated.
8.4 On the superiority of q-GRFs (proof of Theorem 3.2)
Here, we provide a proof of the central result of Theorem 3.2: that the introduction of antithetic termination reduces the variance of estimators of the matrix  $(\mathbf{I}_N - \mathbf{U})^{-2}$ . From App. 8.2, all our results will trivially extend to the 2-regularised Laplacian kernel  $\mathbf{K}_{\mathrm{lap}}^{(2)}$ .
We will begin by assuming that the graph is  $d$ -regular, that all edges have equal weights denoted  $w$ , and that our sampling strategy involves the random walker choosing one of its neighbours with equal probability at each timestep. We will relax these assumptions in App. 8.5.
We have seen that antithetic termination does not modify the walkers' marginal termination behaviour, so the variance of the estimator  $\phi (i)^{\top}\phi (j)$  is only affected via the second-order term  $\mathbb{E}\left[(\phi (i)^{\top}\phi (j))^{2}\right]$ . Writing out the sums,
$$
\begin{array}{l} \left(\phi (i) ^ {\top} \phi (j)\right) ^ {2} = \frac {1}{m ^ {4}} \sum_ {x, y \in \mathcal {N}} \sum_ {k _ {1}, l _ {1}, k _ {2}, l _ {2} = 1} ^ {m} \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \sum_ {\omega_ {3} \in \Omega_ {i y}} \sum_ {\omega_ {4} \in \Omega_ {j y}} \frac {\widetilde {\omega} (\omega_ {1})}{p (\omega_ {1})} \frac {\widetilde {\omega} (\omega_ {2})}{p (\omega_ {2})} \frac {\widetilde {\omega} (\omega_ {3})}{p (\omega_ {3})} \frac {\widetilde {\omega} (\omega_ {4})}{p (\omega_ {4})} \\ \cdot \mathbb {I} \left(\omega_ {1} \in \bar {\Omega} \left(k _ {1}, i\right)\right) \mathbb {I} \left(\omega_ {2} \in \bar {\Omega} \left(l _ {1}, j\right)\right) \mathbb {I} \left(\omega_ {3} \in \bar {\Omega} \left(k _ {2}, i\right)\right) \mathbb {I} \left(\omega_ {4} \in \bar {\Omega} \left(l _ {2}, j\right)\right). \tag {51} \\ \end{array}
$$
To remind the reader: the variables  $x, y$  sum over the nodes of the graph  $\mathcal{N}$ .  $k_{1}$  and  $l_{1}$  enumerate all the  $m$  walks sampled out of node  $i$ , whilst  $k_{2}$  and  $l_{2}$  enumerate walks from  $j$ . The sum over  $\omega_{1} \in \Omega_{ix}$  is over all possible walks between nodes  $i$  and  $x$ .  $\widetilde{\omega}(\omega_{1})$  evaluates the product of edge weights traversed by the walk  $\omega_{1}$ , which is  $w^{\mathrm{len}(\omega_1)}$  in the equal-weights case (with  $\mathrm{len}(\omega_1)$  denoting the number of edges in  $\omega_{1}$ ).  $p(\omega_{1})$  is the marginal probability of the prefix subwalk  $\omega_{1}$ , which is equal to  $((1 - p) / d)^{\mathrm{len}(\omega_1)}$  on a  $d$ -regular graph. Lastly, the indicator function  $\mathbb{I}(\omega_1 \in \bar{\Omega}(k_1, i))$  evaluates to 1 if the  $k_{1}$ th walk out of node  $i$  (denoted  $\bar{\Omega}(k_1, i)$ ) contains the walk  $\omega_{1}$  as a prefix subwalk and 0 otherwise.
We immediately note that our scheme only ever correlates walks leaving the same node, so walks out of different nodes remain independent. Therefore,
$$
\begin{array}{l} \mathbb {E} \left[ \mathbb {I} \left(\omega_ {1} \in \bar {\Omega} \left(k _ {1}, i\right)\right) \mathbb {I} \left(\omega_ {2} \in \bar {\Omega} \left(l _ {1}, j\right)\right) \mathbb {I} \left(\omega_ {3} \in \bar {\Omega} \left(k _ {2}, i\right)\right) \mathbb {I} \left(\omega_ {4} \in \bar {\Omega} \left(l _ {2}, j\right)\right) \right] \tag {52} \\ = p \left(\omega_ {1} \in \bar {\Omega} \left(k _ {1}, i\right), \omega_ {3} \in \bar {\Omega} \left(k _ {2}, i\right)\right) p \left(\omega_ {2} \in \bar {\Omega} \left(l _ {1}, j\right), \omega_ {4} \in \bar {\Omega} \left(l _ {2}, j\right)\right). \\ \end{array}
$$
Consider the term in the sum corresponding to one particular set of walks  $(k_{1},l_{1},k_{2},l_{2})$
$$
\begin{array}{l} \sum_ {x, y \in \mathcal {N}} \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \sum_ {\omega_ {3} \in \Omega_ {i y}} \sum_ {\omega_ {4} \in \Omega_ {j y}} \frac {\widetilde {\omega} (\omega_ {1})}{p (\omega_ {1})} \frac {\widetilde {\omega} (\omega_ {2})}{p (\omega_ {2})} \frac {\widetilde {\omega} (\omega_ {3})}{p (\omega_ {3})} \frac {\widetilde {\omega} (\omega_ {4})}{p (\omega_ {4})} \tag {53} \\ \cdot p \left(\omega_ {1} \in \bar {\Omega} \left(k _ {1}, i\right), \omega_ {3} \in \bar {\Omega} \left(k _ {2}, i\right)\right) p \left(\omega_ {2} \in \bar {\Omega} \left(l _ {1}, j\right), \omega_ {4} \in \bar {\Omega} \left(l _ {2}, j\right)\right). \\ \end{array}
$$
This object will be of central importance and is referred to as the correlation term. In the sum over  $k_{1}, k_{2}, l_{1}, l_{2}$ , there are three possibilities to consider. We stress again that  $k_{1,2}$  refers to a pair of walks out of node  $i$  and  $l_{1,2}$  refers to a pair out of  $j$ .
- Case 1, same-same,  $k_{1} = k_{2}, l_{1} = l_{2}$ : the pair of walks out of  $i$  are identical and the pair of walks out of  $j$  are identical. This term will not be modified by antithetic coupling since the marginal walk behaviour is unmodified and walks out of different nodes remain independent.
- Case 2, different-different,  $k_{1} \neq k_{2}, l_{1} \neq l_{2}$ : the walks out of both  $i$  and  $j$  differ, and each pair may be antithetic or independent. This term will be modified by the coupling.
- Case 3, same-different.  $k_{1} = k_{2}, l_{1} \neq l_{2}$ : the walks out of  $i$  differ – and may exhibit antithetic or independent termination – but the walks out of  $j$  are the same. This term will be modified by the coupling. Note that the  $i$  and  $j$  labels are arbitrary so we have chosen one ordering for concreteness.
If we can reason that the contributions from each of these possibilities  $1 - 3$  either remains the same or is reduced by the introduction of antithetic coupling, then from Eq. 51 we can conclude that the entire sum and therefore the Laplacian kernel estimator variance is suppressed. For completeness, we
write out the entire sum from Eq. 51 with the degeneracy factors below:
$$
\begin{array}{l} \left(\phi (i) ^ {\top} \phi (j)\right) ^ {2} = \frac {1}{m ^ {4}} \sum_ {x, y \in \mathcal {N}} \Big \{ \\ \left. \begin{array}{l} m ^ {2} \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \sum_ {\omega_ {3} \in \Omega_ {i y}} \sum_ {\omega_ {4} \in \Omega_ {j y}} \frac {\tilde {\omega} (\omega_ {1})}{p (\omega_ {1})} \frac {\tilde {\omega} (\omega_ {2})}{p (\omega_ {2})} \frac {\tilde {\omega} (\omega_ {3})}{p (\omega_ {3})} \frac {\tilde {\omega} (\omega_ {4})}{p (\omega_ {4})} \\ \cdot \mathbb {I} (\omega_ {1} \in \bar {\Omega} (k _ {1}, i)) \mathbb {I} (\omega_ {2} \in \bar {\Omega} (l _ {1}, j)) \mathbb {I} (\omega_ {3} \in \bar {\Omega} (k _ {1}, i)) \mathbb {I} (\omega_ {4} \in \bar {\Omega} (l _ {1}, j)) \end{array} \right\} \text {s a m e - s a m e (1)} \\ \left. \begin{array}{l} + m ^ {2} (m - 1) ^ {2} \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \sum_ {\omega_ {3} \in \Omega_ {i y}} \sum_ {\omega_ {4} \in \Omega_ {j y}} \frac {\tilde {\omega} (\omega_ {1})}{p (\omega_ {1})} \frac {\tilde {\omega} (\omega_ {2})}{p (\omega_ {2})} \frac {\tilde {\omega} (\omega_ {3})}{p (\omega_ {3})} \frac {\tilde {\omega} (\omega_ {4})}{p (\omega_ {4})} \\ \cdot \mathbb {I} (\omega_ {1} \in \bar {\Omega} (k _ {1}, i)) \mathbb {I} (\omega_ {2} \in \bar {\Omega} (l _ {1}, j)) \mathbb {I} (\omega_ {3} \in \bar {\Omega} (k _ {2}, i)) \mathbb {I} (\omega_ {4} \in \bar {\Omega} (l _ {2}, j)) \end{array} \right\} d i f f e r e n t - d i f f e r e n t (2) \\ \left. \begin{array}{l} + m ^ {2} (m - 1) \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \sum_ {\omega_ {3} \in \Omega_ {i y}} \sum_ {\omega_ {4} \in \Omega_ {j y}} \frac {\tilde {\omega} (\omega_ {1})}{p (\omega_ {1})} \frac {\tilde {\omega} (\omega_ {2})}{p (\omega_ {2})} \frac {\tilde {\omega} (\omega_ {3})}{p (\omega_ {3})} \frac {\tilde {\omega} (\omega_ {4})}{p (\omega_ {4})} \\ \cdot \mathbb {I} (\omega_ {1} \in \bar {\Omega} (k _ {1}, i)) \mathbb {I} (\omega_ {2} \in \bar {\Omega} (l _ {1}, j)) \mathbb {I} (\omega_ {3} \in \bar {\Omega} (k _ {2}, i)) \mathbb {I} (\omega_ {4} \in \bar {\Omega} (l _ {1}, j)) \\ + m ^ {2} (m - 1) \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \sum_ {\omega_ {3} \in \Omega_ {i y}} \sum_ {\omega_ {4} \in \Omega_ {j y}} \frac {\tilde {\omega} (\omega_ {1})}{\tilde {p} (\omega_ {1})} \frac {\tilde {\omega} (\omega_ {2})}{\tilde {p} (\omega_ {2})} \frac {\tilde {\omega} (\omega_ {3})}{\tilde {p} (\omega_ {3})} \frac {\tilde {\omega} (\omega_ {4})}{\tilde {p} (\omega_ {4})} \\ \cdot \mathbb {I} (\omega_ {1} \in \bar {\Omega} (k _ {1}, i)) \mathbb {I} (\omega_ {2} \in \bar {\Omega} (l _ {1}, j)) \mathbb {I} (\omega_ {3} \in \bar {\Omega} (k _ {1}, i)) \mathbb {I} (\omega_ {4} \in \bar {\Omega} (l _ {2}, j)). \Bigg \} \\ \end{array} \right\} s a m e - d i f f e r e n t (3) \tag {54} \\ \end{array}
$$
We now address each case  $1 - 3$  in turn.
8.4.1 Case 1:  $k_{1} = k_{2}, l_{1} = l_{2}$
As argued above, case 1 is trivial. By design, antithetic termination does not affect the marginal walk behaviour (a sufficient condition for the estimator to remain unbiased). This means that it cannot affect terms that consider a single walk out of node  $i$  and a single walk out of  $j$ , and all terms of case 1 are unchanged by the introduction of antithetic termination.
8.4.2 Case 2:  $k_{1} \neq k_{2}, l_{1} \neq l_{2}$
Now we consider terms where both the walks out of node  $i$  and the walks out of node  $j$  differ. To emphasise, we are considering 4 different random walks: 2 out of  $i$  and 2 out of  $j$ .
Within this setting, we will need to consider the situations where either i) one or ii) both of the pairs exhibit antithetic termination rather than i.i.d.. Terms of both kind will appear when we use ensembles of antithetic pairs. We need to check that in both cases the result is smaller compared to when both pairs are i.i.d..
To evaluate these terms, we first need to understand how inducing antithetic termination modifies the joint distribution  $p(\omega_1 \in \bar{\Omega}(k_1, i), \omega_3 \in \bar{\Omega}(k_2, i))$ : namely, the probability that two randomly sampled walks  $\bar{\Omega}(k_1, i)$  and  $\bar{\Omega}(k_2, i)$  contain the respective subwalks  $\omega_1$  and  $\omega_3$ , given that their termination is either i.i.d. or antithetic. In the i.i.d. case, it is straightforward to convince oneself that
$$
p \left(\omega_ {1} \in \bar {\Omega} (1, i), \omega_ {3} \in \bar {\Omega} (3, i)\right) = \left(\frac {1 - p}{d}\right) ^ {m} \left(\frac {1 - p}{d}\right) ^ {n}, \tag {55}
$$
where  $m$  and  $n$  denote the lengths of subwalks  $\omega_{1}$  and  $\omega_{3}$ , respectively. With antithetic termination, from Eq. 13 it follows that the probability of sampling a walk  $\bar{\Omega}_{3}$  of length  $j$  conditioned on sampling an antithetic partner  $\bar{\Omega}_{1}$  of length  $i$  is
$$
p \left(\operatorname {l e n} \left(\bar {\Omega} _ {3}\right) = j \mid \operatorname {l e n} \left(\bar {\Omega} _ {1}\right) = i\right) = \left\{ \begin{array}{l l} {\left(\frac {1 - 2 p}{1 - p}\right) ^ {j} \frac {p}{1 - p}} & {\text {i f} j <   i,} \\ {0} & {\text {i f} j = i,} \\ {\left(\frac {1 - 2 p}{1 - p}\right) ^ {i} (1 - p) ^ {j - i - 1} p} & {\text {i f} j > i.} \end{array} \right. \tag {56}
$$
Using these probabilities, it is then straightforward but algebraically tedious to derive the joint probabilities over subwalks
$$
p \left(\omega_ {1} \in \bar {\Omega} (1, i), \omega_ {3} \in \bar {\Omega} (3, i)\right) = \left\{ \begin{array}{l l} \frac {1}{d ^ {m + n}} \left(\frac {1 - 2 p}{1 - p}\right) ^ {n} (1 - p) ^ {m} & \text {i f} n <   m, \\ \frac {1}{d ^ {2 m}} (1 - 2 p) ^ {m} & \text {i f} n = m, \\ \frac {1}{d ^ {m + n}} \left(\frac {1 - 2 p}{1 - p}\right) ^ {m} (1 - p) ^ {n} & \text {i f} n > m, \end{array} \right. \tag {57}
$$
where  $m$  is the length of  $\omega_{1}$ ,  $n$  is the length of  $\omega_{3}$  and  $i$  is now the index of a particular node.
To be explicit, we have integrated over the conditional probabilities of walks of particular lengths  $(i,j)$  to obtain the joint probabilities of sampled walks containing subwalks of particular lengths  $(m,n)$ . Let us consider the case of  $n < m$  as an example. Using Eq. 56,
$$
\begin{array}{l} p \left(\omega_ {1} \in \bar {\Omega} (1, i), \omega_ {3} \in \bar {\Omega} (3, i)\right) = \frac {1}{d ^ {m + n}} \sum_ {i = m} ^ {\infty} \left[ \sum_ {j = n} ^ {i - 1} \left(\frac {1 - 2 p}{1 - p}\right) ^ {j} \frac {p}{1 - p} (1 - p) ^ {i} p + \right. \tag {58} \\ + \sum_ {j = i + 1} ^ {\infty} \left(\frac {1 - 2 p}{1 - p}\right) ^ {i} (1 - p) ^ {j - i - 1} p (1 - p) ^ {i} p \Bigg ], \\ \end{array}
$$
where the branching factors of  $d$  appeared because at every timestep the subwalks have  $d$  possible edges to choose from. After we have completed the particular subwalks of lengths  $m$  and  $n$  we no longer care about where the walks go, just their lengths, so we stop accumulating these multiplicative factors. Computing the summations in Eq. 58 (which are all straightforward geometric series), we quickly arrive at the top line of Eq. 57.
Returning to our main discussion, note that in the  $d$ -regular, equal-weights case,