SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Isaac Reid
ir337@cam.ac.uk
Krzysztof Choromanski*
aw665@cam.ac.uk
We present a novel mechanism to improve the accuracy of the recently-introduced class of graph random features (GRFs) [Choromanski, 2023]. Our method induces negative correlations between the lengths of the algorithm's random walks by imposing antithetic termination: a procedure to sample more diverse random walks which may be of independent interest. It has a trivial drop-in implementation. We derive strong theoretical guarantees on the properties of these quasi-Monte Carlo GRFs (q-GRFs), proving that they yield lower-variance estimators of the 2-regularised Laplacian kernel under mild conditions. Remarkably, our results hold for any graph topology. We demonstrate empirical accuracy improvements on a variety of tasks including a new practical application: time-efficient approximation of the graph diffusion process. To our knowledge, q-GRFs constitute the first rigorously studied quasi-Monte Carlo scheme for kernels defined on combinatorial objects, inviting new research on correlations between graph random walks. $^{1}$
Kernel methods are ubiquitous in machine learning [Canu and Smola, 2006, Smola and Scholkopf, 2002, Kontorovich et al., 2008, Campbell, 2002]. Via the kernel trick, they provide a mathematically principled and elegant way to perform nonlinear inference using linear learning algorithms. The positive definite kernel function  $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$ , defined on an input domain  $\mathcal{X}$ , measures the 'similarity' between two datapoints. Examples in Euclidean space include the Gaussian, linear, Matérn, angular and arc-cosine kernels [Williams and Rasmussen, 2006, Cho and Saul, 2011].
Though very effective on small datasets, kernel methods suffer from poor scalability. The need to materialise and invert the kernel matrix typically leads to a time-complexity cubic in the size of the dataset. Substantial research has been dedicated to improving scalability by approximating this matrix, notably including random features (RFs) [Rahimi and Recht, 2007, 2008, Avron et al., 2017, Liu et al., 2022]. These randomised mappings  $\phi : \mathbb{R}^d \to \mathbb{R}^s$  construct low-dimensional feature vectors whose dot product equals the kernel evaluation in expectation:
$$
k (\boldsymbol {x}, \boldsymbol {y}) = \mathbb {E} \left(\phi (\boldsymbol {x}) ^ {\top} \phi (\boldsymbol {y})\right). \tag {1}
$$
This permits a low-rank decomposition of the kernel matrix which enables better time- and space-complexity than exact kernel methods. Random feature methods exist for a variety of Euclidean kernels with properties engineered for the desiderata and symmetries of the particular kernel being
*Senior lead.
<sup>1</sup> Source code is available at https://github.com/isaac-reid/antithetictermination.
approximated [Dasgupta et al., 2010, Johnson, 1984, Choromanski et al., 2020, Goemans and Williamson, 2001, Rahimi and Recht, 2007].
Kernels can also be defined on discrete input spaces such as graphs, which are the natural way to represent data characterised by local relationships (e.g. social networks or interacting chemicals [Albert and Barabási, 2002]) or when data is restricted to a lower-dimensional manifold than the original space [Roweis and Saul, 2000, Belkin and Niyogi, 2003]. We consider graph kernels  $k: \mathcal{N} \times \mathcal{N} \to \mathbb{R}$  on the set of nodes  $\mathcal{N}$  of a graph  $G$ . Examples include the diffusion, regularised Laplacian,  $p$ -step random walk and cosine kernels [Smola and Kondor, 2003, Kondor and Lafferty, 2002, Chung and Yau, 1999], which have found applications including in bioinformatics [Borgwardt et al., 2005], community detection [Kloster and Gleich, 2014] and recommender systems [Yajima, 2006]. More recently, these kernels have been used in manifold learning for deep generative modelling [Zhou et al., 2020] and for solving shortest path problems [Crane et al., 2017]. Substantial research effort has also been devoted to developing and analysing graph kernels  $k: \mathcal{G} \times \mathcal{G} \to \mathbb{R}$ , now taking entire graphs  $G \in \mathcal{G}$  from graph spaces  $\mathcal{G}$  as inputs rather than their nodes [Shervashidze et al., 2009, Vishwanathan et al., 2006, Shervashidze and Borgwardt, 2009], but we stress that these are not the subject of this paper.
The problem of poor kernel scalability is exacerbated in the graph domain because even computing the kernel matrix  $\mathbf{K}$  is typically of at least cubic time-complexity in the number of nodes  $N$ . In contrast to kernels defined on points in  $\mathbb{R}^d$ , random feature methods for fixed graph kernels (c.f. kernel learning [Fang et al., 2021]) have proved challenging to construct. Only recently has a viable graph random feature (GRF) mechanism been proposed, which uses a series of random walkers depositing 'load' at each node they pass through [Choromanski, 2023]. GRFs provide an unbiased estimate of the matrix  $(\mathbf{I}_N - \mathbf{U})^{-d}$ , where  $\mathbf{U}$  is a weighted adjacency matrix of the graph and  $d \in \mathbb{N}$ . The decomposition supports subquadratic time-complexity (again with respect to the number of nodes  $N$ ) in downstream algorithms applying regularised Laplacian kernels. Moreover, the computation of GRFs admits a simple distributed algorithm that can be applied if a large graph needs to be split across machines. The author demonstrates the strong empirical performance of GRFs in speed tests, Frobenius relative error analysis and a  $k$ -means graph clustering task.
In the Euclidean setting, significant research has been dedicated to developing quasi-Monte Carlo (QMC) variants to RF methods that enjoy better convergence properties [Yang et al., 2014, Lyu, 2017, Dick et al., 2013]. By using correlated ensembles rather than i.i.d. random variables in the feature maps, one can suppress the mean squared error (MSE) of the kernel estimator. For example, orthogonal random features (ORFs) [Yu et al., 2016, Choromanski et al., 2020] improve the quality of approximation of the Gaussian kernel when using trigonometric or positive random features, and of the linear kernel in the orthogonal Johnson-Lindenstrauss transformation [Choromanski et al., 2017]. With positive random features, the recently-introduced class of simplex random features (SimRFs) performs even better [Reid et al., 2023]. This has been used to great effect in estimating the attention mechanism of Transformers [Vaswani et al., 2017], overcoming its prohibitive quadratic time-complexity scaling with token sequence length.
This invites the central question of this work: how can we implement a QMC mechanism for random walks on a graph? What do we mean by a 'diverse' sample in this context? Choromanski [2023] first identified the challenge of constructing quasi-Monte Carlo GRFs (q-GRFs). They suggested a high-level approach of reinforced random walks, but left to future work its theoretical and empirical analysis. In this paper, we provide a first concrete implementation of q-GRFs, proposing an unbiased scheme that correlates the length of random walks by imposing antithetic termination. We derive strong theoretical guarantees on the properties of this new class, proving that the correlations reduce the variance of estimators of the 2-regularised Laplacian kernel under mild conditions. Our results hold for any graph topology. We also demonstrate empirical accuracy improvements on a variety of tasks. We hope our new algorithm (hereafter referred to as 'q-GRFs' for brevity) will spur further research on correlations between graph random walks in machine learning.
We emphasise that, although we present antithetic termination through the lens of q-GRFs (an analytically tractable and important use case), it is fundamentally a procedure to obtain a more diverse ensemble of graph random walks and may be of independent interest. To illustrate this, in App 8.1 we show how it can alternatively be used to improve approximations of the PageRank vector [Page et al., 1998]. It could also be relevant e.g. for estimating graphlet statistics for kernels between graphs
[Chen et al., 2016, Ribeiro et al., 2021] or in some GNN architectures [Nikolentzos and Vazirgiannis, 2020].
The remainder of the manuscript is organised as follows. In Sec. 2 we introduce the mathematical concepts and existing algorithms to be used in the paper, including the  $d$ -regularised Laplacian and diffusion kernels and the GRF mechanism. Sec. 3 presents our novel q-GRFs mechanism and discusses its strong theoretical guarantees - in particular, that it provides lower kernel estimator variance than its regular predecessor (GRFs) under mild conditions. We provide a brief proof-sketch for intuition but defer full technical details to App. 8.4. We conduct an exhaustive set of experiments in Sec. 4 to compare q-GRFs to GRFs, including: (a) quality of kernel approximation via computation of the relative Frobenius norm; (b) simulation of the graph diffusion process; (c) kernelised  $k$ -means node clustering; and (d) kernel regression for node attribute prediction. q-GRFs nearly always perform better and in some applications the difference is substantial.
2 Graph kernels and GRFs
2.1 The Laplacian, heat kernels and diffusion on graphs
An undirected, weighted graph  $\mathrm{G}(\mathcal{N},\mathcal{E})$  is defined by a set of vertices  $\mathcal{N}\coloneqq \{1,\dots ,N\}$  and a set of edges  $\mathcal{E}$  given by the unordered vertex pairs  $(i,j)\in \mathcal{E}$  where  $i$  and  $j$  are neighbours, themselves associated with weights  $W_{ij}\in \mathbb{R}$ . The weighted adjacency matrix  $\mathbf{W}\in \mathbb{R}^{N\times N}$  has matrix elements  $W_{ij}$ : that is, the associated edge weights if  $(i,j)\in \mathcal{E}$  and 0 otherwise. Let  $\mathcal{N}(i)\coloneqq \{j\in \mathcal{N}|(i,j)\in \mathcal{E}\}$  denote the set of neighbours of node  $i$ .
Denote by  $\mathbf{D} \in \mathbb{R}^{N \times N}$  the diagonal matrix with elements  $D_{ii} \coloneqq \sum_{j} W_{ij}$ , the sum of edge weights connecting a vertex  $i$  to its neighbours. The Laplacian of  $G$  is then defined  $\mathbf{L} \coloneqq \mathbf{D} - \mathbf{W}$ . The normalised Laplacian is  $\widetilde{\mathbf{L}} \coloneqq \mathbf{D}^{-\frac{1}{2}} \mathbf{L} \mathbf{D}^{-\frac{1}{2}}$ , which rescales  $\mathbf{L}$  by the (weighted) number of edges per node.  $\mathbf{L}$  and  $\widetilde{\mathbf{L}}$  share eigenvectors in the case of an equal-weights  $d$ -regular graph and play a central role in spectral graph theory; their analytic properties are well-understood [Chung, 1997].
In classical physics, diffusion through continuous media is described by the equation
$$
\frac {d \mathbf {u}}{d t} = \nabla^ {2} \mathbf {u} \tag {2}
$$
where  $\nabla^2 = \frac{\partial^2}{\partial x_1^2} + \frac{\partial^2}{\partial x_2^2} + \dots \frac{\partial^2}{\partial x_N^2}$  is the Laplacian operator on continuous spaces. The natural analogue on discrete spaces is  $\mathbf{L}$ , where we now treat  $\mathbf{L}$  as a linear operator on vectors  $\mathbf{u} \in \mathbb{R}^N$ . This can be seen by noting that, if we take  $\mathbf{W}$  to be the unweighted adjacency matrix,  $\langle \mathbf{u}, \mathbf{L}\mathbf{u} \rangle = \mathbf{u}^\top \mathbf{L}\mathbf{u} = -\frac{1}{2}\sum_{(i,j) \in \mathcal{N}} (u_i - u_j)^2$  so, like its continuous counterpart,  $\mathbf{L}$  measures the local smoothness of its domain. In fact, in this case  $-\mathbf{L}$  is exactly the finite difference discretisation of  $\nabla^2$  on a square grid in  $N$ -dimensional Euclidean space. This motivates the discrete heat equation on  $G$ ,
$$
\frac {d \mathbf {u}}{d t} = - \widetilde {\mathbf {L}} \mathbf {u}, \tag {3}
$$
where we followed literature conventions by using the normalised variant whose spectrum is conveniently contained in [0, 2] [Chung, 1997]. This has the solution  $\mathbf{u}_t = \exp (-\widetilde{\mathbf{L}} t)\mathbf{u}_0$ , where  $\exp (-\widetilde{\mathbf{L}} t) = \lim_{n\to \infty}\left(1 - \frac{t\widetilde{\mathbf{L}}}{n}\right)^n$ . The symmetric and positive semi-definite matrix
$$
\mathbf {K} _ {\text {d i f f}} (t) := \exp (- \widetilde {\mathbf {L}} t) \tag {4}
$$
is referred to as the heat kernel or diffusion kernel [Smola and Kondor, 2003]. The exponentiation of the generator  $\widetilde{\mathbf{L}}$ , which by construction captures the local structure of  $G$ , leads to a kernel matrix  $\mathbf{K}_{\mathrm{diff}}$  which captures the graph's global structure. Upon discretisation of Eq. 3 with the backward Euler step (which is generally more stable than the forward), we have that
$$
\mathbf {u} _ {t + \delta t} = \left(\mathbf {I} _ {N} + \delta t \tilde {\mathbf {L}}\right) ^ {- 1} \mathbf {u} _ {t}, \tag {5}
$$
where the discrete time-evolution operator  $\mathbf{K}_{\mathrm{lap}}^{(1)} = (\mathbf{I}_N + \delta t\widetilde{\mathbf{L}})^{-1}$  is referred to as the 1-regularised Laplacian kernel. This is a member of the more general family of  $d$ -regularised Laplacian kernels,
$$
\mathbf {K} _ {\mathrm {l a p}} ^ {(d)} = \left(\mathbf {I} _ {N} + \delta t \widetilde {\mathbf {L}}\right) ^ {- d}, \tag {6}
$$
for which we can construct an unbiased approximation using GRFs [Choromanski, 2023]. We predominantly consider  $d = 2$  with the understanding that estimators for other values of  $d$  (and the diffusion kernel) are straightforward to obtain - see App. 8.2. This demonstrates the intimate connection between the graph-diffusion and Laplacian kernels, and how a QMC scheme that improves the convergence of  $\mathbf{K}_{\mathrm{lap}}^{(2)}$  will be of broad interest.
2.2 Graph random features (GRFs)
Here we recall the GRF mechanism, which offers a rich playground for our novel antithetic termination QMC scheme. The reader should consult Choromanski [2023] (especially Algorithm 1.1) for a full discussion, but for convenience we provide a cursory summary.
Suppose we would like to estimate the matrix  $(\mathbf{I}_N - \mathbf{U})^{-2}$ , with  $\mathbf{U} \in \mathbb{R}^{N \times N}$  a weighted adjacency matrix of a graph with  $N$  nodes and no loops. Choromanski [2023] proposed a novel algorithm to construct a randomised estimator of this matrix. The author uses graph random features (GRFs)  $\phi(i) \in \mathbb{R}^N$ , with  $i$  the index of one of the nodes, designed such that
$$
\left(\mathbf {I} _ {N} - \mathbf {U}\right) _ {i j} ^ {- 2} = \mathbb {E} \left(\phi (i) ^ {\top} \phi (j)\right). \tag {7}
$$
They construct  $\phi(i)$  by taking  $m$  random walks  $\{\bar{\Omega}(k,i)\}_{k=1}^{m}$  on the graph out of node  $i$ , depositing a 'load' at every node that depends on  $i$ ) the product of edge weights traversed by the subwalk and ii) the marginal probability of the subwalk. Importantly, each walk terminates with probability  $p$  at every timestep. The  $x$ th component of the  $i$ th random feature is given by
$$
\phi (i) _ {x} := \frac {1}{m} \sum_ {k = 1} ^ {m} \sum_ {\omega_ {1} \in \Omega_ {i x}} \frac {\widetilde {\omega} (\omega_ {1})}{p (\omega_ {1})} \mathbb {I} (\omega_ {1} \in \bar {\Omega} (k, i)), \tag {8}
$$
where:  $k$  enumerates  $m$  random walks we sample out of node  $i$ ;  $\omega_{1}$  denotes a particular walk from the set of all walks  $\Omega_{ix}$  between nodes  $i$  and  $x$ ;  $\widetilde{\omega}(\omega_{1})$  is the product of weights of the edges traversed by the walk  $\omega_{1}$ ;  $p(\omega_{1})$  is the marginal probability that a walk contains a prefix subwalk  $\omega_{1}$ , given by  $p(\omega_{1}) = ((1 - p)/d)^{\mathrm{len}(\omega_{1})}$  in the simplest case of a walk of length  $\mathrm{len}(\omega_{1})$  on a  $d$ -regular graph;  $\mathbb{I}(\omega_{1} \in \bar{\Omega}(k, i))$  is an indicator function that evaluates to 1 when the walk  $\omega_{1}$  is a prefix subwalk of the  $k$ th random walk sampled from  $i$  (itself denoted  $\bar{\Omega}(k, i)$ ) and is 0 otherwise.
It is simple to see how Eq. 8 satisfies Eq. 7. By construction
$$
\mathbb {E} \left[ \mathbb {I} \left(\omega_ {1} \in \bar {\Omega} (k, i)\right) \mathbb {I} \left(\omega_ {2} \in \bar {\Omega} (l, j)\right) \right] = p \left(\omega_ {1}\right) p \left(\omega_ {2}\right) \tag {9}
$$
for independent walks, whereupon
$$
\mathbb {E} \left(\phi (i) ^ {\top} \phi (j)\right) = \sum_ {x \in \mathcal {N}} \sum_ {\omega_ {1} \in \Omega_ {i x}} \sum_ {\omega_ {2} \in \Omega_ {j x}} \widetilde {\omega} (\omega_ {1}) \widetilde {\omega} (\omega_ {2}) = \sum_ {\omega \in \Omega_ {i j}} (\operatorname {l e n} (\omega) + 1) \widetilde {\omega} (\omega) = (\mathbf {I} - \mathbf {U}) _ {i j} ^ {- 2}. \tag {10}
$$
This shows us that the estimator is unbiased. The central contribution of this work is a QMC scheme that induces correlations between the  $m$  walks out of each node to suppress the variance of the estimator  $\phi (i)^{\top}\phi (j)$  without breaking this unbiasedness.
3 q-GRFs and antithetic termination
We will now present our novel antithetic termination mechanism. It generalises the notion of antithetic variates – a common, computationally cheap variance-reduction technique when sampling in Euclidean space [Hammersley and Morton, 1956] – to the termination behaviour of random walks.
We have seen that, in the i.i.d. implementation of the GRF algorithm, each walker terminates independently with probability  $p$  at every timestep. For a pair of i.i.d. walkers out of node  $i$ , this is implemented by independently sampling two termination random variables (TRVs) between 0 and 1 from a uniform distribution,  $t_{1,2} \sim \mathrm{Unif}(0,1)$ . Each walker terminates if its respective TRV is less than  $p$ ,  $t_{1,2} < p$ . In contrast, we define the antithetic walker as follows.
Definition 3.1 (Antithetic walkers). We refer to a pair of walkers as antithetic if their TRVs are marginally distributed as  $t_{1,2} = \mathrm{Unif}(0,1)$  but are offset by  $\frac{1}{2}$ ,
$$
t _ {2} = \operatorname {m o d} _ {1} \left(t _ {1} + \frac {1}{2}\right), \tag {11}
$$
(b)
Figure 1: Left: schematic of the i.i.d. (GRF) and antithetic (q-GRF) mechanisms in termination space.  $f(t)$  is the probability density of the termination random variable (TRV)  $t$ . Vertical arrows represent draws of  $t$ , with the walker terminating if they lie in the pink region where  $t < p$ . With q-GRFs the TRVs are offset by  $\frac{1}{2}$ , modifying the joint distribution over walk lengths. Right: demonstration with 4 random walks on the karate graph, beginning at some node labelled  $i$ . The blue pair of antithetic walks (q-GRFs) have very different lengths; they cannot terminate simultaneously. The red pair of i.i.d. walks (GRFs) have similar lengths. We prove that q-GRFs give lower variance estimators of the 2-regularised Laplacian kernel.
such that we have the conditional probability density
$$
p \left(t _ {2} \mid t _ {1}\right) = \delta \left(\operatorname {m o d} _ {1} \left(t _ {2} - t _ {1}\right) - \frac {1}{2}\right). \tag {12}
$$
In both schemes, the TRVs are resampled at every timestep until the corresponding walker terminates.
Computational cost: We note that the computational cost of generating an antithetic TRV according to Eq. 11 is no greater than the cost of generating an independent TRV, so this drop-in replacement in the GRF algorithm is cheap. We provide a schematic in Fig. 1a.
Since the marginal distributions over  $t_i$  are unchanged our estimator remains unbiased, but the couplings between TRVs lead to statistical correlations between the walkers' terminations. Denoting by  $s_1$  the event that walker 1 terminates at some timestep,  $s_2$  the event that walker 2 terminates and  $\bar{s}_{1,2}$  their complements, it is straightforward to convince oneself that for  $p \leq \frac{1}{2}$
$$
p (s _ {1}) = p (s _ {2}) = p, \quad p (\bar {s} _ {1}) = p (\bar {s} _ {2}) = 1 - p, \quad p (s _ {2} | s _ {1}) = 0,
$$
$$
p \left(\bar {s} _ {2} \mid s _ {1}\right) = 1, \quad p \left(s _ {2} \mid \bar {s} _ {1}\right) = \frac {p}{1 - p}, \quad p \left(\bar {s} _ {2} \mid \bar {s} _ {1}\right) = \frac {1 - 2 p}{1 - p}. \tag {13}
$$
This termination coupling modifies the joint probability distribution over walk lengths. In the i.i.d. scheme, the walks are independent and are of expected length
$$
\mathbb {E} (\operatorname {l e n} (\omega)) = \frac {1 - p}{p}. \tag {14}
$$
These marginal expectations are preserved in the antithetic scheme, but now the expected length of one walk conditioned on the length of the other is
$$
\mathbb {E} \left(\operatorname {l e n} \left(\omega_ {2}\right) \mid \operatorname {l e n} \left(\omega_ {1}\right) = m\right) = \frac {1 - 2 p}{p} + 2 \left(\frac {1 - 2 p}{1 - p}\right) ^ {m}, \tag {15}
$$
which we derive in App. 8.3. It is straightforward to see that the two lengths are negatively correlated. Antithetic termination 'diversifies' the lengths of random walks we sample, preventing them from clustering together, and in the spirit of QMC this turns out to suppress the kernel estimator variance. See Fig. 1b for a schematic. We refer to random features constructed with antithetic walkers as quasi-Monte Carlo graph random features (q-GRFs).
Though this paper focuses on antithetic termination for GRFs, it is a more general QMC scheme applicable in algorithms that sample walks with geometrically distributed lengths. To illustrate this, we again direct the reader to App. 8.1 where we show how it can be used to improve numerical estimates of the PageRank vector [Page et al., 1998].
3.1 Theoretical results
In this section, we state and discuss our central theoretical results for the q-GRFs mechanism. Sec. 3.1.1 provides a sketch, but full proofs are deferred to App. 8.4. We remind the reader that results for  $(\mathbf{I}_N - \mathbf{U})^{-2}$  are trivially applied to  $\mathbf{K}_{\mathrm{lap}}^{(2)}$  (see App. 8.2).
Theorem 3.2 (Antithetic termination is better than i.i.d.). For any graph,  $q$ -GRFs will give lower variance on estimators of  $(\mathbf{I}_N - \mathbf{U})^{-2}$  than regular GRFs provided either i) the termination probability  $p$  is sufficiently small or ii) the spectral radius  $\rho(\mathbf{U})$  is sufficiently small.
By 'sufficiently small' we mean that for a fixed  $\rho(\mathbf{U})$  there exists some value of  $p$  below which antithetic termination will outperform i.i.d.. Likewise, for fixed  $p$  there exists some value of  $\rho(\mathbf{U})$ . These conditions turn out to not be too restrictive in our experiments; antithetic termination is actually very effective at  $p = 0.5$  which we use for practical applications.
Considering Eq. 11 carefully, it is easy to see that the termination probabilities in Eq. 13 are not particular to a TRV offset equal to  $\frac{1}{2}$ , and in fact hold for any offset  $\Delta$  satisfying  $p \leq \Delta \leq 1 - p$ . An immediate corollary is as follows.
Corollary 3.3 (Maximum size of an antithetic ensemble). For a termination probability  $p$ , up to  $[p^{-1}]$  random walkers can all be conditioned to exhibit mutually antithetic termination.
This is achieved by offsetting their respective TRVs by  $\Delta = p$ . The resulting antithetic ensemble will have lower kernel estimator variance than the equivalent number of i.i.d. walkers or a set of mutually independent antithetic pairs. We make one further interesting remark.
Theorem 3.4 (Termination correlations beyond antithetic). A pair of random walkers with TRVs offset by  $p(1 - p) < \Delta < p$  will exhibit lower variance on estimators of  $(\mathbf{I}_N - \mathbf{U})^{-2}$  than independent walkers, provided either i) the termination probability  $p$  is sufficiently small or ii) the spectral radius of the weighted adjacency matrix  $\rho(\mathbf{U})$  is sufficiently small.
This provides an upper limit of  $\lfloor (p(1 - p))^{-1}\rfloor$  on the number of walkers we can simultaneously correlate before we can no longer guarantee that coupling in a further walker's TRV will be better than sampling it independently. Intuitively, Theorem 3.4 tells us that we can space TRVs even more closely than  $p$ , allowing us to increase the number of simultaneously anticorrelated random walkers at the cost of the strength of negative correlations between walkers with neighbouring TRVs.
3.1.1 Proof sketch