SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Abhin Shah, Wei-Ning Chen, Johannes Balle, Peter Kairouz, and Lucas Theis. Optimal compression of locally differentially private mechanisms. In International Conference on Artificial Intelligence and Statistics, pp. 7680-7723. PMLR, 2022.
Lucas Theis and Noureldin Yosri. Algorithms for the communication of samples. In International Conference on Machine Learning, 2022.
Lucas Theis, Tim Salimans, Matthew D. Hoffman, and Fabian Mentzer. Lossy compression with Gaussian diffusion. arXiv:2206.08889, 2022.
Jacob Ziv. On universal quantization. IEEE Transactions on Information Theory, 31(3):344-347, 1985.
A Measure-theoretic Construction of Greedy Poisson Rejection Sampling
In this section, we provide a construction of greedy Poisson rejection sampling (GPRS) in its greatest generality. However, we first establish the notation for the rest of the appendix.
Notation: In what follows, we will always denote GPRS's target distribution as  $Q$  and its proposal distribution as  $P$ . We assume that  $Q$  and  $P$  are Borel probability measures over some Polish space  $\Omega$  with Radon-Nikodym derivative  $r \stackrel{\text{def}}{=} dQ / dP$ . We denote the standard Lebesgue measure on  $\mathbb{R}^n$  by  $\lambda$ . All logarithms are assumed to be to the base 2 denoted by  $\log_2$ . Similarly, we use the less common notation of  $\exp_2$  to denote exponentiation with base 2, i.e.  $\exp_2(x) = 2^x$ . The relative entropy of  $Q$  from  $P$  is defined as  $D_{\mathrm{KL}}[Q \| P] \stackrel{\text{def}}{=} \int_{\Omega} \log_2 r(x) dP(x)$  and the Rényi -divergence as  $D_{\infty}[Q \| P] \stackrel{\text{def}}{=} \operatorname*{ess}\sup_{x \in \Omega} \{ \log r(x) \}$ , where the essential supremum is taken with respect to  $P$ .
Restricting a Poisson process "under the graph of a function": We first consider the class of functions "under which" we will be restricting our Poisson processes. Let  $\varphi : \Omega \to \mathbb{R}^+$  be a measurable function, such that  $|\varphi(\Omega)| = |r(\Omega)|$ , i.e. the image of  $\varphi$  has the same cardinality as the image of  $r = dQ / dP$ . This implies that there exist bijections between  $r(\Omega)$  and  $\varphi(\Omega)$ . Since both images are subsets of  $\mathbb{R}^+$ , both images have a linear order. Hence, a monotonically increasing bijection  $\tilde{\sigma}$  exists such that  $\varphi = \tilde{\sigma} \circ r$ . Since  $\tilde{\sigma}$  is monotonically increasing, we can extend it to a monotonically increasing continuous function  $\sigma : \mathbb{R}^+ \to \mathbb{R}^+$ . This fact is significant, as it allows us to reduce questions about functions of interest from  $\Omega$  to  $\mathbb{R}^+$  to invertible functions on the positive reals. Since  $\sigma$  is continuous and monotonically increasing on the positive reals, it is invertible on the extended domain. We now consider restricting a Poisson process under the graph of  $\varphi$ , and work out the distribution of the spatial coordinate of the first arrival.
Let  $\Pi = \{(T_n,X_n)\}_{n = 1}^{\infty}$  be a Poisson process over  $\mathbb{R}^{+}\times \Omega$  with mean measure  $\mu = \lambda \times P$ . Let  $U\stackrel {def}{=}\{(t,x)\in \Omega \mid t\leq \varphi (x)\}$  be the set of points "under the graph" of  $\varphi$ . By the restriction theorem (Kingman, 1992),  $\tilde{\Pi}\stackrel {def}{=}\Pi \cap U$  is a Poisson process with mean measure  $\tilde{\mu} (A) = \mu (A\cap U)$  for an arbitrary Borel set  $A$ . As a slight abuse of notation, we define the shorthand  $\tilde{\mu} (t) = \tilde{\mu} ([0,t)\times \Omega)$  for the average number of points in  $\tilde{\Pi}$  up to time  $t$ . Let  $\mathbf{N}_{\tilde{\Pi}}$  denote the counting measure of  $\tilde{\Pi}$  and let  $(\tilde{T},\tilde{X})$  be the first arrival of  $\tilde{\Pi}$ , i.e. the first point of the original process  $\Pi$  that falls under the graph of  $\varphi$ , assuming that it exists. To develop GPRS, we first work out the distribution of  $\tilde{X}$  for an arbitrary  $\varphi$ .
Remark: Note, that the construction below only holds if the first arrival is guaranteed to exist, otherwise the quantities below do not make sense. From a formal point of view, we would have to always condition on the event  $\mathbf{N}_{\tilde{\Pi}}(\infty) \stackrel{\text{def}}{=} \lim_{t \to \infty} \mathbf{N}_{\tilde{\Pi}}(t) > 0$ , e.g. we would need to write  $\mathbb{P}[\tilde{T} \geq t \mid \mathbf{N}_{\tilde{\Pi}}(\infty) > 0]$ . However, we make a "leap of faith" instead and assume that our construction is valid to obtain our "guesses" for the functions  $\sigma$  and  $\varphi$ . Then, we show that for our specific guesses the derivation below is well-defined, i.e. namely that  $\mathbb{P}[\mathbf{N}_{\tilde{\Pi}}(\infty) > 0] = 1$ , which then implies  $\mathbb{P}[\tilde{T} \geq t \mid \mathbf{N}_{\tilde{\Pi}}(\infty) > 0] = \mathbb{P}[\tilde{T} \geq t]$ . To do this, note that
$$
\begin{array}{l} \mathbb {P} \left[ \mathbf {N} _ {\tilde {\Pi}} (\infty) > 0 \right] = 1 - \mathbb {P} \left[ \mathbf {N} _ {\tilde {\Pi}} (\infty) = 0 \right] (13) \\ = 1 - \lim  _ {t \rightarrow \infty} e ^ {- \tilde {\mu} (t)}. (14) \\ \end{array}
$$
Hence, for the well-definition of our construction it is enough to show that for our choice of  $\sigma$ ,  $\tilde{\mu}(t) \to \infty$  as  $t \to \infty$ .
Constructing  $\sigma$ : We wish to derive
$$
\frac {d \mathbb {P} [ \tilde {X} = x ]}{d P} = \int_ {0} ^ {\infty} \frac {d \mathbb {P} [ \tilde {T} = t , \tilde {X} = x ]}{d (\lambda \times P)} d t. \tag {15}
$$
To do this, recall that we defined  $\mathbf{N}_{\tilde{\Pi}}(t)\stackrel {def}{=}\mathbf{N}_{\tilde{\Pi}}([0,t)\times \Omega)$  and define  $\mathbf{N}_{\tilde{\Pi}}(s,t)\stackrel {def}{=}\mathbf{N}_{\tilde{\Pi}}([s,t]\times \Omega)$ . Similarly, let  $\tilde{\mu} (t)\stackrel {def}{=}\mathbb{E}[\mathbf{N}_{\tilde{\Pi}}(t)]$  and  $\tilde{\mu} (s,t)\stackrel {def}{=}\mathbb{E}[\mathbf{N}_{\tilde{\Pi}}(s,t)] = \tilde{\mu} (t) - \tilde{\mu} (s)$ . Then,
$$
\begin{array}{l} \mathbb {P} [ \tilde {X} \in A, \tilde {T} \in d t ] = \lim  _ {s \rightarrow t} \frac {\mathbb {P} [ \tilde {X} \in A , \tilde {T} \in [ s , t ] ]}{t - s} (16) \\ = \lim  _ {s \rightarrow t} \frac {\mathbb {P} [ \tilde {X} \in A , \mathbf {N} _ {\tilde {\Pi}} (s , t) = 1 , \mathbf {N} _ {\tilde {\Pi}} (s) = 0 ]}{t - s} (17) \\ = \lim  _ {s \rightarrow t} \frac {\mathbb {P} [ \tilde {X} \in A \mid \mathbf {N} _ {\tilde {\Pi}} (s , t) = 1 ] \mathbb {P} [ \mathbf {N} _ {\tilde {\Pi}} (s , t) = 1 ] \mathbb {P} [ \mathbf {N} _ {\tilde {\Pi}} (s) = 0 ]}{t - s}, (18) \\ \end{array}
$$
where the last equality holds, because  $\mathbf{N}_{\bar{\Pi}}(s) \perp \mathbf{N}_{\bar{\Pi}}(s,t)$ , since  $[0,s) \times \Omega$  and  $[s,t] \times \Omega$  are disjoint. By Lemma 3 from Maddison (2016),
$$
\begin{array}{l} \mathbb {P} [ \tilde {X} \in A \mid \mathbf {N} _ {\tilde {\Pi}} (s, t) = 1 ] = \mathbb {P} [ (\tilde {T}, \tilde {X}) \in [ s, t ] \times A \mid \mathbf {N} _ {\tilde {\Pi}} (s, t) = 1 ] (19) \\ = \frac {\tilde {\mu} ([ s , t ] \times A)}{\tilde {\mu} (s , t)}. (20) \\ \end{array}
$$
Substituting this back into Equation (18) and using the fact that  $\mathbf{N}_{\bar{\Pi}}$  is Poisson, we find
$$
\begin{array}{l} \mathbb {P} [ \tilde {X} \in A, \tilde {T} \in d t ] = \lim  _ {s \rightarrow t} \left(\frac {\tilde {\mu} ([ s , t ] \times A)}{\tilde {\mu} (s , t)} \cdot \tilde {\mu} (s, t) e ^ {- \tilde {\mu} (s, t)} \cdot e ^ {- \tilde {\mu} (s)}\right) / (t - s) (21) \\ = \lim  _ {s \rightarrow t} \left(\tilde {\mu} ([ s, t ] \times A) \cdot e ^ {- (\tilde {\mu} (t) - \tilde {\mu} (s))} \cdot e ^ {- \tilde {\mu} (s)}\right) / (t - s) (22) \\ = e ^ {- \tilde {\mu} (t)} \cdot \lim  _ {s \rightarrow t} \frac {\tilde {\mu} ([ s , t ] \times A)}{t - s} (23) \\ = e ^ {- \bar {\mu} (t)} \cdot \int_ {A} \mathbf {1} [ t \leq \varphi (x) ] d P (x), (24) \\ \end{array}
$$
where the last equation holds by the definition of the derivative and the fundamental theorem of calculus. From this, by inspection we find
$$
\frac {d \mathbb {P} [ \tilde {T} = t , \tilde {X} = x ]}{d (\lambda \times P)} = \mathbf {1} [ t \leq \varphi (x) ] e ^ {- \tilde {\mu} (t)} = \mathbf {1} [ t \leq \varphi (x) ] \mathbb {P} [ \tilde {T} \geq t ]. \tag {25}
$$
Finally, by marginalizing out the first arrival time, we find that the spatial distribution is
$$
\begin{array}{l} \frac {d \mathbb {P} [ \tilde {X} = x ]}{d P} = \int_ {0} ^ {\infty} \frac {d \mathbb {P} [ \tilde {T} = t , \tilde {X} = x ]}{d (\lambda \times P)} d t (26) \\ = \int_ {0} ^ {\varphi (x)} \mathbb {P} [ \tilde {T} \geq t ] d t. (27) \\ \end{array}
$$
Deriving  $\varphi$  by finding  $\sigma$ : Note that Equation (27) holds for any  $\varphi$ . However, to get a correct algorithm, we need to set it such that  $\frac{d\mathbb{P}[\tilde{X} = x]}{dP} = \frac{dQ}{dP} = r$ . Since we can write  $\varphi = \sigma \circ r$  by our earlier argument, this problem is reduced to finding an appropriate invertible continuous function  $\sigma: \mathbb{R}^{+} \to \mathbb{R}^{+}$ . Thus, we wish to find  $\sigma$  such that
$$
\forall x \in \Omega , \quad r (x) = \int_ {0} ^ {\sigma (r (x))} \mathbb {P} [ \tilde {T} \geq t ] d t. \tag {28}
$$
Now, introduce  $\tau = \sigma(r(x))$ , so that  $\sigma^{-1}(\tau) = r(x)$ . Note that this substitution only makes sense for  $\tau \in r(\Omega)$ . However, since we are free to extend  $\sigma^{-1}$  to  $\mathbb{R}^+$  in any we like so long as it is monotone, we may require that this equation hold for all  $\tau \in \mathbb{R}^+$ . Then, we find
$$
\begin{array}{l} \sigma^ {- 1} (\tau) = \int_ {0} ^ {\tau} \mathbb {P} [ \tilde {T} \geq t ] d t (29) \\ \Rightarrow \left(\sigma^ {- 1}\right) ^ {\prime} (\tau) = \mathbb {P} [ \tilde {T} \geq \tau ] (30) \\ \end{array}
$$
with  $\sigma^{-1}(0) = 0$ . Before we solve for  $\sigma$ , we define
$$
w _ {P} (h) \stackrel {\text {d e f}} {=} \mathbb {P} _ {Y \sim P} [ r (Y) \geq h ] \tag {31}
$$
$$
w _ {Q} (h) \stackrel {\text {d e f}} {=} \mathbb {P} _ {Y \sim Q} [ r (Y) \geq h ]. \tag {32}
$$
Note that  $w_{P}$  and  $w_{Q}$  are supported on  $[0,r^{*})$ , where  $r^* \stackrel{\text{def}}{=} \operatorname{ess}\sup_{x \in \Omega} \{r(x)\} = \exp_2(D_{\infty}[Q \| P])$ . Now, we can rewrite Equation (30) as
$$
\left(\sigma^ {- 1}\right) ^ {\prime} (t) = \mathbb {P} [ \tilde {T} \geq t ] \tag {33}
$$
$$
= \mathbb {P} [ \tilde {T} \geq t, \varphi (\tilde {X}) \geq t ] + \underbrace {\mathbb {P} [ \tilde {T} \geq t , \varphi (\tilde {X}) <   t ]} _ {= 0 \text {d u e t o m u t u a l e x c l u s i v i t y}} \tag {34}
$$
$$
= \underbrace {\mathbb {P} [ \tilde {T} \geq 0 , \varphi (\tilde {X}) \geq t ]} _ {= \mathbb {P} [ \varphi (\tilde {X}) \geq t ], \text {s i n c e} \tilde {T} \geq 0 \text {a l w a y s}} - \mathbb {P} [ \varphi (\tilde {X}) \geq t \geq \tilde {T} ] \tag {35}
$$
$$
= \mathbb {P} [ r (\tilde {X}) \geq \sigma^ {- 1} (t) ] - \mathbb {P} [ r (\tilde {X}) \geq \sigma^ {- 1} (t) \geq \sigma^ {- 1} (\tilde {T}) ] \tag {36}
$$
$$
= w _ {Q} \left(\sigma^ {- 1} (t)\right) - \sigma^ {- 1} (t) w _ {P} \left(\sigma^ {- 1} (t)\right) \tag {37}
$$
where the second term in the last equation follows by noting that
$$
\begin{array}{l} \mathbb {P} [ r (\tilde {X}) \geq \sigma^ {- 1} (t) \geq \sigma^ {- 1} (\tilde {T}) ] = \int_ {\Omega} \int_ {0} ^ {t} \underbrace {\mathbf {1} [ r (x) \geq \sigma^ {- 1} (t) ] \mathbf {1} [ r (x) \geq \sigma^ {- 1} (\tau) ]} _ {= \mathbf {1} [ r (x) \geq \sigma^ {- 1} (t) ], \text {s i n c e} \tau <   t} \mathbb {P} [ \tilde {T} \geq \tau ] d \tau d P (x) (38) \\ = \int_ {\Omega} \mathbf {1} [ r (x) \geq \sigma^ {- 1} (t) ] \underbrace {\int_ {0} ^ {t} \mathbb {P} [ \tilde {T} \geq \tau ] d \tau} _ {= \sigma^ {- 1} (t), \text {b y e q . (2 9)}} d P (x) (39) \\ = \sigma^ {- 1} (t) w _ {P} \left(\sigma^ {- 1} (t)\right). (40) \\ \end{array}
$$
Now, by the inverse function theorem, we get
$$
\sigma^ {\prime} (h) = \frac {1}{w _ {Q} (h) - h \cdot w _ {P} (h)}, \tag {41}
$$
thus we finally find
$$
\sigma (h) = \int_ {0} ^ {h} \frac {1}{w _ {Q} (\eta) - \eta \cdot w _ {P} (\eta)} d \eta . \tag {42}
$$
Thus, to recapitulate, setting  $\varphi = \sigma \circ r$ , where  $\sigma$  is given by Equation (42) will ensure that the spatial distribution of the first arrival of  $\Pi$  under  $\varphi$  is the target distribution  $Q$ .
An alternate form for  $\mathbb{P}[\tilde{T}\geq t]$ : We make use of the following integral representation of the indicator function to present an alternative form of the survival function of  $\tilde{T}$ :
$$
\text {f o r} a > 0: \quad \mathbf {1} [ a \leq x ] = \int_ {0} ^ {x} \delta (y - a) d y, \tag {43}
$$
where  $\delta$  is the Dirac delta function. Then, we have
$$
\begin{array}{l} w _ {Q} (h) = \int_ {\Omega} \mathbf {1} [ r (x) \geq h ] d Q (x) (44) \\ = 1 - \int_ {\Omega} \int_ {0} ^ {h} r (x) \delta (y - r (x)) d y d P (x) (45) \\ = 1 - \int_ {0} ^ {h} y \int_ {\Omega} \delta (y - r (x)) d P (x) d y (46) \\ = 1 - \left[ y \left(1 - w _ {P} (y)\right) \right] _ {0} ^ {h} + \int_ {0} ^ {h} \left(1 - w _ {P} (y)\right) d y (47) \\ = 1 + h \cdot w _ {P} (h) - \int_ {0} ^ {h} w _ {P} (y) d y. (48) \\ \end{array}
$$
Then, from this identity, we find that
$$
\begin{array}{l} \mathbb {P} [ \tilde {T} \geq t ] = w _ {Q} \left(\sigma^ {- 1} (t)\right) - \sigma^ {- 1} (t) w _ {P} \left(\sigma^ {- 1} (t)\right) (49) \\ = 1 - \int_ {0} ^ {\sigma^ {- 1} (t)} w _ {P} (y) d y. (50) \\ \end{array}
$$
Well-definition of GPRS and Algorithm 3 terminates with probability 1: To ensure that our construction is useful, we need to show that the first arrival under the graph  $\tilde{T}$  exists and is almost surely finite, so that Algorithm 3 terminates with probability 1. First, note that for any  $t > 0$ , we have  $\tilde{\mu}(t) \leq \mu(t) = t < \infty$ . Since  $\mathbb{P}[\mathbf{N}_{\tilde{\Pi}}(t) = 0] = e^{-\tilde{\mu}(t)}$ , this implies that for any finite time  $t$ ,  $\mathbb{P}[\mathbf{N}_{\tilde{\Pi}}(t) = 0] > 0$ . Second, note that for  $h \in [0, r^*]$ ,
$$
w _ {Q} (h) - h \cdot w _ {P} (h) = \int_ {\Omega} \mathbf {1} [ r (x) \geq h ] (r (x) - h) d P (x), \tag {51}
$$
from which
$$
\begin{array}{l} w _ {Q} \left(r ^ {*}\right) - r ^ {*} w _ {P} \left(r ^ {*}\right) = \int_ {\Omega} \mathbf {1} [ r (x) \geq r ^ {*} ] \left(r (x) - r ^ {*}\right) d P (x) (52) \\ = \int_ {\Omega} \mathbf {1} [ r (x) = r ^ {*} ] \left(r (x) - r ^ {*}\right) d P (x) (53) \\ = 0. (54) \\ \end{array}
$$
Thus, in particular, we find that
$$
\lim  _ {h \rightarrow r ^ {*}} e ^ {- \tilde {\mu} (\sigma (h))} = \lim  _ {h \rightarrow r ^ {*}} \mathbb {P} [ \mathbf {N} _ {\tilde {\Pi}} (\sigma (h)) = 0 ] \stackrel {{\text {e q .} (3 7)}} {{=}} \lim  _ {h \rightarrow r ^ {*}} \left(w _ {Q} (h) - h \cdot w _ {P} (h)\right) = 0. \tag {55}
$$
By continuity, this can only hold if  $\tilde{\mu} (\sigma (h))\to \infty$  as  $h\rightarrow r^*$ . Since  $\tilde{\mu} (t)$  is finite for all  $t > 0$  and  $\sigma$  is monotonically increasing, this also implies that  $\sigma (h)\to \infty$  as  $h\to r^{*}$ . Thus, we have shown a couple of facts:
-  $\sigma(h) \to \infty$  as  $h \to r^*$ , hence  $\varphi$  is always unbounded at points in  $\Omega$  that achieve the supremum  $r^*$ . Furthermore, this implies that
$$
\sigma^ {- 1} (t) \rightarrow r ^ {*} \quad \text {a s} \quad t \rightarrow \infty . \tag {56}
$$
Since  $\sigma^{-1}$  is increasing, this implies that it is bounded from above by  $r^*$ .
-  $\tilde{\mu}(t) < \infty$  for all  $t > 0$ , but  $\tilde{\mu}(t) \to \infty$  as  $t \to \infty$ . Hence, by Equation (14) we have  $\mathbb{P}[\mathbf{N}_{\tilde{\Pi}}(\infty) > 0] = 1$ . Thus, the first arrival of  $\tilde{\Pi}$  exists almost surely, and our construction is well-defined. In particular, it is meaningful to write  $\mathbb{P}[\tilde{T} \geq t]$ .
-  $\mathbb{P}[\tilde{T}\geq t] = \mathbb{P}[\mathbf{N}_{\tilde{\Pi}}(t) = 0]\to 0$  as  $t\to \infty$  , which shows that the first arrival time is finite with probability 1. In turn, this implies that Algorithm 3 will terminate with probability one, as desired.
Note, that  $w_{P}$  and  $w_{Q}$  can be computed in many practically relevant cases, see Appendix G.
B Analysis of Greedy Poisson Rejection Sampling
Now that we constructed a correct sampling algorithm in Appendix A, we turn our attention to deriving the expected first arrival time  $\tilde{T}$  in the restricted process  $\tilde{\Pi}$ , the expected number of samples  $N$  before Algorithm 3 terminates and bound on  $N$ 's variance, and an upper bound on its coding cost. The proofs below showcase the true advantage of formulating the sampling algorithm using the language of Poisson processes: the proofs are all quite short and elegant.
B.1 The Expected First Arrival Time
At the end of the previous section, we showed that  $\tilde{T}$  is finite with probability one, which implies that  $\mathbb{E}[\tilde{T}] < \infty$ . Now, we derive the value of this expectation exactly. Since  $\tilde{T}$  is a positive random variable, by the Darth Vader rule (Muldowney et al., 2012), we may write its expectation as
$$
\mathbb {E} \left[ \tilde {T} \right] = \int_ {0} ^ {\infty} \mathbb {P} [ \tilde {T} \geq t ] d t \stackrel {{\text {e q .} (2 9)}} {{=}} \lim  _ {t \rightarrow \infty} \sigma^ {- 1} (t) \stackrel {{\text {e q .} (5 6)}} {{=}} r ^ {*}. \tag {57}
$$
B.2 The Expectation and Variance of the Runtime
Expectation of  $N$ : First, let  $\tilde{\Pi}^C \stackrel{def}{=} \Pi \setminus \tilde{\Pi}$  be the set of points in  $\Pi$  above the graph of  $\varphi$ . Since  $\tilde{\Pi}$  and  $\tilde{\Pi}^C$  are defined on complementary sets, they are independent. Since by definition  $\tilde{T}$  is the first arrival of  $\tilde{\Pi}$  and the  $N$ th arrival of  $\Pi$ , it must mean that the first  $N - 1$  arrivals of  $\Pi$  occurred in  $\tilde{\Pi}^C$ . Thus, conditioned on  $\tilde{T}$ , we have  $N - 1 = |\{(T,X) \in \tilde{\Pi}^C \mid T < \tilde{T}\}|$  is Poisson distributed with mean
$$
\begin{array}{l} \mathbb {E} [ N - 1 \mid \tilde {T} ] = \int_ {0} ^ {\tilde {T}} \int_ {\Omega} \mathbf {1} [ t \geq \varphi (x) ] d P (x) d t (58) \\ = \int_ {0} ^ {\tilde {T}} \int_ {\Omega} 1 - \mathbf {1} [ t <   \varphi (x) ] d P (x) d t (59) \\ = \tilde {T} - \tilde {\mu} (\tilde {T}). (60) \\ \end{array}
$$
By the law of iterated expectations,
$$
\begin{array}{l} \mathbb {E} [ N - 1 ] = \mathbb {E} _ {\tilde {T}} \left[ \mathbb {E} \left[ N - 1 \mid \tilde {T} \right] \right] (61) \\ = \mathbb {E} \left[ \tilde {T} \right] - \mathbb {E} \left[ \tilde {\mu} (\tilde {T}) \right]. (62) \\ \end{array}
$$
Focusing on the second term, we find
$$
\begin{array}{l} \mathbb {E} \left[ \tilde {\mu} (\tilde {T}) \right] = \int_ {0} ^ {\infty} \tilde {\mu} (t) \cdot \mathbb {P} [ \tilde {T} \in d t ] d t (63) \\ = \int_ {0} ^ {\infty} \tilde {\mu} (t) \cdot \tilde {\mu} ^ {\prime} (t) e ^ {- \tilde {\mu} (t)} d t (64) \\ = \int_ {0} ^ {\infty} u e ^ {- u} d u = 1, (65) \\ \end{array}
$$
where the third equality follows by substituting  $u = \tilde{\mu}(t)$ . Finally, plugging the above and Equation (57) into Equation (62), we find
$$
\mathbb {E} [ N ] = 1 + \mathbb {E} [ N - 1 ] = 1 + r ^ {*} - 1 = r ^ {*}. \tag {66}
$$
Variance of  $N$ : We now show that the distribution of  $N$  is super-Poissonian, i.e.  $\mathbb{E}[N] \leq \mathbb{V}[N]$ . Similarly to the above, we begin with the law of iterated variances to find
$$
\begin{array}{l} \mathbb {V} [ N ] = \mathbb {V} [ N - 1 ] = \mathbb {E} _ {\tilde {T}} [ \mathbb {V} [ N - 1 \mid \tilde {T} ] ] + \mathbb {V} _ {\tilde {T}} [ \mathbb {E} [ N - 1 \mid \tilde {T} ] ] (67) \\ = \mathbb {E} \left[ \tilde {T} \right] - \mathbb {E} \left[ \tilde {\mu} (\tilde {T}) \right] + \mathbb {V} [ \tilde {T} ] + \mathbb {V} [ \tilde {\mu} (\tilde {T}) ], (68) \\ \end{array}
$$
where the second equality follows from the fact that the variance of  $N - 1$  matches its mean conditioned on  $\tilde{T}$ , since it is a Poisson random variable. Focussing on the last term, we find
$$
\begin{array}{l} \mathbb {V} \left[ \tilde {\mu} (\tilde {T}) \right] = \mathbb {E} \left[ \tilde {\mu} (\tilde {T}) ^ {2} \right] - \mathbb {E} \left[ \tilde {\mu} (\tilde {T}) \right] ^ {2} (69) \\ = \int_ {0} ^ {\infty} \tilde {\mu} (t) ^ {2} \cdot \mathbb {P} [ \tilde {T} \in d t ] d t - 1 (70) \\ = \int_ {0} ^ {\infty} u ^ {2} e ^ {- u} d t - 1 = 1, (71) \\ \end{array}
$$
where the third equality follows from a similar  $u$ -substitution as in Equation (65). Thus, putting everything together, we find
$$
\mathbb {V} [ N ] = r ^ {*} - 1 + \mathbb {V} [ \tilde {T} ] + 1 = r ^ {*} + \mathbb {V} [ \tilde {T} ] > \mathbb {E} [ N ]. \tag {72}
$$
B.3 The Fractional Moments of the Index
In this section, we follow the work of Liu & Verdu (2018) and analyse the fractional moments of the index  $N$  and prove Theorem 3.2. As before, set  $Q$  and  $P$  as GPRS's target and proposal distribution, respectively, and let  $r = \frac{dQ}{dP}$ . To begin, we define the Rényi  $\alpha$ -divergence between two distributions for  $\alpha \in (0,1) \cup (1,\infty)$  as
$$
D _ {\alpha} [ Q \| P ] = \frac {1}{\alpha - 1} \log_ {2} \mathbb {E} _ {Z \sim Q} [ r (Z) ^ {\alpha - 1} ]. \tag {73}
$$
By taking limits, we can extend the definiton to  $\alpha = 1,\infty$  as
$$
D _ {1} [ Q \| P ] \stackrel {\text {d e f}} {=} \lim  _ {\alpha \rightarrow 1} D _ {\alpha} [ Q \| P ] = D _ {\mathrm {K L}} [ Q \| P ] \tag {74}
$$
$$
D _ {\infty} [ Q \| P ] \stackrel {d e f} {=} \lim  _ {\alpha \rightarrow \infty} D _ {\alpha} [ Q \| P ] = \operatorname * {e s s} _ {x \in \Omega} \left\{\log_ {2} r (x) \right\}. \tag {75}
$$
Now fix  $\alpha \in (0,1)$  and  $x\in \Omega$  and note that by Jensen's inequality,
$$
\mathbb {E} [ N ^ {\alpha} \mid \tilde {X} = x ] \leq \mathbb {E} [ N \mid \tilde {X} = x ] ^ {\alpha}. \tag {76}
$$
Now, to continue, we will show the following bound:
$$
\mathbb {E} [ N \mid \tilde {X} = x ] \leq \frac {1}{w _ {P} (r (x))} \tag {77}
$$
Let  $\Pi$  be the  $(1, P)$ -Poisson process realized by Algorithm 3, and let
$$
M = \min  \left\{m \in \mathbb {N} \mid \left(T _ {m}, X _ {m}\right) \in \Pi , r \left(X _ {m}\right) \geq r (x) \right\}, \tag {78}
$$
i.e. the index of the first arrival in  $\Pi$  such that the density ratio of the spatial coordinate exceeds  $r(x)$ . Since  $X_{i}\sim P$  are i.i.d.,
$$
M \sim \operatorname {G e o m} \left(\mathbb {P} _ {Z \sim P} [ r (Z) \geq r (x) ]\right), \tag {79}
$$
$$
\mathbb {E} [ M ] = \frac {1}{\mathbb {P} _ {Z \sim P} [ r (Z) \geq r (x) ]} = \frac {1}{w _ {P} (r (x))}. \tag {80}
$$
Now, since geometrically distributed random variables are memoryless, we find
$$
\mathbb {E} [ M ] = \mathbb {E} [ M - N \mid M > N ] \tag {81}
$$
$$
\leq \mathbb {E} [ M \mid M > N ], \tag {82}
$$
which then implies
$$
\mathbb {E} [ M \mid M \leq N ] \leq \mathbb {E} [ M ]. \tag {83}
$$
Before we proceed, we first require the following result
Lemma B.1. Let all quantities be as defined above and let  $x, y \in \Omega$  such that  $r(x) \leq r(y)$ . Then,  $N \mid \tilde{X} = y$  stochastically dominates  $N \mid \tilde{X} = x$ , i.e. for all positive integers  $n$  we have
$$
\mathbb {P} [ N \geq n \mid \tilde {X} = x ] \leq \mathbb {P} [ N \geq n \mid \tilde {X} = y ]. \tag {84}
$$
Proof. The proof follows via direct calculation. Concretely,
$$
\begin{array}{l} \mathbb {P} [ N \leq n \mid \tilde {X} = x ] = \mathbb {P} [ N - 1 \leq n - 1 \mid \tilde {X} = x ] (85) \\ = \mathbb {E} _ {\tilde {T} \sim P _ {\tilde {T} | \tilde {X} = x}} \left[ \mathbb {P} [ N - 1 \leq n - 1 \mid \tilde {X} = x, \tilde {T} ] \right] (86) \\ = \frac {1}{(n - 1) !} \mathbb {E} _ {\tilde {T} \sim P _ {\tilde {T} | \tilde {X} = x}} \left[ \Gamma (n, \tilde {T} - \tilde {\mu} (\tilde {T})) \right], (87) \\ \end{array}
$$
where  $\Gamma(s, x)$  is the upper incomplete  $\Gamma$ -function defined as
$$
\Gamma (s, x) \stackrel {\text {d e f}} {=} \int_ {x} ^ {\infty} t ^ {s - 1} e ^ {- t} d t. \tag {88}
$$
Equation (87) follows from the fact that conditioned on  $\tilde{T}$  we have that  $N - 1$  is a Poisson random variable with mean  $\tilde{T} -\tilde{\mu} (\tilde{T})$ . Now, focussing on the expectation term in Equation (87), we let