SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof. Equation (199) follows via the same argument as Equation (181), and Equation (200) follows via the same argument as lemma D.3, combined with lemma E.1.  $\square$
Expected runtime: Combining Equation (200) with Equation (143) yields
$$
\mathbb {E} [ D ] \leq \frac {D _ {\mathrm {K L}} [ Q \| P ] + 2 \cdot \log_ {2} e}{\log_ {2} (4 / 3)}, \tag {201}
$$
which proves Theorem 3.6. In the channel simulation case, taking a pair of correlated random variables  $\mathbf{x},\mathbf{y}\sim P_{\mathbf{x},\mathbf{y}}$  and setting  $Q\gets P_{\mathbf{x}|\mathbf{y}}$  and  $P\gets P_{\mathbf{x}}$ , and taking expectation over  $\mathbf{y}$ , we get
$$
\mathbb {E} [ D ] \leq \frac {I [ \mathbf {x} ; \mathbf {y} ] + 2 \cdot \log_ {2} e}{\log_ {2} (4 / 3)}. \tag {202}
$$
Entropy coding the heap index: Let  $\nu \in T_{\mathrm{on - sample}}$  now denote the node accepted and returned by Algorithm 5. We will encode  $\nu$  using its heap index  $H$ . As we saw in Appendix D,  $H$  can be decomposed into two parts: the depth  $D$  and the residual  $R$ :
$$
H = 2 ^ {D} + R, \quad 0 \leq R <   2 ^ {D}. \tag {203}
$$
$$
\begin{array}{l} \mathbb {H} [ H ] \leq \mathbb {H} [ D, R ] (204) \\ = \mathbb {H} [ R \mid D ] + \mathbb {H} [ D ]. (205) \\ \end{array}
$$
Dealing with the second term first, we note that the maximum entropy distribution for a random variable  $Z$  over the positive integers with the moment constraint  $\mathbb{E}[Z] = 1 / p$  is the geometric distribution with rate  $p$ . Thus, based on Equation (202), we set
$$
p = \frac {\log_ {2} (4 / 3)}{I [ \mathbf {x} ; \mathbf {y} ] + 2 \cdot \log_ {2} e}, \tag {206}
$$
from which we get
$$
\begin{array}{l} \mathbb {H} [ D ] \leq \mathbb {H} [ \operatorname {G e o m} (p) ] (207) \\ = - \log_ {2} p - \frac {1 - p}{p} \log_ {2} (1 - p) (208) \\ \leq \log_ {2} (I [ \mathbf {x}; \mathbf {y} ] + 2 \log_ {2} e) - \log_ {2} \log_ {2} (4 / 3) + \log_ {2} e (209) \\ \leq \log_ {2} (I [ \mathbf {x}; \mathbf {y} ] + 1) + \log_ {2} (2 \log_ {2} e) - \log_ {2} \log_ {2} (4 / 3) + \log_ {2} e, (210) \\ \end{array}
$$
where Equation (208) follows from the fact that the term  $-\frac{1 - p}{p}\log_2(1 - p)$  is bounded from above by  $\log_2 e$ , which it attains as  $I[\mathbf{x};\mathbf{y}]\to \infty$ .
For the  $\mathbb{H}[R\mid D]$  term, we begin by noting that there is a particularly natural encoding for  $R$  given a particular run of the algorithm. Let  $B_{1}^{(D)},\ldots ,B_{2^{D}}^{(D)}$  be the bounds associated with the depth-  $D$  nodes of  $\mathcal{T}_{\mathrm{on - sample}}\mid D$ , the split-on-sample BSP tree given  $D$ . Then, since  $B_{1}^{(D)},\ldots ,B_{2^{D}}^{(D)}$  form a  $P$ -essential partition of  $\Omega$ , we can encode  $R$  by encoding the bounds associated with the accepted sample,  $B_R^{(D)}$ . Here, we can borrow a technique from arithmetic coding (Rissanen & Langdon, 1979): we encode the largest dyadic interval that is a subset of  $B_R^{(D)}$ , which can be achieved using  $-\log_2P(B_R^{(D)}) + 1$  bits. Note, that Algorithm 5 only ever realises a single branch of  $\mathcal{T}_{\mathrm{on - sample}}$ , hence, in this section only, let  $X_{1:\infty}$  denote the infinite sequence of samples along this branch, starting with  $X_{1}\sim P$  at the root. Furthermore, from here onwards, let  $B_{d}$  and  $T_{d}$  denote the bounds and arrival time associated with  $X_{d}$  in the sequence, respectively. Then, by the above reasoning, we have
$$
\begin{array}{l} \mathbb {H} [ R \mid D ] \leq \mathbb {E} _ {X _ {1: \infty}, T _ {1: \infty}} \left[ \mathbb {E} _ {D} \left[ - \log_ {2} P \left(B _ {D}\right) \right] \right] + 1 (211) \\ = - \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X _ {1: k - 1}, T _ {1: k}} \left[ \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k} ] \log_ {2} P (B _ {k}) \right] + 1. (212) \\ \end{array}
$$
We now develop a bound on  $P(B_k)$ . To this end, note, that by the definition of GPRS,
$$
\mathbb {P} \left[ D \geq k + 1 \mid X _ {1: k - 1}, T _ {1: k}, D \geq k \right] = \mathbb {P} \left[ \mathbf {N} _ {\tilde {\Pi}} \left(\left[ T _ {k - 1}, T _ {k} \right] \times B _ {k}\right) = 0 \mid X _ {1: k - 1}, T _ {1: k - 1}, D \geq k \right].
$$
$$
\begin{array}{l} \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k}, D \geq k ] = 1 - \mathbb {P} [ D \geq k + 1 \mid X _ {1: k - 1}, T _ {1: k}, D \geq k ] (213) \\ = \mathbb {P} \left[ \mathbf {N} _ {\tilde {\Pi}} \left(\left[ T _ {k - 1}, T _ {k} \right] \times B _ {k}\right) \geq 1 \mid X _ {1: k - 1}, T _ {1: k - 1}, D \geq k \right] (214) \\ \leq \mathbb {E} \left[ \mathbf {N} _ {\tilde {\Pi}} \left(\left[ T _ {k - 1}, T _ {k} \right] \times B _ {k}\right) \mid X _ {1: k - 1}, T _ {1: k - 1}, D \geq k \right] (215) \\ \leq \mathbb {E} \left[ \mathbf {N} _ {\Pi} \left(\left[ T _ {k - 1}, T _ {k} \right] \times B _ {k}\right) \mid X _ {1: k - 1}, T _ {1: k - 1}, D \geq k \right] (216) \\ = P \left(B _ {k}\right) \left(T _ {k} - T _ {k - 1}\right), (217) \\ \end{array}
$$
where eq. (215) follows from Markov's inequality, and eq. (216) follows from the fact that the mean measure of  $\tilde{\Pi}$  is just a restricted version of  $\Pi$ 's mean measure. For convenience, we now write  $\Delta_k = T_k - T_{k-1}$ . Then, rearranging the above inequality, we get
$$
- \log_ {2} P \left(B _ {k}\right) \leq - \log_ {2} \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}, D \geq k ] + \log_ {2} \Delta_ {k}. \tag {218}
$$
Substituting this into eq. (212), we get
$$
\begin{array}{l} \mathbb {H} [ R \mid D ] (219) \\ \leq - \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}} \left[ \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k} ] \log_ {2} \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k} ] \right] \\ + \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}} \left[ \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k} ] \log_ {2} \Delta_ {k} \right] + 1. (220) \\ \end{array}
$$
We deal with the above two sums separately. In the first sum, we apply Jensen's inequality on the concave function  $x \mapsto -x \log_2 x$  in each summand, from which we get
$$
\begin{array}{l} - \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}} \left[ \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k} ] \log_ {2} \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k} ] \right] \\ \leq - \sum_ {k = 1} ^ {\infty} \mathbb {P} [ D = k ] \log_ {2} \mathbb {P} [ D = k ] (221) \\ = \mathbb {H} [ D ]. (222) \\ \end{array}
$$
For the second sum, consider the  $k$ th summand first:
$$
\begin{array}{l} \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}, D \geq k ] \log_ {2} \Delta_ {k} \tag {223} \\ = \int_ {B _ {k}} \frac {d \mathbb {P} [ D = k , X _ {k} = x \mid X _ {1 : k - 1} , T _ {1 : k - 1} , \Delta_ {k} , D \geq k ]}{d P} \log_ {2} \Delta d P (x). \\ = \frac {1}{P \left(B _ {k}\right)} \int_ {B _ {k}} \mathbf {1} [ r (x) \geq \sigma^ {- 1} \left(T _ {k}\right) ] \log_ {2} \Delta d P (x). \\ \end{array}
$$
To continue, we now examine the integrand:
$$
r (x) \geq \sigma^ {- 1} \left(T _ {k}\right) \overset {\text {e q .} (1 3 2)} {\geq} \mathbb {P} [ \tilde {T} \geq T _ {k} ] \cdot T _ {k} \geq \mathbb {P} [ \tilde {T} \geq T _ {k} ] \cdot \Delta_ {k} \tag {224}
$$
$$
\log_ {2} \Delta_ {k} \leq \log_ {2} r (x) - \log_ {2} \mathbb {P} [ \tilde {T} \geq T _ {k} ]. \tag {225}
$$
Substituting this back, we get
$$
\begin{array}{l} \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}, D \geq k ] \log_ {2} \Delta_ {k} (226) \\ \leq \frac {1}{P \left(B _ {k}\right)} \int_ {B _ {k}} \mathbf {1} [ r (x) \geq \sigma^ {- 1} \left(T _ {k}\right) ] \log_ {2} r (x) d P (x) \\ - \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}, D \geq k ] \log_ {2} \mathbb {P} [ \tilde {T} \geq T _ {k} ]. (227) \\ \end{array}
$$
Therefore, for the second summand, we get
$$
\begin{array}{l} \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k}} \left[ \mathbb {P} [ D = k \mid X _ {1: k - 1}, T _ {1: k - 1}, \Delta_ {k} ] \log_ {2} \Delta_ {k} \right] (228) \\ \leq \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X \sim P} \left[ \frac {d \mathbb {P} [ D = k , X _ {k} = X ]}{d P} \log_ {2} r (X) \right] - \sum_ {k = 1} ^ {\infty} \mathbb {P} [ D = k ] \log_ {2} \mathbb {P} [ \tilde {T} \geq T _ {k} ]. (229) \\ \end{array}
$$
For the first term, by Fubini's theorem and by the correctness of GPRS, we get
$$
\begin{array}{l} \sum_ {k = 1} ^ {\infty} \mathbb {E} _ {X \sim P} \left[ \frac {d \mathbb {P} [ D = k , X _ {k} = X ]}{d P} \log_ {2} r (X) \right] = \mathbb {E} _ {X \sim P} \left[ \sum_ {k = 1} ^ {\infty} \frac {d \mathbb {P} [ D = k , X _ {k} = X ]}{d P} \log_ {2} r (X) \right] (230) \\ = \mathbb {E} _ {X \sim P} [ r (X) \log_ {2} r (X) ] (231) \\ = D _ {\mathrm {K L}} [ Q \| P ]. (232) \\ \end{array}
$$
For the second term, note that
$$
\mathbb {P} [ D = k ] = \mathbb {P} [ \tilde {T} = T _ {k} ]. \tag {233}
$$
$$
\begin{array}{l} - \sum_ {k = 1} ^ {\infty} \mathbb {P} [ D = k ] \log_ {2} \mathbb {P} [ \tilde {T} \geq T _ {k} ] = - \sum_ {k = 1} ^ {\infty} \mathbb {P} [ D = k ] \log_ {2} \mathbb {P} [ D \geq k ] (234) \\ = \sum_ {k = 1} ^ {\infty} \left(\mathbb {P} [ D \geq k ] - \mathbb {P} [ D \geq k + 1 ]\right) \log_ {2} \frac {1}{\mathbb {P} [ D \geq k ]} (235) \\ \leq - \int_ {1} ^ {0} \log_ {2} p d p (236) \\ = \log_ {2} e. (237) \\ \end{array}
$$
Thus, we finally get that
$$
\mathbb {H} [ R \mid D ] \leq D _ {\mathrm {K L}} [ Q \| P ] + \mathbb {H} [ D ] + \log_ {2} e + 1. \tag {238}
$$
In the average case, this yields
$$
\mathbb {H} [ R \mid D ] \leq I [ \mathbf {x}; \mathbf {y} ] + \mathbb {H} [ D ] + \log_ {2} e + 1. \tag {239}
$$
Thus, we finally get that the joint entropy is bounded above by
$$
\begin{array}{l} \mathbb {H} [ R, D ] \leq I [ \mathbf {x}; \mathbf {y} ] + 2 \log_ {2} (I [ \mathbf {x}; \mathbf {y} ] + 1) (240) \\ + 2 \left(\log_ {2} \left(2 \log_ {2} e\right) - \log_ {2} \log_ {2} (4 / 3) + \log_ {2} e\right) + \log_ {2} e + 1 (241) \\ <   I [ \mathbf {x}; \mathbf {y} ] + 2 \log_ {2} (I [ \mathbf {x}; \mathbf {y} ] + 1) + 1 1, (242) \\ \end{array}
$$
which finally proves Theorem 3.7.
F General GPRS with Binary Search
We finally present a generalized version of branch-and-bound GPRS (Algorithm 6) for more general spaces and remove the requirement that  $r$  be unimodal.
Decomposing a Poisson process into a mixture of processes: Let  $\Pi$  be a process over  $\mathbb{R}^+ \times \Omega$  as before, with spatial measure  $P$ , and let  $Q$  be a target measure,  $\varphi = \sigma \circ r$  and  $U = \{(t,x) \in \mathbb{R}^+ \times \Omega \mid t \leq \varphi(x)\}$  as before. Let  $\tilde{\Pi}$  be  $\Pi = \Pi \cap U$  restricted under  $\varphi$  and  $(\tilde{T}_1, \tilde{X}_1)$  its first arrival. Let  $\{L,R\}$  form an  $P$ -essential partition of  $\Omega$ , i.e.  $L \cap R = \emptyset$  and  $P(L \cup R) = 1$ , and let  $\Pi_L = \Pi \cap \mathbb{R}^+ \times L$  and  $\Pi_R = \Pi \cap \mathbb{R}^+ \times R$  be the restriction of  $\Pi$  to  $L$  and  $R$ , respectively. Let  $\tilde{\Pi}_L = \Pi_L \cap U$  and  $\tilde{\Pi}_R = \Pi_R \cap U$  be the restrictions of the two processes under  $\varphi$  as well. Let  $\tilde{\mu}_L(t)$  and  $\tilde{\mu}_R(t)$  be the mean measures of these processes. Thus, the first arrival times in these processes have survival functions
$$
\mathbb {P} \left[ \tilde {T} _ {1} ^ {L} \geq t \right] = e ^ {- \tilde {\mu} _ {L} (t)} \tag {243}
$$
$$
\mathbb {P} \left[ \tilde {T} _ {1} ^ {R} \geq t \right] = e ^ {- \tilde {\mu} _ {R} (t)}. \tag {244}
$$
Note, that by the superposition theorem,  $\Pi_L \cup \Pi_R = \Pi$ , hence the first arrival  $(\tilde{T}_1, \tilde{X}_1)$  occurs in either  $\Pi_L$  or  $\Pi_R$ . Assume now that we have already searched through the points of  $\Pi$  up to time  $\tau$  without finding the first arrival. At this point, we can ask: will the first arrival occur in  $\Pi_L$ , given that  $\tilde{T}_1 \geq \tau$ ? Using Bayes' rule, we find
$$
\mathbb {P} \left[ \tilde {X} _ {1} \in L \mid \tilde {T} _ {1} \geq \tau \right] = \frac {\mathbb {P} \left[ \tilde {X} _ {1} \in L , \tilde {T} _ {1} \geq \tau \right]}{\mathbb {P} \left[ \tilde {T} _ {1} \geq \tau \right]}. \tag {245}
$$
More generally, assume that the first arrival of  $\Pi$  occurs in some set  $A \subseteq \Omega$ , and we know that the first arrival time is larger than  $\tau$ . Then, what is the probability that the first arrival occurs in some set  $B \subseteq A$ ? Similarly to the above, we find
$$
\begin{array}{l} \mathbb {P} \left[ \tilde {X} _ {1} \in B \mid \tilde {T} _ {1} \geq \tau , \tilde {X} _ {1} \in A \right] = \frac {\mathbb {P} \left[ \tilde {X} _ {1} \in B , \tilde {T} _ {1} \geq \tau , \tilde {X} _ {1} \in A \right]}{\mathbb {P} \left[ \tilde {T} _ {1} \geq \tau , \tilde {X} _ {1} \in A \right]} (246) \\ = \frac {\mathbb {P} \left[ \tilde {T} _ {1} \geq \tau , \tilde {X} _ {1} \in B \right]}{\mathbb {P} \left[ \tilde {T} _ {1} \geq \tau , \tilde {X} _ {1} \in A \right]}. (247) \\ \end{array}
$$
Let  $(\tilde{T}_L, \tilde{X}_L)$  be the first arrival of  $\Pi_L$ . Then, the crucial observation is that
$$
\left(\tilde {T} _ {L}, \tilde {X} _ {L}\right) \stackrel {{d}} {{=}} \tilde {T} _ {1}, \tilde {X} _ {1} \mid \tilde {X} _ {1} \in L. \tag {248}
$$
This enables us to search for the first arrival of  $\Pi$  under the graph of  $\varphi$  using an extended BSP tree construction. At each node, if we reject, we draw a Bernoulli random variable  $b$  with mean equal to the probability that the first arrival occurs within the bounds associated with the right child node. Then, if  $b = 1$ , we continue the search along the right branch. Otherwise, we search along the left branch.
Note, however, that in a restricted process  $\Pi_A$ , the spatial measure no longer integrates to 1. Furthermore, our target Radon-Nikodym derivative is  $r(x) \cdot \mathbf{1}[x \in A] / Q(A)$ . This means we need to change
the graph  $\varphi$  to some new graph  $\varphi_A$  to ensure that the spatial distribution of the returned sample is still correct. Therefore, for a set  $A$  we define the restricted versions of previous quantities:
$$
\tilde {\mu} _ {A} (t) \stackrel {\text {d e f}} {=} \int_ {0} ^ {t} \int_ {A} \mathbf {1} [ \tau \leq \varphi_ {A} (x) ] d P (x) d t \tag {249}
$$
$$
w _ {P} (h \mid A) \stackrel {\text {d e f}} {=} \int_ {A} \mathbf {1} [ h \leq r (x) ] d P (x) \tag {250}
$$
Then, via analogous arguments to the ones in Appendix A, we find
$$
\frac {d \mathbb {P} [ \tilde {T} _ {A} = t , \tilde {X} _ {A} = x ]}{d (\lambda \times P)} = \mathbf {1} [ x \in A ] \mathbf {1} [ t \leq \varphi_ {A} (x) ] \mathbb {P} [ \tilde {T} _ {A} \geq t ] \tag {251}
$$
$$
\frac {d \mathbb {P} \left[ \tilde {X} _ {A} = x \right]}{d P} = \mathbf {1} [ x \in A ] \int_ {0} ^ {\varphi_ {A} (x)} \mathbb {P} \left[ \tilde {T} _ {A} \geq t \right] d t \tag {252}
$$
$$
\varphi_ {A} = \sigma_ {A} \circ r. \tag {253}
$$
Similarly, setting  $\frac{d\mathbb{P}[\tilde{X}_A = x]}{dP} = \mathbf{1}[x\in A]\cdot r(x) / Q(A)$ , and setting  $\tau = \sigma_{A}(r(x))$ , we get
$$
\sigma_ {A} ^ {- 1} (\tau) = Q (A) \int_ {0} ^ {\tau} \mathbb {P} [ \tilde {T} _ {A} \geq t ] d t \tag {254}
$$
$$
\Rightarrow \quad \left(\sigma_ {A} ^ {- 1}\right) ^ {\prime} (\tau) = \mathbb {P} \left[ \tilde {T} _ {A} \geq \tau \right] \tag {255}
$$
$$
= \mathbb {P} [ \tilde {T} \geq \tau , \tilde {X} \in A ]. \tag {256}
$$
From this, again using similar arguments to the ones in Appendix A, we find
$$
\left(\sigma_ {A} ^ {- 1}\right) ^ {\prime} (\tau) = w _ {Q} \left(\sigma_ {A} ^ {- 1} (t) \mid A\right) - \sigma_ {A} ^ {- 1} (t) \cdot w _ {P} \left(\sigma_ {A} ^ {- 1} (t) \mid A\right). \tag {257}
$$
G Necessary Quantities for Implementing GPRS in Practice
Ultimately, given a target-proposal pair  $(Q,P)$  with density ratio  $r$ , we would want an easy-to-evaluate expression for the appropriate stretch function  $\sigma$  or  $\sigma^{-1}$  to plug directly into our algorithms. Computing  $\sigma$  requires computing the integral in Equation (5) and finding  $\sigma^{-1}$  requires solving the non-linear ODE in Equation (4), neither of which is usually possible in practice. Hence, we usually resort to computing  $\sigma^{-1}$  numerically by using an ODE solver for Equation (4).
In any case, we need to compute  $w_{P}$  and  $w_{Q}$ , which are analytically tractable in the practically relevant cases. Hence, in this section, we give closed-form expressions for  $w_{P}$  and  $w_{Q}$  for all the examples we consider and give closed-form expressions for  $\sigma$  and  $\sigma^{-1}$  for some of them. If we do not give a closed-form expression of  $\sigma$ , we use numerical integration to compute  $\sigma^{-1}$  instead.
Note on notation: In this section only, we denote the natural logarithm as  $\ln$  and exponentiation with respect to the natural base as  $\exp$ .
G.1 Uniform-Uniform Case
Let  $P$  be the uniform distribution over an arbitrary space  $\Omega$  and  $Q$  a uniform distribution supported on some subset  $\mathcal{X} \subset \Omega$ , with  $P(\mathcal{X}) = C$  for some  $C \leq 1$ . Then,
$$
r (x) = \frac {1}{C} \cdot \mathbf {1} [ x \in \mathcal {X} ] \tag {258}
$$
$$
w _ {P} (h) = C \tag {259}
$$
$$
w _ {Q} (h) = 1 \tag {260}
$$
$$
\sigma (h) = - \frac {1}{C} \log (1 - C \cdot h) \tag {261}
$$
$$
\sigma^ {- 1} (t) = \frac {1}{C} (1 - \exp (- C \cdot h)). \tag {262}
$$
Note that using GPRS in the uniform-uniform case is somewhat overkill, as in this case, it is simply equivalent to standard rejection sampling.
G.2 Triangular-Uniform Case
Let  $P = \mathrm{Unif}(0,1)$  and for some numbers  $0 < a < c < b < 1$ , let  $Q$  be the triangular distribution, defined by the PDF
$$
q (x) = \left\{ \begin{array}{l l} 0 & \text {i f} x <   a \\ \frac {2 (x - a)}{(b - a) (c - a)} & \text {i f} a \leq x <   c \\ \frac {2}{b - a} & \text {i f} x = c \\ \frac {2 (b - x)}{(b - a) (b - c)} & \text {i f} c <   x \leq b \\ 0 & \text {i f} b <   x. \end{array} \right. \tag {263}
$$
In this parameterization, the distribution has support on  $(a,b)$ . Moreover, it is linearly increasing on  $(a,c)$  and linearly decreasing on  $(c,b)$ , with the mode located at  $c$ . For convenience, let  $\ell = b - a$ . Then,
$$
r (x) = q (x) \tag {264}
$$
$$
D _ {\mathrm {K L}} [ Q \| P ] = \log_ {2} \left(\frac {2}{\ell}\right) - \frac {1}{2} \log_ {2} e \tag {265}
$$
$$
D _ {\infty} [ Q \| P ] = 1 - \log_ {2} \ell \tag {266}
$$
$$
D _ {\alpha} [ Q \| P ] = - \log_ {2} \ell + \frac {\alpha - \log_ {2} (1 + \alpha)}{\alpha - 1} \tag {267}
$$
$$
w _ {P} (h) = \ell - \frac {\ell^ {2}}{2} \cdot h \tag {268}
$$
$$
w _ {Q} (h) = 1 - \frac {\ell^ {2}}{4} \cdot h ^ {2} \tag {269}
$$
$$
\sigma (h) = \frac {2 h}{2 - \ell \cdot h} \tag {270}
$$
$$
\sigma^ {- 1} (t) = \frac {2 t}{2 + \ell \cdot t} \tag {271}
$$
Additionally, in this case we find
$$
\mathbb {P} [ \tilde {T} \geq t ] = \frac {4}{(2 + \ell \cdot t) ^ {2}} \tag {272}
$$
$$
\mathbb {P} [ \tilde {T} \in d t ] = \frac {8 \cdot \ell}{(2 + \ell \cdot t) ^ {3}} \tag {273}
$$
$$
\begin{array}{l} \varphi_ {\tilde {T}} (s) = \frac {2 G _ {1 , 3} ^ {3 , 1} \left(\frac {s ^ {2}}{l ^ {2}} \mid \begin{array}{c} 0 \\ 0, 1, 3 / 2 \end{array} \right)}{\sqrt {\pi}} (274) \\ + \frac {2 i s ^ {2} \operatorname {s i g n} (s) \left(l - | s | \left(2 \mathrm {C i} \left(\frac {2 | s |}{l}\right) \sin \left(\frac {2 | s |}{l}\right) + \left(\pi - 2 \mathrm {S i} \left(\frac {2 | s |}{l}\right)\right) \cos \left(\frac {2 | s |}{l}\right)\right)\right)}{l ^ {2} | s |} (275) \\ \end{array}
$$
$$
\mathcal {M} _ {\tilde {T}} (s) = \frac {\ell^ {2} + 2 \ell s - 4 s ^ {2} e ^ {- 2 s / \ell} \operatorname {E i} \left(\frac {2 s}{\ell}\right)}{\ell^ {2}} \quad \text {f o r} s \leq 0, \tag {276}
$$
where  $\varphi_{\tilde{T}}$  and  $\mathcal{M}_{\tilde{T}}$  denote the characteristic and the moment-generating functions of  $\tilde{T}$ , respectively. Furthermore, Si, Ci and Ei are the sine, cosine and exponential integrals, defined as
$$
\operatorname {S i} (x) = \int_ {0} ^ {x} t ^ {- 1} \sin (t) d t \tag {277}
$$
$$
\operatorname {C i} (x) = - \int_ {- x} ^ {\infty} t ^ {- 1} \cos (t) d t \tag {278}
$$
$$
\operatorname {E i} (x) = \int_ {- \infty} ^ {x} t ^ {- 1} e ^ {t} d t. \tag {279}
$$
Finally,  $G_{1,3}^{3,1}$  is the Meijer-G function (Sections 5.1-3; Bateman & Erdélyi, 1953) Interestingly, we can see that in this case, while the expectation of  $\tilde{T}$  is finite, we find that  $\mathbb{V}[\tilde{T}] = \infty$ . Thus, by Equation (72), we find that  $\mathbb{V}[N] = \infty$  as well, where  $N$  is the number of samples simulated by Algorithm 3.
G.3 Finite Discrete-Discrete / Piecewise Constant Case
Without loss of generality, let  $Q$  be a discrete distribution over a finite set with  $K$  elements defined by the probability vector  $(r_1 < r_2 < \ldots < r_K)$  and  $P = \mathrm{Unif}([1:K])$  the uniform distribution on  $K$  elements. Note that any discrete distribution pair can be rearranged into this setup. Now, we can embed this distribution into [0, 1] by mapping  $P$  to  $\hat{P} = \mathrm{Unif}(0,1)$  the uniform distribution over [0, 1] and mapping  $Q$  to  $\hat{Q}$  with density
$$
\hat {q} (x) = \sum_ {k = 1} ^ {K} \mathbf {1} [ \lfloor K \cdot x \rfloor = k - 1 ] \cdot K \cdot r _ {k}. \tag {280}
$$
Then, we can draw a sample  $x$  from  $Q$  by simulating a sample  $\hat{x} \sim \hat{Q}$  and computing  $x = K \cdot \lfloor \hat{x} / K \rfloor + 1$ . Hence, we can instead reduce the problem to sampling from a piecewise constant distribution. Thus, let us now instead present the more general case of arbitrary piecewise constant distributions over [0, 1], with  $\tilde{Q}$  defined by probabilities  $(q_1 < q_2 < \ldots < q_K)$  and corresponding piece widths  $(w_1 < w_2 < \ldots < w_K)$ . We require, that  $\sum_{k=1}^{K} q_k = \sum_{k=1}^{K} w_k = 1$ . Then, the density is
$$
\tilde {q} (x) = \sum_ {k = 1} ^ {K} \mathbf {1} \left[ \sum_ {j = 1} ^ {k - 1} w _ {j} \leq x \leq \sum_ {j = 1} ^ {k} w _ {j} \right] \cdot \frac {q _ {k}}{w _ {k}} \tag {281}
$$
Define  $r_k = q_k / w_k$ . Then,
$$
w _ {P} (h) = \sum_ {k = 1} ^ {K} \mathbf {1} [ h \leq r _ {k} ] \cdot w _ {k} \tag {282}
$$
$$
w _ {Q} (h) = \sum_ {k = 1} ^ {K} \mathbf {1} [ h \leq r _ {k} ] \cdot w _ {k} \cdot r _ {k} \tag {283}
$$
$$
\sigma (h) = \sum_ {k = 1} ^ {K} \mathbf {1} [ h \geq r _ {k - 1} ] \cdot \frac {1}{B _ {k}} \cdot \ln \left(\frac {A _ {k} - B _ {k} r _ {k - 1}}{A _ {k} - B _ {k} \min  \{r _ {k} , h \}}\right) \tag {284}
$$
where we defined
$$
A _ {k} = \sum_ {j = k} ^ {K} w _ {j} r _ {j} = \sum_ {j = k} ^ {K} q _ {j} \tag {285}
$$
$$
B _ {k} = \sum_ {j = k} ^ {K} w _ {j} \tag {286}
$$
G.4 Diagonal Gaussian-Gaussian Case
Without loss of generality, let  $Q = \mathcal{N}(\mu, \sigma^2 I)$  and  $P = \mathcal{N}(0, I)$  be  $d$ -dimensional Gaussian distributions with diagonal covariance. As a slight abuse of notation, let  $\mathcal{N}(x \mid \mu, \sigma^2 I)$  denote the probability density function of a Gaussian random variable with mean  $\mu$  and covariance  $\sigma^2 I$  evaluated
at  $x$ . Then, when  $\sigma^2 < 1$ , we have
$$
r (x) = Z \cdot \mathcal {N} (x \mid m, s ^ {2} I) \tag {287}
$$
$$
m = \frac {\mu}{1 - \sigma^ {2}} \tag {288}
$$
$$
s ^ {2} = \frac {\sigma^ {2}}{1 - \sigma^ {2}} \tag {289}
$$
$$
Z = \frac {\left(1 - \sigma^ {2}\right) ^ {- d}}{\mathcal {N} (\mu \mid 0 , (1 - \sigma^ {2}) I)} \tag {290}
$$
$$
w _ {P} (h) = \mathbb {P} \left[ \chi^ {2} (d, \| m \| ^ {2}) \leq - 2 s ^ {2} \ln h + C \right] \tag {291}
$$
$$
w _ {Q} (h) = \mathbb {P} \left[ \chi^ {2} \left(d, \left\| \frac {m - \mu}{\sigma} \right\| ^ {2}\right) \leq \frac {- 2 s ^ {2} \ln h + C}{\sigma^ {2}} \right] \tag {292}
$$
$$
C = s ^ {2} \left(2 \ln Z - d \ln \left(2 \pi s ^ {2}\right)\right). \tag {293}
$$
Unfortunately, in this case, it is unlikely that we can solve for the stretch function analytically, so in our experiments, we solved for it numerically using Equation (4).
G.5 One-dimensional Laplace-Laplace Case