SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\psi_ {p} ^ {- 1} (x ^ {q}) = (\log (1 + x ^ {q})) ^ {1 / p} \leq (\log (1 + x) ^ {q}) ^ {1 / p} = q ^ {1 / p} \psi_ {p} ^ {- 1} (x).
$$
(iv) When  $x\geq 1$
$$
\operatorname {e} x \geq x + 1 \Rightarrow \log x + 1 \geq \log (x + 1) \Rightarrow \log^ {1 / p} (x) + 1 \geq \psi_ {p} ^ {- 1} (x).
$$
The following simple result is for converting between sums and integrals:
Proposition A.3. For any  $r \geq 2$ ,  $K \in \mathbb{N}$ , and a continuous nonincreasing  $f: (0, +\infty) \to (0, +\infty)$ , we have
$$
\sum_ {k = 1} ^ {K} r ^ {- k} f \left(r ^ {- k}\right) \leq r \int_ {0} ^ {1} f (\varepsilon) \mathrm {d} \varepsilon \leq r ^ {2} \sum_ {k = 0} ^ {\infty} r ^ {- k} f \left(r ^ {- k}\right) \tag {A.1}
$$
Proof. Using the monotonicity of  $f$ , we have
$$
\begin{array}{l} \sum_ {k = 1} ^ {K} r ^ {- k} f (r ^ {- k}) \leq \sum_ {k = 1} ^ {K} r ^ {- k} (r - 1) f (r ^ {- k}) \leq r \sum_ {k = 1} ^ {K} \int_ {r ^ {- k - 1}} ^ {r ^ {- k}} f (\varepsilon) \mathrm {d} \varepsilon \\ \leq r \int_ {0} ^ {1} f (\varepsilon) \mathrm {d} \varepsilon \leq r \sum_ {k = 0} ^ {\infty} \int_ {r ^ {- k}} ^ {r ^ {- k + 1}} f (\epsilon) \mathrm {d} \varepsilon \leq r ^ {2} \sum_ {k = 0} ^ {\infty} r ^ {- k} f (r ^ {- k}). \\ \end{array}
$$
B.1 Proofs for Section 2
Proof of Proposition 1. It follows from the inequality  $x \log (x + 1) \leq x \log x + 1, x \geq 0$ , that
$$
\frac {\mathrm {d} \mu}{\mathrm {d} \nu} \log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1\right) \leq \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \log \frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1.
$$
Using this and Jensen's inequality, we get
$$
\begin{array}{l} \left\langle \mu , \psi_ {p} ^ {- 1} \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu}\right) \right\rangle = \left\langle \mu , \left(\log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1\right)\right) ^ {1 / p} \right\rangle \\ \leq \left(\left\langle \mu , \log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1\right) \right\rangle\right) ^ {1 / p} \\ = \left(\left\langle \nu , \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1\right) \right\rangle\right) ^ {1 / p} \\ \leq \left(\left\langle \nu , \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \log \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \right\rangle + 1\right) ^ {1 / p} \\ = \left(D (\mu \| \nu) + 1\right) ^ {1 / p}. \\ \end{array}
$$
B.2 Proofs for Section 3
Proof of Lemma 1. To prove (4), we start with the Young-type inequality
$$
x y \leq \psi_ {p} ^ {*} (x) + \psi_ {p} (y), \quad x, y \geq 0
$$
$$
\psi_ {p} ^ {*} (x) := \sup  _ {y \geq 0} \left(x y - \psi_ {p} (y)\right)
$$
is the (one-sided) Legendre-Fenchel conjugate of  $\psi_p$ . While a closed-form expression for  $\psi_p^*$  is not available, we claim that we can bound it from above as  $\psi_p^*(x) \leq 2^{1/p} x \psi_p^{-1}(x)$ , resulting in
$$
x y \leq 2 ^ {1 / p} x \psi_ {p} ^ {- 1} (x) + \psi_ {p} (y). \tag {B.1}
$$
To establish the claim, we write
$$
\sup  _ {y \geq 0} (x y - \psi_ {p} (y)) = \sup  _ {y \geq 0} (x y - (e ^ {y ^ {p} / 2} - 1) (e ^ {y ^ {p} / 2} + 1))
$$
and consider two cases:
- if  $y \leq 2^{1/p} \psi_p^{-1}(x)$ , then
$$
x y - \left(e ^ {y ^ {p} / 2} - 1\right) \left(e ^ {y ^ {p} / 2} + 1\right) \leq 2 ^ {1 / p} x \psi_ {p} ^ {- 1} (x).
$$
- if  $y > 2^{1/p} \psi_p^{-1}(x)$ , then
$$
x y - (e ^ {y ^ {p} / 2} - 1) (e ^ {y ^ {p} / 2} + 1) \leq (e ^ {y ^ {p} / 2} - 1) (y - (e ^ {y ^ {p} / 2} + 1)) \leq 0.
$$
Applying (B.1) with  $x = \frac{\mathrm{d}\mu}{\mathrm{d}\nu}$  and  $y = g$  gives
$$
g \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \leq 2 ^ {1 / p} \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \psi_ {p} ^ {- 1} \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu}\right) + \psi_ {p} (g),
$$
$$
\begin{array}{l} \langle \mu , f g \rangle = \left\langle \nu , f g \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \right\rangle \\ \leq \left\langle \nu , \left(2 ^ {1 / p} f \frac {\mathrm {d} \mu}{\mathrm {d} \nu} \psi_ {p} ^ {- 1} \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu}\right) + f \psi_ {p} (g)\right) \right\rangle \\ = 2 ^ {1 / p} \left\langle \mu , f \psi_ {p} ^ {- 1} \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu}\right) \right\rangle + \langle \nu , f \psi_ {p} (g) \rangle . \\ \end{array}
$$
To prove (5), define the event
$$
E := \left\{\frac {\mathrm {d} \mu}{\mathrm {d} \nu} \geq \frac {\exp (g ^ {p} / 4) - 1}{\langle \nu , \exp (g ^ {p}) \rangle} \right\}.
$$
Then, since  $\langle \nu ,\exp (g^p)\rangle \geq 1$
$$
\begin{array}{l} \int_ {E} f g \mathrm {d} \mu \leq 4 ^ {1 / p} \int f \left(\log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} \langle \nu , \exp (g ^ {p}) \rangle + 1\right)\right) ^ {1 / p} \mathrm {d} \mu \\ \leq 4 ^ {1 / p} \int f \left(\log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1\right) + \log \langle \nu , \exp (g ^ {p}) \rangle\right) ^ {1 / p} \mathrm {d} \mu \\ \leq 4 ^ {1 / p} \int f \left(\log \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu} + 1\right)\right) ^ {1 / p} \mathrm {d} \mu + 4 ^ {1 / p} \int f \mathrm {d} \mu \cdot \left(\log \langle \nu , \exp (g ^ {p}) \rangle\right) ^ {1 / p} \\ = 4 ^ {1 / p} \left\langle \mu , f \psi_ {p} ^ {- 1} \left(\frac {\mathrm {d} \mu}{\mathrm {d} \nu}\right) \right\rangle + 4 ^ {1 / p} \| f \| _ {L ^ {1} (\mu)} \left(\log \langle \nu , \exp (g ^ {p}) \rangle\right) ^ {1 / p}. \\ \end{array}
$$
$$
\begin{array}{l} \int_ {E ^ {c}} f g \mathrm {d} \mu \leq \int f g \frac {\exp (g ^ {p} / 4) - 1}{\langle \nu , \exp (g ^ {p}) \rangle} \mathrm {d} \nu \\ \leq 2 ^ {1 / p} \int f \frac {\exp (g ^ {p} / 2)}{\langle \nu , \exp (g ^ {p}) \rangle} d \nu \\ \leq 2 ^ {1 / p} \| f \| _ {L ^ {2} (\nu)}, \\ \end{array}
$$
where the first inequality is by the definition of  $E$ , the second inequality follows from Proposition A.2(ii), and the third inequality is by Cauchy-Schwarz. Putting everything together, we get (5).
B.3 Proofs for Section 4
Proof of Theorem 1. It follows from the independence of  $Z_{1}, \ldots, Z_{n}$  that  $\operatorname{gen}(w, S)$  is  $(\sigma / \sqrt{n})$ -subgaussian, so
$$
\mathbf {E} \left[ \psi_ {2} \left(\frac {| \operatorname {g e n} (w , S) |}{\sigma \sqrt {6 / n}}\right) \right] \leq 1, \quad \forall w \in \mathcal {W}. \tag {B.2}
$$
Using Lemma 1 with  $\mu = P_{W|S}, \nu = Q_W$ ,  $f(w) = \sigma \sqrt{6/n}$ , and  $g(w) = \frac{|\mathrm{gen}(w,S)|}{\sigma \sqrt{6/n}}$ , we have
$$
\langle P _ {W | S}, | \mathrm {g e n} (\cdot , S) | \rangle \leq \sqrt {\frac {1 2 \sigma^ {2}}{n}} \left(\left\langle P _ {W | S}, \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} P _ {W | S}}{\mathrm {d} Q _ {W}}\right) \right\rangle + \left\langle Q _ {W}, \psi_ {2} \left(\frac {| \mathrm {g e n} (\cdot , S) |}{\sigma \sqrt {6 / n}}\right) \right\rangle\right).
$$
Taking expectations of both sides w.r.t.  $P_{S}$  and using Fubini's theorem and (B.2), we get (6).
Proof of Corollary 1. Applying Proposition 1 conditionally on  $S$  gives
$$
\left\langle P _ {W | S}, \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} P _ {W | S}}{\mathrm {d} Q _ {W}}\right) \right\rangle \leq \sqrt {D (P _ {W | S} \| Q _ {W}) + 1},
$$
where the divergence  $D(P_{W|S} \| Q_W)$ , being a function of  $S$ , is a random variable. Substituting this into (6) and using Jensen's inequality, the definition of conditional divergence, and  $\sqrt{a + b} \leq \sqrt{a} + \sqrt{b} \leq \sqrt{2(a + b)}$ , we get
$$
\mathbf {E} [ | \mathrm {g e n} (W, S) | ] \leq \sqrt {\frac {2 4 \sigma^ {2}}{n} \left(D (P _ {W | S} | | Q _ {W} | P _ {S}) + 4\right)}.
$$
Taking the infimum of both sides w.r.t.  $Q_W$  and using (2), we get (7).
Proof of Theorem 2. For each fixed  $(w, \tilde{s})$ , the random variable  $\delta(w, \tilde{s}, \varepsilon) := |\sum_{i=1}^{n} \varepsilon_i (\ell(w, z_i') - \ell(w, z_i))|$  is  $\sigma(w, \tilde{s})$ -subgaussian, where  $\sigma(w, \tilde{s}) := \left( \sum_{i=1}^{n} (\ell(w, z_i') - \ell(w, z_i))^2 \right)^{1/2}$ . Thus,
$$
\mathbf {E} _ {\varepsilon} \left[ \zeta (w, \tilde {s}, \varepsilon) \right] := \mathbf {E} _ {\varepsilon} \left[ \psi_ {2} \left(\frac {\delta (w , \tilde {s} , \varepsilon)}{\sqrt {6} \sigma (w , \tilde {s})}\right) \right] \leq 1, \quad \forall (w, \tilde {s}). \tag {B.3}
$$
Applying Lemma 1 conditionally on  $(\tilde{S},\varepsilon)$  with  $\mu = \bar{P}_{W|\tilde{S}\varepsilon},\nu = Q_{W|\tilde{S}},f(w) = \sigma (w,\tilde{S}),g(w) = \zeta (w,\tilde{S},\varepsilon)$ , we obtain
$$
\begin{array}{l} \left\langle \bar {P} _ {W | \tilde {S} _ {\varepsilon}}, \sigma (\cdot , \tilde {S}) \zeta (\cdot , \tilde {S}, \varepsilon) \right\rangle \\ \leq \sqrt {2} \left\langle \bar {P} _ {W | \tilde {S} \varepsilon}, \sigma (\cdot , \tilde {S}) \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} \bar {P} _ {W | \tilde {S} , \varepsilon}}{\mathrm {d} Q _ {W | \tilde {S}}}\right) \right\rangle + \left\langle Q _ {W | \tilde {S}}, \sigma (\cdot , \tilde {S}) \psi_ {2} (\zeta (\cdot , \tilde {S}, \varepsilon)) \right\rangle \\ \leq \sqrt {2} \| \Delta (\tilde {S}) \| _ {\ell^ {2}} \left(\left\langle \bar {P} _ {W | \tilde {S} \varepsilon}, \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} \bar {P} _ {W | \tilde {S} , \varepsilon}}{\mathrm {d} Q _ {W | \tilde {S}}}\right) \right\rangle + \left\langle Q _ {W | \tilde {S}}, \psi_ {2} (\zeta (\cdot , \tilde {S}, \varepsilon)) \right\rangle\right). \\ \end{array}
$$
Taking expectations of both sides w.r.t.  $\tilde{S}$  and  $\varepsilon$ , then using Fubini's theorem, (B.3), and the inequality  $\mathbf{E}_P[|\mathrm{gen}(W,S)|]\leq \frac{1}{n}\mathbf{E}_{\bar{P}}[\delta (W,\tilde{S},\varepsilon)]$ , we obtain (8).
Proof of Corollary 2. For any  $Q_{W|S}$ , using Proposition 1, Cauchy-Schwarz, and the independence of  $(Z_i', Z_i)$ , we have
$$
\begin{array}{l} \mathbf {E} _ {\bar {P}} \left[ \| \Delta (\tilde {S}) \| _ {\ell^ {2}} \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} \bar {P} _ {W | \tilde {S} , \varepsilon}}{\mathrm {d} Q _ {W | \tilde {S}}}\right) \right] \\ \leq \sqrt {\mathbf {E} _ {\bar {P}} [ \| \Delta (\tilde {S}) \| _ {\ell^ {2}} ^ {2} ] \left(D \left(\bar {P} _ {W | \tilde {S} \varepsilon} \| Q _ {W | \tilde {S}} | \bar {P} _ {\tilde {S} \varepsilon}\right) + 1\right)} \\ = \sqrt {n \mathbf {E} [ \Delta (Z , Z ^ {\prime}) ^ {2} ] \left(D (\bar {P} _ {W | \tilde {S} _ {\varepsilon}} \| Q _ {W | \tilde {S}} | \bar {P} _ {\tilde {S} _ {\varepsilon}}) + 1\right)}. \\ \end{array}
$$
Substituting this estimate into (8), taking the infimum of both sides w.r.t.  $Q_{W|\tilde{S}}$ , and using (3), we get (9).
B.4 Proofs for Section 5
Proof of Theorem 3. Let
$$
\delta (u, v, z, z ^ {\prime}) := \left(\ell (u, z ^ {\prime}) - \ell (v, z ^ {\prime})\right) - \left(\ell (u, z) - \ell (v, z)\right),
$$
$$
\delta (u, v, \tilde {s}) := \sum_ {i = 1} ^ {n} \delta (u, v, z _ {i}, z _ {i} ^ {\prime}),
$$
$$
\zeta (u, v, \tilde {s}) := \frac {| \delta (u , v , \tilde {s}) |}{\sqrt {6} \sigma (u , v , \tilde {s})}.
$$
For each fixed  $(u,v)\in \mathcal{W}^2$ ,  $\delta (u,v,Z_i,Z_i')$ ,  $1\leq i\leq n$ , are i.i.d. symmetric random variables. Therefore, introducing a tuple  $\varepsilon = (\varepsilon_{1},\ldots ,\varepsilon_{n})$  of i.i.d. Rademacher random variables independent of everything else and using the fact that the joint distributions of  $\left(\delta (u,v,Z_i,Z_i')\right)_{i = 1}^{n}$  and  $\left(\varepsilon_{i}\delta (u,v,Z_{i},Z_{i}^{\prime})\right)_{i = 1}^{n}$  are the same, we see that
$$
\mathbf {E} [ \psi_ {2} (\zeta (u, v, \tilde {S})) ] = \mathbf {E} _ {\tilde {S}} \mathbf {E} _ {\varepsilon} \left[ \psi_ {2} \left(\frac {| \sum_ {i = 1} ^ {n} \varepsilon_ {i} \delta (u , v , Z _ {i} , Z _ {i} ^ {\prime}) |}{\sqrt {6} \sigma (u , v , \tilde {S})}\right) \right] \leq 1,
$$
where the inequality follows from the fact that, conditionally on  $S$  and  $S'$ , the random variables  $\sum_{i=1}^{n} \varepsilon_i \delta(u, v, Z_i, Z_i')$  are  $\sigma(u, v, \tilde{S})$ -subgaussian.
Now, given  $Q_W \in \mathcal{P}(\mathcal{W})$  and a family of couplings  $P_{UV|S=s} \in \Pi(P_{W|S=s}, Q_W)$ , it follows from the above definitions and from (10) that
$$
\mathbf {E} [ \operatorname {g e n} (W, S) ] \leq \frac {1}{n} \mathbf {E} [ | \delta (U, V, \tilde {S}) | ] = \frac {\sqrt {6}}{n} \mathbf {E} [ \sigma (U, V, \tilde {S}) \zeta (U, V, \tilde {S}) ]. \tag {B.4}
$$
Picking any  $\rho_{UV} \in \mathcal{P}(\mathcal{W} \times \mathcal{W})$  such that  $P_{UV|S=s} \ll \rho_{UV}$  for all  $s \in \mathcal{Z}^n$  and applying Lemma 1, we get
$$
\begin{array}{l} \langle P _ {U V | S}, \sigma (\cdot , \tilde {S}) \zeta (\cdot , \tilde {S}) \rangle \\ \leq 2 \left\langle P _ {U V | S}, \sigma (\cdot , \tilde {S}) \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} P _ {U V | S}}{\mathrm {d} \rho_ {U V}}\right) \right\rangle + \sqrt {2} \left\langle \rho_ {U V}, \sigma (\cdot , \tilde {S}) \psi_ {2} \left(\frac {\zeta (\cdot , \tilde {S})}{\sqrt {2}}\right) \right\rangle . \\ \end{array}
$$
Using the inequality  $\psi_2^2 (x / \sqrt{2})\leq \psi_2(x)$  (see Proposition A.2(i)), Cauchy-Schwarz, and (B.4), we have
$$
\mathbf {E} \left[ \sigma (u, v, \tilde {S}) \psi_ {2} \left(\frac {\zeta (u , v , \tilde {S})}{\sqrt {2}}\right) \right] \leq \sqrt {\mathbf {E} [ \sigma^ {2} (u , v , \tilde {S}) ]}, \quad \forall (u, v) \in \mathcal {W} \times \mathcal {W}.
$$
Putting everything together and taking expectations w.r.t.  $S$  and  $S^{\prime}$ , we obtain (12).
Proof of Corollary 3. For  $\sigma$  defined in Theorem 3, we have
$$
\sigma^ {2} (u, v, \tilde {S}) \leq 2 \sum_ {i = 1} ^ {n} \left(\left(\ell (u, Z _ {i} ^ {\prime}) - \ell (v, Z _ {i} ^ {\prime})\right) ^ {2} + \left(\ell (u, Z _ {i}) - \ell (v, Z _ {i})\right) ^ {2}\right).
$$
Taking conditional expectations given  $U, V, S$  and using Jensen's inequality gives
$$
\begin{array}{l} \mathbf {E} [ \sigma (U, V, \tilde {S}) | U, V, S ] \leq \sqrt {\mathbf {E} [ \sigma^ {2} (U , V , \tilde {S}) | U , V , S ]} \\ \leq \sqrt {2 n} \left(d _ {\ell} (U, V) + d _ {S, \ell} (U, V)\right). \\ \end{array}
$$
An analogous argument gives
$$
\sqrt {\mathbf {E} [ \sigma^ {2} (\bar {U} , \bar {V} , \tilde {S}) | \bar {U} , \bar {V} ]} \leq 2 \sqrt {n} d _ {\ell} (\bar {U}, \bar {V}).
$$
Substituting these estimates into (12) gives the desired result.
B.5 Proofs for Section 6
Proof of Theorem 5. Using the definition of  $\bar{\ell}$ , we have
$$
\mathbf {E} [ \mathrm {g e n} (W, S) ] = \frac {1}{n} \sum_ {k = 1} ^ {K} \mathbf {E} \left[ \sum_ {i = 1} ^ {n} \left(\bar {\ell} \left(W _ {k}, Z _ {i}\right) - \bar {\ell} \left(W _ {k - 1}, Z _ {i}\right)\right) \right].
$$
Applying Lemma 1 conditionally on  $S$  with  $f(u,v) = d(u,v)$ ,  $g(u,v) = \frac{|\sum_{i=1}^{n}(\bar{\ell}(u,Z_i) - \bar{\ell}(v,Z_i))|}{\sqrt{nd}(u,v)}$ ,  $\mu = P_{W_k W_{k-1}|S}$  and  $\nu = \rho_{W_k W_{k-1}}$ , taking expectations w.r.t.  $P_S$ , and using (13) gives the desired result.
Proof of Corollary 4. For each  $k \geq 1$ , let  $\rho_{W_k W_{k-1}} = P_{W_k W_{k-1}}$ . Then
$$
\frac {\mathrm {d} P _ {W _ {k} W _ {k - 1} | S}}{\mathrm {d} P _ {W _ {k} W _ {k - 1}}} = \frac {\mathrm {d} P _ {W _ {k} W _ {k - 1} S}}{\mathrm {d} (P _ {W _ {k} W _ {k - 1}} \otimes P _ {S})} = \frac {\mathrm {d} P _ {S | W _ {k} W _ {k - 1}}}{\mathrm {d} P _ {S}} = \frac {\mathrm {d} P _ {S | W _ {k}}}{\mathrm {d} P _ {S}},
$$
where we have made use of Bayes' rule and the fact that  $S \perp W_{k-1} | W_k$ . Using this in (14) together with Proposition 1 gives (15). An application of Cauchy-Schwarz and Jensen gives (16).
Proof of Corollary 5. For each  $k \geq 1$ , let  $\rho_{W_k W_{k-1}} = P_{W_k W_{k-1}}$ . Notice that, by disintegration and the choice of couplings,
$$
\mathbf {E} [ d (\bar {W} _ {k}, \bar {W} _ {k - 1}) ] = \mathbf {E} \left[ \int d (u, v) P _ {W _ {k} W _ {k - 1} | S} (\mathrm {d} u, \mathrm {d} v) \right] \leq \mathbf {E} \left[ \mathsf {W} _ {2} \left(P _ {W _ {k} | S}, P _ {W _ {k - 1} | S}\right) \right],
$$
where we have used the fact that  $\mathsf{W}_2(\cdot ,\cdot)$  dominates  $\mathsf{W}_1(\cdot ,\cdot)$  [23]. Since  $P_{W_k|S}$  are points on the geodesic connecting  $P_{W|S}$  and  $P_W$ , we have
$$
\sum_ {k = 1} ^ {K} \mathsf {W} _ {2} (P _ {W _ {k} | S}, P _ {W _ {k - 1} | S}) = \sum_ {k = 1} ^ {K} (t _ {k} - t _ {k - 1}) \mathsf {W} _ {2} (P _ {W | S}, P _ {W}) = \mathsf {W} _ {2} (P _ {W | S}, P _ {W}).
$$
Using this together with Cauchy-Schwarz and Proposition 1, we obtain (17).
B.6 Proofs for Section 7
Proof of Theorem 6. Applying (5) conditionally on  $S$  with  $f(w) = \sigma \sqrt{6 / n}$ ,  $g(w) = \frac{\left|\operatorname{gen}(w,S)\right|}{\sigma\sqrt{6 / n}}$ ,  $\mu = P_{W|S}$ , and  $\nu = Q_W$ , we have
$$
\begin{array}{l} \langle P _ {W | S}, | \mathrm {g e n} (W, S) | \rangle \\ \leq \sqrt {\frac {2 4 \sigma^ {2}}{n}} \left(1 + \left\langle P _ {W | S}, \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} P _ {W | S}}{\mathrm {d} Q _ {W}}\right) \right\rangle + \left(\log \left\langle Q _ {W}, \exp \left(\frac {\operatorname* {g e n} ^ {2} (\cdot , S)}{6 \sigma^ {2} / n}\right) \right\rangle\right) ^ {1 / 2}\right). \\ \end{array}
$$
Since  $\ell (w,S)$  is  $(\sigma /\sqrt{n})$  -subgaussian for all  $w$ , Markov's inequality gives, for any  $0 < \delta < 1$ ,
$$
\mathbf {P} \left[ \left\langle Q _ {W}, \exp \left(\frac {\operatorname {g e n} ^ {2} (\cdot , S)}{6 \sigma^ {2} / n}\right) \right\rangle > \frac {2}{\delta} \right] \leq \frac {\delta}{2} \left\langle P _ {S} \otimes Q _ {W}, \exp \left(\frac {\operatorname {g e n} ^ {2} (\cdot , \cdot)}{6 \sigma^ {2} / n}\right) \right\rangle \leq \delta ,
$$
Proof of Theorem 7. The argument is almost identical to the proof of Theorem 3, with the difference that (5) is used for decorrelation.
To lighten the notation, let  $\pi_k^S \coloneqq P_{W_k W_{k-1}|S}$  and  $\rho_k \coloneqq \rho_{W_k W_{k-1}}$ . Use the same definitions of  $\delta, \sigma, \zeta$  as in the proof of Theorem 3. Then, applying (5) with  $f(\cdot) = \sigma(\cdot, \tilde{S})$ ,  $g(\cdot) = \zeta(\cdot, \tilde{S})$ ,  $\mu = \pi_k^S$ ,  $\nu = \rho_k$ , we have
$$
\begin{array}{l} \langle \pi_ {k} ^ {S}, \sigma (\cdot , \tilde {S}) \zeta (\cdot , \tilde {S}) \rangle \leq \sqrt {2} \| \sigma (\cdot , \tilde {S}) \| _ {L ^ {2} \left(\rho_ {k}\right)} + 2 \left\langle \pi_ {k} ^ {S}, \sigma (\cdot , \tilde {S}) \psi_ {2} ^ {- 1} \left(\frac {\mathrm {d} \pi_ {k} ^ {S}}{\mathrm {d} \rho_ {k}}\right) \right\rangle \\ + 2 \left\| \sigma (\cdot , \tilde {S}) \right\| _ {L ^ {1} \left(\pi_ {k} ^ {S}\right)} \sqrt {\log \left\langle \rho_ {k} , \exp \left(\zeta^ {2} (\cdot , \tilde {S})\right) \right\rangle}. \\ \end{array}
$$
By Markov's inequality and the union bound, for any  $0 < \delta < 1$
$$
\mathbf {P} \left[ \exists k \text {s . t .} \left\langle \rho_ {k}, \exp \left(\zeta^ {2} (\cdot , \tilde {S})\right) \right\rangle > \frac {2}{p _ {k} \delta} \right] \leq \sum_ {k} \frac {p _ {k} \delta}{2} \left\langle P _ {\tilde {S}} \otimes \rho_ {k}, \exp \left(\zeta^ {2} (\cdot , \cdot)\right) \right\rangle \leq \delta .
$$
Using this together with the estimate  $\sigma (\cdot ,\tilde{S})\leq 2\sqrt{n} d_{\tilde{S},\ell}(\cdot)$  yields the result in the statement.
B.7 Proofs for Section 8
Proof of Theorem 8. Without loss of generality, we assume  $\mathrm{diam}(T) = 1$ . Let  $Q$  be the Markov kernel from  $\Omega$  to  $T$  defined by  $Q(\cdot |\omega) = \delta_{\tau (\omega)}(\cdot)$ ; in particular,  $\nu (\cdot) = \int_{\Omega}\mathbb{P}(\mathrm{d}\omega)Q(\cdot |\omega)$ .
Fix some  $r \geq 2$ . For each  $k \geq 0$  and each  $t \in T$ , let  $B_{k}(t) \coloneqq B(t, r^{-k})$ . Since  $T$  is finite, there exists some  $K \in \mathbb{N}$ , such that  $B_{K}(t) = \{t\}$  for all  $t \in T$ . Let  $\mu \in \mathcal{P}(T)$  be given. Define the following sequence of Markov kernels from  $\Omega$  to  $T$ :
$$
Q _ {k} (\cdot | \omega) := \frac {\mu (\cdot \cap B _ {k} (\tau (\omega)))}{\mu (B _ {k} (\tau (\omega)))}, \qquad k = 0, \dots , K.
$$
Observe that  $Q_0 = \mu$  and  $Q_K = Q$ . Then, since
$$
\langle \mathbb {P} \otimes \mu , X \rangle = \int_ {\Omega \times T} \mathbb {P} (\mathrm {d} \omega) \mu (\mathrm {d} t) X _ {t} (\omega) = \int_ {T} \mu (\mathrm {d} t) \mathbf {E} [ X _ {t} ] = 0,
$$
we can write
$$
\begin{array}{l} \mathbf {E} \left[ X _ {\tau} \right] = \langle \mathbb {P} \otimes Q, X \rangle - \langle \mathbb {P} \otimes \mu , X \rangle \\ = \langle \mathbb {P} \otimes Q _ {K}, X \rangle - \langle \mathbb {P} \otimes Q _ {0}, X \rangle \\ = \sum_ {k = 1} ^ {K} \left\langle \mathbb {P} \otimes Q _ {k} - \mathbb {P} \otimes Q _ {k - 1}, X \right\rangle \\ = \sum_ {k = 1} ^ {K} \int_ {\Omega} \left(\int_ {T} X _ {t} (\omega) Q _ {k} (\mathrm {d} t | \omega) - \int_ {T} X _ {t} (\omega) Q _ {k - 1} (\mathrm {d} t | \omega)\right) \mathbb {P} (\mathrm {d} \omega) \\ \leq \sum_ {k = 1} ^ {K} \int_ {\Omega} \int_ {T \times T} | X _ {u} (\omega) - X _ {v} (\omega) | Q _ {k} (\mathrm {d} u | \omega) Q _ {k - 1} (\mathrm {d} v | \omega) \mathbb {P} (\mathrm {d} \omega). \\ \end{array}
$$
Applying (4) conditionally on  $\omega$  with
$$
f (u, v) = \mathbf {1} _ {B _ {k} (\tau (\omega))} (u) \mathbf {1} _ {B _ {k - 1} (\tau (\omega))} (v) d (u, v),
$$
$$
g (u, v) = \frac {\left| X _ {u} (\omega) - X _ {v} (\omega) \right|}{d (u , v)},
$$
$$
\mu (\mathrm {d} u, \mathrm {d} v) = Q _ {k} (\mathrm {d} u | \omega) \otimes Q _ {k - 1} (\mathrm {d} v | \omega),
$$
$$
\nu (\mathrm {d} u, \mathrm {d} v) = \mu (\mathrm {d} u) \otimes \mu (\mathrm {d} v),
$$
and using the fact that  $Q_{k}(\cdot |\omega)$  is supported on  $B_{k}(\tau (\omega))$  and  $B_{k}(\tau (\omega))\subseteq B_{k - 1}(\tau (\omega))$ , we have
$$
\begin{array}{l} \int_ {T \times T} | X _ {u} (\omega) - X _ {v} (\omega) | Q _ {k} (\mathrm {d} u | \omega) Q _ {k - 1} (\mathrm {d} v | \omega) \\ \leq 2 ^ {1 / p} r ^ {- k + 1} \psi_ {p} ^ {- 1} \left(\frac {1}{\mu (B _ {k} (\tau (\omega))) ^ {2}}\right) + r ^ {- k + 1} \int_ {T \times T} \psi_ {p} \left(\frac {| X _ {u} (\omega) - X _ {v} (\omega) |}{d (u , v)}\right) \mu (\mathrm {d} u) \mu (\mathrm {d} v) \\ \leq 2 ^ {2 / p} r ^ {- k + 1} \psi_ {p} ^ {- 1} \left(\frac {1}{\mu (B _ {k} (\tau (\omega)))}\right) + r ^ {- k + 1} \int_ {T \times T} \psi_ {p} \left(\frac {| X _ {u} (\omega) - X _ {v} (\omega) |}{d (u , v)}\right) \mu (\mathrm {d} u) \mu (\mathrm {d} v), \\ \end{array}
$$
where the first term in the last step is due to Proposition A.2(iii). Then, using the increment condition and Proposition A.3, we have
$$
\begin{array}{l} \mathbf {E} \left[ X _ {\tau} \right] \leq 2 ^ {2 / p} \sum_ {k = 1} ^ {K} r ^ {- k + 1} \int_ {T} \psi_ {p} ^ {- 1} \left(\frac {1}{\mu (B (t , r ^ {- k}))}\right) \nu (\mathrm {d} t) + \sum_ {k = 1} ^ {K} r ^ {- k + 1} \\ \leq 1 + 2 ^ {2 / p} r ^ {2} \int_ {T} \int_ {0} ^ {1} \psi_ {p} ^ {- 1} \left(\frac {1}{\mu (B (t , \varepsilon))}\right) \mathrm {d} \varepsilon \nu (\mathrm {d} t). \\ \end{array}
$$
Since  $1 / \mu (B(t,\varepsilon))\geq 1$  , we can apply Proposition A.2(iv) to obtain the inequality
$$
\mathbf {E} [ X _ {\tau} ] \leq 2 ^ {2 / p} r ^ {2} \left(2 + \int_ {T} \int_ {0} ^ {1} \left(\log \frac {1}{\mu (B (t , \varepsilon))}\right) ^ {1 / p} \mathrm {d} \varepsilon \nu (\mathrm {d} t)\right).
$$
We can now take  $r = 2$  to get the desired result when  $\mathrm{diam}(T) = 1$ ; the general finite-diameter case follows by straightforward rescaling.