SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The algorithm terminates when we find a  $d$  such that  $0 = \sum_{s\leqslant t}1_{X_s\in (v,\rho_d]}$ . Since  $\epsilon (d)\uparrow \infty$  as  $d\uparrow \infty$ , we have  $\rho_{d} = \epsilon (d)[\epsilon (d)^{-1}v]\downarrow v$ , so that  $\sum_{s\leqslant t}1_{X_s\in (v,\rho_d]}\downarrow 0$ . So the algorithm must terminate.
At level  $d$ , we have  $\epsilon(d)$  confidence sequences. The  $i^{\text{th}}$  confidence sequence at level  $d$  satisfies
$$
P \left(\exists t: \overline {{\mathrm {C D F}}} _ {t} (i / \epsilon (d)) > \Xi_ {t} (i / \epsilon (d); \delta_ {d}, d, \Psi_ {t})\right) \leqslant \frac {\alpha}{2 ^ {d} \epsilon (d)}. \tag {12}
$$
Taking a union bound over all confidence sequences at all levels, we have
$$
P \left(\exists d \in \mathbb {N}, i \in \{1, \dots , d \}, t \in \mathbb {N}: \overline {{\mathrm {C D F}}} _ {t} (i / \epsilon (d)) > \Xi_ {t} (i / \epsilon (d); \delta , d, \Psi_ {t})\right) \leqslant \alpha . \tag {13}
$$
Thus we are assured that, for any  $v\in \mathbb{R}$
$$
P \left(\forall t, d: \overline {{\mathrm {C D F}}} _ {t} (v) \leqslant \overline {{\mathrm {C D F}}} _ {t} \left(\rho_ {d}\right) \leqslant \Xi_ {t} \left(\rho_ {d}; \delta_ {d}, d, \Psi_ {t}\right)\right) \geqslant 1 - \alpha . \tag {14}
$$
Algorithm 1 will return  $\Xi_t(\rho_d; \delta_d, d, \Psi_t)$  for some  $d$  unless all such values are larger than one, in which case it returns the trivial upper bound of one. This proves the upper-bound half of guarantee (2). A similar argument proves the lower-bound half, and union bound over the upper and lower bounds finishes the argument.
C Proof of Theorem 3.3
Theorem 3.3. Let  $U_{t}(v)$  and  $L_{t}(v)$  be the upper and lower bounds returned by Algorithm 1 and Algorithm 2 respectively, when evaluated with  $\epsilon(d) = 2^{d}$  and the confidence sequences  $\Lambda_{t}$  and  $\Xi_{t}$  of Equation (15). If  $\forall t: \mathbb{P}_{t}$  is  $\xi_{t}$ -smooth wrt the uniform distribution on the unit interval then
where  $q_{t} \doteq \overline{\mathrm{CDF}}_{t}(v)$ ;  $V_{t} \doteq 1 / t + (q_{t} - 1 / 2) / \log (q_{t} / 1 - q_{t})$ ; and  $\tilde{O}()$  elides polylog  $V_{t}$  factors.
Note  $v$  is fixed for the entire argument below, and  $\xi_{t}$  denotes the unknown smoothness parameter at time  $t$ .
We will argue that the upper confidence radius  $U_{t}(v) - t^{-1}\sum_{s\leqslant t}1_{X_{s}\leqslant v}$  has the desired rate. An analogous argument applies to the lower confidence radius  $t^{-1}\sum_{s\leqslant t}1_{X_s\leqslant v} - L_t(v)$ , and the confidence width  $U_{t}(v) - L_{t}(v)$  is the sum of these two.
For the proof we introduce an integer parameter  $\eta \geqslant 2$  which controls both the grid spacing  $(\epsilon (d) = \eta^{d})$  and the allocation of error probabilities to levels  $(\delta_{d} = \alpha /(\eta^{d}\epsilon (d)))$ . In the main paper we set  $\eta = 2$ .
At level  $d$  we construct  $\eta^d$  confidence sequences on an evenly-spaced grid of values  $1 / \eta^d, 2 / \eta^d, \ldots, 1$ . We divide total error probability  $\alpha / \eta^d$  at level  $d$  among these  $\eta^d$  confidence sequences, so that each individual confidence sequence has error probability  $\alpha / \eta^{2d}$ .
For a fixed bet  $\lambda$  and value  $\rho$ ,  $S_{t}$  defined in Section 3.2 is sub-Bernoulli qua Howard et al. [2021, Definition 1] and therefore sub-Gaussian with variance process  $V_{t} \doteq tK(q_{t})$ , where  $K(p) \doteq (2p - 1)/2 \log (p / 1 - p)$  is from Kearns and Saul [1998]; from Howard et al. [2021, Proposition 5] it follows that there exists an explicit mixture distribution over  $\lambda$  such that
$$
M (t; q _ {t}, \tau) \doteq \sqrt {2 (t K (q _ {t}) + \tau) \log \left(\frac {\eta^ {2 d}}{2 \alpha} \sqrt {\frac {t K (q _ {t}) + \tau}{\tau}} + 1\right)} \tag {15}
$$
is a (curved) uniform crossing boundary, i.e., satisfies
$$
\frac {\alpha}{\eta^ {2 d}} \geqslant \mathbb {P} \left(\exists t \geqslant 1: S _ {t} \geqslant \frac {M (t ; q _ {t} , \tau)}{t}\right),
$$
where  $S_{t} \doteq \overline{\mathrm{CDF}}_{t}(\rho) - t^{-1}\sum_{s\leqslant t}1_{X_{s}\leqslant \rho}$  is from Equation (10), and  $\tau$  is a hyperparameter to be determined further below.
Because the values at level  $d$  are  $1 / \eta^d$  apart, the worst-case discretization error in the estimated average CDF value is
$$
\overline {{\mathrm {C D F}}} _ {t} \left(\epsilon (d) \left[ \epsilon (d) ^ {- 1} v \right]\right) - \overline {{\mathrm {C D F}}} _ {t} (v) \leqslant 1 / \left(\xi_ {t} \eta^ {d}\right),
$$
and the total worst-case confidence radius including discretization error is
$$
r _ {d} (t) = \frac {1}{\xi_ {t} \eta^ {d}} + \sqrt {\frac {2 (K (q _ {t}) + \tau / t)}{t} \log \left(\frac {\eta^ {2 d}}{2 \alpha} \sqrt {\frac {t K (q _ {t}) + \tau}{\tau}} + 1\right)}.
$$
Now evaluate at  $d$  such that  $\sqrt{\psi_t} < \xi_t\eta^d \leqslant \eta \sqrt{\psi_t}$  where  $\psi_t \doteq t(K(q_t) + \tau / t)^{-1}$ ,
$$
r _ {d} (t) \leqslant \sqrt {\frac {K (q _ {t}) + \tau / t}{t}} + \sqrt {\frac {2 (K (q _ {t}) + \tau / t)}{t} \log \left(\frac {\xi_ {t} ^ {- 2} \eta^ {2}}{2 \alpha} \left(\frac {t}{K (q _ {t}) + \tau / t}\right) \sqrt {\frac {t K (q _ {t}) + \tau}{\tau}} + 1\right)}.
$$
The final result is not very sensitive to the choice of  $\tau$ , and we use  $\tau = 1$  in practice.
D Extensions
D.1 Arbitrary Support
Algorithm 3 is a variation on Algorithm 1 which does not assume a bounded range, and instead uses a countably discrete dense subset of the entire real line. Using the same argument of Theorem 3.3 with the modified probability from the modified union bound, we have
$$
\begin{array}{l} \left| k _ {d} \right| - 1 <   \eta^ {- d} | v | \leqslant \left| k _ {d} \right|, \\ \xi_ {t} / \sqrt {\psi_ {t}} > \eta^ {- d} \geqslant \eta^ {- 1} \xi_ {t} / \sqrt {\psi_ {t}} \\ \Longrightarrow 1 + \left| k _ {d} \right| <   2 + \xi_ {t} | v | / \sqrt {\psi_ {t}} \\ \Rightarrow r _ {d} (t) \leqslant \tilde {O} \left(\sqrt {\frac {V _ {t}}{t} \log \left(\left(2 + \xi_ {t} | v | t ^ {- 1 / 2}\right) ^ {2} \xi_ {t} ^ {- 2} \alpha^ {- 1} t ^ {3 / 2}\right)}\right), \\ \end{array}
$$
demonstrating a logarithmic penalty in the probe value  $v$  (e.g., Figure 7).
Algorithm 3 Entire Real Line Upper Bound.  $\epsilon (d)$  is an increasing function specifying the resolution of discretization at level  $d$ .  $\Xi_t$  ( $\rho$ ;  $\delta$ ,  $d$ ,  $\Psi_t$ ) is an upper confidence sequence for fixed value  $\rho$  with coverage at least  $(1 - \delta)$ .
Input: value  $v$ ; confidence  $\alpha$ ; sufficient statistic  $\Psi_t$ .
//e.g.  $\Psi_t\dot{=} X_{1:t}$  or  $\Psi_t\dot{=} (W_{1:t},X_{1:t})$
Output:  $U_{t}(v)$  satisfying Equation (2).
$u\gets 1$
for  $d = 1$  to  $\infty$  do
$$
k _ {d} \leftarrow \lceil \epsilon (d) ^ {- 1} v \rceil
$$
$$
\rho_ {d} \leftarrow \epsilon (d) k _ {d}
$$
$$
\delta_ {d} \leftarrow \left(\alpha / 2 ^ {d}\right) \left(3 / \left(\pi^ {2} - 3\right) (1 + | k _ {d} |) ^ {2}\right)
$$
$$
u \leftarrow \min  \left(u, \Xi_ {t} \left(\rho_ {d}; \delta_ {d}, d, \Psi_ {t}\right)\right)
$$
$$
\text {i f} 0 = \sum_ {s \leqslant t} 1 _ {X _ {s} \in (v, \rho_ {d} ]} \text {t h e n}
$$
$$
\text {r e t u r n} u
$$
// Sub-optimal: see text for details
// Union bound over  $d \in \mathbb{N}$  and  $k_d \in \mathbb{Z}$
Sub-optimality of  $k_{d}$  The choice of  $k_{d}$  in Algorithm 3 is amenable to analysis, but unlike in Algorithm 1, it is not optimal. In Algorithm 1 the probability is allocated uniformly at each depth, and therefore the closest grid point provides the tightest estimate. However in Algorithm 3, the probability budget decreases with  $|k_{d}|$  and because  $k_{d}$  can be negative, it is possible that a different  $k_{d}$  can produce a tighter upper bound. Since every  $k_{d}$  is covered by the union bound, in principle we could optimize over all  $k_{d}$  but it is unclear how to do this efficiently. In our implementation we do not search over all  $k_{d}$ , but we do adjust  $k_{d}$  to be closest to the origin with the same empirical counts.
D.2 Discrete Jumps
Known Countably Infinite Suppose  $D$  is smooth wrt a reference measure  $M$ , where  $M$  is of the form
$$
M = \breve {M} + \sum_ {i \in I} \zeta_ {i} 1 _ {v _ {i}},
$$
with  $I$  a countable index set,  $1 \geqslant \sum_{i \in I} \zeta_i$  and  $\check{M}$  a sub-probability measure normalizing to  $(1 - \sum_{i \in I} \zeta_i)$ . Then we can allocate  $(1 - \sum_{i \in I} \zeta_i)$  of our overall coverage probability to bounding  $\check{M}$  using Algorithm 1 and Algorithm 2. For the remaining  $\{v_i\}_{i \in I}$  we can run explicit pointwise bounds each with coverage probability fraction  $\zeta_i$ .
Computationally, early termination of the infinite search over the discrete bounds is possible. Suppose (wlog)  $I$  indexes  $\zeta$  in non-increasing order, i.e.,  $i \leqslant j \Rightarrow \zeta_i \leqslant \zeta_j$ : then as soon as there are no remaining empirical counts between the desired value  $v$  and the most recent discrete value  $v_i$ , the search over discrete bounds can terminate.
E Importance-Weighted Variant
E.1 Modified Bounds
Algorithm 1 and Algorithm 2 are unmodified, with the caveat that the oracles  $\Lambda_t$  and  $\Xi_t$  must now operate on an importance-weighted realization  $(W_{1:t}, X_{1:t})$ , rather than directly on the realization  $X_{1:t}$ .
E.1.1 DDRM Variant
For simplicity we describe the lower bound  $\Lambda_t$  only. The upper bound is derived analogously via the equality  $Y_s = W_s - (W_s - Y_s)$  and a lower bound on  $(W_s - Y_s)$ : see Waudby-Smith et al. [2022, Remark 3] for more details.
This is the Heavy NSM from Mineiro [2022] combined with the  $L^{*}$  bound of Orabona [2019, §4.2.3]. The Heavy NSM allows us to handle importance weights with unbounded variance, while the Adagrad  $L^{*}$  bound facilitates lazy evaluation.
For fixed  $v$ , let  $Y_{t} = W_{t}1_{X_{t}\geqslant v}$  be a non-negative real-valued discrete-time random process, let  $\hat{Y}_t\in [0,1]$  be a predictable sequence, and let  $\lambda \in [0,1)$  be a fixed scalar bet. Then
$$
E _ {t} (\lambda) \doteq \exp \left(\lambda \left(\sum_ {s \leqslant t} \hat {Y} _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) + \sum_ {s \leqslant t} \log \left(1 + \lambda \left(Y _ {s} - \hat {Y} _ {s}\right)\right)\right)
$$
is a test supermartingale [Mineiro, 2022, §3]. Manipulating,
$$
\begin{array}{l} E _ {t} (\lambda) = \exp \left(\lambda \left(\sum_ {s \leqslant t} Y _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) - \sum_ {s \leqslant t} \underbrace {\left(\lambda \left(Y _ {s} - \hat {Y} _ {s}\right) - \log \left(1 + \lambda \left(Y _ {s} - \hat {Y} _ {s}\right)\right)\right)} _ {\doteq h \left(\lambda \left(Y _ {s} - \hat {Y} _ {s}\right)\right)}\right) \\ = \exp \left(\lambda \left(\sum_ {s \leqslant t} Y _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) - \sum_ {s \leqslant t} h \left(\lambda \left(Y _ {s} - \hat {Y} _ {s}\right)\right)\right) \\ \geqslant \exp \left(\lambda \left(\sum_ {s \leqslant t} Y _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) - \left(\sum_ {s \leqslant t} h \left(\lambda \left(Y _ {s} - \hat {Y} _ {t} ^ {*}\right)\right)\right) - \operatorname {R e g} (t)\right) \tag {†} \\ = \exp \left(\lambda \left(t \hat {Y} _ {t} ^ {*} - \sum_ {s \leqslant t} \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) + \sum_ {s \leqslant t} \log \left(1 + \lambda \left(Y _ {s} - \hat {Y} _ {t} ^ {*}\right)\right) - \operatorname {R e g} (t)\right), \\ \end{array}
$$
where for  $(\dagger)$  we use a no-regret learner on  $h(\cdot)$  with regret  $\operatorname{Reg}(t)$  to any constant prediction  $\hat{Y}_t^* \in [0,1]$ . The function  $h(\cdot)$  is  $M$ -smooth with  $M = \frac{\lambda^2}{(1 - \lambda)^2}$  so we can get an  $L^*$  bound [Orabona, 2019, §4.2.3] of
$$
\begin{array}{l} \operatorname {R e g} (t) = 4 \frac {\lambda^ {2}}{(1 - \lambda) ^ {2}} + 4 \frac {\lambda}{1 - \lambda} \sqrt {\sum_ {s \leqslant t} h (\lambda (\hat {Y _ {s}} - \hat {Y _ {t}} ^ {*})))} \\ = 4 \frac {\lambda^ {2}}{(1 - \lambda) ^ {2}} + 4 \frac {\lambda}{1 - \lambda} \sqrt {\left(- t \hat {Y} _ {t} ^ {*} + \sum_ {s \leqslant t} Y _ {s}\right) - \sum_ {s \leqslant t} \log \left(1 + \lambda \left(Y _ {s} - \hat {Y} _ {t} ^ {*}\right)\right)}, \\ \end{array}
$$
thus essentially our variance process is inflated by a square-root. In exchange we do not have to actually run the no-regret algorithm, which eases the computational burden. We can compete with any in-hindsight prediction: if we choose to compete with the clipped running mean  $\overline{Y_t}$  then we end up with
$$
E _ {t} (\lambda) \geqslant \exp \left(\lambda \left(\min  \left(t, \sum_ {s \leqslant t} Y _ {s}\right) - \mathbb {E} _ {s - 1} [ Y _ {s} ]\right) + \sum_ {s \leqslant t} \log \left(1 + \lambda \left(Y _ {s} - \bar {Y _ {t}}\right)\right) - \operatorname {R e g} (t)\right), \tag {16}
$$
which is implemented in the reference implementation as LogApprox:getLowerBoundWithRegret(lam). The  $\lambda$ -s are mixed using DDRM from Mineiro [2022, Thm. 4], implemented via the DDRM class and the getDDRMSLowerBound method in the reference implementation. getDDRMSLowerBound provably correctly early terminates the infinite sum by leveraging
$$
\sum_ {s \leqslant t} \log \left(1 + \lambda \left(Y _ {s} - \overline {{Y _ {t}}}\right)\right) \leqslant \lambda \left(\sum_ {s \leqslant t} Y _ {s} - t \overline {{Y _ {t}}}\right)
$$
as seen in the termination criterion of the inner method logwealth(mu).
To minimize computational overhead, we can lower bound  $\log (a + b)$  for  $b\geq 0$  using strong concavity qua Mineiro [2022, Thm. 3], resulting in the following geometrically spaced collection of sufficient statistics:
$$
(1 + k) ^ {n _ {l}} = z _ {l} \leqslant z <   z _ {u} = (1 + k) z _ {l} = (1 + k) ^ {n _ {l} + 1},
$$
along with distinct statistics for  $z = 0$ .  $k$  is a hyperparameter controlling the granularity of the discretization (tighter lower bound vs. more space overhead): we use  $k = 1/4$  exclusively in our
experiments. Note the coverage guarantee is preserved for any choice of  $k$  since we are lower bounding the wealth.
Given these statistics, the wealth can be lower bounded given any bet  $\lambda$  and any in-hindsight prediction  $\hat{Y}_t^*$  via
$$
f (z) \doteq \log \left(1 + \lambda \left(z - \hat {Y} _ {t} ^ {*}\right)\right),
$$
$$
f (z) \geqslant \alpha f (z _ {l}) + (1 - \alpha) f (z _ {u}) + \frac {1}{2} \alpha (1 - \alpha) m (z _ {l}),
$$
$$
\alpha \doteq \frac {z _ {u} - z}{z _ {u} - z _ {l}},
$$
$$
m (z _ {l}) \doteq \left(\frac {k z _ {l} \lambda}{k z _ {l} \lambda + 1 - \lambda \hat {Y} _ {t} ^ {*}}\right) ^ {2}.
$$
Thus when accumulating the statistics, for each  $Y_{s} = W_{s}1_{X_{s}\geqslant v}$ , a value of  $\alpha$  must be accumulated at key  $f(z_{l})$ , a value of  $(1 - \alpha)$  accumulated at key  $f(z_{u})$ , and a value of  $\alpha (1 - \alpha)$  accumulated at key  $m(z_{l})$ . The LogApprox: : update method from the reference implementation implements this.
Because these sufficient statistics are data linear, a further computational trick is to accumulate the sufficient statistics with equality only, i.e., for  $Y_{s} = W_{s}1_{X_{s} = v}$ ; and when the CDF curve is desired, combine these point statistics into cumulative statistics. In this manner only  $O(1)$  incremental work is done per datapoint; while an additional  $O(t\log (t))$  work is done to accumulate all the sufficient statistics only when the bounds need be computed. The method StreamingDDRMECDF::Frozen::__init__ from the reference implementation contains this logic.
E.1.2 Empirical Bernstein Variant
This is the empirical Bernstein NSM from Howard et al. [2021] combined with the  $L^{*}$  bound of Orabona [2019, §4.2.3]. Relative to DDRM it is faster to compute, has a more concise sufficient statistic, and is easier to analyze; but it is wider empirically, and theoretically requires finite importance weight variance to converge.
$$
\begin{array}{l} E _ {t} (\lambda) \doteq \exp \left(\lambda \left(\sum_ {s \leqslant t} Y _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) - \sum_ {s \leqslant t} \underbrace {\left(\lambda \left(Y _ {s} - \hat {Y} _ {s}\right) - \log \left(1 + \lambda \left(Y _ {s} - \hat {Y} _ {s}\right)\right)\right)} _ {\doteq h \left(\lambda \left(Y _ {s} - \hat {Y} _ {s}\right)\right)}\right) \\ \geqslant \exp \left(\lambda \left(\sum_ {s \leqslant t} Y _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) - h (- \lambda) \sum_ {s \leqslant t} \left(Y _ {s} - \hat {Y} _ {s}\right) ^ {2}\right) (Fan, Lemma 4.1) \\ \geqslant \exp \left(\lambda \left(\sum_ {s \leqslant t} Y _ {s} - \mathbb {E} _ {s - 1} \left[ Y _ {s} \right]\right) - h (- \lambda) \left(\operatorname {R e g} (t) + \sum_ {s \leqslant t} \left(Y _ {s} - Y _ {t} ^ {*}\right) ^ {2}\right)\right) (†) \\ \dot {=} \exp (\lambda S _ {t} - h (- \lambda) V _ {t}), \\ \end{array}
$$
where  $S_{t} = \sum_{s\leqslant t}Y_{s} - \mathbb{E}_{s - 1}\left[Y_{s}\right]$  and for  $(\dagger)$  we use a no-regret learner on squared loss on feasible set  $[0,1]$  with regret  $\mathrm{Reg}(t)$  to any constant in-hindsight prediction  $\hat{Y}_t^* \in [0,1]$ . Since  $Y_{s}$  is unbounded
above, the loss is not Lipschitz and we can't get fast rates for squared loss, but we can run Adagrad and get an  $L^*$  bound,
$$
\begin{array}{l} \operatorname {R e g} (t) = 2 \sqrt {2} \sqrt {\sum_ {s \leqslant t} g _ {s} ^ {2}} \\ = 4 \sqrt {2} \sqrt {\sum_ {s \leqslant t} (Y _ {s} - \hat {Y} _ {s}) ^ {2}} \\ \leqslant 4 \sqrt {2} \sqrt {\operatorname {R e g} (t) + \sum_ {s \leqslant t} \left(Y _ {s} - \hat {Y} _ {t} ^ {*}\right) ^ {2}}, \\ \Rightarrow \operatorname {R e g} (t) \leqslant 1 6 + 4 \sqrt {2} \sqrt {8 + \sum_ {s \leqslant t} \left(Y _ {s} - \hat {Y} _ {t} ^ {*}\right) ^ {2}}. \\ \end{array}
$$
Thus basically our variance process is inflated by an additive square root.
We will compete with  $Y_{t}^{*} = \min \left(1, \frac{1}{t} \sum_{s} Y_{s}\right)$ .
A key advantage of the empirical Bernstein over DDRM is the availability of both a conjugate (closed-form) mixture over  $\lambda$  and a closed-form majorized stitched boundary. This yields both computational speedup and analytical tractability.
For a conjugate mixture, we use the truncated gamma prior from Waudby-Smith et al. [2022, Theorem 2] which yields mixture wealth
$$
M _ {t} ^ {\mathrm {E B}} \doteq \left(\frac {\tau^ {\tau} e ^ {- \tau}}{\Gamma (\tau) - \Gamma (\tau , \tau)}\right) \left(\frac {1}{\tau + V _ {t}}\right) _ {1} F _ {1} (1, V _ {t} + \tau + 1, S _ {t} + V _ {t} + \tau), \tag {17}
$$
where  ${}_{1}F_{1}(\ldots)$  is Kummer's confluent hypergeometric function and  $\Gamma (\cdot ,\cdot)$  is the upper incomplete gamma function. For the hyperparameter, we use  $\tau = 1$ .
$$
\begin{array}{l} \forall t, \forall v: U _ {t} (v) - L _ {t} (v) \leqslant \\ B _ {t} + \sqrt {\frac {(\tau + V _ {t}) / t}{t}} \\ + \tilde {O} \left(\sqrt {\frac {(\tau + V _ {t}) / t}{t} \log \left(\xi_ {t} ^ {- 2} \alpha^ {- 1}\right)}\right) \tag {7} \\ + \tilde {O} \left(t ^ {- 1} \log \left(\xi_ {t} ^ {- 2} \alpha^ {- 1}\right)\right), \\ \end{array}
$$
where  $q_{t} \doteq \overline{\mathrm{CDF}}_{t}(v)$ ,  $K(q_{t}) \doteq (q_{t} - 1/2) / \log (q_{t} / 1 - q_{t})$ ;  $V_{t} = O\left(K(q_{t}) \sum_{s \leqslant t} W_{s}^{2}\right)$ ,  $B_{t} = t^{-1} \sum_{s \leqslant t} (W_{s} - 1)$ , and  $\tilde{O}()$  elides polylog  $V_{t}$  factors.
We will argue that the upper confidence radius  $U_{t}(v) - t^{-1}\sum_{s\leqslant t}W_{s}1_{X_{s}\leqslant v}$  has the desired rate. An analogous argument applies to the lower confidence radius. One difference from the non- importance-weighted case is that, to be sub-exponential, the lower bound is constructed from an upper bound on  $U_{t}^{\prime}(v) = W_{s}(1 - 1_{X_{s}\leqslant v})$  via  $L_{t}(v) - 1 - U_{t}^{\prime}(v)$ , which introduces an additional  $B_{t} = t^{-1}\sum_{s\leqslant t}(W_{s} - 1)$  term to the width. (Note, because  $\forall t:\mathbb{E}_t[W_t - 1] = 0$ , this term will concentrate, but we will simply use the realized value here.)
For a fixed bet  $\lambda$  and value  $\rho$ ,  $S_{t}$  defined in Appendix E.1.2 is sub-exponential qua Howard et al. [2021, Definition 1] and therefore from Lemma E.1 there exists an explicit mixture distribution over  $\lambda$  inducing (curved) boundary
$$
\frac {\alpha}{\eta^ {2 d}} \geqslant \mathbb {P} \left(\exists t \geqslant 1: \frac {S _ {t}}{t} \geqslant \max  \left(\frac {C (\tau)}{t}, u \left(V _ {t}; \tau , \frac {\alpha}{\eta^ {2 d}}\right)\right)\right),
$$
$$
\begin{array}{l} u \left(V _ {t}; \tau , \frac {\alpha}{\eta^ {2 d}}\right) = \sqrt {2 \left(\frac {(\tau + V _ {t}) / t}{t}\right) \log \left(\sqrt {\frac {\tau + V _ {t}}{2 \pi}} e ^ {- \frac {1}{1 2 (\tau + V _ {t}) + 1}} \left(\frac {1 + \eta^ {2 d} \alpha^ {- 1}}{C (\tau)}\right)\right)} \\ + \frac {1}{t} \log \left(\sqrt {\frac {\tau + V _ {t}}{2 \pi}} e ^ {- \frac {1}{1 2 (\tau + V _ {t}) + 1}} \left(\frac {1 + \eta^ {2 d} \alpha^ {- 1}}{C (\tau)}\right)\right), \tag {18} \\ \end{array}
$$
where  $S_{t} \doteq \overline{\mathrm{CDF}}_{t}(\rho) - t^{-1}\sum_{s\leqslant t}W_{s}1_{X_{s}\leqslant \rho}$ , and  $\tau$  is a hyperparameter to be determined further below.
$$
r _ {d} (t) = \frac {1}{\xi_ {t} \eta^ {d}} + \max \left(\frac {C (\tau)}{t}, u \left(V _ {t}; \tau , \frac {\alpha}{\eta^ {2 d}}\right)\right).
$$
Now evaluate at  $d$  such that  $\sqrt{\psi_t} < \xi_t\eta^d \leqslant \eta \sqrt{\psi_t}$  where  $\psi_t \doteq t((\tau + V_t) / t)^{-1}$ ,
$$
\begin{array}{l} r _ {d} (t) \leqslant \frac {1}{\sqrt {\psi_ {t}}} + \max \left(\frac {C (\tau)}{t}, u \left(V _ {t}; \tau , \frac {\alpha}{\eta^ {2} \xi_ {t} ^ {- 2} \psi_ {t}}\right)\right) \\ = \sqrt {\frac {(\tau + V _ {t}) / t}{t}} + \tilde {O} \left(\sqrt {\frac {(\tau + V _ {t}) / t}{t} \log \left(\xi_ {t} ^ {- 2} \alpha^ {- 1}\right)}\right) + \tilde {O} (t ^ {- 1} \log \left(\xi_ {t} ^ {- 2} \alpha^ {- 1}\right)), \\ \end{array}
$$
where  $\tilde{O}()$  elides polylog  $V_{t}$  factors. The final result is not very sensitive to the choice of  $\tau$ , and we use  $\tau = 1$  in practice.
Lemma E.1. Suppose
$$
\exp \left(\lambda S _ {t} - \psi_ {e} (\lambda) V _ {t}\right),
$$
$$
\psi_ {e} (\lambda) \doteq - \lambda - \log (1 - \lambda),
$$
is sub- $\psi_{e}$  qua Howard et al. [2021, Definition 1]; then there exists an explicit mixture distribution over  $\lambda$  with hyperparameter  $\tau > 0$  such that
$$
\begin{array}{l} \alpha \geqslant \mathbb {P} \left(\exists t \geqslant 1: \frac {S _ {t}}{t} \geqslant \max  \left(\frac {C (\tau)}{t}, u \left(V _ {t}; \tau , \alpha\right)\right)\right), \\ u \left(V _ {t}; \tau , \alpha\right) = \sqrt {2 \left(\frac {\left(\tau + V _ {t}\right) / t}{t}\right) \log \left(\sqrt {\frac {\tau + V _ {t}}{2 \pi}} e ^ {- \frac {1}{1 2 \left(\tau + V _ {t}\right) + 1}} \left(\frac {1 + \alpha^ {- 1}}{C (\tau)}\right)\right)} \\ + \frac {1}{t} \log \left(\sqrt {\frac {\tau + V _ {t}}{2 \pi}} e ^ {- \frac {1}{1 2 (\tau + V _ {t}) + 1}} \left(\frac {1 + \alpha^ {- 1}}{C (\tau)}\right)\right), \\ \end{array}
$$
$$
C (\tau) \doteq \frac {\tau^ {\tau} e ^ {- \tau}}{\Gamma (\tau) - \Gamma (\tau , \tau)},
$$
is a (curved) uniform crossing boundary.
Proof. We can form the conjugate mixture using a truncated gamma prior from Howard et al. [2021, Proposition 9], in the form from Waudby-Smith et al. [2022, Theorem 2], which is our Equation (17).
$$
M _ {t} ^ {\mathrm {E B}} \doteq \left(\frac {\tau^ {\tau} e ^ {- \tau}}{\Gamma (\tau) - \Gamma (\tau , \tau)}\right) \left(\frac {1}{\tau + V _ {t}}\right) _ {1} F _ {1} (1, V _ {t} + \tau + 1, S _ {t} + V _ {t} + \tau),
$$
where  ${}_{1}F_{1}(\ldots)$  is Kummer's confluent hypergeometric function. Using Olver et al. [2010, identity 13.6.5],
$$
{ } _ { 1 } F _ { 1 } ( 1 , a + 1 , x ) = e ^ { x } a x ^ { - a } \left( \Gamma ( a ) - \Gamma ( a , x ) \right)
$$
where  $\Gamma(a, x)$  is the (unregularized) upper incomplete gamma function. From Pinelis [2020, Theorem 1.2] we have
$$
\begin{array}{l} \Gamma (a, x) <   \frac {x ^ {a} e ^ {- x}}{x - a} \\ \Rightarrow {} _ {1} F _ {1} (1, a + 1, x) \geqslant e ^ {x} a x ^ {- a} \Gamma (a) - \frac {a}{x - a}. \\ \end{array}
$$