SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} 2 \mathbb {E} \left[ f _ {t} \left(p _ {t}\right) - f _ {t} \left(p ^ {*}\right) \right] \\ \leqslant \frac {1}{\eta} \left[ \mathbb {E} \left[ (p _ {t} - p ^ {*}) ^ {2} \right] - \mathbb {E} \left[ (p _ {t + 1} - p ^ {*}) ^ {2} \right] \right] + \eta \mathbb {E} \left[ g _ {t} ^ {2} \right] + 4 \cdot \mathbf {1} [ p ^ {*} \notin \Delta_ {t} ] \left(\frac {\delta_ {t}}{\eta} + \frac {\delta_ {t}}{2} \mathbb {E} \left[ | g _ {t} | \right]\right) \\ \leqslant \frac {1}{\eta} \left[ \mathbb {E} \left[ (p _ {t} - p ^ {*}) ^ {2} \right] - \mathbb {E} \left[ (p _ {t + 1} - p ^ {*}) ^ {2} \right] \right] + \eta \mathbb {E} \left[ g _ {t} ^ {2} \right] + 4 \cdot \mathbf {1} \left[ t <   \left(\frac {a}{2}\right) ^ {\alpha} \right] \left(\frac {\delta_ {t}}{\eta} + \frac {\delta_ {t}}{2} \mathbb {E} \left[ | g _ {t} | \right]\right) \\ \end{array}
$$
The following lemma derives bounds on the first and second (raw) moments of the gradient estimator at each iteration.
Lemma C.5. For each  $t \in [T]$ , the gradient estimates have bounded first and second moments:
$$
\begin{array}{l} \mathbb {E} \left[ g _ {t} ^ {2} \right] \leqslant 2 ^ {5} t ^ {5 / \alpha} \cdot \left(y _ {t} (1) ^ {4} + y _ {t} (0) ^ {4}\right) \\ \mathbb {E} \left[ | g _ {t} | \right] \leqslant 2 ^ {2} t ^ {2 / \alpha} \cdot \left(y _ {t} (1) ^ {2} + y _ {t} (0) ^ {2}\right) \\ \end{array}
$$
Proof. We begin by handling the  $\mathbb{E}\left[g_t^2\right]$  term. By definition of the gradient estimator, we have that the conditional expectation is at most
$$
\begin{array}{l} \mathbb {E} \left[ g _ {t} ^ {2} \mid D _ {t} \right] = p _ {t} \cdot \left(\frac {y _ {t} (1) ^ {2}}{p _ {t} ^ {3}}\right) ^ {2} + (1 - p _ {t}) \cdot \left(\frac {y _ {t} (0) ^ {2}}{(1 - p _ {t}) ^ {3}}\right) ^ {2} \\ = \frac {y _ {t} (1) ^ {4}}{p _ {t} ^ {5}} + \frac {y _ {t} (0) ^ {4}}{(1 - p _ {t}) ^ {5}} \\ \end{array}
$$
By definition of the algorithm, we have that  $p_t \in [\delta_t, 1 - \delta_t]$  at iteration  $t$ . Thus, we may invoke the bound:
$$
\begin{array}{l} \leqslant \delta_ {t} ^ {- 5} \left(y _ {t} (1) ^ {4} + y _ {t} (1) ^ {4}\right) \\ = \left[ (1 / 2) t ^ {- 1 / \alpha} \right] ^ {- 5} \cdot \left(y _ {t} (1) ^ {4} + y _ {t} (1) ^ {4}\right) \\ = 2 ^ {5} t ^ {5 / \alpha} \cdot \left(y _ {t} (1) ^ {4} + y _ {t} (1) ^ {4}\right), \\ \end{array}
$$
and the desired bound on  $\mathbb{E}[g_t^2]$  follows from applying the law of iterated expectation.
The bound on the  $\mathbb{E}\bigl [|g_t|\bigr ]$  term follows in a similar way. By definition of the gradient estimator, we have that the conditional expectation is at most
$$
\begin{array}{l} \mathbb {E} \left[ \left| g _ {t} \right| \mid D _ {t} \right] = p _ {t} \cdot \left| \frac {y _ {t} (1) ^ {2}}{p _ {t} ^ {3}} \right| + (1 - p _ {t}) \cdot \left| \frac {y _ {t} (0) ^ {2}}{(1 - p _ {t}) ^ {3}} \right| \\ = \frac {y _ {t} (1) ^ {2}}{p _ {t} ^ {2}} + \frac {y _ {t} (0) ^ {2}}{(1 - p _ {t}) ^ {2}} \\ \leqslant \delta_ {t} ^ {- 2} \left(y _ {t} (1) ^ {2} + y _ {t} (1) ^ {2}\right) \\ = \left[ (1 / 2) t ^ {- 1 / \alpha} \right] ^ {- 2} \cdot \left(y _ {t} (1) ^ {2} + y _ {t} (1) ^ {2}\right) \\ = 2 ^ {2} t ^ {2 / \alpha} \cdot \left(y _ {t} (1) ^ {2} + y _ {t} (1) ^ {2}\right), \\ \end{array}
$$
and the desired bound on  $\mathbb{E}[|g_t|]$  follows from applying the law of iterated expectation.
The following proposition bounds the expected Neyman regret for general settings of the projection parameter  $\alpha$ .
Proposition C.1. Suppose Assumption 1 holds. Then, for any choice of projection parameter  $\alpha \geqslant 2$  (possibly depending on  $T$ ) and for the step size  $\eta = \sqrt{\frac{e^{\alpha}}{T^{1 + 5 / \alpha}}}$ , a finite-sample bound on the expected Neyman regret incurred by CLIP-OGD is
$$
\mathbb {E} \left[ \mathcal {R} _ {T} \right] \leqslant \left(2 ^ {2} e ^ {a / 2} + 2 ^ {5} C ^ {4}\right) \sqrt {e ^ {\alpha} T ^ {1 + 5 / \alpha}} + 2 ^ {2} C ^ {2} e ^ {2 + a / 2} \sqrt {e ^ {\alpha} T}.
$$
Proof. By Lemma C.4, we have that the regret is at most
$$
\begin{array}{l} 2 \mathbb {E} \left[ \mathcal {R} _ {T} \right] = \sum_ {t = 1} ^ {T} 2 \mathbb {E} \left[ f _ {t} \left(p _ {t}\right) - f _ {t} \left(p ^ {*}\right) \right] \\ \leqslant \frac {1}{\eta} \sum_ {t = 1} ^ {T} \left[ \mathbb {E} \left[ (p _ {t} - p ^ {*}) ^ {2} \right] - \mathbb {E} \left[ (p _ {t + 1} - p ^ {*}) ^ {2} \right] \right] + \eta \sum_ {t = 1} ^ {T} \mathbb {E} \left[ g _ {t} ^ {2} \right] \\ + 4 \sum_ {t = 1} ^ {T} \mathbf {1} \left[ t <   \left(\frac {a}{2}\right) ^ {\alpha} \right] \left(\frac {\delta_ {t}}{\eta} + \frac {\delta_ {t}}{2} \mathbb {E} \left[ | g _ {t} | \right]\right) \\ \end{array}
$$
Using a telescoping argument, we have that the first term is bounded by
$$
\frac {1}{\eta} \sum_ {t = 1} ^ {T} \left[ \mathbb {E} \left[ (p _ {t} - p ^ {*}) ^ {2} \right] - \mathbb {E} \left[ (p _ {t + 1} - p ^ {*}) ^ {2} \right] \right] \leqslant \frac {1}{\eta} \mathbb {E} \left[ (p _ {1} - p ^ {*}) ^ {2} \right] \leqslant \frac {1}{\eta}.
$$
Using Lemma C.5 and Assumption 1, the sum in the second term may be bounded as
$$
\begin{array}{l} \sum_ {t = 1} ^ {T} \mathbb {E} \left[ g _ {t} ^ {2} \right] \leqslant \sum_ {t = 1} ^ {T} 2 ^ {5} t ^ {5 / \alpha} \left(y _ {t} (1) ^ {4} + y _ {t} (0) ^ {4}\right) \tag {Lemma C.5} \\ \leqslant 2 ^ {5} T ^ {5 / \alpha} \left(\sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4} + \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) \\ \leqslant 2 ^ {5} T ^ {5 / \alpha} \left(2 C ^ {4} T\right) \quad (\text {A s s u m p t i o n} 1) \\ = 2 ^ {6} C ^ {4} T ^ {1 + 5 / \alpha}. \\ \end{array}
$$
Next, we deal with the third term by breaking it up into two more terms. Define  $t^* = \lceil (a/2)^\alpha \rceil$ . The third term can be broken into two terms:
$$
4 \sum_ {t = 1} ^ {T} \mathbf {1} \left[ t <   \left(\frac {a}{2}\right) ^ {\alpha} \right] \left(\frac {\delta_ {t}}{\eta} + \frac {\delta_ {t}}{2} \mathbb {E} \left[ | g _ {t} | \right]\right) = \frac {4}{\eta} \sum_ {t = 1} ^ {t ^ {*} - 1} \delta_ {t} + 2 \sum_ {t = 1} ^ {t ^ {*} - 1} \delta_ {t} \mathbb {E} [ | g _ {t} | ] .
$$
The first of these two terms can be bounded in the following way. Using that  $\alpha \geqslant 2$  we have that
$$
\sum_ {t = 1} ^ {t ^ {*} - 1} \delta_ {t} = \sum_ {t = 1} ^ {t ^ {*} - 1} (1 / 2) t ^ {- 1 / \alpha}
$$
$$
\begin{array}{l} = (1 / 2) \left[ 1 + \sum_ {t = 2} ^ {t ^ {*} - 1} t ^ {- 1 / \alpha} \right] \\ \leqslant (1 / 2) \left[ 1 + \int_ {x = 1} ^ {t ^ {*} - 1} x ^ {- 1 / \alpha} d x \right] \\ = (1 / 2) \left[ 1 + \frac {\alpha}{\alpha - 1} \cdot \left(\left(t ^ {*} - 1\right) ^ {(1 - 1 / \alpha)} - 1\right) \right] \\ \leqslant \left(t ^ {*} - 1\right) ^ {\left(1 - 1 / \alpha\right)} \\ \leqslant (a / 2) ^ {\alpha (1 - 1 / \alpha)} \\ = (a / 2) ^ {\alpha - 1}. \\ \end{array}
$$
The second term can be bounded using Lemma C.5 as follows:
$$
\begin{array}{l} \sum_ {t = 1} ^ {t ^ {*} - 1} \delta_ {t} \mathbb {E} [ | g _ {t} | ] \leqslant \sum_ {t = 1} ^ {t ^ {*} - 1} (1 / 2) t ^ {- 1 / \alpha} 2 ^ {2} t ^ {2 / \alpha} \left(y _ {t} (1) ^ {2} + y _ {t} (0) ^ {2}\right) \tag {Lemma C.5} \\ = 2 \sum_ {t = 1} ^ {t ^ {*} - 1} t ^ {1 / \alpha} \left(y _ {t} (1) ^ {2} + y _ {t} (0) ^ {2}\right) \\ \leqslant 2 \left(\sum_ {t = 1} ^ {t ^ {*} - 1} t ^ {2 / \alpha}\right) ^ {1 / 2} \left[ \left(\sum_ {t = 1} ^ {t ^ {*} - 1} y _ {t} (1) ^ {4}\right) ^ {1 / 2} + \left(\sum_ {t = 1} ^ {t ^ {*} - 1} y _ {t} (0) ^ {4}\right) ^ {1 / 2} \right], \quad (\text {C a u c h y - S c h w a r z}) \\ \end{array}
$$
where the last inequality follows from Cauchy-Schwarz. By extending the sum involving the outcomes to all units, we obtain an upper bound on which we can apply the bounded moment assumption:
$$
\begin{array}{l} \leqslant 2 \left(\sum_ {t = 1} ^ {t ^ {*} - 1} t ^ {2 / \alpha}\right) ^ {1 / 2} \left[ \left(\sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) ^ {1 / 2} + \left(\sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) ^ {1 / 2} \right] \\ \leqslant 2 \left(\sum_ {t = 1} ^ {t ^ {*} - 1} t ^ {2 / \alpha}\right) ^ {1 / 2} \left[ 2 \left(C ^ {4} T\right) ^ {1 / 2} \right] \tag {Assumption1} \\ \end{array}
$$
What remains in this step is to bound the first term above. By replacing each of the  $t$  in the sum with  $t^*$ , we obtain the following upper bound:
$$
\begin{array}{l} \leqslant 2 \left(t ^ {*} - 1\right) ^ {\left(1 / 2 + 1 / \alpha\right)} \left[ 2 \left(C ^ {4} T\right) ^ {1 / 2} \right] \\ \leqslant 2 \left(\left(a / 2\right) ^ {\alpha}\right) ^ {(1 / 2 + 1 / \alpha)} \left[ 2 \left(C ^ {4} T\right) ^ {1 / 2} \right] \\ = 2 ^ {2} C ^ {2} \left(\frac {a}{2}\right) ^ {(1 + \alpha / 2)} T ^ {1 / 2}. \\ \end{array}
$$
Using the above work, we have that the overall regret is bounded as
$$
\begin{array}{l} 2 \mathbb {E} \left[ \mathcal {R} _ {T} \right] \leqslant \frac {1}{\eta} + \eta 2 ^ {6} C ^ {4} T ^ {1 + 5 / \alpha} + \frac {4}{\eta} \left(\frac {a}{2}\right) ^ {(\alpha - 1)} + 2 ^ {3} C ^ {2} \left(\frac {a}{2}\right) ^ {(1 + \alpha / 2)} T ^ {1 / 2} \\ = \frac {1}{\eta} \left(1 + 4 \left(\frac {a}{2}\right) ^ {(\alpha - 1)}\right) + \eta 2 ^ {6} C ^ {4} T ^ {1 + 5 / \alpha} + 2 ^ {3} C ^ {2} \left(\frac {a}{2}\right) ^ {(1 + \alpha / 2)} T ^ {1 / 2}. \\ \end{array}
$$
Next, we separate the constant  $a$  from the projection parameter  $\alpha$ . To this end, we use the inequality  $y^{r} \leqslant e^{1 + y} \cdot e^{r}$  for all  $y \in \mathbb{R}$  and  $r \geqslant 0$ , to obtain
$$
\left(\frac {a}{2}\right) ^ {(\alpha - 1)} \leqslant e ^ {1 + a / 2} e ^ {\alpha - 1} = e ^ {a / 2} e ^ {\alpha} \quad \text {a n d} \quad \left(\frac {a}{2}\right) ^ {(1 + \alpha / 2)} \leqslant e ^ {1 + a / 2} e ^ {1 + \alpha / 2} = e ^ {2 + a / 2} e ^ {\alpha / 2},
$$
where we have used that  $a \geqslant 2$ . Substituting these back into the above, we have that the regret bound is Substituting this quantities into the above, we have that the regret bound is
$$
2 \mathbb {E} [ \mathcal {R} _ {T} ] \leqslant \frac {1}{\eta} \left(1 + 4 e ^ {a / 2} e ^ {\alpha}\right) + \eta 2 ^ {6} C ^ {4} T ^ {1 + 5 / \alpha} + 2 ^ {3} C ^ {2} e ^ {2 + a / 2} e ^ {\alpha / 2} T ^ {1 / 2}
$$
$$
\leqslant \frac {1}{\eta} 2 ^ {3} e ^ {a / 2} e ^ {\alpha} + \eta 2 ^ {6} C ^ {4} T ^ {1 + 5 / \alpha} + 2 ^ {3} C ^ {2} e ^ {2 + a / 2} e ^ {\alpha / 2} T ^ {1 / 2}
$$
By setting  $\eta = \sqrt{\frac{e^{\alpha}}{T^{1 + 5 / \alpha}}}$  to minimize the bound above, we have that the expected Neyman regret is bounded as
$$
= (2 ^ {3} e ^ {a / 2} + 2 ^ {6} C ^ {4}) \sqrt {e ^ {\alpha} T ^ {1 + 5 / \alpha}} + 2 ^ {3} C ^ {2} e ^ {2 + a / 2} \sqrt {e ^ {\alpha} T}.
$$
where we have used that and the result follows by dividing both sides by 2.
Proposition C.1 demonstrates that many different values of  $\alpha$  will guarantee sublinear expected Neyman regret. For example, setting  $\alpha$  to be a constant satisfying  $\alpha > 5$  will ensure sublinear expected Neyman regret. However, by tuning  $\alpha$  according to the analysis above, we can achieve  $\mathcal{O}(\sqrt{T\log(T)})$  expected Neyman regret, as demonstrated by Theorem 4.2.
Theorem 4.2*. Under Assumption 1 the parameter values  $\eta = \sqrt{1 / T}$  and  $\alpha = \sqrt{5\log(T)}$  ensure the expected Neyman regret of CLIP-OGD is bounded as
$$
\mathbb {E} \left[ \mathcal {R} _ {T} \right] \leqslant \left(2 ^ {2} e ^ {a / 2} + 2 ^ {5} C ^ {4} + 2 ^ {2} C ^ {2} e ^ {2 + a / 2}\right) \cdot \sqrt {T} \cdot \exp (\sqrt {5 \log (T)}),
$$
which implies that  $\mathbb{E}\big[\mathcal{R}_T\big]\leqslant \tilde{\mathcal{O}}\big(\sqrt{T}\big)$
Proof. Observe that for  $\alpha = \sqrt{5\log(T)}$ , we have that the step size posited in Proposition C.1 (i.e.  $\eta = \sqrt{\frac{e^{\alpha}}{T^{1 + 5 / \alpha}}}$ ) is equal to  $\eta = \sqrt{1 / T}$ . Thus, by rearranging terms and using the result of Proposition C.1, we have that expected Neyman regret is bounded as
$$
\begin{array}{l} \mathbb {E} \left[ \mathcal {R} _ {T} \right] \leqslant \left(2 ^ {2} e ^ {a / 2} + 2 ^ {5} C ^ {4}\right) \sqrt {e ^ {\alpha} T ^ {1 + 5 / \alpha}} + 2 ^ {2} C ^ {2} e ^ {2 + a / 2} \sqrt {e ^ {\alpha} T} \\ = (2 ^ {2} e ^ {a / 2} + 2 ^ {5} C ^ {4}) \sqrt {T} \cdot \sqrt {e ^ {\alpha} T ^ {5 / \alpha}} + 2 ^ {2} C ^ {2} e ^ {2 + a / 2} \sqrt {T} \cdot \sqrt {e ^ {\alpha}} . \\ \end{array}
$$
The difficulty is now to find which setting of  $\alpha$  will make this bound smallest. Observe that the real tension is in the first term and we can re-write the relevant part of this term as
$$
\sqrt {e ^ {\alpha} T ^ {5 / \alpha}} = \left[ \exp (\alpha + \log (T ^ {5 / \alpha})) \right] ^ {1 / 2} = \left[ \exp (\alpha + (5 / \alpha) \log (T)) \right] ^ {1 / 2}.
$$
To minimize this term, we select  $\alpha = \sqrt{5\log(T)}$  which results in
$$
\sqrt {e ^ {\alpha} T ^ {5 / \alpha}} = \exp \big (\sqrt {5 \log (T)} \big) .
$$
Likewise, this choice of  $\alpha$  results in  $\sqrt{e^{\alpha}} \leqslant e^{\alpha} = \exp(\sqrt{5\log(T)})$ . Putting this together yields the desired finite sample regret bound:
$$
\mathbb {E} \left[ \mathcal {R} _ {T} \right] \leqslant \left(2 ^ {2} e ^ {a / 2} + 2 ^ {5} C ^ {4} + 2 ^ {2} C ^ {2} e ^ {2 + a / 2}\right) \cdot \sqrt {T} \cdot \exp (\sqrt {5 \log (T)}).
$$
The result follows by observing that the terms inside the parenthesis are constant by Assumption 1 and the function  $\exp (\sqrt{5}\log (T))$  is subpolynomial.
C.2 Selecting Parameters When Moment Bounds are Known
We briefly remark on how to select the step size parameter  $\eta$  when the experimenter can correctly specify the constants  $C\geq c$  used in the Assumption 1. The proof of Proposition C.1 shows that for general parameters, the Neyman regret may be bounded as
$$
\mathbb {E} [ \mathcal {R} _ {T} ] \leqslant \frac {1}{\eta} 2 ^ {2} e ^ {a / 2} e ^ {\alpha} + \eta 2 ^ {5} C ^ {4} T ^ {1 + 5 / \alpha} + 2 ^ {2} C ^ {2} e ^ {2 + a / 2} e ^ {\alpha / 2} T ^ {1 / 2}.
$$
To optimize the step size with respect to these constants, one would choose  $\alpha = \sqrt{5\log(T)}$  and
$$
\eta = \frac {e ^ {\frac {1}{4} \cdot (1 + C / c)}}{2 \sqrt {2} C ^ {2}} \cdot \frac {1}{\sqrt {T}} ,
$$
where we have used that  $a = 1 + C / c$ . When these moment bounds are correctly specified, this choice of step size will likely yield improved convergence rates, as our bound on the Neyman regret will have a factor of  $C^2$  rather than  $C^4$ .
D Analysis for Inference in Large Samples
In this section, we provide the necessary statistical tools for constructing asymptotically valid confidence intervals. This can be done in two main steps. First, in Section D.1 we construct a conservative variance estimator which we show is consistent in probability. Then, in Section D.2, we show that the resulting Chebyshev-type intervals are asymptotically valid.
While the main paper used capital letters  $P_{t}$  and  $G_{t}$  to signify that the treatment probability and gradient estimator were random variables, we use lower case letters  $p_{t}$  and  $g_{t}$  in the appendix for the purposes of more aesthetically appealing proofs. Throughout the analysis, we use the parameter settings  $\eta = \sqrt{1 / T}$  and  $\alpha = \sqrt{5\log(T)}$ , which are recommended in the main paper. However, we suspect that many of our results will go through for the class of parameters  $\eta = \sqrt{\frac{e^{\alpha}}{T^{1 + 5 / \alpha}}}$  and  $\alpha > 5$  which appear in Proposition C.1.
The following lemma shows that under Assumptions 1 and 2, the variance of the adaptive Horvitz-Thompson estimator under CLIP-OGD achieves the parametric rate.
Lemma D.1. Assumptions 1 and 2, the variance of the adaptive Horvitz-Thompson estimator under CLIP-OGD achieves the parametric rate:  $\mathrm{Var}(\hat{\tau}) = \Theta (1 / T)$
Proof. Theorem 4.1 shows that under Assumption 1, they Neyman ratio is order equivalent to the  $1 / T$ -scaled expected Neyman regret, i.e.  $\kappa_{T} = \Theta ((1 / T)\mathbb{E}[\mathcal{R}_{T}])$ . Theorem 4.2 shows that under these assumptions, CLIP-OGD achieves sublinear expected Neyman regret  $\mathbb{E}[\mathcal{R}_T] = o(T)$  which implies that  $\lim \sup \kappa_T\leqslant 0$ . Likewise, Assumption 2 states that the negative expected Neyman regret is sublinear,  $-\mathbb{E}[\mathcal{R}_T] = o(T)$ , which implies that  $\lim \inf \kappa_T\geqslant 0$ . Thus, we have that the Neyman ratio converges to zero, e.g.  $\lim \kappa_T = 0$ .
By recalling the definition of the Neyman ratio, we have that
$$
0 = \lim  _ {T \rightarrow \infty} \kappa_ {T} = \lim  _ {T \rightarrow \infty} \frac {V - V _ {\mathrm {N}}}{V _ {\mathrm {N}}} = \lim  _ {T \rightarrow \infty} \frac {T \cdot V - T \cdot V _ {\mathrm {N}}}{T \cdot V _ {\mathrm {N}}} .
$$
Proposition B.1 demonstrates that  $T \cdot V_{\mathrm{N}} = \Theta(1)$ . Together with the above, this implies that  $T \cdot V = \Theta(1)$ .
The following lemma shows that under the recommended parameter settings, the (random) treatment probabilities are bounded away from zero and one.
Lemma D.2. When  $\alpha = \sqrt{5\log(T)}$ , we have that for all iterations  $t \in [T]$ , the inverse of projection parameter is bounded:
$$
\frac {1}{\delta_ {t}} \leqslant 2 \exp (\sqrt {\log (T ^ {1 / 5})}) = \widetilde {\mathcal {O}} (1).
$$
Proof. A uniform upper bound on the inverse of the projection parameters is
$$
\frac {1}{\delta_ {t}} \leqslant \frac {1}{\delta_ {T}} = \frac {1}{(1 / 2) T ^ {- 1 / \alpha}} = 2 T ^ {1 / \alpha} = 2 \exp (\frac {1}{\alpha} \log (T)) = 2 \exp (\sqrt {\log (T ^ {1 / 5})}).
$$
To complete the proof, observe that the function  $h(T) = \exp(\sqrt{1/5} \cdot \sqrt{\log(T)})$  is subpolynomial, so that we can write it as  $\widetilde{\mathcal{O}}(1)$ .
D.1 Conservative Variance Estimator (Theorem 5.1)
In this section, we prove Theorem 5.1 which establishes that the normalized variance estimator is converges in probability to the normalized variance upper bound at a  $\widetilde{\mathcal{O}}_p(T^{-1/2})$  rate. Before continuing, let us review the relevant quantities. Recall that the Neyman variance and the corresponding upper bound are given by
$$
T \cdot V _ {\mathrm {N}} = 2 (1 + \rho) S (1) S (0) \quad \text {a n d} \quad T \cdot \mathbf {V B} = 4 S (1) S (0),
$$
where the second moments  $S(1)$  and  $S(0)$  are defined as
$$
S (1) ^ {2} = \frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} \quad \text {a n d} \quad S (0) ^ {2} = \frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {2}.
$$
Our variance estimator is defined as
$$
T \cdot \widehat {\mathrm {V B}} = \sqrt {\widehat {A (1)} \cdot \widehat {A (0)}} \quad \text {w h e r e} \quad \widehat {A (1)} = \frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} \frac {\mathbf {1} [ Z _ {t} = 1 ]}{p _ {t}} \quad \text {a n d} \quad \widehat {A (0)} = \frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {2} \frac {\mathbf {1} [ Z _ {t} = 0 ]}{1 - p _ {t}}.
$$
The random variables  $\widehat{A(1)}$  and  $\widehat{A(0)}$  are unbiased estimates of  $S(1)^2$  and  $S(0)^2$  which are based on the Horvitz-Thompson principle. However, the fact that the estimators  $\widehat{A(1)}$  and  $\widehat{A(0)}$  are not independent and the square root is introduced means that the variance estimator  $\widehat{\mathrm{VB}}$  is not an unbiased estimator for the variance bound VB. Even so, we will show that the variance estimator is consistent for the variance bound. For aesthetic considerations, we define  $A(1) = S(1)^2$  and  $A(0) = S(0)$  so that  $\widehat{A(1)}$  is an estimator for  $A(1)$  and  $\widehat{A(0)}$  is an estimator for  $A(0)$ .
Our general approach will follow in two steps. First, by bounding its bias and variance, we will show that  $\widehat{A(1)}\cdot \widehat{A(0)} - A(1)A(0)$  converges as  $\tilde{\mathcal{O}}_p(T^{-1 / 2})$ . Next, by appealing to a quantitative Continuous Mapping Theorem, we will argue that the error  $\sqrt{\widehat{A(1)}\cdot\widehat{A(0)}} -\sqrt{A(1)A(0)}$  converges at the same rate. By definition, this is exactly the error of the normalized variance estimator to the normalized variance bound, i.e.  $T\cdot \widehat{\mathrm{VB}} -T\cdot \mathrm{VB}$ .
Before continuing, let us define new auxiliary random variables. For each  $t \in [T]$ , we define the variables  $r_t$  and  $q_t$  as
$$
r _ {t} = \frac {\mathbf {1} \left[ z _ {t} = 1 \right]}{p _ {t}} \quad \text {a n d} \quad q _ {t} = \frac {\mathbf {1} \left[ z _ {t} = 0 \right]}{1 - p _ {t}}.
$$
Below are basic facts about these auxiliary random variables.
Lemma D.3. The auxiliary random variables satisfy the following properties:
1.  $\mathbb{E}[r_tq_s] = \mathbf{1}[t\neq s]$
2.  $\mathbb{E}[r_t^2\mid D_t]\leqslant \frac{1}{\delta_t}$  and  $\mathbb{E}[q_t^2\mid D_t]\leqslant \frac{1}{\delta_t}$
3. The covariance  $\mathrm{Cov}(r_t q_s, r_\ell q_k)$  behave in the following ways:
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) = 0 \quad i f t = s o r \ell = k
$$
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) = 0 \quad i f t \neq s a n d \ell \neq k a n d \{t, s \} \cap \{\ell , k \} = \varnothing
$$
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) = - 1 \quad i f t \neq s a n d \ell \neq k a n d (t = k o r s = \ell)
$$
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) \leqslant \frac {1}{\delta_ {t}} - 1 \quad i f t \neq s a n d \ell \neq k a n d t = \ell a n d s \neq k
$$
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) \leqslant \frac {1}{\delta_ {s}} - 1 \quad i f t \neq s a n d \ell \neq k a n d t \neq \ell a n d s = k
$$
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) \leqslant \frac {1}{\delta_ {t} \delta_ {s}} - 1 \quad i f t \neq s a n d \ell \neq k a n d t = \ell a n d s = k
$$
Proof. First, we show that  $\mathbb{E}[r_t q_s] = \mathbf{1}[t \neq s]$ . Let  $t, s \in [T]$  and suppose that  $t \neq s$ . Without loss of generality, suppose that  $t > s$ . Then by using iterated expectation, we have that
$$
\mathbb {E} \left[ r _ {t} q _ {s} \right] = \mathbb {E} \left[ q _ {s} \mathbb {E} \left[ r _ {t} \mid D _ {t} \right] \right] = \mathbb {E} \left[ q _ {s} \right] = 1.
$$
Otherwise, if  $t = s$ , then  $r_t q_t = (\mathbf{1}[z_t = 1] / p_t) \cdot (\mathbf{1}[z_s = 0] / (1 - p_t)) = 0$  so that  $\mathbb{E}[r_t q_t] = 0$ .
Next, we show that  $\mathbb{E}[r_t^2\mid D_t]\leqslant \frac{1}{\delta_t}$  and  $\mathbb{E}[q_t^2\mid D_t]\leqslant \frac{1}{\delta_t}$ . Observe that  $\mathbb{E}[r_t^2\mid D_t] = p_t(1 / p_t^2) = 1 / p_t\leqslant 1 / \delta_t$ , where the inequality follows by definition of Algorithm 1. A similar argument shows that  $\mathbb{E}[q_t^2\mid D_t]\leqslant 1 / \delta_t$ .
Finally, we establish the covariance terms one by one. We do this in order of the cases that they were presented in.
Case 1 ( $t = s$ ) or  $(\ell = k)$ : If  $t = s$ , then  $r_t q_t$  is almost surely zero, as argued above. Likewise, if  $\ell = k$  then  $r_\ell q_\ell$  is almost surely zero. In either of these cases, we have that  $\mathrm{Cov}(r_t q_s, r_\ell q_k) = 0$ .
Case 2 ( $t \neq s$ ) and  $(\ell \neq k)$  and  $\{t, s\} \cap \{\ell, k\} = \emptyset$ : Note that in this case, all the indices  $t, s, \ell$ , and  $k$  are distinct. Without loss of generality, suppose that  $t < s < \ell < k$ . A repeated use of the iterated expectation yields that
$$
\mathbb {E} \left[ r _ {t} q _ {s} r _ {\ell} q _ {s} \right] = \mathbb {E} \left[ r _ {t} q _ {s} r _ {\ell} \mathbb {E} \left[ q _ {s} \mid D _ {s} \right] \right] = \mathbb {E} \left[ r _ {t} q _ {s} r _ {\ell} \right] = \mathbb {E} \left[ r _ {t} q _ {s} \mathbb {E} \left[ r _ {\ell} \mid D _ {r} \right] \right] = \mathbb {E} \left[ r _ {t} q _ {s} \right] = \dots = 1.
$$
Because all terms of distinct, we have that  $\mathbb{E}[r_t q_s] = 1$  and  $\mathbb{E}[r_\ell q_k] = 1$ . Thus, the covariance is equal to
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) = \mathbb {E} \left[ r _ {t} q _ {s}, r _ {\ell} q _ {k} \right] - \mathbb {E} \left[ r _ {t} q _ {s} \right] \cdot \mathbb {E} \left[ r _ {\ell} q _ {k} \right] = 1 - 1 = 0.
$$