SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Case 3 ( $t \neq s$ ) and ( $\ell \neq k$ ) and ( $t = k$  or  $s = \ell$ ): Suppose that  $t = k$ . In this case, observe that  $r_t q_k$  is zero almost surely. Thus,  $\mathbb{E}[r_t q_s r_\ell q_k] = 0$ . On the other hand,  $t \neq s$  and  $\ell \neq k$  so that  $\mathbb{E}[r_t q_s] = \mathbb{E}[r_\ell q_k] = 1$ . This means that
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) = \mathbb {E} \left[ r _ {t} q _ {s} r _ {\ell} q _ {k} \right] - \mathbb {E} \left[ r _ {t} q _ {s} \right] \cdot \mathbb {E} \left[ r _ {\ell} q _ {k} \right] = 0 - 1 \cdot 1 = - 1.
$$
The same argument shows that  $s = \ell$  yields the same result.
Case 4 ( $t \neq s$ ) and ( $\ell \neq k$ ) and  $t = \ell$  and  $s \neq k$ ): We begin by computing the expectation of the product of these four terms. In this case, we have that  $t = \ell$  so that
$$
\mathbb {E} \left[ r _ {t} q _ {s} r _ {\ell} q _ {k} \right] = \mathbb {E} \left[ r _ {t} ^ {2} q _ {s} q _ {k} \right].
$$
By assumption, the indices  $t$ ,  $s$ , and  $k$  are all distinct. Our approach will be to obtain an upper bound on the expectation of the project of these three terms by iterated expectation. In particular, the inequality we will use is that  $\mathbb{E}[r_t^2 \mid D_t] \leqslant 1 / \delta_t$ . Suppose for now that  $s < k < t$ . In this case, we use iterated expectation to get
$$
\mathbb {E} [ r _ {t} q _ {s} r _ {\ell} q _ {k} ] = \mathbb {E} [ q _ {s} q _ {k} \mathbb {E} [ r _ {t} \mid D _ {t} ] ] \leqslant \frac {1}{\delta_ {t}} \mathbb {E} [ q _ {s} q _ {k} ] = \frac {1}{\delta_ {t}} \mathbb {E} [ q _ {s} \mathbb {E} [ q _ {k} \mid D _ {k} ] ] = \frac {1}{\delta_ {t}} \mathbb {E} [ q _ {s} ] = \frac {1}{\delta_ {t}}.
$$
In the above, we have assumed that  $s < k < t$ , but the same iterated expectation technique can be applied regardless of the ordering of these indices, because they are unique. Thus, in this case,  $\mathbb{E}[r_t q_s r_\ell q_k] \leqslant 1 / \delta_t$ . Because  $t \neq s$  and  $\ell \neq k$ , we have that  $\mathbb{E}[r_t q_s] = \mathbb{E}[r_\ell q_k] = 1$ . This means that
$$
\operatorname {C o v} (r _ {t} q _ {s}, r _ {\ell} q _ {k}) = \mathbb {E} [ r _ {t} q _ {s} r _ {\ell} q _ {k} ] - \mathbb {E} [ r _ {t} q _ {s} ] \cdot \mathbb {E} [ r _ {\ell} q _ {k} ] \leqslant \frac {1}{\delta_ {t}} - 1.
$$
Case 5 ( $t \neq s$ ) and ( $\ell \neq k$ ) and  $t \neq \ell$  and  $s = k$ ): We begin by computing the expectation of the product of these four terms. In this case, we have that  $s = k$  so that
$$
\mathbb {E} \big [ r _ {t} q _ {s} r _ {\ell} q _ {k} \big ] = \mathbb {E} \big [ r _ {t} r _ {\ell} q _ {s} ^ {2} \big ] .
$$
By assumption, the indices  $t,\ell$  , and  $s$  are all distinct. Using a similar argument as the previous case, we can use iterated expectation together with the bound  $\mathbb{E}[q_s^2\mid D_s]\leqslant 1 / \delta_s^2$  to obtain that  $\mathbb{E}[r_t q_s r_\ell q_k]\leqslant 1 / \delta_s^2$  . Because  $t\neq s$  and  $\ell \neq k$  we have that  $\mathbb{E}[r_t q_s] = \mathbb{E}[r_\ell q_k] = 1$  . This means that
$$
\operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) = \mathbb {E} \left[ r _ {t} q _ {s} r _ {\ell} q _ {k} \right] - \mathbb {E} \left[ r _ {t} q _ {s} \right] \cdot \mathbb {E} \left[ r _ {\ell} q _ {k} \right] \leqslant \frac {1}{\delta_ {s}} - 1.
$$
Case 6 ( $t \neq s$ ) and  $(\ell \neq k)$  and  $t = \ell$  and  $s = k$ : Suppose without loss of generality that  $t > s$ . In this case, we can bound the product of the expectation of these four terms using iterated expectation and the proven inequalities:
$$
\mathbb {E} \big [ r _ {t} q _ {s} r _ {\ell} q _ {k} \big ] = \mathbb {E} \big [ r _ {t} ^ {2} q _ {s} ^ {2} \big ] = \mathbb {E} \big [ q _ {s} ^ {2} \mathbb {E} \big [ r _ {t} ^ {2} \mid D _ {t} \big ] \big ] \leqslant \frac {1}{\delta_ {t}} \mathbb {E} \big [ q _ {s} ^ {2} \big ] \leqslant \frac {1}{\delta_ {t} \delta_ {s}}.
$$
Because  $t \neq s$ , we have that  $\mathbb{E}[r_t q_s] = \mathbb{E}[r_\ell, q_k] = 1$ . Thus, the covariance is bounded by
$$
\operatorname {C o v} (r _ {t} q _ {s}, r _ {\ell} q _ {k}) = \mathbb {E} [ r _ {t} q _ {s} r _ {\ell} q _ {k} ] - \mathbb {E} [ r _ {t} q _ {s} ] \cdot \mathbb {E} [ r _ {\ell} q _ {k} ] \leqslant \frac {1}{\delta_ {t} \delta_ {s}} - 1.
$$
First, we show that the difference between the expected value of  $\widehat{A(1)A(0)}$  and the target  $A(1)A(0)$  is decreasing at a linear rate in  $T$ .
Proposition D.1. The absolute bias of the estimated crossing term  $\widehat{A(1)A(0)}$  to its target value  $A(1)A(0)$  is at most
$$
\left| \mathbb {E} \left[ \widehat {A (1)} \widehat {A (0)} \right] - S (1) ^ {2} S (0) ^ {2} \right| \leqslant \frac {C ^ {4}}{T}.
$$
Proof. Using Lemma D.3, we can calculate the expectation of the product  $\widehat{A(1)A(0)}$  as
$$
\begin{array}{l} \mathbb {E} \left[ \widehat {A (1) A (0)} \right] = \mathbb {E} \left[ \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} r _ {t}\right) \left(\frac {1}{T} \sum_ {s = 1} ^ {T} y _ {s} (0) ^ {2} q _ {s}\right) \right] \\ = \frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2} \mathbb {E} \left[ r _ {t} q _ {s} \right] \\ \end{array}
$$
$$
\begin{array}{l} = \frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2} - \frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2} \\ = \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2}\right) \left(\frac {1}{T} \sum_ {s = 1} ^ {T} y _ {s} (0) ^ {2}\right) - \frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2} \\ = A (1) A (0) - \frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2}. \\ \end{array}
$$
We complete the proof by using Cauchy-Schwarz and Assumption 1, to bound the absolute bias as
$$
\begin{array}{l} \left| \mathbb {E} \left[ \widehat {A (1)} \widehat {A (0)} \right] - A (1) A (0) \right| = \frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2} \\ \leqslant \frac {1}{T ^ {2}} \left[ \left(\sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) \cdot \left(\sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) \right] ^ {1 / 2} \\ = \frac {1}{T} \left[ \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) ^ {1 / 4} \cdot \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) ^ {1 / 4} \right] ^ {2} \\ \leqslant \frac {C ^ {4}}{T} . \\ \end{array}
$$
Next, we show that the variance of  $\widehat{A(1)}\widehat{A(0)}$  is going to zero at a sufficiently fast rate.
Proposition D.2. The variance of  $\widehat{A(1)A(0)}$  is bounded as
$$
\operatorname {V a r} (\widehat {A (1)} \widehat {A (0)}) \leqslant \frac {4 e ^ {\sqrt {\log \left(T ^ {1 / 5}\right)}} C ^ {8}}{T} + \frac {4 C ^ {8} e ^ {2 \sqrt {\log \left(T ^ {1 / 5}\right)}}}{T ^ {2}} = \tilde {\mathcal {O}} \left(\frac {1}{T}\right).
$$
Proof. We begin by decomposing the variance of  $\widehat{A(1)A(0)}$  into covariances of products of the auxiliary random variables  $r_t$  and  $q_s$ . To this end, observe that
$$
\begin{array}{l} \operatorname {V a r} (\widehat {A (1) A (0)}) = \operatorname {V a r} \left(\frac {1}{T ^ {2}} \sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {T} y _ {t} (1) ^ {2} y _ {t} (0) ^ {2} r _ {t} q _ {s}\right) \\ = \frac {1}{T ^ {4}} \sum_ {t = 1} ^ {T} \sum_ {s = 1} ^ {T} \sum_ {\ell = 1} ^ {T} \sum_ {k = 1} ^ {T} y _ {t} (1) ^ {2} y _ {s} (0) ^ {2} y _ {\ell} (1) ^ {2} y _ {k} (0) ^ {2} \operatorname {C o v} \left(r _ {t} q _ {s}, r _ {\ell} q _ {k}\right) \\ \end{array}
$$
Next, we will use the result of Lemma D.3 to handle the individual covariance terms. In particular, the first six types of terms, as described in Lemma D.3. The first three types of terms are at most 0, so we may discard them from the sum, as they contribute no positive value. The last three terms have upper bounds, which we use here to obtain the following upper bound:
$$
\begin{array}{l} \leqslant \underbrace {\frac {1}{T ^ {4}} \sum_ {t = 1} ^ {T} \sum_ {s \in [ T ] \backslash \{t \}} \sum_ {k \in [ T ] \backslash \{t , s \}} y _ {t} (1) ^ {4} y _ {s} (0) ^ {2} y _ {k} (0) ^ {2} \left(\frac {1}{\delta_ {t}} - 1\right)} _ {\text {T e r m} T _ {1}} \\ + \underbrace {\frac {1}{T ^ {4}} \sum_ {t = 1} ^ {T} \sum_ {s \in [ T ] \backslash \{t \}} \sum_ {\ell \in [ T ] \backslash \{t , s \}} y _ {t} (1) ^ {2} y _ {\ell} (1) ^ {2} y _ {s} (0) ^ {4} \left(\frac {1}{\delta_ {s}} - 1\right)} _ {\text {T e r m} T _ {2}} \\ + \underbrace {\frac {1}{T ^ {4}} \sum_ {t = 1} ^ {T} \sum_ {s \neq t} y _ {t} (1) ^ {4} y _ {s} (0) ^ {4} \left(\frac {1}{\delta_ {t} \delta_ {s}} - 1\right)} _ {\text {T e r m} T _ {3}} \\ \end{array}
$$
Our goal will now be to bound each of these terms individually.
Terms 1 and 2: Terms 1 and 2 are similar and will be handled in the same way. Let's begin with Term 1. Observe that by Lyapunov's inequality, the moment assumptions, and Lemma D.2, we have that
$$
\begin{array}{l} T _ {1} \leqslant \frac {1}{\delta_ {T} \cdot T ^ {4}} \left(\sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) \left(\sum_ {t = 1} ^ {T} y _ {t} (0) ^ {2}\right) ^ {2} \\ \leqslant \frac {2 e ^ {\sqrt {\log \left(T ^ {1 / 5}\right)}}}{T} \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {2}\right) ^ {2} (Lemma D.2) \\ = \frac {2 e ^ {\sqrt {\log (T ^ {1 / 5})}}}{T} \left[ \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) ^ {1 / 4} \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {2}\right) ^ {1 / 2} \right] ^ {4} \\ \leqslant \frac {2 e ^ {\sqrt {\log (T ^ {1 / 5})}}}{T} \left[ \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) ^ {1 / 4} \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) ^ {1 / 4} \right] ^ {4} \quad (\text {L y a p u n o v ' s i n e q u a l i t y}) \\ \leqslant \frac {2 e ^ {\sqrt {\log (T ^ {1 / 5})}} C ^ {8}}{T} (Assumption 1) \\ \end{array}
$$
A similar argument shows that  $T_{2} \leqslant (2e^{\sqrt{\log(T^{1/5})}}C^{8}) / T$ .
Term 3: The third term may be upper bounded using Lemma D.2 and the moment assumptions. Namely,
$$
\begin{array}{l} T _ {3} \leqslant \frac {1}{\delta_ {T} ^ {2} T ^ {4}} \left(\sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) \left(\sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) \\ = \frac {\left(2 e ^ {\sqrt {\log \left(T ^ {1 / 5}\right)}}\right) ^ {2}}{T ^ {2}} \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (1) ^ {4}\right) \left(\frac {1}{T} \sum_ {t = 1} ^ {T} y _ {t} (0) ^ {4}\right) \tag {Lemma D.2} \\ \leqslant \frac {4 C ^ {8} e ^ {2 \sqrt {\log (T ^ {1 / 5})}}}{T ^ {2}} \quad \text {(A s s u m p t i o n 1)} \\ \end{array}
$$
Taken together, this shows that
This establishes the following corollary, which shows that the error  $\widehat{A(1)}\widehat{A(0)} - A(1)A(0)$  is going to zero at a near-parametric rate, which follows from by Chebyshev's inequality from Propositions D.1 and D.2.
Corollary D.1. The following error goes to zero:  $\widehat{A(1)A(0)} - A(1)A(0) = \widetilde{\mathcal{O}}_p\big(T^{-1 / 2}\big)$ .
Using the results derived above, we are ready to prove Theorem 5.1, which we restate here for convenience.
Theorem 5.1. Under Assumptions 1 and 2, and the parameters stated in Theorem 4.2, the error of the normalized variance estimator under CLIP-OGD is  $T \cdot \widehat{VB} - T \cdot VB = \tilde{\mathcal{O}}_p(T^{-1/2})$ .
Proof of Theorem 5.1. Recall that the variance estimator and the variance bound are equal to  $T \cdot \mathbf{VB} = 4S(1)S(0) = 4\sqrt{A(1)A(0)}$  and  $T \cdot \widehat{\mathbf{VB}} = 4\sqrt{\widehat{A(1)A(0)}}$  so that the error is given by
$$
T \cdot \widehat {\mathbf {V B}} - T \cdot \mathbf {V B} = 4 \left[ \sqrt {A (1) A (0)} - \sqrt {\widehat {A (1)} \widehat {A (0)}} \right].
$$
Corollary D.1 states that the error  $\widehat{A(1)A(0)} - A(1)A(0)$  is on the order of  $\tilde{\mathcal{O}}_p\big(T^{-1 / 2}\big)$ . By Assumption 1, we have that  $A(1)A(0) = \sqrt{S(1)^2S(0)^2} > c^2$ . Observe that the square root function  $g(x) = \sqrt{x}$  is Lipschitz on the interval  $(c^2,\infty)$ . Thus, by a rate-preserving Continuous Mapping Theorem, we have that the error of the normalized variance estimator is on the same order i.e.  $T\cdot \mathrm{VB} - T\cdot \widehat{\mathrm{VB}} = \sqrt{A(1)A(0)} - \sqrt{\widehat{A(1)}\widehat{A(0)}} = \tilde{\mathcal{O}}_p\big(T^{-1 / 2}\big)$ .
D.2 Valid Confidence Intervals (Corollary 5.1)
We now prove the asymptotic validity of the associated Chebyshev-type intervals. This proof is standard in the design-based literature, but we present it here for completeness.
Corollary 5.1. Under Assumptions 1 and 2, and parameters stated in Theorem 4.2, Chebyshev-type intervals are asymptotically valid: for all  $\alpha \in (0,1]$ ,  $\lim \inf_{T\to \infty}\operatorname *{Pr}(\tau \in \hat{\tau}\pm \alpha^{-1 / 2}\sqrt{\widehat{VB}})\geqslant 1 - \alpha$ .
Proof. Define the random variables
$$
Z = \frac {\tau - \hat {\tau}}{\sqrt {\operatorname {V a r} (\hat {\tau})}} \quad \mathrm {a n d} \quad Z ^ {\prime} = \frac {\tau - \hat {\tau}}{\sqrt {\widehat {\mathbf {V B}}}} .
$$
Observe that they are related in the following way:
$$
Z ^ {\prime} = \frac {\tau - \hat {\tau}}{\sqrt {\widehat {\mathbf {V B}}}} = \frac {\tau - \hat {\tau}}{\sqrt {\operatorname {V a r} (\hat {\tau})}} \cdot \left(\sqrt {\frac {\operatorname {V a r} (\hat {\tau})}{\mathbf {V B}}} \cdot \sqrt {\frac {\mathbf {V B}}{\widehat {\mathbf {V B}}}}\right) = Z \cdot \left(\sqrt {\frac {\operatorname {V a r} (\hat {\tau})}{\mathbf {V B}}} \cdot \sqrt {\frac {T \cdot \mathbf {V B}}{T \cdot \widehat {\mathbf {V B}}}}\right).
$$
By definition, we have that  $\lim \sup_{T\to \infty}\mathrm{Var}(\hat{\tau}) / \mathrm{VB}\leqslant 1$ . Recall that by Proposition B.1,  $T\cdot \mathrm{VB}\geqslant T\cdot V_{\mathrm{N}} = \Omega (1)$  so that by Theorem 5.1 and Continuous Mapping Theorem, we have that  $\sqrt{\frac{T\cdot\mathrm{VB}}{T\cdot\widehat{\mathrm{VB}}}}\stackrel {p}{\rightarrow}1$ . Thus, by Slutsky's theorem we have that  $Z^{\prime}$  is asymptotically stochastically dominated by  $Z$ . Now we are ready to compute the coverage probability.
$$
\begin{array}{l} \operatorname * {l i m i n f} _ {T \to \infty} \Pr \left(\tau \in \hat {\tau} \pm \alpha^ {- 1 / 2} \sqrt {\widehat {\mathbf {V B}}}\right) = \operatorname * {l i m i n f} _ {T \to \infty} \Pr \left(\left| \frac {\tau - \hat {\tau}}{\sqrt {\widehat {\mathbf {V B}}}} \right| \leqslant \alpha^ {- 1 / 2}\right) \\ = \lim  _ {T \rightarrow \infty} \inf  \Pr \left(\left| Z ^ {\prime} \right| \leqslant \alpha^ {- 1 / 2}\right) \\ \geqslant \lim  _ {T \rightarrow \infty} \inf  _ {\alpha} \Pr \left(\left| Z \right| \leqslant \alpha^ {- 1 / 2}\right) \\ \geqslant 1 - \alpha , \\ \end{array}
$$
where the last line followed from Chebyshev's inequality and the fact that  $\operatorname{Var}(Z) = 1$ .
E Analysis of Alternative Designs
In this section, we provide analysis on the efficacy of existing adaptive experimental designs for the problem of Adaptive Neyman Allocation. To this end, we show two negative results. In Section E.1, we show that the two-stage design of [Hahn et al., 2011, Blackwell et al., 2022] (i.e. Explore-then-Commit) can suffer linear expected Neyman Regret in the design-based framework for a large class of potential outcome sequences. In Section E.2, we show that mutli-arm bandit algorithms which achieve sublinear expected outcome regret will incur super-linear expected Neyman regret, providing further evidence that these two goals are incompatible.
E.1 Analysis of Explore-then-Commit (Proposition 6.1)
In this setting, we show that Explore-then-Commit designs can sometimes suffer linear Neyman regret, and therefore not recover the Neyman variance in large samples. We formally introduce our definition of Explore-then-Commit designs below. Let  $p_{T_0}^*$  be defined as
$$
p _ {T _ {0}} ^ {*} = \left(1 + \sqrt {\frac {\sum_ {t = 1} ^ {T _ {0}} y _ {t} (0) ^ {2}}{\sum_ {t = 1} ^ {T _ {0}} y _ {t} (1) ^ {2}}}\right) ^ {- 1},
$$
which is the optimal Neyman probability when considering only the sample up to  $T_0$ .
Our definition of ETC encompasses many possible ways of estimating the optimal treatment probability. The only requirement is that the estimation method will converge to  $p_{T_0}^*$  at the rate  $T_0^{-1/2}$ . We consider  $p_{T_0}^*$  rather than the true Neyman probability  $p^*$  because the observed data is informative only of the outcomes in the exploration phase  $T_0$ . Many natural estimators will fall into this class, including Horvitz-Thompson style estimators similar to those used in the construction of our variance estimator.
Algorithm 2: EXPLORE-THEN-COMMIT
Input: Exloration phase size  $T_{0}$    
// Explore Phase   
for  $t = 1\dots T_0$  do Sample treatment assignment  $Z_{t}$  as 1 with probability 1/2 and 0 with probability 1/2 Observe outcome  $Y_{t} = \mathbf{1}[Z_{t} = 1]y_{t}(1) + \mathbf{1}[Z_{t} = 0]y_{t}(0)$    
end   
Construct an estimator  $\widehat{p_{T_0}^*}$  from observed data  $(Z_{1},Y_{1},\ldots Z_{T_{0}},Y_{T_{0}})$  satisfying  $\widehat{p_{T_0}^*} -p_T^* = \mathcal{O}_p(T_0^{-1 / 2})$    
// Exploit Phase   
for  $t = T_0 + 1\dots T$  do Sample treatment assignment  $Z_{t}$  as 1 with probability  $\widehat{p_{T_0}^*}$  and 0 with probability  $1 - p_{T_0}^*$  Observe outcome  $Y_{t} = \mathbf{1}[Z_{t} = 1]y_{t}(1) + \mathbf{1}[Z_{t} = 0]y_{t}(0)$    
end
Before continuing, we provide a few more definitions. We define the second moments of treatment and control outcomes as well as the correlation in the exploration phase as
$$
S _ {T _ {0}} (1) ^ {2} = \frac {1}{T} \sum_ {t = 1} ^ {T _ {0}} y _ {t} (1) ^ {2} \quad S _ {T _ {0}} (0) ^ {2} = \frac {1}{T} \sum_ {t = 1} ^ {T _ {0}} y _ {t} (0) ^ {2} \quad \text {a n d} \quad \rho_ {T _ {0}} = \frac {\frac {1}{T _ {0}} \sum_ {t = 1} ^ {T} y _ {t} (1) y _ {t} (0)}{S _ {T _ {0}} (1) S _ {T _ {0}} (0)}.
$$
We are now ready to state the formal version of Proposition 6.1.
Proposition 6.1*. Suppose that  $T_0 = \Omega (T^\epsilon)$  for some  $\epsilon > 0$  and further suppose that the outcome sequence satisfies the following properties for constants  $C \geqslant c > 0$  and  $c' > 0$ :
- The second moments  $S(1)$ ,  $S(0)$ ,  $S_{T_0}(1)$ , and  $S_{T_1}(0)$  are contained in the interval  $[c, C]$ .
The correlations are bounded away from -1, i.e.  $\rho_{T_0},\rho \geqslant -1 + c$
- The second moments satisfy the following:
$$
S (1) ^ {2} \left(\frac {S (0)}{S (1)} - \frac {S _ {T _ {0}} (0)}{S _ {T _ {0}} (1)}\right) + S (0) ^ {2} \left(\frac {S (1)}{S (0)} - \frac {S _ {T _ {0}} (1)}{S _ {T _ {0}} (0)}\right) \geqslant c ^ {\prime}
$$
Then, the Neyman Regret of Explore-then-Commit is at least linear in probability,  $\mathcal{R}_T = \Omega_p(T)$
The first two conditions are essentially extensions of Assumption 1 to the exploration phase. This ensures that the probability  $p_{T_0}^*$  (which is estimated in the Explore-then-Commit design) does not approach 0 or 1. The third condition is what really makes Explore-then-Commit fail to achieve sublinear Neyman regret. This condition states that the ratio of the second moments in the exploration phase is different than in the larger sequence. For example, if  $S(1) = S(0) = 1$  but  $S_{T_0}(0) / S_{T_0}(1) = 2$  then the condition would hold. In this case, we should not expect Explore-then-Commit to achieve the Neyman variance because the exploration phase does not contain sufficient information about the optimal Neyman probability. We now prove the proposition.
Proof. Let  $p^* = \arg \min_{p \in [0,1]} \sum_{t=1}^{T} f_t(p)$  be the Neyman probability. We begin by re-arranging terms in the Neyman regret:
$$
\begin{array}{l} \mathcal {R} _ {T} = \sum_ {t = 1} ^ {T} f _ {t} \left(p _ {t}\right) - \sum_ {t = 1} ^ {T} f _ {t} \left(p ^ {*}\right) \quad (\text {d e f o f N e y m a n r e g r e t}) \\ = \sum_ {t = 1} ^ {T _ {0}} f _ {t} \left(p _ {t}\right) - f _ {t} \left(p ^ {*}\right) + \sum_ {t = T _ {0} + 1} ^ {T} f _ {t} \left(p _ {t}\right) - f _ {t} \left(p ^ {*}\right) \quad (\text {s p l i t t i n g t e r m s b y p h a s e s}) \\ = \sum_ {t = 1} ^ {T _ {0}} f _ {t} (1 / 2) - f _ {t} \left(p ^ {*}\right) + \sum_ {t = T _ {0} + 1} ^ {T} f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) - f _ {t} \left(p ^ {*}\right) \tag {def of ETC} \\ = \sum_ {t = 1} ^ {T _ {0}} f _ {t} (1 / 2) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) + \sum_ {t = 1} ^ {T} f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) - f _ {t} \left(p ^ {*}\right) \quad (\text {a d d i n g} \\ \end{array}
$$
$$
\geqslant \sum_ {t = 1} ^ {T _ {0}} f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) + \sum_ {t = 1} ^ {T} f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) - f _ {t} \left(p ^ {*}\right),
$$
Where the inequality follows by the optimality of  $p_{T_0}^*$  on the exploration phase. By adding and subtracting the  $f_t(p_{T_0}^*)$  to the second sum, we obtain the following decomposition
$$
= \underbrace {\sum_ {t = 1} ^ {T _ {0}} f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right)} _ {\text {T e r m 1}} + \underbrace {\sum_ {t = 1} ^ {T} f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) - f _ {t} \left(p _ {T _ {0}} ^ {*}\right)} _ {\text {T e r m 2}} + \underbrace {\sum_ {t = 1} ^ {T} f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(p ^ {*}\right)} _ {\text {T e r m 3}}
$$
We handle each of the terms separately in the remainder of the proof.
Term 1: By the assumptions on the second moments and correlation of outcomes in the exploration phase, we have that  $p_{T_0}^*$  is bounded away from 0 and 1 by a constant. Furthermore, the cost functions  $f_t$  are Lipschitz on the interval  $[\gamma, 1 - \gamma]$  for any fixed  $\gamma$ . Thus, we may apply the quantitative Continuous Mapping Theorem to bound the absolute value of Term 1 as follows:
$$
\begin{array}{l} \left| \sum_ {t = 1} ^ {T _ {0}} f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) \right| \leqslant \sum_ {t = 1} ^ {T _ {0}} \left| f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) \right| \quad (\text {t r i a n g l e i n e q u a l i t y}) \\ \leqslant \sum_ {t = 1} ^ {T _ {0}} \mathcal {O} _ {p} \left(T _ {0} ^ {- 1 / 2}\right) \quad (\text {e s t i m a t o r p r o p e r t y + C M T}) \\ = \mathcal {O} _ {p} \left(T _ {0} \cdot T _ {0} ^ {- 1 / 2}\right) \\ = \mathcal {O} _ {p} \left(T _ {0} ^ {1 / 2}\right) \\ = \mathcal {O} _ {p} (T ^ {1 / 2}), \\ \end{array}
$$
where the final inequality follows from the fact that  $T_0 \leqslant T$ .
Term 2: A similar argument may be applied to the second term. Again, we may apply the continuous mapping theorem as before to obtain
$$
\begin{array}{l} \left| \sum_ {t = 1} ^ {T} f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) \right| \leqslant \sum_ {t = 1} ^ {T} \left| f _ {t} \left(p _ {T _ {0}} ^ {*}\right) - f _ {t} \left(\widehat {p _ {T _ {0}} ^ {*}}\right) \right| \quad (\text {t r i a n g l e i n e q u a l i t y}) \\ \leqslant \sum_ {t = 1} ^ {T} \mathcal {O} _ {p} \left(T _ {0} ^ {- 1 / 2}\right) \quad (\text {e s t i m a t o r p r o p e r t y + C M T}) \\ = \mathcal {O} _ {p} (T \cdot T _ {0} ^ {- 1 / 2}) \\ = \mathcal {O} _ {p} (T \cdot T ^ {- \epsilon / 2}) \\ = \mathcal {O} _ {p} (T ^ {1 - \epsilon / 2}), \\ \end{array}
$$
where we have used the assumption that  $T_0 = \Omega (T^\epsilon)$
Term 3: We now handle the third term. Let  $V_{0}$  be the variance of adaptive Horvitz-Thompson estimator under the Bernoulli design when using the treatment probability  $p_{T_0}^*$ . By construction of the cost functions, we have that they are related to the variance as follows: