SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Susan Athey and Stefan Wager. Policy learning with observational data. *Econometrica*, 89(1): 133–161, 2021.
[2] Susan Athey, Julie Tibshirani, and Stefan Wager. Generalized random forests. The Annals of Statistics, 47(2):1148-1178, 2019.
[3] Akshay Balsubramani. Sharp finite-time iterated-logarithm martingale concentration. arXiv preprint arXiv:1405.2639, 2014.
[4] Donald A Berry. Bayesian approaches for comparative effectiveness research. Clinical Trials, 9 (1):37-47, 2012.
[5] Avrim Blum, Adam Kalai, and John Langford. Beating the hold-out: Bounds for k-fold and progressive cross-validation. In Proceedings of the twelfth annual conference on Computational learning theory, pages 203-208, 1999.
[6] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, and Whitney Newey. Double/debiased/neyman machine learning of treatment effects. American Economic Review, 107(5):261-65, 2017.
[7] Thomas Cook and Olive D Buhule. Stopping trials early due to harm. NEJM Evidence, 1(5): EVIDctw2100026, 2022.
[8] Yifan Cui, Michael R Kosorok, Erik Sverdrup, Stefan Wager, and Ruoqing Zhu. Estimating heterogeneous treatment effects with right-censored data via causal survival forests. arXiv preprint arXiv:2001.09887, 2020.
[9] Richard E Deichmann, Marie Krousel-Wood, and Joseph Breault. Bioethics in practice: considerations for stopping a clinical trial early. Ochsner Journal, 16(3):197-198, 2016.
[10] David L Demets and KK Gordon Lan. Interim analysis: the alpha spending function approach. Statistics in Medicine, 13(13-14):1341-1352, 1994.
[11] Alex Deng, Jiannan Lu, and Shouyuan Chen. Continuous monitoring of A/B tests without pain: Optional stopping in bayesian testing. In 2016 IEEE international conference on data science and advanced analytics (DSAA), pages 243-252. IEEE, 2016.
[12] Christiana Drake. Effects of misspecification of the propensity score on estimators of treatment effect. Biometrics, pages 1231-1236, 1993.
[13] Steven R Howard, Aaditya Ramdas, Jon McAuliffe, and Jasjeet Sekhon. Time-uniform, nonparametric, nonasymptotic confidence sequences. The Annals of Statistics, 49(2):1055-1080, 2021.
[14] Ramesh Johari, Pete Koomen, Leonid Pekelis, and David Walsh. Peeking at A/B tests: Why it matters, and what to do about it. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1517-1525, 2017.
[15] Toru Kitagawa and Aleksey Tetenov. Who should be treated? empirical welfare maximization methods for treatment choice. *Econometrica*, 86(2):591–616, 2018.
[16] Martin Kulldorff, Robert L Davis, Margarette Kolczak, Edwin Lewis, Tracy Lieu, and Richard Platt. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Sequential analysis, 30(1):58-78, 2011.
[37] Basil Varkey. Principles of clinical ethics and their application to practice. Medical Principles and Practice, 30(1):17-28, 2021.
[38] Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228-1242, 2018.
[39] A Wald. Sequential tests of statistical hypotheses. Annals of Mathematical Statistics, 1945.
[40] Ian Waudby-Smith and Aaditya Ramdas. Estimating means of bounded random variables by betting. arXiv preprint arXiv:2010.09686, 2020.
[41] Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. A survey on causal inference. ACM Transactions on Knowledge Discovery from Data (TKDD), 15(5):1-46, 2021.
[42] Miao Yu, Wenbin Lu, and Rui Song. A new framework for online testing of heterogeneous treatment effect. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 10310-10317, 2020.
[43] Miao Yu, Wenbin Lu, and Rui Song. Online testing of subgroup treatment effects based on value difference. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1463-1468. IEEE, 2021.
[44] Zhongheng Zhang, Heidi Seibold, Mario V Vettore, Woo-Jung Song, and Vieille François. Subgroup identification in clinical trials: an overview of available methods and their implementations with r. Annals of translational medicine, 6(7), 2018.
A Early Stopping Methods
Table S1: Comparison of stopping tests. We categorize commonly-used stopping tests into three categories—frequentist, sequential, and Bayesian—and describe their key features. Stopping type describes whether the stopping test is used to stop for benefit (B), harm (H), or futility (F). Stopping criteria refers to the statistic that is used to make the stopping decision. Monitoring frequency describes how often the method allows peeking at the data; for example, frequentist methods only allow for a small number of interim checkpoints, while sequential methods allow for continuously monitoring outcomes as they are observed. Finally, the test type describes what kinds of alternate hypotheses the method allows for testing; the SPRT, for example, only allows the investigator to test simple vs. simple hypotheses.
Stopping Test	Stopping Type	Stopping Criteria	Monitoring Frequency	Test Types
Frequentist / group sequential
Pocock [23]	B/H	p-value	Limited checks, pre-planned	All
O'Brien-Fleming [22]	B/H	p-value	Limited checks, pre-planned	All
Alpha-spending [10]	B/H	p-value	Limited checks, flexible	All
Conditional power [18]	F	Conditional power	Limited checks, flexible	All
Sequential / quasi-Bayesian
SPRT [39]	B/H/F	Likelihood ratio	Continuous	Simple
2-SPRT [19]	B/H/F	Likelihood ratio	Continuous	One-sided
Max-SPRT [16]	B/H	Likelihood ratio	Continuous	Two-sided
Mixture-SPRT [26, 14]	B/H	Likelihood ratio / always-valid p-value	Continuous	Two-sided
Martingale-based [3, 13]	B/H	Always-valid p-value	Continuous	All
Bayesian
Parameter estimation [33]	B/H	Posterior expected loss	Continuous	All
Hypothesis testing [11]	B/H	Posterior odds	Continuous	All
Predictive probability [29]	F	Posterior predictive probability	Continuous	All
B Adapting the SPRT for Early Stopping
Here, we briefly describe how to adapt SPRT-based tests for early stopping. This largely follows from Section 4.3 in [14]. Recall that for every participant  $i$ , we observe their treatment assignment  $d_{i}$ , outcome  $y_{i}$ , and covariates  $x_{i}$ . We now slightly update this notation. Assume that at time  $t$ , we observe two participants, one treated, one untreated. Let  $y_{t}^{(1)}$  and  $y_{t}^{(0)}$  denote the outcomes for the treated and untreated participants respectively, and  $x_{t}$  denote their joint covariates. We define the difference in outcomes observed at time  $t$ ,  $z_{t} = y_{t}^{(1)} - y_{t}^{(0)}$ .
We can now use Wald's SPRT [39], mSPRT [14], and other sequential tests in this setup. For example, assume that  $z_{t}$  follows a known distribution with probability density function  $p$  and mean parameter  $\mu$ . Then, to test between the null hypothesis of no effect  $H_0: \mu = 0$  against an alternate hypothesis of a specific harmful effect  $H_1: \mu = \beta$ , Wald's SPRT would use
$$
\lambda_ {2 n} ^ {S P R T} = \sum_ {t = 1} ^ {n} \log \frac {p (z _ {t} | \mu = \beta)}{p (z _ {t} | \mu = 0)}
$$
$$
b ^ {S P R T} (\alpha) = - \log \alpha
$$
Note that we denote the statistic as  $\lambda_{2n}^{SPRT}$ , not  $\lambda_{n}^{SPRT}$ , as it uses  $2n$  observations ( $n$  treated,  $n$  untreated). For the special case where  $p$  is the Gaussian probability density function, the test statistic reduces to
$$
\lambda_ {2 n} ^ {S P R T} = \beta \sum_ {t = 1} ^ {n} z _ {t} - \frac {n}{2} \beta^ {2}.
$$
C Weighted Test Statistics
We present examples of weighted test statistics for some commonly-used stopping tests in Tab. S2. Note that weighted inference is already implemented in standard data analysis packages for a wide range of estimators. For example, for a group-sequential test (e.g., O'Brien-Fleming) that considers p-values from a generalized linear model, standard packages in R [24] and Python [31] allow for user-defined regression weights.
Table S2: Weighted test statistics for commonly used stopping tests.
Stopping Test	Test Statistic	Weighted Test Statistic
Group-sequential z-test	\(\frac{\sqrt{n}}{\sqrt{2\sigma^2}}\left(\frac{\sum_{i=1}^{n}y_i d_i}{\sum_{i=1}^{n}d_i}-\frac{\sum_{i=1}^{n}y_i(1-d_i)}{\sum_{i=1}^{n}(1-d_i)}\right)\)	\(\frac{\sqrt{\sum_{i=1}^{n}w_i}}{\sqrt{2\sigma^2}}\left(\frac{\sum_{i=1}^{n}w_iy_i d_i}{\sum_{i=1}^{n}w_i d_i}-\frac{\sum_{i=1}^{n}w_iy_i(1-d_i)}{\sum_{i=1}^{n}w_i(1-d_i)}\right)\)
Generic SPRT	\(\sum_{t=1}^{n}\log\frac{p(z_t|\mu=\beta)}{p(z_t|\mu=0)}\)	\(\sum_{t=1}^{n}w_t\log\frac{p(z_t|\mu=\beta)}{p(z_t|\mu=0)}\)
Gaussian SPRT	\(\beta\sum_{t=1}^{n}z_t-\frac{n}{2}\beta^2\)	\(\beta\sum_{t=1}^{n}wetz_t-\frac{\sum_{t=1}^{n}w_t}{2}\beta^2\)
Gaussian mSPRT [14]	\(\sqrt{\frac{2\sigma^2}{2\sigma^2+\tau^2n}}\exp\left\{\frac{\tau^2(\sum_{i=1}^{n}z_t-\theta_0n)^2}{4\sigma^2(2\sigma^2+\tau^2n)}\right\}\)	\(\sqrt{\frac{2\sigma^2}{2\sigma^2+\tau^2\sum_{t=1}^{n}w_t}}\exp\left\{\frac{\tau^2(\sum_{i=1}^{n}wetz_t-\theta_0\sum_{t=1}^{n}w_t)^2}{4\sigma^2(2\sigma^2+\tau^2\sum_{t=1}^{n}w_t)}\right\}\)
Gaussian maxSPRT (based on [16])	\(\frac{\sum_{t=1}^{n}z_t}{n}\sum_{t=1}^{n}z_t-\frac{n}{2}\left(\frac{\sum_{t=1}^{n}z_t}{n}\right)^2\)	\(\frac{\sum_{t=1}^{n}wetz_t}{\sum_{t=1}^{n}w_t}\sum_{t=1}^{n}wetz_t-\frac{\sum_{t=1}^{n}w_t}{2}\left(\frac{\sum_{t=1}^{n}wetz_t}{\sum_{t=1}^{n}w_t}\right)^2\)
D CATE Estimation with No Confounding
Most CATE estimation methods are designed to control for confounders, that is, variables that affect both the treatment status  $D$  and observed outcome  $Y$ . Our work, however, considers experiments in which  $D$  is randomly administered; there is no confounding, as  $D$  is independent of all variables except  $Y$ . The CATE estimation method thus only needs to infer how the treatment effect varies with the covariates  $X$  and does not need to correct for the relationship between  $X$  and  $D$ . This simplifies the application of several CATE estimation methods; we discuss a few specific examples below. In short, many methods use estimates of the treatment propensity  $\eta(x) = \mathbb{P}(D = 1 | X = x)$  to correct for confounding. In our setting, we can simply set  $\eta(x) = 0.5 \forall x$ .
- Causal forests While the original causal forest algorithm [38] did not estimate a propensity score, later improvements [2, 21] use estimates of  $\eta(x)$  to "orthogonalize" the estimator. In practice, this orthogonalization is essential in obtaining accurate CATE estimates from observational data [2]. However, it is not required in our experimental setting; these improved algorithms [2, 21] can be used by setting  $\eta(x)$  to its true value (i.e., 0.5).
- Meta-learners T-learners and S-learners (as described in [17]) do not use estimates of the propensity score and thus do not require any modifications. The X-learner [17] uses an estimate of  $\eta(x)$  to weight the predictions of models trained separately on the treatment and control groups [see 17, equation 9]. These weights can be replaced by 0.5 in our setting.
- Linear Models Linear CATE models often use estimates of  $1 / \eta(x)$  as regression weights. This approach ensures that the estimation is "doubly robust": as long as either the model of treatment assignment or outcome heterogeneity is correctly specified, the resulting regression coefficients are unbiased [27]. In our setting, such reweighting is unnecessary, as  $1 / \eta(x)$  is known and the same for all observations.
Not needing to estimate the propensity score has two key benefits. First, it reduces the computational complexity of the problem. Repeated fitting techniques like cross-fitting [6] are often used to reduce the bias of simultaneous propensity score and treatment effect estimation, but these procedures increase the computational cost of CATE estimation, especially for large datasets. Second, it protects against inaccurate estimates of the treatment propensity. Poor estimates of  $\eta(x)$  can lead to highly biased treatment effect estimates [12]. By explicitly specifying  $\eta(x) = 0.5$ , we avoid spurious inferring any relationships between  $X$  and  $D$  that would create such bias.
We now present proofs for all theoretical results described in the main text. Note that we repeatedly use the following property. For any events  $A$  and  $B$ ,
$$
\mathbb {P} (A, B) \leq \min  \left\{\mathbb {P} (A), \mathbb {P} (B) \right\}. \tag {5}
$$
E.1 Proof of Prop. 3.1: Group knowledge improves early stopping
We argue that the stopping probability of the test on  $S_{n}^{*}$  converges to 1. Recall that
$$
\lambda_ {n} ^ {*} = \frac {\Delta_ {n} ^ {*}}{\hat {\sigma} _ {n} ^ {*}}.
$$
By assumption,  $\lambda_n^* - \mu^* / \hat{\sigma}_n^* \xrightarrow{d} \mathcal{N}(0,1)$  and  $\sqrt{|S_n^*|} \hat{\sigma}_n^* / \sigma^* \xrightarrow{p} 1$ . Define  $\rho_n^* = \sqrt{|S_n^*|} \Delta_n^* / \sigma^*$ . Then, by Slutsky's theorem,
$$
\rho_ {n} ^ {*} - \mu^ {*} \sqrt {| S _ {n} ^ {*} |} / \sigma^ {*} \xrightarrow {d} \mathcal {N} (0, 1).
$$
Now, consider the probability that the test does not stop. For notational convenience, fix some  $\alpha > 0$  and denote  $b(\alpha)$  simply as  $b$ . Then,
$$
\begin{array}{l} \mathbb {P} \left(\lambda_ {n} ^ {*} \leq b\right) = \mathbb {P} \left(\rho_ {n} ^ {*} \leq b \rho_ {n} ^ {*} / \lambda_ {n} ^ {*}\right) \\ = \mathbb {P} \left(\rho_ {n} ^ {*} \leq b \rho_ {n} ^ {*} / \lambda_ {n} ^ {*}, \rho_ {n} ^ {*} / \lambda_ {n} ^ {*} \geq 0. 5\right) + \mathbb {P} \left(\rho_ {n} ^ {*} \leq b \rho_ {n} ^ {*} / \lambda_ {n} ^ {*}, \rho_ {n} ^ {*} / \lambda_ {n} ^ {*} <   0. 5\right) \\ \leq \mathbb {P} \left(\rho_ {n} ^ {*} \leq b / 2\right) + \mathbb {P} \left(\rho_ {n} ^ {*} / \lambda_ {n} ^ {*} <   0. 5\right) \tag {6} \\ \end{array}
$$
Now, we know that,
$$
\begin{array}{l} \frac {\rho_ {n} ^ {*}}{\lambda_ {n} ^ {*}} = \frac {\sqrt {| S _ {n} ^ {*} |} \Delta_ {n} ^ {*} / \sigma^ {*}}{\Delta_ {n} ^ {*} / \hat {\sigma} _ {n} ^ {*}} \\ = \frac {\sqrt {| S _ {n} ^ {*} |} \hat {\sigma} _ {n} ^ {*}}{\sigma^ {*}} \\ \xrightarrow {p} 1. \\ \end{array}
$$
Hence,  $\mathbb{P}(\rho_n^* /\lambda_n^* < 0.5)\to 0$ . This bounds the second term in (6). We now focus on the first term. Fix any  $\epsilon >0$ . Since  $|S_{n}^{*}|\rightarrow \infty$ , there must exist some  $n_{\epsilon}$  such that  $b / 2 - \mu^{*}\sqrt{|S_{n}^{*}|} /\sigma^{*} < \Phi^{-1}(\epsilon)$  for all  $n > n_{\epsilon}$ . Assume  $n > n_{\epsilon}$ . Then,
$$
\begin{array}{l} \mathbb {P} \left(\rho_ {n} ^ {*} \leq b / 2\right) = \mathbb {P} \left(\rho_ {n} ^ {*} - \mu^ {*} \sqrt {\left| S _ {n} ^ {*} \right|} / \sigma^ {*} \leq b / 2 - \mu^ {*} \sqrt {\left| S _ {n} ^ {*} \right|} / \sigma^ {*}\right) \\ \leq \mathbb {P} \left(\rho_ {n} ^ {*} - \mu^ {*} \sqrt {\left| S _ {n} ^ {*} \right|} / \sigma^ {*} \leq \Phi^ {- 1} (\epsilon)\right) \\ \rightarrow \Phi (\Phi^ {- 1} (\epsilon)) = \epsilon \\ \end{array}
$$
since  $\rho_n^* - \mu^* \sqrt{|S_n^*|} / \sigma^* \xrightarrow{d} \mathcal{N}(0,1)$ . Since this holds for any value of  $\epsilon$ , we have that  $\mathbb{P}(\rho_n^* \leq b/2) \to 0$ . Hence, from (6), we have that  $\mathbb{P}(\lambda_n^* \leq b) \to 0$ , and thus
$$
\mathbb {P} \left(\lambda_ {n} ^ {*} > b\right)\rightarrow 1
$$
Hence, the stopping probability of the test on the harmed population converges to 1.
E.2 Proof of Thm. 3.2: FLASH weights converge to optimal weights
Our entire proof will be carried out conditional on the value  $x_{i}$ . Recall that we define our weights as
$$
\hat {w} _ {i} ^ {n} = 1 - \Phi \left(\frac {\delta - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right)
$$
Further recall that the functions  $\hat{\tau}_{n, -i}$  and  $\hat{\sigma}_{n, -i}$  are, by construction, independent of  $x_{i}$ .
Error Bound We first establish a bound on the difference between our estimated weights  $\hat{w}_i^n$  and the optimal weights  $w_i^*$ . Consider two cases.
Case 1:  $\tau(x_i) \leq 0$ . In this case,  $w_i^* = 0$  by definition, and so we just need to prove a bound on the magnitude of  $\hat{w}_i^n$ . Using Taylor's theorem with Lagrange remainder, we know that  $\exists h_n \in [0,1]$  such that
$$
\begin{array}{l} \hat {w} _ {i} ^ {n} = 1 - \Phi \left(\frac {\delta - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) \\ = 1 - \Phi \left(\frac {\delta - \tau \left(x _ {i}\right)}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)}\right) - \frac {\tau \left(x _ {i}\right) - \hat {\tau} _ {n , - i} \left(x _ {i}\right)}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)} \phi \left(\frac {\delta - \tau \left(x _ {i}\right)}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)} + h _ {n} \frac {\tau \left(x _ {i}\right) - \hat {\tau} _ {n , - i} \left(x _ {i}\right)}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)}\right) \\ \leq 1 - \Phi \Bigl (\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} \Bigr) + \frac {| \tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i}) |}{\hat {\sigma} _ {n , - i} (x _ {i})} \phi \Bigl (\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} + h _ {n} \frac {\tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} \Bigr) \\ \end{array}
$$
where  $\phi$  is the standard Gaussian probability density function. Since  $\delta -\tau (x_i) > 0$ , we can use the Chernoff inequality to bound the first term,
$$
1 - \Phi \Big (\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} \Big) \leq \exp \Big (- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \Big).
$$
We now focus on the second term, which contains
$$
\begin{array}{l} \phi \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} + h _ {n} \frac {\tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) \\ = \frac {1}{\sqrt {2 \pi}} \exp \left\{- \frac {1}{2} \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} + h _ {n} \frac {\tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) ^ {2} \right\} \\ = \frac {1}{\sqrt {2 \pi}} \exp \left\{- \frac {1}{2} \left[ \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) ^ {2} + h _ {n} ^ {2} \left(\frac {\tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) ^ {2} - \right. \right. \\ \left. \left. 2 h _ {n} \frac {(\delta - \tau (x _ {i})) (\hat {\tau} _ {n , - i} (x _ {i}) - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right] \right\} \\ \leq \exp \left\{- \frac {1}{2} \left[ \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) ^ {2} - 2 h _ {n} \frac {(\delta - \tau (x _ {i})) | \hat {\tau} _ {n , - i} (x _ {i}) - \tau (x _ {i}) |}{\hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right] \right\} \\ \leq \exp \left\{- \frac {1}{2} \left[ \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) ^ {2} - 2 \frac {(\delta - \tau (x _ {i})) | \hat {\tau} _ {n , - i} (x _ {i}) - \tau (x _ {i}) |}{\hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right] \right\}, \\ \end{array}
$$
since  $h_n \in [0,1]$ . Thus, we have that
$$
\hat {w} _ {i} ^ {n} \leq \exp \left\{- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right\} + \frac {| \tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i}) |}{\hat {\sigma} _ {n , - i} (x _ {i})} \exp \left\{- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} + \frac {(\delta - \tau (x _ {i})) | \hat {\tau} _ {n , - i} (x _ {i}) - \tau (x _ {i}) |}{\hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right\}.
$$
Case 2:  $\tau(x_i) > 0$ . In this case,  $w_i^* = 1$ . Thus,
$$
\begin{array}{l} | \hat {w} _ {i} ^ {n} - w _ {i} ^ {*} | = 1 - \hat {w} _ {i} ^ {n} \\ = \Phi \left(\frac {\delta - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {i} (x _ {i})}\right) \\ = \Phi \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) + \frac {| \tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i}) |}{\hat {\sigma} _ {n , - i} (x _ {i})} \phi \left(\frac {\delta - \tau (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})} + h _ {n} \frac {\tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i})}{\hat {\sigma} _ {n , - i} (x _ {i})}\right) \\ = 1 - \Phi \left(\frac {\tau \left(x _ {i}\right) - \delta}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)}\right) + \frac {\left| \tau \left(x _ {i}\right) - \hat {\tau} _ {n , - i} \left(x _ {i}\right) \right|}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)} \phi \left(\frac {\delta - \tau \left(x _ {i}\right)}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)} + h _ {n} \frac {\tau \left(x _ {i}\right) - \hat {\tau} _ {n , - i} \left(x _ {i}\right)}{\hat {\sigma} _ {n , - i} \left(x _ {i}\right)}\right) \\ \leq \exp \left\{- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right\} + \\ \frac {| \tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i}) |}{\hat {\sigma} _ {n , - i} (x _ {i})} \exp \left\{- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} + \frac {(\tau (x _ {i}) - \delta) | \hat {\tau} _ {n , - i} (x _ {i}) - \tau (x _ {i}) |}{\hat {\sigma} _ {n , - i} ^ {2} (x _ {i})}\right) \Bigg \}, \\ \end{array}
$$
using the same Taylor expansion and Chernoff bound from above.
In summary, we have established that
$$
| \hat {w} _ {i} ^ {n} - w _ {i} ^ {*} | \leq \exp \left\{- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} \right\} + \frac {| \tau (x _ {i}) - \hat {\tau} _ {n , - i} (x _ {i}) |}{\hat {\sigma} _ {n , - i} (x _ {i})} \exp \left\{- \frac {(\delta - \tau (x _ {i})) ^ {2}}{2 \hat {\sigma} _ {n , - i} ^ {2} (x _ {i})} + \frac {| \tau (x _ {i}) - \delta | | \hat {\tau} _ {n , - i} (x _ {i}) - \tau (x _ {i}) |}{\hat {\sigma} _ {n , - i} ^ {2} (x _ {i})}\right) \Bigg \}.
$$
Consistency of  $\hat{w}_i^n$  We now establish the consistency of  $\hat{w}_i^n$  using the derived bound. We assume that  $\delta < \inf_{x:\tau(x) > 0} \tau(x)$  and, given  $x_i$ ,  $\hat{\tau}_{n,-i}(x_i) \stackrel{p}{\to} \tau(x_i)$  and  $\hat{\sigma}_{n,-i}(x_i) \stackrel{p}{\to} 0$ .
Define  $a_{i} = |\tau(x_{i}) - \delta| / \sqrt{2}$  and  $Z_{i,n} = \frac{\tau(x_{i}) - \hat{\tau}_{n, -i}(x_{i})}{\hat{\sigma}_{n, -i}(x_{i})}$ . Note that  $a_{i}$  is strictly positive and a constant (given  $x_{i}$ ). From the error bound, we have that
$$
| \hat {w} _ {i} ^ {n} - w _ {i} ^ {*} | \leq \exp (- a _ {i} ^ {2} / \hat {\sigma} _ {n, - i} (x _ {i}) ^ {2}) + | Z _ {i, n} | \exp [ - a _ {i} ^ {2} / \hat {\sigma} _ {n, - i} (x _ {i}) ^ {2} + \sqrt {2} a _ {i} | Z _ {i, n} | / \hat {\sigma} _ {n, - i} (x _ {i}) ].
$$
Now, we fix any  $\epsilon > 0$ . Applying the law of total probability and equation (5), we find that