SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Here, we define
(6)  $\hat{p}_{t - 1,j} = \frac{\sum_{s = 1}^{t - 1}\sum_{i = 1}^{n_s}\mathbb{1}_{(X_{is}\in\mathcal{S}_j)}}{\sum_{s = 1}^{t - 1}n_s},$
$$
\begin{array}{l} \bar {Y} _ {t - 1, j} (1) = \frac {\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} D _ {i s} Y _ {i s}}{\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} D _ {i s}}, \quad \bar {Y} _ {t - 1, j} (0) = \frac {\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} (1 - D _ {i s}) Y _ {i s}}{\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} (1 - D _ {i s})}, \\ \hat {\tau} _ {t - 1, j} = \bar {Y} _ {t - 1, j} (1) - \bar {Y} _ {t - 1, j} (0), \\ \end{array}
$$
$$
\begin{array}{l} \hat {\sigma} _ {t - 1, j} ^ {2} (1) = \frac {\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} D _ {i s} \left(Y _ {i s} - \bar {Y} _ {t - 1 , j} (1)\right) ^ {2}}{\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} D _ {i s}}, \\ \hat {\sigma} _ {t - 1, j} ^ {2} (0) = \frac {\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} (1 - D _ {i s}) \big (Y _ {i s} - \bar {Y} _ {t - 1 , j} (0) \big) ^ {2}}{\sum_ {s = 1} ^ {t - 1} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} (1 - D _ {i s})}. \\ \end{array}
$$
One important feature of Problem B is that we introduce a relaxation of the welfare constraint through  $\delta(N_{t-1})$ . From a theoretical perspective, the function  $\delta(\cdot)$  should be strictly positive, and satisfies  $\lim_{x \to \infty} \delta(x) = 0$  and  $\lim_{x \to \infty} \sqrt{x} \delta(x) = \infty$ . For implementation, we recommend using  $\delta(N_{t-1}) = \sqrt{\log(N_{t-1}) / N_{t-1}}$ . It is also possible to incorporate the standard error of the estimated
subgroup treatment effects into the welfare constraint, which motives the more sophisticated version:
$$
\log \left(\frac {e _ {j}}{1 - e _ {j}}\right) \cdot \frac {\hat {\tau} _ {t - 1 , j}}{\hat {v} _ {t - 1 , j}} \geq - \sqrt {\frac {\log (N _ {t - 1})}{N _ {t - 1}}},
$$
where  $\hat{v}_{t - 1,j}$  is the adaptively estimated standard deviation defined in the Supplementary Materials. Scaling the welfare constraint by  $\hat{v}_{t - 1,j}$ , a measure of randomness in  $\hat{\tau}_{t - 1,j}$ , delivers a more clear interpretation: the above now corresponds to a t-test for the subgroup treatment effect  $\tau_{j}$  with a diverging threshold, and the specific choice stems from Schwarz's minimum BIC rule.
After the final Stage  $T$ , we have the group-level treatment effect estimates  $\hat{\tau}_j \coloneqq \hat{\tau}_{Tj}$ , the variance estimates  $\hat{\sigma}_j^2(d) \coloneqq \hat{\sigma}_{Tj}^2(d)$ , and the group proportions  $\hat{p}_j \coloneqq \hat{p}_{Tj}$  (that is, we omit the time index  $T$  for estimates obtained after the completion of the experiment). Together with valid standard errors, one can conduct statistical inference at some pre-specified level  $\alpha$ . To be precise, the estimated ATE is  $\hat{\tau} = \sum_{j=1}^{m} \hat{p}_j \hat{\tau}_j$ , and we define the following
$$
\begin{array}{l} \hat {v} _ {j} ^ {2} = \frac {1}{\hat {p} _ {j}} \left(\frac {\hat {\sigma} _ {j} ^ {2} (1)}{\hat {e} _ {j}} + \frac {\hat {\sigma} _ {j} ^ {2} (0)}{1 - \hat {e} _ {j}}\right), \quad \text {a n d} \quad \hat {v} ^ {2} = \sum_ {j = 1} ^ {m} \hat {p} _ {j} ^ {2} \hat {v} _ {j} ^ {2} + \sum_ {j = 1} ^ {m} \hat {p} _ {j} \left(\hat {\tau} _ {j} - \hat {\tau}\right) ^ {2}, \tag {7} \\ \text {w h e r e} \hat {e} _ {j} = \frac {\sum_ {s = 1} ^ {T} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})} D _ {i s}}{\sum_ {s = 1} ^ {T} \sum_ {i = 1} ^ {n _ {s}} \mathbb {1} _ {(X _ {i s} \in \mathcal {S} _ {j})}}. \\ \end{array}
$$
Lastly, we can construct the two-sided confidence intervals for  $\hat{\tau}_j$  and  $\hat{\tau}$  as
$$
\left[ \hat {\tau} _ {j} \pm \Phi^ {- 1} (1 - \alpha / 2) \cdot \hat {v} _ {j} / \sqrt {N} \right] \quad \text {a n d} \quad \left[ \hat {\tau} \pm \Phi^ {- 1} (1 - \alpha / 2) \cdot \hat {v} / \sqrt {N} \right]. \tag {8}
$$
3 Theoretical investigations
In this section, we investigate the theoretical properties of our proposed fair adaptive experiment strategy, and we demonstrate that our approach achieves the same desirable properties as in the oracle setting. We work under the following assumptions:
Assumption 1 For  $t = 1, \dots, T$  and  $i = 1, \dots, n_t$ , the covariates and the potential outcomes,  $(X_{it}, Y_{it}(0), Y_{it}(1))$ , are independently and identically distributed; the potential outcomes have bounded fourth moments:  $\mathbb{E}[|Y_{it}(d)|^4] < \infty$  for  $d = 0, 1$ .
Assumption 2 The group proportions  $p_j$  are bounded away from 0: there exists  $\delta > 0$  such that  $p_j \geq \delta$  for all  $j = 1, 2, \ldots, m$ .
Assumption 3 The sample size for each stage,  $n_t$ , are of the same order: there exists  $c \geq 1$  such that  $\frac{N}{cT} \leq n_t \leq \frac{cN}{T}$ .
Assumption 1 imposes a mild moment condition on the potential outcomes over different stages. Assumption 2 assumes that the proportion of each group is nonzero. Assumption 3 requires that the sample size in each stage are of the same order. We remark that this assumption can be easily relaxed.
Theorem 1 (Consistent treatment effect and variance estimation) Assume Assumptions 1-3 hold. Then, the estimated group-level treatment effects and the associated variances are consistent:
$$
\hat {\tau} _ {t j} - \tau_ {j} = O _ {p} \big (1 / \sqrt {N _ {t}} \big), \quad \hat {\sigma} _ {t j} ^ {2} (d) - \sigma_ {j} ^ {2} (d) = O _ {p} \big (1 / \sqrt {N _ {t}} \big),
$$
where  $N_{t} = \sum_{s = 1}^{t}n_{s}$ . As a result, after stage  $T$ ,
$$
\hat {\tau} _ {j} - \tau_ {j} = O _ {p} \big (1 / \sqrt {N} \big), \quad \hat {\tau} - \tau = O _ {p} \big (1 / \sqrt {N} \big), \quad \hat {\sigma} _ {j} ^ {2} (d) - \sigma_ {j} ^ {2} (d) = O _ {p} \big (1 / \sqrt {N} \big).
$$
Theorem 1 shows consistency of the group-level treatment effects and the associated variance estimators. This further implies the consistency of the average treatment effect estimator. The proof of Theorem 1 leverages the martingale methods [21]. Building on Theorem 1, we can establish the theoretical properties of the actual treatment allocation under our design strategy.
Theorem 2 (Convergence of actual treatment allocation) Assume Assumptions 1-3 hold. Then the actual treatment allocation, defined in Eq (7), converges to the oracle allocation:  $\hat{e}_j - e_j^* = o_p(1)$ .
Theorem 2 is a key result. It suggests that despite having minimum knowledge regarding the distribution of the potential outcomes at the experiment's outset, we are able to adaptively revise the treatment allocation probability using the accrued information, and the actual treatment probabilities under our proposed fair adaptive experiment strategy converge to their oracle counterparts. Building on Theorem 1 and Theorem 2, we are able to establish the asymptotic normality results of our proposed estimators and show that the standard errors are valid.
Theorem 3 (Asymptotic normality and valid standard errors) Assume Assumptions 1-3 hold. Then, the estimated group-level treatment effects and the estimated ATE are asymptotically normally distributed:
$$
\sqrt {N} \left(\hat {\tau} _ {j} - \tau_ {j}\right)\rightsquigarrow \mathcal {N} \left(0, v ^ {2} \left(e _ {j} ^ {*}\right)\right), \quad a n d \quad \sqrt {N} \left(\hat {\tau} - \tau\right)\rightsquigarrow \mathcal {N} \left(0, v ^ {2} \left(e ^ {*}\right)\right),
$$
$$
v _ {j} ^ {2} (e _ {j} ^ {*}) = \frac {1}{p _ {j}} \Big (\frac {\sigma_ {j} ^ {2} (1)}{e _ {j} ^ {*}} + \frac {\sigma_ {j} ^ {2} (0)}{1 - e _ {j} ^ {*}} \Big), \quad a n d \quad v ^ {2} (e ^ {*}) = \sum_ {j = 1} ^ {m} p _ {j} ^ {2} v _ {j} ^ {2} (e _ {j} ^ {*}) + \sum_ {j = 1} ^ {m} p _ {j} (\tau_ {j} - \tau) ^ {2}.
$$
In addition, the standard errors in Eq (7) are consistent:  $\hat{v}_j^2 - v_j^2(e_j^*) = o_p(1)$  and  $\hat{v}^2 - v^2(e^*) = o_p(1)$ .
Theorem 3 shows the asymptotic normality results of the estimated treatment effects under our proposed adaptive experiment strategy. In addition, Theorem 3 verifies that the constructed confidence intervals in Eq (8) attain the nominal coverage thanks to the consistency of standard errors. The proof of Theorem 3 relies on the convergence of the actual treatment allocation in Theorem 2 and and the martingale central limit theorem [21].
4 Simulation evidence
In this section, we evaluate the performance of our proposed fair adaptive experiment strategy through simulation studies. We summarize the takeaways from the simulation studies as follows. First, our proposed fair adaptive experiment strategy achieves higher estimation efficiency than the complete randomization design. Second, compared to a classical adaptive experiment strategy, our method avoids disproportionate treatment assignment probabilities across different groups of participants and accounts for participants' welfare.
Our simulation design generates the potential outcomes under two data-generating processes. DGP 1: Continuous potential outcomes  $Y_{i}(d)|X_{i}\in \mathcal{S}_{j}\sim \mathcal{N}(\mu_{d,j},\sigma_{d,j})$ , where  $\pmb{\mu}_1 = (1,4)^{\intercal}$ ,  $\pmb{\mu}_0 = (4,2)^{\intercal}$ ,  $\pmb{\sigma}_1 = (2.5,1.2)^{\intercal}$ , and  $\pmb{\sigma}_0 = (1.5,3.5)^{\intercal}$ . The group proportions are  $p = (0.5,0.5)^{\intercal}$ . The group-level treatment effects are  $\pmb{\tau} = (-3,2)^{\intercal}$ . DGP 2: Binary potential outcome:  $Y_{i}(d)|X_{i}\in \mathcal{S}_{j}\sim$  Bernoulli  $(\mu_{d,j})$ , where  $\pmb{\mu}_1 = (0.6,0.2,0.3,0.4,0.1)^{\intercal}$ ,  $\pmb{\mu}_0 = (0.1,0.5,0.3,0.4,0.6)^{\intercal}$ . The group proportions are  $p = (0.15,0.25,0.2,0.25,0.15)^{\intercal}$ . To mimic our first case study (in Supplementary Materials), consider the log relative risk as the parameter of interest:  $\log \mathbb{E}[Y(1)] - \log \mathbb{E}[Y(0)]$ . The group-level treatment effects are  $\pmb{\tau} = (1.79, - 0.92,0,0, - 1.79)^{\intercal}$ .
We compare three experiment strategies for treatment assignment: (1) our proposed fair adaptive experiment strategy, (2) the doubly adaptive biased coin design (DBCD) [70], and (3) the complete randomization design, which fixes the treatment allocation probability to be  $1/2$  throughout the experiments. To mimic the fully adaptive experiments, we fix stage 1 sample size at  $n_1 = 40$  and  $n_t = 1$  for  $t = 2, \ldots, T$ , where the total number of stages ranges from  $T \in \{40, \ldots, 400\}$ . We evaluate the performance of each strategy from two angles. First, we compare the standard deviation of the ATE estimates to evaluate the estimation efficiency. Second, we compare the fraction of participants assigned to the treatment arm in each group to evaluate the fairness in treatment allocation. The simulation results are summarized in Figure 1.
We first focus on (A) and (B), which correspond to DGP 1. Panel (A) depicts standard deviations of the treatment effect estimates under the three experiment designs. It clearly demonstrates that our proposed method achieves higher estimation efficiency compared to complete randomization.
Figure 1: Comparison of the proposed adaptive experiment design strategy, the complete randomization design, and the doubly adaptive biased coin design. (A) and (C) show the standard deviation comparisons. (B) and (D) show the percentage of participants allocated to the treatment arm in each group under different experiment strategies.
Panel (B) shows the treatment assignment probabilities produced by the three experiment design strategies. Not surprisingly, complete randomization allocates  $50\%$  participants to the treatment arm regardless of their group status. On the other hand, the DBCD design may produce extreme treatment allocations. In addition, participants in different groups may receive drastically different treatment allocations, which can raise fairness concerns. Encouragingly, our approach generates treatment assignment probabilities that not only are closer to  $50\%$  (i.e., less extreme) but also exhibit less variation across groups. Panel (C) and (D) summarize the simulation evidence for DGP 2 in which the outcome variable is binary. A similar pattern emerges: our fair adaptive experiment design approach improves upon complete randomization, delivering more precise treatment effect estimates. It also accounts for fairness and participants' welfare in assigning treatments.
The simulation results demonstrate the clear trade-off between fairness/welfare and statistical efficiency by adopting our proposed fair adaptive experiment strategy. Although it involves a minor sacrifice in estimation efficiency when contrasted with the DBCD design, our approach delivers more fair treatment allocations and safeguards participant well-being. As our proposed method does not restrict each group to have exactly the same treatment assignment probabilities as in the complete randomization design, it improves the estimation efficiency of ATE. We provide additional simulation results and synthetic data analyses in the Supplementary Materials.
In this work, we propose an adaptive experimental design framework to simultaneously improve statistical estimation efficiency and fairness in treatment allocation, while also safeguarding participants' welfare. One practical limitation of the proposed design is that its objective mainly aligns with the experimenter's interests in estimating the effect of a treatment, as opposed to the interests of the enrolled participants. This aspect offers opportunities for future research exploration.
[58] Sofia S Villar, James Wason, and Jack Bowden. Response-adaptive randomization for multi-arm clinical trials using the forward looking gittins index rule. Biometrics, 71(4):969-978, 2015.
[59] Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228-1242, 2018.
[60] LJ Wei and S Durham. The randomized play-the-winner rule in medical trials. Journal of the American Statistical Association, 73(364):840-843, 1978.
[61] Waverly Wei, Maya Petersen, Mark J van der Laan, Zeyu Zheng, Chong Wu, and Jingshen Wang. Efficient targeted learning of heterogeneous treatment effects for multiple subgroups. Biometrics, 79(3):1934-1946, 2023.
[62] S Faye Williamson, Peter Jacko, Sofia S Villar, and Thomas Jaki. A bayesian adaptive design for clinical trials in rare diseases. Computational Statistics & Data Analysis, 113:136-153, 2017.
[63] Yuhang Wu, Zeyu Zheng, Guangyu Zhang, Zuohua Zhang, and Chu Wang. Adaptive a/b tests and simultaneous treatment parameter optimization. arXiv preprint arXiv:2210.06737, 2022.
[64] Yuhang Wu, Zeyu Zheng, Guangyu Zhang, Zuohua Zhang, and Chu Wang. Non-stationary a/b tests. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2079-2089, 2022.
[65] Yuhang Wu, Zeyu Zheng, and Tingyu Zhu. Best arm identification with fairness constraints on subpopulations. Proceedings of the Winter Simulation Conference, 2023.
[66] Yuhang Wu, Zeyu Zheng, and Tingyu Zhu. Selection of the best policy under fairness and equity constraints. Working Paper, 2023.
[67] Yanxun Xu, Lorenzo Trippa, Peter Müller, and Yuan Ji. Subgroup-based adaptive (suba) designs for multi-arm biomarker trials. Statistics in Biosciences, 8(1):159-180, 2016.
[68] Guosheng Yin, Nan Chen, and J Jack Lee. Phase ii trial design with bayesian adaptive randomization and predictive probability. Journal of the Royal Statistical Society: Series C (Applied Statistics), 61(2):219-235, 2012.
[69] Marvin Zelen. The randomization and stratification of patients to clinical trials. Journal of Chronic Diseases, 27(7-8):365-375, 1994.
[70] Lanju Zhang and William F Rosenberger. Response-adaptive randomization for clinical trials with continuous outcomes. Biometrics, 62(2):562-569, 2006.
[71] Li-Xin Zhang, Feifang Hu, Siu Hung Cheung, and Wai Sum Chan. Asymptotic properties of covariate-adjusted response-adaptive designs. The Annals of Statistics, 35(3):1166-1182, 2007.
[72] Wanying Zhao, Wei Ma, Fan Wang, and Feifang Hu. Incorporating covariates information in adaptive clinical trials for precision medicine. Pharmaceutical Statistics, 21(1):176-195, 2022.