Dynamic Subgroup Identification in Covariate-adjusted Response-adaptive Randomization Experiments
Authors: Yanping Li
Jingshen Wang
Waverly Wei
Venue: NeurIPS 2024 poster (2024)
Keywords: Covariate-adjusted response-adaptive randomization design
Response-adaptive randomization design
Subgroup identification
Causal inference
Treatment effect heterogeneity
Primary area: causal_inference

Abstract: Identifying subgroups with differential responses to treatment is pivotal in randomized clinical trials, as tailoring treatments to specific subgroups can advance personalized medicine. Upon trial completion, identifying best-performing subgroups–those with the most beneficial treatment effects–is crucial for optimizing resource allocation or mitigating adverse treatment effects. However, traditional clinical trials are not customized for the goal of identifying best-performing subgroups because they typically pre-define subgroups at the beginning of the trial and adhere to a fixed subgroup treatment allocation rule, leading to inefficient use of experimental efforts. While some adaptive experimental strategies exist for the identification of the single best subgroup, they commonly do not enable the identification of the best set of subgroups.  To address these challenges, we propose a dynamic subgroup identification covariate-adjusted response-adaptive randomization (CARA) design strategy with the following key features: (i) Our approach is an adaptive experimental strategy that allows the dynamic identification of the best subgroups and the revision of treatment allocation towards the goal of correctly identifying the best subgroups based on collected experimental data. (ii) Our design handles ties between subgroups effectively, merging those with similar treatment effects to maximize experimental efficiency. In the theoretical investigations, we demonstrate that our design has a higher probability of correctly identifying the best set of subgroups compared to conventional designs. Additionally, we prove the statistical validity of our estimator for the best subgroup treatment effect, demonstrating its asymptotic normality and semiparametric efficiency. Finally, we validate our design using synthetic data from a clinical trial on cirrhosis.

Dynamic Subgroup Identification in Covariate-adjusted Response-adaptive Randomization Experiments
Yanping Li
Jingshen Wang Division of Biostatistics University of California, Berkeley j jingshenwangbeereel.y edu
School of Statistics and Data Science Nankai University yanpingli@mail. nankain edu.
Waverly Wei x
Department of Data Sciences and Operations
University of Southern California
waverly@marshall. usc.
Abstract
Identifying subgroups with differential responses to treatment is pivotal in ran- domized clinical trials, as tailoring treatments to specific subgroups can advance personalized medicine. Upon trial completion, identifying best-performing sub- groups-those with the most beneficial treatment effects-is crucial for optimizing resource allocation or mitigating adverse treatment effects. However, traditional clinical trials are not customized for the goal of identifying pess-performing sub- groups because they typically pre-define subgroups at the beginning of the trial and adhere to a fixed subgroup treatment allocation rule, leading to inefficient use of experimental efforts. While some adaptive experimental strategies exist for the identification the single best subgroup, they commonly not enable the identification of the best set of subgroups. To address these challenges, we propose a dynamic subgroup identification covariate- oovariae-adjusted response-adaptin randomization (CARA) design strategy with the following key features: (i) Our approach is an adaptive experimental strategy that allows the dynamic identification of the best subgroups and the revision of treatment allocation towards the goal correctly identifying the best subgroups based on collected experimental data. (ii) Our design handles ties between subgroups effectively, merging those with similar treatment effects to maximize experimental efficiency. In the theoretical investigations, we demonstrate that design has higher probability of correctly identifying the best set of subgroups compared to conventional designs. Addi- tionally, we prove the statistical validity of our estimator for the best subgroup treatment effect, demonstrating its asymptotic normality and semiparametric effi- ciency. Finally, we validate our design using synthetic data from a clinical trial on cirrhosis.
Introduction
Most clinical trial designs adopt "one-size-fits-all" rules for treatment assignment and evaluation based on models that ignore patient heterogeneity. This approach is disconnected from medical practice in recent years, where physicians use each patient's 's diagnosis and prognostic variables to
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

make personalized, precision medicine treatment decisions. As such, identifying patient subgroups with differential responses a treatment plays a pivotal role in designing randomized clinical trials Adaptive clinical trials-that allow randomization probabilities to be adaptively optimized during the trial based on sequentially accrued data-have received much attention due to their potential advantages in promoting precision health. Nevertheless, these trials often involve pre-specifying the patient subgroups to be analyzed [8]. This approach not only leads to inefficient use of experimental efforts but may also reduce the statistical power to detect non-pre pree---peeeified subgroups that exhibit high effect sizes. Consequently, there is a pressing need for novel and statistical clinical trial designs for dynamic subgroup identification to address these issues.
18
18
2 39.
In this paper, we propose a novel statistical design that dynamically performs subgroup identification in CARA experiments. Our contributions are summarized as follows:
1. From the design perspective, there are three highlights of our design: (i) Our design facilitates dynamic identification and sequential refinement of the best subgroups within the framework of covariate-adjuste setting enhances the ability to identify and adjust to the best-performing subgroups over time. (ii) Unlike traditional designs that focus on identifying single best- best-performing subgroup, our design is tailored to identify the best set of subgroups with competitive performance (Section 3). This broader objective makes our approach suitable for more general application settings. (iii) Our design demonstrates a higher probability of correctly identifying the best subgroups compared to conventional designs. This efficiency ensures that experimental efforts are utilized more effectively.
response-adaptive (CARA) experiments (Section [4). This adaptive
2. From theoretical perspective, our proposed design strategy accommodates tied treatment effects among candidate subgroups, whereas many existing methods demand that these effects be distinctly separated. Additionally, our algorithm involves resampling in the presence of dependent data structures caused by adaptive treatment allocation, presenting technical challenges in proving statistical validity. We overcome these challenges and show that the best subgroup identified by our design asymptotically converges to the true set of best subgroups (Theorem [T. We also demonstrate that our proposed design strategy converges to the oracle design, which is the optimal design under the setting that the underlying data- generating distribution is known (Theorem 2 Furthermore, we establish valid statistical inference for the treatment effect of the identified best subgroup and demonstrate that our constructed estimator is semiparametrically efficient (Theorem
In comparison to the existing literature, our design strategy is closely related to CARA designs. Originating from response-adaptive randomization (RAR) designs (5 33 (46). Heuristically, CARA incorporates covariate information along with treatment assignment probabilities based on observed outcomes [4 32 30 [3). Building upon RAR designs, CARA designs utilize both outcome and covariate information to optimize for design objectives. [15] introduces family of CARA designs that balance efficiency and ethics objectives. Further generalizations to incorporate semiparametric estimates have been explored by [4)] Related developments in CARA designs include [22 40 48 47). However, conventional CARA designs often consider subgroups to be pre-specified, which is different from our goal of adaativday identifying the best subgroups.
Our proposed method also connects with the literature on subgroup identification. In post-hoc analyses using previously collected data, the first line of work uses data from prior randomized controlled trials. [20] employs clustering techniques based randomized controlled trial data. A comprehensive review can be found in [23]. The second line of work uses data from prior observational studies. [43] identifies subgroups from existing observational data using machine learning. (36) introduces causal inference tree approach for subgroup identification, which requires specifying the conditional distribution of the outcome given covariates. Similarly, (45 develops causal inference tree types of algorithm that include double robust estimators for constructing subgroup- specific splitting criteria. Other tree-based approaches for subgroup identification include [17 24 16). Besides the tree- based approaches, [10] develops a subgroup identification method within the value function framework. While post-hoc subgroup analyses do not require new data collection, they often rely on untestable causal assumptions, limiting the credibility of causal conclusions. For instance, the unconfoundedness assumption necessary for causal inference in observational studies assumes random treatment assignment based on observed confounders, but unmeasured confounders can

---

compromise these conclusions. Conversely, in randomized experiments, valid causal conclusions do not depend on such assumptions.
In the adaptive experiment literature, relatively few methods have been developed for identifying the best subgroups. While [T]] is in the adaptive experiment setting and proposes Bayesian adaptive pairs,
design that sequentially revises treatment allocation to identify the effective subgroup-treatune their approach carried out under the Bayesian framework, relying on the specification of prior distributions and does not provide theoretical justification for the identified subgroups. In contrast, our design aligned with the frequentist framework and is model-free, avoiding any parametric modeling assumption on the joint distribution of potential outcomes and covariates, and providing theoretical investigations from three aspects.
As our method aims to identify the best subgroups, it shares some similarities with the multi-armed bandit (MAB) literature. Notable MAB algorithms, such as the Thompson sampling method (35), the <-greedy algorithm, and the upper confidence bound algorithm [3]. focus on identifying the best arm. Similar the contextual bandit literature [2] 6 [] our design also incorporates covariate information. However, our design objective diverges significantly from those in traditional MAB approaches as we seek to identify the best subgroups rather than the best arm. Moreover, recognizing that randomized experiments can be time-consuming, costly, and may result in adverse outcomes for patients if treatments are ineffective, our approach seeks to efficiently allocate experimental resources within constrained budget to identify the most beneficial subgroups. Specifically, we focus on scenarios in which the cost per experimental unit (e.g., per patient) significant. For instance, in clinical settings, randomized experiments are often expensive due to the substantial costs of treatment medications. As a result, the primary resource constraint our framework is the limited number of treatments that can be administered, underscoring the need for resource-efficien experimental design.
2 Formulation of CARA
In this section, we shall introduce the formulation of our covariate-adjusted response-adaptive randomization (CARA) experiment framework.
We enroll participants sequentially across T stages, where T < oc. Denote the total number of enrolled participants as N = Σ nu. where nu is the number of participants in Stage t. for
t = 1, . T. The cumulative sample size up to Stage is denoted as N₂ = E Mg- In Stage t, denote the treatment assignment status of participant i as Die E {0, 1). = 1,.... nyy where D' = 1 denotes the treatment arm, and D;+ = 0 denotes the control arm. The observed outcome is denoted as Vv. E R.. We follow the Neyman-Rubin causal model [27 34] to define (d) as the potential outcome we would have observed if participant receives treatment d at Stage I, for d E {0, The observed outcome can then be represented as
Yu = (1) + (1 - (o),, i= L,... The t=
We assume that the outcomes are observed without delay, and their underlying distributions do not shift over time [4]
In CARA experiments, covariate information is also available to practitioners. We denote the covariate information for participant i X₃z E R and assume the covariate space X' can be partitioned into m regions, denoted as 1m In clinical settings, each partition of the sample space commonly referred to as a subgroup
[9 (4). We denote the number of subjects enrolled in subgroup ll (X.,ES)) and the cumulative sample size for group j up to Stage is
at Stage / as Tijj
Noj Σ. Maj. Denote the total number of subjects enrolled in subgroup j as N; NTJ-
As we are interested in assessing the effectiveness of the treatment in each subgroup, we define the subgroup average treatment effeet as
T (1) (O)XXa S,), j= 1.
....... m.
Due to the adaptive nature of CARA experiments, practitioners can sequentially revise treatment allocation based on outcome and covariate information accumulated during the experiment. Formally, we define the treatment assignment probability for participants in subgroup
exj P(D 1 S,, H.-1), t= 1, .....III j == =1, I,..., m,
3

---

where Hi-1 {(Y.. Dis, Xie) denotes the historical information up to Stage t - 1. In CARA experiments, we aim to dynamically revise Etj to reach desired design goals, which shall be introduced in the following section.
3 Design objective for best subgroups identification
In real-world applications, suppose start with m subgroups, and practitioners may only aim to find subgroups with the largest treatment effects. It is possible that the best-performing subgroup is unique. For example, the FORTE trial is clinical trial aiming to investigate the treatment effect of carfilzomib-based
regimens on a patient's progression- free survival rate [25]. Instead of reporting the single best subgroup with the largest survival rate improvement, the trial reports that both the risk myeloma patient subgroup and high-risk patient subgroup show similar survival rate improvement.
Without loss of generality, suppose the population subgroup treatment effects follow the order 7i 72 ... 2 7k ... > Tm- In this case, there are k subgroups exhibiting the largest treatment effects. Thus, it natural to identify all of the k best subgroups instead of the single best subgroup.
To describe the set of best-performing subgroups, we introduce the concept of tie set. We denote the tie set of 71 Ti = { I 7kl o(N-1/2), == 1...., . which contains the indices of the tied subgroups. This tie set also known as the "near tie set" as it captures the subgroups of which the treatment effects lie the looll neighborhood of Tı. We then denote the subgroups that belong to the best set as ST,- Note that when Ti [1), ST, is equivalent to Sj. The population treatment effect under the best subgroups is defined as 7T, = (1) (0) X5t Sj) We further assume that TT Tj, for j $ TT-
Our design objective to correctly identify all the best subgroups. Mathematically, we aim to maximize the correct identification probability:
max P( 2 max R,), where Ti = (1,...,,k)
J
Leveraging the large deviation theory g
formulate our design objective as
) { min G(ST,, 2(VT)(E)) (,,,,) Maximize correct selection probability iETi
max
s.t. 8 S 5 5,
Feasibility constraints
where Ti- [1, ....... kJ. 1 k m, 8 E (0, 1/2), and VT denotes the variance of the best subgroups. Detailed derivations of the equivalence between the correct identification probability and the optimization objective see Appendix (Section C).
However, in practice, solving this optimization problem is challenging. On the one hand, as we do not have knowledge regarding the membership of subgroups that have the largest treatment effects, we do not have any information regarding ST.- On the other hand, because experimenters have no prior information about the joint distribution of the subgroup treatment effects, T, 's and V 's also unknown. To address these two practical challenges, we propose a dynamic subgroup identification algorithm that can adaptively identify and merge the set of best subgroups. Additionally, the dynamic subgroup identification method operates seamlessly under a CARA experimental strategy, which allows experimenters to sequentially learn the unknown parameters and adjust the subgroup treatment allocation to attain our design objective.
Proposed design: Dynamic subgroup identification with CARA
In this section, we shall illustrate our proposed dynamic subgroup identification strategy with CARA design Algorithm [ (Algorithm
the design strategy Section 1.1 encompassing two sub-algorithms
2 and 3, followed by the statistical inference procedure in Section 4.2 We defer several variations of our algorithm to Appendix (Section []. To clarify our design strategy, we also provide a
notation table in the Appendix (Section A, Table 2

---

4.1 Proposed design strategy
In Stage (line 1... we obtain initial estimates of the group-level treatment effect Tiy and the associ-
we perform three tasks: (1) adaptively update treatment allocation and treatment effect estimates, (2) dynamically identify best subgroups, and (3) select hyperparameters that help with dynamic subgroup identification.
ated variances Vıj- Then, in Stage = ................................ T - 1 (Algorithm line 6
For the first task, obtain the optimal treatment allocation ē; (6 .... Cim;) by solving the
following optimization problem based on sequentially collected data:
(.-)1,0)) - 1.))) 25 2 (0)) + 9,
s.t. Σ Pue: S c1 c2 ssss- C2 1= m
mim
V.-1,))(e,))
where 6₁ E (0,1) and 02 (0, Pm E. Ees ((0)) is the estimated subgroup propor- Eis
tion. The total number of subgroups after merging the identified tie set in Stage ( - 1 is denoted as mf - 1 + and the subscript (7) indexes the subgroup with the j-th largest estimated treatment effect. The procedure of finding T 1,1 shall be illustrated in Algorithm 2 In the set of constraints, the first one is the resource constraint, and the second one is the feasibility constraint. Because of the nonlinear objective function of the optimization problem above, we instead work with its equivalent epigraph representation:
(--1,00) F1-1,11))
2 arg IIXX
Puei Se. az Sasl- C2 min 2535m; ((V + 8 (e;)
0). (1)
1=1
To calibrate for the complete randomized treatment allocation in Stage 1, we require an additional calibration step in Stage t:
(2.))) - -N.-1,
j 1, I....,
(2)
Pl.(f)
where (t,()) = Ee (X).))))))) Ne-1.0) (1) = Σ E 1 (X)))) Dis. and Ne.0) =
Ei-1 (s, We then allocate treatments with calibrated probability ε.,() and update the subgroup treatment effects as in Eq (8).
Dynamic identification of the best subgroups (Algorithm 2) The dynamic subgroup identification algorithm for identifying Tr,1 at Stage involves a resampling step that generates bootstrap samples c. from Gaussian distribution centering around 7, at Stage and a resampling step that generates bootstrap samples with accrued data (H.,)== later stages. In line 9 at Stage t as
we identify the best subgroups
Te, {1: (0,11) 1, 1,...,m;}, where 2).1) 1f-c N, (.(1) I (Fth Fi.m)) .N, N. 575 (((1))
where the distance between the upper and lower bounds of the interval for w is of order n' with 5 0.25 guarantee the statistical validity of proposed procedure and balance the trade-off between bias and variance. Note that the dynamic subgroup identification procedure relies on a pair of hyperparameters (ct,c)), ct), which are selected data-adaptively. In what follows, shall illustrate the algorithm for selecting these hyperparameters.
Hyperparameter selection (Algorithm 3). In line of Algorithm 3. adopt bootstrap method and propose several alternative bootstrap methods in Algorithm 5 (line (3) in Appendix (Section D. Algorithm 3 involves resampling step that generates bootstrap samples Ω. from Gaussian
distribution centering around 7;" at Stage 1. In line 2 we compute T = -(7 -- Timm) as
Ttj
A
1=
+ (1 A) Frj, j 1, mi.
(4)
my
5

---

where A min (0.99.
x N7) and 7 E (0, 0.2). We impose a lower bound on A to F.)
N, ensure that A does not equal to We choose 7 0.05 in our simulation studies, and our procedure is shown to be not sensitive to the choice of < 1. In line 7
we compute F," = (f Ttmm ) at
Stage & for > 1 as
j== t
(1
A) Fig.
(5)
m
where Fij computed with the bootstrap samples in Eq (8) In line 10 we compute
m
mi
(1))
Take
Σ
(1)
Busl,, Cx) = 1 (7 S &T,
(6)
k=1
where (1) = 1[ 11- [.(1)
T ((1)) S 2 N, 1.-6.97 V6 (.(1) and the subscript (1) indexes the subgroup with the largest estimated treatment effect. We also propose a double bootstrap-bret based alternative hyperparameter selection procedure in Algorithm in Appendix (Section[]. Finding the optimal hyperparameters involves minimizing a loss function [1( Ca) at each Stage 1, which is defined as
L (a) = (I+o (C,, ca) + Le1 (c2,
L1 (1 = 1(Bllaa000=)) Given
where for 1 0, Lu(c, cx)
a desirable pair of hyperparameters (cL, ca) and value, the indicator function 1BB 02) =
1) is binary, which roughly follows a Bernoulli distribution with probability Σ ((8,(a.))=)
Intuitively, the loss function defined in Eq (7) measures the average of squared differences between 1 (Bw(cc, Cx) I) and the expected value of Bermoulli( Σt ((Ba(q..4)=)) random variables. We would expect that the optimal pair of hyperparameters (cf, cg) at Stage t minimizes such a loss.
B
Algorithm 1 Dynamic subgroup identification CARA design
Stage 1 (Initialization):
1: Enroll n1 participants, and assign treatments in group j with E1, =
2: Compute Fiş and Viz:
3: Choosing hyperparameters (ct, cg) using single bootstrap method (see Algorithm
4: Identify tie set and merge tied subgroups with the best subgroup (see Algorithm Stage t (Adaptive treatment allocation revision):
5: for + 2 to T do
6: With 7,)), and V/-1, estimated using Eq 8 solve the optimization problem in Eq (1
find ı(,))i
7:
Enroll ne participants and assign treatment with calibrated probability Zi,00 in Eq
8:
Update Fr.0y) and Ve.G) as in Eq Choosing hyperparameters (ct, c using single bootstrap (see Algorithm 3):
9:
10:
Identify tie set and merge tied subgroups with the best subgroup (see Algorithm 2
11:
Calculate the merged subgroup ATE estimator T.. in Eq (9. and its variance estimator
T. in Eq 10
end for Stage T (Inference):
13:
Identify the best tie set 7, and construct two-sided confidence intervals for द/, as in Eq
4.2 Statistical inference
Our CARA design also enables making valid statistical inference on the estimated best subgroup treatment effect. We highlight two parts of the statistical inference procedure: (1) estimating unknown

---

parameters based on accrued experimental data while dynamically identifying best subgroups, and (2) constructing valid confidence intervals confirm the estimated best subgroup treatment effect.
First, based on accumulated experimental data, the subgroup treatment effects and associated variances be updated
1-1) == E (,) DioVis E Ei 1 (x,,)))))) (1 Yis
(8)
N1-1,6111
N2-1,0))))
V (..)(()ee)))) Eesl E1=1 ((XIES)))) Dis (Yis -V--,,
(e) (g..LL))))
N (1)1(1
N1-1
Σ Dist (x))) (1- Dis) -b-1,,)(0))
(1-cz) ez)
Ne-1
N1-1,6)(())
No-1
where (1) E i= (x,, Dio (1-1,())0) E.
Di,), N1-1,) = Σ Ei (()))))) ()) = iill (xx)) (s))))
N.,.Y../Ne-1..))000 and (1) = E. Ee Dis.
Additionally, denote the subgroup proportions as Pr...., Pm. Following dynamic subgroup iden- tification at each stage, we merge all subgroups in the Tei and estimate merged best subgroup treatment effect as
-
P
Σ Ps: JETA
(9)
jETA
and estimate the variance of the merged best subgroup treatment effect estimator as
Σ VJ /
2
15
Pj JETA
(10)
JETa
In Stage (line 13
we let 7. := T. के := म, and V == n. Lastly, to confirm the
estimated best subgroup treatment effect, we construct a two-sided level-a confidence interval as
tq. = (1 /V
(11)
Algorithm 2 Dynamic subgroup identification in Stage /
Step (Input):
Input [H,).- 1= Prj. Vej. and (ct, c&) computed from Algorithm Step (Bootstrap):
2: for
B do
3: t then
4: Generate 7p from N (m, where ②., = diag V
Vim)
5: else if > then
6: Generate n, resamples randomly with replacement sequentially from each Hg;
7: Compute Tt as in Eq (8) with the bootstrap samples;
8:
end if
9:
Identify the best subgroups Fa with the bootstrap samples as Eq 3
end for
Step B (Output):
11:
Choose Te with the highest frequency of occurrence and merge subgroups that belong to Fe..
7

---

Theoretical investigation
Assumption 1 (Regularity conditions). (0), (1), xxt are independently identically distributed for =.
= =.....,?? T. In addition, (d)] < oc, d E {0, Lastly, there exists some _Tt, 1.....,,.
ò > such that (d) E S,] 2 8 for d E {0, 1), = I, ..... m.
Assumption (Positivity). The subgroup proportions 8 Σ ph,.... Pm S - 6, ā E (0,
Assumption T says that the potential outcomes have bounded moments and have variability in each subgroup. Assumption 2 says that the subgroup proportions are non-zero the population.
Theorem 1 (Dynamic best subgroup identification consistency). Under Assumptions j ......, m and for O, we have
and for
lim P PILCE FT) 1(j E Ti)l > N-000
Theorem suggests that our dynamic subgroup identification algorithm correctly identifies the best set of subgroups as the sample size tends to infinity.
Theorem 2 (Design strategy consistency). Under Assumptions 7 and for ō > 0, as The - oo,
t= 1....... T,
for the actual treatment allocation, we have
P(le e - 6)
where e" === (ei, em) the optimal treatment allocation.
Theorem 2 says that the actual treatment allocation under our proposed design strategy converges to the optimal treatment allocation asymptotically.
Theorem (Asymptotic normality). Under Assumptions 7 and 2 as N + co.
VN (Tn) N(,,
and
7 - V Opl
Theorem 3 says that the estimated treatment effect of the identified best subgroups converge to Gaussian distribution asymptotically, and our variance estimator consistently estimates the asymptotic variance. Theorem 3 also verifies the validity of our constructed confidence interval.
6 Synthetic real data study
In this section, we investigate the performance of our proposed design strategy for identifying the tie set of best-performing subgroups in a synthetic case study using clinical trial data.
We design our synthetic case study using the dataset from the Mayo Clinic's trial on primary biliary cirrhosis (PBC), containing clinical biomarkers, treatments, and patient outcomes. PBC is progressive autoimmune liver disease marked by inflammation and damage the intrahepatic bile ducts. The Mayo Clinic conducted an extensive trial from 1974 to 1984 to assess the effectiveness of D-penicillamine treating PBC. This dataset includes 424 patients, encompassing both those who were actively enrolled in the trial and additional cases who consented to provide basic measurements 26
In this case study, we work with a subset (n = 312) of patients who participate in the randomized controlled trial. These patients randomly assigned to one of the two arms: the treatment arm (n = 158), who receive D D-penicillamine D) = 1, and the control arm (n = 154), who receive placebos D 0. The outcome of interest is the square root of the survival time, defined as the number of days from registration to the earlier death, transplantation, or the time of study analysis. This dataset includes covariates, and we use median imputation to handle missingness these covariates. We aim to investigate the effectiveness of D- D-penicillamine in improving liver function and symptoms in five subgroups defined by age (in days): (1) patients with age [9, 598, 15, 695), (2) age in (15, 695, 17, 082). (3) age in (17, 082, 20, 440), (4) age in (20, 440, 21, 900). (5) age in (21, 900, 28, 650). We generate synthetic experimental data based on the original dataset, which shall be illustrated in the next section.
8

---

We generate synthetic data that mimic the original PBC dataset. Denote the sub-
Algorithm 3 Hyperparameter selection for dy-
group membership for each participant as S = (x,es)) ..... (x.es,))). We generate
namic subgroup identification
Step (Input):
the potential outcome from [X, E Sj M(
1: Input {H+) Fa.. Vy and (cI.
1,...,,, 5, where μ
2: Compute Tū as Eq (4).
(42.57, 50.44, 44.37, 44.30, 37.71) Po
Step (Bootstrap):
(45.34,39.91) 45.58, 33.42, 39.17) 01
3: for 1 to B do
(10.85, 12.29, 12.64, 14.28, 14.64)
4: ift= then
(11.50, 15.18, 14.57, 13.09, 15.06)
Pj from N (Tii
5:
Generate
The subgroup proportions are P We denote the true subgroup treatment effects as (-2.77, 10.53, - 11221, -146)
(0.28, 0.13, 0.30, 0.11,
where Ω, diag
V 11 Vim)
6:
else if > then
Therefore, Subgroup 2 and Subgroup are the set of best subgroups with merged average treatment effect of 10.70. The treatment assignment D; is decided based on different experiment strategies, which shall be discussed
7:
Generate n,, resamples randomly with replacement sequentially from each H. (the
same Algorithm 2 line 6):
8:
Compute Tj in Eq (8) with the boot-
strap samples, and then T.g as in Eq (5)
later in the section. To generate synthetic data, We mimic CARA experiments where participants are enrolled sequentially across T experimental stages. Here, we set T = 15 and n, 400, for I. T. are conducted with an Intel Core 17-11800H CPU and 16 GB of RAM.
9:
end if
10:
Compute T1,(1) as in Eq
6
and
Bib (cz, CR) in Eq
11: end for
All experiments
Step B (Output):
12: Compute [e(CL, 0x) as in Eq (7). Choose the pair (cf, <f) that minimizes (e(C,,0)).
We compare our proposed design strategy with the complete randomization design and two multi-armed bandit (MAB) algorithms. (1) The complete randomization design refers to a de- sign that fixes Etj 2/2 across all experimental stages, = 1,.... T 1,... m. (2) To customize the MAB algorithms to our setting, for each subgroup, we consider two candidate arms: treatment and control. We set the rewards as the negative asymptotic variance of the treatment effect estimator, i.c. -V (e,). We consider two MAB algorithms: (a) The e-greedy algorithm, which aims to balance the exploration and the exploitation efforts (37. Here, we set 0.1. (b) The upper confidence bound 1 algorithm balances exploration and exploitation using confidence intervals and chooses the arm that maximizes the upper confidence bound on the estimated reward 4 When customizing MAB algorithms to our setting, methods, such as K-means or agglomerative clustering, due to several challenges: the requirement to predefine the number of clusters, the limited applicability of clustering for small number of subgroups, and the inconsistency of these methods in effectively merging the best subgroups.
omit the step of identifying and merging tie sets using conventional
0.9
Correct selection probability
0.8
0.7
10
11
13
14
15
Stage
Methods
Comclete randomization design
Epsiloo groady algorfthim
Single and seporate bootstrap
Upper contidence bound algortthm
Figure 1: Comparison of the correct selection probability among three conventional methods and our proposed design strategy. "Single and separate bootstrap" refers to our proposed design.

---

Table 1: Comparison among three conventional methods and our proposed design strategy based on estimated best tie set or subgroup treatment effect (Est), 95% confidence interval (95% CI). v V --slee bias, and standard deviation (SD).
Method
Est (95% CT)
V /NBias
SD
CR
11.33 13.36.
34.55
80.29
e-greedy UCB
11.26 20.3200113311
28.84
81.19
11.34 (9.29. 13.38)
34.82
80.85
Proposed
10.31 (9.28, 11.35)
29.98
40.94
To evaluate the performance of different design strategies, we assess the effectiveness of each adaptive experiment strategy from two aspects. First, compare the correct selection probability of identifying the best subgroups. The correction selection probability can be written as PFr,,, 2 max Fy). Second, we compare the 95% confidence interval, vN-scaled bias, and standard deviation of the estimated best subgroup treatment effect. In our resampling procedure, we set B = 2, 000. The synthetic case study results are summarized in Figure and Table [
First, from Figure [] our proposed design strategy shows higher correct selection probability than the complete randomization design and the MAB algorithms. Additionally, the correct selection probability under proposed design strategy increases with the number of experimental stages, Specifically, our proposed design has correct selection probability tending to 1 after 15 experimental additional metric compare
stages. We also adopt the normalized mutual information proposed design and three competing methods in the Appendix (Section K.Ip. which further confirms that our proposed design strategy outperforms the conventional methods.
Second, from Table [ we observe that our proposed design strategy has a smaller standard deviation and smaller sca-lel bias, implying that our method is more efficient and less biased. In sum, our proposed adaptive design demonstrates efficient use of experimental data to correctly identify the best-performing subgroups the three competing methods. We defer additional simulation studies to the Appendix (Section K. and additional synthetic real data study to the Appendix (Section K.3). We also extend our proposed dynamic subgroup identification with CARA to the augmented inverse propensity score weighting (AIPW) estimator. Then compare our proposed design with AIPW estimator with the three contextual MAB algorithms in the Appendix (Section K.2
7
Discussion
We propose dynamic subgroup identification method within the CARA design framework that could significantly advance precision medicine. However, we acknowledge some limitations in our approach. We aim to explore and address these challenges in our future research.
First, our method assumes that outcomes CARA experiments are observed immediately at the end of each stage without delay. This assumption, prevalent in adaptive experiments such as [15] and [49] simplifies the modeling process and facilitates quick adjustments based on the latest data. However, in some practical scenarios, outcomes may be observed with delays, complicating the process of adjusting treatment allocations, as highlighted by [31 and (9). future work, we plan to revise our design framework and update our estimators to account for the impact of delayed responses on both treatment effects and variance estimators.
Second, our work addresses scenarios where assigning a treatment is costly, and there is an overall constraint on how many treatments can be deployed. A key challenge arises from the potential misalignment between efficiently estimating the best causal effect and determining the optimal causal decision rule particularly when the best decision rule to treat all patients when there any positive effect. For instance, [9] focuses on ensuring that all individuals who would benefit are accurately assigned to the treatment arm, assuming negligible treatment costs. maximize the welfare for participating subjects, in future work, we plan to incorporate an "early stopping" step which would not only identify the most effective subgroups but also halt enrollment for subgroups exhibiting significantly adverse treatment effects.
10

---

References
[1] Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning, pages 127-135. PMLR, 2013.
[2] Derek C. Angus and Chung-Chon H Chang. Heterogencity of treatment effect: estimating how the effects of interventions vary across individuals. Jama, 326(22)23312-23333 2021.
[3] Susan F Assmann, Stuart J Pocock, Laura E Enos, and Linda E Kasten, Subgroup analysis and other (mis) uses of baseline data in clinical trials. The Lancer, (55(9209):0044 1069, 2000.
[4] Peter Auer. Nicolo Cesa- Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning. 47:235- 256, 2002.
[5] Uttam Bandyopadhyay and Atanu Biswas. Allocation by randomized play-the-winner rule in the presence of prognostic factors. Sankhyä: The Indian Journal of Statistics, Series B, pages 397-412, 1999.
[6] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 208 214. JMLR Workshop and Conference Proceedings, 2011.
[7] Rajeev H Dehejia and Sadck Wahba. Propensity score-matching methods for nonexperimental causal studics, Review of Economics and statistics, 84(1):551-111, 2002.
[8] Amir Dembo and Ofer Zeitouni. Large deviations techniques and applicaaioos, volume 38. Springer Science & Business Media, 2009.
[9] Carlos Fernández- and Foster Provost. Causal decision making and causal effect estimation are not the same... and why it matters. INFORMS Journal on Data Science, 1(1):4166, 16, 2022.
[10] Haoda Fu, Jin Zhou, and Douglas E Faries. Estimating optimal treatment regimes via subgroup identifi- cation in randomized control trials and observational studies. Statistics in medicine, 35(19):3285-330 2016.
[11] Wentian Guo, Yuan Ji, and Daniel VT Catenacci. A subgroup cluster-based bayesian adaptive design for precision medicine. Biometrics, 73(2):367-377, 2017.
[12] Frank Hollander. Large deviations, volume 14. American Mathematical Soc., 2000.
[13] F. L.-X. Zhang. Asymptotic normality of urn models for clinical trials with delayed response. Bernoulli, 10(3)444-463 2004.
[14] Feifang Hu and William Rosenberger. The theory of response-adaptive randomization in clinical trials. John Wiley& & Sons, 2006.
[15] Jianhua Hu, Hongjian Zhu, and Feifang Hu, A unified family of covariate-adjusted response-adaptive designs based on efficiency and ethics. Journal of the American Statistical Association, 10(5509)3357-377, 2015.
[16] Joseph Kang, Xiaogang Su, Brian Hitsman, Kiang Liu, and Donald Lloyd-Jones. Tree-structured analysis of treatment effects with large observational data, Journal of Applied Statistics, 39(3):513 (9(3)2533-529, 2012.
[17] Joseph Kang, Xiaogang Su, Lei Liu, and Martha I. Daviglus. Causal inference of interaction effects with inverse propensity weighting, g-computation and tree-based standardization. Statistical Analysis and Data Mining: The ASA Data Science Journal, 7 2014.
[18] David Kent, Ewout Steyerberg, and David Van Klaveren. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. Bmj, 363, 2018.
[19] K Kubota, Y Ichinose, G Scagliotti, D Spigel, JH Kim, T Shinkai, K Takeda, S-W Kim, T-C Hsia, RK Li, et al. Phase ii study (monetl) of motesanib plus carboplatin/paccittxell in patients with advanced nonsquamous nonsmall-cell lung cancer (nscle): Asian subgroup analysis. Annals of oncology. 25(2): 529-536, 2014.
[20] Beom S Lee et al. A clustering method to identify who benefits most from the treatment group in clinical trials. Health Psychology and Behuvioral Medicine: an Open Access Journal, 2(1):723-73) 2014.
[21] Lihong Li. Wei Chu, John Langford, and Robert Schapire. A cantextual-bandiie approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661-670, 2010.
11

---

[22] Yunzhi Lin. Ming Zhu, and Zheng Su. The pursuit of balance: overview of covariate-adaptive randomization techniques in clinical trials. Contemporary clinical trials, 45:21-25, 2015.
[23] Ilya Lipkovich, Alex Dmitrienko, and Ralph B D' D'Agostino Sr. Tutorial in biostatisties: data-driven subgroup identification and analysis climical trials. Statistics medicine, 36(1):136-196, 2017.
[24] Wei-Yin Loh, Xu He, and Michael Man. A regression tree approach identifying subgroups with differential treatment effects. Statistics in medicine, 34(11) (1111)):818-88333 2015.
[25] Roberto Mina, Pellegrino Musto, Delia Scalabbia,,,in. Laura Paris, Barbara Gamberi, Angelo Palmas, Sara Aquino, Paolo de Fabritiis, Nicola Giuliani, Luca De Rosa, et al. Carfilzomih induction, consolidation, and maintenance with or without autologous stem-cell transplantation in patients with newly diagnosed multiple myeloma: pre-planned cytogenetic subgroup analysis of the randomised, phase 2 forte trial. The Lancel Oncology, 24(1):64-76 2023.
[26] Paul A Murtaugh, Rolland E Dickson, Gooitzen M Van Dam, Michael Malinchoc, Patricia M Grambsch, Alice Langworthy, and Chris Gips. Primary biliary cirrhosis: prediction of short term survival based on repeated patient visits, Hepatology, 20( 126-144,4, 1994.
[27] Jerzy Neyman. On application of probability theory to agricultural experiments. essay on principles. section 9. (translated and edited by dm dabrowska and tp speed, statistical science (1990), 5, 465-480). Annaly of Agricultural Sciences, 10:1-51, 1923.
[28] Mark Pletcher and Charles E McCulloch. The challenges of generating evidence to support precision medicine. JAMA Internal Medicine, 177(4)5611562, 2017.
[29] D. S. Robertson. K. M. Lee, B. C. K.Gpez-Kolooskee and S. S. Villar. Response-adaptive randomization in clinical trials: from myths to practical considerations. Statistical Science: A Review Journal of the Institute of Mathematical Statistics, 38(2):185, 2023.
[30] David Rohertson, Kim May Lee, Boryana C. Lopezz-Kllovvk,, and Sofía Villar. Response-adaptive randomization in clinical trials: from myths to practical considerations. Statistical science: a review journal of the Institute of Mathematical Statistics, 38(2):185, 2023.
[31] W. F. Rosenberger, O. Sverdlov, and F. Hu. Adaptive randomization for clinical trials. Journal of Biopharmaceutical Statistics, 22(4):719-7366 2012.
[32] William F Rosenberger. Randomized um models and sequential design. Sequential Analysis, 21(1-2):1-28, 2002.
[33] William F Rosenberger, AN Vidyashankar, and Deepak K Agarwal. Covariate- response-adaptive designs for binary response. Journal of biopharmaceutical statistics, 11(4):227 236, 2001.
[34] Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5)6888, 1974.
[35] Daniel Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al. A tutorial on thompson sampling. Foundations and Trends@ Machine Learning, 11(1):1 96, 2018.
[36] Xiaogang Su, Joseph Kang, Juanjuan Fan, Richard Levine, and Xin Yan. Facilitating score and causal inference trees for large observational studies. Journal of Machine Learning Research, 13:2955, 2012.
[37] Richard Sutton and Andrew Barto. Introduction reinforcement learning, mit press. Cambridge. MA, 1998.
[38] Julien Tanniou, Ingeborg Van Der Tweel, Steven Teerenstra, and Kit CB Rocs. Subgroup analyses in confirmatory clinical trials: time to be specific about their purposes. BMC medical research methodology.
16: 1-1, 2016.
[39] U.S. Food and Drug Administration. ity of treatment effects in patient
Assessing and communicating heterogene- subpopulations: Challenges and opportuni-
ties.
https::/www. fda. wtw.:/iiieeeeeeeeeecercc/aaaaaaa scciacrrrryyiicceeeeeeecc
and-
and
2024.
[40] Soffa S Villar and William F Rosenberger. Covariate-adjuster response- adpatie-adaptive randomization for multi-arm clinical trials using a modified forward looking gittins index rule. Biometrics, 74(1):49-5) 57, 2018.
12

---

[41] Waverly Wei, Xinwei Ma, and Jingshen Wang. Adaptive experiments toward learning treatment effect heterogeneity. arXiv preprini urXi::2312.06883, 2023.
[42] Waverly Wei, Yuqing Zhou, 7eyu Zheng, and Jingshen Wang. Inference on the best policies with many covariates. Journal of Econometrics, page 105460, 2023.
[43] Jie Xu et al. Machine learning enabled subgroup analysis with real-world data to inform clinical trial eligibility criteria design. Scientific Reports, 13(1):613, 2023.
[44] Yanxun Xu, Lorenzo Trippa, Peter Mäller, and Yuan Ji. Subgroup-bssed based adaptive (suba) designs for multi-arm biomarker trials. Statistics in Biosciences, 8(1):199-800, 2016.
[45] Jiabei Yang. Issa Dahabreh, and Jon A Steingrimsson. Causal interaction trees: Finding subgroups with heterogeneous treatment effects observational data. Biometrics, 78(2):624-635, 2022.
[46] Li-Xin Zhang, Feifang Hu, Siu Hung Cheung, and Wai Sum Chan. Asymptotic properties of covariate- adjusted response-adaptive designs. The Annals of Statistics, 166 182, 2007.
[47] Jinglong Zhao. Adaptive neyman allocation. arXiv preprint arXiv: 2023.
[48] Wanying Zhao, Wei Ma, Fan Wang, and Feifang Hu. Incorporating covariates information in adaptive clinical trials for precision medicine. Pharmaceutical Statistics, 21(1):176-13) 195, 2022.
[49] Hai Zhu and Hongjian Zhu. Covariate-adjustee response-adaptive designs based on semiparamctric approaches. Biometrics, 2023.
13

---

Technical Appendix
A
Notations
We present a notation table, as shown Table 2 to illustrate the key symbols and their descriptions used in our covariate-adjusted response-adaptive randomization (CARA) experiment framework.
Table 2: Notation table for the proposed dynamic subgroup identification strategy with CARA design.
Symbol
Description
T
Total number of stages in the experiment.
N
Total number of enrolled participants across all stages.
Number subjects enrolled in subgroup j Stage Number of participants enrolled in Stage t.
No
Cumulative sample size for subgroup j up to Stage t.
Nr
Cumulative sample size up to Stage f. Treatment assignment status of participant in Stage t.
Da
Yu
Observed outcome for participant in Stage t.
(d(
Potential outcome for participant if assigned treatment in Stage t. Covariate information for participant in Stage
Xa
X
Covariate space.
H
Historical information up Stage Total number of subgroups. Number of subgroups after merging the identified tie set in Stage 1.
m
Pj
The j-th subgroup proportion.
Ptj
Estimated subgroup proportion of subgroup at Stage t. Treatment assignment probability for participants in subgroup j at Stage t.
C
Optimal treatment allocation probabilities Stage
T
Tie set of the highest treatment effect T1, containing indices of tied subgroups. The j-th subgroup.
S
ST.
Subgroups that belong to the best set identified by Tı.
G
Subgroup average treatment effect for subgroup j. Population treatment effect under the best subgroups in T₁.
B Assumptions, lemmas, and corollaries
Before discussions, we define some additional notations first. For f = 1, 1,...,,,72 =1, 1,..., m, and
e > 0, denote
(3,0(1) (1) E Sy). Mg E|{); E (1j,c(1) Eim En (xx,es,) Y (0)
1 (x.,es)) Drx)
(1y(1)
N2²(0)
Assumption 3 (Subgroup treatment effects). For å E 0.5), the asymptotic distance between treatment effects of Subgroup and Subgroup 1 diverges as N + co:
66 min JETi
IT
Ty xc, Vj I...., m.
Lemma 1. Assume Assumption 7 and 2 holds. Let Pj P(X,t E 891H4-1) be the subgroup proportions, and Ł7 P(D = Xxt E SJ. 1 be the treatment probabilities. Assume there exists some E (0, 1/2) such that for all j = 1, 2....... m and t = 1,2....
8 < 2 $ 8.
Then for any j = 1,2.... 2....... m, d = 0, 1, and any satisfying oc.
(c) = (d) + Op
VN,
14

---

Corollary 1. Ler d E (0, 1/2) be some constant. Assume Assumptions 7 and hold,
sup 85e51-8 1-8
Veyle) V ((( =
=0p VN..
Corollary 2. Assume Assumptions and hold. Then
Fij T Op VN.
Theorem 4. Under Assumptions and N + 00,
VN - N
v Op
Lemma 0 T and 2 and Theorem 4 are similar to the theoretical results in 41 We defer readers to [1 for more technical details.
and
Lemma 2. Under Assumptions 3 for j = -,...., m, 1, 1,..., T, any positive constant C and
ô E (0, }) the following statement holds
lim (( Ty No C) =0 No
Lemma 3. Let 5 (0,1/2) be some constant. Assume Assumptions Hand 2 hold,
T, TTi = Op
sup 65e51-6
(... (1,))v -v(le) Op (vN..
Lemma 4. Assume Assumptions 7 and hold. Let Ex be the solution(s) to the problem as follows:
(T, - )T,
(T,
max
min k+1sism
G(S S(ien.e))))))
+
s.l.
Pjey Sc,
02 <e < c2 j = -,,,..... I,..., m
k+11
where aı (0,1), 02 1/21, Ti = (1..... k}. 15 m, and VT, denotes the variance of the top subgroups. Let E. be the optimized treaiment allocations solved from the sample analog of this problem. Then for any 8 >> 0, and any l. satisfying N₁-1 - 00,
PP sup inf || - -e'l s +1 e e'
C Optimization problem objective function derivations
Proof. Because maximizing the correct selection probability is equivalent to minimizing the incorrect selection probability, we formulate the incorrect selection probability IP (fr, S max 7g) and denote the cardinality of Ti as k, S k < m 1, where m denotes the total number of subgroups. The incorrect selection probability can be bounded as
max P (fr, S
P
fr.
max J471
I (m k) max ( S Fy),
JETi
JET
where the inequality follows from the union bound. For both sides, take the logarithm and divide by N.
log max (1 - P (Fr 2 7311)
log max (1 - (fr
log (m - k) +
jēTi Additionally, by large deviation theory g there exists rate function G(ST, Syi E1, ,,),,,,)))).) ej), such that
N
N
JēTi
lim No
log (1 - 1-P (f 2 9)) = -(,,,, 14
N
15

---

Therefore, jaxjeT:
(1-P(fT, 2 f1)) is equivalent to exp
(opp-mmmng minjer C(ST, ej)). We
obtain
1
min G( J071
S
N log -
PT
max
J
log (m - k))
min
N
je
As result, based on the Gartner-Ellis Theorem 8 are able to obtain:
ch.2.3], when the sample size tends to infinity,
Jim
N log (1
P
max f
min Sfje,,)) J471
3471
(T) Tr )² 2(V7, (61) + (e,)))
D Proof of Lemma 2
Proof. Note that
VN. (T) - -Ty) = (f Ty) /No (Ti - fv))
As for the first term /N, (Fe) T3), by Corollary 2 we have
No (f. - - = 0,
As for the second term v N. (fg F.y). we consider two situations under / = 1 and <>> 1 separately.
When Fig is generated from N (Tip /15/N1) so have
VN ( -ij) = (1).
When > we compute F the same way as Rtj with the bootstrap samples instead, so Fi and Tty are consistent and we have
VN, (T) n.)) O1)
Therefore, we obtain for 1, N1 ( T2) Op (1), i.e., for any given >> there exists an M, such that
V N₁ + Ty M) <
Then for any N₂ such that N, >
M 8 we have
P (17 Ty
6 <
and thus
limsup P (1 Ty 2 N₂ C) <
n+000
Note that the above inequality holds for arbitrary 0. Hence, we have
lim (1i Tyl > N C) = 0,
n
which completes the proof.
16

---

E Proof of Theorem I
Proof. Recall our definition of the tie of best-performing subgroups:
T = [j In Tyl = o(N j = =1, m..
This suggests that Vj Ti, there exists sequence 8, - as N - such that
Tj + N 8n Vj E TT.
We further define a set of subgroups with treatment effects smaller than those in the set Ti:
7 = (j:: :
min {Tm), j m.. mET:
As for the estimated near tie set of Ti in Stage t for t = 1, 1,..., T, we have for any E Tri, and a pair of hyperparameters (cf. c)) that
- N, L()) S -())) S ch No L())
where 8 0.25. Thus, there exists positive constant C such that
i Plol < C, Vj E Fe1.
Similar to the proof of Lemma in 42 our proof is composed of the following three steps:
Step 1. We first show that the subgroup with the largest treatment effect in the resampled statistics falls into the set Ti with high probability, that is
lim P (I, E Tı) = 1, No0
(12)
where i, = Ema j 1 (())) fort L..... T.
Because 72,1, [minjeT Fzz. max Fij by definition, coupled with the fact that
max 1k KET
min Fo JET
max fij JETi
(I+ TT).
Under Assumption by Lemma 2 for any A T and j E Tı, we have
lim P ( < Tt 1,
N
so we can obtain lim 00 P max t < minjen Fi & maxjeTi Ti)
= =1 Thus we have
shown that lim P (I, Ti)= 1.
Step 2. We then show, for $ Ti, and I = I.
lim ((11 f1)eef))ee, Ti) = 0. No0
(13)
For any e > and & Tı, we have the following holds
(l1ue faliefa)lpp >))
=P -P (L1ie inn) > > Fni) -P (jEJT.))
PP (10 Fni) > e/j $ Trl) (j Fm
<P
[j E Fn
[ [((
Defp
cc)
N.-
17

---

7
N₂ <0
(1) 1..1 < No... c))
(16)) 77) (+)) 7) + -r,)) N.O.CO 0)
(Nz 7. - N2 [3
No [.. - - <C)
=P (Nº [+) - 7 + N1 |1 - -
> 17, - 7 + +C).
By definition, for j $ Tı.
P (N. IT Til < () s P( Ty- -
< -
+P
max (N - C,k Ti) (T) kE
Under Assumption Lemma 2 and the conclusion in Eq 12) in Step 1 suggest that by letting Ne + oo on both sides, we have the above probability converges to zero. Based on above derivation,
we have shown that No P (1( E Ta) > e.j & Ti
Step 3. We are left to prove that for all j E Ti and t = 1,...,? T, the following holds for all E > O:
(11G F.)) - 1 1 > >z)= =0.
lim No
Following similar arguments, for a positive constant C, we have for k. E Tı, the following statement holds:
P ( [(j E tn) - 1 E)
=P (P(n E ta) - 1 > elj E Ta)) -P PP TTa)
( E Fr) - 1 > e/j $ kml) PP (j $ Fm
+
<P
( $ Fa
Del p
2 C
N
|70 1
=P
N:6 20)
(+) - 7-)) - ( -7) + (T, - (1)--)]1 1))22 No c)
<P
[7 - [3| + |Tt) $| +
[5. -
2 N c)
<P N [0 Ty| + N + IT, ল|
No
/, न 2 22-55.0) c).
By definition of the near-tie set, for j E Ti. we have
P N (N2 br, . c) <P (Nr' IT,
ল| <
C, Ti) ( Ti)
maxP (N 1 C,k E Tı) + P (i)
RETI
Again, under Assumption Lemma and the conclusion in Eq (33) we have derived in Step 1, by letting N₂ + so on both side, we have the above probability converges to 1. According to above
discussion, we have shown that lim P [10 E F.) - > 8 = O.
Combining the results obtained in the aforementioned three steps, we have for 1, -,..., m and for E > 0, the following holds:
lim P (ILU) F.. - 1(j E TI) > $) 0, Vt = 1,..., T.
No0
Then it is straightforward to get the final results by taking t T.
18

---

Proof of Lemma
Proof. Note that
T 7 17i = (7, - (T..)) + (F.,. TT,) T. (e) (e) ( F (e) V + (e) (e))
Thus this lemma follows directly from Corollary Corollary and Theorem
G Proof of Lemma
Proof. Denote the oracle objective function as
(T) - (T,-,,)²
(T6-1T1H+1) mT,
f(e)
min m +
min ITi+isism
VT, Vo-Minll)))
where the second equality follows from regarding the tie set Ti as whole after merging the data, and then Tj ranks and other subgroups rank from to m [T + Then, the original optimization problem can be written
max f(e),
S.L.
Σ
S
2 S 02. j [T L....,, m.
1= +1
For notational simplicity, let be the set candidate allocation rules (i.e., all allocation rules satisfying the constraints).
Similarly, let Fi(e) be the estimated objective function, that is, with m 1T
(fc-1.0) 1...17._111 }
fi(e) = min
25
1-177 (0i) + 8 (1)
(74-1.15 1.1 +1) T -
min
H1Sism /,1, (e1) + V/ 2- 1 (t-na)+))ee)))
We solve the following optimization problem:
max Id(e),
Σ
Dje S1, 02 Ss 62 j == T ,1...,, m.
S.L.
We denote the set of maximizers to the above optimization problem E.. Lastly, let the enlargement of E be E" + Bg, where Bş is the Euclidean ball centered at the origin with radius &:
3 B3 fe u: e Ex ||ul| = 5}. Finally, notice that supees (f((e) f(e) (1()) due
to Lemma 3 and Theorem which closes the proof.
By the same proof of Lemma C.2 in [41, we have e- C E + Bs.
H Proof of Theorem 2
Proof. Adopting the same proof of Theorem
41, we obtain that
T En 1l (x,,,s,)
(Di
Σ E lxnnss
ê Σr En ((x(ES))
Σn En ((xnES))
1
(1+
Σ
m N
=0p VN
From Lemma 4 one has
sup
eg = (1).
2513T
19

---

Then with the calibration procedure presented in Section 4.1 by replacing Bi with εzy. we have that
Σ
neey
T
74
(Gig(ne) + M2-13) (1)
N
N
==1
T
T
Cigy Nc - N1-15(1)
Ct
N
N
t=1
1=1
T
1-1
= =
(X.ES))
(Di--Eg)) + o,((1)
Npj
oj + o1(1),
where the second term in the last equality has the order of Op(I/NN with variance calculation. Thus, we complete the proof.
I Proof of Theorem 3
Proof. Note that
77. - 77 = =(T7 - (T)) + (fT - -TT));
and
Σ Pj
ΣDJ
Pj - Σ PsTj
Σrj
TTi
jETi
jET
jET1
jETI
=
Σ
Pÿ G T²)
Σ Pj.
jET
JETI
Applying the result presented in Theorem
-T) (,N,
we have that
VN(TT, TT,) 3 N N((1, (oii
(Ejen, pi) By Theorem
where Vy, (ei = Ejen vjv, (et)
we also have (4. - tt 0.
Thus by Slutsky's theorem,
VN - TT) -n) P (0
Consistency of the variance estimator follows from Theorem and Lemma which similar the proof of Lemma 3
J Additional algorithms
In this section, we provide another data-adaptive algorithm for choosing hyperparameters for Stage and another bootstrap-based merging data bootstrap for later stages.
We present a double bootstrap method for choosing hyperparameters Algorithm 4 as an alternative to Algorithm]
20

---

Algorithm 4 Hyperparameter selection in Stage 1
Step (Input):
1: Input Hi, Tij. ViJ. and (ct, ca).
2: Compute Ti as in Eq
Step (Bootstrap):
3: for b + 1 to B do
4: Generate 71 from N (nn where ŋ2, - diag (Vn...., Vam)
5: r 1 to R do
6: Generate F1 from M/ (7i. fr..//
7: Compute 7** as in Eq (6: '1.(1)
8: end for
9: Record (b) (CL, 02) as in Eq 14
10:
end for Step B (Output):
11:
Compute L1 (c,,, C2) as in Eq (14.. Choose the pair (ct, Gg) that minimizes (1,,,00 Cz).
Here, we define the loss function
1 R (I,(1) (cL, on) R
2
Σ
(a)) sT)),, L1 (c,, (x) =
B Σ
(1,0))(2,
B +
(14)
where (I,(b) (Cu, CR) is the 6-th smallest statistics in Bi (c,, ...., BIB Ca).
We could know that P(Fi.(1) I (Y Dis, x2+) roughly follows Unif(0,1) when the sample size n1 is large. Given desirable pair of hyperparameters (c,, Ca). we would thus expect that B1 (1 (R), ...,................... B BB,(B) (ct, CR shhree similar distribution with the ordered statistics of i.i.d. Unif(O, 1) random variables. The loss function defined in Eq (14) measures the average of squared differences between (1,(0) (Ct, Ca) and the expected value of the order statistics of the Unif(0,, random variables. Given the rational above, we would expect that the optimal pair of hyperparameters (cl, og) Stage 1 minimizes such a loss.
We present naïve bootstrap method for identifying and merging data for later stages in Algorithm as an alternative to Algorithm 2
Algorithm 5 Subgroup identification in Stage t (t > 1)
Step (Input):
1: Input (H,). 14 Pej. V Vej. and (ct, ck) computed from Algorithm 3 or Algorithm 4
Step (Bootstrap):
2: for 1 to B do
3: Generate totally E =1=1 n. resamples randomly with replacement from (H.,).
4: Compute FG in Eq ( with the bootstrap samples;
5: Identify the best subgroups Ti with the bootstrap samples as in Eq 33
6: end for
Step B (Output):
7: Choose Tn with the highest frequency of occurrence and merge subgroups that belong to Fe..
K
Additional details on synthetic real data study
K.1 Additional simulation results of our proposed algorithms
In this section, we mainly focus on the comparison of four variations of our proposed design strategy adopted with single or double bootstrap for selecting hyperparameters and naïve or separate bootstrap for data merging. We adopt the same setting mentioned in Section
21

---

Correct selection probability
0.90
11
15
Stage
Mathods
soparate beotstrap
Single and naive bootstrap
Singlo and separate bootstrap
Figure 2: Comparison of the correct selection probability among four variations of our proposed design strategies.
Table 3: Comparison among four variations of proposed design strategy based estimated best tie treatment effect (Est), 95% confidence interval (95% CI), V--scaled bias (v (vNBias), and standard deviation (SD).
Method
Est (95% CI)
VNBias
SD
Double and naive bootstrap
10.16 (9.14,11.18)
41.69
40.40
Double and separate bootstrap
10.25 (9.22,11.28)
34.44
40.71
Single and naive bootstrap
10.26 (9.22,11.29)
34.20
40.85
Single and separate bootstrap
10.31 (9.28, 11.35)
29.98
40.94
First, from the comparison in Figure 2 as the adaptive design goes on, four variations of our proposed design strategy do not present distinetly different performance, and they are all efficient with correct selection probability gradually rising over 0.9. Our design conducted with a single bootstrap for choosing hyperparameters and naïve bootstrap for identifying and merging tie perform little better than other variations after 15 stages.
Second, from Table 3 obtain that proposed strategy conducted with single bootstrap for selecting hyperparameters and separate bootstrap for identifying and merging data has the smallest vN -calld bias. Overall, these four variations demonstrate similar efficiency in identifying and merging the best subgroups with almost the same estimated treatment effeet and standard deviation.
Narmal and gusnal ilfommaton
Narmat bad gruttall Hiformatioo
00
Stage
Mothods
sastiiazae Easl eigantin Sirge ans reporme becerno Upper confosncc hound bgodthn
Mothods
Deeile bechiius apenae bostatee Sig and eaive hoettiig Single sepsors bocnmip.
(a)
(b)
Figure 3: Comparison of the normalized mutual information among three conventional methods and four variations of proposed design strategies.
We add normalized mutual information an additional metric to evaluate the similarity of estimated tie set and true tie set of pest-performing subgroups. With the results presented Figure (b), the normalized mutual information for proposed design strategy and the three conventional
3 (a) and
22

---

methods exhibits similar trend in terms of correct selection probability. This observation further confirms that our proposed design strategy outperforms the conventional methods.
K.2 Extension: AIPW estimator
When contextual information (or additional covariate information) is considered in our design, the subgroup treatment effects and associated variances in Eq E be replaced by
Die (Yis --i + (Zis)
1--1 = N 1,6))
(X))
(t-16))
1
1-1
1 (Yis - -mo(Zii))) + (2i)
1 (XL,
N1-1
(1-
s=1 ==1
Σ Di=1
(XuES Dio(Vis -pi (21())
(e.. No.1 NNN-
V.
V=-1,))(e;)
(1)
Nr-1
E Ei= ((X)NE)))) (1 - (Ys) -To(Zis))) (11 (11 (1-ez)- eg) NN-101)-
+
N (1)
N1-1
1 (S)))) (11 - - s=1
+ N 1o-1
where (Z,.) E|Y,,| Dis d, Zis &,y) de {0.1). We still generate synthetic data that mimic the original PBC dataset with the potential outcome from Y;(d) E Sj Paj + +0.5% Zi ~ ((,,1), Edj ~ ~N (0, j = I..... 5 to investigate the performance our proposed stratgy with AIPW estimator.
Through simulation studies in Figure we found that our method (whether using IPW or AIPW) has the highest accuracy in identifying the best subgroups. The performance of the causal tree model is comparable to that of the contextual bandit algorithms where treatment effects are estimated with the AIPW estimator, thereby including contextual information. Regarding the causal tree model, we would like to note that the comparison may not be entirely fair for the causal tree model. This because the causal tree method is method used identify subgroups after the data has already been collected, whereas our method focuses on designing data collection mechanisms to accurately identify the best subgroups. Therefore, it can be expected that our approach will have higher accuracy in identifying subgroups.
Cariedt salect on protua
Stage
Stage
Crasdtive
Causi
Compleo rodomno PN)
Nethods 0 Fridon (podly sgurth (tiee
Bingl on sepiote bemsove sAPM Si aeeaanaaaee antruguuaaa zoctimp wy Upper parilenca de algratee JAIM!
Methods
Taoker evetly Jeoite (1PM))
Single sepance beatary (MPP:
Serge auparaks bectmne JP
Upper fimnce Sound sagertien JAPV)
(a)
(b)
Figure 4: Comparison of the correct selection probability and normalized mutual information among our proposed design with IPW estimator, three conventional algorithms and our proposed design with AIPW estimator and causal tree.
K.3 Additional synthetic real data study
In this section, we provide additional synthetic real study in another domain to showcase the performance of our proposed design. We consider a job dataset from the National Supported Work (NSW) program. This program, initiated in the 1970s, aims to provide disadvantaged workers
23

---

with work experience. We utilize the dataset from a field experiment referenced in 7 (n = 455), which includes 185 workers the treatment group and 260 workers in the control group. The dataset features treatment indicator variable, outcome variable representing participant earnings post-treatment 1978, and eight baseline variables. These baseline variables include age, years of education, an indicator for high school graduation, indicators for Black and Hispanic ethnicity, marital status, and pre-treatment earnings for the years 1974 and 1975.
We aim to investigate whether the job training program is indeed beneficial for certain groups of workers in for subgroups defined by: (1) education years 2 11 and age 2 26, (2) education years < 11 and age 2 26, (3) education years 2 11 and age < 26, (4) education years < 11 and age < 26. is a suitable dataset because by conducting two- aro-sample t-tests between every two subgroups the pairwise similarity of the distribution of subgroups, Subgroups (1), (2) and (3) are regarded as tie sets of the best subgroups under the significance level of = 0.05. We provide the results in Figure which can validate the performance of our proposed strategy.
Connect sellciion grobunsiity 8
Connect Cornect salustoo protaaattty
Stage
Stage
dingle
Single
Ner maliaed masual ai intermatinn
Mor mall maaddadd mnstua intormaation
0.0
Stage
Stage
Comples design Spdion vedy agaattnn
MaFrode Doubde nake boutatap Doubls andisp bookineo
Figure 5: Comparison of the correct selection probability and normalized mutual information among three conventional methods and our proposed design strategy based on job dataset.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our work in Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
25

---

Justification:
Please see Section S
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please Section and Supplementary Materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
26

---

Answer: [Yes]
Justification:
Please see Section 6 and Supplementary Materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pipps.ccppullc//uuuii wwwiiicgguie///eeeeeeeeic CoodeSubeiiieeee ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please see Section 6 and Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Please see Section 6
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
27

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question:
Does the research conducted in the paper conform, in every respect, with the httprr/ps...pppll///hhhcceeeeaid
NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research in the paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discussed the potential positive social impacts in Section | social impacts are not applicable.
The negative
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have properly credited the original data source.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The manuscript does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
30


Reviews:
[{"review_id": "xKtyMjqPBv", "reviewer": "Reviewer_pM3n", "summary": "The paper introduces a dynamic subgroup identification strategy within the framework of covariate-adjusted response-adaptive randomization, addressing the need for more nuanced subgroup analysis in clinical trials. This strategy aims to optimize treatment allocation dynamically and identify subgroups that demonstrate significant treatment effects, which is crucial in practice.", "questions": "See previous comments.", "limitations": "See previous comments.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tThe method is highly relevant to modern clinical trial designs, where there is a pressing need to identify patient subgroups with differential treatment responses efficiently.\n2.\tThe paper is strong in theoretical development, providing rigorous proofs and formulations that demonstrate the statistical validity and efficiency of the estimator for the best subgroup treatment effect.\n3.\tThe approach looks novel, from my perspective.", "weaknesses": "1.\tI understand the paper is a very technical paper, and the authors have spent a lot of efforts in making the paper easy to follow. My minor comment is that maybe providing more intuition on the reasons why the complicated design is needed will be helpful. For example, why we need resampling and bootstrap? What kind of technical challenge bootstrap can help to overcome?\n2.\tThe objective function in line 128 seems a little mismatched with the objective of best subgroup identification. For example, if there are two arms in $\\mathcal{T}_1$  with variance zero and very large respectively, it seems that optimizing the objective in line 128 will lead to the solution that we forget about the super large one and devote most efforts to the one with variance 0.  Thus, I am thinking about whether there is some analysis on the probability that we successfully identify $\\mathcal{T}_1$? More specifically, can we always guarantee we identify everything in $\\mathcal{T}_1$ and we never include something out of $\\mathcal{T}_1$?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- Thank you for your encouragement. Below, we provide our understanding of why adaptive designs can be preferable in identifying the best subgroups compared to other alternatives, why resampling and bootstrapping are critical in our procedure, and what technical challenges bootstrapping addresses.\n\n  - To illustrate why adaptive designs can be preferable in identifying the best subgroups, we first review the literature on identifying subgroups and put our method into perspective. There are two lines of literature: The first line conducts a post-hoc analysis of already collected data from previous studies, while the second line focuses on adaptively collecting new data via randomized experiments to ascertain the best subgroup. Therefore, designing an adaptive data collection mechanism is the focus of the second line of literature, and our method aligns with the second line. We follow the second line for subgroup identification because While the first line of literature adopts post hoc subgroup analyses that do not require new data collection, they rely on untestable causal assumptions, limiting the credibility of causal conclusions (e.g., the unconfoundedness assumption is necessary for causal inference in observational studies, but unmeasured confounders can compromise these conclusions). Conversely, in randomized experiments, valid causal conclusions do not depend on such assumptions, but analysis of subgroup treatment effects may still face biases like the winner's curse, mainly if heterogeneous effects are selected from the data in an ad hoc manner (Guo and He, 2021;Andrews et al., 2019). Our approach aligns with the second line of literature and involves collecting data through randomized experiments to identify subgroups that benefit most from the treatment accurately. This approach is preferable because it avoids the limitations of analyzing existing data, which is the case with first-line approaches.  It ensures that treatments are randomly assigned, thus eliminating the influence of unmeasured confounders and allowing for robust, valid causal inferences without the need for untestable assumptions—distinctly superior to using observational data. Moreover, our adaptive design method permits iterative adjustments based on new information, enhancing trial flexibility and efficiency. We also hope to gently note that we are not deliberately constructing a complex adaptive experimental design strategy. Given randomized experiments can be time-consuming, costly, and potentially result in adverse patient outcomes if unsuccessful, our approach aims to efficiently allocate experimental resources within a limited budget to identify the most beneficial subgroup.\n\n  - Bootstrap and resampling-based methods play a crucial role when dealing with subgroups that have tied effect sizes in the population. To shed light on this issue, we consider a more straightforward scenario without adaptive data collection. In this simplified scenario, our objective is to identify the subgroup clusters (referred to as \"tie sets\" in our paper) and rank these clusters according to their average effect sizes with statistical confidence. Suppose we have in total $p$ subgroups forming $m$ clusters, and their population effect sizes and the averaged cluster effect sizes are $\\underbrace{\\beta\\_1= \\ldots =\\beta\\_k}\\_{\\text{Cluster 1}}, \\beta\\_{k+1}, \\ldots, \\beta\\_{l-1},\\underbrace{\\beta\\_{l} = \\ldots = \\beta\\_{p}}\\_{\\text{Cluster m}}$, $\\alpha\\_{j} = \\sum\\_{l\\in \\text{Cluster j}} w\\_l \\beta\\_l,  \\sum\\_{l\\in \\text{Cluster j}} w\\_l = 1,  j = 1, \\ldots, m$. Then, the bootstrap procedure proposed in our method helps us to achieve the following two goals simultaneously: Provide valid statistical inference (confidence intervals and consistent point estimates) on the ``ordered\" averaged cluster effect sizes, that are $\\alpha\\_{(1)},\\alpha\\_{(2)}, \\ldots, \\alpha\\_{(m)}$; Identify the clusters with a high probability. While numerous methods exist for subject clustering, providing confidence intervals for their ordered estimated mean effect sizes is a challenging task due to the winner's curse bias. This issue is well-known in the existing literature, where constructing confidence intervals for order statistics is particularly difficult. Therefore, our bootstrap procedure is important in identifying the tie set while delivering valid statistical inference on the identified best set of subgroups.  \n\n- Thank you for your question. When there are two arms with the same treatment effects but different variances, our algorithm can effectively manage the allocation efforts, avoiding the scenario when most efforts are allocated to the arm with zero variance. We shall illustrate this in a two-stage adaptive experiment. At the start of Stage 1, we adopt the same treatment allocation for all subgroups. At the end of Stage 1, if there are two arms (or subgroups) in $\\mathcal{T}\\_1$  with similar treatment effects but one with zero variance and the other with large variance, our algorithm will be able to identify these two competing subgroups and then merge these two subgroups into the best \"set\"  $\\hat{\\mathcal{T}}\\_1$. In Stage 2, since both of these two subgroups belong to $\\hat{\\mathcal{T}}\\_1$, they will be assigned the same treatment allocation, denoted as $\\hat{e}\\_1^*$. Additionally, Theorem 1 in our manuscript provides an analysis of the probability of successfully identifying $\\mathcal{T}\\_1$. We show that as the sample size tends to infinity, we can always correctly identify the best set of subgroups. Therefore, we can guarantee the identification of everything in $\\mathcal{T}\\_1$ and ensure that nothing out of $\\mathcal{T}\\_1$ is included.\n\nReference: \n- Andrews, I., Kitagawa, T., and McCloskey, A. (2019). Inference on winners. Technical report, National Bureau of Economic Research\n- Guo, X. and He, X. (2021). Inference on selected subgroups in clinical trials. Journal of the American Statistical Association, 116(535):1498–1506.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kykhkLJOsp", "reviewer": "Reviewer_Rfeb", "summary": "This paper studies an interesting problem in clinical trials to identify patient subgroups with the most beneficial responses to the treatment, which is essential for clinicians to create personalized treatment plans for their patients. However, most existing strategies for the design of clinical trials rely on domain knowledge or past experience of expert clinicians and stick to several pre-defined patient subgroups throughout the trial, which discards the valuable information collected from different trial stages. Some adaptive experimental strategies are developed to identify the best performing patient subgroup based on the trial outcomes; but their negligence of other subgroups where the treatment could be equally effective usually causes inefficient utilization of the experimental efforts. To tackle these challenges, the authors propose a dynamic subgroup identification approach to enable the construction of more effective experimental strategies for practical clinical trials. The authors claim three major contributions in this study: 1) their approach allows the dynamic identification for best patient subgroups based on experimental data collected during the trial process; 2) the authors develop new algorithms to effectively merge patient subgroups with similar (highest) responses to the treatment and provide theoretical results to support their analyses; and 3) the proposed method is validated using a synthetic dataset constructed from a clinical trial on cirrhosis.", "questions": "In summary, I have the following concerns about this paper.\n1. The related works are not sufficiently discussed. The difference between this paper and relevant studies should be clearly illustrated.\n2. It is unclear how the proposed method is different from conventional methods considering the RAR settings. There is no discussion on the contribution or benefit of including patient covariates ($X$) in the design of clinical trials. \n3. Many symbols are introduced in the analysis of this paper without proper explanation.\n4. The derivation of some key results is not sufficiently explained. For instance, the optimization objective below LN 129 and the optimal treatment allocation in Eq. 1.\n5. The correctness of some proofs seems to be questionable. Specifically, the proof for Theorem 1. Please see the weakness section above for details.\n6. The proposed method needs to be tested on more datasets, and the authors are encouraged to include more performance metrics in the benchmark.\n7. There should be more baselines, e.g., causal tree and contextual bandit, in the benchmark to highlight the advantage of the proposed method.\n8. How does the number of stages $T$ in a trial affect the convergence of the proposed method? What if there is only one stage? What if $T$ is small (Below 4)?", "limitations": "The limitations of this study are briefly discussed at the end of this paper. The authors identify the potential mismatch between their assumption on immediate observation of treatment responses and the delays commonly observed in real-world clinical trials. However, the potential negative societal impact of the clinical trial design approach in this paper is not sufficiently discussed. In addition, the authors are encouraged to discuss whether their approach can be generalized to deal with scenarios with multiple treatment options.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "## Originality\nThe method presented in this paper looks to be novel. The authors have provided substantial theoretical analysis to evident the originality and validity of their approach.\n\n## Quality\nMost parts of this paper are well-written and properly organized. The experimental results are reported with relevant statistics and are clearly evaluated and discussed with texts and visualizations.\n\n## Clarity\nThe general clarity of this paper is fair. Experimental results in this paper are sufficiently discussed.\n\n## Significance\nThe problem of patient subgroup identification is essential in clinical trials for the selection of target patient cohorts. For better utilization of the resources in a trial, dynamic identification of best performing patient subgroups and adaptive treatment assignments are imperative. The approach proposed in this paper enables effective patient subgroup identification using patient characteristics and treatment responses collected during different trial stages and allows adaptive optimization of the treatment assignment strategy.", "weaknesses": "## Related Works\n\nThere are two short paragraphs in the Introduction discussing the literature related to this study. However, to distinguish the experimental design approach in this paper from other research and highlight the contributions of this study, a more comprehensive comparison with related works is needed. It seems that there are many other studies, with similar focuses on subgroup identification in clinical trials, not sufficiently discussed in this paper. For instance:\n\n- Adaptive identification and assessment of patient subgroups [1]\n- Identification of patient subgroups with similar clinical characteristics (covariates) [2]\n- Clustering of patient subgroups with different levels of benefits in clinical trials [3]\n\nThe authors are encouraged to provide discussions and comparisons with related works on patient subgroup identification to better demonstrate the novelty and advantage of this study.\n\nFurther, although the authors emphasize that their method focuses on the setting of covariate-adjusted response-adaptive (CARA) experiments which is an under-explored area, the reviewer cannot find discussions on the contribution or benefit of including patient covariates ($X$) in the design of clinical trials. The covariates-based patient subgroup identification has already been studied in the machine learning literature, e.g., [2,3]. \n\nReferences: \n\n[1] Guo, Wentian, Yuan Ji, and Daniel VT Catenacci. \"A subgroup cluster‐based Bayesian adaptive design for precision medicine.\" Biometrics 73.2 (2017): 367-377.\n\n[2] Lee, Beom S., et al. \"A clustering method to identify who benefits most from the treatment group in clinical trials.\" Health Psychology and Behavioral Medicine: an Open Access Journal 2.1 (2014): 723-734.\n\n[3] Xu, Jie, et al. \"Machine learning enabled subgroup analysis with real-world data to inform clinical trial eligibility criteria design.\" Scientific Reports 13.1 (2023): 613.\n\n\n\n\n\n## Clarity\n\n### The role of patient covariates\n\nAs mentioned earlier, the role of patient covariates is not sufficiently discussed in the proposed experimental design approach. The only place related to the covariates seems to be line 7 of Algorithm 2 where previous trial results are randomly resampled. It is unclear how the experimental design is adjusted based on patient covariates. Since this paper focuses on the CARA setting and considers both treatment responses and patient covariates in subgroup identification, it is important for the authors to elaborate the contribution of patient covariates in the proposed algorithms and how is this study different from conventional research on response-adaptive randomization (RAR) settings.\n\n### Notations\n\nThere are many symbols introduced in the analysis of this paper without any explanation. \nFor instance, the variable $\\hat{p}\\_{tl}$ in Eq. 1 has never been explained. \nThe symbol $\\mathcal{B}_{t,b}$ in Eq. 6 comes from nowhere. \nIn the meantime, there are so many similar symbols used in the discussion, and it is very difficult for the reviewer to tell their difference. The authors are encouraged to ensure the consistency in their notation. If possible, a notation table could help a lot to improve the clarity of this paper.\n\n### Insufficient explanations\n\nSome results or derivations in this paper are introduced without proper explanation. For instance, although the authors provide citations to the large deviation theory in LN 129, the equivalence between the correct identification probability and the optimization objective remains obscure. Similarly, it is unclear why the optimal treatment allocation can be derived from Eq. 1. Additionally, the hard-coded exponent 0.05 for $\\Delta$ in LN 162 appears without any explanation.\n\n## Correctness\n\nThe correctness of some proofs seems to be questionable. For instance, in the proof of Theorem 1, it is unclear why $\\tau_{t,l} = \\tau_{l}$. The inequality in LN 450 cannot be directly established according to the analysis in LN 451 – 453. Similarly, the final inequality in LN 457 is non-trivial and the authors should provide analysis to prove its correctness.\n\n## Evaluation\n\n### Dataset\n\nThe proposed experimental design approach is only evaluated on a synthetic dataset. To validate the general applicability and performance of this method, benchmark results on more datasets are necessary.\n\n### Baselines\n\nThe authors have discussed many relevant studies in the Introduction. However, only three baselines are considered in the experiment. For a more comprehensive comparison, the authors are encouraged to include more baselines from related works to demonstrate the advantage of their method. Particularly, the reviewer is interested in the performance of contextual bandit algorithms and causal tree models.\n\n### Metrics\n\nNote that the experimental results are obtained on a synthetic dataset where the ground truth subgroup labels are available. The authors are encouraged to include additional metrics on clustering accuracy, e.g., purity score, normalized mutual information, etc., in the benchmark.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Thank you for pointing us to the references. While both Guo et al. (2017) and ours are in an adaptive experiment setting, there are two major differences. (i)Their design is Bayesian, relying on prior specification, while ours is frequentist and model-free. (ii)They do not discuss theoretical properties for the identified subgroups, while we provide theoretical results justifying the reliability of our approach. Next, there are two lines of literature for subgroup identification: (1)post-hoc analyses using previously collected data, and (2)adaptive data collection through randomized experiments. Lee (2014) and Xu (2023) align with the first one, whereas our method focuses on the second. Lee et al. (2014) use clustering techniques based on data from prior randomized controlled trials, and Xu et al. (2023) identify subgroups from existing observational data using machine learning, both differing from our adaptive experiment setting.\n\n2. In classical CARA design (so does our design), patient covariates are used to define subgroups. Concretely, denote the covariate as $X_{it}$ and assume the covariate space $\\mathcal{X}$ is partitioned into $m$ regions: $\\mathcal{S}_{j=1}^m$ (lines 98-100).  CARA designs differ from RAR as RAR does not consider covariates at all, optimizing treatment allocation solely based on past treatment assignments and outcomes. This oversight can lead to less effective treatment strategies by ignoring valuable patient-specific information that could significantly influence treatment responses. We hope to note that in response to your Q7, we refined our approach using the augmented Inverse Probability Weighting estimator, allowing us to incorporate other covariates that are not used to define subgroups, enabling us to adjust treatment allocation accordingly. \n\n3.We apologize for missing the definition of $\\hat{p}_{tl}$. It is defined as $\\hat{p}\\_{tl}=\\frac{\\sum\\_{s=1}^{t}\\sum\\_{i=1}^{n\\_s}\\mathbb{1}\\_{(X\\_{is}\\in\\mathcal{S}\\_{(l)})}}{\\sum\\_{s=1}^{t}n_s}$. ${B}\\_{t,b}$ is defined in Eq.6 in preparation for the calculation of the loss function in Eq.7. Additionally, $\\Delta=\\min(1,R\\cdot(\\frac{\\sqrt{\\sum\\_{j=1}^{m}n\\_{tj}\\hat{\\mathbb{V}}\\_{tj}/m}}{\\sqrt{\\sum\\_{j=1}^{m}\\hat{\\mathbb{V}}\\_{tj}/n}})^{2*\\gamma})\\approx \\min(1,R\\cdot n^{\\gamma})$, where $\\gamma$ is a small tuning parameter to ensure $\\Delta<1$. We choose $\\gamma=0.05$ and show that our procedure is not sensitive to the tuning parameter(Table 1 in the attached pdf).  \n\n4. Due to the space limit in our first submission, we neglected technical details to justify Eq.1. In fact, the rate function in the optimization problem's objective is a monotone transformation of the correct identification probability in an asymptotic sense. Below is a brief derivation due to the space limit: $\\lim\\_{N\\rightarrow\\infty}\\frac{1}{N}\\log(1-\\mathbb{P}( \\hat{\\tau}\\_{\\mathcal{T}\\_1}\\geq\\max\\_{j \\notin\\mathcal{T}\\_1}\\hat{\\tau}\\_j))=-\\min\\_{j\\notin \\mathcal{T}\\_1}G(\\mathcal{S}\\_1,\\mathcal{S}\\_j;e\\_1,e\\_j)$. We are happy to provide more details if you raise additional concerns. \n\n5. After inspecting our proof, we believe that our proof in the three places you pointed to us is correct. $\\tau\\_{t,l}=\\tau\\_l$ naturally holds by Assumption 1. This is because potential outcomes are independently identically distributed for $i=1,\\ldots,n_t$, $t=1,\\ldots,T$, implying $\\tau\\_{t,l}=\\tau\\_l$ in the proof of Theorem 1. This i.i.d assumption is commonly assumed in adaptive design literature.  With the analysis in LN 451-453, we have $\\lim\\_{N\\rightarrow\\infty}\\mathbb{P}(N^{\\delta}|\\tau\\_{j}-\\tau\\_{\\check{1}}|<C)=0$ and thus $\\lim\\_{N\\rightarrow\\infty}\\mathbb{P}( N^{\\delta}|\\tau\\_{j}-\\tau\\_{\\check{1}}|+C< 2C)=0$. By Lemma 2 in the Appendix, we obtain $\\lim\\_{N\\rightarrow\\infty}\\mathbb{P}(N^{\\delta}|\\hat{\\tau}\\_{tj}^*-\\tau\\_{j}| +N^{\\delta}|\\hat{\\tau}\\_{t,\\check{1}}^*-\\tau\\_{\\check{1}}|\\geq 2C)=0$. Then we reach the conclusion in Eq.13. Similarly, with the analysis in LN 458-461, we have $\\lim\\_{N\\rightarrow\\infty}\\mathbb{P}( N^{\\frac{1}{2}}|\\tau\\_{j}-\\tau\\_{\\check{1}}|<C)=1$. We also derive \n\n$\\lim\\_{N\\rightarrow\\infty}\\mathbb{P}(N^{\\delta}|\\hat{\\tau}\\_{tj}^*-\\tau\\_{j}|<C)=1$ and \n$\\lim\\_{N\\rightarrow\\infty}\\mathbb{P}(N^{\\delta}|\\hat{\\tau }\\_{t,\\check{1}}^*-\\tau\\_{\\check{1}}|<C)=1$. With $\\delta<\\frac{1}{2}$, we establish the result in LN 455.\n\n6. We have added an additional case study using the National Supported Work program data(Figure 1 in the attached pdf).\n\n7. For the comparison with various contextual MAB algorithms, we now extend our proposed design to an augmented inverse propensity score weighting (AIPW) estimator incorporating contextual information (Figure 2 in the attached PDF). For the causal tree, the comparison may not be entirely fair since the causal tree identifies subgroups after data collection, whereas our method designs the data collection mechanism to identify the best subgroups accurately. Thus, our approach is expected to have higher accuracy in identifying subgroups. We have provided a comparison (Figure 2 in the attached PDF). The performance of the causal tree model is similar to contextual bandit algorithms, and our proposed algorithm has the highest probability of identifying subgroups. \n\n8. The purity score is similar to correct selection probability, as both measure accuracy: purity score assesses how well clusters contain a single class, while correct selection probability evaluates identifying the best subgroups. Both range from 0 to 1, with higher values indicating better performance. Figure 1(c) (d) in the attached PDF compares the normalized mutual information for our proposed design and three competing methods.\n\nPer your suggestions, we will update our manuscript as follows:\n- Add a more thorough literature review on posthoc subgroup analysis\n- Add a notation table\n- Add additional simulation results, including contextual bandit, AIPW estimator, and normalized mutual information as an additional metric", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H3yajwqy3f", "reviewer": "Reviewer_Sgje", "summary": "This paper introduces a new dynamic treatment assignment for clinical trial to target treatment to the group most likely to benefit from it.", "questions": "Following the previous point, I would recommend a more in-depth review of the literature to convince the reader that this problem does not have alternative solution in the literature.", "limitations": "The paper discusses the critical limitation of the proposed approach's assumption of instantaneous access to the outcome following treatment. It would be interesting to discuss this assumption in the context of the existing literature. Is it a common assumption? If not, it would be important to justify further.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper studies a critical problem of clinical trial design. It is clear and provides both theoretical justification and synthetic validation, demonstrating the utility of the proposed method.", "weaknesses": "While the paper tackles a critical problem, the problem is studied in depth in biostatistics. While I am unfamiliar with the literature on this topic, I am surprised there is none that could be considered for comparison. A more in-depth analysis of the literature on this topic should be presented in the Appendix to justify the choice of compared methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable questions and suggestions!\n\nA main limitation of our proposed approach is the assumption that outcomes are observed instantaneously following treatment. This assumption, prevalent in adaptive experiments such as Hu et al. (2015) and Zhu and Zhu (2023), simplifies the modeling process and allows for quick adjustments based on the latest data. While this assumption is common, it may not always reflect real-world scenarios where response delays. To provide a more comprehensive understanding, we review the literature addressing delayed responses.\n\n- The importance of incorporating delayed responses in adaptive experiments is well-recognized in the literature. Rosenberger et al. (2012) discuss the effects of delayed responses on response-adaptive randomization. Early work by Wei and Durham (1978) introduces the randomized play-the-winner rule, which updates the contents of an urn only upon receiving patient responses, thus naturally accommodating delayed responses. This approach offers inherent flexibility in managing delays and represents an improvement over more rigid methods. Bai et al. (2002) and Hu and Zhang (2004) establish asymptotic normality results under urn models with delayed responses, explicitly concerning the asymptotic normality of the fraction of patients assigned to each treatment arm. Their findings show that limiting distributions remain unaffected by delayed responses under reasonable conditions, providing a solid theoretical basis for handling such delays. Zhang et al. (2007) extend this work by introducing a generalized drop-the-loser urn model, demonstrating that asymptotic properties are preserved despite response delays. This generalized model offers broader applicability and enhanced flexibility for managing various delay mechanisms.\n- Regarding the doubly-adaptive biased coin design (DBCD), Zhang and Rosenberger (2006) have shown through simulations that moderate delays in responses have minimal impact on the power and skewness of the DBCD. Their results suggest that the design remains robust even in the presence of delays, although the effects of more severe delays are less clear. Hu et al. (2008) study the asymptotic properties of the DBCD with delayed responses, showing that these properties remain unaffected by such delays. They also provide strong consistency results for the constructed variance estimator that incorporates delayed\nresponses, enhancing the design's reliability in practical scenarios. \n- In addition to response-adaptive randomization designs, some group-sequential designs address the challenges brought by delayed responses. Hampson (2013) proposes to incorporate short-term endpoints to enhance the efficiency of group-sequential tests when long-term responses are delayed. This method effectively balances immediate data needs with constraints imposed by delayed outcomes. Schuurhuis et al. (2024) expand this framework by suggesting the integration of pipeline data into group-sequential designs, enabling the trial to restart after a temporary halt. This integration provides added flexibility and robustness in managing trials with delayed responses. \n\n- Overall, handling delayed responses requires challenging adjustments to our design. First, our objective function, based on the semiparametric efficiency bound of the subgroup treatment effect, must be revised. The difficulty stems from the function's current assumption of immediate responses, and modifying it to account for delays significantly complicates the task of ensuring accuracy.\nSecond, our estimators need updating. Delayed responses impact both the treatment effect and variance estimators, necessitating an additional step to address these delays. While this step is crucial for maintaining precision and reliability, it also introduces complexities that must be managed to ensure a robust estimation process.\n\nPer your suggestions, we plan to make the following updates to our manuscript:\n- Present a more in-depth analysis of the literature in our revised manuscript.\n- Discuss the critical limitation in the context of the existing literature in our revised manuscript.\n\nReference:\n- Hu, J., Zhu, H., and Hu, F. (2015). A unified family of covariate-adjusted response-adaptive designs based on efficiency and ethics. Journal of the American Statistical Association, 110(509):357–367.\n- Zhu, H. and Zhu, H. (2023). Covariate-adjusted response-adaptive designs based on semiparametric approaches. Biometrics.\n- Rosenberger, W. F., Sverdlov, O., and Hu, F. (2012). Adaptive randomization for clinical trials. Journal of Biopharmaceutical Statistics, 22(4):719–736.\n- Wei, L. and Durham, S. (1978). The randomized play-the-winner rule in medical trials. Journal of the American Statistical Association, 73(364):840–843.\n- Bai, Z., Hu, F., and Rosenberger, W. F. (2002). Asymptotic properties of adaptive designs for clinical trials with delayed response. The Annals of Statistics, 30(1):122–139.\n- Hu, F. and Zhang, L.-X. (2004). Asymptotic normality of urn models for clinical trials with delayed response. Bernoulli, 10(3):447–463.\n- Zhang, L.-X., Chan, W. S., Cheung, S. H., and Hu, F. (2007). A generalized drop-the-loser urn for clinical trials with delayed responses. Statistica Sinica, 17(1):387–409.\n- Zhang, L. and Rosenberger, W. F. (2006). Response-adaptive randomization for clinical trials with continuous outcomes. Biometrics, 62(2):562–569.\n- Hu, F., Zhang, L.-X., Cheung, S. H., and Chan, W. S. (2008). Doubly adaptive biased coin designs with delayed responses. Canadian Journal of Statistics, 36(4):541–559.\n- Hampson, L. V. and Jennison, C. (2013). Group sequential tests for delayed responses (with discussion). Journal of the Royal Statistical Society Series B: Statistical Methodology, 75(1):3–54\n- Schuurhuis, S., Konietschke, F., and Kunz, C. U. (2024). A two-stage group-sequential design for delayed treatment responses with the possibility of trial restart. Statistics in Medicine.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yGU2c4dNiY", "reviewer": "Reviewer_JR9y", "summary": "The paper proposes a dynamic subgroup identification strategy within covariate-adjusted response-adaptive randomization (CARA) for clinical trials. This adaptive method dynamically identifies and adjusts treatment allocation to the best-performing subgroups based on ongoing trial data, and thus extend the traditional fixed-design trials to an online setup. Moreover, the paper also makes theoretical contributions by showing validity, asymptotic normality and efficiency of the estimator for the best subgroup treatment effect.", "questions": "1. Page 3, bottom equation: In the equation at the bottom of this page, should $=$ be $\\coloneqq$, i.e., a definition?\n\n2. Page 3, Line 127-128: The goal of the design is to maximize the correct identification probability, based on which the paper proposes CARA and develops the corresponding theory. However, why maximizing the correct identification probability is the ultimate goal? In precision medicine, a more plausible goal is to maximize the overall welfare for the patients, i.e., finding the design that best improves the medical outcome for all patients. How do you compare your design objective to this welfare maximization objective? Can you modify your design for the second objective?\n\n3. Page 4, Eq (1): This equation is confusing. What is the objective function for $\\mathbf{e}$? I only saw the constraint set inside the parenthesis.", "limitations": "As discussed in the paper, one main limitation is that the method cannot handle delayed outcomes, which can be restrictive in the real-world clinical trials. In my opinion, the authors have adequately addressed the limitations.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Originality\nThe paper makes a significant original contribution by introducing a novel dynamic subgroup identification strategy within covariate-adjusted response-adaptive randomization (CARA) for clinical trials. This approach is innovative in its dynamic adjustment of treatment allocation based on real-time data, addressing inefficiencies associated with traditional fixed-design trials. This work also derives new theoretical results justifying the use of their proposed design. \n\nQuality\nThe research is of high quality, demonstrated through both theoretical and empirical validations. The authors provide rigorous theoretical results, including validity, asymptotic normality and semiparametric efficiency. Additionally, the empirical validation using a synthetic clinical trial on cirrhosis data is well-executed and convincingly demonstrates the effectiveness of the proposed design. \n\nClarity\nThe paper is clearly written and well-structured. Complex definitions and algorithms are clearly demonstrated. \n\nSignificance\nI consider this a significant work in the field of sequential experimental design and subgroup identification. It explores how to best identify and estimate subgroup effects in a dynamic regime, which is underexplored in the literature. What's more, it has broader impact in precision medicine and clinical trials.", "weaknesses": "Overall the paper is technically solid, though I notice some issues that can be addressed in the revision. First, the clarity of this paper can be improved as I identified some sentences/equations that are confusing. Please check the question section for details. Second and more importantly, the way that the paper formulates the problem needs some further justification, which is also related to Q2 in the question section.  The paper proposes a design to maximize the probability of correctly identifying the best subgroup, and estimate the best treatment effect; the theory and experiments all focus on this best subgroup identification. However, in personalized medicine which is the motivation for this work, I suppose practitioners care more about whether the treatment is beneficial to a certain group of patients or not, rather than the best treatment effect. In other words, even one can efficiently estimate the best treatment effect using CARA, it does not guarantee the best causal decision rule in personalized medicine. See https://pubsonline.informs.org/doi/10.1287/ijds.2021.0006 for a relevant discussion. Therefore, I suggest the authors discuss this gap between your work and practical considerations; this will help further justify the necessity of best subgroup identification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "- Thank you for your insightful questions regarding the design objective and for kindly pointing us to the reference.\n  - We completely understand your concern about the practicality of identifying the best set of subgroups rather than all the benefitted ones. In many clinical settings, treating all the benefitted patients is indeed a natural and ethical goal from the practitioner's standpoint when the number of treatments is unlimited. Different from this setting, our design is tailored to situations where resources are limited, making it preferable to identify only the best set of subgroups. For example, during the onset of COVID-19, when the total number of vaccines was limited, only a subset of the population could be treated. The design objective of identifying the best set of subgroups thus becomes more relevant.  \n  -  Furthermore, we would like to point out a subtle difference between our design objective and the design objective of maximizing overall welfare.  Our work considers situations where assigning a treatment is costly, and there is an overall constraint on how many treatments can be deployed.  In Fernandez-Loria and Provost (2022), for example, the goal is to ensure all the benefitted subjects are precisely assigned to the treatment arm while the cost of treatments is negligible. They aim to learn the optimal policy that maps a given covariate to the best arm for that patient profile. In our work, we consider scenarios where implementing the treatment can be costly per sample of a patient (experimentation unit). For example, in clinical settings, randomized experiments are often expensive due to the substantial costs of treatment medications. Consequently, the resource constraints in our problem are equivalent to the number of treatments that can be administered. \n  - We appreciate the reference you provided, which we found very helpful.  We agree that efficiently estimating the best causal effect can sometimes misalign with the best causal decision rule, especially when the best decision rule is to always treat a patient when there is a positive effect. In such cases, accurately estimating the magnitude of the treatment effect becomes less important. Since our primary objective is not to assign all subjects with positive treatment effects to the treatment arm, there could be a misalignment with the best causal decision rule discussed in Fernandez-Loria and Provost (2022). Nevertheless, if we are in a scenario similar to the previously mentioned COVID-19 vaccine example, where the best decision rule is to identify the most benefitted subgroups to prioritize treatment assignments, the accurate causal estimation aligns well with the best causal decision-making. We agree that it is important to be mindful of whether the design objective leads to the best causal decision rule, and the potential misalignment between causal effect estimation and causal decision-making should be carefully discussed. We hope to add this discussion to our revised manuscript. \n  - While our current design is not tailored to welfare maximization, we shall discuss a potential approach to refining our design toward maximizing participant welfare. Consider a two-stage experiment with four candidate subgroups and their population treatment effects follow the order: $\\tau\\_1=\\tau\\_2 > \\tau\\_3 > \\tau\\_4$, where $\\tau\\_1 = \\tau\\_2 > \\tau\\_3 > 0 \\geq  \\tau\\_4$. At the end of Stage 1, our current procedure will be able to correctly identify subgroups 1 and 2 as the best set of subgroups with a high probability. If we were to maximize the patient's welfare, we would add an additional ``early stopping\" step. That is, besides identifying the best set, we will also identify subgroups that exhibit significantly adverse treatment effects. To protect the patient's welfare, we will stop Subgroup 4 from enrollment in the next stage. In Stage 2, we not only avoid impairing the welfare of Subgroup 4 but also maximize the resources (treatments) available to the rest of the benefitted subgroups. \n- Thank you for your careful reading of our manuscript. We shall clarify our optimization problem formulation as follows. Our original optimization problem is formulated as $\\max_{ \\mathbf{e}} \\min\\_{2\\leq j\\leq m^\\ast\\_{t}} \\frac{( \\hat{\\tau}\\_{t-1,(j)} -  \\hat{\\tau}\\_{t-1,(1)})^2}{2\\big(\\hat{\\mathbb{V}}\\_{t-1,(1)}(e\\_{1}) + \\hat{\\mathbb{V}}\\_{t-1,(j)}(e\\_{j})\\big)}, \\text{s.t.}\\ \\sum\\_{l=1}^{m^\\ast\\_{t}} \\hat{p}\\_{tl} e\\_l \\leq c\\_1,    \\  c_2\\leq e\\_l \\leq 1-c\\_2,\\ l=1, \\ldots,m^\\ast\\_{t}$. The set of constraints includes the resource constraint: $\\sum\\_{l=1}^{m^\\ast\\_{t}} \\hat{p}\\_{tl}\\leq c\\_l$, and the feasibility constraint: $c_2\\leq e\\_l \\leq 1-c\\_2,\\ l=1, \\ldots,m^\\ast\\_{t}$.  Because the original objective function takes the minimum of $m^*-1$ rate functions, the original optimization problem is nonlinear. We instead work with its equivalent epigraph representation: $\\max\\_{ \\mathbf{e}} z$, s.t. $ \\min\\_{2\\leq j\\leq m^\\ast\\_{t}}\\frac{( \\hat{\\tau}\\_{t-1,(j)} -  \\hat{\\tau}\\_{t-1,(1)})^2}{2\\big(\\hat{\\mathbb{V}}\\_{t-1,(1)}(e\\_{1}) + \\hat{\\mathbb{V}}\\_{t-1,(j)}(e\\_{j})\\big)} -z \\geq 0,  \\ \\sum\\_{l=1}^{m^\\ast\\_{t}} \\hat{p}\\_{tl} e\\_l \\leq c\\_1,    \\  c_2\\leq e\\_l \\leq 1-c\\_2,\\ l=1, \\ldots,m^\\ast\\_{t}$, which is Eq (1) in our current submission. The original formulation of the objective function was omitted due to space limits. We will clarify the objective function in our revision. \n\nPer your suggestions, we plan to make the following updates to our manuscript:\n- We will add a discussion on the potential gap between our design and the best causal decision rule in precision medicine scenarios in our revised manuscript. \n- We will revise sentences and notations for clarity. \n- We will provide more illustrations of Eq (1). \n\nReference:\n- Fernandez-Loria, C. and Provost, F. (2022). Causal decision making and causal effect estimation are not the same... and why it matters. INFORMS Journal on Data Science, 1(1):4–16.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We want to thank all of our reviewers for their very insightful suggestions and comments. We have made our best efforts to address the questions and comments raised by our reviewers. 

To supplement our simulation studies and to provide additional information in response to some questions, we provide a pdf file which includes the following figures and a table:

- Figure 1: Comparison of the correct selection probability and normalized mutual information among three conventional methods and our proposed design strategy based on an additional dataset.
- Table 1: Simulation results that demonstrate the insensitivity of our method to the choice of tuning parameters. 
- Figure 2: Comparison of the correct selection probability and normalized mutual information among causal tree model, complete randomization with AIPW estimator, contextual bandit algorithms including epsilon greedy algorithm and upper confidence bound 1 algorithm with AIPW estimator, and our proposed design with IPW estimator and AIPW estimator.

We greatly appreciate the reviewers for taking the time and effort to provide their valuable feedback.

Meta-review: This paper presents a design strategy to dynamically identify the best subgroups that demonstrate significant treatment effects in randomized trials. It theoretically demonstrates that the proposed design has a higher probability of identifying the best subgroups compared to conventional designs. The paper also validates the design using both synthetic and clinical trial data.

All reviewers agree that the proposed design is important in practice and novel. The design is supported by sound theoretical foundations. Reviewers have expressed some concerns about the presentation and explanations. Specifically, one reviewer raised concerns regarding the related work, presentation clarity, and the experimental comparison baselines. During the discussion phase, most of the reviewers' questions were addressed, and the remaining concerns do not affect the soundness or novelty of the paper. Therefore, the paper is acceptable.
Decision: Accept (poster)