Title: Dynamic Subgroup Identification in Covariate-adjusted Response-adaptive Randomization Experiments
Venue: NeurIPS 2024
Accepted: yes
Keywords: Covariate-adjusted response-adaptive randomization design

Response-adaptive randomization design

Subgroup identification

Causal inference

Treatment effect heterogeneity
Abstract: Identifying subgroups with differential responses to treatment is pivotal in randomized clinical trials, as tailoring treatments to specific subgroups can advance personalized medicine. Upon trial completion, identifying best-performing subgroups–those with the most beneficial treatment effects–is crucial for optimizing resource allocation or mitigating adverse treatment effects. However, traditional clinical trials are not customized for the goal of identifying best-performing subgroups because they typically pre-define subgroups at the beginning of the trial and adhere to a fixed subgroup treatment allocation rule, leading to inefficient use of experimental efforts. While some adaptive experimental strategies exist for the identification of the single best subgroup, they commonly do not enable the identification of the best set of subgroups.  To address these challenges, we propose a dynamic subgroup identification covariate-adjusted response-adaptive randomization (CARA) design strategy with the following key features: (i) Our approach is an adaptive experimental strategy that allows the dynamic identification of the best subgroups and the revision of treatment allocation towards the goal of correctly identifying the best subgroups based on collected experimental data. (ii) Our design handles ties between subgroups effectively, merging those with similar treatment effects to maximize experimental efficiency. In the theoretical investigations, we demonstrate that our design has a higher probability of correctly identifying the best set of subgroups compared to conventional designs. Additionally, we prove the statistical validity of our estimator for the best subgroup treatment effect, demonstrating its asymptotic normality and semiparametric efficiency. Finally, we validate our design using synthetic data from a clinical trial on cirrhosis.

Dynamic Subgroup Identification in Covariate-adjusted Response-adaptive Randomization Experiments

Abstract

Identifying subgroups with differential responses to treatment is pivotal in randomized clinical trials, as tailoring treatments to specific subgroups can advance personalized medicine. Upon trial completion, identifying best-performing subgroups-those with the most beneficial treatment effects-is crucial for optimizing resource allocation or mitigating adverse treatment effects. However, traditional clinical trials are not customized for the goal of identifying best-performing subgroups because they typically pre-define subgroups at the beginning of the trial and adhere to a fixed subgroup treatment allocation rule, leading to inefficient use of experimental efforts. While some adaptive experimental strategies exist for the identification of the single best subgroup, they commonly do not enable the identification of the best set of subgroups. To address these challenges, we propose a dynamic subgroup identification covariate-adjusted response-adaptive randomization (CARA) design strategy with the following key features: (i) Our approach is an adaptive experimental strategy that allows the dynamic identification of the best subgroups and the revision of treatment allocation towards the goal of correctly identifying the best subgroups based on collected experimental data. (ii) Our design handles ties between subgroups effectively, merging those with similar treatment effects to maximize experimental efficiency. In the theoretical investigations, we demonstrate that our design has a higher probability of correctly identifying the best set of subgroups compared to conventional designs. Additionally, we prove the statistical validity of our estimator for the best subgroup treatment effect, demonstrating its asymptotic normality and semiparametric efficiency. Finally, we validate our design using synthetic data from a clinical trial on cirrhosis.

1 Introduction

Most clinical trial designs adopt 'one-size-fits-all" rules for treatment assignment and evaluation based on models that ignore patient heterogeneity. This approach is disconnected from medical practice in recent years, where physicians use each patient's diagnosis and prognostic variables to

make personalized, precision medicine treatment decisions. As such, identifying patient subgroups with differential responses to a treatment plays a pivotal role in designing randomized clinical trials [28, 18, 2, 39]. Adaptive clinical trials-that allow randomization probabilities to be adaptively optimized during the trial based on sequentially accrued data-have received much attention due to their potential advantages in promoting precision health. Nevertheless, these trials often involve pre-specifying the patient subgroups to be analyzed [3, 38]. This approach not only leads to inefficient use of experimental efforts but may also reduce the statistical power to detect non-pre-specified subgroups that exhibit high effect sizes. Consequently, there is a pressing need for novel and statistical clinical trial designs for dynamic subgroup identification to address these issues.

In this paper, we propose a novel statistical design that dynamically performs subgroup identification in CARA experiments. Our contributions are summarized as follows:

From the design perspective, there are three highlights of our design: (i) Our design facilitates dynamic identification and sequential refinement of the best subgroups within the framework of covariate-adjusted response-adaptive (CARA) experiments (Section 4). This adaptive setting enhances the ability to identify and adjust to the best-performing subgroups over time. (ii) Unlike traditional designs that focus on identifying a single best-performing subgroup, our design is tailored to identify the best set of subgroups with competitive performance (Section 3). This broader objective makes our approach suitable for more general application settings. (iii) Our design demonstrates a higher probability of correctly identifying the best subgroups compared to conventional designs. This efficiency ensures that experimental efforts are utilized more effectively.

From a theoretical perspective, our proposed design strategy accommodates tied treatment effects among candidate subgroups, whereas many existing methods demand that these effects be distinctly separated. Additionally, our algorithm involves resampling in the presence of dependent data structures caused by adaptive treatment allocation, presenting technical challenges in proving statistical validity. We overcome these challenges and show that the best subgroup identified by our design asymptotically converges to the true set of best subgroups (Theorem 1). We also demonstrate that our proposed design strategy converges to the oracle design, which is the optimal design under the setting that the underlying datagenerating distribution is known (Theorem 2). Furthermore, we establish valid statistical inference for the treatment effect of the identified best subgroup and demonstrate that our constructed estimator is semiparametrically efficient (Theorem 3).

In comparison to the existing literature, our design strategy is closely related to CARA designs. Originating from response-adaptive randomization (RAR) designs [5, 33, 46]. Heuristically, CARA incorporates covariate information along with treatment assignment probabilities based on observed outcomes [14, 32, 30, 13]. Building upon RAR designs, CARA designs utilize both outcome and covariate information to optimize for design objectives. [15] introduces a family of CARA designs that balance efficiency and ethics objectives. Further generalizations to incorporate semiparametric estimates have been explored by [49]. Related developments in CARA designs include [22, 40, 48, 47]. However, conventional CARA designs often consider subgroups to be pre-specified, which is different from our goal of data-adaptively identifying the best subgroups.

Our proposed method also connects with the literature on subgroup identification. In post-hoc analyses using previously collected data, the first line of work uses data from prior randomized controlled trials. [20] employs clustering techniques based on randomized controlled trial data. A comprehensive review can be found in [23]. The second line of work uses data from prior observational studies. [43] identifies subgroups from existing observational data using machine learning. [36] introduces a causal inference tree approach for subgroup identification, which requires specifying the conditional distribution of the outcome given covariates. Similarly, [45] develops causal inference tree types of algorithm that include double robust estimators for constructing subgroupspecific splitting criteria. Other tree-based approaches for subgroup identification include [17, 24, 16]. Besides the tree-based approaches, [10] develops a subgroup identification method within the value function framework. While post-hoc subgroup analyses do not require new data collection, they often rely on untestable causal assumptions, limiting the credibility of causal conclusions. For instance, the unconfoundedness assumption necessary for causal inference in observational studies assumes random treatment assignment based on observed confounders, but unmeasured confounders can

compromise these conclusions. Conversely, in randomized experiments, valid causal conclusions do not depend on such assumptions.

In the adaptive experiment literature, relatively few methods have been developed for identifying the best subgroups. While [11] is in the adaptive experiment setting and proposes a Bayesian adaptive design that sequentially revises treatment allocation to identify the effective subgroup-treatment pairs, their approach is carried out under the Bayesian framework, relying on the specification of prior distributions and does not provide theoretical justification for the identified subgroups. In contrast, our design is aligned with the frequentist framework and is model-free, avoiding any parametric modeling assumption on the joint distribution of potential outcomes and covariates, and providing theoretical investigations from three aspects.

As our method aims to identify the best subgroups, it shares some similarities with the multi-armed bandit (MAB) literature. Notable MAB algorithms, such as the Thompson sampling method [35], the ϵ -greedy algorithm, and the upper confidence bound algorithm [37], focus on identifying the best arm. Similar to the contextual bandit literature [21, 6, 1], our design also incorporates covariate information. However, our design objective diverges significantly from those in traditional MAB approaches as we seek to identify the best subgroups rather than the best arm. Moreover, recognizing that randomized experiments can be time-consuming, costly, and may result in adverse outcomes for patients if treatments are ineffective, our approach seeks to efficiently allocate experimental resources within a constrained budget to identify the most beneficial subgroups. Specifically, we focus on scenarios in which the cost per experimental unit (e.g., per patient) is significant. For instance, in clinical settings, randomized experiments are often expensive due to the substantial costs of treatment medications. As a result, the primary resource constraint in our framework is the limited number of treatments that can be administered, underscoring the need for a resource-efficient experimental design.

2 Formulation of CARA

In this section, we shall introduce the formulation of our covariate-adjusted response-adaptive randomization (CARA) experiment framework.

We enroll participants sequentially across T stages, where T < ∞ . Denote the total number of enrolled participants as N = ∑ T t =1 n t , where n t is the number of participants in Stage t , for t = 1 , . . . , T . The cumulative sample size up to Stage t is denoted as N t = ∑ t s =1 n s . In Stage t , we denote the treatment assignment status of participant i as D it ∈ { 0 , 1 } , i = 1 , . . . , n t , where D it = 1 denotes the treatment arm, and D it = 0 denotes the control arm. The observed outcome is denoted as Y it ∈ R . We follow the Neyman-Rubin causal model [27, 34] to define Y it ( d ) as the potential outcome we would have observed if participant i receives treatment d at Stage t , for d ∈ { 0 , 1 } . The observed outcome can then be represented as

We assume that the outcomes are observed without delay, and their underlying distributions do not shift over time [14].

In CARA experiments, covariate information is also available to practitioners. We denote the covariate information for participant i as X it ∈ R and assume the covariate space X can be partitioned into m regions, denoted as {S j } j m =1 . In clinical settings, each partition of the sample space is commonly referred to as a subgroup [3, 19, 44]. We denote the number of subjects enrolled in subgroup j at Stage t as n tj = ∑ n t i =1 1 ( X it ∈S j ) and the cumulative sample size for group j up to Stage t is N tj = ∑ t s =1 n sj . Denote the total number of subjects enrolled in subgroup j as N j = N Tj .

As we are interested in assessing the effectiveness of the treatment in each subgroup, we define the subgroup average treatment effect as

Due to the adaptive nature of CARA experiments, practitioners can sequentially revise treatment allocation based on outcome and covariate information accumulated during the experiment. Formally, we define the treatment assignment probability for participants in subgroup j as

where H t -1 = { ( Y is , D is , X is ) n s i =1 } t -1 s =1 denotes the historical information up to Stage t -1 . In CARA experiments, we aim to dynamically revise e tj to reach desired design goals, which shall be introduced in the following section.

3 Design objective for best subgroups identification

In real-world applications, suppose we start with m subgroups, and practitioners may only aim to find subgroups with the largest treatment effects. It is possible that the best-performing subgroup is not unique. For example, the FORTE trial is a clinical trial aiming to investigate the treatment effect of carfilzomib-based induction-intensification-consolidation regimens on a patient's progressionfree survival rate [25]. Instead of reporting the single best subgroup with the largest survival rate improvement, the trial reports that both the risk myeloma patient subgroup and high-risk patient subgroup show similar survival rate improvement.

Without loss of generality, suppose the population subgroup treatment effects follow the order τ 1 ≥ τ 2 ≥ . . . ≥ τ k > . . . > τ m . In this case, there are k subgroups exhibiting the largest treatment effects. Thus, it is natural to identify all of the k best subgroups instead of the single best subgroup.

To describe the set of best-performing subgroups, we introduce the concept of a tie set. We denote the tie set of τ 1 as T 1 = { k : | τ 1 -τ k | = o ( N -1 / 2 ) , k = 1 , . . . , m } which contains the indices of the tied subgroups. This tie set is also known as the 'near tie set" as it captures the subgroups of which the treatment effects lie in the √ N -local neighborhood of τ 1 . We then denote the subgroups that belong to the best set as S T 1 . Note that when T 1 = { 1 } , S T 1 is equivalent to S 1 . The population treatment effect under the best subgroups is defined as τ T 1 = E [ Y it (1) -Y it (0) | X it ∈ ∪ j ∈T 1 S j ] . We further assume that τ T 1 > τ j , for j / ∈ T 1 .

Our design objective is to correctly identify all the best subgroups. Mathematically, we aim to maximize the correct identification probability:

Leveraging the large deviation theory [8, 12], we can formulate our design objective as

where T 1 = { 1 , . . . , k } , 1 ≤ k < m , δ ∈ (0 , 1 / 2) , and V T 1 denotes the variance of the best subgroups. Detailed derivations of the equivalence between the correct identification probability and the optimization objective see Appendix (Section C).

However, in practice, solving this optimization problem is challenging. On the one hand, as we do not have knowledge regarding the membership of subgroups that have the largest treatment effects, we do not have any information regarding S T 1 . On the other hand, because experimenters have no prior information about the joint distribution of the subgroup treatment effects, τ j 's and V j 's are also unknown. To address these two practical challenges, we propose a dynamic subgroup identification algorithm that can adaptively identify and merge the set of best subgroups. Additionally, the dynamic subgroup identification method operates seamlessly under a CARA experimental strategy, which allows experimenters to sequentially learn the unknown parameters and adjust the subgroup treatment allocation to attain our design objective.

4 Proposed design: Dynamic subgroup identification with CARA

In this section, we shall illustrate our proposed dynamic subgroup identification strategy with CARA design in Algorithm 1: the design strategy in Section 4.1, encompassing two sub-algorithms (Algorithm 2 and 3), followed by the statistical inference procedure in Section 4.2. We defer several variations of our algorithm to Appendix (Section J). To clarify our design strategy, we also provide a notation table in the Appendix (Section A, Table 2).

4.1 Proposed design strategy

In Stage 1 (line 1-4), we obtain initial estimates of the group-level treatment effect ̂ τ 1 j and the associated variances ̂ V 1 j . Then, in Stage t = 2 , . . . , T -1 (Algorithm 1 line 6-11), we perform three tasks: (1) adaptively update treatment allocation and treatment effect estimates, (2) dynamically identify best subgroups, and (3) select hyperparameters that help with dynamic subgroup identification.

For the first task, we obtain the optimal treatment allocation ̂ e ∗ t = ( ̂ e ∗ t 1 , . . . , ̂ e ∗ tm ∗ t ) by solving the following optimization problem based on sequentially collected data:

where c 1 ∈ (0 , 1) and c 2 ∈ (0 , 1 / 2) , ̂ p tl = ∑ t s =1 ∑ ns i =1 1 ( Xis ∈S ( l ) ) ∑ t s =1 n s is the estimated subgroup proportion. The total number of subgroups after merging the identified tie set in Stage t -1 is denoted as m ∗ t = m -| ̂ T t -1 , 1 | +1 and the subscript ( j ) indexes the subgroup with the j -th largest estimated treatment effect. The procedure of finding ̂ T t -1 , 1 shall be illustrated in Algorithm 2. In the set of constraints, the first one is the resource constraint, and the second one is the feasibility constraint. Because of the nonlinear objective function of the optimization problem above, we instead work with its equivalent epigraph representation:

To calibrate for the complete randomized treatment allocation in Stage 1, we require an additional calibration step in Stage t :

where n t, ( j ) = ∑ n t i =1 1 ( X it ∈S ( j ) ) , N t -1 , ( j ) (1) = ∑ t -1 s =1 ∑ n s i =1 1 ( X is ∈S ( j ) ) D is , and N t, ( j ) = ∑ t s =1 n s, ( j ) . We then allocate treatments with calibrated probability ˜ e t, ( j ) and update the subgroup treatment effects as in Eq (8).

Dynamic identification of the best subgroups (Algorithm 2). The dynamic subgroup identification algorithm for identifying ̂ T t, 1 at Stage t involves a resampling step that generates bootstrap samples ̂ τ ◦ t from a Gaussian distribution centering around ̂ τ t at Stage 1 and a resampling step that generates bootstrap samples with accrued data {H s } t s =1 at later stages. In line 9, we identify the best subgroups at Stage t as

where the distance between the upper and lower bounds of the interval for w ◦ k, (1) is of order n δ with δ = 0 . 25 to guarantee the statistical validity of our proposed procedure and balance the trade-off between bias and variance. Note that the dynamic subgroup identification procedure relies on a pair of hyperparameters ( c t L , c t R ) , which are selected data-adaptively. In what follows, we shall illustrate the algorithm for selecting these hyperparameters.

Hyperparameter selection (Algorithm 3) . In line 7 of Algorithm 3, we adopt a bootstrap method and propose several alternative bootstrap methods in Algorithm 5 (line 3) in Appendix (Section J). Algorithm 3 involves a resampling step that generates bootstrap samples ̂ τ ∗ t from a Gaussian distribution centering around τ ∗ t at Stage 1. In line 2, we compute τ ∗ t = ( τ ∗ t 1 , . . . , τ ∗ t,m ∗ t ) ′ as

where ∆ = min { 0 . 99 , ∑ m ∗ t j =1 ̂ V tj N t ∑ m ∗ t j =1 ( ̂ τ tj -̂ τ t ) 2 × N γ t } and γ ∈ (0 , 0 . 2) . We impose a lower bound on ∆ to ensure that ∆ does not equal to 1. We choose γ = 0 . 05 in our simulation studies, and our procedure is shown to be not sensitive to the choice of γ < 1 . In line 7, we compute ̂ τ ∗ t = ( ̂ τ ∗ t 1 , . . . , ̂ τ ∗ t,m ∗ t ) ′ at Stage t for t > 1 as

where ̂ τ ◦ tj is computed with the bootstrap samples as in Eq (8). In line 10, we compute

where w ∗ k, (1) = 1 {-c t L · N -δ t · ̂ V δ t, (1) ≤ ( ̂ τ ∗ tk -̂ τ ∗ t, (1) ) ≤ c t R · N -δ t · ̂ V δ t, (1) } and the subscript (1) indexes the subgroup with the largest estimated treatment effect. We also propose a double bootstrap-based alternative hyperparameter selection procedure in Algorithm 4 in Appendix (Section J). Finding the optimal hyperparameters involves minimizing a loss function L t ( c L , c R ) at each Stage t , which is defined as

where for l = 0 , 1 , L tl ( c L , c R ) = 1 B ∑ B b =1 ( 1 ( B tb ( c L , c R ) = l ) -∑ B b =1 1 ( B tb ( c L ,c R )= l ) B ) 2 . Given a desirable pair of hyperparameters ( c L , c R ) and l value, the indicator function 1 ( B tb ( c L , c R ) = l ) is binary, which roughly follows a Bernoulli distribution with probability ∑ B b =1 1 ( B tb ( c L ,c R )= l ) B . Intuitively, the loss function defined in Eq (7) measures the average of squared differences between 1 ( B tb ( c L , c R ) = l ) and the expected value of Bernoulli ( ∑ B b =1 1 ( B tb ( c L ,c R )= l ) B ) random variables. We would expect that the optimal pair of hyperparameters ( c t L , c t R ) at Stage t minimizes such a loss.

Algorithm 1 Dynamic subgroup identification CARA design

Stage 1 (Initialization) :

1: Enroll n 1 participants, and assign treatments in group j with e 1 j = 1 2 ;

2: Compute ̂ τ 1 j and ̂ V 1 j ;

3: Choosing hyperparameters ( c 1 L , c 1 R ) using single bootstrap method (see Algorithm 3);

4: Identify tie set and merge tied subgroups with the best subgroup (see Algorithm 2). Stage t (Adaptive treatment allocation revision) :

5: for t → 2 to T do

6: With ̂ τ t -1 , ( j ) and ̂ V t -1 , ( j ) estimated using Eq 8, solve the optimization problem in Eq (1) to find ̂ e ∗ t, ( j ) ;

7: Enroll n t participants and assign treatment with calibrated probability ˜ e ∗ t, ( j ) as in Eq (2);

8: Update ̂ τ t, ( j ) and ̂ V t, ( j ) as in Eq (8);

9: Choosing hyperparameters ( c t L , c t R ) using single bootstrap (see Algorithm 3);

10: Identify tie set and merge tied subgroups with the best subgroup (see Algorithm 2).

11: Calculate the merged subgroup ATE estimator ̂ τ t, ̂ T t 1 as in Eq (9), and its variance estimator ̂ V t, ̂ T t 1 as in Eq (10).

12: end for

Stage T (Inference):

13: Identify the best tie set ̂ T 1 , and construct two-sided confidence intervals for ̂ τ ̂ T 1 as in Eq (11).

4.2 Statistical inference

Our CARA design also enables making valid statistical inference on the estimated best subgroup treatment effect. We highlight two parts of the statistical inference procedure: (1) estimating unknown

parameters based on accrued experimental data while dynamically identifying best subgroups, and (2) constructing valid confidence intervals to confirm the estimated best subgroup treatment effect.

First, based on accumulated experimental data, the subgroup treatment effects and associated variances can be updated as

Additionally, denote the subgroup proportions as p 1 , . . . , p m . Following dynamic subgroup identification at each stage, we merge all subgroups in the ̂ T t 1 and estimate the merged best subgroup treatment effect as

and estimate the variance of the merged best subgroup treatment effect estimator as

In Stage T (line 13), we let ̂ T 1 := ̂ T T 1 , ̂ τ ̂ T 1 := ̂ τ T, ̂ T 1 and ̂ V ̂ T 1 := ̂ V T, ̂ T 1 . Lastly, to confirm the estimated best subgroup treatment effect, we construct a two-sided levelα confidence interval as

Algorithm 2 Dynamic subgroup identification in Stage t

Step 1 (Input) :

1: Input {H s } t s =1 , ̂ τ tj , ̂ V tj , and ( c t L , c t R ) computed from Algorithm 3. Step b (Bootstrap) :

2: for b ← 1 to B do

3: if t = 1 then

4: Generate ̂ τ ◦ 1 from N ( ̂ τ 1 , ̂ Ω n /n 1 ) , where ̂ Ω n = diag ( ̂ V 11 , . . . , ̂ V 1 m ) ;

5: else if t > 1 then

6: Generate n s resamples randomly with replacement sequentially from each H s ;

8: end if

7: Compute ̂ τ ◦ tj as in Eq (8) with the bootstrap samples;

9: Identify the best subgroups ̂ T t 1 with the bootstrap samples as in Eq (3).

10: end for Step B

(Output):

11: Choose ̂ T t 1 with the highest frequency of occurrence and merge subgroups that belong to ̂ T t 1 .

5 Theoretical investigation

Assumption 1 (Regularity conditions) . ( Y it (0) , Y it (1) , X it ) are independently identically distributed for i = 1 , . . . , n t , t = 1 , . . . , T . In addition, E [ | Y it ( d ) | ] 4 < ∞ , d ∈ { 0 , 1 } . Lastly, there exists some δ > 0 , such that V [ Y it ( d ) | X it ∈ S j ] ≥ δ for d ∈ { 0 , 1 } , j = 1 , . . . , m .

Assumption 2 (Positivity) . The subgroup proportions δ ≤ p 1 , . . . , p m ≤ 1 -δ , δ ∈ (0 , 1 / 2) .

Assumption 1 says that the potential outcomes have bounded moments and have variability in each subgroup. Assumption 2 says that the subgroup proportions are non-zero in the population.

Theorem 1 (Dynamic best subgroup identification consistency) . Under Assumptions 1 and 2, for j = 1 , . . . , m and for ε > 0 , we have

Theorem 1 suggests that our dynamic subgroup identification algorithm correctly identifies the best set of subgroups as the sample size tends to infinity.

Theorem 2 (Design strategy consistency) . Under Assumptions 1 and 2, for δ > 0 , as n t → ∞ , t = 1 , . . . , T , for the actual treatment allocation, we have

where e ∗ = ( e ∗ 1 , . . . , e ∗ m ) is the optimal treatment allocation.

Theorem 2 says that the actual treatment allocation under our proposed design strategy converges to the optimal treatment allocation asymptotically.

Theorem 3 (Asymptotic normality) . Under Assumptions 1 and 2, as N →∞ , and

Theorem 3 says that the estimated treatment effect of the identified best subgroups converge to a Gaussian distribution asymptotically, and our variance estimator consistently estimates the asymptotic variance. Theorem 3 also verifies the validity of our constructed confidence interval.

6 Synthetic real data study

In this section, we investigate the performance of our proposed design strategy for identifying the tie set of best-performing subgroups in a synthetic case study using clinical trial data.

We design our synthetic case study using the dataset from the Mayo Clinic's trial on primary biliary cirrhosis (PBC), containing clinical biomarkers, treatments, and patient outcomes. PBC is a progressive autoimmune liver disease marked by inflammation and damage to the intrahepatic bile ducts. The Mayo Clinic conducted an extensive trial from 1974 to 1984 to assess the effectiveness of D-penicillamine in treating PBC. This dataset includes 424 patients, encompassing both those who were actively enrolled in the trial and additional cases who consented to provide basic measurements [26].

In this case study, we work with a subset ( n = 312 ) of patients who participate in the randomized controlled trial. These patients are randomly assigned to one of the two arms: the treatment arm ( n = 158 ), who receive D-penicillamine D = 1 , and the control arm ( n = 154 ), who receive placebos D = 0 . The outcome of interest is the square root of the survival time, defined as the number of days from registration to the earlier death, transplantation, or the time of study analysis. This dataset includes 17 covariates, and we use median imputation to handle missingness in these covariates. We aim to investigate the effectiveness of D-penicillamine in improving liver function and symptoms in five subgroups defined by age (in days): (1) patients with age in [9 , 598 , 15 , 695] , (2) age in (15 , 695 , 17 , 082] , (3) age in (17 , 082 , 20 , 440] , (4) age in (20 , 440 , 21 , 900] , (5) age in (21 , 900 , 28 , 650] . We generate synthetic experimental data based on the original dataset, which shall be illustrated in the next section.

We generate synthetic data that mimic the original PBC dataset. Denote the group membership for each participant i as S = ( 1 ( X i ∈S 1 ) , . . . , 1 ( X i ∈S 5 ) ) ⊺ . We generate the potential outcome from Y ( d ) | X ∈ S ∼

subi i j N ( µ dj , σ 2 dj ) , j = 1 , . . . , 5 , where µ 1 = (42 . 57 , 50 . 44 , 44 . 37 , 44 . 30 , 37 . 71) ⊺ , µ 0 = (45 . 34 , 39 . 91 , 45 . 58 , 33 . 42 , 39 . 17) ⊺ , σ 1 = (10 . 85 , 12 . 29 , 12 . 64 , 14 . 28 , 14 . 64) ⊺ , σ 0 = (11 . 50 , 15 . 18 , 14 . 57 , 13 . 09 , 15 . 06) ⊺ . The subgroup proportions are p = (0 . 28 , 0 . 13 , 0 . 30 , 0 . 11 , 0 . 19) ⊺ . We denote the true subgroup treatment effects as τ = ( -2 . 77 , 10 . 53 , -1 . 21 , 10 . 89 , -1 . 46) ⊺ . Therefore, Subgroup 2 and Subgroup 4 are the set of best subgroups with a merged average treatment effect of 10.70. The treatment assignment D i is decided based on different experiment strategies, which shall be discussed later in the section. To generate synthetic data, We mimic CARA experiments where participants are enrolled sequentially across T experimental stages. Here, we set T = 15 and n t = 400 , for t = 1 , . . . , T . All experiments are conducted with an Intel Core i7-11800H CPU and 16 GB of RAM.

We compare our proposed design strategy with the complete randomization design and two multi-armed bandit (MAB) algorithms. (1) The complete randomization design refers to a design that fixes e tj = 1 2 across all experimental stages, t = 1 , . . . , T, j = 1 , . . . , m . (2) To customize the MAB algorithms to our setting, for each subgroup, we consider two candidate arms: treatment and control. We set the rewards as the negative asymptotic variance of the treatment effect estimator, i.e., -V j ( e j ) . We consider two MAB algorithms: (a) The ϵ -greedy algorithm, which aims to balance the exploration and the exploitation efforts [37]. Here, we set ϵ = 0 . 1 . (b) The upper confidence bound 1 algorithm balances exploration and exploitation using confidence intervals and chooses the arm that maximizes the upper confidence bound on the estimated reward [4]. When customizing MAB algorithms to our setting, we omit the step of identifying and merging tie sets using conventional methods, such as K-means or agglomerative clustering, due to several challenges: the requirement to predefine the number of clusters, the limited applicability of clustering for a small number of subgroups, and the inconsistency of these methods in effectively merging the best subgroups.

0.5

0.6

0.7

0.8

0.9

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Correct selection probability

Methods

Complete randomization design

Epsilon greedy algorithm

Single and separate bootstrap

Upper confidence bound 1 algorithm

Algorithm 3 Hyperparameter selection for dynamic subgroup identification

Step 1 (Input) :

1: Input {H s } t s =1 , ̂ τ tj , ̂ V tj , and ( c L , c R ) ;

2: Compute τ ∗ tj as in Eq (4).

Step

b

(Bootstrap)

:

3: for b ← 1 to B do

4: if t = 1 then

5: Generate ̂ τ ∗ 1 from N ( τ ∗ 1 , ̂ Ω n /n 1 ) where ̂ Ω n = diag ( ̂ V 11 , . . . , ̂ V 1 m ) ;

6:

7:

else if t > 1 then

Generate n s resamples randomly with replacement sequentially from each H s (the same as Algorithm 2 line 6);

8:

9: end if

Compute ̂ τ ◦ tj as in Eq (8) with the bootstrap samples, and then ̂ τ ∗ tj as in Eq (5);

10: Compute ˜ τ ∗ t, (1) as in Eq (6), and B tb ( c L , c R ) as in Eq (7).

11: end for Step B

(Output):

12: Compute L t ( c L , c R ) as in Eq (7). Choose the pair ( c t L , c t R ) that minimizes L t ( c L , c R ) .

,

To evaluate the performance of different design strategies, we assess the effectiveness of each adaptive experiment strategy from two aspects. First, we compare the correct selection probability of identifying the best subgroups. The correction selection probability can be written as P ( ̂ τ T 1 ≥ max j / ∈T 1 ̂ τ j ) . Second, we compare the 95% confidence interval, √ N -scaled bias, and standard deviation of the estimated best subgroup treatment effect. In our resampling procedure, we set B = 2 , 000 . The synthetic case study results are summarized in Figure 1 and Table 1.

First, from Figure 1, our proposed design strategy shows a higher correct selection probability than the complete randomization design and the MAB algorithms. Additionally, the correct selection probability under our proposed design strategy increases with the number of experimental stages. Specifically, our proposed design has a correct selection probability tending to 1 after 15 experimental stages. We also adopt the normalized mutual information as an additional metric to compare our proposed design and three competing methods in the Appendix (Section K.1), which further confirms that our proposed design strategy outperforms the conventional methods.

Second, from Table 1, we observe that our proposed design strategy has a smaller standard deviation and smaller √ N -scaled bias, implying that our method is more efficient and less biased. In sum, our proposed adaptive design demonstrates efficient use of experimental data to correctly identify the best-performing subgroups of the three competing methods. We defer additional simulation studies to the Appendix (Section K.1) and an additional synthetic real data study to the Appendix (Section K.3). We also extend our proposed dynamic subgroup identification with CARA to the augmented inverse propensity score weighting (AIPW) estimator. Then we compare our proposed design with AIPW estimator with the three contextual MAB algorithms in the Appendix (Section K.2).

7 Discussion

We propose a dynamic subgroup identification method within the CARA design framework that could significantly advance precision medicine. However, we acknowledge some limitations in our approach. We aim to explore and address these challenges in our future research.

First, our method assumes that outcomes in CARA experiments are observed immediately at the end of each stage without delay. This assumption, prevalent in adaptive experiments such as [15] and [49], simplifies the modeling process and facilitates quick adjustments based on the latest data. However, in some practical scenarios, outcomes may be observed with delays, complicating the process of adjusting treatment allocations, as highlighted by [31] and [29]. In future work, we plan to revise our design framework and update our estimators to account for the impact of delayed responses on both treatment effects and variance estimators.

Second, our work addresses scenarios where assigning a treatment is costly, and there is an overall constraint on how many treatments can be deployed. A key challenge arises from the potential misalignment between efficiently estimating the best causal effect and determining the optimal causal decision rule-particularly when the best decision rule is to treat all patients when there is any positive effect. For instance, [9] focuses on ensuring that all individuals who would benefit are accurately assigned to the treatment arm, assuming negligible treatment costs. To maximize the welfare for participating subjects, in future work, we plan to incorporate an 'early stopping" step which would not only identify the most effective subgroups but also halt enrollment for subgroups exhibiting significantly adverse treatment effects.

References

Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning , pages 127-135. PMLR, 2013.

Derek C Angus and Chung-Chou H Chang. Heterogeneity of treatment effect: estimating how the effects of interventions vary across individuals. Jama , 326(22):2312-2313, 2021.

Susan F Assmann, Stuart J Pocock, Laura E Enos, and Linda E Kasten. Subgroup analysis and other (mis) uses of baseline data in clinical trials. The Lancet , 355(9209):1064-1069, 2000.

Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine learning , 47:235-256, 2002.

Uttam Bandyopadhyay and Atanu Biswas. Allocation by randomized play-the-winner rule in the presence of prognostic factors. Sankhy¯ a: The Indian Journal of Statistics, Series B , pages 397-412, 1999.

Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages 208-214. JMLR Workshop and Conference Proceedings, 2011.

Rajeev H Dehejia and Sadek Wahba. Propensity score-matching methods for nonexperimental causal studies. Review of Economics and statistics , 84(1):151-161, 2002.

Amir Dembo and Ofer Zeitouni. Large deviations techniques and applications , volume 38. Springer Science & Business Media, 2009.

Carlos Fernández-Loría and Foster Provost. Causal decision making and causal effect estimation are not the same. . . and why it matters. INFORMS Journal on Data Science , 1(1):4-16, 2022.

Haoda Fu, Jin Zhou, and Douglas E Faries. Estimating optimal treatment regimes via subgroup identification in randomized control trials and observational studies. Statistics in medicine , 35(19):3285-3302, 2016.

Wentian Guo, Yuan Ji, and Daniel VT Catenacci. A subgroup cluster-based bayesian adaptive design for precision medicine. Biometrics , 73(2):367-377, 2017.

Frank Hollander. Large deviations , volume 14. American Mathematical Soc., 2000.

F. Hu and L.-X. Zhang. Asymptotic normality of urn models for clinical trials with delayed response. Bernoulli , 10(3):447-463, 2004.

Feifang Hu and William F Rosenberger. The theory of response-adaptive randomization in clinical trials . John Wiley & Sons, 2006.

Jianhua Hu, Hongjian Zhu, and Feifang Hu. A unified family of covariate-adjusted response-adaptive designs based on efficiency and ethics. Journal of the American Statistical Association , 110(509):357-367, 2015.

Joseph Kang, Xiaogang Su, Brian Hitsman, Kiang Liu, and Donald Lloyd-Jones. Tree-structured analysis of treatment effects with large observational data. Journal of Applied Statistics , 39(3):513-529, 2012.

Joseph Kang, Xiaogang Su, Lei Liu, and Martha L Daviglus. Causal inference of interaction effects with inverse propensity weighting, g-computation and tree-based standardization. Statistical Analysis and Data Mining: The ASA Data Science Journal , 7(5):323-336, 2014.

David M Kent, Ewout Steyerberg, and David Van Klaveren. Personalized evidence based medicine: predictive approaches to heterogeneous treatment effects. Bmj , 363, 2018.

K Kubota, Y Ichinose, G Scagliotti, D Spigel, JH Kim, T Shinkai, K Takeda, S-W Kim, T-C Hsia, RK Li, et al. Phase iii study (monet1) of motesanib plus carboplatin/paclitaxel in patients with advanced nonsquamous nonsmall-cell lung cancer (nsclc): Asian subgroup analysis. Annals of oncology , 25(2): 529-536, 2014.

Beom S Lee et al. A clustering method to identify who benefits most from the treatment group in clinical trials. Health Psychology and Behavioral Medicine: an Open Access Journal , 2(1):723-734, 2014.

Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web , pages 661-670, 2010.

Yunzhi Lin, Ming Zhu, and Zheng Su. The pursuit of balance: an overview of covariate-adaptive randomization techniques in clinical trials. Contemporary clinical trials , 45:21-25, 2015.

Ilya Lipkovich, Alex Dmitrienko, and Ralph B D'Agostino Sr. Tutorial in biostatistics: data-driven subgroup identification and analysis in clinical trials. Statistics in medicine , 36(1):136-196, 2017.

Wei-Yin Loh, Xu He, and Michael Man. A regression tree approach to identifying subgroups with differential treatment effects. Statistics in medicine , 34(11):1818-1833, 2015.

Roberto Mina, Pellegrino Musto, Delia Rota-Scalabrini, Laura Paris, Barbara Gamberi, Angelo Palmas, Sara Aquino, Paolo de Fabritiis, Nicola Giuliani, Luca De Rosa, et al. Carfilzomib induction, consolidation, and maintenance with or without autologous stem-cell transplantation in patients with newly diagnosed multiple myeloma: pre-planned cytogenetic subgroup analysis of the randomised, phase 2 forte trial. The Lancet Oncology , 24(1):64-76, 2023.

Paul A Murtaugh, Rolland E Dickson, Gooitzen M Van Dam, Michael Malinchoc, Patricia M Grambsch, Alice L Langworthy, and Chris H Gips. Primary biliary cirrhosis: prediction of short-term survival based on repeated patient visits. Hepatology , 20(1):126-134, 1994.

Jerzy S Neyman. On the application of probability theory to agricultural experiments. essay on principles. section 9.(translated and edited by dm dabrowska and tp speed, statistical science (1990), 5, 465-480). Annals of Agricultural Sciences , 10:1-51, 1923.

Mark J Pletcher and Charles E McCulloch. The challenges of generating evidence to support precision medicine. JAMA Internal Medicine , 177(4):561-562, 2017.

D. S. Robertson, K. M. Lee, B. C. López-Kolkovska, and S. S. Villar. Response-adaptive randomization in clinical trials: from myths to practical considerations. Statistical Science: A Review Journal of the Institute of Mathematical Statistics , 38(2):185, 2023.

David S Robertson, Kim May Lee, Boryana C López-Kolkovska, and Sofía S Villar. Response-adaptive randomization in clinical trials: from myths to practical considerations. Statistical science: a review journal of the Institute of Mathematical Statistics , 38(2):185, 2023.

W. F. Rosenberger, O. Sverdlov, and F. Hu. Adaptive randomization for clinical trials. Journal of Biopharmaceutical Statistics , 22(4):719-736, 2012.

William F Rosenberger. Randomized urn models and sequential design. Sequential Analysis , 21(1-2):1-28, 2002.

William F Rosenberger, AN Vidyashankar, and Deepak K Agarwal. Covariate-adjusted response-adaptive designs for binary response. Journal of biopharmaceutical statistics , 11(4):227-236, 2001.

Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology , 66(5):688, 1974.

Daniel J Russo, Benjamin Van Roy, Abbas Kazerouni, Ian Osband, Zheng Wen, et al. A tutorial on thompson sampling. Foundations and Trends® in Machine Learning , 11(1):1-96, 2018.

Xiaogang Su, Joseph Kang, Juanjuan Fan, Richard A Levine, and Xin Yan. Facilitating score and causal inference trees for large observational studies. Journal of Machine Learning Research , 13:2955, 2012.

Richard S Sutton and Andrew G Barto. Introduction to reinforcement learning. mit press. Cambridge, MA , 1998.

Julien Tanniou, Ingeborg Van Der Tweel, Steven Teerenstra, and Kit CB Roes. Subgroup analyses in confirmatory clinical trials: time to be specific about their purposes. BMC medical research methodology , 16:1-15, 2016.

U.S. Food and Drug Administration. Assessing and communicating heterogeneity of treatment effects in patient subpopulations: Challenges and opportunities. https://www.fda.gov/science-research/advancing-regulatory-science/ assessing-and-communicating-heterogeneity-treatment-effects-patient-subpopulations-challenges-and , 2024.

Sofía S Villar and William F Rosenberger. Covariate-adjusted response-adaptive randomization for multi-arm clinical trials using a modified forward looking gittins index rule. Biometrics , 74(1):49-57, 2018.

Waverly Wei, Xinwei Ma, and Jingshen Wang. Adaptive experiments toward learning treatment effect heterogeneity. arXiv preprint arXiv:2312.06883 , 2023.

Waverly Wei, Yuqing Zhou, Zeyu Zheng, and Jingshen Wang. Inference on the best policies with many covariates. Journal of Econometrics , page 105460, 2023.

Jie Xu et al. Machine learning enabled subgroup analysis with real-world data to inform clinical trial eligibility criteria design. Scientific Reports , 13(1):613, 2023.

Yanxun Xu, Lorenzo Trippa, Peter Müller, and Yuan Ji. Subgroup-based adaptive (suba) designs for multi-arm biomarker trials. Statistics in Biosciences , 8(1):159-180, 2016.

Jiabei Yang, Issa J Dahabreh, and Jon A Steingrimsson. Causal interaction trees: Finding subgroups with heterogeneous treatment effects in observational data. Biometrics , 78(2):624-635, 2022.

Li-Xin Zhang, Feifang Hu, Siu Hung Cheung, and Wai Sum Chan. Asymptotic properties of covariateadjusted response-adaptive designs. The Annals of Statistics , 35(3):1166-1182, 2007.

Jinglong Zhao. Adaptive neyman allocation. arXiv preprint arXiv:2309.08808 , 2023.

Wanying Zhao, Wei Ma, Fan Wang, and Feifang Hu. Incorporating covariates information in adaptive clinical trials for precision medicine. Pharmaceutical Statistics , 21(1):176-195, 2022.

Hai Zhu and Hongjian Zhu. Covariate-adjusted response-adaptive designs based on semiparametric approaches. Biometrics , 2023.

Technical Appendix

A Notations

We present a notation table, as shown in Table 2, to illustrate the key symbols and their descriptions used in our covariate-adjusted response-adaptive randomization (CARA) experiment framework.

B Assumptions, lemmas, and corollaries

Before discussions, we define some additional notations first. For t = 1 , . . . , T , j = 1 , . . . , m , and c > 0 , denote

Assumption 3 (Subgroup treatment effects) . For δ ∈ (0 , 0 . 5) , the asymptotic distance between treatment effects of Subgroup j and Subgroup 1 diverges as N →∞ :

Lemma 1. Assume Assumption 1 and 2 holds. Let p j = P ( X it ∈ S j | H t -1 ) be the subgroup proportions, and ̂ e ∗ tj = P ( D it = 1 | X it ∈ S j , H t -1 ) be the treatment probabilities. Assume there exists some δ ∈ (0 , 1 / 2) such that for all j = 1 , 2 , . . . , m and t = 1 , 2 , . . . T ,

Then for any j = 1 , 2 , . . . , m , d = 0 , 1 , and any t satisfying n t →∞ ,

Corollary 1. Let δ ∈ (0 , 1 / 2) be some constant. Assume Assumptions 1 and 2 hold,

Corollary 2. Assume Assumptions 1 and 2 hold. Then

Theorem 4. Under Assumptions 1 and 2, as N →∞ ,

Lemma 1, Corollaries 1 and 2, and Theorem 4 are similar to the theoretical results in [41]. We defer readers to [41] for more technical details.

Lemma 2. Under Assumptions 3, for j = 1 , . . . , m , t = 1 , . . . , T , any positive constant C and δ ∈ (0 , 1 2 ) , the following statement holds

Lemma 3. Let δ ∈ (0 , 1 / 2) be some constant. Assume Assumptions 1 and 2 hold,

Lemma 4. Assume Assumptions 1 and 2 hold. Let E ∗ be the solution(s) to the problem as follows:

where c 1 ∈ (0 , 1) , c 2 ∈ (0 , 1 / 2) , T 1 = { 1 , . . . , k } , 1 ≤ k < m , and V T 1 denotes the variance of the top subgroups. Let ̂ E ∗ be the optimized treatment allocations solved from the sample analog of this problem. Then for any δ > 0 , and any t satisfying N t -1 →∞ ,

C Optimization problem objective function derivations

Proof. Because maximizing the correct selection probability is equivalent to minimizing the incorrect selection probability, we formulate the incorrect selection probability as P ( ̂ τ T 1 ≤ max j / ∈T 1 ̂ τ j ) and denote the cardinality of T 1 as k , 1 ≤ k ≤ m -1 , where m denotes the total number of subgroups. The incorrect selection probability can be bounded as

where the inequality follows from the union bound. For both sides, take the logarithm and divide by N ,

Additionally, by the large deviation theory [8], there exists a rate function G ( S T 1 , S j ; e 1 , e j ) , such that

Therefore, max j / ∈T 1 ( 1 -P ( ̂ τ T 1 ≥ ̂ τ j ) ) is equivalent to exp ( -min j / ∈T 1 G ( S T 1 , S j ; e 1 , e j ) ) . We obtain

As a result, based on the Gartner-Ellis Theorem [8, ch.2.3], when the sample size tends to infinity, we are able to obtain:

D Proof of Lemma 2

Proof. Note that

As for the first term √ N t ( ̂ τ tj -τ j ) , by Corollary 2, we have

As for the second term √ N t ( ̂ τ ◦ tj -̂ τ tj ) , we consider two situations under t = 1 and t > 1 separately. When t = 1 , ̂ τ ◦ 1 j is generated from N ( τ ◦ 1 j , ̂ V 1 j /N 1 ) , so we have

When t > 1 , we compute ̂ τ ◦ tj the same way as ̂ τ tj with the bootstrap samples instead, so ̂ τ ◦ tj and ̂ τ tj are consistent and we have

Therefore, we obtain for t ≥ 1 , √ N t ( ̂ τ ◦ tj -τ j ) = O p (1) , i.e., for any given ε > 0 , there exists an M , such that

Then for any N t such that N t > ( M C ) 1 1 2 -δ , we have

Note that the above inequality holds for arbitrary ε > 0 . Hence, we have

and thus which completes the proof.

E Proof of Theorem 1

Proof. Recall our definition of the tie set of best-performing subgroups:

This suggests that ∀ j ∈ T 1 , there exists a sequence δ n → 0 as N → 0 , such that

We further define a set of subgroups with treatment effects smaller than those in the set T 1 :

As for the estimated near tie set of τ 1 in Stage t for t = 1 , . . . , T , we have for any j ∈ ̂ T t 1 , and a pair of hyperparameters ( c t L , c t R ) that

where δ = 0 . 25 . Thus, there exists a positive constant C such that

Similar to the proof of Lemma 1 in [42], our proof is composed of the following three steps:

Step 1. We first show that the subgroup with the largest treatment effect in the resampled statistics falls into the set T 1 with high probability, that is

where ˇ 1 t = ∑ m ∗ t j =1 j · 1 ( ̂ τ ◦ tj = ̂ τ ◦ t, (1) ) for t = 1 , . . . , T .

Because ̂ τ ◦ t, ˇ 1 t ∈ [min j ∈T 1 ̂ τ tj , max j ∈T 1 ̂ τ ◦ tj ] by definition, coupled with the fact that

Under Assumption 3, by Lemma 2, for any k ∈ T L 1 and j ∈ T 1 , we have

so we can obtain lim N t →∞ P ( max k ∈T L 1 ̂ τ ◦ tk < min j ∈T 1 ̂ τ ◦ tj ≤ max j ∈T 1 ̂ τ ◦ tj ) = 1 . Thus we have shown that lim N t →∞ P ( ˇ 1 t ∈ T 1 ) = 1 .

Step 2. We then show, for j / ∈ T 1 , and t = 1 , . . . , T ,

For any ε > 0 and j / ∈ T 1 , we have the following holds

By definition, for j / ∈ T 1 ,

Under Assumption 3, Lemma 2 and the conclusion in Eq (12) in Step 1 suggest that by letting N t →∞ on both sides, we have the above probability converges to zero. Based on above derivation, we have shown that lim N t →∞ P ( 1 ( j ∈ ̂ T t 1 ) > ε, j / ∈ T 1 ) = 0 .

Step 3. We are left to prove that for all j ∈ T 1 and t = 1 , . . . , T , the following holds for all ε > 0

: (∣ ∣

Following similar arguments, for a positive constant C , we have for k, j ∈ T 1 , the following statement holds:

By definition of the near-tie set, for j ∈ T 1 , we have

Again, under Assumption 3, Lemma 2 and the conclusion in Eq (13) we have derived in Step 1, by letting N t →∞ on both side, we have the above probability converges to 1. According to above discussion, we have shown that lim N t →∞ P (∣ ∣ ∣ 1 ( j ∈ ̂ T t 1 ) -1 ∣ ∣ ∣ > ε ) = 0 .

Combining the results obtained in the aforementioned three steps, we have for j = 1 , . . . , m ∗ t and for ε > 0 , the following holds:

Then it is straightforward to get the final results by taking t = T .

F Proof of Lemma 3

Proof. Note that

Thus this lemma follows directly from Corollary 1, Corollary 2 and Theorem 1.

G Proof of Lemma 4

Proof. Denote the oracle objective function as

where the second equality follows from regarding the tie set T 1 as a whole after merging the data, and then T 1 ranks 1 and other subgroups rank from 2 to m -|T 1 | +1 . Then, the original optimization problem can be written as

For notational simplicity, let E be the set of candidate allocation rules (i.e., all allocation rules satisfying the constraints).

Similarly, let ̂ f t ( e ) be the estimated objective function, that is, with m ∗ t = m -| ̂ T t -1 , 1 | +1 ,

We solve the following optimization problem:

We denote the set of maximizers to the above optimization problem as ̂ E ∗ . Lastly, let the δ -enlargement of E ∗ be E ∗ + B δ , where B δ is the Euclidean ball centered at the origin with radius δ : E ∗ + B δ = { e + u : e ∈ E ∗ , ∥ u ∥ = δ } . Finally, we notice that sup e ∈E | ̂ f t ( e ) -f ( e ) | = o p (1) due to Lemma 3, and Theorem 1. By the same proof of Lemma C.2 in [41], we have ̂ E ∗ ⊆ E ∗ + B δ , which closes the proof.

H Proof of Theorem 2

Proof. Adopting the same proof of Theorem 1 in [41], we obtain that

Then with the calibration procedure presented in Section 4.1, by replacing ̂ e ∗ tj with ˜ e tj , we have that

where the second term in the last equality has the order of O p (1 / √ N ) with variance calculation. Thus, we complete the proof.

I Proof of Theorem 3

Proof. Note that

and

Applying the result presented in Theorem 4,

where V T 1 ( e ∗ 1 ) = ∑ j ∈T 1 p 2 j V j ( e ∗ j ) /( ∑ j ∈T 1 p j ) 2 . By Theorem 1, we also have ̂ τ ̂ T 1 -̂ τ T 1 → p 0 . Thus by Slutsky's theorem,

Consistency of the variance estimator follows from Theorem 1 and Lemma 1, which is similar to the proof of Lemma 3.

J Additional algorithms

In this section, we provide another data-adaptive algorithm for choosing hyperparameters for Stage 1 and another bootstrap-based merging data bootstrap for later stages.

We present a double bootstrap method for choosing hyperparameters in Algorithm 4 as an alternative to Algorithm 3.

we have that

Algorithm 4 Hyperparameter selection in Stage 1

Step 1 (Input) :

1: Input H 1 , ̂ τ 1 j , ̂ V 1 j , and ( c L , c R ) .

2: Compute τ as in Eq (4).

Step b (Bootstrap)

∗ 1 j :

3: for b ← 1 to B do

4: Generate ̂ τ ∗ 1 from N ( τ ∗ 1 , ̂ Ω n /n 1 ) , where ̂ Ω n = diag ( ̂ V 11 , . . . , ̂ V 1 m ) ;

5: for r ← 1 to R do

6: Generate ̂ τ ∗∗ 1 from N ( ̂ τ ∗ 1 , ̂ Ω n /n 1 ) ;

7: Compute ˜ τ ∗∗ 1 , (1) as in Eq (6);

8: end for

9: Record B 1 , ( b ) ( c L , c R ) as in Eq (14).

10: end for

Step B (Output):

11: Compute L 1 ( c L , c R ) as in Eq (14). Choose the pair ( c 1 L , c 1 R ) that minimizes L 1 ( c L , c R ) .

Here, we define the loss function

where B 1 , ( b ) ( c L , c R ) is the b -th smallest statistics in B 11 ( c L , c R ) , . . . , B 1 B ( c L , c R ) .

We could know that P ( ˜ τ ∗ 1 , (1) ≤ τ 1 | ( Y is , D is , X is ) n 1 i =1 ) roughly follows Unif(0,1) when the sample size n 1 is large. Given a desirable pair of hyperparameters ( c L , c R ) , we would thus expect that B 1 , (1) ( c L , c R ) , . . . , B 1 , ( B ) ( c L , c R ) share a similar distribution with the ordered statistics of i.i.d. Unif(0, 1) random variables. The loss function defined in Eq (14) measures the average of squared differences between B 1 , ( b ) ( c L , c R ) and the expected value of the order statistics of the Unif(0, 1) random variables. Given the rational above, we would expect that the optimal pair of hyperparameters ( c 1 L , c 1 R ) at Stage 1 minimizes such a loss.

We present naïve bootstrap method for identifying and merging data for later stages in Algorithm 5 as an alternative to Algorithm 2.

Algorithm 5 Subgroup identification in Stage t ( t > 1 )

Step 1 (Input) :

1: Input {H s } t s =1 , ̂ τ tj , ̂ V tj , and ( c t L , c t R ) computed from Algorithm 3 or Algorithm 4. Step b (Bootstrap) :

2: for b ← 1 to B do

4: Compute ̂ τ ◦ tj as in Eq (8) with the bootstrap samples;

3: Generate totally ∑ t s =1 n s resamples randomly with replacement from {H s } t s =1 .

5: Identify the best subgroups ̂ T t 1 with the bootstrap samples as in Eq (3).

6: end for Step B

(Output):

7: Choose ̂ T t 1 with the highest frequency of occurrence and merge subgroups that belong to ̂ T t 1 .

K Additional details on synthetic real data study

K.1 Additional simulation results of our proposed algorithms

In this section, we mainly focus on the comparison of four variations of our proposed design strategy adopted with single or double bootstrap for selecting hyperparameters and naïve or separate bootstrap for data merging. We adopt the same setting mentioned in Section 6.

0.75

0.80

0.85

0.90

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Correct selection probability

Methods

Double and naive bootstrap

Double and separate bootstrap

Single and naive bootstrap

Single and separate bootstrap

√

First, from the comparison in Figure 2, as the adaptive design goes on, four variations of our proposed design strategy do not present a distinctly different performance, and they are all efficient with correct selection probability gradually rising over 0.9. Our design conducted with a single bootstrap for choosing hyperparameters and naïve bootstrap for identifying and merging tie sets perform a little better than other variations after 15 stages.

Second, from Table 3, we obtain that our proposed strategy conducted with a single bootstrap for selecting hyperparameters and a separate bootstrap for identifying and merging data has the smallest √ N -scaled bias. Overall, these four variations demonstrate similar efficiency in identifying and merging the best subgroups with almost the same estimated treatment effect and standard deviation.

0.5

0.6

0.7

0.8

0.9

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Normalized mutual information

Methods

Complete randomization design

Epsilon greedy algorithm

Single and separate bootstrap

Upper confidence bound 1 algorithm

(a)

0.80

0.85

0.90

0.95

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Normalized mutual information

Methods

Double and naive bootstrap

Double and separate bootstrap

Single and naive bootstrap

Single and separate bootstrap

(b)

We add normalized mutual information as an additional metric to evaluate the similarity of estimated tie set and true tie set of best-performing subgroups. With the results presented in Figure 3 (a) and (b), the normalized mutual information for our proposed design strategy and the three conventional

methods exhibits a similar trend in terms of correct selection probability. This observation further confirms that our proposed design strategy outperforms the conventional methods.

K.2 Extension: AIPW estimator

When contextual information (or additional covariate information) is considered in our design, the subgroup treatment effects and associated variances in Eq.8 can be replaced by

where ̂ µ d ( Z is ) = ̂ E [ Y is | D is = d, Z is ∈ S j ] , d ∈ { 0 , 1 } . We still generate synthetic data that mimic the original PBC dataset with the potential outcome from Y i ( d ) | X i ∈ S j = µ dj +0 . 5 Z i + ε dj , Z i ∼ N (0 , 1) , ε dj ∼ N (0 , σ 2 dj ) , j = 1 , . . . , 5 to investigate the performance of our proposed stratgy with AIPW estimator.

Through simulation studies in Figure 4, we found that our method (whether using IPW or AIPW) has the highest accuracy in identifying the best subgroups. The performance of the causal tree model is comparable to that of the contextual bandit algorithms where treatment effects are estimated with the AIPW estimator, thereby including contextual information. Regarding the causal tree model, we would like to note that the comparison may not be entirely fair for the causal tree model. This is because the causal tree method is a method used to identify subgroups after the data has already been collected, whereas our method focuses on designing data collection mechanisms to accurately identify the best subgroups. Therefore, it can be expected that our approach will have higher accuracy in identifying subgroups.

0.4

0.5

0.6

0.7

0.8

0.9

1

2

3

4

Stage

Correct selection probability

Methods

Causal tree model

Complete randomization design (AIPW)

Epsilon greedy algorithm (AIPW)

Single and separate bootstrap (AIPW)

Single and separate bootstrap (IPW)

Upper confidence bound 1 algorithm (AIPW)

(a)

0.5

0.6

0.7

0.8

0.9

1

2

3

4

Stage

Normalized mutual information

Methods

Causal tree model

Complete randomization design (AIPW)

Epsilon greedy algorithm (AIPW)

Single and separate bootstrap (AIPW)

Single and separate bootstrap (IPW)

Upper confidence bound 1 algorithm (AIPW)

(b)

K.3 Additional synthetic real data study

In this section, we provide an additional synthetic real study in another domain to showcase the performance of our proposed design. We consider a job dataset from the National Supported Work (NSW) program. This program, initiated in the 1970s, aims to provide disadvantaged workers

with work experience. We utilize the dataset from a field experiment referenced in [7] ( n = 455 ), which includes 185 workers in the treatment group and 260 workers in the control group. The dataset features a treatment indicator variable, an outcome variable representing participant earnings post-treatment in 1978, and eight baseline variables. These baseline variables include age, years of education, an indicator for high school graduation, indicators for Black and Hispanic ethnicity, marital status, and pre-treatment earnings for the years 1974 and 1975.

We aim to investigate whether the job training program is indeed beneficial for certain groups of workers in for subgroups defined by: (1) education years ≥ 11 and age ≥ 26 , (2) education years < 11 and age ≥ 26 , (3) education years ≥ 11 and age < 26 , (4) education years < 11 and age < 26 . It is a suitable dataset because by conducting two-sample t-tests between every two subgroups the pairwise similarity of the distribution of subgroups, Subgroups (1), (2) and (3) are regarded as tie sets of the best subgroups under the significance level of α = 0 . 05 . We provide the results in Figure 5, which can validate the performance of our proposed strategy.

0.4

0.5

0.6

0.7

0.8

0.9

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Correct selection probability

Methods

Complete randomization design

Epsilon greedy algorithm

Single and separate bootstrap

Upper confidence bound 1 algorithm

(a)

0.4

0.6

0.8

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Correct selection probability

Methods

Double and naive bootstrap

Double and separate bootstrap

Single and naive bootstrap

Single and separate bootstrap

(b)

0.5

0.6

0.7

0.8

0.9

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Normalized mutual information

Methods

Complete randomization design

Epsilon greedy algorithm

Single and separate bootstrap

Upper confidence bound 1 algorithm

(c)

0.5

0.6

0.7

0.8

0.9

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Stage

Normalized mutual information

Methods

Double and naive bootstrap

Double and separate bootstrap

Single and naive bootstrap

Single and separate bootstrap

(d)

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Please see Section 1.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitations of our work in Section 7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Please see Section 5.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please see Section 6 and Supplementary Materials.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Please see Section 6 and Supplementary Materials.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please see Section 6 and Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Please see Section 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please see Section 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research in the paper conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We discussed the potential positive social impacts in Section 1. The negative social impacts are not applicable.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have properly credited the original data source.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The manuscript does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a dynamic subgroup identification strategy within covariate-adjusted response-adaptive randomization (CARA) for clinical trials. This adaptive method dynamically identifies and adjusts treatment allocation to the best-performing subgroups based on ongoing trial data, and thus extend the traditional fixed-design trials to an online setup. Moreover, the paper also makes theoretical contributions by showing validity, asymptotic normality and efficiency of the estimator for the best subgroup treatment effect.\n\nSTRENGTHS:\nOriginality\nThe paper makes a significant original contribution by introducing a novel dynamic subgroup identification strategy within covariate-adjusted response-adaptive randomization (CARA) for clinical trials. This approach is innovative in its dynamic adjustment of treatment allocation based on real-time data, addressing inefficiencies associated with traditional fixed-design trials. This work also derives new theoretical results justifying the use of their proposed design. \n\nQuality\nThe research is of high quality, demonstrated through both theoretical and empirical validations. The authors provide rigorous theoretical results, including validity, asymptotic normality and semiparametric efficiency. Additionally, the empirical validation using a synthetic clinical trial on cirrhosis data is well-executed and convincingly demonstrates the effectiveness of the proposed design. \n\nClarity\nThe paper is clearly written and well-structured. Complex definitions and algorithms are clearly demonstrated. \n\nSignificance\nI consider this a significant work in the field of sequential experimental design and subgroup identification. It explores how to best identify and estimate subgroup effects in a dynamic regime, which is underexplored in the literature. What's more, it has broader impact in precision medicine and clinical trials.\n\nWEAKNESSES:\nOverall the paper is technically solid, though I notice some issues that can be addressed in the revision. First, the clarity of this paper can be improved as I identified some sentences/equations that are confusing. Please check the question section for details. Second and more importantly, the way that the paper formulates the problem needs some further justification, which is also related to Q2 in the question section.  The paper proposes a design to maximize the probability of correctly identifying the best subgroup, and estimate the best treatment effect; the theory and experiments all focus on this best subgroup identification. However, in personalized medicine which is the motivation for this work, I suppose practitioners care more about whether the treatment is beneficial to a certain group of patients or not, rather than the best treatment effect. In other words, even one can efficiently estimate the best treatment effect using CARA, it does not guarantee the best causal decision rule in personalized medicine. See https://pubsonline.informs.org/doi/10.1287/ijds.2021.0006 for a relevant discussion. Therefore, I suggest the authors discuss this gap between your work and practical considerations; this will help further justify the necessity of best subgroup identification.\n\nQUESTIONS:\n1. Page 3, bottom equation: In the equation at the bottom of this page, should $=$ be $\\coloneqq$, i.e., a definition?\n\n2. Page 3, Line 127-128: The goal of the design is to maximize the correct identification probability, based on which the paper proposes CARA and develops the corresponding theory. However, why maximizing the correct identification probability is the ultimate goal? In precision medicine, a more plausible goal is to maximize the overall welfare for the patients, i.e., finding the design that best improves the medical outcome for all patients. How do you compare your design objective to this welfare maximization objective? Can you modify your design for the second objective?\n\n3. Page 4, Eq (1): This equation is confusing. What is the objective function for $\\mathbf{e}$? I only saw the constraint set inside the parenthesis.\n\nLIMITATIONS:\nAs discussed in the paper, one main limitation is that the method cannot handle delayed outcomes, which can be restrictive in the real-world clinical trials. In my opinion, the authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-05"}

comments: SUMMARY:
This paper introduces a new dynamic treatment assignment for clinical trial to target treatment to the group most likely to benefit from it.

STRENGTHS:
The paper studies a critical problem of clinical trial design. It is clear and provides both theoretical justification and synthetic validation, demonstrating the utility of the proposed method.

WEAKNESSES:
While the paper tackles a critical problem, the problem is studied in depth in biostatistics. While I am unfamiliar with the literature on this topic, I am surprised there is none that could be considered for comparison. A more in-depth analysis of the literature on this topic should be presented in the Appendix to justify the choice of compared methods.

QUESTIONS:
Following the previous point, I would recommend a more in-depth review of the literature to convince the reader that this problem does not have alternative solution in the literature.

LIMITATIONS:
The paper discusses the critical limitation of the proposed approach's assumption of instantaneous access to the outcome following treatment. It would be interesting to discuss this assumption in the context of the existing literature. Is it a common assumption? If not, it would be important to justify further.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies an interesting problem in clinical trials to identify patient subgroups with the most beneficial responses to the treatment, which is essential for clinicians to create personalized treatment plans for their patients. However, most existing strategies for the design of clinical trials rely on domain knowledge or past experience of expert clinicians and stick to several pre-defined patient subgroups throughout the trial, which discards the valuable information collected from different trial stages. Some adaptive experimental strategies are developed to identify the best performing patient subgroup based on the trial outcomes; but their negligence of other subgroups where the treatment could be equally effective usually causes inefficient utilization of the experimental efforts. To tackle these challenges, the authors propose a dynamic subgroup identification approach to enable the construction of more effective experimental strategies for practical clinical trials. The authors claim three major contributions in this study: 1) their approach allows the dynamic identification for best patient subgroups based on experimental data collected during the trial process; 2) the authors develop new algorithms to effectively merge patient subgroups with similar (highest) responses to the treatment and provide theoretical results to support their analyses; and 3) the proposed method is validated using a synthetic dataset constructed from a clinical trial on cirrhosis.\n\nSTRENGTHS:\n## Originality\nThe method presented in this paper looks to be novel. The authors have provided substantial theoretical analysis to evident the originality and validity of their approach.\n\n## Quality\nMost parts of this paper are well-written and properly organized. The experimental results are reported with relevant statistics and are clearly evaluated and discussed with texts and visualizations.\n\n## Clarity\nThe general clarity of this paper is fair. Experimental results in this paper are sufficiently discussed.\n\n## Significance\nThe problem of patient subgroup identification is essential in clinical trials for the selection of target patient cohorts. For better utilization of the resources in a trial, dynamic identification of best performing patient subgroups and adaptive treatment assignments are imperative. The approach proposed in this paper enables effective patient subgroup identification using patient characteristics and treatment responses collected during different trial stages and allows adaptive optimization of the treatment assignment strategy.\n\nWEAKNESSES:\n## Related Works\n\nThere are two short paragraphs in the Introduction discussing the literature related to this study. However, to distinguish the experimental design approach in this paper from other research and highlight the contributions of this study, a more comprehensive comparison with related works is needed. It seems that there are many other studies, with similar focuses on subgroup identification in clinical trials, not sufficiently discussed in this paper. For instance:\n\n- Adaptive identification and assessment of patient subgroups [1]\n- Identification of patient subgroups with similar clinical characteristics (covariates) [2]\n- Clustering of patient subgroups with different levels of benefits in clinical trials [3]\n\nThe authors are encouraged to provide discussions and comparisons with related works on patient subgroup identification to better demonstrate the novelty and advantage of this study.\n\nFurther, although the authors emphasize that their method focuses on the setting of covariate-adjusted response-adaptive (CARA) experiments which is an under-explored area, the reviewer cannot find discussions on the contribution or benefit of including patient covariates ($X$) in the design of clinical trials. The covariates-based patient subgroup identification has already been studied in the machine learning literature, e.g., [2,3]. \n\nReferences: \n\n[1] Guo, Wentian, Yuan Ji, and Daniel VT Catenacci. \"A subgroup cluster‐based Bayesian adaptive design for precision medicine.\" Biometrics 73.2 (2017): 367-377.\n\n[2] Lee, Beom S., et al. \"A clustering method to identify who benefits most from the treatment group in clinical trials.\" Health Psychology and Behavioral Medicine: an Open Access Journal 2.1 (2014): 723-734.\n\n[3] Xu, Jie, et al. \"Machine learning enabled subgroup analysis with real-world data to inform clinical trial eligibility criteria design.\" Scientific Reports 13.1 (2023): 613.\n\n\n\n\n\n## Clarity\n\n### The role of patient covariates\n\nAs mentioned earlier, the role of patient covariates is not sufficiently discussed in the proposed experimental design approach. The only place related to the covariates seems to be line 7 of Algorithm 2 where previous trial results are randomly resampled. It is unclear how the experimental design is adjusted based on patient covariates. Since this paper focuses on the CARA setting and considers both treatment responses and patient covariates in subgroup identification, it is important for the authors to elaborate the contribution of patient covariates in the proposed algorithms and how is this study different from conventional research on response-adaptive randomization (RAR) settings.\n\n### Notations\n\nThere are many symbols introduced in the analysis of this paper without any explanation. \nFor instance, the variable $\\hat{p}\\_{tl}$ in Eq. 1 has never been explained. \nThe symbol $\\mathcal{B}_{t,b}$ in Eq. 6 comes from nowhere. \nIn the meantime, there are so many similar symbols used in the discussion, and it is very difficult for the reviewer to tell their difference. The authors are encouraged to ensure the consistency in their notation. If possible, a notation table could help a lot to improve the clarity of this paper.\n\n### Insufficient explanations\n\nSome results or derivations in this paper are introduced without proper explanation. For instance, although the authors provide citations to the large deviation theory in LN 129, the equivalence between the correct identification probability and the optimization objective remains obscure. Similarly, it is unclear why the optimal treatment allocation can be derived from Eq. 1. Additionally, the hard-coded exponent 0.05 for $\\Delta$ in LN 162 appears without any explanation.\n\n## Correctness\n\nThe correctness of some proofs seems to be questionable. For instance, in the proof of Theorem 1, it is unclear why $\\tau_{t,l} = \\tau_{l}$. The inequality in LN 450 cannot be directly established according to the analysis in LN 451 – 453. Similarly, the final inequality in LN 457 is non-trivial and the authors should provide analysis to prove its correctness.\n\n## Evaluation\n\n### Dataset\n\nThe proposed experimental design approach is only evaluated on a synthetic dataset. To validate the general applicability and performance of this method, benchmark results on more datasets are necessary.\n\n### Baselines\n\nThe authors have discussed many relevant studies in the Introduction. However, only three baselines are considered in the experiment. For a more comprehensive comparison, the authors are encouraged to include more baselines from related works to demonstrate the advantage of their method. Particularly, the reviewer is interested in the performance of contextual bandit algorithms and causal tree models.\n\n### Metrics\n\nNote that the experimental results are obtained on a synthetic dataset where the ground truth subgroup labels are available. The authors are encouraged to include additional metrics on clustering accuracy, e.g., purity score, normalized mutual information, etc., in the benchmark.\n\nQUESTIONS:\nIn summary, I have the following concerns about this paper.\n1. The related works are not sufficiently discussed. The difference between this paper and relevant studies should be clearly illustrated.\n2. It is unclear how the proposed method is different from conventional methods considering the RAR settings. There is no discussion on the contribution or benefit of including patient covariates ($X$) in the design of clinical trials. \n3. Many symbols are introduced in the analysis of this paper without proper explanation.\n4. The derivation of some key results is not sufficiently explained. For instance, the optimization objective below LN 129 and the optimal treatment allocation in Eq. 1.\n5. The correctness of some proofs seems to be questionable. Specifically, the proof for Theorem 1. Please see the weakness section above for details.\n6. The proposed method needs to be tested on more datasets, and the authors are encouraged to include more performance metrics in the benchmark.\n7. There should be more baselines, e.g., causal tree and contextual bandit, in the benchmark to highlight the advantage of the proposed method.\n8. How does the number of stages $T$ in a trial affect the convergence of the proposed method? What if there is only one stage? What if $T$ is small (Below 4)?\n\nLIMITATIONS:\nThe limitations of this study are briefly discussed at the end of this paper. The authors identify the potential mismatch between their assumption on immediate observation of treatment responses and the delays commonly observed in real-world clinical trials. However, the potential negative societal impact of the clinical trial design approach in this paper is not sufficiently discussed. In addition, the authors are encouraged to discuss whether their approach can be generalized to deal with scenarios with multiple treatment options.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a dynamic subgroup identification strategy within the framework of covariate-adjusted response-adaptive randomization, addressing the need for more nuanced subgroup analysis in clinical trials. This strategy aims to optimize treatment allocation dynamically and identify subgroups that demonstrate significant treatment effects, which is crucial in practice.\n\nSTRENGTHS:\n1.\tThe method is highly relevant to modern clinical trial designs, where there is a pressing need to identify patient subgroups with differential treatment responses efficiently.\n2.\tThe paper is strong in theoretical development, providing rigorous proofs and formulations that demonstrate the statistical validity and efficiency of the estimator for the best subgroup treatment effect.\n3.\tThe approach looks novel, from my perspective.\n\nWEAKNESSES:\n1.\tI understand the paper is a very technical paper, and the authors have spent a lot of efforts in making the paper easy to follow. My minor comment is that maybe providing more intuition on the reasons why the complicated design is needed will be helpful. For example, why we need resampling and bootstrap? What kind of technical challenge bootstrap can help to overcome?\n2.\tThe objective function in line 128 seems a little mismatched with the objective of best subgroup identification. For example, if there are two arms in $\\mathcal{T}_1$  with variance zero and very large respectively, it seems that optimizing the objective in line 128 will lead to the solution that we forget about the super large one and devote most efforts to the one with variance 0.  Thus, I am thinking about whether there is some analysis on the probability that we successfully identify $\\mathcal{T}_1$? More specifically, can we always guarantee we identify everything in $\\mathcal{T}_1$ and we never include something out of $\\mathcal{T}_1$?\n\nQUESTIONS:\nSee previous comments.\n\nLIMITATIONS:\nSee previous comments.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}