Dynamic Subgroup Identification in Covariate-adjusted Response-adaptive Randomization Experiments

Authors: yanping  li
jingshen  wang
waverly  wei
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 19:14:35

Abstract: Identifying subgroups with differential responses to treatment is pivotal in randomized clinical trials, as tailoring treatments to specific subgroups can advance personalized medicine. Upon trial completion, identifying best-performing subgroups–those with the most beneficial treatment effects–is crucial for optimizing resource allocation or mitigating adverse treatment effects. However, traditional clinical trials are not customized for the goal of identifying best-performing subgroups because they typically pre-define subgroups at the beginning of the trial and adhere to a fixed subgroup treatment allocation rule, leading to inefficient use of experimental efforts. While some adaptive experimental strategies exist for the identification of the single best subgroup, they commonly do not enable the identification of the best set of subgroups.  To address these challenges, we propose a dynamic subgroup identification covariate-adjusted response-adaptive randomization (CARA) design strategy with the following key features: (i) Our approach is an adaptive experimental strategy that allows the dynamic identification of the best subgroups and the revision of treatment allocation towards the goal of correctly identifying the best subgroups based on collected experimental data. (ii) Our design handles ties between subgroups effectively, merging those with similar treatment effects to maximize experimental efficiency. In the theoretical investigations, we demonstrate that our design has a higher probability of correctly identifying the best set of subgroups compared to conventional designs. Additionally, we prove the statistical validity of our estimator for the best subgroup treatment effect, demonstrating its asymptotic normality and semiparametric efficiency. Finally, we validate our design using synthetic data from a clinical trial on cirrhosis.

Main paper: 
 introduction: 
 Most clinical trial designs adopt "one-size-fits-all" rules for treatment assignment and evaluation based on models that ignore patient heterogeneity. This approach is disconnected from medical practice in recent years, where physicians use each patient's diagnosis and prognostic variables to make personalized, precision medicine treatment decisions. As such, identifying patient subgroups with differential responses to a treatment plays a pivotal role in designing randomized clinical trials [28, 18, 2, 39] . Adaptive clinical trials-that allow randomization probabilities to be adaptively optimized during the trial based on sequentially accrued data-have received much attention due to their potential advantages in promoting precision health. Nevertheless, these trials often involve pre-specifying the patient subgroups to be analyzed [3, 38] . This approach not only leads to inefficient use of experimental efforts but may also reduce the statistical power to detect non-pre-specified subgroups that exhibit high effect sizes. Consequently, there is a pressing need for novel and statistical clinical trial designs for dynamic subgroup identification to address these issues.In this paper, we propose a novel statistical design that dynamically performs subgroup identification in CARA experiments. Our contributions are summarized as follows:1. From the design perspective, there are three highlights of our design: (i) Our design facilitates dynamic identification and sequential refinement of the best subgroups within the framework of covariate-adjusted response-adaptive (CARA) experiments (Section 4). This adaptive setting enhances the ability to identify and adjust to the best-performing subgroups over time.(ii) Unlike traditional designs that focus on identifying a single best-performing subgroup, our design is tailored to identify the best set of subgroups with competitive performance (Section 3). This broader objective makes our approach suitable for more general application settings. (iii) Our design demonstrates a higher probability of correctly identifying the best subgroups compared to conventional designs. This efficiency ensures that experimental efforts are utilized more effectively.2. From a theoretical perspective, our proposed design strategy accommodates tied treatment effects among candidate subgroups, whereas many existing methods demand that these effects be distinctly separated. Additionally, our algorithm involves resampling in the presence of dependent data structures caused by adaptive treatment allocation, presenting technical challenges in proving statistical validity. We overcome these challenges and show that the best subgroup identified by our design asymptotically converges to the true set of best subgroups (Theorem 1). We also demonstrate that our proposed design strategy converges to the oracle design, which is the optimal design under the setting that the underlying datagenerating distribution is known (Theorem 2). Furthermore, we establish valid statistical inference for the treatment effect of the identified best subgroup and demonstrate that our constructed estimator is semiparametrically efficient (Theorem 3).In comparison to the existing literature, our design strategy is closely related to CARA designs. Originating from response-adaptive randomization (RAR) designs [5, 33, 46] . Heuristically, CARA incorporates covariate information along with treatment assignment probabilities based on observed outcomes [14, 32, 30, 13] . Building upon RAR designs, CARA designs utilize both outcome and covariate information to optimize for design objectives. [15] introduces a family of CARA designs that balance efficiency and ethics objectives. Further generalizations to incorporate semiparametric estimates have been explored by [49] . Related developments in CARA designs include [22, 40, 48, 47] . However, conventional CARA designs often consider subgroups to be pre-specified, which is different from our goal of data-adaptively identifying the best subgroups.Our proposed method also connects with the literature on subgroup identification. In post-hoc analyses using previously collected data, the first line of work uses data from prior randomized controlled trials. [20] employs clustering techniques based on randomized controlled trial data.A comprehensive review can be found in [23] . The second line of work uses data from prior observational studies. [43] identifies subgroups from existing observational data using machine learning. [36] introduces a causal inference tree approach for subgroup identification, which requires specifying the conditional distribution of the outcome given covariates. Similarly, [45] develops causal inference tree types of algorithm that include double robust estimators for constructing subgroupspecific splitting criteria. Other tree-based approaches for subgroup identification include [17, 24, 16] .Besides the tree-based approaches, [10] develops a subgroup identification method within the value function framework. While post-hoc subgroup analyses do not require new data collection, they often rely on untestable causal assumptions, limiting the credibility of causal conclusions. For instance, the unconfoundedness assumption necessary for causal inference in observational studies assumes random treatment assignment based on observed confounders, but unmeasured confounders can compromise these conclusions. Conversely, in randomized experiments, valid causal conclusions do not depend on such assumptions.In the adaptive experiment literature, relatively few methods have been developed for identifying the best subgroups. While [11] is in the adaptive experiment setting and proposes a Bayesian adaptive design that sequentially revises treatment allocation to identify the effective subgroup-treatment pairs, their approach is carried out under the Bayesian framework, relying on the specification of prior distributions and does not provide theoretical justification for the identified subgroups. In contrast, our design is aligned with the frequentist framework and is model-free, avoiding any parametric modeling assumption on the joint distribution of potential outcomes and covariates, and providing theoretical investigations from three aspects.As our method aims to identify the best subgroups, it shares some similarities with the multi-armed bandit (MAB) literature. Notable MAB algorithms, such as the Thompson sampling method [35] , the ϵ-greedy algorithm, and the upper confidence bound algorithm [37] , focus on identifying the best arm. Similar to the contextual bandit literature [21, 6, 1] , our design also incorporates covariate information. However, our design objective diverges significantly from those in traditional MAB approaches as we seek to identify the best subgroups rather than the best arm. Moreover, recognizing that randomized experiments can be time-consuming, costly, and may result in adverse outcomes for patients if treatments are ineffective, our approach seeks to efficiently allocate experimental resources within a constrained budget to identify the most beneficial subgroups. Specifically, we focus on scenarios in which the cost per experimental unit (e.g., per patient) is significant. For instance, in clinical settings, randomized experiments are often expensive due to the substantial costs of treatment medications. As a result, the primary resource constraint in our framework is the limited number of treatments that can be administered, underscoring the need for a resource-efficient experimental design. 
formulation of cara: 
 In this section, we shall introduce the formulation of our covariate-adjusted response-adaptive randomization (CARA) experiment framework.We enroll participants sequentially across T stages, where T < ∞. Denote the total number of enrolled participants as N = T t=1 n t , where n t is the number of participants in Stage t, for t = 1, . . . , T . The cumulative sample size up to Stage t is denoted as N t = t s=1 n s . In Stage t, we denote the treatment assignment status of participant i as D it ∈ {0, 1}, i = 1, . . . , n t , where D it = 1 denotes the treatment arm, and D it = 0 denotes the control arm. The observed outcome is denoted as Y it ∈ R. We follow the Neyman-Rubin causal model [27, 34] to define Y it (d) as the potential outcome we would have observed if participant i receives treatment d at Stage t, for d ∈ {0, 1}. The observed outcome can then be represented asY it = D it Y it (1) + (1 -D it )Y it (0), i = 1, . . . , n t , t = 1, . . . , T.We assume that the outcomes are observed without delay, and their underlying distributions do not shift over time [14] .In CARA experiments, covariate information is also available to practitioners. We denote the covariate information for participant i as X it ∈ R and assume the covariate space X can be partitioned into m regions, denoted as {S j } m j=1 . In clinical settings, each partition of the sample space is commonly referred to as a subgroup [3, 19, 44] . We denote the number of subjects enrolled in subgroup j at Stage t as n tj = nt i=1 1 (Xit∈Sj ) and the cumulative sample size for group j up to Stage t is N tj = t s=1 n sj . Denote the total number of subjects enrolled in subgroup j as N j = N T j . As we are interested in assessing the effectiveness of the treatment in each subgroup, we define the subgroup average treatment effect asτ j = E[Y it (1) -Y it (0)|X it ∈ S j ], j = 1, . . . , m.Due to the adaptive nature of CARA experiments, practitioners can sequentially revise treatment allocation based on outcome and covariate information accumulated during the experiment. Formally, we define the treatment assignment probability for participants in subgroup j ase tj = P(D it = 1|X it ∈ S j , H t-1 ), t = 1, . . . , T, j = 1, . . . , m, where H t-1 = {(Y is , D is , X is ) ns i=1 } t-1s=1 denotes the historical information up to Stage t -1. In CARA experiments, we aim to dynamically revise e tj to reach desired design goals, which shall be introduced in the following section.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: We have discussed the limitations of our work in Section 7.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
design objective for best subgroups identification: 
 In real-world applications, suppose we start with m subgroups, and practitioners may only aim to find subgroups with the largest treatment effects. It is possible that the best-performing subgroup is not unique. For example, the FORTE trial is a clinical trial aiming to investigate the treatment effect of carfilzomib-based induction-intensification-consolidation regimens on a patient's progressionfree survival rate [25] . Instead of reporting the single best subgroup with the largest survival rate improvement, the trial reports that both the risk myeloma patient subgroup and high-risk patient subgroup show similar survival rate improvement.Without loss of generality, suppose the population subgroup treatment effects follow the order τ 1 ≥ τ 2 ≥ . . . ≥ τ k > . . . > τ m . In this case, there are k subgroups exhibiting the largest treatment effects. Thus, it is natural to identify all of the k best subgroups instead of the single best subgroup.To describe the set of best-performing subgroups, we introduce the concept of a tie set. We denote the tie set of τ 1 asT 1 = {k : |τ 1 -τ k | = o(N -1/2), k = 1, . . . , m} which contains the indices of the tied subgroups. This tie set is also known as the "near tie set" as it captures the subgroups of which the treatment effects lie in the √ N -local neighborhood of τ 1 . We then denote the subgroups that belong to the best set as S T1 . Note that when T 1 = {1}, S T1 is equivalent to S 1 . The population treatment effect under the best subgroups is defined asτ T1 = E[Y it (1) -Y it (0)|X it ∈ ∪ j∈T1 S j ].We further assume that τ T1 > τ j , for j / ∈ T 1 .Our design objective is to correctly identify all the best subgroups. Mathematically, we aim to maximize the correct identification probability:max e P τ T1 ≥ max j / ∈T1τ j , where T 1 = {1, . . . , k}.Leveraging the large deviation theory [8, 12] , we can formulate our design objective as max e min j / ∈T1 G(S T1 , S j ; e 1 , e j ) := (τ j -τ T1 ) 2 2 V T1 (e 1 ) + V j (e j ), ← Maximize correct selection probability s.t. δ ≤ e j ≤ 1 -δ, ← Feasibility constraints where T 1 = {1, . . . , k}, 1 ≤ k < m, δ ∈ (0, 1/2), and V T1 denotes the variance of the best subgroups. Detailed derivations of the equivalence between the correct identification probability and the optimization objective see Appendix (Section C).However, in practice, solving this optimization problem is challenging. On the one hand, as we do not have knowledge regarding the membership of subgroups that have the largest treatment effects, we do not have any information regarding S T1 . On the other hand, because experimenters have no prior information about the joint distribution of the subgroup treatment effects, τ j 's and V j 's are also unknown. To address these two practical challenges, we propose a dynamic subgroup identification algorithm that can adaptively identify and merge the set of best subgroups. Additionally, the dynamic subgroup identification method operates seamlessly under a CARA experimental strategy, which allows experimenters to sequentially learn the unknown parameters and adjust the subgroup treatment allocation to attain our design objective.4 Proposed design: Dynamic subgroup identification with CARAIn this section, we shall illustrate our proposed dynamic subgroup identification strategy with CARA design in Algorithm 1: the design strategy in Section 4.1, encompassing two sub-algorithms (Algorithm 2 and 3), followed by the statistical inference procedure in Section 4.2. We defer several variations of our algorithm to Appendix (Section J). To clarify our design strategy, we also provide a notation table in the Appendix (Section A, Table 2 ).Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer • The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
proposed design strategy: 
 In Stage 1 (line 1-4), we obtain initial estimates of the group-level treatment effect τ 1j and the associated variances V 1j . Then, in Stage t = 2, . . . , T -1 (Algorithm 1 line 6-11), we perform three tasks:(1) adaptively update treatment allocation and treatment effect estimates, (2) dynamically identify best subgroups, and (3) select hyperparameters that help with dynamic subgroup identification.For the first task, we obtain the optimal treatment allocation e * t = ( e * t1 , . . . , e * tm * t ) by solving the following optimization problem based on sequentially collected data:max e min 2≤j≤m * t ( τ t-1,(j) -τ t-1,(1) ) 2 2 V t-1,(1) (e 1 ) + V t-1,(j) (e j ), s.t.m * t l=1 p tl e l ≤ c 1 , c 2 ≤ e l ≤ 1 -c 2 , l = 1, . . . , m * t ,where c 1 ∈ (0, 1) and c 2 ∈ (0, 1/2),p tl = t s=1 ns i=1 1 (X is ∈S (l) ) t s=1 nsis the estimated subgroup proportion. The total number of subgroups after merging the identified tie set in Stage t -1 is denoted asm * t = m -| T t-1,1 | + 1and the subscript (j) indexes the subgroup with the j-th largest estimated treatment effect. The procedure of finding T t-1,1 shall be illustrated in Algorithm 2. In the set of constraints, the first one is the resource constraint, and the second one is the feasibility constraint. Because of the nonlinear objective function of the optimization problem above, we instead work with its equivalent epigraph representation:EQUATIONTo calibrate for the complete randomized treatment allocation in Stage 1, we require an additional calibration step in Stage t: e t,(j) = e * t,(j) N t,(j) -N t-1,(j) (1) n t,(j) , j = 1, . . . , m * t ,where n t,(j) = nt i=1 1 (Xit∈S (j) ) , N t-1,(j) (1) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) D is , and N t,(j) = t s=1 n s,(j) . We then allocate treatments with calibrated probability e t,(j) and update the subgroup treatment effects as in Eq (8) .Dynamic identification of the best subgroups (Algorithm 2). The dynamic subgroup identification algorithm for identifying T t,1 at Stage t involves a resampling step that generates bootstrap samples τ • t from a Gaussian distribution centering around τ t at Stage 1 and a resampling step that generates bootstrap samples with accrued data {H s } t s=1 at later stages. In line 9, we identify the best subgroups at Stage t asEQUATIONwherew • k,(1) = 1{-c t L • N -δ t • V δ t,(1) ≤ ( τ • tk -τ • t,(1) ) ≤ c t R • N -δ t • V δ t,(1) },where the distance between the upper and lower bounds of the interval for w • k,(1) is of order n δ with δ = 0.25 to guarantee the statistical validity of our proposed procedure and balance the trade-off between bias and variance. Note that the dynamic subgroup identification procedure relies on a pair of hyperparameters (c t L , c t R ), which are selected data-adaptively. In what follows, we shall illustrate the algorithm for selecting these hyperparameters.Hyperparameter selection (Algorithm 3). In line 7 of Algorithm 3, we adopt a bootstrap method and propose several alternative bootstrap methods in Algorithm 5 (line 3) in Appendix (Section J). Algorithm EQUATIONwhere ∆ = min{0.99,m * t j=1 Vtj Nt m * t j=1 ( τtj -τ t)2 × N γ t } and γ ∈ (0, 0.2). We impose a lower bound on ∆ to ensure that ∆ does not equal to 1. We choose γ = 0.05 in our simulation studies, and our procedure is shown to be not sensitive to the choice of γ < 1. In line 7, we computeEQUATIONwhere τ • tj is computed with the bootstrap samples as in Eq (8) . In line 10, we computeEQUATIONwhereEQUATION) ) ≤ c t R •N -δ t • V δ t,(1)} and the subscript (1) indexes the subgroup with the largest estimated treatment effect. We also propose a double bootstrap-based alternative hyperparameter selection procedure in Algorithm 4 in Appendix (Section J). Finding the optimal hyperparameters involves minimizing a loss function L t (c L , c R ) at each Stage t, which is defined asEQUATIONwhere for l = 0, 1,L tl (c L , c R ) = 1 B B b=1 (1(B tb (c L , c R ) = l) - B b=1 1(B tb (cL,cR)=l) B) 2 . Given a desirable pair of hyperparameters (c L , c R ) and l value, the indicator function1(B tb (c L , c R ) = l) is binary, which roughly follows a Bernoulli distribution with probability B b=1 1(B tb (cL,cR)=l) B. Intuitively, the loss function defined in Eq (7) measures the average of squared differences between1(B tb (c L , c R ) = l) and the expected value of Bernoulli( B b=1 1(B tb (cL,cR)=l) B) random variables. We would expect that the optimal pair of hyperparameters (c t L , c t R ) at Stage t minimizes such a loss.Stage 1 (Initialization): 1: Enroll n 1 participants, and assign treatments in group j with e 1j = 1 2 ; 2: Compute τ 1j and V 1j ; 3: Choosing hyperparameters c 1 L , c 1 R using single bootstrap method (see Algorithm 3); 4: Identify tie set and merge tied subgroups with the best subgroup (see Algorithm 2).Stage t (Adaptive treatment allocation revision):5: for t → 2 to T do 6:With τ t-1,(j) and V t-1,(j) estimated using Eq 8, solve the optimization problem in Eq (1) to find e * t,(j) ; 7:Enroll n t participants and assign treatment with calibrated probability e * t,(j) as in Eq (2); 8:Update τ t,(j) and V t,(j) as in Eq (8);9:Choosing hyperparameters (c t L , c t R ) using single bootstrap (see Algorithm 3); 10:Identify tie set and merge tied subgroups with the best subgroup (see Algorithm 2).11:Calculate the merged subgroup ATE estimator τ t, Tt1 as in Eq (9) , and its variance estimator V t, Tt1 as in Eq (10) . 12: end for Stage T (Inference): 13: Identify the best tie set T 1 , and construct two-sided confidence intervals for τ T1 as in Eq (11) .Our CARA design also enables making valid statistical inference on the estimated best subgroup treatment effect. We highlight two parts of the statistical inference procedure: (1) estimating unknown parameters based on accrued experimental data while dynamically identifying best subgroups, and (2) constructing valid confidence intervals to confirm the estimated best subgroup treatment effect.First, based on accumulated experimental data, the subgroup treatment effects and associated variances can be updated asEQUATIONV t-1,(j) (e j ) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) D is Y is -Ȳt-1,(j) (1) 2 N t-1,(j) (1) e j • N t-1,(j) N t-1 -1 + t-1 s=1 ns i=1 1 (Xis∈S (j) ) (1 -D is ) Y is -Ȳt-1,(j) (0) 2 N t-1,(j) (0) (1 -e j ) • N t-1,(j) N t-1 -1, (1) . Additionally, denote the subgroup proportions as p 1 , . . . , p m . Following dynamic subgroup identification at each stage, we merge all subgroups in the T t1 and estimate the merged best subgroup treatment effect aswhere N t-1,(j) (1) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) D is , N t-1,(j) (0) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) (1 - D is ), N t-1,(j) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) , Ȳt-1,(j) (0) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) (1 - D is )Y is /N t-1,(j) (0), and Ȳt-1,(j) (1) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) D is Y is /N t-1,(j)EQUATIONand estimate the variance of the merged best subgroup treatment effect estimator asEQUATION)In Stage T (line 13), we let T 1 := T T 1 , τ T1 := τ T, T1 and V T1 := V T, T1 . Lastly, to confirm the estimated best subgroup treatment effect, we construct a two-sided level-α confidence interval asEQUATIONAlgorithm 2 Dynamic subgroup identification in Stage tStep 1 (Input):1: Input {H s } t s=1 , τ tj , V tj , and (c t L , c t R ) computed from Algorithm 3. Step b (Bootstrap): 2: for b ← 1 to B do 3: if t = 1 then 4: Generate τ • 1 from N τ 1 , Ω n /n 1 , where Ω n = diag V 11 , . . . , V 1m ; 5: else if t > 1 then 6:Generate n s resamples randomly with replacement sequentially from each H s ;7:Compute τ • tj as in Eq (8) with the bootstrap samples;8:end if 9:Identify the best subgroups T t1 with the bootstrap samples as in Eq (3). 10: end forStep B (Output): 11: Choose T t1 with the highest frequency of occurrence and merge subgroups that belong to T t1 . 
theoretical investigation: 
 Assumption 1 (Regularity conditions). (Y it (0), Y it (1), X it ) are independently identically distributed for i = 1, . . . , n t , t = 1, . . . , T . In addition,E[|Y it (d)|] 4 < ∞, d ∈ {0, 1}. Lastly, there exists some δ > 0, such that V[Y it (d)|X it ∈ S j ] ≥ δ for d ∈ {0, 1}, j = 1, . . . , m. Assumption 2 (Positivity). The subgroup proportions δ ≤ p 1 , . . . , p m ≤ 1 -δ, δ ∈ (0, 1/2).Assumption 1 says that the potential outcomes have bounded moments and have variability in each subgroup. Assumption 2 says that the subgroup proportions are non-zero in the population. Theorem 1 (Dynamic best subgroup identification consistency). Under Assumptions 1 and 2, for j = 1, . . . , m and for ε > 0, we have limN →∞ P |1(j ∈ T 1 ) -1(j ∈ T 1 )| > ε = 0.Theorem 1 suggests that our dynamic subgroup identification algorithm correctly identifies the best set of subgroups as the sample size tends to infinity. Theorem 2 (Design strategy consistency). Under Assumptions 1 and 2, for δ > 0, as n t → ∞, t = 1, . . . , T , for the actual treatment allocation, we haveP(|| e t -e * || ≤ δ) → 1,where e * = (e * 1 , . . . , e * m ) is the optimal treatment allocation. Theorem 2 says that the actual treatment allocation under our proposed design strategy converges to the optimal treatment allocation asymptotically. Theorem 3 (Asymptotic normality). Under Assumptions 1 and 2, asN → ∞, √ N ( τ T1 -τ T1 ) → N 0, V T1 (e * 1 ), andV T1 -V T1 (e * 1 ) = O p ( 1 √ N ).Theorem 3 says that the estimated treatment effect of the identified best subgroups converge to a Gaussian distribution asymptotically, and our variance estimator consistently estimates the asymptotic variance. Theorem 3 also verifies the validity of our constructed confidence interval. 
synthetic real data study: 
 In this section, we investigate the performance of our proposed design strategy for identifying the tie set of best-performing subgroups in a synthetic case study using clinical trial data.We design our synthetic case study using the dataset from the Mayo Clinic's trial on primary biliary cirrhosis (PBC), containing clinical biomarkers, treatments, and patient outcomes. PBC is a progressive autoimmune liver disease marked by inflammation and damage to the intrahepatic bile ducts. The Mayo Clinic conducted an extensive trial from 1974 to 1984 to assess the effectiveness of D-penicillamine in treating PBC. This dataset includes 424 patients, encompassing both those who were actively enrolled in the trial and additional cases who consented to provide basic measurements [26] .In this case study, we work with a subset (n = 312) of patients who participate in the randomized controlled trial. These patients are randomly assigned to one of the two arms: the treatment arm (n = 158), who receive D-penicillamine D = 1, and the control arm (n = 154), who receive placebos D = 0. The outcome of interest is the square root of the survival time, defined as the number of days from registration to the earlier death, transplantation, or the time of study analysis. Compute τ * t,(1) as in Eq (6) , and B tb (c L , c R ) as in Eq (7) . 11: end forStep B (Output): 12: Compute L t (c L , c R ) as in Eq (7) . Choose the pair(c t L , c t R ) that minimizes L t (c L , c R ).We generate synthetic data that mimic the original PBC dataset. Denote the subgroup membership for each participant i as S = (1 (Xi∈S1) , . . . , 1 (Xi∈S5) ) ⊺ . We generate the potential outcome from We compare our proposed design strategy with the complete randomization design and two multi-armed bandit (MAB) algorithms. (1) The complete randomization design refers to a design that fixes e tj = 1 2 across all experimental stages, t = 1, . . . , T, j = 1, . . . , m. (2) To customize the MAB algorithms to our setting, for each subgroup, we consider two candidate arms: treatment and control. We set the rewards as the negative asymptotic variance of the treatment effect estimator, i.e., -V j (e j ). We consider two MAB algorithms: (a) The ϵ-greedy algorithm, which aims to balance the exploration and the exploitation efforts [37] . Here, we set ϵ = 0.1. (b) The upper confidence bound 1 algorithm balances exploration and exploitation using confidence intervals and chooses the arm that maximizes the upper confidence bound on the estimated reward [4] . When customizing MAB algorithms to our setting, we omit the step of identifying and merging tie sets using conventional methods, such as K-means or agglomerative clustering, due to several challenges: the requirement to predefine the number of clusters, the limited applicability of clustering for a small number of subgroups, and the inconsistency of these methods in effectively merging the best subgroups. To evaluate the performance of different design strategies, we assess the effectiveness of each adaptive experiment strategy from two aspects. First, we compare the correct selection probability of identifying the best subgroups. The correction selection probability can be written as P( τ T1 ≥ max j / ∈T1 τ j ). Second, we compare the 95% confidence interval, √ N -scaled bias, and standard deviation of the estimated best subgroup treatment effect. In our resampling procedure, we set B = 2, 000. The synthetic case study results are summarized in Figure 1 and Table 1 .Y i (d)|X i ∈ S j ∼ N (µ dj , σ 2 dj ), j = 1, . . . ,First, from Figure 1 , our proposed design strategy shows a higher correct selection probability than the complete randomization design and the MAB algorithms. Additionally, the correct selection probability under our proposed design strategy increases with the number of experimental stages. Specifically, our proposed design has a correct selection probability tending to 1 after 15 experimental stages. We also adopt the normalized mutual information as an additional metric to compare our proposed design and three competing methods in the Appendix (Section K.1), which further confirms that our proposed design strategy outperforms the conventional methods.Second, from Table 1 , we observe that our proposed design strategy has a smaller standard deviation and smallerN -scaled bias, implying that our method is more efficient and less biased. In sum, our proposed adaptive design demonstrates efficient use of experimental data to correctly identify the best-performing subgroups of the three competing methods. We defer additional simulation studies to the Appendix (Section K.1) and an additional synthetic real data study to the Appendix (Section K.3). We also extend our proposed dynamic subgroup identification with CARA to the augmented inverse propensity score weighting (AIPW) estimator. Then we compare our proposed design with AIPW estimator with the three contextual MAB algorithms in the Appendix (Section K.2). 
discussion: 
 We propose a dynamic subgroup identification method within the CARA design framework that could significantly advance precision medicine. However, we acknowledge some limitations in our approach. We aim to explore and address these challenges in our future research.First, our method assumes that outcomes in CARA experiments are observed immediately at the end of each stage without delay. This assumption, prevalent in adaptive experiments such as [15] and [49] , simplifies the modeling process and facilitates quick adjustments based on the latest data. However, in some practical scenarios, outcomes may be observed with delays, complicating the process of adjusting treatment allocations, as highlighted by [31] and [29] . In future work, we plan to revise our design framework and update our estimators to account for the impact of delayed responses on both treatment effects and variance estimators.Second, our work addresses scenarios where assigning a treatment is costly, and there is an overall constraint on how many treatments can be deployed. A key challenge arises from the potential misalignment between efficiently estimating the best causal effect and determining the optimal causal decision rule-particularly when the best decision rule is to treat all patients when there is any positive effect. For instance, [9] focuses on ensuring that all individuals who would benefit are accurately assigned to the treatment arm, assuming negligible treatment costs. To maximize the welfare for participating subjects, in future work, we plan to incorporate an "early stopping" step which would not only identify the most effective subgroups but also halt enrollment for subgroups exhibiting significantly adverse treatment effects.We present a notation table, as shown in Table 2 , to illustrate the key symbols and their descriptions used in our covariate-adjusted response-adaptive randomization (CARA) experiment framework. Tie set of the highest treatment effect τ 1 , containing indices of tied subgroups.The j-th subgroup.Subgroups that belong to the best set identified by T 1 . τ j Subgroup average treatment effect for subgroup j. τ T1Population treatment effect under the best subgroups in T 1 .Before discussions, we define some additional notations first. For t = 1, . . . , T , j = 1, . . . , m, and c > 0, denoteµ j,c (1) = E[Y it (1) c |X it ∈ S j ], µ j,c (0) = E[Y it (0) c |X it ∈ S j ], µ tj,c (1) = t s=1 ns i=1 1 (Xis∈Sj ) D is Y c is N tj (1) , µ tj,c (0) = t s=1 ns i=1 1 (Xis∈Sj ) (1 -D is )Y c is N tj (0) .Assumption 3 (Subgroup treatment effects). For δ ∈ (0, 0.5), the asymptotic distance between treatment effects of Subgroup j and Subgroup 1 diverges as N → ∞:N δ • min j / ∈T1 |τ 1 -τ j | → ∞, ∀j = 1, . . . , m.Lemma 1. Assume Assumption 1 and 2 holds. Let p j = P(X it ∈ S j |H t-1 ) be the subgroup proportions, and e * tj = P(D it = 1|X it ∈ S j , H t-1 ) be the treatment probabilities. Assume there exists some δ ∈ (0, 1/2) such that for all j = 1, 2, . . . , m and t = 1, 2, . . . T , δ ≤ e * tj ≤ 1 -δ. Then for any j = 1, 2, . . . , m, d = 0, 1, and any t satisfying n t → ∞,µ tj,c (d) = µ j,c (d) + O p 1 √ N t .Corollary 1. Let δ ∈ (0, 1/2) be some constant. Assume Assumptions 1 and 2 hold,sup δ≤e≤1-δ V tj (e) -V j (e) = O p 1 √ N t .Corollary 2. Assume Assumptions 1 and 2 hold. Thenτ tj -τ j = O p 1 √ N t .Theorem 4. Under Assumptions 1 and 2, asN → ∞, √ N ( τ j -τ j ) → N 0, V j (e * j ) , V j -V j (e * j ) = O p ( 1 √ N ).Lemma 1, Corollaries 1 and 2, and Theorem 4 are similar to the theoretical results in [41] . We defer readers to [41] for more technical details. Lemma 2. Under Assumptions 3, for j = 1, . . . , m, t = 1, . . . , T , any positive constant C and δ ∈ (0, 1 2 ), the following statement holds limNt→∞ P τ • tj -τ j ≥ N -δ t • C = 0.Lemma 3. Let δ ∈ (0, 1/2) be some constant. Assume Assumptions 1 and 2 hold, τ t, Tt1 -τ T1 = O p 1 √ N t , sup δ≤e≤1-δ V t, Tt1 (e) -V T1 (e) = O p 1 √ N t .Proof. Because maximizing the correct selection probability is equivalent to minimizing the incorrect selection probability, we formulate the incorrect selection probability as P τ T1 ≤ max j / ∈T1 τ j and denote the cardinality of T 1 as k, 1 ≤ k ≤ m -1, where m denotes the total number of subgroups. The incorrect selection probability can be bounded asmax j / ∈T1 P ( τ T1 ≤ τ j ) ≤ P τ T1 ≤ max j / ∈T1 τ j ≤ (m -k) • max j / ∈T1 P ( τ T1 ≤ τ j ) ,where the inequality follows from the union bound. For both sides, take the logarithm and divide by N ,1 N log max j / ∈T1 (1 -P ( τ T1 ≥ τ j )) ≤ 1 N log (m -k) + 1 N log max j / ∈T1 (1 -P ( τ T1 ≥ τ j )) .Additionally, by the large deviation theory [8] , there exists a rate function G(S T1 , S j ; e 1 , e j ), such thatlim N →∞ 1 N log (1 -P ( τ T1 ≥ τ j )) = -G(S T1 , S j ; e 1 , e j ), j / ∈ T 1 .Therefore, max j / ∈T1 1 -P τ T1 ≥ τ j is equivalent to exp -min j / ∈T1 G(S T1 , S j ; e 1 , e j ) . We obtain-min j / ∈T1 G(S T1 , S j ; e 1 , e j ) ≤ 1 N log 1 -P τ T1 ≥ max j / ∈T1 τ j ≤ log (m -k) N -min j / ∈T1G(S T1 , S j ; e 1 , e j ).As a result, based on the Gartner-Ellis Theorem [8, ch.2.3], when the sample size tends to infinity, we are able to obtain:lim N →∞ 1 N log 1 -P τ T1 ≥ max j / ∈T1 τ j = -min j / ∈T1 G(S 1 , S j ; e 1 , e j ),G(S T1 , S j ; e 1 , e j ) = (τ j -τ T1 ) 2 2 V T1 (e 1 ) + V j (e j ) .Proof. Note thatN t τ • tj -τ j = N t ( τ tj -τ j ) + N t τ • tj -τ tj .As for the first term √ N t ( τ tj -τ j ), by Corollary 2, we haveN t ( τ tj -τ j ) = O p (1) .As for the second term √ N t τ • tj -τ tj , we consider two situations under t = 1 and t > 1 separately. When t = 1, τ • 1j is generated from N τ • 1j , V 1j /N 1 , so we haveN 1 τ • 1j -τ • 1j = O p (1) .When t > 1, we compute τ • tj the same way as τ tj with the bootstrap samples instead, so τ • tj and τ tj are consistent and we haveN t τ • tj -τ tj = O p (1).Therefore, we obtain for t ≥ 1, √ N t τ • tj -τ j = O p (1), i.e., for any given ε > 0, there exists an M , such thatP N t | τ • tj -τ j | > M < ε.Then for any N t such thatN t > M C 1 1 2-δ , we haveP | τ • tj -τ j | > N -δ t • C < ε,and thuslim sup n→∞ P | τ • tj -τ j | ≥ N -δ t • C < ε.Note that the above inequality holds for arbitrary ε > 0. Hence, we havelim sup n→∞ P | τ • tj -τ j | ≥ N -δ t • C = 0,which completes the proof.Proof. Recall our definition of the tie set of best-performing subgroups:T 1 = {j : |τ 1 -τ j | = o(N -1/2 ), j = 1, . . . , m}.This suggests that ∀j ∈ T 1 , there exists a sequence δ n → 0 as N → 0, such thatτ j = τ 1 + N -1 2 • δ n , ∀j ∈ T 1 .We further define a set of subgroups with treatment effects smaller than those in the set T 1 :T L 1 = {j : τ j < min m∈T1 {τ m }, j = 1, . . . , m}.As for the estimated near tie set of τ 1 in Stage t for t = 1, . . . , T , we have for any j ∈ T t1 , and a pair of hyperparameters ) , where δ = 0.25. Thus, there exists a positive constant C such thatEQUATION| τ • tj -τ • t,(1) | N -δ t < C, ∀j ∈ T t1 .Similar to the proof of Lemma 1 in [42] , our proof is composed of the following three steps:Step 1. We first show that the subgroup with the largest treatment effect in the resampled statistics falls into the set T 1 with high probability, that isEQUATIONwhere 1t =EQUATION) ) for t = 1, . . . , T . Because τ • t, 1t ∈ [min j∈T1 τ tj , max j∈T1 τ • tj ]by definition, coupled with the fact thatmax k∈T L 1 τ • tk < min j∈T1 τ • tj ≤ max j∈T1 τ • tj ⊂ 1t ∈ T 1 .Under Assumption 3, by Lemma 2, for any k ∈ T L 1 and j ∈ T 1 , we have limNt→∞ P τ • tk < τ • tj = 1, so we can obtain lim Nt→∞ P max k∈T L 1 τ • tk < min j∈T1 τ • tj ≤ max j∈T1 τ • tj = 1. Thus we have shown that lim Nt→∞ P 1t ∈ T 1 = 1.Step 2. We then show, for j / ∈ T 1 , and t = 1, . . . , T ,EQUATION)For any ε > 0 and j / ∈ T 1 , we have the following holdsP 1(j ∈ T t1 ) > ε =P 1(j ∈ T t1 ) > ε|j ∈ T t1 • P j ∈ T t1 + P 1(j ∈ T t1 ) > ε|j / ∈ T t1 • P j / ∈ T t1 ≤P j ∈ T t1 Def = P | τ • tj -τ • t,(1) | N -δ t < C =P | τ • tj -τ • t, 1| N -δ t < C =P | τ • tj -τ • t, 1| < N -δ t • C =P | τ • tj -τ j -τ • t, 1 -τ1 + τ j -τ t, 1 | < N -δ t • C ≤P N δ t |τ j -τ1| -N δ t τ • tj -τ j -N δ t τ • t, 1 -τ1 < C =P N δ t τ • tj -τ j + N δ t τ • t, 1 -τ1 > N δ t |τ j -τ1| + C . By definition, for j / ∈ T 1 , P N δ t |τ j -τ1| < C ≤ P N δ t |τ j -τ1| < C, 1 ∈ T 1 + P 1 / ∈ T 1 ≤ max k∈T1 P N δ t |τ j -τ k | < C, k ∈ T 1 + P 1 / ∈ T 1 .Under Assumption 3, Lemma 2 and the conclusion in Eq (12) in Step 1 suggest that by letting N t → ∞ on both sides, we have the above probability converges to zero. Based on above derivation, we have shown that lim Nt→∞ P 1(j ∈ T t1 ) > ε, j / ∈ T 1 = 0.Step 3. We are left to prove that for all j ∈ T 1 and t = 1, . . . , T , the following holds for all ε > 0:lim Nt→∞ P 1(j ∈ T t1 ) -1 > ε = 0.Following similar arguments, for a positive constant C, we have for k, j ∈ T 1 , the following statement holds:P 1(j ∈ T t1 ) -1 > ε =P 1(j ∈ T t1 ) -1 > ε|j ∈ T t1 • P j ∈ T t1 + P 1(j ∈ T t1 ) -1 > ε|j / ∈ T t1 • P j / ∈ T t1 ≤P j / ∈ T t1 Def = P | τ • tj -τ • t,(1) | N -δ t ≥ C =P | τ • tj -τ • t, 1| N -δ t ≥ C =P | τ • tj -τ j -τ • t, 1 -τ1 + (τ j -τ1) | ≥ N -δ t • C ≤P τ • tj -τ j + |τ tj -τ1| + τ • t, 1 -τ1 ≥ N -δ t • C ≤P N 1 2 t τ • tj -τ j + N 1 2 t |τ j -τ1| + N 1 2 t τ • t, 1 -τ1 ≥ N 1 2 -δ t • C .By definition of the near-tie set, for j ∈ T 1 , we haveP N 1 2 t |τ j -τ1| < C ≤P N δ t |τ j -τ1| < C, 1 ∈ T 1 + P 1 / ∈ T 1 = max k∈T1 P N δ t |τ j -τ k | < C, k ∈ T 1 + P 1 / ∈ T 1Again, under Assumption 3, Lemma 2 and the conclusion in Eq (13) we have derived in Step 1, by letting N t → ∞ on both side, we have the above probability converges to 1. According to above discussion, we have shown that lim Nt→∞ P 1(j ∈ T t1 ) -1 > ε = 0.Combining the results obtained in the aforementioned three steps, we have for j = 1, . . . , m * t and for ε > 0, the following holds:lim Nt→∞ P |1(j ∈ T t1 ) -1(j ∈ T 1 )| > ε = 0, ∀t = 1, . . . , T.Then it is straightforward to get the final results by taking t = T .Proof. Note that τ t, Tt1 -τ T1 = ( τ t, Tt1 -τ t,T1 ) + ( τ t,T1 -τ T1 ), V t, Tt1 (e) -V T1 (e) = ( V t, Tt1 (e) -V t,T1 (e)) + ( V t,T1 (e) -V T1 (e)).Thus this lemma follows directly from Corollary 1, Corollary 2 and Theorem 1.Proof. Denote the oracle objective function asf (e) = min |T1|+1≤j≤m (τ j -τ T1 ) 2 V T1 (e 1 ) + V j (e j ) = min |T1|+1≤j≤m (τ (j-|T1|+1) -τ T1 ) 2 V T1 (e 1 ) + V (j-|T1|+1) (e j ),where the second equality follows from regarding the tie set T 1 as a whole after merging the data, and then T 1 ranks 1 and other subgroups rank from 2 to m -|T 1 | + 1. Then, the original optimization problem can be written asmax e f (e), s.t. m j=|T1|+1 p j e j ≤ c 1 , c 2 ≤ e j ≤ 1 -c 2 , j = |T 1 | + 1, . . . , m.For notational simplicity, let E be the set of candidate allocation rules (i.e., all allocation rules satisfying the constraints).Similarly, let f t (e) be the estimated objective function, that is, withm * t = m -| T t-1,1 | + 1, f t (e) = min 2≤j≤m * t ( τ t-1,(j) -τ t-1, Tt-1,1 ) 2 V t-1, Tt-1,1 (e 1 ) + V t-1,(j) (e j ) = min | Tt-1,1|+1≤j≤m ( τ t-1,(j-| Tt-1,1|+1) -τ t-1, Tt-1,1 ) 2 V t-1, Tt-1,1 (e 1 ) + V t-1,(j-| Tt-1,1|+1) (e j ).We solve the following optimization problem:max e f t (e), s.t. m j=| Tt-1,1|+1 p j e j ≤ c 1 , c 2 ≤ e j ≤ 1 -c 2 , j = | T t-1,1 | + 1, . . . , m.We denote the set of maximizers to the above optimization problem as E * . Lastly, let the δenlargement of E * be E * + B δ , where B δ is the Euclidean ball centered at the origin with radius δ: EQUATIONProof. Adopting the same proof of Theorem 1 in [41] , we obtain that Then with the calibration procedure presented in Section 4.1, by replacing e * tj with e tj , we have thatEQUATIONEQUATIONEQUATIONn tj = e * j - T t=1 1 N p j t-1 s=1 ns i=1 1 (Xis∈Sj ) D is -e * tj + o p (1) = e * j + o p (1),where the second term in the last equality has the order of O p (1/ √ N ) with variance calculation. Thus, we complete the proof.Proof. Note thatEQUATIONτ T1 -τ T1 = j∈T1 p j τ j j∈T1 p j - j∈T1 p j τ j j∈T1 p j = j∈T1 p j ( τ j -τ j ) j∈T1 p j .Applying the result presented in Theorem 4,√ N ( τ j -τ j ) D → N 0, V j (e * j ) , we have that √ N ( τ T1 -τ T1 ) D → N 0, V T1 (e * 1 ),where V T1 (e * 1 ) = j∈T1 p 2 j V j e * j j∈T1 p j 2. By Theorem 1, we also have τ T1 -τ T1 → p 0.Thus by Slutsky's theorem,√ N ( τ T1 -τ T1 ) D → N 0, V T1 (e * 1 ).Consistency of the variance estimator follows from Theorem 1 and Lemma 1, which is similar to the proof of Lemma 3.In this section, we provide another data-adaptive algorithm for choosing hyperparameters for Stage 1 and another bootstrap-based merging data bootstrap for later stages.We present a double bootstrap method for choosing hyperparameters in Algorithm 4 as an alternative to Algorithm 3.Step 1 (Input):1: Input H 1 , τ 1j , V 1j , and (c L , c R ). 2: Compute τ * 1j as in Eq (4). Step b (Bootstrap): 3: for b ← 1 to B do 4: Generate τ * 1 from N τ * 1 , Ω n /n 1 , where Ω n = diag V 11 , . . . , V 1m ; 5: for r ← 1 to R do 6: Generate τ * * 1 from N τ * 1 , Ω n /n 1 ; 7:Compute τ * * 1,(1) as in Eq (6); 8:end for9:Record B 1,(b) (c L , c R ) as in Eq (14) . 10: end forStep B (Output): (14) . Choose the pair11: Compute L 1 (c L , c R ) as in Eq(c 1 L , c 1 R ) that minimizes L 1 (c L , c R ).Here, we define the loss functionEQUATIONwhereB 1,(b) (c L , c R ) is the b-th smallest statistics in B 11 (c L , c R ) , . . . , B 1B (c L , c R ).We could know thatP( τ * 1,(1) ≤ τ 1 |(Y is , D is , X is ) n1 i=1) roughly follows Unif(0,1) when the sample size n 1 is large. Given a desirable pair of hyperparameters (c L , c R ), we would thus expect that B 1,(1) (c L , c R ), . . . , B 1,(B) (c L , c R )share a similar distribution with the ordered statistics of i.i.d. Unif(0, 1) random variables. The loss function defined in Eq (14) measures the average of squared differences between B 1,(b) (c L , c R ) and the expected value of the order statistics of the Unif(0, 1) random variables. Given the rational above, we would expect that the optimal pair of hyperparameters (c 1 L , c 1 R ) at Stage 1 minimizes such a loss.We present naïve bootstrap method for identifying and merging data for later stages in Algorithm 5 as an alternative to Algorithm 2.Step 1 (Input): Identify the best subgroups T t1 with the bootstrap samples as in Eq (3). 6: end for1: Input {H s } t s=1 , τ tj , V tj ,Step B (Output): 7: Choose T t1 with the highest frequency of occurrence and merge subgroups that belong to T t1 .In this section, we mainly focus on the comparison of four variations of our proposed design strategy adopted with single or double bootstrap for selecting hyperparameters and naïve or separate bootstrap for data merging. We adopt the same setting mentioned in Section 6. First, from the comparison in Figure 2 , as the adaptive design goes on, four variations of our proposed design strategy do not present a distinctly different performance, and they are all efficient with correct selection probability gradually rising over 0.9. Our design conducted with a single bootstrap for choosing hyperparameters and naïve bootstrap for identifying and merging tie sets perform a little better than other variations after 15 stages.Second, from Table 3 , we obtain that our proposed strategy conducted with a single bootstrap for selecting hyperparameters and a separate bootstrap for identifying and merging data has the smallest √ N -scaled bias. Overall, these four variations demonstrate similar efficiency in identifying and merging the best subgroups with almost the same estimated treatment effect and standard deviation. We add normalized mutual information as an additional metric to evaluate the similarity of estimated tie set and true tie set of best-performing subgroups. With the results presented in Figure 3 (a) and (b), the normalized mutual information for our proposed design strategy and the three conventional methods exhibits a similar trend in terms of correct selection probability. This observation further confirms that our proposed design strategy outperforms the conventional methods.When contextual information (or additional covariate information) is considered in our design, the subgroup treatment effects and associated variances in Eq.8 can be replaced byτ t-1,(j) = 1 N t-1,(j) t-1 s=1 ns i=1 1 (Xis∈S (j) ) D is e t-1,(j) (Y is -µ 1 (Z is )) + µ 1 (Z is ) - 1 N t-1,(j) t-1 s=1 ns i=1 1 (Xis∈S (j) ) 1 -D is 1 -e t-1,(j) (Y is -µ 0 (Z is )) + µ 0 (Z is ) V t-1,(j) (e j ) = t-1 s=1 ns i=1 1 (Xis∈S (j) ) D is (Y is -µ 1 (Z is )) 2 N t-1,(j) (1)e j • N t-1,(j) N t-1 -1 + t-1 s=1 ns i=1 1 (Xis∈S (j) ) (1 -D is )(Y is -µ 0 (Z is )) 2 N t-1,(j) (0) (1 -e j ) • N t-1,(j) N t-1 -1 + 1 N t-1,(j) t-1 s=1 ns i=1 1 (Xis∈S (j) ) µ 1 (Z is ) -µ 0 (Z is ) -τ t-1,(j) 2 , where µ d (Z is ) = E[Y is |D is = d, Z is ∈ S j ], d ∈ {0, 1}.We still generate synthetic data that mimic the original PBC dataset with the potential outcome from Y i (d)|X i ∈ S j = µ dj +0.5Z i +ε dj , Z i ∼ N (0, 1), ε dj ∼ N (0, σ 2 dj ), j = 1, . . . , 5 to investigate the performance of our proposed stratgy with AIPW estimator.Through simulation studies in Figure 4 , we found that our method using IPW or AIPW) has the highest accuracy in identifying the best subgroups. The performance of the causal tree model is comparable to that of the contextual bandit algorithms where treatment effects are estimated with the AIPW estimator, thereby including contextual information. Regarding the causal tree model, we would like to note that the comparison may not be entirely fair for the causal tree model. This is because the causal tree method is a method used to identify subgroups after the data has already been collected, whereas our method focuses on designing data collection mechanisms to accurately identify the best subgroups. Therefore, it can be expected that our approach will have higher accuracy in identifying subgroups. In this section, we provide an additional synthetic real study in another domain to showcase the performance of our proposed design. We consider a job dataset from the National Supported Work (NSW) program. This program, initiated in the 1970s, aims to provide disadvantaged workers with work experience. We utilize the dataset from a field experiment referenced in [7] (n = 455), which includes 185 workers in the treatment group and 260 workers in the control group. The dataset features a treatment indicator variable, an outcome variable representing participant earnings post-treatment in 1978, and eight baseline variables. These baseline variables include age, years of education, an indicator for high school graduation, indicators for Black and Hispanic ethnicity, marital status, and pre-treatment earnings for the years 1974 and 1975.We aim to investigate whether the job training program is indeed beneficial for certain groups of workers in for subgroups defined by: ( 1 It is a suitable dataset because by conducting two-sample t-tests between every two subgroups the pairwise similarity of the distribution of subgroups, Subgroups (1), ( 2) and (3) are regarded as tie sets of the best subgroups under the significance level of α = 0.05. We provide the results in Figure 5 , which can validate the performance of our proposed strategy. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments? Answer: [Yes] Justification: Please see Section 6. Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have properly credited the original data source.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix:  


Decision: This paper presents a design strategy to dynamically identify the best subgroups that demonstrate significant treatment effects in randomized trials. It theoretically demonstrates that the proposed design has a higher probability of identifying the best subgroups compared to conventional designs. The paper also validates the design using both synthetic and clinical trial data.

All reviewers agree that the proposed design is important in practice and novel. The design is supported by sound theoretical foundations. Reviewers have expressed some concerns about the presentation and explanations. Specifically, one reviewer raised concerns regarding the related work, presentation clarity, and the experimental comparison baselines. During the discussion phase, most of the reviewers' questions were addressed, and the remaining concerns do not affect the soundness or novelty of the paper. Therefore, the paper is acceptable.

Reviews: review: paper_summary: paper_summary: The paper introduces a dynamic subgroup identification strategy within the framework of covariate-adjusted response-adaptive randomization, addressing the need for more nuanced subgroup analysis in clinical trials. This strategy aims to optimize treatment allocation dynamically and identify subgroups that demonstrate significant treatment effects, which is crucial in practice.
strength_weakness: strength_weakness: 1.	I understand the paper is a very technical paper, and the authors have spent a lot of efforts in making the paper easy to follow. My minor comment is that maybe providing more intuition on the reasons why the complicated design is needed will be helpful. For example, why we need resampling and bootstrap? What kind of technical challenge bootstrap can help to overcome?
2.	The objective function in line 128 seems a little mismatched with the objective of best subgroup identification. For example, if there are two arms in $\mathcal{T}_1$  with variance zero and very large respectively, it seems that optimizing the objective in line 128 will lead to the solution that we forget about the super large one and devote most efforts to the one with variance 0.  Thus, I am thinking about whether there is some analysis on the probability that we successfully identify $\mathcal{T}_1$? More specifically, can we always guarantee we identify everything in $\mathcal{T}_1$ and we never include something out of $\mathcal{T}_1$?
questions: questions: See previous comments.
limitations: limitations: See previous comments.
paper_summary: paper_summary: This paper studies an interesting problem in clinical trials to identify patient subgroups with the most beneficial responses to the treatment, which is essential for clinicians to create personalized treatment plans for their patients. However, most existing strategies for the design of clinical trials rely on domain knowledge or past experience of expert clinicians and stick to several pre-defined patient subgroups throughout the trial, which discards the valuable information collected from different trial stages. Some adaptive experimental strategies are developed to identify the best performing patient subgroup based on the trial outcomes; but their negligence of other subgroups where the treatment could be equally effective usually causes inefficient utilization of the experimental efforts. To tackle these challenges, the authors propose a dynamic subgroup identification approach to enable the construction of more effective experimental strategies for practical clinical trials. The authors claim three major contributions in this study: 1) their approach allows the dynamic identification for best patient subgroups based on experimental data collected during the trial process; 2) the authors develop new algorithms to effectively merge patient subgroups with similar (highest) responses to the treatment and provide theoretical results to support their analyses; and 3) the proposed method is validated using a synthetic dataset constructed from a clinical trial on cirrhosis.
strength_weakness: strength_weakness: ## Related Works

There are two short paragraphs in the Introduction discussing the literature related to this study. However, to distinguish the experimental design approach in this paper from other research and highlight the contributions of this study, a more comprehensive comparison with related works is needed. It seems that there are many other studies, with similar focuses on subgroup identification in clinical trials, not sufficiently discussed in this paper. For instance:

- Adaptive identification and assessment of patient subgroups [1]
- Identification of patient subgroups with similar clinical characteristics (covariates) [2]
- Clustering of patient subgroups with different levels of benefits in clinical trials [3]

The authors are encouraged to provide discussions and comparisons with related works on patient subgroup identification to better demonstrate the novelty and advantage of this study.

Further, although the authors emphasize that their method focuses on the setting of covariate-adjusted response-adaptive (CARA) experiments which is an under-explored area, the reviewer cannot find discussions on the contribution or benefit of including patient covariates ($X$) in the design of clinical trials. The covariates-based patient subgroup identification has already been studied in the machine learning literature, e.g., [2,3]. 

References: 

[1] Guo, Wentian, Yuan Ji, and Daniel VT Catenacci. "A subgroup cluster‐based Bayesian adaptive design for precision medicine." Biometrics 73.2 (2017): 367-377.

[2] Lee, Beom S., et al. "A clustering method to identify who benefits most from the treatment group in clinical trials." Health Psychology and Behavioral Medicine: an Open Access Journal 2.1 (2014): 723-734.

[3] Xu, Jie, et al. "Machine learning enabled subgroup analysis with real-world data to inform clinical trial eligibility criteria design." Scientific Reports 13.1 (2023): 613.





## Clarity

### The role of patient covariates

As mentioned earlier, the role of patient covariates is not sufficiently discussed in the proposed experimental design approach. The only place related to the covariates seems to be line 7 of Algorithm 2 where previous trial results are randomly resampled. It is unclear how the experimental design is adjusted based on patient covariates. Since this paper focuses on the CARA setting and considers both treatment responses and patient covariates in subgroup identification, it is important for the authors to elaborate the contribution of patient covariates in the proposed algorithms and how is this study different from conventional research on response-adaptive randomization (RAR) settings.

### Notations

There are many symbols introduced in the analysis of this paper without any explanation. 
For instance, the variable $\hat{p}\_{tl}$ in Eq. 1 has never been explained. 
The symbol $\mathcal{B}_{t,b}$ in Eq. 6 comes from nowhere. 
In the meantime, there are so many similar symbols used in the discussion, and it is very difficult for the reviewer to tell their difference. The authors are encouraged to ensure the consistency in their notation. If possible, a notation table could help a lot to improve the clarity of this paper.

### Insufficient explanations

Some results or derivations in this paper are introduced without proper explanation. For instance, although the authors provide citations to the large deviation theory in LN 129, the equivalence between the correct identification probability and the optimization objective remains obscure. Similarly, it is unclear why the optimal treatment allocation can be derived from Eq. 1. Additionally, the hard-coded exponent 0.05 for $\Delta$ in LN 162 appears without any explanation.

## Correctness

The correctness of some proofs seems to be questionable. For instance, in the proof of Theorem 1, it is unclear why $\tau_{t,l} = \tau_{l}$. The inequality in LN 450 cannot be directly established according to the analysis in LN 451 – 453. Similarly, the final inequality in LN 457 is non-trivial and the authors should provide analysis to prove its correctness.

## Evaluation

### Dataset

The proposed experimental design approach is only evaluated on a synthetic dataset. To validate the general applicability and performance of this method, benchmark results on more datasets are necessary.

### Baselines

The authors have discussed many relevant studies in the Introduction. However, only three baselines are considered in the experiment. For a more comprehensive comparison, the authors are encouraged to include more baselines from related works to demonstrate the advantage of their method. Particularly, the reviewer is interested in the performance of contextual bandit algorithms and causal tree models.

### Metrics

Note that the experimental results are obtained on a synthetic dataset where the ground truth subgroup labels are available. The authors are encouraged to include additional metrics on clustering accuracy, e.g., purity score, normalized mutual information, etc., in the benchmark.
questions: questions: In summary, I have the following concerns about this paper.
1. The related works are not sufficiently discussed. The difference between this paper and relevant studies should be clearly illustrated.
2. It is unclear how the proposed method is different from conventional methods considering the RAR settings. There is no discussion on the contribution or benefit of including patient covariates ($X$) in the design of clinical trials. 
3. Many symbols are introduced in the analysis of this paper without proper explanation.
4. The derivation of some key results is not sufficiently explained. For instance, the optimization objective below LN 129 and the optimal treatment allocation in Eq. 1.
5. The correctness of some proofs seems to be questionable. Specifically, the proof for Theorem 1. Please see the weakness section above for details.
6. The proposed method needs to be tested on more datasets, and the authors are encouraged to include more performance metrics in the benchmark.
7. There should be more baselines, e.g., causal tree and contextual bandit, in the benchmark to highlight the advantage of the proposed method.
8. How does the number of stages $T$ in a trial affect the convergence of the proposed method? What if there is only one stage? What if $T$ is small (Below 4)?
limitations: limitations: The limitations of this study are briefly discussed at the end of this paper. The authors identify the potential mismatch between their assumption on immediate observation of treatment responses and the delays commonly observed in real-world clinical trials. However, the potential negative societal impact of the clinical trial design approach in this paper is not sufficiently discussed. In addition, the authors are encouraged to discuss whether their approach can be generalized to deal with scenarios with multiple treatment options.
paper_summary: paper_summary: This paper introduces a new dynamic treatment assignment for clinical trial to target treatment to the group most likely to benefit from it.
strength_weakness: strength_weakness: While the paper tackles a critical problem, the problem is studied in depth in biostatistics. While I am unfamiliar with the literature on this topic, I am surprised there is none that could be considered for comparison. A more in-depth analysis of the literature on this topic should be presented in the Appendix to justify the choice of compared methods.
questions: questions: Following the previous point, I would recommend a more in-depth review of the literature to convince the reader that this problem does not have alternative solution in the literature.
limitations: limitations: The paper discusses the critical limitation of the proposed approach's assumption of instantaneous access to the outcome following treatment. It would be interesting to discuss this assumption in the context of the existing literature. Is it a common assumption? If not, it would be important to justify further.
paper_summary: paper_summary: The paper proposes a dynamic subgroup identification strategy within covariate-adjusted response-adaptive randomization (CARA) for clinical trials. This adaptive method dynamically identifies and adjusts treatment allocation to the best-performing subgroups based on ongoing trial data, and thus extend the traditional fixed-design trials to an online setup. Moreover, the paper also makes theoretical contributions by showing validity, asymptotic normality and efficiency of the estimator for the best subgroup treatment effect.
strength_weakness: strength_weakness: Overall the paper is technically solid, though I notice some issues that can be addressed in the revision. First, the clarity of this paper can be improved as I identified some sentences/equations that are confusing. Please check the question section for details. Second and more importantly, the way that the paper formulates the problem needs some further justification, which is also related to Q2 in the question section.  The paper proposes a design to maximize the probability of correctly identifying the best subgroup, and estimate the best treatment effect; the theory and experiments all focus on this best subgroup identification. However, in personalized medicine which is the motivation for this work, I suppose practitioners care more about whether the treatment is beneficial to a certain group of patients or not, rather than the best treatment effect. In other words, even one can efficiently estimate the best treatment effect using CARA, it does not guarantee the best causal decision rule in personalized medicine. See https://pubsonline.informs.org/doi/10.1287/ijds.2021.0006 for a relevant discussion. Therefore, I suggest the authors discuss this gap between your work and practical considerations; this will help further justify the necessity of best subgroup identification.
questions: questions: 1. Page 3, bottom equation: In the equation at the bottom of this page, should $=$ be $\coloneqq$, i.e., a definition?

2. Page 3, Line 127-128: The goal of the design is to maximize the correct identification probability, based on which the paper proposes CARA and develops the corresponding theory. However, why maximizing the correct identification probability is the ultimate goal? In precision medicine, a more plausible goal is to maximize the overall welfare for the patients, i.e., finding the design that best improves the medical outcome for all patients. How do you compare your design objective to this welfare maximization objective? Can you modify your design for the second objective?

3. Page 4, Eq (1): This equation is confusing. What is the objective function for $\mathbf{e}$? I only saw the constraint set inside the parenthesis.
limitations: limitations: As discussed in the paper, one main limitation is that the method cannot handle delayed outcomes, which can be restrictive in the real-world clinical trials. In my opinion, the authors have adequately addressed the limitations.

Comments: title: Maintain score
Two days remain for discussion
comment: Thank you for your reply and recognition. We regret that there are still concerns regarding our paper.

**More Effective Baselines**

Thank you for suggesting an additional baseline. We will include a comparison with this baseline in the revised manuscript.

**Tuning Parameter**

- **Tie Set Selection:** Our hyperparameter selection process is independent of the dynamic identification of the best subgroups. Specifically, there are two separate bootstrap procedures: one for hyperparameter selection and another for dynamic identification. In both procedures, we compute $\hat{\tau }\_{tj}^{\circ}$ with bootstrap samples independently. For hyperparameter selection, $\hat{\tau }\_{tj}^{\circ}$, $\gamma$, and $\Delta$ are used to calculate $\hat{\tau }\_{tj}^*$ and select the hyperparameter pair $(c\_\texttt{L},c\_\texttt{R})$. For dynamic identification, $\hat{\tau }\_{tj}^{\circ}$ is used to identify the tie set in Eq (5), and this process does not depend on $\gamma$ and $\Delta$.

- **Upper Bound of $\Delta$:** As previously mentioned, $\gamma$ is a small tuning parameter to ensure $\Delta < 1$. We have set the upper bound at 0.99 as a precautionary measure.

- **Typo in New Results:** We apologize for the typo in Table 1. To address this issue, we are prepared to provide the code for the simulation to clarify any discrepancies.
I appreciate the authors' additional rebuttal which has addressed my concerns on subgroup initialization and the contribution of AIPW.

#### More effective baselines
According to the authors' response, a more reasonable baseline should be contextual bandits with AIPW estimation as feature maps for treatment assignment (and using clustering algorithms like K-means or agglomerative clustering to merge subgroups with similar treatment responses as desired in this paper). The current baselines seem to be quite weak.

#### Tuning parameter
I am not fully convinced by the authors' response.
- **Tie set selection**: The tie set selection in Eq. (5) is dependent of $\\hat{\\tau}^{o}_{tk}$ which is computed with the bootstrap samples as clarified by the authors. This implies that the tie set selection is in fact affected by $\\Delta$ and $\\gamma$.
-  **Upper bound of $\\Delta$**: Setting the upper bound of $\\Delta$ to 0.99 is kind of arbitrary and lacks rigor.
- **Typo in new results**:  I appreciate the new results by the authors. However, it is difficult to verify whether the range of parameter $\\gamma$ reported in Table 1 of the rebuttal is a typo.

Therefore, I can only increase my rating of this paper from 3 to 4.
The second comment below addresses your second concern:

**Tuning parameter**: We are sorry that the previous presentation of our procedure might have caused some confusion. In the following, we would like to clarify that our bootstrap procedure is not intended to select $\gamma$, and both $\gamma$ and $\Delta$ are only adopted to select the hyperparameter pair $(c\_\texttt{L},c\_\texttt{R})$, which determines the neighborhood for merging subgroups. 

First, we will replace lines 159-170 with the following in our revision to clarify our procedure:

- **Dynamic identification of the best subgroups (Algorithm 2)**:The dynamic subgroup identification algorithm involves a resampling step at each stage. Specifically, at each stage $t$, we generate bootstrap samples  $\hat{\mathbf{\tau }}\_t^{\circ}$ from a Gaussian distribution centering around $\hat{\mathbf{\tau }}\_t$, which is estimated using the data collected up to stage $t$ using Eq (8). We then identify the best subgroups at Stage $t$ using 
\begin{align*}
\hat{\mathcal{T}}\_{t1} = \\{k: w\_{k,(1) }^{\circ} = 1,k = 1,\ldots,m\\}, \ (5)
\end{align*} where $w\_{k,(1)}^{\circ}=\mathbb{1}\\{-c^t\_{\texttt{L}}\cdot N\_t^{-\delta}\cdot \hat{\mathbb{V}}\_{t,( 1)}^{\delta}\leqslant (\hat{\tau}\_{tk}^{\circ}-\hat{\tau}\_{t,(1) }^{\circ})\leqslant c^t\_{\texttt{R}}\cdot N\_t^{-\delta}\cdot \hat{\mathbb{V}}\_{t,(1)}^{\delta }\\}$. $N_t = \sum\_{s=1}^t n\_s$ and $\delta = 0.25$. Note that the above formulation relies on a pair of hyperparameters $(c^t\_{\texttt{L}},c^t\_{\texttt{R}})$, which are selected data-adaptively. In what follows, we shall illustrate the algorithm for selecting hyperparameters.
 
- **Hyperparameter selection (Algorithm 3)**: In line 7 of Algorithm 3, we adopt a bootstrap method and propose several alternative bootstrap methods in Algorithm 3(line 7) in Appendix (Section H).Algorithm 3 involves a resampling step that generates bootstrap samples $\hat{\mathbf{\tau}}\_t^*$  from a Gaussian distribution centering around $\mathbf{\tau}\_t^*$ at Stage 1. In line 2, we compute $\mathbf{\tau}\_t^*=(\tau\_{t1}^*,\ldots,\tau\_{tm\_t^*}^{* })^{\prime}$ as \begin{align*} \tau\_{tj}^*=\Delta\_t \cdot \frac{\sum\_{j=1}^{m\_t^*}\hat\tau\_{tj}}{m\_t^*}+(1-\Delta\_t) \cdot \hat\tau\_{tj}, \ j = 1,\ldots,m, \ (3) \end{align*} where $\Delta\_t =\min\\{0.99,\frac{\sum\_{j=1}^{m\_t^*}\hat{\mathbb{V}}\_{tj}}{ N\_t\sum\_{j=1}^{m\_t^*}(\hat{\tau}\_{tj}-\overline{\hat{\tau}}\_t)^{2}}\times N\_t^{\gamma}\\}$ and $\gamma \in (0,0.2)$. We choose $\gamma = 0.05$ in our simulation studies, and our procedure is shown not sensitive to $\gamma <1$.  In line 8, we compute  $\hat{\mathbf{\tau}}\_t^*=(\hat{\tau}\_{t1}^*,\ldots,\hat{\tau}\_{tm\_t^*}^*)^{\prime}$ at Stage $t$ for $t>1$ as 
\begin{align*}
\hat{\tau}\_{tj}^*=\Delta\_t \cdot \frac{\sum\_{j=1}^{m\_t^*}\hat{\tau}\_{tj}^{\circ}}{m\_t^*}+(1-\Delta\_t) \cdot \hat{\tau }\_{tj}^{\circ},  \ (4)
\end{align*} where $\hat{\tau}\_{tj}^{\circ}$ is computed with the bootstrap samples as in Eq (8). 

Through this revision, we hope to clarify that 
- The final tie set is selected using Eq (5), not depending on $\Delta$ and $\gamma$. 
- You are absolutely correct that $\Delta$ cannot be 1; we will revise our algorithm to put a lower bound of 0.99. Additionally, there was a typo in Table 1; the magnitude of $\gamma$ we have tested is actually from $0.00$ to $0.10$. We hope to provide additional simulation results below. Due to time limit, we are only able to provide results for reduced sample size with $n_t = 400$, $T=4$ and reduced $B=400$ on the correct selection probability, Monte Carlo bias, and variance for a various choice of $\gamma$: 
  -  $\gamma = 0.05$, CSP: $0.68, 0.81, 0.85, 0.86$; $\sqrt{N}$Bias: $35.20$; SD: $44.71$; 
  - $\gamma = 0.10$, CSP: $0.68, 0.81, 0.85, 0.86$; $\sqrt{N}$Bias: $35.20$; SD: $44.71$; 
  -  $\gamma = 0.15$,CSP: $0.67, 0.81, 0.86, 0.86$; $\sqrt{N}$Bias: $35.21$; SD: $44.72$;
  - $\gamma = 0.20$, CSP: $0.68, 0.81, 0.86, 0.86$; $\sqrt{N}$Bias: $34.42$; SD: $44.75$. 

The performance of our method in this response may not be as strong as the results in the submitted manuscript due to time constraints in addressing your concerns. The results still demonstrate that our method is not sensitive to the choice of $\gamma$. 

We hope this revision of our manuscript will address your concerns!
Thank you for raising additional concerns. Due to space constraints, we hope to break down our response into two comments. The first comment below addresses your first and third concerns. 

**Usage of covariates**: To ensure our proposed design is practically relevant, our design does not incorporate any data-driven methods for identifying initial subgroup divisions after the trial has started. This is because regulatory agencies often strongly encourage that subgroups be pre-specified to enhance the interpretability of trial results and prevent data mining during the planning phase of a clinical trial; see some sample RCT designs with pre-fixed subgroups in Murray et al.(2018), Thall et al.,(2003), and Hu et al.,(2015).In accordance with this regulation, our design aims to dynamically merge subgroups if certain subgroups show homogeneous effects (note that merging does not create new divisions of subgroups) and sequentially adjusts treatment assignment probabilities within those merged subgroups to identify the best one efficiently. We do see that adding an initialization stage can be helpful when the initial subgroups are not informative of the treatment effect heterogeneity. As a future direction, we plan to explore the possibility of using tree-based methods to identify subgroups and sequentially merge them. 

**Extension with augmented IPW**: Indeed, it may seem counterintuitive that including additional covariates does not significantly improve the empirical performance. This is because our proposed method employs the IPW estimator with estimated propensity scores, which, as justified by Hirano et al. (2003), already attains the semiparametric efficiency bound. Thus, in the supplementary simulation study, adjusting for additional covariates with AIPW does not further improve the empirical performance of the treatment effect estimator, and therefore, the empirical performance of our design remains unchanged. We hope this addresses your concern! Thank you very much for carefully going through our new simulation results. 

Reference:
- Thall, P. F., Wathen, J. K., Bekele, B. N., Champlin, R. E., Baker, L. H., and Benjamin, R. S.(2003). Hierarchical bayesian approaches to phase ii trials in diseases with multiple subtypes. Statistics in medicine, 22(5):763–780.
- Murray, T. A., Yuan, Y., Thall, P. F., Elizondo, J. H., and Hofstetter, W. L. (2018). A utility-based design for randomized comparative trials with ordinal outcomes and prognostic subgroups.Biometrics, 74(3):1095–1103
- Hu, J., Zhu, H., and Hu, F. (2015). A unified family of covariate-adjusted response-adaptive designs
based on efficiency and ethics. Journal of the American Statistical Association, 110(509):357–367
- Hirano, K., Imbens, G. W., and Ridder, G. (2003). Efficient estimation of average treatment effects using the estimated propensity score. Econometrica, 71(4):1161–1189.
Thank you for your response. I appreciate the authors' clarification on related works and details in their proof.
However, regarding the new results presented in the rebuttal, I still have the following concerns.

#### The usage of patient covariates
Thanks for the clarification on the differences between the proposed method and RAR and CARA approaches.
It seems that the trial strategy proposed in this paper relies on some conventional covariate-adjusted strategies to generate the initial subgroup division.
However, its robustness with respect to the subgroup initialization is not properly evaluated.
What if the initial subgroups are not correctly aligned to the distribution of treatment responses in a population (which could be common in real-world applications)?

#### Tuning parameter $\\gamma$
The authors mention that the number $\\gamma=0.05$ is a small tuning parameter to ensure that the bootstrap factor $\\Delta<1$ in Eq. (3) and have provided ablation study about $\\gamma$. However, this tuning parameter seems to be useless according to Table 1 provided in the authors' rebuttal. As affirmed by the authors, their proposed procedure is insensitive to $\\gamma$. When $\\gamma=1$, $\\Delta$ could always be equal to 1, which completely discards the proposed bootstrap strategy. This raise further concerns on the novelty of this paper.

#### Extension with augmented IPW.
Comparing Fig. 1 in the main manuscript and Fig. 2(a) in the authors' rebuttal, the inclusion of IPW estimator leads to no obvious improvement in performance.
Why is this happening? Contextual information shall allow more precise treatment allocation during the trial and contributes to faster convergence (e.g., high correct selection probability with fewer stages). Therefore, the improvement mentioned in the authors rebuttal doesn't seem to be effective.
Further, this may suggest that the synthetic dataset used in the experiment is inappropriate for serious performance evaluation.
Thank you for adding the literature review, I am maintaining my score
I thank you all for supporting NeurIPS 2024, and I extend my gratitude to the reviewers who have engaged in discussions with the authors. May I kindly remind the reviewers who have not yet responded to the authors’ rebuttals that two days remain to do so. Please review the authors' rebuttal and indicate whether it addresses your concerns and if you will be adjusting your scores. 

AC
Thanks for the clarification, especially on the distinctions between your work and welfare maximization objective. I have no further concerns.
I really appreciate the authors efforts in clarifying my concerns, which are very helpful. Thanks!