Title: Using Surrogates in Covariate-adjusted Response-adaptive Randomization Experiments with Delayed Outcomes
Venue: NeurIPS 2024
Accepted: yes
Keywords: Covariate-adjusted response-adaptive randomization design

Response-adaptive randomization design

Surrogate biomarker

Causal inference
Abstract: Covariate-adjusted response-adaptive randomization (CARA) designs are gaining increasing attention. These designs combine the advantages of randomized experiments with the ability to adaptively revise treatment allocations based on data collected across multiple stages, enhancing estimation efficiency. Yet, CARA designs often assume that primary outcomes are immediately observable, which is not the case in many clinical scenarios where there is a delay in observing primary outcomes. This assumption can lead to significant missingness and inefficient estimation of treatment effects. To tackle this practical challenge, we propose a CARA experimental strategy integrating delayed primary outcomes with immediately observed surrogate outcomes. Surrogate outcomes are intermediate clinical outcomes that are predictive or correlated with the primary outcome of interest. Our design goal is to improve the estimation efficiency of the average treatment effect (ATE) of the primary outcome utilizing surrogate outcomes. From a methodological perspective, our approach offers two benefits: First, we accommodate arm and covariates-dependent delay mechanisms without imposing any parametric modeling assumptions on the distribution of outcomes. Second, when primary outcomes are not fully observed, surrogate outcomes can guide the adaptive treatment allocation rule. From a theoretical standpoint, we prove the semiparametric efficiency bound of estimating ATE under delayed primary outcomes while incorporating surrogate outcomes. We show that the ATE estimator under our proposed design strategy attains this semiparametric efficiency bound and achieves asymptotic normality. Through theoretical investigations and a synthetic HIV study, we show that our design is more efficient than the design without incorporating any surrogate information.

Using Surrogates in Covariate-adjusted Response-adaptive Randomized Experiments with Delayed Outcomes

Abstract

Covariate-adjusted response-adaptive randomization (CARA) designs are gaining increasing attention. These designs combine the advantages of randomized experiments with the ability to adaptively revise treatment allocations based on data collected across multiple stages, enhancing estimation efficiency. Yet, CARA designs often assume that primary outcomes are immediately observable, which is not the case in many clinical scenarios where there is a delay in observing primary outcomes. This assumption can lead to significant missingness and inefficient estimation of treatment effects. To tackle this practical challenge, we propose a CARA experimental strategy integrating delayed primary outcomes with immediately observed surrogate outcomes. Surrogate outcomes are intermediate clinical outcomes that are predictive or correlated with the primary outcome of interest. Our design goal is to improve the estimation efficiency of the average treatment effect (ATE) of the primary outcome utilizing surrogate outcomes. From a methodological perspective, our approach offers two benefits: First, we accommodate arm and covariates-dependent delay mechanisms without imposing any parametric modeling assumptions on the distribution of outcomes. Second, when primary outcomes are not fully observed, surrogate outcomes can guide the adaptive treatment allocation rule. From a theoretical standpoint, we prove the semiparametric efficiency bound of estimating ATE under delayed primary outcomes while incorporating surrogate outcomes. We show that the ATE estimator under our proposed design strategy attains this semiparametric efficiency bound and achieves asymptotic normality. Through theoretical investigations and a synthetic HIV study, we show that our design is more efficient than the design without incorporating any surrogate information.

1 Introduction

1.1 Motivation

In recent years, covariate-adjusted response-adaptive randomization (CARA) designs have become increasingly prominent in clinical research for evaluating the effects of treatment plans or drugs on

primary outcomes of interest [43, 36, 49]. Specifically, CARA designs are experiment strategies that adaptively revise treatment allocation based on observed outcomes and covariate information accumulated during multi-stage clinical trials [22]. Inheriting the advantage of traditional clinical trial designs, CARA designs can also help with providing reliable causal evidence to support clinical discoveries. Nevertheless, different from traditional clinical trial designs, CARA designs offer several major benefits. On the one hand, CARA designs enhance the precision of medicine by improving the statistical power to estimate treatment effects. On the other hand, CARA designs allow for adaptive modifications of treatment allocation based on intermediate findings. As highlighted in the recent Food and Drug Administration (FDA)'s adaptive design clinical trials for drugs and biologics guidance for industry [43], the key advantage of adaptive clinical trial designs is their ability to adapt to new information that emerges during the trial. This fundamental characteristic fosters more flexible and efficient clinical trials. In sum, CARA designs can utilize clinical trial data efficiently by performing sequentially adaptive subgroup-specific optimization.

While CARA designs are beneficial, they face a practical challenge when outcomes are observed with delays. Such delays make it impossible to sequentially revise the randomization probability to optimize trial objectives. Delayed outcomes are common in clinical practice, as treatments may take time to show their effects, especially for chronic conditions or diseases with slow progression. For instance, consider an HIV study conducted in Tanzania that investigates the effect of cash incentives on viral load suppression [15]. The primary outcome is the viral load measured six months after starting antiretroviral therapy (ART) because achieving viral suppression (below 1000 copies per mL) is critical for preventing HIV transmission [33, 31]. However, measuring viral load depends on regular clinic visits by study subjects, which are often delayed due to factors such as limited access to care, poverty, or social stigma [19]. These delays lead to significant missingness in primary outcome data, which undermines the efficiency of estimating treatment effects on the primary outcome.

In situations where primary outcomes are delayed, surrogate outcomes-intermediate or substitute biomarkers that can be measured immediately and are predictive of primary outcomes-emerge as a natural remedy [20, 29, 5]. As mentioned in the FDA's surrogate endpoint resources for drug and biologic development, from 2010 to 2012, the FDA granted approval to 45% of new drugs based on a surrogate endpoint [44]. Using surrogate outcomes provides early indicators of a treatment's efficacy, which accelerates the decision-making process and can lead to faster delivery of beneficial treatments to patients. Surrogate outcomes are particularly useful in CARA designs because they enable modifications of treatment allocation without waiting for delayed primary outcomes during the interim stages.

As such, there is a pressing need to develop new CARA designs to answer the following questions: How can both delayed primary outcomes and immediately observed surrogate outcomes be used simultaneously to guide treatment allocation revisions, optimizing the estimation efficiency of the primary outcome's treatment effect? In this manuscript, we propose a multi-stage CARA design with the goal of improving the estimation efficiency of primary outcome treatment effects by utilizing surrogate outcomes in the presence of delayed primary outcomes.

Our contributions can be summarized as follows:

From a methodological perspective, we propose a novel CARA design strategy that significantly improves the estimation efficiency of the primary outcome's ATE by incorporating surrogate outcomes. First, our approach uniquely integrates both primary and surrogate outcomes synergistically, unlike traditional clinical trial designs that depend exclusively on one or the other. Second, diverging from some adaptive experimental strategies that incorporate both surrogate and primary outcomes, we explore strategies that tackle the arm-dependent delay mechanism of primary outcomes and optimize the statistical efficiency in the presence of temporary or permanent missingness.

From a theoretical perspective, we prove the semiparametric efficiency bound under armdependent delayed primary outcomes while incorporating surrogate outcomes for estimating the ATE (Theorem 3). We show that our proposed design strategy converges to the oracle design, which represents the optimal design assuming perfect knowledge of the data-generating distribution. We also prove the statistical validity of our inference procedure by establishing the asymptotic normality of our proposed ATE estimator (Theorem 1). Furthermore, the proposed ATE estimator attains the semiparametric efficiency bound (Theorem 2), which verifies the efficiency gain of adopting the proposed design. Theorem 1 further provides

practical guidance for the general setup of the design in terms of the number of experimentation stages and the portion of enrolled units within each stage to achieve optimality. Lastly, we illustrate that our design largely enhances statistical efficiency compared to the design that only uses delayed primary outcomes.

1.2 Related literature

CARA designs. When covariate information is available to experimenters, CARA design can utilize both outcome information and covariate information to optimize for the experimental goals [37, 47]. CARA designs originate from response-adaptive randomization designs, yet they further utilize covariate information to assist the design [34, 21, 45]. [22] introduced a family of CARA designs that consider both efficiency and ethics. Generalizations of CARA to incorporate semiparametric estimates are discussed by [49]. Related CARA designs are also explored in studies such as [35, 1, 48]. However, unlike conventional CARA designs that assume immediate observation of outcomes, our approach accommodates delayed outcomes.

Criteria of surrogacy. [32] proposes the statistical surrogacy criterion, which states that the observed primary outcome should be conditionally independent of the assigned treatment given the observed value of the post-treatment variable (surrogate outcome). The proposal is further extended by [17] and [9]. [16] argues that [32]'s proposal does not satisfy the causal necessity properties and proposes to study the principal surrogate. [27] introduces the notion of a strong surrogate using a graphical model, which states that treatment serves as an instrument for the effect of the surrogate on the primary outcome. [12] makes a detailed comparison of different notions of surrogacy. The authors propose (strictly) consistent surrogacy to avoid the surrogacy paradox and evaluate several conditions for the criteria to hold. [24] studies criteria of surrogacy based on distributional average causal effects.

Surrogates in randomized experiments. Existing literature focuses on using surrogates can be roughly divided into two lines of work. The first line of work utilizes surrogates from a data fusion perspective. [4] proposes to use the surrogate index to estimate long-term causal effects when primary outcomes are missing in experimental data and only observed in observational data. This work relies on the statistical surrogacy assumption. [3] proposes a data combination framework to incorporate surrogate outcomes under the assumption of latent unconfoundedness . [23] proposes methods to use surrogate outcomes as proxy variables for persistent confounders. The second line of work uses surrogates as supplements for the primary outcomes. [13] investigates using surrogate data to improve efficiency when inferring parameters from estimating functions. [25] proposes to view surrogates as supplements instead of replacements for the primary outcome and investigates how this proposal can improve the efficiency of treatment effect estimation. [10] proposes to combine primary data and auxiliary data where both datasets are assumed to satisfy ignorability. [2] uses surrogate outcomes to assist with the design of adaptive clinical trials and show their relative and absolute benefits. Diverging from the usage of surrogate outcomes in randomized experiments, we explicitly model the arm and covariates delay mechanisms of primary outcomes while incorporating surrogates.

2 Problem setup

Suppose we are designing an adaptive experiment with a total of T stages. Let X it denote the covariates, S it denote the surrogate outcome, and Y it denote the primary outcome for subject i at stage t = 1 , . . . , T . Let A it denote the treatment assignment status, where A it = 1 corresponds to the treatment arm, while A it = 0 corresponds to the control. Following the Neyman-Rubin causal inference framework [30, 38], we let Y it ( a ) and S it ( a ) represent the primary potential outcome and surrogate potential outcomes under treatment a , where a ∈ { 0 , 1 } . Assume that ( Y it (1) , Y it (0) , S it (1) , S it (0)) are i.i.d. copies of a population tuple ( Y (1) , Y (0) , S (1) , S (0)) . Our parameter of interest is the average treatment effect corresponding to the primary outcome:

Weare in a setting where, at the end of Stage t , we cannot observe the primary outcome for all subjects due to possible delay in the data collection process, but we can observe the surrogate outcomes for all subjects. The surrogate outcomes can inform the primary outcomes. We assume the primary outcome is observed after D it stages and denote the conditional cumulative distribution function of D it as

D it represents the number of stages for which an observation is delayed at stage t . For example, in a two-stage experiment, D i 1 = 0 means no delay for the i -th observation. D i 1 = 1 means that the i -th primary outcome will be observed at stage 2 . When D i 1 = ∞ , we never collect the i -th primary outcome back. A primary outcome can only be observed by the end of the experiment if D it ≤ T -t . Moreover, we allow D it = ∞ , which means that the primary outcome Y it is missing (censored) at the end of the experiment.

The observed primary outcome can be written as

The observed surrogate outcome can be written as S it = A it S it (1) + (1 -A it ) S it (0) . Therefore, our observed data has the following structure:

In our designs, experimenters can sequentially revise the treatment assignment based on the historical data. We define the treatment assignment probability for subjects in stage t as

where H t -1 = { ( X is , A is , D is , S is , Y is ) n s i =1 } t -1 s =1 is the historical information collected up to Stage t -1 . In our manuscript, we consider a multi-stage clinical trial setting where T < ∞ and n t →∞ for t = 1 , . . . , T .

We have the following unconfounded assumption for arm-dependent delayed outcomes: A ⊥ ⊥ ( Y (1) , Y (0) , S (1) , S (0)) | X . We also assume the delay is arm and covariates dependent: D ⊥ ⊥ ( Y (1) , Y (0) , S (1) , S (0)) | X,A . As we are in the randomized experiment setting, the unconfoundedness assumption holds by design. The delay mechanism says that the delay variable is independent of potential primary outcomes and potential surrogate outcomes when conditional one covariates and treatment arms. This rules out the arrow between the delay and the outcome, which simplifies the technical discussion of the paper. Moreover, such a setup aims to reflect real-world randomized experiments where the delay is mainly due to protocol or structural factors instead of the realized outcome (such as the severity of some diseases). For example, if we are testing the effect of a vitamin supplement on blood vitamin levels and ask patients for regular check-ins, the delay is less likely to depend on the outcome since the outcome does not cause severe symptoms that affect patient check-in.

3 Design objective

In this section, we shall formulate our design objective. Our goal of the CARA design is to improve the estimation efficiency of the ATE. Heuristically, we aim to find the optimal treatment allocation that minimizes the semiparametric efficiency bound (SPEB) of estimating the ATE.

The formulation of the design objective relies on the derivation of the SPEB of the ATE under the delayed outcome setting while incorporating surrogate information. In Theorem 3 of the Appendix, we establish the SPEB for estimating τ . If we introduce the following notations on the conditional expectation of the outcomes: τ ( a, x, s ) = E [ Y | A = a, X = x, S = s ] , τ ( a, x ) = E [ τ ( A,X,S ) | A = a, X = x ] , our design objective can be formally defined as follows:

The SPEB is a theoretical result derived from the work of [28]. Our design objective says that we aim to find the sequence of optimal treatment allocation that minimizes the semiparametric efficiency bound. Furthermore, we require that the treatment allocation should be bounded by δ and 1 -δ for

δ ∈ (0 , 1 / 2) . Note that our design objective depends on the conditional expectation of the primary outcome on covariates and surrogates and the delay mechanism. However, in practical settings, both the conditional expectations and the delay mechanisms are unknown to practitioners. Therefore, we propose to use CARA designs that allow us to adaptively learn the unknown parameters based on sequentially collected data. We shall introduce our proposed design strategy in the following section.

4 Proposed CARA design strategy

In this section, we shall formally introduce our proposed design strategy. We consider a delay variable D with finite support: D = { 0 , 1 , . . . , D ∗ } ∪ {∞} , where D ∗ is pre-specified based on domain knowledge. Also, we allow D to take ∞ , meaning the outcome is permanently missing. For simplicity, we assume that the enrollment proportion at each stage is r t = 1 /T , for t = 1 , . . . , T , T < ∞ . We assume both X and S take discrete values to avoid unnecessary technical complexity.

From a high level, our proposal consists of four steps. Step 1 consists of D ⋆ +1 stages and serves as an exploration step that learns parameters like the delay distribution, outcome model, etc. Step 2 is a policy optimization step after Stage D ⋆ +1 , to compute the optimal allocation with the learned parameters. Step 3 is a policy calibration step from Stage D ⋆ + 1 to T , where we calibrate the allocation to match the optimal strategy. Step 4 constructs point and variance estimators based on the data. The full procedure is summarized in Algorithm 1.

Algorithm 1 Surrogate-enhanced adaptive experiment with delayed outcomes

Stage 1 (Initialization) :

1: Enroll n 1 participants, and assign treatments with e 1 = 1 2 ;

2: Estimate ̂ ρ (1) a (0 | x ) and let ̂ ρ (1) a ( l | x ) = ̂ ρ (1) a (0 | x ) , l = 1 , . . . , T -1 . Stage 2 to D ∗ +1 (Learn delay mechanism) :

3: for t → 2 to D ∗ +1 do

4: Enroll n t subjects and assign treatment with probability e t = 1 2 ;

5: Update ̂ ρ ( t ) a ( l | x ) for l ≤ t -1 .

7: Obtain the sequence of optimal treatment allocation ̂ e ( D ∗ +1) by solving (1). Stage D ∗ +2 to T (Calibration of treatment allocation):

6: end for

8: for t → D ∗ +2 to T do

9: Enroll subjects and assign treatments with calibrated probability ˜ e ∗ t .

10: end for After Stage T (Inference):

11: Update ̂ τ and ̂ V .

12: Construct a two-sided α -level confidence interval for ̂ τ .

Below we take a deeper dive into the details and motivations for each step in Algorithm 1.

Stage 1 . In line 1 -2, we initialize the experiment by assigning treatments with probability ̂ e 1 = 1 2 . After Stage 1, we are able to observe all the surrogate outcomes for n 1 subjects and part of the primary outcomes. One can estimate arm-dependent delay with ρ a (0 | x ) by counting the portion of the observed sample:

As we only observe outcomes whose delay variable D i 1 ≤ 0 at the end of Stage 1, we let ̂ ρ (1) a ( l | x ) = ̂ ρ (1) a (0 | x ) , l ≥ 1 . We then estimate ̂ τ (1) ( a, x, s ) and ̂ τ (1) ( a, x ) by taking a sample average of the observed outcome stratified by ( a, x, s ) and ( a, x ) , respectively:

Stage 2 to D ∗ +1 . In line 4 to 5, we keep enrolling subjects and assign subjects to the treatment arm with probability ̂ e t = 1 / 2 . Update the estimates of the delay mechanism by counting portions of the observed sample up to the current stage:

and let ̂ ρ ( t ) a ( l | x ) = ̂ ρ ( t ) a ( t -1 | x ) for l ≥ t . Then update the estimates ̂ τ ( t ) ( a, x, s ) and ̂ τ ( t ) ( a, x ) as the new stratified sample averages:

At the end of Stage D ∗ + 1 , we can obtain estimates of all the delay mechanisms: ̂ ρ ( D ∗ +1) a (0 | x ) , . . . , ̂ ρ ( D ∗ +1) a ( D ∗ | x ) . We then solve the following optimization problem to find the optimal sequence of ˜ e = ( ˜ e 1 , . . . , ˜ e T ) :

In particular, confronted with the possibility that multiple minima could exist for the oracle optimization program, we introduced the penalty term λ N ∥ e ∥ 2 2 , which regularizes the solution. In Theorem 2, we will showcase that, theoretically, if we choose the penalty level to be λ N = C √ log N/N , ˜ e will converge in probability to an oracle minima of the population program. In practice, we propose to choose the tuning parameter based on cross-validation, where we split the collected data into multiple folds to tune the best scaling factor C for λ N = C √ log N/N .

Stage ( D ∗ +2) to T . In line 7 to 9, we calibrate the treatment allocation based on the derived Stage D ∗ +1 optimal treatment allocation strategy. From Stage ( D ∗ +2) to Stage ( T -D ∗ ) , we assign treatments with calibrated probability ̂ e l , where

and ̂ e s is the actual treatment allocation at each Stage s . Then, from Stage s = ( T -D ∗ +1) to T , we assign treatments with probability ̂ e s = ˜ e s , where s = T -D ∗ +1 , . . . , T . After Stage T (line 11 to 12), we update the delay mechanism and the treatment assignment probability using data collected from the whole experiment.

After Stage T . We obtain the final ATE estimate ̂ τ based on the efficient influence curve derived in Theorem 3 as

and the variance estimates as

5 Theoretical investigation

In this section, we investigate the theoretical properties of our proposed design strategy. In Theorem 1, we demonstrate the asymptotic normality of the ATE estimator with any sequence of propensity scores that are generated sequentially from history and converge to some fixed scores. Furthermore, we show that our design strategy asymptotically converges to the oracle design strategy in Theorem 2, which suggests that the constructed estimator is semi-parametrically efficient.

Theorem 1 (Asymptotic normality) . Assume the delays are conditionally independent of the surrogate and potential outcomes, given covariates and historical data. Assume the following conditions hold:

Bounded conditional moments: max { m 4 (0 , s, x ) , m 4 (1 , s, x ) } ≤ M 4 , where

Convergent scores: there exists a set of propensity scores e , such that

Let n t = r t N with r t > 0 , ∑ t r t = 1 . Then when N →∞ , we have

Moreover, the variance estimator satisfies

Theorem 1 is a general theoretical result that works for any sequence of historic-dependent treatment allocation strategy ̂ e , as long as ̂ e converges to an oracle policy in probability asymptotically. For example, one can apply complete randomization through the experiment, which corresponds to ̂ e t, cr = e t, cr = 1 / 2 and establish the asymptotic normality of ̂ τ with variance V ( e cr ) . In particular, if the proposed adaptive designing strategy (Algorithm 1) is implemented, the treatment allocation probability will converge to one set of optimal policies that minimizes the semiparametric efficiency bound (4), which is established in Theorem 2 below.

Theorem 2 (Convergence of the proposed design strategy) . Let n t = r t N with r t > 0 , ∑ t r t = 1 . Assume the bounded conditional moments hold as in Theorem 1. Assume sufficient enrollment at the first ( T -D ⋆ ) (non-delayed) stages:

Set λ N = C √ log N/N . When N →∞ , we have

(b) e -e ∗∗ = o (1) , where e ∗∗ is a minima of L ( e )

Theorem 2 formally established the convergence property of the propensity scores generated by Algorithm 1. More concretely, Theorem 2 showed that two sets of estimated treatment allocation probability converge to the minima of the population optimal variance: one is the scores ˜ e generated by the optimization program (1), and the other is the sequence of propensity scores ̂ e upon calibration which is implemented through the experiment. Therefore, Theorem 1 and Theorem 2 together imply that the point estimator based on the proposed treatment strategy attains the semiparametric efficient bound asymptotically.

Theorem 2 imposes an additional condition (3), which requires a sufficient number of enrollment in the non-delayed stages. This is an important sufficient condition for the optimality of the adaptive allocation strategy, which ensures the optimization program has multiple global minima and enables experimenters to calibrate the propensity scores based on the history to achieve optimality. In practice, Condition (3) can also provide guidance on the design of the experiment to decide the length of the study and the portion of units to recruit in the initial stages.

6 Synthetic case study: HIV trial

In this synthetic case study, we calibrate our data-generating process using data collected from an HIV randomized controlled trial (RCT) conducted in Tanzania [15]. In this study, n = 530 participants are initially enrolled and randomized to one of the three treatment arms: (1) receive no cash, (2) receive 10,000 TZS, and (3) receive 22,500 TZS. In our case study, we combine the latter two arms and define the control arm A = 0 as "receive no cash transfer" and the treatment arm A = 1 as "receive any cash transfer." For the treatment arm, the cash incentives were administered six times from May 21, 2018, to May 31, 2019. In our case study, we consider June 1, 2019, as the starting point of our trial because all the delays observed after June 1, 2019, are not due to the delay in cash transfer administration. After all the cash incentives were administered, n = 511 subjects were still alive on June 1, 2019, with n = 177 subjects in the control arm and n = 334 in the treatment arm. As the very last test was taken on June 29, 2021, we consider June 29, 2021, as the last time point where we gathered outcome data.

Motivated by the original HIV trial, we also consider a multi-stage trial where each stage consists of six months, with the primary outcome Y being the viral load measured at the end of a stage. The viral load can be delayed to be measured because HIV patients may miss their scheduled test appointments. Therefore, we consider the World Health Organization (WHO) stage as the surrogate outcome S ∈ { 1 , 2 , 3 } because the WHO stage is a clinical indicator of HIV infection progression and it is measured without delay, where S = 1 , 2 , 3 indicates the asymptomatic, mild, and advanced symptoms respectively. We use 'sex" as the covariate X in our study, where X = 1 indicates male and X = 0 indicates female. We further observe that the delay mechanism depends on both the treatment arm and the covariate. We summarize the delay mechanism observed from the real data in Table B. We summarize the expectation and standard deviation of the primary outcome under arm A given X and S in Table 2.

In our synthetic data generation, we generate the surrogate outcome from a multinomial distribution. The delay mechanism is also generated following multinomial distribution using the parameters in Table B. We generate X as X it ∼ Bernoulli (0 . 64) , and the primary outcome variable Y as Y it | A it = a, X it = x, S it = s ∼ N ( τ ( a, x, s )) . The true average treatment effect of the primary outcome is τ = 0 . 04 . Following our proposed design strategy, we set the total number of experimental stages as T = 2( D ∗ +1) = 8 . In this case study, We compare three designs: (1) Our proposed design strategy; (2) the complete randomization design which sets e t = 1 / 2 for t = 1 , . . . , T ; (3) CARA design that only utilizes primary outcomes but not surrogate outcomes.

We present the case study results in Figure 1. In terms of bias, Figure 1(A) suggests that the point estimators from all three designs have a vanishing bias as the sample size grows, validating the asymptotic unbiasedness of all strategies. Nevertheless, in terms of variance, Figure 1(B) demonstrates

that our proposed design yields smaller standard deviations and thus higher estimation efficiency for estimating the ATE. The synthetic case study results verify our efficiency comparison in Section 5 and demonstrate the efficiency gain of our proposed design strategy.

(A) √N(IBiasl)

(B) √N(Standard Deviation)

1.50

4.00

1.00

(se!gl)N/)

3.60

√N(SD

0.50

3.20

0.00-

-0.50-

2.80

400

800

1200

1600

400

800

1200

1600

Sample Size

SampleSize

Method

Complete randomization

NoSurrogate

Proposed

7 Discussion

In this work, we proposed a CARA design strategy that improves the efficiency of ATE estimation in the presence of delay by incorporating surrogate outcomes. The strategy leads to a point estimator that achieves semiparametric efficiency under the arm-covariate-dependent delay setting. It also demonstrates efficiency gain compared with the strategy that uses only primary endpoints. There are some future directions to explore. First, we mainly considered an arm-dependent delay setting [18, 41]. A more general setup is to consider the delay mechanism that is also dependent on the outcomes [42]. Another possible framework is to consider outcomes that are also impacted by delays, for which the delay variable serves as a special set of surrogate variables [11]. Moreover, the objective of this work focuses on statistical efficiency gain for ATE estimation. It is also important to incorporate realistic constraints for practical implementation, such as fairness among subgroups [46], safety concerns [6], the discrepancy between algorithmic results and human knowledge [14], etc. More generally, it is also interesting to draw connection to other relevant problems in causal inference and adaptive experiments, such as dealing with many treatment arms [26, 39, 40], targeting other causal parameters [7], incorporating instrumental variables [8], among others. We leave these questions as future research work.

References

Giacomo Aletti, Andrea Ghiglietti, and William F Rosenberger. Nonparametric covariateadjusted response-adaptive design based on a functional urn model. 2018.

Arielle Anderer, Hamsa Bastani, and John Silberholz. Adaptive clinical trial designs with surrogates: When should we bother? Management Science , 68(3):1982-2002, 2022.

Susan Athey, Raj Chetty, and Guido Imbens. Combining experimental and observational data to estimate treatment effects on long term outcomes. arXiv preprint arXiv:2006.09676 , 2020.

Susan Athey, Raj Chetty, Guido W Imbens, and Hyunseung Kang. The surrogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. Technical report, National Bureau of Economic Research, 2019.

David J Benjamin, Vinay Prasad, and Mark P Lythgoe. Fda decisions on new oncological drugs. The Lancet Oncology , 23(5):585-586, 2022.

Andrew Bennett, Dipendra Misra, and Nathan Kallus. Provable safe reinforcement learning with binary feedback. In International Conference on Artificial Intelligence and Statistics , pages 10871-10900. PMLR, 2023.

Aurélien Bibaut, Winston Chou, Simon Ejdemyr, and Nathan Kallus. Learning the covariance of treatment effects across many weak experiments. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 153-162, 2024.

Aurélien Bibaut, Nathan Kallus, and Apoorva Lal. Nonparametric jackknife instrumental variable estimation and confounding robust surrogate indices, 2024.

Marc Buyse and Geert Molenberghs. Criteria for the validation of surrogate endpoints in randomized experiments. Biometrics , pages 1014-1029, 1998.

Hengrui Cai, Wenbin Lu, and Rui Song. Calibrated optimal decision making with multiple data sources and limited outcome. arXiv preprint arXiv:2104.10554 , 2021.

Leonardo Cella and Nicolò Cesa-Bianchi. Stochastic bandits with delay-dependent payoffs. In International Conference on Artificial Intelligence and Statistics , pages 1168-1177. PMLR, 2020.

Hua Chen, Zhi Geng, and Jinzhu Jia. Criteria for surrogate end points. Journal of the Royal Statistical Society Series B: Statistical Methodology , 69(5):919-932, 2007.

Song Xi Chen, Denis HY Leung, and Jing Qin. Improving semiparametric estimation by using surrogate data. Journal of the Royal Statistical Society Series B: Statistical Methodology , 70(4):803-823, 2008.

Changyu Deng, Xunbi Ji, Colton Rainey, Jianyu Zhang, and Wei Lu. Integrating machine learning with human knowledge. Iscience , 23(11), 2020.

Carolyn A Fahey, Prosper F Njau, Emmanuel Katabaro, Rashid S Mfaume, Nzovu Ulenga, Natalino Mwenda, Patrick T Bradshaw, William H Dow, Nancy S Padian, Nicholas P Jewell, et al. Financial incentives to promote retention in care and viral suppression in adults with hiv initiating antiretroviral therapy in tanzania: a three-arm randomised controlled trial. The Lancet HIV , 7(11):e762-e771, 2020.

Constantine E Frangakis and Donald B Rubin. Principal stratification in causal inference. Biometrics , 58(1):21-29, 2002.

Laurence S Freedman, Barry I Graubard, and Arthur Schatzkin. Statistical validation of intermediate endpoints for chronic diseases. Statistics in medicine , 11(2):167-178, 1992.

Manegueu Anne Gael, Claire Vernade, Alexandra Carpentier, and Michal Valko. Stochastic bandits with arm-dependent delays. In International Conference on Machine Learning , pages 3348-3356. PMLR, 2020.

Tessa Heestermans, Joyce L Browne, Susan C Aitken, Sigrid C Vervoort, and Kerstin KlipsteinGrobusch. Determinants of adherence to antiretroviral therapy among hiv-positive adults in sub-saharan africa: a systematic review. BMJ global health , 1(4):e000125, 2016.

Spencer Phillips Hey, Aaron S Kesselheim, Pranav Patel, Preeti Mehrotra, and John H Powers. Us food and drug administration recommendations on the use of surrogate measures as end points in new anti-infective drug approvals. JAMA internal medicine , 180(1):131-138, 2020.

Feifang Hu and William F Rosenberger. Optimality, variability, power: evaluating responseadaptive randomization procedures for treatment comparisons. Journal of the American Statistical Association , 98(463):671-678, 2003.

Jianhua Hu, Hongjian Zhu, and Feifang Hu. A unified family of covariate-adjusted responseadaptive designs based on efficiency and ethics. Journal of the American Statistical Association , 110(509):357-367, 2015.

Guido Imbens, Nathan Kallus, Xiaojie Mao, and Yuhao Wang. Long-term causal inference under persistent confounding via data combination. arXiv preprint arXiv:2202.07234 , 2022.

Chuan Ju and Zhi Geng. Criteria for surrogate end points based on causal distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology , 72(1):129-142, 2010.

Nathan Kallus and Xiaojie Mao. On the role of surrogates in the efficient estimation of treatment effects with limited outcome data. arXiv preprint arXiv:2003.12408 , 2020.

Anand Kalvit and Assaf Zeevi. From finite to countable-armed bandits. Advances in Neural Information Processing Systems , 33:8259-8269, 2020.

Steffen L Lauritzen, Odd O Aalen, Donald B Rubin, and Elja Arjas. Discussion on causality [with reply]. Scandinavian Journal of Statistics , 31(2):189-201, 2004.

Xinwei Ma, Waverly Wei, and Jingshen Wang. Covariate adjusted response adaptive design with delayed outcomes. Forthcoming, 2024.

Anthony Muchai Manyara, Philippa Davies, Derek Stewart, Christopher J Weir, Amber E Young, Valerie Wells, Jane Blazeby, Nancy J Butcher, Sylwia Bujkiewicz, An-Wen Chan, et al. Definitions, acceptability, limitations, and guidance in the use and reporting of surrogate endpoints in trials: A scoping review. Journal of Clinical Epidemiology , 2023.

J. Neyman. On the application of probability theory to agricultural experiments. essay on principles. section 9. Statistical Science , pages 465-472, 1923/1990.

Joint United Nations Programme on HIV/AIDS et al. Understanding fast-track: accelerating action to end the aids epidemic by 2030. 2015.

Ross L Prentice. Surrogate endpoints in clinical trials: definition and operational criteria. Statistics in medicine , 8(4):431-440, 1989.

Angela Ramadhani, Robert M Josiah, Anath Rwebembera, Emma Lekashingo Msuya, Roland Swai, Peris Urassa, Samuel Kalluvya, Salehe Omari, Josephine Kaganda, Ayoub Kibao, et al. National guidelines for the management of hiv and aids, 2012.

William F Rosenberger and John M Lachin. Randomization in clinical trials: theory and practice . John Wiley & Sons, 2015.

William F Rosenberger and Oleksandr Sverdlov. Handling covariates in the design of clinical trials. 2008.

William F Rosenberger, Oleksandr Sverdlov, and Feifang Hu. Adaptive randomization for clinical trials. Journal of biopharmaceutical statistics , 22(4):719-736, 2012.

William F Rosenberger, AN Vidyashankar, and Deepak K Agarwal. Covariate-adjusted responseadaptive designs for binary response. Journal of biopharmaceutical statistics , 11(4):227-236, 2001.

Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology , 66(5):688, 1974.

Lei Shi and Peng Ding. Berry-esseen bounds for design-based causal inference with possibly diverging treatment levels and varying group sizes. arXiv preprint arXiv:2209.12345 , 2022.

Lei Shi and Xinran Li. Some theoretical foundations for the design and analysis of randomized experiments. arXiv preprint arXiv:2406.10444 , 2024.

Lei Shi, Jingshen Wang, and Tianhao Wu. Statistical inference on multi-armed bandits with delayed feedback. In International Conference on Machine Learning , pages 31328-31352. PMLR, 2023.

Yifu Tang, Yingfei Wang, and Zeyu Zheng. Stochastic multi-armed bandits with strongly reward-dependent delays. In International Conference on Artificial Intelligence and Statistics , pages 3043-3051. PMLR, 2024.

U.S. Food and Drug Administration. Adaptive design clinical trials for drugs and biologics guidance for industry, 2019.

U.S. Food and Drug Administration. Surrogate endpoint resources for drug and biologic development, 2022.

Sofía S Villar, Jack Bowden, and James Wason. Response-adaptive designs for binary responses: how to offer patient benefit while being robust to time trends? Pharmaceutical statistics , 17(2):182-197, 2018.

Waverly Wei, Xinwei Ma, and Jingshen Wang. Fair adaptive experiments. Advances in Neural Information Processing Systems , 36, 2024.

Li-Xin Zhang, Feifang Hu, Siu Hung Cheung, and Wai Sum Chan. Asymptotic properties of covariate-adjusted response-adaptive designs. 2007.

Wanying Zhao, Wei Ma, Fan Wang, and Feifang Hu. Incorporating covariates information in adaptive clinical trials for precision medicine. Pharmaceutical Statistics , 21(1):176-195, 2022.

Hai Zhu and Hongjian Zhu. Covariate-adjusted response-adaptive designs based on semiparametric approaches. Biometrics , 79(4):2895-2906, 2023.

Technical Appendix

A Additional results and theoretical proofs

A.1 Derivation of the semi-parametric efficiency bound

We use R i ∈ { 1 , . . . , T } to record the stage at which the subject i is enrolled, such that P ( R i = t ) = r t , ∑ T t =1 r t = 1 .

Theorem 3. Under the unconfoundedness assumption and the arm-dependent delay assumption: (a) The efficient influence function for estimating τ is

(b) The semiparametric efficiency bond for estimating τ is

Proof. (a) Proof of Efficient influence function

Step 1. First, we write out the joint density the observed sample as f η ( x, a, d, v, s, y ) , indexed by η .

Step 2. Next, we derive the tangent spaces. We use f ( · ) , g ( · ) , e t ( · ) , p t ( · ) to denote ∂ ∂η log f ( · ) , ∂ ∂η log g ( · ) , ∂ ∂η log e ( · ) , and ∂ ∂η log ρ ( · ) .

T

Therefore, the tangent space is

Step 3. We consider the estimation of the target parameter E [ Y (1)] via a parametric submodel indexed by η . Under identifiability assumptions, the target parameter can be written as

Taking pathwise derivative of E η [ Y (1)] at η = 0 , we have

Let s ( X,A,D,V,S,Y ) denote the score function of f ( x, a, d, v, s, y ) .

where τ (1 , S, X ) = E [ Y | A = 1 , V = t, D ≤ T -t, S, X ] .

As a result,

where

We obtain

Step 4. Now we project the score functions to the corresponding tangent spaces. First, we project φ 1 ( X ) onto T X .

Next, we project φ 2 ( A,S,X,V ) onto T S (1) .

Then,

Because we obtain

Lastly, we project φ 3 ( X ) onto T Y (1) .

Because

we obtain

In sum, the efficient influence function is

(b) Proof of semiparametric efficiency bound

Proof.

We work with each term separately.

Therefore,

,

A.2 Proof of Theorem 1

Proof. Our proposed ATE estimator is

Denote

Step 0. A basic decomposition.

Note that for ̂ τ 2 , we have a similar decomposition; we omit the presentation for conciseness.

Step 1. Asymptotics for the nuisance terms.

Step 1.1. Convergence of weighted propensity scores. Under Condition (2), we prove the following results:

WLOG, we only prove the first one.

By Condition (2), we have

Meanwhile,

The summands in the outer layer, indexed by the t , form a martingale difference sequence. Hence, its variance is given by

Therefore, we have

Step 1.2. Convergence of the mean functionals. In this part, we prove the following results:

WLOG, we check the first result.

The numerator forms a martingale difference triangular array:

For convenience we denote F ( n ) 0 s = F ( n ) n s -1 ( s -1) , then we can verify that

The variance of the numerator is then given by

Therefore,

For the denominator, similarly, we can show that

Therefore,

Step 1.3. Concentration of propensity weights.

We prove that

WLOG, we prove the first result. We can compute

Step 2. Approximate the main terms. We prove that the sum of ( A. 1) and ( B. 1) can be approximated by the following:

and

The first approximation can be done by simply noticing that

The second approximation can be done similarly.

Step 3. Compute the variance of the main part.

By Step 0-2, we have the following approximation for ̂ τ -τ :

where

We can verify that the variance of ̂ τ 1 , apr -̂ τ 0 , apr is given by the following form:

To sum up,

Step 4. Prove the asymptotic normality of the approximation ̂ τ apr upon standardization. We check that the Lyapunov condition holds:

Similarly, we have

and

Therefore, we have proved that the Lyapunov condition holds and

Step 5. Prove the convergence of the variance estimator.

First, we can prove that

The results follow from the martingale structure and the approximation given in Steps 1.1 and 1.2. Analogously, with similar ways of decomposition, we can prove that

Moreover, we have

Summarizing the above parts, we can conclude

A.3 Proof of Theorem 2

Proof. The proof is delivered in two steps:

Step 1. A sufficient condition for the existence of non-trivial oracle propensity scores in the non-delay stages.

For the oracle optimization problem 3, by considering covariates X and S with finite discrete support values, we are essentially solving the following program:

For simplicity, we can omit the dependence on x and s values and focus instead on the following:

We want to compute how many stages it takes to obtain a nontrivial solution for the adaptive experiment. The last D ∗ stages will just come with delay; the first T -( D ∗ +1) stages will have ρ 1 ( T -t ) = ρ 1 , ∞ and ρ 0 ( T -t ) = ρ 0 , ∞ . We can condense the first T -( D ∗ +1) variables as a single one by representing it as an average e :

We want to introduce a solution for e that is bounded away from the boundary. By KKT conditions, for the optimal solution variable e ⋆ , we have

Note that the derivative function ∂ L /∂e is continuous and monotonically increasing in e . We can upper bound the minimum value of ∂ L /∂e and set it to negative:

Denote which further simplifies to

Similarly and symmetrically, we can lower bound the maximal value of the derivative and set it to positive, which gives

Summarizing both parts, we obtain

Step 2. Convergence of the empirical objective.

We prove that the empirical version of the objective function converges uniformly to the population program:

For the first D ∗ +1 stages, because the data collection policy is complete randomization with equal probability, the estimates for conditional variances, conditional means, and delay distribution are all √ N -consistent. Noting that the denominators of ̂ L and L are strictly bounded away from 0 and 1, (5) can be verified.

Step 3. Convergence of the solution. Using the fact that ˜ e N minimizes the empirical objective, we have

Then we have

where the last inequality follows that e ∗ minimizes L ( e ) .

Meanwhile, using

we have

Here R f stands for the portion of the whole sample before and including stage T -D ∗ , and R d stands for the portion beyond stage T -D ∗ . After simplification, we obtain

To sum up, we have proved

Therefore,

Now letting N →∞ , we know that

Hence we can conclude that

On the other hand,

(6) states that the distance between the ˜ e N and the Euclidean ball B ⋆ = { e : ∥ e ∥ 2 = ∥ e ⋆ ∥ 2 } converges to zero: d ( ˜ e N , B ⋆ ) P - → 0 . (7) states that the distance between ˜ e N and the minimal point set M ⋆ converges to zero: d ( ˜ e N , M ⋆ ) P - → 0 . But, B ⋆ ∩ M ⋆ = { e ⋆ } . Therefore it must be that

˜

Step 4. Convergence of the implemented propensity scores.

The rest of the result, ̂ e P - → e ∗∗ , is now a direct result from Step 2 and Step 3.

B Synthetic case study details

In this section, we provide the true parameters generated from the real data adopted in our synthetic case study. The parameters are summarized in Tables B and 2.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Please see Section 1.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please see Section 7.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Please see Section 5.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please see Section 6 and Supplementary Materials.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide open access to the code in the Supplementary Materials. However, we are not able to provide open access to the data as the HIV dataset is not publicly available.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please see Section 6 and Supplementary Materials.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Please see Section 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The simulation studies require approximately 160 minutes to complete when running on a MacBook Pro equipped with an 8-core CPU.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Please see Section 7.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The creators or original owners of the original dataset used in the paper have been properly credited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [Yes]

Justification: We have included the details related to the original HIV trial. Please see Section 6.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [Yes]

Justification: We have obtained IRB approvals when obtaining access to the original data.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nClinical trials try to achieve the highest statistical precision using the fewest number of enrolled participants. One way to do this is by assigning more participants to the {covariate, arm} combinations with the highest outcome variance. We don't know the outcome variance before running the trial, which motivates the adaptive CARA method to adaptively assign more units with high outcome variance to treatment. This paper considers the setting where we don't directly observe the outcome, but instead observe a noisy surrogate along with a delayed outcome. The paper proposes a design objective (minimizing the semiparametric efficiency bound) and an explore-then-commit style algorithm for learning then applying a covariate-adaptive randomization using surrogate outcomes. The paper shows that this algorithm attains the semiparametric efficiency bound, and provides synthetic experiments based on an HIV study comparing this algorithm to complete randomization and CARA on observed outcomes only.\n\nSTRENGTHS:\nI am not very familiar with real-world clinical trials. However, from what I can tell, the work is well-grounded in problems that clinical trial designers care about. The mathematical model of surrogacy and delay is simple to state and understand while capturing the important aspects of the problem. The problem is explained clearly with explicit examples.\n\nWEAKNESSES:\nThe empirical results are missing error bars. These should be added to show variation in the results across many replicates. This is especially important for two reasons: (a) to support the claim that the proposed method has lower bias than complete randomization, when the two are extremely close on the plots, and (b) to understand whether the \"no surrogate\" method is really worse in both bias and variance than complete randomization, as suggested by figure 2. If the \"no surrogate\" method is actually worse than complete randomization, it would be interesting to have a comment on this in the text.\n\nQUESTIONS:\nI had trouble understanding the setup of the real-world study. If I understand correctly, measurements are taken every 6 months. The actual outcome Y is viral load (measured every 6 months), while the surrogate outcome is WHO stage (also measured every 6 months). I'm confused by the claim that WHO stage is available immediately, which I took to mean that the experimenters receive the WHO stage for every patient every 6 months. Does that not require the patient to visit a clinic, which might also lead to missingness? This isn't a question you need to address in your response, but if I have misunderstood something it might be worth clarifying.\n\nRequest: Neurips style is for citations to be by (Author, Year) instead of numerical. Please fix this in the next revision.\n\nThe introduction states that \"we allow D_it = infty\" (Line 139), but the algorithm only works with finite, known upper bound on D. It would be helpful for the authors to address the sensitivity of their results to misspecification of D*.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis article studies how to use interminate surrogate outcomes to estimate causal effects when the primary outcome is delayed in clinical trials. The author first proposes a novel Covariate-adjusted Response-adaptive (CARA)  design that supports efficient estimation of ATE using both surrogate and primary outcomes.  The authors prove the semiparametric efficiency bound for their ATE estimator and demonstrate the proposed design's efficiency through theoretical analyses and a synthetic HIV study.\n\nSTRENGTHS:\n1. The delay outcome problem studied in this paper is well-motivated. The covariate-adjusted response-adaptive (CARA)  design allows us to modify the treatment allocation mechanism over time, providing an interesting scenario to address the problem. \n2. The authors consider improving efficiency in both design and estimation. They show that the proposed design strategy converges to an oracle design and proposes a semiparametric efficient estimator of the ATE.\n3. The paper includes a synthetic HIV trial to illustrate the practical application and efficiency gains of the proposed design. The study demonstrates that the design reduces the standard deviation in estimating the ATE compared to other methods.\n\nWEAKNESSES:\nThe main weakness of this article is that the presentation is too messy and lacks explanations in multiple places. \n1. In section 3, from line 159 to 179, the author introduces the efficient influence function (EIF) and variance bound in estimating $\\tau$. However, it seems like the main motivation of CARA is that the expectation of the delay outcomes and the delay mechanisms are unknown.   It is unclear why EIF and variance bound are presented at the beginning of section 3.\n\n2. Section 4 is also presented in a poor way. The authors go through the design steps with dense notation. The authors should provide more explanation of the steps taken in the design. The current Section 4 only shows that the design allows us to estimate the unknown quantity. But it is unclear how the design improves efficiency.\n\nQUESTIONS:\nN.A.\n\nLIMITATIONS:\nN.A.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an approach to optimizing treatment allocation for variance reduction, in the context of adaptive clinical trials.  The particular setting is one where there are delays in observing outcomes. These delays that are independent of the outcomes themselves, but they impact efficiency of estimation.  Moreover, there are short-term surrogate variables that can be used to improve estimation efficiency.  In this setting, this paper proposes to estimate the variance of different treatment allocation schemes from an initial stage of experimentation, and then use those estimates to choose an optimal treatment allocation scheme for the remainder of experimentation.\n\n---------------\nREBUTTAL UPDATE: I have updated my score from 5->6\n\nSTRENGTHS:\nOverall, I found the setup of the problem to be original, and while I did not have time to review the proofs in depth, the results seems correct based on my knowledge of semiparametrics.  The theoretical results are fairly thorough: For instance, I appreciated Theorem 3, which speaks to the convergence of the chosen strategy to the optimal one.  In terms of clarity, the presentation of the technical details is similarly thorough, and while a bit dense and harder to skim, I found it straightforward to follow as written.  I also appreciated the experimental results, which I found to be reasonably compelling.\n\nWEAKNESSES:\nI noted a few areas of the paper that appeared a bit weaker, or at least warrant some clarification.  I'll number my points to make it easier to respond during the rebuttal period, and put them in rough order of priority.\n\n(W1) Significance / Novelty of Theorem 1: I understood Theorem 1 to be a necessary pre-cursor to the proposed adaptive strategy, but not the main contribution of the paper, so this concern is not as pressing as it could be.  However, it seems that Theorem 1 is essentially the same as Theorem 2.1 of Kallus & Mao 2020 [1], with the minor complication that different observations appear at different times, which leads to some additional notation to ensure the correct $e_{t}(X)$ is used in the relevant denominators.  Since the paper cites [1] (as [23] in the paper), it may be worth clarifying the similarity in the main text, and framing the contributions appropriately (e.g., my read is that Theorem 2 and 3 are the novel theoretical thrust of the paper).  I'm open to clarifications / corrections on this front.\n\n[1] https://arxiv.org/abs/2003.12408\n\n(W2) Significance of the proposed problem setting: From a practical perspective, I struggle to think of a setting where we would expect the causal graph in Figure 1 to hold, where the delay itself provides no information on the outcome $Y$, i.e., delays are not informative. E.g., in the HIV application, the outcome is viral load, and delay is in coming in for a visit (Lines 274-275).  It seems to be a classic case where delay could be caused by more severe illness, giving us some indication of $Y$.\n\n(W3) I found it slightly hard to parse the optimality claim (Theorems 2 and 3 and subsequent discussion).  It's clear that the estimated treatment allocation converges in probability to the optimal treatment allocation, but does that imply that this approach yields the optimal variance?  Could there be other ways to estimate the treatment allocation that also converges to the optimal one, but has better variance properties along the way, or is that somehow ruled out?\n\n(W4) Overall, I would have preferred to see more discussion of intuition and insights, rather than just going through the technical steps to derive the results.  It seemed that some parts could be made shorter to make space for this, e.g., Section 4 could be trimmed down a bit (e.g., define $\\hat{\\tau}^t(a, x, s), \\hat{\\tau}^t(a, x)$ and then use that notation in both Stage 1 and Stage 2 to $D^*+1$, or even just give the high-level comment that you estimate via empirical counts among the population where $Y$ is observed.)\n\nIn terms of missing intuition, I still lack some intuition for the \"signal\" being used in the optimization of the treatment allocation, and why we should expect it to vary across time periods.  I suppose the variance reduction comes from some understanding of which combinations of $X, A$ are likely to have missing data by the end due to delay, and so should be prioritized earlier in the treatment allocation process?\n\n(W5) There is a slight mismatch between the discussion in the introduction and the actual setting where the algorithm can be applied, particularly the restriction that the delay must be upper-bounded (see 181-182).  In contrast, Line 75-76 refers to the contribution \"optimize the statistical efficiency in the presence of temporary or permanent missingness\", where I would have interpreted \"permanent missingness\" to be $D = \\infty$. \n\n(W6) Interaction (or lack thereof) between delay and surrogates: This is a subjective / aesthetic point, so I place it last, but I was expecting more of a nuanced interaction between the delay and the surrogates.  In particular, the problem seems to factorize cleanly into (a) deriving an efficient estimator and variance lower-bound for any treatment allocation, (b) optimizing the allocation in a backward-looking period of data, and then (c) applying that allocation to the next period of data.  It's not clear the role of surrogates in this recipe, other than the influence they have on (a), because observing surrogates gives you no insight into future delays and therefore the optimal allocation going forward.\n\nMinor nits:\n* Algorithm 1 mentions \"Problem B\" on line 7, but I think that's really just Equation (2), Page 6, line 199.\n\nQUESTIONS:\n1. (W1) Could you clarify the similarity/difference of Theorem 1 to Theorem 2.1 of Kallus & Mao 2020?  It's fine if they are essentially the same, in my view, b/c the main contribution of the paper seems to lie in Theorems 2 & 3, but I just wanted to get some clarification in case I'm missing something.\n2. (W2) Is there a real-world scenario you have in mind where you would expect this causal structure to hold (i.e., delays are not informative of outcomes)?\n3. (W3) Could you clarify the optimality claim?  Is it just that the allocation converges in probability to the optimal allocation?  Or is there a stronger claim being made about the optimality of the procedure which uses the estimated allocation?\n4. (W4) Is my intuition correct, for where the variance reduction is coming from?  Or do you have other intuition on this point?\n\nThe authors are welcome to react to my points (W5) and (W6), but I don't think they need to do so, and they factored less into my score.\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of covariate adaptive experimental design in the presence of time delayed outcomes. More specifically, we assume that participants are enrolled in waves, and the probability of treatment is given conditional on available user covariates. The target estimand is the average treatment effect for a long term/delayed outcome. To circumvent the issue of covariate adaptive randomization in the absence of observed outcomes, the authors consider a surrogate metrics. To derive the assignment procedure, the authors derive the influence function and semi-parametric efficiency bound, and then derive an objective based on them. A small number of empirical evaluations is provided which show the improvement in variance of the proposed estimator over complete randomization and approaches which do not incorporate surrogate information.\n\nSTRENGTHS:\nThis is an interesting addition to the literature on adaptive experimentation, and covariate adaptive experimentation. The authors do a nice job of clearly describing the task, motivating and deriving the influence function and semiparametric efficiency bound, and introducing a relatively simple algorithm for optimizing the bound. The proofs, to my reading, are correct and well presented. The problem is of clear practical importance, the authors motivate their approach through drug trials, but there are also a number of applications in both social scientific and industrial settings where the problem of delayed and long term outcomes arises and adaptive design is desirable. The authors also do a nice job of clearly walking through each step of the proposed algorithm and describing it's function and intuition.\n\nWEAKNESSES:\nThe paper focuses on asymptotic results, which is sensible and provides good empirical performance. However, it would be useful to have finite sample analysis as well, since many of the applications of experimental design are sample-starved. It would have also been nice to have seen a slightly larger set of empirical results. The authors provide a small demonstration, but ideally the behavior of the propose algorithm is more rigorously evaluated empirically.\n\nQUESTIONS:\nIt wasn't clear to me what is being assumed about the size of each wave of participants. I assume that the authors are assuming some conditions on the number of arriving participants? Also that these participants are arriving i.i.d. over time?  Does the proposed procedure give any guidance to the setting where the experimenter can choose the number of participants to enroll at each stage?\n\nLIMITATIONS:\nYes. See above for questions.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper considers the covariate-adjusted response-adaptive randomization design for settings with delayed outcomes and surrogate outcomes. The paper first characterizes the efficient influence function for estimating the primary outcome under the delayed setting with surrogate outcomes. They then characterize the semiparametric efficiency bound of the estimate. Using the semi parametric efficiency bound they devise an optimization approach to construct an adaptive randomization design that optimizes the semi-parametric efficient bound. They provide a synthetic case study to demonstrate the effectiveness of their approach.\n\nSTRENGTHS:\nThe paper overall feels well written and well organized. The problem set-up as well as the key results follow a logical flow and it is straightforward to understand how the adaptive randomization proposed in the paper is derived. The authors also provide insightful comments on the results in the paper which helped point out its main contributions compared to existing work. \n\nThe paper's contributions are interesting as their adaptive randomization design in the delayed outcome setting with surrogate outcomes is a realistic setting seen in clinical trials performed by the FDA. The synthetic case study helps highlight the potential application of their work. They also provide theoretical results that give guarantees on the quality of the estimation approach\n\nWEAKNESSES:\nThe paper leaves out some details which may be obvious but are worth including for clarity. For example, the unconfoundedness and arm-dependent delay assumptions are briefly mentioned in passing, but are not formally defined even though they are used in the statements of the theorems. The authors also do not comment on the tractability of optimization problem that determines the treatment allocation probabilities. Other details omitted include details of the cross-validation set-up used to tune the regularization penalty parameter and the the details of the \"No Surrogate\" approach in the case study. \n\nDiscussion on the settings where the proposed method has the most benefit is also lacking. The numeric case study primarily studies the impact on sample size on different approaches for estimating the primary outcome. The results show that while there is benefit from using the proposed method, the most naive method with complete randomization also performs better than other adaptive randomization approaches. Adaptive randomization design may be more challenging to implement so a deeper study on settings that benefit the most from the proposed method would shed more light on the benefits and robustness of the proposed approach.\n\nQUESTIONS:\n1. In the numerics, the \"No Surrogate\" approach seems to perform worse than complete randomization. Is there any insight on why this adaptive approach performs worse than the complete randomization approach? Does the \"No Surrogate\" approach also account for delays or does it utilize an existing adaptive randomization design approach?\n\n2. The proposed method suggests applying regularization to the oracle optimization program to deal with potential multiple optima. Does the procedure tune the regularization parameter so it produces calibrated treatment allocation probabilities with better estimation error? Does this align cross-validation set-up briefly mentioned in the paper? If so, what is the exact cross-validation set-up and if not how is the the tuning parameter chosen?\n\nLIMITATIONS:\nThe authors have adequately addressed limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-17"}