Using Surrogates in Covariate-adjusted Response-adaptive Randomization Experiments with Delayed Outcomes
Authors: Lei Shi
Waverly Wei
Jingshen Wang
Venue: NeurIPS 2024 poster (2024)
Keywords: Covariate-adjusted response-adaptive randomization design
Response-adaptive randomization design
Surrogate biomarker
Causal inference
Primary area: causal_inference

Abstract: Covariate-adjusted response-adaptive randomization (CARA) designs are gaining increasing attention. These designs combine the advantages of randomized experiments with the ability to adaptively revise treatment allocations based on data collected across multiple stages, enhancing estimation efficiency. Yet, CARA designs often assume that primary outcomes are immediately observable, which is not the case in many clinical scenarios where there is a delay in observing primary outcomes. This assumption can lead to significant missingness and inefficient estimation of treatment effects. To tackle this practical challenge, we propose a CARA experimental strategy integrating delayed primary outcomes with immediately observed surrogate outcomes. Surrogate outcomes are intermediate clinical outcomes that are predictive or correlated with the primary outcome of interest. Our design goal is to improve the estimation efficiency of the average treatment effect (ATE) of the primary outcome utilizing surrogate outcomes. From a methodological perspective, our approach offers two benefits: First, we accommodate arm and covariates-dependent delay mechanisms without imposing any parametric modeling assumptions on the distribution of outcomes. Second, when primary outcomes are not fully observed, surrogate outcomes can guide the adaptive treatment allocation rule. From a theoretical standpoint, we prove the semiparametric efficiency bound of estimating ATE under delayed primary outcomes while incorporating surrogate outcomes. We show that the ATE estimator under our proposed design strategy attains this semiparametric efficiency bound and achieves asymptotic normality. Through theoretical investigations and a synthetic HIV study, we show that our design is more efficient than the design without incorporating any surrogate information.

Using Surrogates in Covariate-adjusted Response-adaptive Randomized Experiments with Delayed Outcomes
Lei Shi Division of Biostatistics University of California, Berkeley 2121 Berkeley Way, Berkeley, CA 94704 leishi @eerheeeeeeeeyyy. edu
Waverly Wei
Marshall School of Business University of Southern California 3670 Trousdale Pkwy, Los Angeles, CA 90089 haverly@marshal usc. edu
Jingshen Wang Division of Biostatistics
2121 Berkeley Way, Berkeley, CA 94704 j ingshenaang@berrxleeyycco edu
Abstract
adjuueee response-adaptive randomization (CARA) designs gain- ing increasing attention. These designs combine the advantages of randomized experiments with the ability adaptively revise treatment allocations based data collected across multiple stages, enhancing estimation efficiency. Yet, CARA designs often assume that primary outcomes are immediately observable, which is not the case in many clinical scenarios where there is a delay in observing primary outcomes. This assumption can lead significant missingness and inefficient estimation of treatment effects. To tackle this practical challenge, we propose a CARA experimental strategy integrating delayed primary outcomes with immedi- ately observed surrogate outcomes. Surrogate outcomes are intermediate clinical outcomes that are predictive or correlated with the primary outcome of interest. Our design goal is to improve the estimation efficiency of the average treatment effect (ATE) of the primary outcome utilizing surrogate outcomes. From method- ological perspective, our approach offers two benefits: First, we accommodate arm and deepritte-dependeent delay mechanisms without imposing any parametric modeling assumptions the distribution of outcomes. Second, when primary outcomes are not fully observed, surrogate outcomes can guide the adaptive treat- ment allocation rule. From theoretical standpoint, efficiency bound of estimating ATE under delayed primary outcomes while incor- porating surrogate outcomes. We show that the ATE estimator under proposed design strategy attains this semiparametric efficiency bound and achieves asymp- totic normality. Through theoretical investigation and synthetic HIV study, we show that our design is more efficient than the design without incorporating any surrogate information.
prove the semiparametric
Introduction
1.1 Motivation
In recent years, covariate-adiee adjussed issponse-adppiin randomization (CARA) designs have become increasingly prominent in clinical research for evaluating the effects of treatment plans or drugs on
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

primary outcomes of interest 43 36 49).
Specifically, CARA designs are experiment strategies that adaptively revise treatment allocation based on observed outcomes and covariate information accumulated during multi-stage clinical trials [22]. Inheriting the advantage of traditional clinical trial designs. CARA designs can also help with providing reliable causal evidence to support clinical discoveries. Nevertheless, different from traditional clinical trial designs, CARA designs offer several major benefits. On the one hand, CARA designs enhance the precision of medicine by improving the statistical power to estimate treatment effects. On the other hand, CARA designs allow for adaptive modifications of treatment allocation based on intermediate findings. As highlighted in the recent Food and Drug Administration (FDA)'s 'ss adaptive design clinical trials for drugs and biologics guidance for industry (43), the key advantage of adaptive clinical trial designs is their ability to adapt to new information that emerges during the trial. This fundamental characteristic fosters more flexible and efficient clinical trials. In sum, CARA designs can utilize clinical trial data efficiently by performing sequentially adaptive subgroup-specific optimization.
While CARA designs are beneficial, they face a practical challenge when outcomes are observed with delays. Such delays make it impossible to sequentially revise the randomization probability to optimize trial objectives. Delayed outcomes are common in clinical practice, as treatments may take time to show their effects, especially for chronic conditions or diseases with slow progression. For instance, consider an HIV study conducted in Tanzania that investigates the effect of cash incentives on viral load suppression [15). The primary outcome is the viral load measured six months after starting antiretroviral therapy (ART) because achieving viral suppression (below 1000 copies per mL) is critical for preventing HIV transmission [33 31 However, measuring viral load depends on regular clinic visits by study subjects, which are often delayed due to factors such as limited access to care, poverty, social stigma [19]. These delays lead to significant missingness in primary outcome data, which undermines the efficiency of estimating treatment effects on the primary outcome.
In situations where primary outcomes are delayed, surrogate outcomes-intermediin or substitute biomarkers that can be measured immediately and are predictive of primary outeomes-emerge as a natural remedy [00 29 5 As mentioned in the FDA's surrogate endpoint resources for drug and biologic development, from 2010 to 2012, the FDA granted approval to 45% of new drugs based on surrogate endpoint [44). Using surrogate outcomes provides early indicators of treatment's efficacy, which accelerates the decision maasing process and can lead to faster delivery of beneficial treatments patients. Surrogate outcomes are particularly useful in CARA designs because they enable modifications of treatment allocation without waiting for delayed primary outcomes during the interim stages.
As such, there is a pressing need to develop new CARA designs to answer the following questions: How can both delayed primary outcomes and immediately observed surrogate outcomes be used simultaneously to guide treatment allocation revisions, optimizing the estimation efficiency of the primary outcome's treatment effect? In this manuscript, propose multi-stage CARA design with the goal of improving the estimation efficiency of primary outcome treatment effects by utilizing surrogate outcomes in the presence of delayed primary outcomes.
Our contributions can be summarized as follows:
1. From methodological perspective, we propose a novel CARA design strategy that signifi- cantly improves the estimation efficiency of the primary outcome's ATE by incorporating surrogate outcomes. First, our approach uniquely integrates both primary and surrogate outcomes synergistically, unlike traditional clinical trial designs that depend exclusively on one or the other. Second, diverging from some adaptive experimental strategies that incorporate both surrogate and primary outcomes, we explore strategies that tackle the arm-dependent delay mechanism of primary outcomes and optimize the statistical efficiency in the presence of temporary or permanent missingness.
2. From a theoretical perspective, we prove the semiparametric efficiency bound under arm- dependent delayed primary outcomes while incorporating surrogate outcomes for estimating the ATE (Theorem (3). We show that our proposed design strategy converges to the oracle de- sign, which represents the optimal design assuming perfect knowledge of the data lata-generating distribution. We also prove the statistical validity of our inference procedure by establishing the asymptotic normality of our proposed ATE estimator (Theorem (]). Furthermore, the proposed ATE estimator attains the semiparametric efficiency bound (Theorem) 2) which verifies the efficiency gain of adopting the proposed design. Theorem I further provides
2

---

practical guidance for the general setup of the design in terms of the number of experimenta- tion stages and the portion of enrolled units within each stage to achieve optimality. Lastly, we illustrate that our design largely enhances statistical efficiency compared to the design that only uses delayed primary outcomes.
1.2 Related literature
CARA designs. When covariate information is available to experimenters, CARA design can utilize both outcome information and covariate information optimize for the experimental goals [37 47 CARA designs originate from response-adaptive randomization designs, yet they further utilize covariate information to assist the design [34 21 45). [22] introduced a family of CARA designs that consider both efficiency and ethics. Generalizations of CARA to incorporate semiparametric estimates are discussed by 49 Related CARA designs are also explored in studies such as [35 However, unlike conventional CARA designs that assume immediate observation of outcomes, our approach accommodates delayed outcomes.
Criteria of surrogacy. 32 proposes the statistical surrogacy criterion, which states that the observed primary outcome should be conditionally independent of the assigned treatment given the observed value of the post-treatment variable (surrogate outcome). The proposal is further extended by [17] and [9]. [16] argues that [32 's proposal does not satisfy the causal necessity properties and proposes to study the principal surrogate. (27] introduces the notion of strong surrogate using a graphical model, which states that treatment serves as an instrument for the effect of the surrogate on the primary outcome. [12] makes detailed comparison of different notions of surrogacy. The authors propose (strictly) consistent surrogacy to avoid the surrogacy paradox and evaluate several conditions for the criteria to hold. 24 studies criteria of surrogacy based on distributional average causal effects.
Surrogates in randomized experiments. Existing literature focuses on using surrogates can be roughly divided into two lines of work. The first line of work utilizes surrogates from data fusion perspective. 4 proposes to use the surrogate index to estimate long-term causal effects when primary outcomes are missing in experimental data and only observed in observational data. This work relies on the statistical surrogacy assumption. surrogate outcomes under the assumption of latent unconfoundedness. (23) proposes methods to use surrogate outcomes as proxy variables for persistent confounders. The second line of work uses surrogates as supplements for the primary outcomes. [13] investigates using surrogate data to improve efficiency when inferring parameters from estimating functions.
proposes data combination framework incorporate
[25] proposes to view surrogates supplements instead of replacements for the primary outcome and investigates how this proposal can improve the efficiency of treatment effect estimation. [10 proposes to combine primary data and auxiliary data where both datasets are assumed to satisfy ignorability. (2) uses surrogate outcomes to assist with the design of adaptive clinical trials and show their relative and absolute benefits. Diverging from the usage of surrogate outcomes in randomized experiments, explicitly model the arm and covariates delay mechanisms of primary outcomes while incorporating surrogates.
2 Problem setup
Suppose are designing adaptive experiment with total of T stages. Let X, denote the covariates, Sa denote the surrogate outcome, and Yú, denote the primary outcome for subject at stage t = 1,..., T. Let Ar denote the treatment assignment status, where Aa
corresponds
to the treatment arm, while Ast O corresponds to the control. Following the Neyman-
causal inference framework [30 38), we let (a) and (a(()) represent the primary potential outcome and surrogate potential outcomes under treatment a, where E {0, 1). Assume that (1,(1), (0), (1), (0)) are i.i.d. copies of a population tuple (Y)1,, Y (0), S(1), ((O). Our parameter of interest is the average treatment effect corresponding to the primary outcome:
T (1) (0)).
We are in setting where, at the end of Stage f, we cannot observe the primary outcome for all subjects due to possible delay in the data collection process, but we can observe the surrogate outcomes for all subjects. The surrogate outcomes can inform the primary outcomes. We assume the primary outcome observed after Di stages and denote the conditional cumulative distribution function of D; as
(d(x))) P(Da S dlAu Xa. z), d 10,1,2, ... - 1} (oo), a (0.
3

---

Di represents the number of stages for which an observation is delayed at stage t. For example, in two-stage experiment, D means no delay for the i-th observation. D₁ means that the i-th primary outcome will be observed at stage 2. When Da = oo, we never collect the i-th primary outcome back. A primary outcome can only be observed by the end of the experiment if Du. S T t. Moreover, we allow Di = oc, which means that the primary outcome Yıc is missing (censored) at the end of the experiment.
The observed primary outcome can be written as
Y =
Aa + (x)
Da < T -t;
missing,
D3 > t.
The observed surrogate outcome can be written as Saa = St + (1 Sa)) Therefore, observed data has the following structure:
{(Xa, Au, Su. Yu, D.O
In our designs, experimenters can sequentially revise the treatment assignment based on the historical data. We define the treatment assignment probability for subjects in stage as
P(An = 1|x,,, Hi-1), t= l.,,..,,
where H4-1 = {(X,, Als, Dis. Sis: Y.. )==))= is the historical information collected up to Stage t t-1 1. In our manuscript, we consider a multi-stage clinical trial setting where T < oo and + co for = I....,T.
We have the following unconfounded assumption for arm-dependent delayed outcomes: A 1 (Y(1), (O), S(1), S(0)) X. We also assume the delay is and covariates dependent: D 1 (L.,). (O), S(O)) A. As we are in the randomized experiment setting, the unconfoundedness assumption holds by design. The delay mechanism says that the delay variable is independent of potential primary outcomes and potential surrogate outcomes when conditional one covariates and treatment arms. This rules out the arrow between the delay and the outcome, which simplifies the technical discussion of the paper. Moreover, such setup aims to reflect real-world randomized experiments where the delay is mainly due to protocol or structural factors instead of the realized outcome (such as the severity of some diseases). For example, if we are testing the effect of a vitamin supplement on blood vitamin levels and ask patients for regular check-ins, the delay is less likely depend on the outcome since the outcome does not cause severe symptoms that affect patient check-in.
Design objective
In this section, shall formulate our design objective. Our goal of the CARA design is to improve the estimation efficiency of the ATE, Heuristically, we aim to find the optimal treatment allocation that minimizes the semiparametric efficiency bound (SPEB) of estimating the ATE.
The formulation of the design objective relies on the derivation of the SPEB of the ATE un- der the delayed outcome setting while incorporating surrogate information. In Theorem 3 of the Appendix, we establish the SPEB for estimating T. If we introduce the following notations on the conditional expectation of the outcomes: s) E[Y]A a H, s). T(a, = ([T X, 5)|A= a, X = z)., our design objective can be formally defined as follows:
min e eE
V SPEB
( -(1,X,
(Y (0,
min e 6,1-8
EE
+ ΣT 7((1 ex())) x|x)]
ΣI (e)( -tXX)
(T(1, XX,) - 7( ((,, + (7(0,X, X) - ())
+E
Σi= (1
The SPEB is a theoretical result derived from the work of 28 Our design objective says that we aim to find the sequence of optimal treatment allocation that minimizes the semiparametric efficiency bound. Furthermore, we require that the treatment allocation should be bounded by 8 and 1 8 for

---

5 E (0, 1/2). Note that our design objective depends on the conditional expectation of the primary outcome on covariates and surrogates and the delay mechanism. However, practical settings, both the conditional expectations and the delay mechanisms are unknown to practitioners. Therefore, we propose to use CARA designs that allow us to adaptively learn the unknown parameters based on sequentially collected data. We shall introduce our proposed design strategy in the following section.
4 Proposed CARA design strategy
In this section, we shall formally introduce our proposed design strategy. We consider delay variable D with finite support: domain knowledge. Also, we allow D to take meaning the outcome is permanently missing, For simplicity, we assume that the enrollment proportion at each stage is rt = 1/", for / I....,T. T,
[o., {oo), where D* pre-specified based on
T < 0o. We assume both X and S take discrete values to avoid unnecessary technical complexity.
From a high level, our proposal consists of four steps. Step 1 consists D + stages and serves as an exploration step that learns parameters like the delay distribution, outcome model, etc. Step 2 is a policy optimization step after Stage D + 1, to compute the optimal allocation with the learned parameters. Step 3 is a policy calibration step from Stage D* + to T, where we calibrate the allocation to match the optimal strategy. Step constructs point and variance estimators based on the data. The full procedure is summarized in Algorithm |
Algorithm 1 SSrrogate-enhancee adaptive experiment with delayed outcomes
Stage (Initialization):
1: Enroll nı participants, and assign treatments with 1;
2: Estimate (( (0 1z) and let p! p. (0)z). I....,
T
Stage 1 + (Learn delay mechanism):
3: for + to D + do
4: Enroll n, subjects and assign treatment with probability e 2::
5: Update p (1]z) for Stt-l
6: end for
7: Obtain the sequence of optimal treatment allocation c(D" +11 by solving (
Stage D + 2 to T (Calibration of treatment allocation):
8: for f + 1 + 2 to I' do
9:
Enroll subjects and assign treatments with calibrated probability ě
end for After Stage (Inference):
Update and V.
12: Construct a two-sided allevel confidence interval for P.
Below we take a deeper dive into the details and motivations for each step in Algorithm !
we initialize the experiment by assigning treatments with probability ē = 1 2- After Stage 1, we are able to observe all the surrogate outcomes for nı subjects and part of the primary outcomes. One can estimate arm-dependent delay with pa (0]ær by counting the portion of the observed sample:
Stage 1. In line 1 2
(0 = Σn L(D,, 50) [(A
Σ ((a==)) 1 (An=a)
As we only observe outcomes whose delay variable D₁ S at the end of Stage 1, we let Pa ph (0]=), 121 1. We then estimate (()) x,, s) and *(1) (a) by taking a sample average of the observed outcome stratified by (a, 8),)) and (a, 2)), respectively:
((((()))))
(a, = En ( so) Yn [ =a) (X,
I Soj (An=a) (xn=)))l(snse
(Dn so) Ya 1l (0)
(a,
1= 1(D 1 (An I (Xn=a)
5

---

Stage 2 to D +. In line 4 to
5 we keep enrolling subjects and assign subjects to the treatment
arm with probability é, = 1/2. Update the estimates of the delay mechanism by counting portions of the observed sample up to the current stage:
Σ:=1
=== E Il (D. 11 (X,, (Au for l = O....,t 1,
p (1xx)
(Xx==)) (Au=a)
and let p (1zz) Pa (t- 1/x) for 1 2 t. Then update the estimates 700 (a, x, 8) and 7()) (a, as the new stratified sample averages:
Yu St- (Au=a)) (X (X)==)
700 (a, 21)
1 1(D,, 1) (a==a) 1 (x==z) (8
(a, = 1=1 M Yu st
E. (DD. "l (Am=)) (Xu=m)
At the end of Stage D + 1, we can obtain estimates of all the delay mechanisms: +1
Pa 1D" 11*++)( (D" (z). We then solve the following optimization problem to find the optimal sequence of é (T)
(0fz)..... (0
D1 ny
(Yis 7(t) (1, (i:: Sia))²
(Yio - (1, Xio, Sta))
min eE[6,1-5]
2t=l TTet p t(x
E1 ro(1 - et) .b (T xt)
s= 11
(711 Xis iss Sin) - -10 (1,
(010 Xis, Sis) - - (1, Xis)) 2
+ Axllell
Et-rree TeEt
E ro -e) ex)
(1)
In particular, confronted with the possibility that multiple minima could exist for the oracle optimiza- tion program, we introduced the penalty term A / which regularizes the solution. In Theorem 2 we will showcase that, theoretically, if we choose the penalty level to be AN C /log N/N, ē will converge in probability to an oracle minima of the population program. In practice, we propose to choose the tuning parameter based on validatin,,,,, where we split the collected data into multiple folds to tune the best scaling factor C' for AN C./To N/N.
Stage (D*+2)
+ 2) to T. In line too g we calibrate the treatment allocation based on the derived Stage D + optimal treatment allocation strategy. From Stage (D* + 2) to Stage (T - DD)) we assign treatments with calibrated probability ē₁ where
D n₂ E, -P' no C for l = D= +
D
E n
and Ø₂2 is the actual treatment allocation at each Stage Then, from Stage s = (T - D' + to T. we assign treatments with probability é. = é. where s = T D* + I,..., T. After Stage T (line to 12). we update the delay mechanism and the treatment assignment probability using data collected from the whole experiment.
After Stage T. We obtain the final ATE estimate f based on the efficient influence curve derived in Theorem
Aa Pt (x (D ST
XI,,
(1 (D)
(Yi Vie-Flm Sit. Sis)
Tt (1- (r(Xit)r, S ST' t|x)
Aat
(1t, Sit) - -7(1, x(1,))))
Σ 1 Pr. . (x.(X
(1 AAt)
X1, (F())-F(0xx)) x1). + Xa), - -700
Σr rt: (1-2((X,))

---

and the variance estimates as
(D - f(1, St
AAn Ann - Xi. Sie)²
E (x(x)) x|x
12 Fr' (x (1 (|xxa)
An(, Std) - xx1)2
(1 Au) (F Xix, Ste) F(0,
+
E Nt {X{x))
1E 't" ex
- -(0, -7)²
Theoretical investigation
In this section, we investigate the theoretical properties of our proposed design strategy. In Theorem I. we demonstrate the asymptotic normality of the ATE estimator with any sequence of propensity scores that are generated sequentially from history and converge to some fixed scores. Furthermore, we show that our design strategy asymptotically converges to the oracle design strategy in Theorem 2 which suggests that the constructed estimator is semi parametrically efficient.
Theorem 1 (Asymptotic normality). Assume the delays are conditionally independent of the surrogate and potential outcomes, given covariates and historical data. Assume the following conditions hold:
1. Bounded conditional moments: max(ma(0. s, (1, < Ma. where
X = 2) = 0. ma s, = E (a)( S(a) = s,
2. Convergent scores: there exists set of propensity scores e, such that
max 'ET))
[C((x)) cx(x)] = (1).
(2)
Let n4 = N with r > O, E. = 1. Then when N - oc, we have
VN(F -) (o,((O()()))
Moreover, the variance estimator satisfies
(())- Op(
Theorem is general theoretical result that works for any sequence of historie-dependent treatment allocation strategy é, long è converges an oracle policy in probability asymptotically. For example, one can apply complete randomization through the experiment, which corresponds to êr,er Ct,er 1/2 and establish the asymptotic normality of f with variance V(c,r). In particular, the proposed adaptive designing strategy (Algorithm) T is implemented, the treatment allocation probability will converge one set of optimal policies that minimizes the semiparametric efficiency bound (4), which is established in Theorem below.
Theorem (Convergence of the proposed design strategy). n4 = r,N with r, > E. =1 Assume the bounded conditional moments hold as in Theorem 7 Assume sufficient enrollment at the first (T - ( (non-delayed) stages:
D
max
0// Pu.co + J v 0f/01,00 + +
Σ
r 1
(3)
0 + + v + +
Set AN Co/log N/N. When N + oc, we have
(a) ē - e" = (1)), where e = lim e" (A), and
e e²(A) = arg min L(e) + Alle||
eE(6,1-0]²
7

---

(b) é - =
--- =p(1), where exx is a minima of L(e) defined as follows:
1/2,
5 D* +
ET-D" r. - 1/2 De fx 6
et
D
12 T D'+1.
Theorem formally established the convergence property of the propensity scores generated by Algorithm|] | More concretely, Theorem showed that two sets of estimated treatment allocation probability converge to the minima of the population optimal variance: one is the scores é generated by the optimization program (I). and the other the sequence of propensity scores é upon calibration which is implemented through the experiment. Therefore, Theorem I and Theorem 2 together imply that the point estimator based on the proposed treatment strategy attains the semiparametric efficient bound asymptotically.
Theorem imposes an additional condition which requires a sufficient number of enrollment in the non-delan non-delayed stages. This is an important sufficient condition for the optimality of the adaptive allocation strategy, which ensures the optimization program has multiple global minima and enables experimenters to calibrate the propensity scores based on the history to achieve optimality. In practice, Condition (3) can also provide guidance on the design of the experiment to decide the length of the study and the portion of units to recruit in the initial stages.
6 Synthetic case study: HIV trial
In this synthetic case study, we calibrate our data- eata-generating process using data collected from an HIV randomized controlled trial (RCT) conducted in Tanzania [13]. In this study, n 530 participants
are initially enrolled and randomized to one of the three treatment arms: (1) receive no cash, (2) receive 10,000 TZS, and (3) receive 22,500 TZS. In our case study, we combine the latter two arms and define the control arm A 0 as "receive no cash transfer" and the treatment arm A 1 as "receive any cash transfer." For the treatment arm, the cash incentives were administered six times from May 21, 2018, to May 31, 2019. In our case study, we consider June 1, 2019, as the starting point of our trial because all the delays observed after June 1, 2019, not due to the delay in cash transfer administration. After all the cash incentives were administered, n = 511 subjects were still alive on June 1, 2019, with As the very last test was taken on June 29, 2021, we consider June 29, 2021, as the last time point where we gathered outcome data.
177 subjects in the control arm and 334 in the treatment arm.
Motivated by the original HIV trial, we also consider a multi-stage trial where each stage consists of six months, with the primary outcome Y being the viral load measured at the end of stage. The viral load can be delayed to be measured because HIV patients may miss their scheduled test appointments. Therefore,
consider the World Health Organization (WHO) stage the surrogate outcome S (1,2,3) 3} because the WHO stage is a clinical indicator of HIV infection progression and it is measured without delay, where S 1, 2, 3 indicates the asymptomatic, mild, and advanced symptoms respectively. We use "sex" as the covariate X in our study, where X = 1 indicates male and X 0 indicates female. We further observe that the delay mechanism depends on both the treatment arm and the covariate. We summarize the delay mechanism observed from the real data in Table B We summarize the expectation and standard deviation of the primary outcome under arm A given X and S in Table
In our synthetic data generation, we generate the surrogate outcome from multinomial distribution. The delay mechanism is also generated following multinomial distribution using the parameters in Table B We generate X Xa Bermouli and the primary outcome variable Y as Yu AAe = a, Xxt X, Set = s ~ (T z, s)). The true average treatment effect of the primary outcome is
0.04. Following our proposed design strategy, we set the total number of experimental stages as T = 2(D* + 1) = 8. In this case study, We compare three designs: (1) Our proposed design strategy; (2) the complete randomization design which sets e 1/2 for t = 1,..., T; (3) CARA design that only utilizes primary outcomes but not surrogate outcomes.
We present the case study results in Figure I In terms of bias, Figure [[A) suggests that the point estimators from all three designs have vanishing bias as the sample size grows, validating the asymptotic unbiasedness of all strategics. Nevertheless, in terms of variance, Figure B) demonstrates

---

that our proposed design yields smaller standard deviations and thus higher estimation efficiency for estimating the ATE. The synthetic case study results verify our efficiency comparison in Section and demonstrate the efficiency gain of our proposed design strategy.
(A) (N((ai))))
(B) (SSaanddrdd Deviation)
1.50
4.00
1.00
(i((B))s)))
(N(SD)
3.60
0.50
3.20
-0.50
2.80
400
800
1200
1600
400
800
1200
1600
Sample Size
Sample Size
Method
Complete randomization
No Surrogate
Proposed
Figure 1: Bias and standard deviation comparison of the three design strategies.
7 Discussion
In this work, we proposed a CARA design strategy that improves the efficiency of ATE estimation in the presence of delay by incorporating surrogate outcomes. The strategy leads to point estimator that achieves semiparametric efficiency under the aun--ovarriate-deeddeent delay setting. It also demonstrates efficiency gain compared with the strategy that uses only primary endpoints. There are some future directions to explore. First, we mainly considered an arm-dependen delay setting 18 A more general setup is to consider the delay mechanism that is also dependent the outcomes [42). Another possible framework is to consider outcomes that are also impacted by delays, for which the delay variable serves special of surrogate variables [Л]. Moreover, the objective of this work focuses on statistical efficiency gain for ATE estimation. It is also important incorporate realistic constraints for practical implementation, such as fairness among subgroups [66, safety concerns [6], the discrepancy between algorithmic results and human knowledge [14], etc. More generally, it is also interesting to draw connection other relevant problems in causal inference and adaptive experiments, such as dealing with many treatment arms [26] 39 40), targeting other causal parameters (7). incorporating instrumental variables g among others. We leave these questions as future research work.
References
[1] Giacomo Aletti, Andrea Ghiglietti, and William F Rosenberger. Nonparametric covariate- adjusted response-adaptive design based on a functional urn model. 2018.
[2] Arielle Anderer, Hamsa Bastani, and John Silberholz. Adaptive clinical trial designs with surrogates: When should we bother? Management Science, 188-2202,, 2022.
[3] Susan Athey, Raj Chetty, and Guido Imbens. Combining experimental and observational data estimate treatment effects on long term outcomes. arXiv preprint urXiv:2006.09677 2020.
[4] Susan Athey, Raj Chetty, Guido W Imbens, and Hyunseung Kang. The surrogate index: Combining short-term proxies estimate long-term treatment effects more rapidly and precisely. Technical report, National Bureau of Economic Research, 2019.
[5] David Benjamin, Vinay Prasad, and Mark P Lythgoe. Fda decisions on new oncological drugs. The Lancet Oncology, 23(5):585-33)) 2022.

---

[6] Andrew Bennett, Dipendra Misra, and Nathan Kallus. Provable safe reinforeement learning with binary feedback. In International Conference on Artificial Intelligence and Statistics, pages 10871-10900. PMLR, 2023.
[7] Aurélien Bibaut, Winston Chou, Simon Ejdemyr, and Nathan Kallus. Learning the covariance of treatment effects across many weak experiments. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 153-162, 2024.
[8] Aurélien Bibaut, Nathan Kallus, and Apoorva Lal. Nonparametric jackknife instrumental variable estimation and confounding robust surrogate indices, 2024.
[9] Marc Buyse and Geert Molenberghs. Criteria for the validation of surrogate endpoints in randomized experiments. Biometrics, pages 1014-1029, 1998.
[10] Hengrui Cai, Wenbin Lu, and Rui Song. Calibrated optimal decision making with multiple data sources and limited outcome. arXiv preprint arXiv:2104. 10554, 2021.
[11] Leonardo Cella and Nicolò Cesa-Bianchi. Stochastic bandits with delyndee payoffs. In International Conference on Artificial Intelligence and Statistics, pages 1168-1177. PMLR, 2020.
[12] Hua Chen, Zhi Geng, and Jinzhu Jia. Criteria for surrogate end points. Journal of the Royal Statistical Society Series B: Statistical Methodology, 69(5):919-932, 2007.
[13] Song Xi Chen, Denis HY Leung, and Jing Qin. Improving semiparametric estimation by using surrogate data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 70(4):803-823, 2008
[14] Changyu Deng, Xunbi Ji, Colton Rainey, Jianyu Zhang, and Wei Lu. Integrating machine learning with human knowledge. Iscience, 23(11), 2020.
[15] Carolyn A Fahey, Prosper F Njau, Emmanuel Katabaro, Rashid S Mfaume. Nzovu Ulenga, Natalino Mwenda, Patrick Bradshaw, William Dow, Nancy S Padian, Nicholas P Jewell, et al. Financial incentives to promote retention in care and viral suppression in adults with hiv initiating antiretroviral therapy in tanzania: three-arm randomised controlled trial. The Lancet HIV, 7(1 (11):e762-e771 2020.
[16] Constantine E Frangakis and Donald B Rubin. Principal stratification in causal inference. Biometrics, 58(1):21-29) 2002.
[17] Laurence S Freedman, Barry Graubard, and Arthur Schatzkin. Statistical validation of intermediate endpoints for chronic diseases. Statistics in medicine, 11(2):167-178, 1992.
[18) Manegueu Anne Gael, Claire Vernade, Alexandra Carpentier, and Michal Valko. Stochastic bandits with arm-dependent delays. In International Conference on Machine Learning, pages 3348-3356. PMLR, 2020.
[19] Tessa Heestermans, Joyce I. Browne, Susan C Aitken, Sigrid C Vervoort, and Kerstin Klipstein- Grobusch. Determinants of adherence to antiretroviral therapy among hiv-positive adults in sub-saharan africc: a systematic review. BMJ global health, 1(4):e000125 2016.
[20] Spencer Phillips Hey, Aaron Kesselheim. Pranav Patel, Preeti Mehrotra, and John H Powers. Us food and drug administration recommendations the use of surrogate measures end points in new anti-infective drug approvals. JAMA internal medicine, 181(13) 2020.
[21] Feifang Hu and William F Rosenberger. Optimality, variability, power: evaluating response- adaptive randomization procedures for treatment comparisons. Journal of the American Statisti- cal Association, 98(463):671-61) 177-678 2003.
[22] Jianhua Hu, Hongjian Zhu, and Feifang Hu. A unified family of covariate-adjusted response- adaptive designs based on efficiency and ethics. Journal of the American Statistical Association, 1106009)357-367 2015.
[23] Guido Imbens, Nathan Kallus, Xiaojie Mao, and Yuhao Wang. Long-term causal inference under persistent confounding via data combination. arXiv preprint arXiv:2202.07234, 2022.
10

---

[24] Chuan Ju and Zhi Geng. Criteria for surrogate end points based on causal distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology. 72(1):129-142) 2010.
[25] Nathan Kallus and Xiaojie Mao. On the role of surrogates in the efficient estimation of treatment effects with limited outcome data. arXiv preprini arXiv:2003. 12408, 2020.
[26] Anand Kalvit and Assaf Zeevi. From finite to countable-armed bandits. Advances in Neural Information Processing Systems, 53:8259-8269, 2020.
[27] Steffen L Lauritzen, Odd O Aalen, Donald B Rubin, and Elja Arjas. Discussion on causality [with reply]. Scandinavian Journal of Statistics, 31(2):189-20) 2004.
[28] Xinwei Ma, Waverly Wei, and Jingshen Wang. Covariate adjusted response adaptive design with delayed outcomes. Forthcoming, 2024.
[29] Anthony Muchai Manyara, Philippa Davies, Derek Stewart, Christopher J Weir, Amber Young, Valerie Wells, Jane Blazeby, Nancy J Butcher, Sylwia Bujkiewicz, An- Wen Chan, et al. Definitions, acceptability, limitations, and guidance in the use and reporting of surrogate endpoints in trials: scoping review. Journal of Clinical Epidemiology, 2023.
[30] J. Neyman. On the application of probability theory agricultural experiments. essay on principles. section 9. Statistical Science, pages 465-472, 1923/1990.
[31] Joint United Nations Programme on HIV/AIDS et al. Understanding fast-track: accelerating action to end the aids epidemic by 2030. 2015.
[32] Ross L Prentice. Surrogate endpoints in clinical trials: definition and operational criteria. Statistics in medicine, 8(4): 440, 1989.
[33] Angela Ramadhani, Robert M Josiah, Anath Rwebembera, Emma Lekashingo Msuya, Roland Swai, Peris Urassa, Samuel Kalluvya, Salehe Omari, Josephine Kaganda, Ayoub Kibao, et al. National guidelines for the management of hiv and aids, 2012.
[34] William F Rosenberger and John M Lachin. Randomization in clinical trials: theory and practice. John Wiley Sons, 2015.
[35] William F Rosenberger and Oleksandr Sverdlov. Handling covariates in the design of clinical trials. 2008
[36] William F Rosenberger, Oleksandr Sverdlov, and Feifang Hu. Adaptive randomization for clinical trials. Journal of biopharmaceutical statistics, 22(():719 1766999736711 2012.
[37] William F Rosenberger, AN Vidyashankar, and Deepak K Agarwal. Covariate-adjusee response- adaptive designs for binary response. Journal of biopharmaceutical statistics, 1(44):277-236 2001.
[38] Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5):688, 1974.
[39] Lei Shi and Peng Ding. Berry-esseen bounds for design-based causal inference with possibly diverging treatment levels and varying group sizes. arXiv preprint arXiv:2209. /2345, 2022.
[40] Lei Shi and Xinran Li. Some theoretical foundations for the design and analysis of randomized experiments. arXiv preprint arXiv:2406.10444, 2024.
[41] Lei Shi, Jingshen Wang, and Tianhao Wu. Statistical inference on multi-armed bandits with delayed feedback. In International Conference on Machine Learning, pages 31328-3111 -1352.. PMLR, 2023.
[42] Yifu Tang, Yingfei Wang, and Zeyu Zheng. Stochastic multi-armed bandits with strongly reward-dependent delays. In International Conference on Artificial Intelligence and Statistics, pages 3053-3051. PMLR, 2024.
[43] U.S. Food and Drug Administration. Adaptive design clinical trials for drugs and biologics guidance for industry, 2019.
11

---

[44] U.S. Food and Drug Administration. Surrogate endpoint resources for drug and biologic development. 2022.
[45] Soffa Villar, Jack Bowden, and James Wason. Response-adaptive designs for binary responses: how to offer patient benefit while being robust to time trends? Phurmaceutical statistics, 17(2):12--197, 197, 2018.
[46] Waverly Wei, Xinwei Ma, and Jingshen Wang. Fair adaptive experiments. Advances in Neural Information Processing Systems, 36, 2024.
[47] Li-Xin Zhang. Feifang Hu, Siu Hung Cheung, and Wai Sum Chan, Asymptotic properties of covariate-adjusted cesponse-adaptivy designs. 2007.
[48] Wanying Zhao, Wei Ma, Fan Wang, and Feifang Hu. Incorporating covariates information in adaptive clinical trials for precision medicine. Pharmaceutical Statistics, 21(1):176-195 2022.
[49] Hai Zhu and Hongjian Zhu. Covariate-adjusted response-adaptive designs based on semipara- metric approaches. Biometrics, 79(4):2895-29) 2906, 2023.
12

---

Technical Appendix
A
Additional results and theoretical proofs
A.1 Derivation of the semi-parame remipparametric efficiency bound
We use Rí E (L,...,, r=1== Theorem 3. Under the unconfoundedness assumption and the arm-dependent delay assumption:
I') to record the stage at which the subject is enrolled, such that P(R: = 1)
Tt, -e=rr=
(a) The efficient influence function for estimating
((YYA, ((0,X) -
E ((R=t)
(1- ()-A)EIT (R=tt) (11XX,))-(00,1))
(11
Ei=1 (X)
T
Ei= rr(l - ex()
A E (R=1) (DST-1) ((,,(1,X,S)) Σ ri(1 ex(X)) +|X)
(1- A) Σ (R=1) (DST-1) ( =(0 (.)..
(T) f|x)
(b) The semiparametric efficiency bond for estimating T is
SPEB
(T(1,x,)))-((xx)))
(r(0,X,S)-7(0) X))2
E
Σtz Txe((X)
Etmr ri(1 ex())
(Y -(1,
(Y - - S))2
E
(4)
E (ie())p))) (T -(X)
Σ re(1 - (x))))) 41x
Proof. (a) Proof Efficient influence function
Step 1. First, we write out the joint density the observed sample as fy a, d, v, s,8), indexed by
fx a, v.u.
= (x),(x)) 1=1
II { (rnllennnt))sssxxx..ll (Pa.,(T' (xa (1ss-11)
(1 - -- (T (1))) (1sT o1e
[(1 enn (z)) (5(0))10000)) (1(x) (PI-an(T (T fv() (ul,,()) (1as7-11
(1 (1-P1-a.m) Pi-a.n (T t(x)) (
(15T-11)1-0) 1-a 1wme)
ETsr Dis ((==))
fx,(())
T
(ct, (1-m)) (1 (2)) (1-a))
x (15(1)X,) (s|x)) ET- al (s)) (0--))))
T
II (Pa,n(T t|x) -l Pa.n (ast-n)
[=1
T
11 (1 P1-a.n f/x)) -MMor-ooo
(t)))
x
1=1
x (frajss...vuss.))) 2i
2n al of4--)
13

---

x (o x)) 2)) (asss-o0 2as (o=t))
Step 2. Next, we derive the tangent spaces. We use f(-), 0()), ex()), Pr(-) to denote by logg())) s())
log (.gg()).
& e()) and by log p(-).
Bq
oy
Tx (fx(x) Elf (X,)] = =0
al (dgT- (v=t) f1(y1s,)) XX) =0 ))
Ty()
TY(0)
I (dST-)) fo(vls, :lfo) (10. X,IS,
Ts()
a1 (-=1) (v=1) (1 (lg(((()))) = =0}
1=1
a) () Blgo (S. =o}
Ts(o)
t=1
TA,1
{ (1=
(1 a> (-= (1 c=(x))
= =1(1=)) (x(x)
1- ex(x) )e(())
1-a a
TD.,tt
al (1l() (dST- - (/)) + 1 [d>T P1
Pa(T - dx)))
11 (JST-1) (12T-11) Pa (T-T-1 - t12)
= (
Pa (T' - -1/z) 1 - Px()
TD,1-a, = (1 a) ( ((SST-))P--a( t|z) + (
() (1 P1-a(T t[x)))
(1 a) 1l (0=1)
PT-a(T - t/xr) 1 - 1-Pi-a(1) M/ PT - -110)
(dST
(d)TT-1)
Therefore, the tangent space is
T Tx Ty() (y @ Ts() Ts(o) (TA,1 TDA.1
e... (TA, + TD.A, T--DAA,).
Step 3. We consider the estimation of the target parameter E[Y(1) via a parametric submodel indexed by Under identifiability assumptions, the target parameter can be written
[Y E, E,y E. [ 1,DT
Taking pathwise derivative of E,, [Y]] at = 0, we have
[Y())
On
7
E,
On
(1)
Ey
= LV
Om
(11)
Om
(mTl)
14

---

Let (X denote the score function of f(z d, s.
(7) E
-11, x))x(())
E
(T(I,)) (r(1,))- -x1xx(x)
E
(T(1, (F(1, -(),x))7)9(XX,4) (x,,
where
= =
A
(II) E
E E(11,x,99(( 1||
E (
E
(- (1.x59))-((1x))xxxx........ (x())
where 51, - T [,S,
(111) = E -10-
- -(1, 1FF
A 1(v=)) ((DTT-I) (T-r((..5xx))x70741...xx ex(X)riaa - x\X)
A. 1. 1(STT-) (Y (V--11...x))01..55xxxVV)
E
(x( Te
As a result,
,E,,(Y(O
On
D=
(T(1.)) T1) s(Y, s(Y,A,,,x,V,D)
A 1( -((x)) A A-1(v=)) 1(DTT-0 (Y (X,,
E
x(1,x))
ex(X)r
ex(X)r:pa( t(X)
- ==
-B
where
= = T(1,X) 71,
52 = A. 1- (T(1,X, - x(1,))),
ex(xrr
15

---

A L(v- (DDST- (Y (XX,)X ex(X)r: - 11x)
P,,Y,
(11,
Step 4. Now we project the score functions to the corresponding tangent spaces. First, we project (p (X) onto Tx.
E (1(()) ((xxx))(xx) x [(X) = O.
We obtain
f(X) (x().
Next, we project S, V) onto Ts(1)-
T x ( (v==) )(8Ix)
E
(4
Σ
1(v=0) ((81x)
= =0.
1=1
(A,
E
A (. - (I) x (A) 2 1(v=)) [(|x)) )
E
ex(X)r
A ((==)) (11,,,,)--((4, x (A Σ 1(v=0) 1(V=)) [((X))I)]]]] ex(X)re
E
E
t=1
=
E
-(1, 7(,
Then.
1 [(v=t) [(((x))
(A) 1(v=)) )(II
E
A
1=1
[(V=t)) |(111}
E
T
E
Σ
1 ((==)) [(11))111]]
T
E
Because
T
E
(f(1,8,x)-7(11. ((1x)))(((
= E Σ [(88)1[(11x))11
we obtain
f(Y|X, x(1,8, - (1,,
Eizee (X()r,
Lastly, we project (X) onto Tr())-
E
(,(Y.A,,S,.VV. - (A)
1 Lv -
<A - =0.
16

---

I
Pa 9,,X,A,,,x, x (A, Σ (
1=1
A. ( L(D (Y (1,, x (A)
T
E
E
1 (
e((
t=1
E
E
A A. ((ve)) ((DST-1) (Y -(1, x (A ex(X)roPa(T t|x)
Σ
1 1 DSs-
1=1
E
(Y - (1,(1XX)))))
Σ [[v= I (1
<AΣ
E
A
t=1
E
A
[ 10==)
T
1 1 [w
E
T
=
E
Because
T =
Σ -X
obtain
Y -1,(1,
HY|X, Σi= (T
In sum, the efficient influence function is
A Et 1(v=1) M(DST-1) (1,))1,
XV,
Σ (X ((x)
(1 A) Σt L(V=1) 1(DST-1) (Y -00,
EL (1 (X)))i(i -t())
A. L(Ve) (T(1, x(1,X))
+
Σ ex(x)r
(1 - A) Et= ((vet) (0,, 8) (00,
Σt (1 - -Xi(X))
+ (7(1, X) - x(0, -T).
(b) Proof of semiparametrie efficiency bound
Proof.
VIp R,
= -V[7(1,)) - - -( -
17

---

A AE (( (r(1,X,5) -1,,))
(1- A) Ei= (R- )
(e((X)
Σi=1 11(1 ee(())
E (R=t) (DST-1) (Y-r(1,X,,)) (1- A) E ((R= (X -0-r(0,x.5))
E mee (X) (x)p1 - x
Eis r2(1 (x( f/x)
X) - X) -1
(A)
T t= ((==)) ()),
(1 1)- ET= 1 ((R -(0,
+V
Ei=1 (X)
Ei=1 rr(1 - ex()))
(B)
D ((E=)) (8-
(1- A) I(R= (Y =(0,
+V
Σ (t)
Σt= r2(1 (X)))
(0)
(1(,x))-(0.x)-))
+ 2Cov
4 1 (R=1)
(1- A) E) (n=!) (r(0, - )
(F(1,X,9) -(1,
E (X)
Σt= r2(1 - ex))
(D)
+ 2Cov
A 1 (1S)))))) ( - -(1,x,))
A1-A)ET- ( (V 7(0, X, (0,,0 X,S))
Ei= 1== (1)
Et= ro(1 - -e(()))oo(- (T
(E)
A A 1=1 (R= ( (1 - A) ΣT= (R(=tt)
(0,x)))
+ 2Cov
El (X)
- Et=1r(1-
r((1
ex))
AT 1(R= (Y- -(1,X,))
(1- 4n= (X.00.
Ei=1 (p -T
Et= re - re(1- - -(X)
(F)
We work with each term separately.
(A) = v 71,, X) - -(0, -r]
=E((T11,x) - -(0,)) -7)
ET 1 (RR
(1 A) ET (R (f(0, -(0))))))))))))
(B) = V
()(1,x - (1,,x)
Ee (X)
Σt r((1 - ex)())
1 ((R=)))
+ (1-A) A) ET= ((R=() () -(0
V
(T(1,X,)) S) -11,x,)))7((xx))
El 1 (X)
Σt re(1 - ex(())
A Et ((R=)) (r(1,X, - (1),))
2Cov
Σ (te()
(1- A) Σt (R=()) ()(0,X,))- - -(0,
Σ rt(1 - (x))
18

---

A - (R-1))
(T(1,X,5)- ((4xx))) 2)
(1 (1-A) A) E (1-1)) (1)
E
+ I
(=1 (X((X)
Ex= rc(1 - ex))
i=1 1 (1t=t) -T(,,x,)))(((x))))
-T(1
A) E|=1 1 (1==1) (T(0,X,5)) -(0, -((,
21E
B
2i rr(1 ex(X))
E 1=1 (R==))
(1 A) E (R=t) (1(0,X,5)- - -00x)))
E
- (r(1,X,5)-
+ E
Ei (X)
Ei=1 I:( - (X))
A A (DST)) (Y (1,,
(1- A) E 1= 11==)) (Y (Y-+
(C) V
Σ re(1 - (X
E'= 11E=) -
V
(1,, (11,
E
(1 A) (R=z) (DST-1) ( -7,-
V
E r(11 - (X))) t|x)
- 2Cov A AE (
- (I(,
Ei (e( f|x)
(1 A) EL LIR ((ST--1) (Y - -0, (0,))
1=1 ro(1 - (X)) (Ix))
A A 1 ( 1)
(V-7(1,X.5)) 2
E
E 1 -0.. (0) 2]
- (1x)
+ E
-
Ex1r (x)1)c(())))((7---- -(x)
Et (T (1x) (Y (X, S)))
A
2E
(1 - A) ET= ( (Y- (y.-(0,x,0))) x0))
E
EL re(1 - (X)) x(x)
A AT 11 I1
E
E/= (t (T (Y - S1)
(1 - A) 1 -
-0, x0,x)
+ E
Σl ri(1 - ea (x)) x1x)
((11x))=(0,)) - 7).
(D) = 2Cov
A A ((==)) (T(1,X,S) - -(1,x)) (1- A)- [(n=)) (-(0,x,5)-7(0 -(.x.))
MT xx)
ΣT - -=(x)(eeex
(((1,))-((0-) -(0,,) -7). A A 1 (R=1) (r(1,x,5)- (11x)))
= 2Cov
Emmree(( (X)
D1
(1-A) ΣT (R=z))
- 2Cov
- (+0.x,,)--(0.)))
Σi=1 rr(1 - (x))
D2
19

---

= D1 - D2.
A AE ((m=t) (T(1,X,5)) -((x)))]
D1 = 2E
(11,x)-=(0,80) -7)
Ei= (X)
E E [(R=)) (T(1,X, -T(1,x)))))((xx))
(T(1,X)- (1lxx)-((0.x)) -(0,,)) - +)
2E
Ei (X)
1 Σラ= ((R=1) (T (T(1,X,5) (+(1,x,5))<((xxx)]
2E
(11, - x)) - -)
E E/= (X)
(1- A) Et= 1(R-L) (0)) -(0, -(0,
D2
2E
(T(1,x) x( -)
Ei=r rr(1 (X))
()(1,)) - (00,
-r) :))
(1- - A) Ei [((=))) (T(0,X, --0,x,,))=(0xx1))
2E
E
Σi= re(l ex
(1
A) Et= 1(R=11) (r(0,X, -(,
((,
2E
Σ -1=1 re(1 - ex()))
Therefore.
(D) D1 D22
A DTT 1(R=1) (T) (+(1xx)))=((xxx))
- -(O,)) - 7) (
= 2E
Σ/m (X)
((1,x)-T(0.4) -r) (1( A) E/= (R=1) (100,x,5)-
2E
-t=1 ri(1 -e((X)
= 0.
(71.8))-((xx)-
(E)= = 2Cov
A ET= ((R0) (X,,(1,X,5))
El (e) (T
(1- (1-A) ET= ( (V - -(0, ((,,
Ex=l re(1 ex(()))p0 -((x)
A AΣT
((1.x)--(0.x)) - -))
(Y - SI)
2Cov
D E rer (T) -
E.
(1 - A) T
11 (R= ( (Y -(0,
2Cov
r1 - (P)) -x(x)
1==
E1 Eg.
A A ( (V- -((,,))7(0xxx) -7)
E = 2E
E/ (e)(() - (1x)
A AE ( (Y (1,,
2E
-(0
E
Dta (e)((X) t|x)
20

---

=0.
(1 A) Ei 1 (R= (0,
E2 2B
((1,))-=(0,30)
Ei= ri(1 - (X - x(x)
(1- A) A) (R (x- (x-770,x.51))
-(. -7) E
2E
Ei=1 - (X))))(T t|x)
=0.
Therefore, (E) =9 E1 - E2 0.
A - r-1 ((=))) -((xx) (1 - A) 1 (1==)) (F(0.X,5)- -((0,))))
(F) 2Cov
1- Me)((
S0-1 - e(x))
ET= ((R=)) (Y -Y-
-(1,x,
Et=1 (T
(1 (1-1) (R-1) (R=)) (Y - T(,,, x.)))
E rr(1
A Σ 1=1 ((R
E
((1, (1 (1- A) E (R (100,x,5)-((0..)
2E
1-1 (X)(X)
Et=rr(1 ri (x))) ex))
-
A E (D=S) (Y -8--((1) -(1,())
S1 (T (|x)
E)
- m (x)) t|xx (Y (0.x).1))
T
Eim ((R=() (7(1, X, x(, (1- A) E/ (R=t) (T(0, - -((,,,)))-((xx)))
2E
1 (X)
Ei=rrel1 - ex(1) ex)))
EL (DSTT (V (1,,(1,X,S)
A) ET (
(Y -V-1(0m -0,
1=1 rr(1 - ()(X)))ol (xxx))
A 21=1 (R==)) (T( -),(x)))))
A A (D=T) (Y - -((,X,,)))
2E
Di=1 (X)
Σ mx)) (1x)
(1 A) Σt= ((==t) (r) -(0,
+ 2E
Et= re(1 ex())
(1 A) ΣT= | ( (o -(0, (0)00_0)0000
Σ re(1 (T
00.
In sum, terms (D) = (F)) = (F) = (Xx, (I,
+ (C):
Vls
= E
(T(1. - (1(1,))7(00x8 X), - 7)
A D ((==)) (T(1,X,5) -
E
Σnn (X)
(1 A) Et= 1 (R=1))
+
-
Σ re(1 ex))
21

---

A A T
1 ((S-1) (Y- (X-
E
Ei=1 (1ex)))(]](
(1 - A) Ei 1u=
Eim 1i - (X))) (Y - -=(00
+ E
A.2 Proof of Theorem 1
Proof. Our proposed ATE estimator is
(D
7 t= P (Xu),,, (D - (Yx Vv--7(1, Xet, Si)s
(1 (1- (D)
D1=1 r4 (Xx))r0 Die (11 (Kie- Yyc--1(0 x.., Stn)
Aa
(f(1,Xic. Stt) - -F(1,
1= re (Xa))
(1 Aia)
(1,, - -F(0, X,1))
Di Tr. (1- (Xir))
T
+111 -7(0,
Denote
(1) h
(Yu =(1, X Sil)
Σ 2t=1 TE (X -T
tm
An
(11,,Xu, xx), -
F(11, +(()
21=1 (Xit
T 14
(1 An)
(2)
To
E Tr: (1 -1-2a (Xit) ST- (Yu F(0, Xic, Sit)
(1-
x(.) + +7 xx)) F(0, )()
Els Pe - (X,(
Step O. A basic decomposition.
1
T
n
A
(Yr - -0,, Sa)
N
Ei r. (Dr I -(x
(A.1)
Sw) - =(11, X,,, Str)
E TI
(X <T-
(A.2)
A₁
(T(,,Xo, Sit) - -((1,
t=1 Ti (x(ss))
T
(B.1)
Ast
-1
(111, X,,, Sa) T(1, Sa) - (7(1,
- T(1, (F)1)())))) X,)))
E T2 (Xxx))
(B.2)
22

---

(()) TI (C.1)
Note that for f2, we have similar decomposition; omit the presentation for conciseness.
Step Asymptoties for the nuisance terms.
Step 1.1. Convergence of weighted propensity scores. following results:
Under Condition
we prove the
T (D= Σ
(x +,,())
T rr(1 (X ST
ST
Σ
n(1 ((xx)))o(DDe - + o,(1,,
t=1
T
Σ (Xi)
Σ red (XH) + +0),
T En(1-
Σ
ri(1 c: (x(xx)) + (1).
WLOG, only prove the first one.
(X,)) ST x|X)
- c(Xx)) -T -U))) +
Σ (D ST (|x) pi -T- -(x))
Σ
ST ((x)).
By Condition
we have
T
rr (2((xa) - cx(xa) ST
< max (z) cx(x)) Σ
r (D. ST- = (1)
f
t=1
Meanwhile,
Σ (P -
- ST -(())))
LIDAST-AAA (((2)
(x)
- ST TT -Ta(a)) (xn=z))
The summands the outer layer, indexed by the f, form a martingale difference sequence. Hence, its variance is given by
Σ
E
(1
pi(p ST ())² Xt = =)p(o) E n
n4
7,(x))
p(x)
7
50 max
n4
(()) + 1)
23

---

1
where we applied E
s
1 + Bin(n, p)
(1 +1+ n)pp
Therefore, we have
T (e((Ka) ST - + o,))
(a((X))i(Da -T- E
Step 1.2. Convergence of the mean functionals. In this part, we prove the following results:
1
- (1.))=(1,.))) Op(
VN
F(0, 2)) =00,)) Op(
VN
1
2(,,2,s) (),, 0p(
VN
7(0, -(0 x0) = Op(
WLOG, we check the first result.
11= 1o N- 1=1 E.n -0 (Xx==))
7(1,2) - -(1,,))
The numerator forms a martingale difference triangular array:
w!!") (L)(1)) - (:d1)-=-((i)))dddd..-------
F o((Y),()) AJar Djoeesussi. Aisosssus
For convenience we denote F(l)) = =(n) 1(s-1); then we can verify that
E W B[W(())
(m)
1jr = O, for SLST, 7, sis ny.
The variance of the numerator is then given by
r (L,)) ())(())
0² (1,x)(())
Therefore,
T - ((,)))-(11x))441) ((000) = Op( VV
N 1
==
For the denominator, similarly, we can show that
T
N
-o Σ -))
Therefore,
- Op(
Step 1.3. Concentration of propensity weights.
24

---

We prove that
T
n
1 =x
Ei=1 (T - 1 0p(
N
[=1
T
LTD A1r)
1
1 1
Op(
N
Elar r1(1 - (o) - t/xx)
VN
1=1
1m1
2t=1 EE Pa(x)) -1) =0p(
Au
1
=X)=z)
VN
T T4
It (1-2 -1) =0p(
Au
1
(X
VN
[=1
i=1
WLOG, we prove the first result. We can compute
T
-1) 1
((u==))
r 1/1)
+ +02( (xx)
-(p(s) + Opl 1
(1 + 0pl )
0pp VN
Step 2. Approximate the main terms.
We prove that the sum of (A.1) and (B.1) can be
approximated by the following:
T
(D
(Yr 11,--(1,11,11 X
(1, Six)
N
To
(I
(D ST -T-t|Xi
A
(Yu ((, + topl VN
((xu))((De -Tt(/Xu)
[=1
1=1
1e=
and
T m4
Aa
Sst -
[=1 1=
1= To
(((it)
T
Aut
(T(1,Xu;,s) - (11) + top( 1
Ei= Tt (X(s
[=1
i=
The first approximation can be done by simply noticing that
T
Aa (x (Ya (vu - =(1,,,)))
N
1
1
=0p( ) = opl
E r. (D) TT x(xaa En Ti - (e(2)) ST f|Xa)
The second approximation can be done similarly.
Step Compute the variance of the main part.
By Step 0-2, we have the following approximation for 7 - T:
Ã (fo To)
Ti - (oo TO) + op(
.pp
VN
25

---

where
Rt
f Pap
(Y - 7(1, Xu,, Xi))
N
-- Ti ST
Am
Aa
- (7(1,
E 1=l " ((()
1
Em T'' (1 (x=e((X)))nn(De ST- (Va (V=- 70, Xit, Sit)
(1- (I-Ar)l(DiST-)
0.aap
N
(1- Ait)
(T0, Sit) -
- -(0 (0. + F(00, Xx)
E I'' (1 - (xiit)
We can verify that the variance of नpp ap is given by the following form:
1
0² (1,
(0,
V(Fup)) = N
E
+ Ei m (1-x(())0) ST - (|x)
E 1 tt ce X (
S T ((x)
(1,
x0
7) }} ol 1
+
-
=- F' (x(())
E/= P2 : - (x))
N
To sum up,
1 ((e)
(FFopt)
+ o( N
N
Step 4. Prove the asymptotic normality of the approximation apr upon standardization. We check that the Lyapunov condition holds:
n
Aul 1 ro -T-T-((Xa) (Ya (Ya- 7(1, Xxe, Shi))
E
N2V(C)
(1xa)
CM4
1
CM4
E
S
NV(e)283
Similarly, we have
T
(1 A
(Yu - 7(0, Xies XxsssSe)))
22
Σm ri
(1- (xx:))) ST - x|xx,)
CM1
1
CM₁
NV(e)²
1
-i=1"
ro. (1
NV(e)283
and
T
Avt
CM4
E
-(,,,
Nc)(
Σ 1 re (x,)
NV
1=1 1=1
Σ
T
Ave
(10, ((00)))))
CM
IE
N2V(c)3
1 re: . (1 - (X(e))
NV(6)283
t=1 1=1
1
T
- -11 < CM₁
E - ((,
N2V())2
NV(())²'
1=1 im1
26

---

Therefore, we have proved that the Lyapunov condition holds and
VN( T) ~ (O, V(e)).
Step 5. Prove the convergence of the variance estimator.
First, we can prove that
L ((,, EL=1 ro ((xN) - x[x,,]²
(DD ST Au (Yu (Yn
X1, S.) (f(1, X,. Sa) r(1, Xa, Sa)))²
E 1o (X,)) (Da S
- x(x,)}²
1
(T) Aa - X1, x Szt))
(Xx))
Au(T(1) Xu, Sa) - (1,, X Sti)s EE I't 2 (X )m (D ST - i)
(Yu 7(1, Xu. (F), S.1 T(, Saa) ST ST 1) (Yn
{X,))
(1
=E EE T. (X)) (D TT
+ Opp
The results follow from the martingale structure and the approximation given in Steps 1.1 and 1.2. Analogously, with similar ways of decomposition, we can prove that
T n
11p.sr-ol1AA.0Y. Xi, Stt))
E/=1 TI (1= - i() )0,))
+ OO VN"
t= I' (1- (x)) < T (1x) /
T
An Xir, Swt) - -(l,, Xxr))²
N
EE Tr (X
jº (1,X)
=E
+ +0p
El (X)
T
(D:STT- - (F(,, X Sa)) - F(O, Xx))
N
1Et Ti (1 - E -x)))))
(0,
+ +0pp 1 )
=E
2tmi Tt (1 - cx()))
Moreover, have
T
X1) - -FiLOR =)
-(0, (F(11 - (7, xx
T
(x11,x)) - - -TTI -
11
27

---

T n1
- - (r( - -(,
-((fL,oR - -b,oR) - (111 - 2o))
=B ((7(1.)) =(0, X) +) + Op(
Summarizing the above parts, we can conclude
V(e) v(e) Opl VN
A.3 Proof of Theorem 2
Proof. The proof is delivered two steps:
Step 1. A sufficient condition for the existence of non-trivial oracle propensity scores in the non-delay stages.
For the oracle optimization problem 3 by considering covariates X and S with finite discrete support
values, we essentially solving the following program:
(1,28) min Σ. mee())))((T -t
(0,22,8)) e() t(ax
(1,,2) E. (z)(
x²00,z)
Err((-e((z)
For simplicity, we can omit the dependence on x and 8 values and focus instead on the following:
of
of t) + E.re + E.r((1-e)
of
of
min 85e E, Eepp - t) +
We want to compute how many stages it takes to obtain nontrivial solution for the adaptive experiment. The last D stages will just come with delay; the first T (D + 1) stages will have (T t) Pl and Po(T single one by representing it as an average e:
Pa We can condense the first T (D* + 1) variables
of
min
EisT 00 + E ( 65a51-88
o/
EigT-D--11(11 e) EIST-D ri(1
+ -EST-D-1
EisT-D- -is--- TeC:
r1 EE>T-D rr(1 e,)
We want to introduce solution for that is bounded away from the boundary, By KKT conditions, for the optimal solution variable e*, we have
dL(ecet de
2 Σ 1 1tP1,x
EIST-D rE P1,00 + 1iST----TTEP11 t)}2
-IST-D--1 DD- 12(1 10,00 + EtsT-D re(1 (p) 1))z
07 Hi -D-11' T't
EIST DITT-D-1 D Px6* "E>T---- 1
o EIST_D'-1' DD
E1 11 c*) + EEST-D-- T: -1)=)))
et) =0
28

---

Note that the derivative function 0C/0e is continuous and monotonically increasing in e. We can upper bound the minimum value of 0c/de and set to negative:
dc
( + o3) EiST-D T1
(0/P0 + 00) LIST-Do Tt
+
(EisT-D Dr re(1 - 8)}2 S 0.
de l=
(EIST-D rid + EiST-D- rz)
Denote
R Er. Th: RR
Σ
Tt.
I-T-D-
T>T-D-
Here R stands for the portion of the whole sample before and including stage T D*, and Ra stands for the portion beyond stage - DD. After simplification, we obtain
6 + Rd
P1.01.0 + o
R2 -6)
0 P0,00 + 0
which further simplifies to
0 10,00 +
R 2 1-8"
Vol/pl,o + + + ooll Pu,cc + +
Similarly and symmetrically, we can lower bound the maximal value of the derivative and set it to positive, which gives
V vilpp,.0 +o¹
R 1-8 V 0 + J9 Vog/pu.c +o
Summarizing both parts, we obtain
max
(V (V o Po. + +0 V
vot /f//11.0 + +07
RS 2 1-5
VolpI +o oo// / P0,cc +o
Step Convergence of the empirical objective.
We prove that the empirical version of the objective function converges uniformly to the population program:
EN max 12(e) C(e)) Op( VN
(5)
eE(6,1-8,7
For the first D + 1 stages, because the data collection policy is complete randomization with equal probability, the estimates for conditional variances, conditional means, and delay distribution are all VN //N-consistent. Noting that the denominators C and L are strictly bounded away from and 1, (5) can be verified.
Step Convergence of the solution. Using the fact that ē minimizes the empirical objective, we have
Σ(ēN) ANllEN11 < Ō(ee) + AN
Then we have
An A AN Z(e") Ō(ēx) s L(e") L(ēN) + 2EN & 2EN. where the last inequality follows that e* minimizes C(e). Meanwhile, using
[(e')) + AN 2 L(ēN) + AN 1
we have
>vllev -N -
L(ēx) - C(exx) < 2(ex) Z(er) + +EN S 2EN
29

---

To sum up, we have proved
2 An
NE AN
le Lev
_ 1ivl2 - ||e112 +
Now letting N + we know that
NEN = Op
AN
Vlog N
Hence we can conclude that
||ē 12 & ||e1|
(6)
On the other hand,
((e*) sC((N)
<ā(ēN) + AN ANl Ax 1|xx + En
<ā(eN) + +>v|lexi1 AN [ 12 EN
<C(eN) + +Av1ee>11 AN ēN 28N
Therefore,
[(ēN) 3, L(ee)
( states that the distance between the ēN and the Euclidean ball 2* {e |ella = ||]|||
converges zero: d(ēN. B*) P, O. 7 states that the distance between ěN and the minimal point
set 2N' converges zero: d(ēN, wV) E, O. But, 2* n n {e*), }. Therefore it must be that
P
Step 4. Convergence of the implemented propensity scores.
The of the result, P ex is direct result from Step and Step
B Synthetic case study details
In this section, we provide the parameters generated from the real data adopted in synthetic case study. The parameters are summarized Tables
B anddd 2
Table
Delay mechanism from the real data
T
X
PD T x\))
Pu D
T
t X
0
0.69
0.60
0.89
0.86
2
0.98
0.97
0
0
0.68
0.67
0.92
0.93
2
0
0.98
30

---

Table 2: Parameters generated from real data (Continuous outcome)
X
T1,,088)
70,,
(1,2,, 8)
a a0,, 21, 8)
0
2.50
2.98
0.36
2.06
1 2.72
2.47
0.82
0.31
0
3.03
3.02
2.06
1.70
1
2.68
2.92
0.66
0.85
0
2.94
2.59
1.27
0.48
1
3.13
2.84
2.01
0.78
31

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
32

---

Justification:
Please see Section S
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please Section and Supplementary Materials.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
33

---

Answer: [Yes]
Justification: We provide open access to the code in the Supplementary Materials. However, we are not able to provide open access to the data as the HIV dataset is not publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cr cc/ public/guides/ htwp://oooeeeuaaoisssoooooooo
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips. //nips.cc/pil wwcppbblc/guuiicuceccccccoe.oopp /eeggudddsseCeebbbinninn htt.SSbbbiiiiooooiiiiiooooi for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please Section 0 and Supplementary Materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Please see Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
34

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπхо rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
The simulation studies require approximately 160 minutes to complete when running on MacBook Pro equipped with an 8-core CPU.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted in the paper conforms with the NeurIPS Code of
Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see Section 7
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
35

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators or original owners of the original dataset used in the paper have been properly credited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
36

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: We have included the details related to the original HIV trial. Please see Section 6
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [Yes]
Justification: We have obtained IRB approvals when obtaining access the original data.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
37


Reviews:
[{"review_id": "qfa6FIFaZC", "reviewer": "Reviewer_32pT", "summary": "The paper considers the covariate-adjusted response-adaptive randomization design for settings with delayed outcomes and surrogate outcomes. The paper first characterizes the efficient influence function for estimating the primary outcome under the delayed setting with surrogate outcomes. They then characterize the semiparametric efficiency bound of the estimate. Using the semi parametric efficiency bound they devise an optimization approach to construct an adaptive randomization design that optimizes the semi-parametric efficient bound. They provide a synthetic case study to demonstrate the effectiveness of their approach.", "questions": "1. In the numerics, the \"No Surrogate\" approach seems to perform worse than complete randomization. Is there any insight on why this adaptive approach performs worse than the complete randomization approach? Does the \"No Surrogate\" approach also account for delays or does it utilize an existing adaptive randomization design approach?\n\n2. The proposed method suggests applying regularization to the oracle optimization program to deal with potential multiple optima. Does the procedure tune the regularization parameter so it produces calibrated treatment allocation probabilities with better estimation error? Does this align cross-validation set-up briefly mentioned in the paper? If so, what is the exact cross-validation set-up and if not how is the the tuning parameter chosen?", "limitations": "The authors have adequately addressed limitations.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper overall feels well written and well organized. The problem set-up as well as the key results follow a logical flow and it is straightforward to understand how the adaptive randomization proposed in the paper is derived. The authors also provide insightful comments on the results in the paper which helped point out its main contributions compared to existing work. \n\nThe paper's contributions are interesting as their adaptive randomization design in the delayed outcome setting with surrogate outcomes is a realistic setting seen in clinical trials performed by the FDA. The synthetic case study helps highlight the potential application of their work. They also provide theoretical results that give guarantees on the quality of the estimation approach", "weaknesses": "The paper leaves out some details which may be obvious but are worth including for clarity. For example, the unconfoundedness and arm-dependent delay assumptions are briefly mentioned in passing, but are not formally defined even though they are used in the statements of the theorems. The authors also do not comment on the tractability of optimization problem that determines the treatment allocation probabilities. Other details omitted include details of the cross-validation set-up used to tune the regularization penalty parameter and the the details of the \"No Surrogate\" approach in the case study. \n\nDiscussion on the settings where the proposed method has the most benefit is also lacking. The numeric case study primarily studies the impact on sample size on different approaches for estimating the primary outcome. The results show that while there is benefit from using the proposed method, the most naive method with complete randomization also performs better than other adaptive randomization approaches. Adaptive randomization design may be more challenging to implement so a deeper study on settings that benefit the most from the proposed method would shed more light on the benefits and robustness of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your careful review and critical questions. Below, we add a point-by-point response to your questions:\n\n- _In the numerics, the \"No Surrogate\" approach seems to perform worse than complete randomization. Is there any insight on why this adaptive approach performs worse than the complete randomization approach? Does the \"No Surrogate\" approach also account for delays or does it utilize an existing adaptive randomization design approach?_\n\nRe: Sorry for the confusion. The complete randomization strategy **incorporates surrogates and adjusts for delays** in the primary outcome. The \"No Surrogate\" strategy **accounts for delays but does not incorporate any surrogates**. In general, these two strategies are not directly comparable, and the point we want to emphasize here is that **both strategies are inferior to the proposed CARA design**. Between complete randomization and the \"No surrogate\" strategy, which performs better **depends on the surrogates' quality**. If the surrogates are more predictive of the outcome, then incorporating the surrogates can greatly reduce the bias and variance, leading to an estimator that outperforms the \"No Surrogate\" strategy. According to the summary statistics in Table 2 in the Appendix, different levels of the WHO HIV stage lead to a significant variation in the conditional mean and variance of primary outcomes, which suggests the importance of incorporating surrogates and explains the superiority of this strategy in this case.\n\n- _The proposed method suggests applying regularization to the oracle optimization program to deal with potential multiple optima. Does the procedure tune the regularization parameter so it produces calibrated treatment allocation probabilities with better estimation error? Does this align with the cross-validation set-up briefly mentioned in the paper? If so, what is the exact cross-validation setup and if not how is the the tuning parameter chosen?_\n\nRe: Thank you for your question. We provide a comparison of various tuning parameter selections in Figure 1 of the attached pdf file. Overall, our design is not sensitive to tuning parameter choices, especially when the sample size is large. Based on Figure 1, as heuristic guidance, we recommend choosing a tuning parameter between 0 and 1 because a large tuning parameter may push the treatment allocation solutions to the boundary. As demonstrated in Figure 1, when the tuning parameter is chosen to be 10, the design is less efficient than the performance under a smaller tuning. We agree that having a cross-validation procedure for tuning parameter selection would be more interesting and systematic. We hope to explore this piece of methodological development in future work.\n\nThanks again for your engagement with our paper. We will incorporate your comments in the next revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DA9yf6yvAw", "reviewer": "Reviewer_Xjru", "summary": "This paper addresses the problem of covariate adaptive experimental design in the presence of time delayed outcomes. More specifically, we assume that participants are enrolled in waves, and the probability of treatment is given conditional on available user covariates. The target estimand is the average treatment effect for a long term/delayed outcome. To circumvent the issue of covariate adaptive randomization in the absence of observed outcomes, the authors consider a surrogate metrics. To derive the assignment procedure, the authors derive the influence function and semi-parametric efficiency bound, and then derive an objective based on them. A small number of empirical evaluations is provided which show the improvement in variance of the proposed estimator over complete randomization and approaches which do not incorporate surrogate information.", "questions": "It wasn't clear to me what is being assumed about the size of each wave of participants. I assume that the authors are assuming some conditions on the number of arriving participants? Also that these participants are arriving i.i.d. over time?  Does the proposed procedure give any guidance to the setting where the experimenter can choose the number of participants to enroll at each stage?", "limitations": "Yes. See above for questions.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This is an interesting addition to the literature on adaptive experimentation, and covariate adaptive experimentation. The authors do a nice job of clearly describing the task, motivating and deriving the influence function and semiparametric efficiency bound, and introducing a relatively simple algorithm for optimizing the bound. The proofs, to my reading, are correct and well presented. The problem is of clear practical importance, the authors motivate their approach through drug trials, but there are also a number of applications in both social scientific and industrial settings where the problem of delayed and long term outcomes arises and adaptive design is desirable. The authors also do a nice job of clearly walking through each step of the proposed algorithm and describing it's function and intuition.", "weaknesses": "The paper focuses on asymptotic results, which is sensible and provides good empirical performance. However, it would be useful to have finite sample analysis as well, since many of the applications of experimental design are sample-starved. It would have also been nice to have seen a slightly larger set of empirical results. The authors provide a small demonstration, but ideally the behavior of the propose algorithm is more rigorously evaluated empirically.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks so much for your careful review. We add a point-by-point response to your questions below.\n\n- *It would be useful to have a finite sample analysis.*\n\nRe: Thanks for the suggestion. We pursue asymptotic analysis to establish the distributional convergence and construct confidence intervals based on normal distribution approximations. In settings like A/B testing, many users are involved and asymptotic analysis suffices to help with inference. Nevertheless, we agree that in other settings such as clinical trials, sample size becomes a more important constraint which calls for a more delicate finite sample quantification. To translate the asymptotic results into a finite sample analysis, we can utilize the martingale concentration inequalities to quantify the tail of the estimates and also use Berry-Esseen bounds to obtain rates of distribution convergence. To avoid an overly lengthy paper, we hope to save these for future work.\n\n\n- *It would have also been nice to have seen a slightly larger set of empirical results.*\n\nRe: Thanks for the suggestion. In the rebuttal, we have added a new experiment to compare different tuning strategies for our design optimization program (Figure 1 in the attached pdf). Also, we have refined the presentation of the previous experiments by adding confidence bands to the curves (Figure 2 in the attached pdf). \n\n- *It wasn’t clear to me what is being assumed about the size of each wave of participants.*\n\nRe: In our setting, we assume that in each stage, the portions of participants to be recruited in each stage (the $r_t$'s) are decided a priori. This aligns with the practice in many real-world applications. For example, in clinical trials, researchers need to decide the number of patients to enroll as part of the protocol before the trial starts. Theoretically, this assumption makes it easier to discuss asymptotic regimes when $N$ goes to $\\infty$. \n\n- *Also these participants are arriving i.i.d. over time?*\n\nRe: In our setting, we assumed that the **potential outcomes of the participants across stages are i.i.d., but the truly observed outcomes are not**. The i.i.d. assumption is only imposed on the potential outcomes, which can be interpreted as the patients are drawn from the same target population. Nevertheless, due to the adaptive recruitment process, the observed outcomes are decided by both the distribution of the potential outcomes and the history-dependent data collection policy and thus are not i.i.d. anymore. \n\n- *Does the proposed procedure give any guidance to the setting where the experimenter can choose the number of participants to enroll at each stage?*\n\nRe: Thanks for this insightful point. Our results can provide some guidance in the following two aspects:\n\n1. For Stage $1$ to $D^\\star$, Condition (4) in Theorem 3 provides a sufficient condition to quantify the portion of people to recruit for the first $D^\\star$ stages to achieve the optimal treatment allocation strategy, which depends on the variance and the delay mechanism. Therefore, a promising strategy is to use the first stage to estimate the size of the quantity on the right-hand side of Condition (4), then adjust the portions in the following stages to satisfy the constraint. \n\n2. For Stage $D^\\star + 1$ to $T$, we apply the calibrated treatment allocation probabilities $\\hat{e}\\_l$. To make sure $\\hat{e}\\_l$ is bounded away from $0$ and $1$, we could set up a sequence of $n_s$ that satisfy the following inequality:\n$$\n    \\delta \\le \\frac{\\sum\\_{s=1}^{T-D^\\star} n_s \\cdot \\tilde{e}\\_{s} - \\sum_{s=1}^{D^*+1} n\\_s\\cdot \\hat{e}\\_{s}}{\\sum\\_{s = D^* + 2}^{T-D^*}n\\_{s}} \\le 1 - \\delta. \n$$\nThis will ensure a sufficient portion of participants in both the treatment and control groups for Stage $D^\\star + 1$ to $T$.\n\n\nThanks again for the insightful comments! We will incorporate these points in the next revision of our manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YPTlDofxe2", "reviewer": "Reviewer_AMr4", "summary": "This paper introduces an approach to optimizing treatment allocation for variance reduction, in the context of adaptive clinical trials.  The particular setting is one where there are delays in observing outcomes. These delays that are independent of the outcomes themselves, but they impact efficiency of estimation.  Moreover, there are short-term surrogate variables that can be used to improve estimation efficiency.  In this setting, this paper proposes to estimate the variance of different treatment allocation schemes from an initial stage of experimentation, and then use those estimates to choose an optimal treatment allocation scheme for the remainder of experimentation.\n\n---------------\nREBUTTAL UPDATE: I have updated my score from 5->6", "questions": "1. (W1) Could you clarify the similarity/difference of Theorem 1 to Theorem 2.1 of Kallus & Mao 2020?  It's fine if they are essentially the same, in my view, b/c the main contribution of the paper seems to lie in Theorems 2 & 3, but I just wanted to get some clarification in case I'm missing something.\n2. (W2) Is there a real-world scenario you have in mind where you would expect this causal structure to hold (i.e., delays are not informative of outcomes)?\n3. (W3) Could you clarify the optimality claim?  Is it just that the allocation converges in probability to the optimal allocation?  Or is there a stronger claim being made about the optimality of the procedure which uses the estimated allocation?\n4. (W4) Is my intuition correct, for where the variance reduction is coming from?  Or do you have other intuition on this point?\n\nThe authors are welcome to react to my points (W5) and (W6), but I don't think they need to do so, and they factored less into my score.", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "Overall, I found the setup of the problem to be original, and while I did not have time to review the proofs in depth, the results seems correct based on my knowledge of semiparametrics.  The theoretical results are fairly thorough: For instance, I appreciated Theorem 3, which speaks to the convergence of the chosen strategy to the optimal one.  In terms of clarity, the presentation of the technical details is similarly thorough, and while a bit dense and harder to skim, I found it straightforward to follow as written.  I also appreciated the experimental results, which I found to be reasonably compelling.", "weaknesses": "I noted a few areas of the paper that appeared a bit weaker, or at least warrant some clarification.  I'll number my points to make it easier to respond during the rebuttal period, and put them in rough order of priority.\n\n(W1) Significance / Novelty of Theorem 1: I understood Theorem 1 to be a necessary pre-cursor to the proposed adaptive strategy, but not the main contribution of the paper, so this concern is not as pressing as it could be.  However, it seems that Theorem 1 is essentially the same as Theorem 2.1 of Kallus & Mao 2020 [1], with the minor complication that different observations appear at different times, which leads to some additional notation to ensure the correct $e_{t}(X)$ is used in the relevant denominators.  Since the paper cites [1] (as [23] in the paper), it may be worth clarifying the similarity in the main text, and framing the contributions appropriately (e.g., my read is that Theorem 2 and 3 are the novel theoretical thrust of the paper).  I'm open to clarifications / corrections on this front.\n\n[1] https://arxiv.org/abs/2003.12408\n\n(W2) Significance of the proposed problem setting: From a practical perspective, I struggle to think of a setting where we would expect the causal graph in Figure 1 to hold, where the delay itself provides no information on the outcome $Y$, i.e., delays are not informative. E.g., in the HIV application, the outcome is viral load, and delay is in coming in for a visit (Lines 274-275).  It seems to be a classic case where delay could be caused by more severe illness, giving us some indication of $Y$.\n\n(W3) I found it slightly hard to parse the optimality claim (Theorems 2 and 3 and subsequent discussion).  It's clear that the estimated treatment allocation converges in probability to the optimal treatment allocation, but does that imply that this approach yields the optimal variance?  Could there be other ways to estimate the treatment allocation that also converges to the optimal one, but has better variance properties along the way, or is that somehow ruled out?\n\n(W4) Overall, I would have preferred to see more discussion of intuition and insights, rather than just going through the technical steps to derive the results.  It seemed that some parts could be made shorter to make space for this, e.g., Section 4 could be trimmed down a bit (e.g., define $\\hat{\\tau}^t(a, x, s), \\hat{\\tau}^t(a, x)$ and then use that notation in both Stage 1 and Stage 2 to $D^*+1$, or even just give the high-level comment that you estimate via empirical counts among the population where $Y$ is observed.)\n\nIn terms of missing intuition, I still lack some intuition for the \"signal\" being used in the optimization of the treatment allocation, and why we should expect it to vary across time periods.  I suppose the variance reduction comes from some understanding of which combinations of $X, A$ are likely to have missing data by the end due to delay, and so should be prioritized earlier in the treatment allocation process?\n\n(W5) There is a slight mismatch between the discussion in the introduction and the actual setting where the algorithm can be applied, particularly the restriction that the delay must be upper-bounded (see 181-182).  In contrast, Line 75-76 refers to the contribution \"optimize the statistical efficiency in the presence of temporary or permanent missingness\", where I would have interpreted \"permanent missingness\" to be $D = \\infty$. \n\n(W6) Interaction (or lack thereof) between delay and surrogates: This is a subjective / aesthetic point, so I place it last, but I was expecting more of a nuanced interaction between the delay and the surrogates.  In particular, the problem seems to factorize cleanly into (a) deriving an efficient estimator and variance lower-bound for any treatment allocation, (b) optimizing the allocation in a backward-looking period of data, and then (c) applying that allocation to the next period of data.  It's not clear the role of surrogates in this recipe, other than the influence they have on (a), because observing surrogates gives you no insight into future delays and therefore the optimal allocation going forward.\n\nMinor nits:\n* Algorithm 1 mentions \"Problem B\" on line 7, but I think that's really just Equation (2), Page 6, line 199.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review and critical comments. Below we add a point-by-point response.\n\n- *(W1)*: We agree that our efficiency bound may share some similarities with [1], but also hope to emphasize that deriving a neat decomposition of the tangent space is not straightforward when data come in multi-stages with delays (Step 2 in the proof of Theorem 1). This step is crucial for score function projection and finding the efficient influence function and requires more careful treatment in our setting than [1]. \n\nOn the other hand, as you have correctly pointed out, this result is the building block for the proposed adaptive design strategy and motivates point and variance estimators for ATE. Grounded by Theorem 1, Theorem 2 and 3 further provide guarantees for the optimality of design and validity of inference. We will further clarify these points in the next revision of our manuscript. \n\n- *(W2)*: In HIV trials, delays can occur for many reasons. For instance, viral load is typically measured every six months due to the protocol, which are independent of outcomes. Also, factors like economic status or distance to clinics are confounders for delay and outcome. Once adjusted, these delays become non-informative. Meanwhile, delays due to severity of HIV, as you mentioned, are informative for the outcome. In reality, the interplay among these factors is complex. In our simulation, we simplify the setup by considering a hypothetical setting where delays are due to protocol or structural factors.\n\n  In some cases, the delay is conditionally independent of the outcome. For example, if we are testing the effect of a vitamin supplement on blood vitamin levels and ask patients for regular check-ins, the delay is less likely to depend on the outcome since it does not cause severe symptoms like HIV. In general, there is a complicated delay-outcome interaction. We leave the extension as future work.\n\n- *(W3)*: Apologize for the confusion. The optimal allocation means the sequence of propensity scores that gives the minimal asymptotic variance matching the efficiency bound. So optimal allocation implies optimal variance. Nevertheless, the optimal treatment allocation that achieves the efficiency bound might not be unique, and one can add additional constraints or penalization (say $\\ell\\_2$, entropy, and so on). We will clarify this point in the next revision. \n\n- *(W4)*: Thanks for the suggestion. In the next revision, we will incorporate your suggestions to improve the presentation. Here is a modification plan:\n   \n   1. More discussion on the intuition. Our proposal has four steps. Step 1 is from Stage $1$ to $D^\\star+1$, an exploration step that learns parameters like the delay distribution, outcome model, etc. Step 2 is the policy optimization step after Stage $D^\\star + 1$, to compute the optimal allocation. Step 3 is policy calibration from Stage $D^\\star + 1$ to $T$, where we calibrate the allocation to match the optimal strategy. Step 4 is constructing point and variance estimators based on the data.  \n\n   2. Simplify the notation. The elements such as outcome models $\\hat{\\tau}^{(t)}(a,x,s)$ and $\\hat{\\tau}^{(t)}(a,x)$, and delay distribution $\\hat{\\rho}_a^{(t)}(0 | x)$ can be obtained by taking a stratified average over each combination of the $(A,X,S)$ levels. Also, we will defer the form of $\\hat{\\tau}$ and $\\hat{\\mathbb{V}}$ to the Appendix. \n\n   Moreover, your intuition behind variance reduction is correct. In our setting, the missing probability is different across stages and combinations of $(X, A)$ levels. We need enough data in the early stage to learn the mean and variance across arms, especially for those that are more likely to delay, so that we can optimize variance in later stages. As a toy example, consider a two-stage example, with the same $n_t$ for each stage.  We omit surrogates here for simplicity. The bound in Theorem 1 becomes:  \n\\begin{align}\n    & \\mathbb{V}\\_{SPEB}\n    =\\mathbb{E}[\\big(\\tau(1,X) -\\tau(0,X) - \\tau\\big)^2]\\\\\\\\\n    & + 2\\mathbb{E}\\Big[\\frac{\\sigma\\_1^2(X)}{ e\\_1(X) \\rho_1(1|X)}  + \\frac{\\sigma\\_0^2(X)}{(1-e\\_1(X)) \\rho_0(1|X)}\\Big]\\\\\\\\\n    & + 2\\mathbb{E}\\Big[\\frac{\\sigma\\_1^2(X)}{ e\\_2(X) \\rho_1(0|X)}  + \\frac{\\sigma\\_0^2(X)}{(1-e\\_2(X)) \\rho_0(0|X)}\\Big]. \n\\end{align}\n   The first stage is exploration for estimating the nuisance. We set $e_1(X) = 1/2$ to guarantee each $(x,a)$ level has enough sample. For the second stage, we minimize $\\mathbb{V}_{SPEB}$ over $e_2(X)$, which gives\n   \\begin{align}\n       e_2^\\star(x) = \\frac{\\sigma_1(x)/\\sqrt{\\rho_1(0\\mid x)}}{\\sigma_1(x)/\\sqrt{\\rho_1(0\\mid x)} \n       + \n       \\sigma_0(x)/\\sqrt{\\rho_0(0\\mid x)}}\n   \\end{align}\n   Therefore, the allocation depends on $\\sigma_a(x)/\\sqrt{\\rho_a(0\\mid x)}$. If some $(x,a)$ levels lead to larger variance and fewer delays, the design will sample more from those levels to facilitate variance reduction. \n\n  - *(W5)*: We apologize for the confusion. More rigorously, our framework allows the delay to take a finite number of values, instead of having a finite upper bound. Therefore, we can also add in $\\infty$ as long as there is a finite support for the rest of the values. In the next revision, we will further clarify this point. \n\n   - *(W6)*: One role of surrogates here is, as you have pointed out, to facilitate the derivation of the point estimator variance bound, which suggests that incorporating surrogates reduces variance (Remark 1). Another role is that it helps with policy optimization in the early stages when many primary outcomes are missing. It is interesting to think about the interplay between delay and surrogates. As a special setup, Kallus \\& Mao (2020) discussed missingness that depends on surrogates. We believe such an extension also works in our case and will leave this as a future work.\n\n  - Minor nits: will correct in the next revision.\n\nThanks again for the insightful comments! We will incorporate these points in the next revision of our manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CYGFanxD9a", "reviewer": "Reviewer_GAcb", "summary": "This article studies how to use interminate surrogate outcomes to estimate causal effects when the primary outcome is delayed in clinical trials. The author first proposes a novel Covariate-adjusted Response-adaptive (CARA)  design that supports efficient estimation of ATE using both surrogate and primary outcomes.  The authors prove the semiparametric efficiency bound for their ATE estimator and demonstrate the proposed design's efficiency through theoretical analyses and a synthetic HIV study.", "questions": "N.A.", "limitations": "N.A.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "1. The delay outcome problem studied in this paper is well-motivated. The covariate-adjusted response-adaptive (CARA)  design allows us to modify the treatment allocation mechanism over time, providing an interesting scenario to address the problem. \n2. The authors consider improving efficiency in both design and estimation. They show that the proposed design strategy converges to an oracle design and proposes a semiparametric efficient estimator of the ATE.\n3. The paper includes a synthetic HIV trial to illustrate the practical application and efficiency gains of the proposed design. The study demonstrates that the design reduces the standard deviation in estimating the ATE compared to other methods.", "weaknesses": "The main weakness of this article is that the presentation is too messy and lacks explanations in multiple places. \n1. In section 3, from line 159 to 179, the author introduces the efficient influence function (EIF) and variance bound in estimating $\\tau$. However, it seems like the main motivation of CARA is that the expectation of the delay outcomes and the delay mechanisms are unknown.   It is unclear why EIF and variance bound are presented at the beginning of section 3.\n\n2. Section 4 is also presented in a poor way. The authors go through the design steps with dense notation. The authors should provide more explanation of the steps taken in the design. The current Section 4 only shows that the design allows us to estimate the unknown quantity. But it is unclear how the design improves efficiency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful review and critical comments. Below we add a point-by-point response to the weaknesses you listed. \n\n- *The presentation is too messy and lacks explanations in multiple places. It is unclear why EIF and variance bound are presented at the beginning of section 3.*\n\nRe: We apologize for the confusion. The semiparametric efficiency bound is the building block for our adaptive design proposal, so we use the majority of Section 3 to present it. We agree that we should try to include more intuition on why we introduce the EIF and variance bound before diving into the technical details. In the next revision, we plan to add the following explanation in Section 3:\n\n> The idea of adaptation is to adaptively collect data and adjust the allocation probability across multiple experiment stages to minimize the variance of a certain estimator for ATE. One question is: what estimator should be used for treatment effect evaluation at the end of the study? For ATE estimation, one popular choice is the semiparametric efficient estimator, whose variance matches the efficiency lower bound across all semiparametric estimators. Therefore, it is a natural idea to set the semiparametric efficiency bound as the design objective and to optimize this objective with adaptive treatment allocation. As the building block, we first present the EIF and semiparametric efficiency bound in Theorem 1 to serve as the building block for our design strategy.  \n\n\n\n- *Section 4 is also presented in a poor way. The authors go through the design\nsteps with dense notation. The authors should provide more explanation of the\nsteps taken in the design.*\n\nRe: We apologize for the confusion. In the next revision, we will try to improve the presentation of Section 4. Here is a plan for modifying Section 4:\n   \n   1. We want to add more discussion on the intuition. From a high-level perspective, our proposed design consists of four steps. The first step is from Stage $1$ to $D^\\star+1$, a parameter exploration step that learns the nuisance parameters such as the delay distribution, outcome function, etc. The second step is the policy optimization step after Stage $D^\\star + 1$, where the experimenter uses the estimated parameter information to compute the optimal treatment allocation strategy. The third step is the policy calibration step from Stage $D^\\star + 1$ to $T$, where we calibrate the treatment probability to match the optimal treatment allocation strategy computed from the policy optimization step. The final step is constructing point and variance estimators based on the collected data.  \n\n   2. We can also simplify the notation by introducing some high-level explanations for the quantity estimation part. The nuisance parameters such as outcome regressions $\\hat{\\tau}^{(t)}(a,x,s)$ and $\\hat{\\tau}^{(t)}(a,x)$ as well delay distribution $\\hat{\\rho}_a^{(t)}(0 | x)$ can be obtained by taking a stratified average over each combination of the treatment and covariate levels. Also, we can deliver the expression of the point estimator $\\hat{\\tau}$ and the variance estimator $\\hat{\\mathbb{V}}$ to the Appendix.\n\nThanks very much again for your engagement with our paper. We will incorporate your comments in the next revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QfhTKsBB6x", "reviewer": "Reviewer_oWF4", "summary": "Clinical trials try to achieve the highest statistical precision using the fewest number of enrolled participants. One way to do this is by assigning more participants to the {covariate, arm} combinations with the highest outcome variance. We don't know the outcome variance before running the trial, which motivates the adaptive CARA method to adaptively assign more units with high outcome variance to treatment. This paper considers the setting where we don't directly observe the outcome, but instead observe a noisy surrogate along with a delayed outcome. The paper proposes a design objective (minimizing the semiparametric efficiency bound) and an explore-then-commit style algorithm for learning then applying a covariate-adaptive randomization using surrogate outcomes. The paper shows that this algorithm attains the semiparametric efficiency bound, and provides synthetic experiments based on an HIV study comparing this algorithm to complete randomization and CARA on observed outcomes only.", "questions": "I had trouble understanding the setup of the real-world study. If I understand correctly, measurements are taken every 6 months. The actual outcome Y is viral load (measured every 6 months), while the surrogate outcome is WHO stage (also measured every 6 months). I'm confused by the claim that WHO stage is available immediately, which I took to mean that the experimenters receive the WHO stage for every patient every 6 months. Does that not require the patient to visit a clinic, which might also lead to missingness? This isn't a question you need to address in your response, but if I have misunderstood something it might be worth clarifying.\n\nRequest: Neurips style is for citations to be by (Author, Year) instead of numerical. Please fix this in the next revision.\n\nThe introduction states that \"we allow D_it = infty\" (Line 139), but the algorithm only works with finite, known upper bound on D. It would be helpful for the authors to address the sensitivity of their results to misspecification of D*.", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "I am not very familiar with real-world clinical trials. However, from what I can tell, the work is well-grounded in problems that clinical trial designers care about. The mathematical model of surrogacy and delay is simple to state and understand while capturing the important aspects of the problem. The problem is explained clearly with explicit examples.", "weaknesses": "The empirical results are missing error bars. These should be added to show variation in the results across many replicates. This is especially important for two reasons: (a) to support the claim that the proposed method has lower bias than complete randomization, when the two are extremely close on the plots, and (b) to understand whether the \"no surrogate\" method is really worse in both bias and variance than complete randomization, as suggested by figure 2. If the \"no surrogate\" method is actually worse than complete randomization, it would be interesting to have a comment on this in the text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the careful review and critical comments. Below we give a point-by-point review to your comments.\n\n- *I'm confused by the claim that WHO stage is available immediately, which I took to mean that the experimenters receive the WHO stage for every patient every 6 months. Does that not require the patient to visit a clinic, which might also lead to missingness?*\n\nRe: Thanks for your question. WHO stage is a measure of the severity of HIV symptoms for the patients, and it can be collected from phone call check-in. However, clinical visits are required to measure HIV viral load. We will clarify this point in the next revision. \n\n- *Request: Neurips style is for citations to be by (Author, Year) instead of numerical. Please fix this in the next revision.*\n\nThanks for pointing this out. We will correct the style in the next revision. \n\n- *The introduction states that \"we allow $D\\_{it} = \\\\infty$\" (Line 139), but the algorithm only works with finite, known upper bound on $D$. It would be helpful for the authors to address the sensitivity of their results to the misspecification of $D^\\*$.*\n\nRe: We apologize for the confusion. More rigorously, Our framework actually allows the delay to take a finite number of values, instead of having a finite upper bound. Therefore, we can also add in $\\infty$ as long as there is a finite support for the rest of the values. In the next revision, we will further clarify this point. \n\n\n\n- *Weakness: The empirical results are missing error bars.*\n\nRe: Thank you for your suggestion. We have added confidence bands to the bias and standard deviation plots. See Figure 2 of the attached PDF file. From the bias plot Figure (A), we observe that the complete randomization design becomes unbiased when the sample size is over 800, while our proposed design is unbiased at a much smaller sample size. This suggests that our proposed design has a smaller finite sample bias. In Figure (B), our method demonstrates a significantly smaller standard deviation compared to the other two designs. For the \"No surrogate\" design (the green bands in both plots), we observe that its bias is rather significant in plot (A) as the green band fails to cover 0 when the sample size is smaller than 1600. We will further clarify this point in the next revision. \n\n\nThanks very much again for your engagement with our paper. We will incorporate your comments in the next revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely appreciate all the constructive feedback provided by our reviewers. We have made our best efforts to respond to the questions and comments raised by our reviewers. 

To answer some of the questions raised by our reviewers, we have attached a pdf file containing the following two figures:
- Figure 1: Comparison of bias and standard deviation across various tuning parameter selections.
- Figure 2: Addition of confidence bands to the bias and standard deviation comparison plot.

Thank you once again for your time and effort in reviewing our manuscript!

Meta-review: I concur with the reviewers' positive assessment of the paper. In particular, I think the problem is very well motivated from practice and merits attention, and I think the paper takes a very reasonable approach and studies it very nicely. Well done! It is thus a great addition to the literature on the subject, and I am very glad to recommend acceptance. There are some edits discussed in the rebuttal that I request the authors undertake. Also, while the authors do a nice job of reviewing the landscape of the literature, they could be more transparent in connections in their technical results. In particular, as one reviewer highlights, the connection of Theorem 1 to Theorems 2.1 and 2.2 of [23] and any significant differences should be discussed upfront (you mention some non-specific technical difficulties in the rebuttal but I did not follow since the decomposition seems to just follow from the ancillarity of the design, no?). Moreover, drawing the connection between an adaptive experiment and a sequence of different experiments and discussing the recent works of Bibaut et al. "Learning the Covariance ..." and "Nonparametric Jackknife ..." would add to the (already very nice) discussion of the landscape. I fully trust that the authors can undertake all this appropriately within the preparation of a camera ready version as this mostly involves adding a couple sentences here and there.
Decision: Accept (poster)