---
abstract: 'In randomized trials, appropriately adjusting for baseline variables and short-term outcomes can lead to increased precision and reduced sample size.  We examine the impact of such adjustment in group sequential designs, i.e., designs with preplanned interim analyses where enrollment may be stopped early for efficacy or futility.  We address the following questions: how much precision gain can be obtained by appropriately adjusting for baseline variables and a short-term outcome?  How is this precision gain impacted by factors such as the proportion of pipeline participants (those who enrolled but haven’t yet had their primary outcomes measured) and treatment effect heterogeneity?  What is the resulting impact on power and average sample size in a group sequential design?  We derive an asymptotic formula that decomposes the overall precision gain from adjusting for baseline variables and a short-term outcome into contributions from factors mentioned above, for efficient estimators in the model that only assumes randomization and independent censoring.  We use our formula to approximate the precision gain from a targeted minimum loss-based estimator applied to data from a completed trial of a new surgical intervention for stroke.  Our formula implies that (for an efficient estimator) adjusting for a prognostic baseline variable leads to at least as much asymptotic precision gain as adjusting for an equally prognostic short-term outcome.  In many cases, such as our stroke trial application, the former leads to substantially greater precision gains than the latter.  In our simulation study, we show how precision gains from adjustment can be converted into sample size reductions (even when there is no treatment effect).' 
author:
- 'Tianchen Qian[^1], Michael Rosenblum[^2], and Huitong Qiu[^3]'
bibliography:
- 'MISTIE\_TMLE\_short-term.bib'
title: 'Improving Precision through Adjustment for Prognostic Variables in Group Sequential Trial Designs: Impact of Baseline Variables, Short-Term Outcomes, and Treatment Effect Heterogeneity'
---

[*Keywords:*]{} Short-term outcome; Semiparametric efficiency; Targeted minimum loss-based estimator

Introduction {#sec:introduction}
============

Group sequential designs for randomized clinical trials involve interim analyses that may result in early stopping for efficacy or futility.  We consider trial designs where the primary outcome is measured at a fixed time (called the delay) after enrollment.  Prior to observing the primary outcome, participants may have baseline variables and a short-term outcome measured.  For example, in the MISTIE-II trial [@MISTIE_trial2008] for evaluating a surgical treatment for intracerebral hemorrhage the primary outcome is the modified Rankin Scale (mRS), which measures degree of disability, 180 days after enrollment.  A short-term outcome is mRS 30 days after enrollment.  The baseline (pre-randomization) variables include age and measures of stroke severity. 

Estimators that adjust for baseline variables are used in randomized trials because of the potential for increased precision and reduced sample size.  @pocock2002review, who surveyed 50 clinical trial reports from major medical journals, found that 36 adjusted for baseline variables.  Adjusting for prognostic baseline variables is recommended by regulators, e.g., @ICH9 [@EMAguideline2015; @FDA-guideline2019].  Also, in order to address missing outcomes, it may be useful to appropriately adjust for baseline and post-randomization variables (e.g., short-term outcomes) [@committee2010national; @EMAmissingdata].  Various methods for adjustment in randomized trials have been developed; see, e.g., @leon2003semiparametric [@davidian2005semiparametric; @tsiatis2008covariate; @rubin2008covariate; @moore2009binary; @moore2011relativeefficiency].  There has also been discussion on the benefits and limitations of adjustment, compared to the standard, unadjusted estimator [@yang2001efficiency; @freedman2008regression; @lin2013agnostic].  @moore2011relativeefficiency present a formula for the precision gain from adjusting for baseline variables in single stage trials. 

We focus on estimating the average treatment effect for continuous or binary outcomes.  We derive a formula for the asymptotic precision gain (measured by the relative efficiency compared to the unadjusted estimator) due to adjustment for baseline variables and a short-term outcome using efficient estimators, that is, estimators that extract the maximum prognostic information from these variables.  The formula can be used in trial planning to approximate the precision gain from adjustment, which can translate to reduced sample size. 

We show, using data of the MISTIE-II trial, how a modified version of these formulas can be used to approximate the precision gain from using the targeted minimum loss-based estimator (TMLE) of @Gruber2012TMLE while accounting for model misspecification.  We highlight a few implications of our formula.  Holding other factors fixed, larger treatment effect heterogeneity decreases the value added from adjusting for baseline variables of all participants; however, for the pipeline participants, adjusting for their baseline variables leads to increased precision only when there is treatment effect heterogeneity.  Adjusting for prognostic baseline variables typically leads to precision gains even if all participants have their primary outcomes observed.  In contrast, adjusting for a prognostic short-term outcome can only improve precision when there are participants with the short-term outcome observed but the primary outcome unobserved.  In Section \[sec:notation\], we introduce notation and assumptions.  In Section \[sec:theory-ate\], we present our formula for precision gain.  We show how to approximate the precision gain from using a TMLE in Section \[sec:tmle\].  In Section \[sec:application\], we apply our formula to data from the MISTIE-II trial.  In Section \[sec:simulation\], we conduct simulation studies.  Limitations and open problems are discussed in Section \[sec:discussion\]. 

Notation and Assumptions {#sec:notation}
========================

For participant $i$, let $A_i$ denote the indicator of study arm assignment.  We assume $A_i$ is binary-valued with $A_i=1$ for treatment and $A_i=0$ for control.  Denote by $W_i$ a vector of baseline variables measured before randomization.  All variables must be preplanned in the trial protocol.  Denote by $L_i$ a short-term outcome, which is observed at time $d_L$ after enrollment.  Though all of our results hold if $L_i$ is any prespecified variable measured after randomization, not necessarily the outcome measured at an earlier time, we focus on the case of short-term outcomes.  Also, for simplicity, we focus on cases with binary $L_i$; the results can be generalized to multidimensional, continuous $L_i$. Denote by $Y_i$ the primary outcome (continuous or binary-valued), which is observed at time $d_Y$ after enrollment with $d_Y \geq d_L$. The delays $d_L$ and $d_Y$ are prespecified and are common to all participants.  When followed up completely, participant $i$ contributes full data $(W_i, A_i, L_i, Y_i)$.

Let $n$ denote the sample size.  We assume that the set of vectors $\{(W_i, A_i, L_i, Y_i)\}_{1\leq i \leq n}$ are independent and identically distributed draws from the unknown joint distribution $P_0$ on generic data vector $(W, A, L, Y)$. The following assumption follows from randomization:

\[assump:randomization\] The study arm assignment $A$ is independent of the baseline variables $W$ and $P(A=1)=P(A=0)=1/2$.

Results for randomization probabilities other than $1/2$ are in Supplementary Material \[proof:thm:avar-ate\]. 

Define the average treatment effect as $\Delta = E(Y|A=1)-E(Y|A=0)$. The goal is to test the null hypothesis of no average treatment benefit: $H_0: \Delta \leq 0$ versus $H_1: \Delta > 0$.

We assume that participants are enrolled at a constant rate.  Since not all participants have full data observed at an interim analysis, we introduce indicators $C^L$ and $C^Y$ to denote that $L$ and $Y$, respectively, have been observed at or before a given analysis time.  For a participant, $C^L = 1$ if and only if $L$ is observed, and $C^Y = 1$ if and only if $Y$ is observed.  These variables get updated at each analysis.  We make the following assumptions:

\[assump:independent-censoring\] (Independent Censoring) $(C^L, C^Y)$ is independent of $(W,A,L,Y)$.

\[assump:monotone-censoring\] (Monotone Censoring) $C^L \geq C^Y$ with probability $1$.

An example where Assumptions \[assump:independent-censoring\] and \[assump:monotone-censoring\] hold is when administrative censoring is the only source of missingness and there are no changes over time in the population who are enrolled; this is what we simulate in Section \[sec:simulation\].  At any analysis time, each enrolled participant has one of the following missingness patterns:

-   $(C^L, C^Y) = (0,0)$: a pipeline participant with only $(W,A)$ observed;

-   $(C^L, C^Y) = (1,0)$: a pipeline participant with $(W,A,L)$ observed;

-   $(C^L, C^Y) = (1,1)$: a participant with $(W,A,L,Y)$ observed. 

We assume a nonparametric model for the joint distribution of the variables $(W, A, L, Y,C_L,C_Y)$, except that we make Assumptions 1-3.  The semiparametric efficiency results in the following sections are with respect to this model. 

For a group sequential design with $K$ stages, we consider the asymptotic setting where the maximum sample size goes to infinity such that at each interim analysis the proportions of enrolled participants with final and short-term outcomes observed converge to constants $p_y$ and $p_l$, respectively (where these limit proportions may differ by analysis time, but for conciseness we suppress their dependence on the analysis time).  This asymptotic regime corresponds to fixing the delay times $d_L, d_Y$ and analysis times, while increasing the enrollment rate.  We only consider analysis times with $p_y > 0$. The unadjusted estimator of $\Delta$ is the difference between the two arms of the sample mean of the primary outcome $Y$, using data from all participants with $Y$ observed, i.e., $\sum_i A_i C^Y_i Y_i/\sum_i A_i C^Y_i - \sum_i (1-A_i) C^Y_i Y_i/\sum_i (1-A_i) C^Y_i.$ All estimators of the average treatment effect $\Delta$ that we consider are regular, asymptotically linear (RAL), defined, e.g., by [@BKRW1993].  Any such estimator $\widehat{\Delta}$ of $\Delta$ is asymptotically normal, i.e., $\sqrt{n}(\widehat{\Delta} - \Delta) \stackrel{d}{\to} N(0, \sigma^2)$ as sample size $n$ goes to infinity, where $\stackrel{d}{\to}$ denotes convergence in distribution; the variance $\sigma^2$ is called the asymptotic variance of $\widehat{\Delta}$ and is denoted by ${\text{AVar}}(\widehat{\Delta})$. For example, the asymptotic variance of the unadjusted estimator is $2\{{\text{Var}}(Y|A=1)+{\text{Var}}(Y|A=0)\}/p_y$ at an analysis time where $p_y$ fraction of the enrolled participants have their primary outcome observed.  The asymptotic relative efficiency (ARE) between two RAL estimators $\widehat{\Delta}_1$ and $\widehat{\Delta}_2$ of $\Delta$ is the inverse of the ratio of their asymptotic variances: ${\text{ARE}}(\widehat{\Delta}_1, \widehat{\Delta}_2) = {\text{AVar}}(\widehat{\Delta}_2)/{\text{AVar}}(\widehat{\Delta}_1).$ The asymptotic sample size reduction for a Wald test of single stage trial by using estimator $\widehat{\Delta}_1$ instead of $\widehat{\Delta}_2$ is $1 - {\text{ARE}}(\widehat{\Delta}_1, \widehat{\Delta}_2)^{-1}$. See Section \[appen:are-sample-size-reduction\] of the Supplementary Material for a more detailed discussion. 

For any random vector $X$ and $a\in\{0,1\}$, let $E_a(Y \mid X)$ and ${\text{Var}}_a(Y \mid X)$ denote $E(Y\mid X, A=a)$ and ${\text{Var}}(Y\mid X, A=a)$, respectively.  For example, $E_1(Y \mid W) = E(Y \mid W, A=1)$, and $E_1(Y) = E(Y \mid A=1)$. When no subscript is used on $E$ or ${\text{Var}}$, these refer to expectation and variance, respectively, under the population distribution of the corresponding variables. 

Best Possible Precision Gain {#sec:theory-ate}
============================

Formula for Precision Gain
--------------------------

The following result gives the best possible asymptotic variance that can be achieved by a RAL estimator of the average treatment effect $\Delta$, in terms of the proportions $p_l, p_y$ of enrolled participants with $L$ and $Y$ observed, respectively, at a given analysis time:

\[thm:avar-ate\] Suppose Assumptions \[assump:randomization\], \[assump:independent-censoring\], and \[assump:monotone-censoring\] hold.  The asymptotic variance of any RAL estimator of $\Delta$ is at least $$\begin{aligned}
   & {\text{Var}}\{E_1(Y \mid W) - E_0(Y \mid W) \}
+  \sum_{a\in\{0,1\}} \frac{2}{p_l} {\text{Var}}_a\{ E_a(Y \mid L,W) - E_a(Y \mid W) \} \nonumber \\
+ &  \sum_{a\in\{0,1\}} \frac{2}{p_y} {\text{Var}}_a\{ Y - E_a(Y \mid L,W) \}.  \label{eq:thm:var-bound-ate}\end{aligned}$$

Lemma \[thm:avar-ate\] is a consequence of the efficient influence function of $\Delta$ in our semiparametric model that only makes the assumptions in Section \[sec:notation\].  This efficient influence function is given in Section \[proof:thm:avar-ate\] of the Supplementary Material, and follows from @robins1992semipara [@Scharfstein1999; @van2003semipara].  The TMLE estimator of @Gruber2012TMLE, which will be discussed in Section \[sec:tmle\], achieves this variance lower bound when all working models are correct.  When certain working models for this estimator are misspecified, one can approximate the corresponding ${\text{ARE}}$ by substituting regression model fits for conditional expectations in (\[eq:thm:var-bound-ate\]); see Section \[subsec:are-fitted\]. 

The first term in (\[eq:thm:var-bound-ate\]) characterizes the variance in the conditional treatment effect across different levels of $W$. We define the *treatment effect heterogeneity*, denoted by $\gamma$, by dividing the variance of the conditional treatment effect by the sum of variances of $Y$ in each arm: $$\begin{aligned}
\gamma = \frac{{\text{Var}}\{E_1(Y \mid W) - E_0(Y \mid W) \}}{\sum_{a\in\{0,1\}} {\text{Var}}_a(Y)}.  \label{def:gamma}\end{aligned}$$ The treatment effect heterogeneity $\gamma$ is invariant to linear transformations of $Y$, and is nonnegative.  When $\gamma = 0$ there is no treatment effect heterogeneity, i.e., the conditional treatment effect $E_1(Y \mid W)-E_0(Y \mid W)=\Delta$ with probability 1. 

For each arm $a \in \{0,1\}$, the variance of $Y$ given $A=a$ can be decomposed as follows: $${\text{Var}}_a(Y) = {\text{Var}}_a\{Y - E_a(Y\mid L,W)\} +  {\text{Var}}_a\{ E_a(Y\mid L,W) - E_a(Y \mid W) \} + {\text{Var}}\{E_a(Y\mid W)\}, \label{var_decomp_simple}$$ as proved in Section \[proof:lem:vardecomp\] in the Supplementary Material.  The last term in the display above motivates the following definition of the proportion of the variance in $Y$ explained by $W$ (summed across arms): $$R^2_W = \frac{\sum_{a\in\{0,1\}} {\text{Var}}\{E_a(Y \mid W)\} }{\sum_{a\in\{0,1\}} {\text{Var}}_a(Y)}  = 1 - \frac{\sum_{a\in\{0,1\}} {\text{Var}}_a\{Y - E_a(Y \mid W)\} }{\sum_{a\in\{0,1\}} {\text{Var}}_a(Y)} 
.$$ Similarly, the middle term in (\[var\_decomp\_simple\]) motivates the following definition of the proportion of additional variance in $Y$ explained by $L$ after accounting for $W$ (summed across arms): $$R^2_{L \mid W} = \frac{\sum_{a\in\{0,1\}} {\text{Var}}_a\{E_a(Y \mid L, W) - E_a(Y \mid W)\} }{\sum_{a\in\{0,1\}} {\text{Var}}_a(Y) }.  \label{def:RsquaredLgivenW}$$ A RAL estimator of $\Delta$ is efficient (at a given data generating distribution) if it achieves the asymptotic variance lower bound (\[eq:thm:var-bound-ate\]) in the semiparametric model. 

\[cor:re-ate\]

Suppose Assumptions \[assump:randomization\], \[assump:independent-censoring\], and \[assump:monotone-censoring\] hold.  Consider any analysis time.  The ARE between any efficient RAL estimator of $\Delta$ and the unadjusted estimator is $${\text{ARE}}= \frac{1}{ 1 + (p_y/2) \gamma - R^2_W -  (1 - p_y / p_l) R^2_{L\mid W} }.  \label{eq:re-ate}$$

The denominator of the right side of the above display never exceeds 1, which follows since $\gamma \leq 2 R^2_W$ by the Cauchy-Schwarz inequality. 

Impact of Baseline Variables on Precision Gain {#sec:impactbaseline}
----------------------------------------------

In order to isolate the impact of baseline variables $W$ on the ARE , we consider the case where there is no impact of $L$, i.e., when $R^2_{L \mid W} = 0$. This is the case, for example, when $L$ is independent of $Y$ given $A$ and $W$. Then the ${\text{ARE}}$ from adjusting for baseline variables $W$ depends on three factors: the proportion of variance in $Y$ explained by $W$ ($R^2_W$), the proportion of participants with $Y$ observed among those enrolled ($p_y$), and the treatment effect heterogeneity ($\gamma$).  We plot in Figure \[fig:are-w-ate\] the ${\text{ARE}}$ versus $p_y$, considering different combinations of $R^2_W$ and $\gamma$.

![ Asymptotic relative efficiency between an efficient estimator and the unadjusted estimator for estimating $\Delta$, when only the baseline variable $W$ is prognostic ($R^2_{L \mid W} = 0$ in all curves).[]{data-label="fig:are-w-ate"}](REW_ATE_onlylinetype.png){width="80.00000%"}

A summary of what is happening in Figure \[fig:are-w-ate\] is that with $p_y=1$ and $R^2_W$ fixed, greater treatment effect heterogeneity ($\gamma$) lessens the precision gain from prognostic baseline variables.  However, with fixed $R^2_W>0$ and $\gamma>0$, some precision gain is restored as we decrease $p_y$ from $1$ to $0$ (i.e., as we move from right to left along any line with $\gamma>0$ in Figure \[fig:are-w-ate\]).  We describe the intuition for these phenomena below.  The key idea is that precision gains result from adjusting for chance imbalances in $W$ both (i) between study arms among the participants with $Y$ observed, and (ii) between the full sample and the subset with $Y$ observed.  Whether the adjustments in (i) and (ii) lead to precision gains or not is determined by the treatment effect heterogeneity $\gamma$, which influences whether the estimated means in each arm are adjusted in opposite directions (leading to variance reduction) or in the same direction (leading to cancellation and no impact).  This is analogous to constructive versus destructive interference between waves, which can lead to cancellation or amplification as the waves come together. 

To give intuition for the phenomena in Figure \[fig:are-w-ate\], for the remainder of this subsection we fix $R^2_W$ and consider the simple case of a single, binary-valued $W$ representing being less than 65 years old at baseline, and primary outcome $Y$ being the indicator of having mRS at most 3 (a good outcome) at 180 days.  We focus on the stratum $W=1$ and assume that the under 65 subset of the study population would have better outcomes on average than the overall study population if all were assigned to the control arm, i.e., $E(Y|A=0,W=1)-E(Y|A=0)>0$. We next consider $p_y=1$ and the opposite extremes of no treatment effect heterogeneity (where efficiency gains from adjusting for $W$ are maximal) and maximum treatment effect heterogeneity (where there are no efficiency gains from adjusting for $W$). 

First, consider the case of $p_y=1$ and no treatment effect heterogeneity ($\gamma=0$), i.e., $E(Y|A=1,W)-E(Y|A=0,W)$ equals the constant $E(Y|A=1)-E(Y|A=0)=\Delta$. Combining this with the assumption above that those under 65 have better outcomes on average than the overall study population under assignment to the control arm ($E(Y|A=0,W=1)-E(Y|A=0)>0$), it follows that those under 65 also have better outcomes on average than the overall study population under assignment to the [*treatment*]{} arm, i.e., $E(Y|A=1,W=1)-E(Y|A=1)=E(Y|A=0,W=1)-E(Y|A=0)$. If by chance there are proportionally more under 65 participants assigned to the treatment arm than the control arm, then the unadjusted estimator of the mean outcome in the treatment (control) arm is biased upward (downward) conditional on the chance imbalance [@jiang2019robust].  An efficient estimator adjusts for this by decreasing the unadjusted estimate of $E(Y|A=1)$ and by increasing the unadjusted estimate of $E(Y|A=0)$; the net impact is to decrease the estimate of $\Delta$. Alternatively, if by chance there are proportionally fewer under 65 participants assigned to the treatment arm, then by a symmetric argument an efficient estimator adjusts for this by increasing the unadjusted estimate of $\Delta$. The overall impact of such adjustments across many hypothetical trials, is to remove the variance caused by chance imbalances across arms in the stratum $W=1$. This is, intuitively, why there is an efficiency gain when $\gamma=0$.

Now consider $p_y=1$ and the opposite extreme of maximum treatment effect heterogeneity ($\gamma=2R^2_W$), which occurs when $E(Y|A=1,W)+E(Y|A=0,W)$ equals the constant $E(Y|A=1)+E(Y|A=0)$. An analogous argument (given in Section \[appen:adjustmentintuition\] of the Supplementary Material) to the previous paragraph shows that adjustment for chance imbalances in $W$ leads to equal changes (in the same direction) in the estimated means in each arm; the net impact is that the difference between estimated means across arms is not changed (due to cancellation).  Adjustment has no impact in this case, and leads to no variance reduction. 

Define the chance imbalance between arms in the proportion of participants with $W=1$ as $\widehat{I}=\sum_{i=1}^n A_iW_i / \sum_{i=1}^n A_i - \sum_{i=1}^n (1-A_i)W_i / \sum_{i=1}^n (1-A_i)$. E.g., $\widehat{I}>0$ means that there are proportionally more over 65 participants in the treatment arm than the control arm.  An efficient estimator will adjust for the resulting conditional bias (see e.g., [@jiang2019robust]) by decreasing the unadjusted estimate of the treatment arm mean outcome $E(Y|A=1)$ by $E(Y|A=1,W=1)*\widehat{I}$. Analogously, an efficient estimator will increase the unadjusted estimator of the control arm mean outcome $E(Y|A=0)$ by $E(Y|A=0,W=1)*\widehat{I}$. The net impact on the efficient estimator of the average treatment effect $\Delta=E(Y|A=1)-E(Y|A=0)$ is to subtract $\{E(Y|A=1,W=1)+E(Y|A=0,W=1)\}\widehat{I}$ from the unadjusted estimator.  In this way, the efficient estimator adjusts for chance imbalances in $W$ across arms by subtracting $\{E(Y|A=1,W=1)+E(Y|A=0,W=1)\}|\widehat{I}|$ for trials with $\widehat{I}>0$ and adding $\{E(Y|A=1,W=1)+E(Y|A=0,W=1)\}|\widehat{I}|$ for trials with $\widehat{I}<0$. Removing this variation, which is due to imbalances in $W$ across arms and not due to the treatment effect, from the unadjusted estimator leads to a variance reduction of $Var\{E(Y|A=1,W)+E(Y|A=0,W)\}$, asymptotically, in the efficient estimator [@jiang2019robust].  We next relate this characterization of the asymptotic variance reduction to the impact of $R^2_W$, $p_y$, and $\gamma$ on the ${\text{ARE}}$.

Given fixed $R^2_W>0$, $p_y=1$, and $Var_0(Y)=Var_1(Y)=1/2$, the variance reduction due to adjusting for $W$ is maximal when there is no treatment effect heterogeneity, i.e., when $E_1(Y|W)-E_0(Y|W)$ is the same (equal to $\Delta$) across all strata of $W$. This is because of the relationship $Var\{E(Y|A=1,W)+E(Y|A=1,W)\}= 2R^2_W - \gamma$, i.e., the variance reduction from adjusting for baseline variables $W$ as described above equals twice the variance in $Y$ explained by $W$ minus the treatment effect heterogeneity.  This relationship implies that greater treatment effect heterogeneity leads to lower precision gains.  In the case where $\gamma$ is the maximum possible value, i.e., $2R^2_W$, there is no variance reduction due to adjustment.  This case corresponds to $E(Y|A=1,W)=-E(Y|A=0,W)$, i.e., the adjustment $\{E(Y|A=1,W)+E(Y|A=0,W)\} \widehat{I}$ in the previous paragraph is always $0$ (and so has no impact).  Above, we considered $p_y=1$, i.e., no pipeline participants.  We now fix $R^2_W>0$ and $\gamma>0$, and consider the impact of decreasing $p_y$ from $1$ to $0$. This leads to increased ${\text{ARE}}$ because an efficient estimator can extract information from pipeline participants if $\gamma>0$; in contrast, the unadjusted estimator does not use any information from pipeline participants.  Intuition for why $\gamma>0$ is necessary in order for pipeline participant information to be useful is given in Section \[appen:adjustmentintuition\] of the Supplementary Material. 

Impact of a Short-term Outcome on Precision Gain {#impactshortterm}
------------------------------------------------

To isolate the impact of a short-term outcome $L$ on precision gain, we set $R^2_W = 0$ in so that only $L$ is prognostic.  The precision gain from adjusting for short-term outcome $L$ depends on two factors: the proportion of variance in $Y$ explained by $L$ after accounting for $W$ ($R^2_{L \mid W}$), and the proportion of participants with $Y$ observed among those with $L$ observed ($p_y / p_l$).  In Figure \[fig:are-l-ate\], we plot the ${\text{ARE}}$ against $p_y / p_l$, and consider two values of $R^2_{L \mid W}$.

![ Asymptotic relative efficiency between an efficient estimator and the unadjusted estimator for estimating $\Delta$, when only the short-term outcome $L$ is prognostic ($R^2_W = 0$ in both curves).  The solid line corresponds to a higher prognostic value $R^2_{L \mid W}=0.25$ and the dashed line corresponds to a lower prognostic value $R^2_{L \mid W}=0.1$. []{data-label="fig:are-l-ate"}](REL_ATE.png){width="48.00000%"}

Smaller $p_y/p_l$ and larger $R^2_{L \mid W}$ generally increase the precision gain from adjusting for $L$; this is because $L$ adds value just for participants with $L$ but not $Y$ observed, and $R^2_{L \mid W}$ quantifies the prognostic value of $L$ beyond the variance in $Y$ explained by $W$. When there are no pipeline participants, which would occur at the end of a trial with no early stopping or dropout, we have $p_y/p_l = 1$ and the ${\text{ARE}}$ is 1, i.e., adjusting for $L$ is useless.  This is because adjusting for $L$ helps by, in each arm separately, accounting for chance imbalances between the participants with $Y$ observed and the participants with $L$ but not $Y$ observed; when $p_y/p_l = 1$, these groups are identical and no adjustment can be made. 

Recall that we are operating in the semiparametric model defined by Assumptions 1-3; in particular, we are not making any assumptions about the relationships among the variables $(Y,A,L)$. Therefore, we cannot adjust for observed imbalances in $L$ between arms $A \in \{0,1\}$, since these imbalances may be due to the impact of arm assignment.  Given fixed $R^2_{L \mid W}>0$, larger $p_y / p_l$ decreases the precision gain from adjusting for $L$, as seen in Figure \[fig:are-l-ate\].  In a trial with constant enrollment rate and $d_L < d_Y$, the precision gain from adjusting for $L$ typically attenuates at later stages of a group sequential design due to $p_y / p_l$ being nondecreasing over time.  In the case where missingness is only due to administrative censoring, $p_y / p_l$ starts at $0$ at time $d_L$ and eventually becomes 1 after enrollment stops and enough time has elapsed for all enrolled participants to have $Y$ observed. 

Comparison of Equally Prognostic Baseline Variable and Short-term Outcome {#impactcomparison}
-------------------------------------------------------------------------

For any $q: 0 < q \leq 1$, we compare the ${\text{ARE}}$ between two cases: $R^2_W = q, R^2_{L \mid W} = 0$ (only baseline variable prognostic) and $R^2_W = 0, R^2_{L \mid W} = q$ (only short-term outcome prognostic).  The ${\text{ARE}}$ in the former case is larger or equal to that in the latter case.  Equality occurs if and only if $p_l=1$ and treatment effect heterogeneity in the first case is the maximum possible ($\gamma=2R^2_W=2q)$, as proved in Section \[appen:proof-section3.4\] of the Supplementary Material.  Intuition for this is given in Section \[appen:adjustmentintuition\] of the Supplementary Material.  The equality can be seen graphically in that the two lines in Figure \[fig:are-l-ate\] (corresponding to prognostic value $0.25$ and $0.1$) are identical to the two corresponding lines in Figure \[fig:are-w-ate\] with $\gamma=2R^2_W$; the horizontal axes of the two figures are identical in the special case of $p_l=1$.

Estimator that Adjusts for Baseline Variables and Short-term Outcomes {#sec:tmle}
=====================================================================

Targeted Minimum Loss-Based Estimator
-------------------------------------

The targeted minimum loss-based estimator (TMLE) of [@Gruber2012TMLE], which builds on the ideas of , is implemented in the package [@ltmlePackage].  We use it to estimate $\Delta$ and call it the *adjusted estimator*. 

Our implementation of this TMLE for a binary outcome $Y$ involves fitting logistic regression working models.  Such models are fit first for the conditional probabilities of censoring given the observed history before censoring: $P(C^L=1 \mid A, W)$ and $P(C^Y=1 \mid L, A, W)$. We also fit such a model for study arm assignment $P(A=1 \mid W)$. Lastly, we fit logistic regression models for $E(Y| L,A,W)$ and for $E(Y| A,W)$, given the corresponding observed histories; we refer to these as outcome regression models.  The last model is fit using sequential regression, an idea from [@Robins2000TMLE; @BangRobins2005TMLE]; see our R code (Github link given at end of paper) for the implementation of the software of [@ltmlePackage] that we used.  All models involve an intercept and main terms for each regressor variable.  Each regression is performed using all participants for whom the relevant variables are uncensored.  For a continuous-valued outcome $Y$, linear outcome regression models could be used or, if the outcome is bounded then it can be rescaled to the interval $[0,1]$ and logistic regression can be used as described by [@grubervanderLaan2010].  The adjusted estimator initially uses the aforementioned regression model fits.  It then updates each outcome regression model by adding a new covariate built from the other model fits, as described by [@Gruber2012TMLE].  The final estimator of the average treatment effect $\Delta$ is based on the updated outcome regression model fit of $E(Y|A,W)$. It is computed by first generating a prediction $Y_{a,i}$ of the outcome $Y$ under each hypothetical assignment to arm $a \in \{0,1\}$ for each participant $i$ by substituting $W=W_i$ and $A=a$ into this model fit.  The estimator of $\Delta$ is $(1/n)\sum_{i=1}^n (Y_{1,i}-Y_{0,i})$.

Assumptions 1-3 from Section \[sec:notation\] imply that the censoring models and the model for $P(A=1|W)$ are correctly specified.  The results of @Gruber2012TMLE [Section 4] imply that the adjusted estimator is a consistent estimator of the average treatment effect $\Delta$. This holds regardless of the correctness of the outcome regression models.  That is, these models could be arbitrarily misspecified and still the adjusted estimator converges in probability to $\Delta$ as sample size goes to infinity.  If the outcome regression models are correct, then the adjusted estimator achieves the asymptotic variance lower bound in (\[eq:thm:var-bound-ate\]), in which case it is semiparametric, locally efficient [@Gruber2012TMLE Section 4].  Other estimators with the above properties include the augmented, inverse probability of treatment (and censoring) estimators of @Robins2000TMLE and @BangRobins2005TMLE. 

Approximating the Relative Efficiency between the Adjusted Estimator and the Unadjusted Estimator {#subsec:are-fitted}
-------------------------------------------------------------------------------------------------

We focused on the asymptotic relative efficiency between an efficient estimator and the unadjusted estimator in Section \[sec:theory-ate\].  To connect our results regarding efficient estimators to the adjusted estimator, we use results from @Gruber2012TMLE [Section 5.3].  These results imply that if all regression models used in the TMLE are correctly specified, then plugging the corresponding model fits into the formulas (\[def:gamma\])-(\[def:RsquaredLgivenW\]) and then using these estimates of $\gamma,R^2_W,R^2_{L \mid W}$ in (\[eq:re-ate\]) results in a consistent estimator of the ${\text{ARE}}$ (called the plug-in estimator).  This estimator of the ${\text{ARE}}$ is asymptotically conservative, meaning that the ${\text{ARE}}$s may be underestimated but not overestimated, if the models for censoring and for $P(A=1|W)$ are correctly specified; since these models are correct under Assumptions \[assump:randomization\]-\[assump:monotone-censoring\], it follows that the plug-in estimator can serve as a (possibly conservative) asymptotic approximation to the ${\text{ARE}}$ between the adjusted estimator and the unadjusted estimator. 

Denote by $\widehat{E}(Y \mid X=x)$ the predicted value of $E(Y \mid X=x)$ from a model fit.  Let $\widehat{{\text{Var}}}(\cdot)$ denote the sample variance over participants from both arms, and $\widehat{{\text{Var}}}_a(\cdot)$ denote the sample variance over participants from arm $a$. We estimate $R^2_W$, $R^2_{L\mid W}$, and $\gamma$, respectively, by $$\widehat{R}^2_W  = \frac{ \sum_{a\in\{0,1\}} \widehat{{\text{Var}}}\{ \widehat{E}(Y \mid W, A=a) \}}{ \sum_{a\in\{0,1\}} \widehat{{\text{Var}}}_a(Y) },$$ $$\widehat{R}^2_{L \mid W}  = \frac{ \sum_{a\in\{0,1\}} \widehat{{\text{Var}}}_a\{\widehat{E}(Y \mid W, L, A=a) - \widehat{E}(Y \mid W, A=a)\} }{ \sum_{a\in\{0,1\}} \widehat{{\text{Var}}}_a(Y) },$$ $$\widehat\gamma = \frac{\widehat{\text{Var}}\{ \widehat{E}(Y \mid W, A=1) - \widehat{E}(Y \mid W, A=0) \}}{\sum_{a\in\{0,1\}} \widehat{{\text{Var}}}_a(Y)}.$$ In Sections \[sec:application\]-\[sec:simulation\] when calculating the conditional expectations $\widehat{E}$ in the formulas for $\widehat{R}^2_W$ and $\widehat{R}^2_{L \mid W}$, we used logistic regression model fits with main terms.  When calculating $\widehat\gamma$, we used similar model fits except including interactions, since we wanted to give an opportunity to detect treatment effect heterogeneity.  The estimated $R$-squared quantities above are relevant for continuous and binary outcomes in measuring the precision gain from adjustment when estimating $\Delta$; this generalizes an idea from @moore2009binary, who give an R-squared formula related to $\widehat{R}^2_W$ except that it marginalizes over $A$. 

Applying Precision Gain Formula to Trial Data {#sec:application}
=============================================

Trial Example: MISTIE-II {#subsec:mistie}
------------------------

MISTIE-II is a Phase II randomized clinical trial evaluating a new surgical treatment for intracerebral hemorrhage.  The treatment is called Minimally-Invasive Surgery Plus rt-PA for Intracerebral Hemorrhage Evacuation, abbreviated as MISTIE [@MISTIE_trial2008].  In the MISTIE-II dataset, the primary and short-term outcomes of each participant correspond to the Modified Rankin Scale (mRS), measured at different times after enrollment.  We use the variables in Table \[tab:MISTIEvars\].  The primary outcome $Y$ is a binary indicator of a successful outcome ($\mbox{mRS}\leq3$) at 180 days after enrollment.  The short-term outcomes $L^{(1)}$ and $L^{(2)}$ are the indicators of mRS no larger than 4 at 30 and 90 days after enrollment, respectively.  The cutoff 4 for $L^{(1)}$ and $L^{(2)}$ was chosen because there were relatively few participants with mRS 3 or less at 30 or 90 days after enrollment.  The treatment assignment indicator $A$ denotes whether a participant is assigned to the surgical treatment ($A=1$) or to standard of care ($A=0$).  Baseline variables $W^{(1)}$-$W^{(4)}$ are age (dichotomized at 65 years), NIHSS (NIH Stroke Scale total score, quantifying stroke-caused impairment), ICH (intracerebral hemorrhage volume), and GCS (Glasgow Coma Scale), all measured before randomization.  The baseline variables (except for age) are treated as continuous variables in the regression models.  The dataset has 100 participants. 

  Name        Description                                                 
  ----------- ----------------------------------------------------------- --
  $W^{(1)}$   baseline age (years), dichotomized at $\leq 65$             
  $W^{(2)}$   baseline NIHSS, ordinal                                     
  $W^{(3)}$   baseline ICH, ordinal                                       
  $W^{(4)}$   baseline GCS, ordinal                                       
  $A$         treatment indicator, 1 being MISTIE                         
  $L^{(1)}$   mRS at 30 days, dichotomized by thresholding at $\leq 4$    
  $L^{(2)}$   mRS at 90 days, dichotomized by thresholding at $\leq 4$    
  $Y$         mRS at 180 days, dichotomized by thresholding at $\leq 3$   

  : Variables that we use from the MISTIE-II dataset.  Ordinal-valued basline variables below are treated as continuous in our regression models. []{ data-label="tab:MISTIEvars"}

We let $W^\text{full} = (W^{(1)}, W^{(2)}, W^{(3)}, W^{(4)})$ and $L^\text{full} = (L^{(1)}, L^{(2)})$. These variables are used in constructing data generating distributions for our simulations in Section \[sec:simulation\].  However, only the smaller subsets of variables $W = (W^{(1)}, W^{(4)})$ and $L = L^{(1)}$ are made available to the adjusted estimator (both in Section \[subsec:application-precision-gain\] and \[sec:simulation\]).  We included the “extra” variables $W^{(2)}, W^{(3)}, L^{(2)}$ (which the adjusted estimator does not have access to) in our data generating distributions for simulations so that the regression models used by the adjusted estimator will be misspecified; this was done since we expect at least some misspecification to occur in practice. 

Approximate Precision Gain by Using the Adjusted Estimator on MISTIE-II Data {#subsec:application-precision-gain}
----------------------------------------------------------------------------

Applying the method in Section \[subsec:are-fitted\] to the MISTIE-II data, the estimated R-squared quantities are $\widehat{R}^2_W = 0.36$, $\widehat{R}^2_{L \mid W} = 0.08$, and $\widehat\gamma = 0.02$. This indicates that the baseline variables are strongly prognostic, that (after accounting for the baseline variables) the short-term outcome $L$ is only mildly prognostic, and that there is little (if any) treatment effect heterogeneity.  At the end of a trial, if every participant has $Y$ observed, we have $p_l = p_y = 1$. Substituting these and the estimated values $\widehat{R}^2_W, \widehat{R}^2_{L \mid W}, \widehat{\gamma}$ into (\[eq:re-ate\]) gives an approximate ${\text{ARE}}$ of $1.53$ comparing the TMLE versus the unadjusted estimator.  If instead there had been no treatment effect heterogeneity ($\widehat \gamma = 0$), then the approximate ${\text{ARE}}$ would be slightly larger ($1.55$), assuming $\widehat R^2_W$ and $\widehat R^2_{L \mid W}$ remain fixed.  Now consider a hypothetical interim analysis where 80% of the enrolled participants have $Y$ observed and 95% of the enrolled participants have $L$ observed (e.g., at the 3rd interim analysis of the group sequential design with ${n_{\rm max}}= 480$ considered in Section \[subsec:simulationresults-2\]; see Table \[tab:sample-size\] in the Supplementary Material).  Then we have $p_y = 0.8$, $p_l = 0.95$, and the approximate ${\text{ARE}}$ is $1.56$.

Equivalently, in each of the three aforementioned situations, the adjusted estimator approximately reduces the required sample size to achieve a desired power by $35\%$, $36\%$, and $36\%$, respectively, compared to the unadjusted estimator. ( The asymptotic sample size reduction is computed as $1-1/{\text{ARE}}$ as described in Section \[sec:theory-singlearm\] of the Supplementary Material.)  These are all quite similar, which is primarily due to the relatively small values of $\widehat \gamma$ and $\widehat R^2_{L \mid W}$. The magnitudes of the AREs above are not unusual in the MISTIE-II population–see @optimising2009should–and result from some baseline variables (especially NIHSS) being strongly prognostic for the outcome.  Approximations of the ${\text{ARE}}$ as above can be applied to trial planning, where one may use previous data to estimate the variance explained by baseline variables and short-term outcomes and the treatment effect heterogeneity, and use to get a rough projection of the precision gain from adjusting for baseline variables and short-term outcomes.  If treatment effect heterogeneity is difficult to estimate *a priori*, one could use $\gamma = 0$ as the best case scenario (no treatment effect heterogeneity, corresponding to maximal precision gain), with the expectation that $\gamma > 0$ will result in attenuated precision gain.  A conservative approach is to assume no precision gain when selecting the initial sample size per stage, but then use preplanned sample-size re-estimation (using the estimated variance) to potentially shrink the per-stage sample size to reflect the estimated precision gains based on accrued data.  Such adaptations based on estimating a nuisance parameter are generally acceptable to regulators such as the U.S. Food and Drug Adminsitration [@fda:2016; @FDA].  This approach has important limitations given in Section \[sec:discussion\]. 

Simulations of Group Sequential Design {#sec:simulation}
======================================

Data Generating Distributions for Simulated Trials Based on MISTIE-II {#subsec:dgm}
---------------------------------------------------------------------

We conducted simulation studies to assess how good of an approximation is provided by the precision gain formula (\[eq:re-ate\]), and to examine the sample size reduction by using the adjusted versus unadjusted estimator in a group sequential design.  In order to mimic key features of the MISTIE-II trial, we use a resampling-based algorithm to generate participants for our simulated trials.  This algorithm generates simulated trials with the following properties:

1.   The treatment assignment is independent of baseline variables. 

2.   The relative efficiency between the adjusted estimator and the unadjusted estimator is similar to that calculated directly from the MISTIE-II dataset. 

We define eight data generating distributions, called “settings”.  The goal is to consider four situations involving $W$ and/or $L$ being prognostic or not for $Y$; these are denoted by ${progn_{W,L}}$ (both prognostic), ${progn_{W}}$ (only $W$ prognostic), ${progn_{L}}$ (only $L$ prognostic), and ${progn_{\emptyset}}$ (neither prognostic).  For each of these four situations (called “prognostic settings”), we construct distributions with the following two average treatment effects: $\Delta=0$ (no effect) or $\Delta=0.122$ (benefit equal to the unadjusted estimate from the MISTIE-II dataset).  Details of the data generating algorithm for the eight settings are in Section \[appen:dgm\] of the Supplementary Material.  Table \[tab:rsq-mistie\] gives the approximate $R$-squared and $\gamma$ values for the eight settings.  The only source of missing data in our simulations is administrative censoring, i.e., participants who enrolled but have not been in the trial long enough to have outcomes measured. 

  ------------------ ------------ ---------------- -- ------------ ---------------- -- ------------ ---------------- -- ------------ ----------------
                                                                                                                                     
                     $\Delta=0$   $\Delta=0.122$      $\Delta=0$   $\Delta=0.122$      $\Delta=0$   $\Delta=0.122$      $\Delta=0$   $\Delta=0.122$
  $R^2_{W}$          0.35         0.36                0.35         0.36                0            0                   0            0
  $R^2_{L \mid W}$   0.08         0.07                0            0                   0.30         0.30                0            0
  $\gamma$           0            0.01                0            0.01                0            0                   0            0
  ------------------ ------------ ---------------- -- ------------ ---------------- -- ------------ ---------------- -- ------------ ----------------

  : Approximate $R$-squared and $\gamma$ for each of the eight data generating distributions (settings) in our simulation study, computed by simulating $10^6$ participants under each distribution and computing $\widehat{R}^2_W, \widehat{R}^2_{L \mid W}, \widehat{\gamma}$, respectively.[]{data-label="tab:rsq-mistie"}

Group Sequential Trial Design Analysis Timing and Early Stopping Rule {#subsec:design}
---------------------------------------------------------------------

We assume that $d_L = 30$ days and $d_Y = 180$ days.  The goal is to control Type I error at level 0.025 and to have at least 80% power to reject the null hypothesis $H_0: \Delta \leq 0$ when $\Delta = 0.122$.

We used the group sequential design framework in @hampson2013group, which involves interim analyses where a choice is made to stop or continue enrollment; if enrollment is stopped, then one waits until all pipeline participants complete the trial and then a hypothesis test (called a decision analysis) is conducted.  Full details of the design are in Supplementary Material \[appen:gst-multiple-testing\].  Denote by $K$ the total number of stages.  We set $K=5$ stages in our simulated trials.  Participants were enrolled at the constant rate of 140 participants per year based on the projection for the enrollment rate in the MISTIE-III trial [@MistieIIITrial].  Given the maximum sample size ${n_{\rm max}}$, the timing of interim analyses is chosen such that for $1\leq k \leq 4$, at the $k$-th interim analysis there are $(k/K){n_{\rm max}}$ participants with $Y$ observed. 

For either estimator the corresponding Wald statistics at different analysis times have (asymptotically) a multivariate normal distribution, where the information at a given analysis time is the reciprocal of the estimator variance. ( This relies on Assumptions 1-3 and holds regardless of the correctness of the outcome regression models used in the adjusted estimator.)  The covariance matrix can be estimated, e.g., using the nonparametric bootstrap.  To reduce the computational burden in our simulation studies, for any data generating distribution and ${n_{\rm max}}$ we precompute the covariance matrix of our statistics at different analysis times, which is then used to determine stopping boundaries.  The efficacy/futility stopping boundaries at each stage are calculated using the error spending approach presented in Supplementary Material \[appen:gst-boundary\].  We used error spending functions $f(t) = 0.025\min(t^2,1)$ for Type I error and $g(t) = 0.2\min(t^2,1)$ for Type II error, where $t$ denotes the information time (observed information divided by maximum information). 

For each combination of prognostic setting (${{\rm progn}_{W,L}}, {{\rm progn}_{W}}, {{\rm progn}_{L}}, {{\rm progn}_{\emptyset}}$) and estimator (unadjusted or adjusted), we use binary search to find the minimum total sample size ${n_{\rm max}}$ required to achieve approximately 80% power under $\Delta=0.122$ at $0.025$ Type I error using the group sequential design.  In practice, this could be approximated from accruing data by estimating the variance of the adjusted estimator and then using sample size re-estimation to set ${n_{\rm max}}$. However, we assumed that this was known in order to reduce the computational burden.  For the unadjusted estimator, in every setting we let ${n_{\rm max}}$ be 480.  For the adjusted estimator, ${n_{\rm max}}$ is set to be 300 under all settings with ${{\rm progn}_{W,L}}$ and ${{\rm progn}_{W}}$, and 480 under all settings with ${{\rm progn}_{L}}$ and ${{\rm progn}_{\emptyset}}$. We simulated 50,000 trials under each of the eight settings.  The number of accrued participants, error spending functions, and the stopping boundaries for each analysis are listed in Table \[tab:error-boundary\] in the Supplementary Material. 

Simulation Results: Sample Size Reduction from Adjustment {#subsec:simulationresults-2}
---------------------------------------------------------

Table \[tab:poweress\] lists the simulation-based ${n_{\rm max}}$, Type I error (obtained under $\Delta=0$), power (obtained under $\Delta=0.122$), and the expected sample size (ESS) for each of the eight settings for each estimator.  For each setting, the empirical Type I error rate is controlled at 0.025 and the power is approximately 80%.  ESS is calculated as the number of enrolled participants when the trial stops, averaging over 50,000 simulations.  The performance of the unadjusted estimator is the same under all prognostic settings and is summarized in the first row. 

Compared to the unadjusted estimator, the adjusted estimator substantially reduces the sample size when the baseline variables are prognostic.  In particular, comparing the first row (unadjusted) versus the third row (adjusted ${{\rm progn}_{W}}$), the adjusted estimator reduces the maximum sample size from 480 to 300, the expected sample size under $\Delta=0$ from 318 to 227, and the expected sample size under $\Delta=0.122$ from 382 to 260.  The sample size reduction due to the prognostic short-term outcome is very small.  When neither $W$ nor $L$ is prognostic (${{\rm progn}_{\emptyset}}$), performance of the two estimators is similar. 

  Estimator    Progn.  set.                      ${n_{\rm max}}$   Type I error   Power   ESS($\Delta=0$)   ESS($\Delta=0.122$)
  ------------ ----------------------------- ----------------- -------------- ------- ----------------- ---------------------
  unadjusted   -                                           480         0.0250   0.811               318                   382
                                                                                                        
  adjusted     ${{\rm progn}_{W,L}}$                       300         0.0254   0.791               225                   259
               ${{\rm progn}_{W}}$                         300         0.0256   0.805               227                   260
               ${{\rm progn}_{L}}$                         480         0.0253   0.805               309                   375
               ${{\rm progn}_{\emptyset}}$                 480         0.0248   0.811               321                   384

  : The maximum sample size (${n_{\rm max}}$) and empirical Type I error, power, and expected sample size (ESS) under $\Delta=0$ and $\Delta=0.122$ for each estimator under each prognostic setting.  The ${n_{\rm max}}$ was chosen for each prognostic setting in order to achieve approximately 80% power under $\Delta=0.122$.[]{data-label="tab:poweress"}

Simulation Results: Relative Efficiency {#subsec:simulationresults}
---------------------------------------

In this subsection, we simulated trials with no early stopping, i.e., each simulated trial always enrolls ${n_{\rm max}}$ participants and continues follow-up until all participants have $Y$ observed.  We then look back at each simulated trial and compute what each estimator’s value would be at each of the interim and decision analysis times.  At any analysis time, the number of participants with $Y$ observed, $L$ observed, and only $(A,W)$ observed, respectively, is fixed; therefore, we can make a direct comparison between the ratio of estimator variances in our simulations and the predicted ${\text{ARE}}$ from our formula (which assumes constant $p_l,p_y$). 

In order to make direct comparisons between estimators at the same sample size, we change (just for this subsection) the setting of ${n_{\rm max}}$. The maximum sample size ${n_{\rm max}}$ is set to be 300 under ${{\rm progn}_{W,L}}$ and ${{\rm progn}_{W}}$, and is set to 480 under ${{\rm progn}_{L}}$ and ${{\rm progn}_{\emptyset}}$. Under each setting the same ${n_{\rm max}}$ is used for both the unadjusted estimator and the adjusted estimator.  Table \[tab:re\] lists the approximate, asymptotic relative efficiency (ARE) computed by evaluating (\[eq:re-ate\]) but with conditional expectations replaced by empirical estimates as discussed in Section \[subsec:are-fitted\], and the relative efficiency (RE) from simulation.  Under most settings, the ARE predicted by the theory is similar to the RE computed from the simulation.  There are some discrepancies between these at the earlier analysis times.  We think that this is due to the relatively small sample sizes at these analysis times compared to the number of variables adjusted for, which can lead to model overfit and so reduced performance of the adjusted estimator.  For example, at interim analysis 1 under ${{\rm progn}_{W,L}}$ there are only 60 participants with the primary outcome observed but 4 variables get adjusted for in the regression model for $E(Y|W,A,L)$; this violates the rule of thumb that one should have at least 20 observations per term in the regression model.  We discuss possible remedies for this in Section \[sec:discussion\]. 

A comparison between ${{\rm progn}_{W}}$ (when only $W$ is prognostic) and ${{\rm progn}_{L}}$ (when only $L$ is prognostic) shows that there is a much larger precision gain in the former case.  This is despite the fact that $W$ and $L$ have roughly similar prognostic values for $Y$ marginally, as shown in Table \[tab:rsq-mistie\].  The reason, as discussed in Sections \[sec:impactbaseline\]-\[impactcomparison\], is that there is relatively little treatment effect heterogeneity, which means that the baseline variables contribute to precision gains in two ways: allowing adjustment for chance imbalances (i) between study arms among the participants with $Y$ observed, and (ii) between the full sample and the subset with $Y$ observed.  The short-term outcome can only contribute in the second way, and so may lead to smaller gains.  Under ${{\rm progn}_{L}}$, comparing interim analyses to decision analyses shows that $L$ improves estimation precision only at interim analyses (i.e., when there are pipeline participants), which is in line with our theoretical results. 

Discussion {#sec:discussion}
==========

We considered independent censoring, and the only source of missing data in our simulations was administrative censoring.  If outcomes are missing at random [@rubin1976MCAR], then the adjusted estimator may still be consistent under correct specification of certain working models, while the unadjusted estimator may be biased.  If dropout is missing not at random [@rubin1976MCAR], then both estimators may be inconsistent and our formulas will not work. 

The theoretical ARE can be larger than the simulation-based ARE, especially at the early interim analyses.  One reason is that in practice there is a finite sample penalty for adjustment (for each degree of freedom in the models fit) that is not reflected in the theoretical calculations, and that dissipates as sample size grows.  This issue may be mitigated by only scheduling interim analyses for times when at least 50% of the participants have primary outcomes observed, and by constraining the number of variables adjusted for such that one has at least 20 participants with primary outcomes observed per variable. 

The sample size reductions due to improved precision from adjustment came primarily from selecting smaller ${n_{\rm max}}$. Since the prognostic value of variables is typically not precisely known before the trial starts, one may use a preplanned rule for sample size re-estimation to set ${n_{\rm max}}$ based on accruing data.  First, the original, maximum sample size is set conservatively assuming no gains from adjustment.  During the trial the asymptotic variance of the adjusted estimator is estimated and ${n_{\rm max}}$ is set to be the sample size for which the desired power is achieved when $\Delta$ equals the a priori specified, clinically meaningful, minimum treatment effect.  The sample size re-estimation should only be conducted after a sufficient number of participants have had their primary outcomes measured, e.g., when roughly 50% of the originally planned sample size have this measured. 

This approach is only feasible if the delay time $d_Y$ is not too long compared to the enrollment rate (otherwise the originally planned sample size will all be enrolled before a sufficient number of primary outcomes are observed).  This was feasible in the MISTIE trial context with $d_Y=180$ days and enrollment rate 140 per year, since sample size re-estimation could be done at 2.1 years.  At this time, 230 (48%) of the originally planned sample size (480) have primary outcomes observed.  So restricting to approximately 300 participants (the required ${n_{\rm max}}$ when $W$ is as prognostic as in the MISTIE-II data) is possible. 

A limitation of our simulation study is that we precomputed ${n_{\rm max}}$ for each setting, in order to save computation time.  In practice, this would be calculated as a function of the estimated variance using accruing data in the trial.  A future area of research is to run simulation studies to evaluate the resulting impact on power and sample size. 

code [@R] for our simulations can be downloaded at <https://github.com/tqian/gst_tmle>. 

Acknowledgments {#acknowledgments .unnumbered}
===============

This work was supported by the Patient-Centered Outcomes Research Institute (ME-1306-03198), the US Food and Drug Administration (HHSF223201400113C), and NIH grant UL1TR001079.  This work is solely the responsibility of the authors and does not represent the views of the above agencies.  We thank Mary Joy Argo for helpful comments. 

Supplementary Material {#supplementary-material .unnumbered}
======================

In Section \[appen:are-sample-size-reduction\], we discuss the relationship between asymptotic relative efficiency and sample size reduction for a Wald test of a single stage trial.  In Section \[appen:gst\], we present the group sequential design used in the simulation studies in Section \[sec:simulation\].  In Section \[appen:dgm\], we specify the data generating distribution used in the simulation studies in Section \[sec:simulation\].  Intuition for the impact of treatment effect heterogeneity on precision gains is given in Section \[appen:adjustmentintuition\].  In Section \[appen:theory-predictive-prognostic\], we provide additional theoretical results regarding the precision gain from adjusting for prognostic baseline variables, when the baseline variable is either purely predictive or purely prognostic.  In Section \[sec:theory-singlearm\], we provide theoretical results regarding the precision gain from adjusting for prognostic baseline variables and the short-term outcome, when the parameter is $E(Y \mid A=a)$ for each $a\in\{0,1\}$ (rather than the average treatment effect).  Simulations to evaluate the theory in Section \[sec:theory-singlearm\] are provided in Section \[appen:re-h1\].  Section \[appen:sec:proof\] includes proofs of the results in the main paper as well as results in Section \[appen:theory-predictive-prognostic\] and Section \[sec:theory-singlearm\].  Auxiliary lemmas that are used in the proofs in Section \[appen:sec:proof\] are themselves proven in Section \[appen:proof-aux\]. 

Relationship between asymptotic relative efficiency and sample size reduction for a Wald test of a single stage trial {#appen:are-sample-size-reduction}
=====================================================================================================================

Consider one-sided Wald tests of the null hypothesis $H_0:\Delta \leq 0$ versus the alternative $H_1:\Delta > 0$, using RAL estimators $\widehat{\Delta}_1$ and $\widehat{\Delta}_2$, respectively (where the test statistics are the estimators divided by their standard errors).  The asymptotic relative efficiency can be interpreted as the limit as sample size goes to infinity of the inverse of the ratio of the required sample sizes for the two estimators to achieve a given power at local alternatives [@vdv Section 8.2].  For example, ${\text{ARE}}(\widehat{\Delta}_1, \widehat{\Delta}_2) = 1.2$ means that by using $\widehat\Delta_1$ instead of $\widehat\Delta_2$, the required sample size is reduced by $1 - 1 / 1.2 \approx 17\%$ asymptotically. 

Full Description of Group Sequential Design Used in Simulation {#appen:gst}
==============================================================

Components of the group sequential design used in simulation are presented in the following order: the multiple testing procedure (Section \[appen:gst-multiple-testing\]), the computation of the test boundaries (Section \[appen:gst-boundary\]), the sample size at each analysis for the simulated trials (Section \[appen:sample-size-each-analysis\]), and the value of Type I error and Type II error spent and the testing boundary at each analysis (Section \[appen:testing-boundary\]). 

Multiple Testing Procedure {#appen:gst-multiple-testing}
--------------------------

In the simulation studies in Section \[sec:simulation\] we use the group sequential test from @hampson2013group with $K$ stages.  Given $\alpha, \beta \in (0,1)$, the design goal is to control the Type I error rate at level $\alpha$ and have power $1-\beta$ at alternative $\Delta=\delta>0$. Such a group sequential test can terminate enrollment at an interim analysis, and if such early stopping happens follow-up continues until all pipeline participants have $Y$ observed before conducting a decision analysis to reject or accept $H_{0}$. For each stage $k$, denote by $S_{k}$ and $\tilde{S}_{k}$ the test statistics at the $k$th interim analysis (where the decision to stop or continue enrollment occurs) and the $k$th decision analysis (where the hypothesis test is conducted), respectively; let $u_{k}$ and $l_{k}$ denote the efficacy and futility boundaries for terminating enrollment at interim analysis $k$, and let $c_{k}$ denote the critical value for the corresponding decision analysis.  These are used in the group sequential testing procedure below, reproduced from @hampson2013group [Figure 1]:

Computation of Test Boundaries {#appen:gst-boundary}
------------------------------

Following @hampson2013group, consider a Type I error spending function $f(\cdot)$ and Type II error spending function $g(\cdot)$, which are non-decreasing with $f\left(0\right)=g\left(0\right)=0$ and $f\left(t\right)=\alpha$ and $g\left(t\right)=\beta$ for $t\geq1$. The maximum information level ${\cal I}_{\max}$ is chosen depending on the power goal and the error spending functions.  Denote by ${\cal I}_{k}$ and $\tilde{{\cal I}}_{k}$ the information levels at the $k$-th interim analysis and decision analysis, respectively.  Denote by ${\cal C}_{k}=\left(l_{k},u_{k}\right)$ the critical region at interim analysis $k$, $1\leq k \leq K$. The test boundaries $u_k$, $l_k$, and $c_k$ are calculated by (12)-(15) in @hampson2013group [Section 4.1.1], and we paraphrase as follows.  Let $u_1$ and $l_1$ be the solutions to $$P(S_1\geq u_1;\Delta=0)=f({\cal I}_1/{\cal I}_{\max})\qquad\mbox{and}\qquad P(S_1\leq l_1;\Delta=\delta)=g({\cal I}_1/{\cal I}_{\max}).  \nonumber$$ For $2\leq k\leq K-1$, $u_k$ is the solution to $$P(S_1\in{\cal C}_1,\ldots,S_{k-1}\in{\cal C}_{k-1},S_k\geq u_k;\Delta=0)=f({\cal I}_k/{\cal I}_{\max})-f({\cal I}_{k-1}/{\cal I}_{\max}),  \label{eq:err_bdry_effic}$$ and $l_k$ is the solution to $$P(S_1\in{\cal C}_1,\ldots,S_{k-1}\in{\cal C}_{k-1},S_k\leq l_k;\Delta=\delta)=g({\cal I}_k/{\cal I}_{\max})-g({\cal I}_{k-1}/{\cal I}_{\max}).  \label{eq:err_bdry_fut}$$ For $1\leq k\leq K-1$, the critical value $c_k$ is the solution to $$\begin{aligned}
P(S_1\in{\cal C}_1,\ldots,S_{k-1}\in{\cal C}_{k-1}, & S_k\geq u_k,\tilde{S}_k<c_k;\Delta=0) \nonumber \\
& = P(S_1\in{\cal C}_1,\ldots,S_{k-1}\in{\cal C}_{k-1},S_k\leq l_k,\tilde{S}_k\geq c_k;\Delta=0).   \nonumber $$ And the critical value $c_K$ for the last stage is the solution to $$P(S_1\in{\cal C}_1,\ldots,S_{K-1}\in{\cal C}_{K-1},\tilde{S}_K\geq c_K;\Delta=0)=\alpha-f({\cal I}_{K-1}/{\cal I}_{\max}).   \nonumber $$

Sample Size at Each Analysis {#appen:sample-size-each-analysis}
----------------------------

Table \[tab:sample-size\] lists the sample size and analysis timing of the group sequential designs with ${n_{\rm max}}=480$ and ${n_{\rm max}}=300$ used in Section \[subsec:simulationresults\].  For $1\leq k \leq 4$, Analysis $k$ indicates interim analysis at stage $k$ and $k^*$ indicates the corresponding decision analysis if enrollment is early stopped at that stage.  Analysis $5^*$ indicates the final decision analysis.  There is not any interim analysis for the final stage.  Fully observed participants are those with $W,L,Y$ observed; partially observed participants are those with $W,L$ but not $Y$ observed.  The three groups of participants listed in Table \[tab:sample-size\] are inclusive of all enrollees and mutually exclusive. 

[llrrrrrrrrr]{} Analysis & & 1 & $1^*$ & 2 & $2^*$ & 3 & $3^*$ & 4 & $4^*$ & $5^*$\
& &\
Time (year) & & 1.2 & 1.7 & 1.9 & 2.4 & 2.6 & 3.0 & 3.2 & 3.7 & 3.9\
\# Fully observed ($W,L,Y$) & & 96 & 165 & 192 & 261 & 288 & 357 & 384 & 453 & 480\
\# Partially observed ($W,L$) only & & 57 & 0 & 57 & 0 & 57 & 0 & 57 & 0 & 0\
\# Pipeline with only $W$ observed & & 12 & 0 & 12 & 0 & 12 & 0 & 12 & 0 & 0\
\
& &\
Time (year) & & 0.9 & 1.4 & 1.4 & 1.8 & 1.8 & 2.3 & 2.2 & 2.6 & 2.6\
\# Fully observed ($W,L,Y$) & & 60 & 129 & 120 & 189 & 180 & 249 & 240 & 300 & 300\
\# Partially observed ($W,L$) only & & 57 & 0 & 57 & 0 & 57 & 0 & 57 & 0 & 0\
\# Pipeline with only $W$ observed & & 12 & 0 & 12 & 0 & 12 & 0 & 3 & 0 & 0\
\

Type I Error and Type II Error Spent and Testing Boundary {#appen:testing-boundary}
---------------------------------------------------------

Table \[tab:error-boundary\] lists the Type I error per stage $f({\cal I}_k/{\cal I}_{\max})-f({\cal I}_{k-1}/{\cal I}_{\max})$, Type II error per stage $g({\cal I}_k/{\cal I}_{\max})-g({\cal I}_{k-1}/{\cal I}_{\max})$, and the testing boundaries at each stage under different settings used in the simulation studies.  Given any prognostic setting ${{\rm progn}_{W,L}}, {{\rm progn}_{W}}, {{\rm progn}_{L}}, {{\rm progn}_{\emptyset}}$ and estimator pair, the designs are the same for $\Delta=0$ and $\Delta=0.122$.

[lrrrrr]{} Analysis $(k)$ & 1 & 2 & 3 & 4 & 5\
&\
Type I error per stage $\times 10^{-3}$ & 0.9 & 3.2 & 5.1 & 7.2 & 8.6\
Type II error per stage $\times 10^{-3}$ & 7.5 & 25.4 & 41.1 & 57.3 & 68.6\
Efficacy boundary at interim analysis ($u_k$) & 3.11 & 2.71 & 2.47 & 2.27 & -\
Futility boundary at interim analysis ($l_k$) & -1.20 & -0.08 & 0.70 & 1.36 & -\
Critical value at decision analysis ($c_k$) & 1.30 & 1.54 & 1.74 & 1.91 & 2.07\
\
&\
Type I error per stage $\times 10^{-3}$ & 0.8 & 2.9 & 5.0 & 7.0 & 9.3\
Type II error per stage $\times 10^{-3}$ & 6.5 & 23.4 & 40.1 & 55.8 & 74.1\
Efficacy boundary at interim analysis ($u_k$) & 3.16 & 2.73 & 2.48 & 2.28 & -\
Futility boundary at interim analysis ($l_k$) & -1.26 & -0.14 & 0.66 & 1.33 & -\
Critical value at decision analysis ($c_k$) & 1.32 & 1.55 & 1.75 & 1.91 & 2.06\
\
&\
Type I error per stage $\times 10^{-3}$ & 1.2 & 3.2 & 5.3 & 7.5 & 7.8\
Type II error per stage $\times 10^{-3}$ & 9.3 & 25.8 & 42.6 & 60.2 & 62.1\
Efficacy boundary at interim analysis ($u_k$) & 3.05 & 2.68 & 2.44 & 2.24 & -\
Futility boundary at interim analysis ($l_k$) & -0.99 & 0.04 & 0.80 & 1.47 & -\
Critical value at decision analysis ($c_k$) & 1.24 & 1.50 & 1.72 & 1.91 & 2.06\
\
&\
Type I error per stage $\times 10^{-3}$ & 0.9 & 2.9 & 5.1 & 6.9 & 9.3\
Type II error per stage $\times 10^{-3}$ & 7.3 & 22.9 & 40.6 & 54.9 & 74.3\
Efficacy boundary at interim analysis ($u_k$) & 3.12 & 2.73 & 2.47 & 2.28 & -\
Futility boundary at interim analysis ($l_k$) & -1.16 & -0.08 & 0.72 & 1.37 & -\
Critical value at decision analysis ($c_k$) & 1.24 & 1.50 & 1.72 & 1.91 & 2.04\
\
&\
Type I error per stage $\times 10^{-3}$ & 1.0 & 3.0 & 4.9 & 6.9 & 9.2\
Type II error per stage $\times 10^{-3}$ & 7.8 & 24.1 & 39.6 & 54.9 & 73.6\
Efficacy boundary at interim analysis ($u_k$) & 3.10 & 2.72 & 2.48 & 2.29 & -\
Futility boundary at interim analysis ($l_k$) & -1.37 & -0.43 & 0.24 & 0.82 & -\
Critical value at decision analysis ($c_k$) & 1.18 & 1.36 & 1.53 & 1.67 & 2.11\
\

Resampling-based Algorithm to Simulate Trial Data {#appen:dgm}
=================================================

For the data generating distribution to have the two properties in Section \[subsec:dgm\], we design the algorithm to generate a simulated trial of $n$ independent and identically distributed samples from the 100 participants in MISTIE-II dataset.  Recall that the three properties are:

1.   The treatment assignment is independent of baseline variables. 

2.   The relative efficiency between the adjusted estimator and the unadjusted estimator in the simulated data is similar to that calculated directly from the MISTIE-II dataset. 

Briefly, the algorithm ensures property (i) by adding a “twin” with identical baseline variables and opposite treatment assignment to each participant in the MISTIE-II dataset.  The variables $L^\text{full}$ and $Y$ for each “twin” are then generated using regression model fits.  The original MISTIE-II data and the set of “twins” results in a 200 participant data set.  This is done once, before any of our simulations are conducted.  To generate each simulated data set, we resample participant vectors from this 200 participant data set with replacement and then make modifications to the replicate data set depending on the desired simulation setting.  Details of the algorithm are given below. 

*Step 1: Construct a set of 100 pairs of “twins”.*  We start with the 100-participant MISTIE-II dataset, and we augment the data with a hypothetical “twin” for each participant.  A “twin” has identical baseline variables as the original participant, but opposite treatment assignment.  We fit logistic regression models for $L^{(1)}$ on $(W^\text{full},A)$, for $L^{(2)}$ on $(W^\text{full},A,L^{(1)})$, and for $Y$ on $(W^\text{full},A,L^{(1)},L^{(2)})$, using the original 100 participants in MISTIE-II dataset.  The preliminary $L_i$ and $Y_i$ of each newly added twin are then predicted based on these logistic regression models by rounding the predicted success probability to be $0$ or $1$. The indicator of whether a participant is an original participant in the MISTIE-II data set or a hypothetical twin is included as a variable in this augmented data set of 200 participants, which will be used in Step 2.  Step 1 is only done once at the beginning of the simulation. 

*Step 2: Sample $n$ participants from the augmented data set with 200 participants and calibrate the treatment effect on the sampled participants.*  We sample $n$ participants uniformly with replacement from the augmented dataset constructed in Step 1.  Then, for each participant in the $n$ participants whose “hypothetical twin indicator” is true, with probability $0.03$ we reset its $Y_i$ to equal $A_i$. This resetting step increases the treatment effect of the augmented data to $0.122$, matching that of the original data. 

*Step 3: Calibrate relative efficiency between the adjusted estimator and the unadjusted estimator.*  For $a\in\{ 0,1\}$, we empirically estimate the marginal distribution $p_{Y;a}=P(Y=1\mid A=a)$ using the corresponding sample proportions in the original data.  Then, for each participant in the $n$ participants whose “hypothetical twin indicator” is true, with probability $0.164$ we reset $Y_i$ by a realization of an independent Bernoulli random draw with success probability $p_{Y;A_i}$. This resetting step adds random noise to reduce the prognostic value in $W$ and $L$, so that the relative efficiency between the adjusted estimator and the unadjusted estimator for the simulated data is comparable to the estimated ${\text{ARE}}$ from the original MISTIE-II dataset. 

We then make modifications that are described next to assess validity of the theory under various settings.  We define eight data generating distributions, called “settings”, each based on modifying the aforementioned data generating process.  The goal is to consider four situations involving $W$ and/or $L$ being prognostic or not for $Y$; these are denoted by ${progn_{W,L}}$ (both prognostic), ${progn_{W}}$ (only $W$ prognostic), ${progn_{L}}$ (only $L$ prognostic), and ${progn_{\emptyset}}$ (neither prognostic).  For each of these four situations, we consider the following two average treatment effects: $\Delta=0$ (no effect) or $\Delta=0.122$ (benefit). 

We first describe how we generated simulated trials with $\Delta=0.122$. The aforementioned data generating algorithm corresponds to ${progn_{W,L}}$. To create data generating distributions with ${progn_{W}}$, we first generate each participant’s data as in ${progn_{W,L}}$ and then replace $L$ by an independent draw from the marginal distribution in the MISTIE-II data.  Similarly, in order to make only $L$ prognostic for $Y$ (${progn_{L}}$), we first generate each participant’s data as in ${progn_{W,L}}$ and then replace $W$ by an independent draw from the marginal distribution in the MISTIE-II data.  Lastly, to make neither $W$ nor $L$ prognostic, we first generate each participant’s data as in ${progn_{W,L}}$ and then replace each of $W$ and $L$ by independent draws from the corresponding marginal distributions in the MISTIE-II data, respectively. 

To generate participants under $\Delta=0$, the data generating process above is followed, and then each study arm assignment $A$ is replaced by an independent Bernoulli draw with probability $1/2$. Under the data generating process with $\Delta=0.122$, there is slight treatment effect heterogeneity $\gamma \approx 0.01$. By construction, under $\Delta=0$ there is no treatment effect heterogeneity ($\gamma = 0$). 

Intuition for Impact of Treatment Effect Heterogeneity on Precision Gains {#appen:adjustmentintuition}
=========================================================================

To give intuition for the phenomena in Figure \[fig:are-w-ate\] in Section \[sec:impactbaseline\], for the remainder of this appendix we fix $R^2_W$ and consider the simple case of a single, binary-valued $W$ representing being less than 65 years old at baseline, and primary outcome $Y$ being the indicator of having mRS at most 3 (a good outcome) at 180 days.  We focus on the stratum $W=1$ and assume that the under 65 subset of the study population would have better outcomes on average than the overall study population if all were assigned to the control arm, i.e., $E(Y|A=0,W=1)-E(Y|A=0)>0$. We next consider $p_y=1$ and the extreme of maximum treatment effect heterogeneity (where there are no efficiency gains from adjusting for $W$). ( The opposite extreme of $p_y=1$ and no treatment effect heterogeneity is discussed in detail in Section \[sec:impactbaseline\].) 

Consider $p_y=1$ and with maximum treatment effect heterogeneity ($\gamma = 2R^2_W$), which occurs when $E(Y|A=1,W)+E(Y|A=0,W)$ equals the constant $E(Y|A=1)+E(Y|A=0)$. It follows from the assumption above that those under 65 have better outcomes on average than the overall study population in the control arm ($E(Y|A=0,W=1)-E(Y|A=0)>0$), that those under 65 have worse outcomes on average than the overall study population under assignment to the treatment arm (and by precisely the same magnitude), which follows since $E(Y|A=1,W=1)-E(Y|A=1)= -\{E(Y|A=0,W=1)-E(Y|A=0)\}$. If by chance there are proportionally more under 65 participants assigned to the treatment arm than the control arm, then the unadjusted estimator of the mean outcome in each arm is biased downward, and by the same amount, conditional on the chance imbalance.  Since this bias cancels out when estimating $\Delta$, an efficient estimator of $\Delta$ makes no adjustment.  Alternatively, if by chance there are proportionally fewer under 65 participants assigned to the treatment arm, then by a symmetric argument an efficient estimator of $\Delta$ makes no adjustment.  The overall impact is no adjustment due to chance imbalances in the stratum $W=1$, which means that no improvement in variance is made compared to the unadjusted estimator.  This is, intuitively, why there is no efficiency gain under maximum treatment effect heterogeneity ($\gamma=2R^2_W$) when there are no pipeline participants ($p_y=1$). 

We next give intuition for why one needs $\gamma>0$ in order for pipeline participants to contribute useful information toward estimating $\Delta$ (when $p_y < 1$).  Consider the case where those in stratum $W=1$ benefit more than the overall population, i.e., $E(Y|A=1,W=1)-E(Y|A=0,W=1)>\Delta$; this is only possible if $\gamma>0$. Assume that adjustment for chance imbalance in the stratum $W=1$ between arms has already been done to the unadjusted estimator (using only those with $Y$ observed), as described above.  Consider the proportion of participants having $W=1$ among all participants and also among the subset of participants with $Y$ observed.  If the former proportion is larger than the latter, then the stratum $W=1$ (of participants who benefit more than average) is underrepresented among those with outcomes observed.  An efficient estimator adjusts the estimate of $\Delta$ upward to compensate.  Alternatively, if the former proportion is smaller than the latter, by a symmetric argument an efficient estimator adjusts the estimate of $\Delta$ downward.  In this way, variance due to imbalance in the proportion with $W=1$ between the overall sample (including pipeline participants) and those with $Y$ observed is removed.  The only cases where adjusting for the aforementioned imbalance at each level of $W$ has no impact are when $E(Y|A=1,W)-E(Y|A=0,W) = \Delta$, i.e., zero treatment effect heterogeneity, or $p_y=1$ (no pipeline participants). 

Additional Results: When Baseline Variable is Purely Predictive or Purely Prognostic. {# appen:theory-predictive-prognostic}
=====================================================================================

The following theorem examines the impact of treatment effect heterogeneity on precision gain from adjusting for prognostic baseline variables under two extreme cases.  Define ${\text{ARE}}({\text{efficient}}, {\text{unadjusted}})$ to be the asymptotic relative efficiency in (\[eq:re-ate\]).  More generally, ${\text{ARE}}(\widehat{\Delta}_1, \widehat{\Delta}_2)$ denotes the asymptotic relative efficiency of the estimators $\widehat{\Delta}_1, \widehat{\Delta}_2$.

\[cor:prognostic-predictive\] Suppose Assumptions \[assump:randomization\], \[assump:independent-censoring\], and \[assump:monotone-censoring\] hold.  In addition, assume $R^2_{L \mid W} = 0$. For the estimand $E(Y\mid A=1) - E(Y\mid A=0)$, we have the following result regarding the asymptotic relative efficiency between an efficient RAL estimator and the unadjusted estimator. 

1.   (Maximal treatment effect heterogeneity.) 

    If ${\text{Var}}\{E_1(Y \mid W) - E_0(Y \mid W)\} > 0$ and ${\text{Var}}\{E(Y \mid W)\} = 0$, then $\gamma = 2 R_W^2$ and $${\text{ARE}}({\text{efficient}}, {\text{unadjusted}}) = \frac{1}{1 - (1-p_y) R^2_W}.  \label{eq:rsq-onlyW-ate-predictive}$$

2.   (No treatment effect heterogeneity.) 

    If ${\text{Var}}\{E_1(Y \mid W) - E_0(Y \mid W)\} = 0$ and ${\text{Var}}\{E(Y \mid W)\} > 0$, then $\gamma = 0$ and $${\text{ARE}}({\text{efficient}}, {\text{unadjusted}}) = \frac{1}{1 - R^2_W}.  \label{eq:rsq-onlyW-ate-prognostic}$$

We call a baseline variable $W$ *purely predictive* if it satisfies the conditions in Corollary \[cor:prognostic-predictive\](i), as it predicts the conditional average treatment effect (i.e., it identifies which strata of $W$ benefit from the treatment on average), but it does not explain the variance in $Y$ marginally (i.e., margining out $A$).  For such a baseline variable, ${\text{ARE}}= 1$ in (\[eq:rsq-onlyW-ate-predictive\]) when $p_y=1$. This means that adjusting for a purely predictive baseline variable results in no precision gain when there are no missing primary outcomes.  An extreme case of this, for illustration only, is the following data generating distribution where the baseline variable is perfectly correlated with the outcome within each arm but still contributes nothing to the precision gain: $W,A$ are independently distributed as $\text{Bernoulli}(0.5)$ and $Y = {{\mathds 1}}(A=W)$, where ${{\mathds 1}}(\cdot)$ is the indicator function.  In this case it is straightforward to verify that $R^2_W = 1$, but adjusting for $W$ would not impact asymptotic variance compared to the unadjusted estimator if all participants have $Y$ observed $(p_y=1)$ since $\gamma=2$. Though we don’t expect to encounter a purely predictive variable in practice, the above thought experiment shows how $W$ could explain treatment effect heterogeneity (which is useful on its own) while being useless for improving precision in estimating the average treatment effect. 

We call a baseline variable $W$ *purely prognostic* if it satisfies the conditions in Corollary \[cor:prognostic-predictive\](ii), as the treatment effect is constant across strata of $W$, and $W$ explains some of the marginal variance in $Y$. For such a baseline variable, the ${\text{ARE}}$ in (\[eq:rsq-onlyW-ate-prognostic\]) does not depend on $p_y$. An intuition for why this holds, as discussed in Section \[sec:impactbaseline\], is that adjusting for such a baseline variable reduces variance by correcting for chance imbalances between arms among those who have $Y$ observed, while the baseline variables for the pipeline participants contain no information about $\Delta$ since $\gamma=0$ (and we are ignoring $L$ here). 

Precision Gain When Estimating the Treatment Specific Mean $E(Y|A=a)$ {#sec:theory-singlearm}
=====================================================================

In this section we focus on estimating the population mean outcome under assignment to a single arm (called the treatment specific mean): $E(Y\mid A=a)$ for each $a\in\{0,1\}$ separately.  The following theorem gives the lower bound on the asymptotic variance for all regular asymptotically linear (RAL) estimators of $E(Y\mid A=a)$.

\[thm:avar\]

Denote by $p_a = P(A=a)$ for $a\in\{0, 1\}$. Assume $p_y > 0$ and $0<p_a<1$. Under Assumptions \[assump:randomization\], \[assump:independent-censoring\], and \[assump:monotone-censoring\] (except allowing $p_a \neq 1/2$) the semiparametric lower bound on the asymptotic variance of all RAL estimators for $E(Y\mid A=a)$ is $${\text{Var}}_a\{E_a(Y \mid W)\} + \frac{1}{p_a p_l} {\text{Var}}_a\{ E_a(Y \mid L,W) - E_a(Y \mid W) \} + \frac{1}{p_a p_y} {\text{Var}}_a\{ Y - E_a(Y \mid L,W) \}.  \label{eq:thm:var-bound}$$

Analogous to the $R$-squared quantities defined in Section \[sec:theory-ate\], we define new $R$-squared quantities relevant to estimating the population mean of the primary outcome $Y$ in each arm separately.  For each $a \in \{0,1\}$, define the proportion of variance in $Y$ explained by $W$ in arm $a$ as $
R^2_{W;a} = {\text{Var}}\{E_a(Y \mid W)\} / {\text{Var}}_a(Y)
$; the proportion of additional variance in $Y$ explained by $L$ after accounting for $W$ in arm $a$ as $
R^2_{L \mid W;a} = {\text{Var}}_a\{E_a(Y \mid L, W) - E_a(Y \mid W)\} / {\text{Var}}_a(Y)
$.  We also will refer to the proportion of variance in $Y$ explained by $L$ alone in arm $a$, defined as $R^2_{L;a} = {\text{Var}}_a\{E_a(Y \mid L)\} / {\text{Var}}_a(Y)$.

The following result gives the ${\text{ARE}}$ between an efficient RAL estimator and the unadjusted estimator for $E(Y\mid A=a)$.

\[cor:re\]

Assume all conditions in Theorem \[thm:avar\] hold.  For each arm $a\in\{0,1\}$, the asymptotic relative efficiency between an efficient RAL estimator and the unadjusted estimator of the treatment specific mean $E(Y\mid A=a)$ is $$\frac{1}{1 - (1 - p_a p_y) R^2_{W;a} - (1 - p_y / p_l) R^2_{L\mid W;a}}.  \label{eq:re}$$

[.5]{} ![[]{ data-label="fig:are-l"}](REW.png "fig:"){width="\linewidth"}

[.5]{} ![[]{ data-label="fig:are-l"}](REL.png "fig:"){width="\linewidth"}

When only $W$ is prognostic ($R^2_{L \mid W;a} = 0$), the asymptotic relative efficiency between an efficient RAL estimator and the unadjusted estimator is equal to when the $L$ variable is not available to the efficient RAL estimator (i.e., when the efficient RAL estimator adjusts for $W$ alone).  Similarly, when only $L$ is prognostic ($R^2_{W;a} = 0$), the asymptotic relative efficiency is equal to when the $W$ variable is not available to the efficient RAL estimator (i.e., when only adjusts for $L$ alone).  In Figure \[fig:are-w\] we illustrate the implication of Theorem \[cor:re\] under those special cases.  We plot ${\text{ARE}}$ against $p_y$ when only $W$ is prognostic ($R^2_{L \mid W;a} = 0$), and in Figure \[fig:are-l\] we plot ${\text{ARE}}$ against $p_y / p_l$ when only $L$ is prognostic ($R^2_{W;a} = 0$).  In each plot, we separately consider prognostic value being $0.1$ and $0.25$. We consider three implications of the curves in Figure \[fig:are\].  First, the precision gain (${\text{ARE}}$) from adjusting for a prognostic baseline variable increases with a greater proportion of pipeline participants (i.e., smaller $p_y$).  The reason is that each additional pipeline participant (who has observed baseline variables but missing outcome) contributes some information to the adjusted estimator (through the baseline variables) but no information to the unadjusted estimator.  For a similar reason, the precision gain from adjusting for a prognostic short-term outcome increases with a greater proportion of participants with $W,L$ but not $Y$ observed (i.e., smaller $p_y/p_l$).  Second, when every participant has their primary outcome observed (e.g., at the final analysis time of a group sequential design with no dropouts), adjusting for the prognostic baseline variables still improves estimation precision as long as $R^2_{W;a} > 0$, but adjusting for prognostic short-term outcome no longer does.  Third, for any given $(p_y,p_l)$, adjusting for a prognostic baseline variable alone always leads to larger precision gain than adjusting for an equally prognostic short-term outcome alone. 

Following the discussion in Section \[appen:are-sample-size-reduction\], we define the asymptotic equivalent reduction in sample size (${\text{AERSS}}$) of $\widehat{\Delta}_1$ compared to $\widehat{\Delta}_2$ as $${\text{AERSS}}(\widehat{\Delta}_1, \widehat{\Delta}_2) = 1 - {\text{ARE}}(\widehat{\Delta}_1, \widehat{\Delta}_2)^{-1}.  \label{def:redss}$$ Adjusting for prognostic $W$ alone with $R^2_{W;a}=q$ (for $0<q<1$) yields ${\text{AERSS}}= q(1 - p_y/2)$, and adjusting for prognostic $L$ alone with $R^2_{L;a} = q$ yields ${\text{AERSS}}= q(1 - p_y/p_l)$. The ratio of the two ${\text{AERSS}}$s equals (assuming $q>0$) $$r = \frac{{\text{AERSS}}\text{ from adjusting only for prognostic } W \text{ with } R^2_{W;a}=q}{{\text{AERSS}}\text{ from adjusting only for prognostic } L \text{ with } R^2_{L;a}=q} = \frac{1 - p_y/2}{1 - p_y/p_l}.  \label{def:ratio-redss}$$ $r>1$ means that the sample size reduction from adjusting for a prognostic $W$ is larger than that from adjusting for an equally prognostic $L$; $r<1$ means the opposite.  Figure \[fig:contour\] plots $r$ against $(p_l, p_y)$ under the constraint $0 < p_y \leq p_l \leq 1$. For all such $(p_l, p_y)$, $r > 1$. In addition, we have $r \geq 2$ if $p_y \geq 2/3$. This means that if at most $1/3$ of the enrollees are in the pipeline, the sample size reduction from adjusting for a prognostic short-term outcome alone is at most half of that from adjusting for an equally prognostic baseline variable alone.  Roughly speaking, small $p_y$ makes $r$ close to 1, meaning that adjusting for $W$ or adjusting for $L$ results in a similar sample size reduction when there are relatively few with $Y$ observed.  This may occur at early stages of a group sequential design, if the delay of the primary outcome is long relative to the enrollment rate. 

Simulation Results for Relative Efficiency When Estimating the Treatment Specific Mean {#appen:re-h1}
======================================================================================

The simulation setup, including the data generating distributions (settings), are the same as Section \[sec:simulation\].  Table \[tab:rsq-mistie-a0a1\] gives the values of $R^2_{W;a}$ and $R^2_{L \mid W;a}$ for $a\in\{0,1\}$, which are defined in the paragraph after Theorem \[thm:avar\].  Tables \[tab:re-H0\] and \[tab:re-H1\] list the asymptotic relative efficiency (ARE) approximated by substituting model-based estimates of the conditional expectations in the quantities in (\[eq:re\]). 

  -------------------- ------------ ---------------- -- ------------ ---------------- -- ------------ ---------------- -- ------------ ----------------
                                                                                                                                       
                       $\Delta=0$   $\Delta=0.122$      $\Delta=0$   $\Delta=0.122$      $\Delta=0$   $\Delta=0.122$      $\Delta=0$   $\Delta=0.122$
  $R^2_{W;0}$          0.35         0.33                0.35         0.33                0            0                   0            0
  $R^2_{L \mid W;0}$   0.08         0.06                0            0                   0.30         0.28                0            0
  $R^2_{W;1}$          0.35         0.38                0.35         0.38                0            0                   0            0
  $R^2_{L \mid W;1}$   0.08         0.08                0            0                   0.30         0.31                0            0
  -------------------- ------------ ---------------- -- ------------ ---------------- -- ------------ ---------------- -- ------------ ----------------

  : $R^2_{W;a}$ and $R^2_{L \mid W;a}$ for $a\in\{0,1\}$ for the data generating distributions from the settings in Section \[sec:simulation\], approximated based on a simulated dataset with 1,000,000 participants under each setting.[]{data-label="tab:rsq-mistie-a0a1"}

Proofs {#appen:sec:proof}
======

We prove the results in the main paper as well as results in Section \[appen:theory-predictive-prognostic\] and Section \[sec:theory-singlearm\].  Auxiliary lemmas (used in these proofs) are proved Section \[appen:proof-aux\]. 

Identification of the average treatment effect
----------------------------------------------

Under Assumptions \[assump:randomization\]-\[assump:monotone-censoring\], the average treatment effect can be expressed as follows: $$\begin{aligned}
E(Y \mid A=1) - E(Y \mid A=0) & =& E [ E \{ E (Y \mid L,W,A=1,C^Y=1) \mid W,C^L=1,A=1\} ] \nonumber \\
&& -  E [ E \{ E (Y \mid L,W,A=0,C^Y=1) \mid W,C^L=1,A=0\} ]  \label{eq:gcomputation}\end{aligned}$$ The proof of (\[eq:gcomputation\]) is given as follows. 

It suffices to show that for each $a\in\{0,1\}$ the following holds: $$E(Y \mid A=a) = E [ E \{ E (Y \mid L,W,A=a,C^Y=1) \mid W,A=a,C^L=1\} ].  \label{eq:gcomputation-proofuse-1}$$ Using the law of iterated expectation twice, we deduce $$E(Y \mid A=a) = E [ E \{ E (Y \mid L,W,A=a) \mid W,A=a\} ].  \label{eq:gcomputation-proofuse-3}$$ Let $f(L,W) = E (Y \mid L,W,A=a)$. By Assumption \[assump:independent-censoring\] we have $$E(f(L,W) \mid A=a) = E(f(L,W) \mid A=a, C^L=1), \label{eq:gcomputation-proofuse-4}$$ and $$E(Y \mid L,W,A=a) = E(Y \mid L,W,A=a, C^L=1, C^Y=1).  \label{eq:gcomputation-proofuse-5}$$ Equations (\[eq:gcomputation-proofuse-3\])-(\[eq:gcomputation-proofuse-5\]) together yield $$E(Y \mid A=a) = E [ E \{ E (Y \mid L,W,A=a,C^L=1,C^Y=1) \mid W,A=a,C^L=1\} ].  \label{eq:gcomputation-proofuse-6}$$ Equation (\[eq:gcomputation-proofuse-6\]) and Assumption \[assump:monotone-censoring\] yield (\[eq:gcomputation-proofuse-1\]).  This completes the proof. 

Lemma on Variance Decomposition {#proof:lem:vardecomp}
-------------------------------

\[lem:vardecomp\]

For study arm $a\in\{0,1\}$, we have the following decomposition of the variance of $Y$ in that arm: $${\text{Var}}_a(Y) = {\text{Var}}_a\{E_a(Y\mid W)\} + {\text{Var}}_a\{ E_a(Y\mid L,W) - E_a(Y \mid W) \} + {\text{Var}}_a\{Y - E_a(Y\mid L,W)\}.  \label{eq:vardecomp}$$ In addition, we have $$R^2_{W;a} + R^2_{L\mid W;a} + R^2_{r;a} = 1, \quad \mbox{ for each } a \in \{0,1\}.  \label{eq:rsqsum1}$$

Adding and subtracting terms, we have $${\text{Var}}_a(Y) = {\text{Var}}_a \big[ \{Y - E_a(Y\mid L,W)\} + \{E_a(Y\mid L,W) - E_a(Y \mid W)\} + E_a(Y\mid W) \big].$$ So for proving (\[eq:vardecomp\]), it suffices to establish the following: $$\begin{aligned}
{\text{Cov}}_a \big\{ Y - E_a(Y\mid L,W), E_a(Y\mid L,W) - E_a(Y \mid W) \big\} &= 0, \label{eq:vardecomp-proofuse-1} \\
{\text{Cov}}_a \big\{ Y - E_a(Y\mid L,W), E_a(Y\mid W) \big\} &= 0, \label{eq:vardecomp-proofuse-2} \\
{\text{Cov}}_a \big\{ E_a(Y\mid L,W) - E_a(Y \mid W), E_a(Y\mid W) \big\} &= 0, \label{eq:vardecomp-proofuse-3}\end{aligned}$$ where ${\text{Cov}}_a$ denotes the conditional covariance given $A=a$.

First, we have $$\begin{aligned}
& {\text{Cov}}_a \big\{ Y - E_a(Y\mid L,W), E_a(Y\mid L,W) - E_a(Y \mid W) \big\} \nonumber\\
= &  E_a \big\{ Y E_a(Y\mid L,W) - E_a(Y\mid L,W)^2
  - Y E_a(Y \mid W) + E_a(Y\mid L,W) E_a(Y \mid W) \big\}.  \label{eq:vardecomp-proofuse-4}\end{aligned}$$ By Lemma \[lem:cond-exp\] with $X = Z = (L,W)$, we have $$E_a \big\{ Y E_a(Y\mid L,W) \big\} = E_a \big\{ E_a(Y \mid L,W)^2 \big\}.  \label{eq:vardecomp-proofuse-5}$$ By Lemma \[lem:cond-exp\] with $X = W$ and $Z = (L,W)$, we have $$E_a \big\{ Y E_a(Y \mid W) \big\} = E_a \big\{ E_a(Y \mid L,W) E_a(Y\mid W) \big\}.  \label{eq:vardecomp-proofuse-6}$$ Equations (\[eq:vardecomp-proofuse-4\]), (\[eq:vardecomp-proofuse-5\]), and (\[eq:vardecomp-proofuse-6\]) imply (\[eq:vardecomp-proofuse-1\]). 

Second, we have $${\text{Cov}}_a \big\{ Y - E_a(Y\mid L,W), E_a(Y\mid W) \big\} = E_a \big\{ Y E_a(Y\mid W) - E_a(Y\mid L,W) E_a(Y\mid W) \big\}.  \label{eq:vardecomp-proofuse-7}$$ Equations (\[eq:vardecomp-proofuse-7\]) and (\[eq:vardecomp-proofuse-6\]) imply (\[eq:vardecomp-proofuse-2\]). 

Third, since $E_a(Y\mid L,W) - E_a(Y \mid W)$ has expectation zero, we have $${\text{Cov}}_a \big\{ E_a(Y\mid L,W) - E_a(Y \mid W), E_a(Y\mid W) \big\}
= E_a \big\{ E_a(Y\mid L,W) E_a(Y\mid W) - E_a(Y \mid W)^2 \big\}.  \label{eq:vardecomp-proofuse-8}$$ In Lemma \[lem:cond-exp\], letting $X=Z=W$ and replacing $Y$ in the lemma by $E(Y \mid L,W)$ implies $$E_a \big\{ E_a(Y \mid L,W) E_a(Y\mid W) \big\} = E_a \big\{ E_a(Y \mid W)^2 \big\}.  \label{eq:vardecomp-proofuse-9}$$ Equations (\[eq:vardecomp-proofuse-8\]) and (\[eq:vardecomp-proofuse-9\]) imply (\[eq:vardecomp-proofuse-3\]). 

This proves (\[eq:vardecomp\]).  Equation (\[eq:rsqsum1\]) follows immediately from (\[eq:vardecomp\]) and the definition of $R^2_{W;a}$, $R^2_{L \mid W;a}$ and $R^2_{r;a}$. This completes the proof for Lemma \[lem:vardecomp\]. 

Proof of Theorem \[thm:avar\] {#proof:thm:avar}
-----------------------------

In the proof, we will use equation (\[eq:gcomputation-proofuse-6\]) derived earlier; we rewrite it below: $$E(Y \mid A=a) = E [ E \{ E (Y \mid L,W,A=a,C^L=1,C^Y=1) \mid W,A=a,C^L=1\} ].  \label{eq:thm:avar-proofuse-1}$$ Treating the missingness indicators $C^L$ and $C^Y$ as binary treatments, the right-hand side of (\[eq:thm:avar-proofuse-1\]) becomes the average of outcome $Y$ under time dependent treatment assignment: $A=a, C^L=1, C^Y=1$. Because there is no measurement made between $A$ and $C^L$, we can combine the two as a single treatment $\tilde{A}$, with $\tilde{A}=1$ if and only if $A=a$ and $C^L=1$. Equation (\[eq:thm:avar-proofuse-1\]) becomes $$E(Y \mid A=a) = E [ E \{ E (Y \mid L,W,\tilde{A}=1,C^Y=1) \mid W,\tilde{A}=1\} ].  \label{eq:thm:avar-proofuse-1.5}$$ Using the fact that $L$ is binary-valued, by equations (24) and (28) in @rosenblum2011EIF or Theorem 1 in @van2010EIF, the efficient influence function for (\[eq:thm:avar-proofuse-1.5\]) is $$D(W,\tilde{A},L,C^Y,Y) = D_0(W) + D_1(W,\tilde{A},L) + D_2(W,\tilde{A},L,C^Y,Y), \label{eq:thm:avar-proofuse-d}$$ where $$D_0(W) = E(Y \mid W,\tilde{A}=1,C^Y=1) - E(Y \mid \tilde{A}=1), \label{eq:thm:avar-proofuse-d0}$$ $$D_1(W,\tilde{A},L) = \frac{
{{\mathds 1}}(\tilde{A}=1) \big\{ E(Y \mid L,W, \tilde{A}=1, C^Y=1) - E(Y \mid W,\tilde{A}=1,C^Y=1) \big\}
}{P(\tilde{A}=1\mid W)}, \label{eq:thm:avar-proofuse-d1}$$ and $$D_2(W,\tilde{A},L,C^Y,Y) = \frac{
{{\mathds 1}}(C^Y=1) {{\mathds 1}}(\tilde{A}=1) \big\{ Y - E(Y \mid L,W, \tilde{A}=1, C^Y=1) \big\}
}{P(C^Y=1 \mid L,W,\tilde{A}=1)P(\tilde{A}=1\mid W)}.  \label{eq:thm:avar-proofuse-d2}$$

By randomization and independent censoring assumptions, (\[eq:thm:avar-proofuse-d0\]) simplifies to $$D_0(W) = E(Y \mid W, A=a) - E(Y \mid A=a); \label{eq:thm:avar-proofuse-d0-simp}
$$ equation (\[eq:thm:avar-proofuse-d1\]) simplifies to $$D_1(W,\tilde{A},L) = \frac{
{{\mathds 1}}(\tilde{A}=1) \big\{ E(Y \mid L,W, A=a) - E(Y \mid W,A=a) \big\}
}{P(\tilde{A}=1)}; \label{eq:thm:avar-proofuse-d1-simp}
$$ equation (\[eq:thm:avar-proofuse-d2\]) simplifies to $$D_2(W,\tilde{A},L,C^Y,Y) = \frac{
{{\mathds 1}}(C^Y=1) {{\mathds 1}}(\tilde{A}=1) \big\{ Y - E(Y \mid L,W,A=a) \big\}
}{P(C^Y=1 \mid C^L=1)P(\tilde{A}=1)}.  \label{eq:thm:avar-proofuse-d2-simp}
$$

The following lemma states that $D_0$, $D_1$, and $D_2$ are pairwise uncorrelated. 

\[lem:thm-proofuse\]

We have $$\begin{aligned}
{\text{Cov}}\{ D_0(W), D_1(W,\tilde{A},L)\} & = 0, \label{eq:lem:thm-proofuse-toshow-d0d1}\\
{\text{Cov}}\{ D_0(W), D_2(W,\tilde{A},L,C^Y,Y)\} & = 0, \label{eq:lem:thm-proofuse-toshow-d0d2}\\
{\text{Cov}}\{ D_1(W,\tilde{A},L), D_2(W,\tilde{A},L,C^Y,Y)\} & = 0.  \label{eq:lem:thm-proofuse-toshow-d1d2}\end{aligned}$$

Lemma \[lem:thm-proofuse\] implies $${\text{Var}}\{D(W,\tilde{A},L,C^Y,Y)\} = {\text{Var}}\{D_0(W)\} + {\text{Var}}\{D_1(W,\tilde{A},L)\} + {\text{Var}}\{D_2(W,\tilde{A},L,C^Y,Y)\}.  \label{eq:thm:avar-proofuse-2}$$ By (\[eq:thm:avar-proofuse-d0-simp\]) we have $${\text{Var}}\{D_0(W)\} = {\text{Var}}\{E_a(Y \mid W)\} = {\text{Var}}_a\{E_a(Y \mid W)\}, \label{eq:thm:avar-proofuse-vard0}$$ where the last equality follows from randomization assumption.  By (\[eq:thm:avar-proofuse-d1-simp\]) we have $E\{D_1(W,\tilde{A},L)\}=0$, so it follows from randomization and independent censoring that $$\begin{aligned}
{\text{Var}}\{D_1(W,\tilde{A},L)\} & = E \Big[ \frac{ {{\mathds 1}}(\tilde{A}=1)^2 \big\{ E(Y \mid L,W, A=a) - E(Y \mid W,A=a) \big\}^2 }{P(\tilde{A}=1)^2} \Big] \nonumber \\
& = \frac{E\{{{\mathds 1}}(\tilde{A}=1)\}}{P(\tilde{A}=1)^2} E\big[ \big\{ E(Y \mid L,W, A=a) - E(Y \mid W,A=a) \big\}^2 \mid A=a \big].  \label{eq:thm:avar-proofuse-vard1-1}\end{aligned}$$ By independent censoring we have $P(\tilde{A}=1) = p_a p_l$. It then follows from (\[eq:thm:avar-proofuse-vard1-1\]) and randomization assumption that $${\text{Var}}\{D_1(W,\tilde{A},L)\} = \frac{1}{p_a p_l} {\text{Var}}_a\big\{ E_a(Y \mid L,W) - E_a(Y \mid W) \big\}.  \label{eq:thm:avar-proofuse-vard1}$$ Similarly, (\[eq:thm:avar-proofuse-d2-simp\]) together with randomization and monotone censoring imply $${\text{Var}}\{D_2(W,\tilde{A},L,C^Y,Y)\} = \frac{1}{p_a p_y} {\text{Var}}_a\big\{ Y - E_a(Y \mid L,W) \big\}.  \label{eq:thm:avar-proofuse-vard2}$$

Because the semiparametric lower bound on the asymptotic variance for an estimand equals the variance of the efficient influence function, by (\[eq:thm:avar-proofuse-2\]), (\[eq:thm:avar-proofuse-vard0\]), (\[eq:thm:avar-proofuse-vard1\]), and (\[eq:thm:avar-proofuse-vard2\]) we proved (\[eq:thm:var-bound\]). 

Proof of Theorem \[cor:re\] {#proof:cor:re}
---------------------------

The unadjusted estimator $\widehat{\tau}$ for $E(Y \mid A=a)$ is defined as $$\widehat{\tau} = \frac{\sum_{i=1}^n Y_i {{\mathds 1}}(A_i = a, C^Y_i = 1)}{\sum_{i=1}^n {{\mathds 1}}(A_i = a, C^Y_i = 1)}.  \label{def:unadj-noncausal}$$ Under Assumptions \[assump:randomization\] and \[assump:independent-censoring\], $\widehat{\tau}$ is unbiased: $$\begin{aligned}
E(\widehat{\tau}) = E \Big[ E\Big\{ \frac{\sum_{i=1}^n Y_i {{\mathds 1}}(A_i = a, C^Y_i = 1)}{\sum_{i=1}^n {{\mathds 1}}(A_i = a, C^Y_i = 1)} \Big| A_1,\ldots,A_n,C_{Y_1},\ldots,C_{Y_n}  \Big\} \Big] = E(Y \mid A = a).\end{aligned}$$

In the following we calculate the asymptotic variance of $\widehat{\tau}$. $$\begin{aligned}
\sqrt{n}\{\widehat{\tau} - E(Y \mid A=a)\} & = \frac{ \frac{1}{\sqrt{n}}\sum_{i=1}^n Y_i {{\mathds 1}}(A_i = a, C^Y_i = 1)}{ \frac{1}{n}\sum_{i=1}^n {{\mathds 1}}(A_i = a, C^Y_i = 1)} - \sqrt{n}E(Y \mid A=a) \nonumber \\
&= \frac{ \frac{1}{\sqrt{n}}\sum_{i=1}^n \{Y_i - E(Y \mid A=a)\} {{\mathds 1}}(A_i = a, C^Y_i = 1)}{ \frac{1}{n}\sum_{i=1}^n {{\mathds 1}}(A_i = a, C^Y_i = 1)}.  \label{eq:unadj-asymp-1}\end{aligned}$$ By Weak Law of Large Numbers and the independent censoring assumption, $$\frac{1}{n}\sum_{i=1}^n {{\mathds 1}}(A_i = a, C^Y_i = 1) \stackrel{P}{\to} p_a p_y, \label{eq:unadj-asymp-2}$$ where $\stackrel{P}{\to}$ denotes convergence in probability.  By Central Limit Theorem, $$\frac{1}{\sqrt{n}}\sum_{i=1}^n \{Y_i - E(Y \mid A=a)\} {{\mathds 1}}(A_i = a, C^Y_i = 1) \stackrel{d}{\to} N(0, \sigma^2), \label{eq:unadj-asymp-3}$$ where by randomization and independent censoring we have $$\begin{aligned}
\sigma^2 & = {\text{Var}}\big[ \{Y - E(Y \mid A=a)\} {{\mathds 1}}(A = a, C^Y = 1) \big] = E \big[ \{Y - E(Y \mid A=a)\}^2 {{\mathds 1}}(A = a, C^Y = 1)^2 \big] \nonumber \\
& = p_a p_y {\text{Var}}(Y \mid A=a).  \label{eq:unadj-asymp-4}\end{aligned}$$ Combining (\[eq:unadj-asymp-1\])-(\[eq:unadj-asymp-4\]), it follows from Slutsky’s theorem that $$\sqrt{n}\{\widehat{\tau} - E(Y \mid A=a)\} \stackrel{d}{\to} N\big(0, (p_a p_y)^{-1} {\text{Var}}(Y \mid A=a)\big).  \nonumber$$ So the asymptotic variance of $\widehat{\tau}$ is $(p_a p_y)^{-1} {\text{Var}}(Y \mid A=a)$, which by randomization yields (recall that by definition ${\text{Var}}_a(Y) = {\text{Var}}(Y \mid A=a)$) $${\text{AVar}}({\text{unadjusted}}) = \frac{1}{p_a p_y} {\text{Var}}_a(Y).  \label{eq:cor-proofuse-1}$$ Equations (\[eq:cor-proofuse-1\]) and (\[eq:rsqsum1\]) imply $${\text{AVar}}({\text{unadjusted}}) = \frac{1}{p_a p_y} {\text{Var}}_a(Y) (R^2_{W;a} + R^2_{L\mid W;a} + R^2_{r;a}).  \label{eq:cor-proofuse-2}$$

On the other hand, Theorem \[thm:avar\] and the definition of $R^2_W$ and $R^2_r$ imply $${\text{AVar}}({\text{efficient}}) = {\text{Var}}_a(Y) ( R^2_{W;a} + \frac{1}{p_a p_l} R^2_{L\mid W;a} + \frac{1}{p_a p_y} R^2_{r;a} ).  \label{eq:cor-proofuse-3}$$ Equations (\[eq:cor-proofuse-2\]), (\[eq:cor-proofuse-3\]), and (\[eq:rsqsum1\]) yield (\[eq:re\]).  The proof is thus finished. 

Generalization and proof of Lemma \[thm:avar-ate\] {#proof:thm:avar-ate}
--------------------------------------------------

We provide proof for a generalization of Lemma \[thm:avar-ate\], which allows for constant randomization probability other than $1/2$.

\[thm:avar-ate-2\] Suppose Assumptions \[assump:randomization\], \[assump:independent-censoring\], and \[assump:monotone-censoring\] hold.  Define $P(A=a) = p_a$ for $a \in \{0,1\}$. The lower bound on the asymptotic variance of all RAL estimators of $E(Y\mid A=1) - E(Y\mid A=0)$ is $$\begin{aligned}
   & {\text{Var}}\{E_1(Y \mid W) - E_0(Y \mid W) \}
+  \sum_{a\in\{0,1\}} \frac{1}{p_a p_l} {\text{Var}}_a\{ E_a(Y \mid L,W) - E_a(Y \mid W) \} \nonumber \\
+ &  \sum_{a\in\{0,1\}} \frac{1}{p_a p_y} {\text{Var}}_a\{ Y - E_a(Y \mid L,W) \}.  \label{eq:thm:var-bound-ate-2}\end{aligned}$$

For notation simplicity, denote by $Q(\cdot)$ the conditional expectation $E(Y\mid \cdot)$. Using the derivation in (\[eq:thm:avar-proofuse-1\])-(\[eq:thm:avar-proofuse-d2-simp\]) twice for $A=1$ and $A=0$, we get the efficient influence function $D(W,A,C^L,L,C^Y,Y)$ for $E(Y \mid A=1) - E(Y \mid A=0)$: $$D(W,A,C^L,L,C^Y,Y) = D_0(W) + D_1(W,A,C^L,L) + D_2(W,A,C^L,L,C^Y,Y),$$ where $$D_0(W) = \big\{ Q(W,A=1) - Q(A=1) \big\} - \big\{ Q(W,A=0) - Q(A=0) \big\}, \label{eq:thm:avar-ate-proofuse-d0}$$ $$\begin{aligned}
D_1(W,A,C^L,L) = & \frac{A C^L}{p_1 p_l} \big\{ Q(W,L,A=1) - Q(W,A=1) \big\} \nonumber \\
& - \frac{(1-A)C^L}{p_0 p_l} \big\{ Q(W,L,A=0) - Q(W,A=0) \big\} , \label{eq:thm:avar-ate-proofuse-d1}\end{aligned}$$ and $$D_2(W,A,C^L,L,C^Y,Y) = \frac{A C^Y}{p_1 p_y} \big\{ Y - Q(W,L,A=1) \big\} - \frac{(1-A) C^Y}{p_0 p_y} \big\{ Y - Q(W,L,A=0) \big\}.  \label{eq:thm:avar-ate-proofuse-d2}$$

The following lemma states that $D_0$, $D_1$, and $D_2$ are pairwise uncorrelated. 

\[lem:thm-ate-proofuse\]

We have $$\begin{aligned}
{\text{Cov}}\{ D_0(W), D_1(W,A,C^L,L)\} & = 0, \label{eq:lem:thm-ate-proofuse-toshow-d0d1}\\
{\text{Cov}}\{ D_0(W), D_2(W,A,C^L,L,C^Y,Y)\} & = 0, \label{eq:lem:thm-ate-proofuse-toshow-d0d2}\\
{\text{Cov}}\{ D_1(W,A,C^L,L), D_2(W,A,C^L,L,C^Y,Y)\} & = 0.  \label{eq:lem:thm-ate-proofuse-toshow-d1d2}\end{aligned}$$

Lemma \[lem:thm-ate-proofuse\] implies $$\begin{aligned}
{\text{Var}}\{D(W,A,C^L,L,C^Y,Y)\} = & {\text{Var}}\{D_0(W)\} + {\text{Var}}\{D_1(W,A,C^L,L)\} \nonumber \\
& + {\text{Var}}\{D_2(W,A,C^L,L,C^Y,Y)\}.  \label{eq:thm:avar-ate-proofuse-2}\end{aligned}$$ By \[eq:thm:avar-ate-proofuse-d0\] we have $${\text{Var}}\{D_0(W)\} = {\text{Var}}\{Q(W,A=1) - Q(W,A=0)\}.  \label{eq:thm:avar-ate-proofuse-vard0}$$ By (\[eq:thm:avar-ate-proofuse-d1\]) we have $$\begin{aligned}
{\text{Var}}\{D_1(W,A,C^L,L)\} = & E \Big[ \frac{A C^L}{p_1^2 p_l^2} \big\{ Q(W,L,A=1) - Q(W,A=1) \big\}^2 \Big] \nonumber \\
& + E \Big[ \frac{(1-A) C^L}{p_0^2 p_l^2} \big\{ Q(W,L,A=0) - Q(W,A=0) \big\}^2 \Big] \label{eq:thm:avar-ate-proofuse-vard1-1} \\
& = \sum_{a\in\{0,1\}} \frac{1}{p_a p_l} E \Big[ \big\{ Q(W,L,A=a) - Q(W,A=a) \big\}^2 \mid A=a \Big] \label{eq:thm:avar-ate-proofuse-vard1-2} \\
& = \sum_{a\in\{0,1\}} \frac{1}{p_a p_l} {\text{Var}}\big\{ Q(W,L,A=a) - Q(W,A=a) \mid A=a \big\}.  \label{eq:thm:avar-ate-proofuse-vard1}\end{aligned}$$ The step from (\[eq:thm:avar-ate-proofuse-vard1-1\]) to (\[eq:thm:avar-ate-proofuse-vard1-2\]) utilizes the independent censoring and randomization assumptions.  Similarly, (\[eq:thm:avar-ate-proofuse-d2\]) together with randomization and independent censoring imply $${\text{Var}}\{D_2(W,A,C^L,L,C^Y,Y)\} = \sum_{a\in\{0,1\}} \frac{1}{p_a p_l} {\text{Var}}\big\{ Y - Q(W,L,A=a) \mid A=a \big\}.  \label{eq:thm:avar-ate-proofuse-vard2}$$

Because the semiparametric lower bound on the asymptotic variance for an estimand equals the variance of the efficient influence function, by (\[eq:thm:avar-ate-proofuse-2\]), (\[eq:thm:avar-ate-proofuse-vard0\]), (\[eq:thm:avar-ate-proofuse-vard1\]), and (\[eq:thm:avar-ate-proofuse-vard2\]) we proved Lemma \[thm:avar-ate\]. 

Proof of Theorem \[cor:re-ate\] {#proof:cor:re-ate}
-------------------------------

The unadjusted estimator for the average treatment effect is $$\widehat{\tau} = \frac{\sum_{i=1}^n Y_i {{\mathds 1}}(A_i = 1, C^Y_i = 1)}{\sum_{i=1}^n {{\mathds 1}}(A_i = 1, C^Y_i = 1)} - \frac{\sum_{i=1}^n Y_i {{\mathds 1}}(A_i = 0, C^Y_i = 1)}{\sum_{i=1}^n {{\mathds 1}}(A_i = 0, C^Y_i = 1)}.  \nonumber$$ Similar to the derivation from (\[def:unadj-noncausal\]) to (\[eq:cor-proofuse-1\]), when estimating the average treatment effect with $P(A=1) = P(A=0) = 1/2$, we have $${\text{AVar}}({\text{unadjusted}}) = \frac{2}{p_y} \sum_{a \in \{0,1\}} {\text{Var}}_a(Y).$$ The result in Theorem \[cor:re-ate\] then follows immediately from Lemma \[thm:avar-ate\]. 

Proof of Corollary \[cor:prognostic-predictive\] {#proof:cor:prognostic-predictive}
------------------------------------------------

1.   Because $$\begin{aligned}
    {\text{Var}}\{E(Y \mid W)\} &= {\text{Var}}\{0.5 E(Y \mid W, A=1) + 0.5 E(Y \mid W, A=1)\}, \nonumber\end{aligned}$$ ${\text{Var}}\{E(Y \mid W)\} = 0$ implies $$\begin{aligned}
    {\text{Var}}\{E(Y \mid W, A=1)\} + {\text{Var}}\{E(Y \mid W, A=0)\} = -2{\text{Cov}}\{E(Y \mid W, A=1),E(Y \mid W, A=0).  \label{eq:proofuse:cor:prognostic-predictive-1}\end{aligned}$$ By (\[eq:proofuse:cor:prognostic-predictive-1\]) and the definition of $\gamma$ and $R^2_W$, we have $\gamma = 2 R^2_W$. This combined with (\[eq:re-ate\]) with $R^2_{L \mid W} = 0$ imply (\[eq:rsq-onlyW-ate-predictive\]). 

2.   By the definition of $\gamma$ and ${\text{Var}}\{E(Y \mid W, A=1) - E(Y \mid W, A=0)\} = 0$, (\[eq:rsq-onlyW-ate-prognostic\]) follows immediately from (\[eq:re-ate\]) with $R^2_{L \mid W} = 0$.

Proof of statements in Section \[impactcomparison\] {#appen:proof-section3.4}
---------------------------------------------------

For ease of reading we restate the claims to be proven in Section \[impactcomparison\]: We compare the ${\text{ARE}}$ between two cases: $R^2_W = q >0, R^2_{L \mid W} = 0$ (only baseline variable prognostic) and $R^2_W = 0, R^2_{L \mid W} = q >0$ (only short-term outcome prognostic).  Regardless of the value of $q>0$, the ${\text{ARE}}$ in the former case is larger or equal to that in the latter case.  Equality occurs if and only if $p_l=1$ and $W$ is uncorrelated with $Y$ (marginally) in the former case.  The latter condition is equivalent to the treatment effect heterogeneity being the maximum possible $\gamma=2R^2_W=2q$.

We first show that $\gamma \leq 2R^2_W$ with equality holds only when $E_1(Y\mid W) = - E_0(Y \mid W)$ almost surely.  By Cauchy-Schwarz inequality we have $$\begin{aligned}
    & {\text{Var}}\{ E_1(Y \mid W) - E_0(Y \mid W) \} \\
    & = {\text{Var}}\{ E_1(Y \mid W)\} + {\text{Var}}\{ E_0(Y \mid W) \} - 2 {\text{Cov}}\{E_1(Y \mid W), E_0(Y \mid W) \} \\
    & \leq {\text{Var}}\{ E_1(Y \mid W)\} + {\text{Var}}\{ E_0(Y \mid W) \} + 2 \left[{\text{Var}}\{ E_1(Y \mid W)\}{\text{Var}}\{ E_0(Y \mid W) \}\right]^{1/2} \\
    & \leq 2 {\text{Var}}\{ E_1(Y \mid W)\} + 2 {\text{Var}}\{ E_0(Y \mid W) \},\end{aligned}$$ where both inequalities becomes equality if and only if $E_1(Y \mid W) = - E_0(Y \mid W)$ almost surely.  This proves the claim. 

Theorem \[cor:re-ate\] states that the ${\text{ARE}}$ equals (where we indicate dependence on the arguments by explicitly writing them out) $${\text{ARE}}(R^2_W, R^2_{L\mid W}, \gamma, p_y, p_l) = \frac{1}{ 1 + (p_y/2) \gamma - R^2_W -  (1 - p_y / p_l) R^2_{L\mid W} }.  \nonumber$$ When $R^2_W = q >0, R^2_{L \mid W} = 0$, $$\begin{aligned}
    {\text{ARE}}(q, 0, \gamma, p_y, p_l) = \frac{1}{ 1 + (p_y/2) \gamma - q } \geq \frac{1}{1 - (1 - p_y)q},\end{aligned}$$ with equality holds if and only if $\gamma = 2q$. When $R^2_W = 0, R^2_{L \mid W} = q >0$ (which implies $\gamma = 0$ because $\gamma \leq 2R^2_W$), $${\text{ARE}}(0, R^2_{L\mid W}, 0, p_y, p_l) = \frac{1}{ 1 -  (1 - p_y / p_l) q }.  \nonumber$$ Because $p_l \leq 1$, ${\text{ARE}}(q, 0, \gamma, p_y, p_l) \geq {\text{ARE}}(0, R^2_{L\mid W}, 0, p_y, p_l)$ with equality holds if and only if $p_l = 1$ and $\gamma = 2q$, i.e., $E_1(Y\mid W) = - E_0(Y \mid W)$ almost surely. 

Lastly, we show that $E_1(Y\mid W) = - E_0(Y \mid W)$ almost surely implies that $W$ is uncorrelated with $Y$ (marginally): $$\begin{aligned}
    & {\text{Cov}}(Y, W) = E\{Y (W - E(W))\} = E\{E(Y \mid W) (W - E(W))\} \\
    & = E\{E(Y \mid W) (W - E(W))\} = E\left[\{0.5 E_1(Y\mid W) + 0.5 E_0(Y\mid W)\} (W - E(W))\right] = 0.\end{aligned}$$ This completes the proof. 

Proof of Auxiliary Lemmas {#appen:proof-aux}
=========================

Additional Supporting Lemmas
----------------------------

\[lem:cond-exp\]

Consider three random variables $X$, $Y$, and $Z$. Denote by $\sigma(Z)$ the $\sigma$-field generated by $Z$. If $X \in \sigma(Z)$, then $$E\big\{Y E(Y \mid X)\big\} = E\big\{ E(Y\mid Z) E(Y\mid X) \big\}.  \label{eq:cond-exp}$$

By the law of iterated expectation, we have $$E\big\{Y E(Y \mid X)\big\} = E\big[ E \big\{Y E(Y \mid X) \mid Z\big\} \big].  \label{eq:cond-exp-proofuse-1}$$ Because $X \in \sigma(Z)$, we have $E(Y \mid X) \in \sigma(X) \subset \sigma(Z)$. This implies $$E \big\{Y E(Y \mid X) \mid Z\big\} = E(Y \mid X) E (Y \mid Z).  \label{eq:cond-exp-proofuse-2}$$ Equations (\[eq:cond-exp-proofuse-1\]) and (\[eq:cond-exp-proofuse-2\]) imply (\[eq:cond-exp\]).  This completes the proof. 

\[lem:cond-exp-ate\]

Consider three random variables $W$, $L$, and $Y$. For any measurable functions $f(W)$ and $g(W,L)$, we have $$\begin{aligned}
E\big[ f(W) \big\{ E(Y \mid W,L) - E(Y \mid W) \big\} \big] & = 0, \label{eq:cond-exp-ate-1} \\
E\big[ g(W,L) \big\{ Y - E(Y \mid W,L) \big\} \big] & = 0.  \label{eq:cond-exp-ate-2}\end{aligned}$$

By the law of iterated expectation, we have $$\begin{aligned}
E \big\{ g(W) E(Y \mid W,L) \big\} &= E \big[ E \big\{ g(W) E(Y \mid W,L) \mid W \big\} \big ] \nonumber \\
&= E \big[ g(W) E \big\{ E(Y \mid W,L) \mid W \big\} \big ] \nonumber \\
&= E \big\{ g(W) E(Y \mid W) \big\}, \nonumber\end{aligned}$$ which proves (\[eq:cond-exp-ate-1\]). 

Similarly, we have $$E \big\{ g(W,L) Y \big\} = E \big[ E \big\{ g(W,L) Y \mid W,L \big\} \big] = E \big\{ g(W,L) E(Y \mid W,L) \big\}, \nonumber$$ which proves (\[eq:cond-exp-ate-2\]). 

Proof of Lemma \[lem:thm-proofuse\] {#proof:lem:thm-proofuse}
-----------------------------------

By (\[eq:thm:avar-proofuse-d0-simp\]) and (\[eq:thm:avar-proofuse-d1-simp\]) we have $${\text{Cov}}(D_0, D_1) \propto E \big[ {{\mathds 1}}(\tilde{A}=1)
\{ E_a(Y \mid W) - E_a(Y) \} \{ E_a(Y \mid L,W) - E_a(Y \mid W) \} \big].  \label{eq:lem:thm-proofuse-d0d1}$$ Combining (\[eq:vardecomp-proofuse-3\]), (\[eq:lem:thm-proofuse-d0d1\]), and Assumptions \[assump:randomization\] and \[assump:independent-censoring\], we derive (\[eq:lem:thm-proofuse-toshow-d0d1\]). 

By (\[eq:thm:avar-proofuse-d0-simp\]) and (\[eq:thm:avar-proofuse-d2-simp\]) we have $${\text{Cov}}(D_0, D_2) \propto E \big[ {{\mathds 1}}(\tilde{A}=1,C^Y=1)
\{ E_a(Y \mid W) - E_a(Y) \} \{ Y - E_a(Y \mid L,W) \} \big].  \label{eq:lem:thm-proofuse-d0d2}$$ Combining (\[eq:vardecomp-proofuse-2\]), (\[eq:lem:thm-proofuse-d0d2\]), and Assumptions \[assump:randomization\] and \[assump:independent-censoring\], we derive (\[eq:lem:thm-proofuse-toshow-d0d2\]). 

By (\[eq:thm:avar-proofuse-d1-simp\]) and (\[eq:thm:avar-proofuse-d2-simp\]) we have $${\text{Cov}}(D_1, D_2) \propto E \big[ {{\mathds 1}}(\tilde{A}=1,C^Y=1)
\{ E_a(Y \mid L,W) - E_a(Y \mid W) \} \{ Y - E_a(Y \mid L,W) \} \big].  \label{eq:lem:thm-proofuse-d1d2}$$ Combining (\[eq:vardecomp-proofuse-1\]), (\[eq:lem:thm-proofuse-d1d2\]), and Assumptions \[assump:randomization\] and \[assump:independent-censoring\], we derive (\[eq:lem:thm-proofuse-toshow-d1d2\]). 

This completes the proof. 

Proof of Lemma \[lem:thm-ate-proofuse\] {#proof:lem:thm-ate-proofuse}
---------------------------------------

For notation simplicity, we use $E_1(\cdot)$ and $E_0(\cdot)$ to denote $E(\cdot \mid A=1)$ and $E(\cdot \mid A=0)$, respectively.  By (\[eq:thm:avar-ate-proofuse-d0\]) and (\[eq:thm:avar-ate-proofuse-d1\]) we have $$\begin{aligned}
{\text{Cov}}(D_0, D_1) \propto & E \big[ A \big\{ Q(W,A=1) - Q(W,A=0) \big\} \big\{ Q(W,L,A=1) - Q(W,A=1) \big\} \big] \nonumber \\
& - E \big[ (1-A) \big\{ Q(W,A=1) - Q(W,A=0) \big\} \big\{ Q(W,L,A=0) - Q(W,A=0) \big\} \big] \nonumber \\
&= \sum_{a\in\{0,1\}} E_a \big[ \big\{ E_1(Y \mid W) - E_0(Y \mid W) \big\} \big\{ E_a(Y \mid W,L) - E_a(Y \mid W) \big\} \times P(A=a).  \label{eq:lem:thm-ate-proofuse-d0d1}\end{aligned}$$ Both terms in (\[eq:lem:thm-ate-proofuse-d0d1\]) equals 0 by (\[eq:cond-exp-ate-1\]) in Lemma \[lem:cond-exp-ate\] with $f(W) = E_1(Y \mid W) - E_0(Y \mid W)$. This yields (\[eq:lem:thm-ate-proofuse-toshow-d0d1\]). 

By (\[eq:thm:avar-ate-proofuse-d0\]) and (\[eq:thm:avar-ate-proofuse-d2\]) we have $$\begin{aligned}
{\text{Cov}}(D_0, D_2) \propto & E \big[ A \big\{ Q(W,A=1) - Q(W,A=0) \big\} \big\{ Y - Q(W,L,A=1) \big\} \big] \nonumber \\
& - E \big[ (1-A) \big\{ Q(W,A=1) - Q(W,A=0) \big\} \big\{ Y - Q(W,L,A=0) \big\} \big] \nonumber \\
&= \sum_{a\in\{0,1\}} E_a \big[ \big\{ E_1(Y \mid W) - E_0(Y \mid W) \big\} \big\{ Y - E_a(Y \mid W,L) \big\} \times P(A=a).  \label{eq:lem:thm-ate-proofuse-d0d2}\end{aligned}$$ Both terms in (\[eq:lem:thm-ate-proofuse-d0d2\]) equals 0 by (\[eq:cond-exp-ate-2\]) in Lemma \[lem:cond-exp-ate\] with $g(W,L) = E_1(Y \mid W) - E_0(Y \mid W)$. This yields (\[eq:lem:thm-ate-proofuse-toshow-d0d2\]). 

By (\[eq:thm:avar-ate-proofuse-d1\]) and (\[eq:thm:avar-ate-proofuse-d2\]) we have $$\begin{aligned}
{\text{Cov}}(D_1, D_2) \propto & E \big[ A \big\{ Q(W,L,A=1) - Q(W,A=1) \big\} \big\{ Y - Q(W,L,A=1) \big\} \big] \nonumber \\
& - E \big[ (1-A) \big\{ Q(W,L,A=0) - Q(W,A=0) \big\} \big\{ Y - Q(W,L,A=0) \big\} \big] \nonumber \\
&= \sum_{a\in\{0,1\}} E_a \big[ \big\{ E_a(Y \mid W,L) - E_a(Y \mid W) \big\} \big\{ Y - E_a(Y \mid W,L) \big\} \times P(A=a).  \label{eq:lem:thm-ate-proofuse-d1d2}\end{aligned}$$ Both terms in (\[eq:lem:thm-ate-proofuse-d1d2\]) equals 0 by (\[eq:cond-exp-ate-2\]) in Lemma \[lem:cond-exp-ate\] with $g(W,L) = E_a(Y \mid W,L) - E_a(Y \mid W)$. This yields (\[eq:lem:thm-ate-proofuse-toshow-d1d2\]). 

This completes the proof. 

[^1]: Department of Statistics, Harvard University.  qiantianchen@fas.harvard.edu

[^2]: Department of Biostatistics, Johns Hopkins University. 

[^3]: Vatic Labs, New York.