ariable importance measures for heterogeneous causal effect

The conditional average treatment effect (CATE) of a binary exposure on an outcome is often studied on the basis that personalised treatment decisions lead to better clinical outcomes. The CATE, however, may be a complicated function of several covariates representing patient characteristics. As such, clinicians rely on researchers to summarise insights related to treatment effect heterogeneity. Clinical research usually focuses on the ATE, which averages the CATE across all patient characteristics, or considers the treatment effect in patient strata, though strata are rarely chosen systematically. Here we present new nonparametric treatment effect variable importance measures (TE-VIMs). These may guide clinicians as to which patient characteristics are important to consider when making treatment decisions, and help researchers stratify patient populations for further study. TE-VIMs extend recent regression-VIMs, viewed as nonparametric analogues to ANOVA statistics. Moreover, TE-VIMs are not tied to a particular model, thus are amenable to data-adaptive (machine learning) estimation of the CATE, itself an active area of research. Estimators for the proposed statistics are derived from their efficient influence curves and these are illustrated through a simulation study and an applied example.

<span>*Keywords:*</span> Causal inference; Conditional effects; Treatment effects; Data-adaptive estimation

Introduction
============

In the medical and social sciences there has been a longstanding interest in quantifying heterogeneity in the effects of treatments or interventions between groups of individuals. Understanding such heterogeneity is essential, for instance, in informing scientific research and optimizing treatment decisions. Suggestions to exploit this for personalised medicine have been expressed as early as the 1970's (if not before) [@Senn2001; @Rosenkranz2020], leading up to the domain of pharmacogenetics. Attention focused initially on subgroup analyses, which identify population subgroups (defined in terms of pre-treatment covariates) which benefit most/least from treatment, to be evaluated further in potential future studies; see e.g. @Rothwell2005 for a review, and @Slamon2001 for the first clinical trial in oncology that was restricted to targeted trial populations. Typical challenges of subgroup analyses are how to select stratification variables in a systematic way, and how to handle the resulting multiplicity problem. Endeavours to address these were soon followed by methodological developments on personalised medicine in the causal inference literature, pioneered by @Murphy2003.

For many years, the primary focus was on policy learning; that is, determining the optimal treatment policy (which assigns the same treatment to individuals with the same measured covariate values) with the aim to minimise (some measure of) the population risk that would be seen if that policy were applied, uniformly in the population [@VanderLaan2014; @Kallus2020; @Athey2021]. More recently, attention has shifted towards (machine-learning based) estimation of conditional average treatment effects (CATEs) [@Abrevaya2015; @Athey2016; @Nie2017; @Kallus2018; @Wager2018; @Kunzel2019; @Kennedy2020; @Knaus2021]. Letting $Y^a$ denoting the outcome that would be observed if treatment $A$ were set to the value $ a\in \{0,1\}$, and $X$ a vector of pre-treatment covariates, the CATE can be defined as $\tau(x) \equiv {E}(Y^1-Y^0|X=x)$ [@Rubin1974]. Estimates of the CATE can also been used for policy learning, for instance, the so-called optimal dynamic treatment rule (OTR) for an individual with covariate value $x$, assigns treatment based on the sign of $\tau(x)$. The CATE, however, additionally provides insight into the magnitude of the treatment effect for these individuals.

These foregoing developments are extremely useful and important, but they leave unanswered a key question that researchers commonly have when being presented an estimated CATE or OTR: namely, what are the key drivers of treatment effect heterogeneity? In the context of policy learning, attempts to find the optimal policy within a restricted class of 'simple' policies go some way towards answering this question, albeit at the risk of targeting a suboptimal policy [@Zhang2015]. Indeed, by comparing the mean outcome under the optimal policy, with the mean outcome under a suboptimal policy, which depends only on certain covariates, one may quantify the importance of the excluded covariates in determining the optimal policy [@Williamson2021_general].

In this paper, we will address this question by instead quantifying the importance of variable subsets in determining the CATE. By shifting focus from the OTR to the CATE, we argue that the resulting variable importance measures are easier to infer as well as more interesting from a scientific perspective, since they provide greater insight into effect modifiers. For instance, it may be the case that a particular treatment is uniformly beneficial, in which case the optimal policy is to always treat, despite extremely large treatment effect heterogeneity. An understanding of such heterogeneity may, e.g. provide information about treatment mechanism, suggest future therapies, be used to compare clinical trial populations, and be used to quantify systematic treatment biases (such as on the basis of race or socio-economic status).

One existing proposal for CATE variable attribution is based on the 'causal forest' CATE estimator, which extends the widely used random forest algorithm to CATE estimation [@Athey2019; @Wager2018; @Athey2016; @Breiman2001_rf]. The resulting variable importance measures (VIMs) rely on the 'tree architecture' of causal forest models, thus are inherently tied to the CATE estimation strategy, and have also been criticised as they tend to assign greater importance to continuous variables, or categorical variables with many categories [@Gromping2009; @Strobl2007]. Generally, VIMs which depend on a particular modelling strategy (e.g. random forests/ linear regression) are referred to as 'algorithmic' [@Williamson2020], with the disadvantage that algorithmic VIMs are well defined and interpretable only within their particular modelling strategy. The need to define generic nonparametric VIMs is closely related to the need for methods which explain the output of 'black box' machine learning prediction algorithms, itself an active area of research in the computer science literature. One popular approach is based on so-called Shapley additive explanation (SHAP) values [@Lundberg2017unified], which quantify the direction and magnitude of each covariate in obtaining model predictions. Applications of SHAP to CATE estimation are rare, with @Syrgkanis2019 being a notable recent example. Debate remains, however, over exactly how SHAP values should be defined and interpreted causally [@janzing2020feature; @chen2020true].

In view of these issues, we propose treatment effect variable importance measures (TE-VIMs), which are model-free scalar summary statistics intended to measure the importance of subsets of covariates in predicting the CATE. In particular, our proposed estimands quantify the contribution of covariate subsets towards the variance ${\mathrm{var}}\{\tau(X)\}$ of the treatment effect (VTE) [@Levy2021]; the latter quantifies treatment effect heterogeneity by capturing the extent to which varying effects of treatment can be explained by observed covariates. More precisely, we will consider the estimand $$\begin{aligned}
\Theta_s &\equiv {E}[{\mathrm{var}}\{\tau(X)|X_{-s}\}] = {\mathrm{var}}\{\tau(X)\} - {\mathrm{var}}\{\tau_s(X)\} , \label{estimand_theta}\end{aligned}$$ where $X\in \mathbb{R}^p$ represents a $p$-dimensional covariate vector, the symbol $u_{-s}$ denotes the vector of all the components of $u$ with index not in $s \subseteq \{1,...,p\}$, and $\tau_s(x) \equiv {E}\{\tau(X)|X_{-s}=x_{-s}\}$ denotes the CATE conditional on $X_{-s}$; note that $\tau_s(x)$ only depends on $x_{-s}$, but we write it as a function of $x$ for simplicity of notation. We interpret $\Theta_s \geq 0$ as a difference in VTEs, quantifying the amount by which the VTE changes when variables in the set $s$ are excluded from the model. More formally, it expresses the additional treatment effect heterogeneity explained by $X_s$, over and above that already explained by $X_{-s}$, where for a vector $u$, we let $u_s$ denote the vector of all components of $u$ with index in $s$.

The proposed TE-VIMs rescale $\Theta_s$ by the VTE to express this difference as a proportion, $$\begin{aligned}
\Psi_s &\equiv \frac{\Theta_s}{{\mathrm{var}}\{\tau(X)\}} = 1 - \frac{{\mathrm{var}}\{\tau_s(X)\}}{{\mathrm{var}}\{\tau(X)\}}. \label{estimand}\end{aligned}$$ Assuming that the VTE is non-zero, we interpret $\Psi_s \in [0,1]$ as the proportion of treatment effect heterogeneity explained by $X_{-s}$ compared with $X$. This interpretation is analogous to the familiar coefficient of determination ($R^2$ statistic). The proposed TE-VIMs connect VTE estimands [@Levy2021] to recently proposed regression-VIMs [@Williamson2021; @Zhang2020], also referred to as 'leave out covariates' [@Verdinelli2021; @Lei2018], and the more general VIM framework by @Williamson2021_general. In this way, our work represents a step towards extending VIMs to the analysis of more general statistical functionals, as discussed in Section \[sect:extend\].

In Section \[sec\_methods\] we motivate TE-VIMs, and provide estimators which are efficient under the nonparametric model. These rely on estimating working models, relating our proposal to the DR-learner of the CATE through an interpretation based on so-called pseudo-outcomes [@Kennedy2020; @Luedtke2016; @VanderLaan2013]. Experimental results on simulated data are provided in Section \[sect:simulation\] and Section \[sect:example\] demonstrates an application to the AIDS Clinical Trials Group Protocol 175 [@Hammer1996].

Methodology {#sec_methods}
===========

Motivating the estimand
-----------------------

Suppose we have $n$ i.i.d. observations $(z_1,...,z_n)$ of a random variable $Z$ distributed according to an unknown distribution $P$, such that $Z$ consists of $(Y,A,X)$, where $Y\in \mathbb{R}$ is an 'outcome', $A\in\{0,1\}$ is an 'exposure' or 'treatment' and $X \in \mathbb{R}^p$ is a $p$-dimensional vector of covariates. Under standard identification assumptions of consistency ($A=a \implies Y=Y^a$), conditional exchangeability ($Y^a {\perp\!\!\!\perp}A|X$ for $a=0,1$), and positivity ($0<\pi(X)<1$ w.p.1), the CATE is identified by $\tau(x)\equiv {E}(Y^1-Y^0|X=x)=\mu(1,x)-\mu(0,x)$, where $\mu(a,x)\equiv E(Y|A=a,X)$ and $\pi(x)\equiv {E}(A|X=x)$ denotes the 'propensity score'.

Assume that $||\tau|| < \infty$, where $||f|| \equiv {E}\{f(X)^2\}^{1/2}$ is the $L_2(P)$ norm. With this choice our estimand is finite and well defined, since $$\begin{aligned}
\Theta_{s} = {E}[\{\tau(X)-\tau_s(X)\}^2] = ||\tau - \tau_s||^2  <\infty,\end{aligned}$$ Notice that the VTE is $\Theta_{p}$, where, with a slight abuse of notation, $p$ denotes the index set $\{1,...,p\}$ and $\tau_p$ is the ATE. We further assume that the VTE is non-zero, i.e. $\Theta_{p}>0$ and since $\Theta_{p} \geq \Theta_{s}$, it follows that $\Psi_s =  \Theta_{s}/\Theta_{p} \in [0,1]$. The regression-VIM in @Williamson2021 is analogous to our proposal, in the sense that the former replaces $\tau(x)$ with $\mu(x)$ and $\tau_s(x)$ with $\mu_s(x)\equiv{E}(Y|X_{-s}=x_{-s})$. Specifically, they consider, $$\begin{aligned}
\theta_s \equiv {E}[{\mathrm{var}}\{\mu(X)|X_{-s}\}] = {E}[\{\mu(X)-\mu_s(X)\}^2]\end{aligned}$$ which is analogous to $\Theta_s$. The two proposals, however, differ in how this mean conditional variance is scaled. @Williamson2021 consider scaling by the outcome variance, i.e. by defining $\psi_s\equiv \theta_s/{\mathrm{var}}(Y)$, whereas we scale by the VTE, i.e. $\Psi_s = \Theta_s/\Theta_p$. We scale by the VTE because the treatment effect variance, ${\mathrm{var}}(Y^1-Y^0)$, is generally not identifiable without strong assumptions [@Levy2021; @Ding2016; @Heckman1997]. The VTE, however, is a convenient scaling parameter which bounds $\Psi_s\in[0,1]$, aiding interpretability since, when the VTE is non-zero, $\Psi_s$ behaves like a coefficient of determination ($R^2$).

In practice, the scaling factor makes little difference to the interpretation of our estimands, since investigators are likely to compare the relative importance of covariate sets $s$ and $s^\prime$ by comparing the magnitudes of $\Psi_s$ and $\Psi_{s^\prime}$. This approach is demonstrated in Section \[sect:example\], where the importance of each covariate is ranked individually using $\Psi_s$ where $s$ is a set containing a single covariate of interest. Quantifying variable importance in this way, however, may be problematic when covariates are themselves highly correlated. An alternative could be to define importance with reference to $\Psi_s$ where $s$ is the set of all covariates except the covariate of interest, or else to define variable importance with reference to so-called Shapley population VIMs, which consider all possible covariate permutations that do not include the variable of interest [@Owen2017; @Williamson2020]. To rank all covariates, the latter are highly computationally intensive, requiring CATE estimates for each of the $2^p$ possible covariate subsets. Instead, @Williamson2021_general recommend using domain specific knowledge to group covariates, e.g. one might compare the relative importance of biological factors vs. non-biological factors in determining the CATE. We remark that decreasing the index set can never increase the TE-VIM, in the sense that $s^\prime \subseteq s$ implies that $\Psi_{s^\prime} \leq \Psi_s$, i.e. the covariate set $s^\prime$ cannot be more important than $s$.

CATE estimation {#sect_cates}
---------------

Estimation of the proposed TE-VIM will rely on initial CATE estimates, obtained via flexible machine learning based methods [@Knaus2021], which we review first. CATE estimation is challenging since common machine learning algorithms (random forests, neural networks, boosting etc.) are instead designed for mean outcome regression, such as by minimising the mean squared error loss. CATE estimation strategies therefore either modify existing machine learning methods to target CATEs, e.g. @Athey2019 [@Wager2018] and @Athey2016 modify the random forest algorithm for CATE estimation. Alternatively, 'metalearning' strategies decompose CATE estimation into a sequence of sub-regression problems, which can be solved using off-the-shelf machine learning algorithms, see e.g. @Kunzel2019 [@Nie2017; @Kennedy2020].

In the current work we focus on two metalearning algorithms which, following the naming convention of @Kunzel2019, we refer to as the T-learner and the DR-learner. The T-learner is based on the decomposition $\tau(x)=\mu(1,x)-\mu(0,x)$, and the T-learner estimate of the CATE is $\hat{\tau}^{(T)}(x)\equiv \hat{\mu}(1,x)-\hat{\mu}(0,x)$, where $\hat{\mu}(a,x)$ represents an estimate of $\mu(a,x)$ obtained by a regression of $Y$ on $(A,X)$. The T-learner, however is problematic for two main reasons. Firstly, whilst regularisation methods can be used to control the smoothness of $\hat{\mu}(a,x)$, the same is not true of $\hat{\tau}^{(T)}(x)$ which may be highly erratic. Slow convergence rates affecting $\hat{\mu}(a,x)$ may therefore propagate into the CATE estimator $\hat{\tau}^{(T)}(x)$. Secondly, $\hat{\mu}(1,x)$ is chosen to make an optimal bias-variance trade-off over the covariate distribution of the treated population. Likewise, $\hat{\mu}(0,x)$ is chosen to make an optimal bias-variance trade-off over the covariate distribution of the untreated population. When there is poor overlap between the treated and untreated subgroups, then the difference $\hat{\mu}(1,x)-\hat{\mu}(0,x)$ may fail to deliver an optimal bias-variance trade-off over the population covariate distribution, making the T-learner potentially poorly targeted towards CATE estimation.

The DR-learner [@Kennedy2020; @Luedtke2016; @VanderLaan2013] is an alternative metalearning algorithm based on the decomposition $\tau(x)={E}\{\varphi(Z)|X=x\}$ where $$\begin{aligned}
\varphi(z) \equiv \{y-\mu(a,x)\}\frac{a-\pi(x)}{\pi(x)\{1-\pi(x)\}} + \mu(1,x) - \mu(0,x).\end{aligned}$$ is called the 'pseudo outcome', or the augmented inverse propensity weighted score [@Robins1994], which acts like the causal contrast, $Y^1-Y^0$, in expectation. The DR-learning procedure first estimates $\mu(a,x)$ and $\pi(x)$ to obtain the pseudo-outcome plug-in estimator $$\begin{aligned}
\hat{\varphi}(z) \equiv \{y-\hat{\mu}(a,x)\}\frac{a-\hat{\pi}(x)}{\hat{\pi}(x)\{1-\hat{\pi}(x)\}} + \hat{\mu}(1,x) - \hat{\mu}(0,x),\end{aligned}$$ In a second step, the estimated pseudo-outcome, $\hat{\varphi}(Z)$ is regressed on covariates $X$ to obtain $\hat{\tau}^{(DR)}(x)$. A sample splitting scheme is also recommended, whereby the regression steps to obtain $\hat{\mu}(a,x)$, $\hat{\pi}(x)$, and $\hat{\tau}^{(DR)}(x)$ are performed on three independent samples, see @Kennedy2020 for details.

The DR-learner alleviates the issues related to the T-learner since the complexity of $\hat{\tau}^{(DR)}(x)$ can be controlled by regularising the regression in the final stage of the procedure, mitigating concerns regarding the smoothness of the T-learner. With regard to consistency, the square of ${E}\{\hat{\varphi}(Z)|X=x\}-\tau(x)$ is bounded by at most the product of the squared errors of the propensity score and regression estimators (up to constant scaling). In practice, this means that the final regression step, where $\hat{\varphi}(Z)$ is regressed on $X$, mimics the oracle regression of $\varphi(Z)$ on $X$ provided that

1.  The propensity score and outcome estimators are 'rate double robust' in the sense that $\{\pi(x)-\hat{\pi}(x)\}\{\mu(a,x)-\hat{\mu}(a,x)\}$ is $o_P(n^{-1/2})$ in $L_2(P)$ norm for $a=0,1$.

This requirement implies that one can trade-off accuracy in the outcome and propensity score estimators, a property which is known as rate double robustness, hence the name 'DR-learner'.

Estimation of the CATE $\tau_s(x)$ is complicated by the fact that one cannot assume that $Y{\perp\!\!\!\perp}A|X_{-s}$ for an arbitrary subset of covariates $s$, a problem that is sometimes referred to as 'runtime confounding' [@Coston2020]. The DR-learner readily extends to the setting of runtime confounding through the decomposition $\tau_s(x)={E}\{\varphi(Z)|X_{-s}=x_{-s}\}$. This expression implies that one may estimate $\tau_s(x)$ by regressing $\hat{\varphi}(Z)$ on $X_{-s}$, i.e. by modifying the final regression step of the DR-learner. In this paper we propose a metalearner for $\tau_s(x)$ based on the decomposition $\tau_s(x)={E}\{\tau(X)|X_{-s}=x_{-s}\}$. Specifically we propose regressing an initial estimate of the CATE, $\hat{\tau}(X)$ on $X_{-s}$ to obtain an estimate of $\tau_s(x)$. This approach is agnostic the initial CATE estimator used and, like the DR-learner, one can control the complexity of the resulting CATE estimator $\hat{\tau}_s(x)$ by regularisation. We advocate this approach since it generally results in estimates of $\tau_s(x)$, which are compatible with those of $\tau(x)$, similar to @Williamson2021, who recommend estimating $\mu_s(x)$ by regressing an estimate $\hat{\mu}(x)$ of $\mu(x)$ on $X_{-s}$.

TE-VIM estimation
-----------------

### Efficient estimators of $\Theta_s$ {#efficient-estimators-of-theta_s .unnumbered}

Next we consider estimators based on the efficient influence curve (IC) of $\Theta_s$ under the nonparametric model. Briefly, ICs are model-free, mean zero, functionals that characterise the sensitivity of an estimand to small changes in the data generating law. As such, ICs are useful for constructing efficient estimators and determining their asymptotic distribution, see e.g. @Hines2021 [@Fisher2020] for an introduction to these methods. In the Appendix we derive the IC of $\Theta_s$ at a single observation $z = (y,a,x)$ of $Z$ to be $$\begin{aligned}
\phi_s(z) &= \{\varphi(z)-\tau_s(x)\}^2 - \{\varphi(z)-\tau(x)\}^2  - \Theta_{s}. \label{IC:theta}\end{aligned}$$ The interpretation of $\varphi(Z)$ as a pseudo-outcome which plays the role of the unobserved causal contrast $Y^1-Y^0$ holds in the present context. To see why, we compare the IC in with that of $\theta_s$ given by @Williamson2021, $$\begin{aligned}
 \{y-\mu_s(x)\}^2 - \{y-\mu(x)\}^2  - \theta_s.\end{aligned}$$ The IC of $\theta_s$ has the same form as , since the latter is recovered by replacing the outcome $y$ with the pseudo-outcome, $\varphi(z)$, replacing the conditional mean outcomes with CATEs, i.e. conditional means of $\varphi(Z)$, and replacing $\theta_s$ with $\Theta_s$. Efficient estimating equations estimators can be derived from ICs by setting the sample mean of (an estimate of) the IC to zero. In the current setting, this strategy is equivalent to the so-called one-step correction which we outline in the Appendix. For $\Theta_s$ and $\theta_s$, we thus obtain the estimators $$\begin{aligned}
\hat{\Theta}_{s} &\equiv n^{-1}\sum_{i=1}^n \{\hat{\varphi}(z_i)-\hat{\tau}_s(x_i)\}^2 - \{\hat{\varphi}(z_i)-\hat{\tau}(x_i)\}^2 \label{Est:theta} \\
\hat{\theta}_{s} &\equiv n^{-1}\sum_{i=1}^n \{y_i-\hat{\mu}_s(x_i)\}^2 - \{y_i-\hat{\mu}(x_i)\}^2, \nonumber\end{aligned}$$ where $\hat{\tau}(x)$ and $\hat{\tau}_s(x)$ and $\hat{\varphi}(z)$ are consistent estimators fitted on an independent sample. In practice, a cross-fitting approach may be used to obtain the fitted models and evaluate $\hat{\Theta}_s$ from a single sample [@van_der_laan_cross-validated_2011; @Chernozhukov2018]. Theorem 1 below states the asymptotic distribution of $\hat{\Theta}_{s}$ under (A1) and when,

1.  The differences $\tau(x)-\hat{\tau}(x)$ and $\tau_s(x)-\hat{\tau}_s(x)$ are both $o_P(n^{-1/4})$ in $L_2(P)$ norm.

The requirement for $n^{1/4}$ rate convergence in (A2) is standard in the recent VIM framework of @Williamson2021_general, for instance, an assumption similar to (A2) appears in the regression-VIM setting for the outcome models $\hat{\mu}(x)$ and $\hat{\mu}_s(x)$. In the TE-VIM setting, however, one is also required to estimate the pseudo-outcomes, with (A1) required to control the error which arises from doing so.

Assumptions (A1) and (A2) suggest that the DR-learner of the CATE may be preferable to the T-learner due to its robustness properties. In particular, consider that the T-learner of the CATE $\hat{\tau}^{(T)}(x)$ satisfies the first condition of (A2), provided that $\mu(a,x)-\hat{\mu}(a,x)$ is $o_P(n^{-\delta})$ in $L_2(P)$ norm, with $\delta \geq 1/4$. (A1) then implies that $\pi(x)-\hat{\pi}(x)$ must be at least $o_P(n^{-1/2 + \delta})$ in $L_2(P)$ norm. In other words, the propensity score estimator is allowed to converge at a slower rate, provided the outcome estimator converges at a faster rate, but the converse is not true. This is unsatisfying for example in clinical trial settings, where the exposure is randomised and the propensity score model is known, since the T-learner of the CATE would still require $n^{1/4}$ rate convergence of the outcome model.

Conversely, the DR-learner $\hat{\tau}^{(DR)}(x)$ satisfies the first condition of (A2), provided that (A1) holds and ${E}\{\hat{\varphi}(Z)|X=x\}-\hat{\tau}^{(DR)}(x)$ is $o_P(n^{-1/4})$ in $L_2(P)$ norm, i.e. provided the final DR-learning regression estimator is consistent at $n^{-1/4}$ rate. Applying the same reasoning as before, we see that (A1) implies that $\mu(a,x)-\hat{\mu}(a,x)$ can be $o_P(n^{-\delta})$ in $L_2(P)$ norm, provided that $\pi(x)-\hat{\pi}(x)$ is $o_P(n^{-1/2 + \delta})$ in $L_2(P)$ norm, for any $\delta \geq 0$. In other words, the outcome estimator is allowed to converge at a slower rate, provided the propensity score estimator converges at a faster rate and vice-versa, which marks an improvement over the T-learner. The asymptotic distribution of $\Theta_s$ under (A1) and (A2) is given Theorem \[asym\_theorem\] below.

\[asym\_theorem\] Under (A1), (A2) and regularity assumptions given in the Appendix, $\hat{\Theta}_{s}$ is asymptotically linear with IC, $\phi_s(Z)$, and hence $\hat{\Theta}_{s}$ converges to $\Theta_{s}$ in probability, and for $\Theta_{s} >0$ then $n^{1/2}(\hat{\Theta}_{s}-\Theta_{s})$ converges in distribution to a mean-zero normal random variable with variance ${E}\{\phi^2_s(Z)\}$.

### Importance testing {#importance-testing .unnumbered}

As well as being used to derive efficient estimators, the IC characterises the asymptotic distribution of such estimators, as in Theorem \[asym\_theorem\]. Since $\hat{\Theta}_{s}$ and $\hat{\theta}_{s}$ are analogous, we expect them to share similar properties. One such property concerns the behaviour under the zero-importance null hypothesis, $H_0: \Theta_{s} = 0$, i.e. when $\tau(x)=\tau_s(x)$. In practice, this hypothesis corresponds to treatment effect homogeneity over $X_s$ given $X_{-s}$, thus the IC in is exactly zero. The fact that the IC degenerates in this way makes $H_0$ difficult to test, since Wald-type tests, i.e. those based on the asymptotic distribution of $\hat{\Theta}_s$, tend to overestimate the variance of $\hat{\Theta}_s$ and are therefore conservative. For this reason, Theorem \[asym\_theorem\] gives the behaviour of the estimator only when $\Theta_{s} > 0$.

One solution to the IC degeneracy problem under $H_0$, proposed by @Williamson2021_general, is to estimate ${\mathrm{var}}\{\tau(X)\}$ and ${\mathrm{var}}\{\tau_s(X)\}$ using efficient estimators in separate samples, with each estimand having a non-zero IC provided that ${\mathrm{var}}\{\tau_s(X)\}>0$, despite the two ICs being identical under $H_0$. With this condition, both estimators would be independent and asymptotically normal, hence their difference, which represents an estimator of $\Theta_s$, would also be asymptotically normal even when $\Theta_s=0$. One therefore obtains a valid Wald-type test for $H_0$, at the expense of using an estimator for $\Theta_s$, which is inefficient due to the requirement for sample splitting. Similarly, one could test the zero-VTE null hypothesis $({\mathrm{var}}\{\tau(X)\}=0)$ by estimating ${E}\{\tau^2(X)\}$ and ${E}\{\tau(X)\}^2$ using efficient estimators in separate samples and taking their difference. Fundamentally, however, the distribution of $\hat{\Theta}_s$ under $H_0$ depends on higher-order pathwise derivatives of the estimand, see e.g. @Carone2018, and remains generally an open problem.

### Targeted maximum likelihood estimation {#targeted-maximum-likelihood-estimation .unnumbered}

For the index set $s=p$, $\hat{\Theta}_{p}$ is an estimator of the VTE, which is distinct from the targeted maximum likelihood estimation (TMLE) VTE estimator proposed by @Levy2021. Both are based on initial estimates of $\mu(a,x)$ and $\pi(x)$ and are regular asymptotically linear in the sense of ensuring that the sample mean of the estimated IC is negligible. The TMLE estimator achieves this by replacing the initial estimates $\hat{\mu}(a,x)$, with 'retargeted' estimates $\hat{\mu}^*(a,x)$. These retargeted estimates are used to estimate the CATE as $\hat{\tau}^*(x)\equiv \hat{\mu}^*(1,x) - \hat{\mu}^*(0,x)$, with the ATE and VTE obtained respectively as the sample mean and sample variance of $\hat{\tau}^*(X)$. Unlike $\hat{\Theta}_p$, the TMLE VTE estimator is a 'plug-in' estimator, in the sense that it maps a probability distribution (implied by the empirical measure of the covariates, the propensity score estimator $\hat{\pi}(x)$ and the targeted outcome estimator $\hat{\mu}^*(a,x)$) onto an estimated value using the definition of $\Theta_p$. See e.g. @Hines2021 for an introductory comparison of TMLE estimators with one-step bias correction estimators.

### Efficient estimators of $\Psi_s$ {#efficient-estimators-of-psi_s .unnumbered}

Our main goal is to consider efficient estimation of the rescaled TE-VIM $\Psi_s=\Theta_s/\Theta_p$, which has IC, $$\begin{aligned}
\Phi_s(z) &= \{\phi_s(z) - \Psi_{s}\phi_{p}(z)\}/\Theta_{p}, \label{IC:psi}\end{aligned}$$ where $\phi_p(.)$ denotes for the index set $s=p$. This IC implies an estimating equations estimator, $\hat{\Psi}_{s} = \hat{\Theta}_{s}/\hat{\Theta}_{p}$. Like $\phi_s(z)$, the IC $\Phi_s(z)$ is also degenerate, with $\Phi_s(z)=0$ when $\Psi_{s}=0$ and when $\Psi_{s}=1$, corresponding to $\Theta_s=0$ and $\Theta_s=\Theta_p$ respectively. For this reason, the asymptotic result in Theorem \[asym\_theorem2\] below, holds only when $\Psi_s \in (0,1)$, i.e. when the covariate set $s$ accounts for some, but not all, CATE variability. We make the additional assumption (B1) that the difference $\tau_p-\hat{\tau}_p$ is $o_P(n^{-1/4})$.

\[asym\_theorem2\] Assume that $\Theta_p\neq 0 $. Under (A1), (A2), (B1) and regularity assumptions given in the Appendix, $\hat{\Psi}_{s}$ is asymptotically linear with IC, $\Phi_s(Z)$, and hence $\hat{\Psi}_{s}$ converges to $\Psi_{s}$ in probability, and for $\Psi_{s} \in (0,1)$ then $n^{1/2}(\hat{\Psi}_{s}-\Psi_{s})$ converges in distribution to a mean-zero normal random variable with variance ${E}\{\Phi_s(Z)^2\}$.

### Proposed algorithms for estimating $\Psi_s$ {#proposed-algorithms-for-estimating-psi_s .unnumbered}

The estimator $\hat{\Theta}_{s}$ is indexed by the choice of pseudo-outcome and CATE estimators. Generally, we are not constrained to any particular learning method. Throughout, we consider a plug-in estimator for $\varphi(.)$, where models for $\mu(a,x)$ and $\pi(x)$ are obtained from the same sample, and these are used to construct $\hat{\varphi}(z)$. We consider two approaches to learning $\tau(x)$ based on the T- and DR-learning strategies, though for computational simplicity, our 'DR-learner' does not use the sample splitting scheme proposed by @Kennedy2020.

We expect the estimator based on the DR-learning approach to be less biased in finite samples than that based on the T-learning approach, due to the robustness properties of the DR-learner and since it explicitly seeks to minimise the term, $n^{-1}\sum_{i=1}^n \{\hat{\varphi}(z_i)-\hat{\tau}(x_i)\}^2$, which appears in . This term can be problematic in practice, since it may give negative $\hat{\Theta}_s$ estimates, when $\hat{\tau}(.)$ converges slowly to $\tau(.)$. In the regression-VIM work of @Williamson2021, the authors found that regressing the observed outcome on $X$ and $X_{-s}$ returns conditional mean models $\hat{\mu}(x)$ and $\hat{\mu}_s(x)$ which do not take into account that the two conditional means are related, generally resulting in incompatible estimates. Their solution was to first regress the outcome on $X$ then regress predictions from the resulting conditional mean model on $X_{-s}$. We propose a similar approach to learning $\hat{\tau}_s(x)$, effectively using the 'runtime confounding' CATE metalearning strategy described in Section \[sect\_cates\].

The proposed working function estimators are implemented in Algorithms 1 and 2 below. These algorithms return pseudo-outcome and CATE estimates, $\{\hat{\varphi}_i\}_{i=1}^n,\{\hat{\tau}_i\}_{i=1}^n,$, $\{\hat{\tau}_{s,i}\}_{i=1}^n$, and $\{\hat{\tau}_{p,i}\}_{i=1}^n$, which can be used to obtain $\hat{\Psi}_s = \hat{\Theta}_s/\hat{\Theta}_p$, with variance estimated by $n^{-2}\sum_{i=1}^n \hat{\phi}_i^2$, where, $$\begin{aligned}
\hat{\Theta}_s &= n^{-1} \sum_{i=1}^n \{\hat{\varphi}_i-\hat{\tau}_{s,i}\}^2 - \{\hat{\varphi}_i-\hat{\tau}_{i}\}^2 \\
\hat{\phi}_i &= \frac{1}{\hat{\Theta}_p}\left[\{\hat{\varphi}_i-\hat{\tau}_{s,i}\}^2 - \hat{\Psi}_s \{\hat{\varphi}_i-\hat{\tau}_{p,i}\}^2 + (\hat{\Psi}_s-1)\{\hat{\varphi}_i-\hat{\tau}_{i}\}^2\right]\end{aligned}$$ and $\hat{\Theta}_p$ is defined in a similar manner. Algorithm 2 uses a cross-fitting regime to ensure that $\hat{\varphi}_i,\hat{\tau}_i,\hat{\tau}_{s,i},$ and $\hat{\tau}_{p,i}$ are constructed from estimators which are not fitted using the $i$th observation. This is useful in controlling the so-called empirical process term, see e.g. @Newey2018 [@Hines2021]. Both algorithms are also indexed by the choice of CATE learner in steps 2 and 3 of each algorithm respectively, with the substeps marked (A) and (B) referring to the T- and DR-learning strategies. We note that where the algorithms require models to be 'fitted', any suitable regression method can be used.

**Algorithm 1 - Without sample splitting**

1.  Fit $\hat{\mu}(.,.)$ and $\hat{\pi}(.)$. Use these fitted models to obtain $\hat{\varphi}_i \equiv\hat{\varphi}(z_i)$.

2.  \(A) Use the model for $\hat{\mu}(.,.)$ from Step 1, to obtain $\hat{\tau}(x) \equiv\hat{\mu}(1,x)-\hat{\mu}(0,x)$. Or (B) Fit $\hat{\tau}(.)$ by regressing $\hat{\varphi}(Z)$ on $X$. After doing (A) or (B), use the fitted models to obtain $\hat{\tau}_i \equiv \hat{\tau}(x_i)$.

3.  Fit $\hat{\tau}_s(.)$ by regressing $\hat{\tau}(X)$ on $X_{-s}$. Use the fitted model to obtain $\hat{\tau}_{s,i} \equiv\hat{\tau}_s(x_i)$.

4.  Repeat Step 3 for the covariate set $p$ and (optionally) any other covariate sets of interest.

**Algorithm 2 - With sample splitting**

1.  Split the data into $K$ folds.

2.  **For** each fold $k$: Fit $\hat{\mu}(.,.)$ and $\hat{\pi}(.)$ using the data set excluding fold $k$. Use these fitted models to obtain $\hat{\varphi}_i \equiv\hat{\varphi}(z_i)$ for $i$ in fold $k$.

3.  \(A) Use the model for $\hat{\mu}(.,.)$ from Step 2, to obtain $\hat{\tau}(x) \equiv\hat{\mu}(1,x)-\hat{\mu}(0,x)$. Or (B) Fit $\hat{\tau}(.)$ by regressing $\hat{\varphi}(Z)$ on $X$ using the data excluding fold $k$. After doing (A) or (B), use the fitted models to obtain $\hat{\tau}_i \equiv \hat{\tau}(x_i)$ for $i$ in fold $k$.

4.  Fit $\hat{\tau}_s(.)$ by regressing $\hat{\tau}(X)$ on $X_{-s}$ using the data excluding fold $k$. Use the fitted model to obtain $\hat{\tau}_{s,i} \equiv\hat{\tau}_s(x_i)$ for $i$ in fold $k$.

5.  Repeat Step 4 for the covariate set $p$ and (optionally) any other covariate sets of interest. **End for.**

Simulation study {#sect:simulation}
================

In our simulation study we compared Algorithms 1A, 1B, 2A and 2B on generated data in finite samples, using $K=5$ fold sample splitting. We generated $1000$ datasets of size $n\in \{500,1000,2000,3000,4000\}$ from the following structural equation model $$\begin{aligned}
X_1,X_2 &\sim \text{Uniform}(-1,1) \\
A &\sim \text{Bernoulli}\{\text{expit}(- 0.4X_1 + 0.1X_1 X_2)\} \\
Y &\sim {\mathcal{N}\left( \{X_1X_2 + 2X_2^2 - X_1\} + A\tau(X),1\right)}\end{aligned}$$ where the CATE is given by $\tau(X) = X_1^2(X_1 + 7/5) + 25 X_2^2/9$. Analytically computing the true estimand values gives that the TE-VIMs are $\Psi_1 = 0.32$ and $\Psi_2 = 0.68$, with the ATE and VTE taking the values $\tau_p = 1.39$ and $\Theta_p= 1.00$ respectively.

For each dataset, $\hat{\Psi}_s$ was estimated along with its variance (using the variance estimators above), standard error (as the square root of the variance), and associated Wald based (95%) confidence intervals for the index sets, $s = \{1\},\{2\}$. Two regression algorithms were considered for estimation of $\mu(a,x)$, $\pi(x)$, $\tau_s(x)$, and in the case of the DR-learner, $\tau(x)$. The first regression algorithm used generalised additive models, as implemented through the `mgcv` package in R [@Wood2016]. These are flexible spline smoothing models with interaction terms and for the propensity score model a logistic link function was used. The second regression algorithm used random forest learners available through the `ranger` package in R [@Wright2017].

Figure \[sim\_results\_plot\] shows empirical estimates of the bias and variance of $\hat{\Psi}_1$ scaled by $n^{1/2}$ and $n$ respectively, as well as 95% Wald based confidence-interval coverage probabilities. Similar plots for $\hat{\Psi}_2$ are in the Appendix. Comparing Algorithms 1 and 2 (i.e. no sample splitting vs sample splitting), we notice a greater difference in the results when random forest learning is used, with sample splitting generally reducing bias, increasing variance and improving confidence interval coverage.

Additionally, the DR-learning approach (Algorithm B) outperforms the T-learning approach (Algorithm A) in the sense of reducing the bias and achieving confidence interval coverage closer to the 95% level across all sample sizes. This improvement is due to the DR-learner making better use of the propensity score model to improve estimation of the CATE. On the basis of these results, we recommend Algorithm 2B for TE-VIM learning.

![Bias, variance and coverage for $\hat{\Psi}_1$ using 1000 sampled datasets. Red and blue points indicate that working models are fitted using generalised additive modelling and random forests respectively. Top row of plots corresponds to Algorithm 1 (no sample splitting) and the bottom row corresponds to Algorithm 2 (sample splitting). Square and crossed points indicate that the algorithm used the T-learner and DR-learner respectively for CATE estimation. Similar plots for $\hat{\Psi}_2$ are given in the Appendix. <span data-label="sim_results_plot"></span>](sim_plotX1){width="0.8\linewidth"}

Applied example: variable importance of treatment effect heterogeneity in HIV {#sect:example}
=============================================================================

We demonstrate our estimators on data from the AIDS Clinical Trials Group Protocol 175 (ACTG175) [@Hammer1996], which evaluated 2139 patients infected with HIV whose CD4 T-cell count was in the range 200 to 500 $mm^{-3}$. Patients were randomised to 4 treatment groups: (i) zidovudine (ZDV) monotherapy, (ii) ZDV+didanosine (ddI), (iii) ZDV+zalcitabine, and (iv) ddI monotherapy. We compare treatment groups (iv) and (ii) as in @Lu2013 [@Cui2020]. These two groups are represented with the binary indicator, $A=0,1$, with $n=561$ and $n=522$ patients in each group respectively.

Previous studies have used ACTG175 data to analyse the causal effect of $A$ on a survival time endpoint, and the data is available through the `speff2trial` package in R. We consider CD4 count at 20±5 weeks as a continuous outcome, $Y$, and consider 12 baseline covariates, 5 continuous: age, weight, Karnofsky score, CD4 count, CD8 count; and 7 binary: sex, homosexual activity (y/n), race (white/non-white), symptomatic status (symptomatic/asymptomatic), history of intravenous drug use (y/n), hemophilia (y/n), and antiretroviral history (experienced/naive).

TE-VIMs for each covariates were estimated using all algorithms with $K=20$ folds (between 10 to 20 folds is typical for cross-fitting procedures). Propensity score estimates were obtained as the mean of the treatment indicator in the training set. This model is correctly specified since treatment is randomised. Other fitted models (i.e. those for the outcome and CATEs) were obtained using the Super Learner [@VanDerLaan2007], an ensemble learning method, implemented in the `SuperLearner` package in R. This used 20 cross validation folds, and a 'learner library' containing various routines (`glm`, `glmnet`, `gam`, `xgboost`, `ranger`). Additional results which use the 'discrete' Super Learner for model fitting are presented in the Appendix. The discrete Super Learner selects the regression algorithm in the learner library which minimises a cross validated estimate of e.g. the mean squared error loss, whereas the Super Learner minmizes the same loss by taking a convex combination of learners.

![TE-VIM estimates from the ACTG175 study using the Super Learner for functional estimation. Top row: no sample splitting (Algorithm 1). Bottom row: with sample splitting (Algorithm 2). Left col: T-learner (A). Right col: DR-learner (B). Black lines indicate 95% Confidence intervals. In each plot, covariates are sorted according to their TE-VIM point estimate. Dashed lines indicate the $[0,1]$ support of the TE-VIM.<span data-label="results_plot"></span>](plot_full_non_discrete){width="0.75\linewidth"}

The cross-fitted augmented inverse propensity weighted estimate of the ATE was 29.4$mm^{-3}$ (CI: 14.8, 44.1; p$<$0.01) where all confidence-intervals (CIs) are reported at 95% significance and Wald-type p-values are reported. CIs for the VTE were constructed using both algorithms, and we report the square root of these intervals after jberry@example.org. Algorithm 2A gave a root-VTE estimate of 6.44$mm^{-3}$ (CI: 0,55.3; p=0.79) and Algorithm 2B gave a root-VTE estimate of 28.6$mm^{-3}$ (CI: 0, 74.2; p=0.21). By comparison, the CV-TMLE estimator of @Levy2021 gave an ATE estimate of 29.3$mm^{-3}$ (CI: 14.7, 39.3; p$<$0.01) and a root-VTE estimate of 6.39$mm^{-3}$ (CI: 0,39.3; p=0.70). The results from the CV-TMLE algorithm are more similar to those of Algorithm 2A than Algorithm 2B, since the CV-TMLE algorithm uses the T-learner to obtain initial CATE estimates. CV-TMLE is so-called since the initial propensity score and outcome regression estimates obtained in a 'cross validated' way.

These VTE estimates suggest that treatment effect homogeneity is a possible concern for our analysis, however, large treatment effect heterogeneity (relative to the ATE) cannot be ruled out. In practice, TE-VIMs are known to lie on the interval $[0,1]$ therefore, we recommend truncating CIs at these values. For the purposes of comparing the algorithms, however, we have chosen not do so here. The results (Figure \[results\_plot\]) suggest CD4 T-cell count at baseline is the most important variable in determining an individual's treatment effect, with weak evidence that any other single factor is a good predictor of CATE variability. Algorithms 2A and 2B respectively estimated the TE-VIM for CD4 T-cell count to be 3.26 (CI: -18.1, 24.6; p=0.76) and 1.00 (CI: -0.028, 2.03; p=0.057).

On balance, Algorithm B provides more credible TE-VIM estimates in this applied example than Algorithm A. This is evident in the extremely wide confidence intervals provided by Algorithm 1B in Figure \[results\_plot\]. Algorithms A and B both use the same pseudo-outcome estimates, however the former makes better use of the propensity score model (i.e. the fact that this is a random trial), when estimating the CATE, whereas the latter estimates the CATE based on outcome regression alone. This distinction means that we expect the CATE estimates from Algorithm B to be robust to slow convergence of the outcome model with sample size, whereas those from Algorithm A are not. The cost of this robustness, however, is that Algorithm B relies on $n^{1/4}$ rate convergence of the pseudo-outcome regression, i.e. the final regression step of the DR-learner.

Additionally, Algorithm 2 (with sample splitting) provides more credible confidence intervals than Algorithm 1 (without sample splitting), effectively by using out-of-sample predictions to control for overfitting. This is evident from the narrow confidence intervals provided by Algorithms 1A and 1B in Figure \[results\_plot\], which suggest that all covariates account for a substantial amount of treatment effect heterogeneity. As with the simulation study, we recommend Algorithm 2B for TE-VIM inference.

Related work and extensions {#sect:extend}
===========================

### Optimal treatment rule - VIMs {#optimal-treatment-rule---vims .unnumbered}

The proposed TE-VIMs complement VTEs and ATEs as an additional set of scalar summary estimands for the CATE. Fundamentally, these estimands summarise different aspects of the CATE function which are of scientific interest. Whilst TE-VIMs capture the importance of variable subsets in explaining the CATE, we remark that this should not be confused with the importance of those variables in contributing to optimal treatment decisions. For example, it is possible that a single covariate is important in explaining the magnitude of the CATE, but unimportant in determining the effect direction.

From the perspective of policy learning, one might instead be interested in the importance of variable subsets in explaining the optimal dynamic treatment rule (OTR), defined as $d(x)\equiv\mathbb{I}\{\tau(x)>0\}$, where $\mathbb{I}(.)$ denotes an indicator function and we assume w.l.o.g. that a more positive outcome is preferred. The current approach might be extended by considering an OTR-VIM estimand, $$\begin{aligned}
\Gamma_s \equiv {E}[{\mathrm{var}}\{d(X)|X_{-s}\}] = {E}[d_s(X)\{1-d_s(X)\}]\end{aligned}$$ where $d_s(x) \equiv {E}\{d(X)|X_{-s}=x_{-s}\}=Pr\{\tau(X) >0 |X_{-s}=x_{-s}\}$, and we note that $d(X)\in\{0,1\}$ and $d_s(X)\in[0,1]$. We argue that $\Gamma_s \in [0,0.25]$ is analogous to $\Theta_s$ and $\theta_s$, with the OTR $d(X)$ used in place of $\tau(X)$ and $\mu(X)$ respectively. Alternatively, @Williamson2021_general propose an OTR-VIM based on the estimand, $$\begin{aligned}
\Gamma^*_s  \equiv {E}\{\mu(d(X),X)\} - {E}\{\mu(d^*_s(X),X)\} &= {E}\left[\tau(X)\{d(X)-d^*_s(X)\}\right]\end{aligned}$$ where $d^*_s(x) \equiv\mathbb{I}\{\tau_s(x)>0\}$ is the optimal treatment rule given covariates $X_{-s}$, and $\Gamma^*_s \geq 0$. Unlike the TE-VIM, both $\Gamma_s$ and $\Gamma^*_s$ are not pathwise differentiable, which complicates inference. Estimators for e.g. $\Gamma^*_s$, however, are typically based on efficient estimators derived in the setting where the OTR is a known function.

### Treatment effect cumulative distribution function {#treatment-effect-cumulative-distribution-function .unnumbered}

Another related proposal considers the treatment effect cumulative distribution function (TE-CDF) [@Levy2018], which is a curve $\beta: \mathbb{R} \mapsto [0,1]$, with $$\begin{aligned}
\beta(t) = Pr\{\tau(X)\leq t\}.\end{aligned}$$ Motivated by optimal treatments, the value $\beta(0)$ is of particular interest since it captures the marginal probability that an individual has a negative CATE, and therefore the proportion of the population which is not treated under the OTR. We note that $\beta(0)$ is not the same as $Pr(Y^1 - Y^0 \leq 0)$ which suffers similar identifiability issues regarding the joint distribution of $(Y^1,Y^0)$ as the quantity ${\mathrm{var}}(Y^1-Y^0)$ mentioned previously. Like the OTR-VIMs above, the TE-CDF is generally not pathwise differentiable, hence @Levy2018 focus instead on a kernel smoothed analogue of $\beta(t)$. It is mentioned by @Levy2021 that, provided $\tau_p>0$, then Chebyshev's inequality implies $$\begin{aligned}
\beta(0) \leq \Lambda \equiv \frac{\Theta_p}{\tau_p^2}.\end{aligned}$$ Thus, the VTE is also of scientific interest since it can be used to bound $\beta(0)$, informing investigators about the marginal probability of negative CATEs once a positive ATE has been established. Estimation of $\Lambda$ could be carried out using estimating equations estimators, as in the current work, or targeted methods [@Levy2021], using the IC for $\Lambda$, $$\begin{aligned}
\frac{\{\varphi(Z)-\tau_p\}^2 - \{\varphi(Z)-\tau(X)\}^2 - \Lambda \tau_p\{2\varphi(Z)-\tau_p\} }{\tau_p^2}.\end{aligned}$$ Such estimators are beyond the scope of the current work, though we refer the interested reader to the Appendix for a sketch of the details for the former.

### Continuous treatments {#continuous-treatments .unnumbered}

The idea of treating the CATE as a statistical functional that we would like to summarise enables similar VIM estimands to be defined in settings where one is interested in other statistical functionals. For instance, @Hines2021b propose an analogue of the CATE $$\begin{aligned}
  \lambda(x) \equiv \frac{{\mathrm{cov}}(A,Y|X=x)}{{\mathrm{var}}(A|X=x)},\end{aligned}$$ which is well defined even when $A$ is a continuous exposure, and which identifies the CATE under standard causal assumptions (consistency, positivity, exchangeability) when $A$ is binary, i.e. $\lambda(x)=\mu(1,x)-\mu(0,x)$. One might, therefore, extend the ATE, VTE, and TE-VIMs to continuous exposures by defining the estimands: ${E}\{\lambda(X)\}$, ${\mathrm{var}}\{\lambda(X)\}$, and $$\begin{aligned}
&\frac{{E}[{\mathrm{var}}\{\lambda(X)|X_{-s}\}]}{{\mathrm{var}}\{\lambda(X)\}},\end{aligned}$$ which reduce to their CATE counterparts when $A$ is binary. The ICs for these estimands are obtained by replacing the pseudo-outcome $\varphi(z)$ with $$\begin{aligned}
\left[y-\mu(x)-\lambda(x)\{a-\pi(x)\} \right] \frac{a-\pi(x)}{{\mathrm{var}}(A|X=x)} + \lambda(x)\end{aligned}$$ which reduces to $\varphi(z)$ when $A$ is binary. See Appendix for details.

Conclusion
==========

We propose TE-VIMs, which extend the VIM framework of @Williamson2021_general to include the CATE as a functional of interest. These have immediate applications to the analysis of observational and clinical trial data, and provide insight into scientific questions related to treatment effect heterogeneity. Our methods complement VTE analyses, which quantify treatment effect heterogeneity [@Levy2021]. We derive efficient estimating equation estimators which are amenable to data-adaptive estimation of working models. These are broadly applicable, since they are not tied to a particular model or regression algorithm, unlike existing proposals based on causal random forests [@Athey2019].

We elucidate links between our estimators and regression-VIM counterparts, by interpreting our estimators in terms of pseudo-outcomes [@Kennedy2020]. We believe pseudo-outcome based approaches might generalise to other statistical functionals, where analogous pseudo-outcomes could be derived. For instance, 'derivative effect VIMs' could be derived which are well defined for continuous exposures and which identify the proposed TE-VIMs when the exposure is binary [@Hines2021b], or VIMs for policy learning could be developed using double robust scores [@Athey2021].

We recommend that TE-VIM inference be incorporated into a more broad treatment effect analysis, where primary interest is in inferring the ATE and VTE. We believe that VTE inference should form part of a primary analysis, since the ATE and VTE may be used to bound the marginal probability of adverse CATEs, and since it is possible that the population ATE is zero, but some (or indeed all) individuals experience a large CATE. One may then infer TE-VIMs as part of a secondary analysis, when large treatment effect heterogeneity cannot be ruled out, since TE-VIM estimands are not of scientific interest when there is little variability in the CATE to account for.

Derivation of Efficient Influence Curve {#ic_deriv_append}
=======================================

To derive the ICs in and we adopt the formalism given in @Hines2021. Specifically we let $P$ denote the true distribution of $(Y,A,X)$ and let $\tilde{P}$ denote a point mass at $(\tilde{y},\tilde{a},\tilde{x})$. We further denote the parametric submodel $P_t = t\tilde{P} + (1-t)P$ where $t\in[0,1]$ is a scalar parameter, and we let $\partial_t$ denote an operator such that for some function of $f(t)$, $\partial_t f(t)\equiv \frac{d f(t)}{dt}|_{t=0}$.

We make use of the following lemma, which we demonstrate later in the proof. Letting $g_P(X)$ denote some functional of $P$, then $$\begin{aligned}
\partial_t {E}_{P_t}\{g_{P_t}(X)|X_{-s}=x_{-s}\} &= \frac{\tilde{f}(x_{-s})}{f(x_{-s})} \left[ g_{P}(\tilde{x}) - {E}_{P}\{g_{P}(X)|X_{-s}=x_{-s}\}\right] + {E}_{P}\{\partial_t g_{P_t}(X)|X_{-s}=x_{-s}\} \label{quick_lemma}\end{aligned}$$ where $\tilde{f}(.)$ and $f(.)$ denote the marginal 'densities' of $X_{-s}$ under $\tilde{P}$ and $P$ respectively, which are both assumed to be absolutely continuous w.r.t. to a dominating measure. In practice this expression means that for discrete $X_{-s}$ then $f(.)$ is a probability mass function and $\tilde{f}(.)$ is an indicator function. Similarly for continuous $X_{-s}$ then $f(.)$ is a probability density function and $\tilde{f}(.)$ is a dirac delta function. In both cases $\tilde{f}(x_{-s})$ is a probability point mass, which is zero when $\tilde{x}_{-s} \neq x_{-s}$.

It follows immediately from that, $$\begin{aligned}
\partial_t {E}_{P_t}\{g_{P_t}(X)\} &= g_{P}(\tilde{x}) - {E}_{P}\{g_{P}(X)\} + {E}_{P}\{\partial_t g_{P_t}(X)\} \label{quick_lemma2}\end{aligned}$$ By considering that $$\begin{aligned}
{\mathrm{var}}_{P}\{g_{P}(X)|X_{-s}=x_{-s}\} &= {E}_P\{g_{P}^2(X)|X_{-s}=x_{-s}\} - {E}_P\{g_{P}(X)|X_{-s}=x_{-s}\}^2\end{aligned}$$ We can use to show that that $$\begin{aligned}
\partial_t {\mathrm{var}}_{P_t}\{g_{P_t}(X)|X_{-s}=x_{-s}\} &= \frac{\tilde{f}(x_{-s})}{f(x_{-s})} \left[ \{g_{P}(\tilde{x}) - {E}_P(g_P(X)|X_{-s}=x_{-s})\}^2 - {\mathrm{var}}_{P}\{g_{P}(X)|X_{-s}=x_{-s}\} \right] \label{IC:var} \\
&+ 2 {\mathrm{cov}}_P\{g_{P}(X),\partial_t g_{P_t}(X)|X_{-s}=x_{-s} \} \nonumber\end{aligned}$$ where ${\mathrm{cov}}(A,B|C)\equiv{E}(\{A-{E}(A|C)\}B|C)$ denotes the conditional covariance. Using the results in and , we obtain $$\begin{aligned}
\partial_t {E}_{P_t}\left[{\mathrm{var}}_{P_t}\{g_{P_t}(X)|X_{-s}\}\right] &=  \{g_{P}(\tilde{x}) - {E}_P(g_P(X)|X_{-s}=\tilde{x}_{-s})\}^2 - {E}_{P}\left[{\mathrm{var}}_{P}\{g_{P}(X)|X_{-s}\}\right]  \nonumber \\ &+ 2 {E}_P\{{\mathrm{cov}}_P\{g_{P}(X),\partial_t g_{P_t}(X)|X_{-s}\}\} \label{IC_general}\end{aligned}$$ Setting $g_P(X)=\tau(X)$, we use and the fact that $\tau(x)=\mu(1,x)-\mu(0,x)$ to show that, $$\begin{aligned}
\partial_t g_{P_t}(x) &= \frac{\tilde{f}(x)}{f(x)}\{\tilde{y}-\mu(\tilde{a},x)\}\frac{\tilde{a} - \pi(x)}{\pi(x)\{1-\pi(x)\}}\end{aligned}$$ Hence, implies the IC, $$\begin{aligned}
\phi_s(\tilde{z})&=    \{\tau(\tilde{x})-\tau_s(\tilde{x})\}^2 - \Theta_{s}  + 2\{\tau(\tilde{x})-\tau_s(\tilde{x})\}\{\tilde{y}-\mu(\tilde{a},\tilde{x})\}\frac{\tilde{a} - \pi(\tilde{x})}{\pi(\tilde{x})\{1-\pi(\tilde{x})\}}  \\
&=    \{\tau(\tilde{x})-\tau_s(\tilde{x})\}^2 - \Theta_{s}  + 2\{\tau(\tilde{x})-\tau_s(\tilde{x})\}\{\varphi(\tilde{z}) - \tau(\tilde{x})\}\end{aligned}$$ Completing the square of the expression above gives the result in . In replicating this proof, it is useful to note that for an arbitrary function $h(x)$ $$\begin{aligned}
{E}_P\left\{ \frac{\tilde{f}(X)}{f(X)} h(X) \right\} &= h(\tilde{x})\end{aligned}$$

Proof of Lemma in  {#proof-of-lemma-in .unnumbered}
------------------

To demonstrate we write the lefthand side as $$\begin{aligned}
\partial_t \int g_{P_t}(x^*) dP_{t,X_s|x_{-s}}(x^*_s) &= \int g_{P}(x^*) \partial_t dP_{t,X_s|x_{-s}}(x^*_s) + \int \{\partial_t g_{P_t}(x^*)\} dP_{X_s|x_{-s}}(x^*_s)\end{aligned}$$ where $dP_{t,X_s|x_{-s}}(.)$ is the conditional distribution of $X_s$ given $X_{-s}=x_{-s}$ under the parametric submodel and $x^*_{-s} = x_{-s}$. The second integral on the righthand side recovers the final term in . Hence the lemma follows once we show that $$\begin{aligned}
 \partial_t dP_{t,X_s|x_{-s}}(x^*_s) = \frac{\tilde{f}(x_{-s})}{f(x_{-s})} \{ d\tilde{P}_{X_s}(x^*_s) - dP_{X_s|x_{-s}}(x^*_s)\}\end{aligned}$$ To do so, let $\mu$ denote a dominating measure and write $$\begin{aligned}
dP_{t,X_s|x_{-s}}(x^*_s) &= f_{t,X_s|x_{-s}}(x^*_s) d\mu(x^*_s) \\
&= \frac{f_{t,X}(x^*)}{f_{t,X_{-s}}(x_{-s})} d\mu(x^*_s)\end{aligned}$$ where $f_{t,X}(.)$ and $f_{t,X_{-s}}(.)$ denote the marginal densities of $X$ and $X_{-s}$ under the parametric submodel, $P_t$, i.e. they are the Radon-Nikodym derivatives w.r.t. $\mu$. Applying the quotient rule, we obtain $$\begin{aligned}
\partial_t dP_{t,X_s|x_{-s}}(x^*_s) &= \frac{1}{f_{X_{-s}}(x_{-s})} \left[ \partial_t f_{t,X}(x^*) -
\frac{f_{X}(x^*)}{f_{X_{-s}}(x_{-s})}   \partial_t f_{t,X_{-s}}(x_{-s}) \right]d\mu(x^*_s)\end{aligned}$$ We now evaluate the derivative parts. Since $\partial_t P_t = \tilde{P}-P$, the marginal density derivatives will have a similar structure, as shown in the first expression below, where $f_X(.)$ and $\tilde{f}_X(.)$ denote marginal densities of $X$ under $\tilde{P}$ and $P$, with likewise for $X_{-s}$ $$\begin{aligned}
\partial_t dP_{t,X_s|x_{-s}}(x^*) &=\frac{1}{f_{X_{-s}}(x_{-s})} \left[ \{\tilde{f}_{X}(x^*)-f_{X}(x^*)\} -
  \frac{f_{X}(x^*)}{f_{X_{-s}}(x_{-s})}   \{\tilde{f}_{X_{-s}}(x_{-s})-f_{X_{-s}}(x_{-s})\} \right]d\mu(x^*_s)\end{aligned}$$ Since $\tilde{P}$ is a point mass, $\tilde{f}_X(x^*) = \tilde{f}_{X_s}(x^*_s)\tilde{f}_{X_{-s}}(x^*_{-s})$. Also $x^*_{-s} = x_{-s}$ hence, $$\begin{aligned}
\partial_t dP_{t,X_s|x_{-s}}(x^*) &=\frac{\tilde{f}_{X_{-s}}(x_{-s})}{f_{X_{-s}}(x_{-s})} \left[ \tilde{f}_{X_s}(x^*_s)  -
  \frac{f_{X}(x^*) }{f_{X_{-s}}(x_{-s})}   \right]d\mu(x^*_s) \\
  &= \frac{\tilde{f}_{X_{-s}}(x_{-s})}{f_{X_{-s}}(x_{-s})} \left[ \tilde{f}_{X_s}(x^*_s) -
    f_{X_s|x_{-s}}(x^*_s)   \right]d\mu(x^*_s)\end{aligned}$$ Thus, the result follows.

Corollary {#corollary .unnumbered}
---------

An immediate consequence of these IC derivations is that the IC of ${\mathrm{var}}\{\tau_s(X)\}=\Theta_p-\Theta_s$ is, $$\begin{aligned}
\phi_p(Z) - \phi_s(Z) =  \{\varphi(Z)-\tau_p\}^2 - \{\varphi(Z)-\tau_s(X)\}^2 - {\mathrm{var}}\{\tau_s(X)\}\end{aligned}$$ This result is interesting since it holds even when $Y$ is not independent of $A$ given $X_{-s}$.

Estimator Asymptotic Distributions {#Appen_assymp}
==================================

Proof of Theorem \[asym\_theorem\]
----------------------------------

We demonstrate asymptotic regularity for the estimator $\hat{\Theta}_s$, with the result for $\hat{\Theta}_p$ following from the case $s=p$. Asymptotic regularity of $\Psi_s$ follows using the ratio argument above.

Throughout we use superscript hat to denote functional estimators obtained from an independent sample, and we define, $$\begin{aligned}
\hat{\varphi}(z) &=\{y-\hat{\mu}(a,x)\}\frac{a-\hat{\pi}(x)}{\hat{\pi}(x)\{1-\hat{\pi}(x)\}} + \hat{\mu}(1,x) - \hat{\mu}(0,x) \\
\hat{\phi}_s(z) &= \{\hat{\varphi}(z)-\hat{\tau}_s(x)\}^2 - \{\hat{\varphi}(z)-\hat{\tau}(x)\}^2  - \hat{\Theta}_s^0\end{aligned}$$ where $\hat{\Theta}_s^0$ is an initial plug-in estimate of $\Theta_s$. We make the following assumptions about these functional estimators,

1.  The propensity score and outcome estimators are 'double robust' in the sense that $\{\pi(x)-\hat{\pi}(x)\}\{\mu(a,x)-\hat{\mu}(a,x)\}$ is $o_P(n^{-1/2})$ in $L_2(P)$ norm for $a=0,1$.

2.  The differences $\tau(x)-\hat{\tau}(x)$ and $\tau_s(x)-\hat{\tau}_s(x)$ are both $o_P(n^{-1/4})$ in $L_2(P)$ norm.

3.  The CATE difference estimates are bounded as $\{\hat{\tau}(x)-\hat{\tau}_s(x)\}^2 \leq \delta $ for some $\delta<\infty$ with probability 1.

4.  The propensity score estimates are bounded as $\epsilon\leq \hat{\pi}(x) \leq 1-\epsilon$ for some $\epsilon>0$ with probability 1.

5.  There exists a $P$-Donsker class $\mathcal{G}_0$ such that $P(\hat{\phi}_s(.) \in \mathcal{G}_0)\to 1$.

6.  There exists a constant $K>0$ such that each of $\tau(x)$, $\hat{\tau}(x)$, $\hat{\tau}_s(x)$ and ${\mathrm{var}}(\varphi(Z)|X=x)$ has range uniformly contained in $(-K,K)$ with probability one as $n\to\infty$.

7.  There exists a constant $K>0$ such that ${\mathrm{var}}(Y|X=x)$ and $\hat{\mu}(a,x)$ have range uniformly contained in $(-K,K)$ with probability one as $n\to\infty$.

Under these assumptions we show that the remainder term, $R$, in the expansion below is $o_P(n^{-1/2})$ $$\begin{aligned}
\hat{\Theta}_s^0 - \Theta_s &= -{E}\{\hat{\phi}_s(Z)\} + R\end{aligned}$$ where we highlight that the expectation is conditional on the functional estimators, i.e. $\hat{\varphi}(z)$ is treated as a fixed function. We then show that $$\begin{aligned}
-{E}\{\hat{\phi}_s(Z)\} = n^{-1} \sum_{i=1}^n \phi_s(z_i) + H_n - n^{-1} \sum_{i=1}^n \hat{\phi}_s(z_i) \label{empirical_2}\end{aligned}$$ where $H_n$ is an empirical process term, which is $o_P(n^{-1/2})$ under our assumptions. It follows therefore that for $$\begin{aligned}
\hat{\Theta}_s &= \hat{\Theta}_s^0 + n^{-1} \sum_{i=1}^n \hat{\phi}_s(z_i) \\
\hat{\Theta}_s - \Theta_s &= n^{-1} \sum_{i=1}^n \phi_s(z_i) + o_P(n^{-1/2}).\end{aligned}$$ Formally $\hat{\Theta}_s$ is one-step plug-in bias correction estimator, but it is seen to be equivalent to the estimating equations estimator in the main text.

The remainder term {#the-remainder-term .unnumbered}
------------------

To simplify notation, in this subsection we largely omit function arguments, for example $\tau = \tau(X)$ with similar for $\hat{\tau},\tau_s,\hat{\tau}_s,\pi,\hat{\pi},\hat{\varphi}$. Evaluating the remainder $R \equiv {E}\{\hat{\phi}_s(Z) + \hat{\Theta}_s^0  - \Theta_s \} $ gives $$\begin{aligned}
R &= {E}\left[ \{\hat{\varphi}-\hat{\tau}_s\}^2 - \{\hat{\varphi}-\hat{\tau}\}^2 - \{\tau-\tau_s\}^2  \right]\end{aligned}$$ where we have used the fact that $\Theta_s = {E}[\{\tau-\tau_s\}^2]$. By algebraic manipulation, we write $$\begin{aligned}
R &= {E}\left[ \{\hat{\tau}-\hat{\tau}_s\}^2 - \{\tau-\tau_s\}^2  + 2 \{\hat{\tau}-\hat{\tau}_s\}\{\hat{\varphi}-\hat{\tau}\} \right]\end{aligned}$$ We then use the identity, $$\begin{aligned}
{E}\left[ \{\hat{\tau}-\hat{\tau}_s\}^2 - \{\tau-\tau_s\}^2\right] &= {E}\left[ \{\tau_s-\hat{\tau}_s\}^2-\{\tau-\hat{\tau}\}^2 + 2\{\hat{\tau}-\hat{\tau}_s\}\{\hat{\tau}-\tau\} \right]\end{aligned}$$ to rewrite the remainder term as the sum of two error terms, $$\begin{aligned}
R &= {E}\left[ \{\hat{\tau}-\hat{\tau}_s\}^2 - \{\tau-\tau_s\}^2  + 2 \{\hat{\tau}-\hat{\tau}_s\}\{\hat{\varphi}-\hat{\tau}\} \right] \\
&= {E}\left[ \{\tau_s-\hat{\tau}_s\}^2-\{\tau-\hat{\tau}\}^2  + 2 \{\hat{\tau}-\hat{\tau}_s\}\{\hat{\tau} - \tau\} + 2 \{\hat{\tau}-\hat{\tau}_s\}\{\hat{\varphi}-\hat{\tau}\}  \right] \\
  &= {E}\left[ \{\tau_s-\hat{\tau}_s\}^2-\{\tau-\hat{\tau}\}^2 + 2 \{\hat{\tau}-\hat{\tau}_s\}\{\hat{\varphi} -\tau\}\right] \\
  &= \underbrace{{E}\left[ \{\tau_s-\hat{\tau}_s\}^2-\{\tau-\hat{\tau}\}^2\right]}_{\text{CATE error}} + \underbrace{2{E}\left[\{\hat{\tau}-\hat{\tau}_s\} r \right]}_{\text{Pseudo-outcome error}}\end{aligned}$$ where $r=r(X)$ is defined by $$\begin{aligned}
r(x) &\equiv {E}\left[ \hat{\varphi} | X=x  \right] - \tau(x)\end{aligned}$$ This represents a pseudo-outcome error in the sense that $r(x)={E}\left[ \hat{\varphi} -\varphi| X=x  \right]$. Splitting the remainder in to two error terms allows us to consider that the CATE error is $o_P(n^{-1/2})$ when (A2) holds. For the pseudo-outcome error we use the Cauchy-Schwarz inequality to show that $$\begin{aligned}
{E}\left[\{\hat{\tau}-\hat{\tau}_s\} r \right]^2 &\leq {E}\left[\{\hat{\tau}-\hat{\tau}_s\}^2 \right] {E}\left[r^2 \right] \leq \delta {E}\left[r^2 \right]\end{aligned}$$ with the second inequality following from (A3). Hence the pseudo-outcome error term is $o_P(n^{-1/2})$ if $r$ is $o_P(n^{-1/2})$. By iterated expectation $$\begin{aligned}
r(x) &= \left\{\frac{\pi(x)}{\hat{\pi}(x)} - 1 \right\}\left\{\mu(1,x) - \hat{\mu}(1,x) \right\} - \left\{\frac{1-\pi(x)}{1-\hat{\pi}(x)} - 1 \right\}\left\{\mu(0,x) - \hat{\mu}(0,x) \right\}\end{aligned}$$ Using the inequality $(a+b)^2 \leq 2(a^2 + b^2)$ then $$\begin{aligned}
r^2(x) &\leq 2\left\{\frac{\pi(x)}{\hat{\pi}(x)} - 1 \right\}^2\left\{\mu(1,x) - \hat{\mu}(1,x) \right\}^2 +2 \left\{\frac{1-\pi(x)}{1-\hat{\pi}(x)} - 1 \right\}^2\left\{\mu(0,x) - \hat{\mu}(0,x) \right\}^2 \\
&\leq \left(\frac{2}{\epsilon^2}\right) \left\{ \pi(x) - \hat{\pi}(x) \right\}^2 \left[ \left\{\mu(1,x) - \hat{\mu}(1,x) \right\}^2 + \left\{\mu(0,x) - \hat{\mu}(0,x) \right\}^2\right]\end{aligned}$$ with the second inequality following from (A4). The final expression above is $o_P(n^{-1})$ under (A1), which completes the proof that $R$ itself is $o_P(n^{-1/2})$.

The empirical process term {#the-empirical-process-term .unnumbered}
--------------------------

In this subsection we use a common empirical processes notation, where we define linear operators $P$ and $P_n$ such that for some function $h(Z)$, $P\{ h(Z)\} \equiv {E}\{h(Z)\}$ and $P_n \{h(Z)\} \equiv n^{-1} \sum_{i=1}^n h(z_i)$. Hence we write $-{E}\{\hat{\phi}_s(Z)\}$ as $$\begin{aligned}
(P_n - P)\{\phi_s(Z)\} +(P_n-P)\{\hat{\phi}_s(Z) - \phi_s(Z)\}- P_n \{\hat{\phi}_s(Z)\}\end{aligned}$$ which follows from adding and subtracting $(P_n-P)\{\phi_s(Z)\}$ and $P_n \{\hat{\phi}_s(Z)\}$ to $-P\{\hat{\phi}_s(Z)\}$. This expression recovers since the IC is mean zero, in the sense that $P\{\phi_s(Z)\} = 0$, and we define the empirical process term $$\begin{aligned}
H_n \equiv (P_n-P)\{\hat{\phi}_s(Z) - \phi_s(Z)\}\end{aligned}$$ By e.g. Lemma 19.24 of @Vaart1998, $H_n$ is $o_P(n^{-1/2})$ under (A5) provided that $P\left[\left\{\hat{\phi}_s(Z) - \phi_s(Z)\right\}^2\right]$ converges to zero in probability.

Start by writing, $$\begin{aligned}
\hat{\phi}_s - \phi_s =& 2(\hat{\varphi} - \varphi)(\hat{\tau}-\hat{\tau}_s) \\
&+ 2(\varphi-\tau_s)(\tau_s-\hat{\tau}_s) + (\tau_s-\hat{\tau}_s)^2\\
&- 2(\varphi-\tau)(\tau-\hat{\tau}) - (\tau-\hat{\tau})^2 \\
&- (\hat{\Theta}_s^0-\Theta_s)\end{aligned}$$ Using the inequality $(a+b)^2 \leq 2(a^2+b^2)$, $$\begin{aligned}
P\left[\left\{\hat{\phi}_s(Z) - \phi_s(Z)\right\}^2\right] \leq &\quad  8P\left\{(\hat{\varphi} - \varphi)^2(\hat{\tau}-\hat{\tau}_s)^2 \right]\\
&+ 2P\Big[\Big\{
 2(\varphi-\tau_s)(\tau_s-\hat{\tau}_s) + (\tau_s-\hat{\tau}_s)^2\\
&\quad\quad\quad - 2(\varphi-\tau)(\tau-\hat{\tau}) - (\tau-\hat{\tau})^2 \\
&\quad\quad\quad - (\hat{\Theta}_s^0-\Theta_s)\Big\}^2\Big]\end{aligned}$$ Letting $\hat{\Theta}_s^0 = P_n\{(\hat{\tau}-\hat{\tau}_s)^2\}$ then, in view of Theorem 1 of @Williamson2021, the second term converges to zero under (A2) and (A6). For the first of terms, we note that (A3) implies $$\begin{aligned}
P\left\{(\hat{\varphi} - \varphi)^2(\hat{\tau}-\hat{\tau}_s)^2 \right] \leq \delta P\left\{(\hat{\varphi} - \varphi)^2\right].\end{aligned}$$

Similar terms to $P\left\{(\hat{\varphi} - \varphi)^2\right]$ appear in the ATE empirical process literature. In view of Theorem 5.1 of @Chernozhukov2018, this term is also converges to zero under (A1), (A4) and (A7).

Thus $H_n = o_P(n^{-1/2})$ which completes the proof.

Proof of Theorem \[asym\_theorem2\]
-----------------------------------

Under (A1)-(A7) and

1.  The difference $\tau_p-\hat{\tau}_p$ is $o_P(n^{-1/4})$.

2.  The CATE difference estimates are bounded as $\{\hat{\tau}(x)-\hat{\tau}_p\}^2 \leq \delta $ for some $\delta<\infty$ with probability 1.

3.  There exists a $P$-Donsker class $\mathcal{G}_0$ such that $P(\hat{\phi}_p(.) \in \mathcal{G}_0)\to 1$.

4.  There exists a constant $K>0$ such that $\hat{\tau}_p \in (-K,K)$.

Then we have regular asymptotically linear estimators such that, $$\begin{aligned}
\hat{\Theta}_s - \Theta_s &= n^{-1}\sum_{i=1}^n \phi_s(z_i) + o_p(n^{-1/2}) \\
\hat{\Theta}_p - \Theta_p &= n^{-1}\sum_{i=1}^n \phi_p(z_i) + o_p(n^{-1/2})\end{aligned}$$ It follows by algebraic manipulations that, $$\begin{aligned}
\sqrt{n}(\hat{\Psi}_s - \Psi_s) &= \frac{\Theta_p}{\hat{\Theta}_p} \left[ n^{-1/2}\sum_{i=1}^n \Phi_s(z_i) + o_p(1) \right]\end{aligned}$$ where $\Phi_s(z) = \{\phi_s(z) - \Psi_s\phi_p(z)\}/\Theta_p$ is the IC of $\Psi_s$. Next we use Slutsky's Theorem and the fact that $\hat{\Theta}_p/\Theta_p$ converges to 1 in probability, to write, $$\begin{aligned}
\lim_{n\to \infty} \sqrt{n}(\hat{\Psi}_s - \Psi_s) &= \lim_{n\to \infty} n^{-1/2}\sum_{i=1}^n \Phi_s(z_i)\end{aligned}$$ which gives the desired result due to the central limit theorem. We note that this set up is quite general when one considers estimands which are written as the ratio of two other estimands, such as $\Psi_s$ in the present context.

Additional Plots
================

Simulation plots
----------------

Figure \[sim\_results\_plot2\] shows the performance of the estimator $\hat{\Psi}_2$, of the TE-VIM $\Psi_2$, as described in the simulation study in Section \[sect:simulation\].

![Bias, variance and coverage for $\hat{\Psi}_2$ using 1000 sampled datasets. Red and blue points indicate that working models are fitted using generalised additive modelling and random forests respectively. Top row of plots corresponds to Algorithm 1 (no sample splitting) and the bottom row corresponds to Algorithm 2 (sample splitting). Square and crossed points indicate that the algorithm used the T-learner and DR-learner respectively for CATE estimation. <span data-label="sim_results_plot2"></span>](sim_plotX2){width="0.8\linewidth"}

Applied example
---------------

Figure \[results\_plot2\] gives TE-VIM estimates from the ACTG175 where the discrete Super Learner (20 cross validation folds) is used for functional estimation. The plots for Algorithms 1A and 2A, i.e. the T-learner without and with sample splitting, appear highly uninformative. For Algorithm 1A this is due to the point estimate of the VTE being very close to zero -3.80$\times 10^{-9} mm^{-6}$ (CI: -1.7$\times 10^{-3}$,1.7$\times 10^{-3}$). For Algorithm 2A, the VTE estimate is also negative -107 $mm^{-6}$ (CI:-159,-57), and does not overlap with zero. Since both point estimates are negative, the null hypothesis that the VTE$\leq 0$ has p-value exactly equal to 1.

For Algorithm 2A, all of the TE-VIM point estimates are negative, meaning that all of the corresponding $\Theta_s$ estimates are positive. The negative VTE, therefore has the effect of reversing the order of importance, with CD4 count at baseline appearing at the bottom of the plot for Algorithm 2A in Figure \[results\_plot2\].

![TE-VIM estimates from the ACTG175 study using the discrete Super Learner for functional estimation. Top row: no sample splitting (Algorithm 1). Bottom row: with sample splitting (Algorithm 2). Left col: T-learner (A). Right col: DR-learner (B). Black lines indicate 95% Confidence intervals. Confidence intervals in the bottom left plot are so small that they are not visible. In each plot, covariates are sorted according to their TE-VIM point estimate. Dashed lines indicate the $[0,1]$ support of the TE-VIM.<span data-label="results_plot2"></span>](plot_full_discrete){width="0.75\linewidth"}

TE-CDF bounds
=============

First note Chebyshev's inequality: For a variable $V$ with mean $\mu$ and variance $\sigma^2$, for $k>0$ $$\begin{aligned}
Pr (|V-\mu|\geq k\sigma) &\leq k^{-2} \\
Pr (V \geq \mu + k\sigma) + Pr (V \leq \mu - k\sigma) &\leq k^{-2}\end{aligned}$$ which implies the weaker inequality, $$\begin{aligned}
Pr (V \leq \mu - k\sigma) &\leq k^{-2}\end{aligned}$$ Let $\tau(X)$ be the CATE with ATE $\tau_p$ and VTE $\Theta_p$ then, $$\begin{aligned}
\beta(0) = Pr \left\{\tau(X) \leq 0\right\} &= Pr \left\{\tau(X) \leq \tau_p - \left(\frac{\tau_p}{\sqrt{\Theta_p}}\right)\sqrt{\Theta_p} \right\} \leq \frac{\Theta_p}{\tau^2_p}\end{aligned}$$ Where the inequality applies only when $\tau_p>0$. It follows that, when the ATE is positive, the quantity on the RHS bounds $\beta(0)$ from above. The quotient rule gives that the IC (pathwise derivative) is, $$\begin{aligned}
\phi_\beta(Z) &= \frac{1}{\tau_p^2}\phi_p(Z) -2\left(\frac{\Theta_p}{\tau^3_p}\right)\{\varphi(Z)-\tau_p\} \\
&= \frac{\{\varphi(Z)-\tau_p\}^2 - \{\varphi(Z)-\tau(X)\}^2 - \left(\frac{\Theta_p}{\tau^2_p}\right) \tau_p\{2\varphi(Z)-\tau_p\} }{\tau_p^2}\end{aligned}$$ where $\{\varphi(Z)-\tau_p\}$ is the IC of $\tau_p$. An estimating equations estimator is that which solves $$\begin{aligned}
n^{-1}\sum_{i=1}^n \hat{\phi}_\beta(z_i) &= 0\end{aligned}$$ where $\hat{\phi}_\beta(z)$ is an estimate of $\phi_\beta(z)$. Therefore $\hat{\Theta}_p/\hat{\tau}_p^2$ is an estimating equations estimator where $\hat{\Theta}_p$ is the VTE estimator in the current paper and $$\begin{aligned}
\hat{\tau}_p &= n^{-1}\sum_{i=1}^n \hat{\varphi}(z_i)\end{aligned}$$ is the AIPW estimator of the ATE.

IC for continuous analogue estimands
====================================

Here we use the same fomalism as in Appendix \[ic\_deriv\_append\]. To derive the ICs of interest we consider the results in and in the setting where we set $g_{P}(x)=\lambda(x)$. We will show that, $$\begin{aligned}
\partial_t g_{P_t}(x) &= \frac{\tilde{f}(x)}{f(x)}\{\tilde{y}-\mu(x) - \lambda(x)\{\tilde{a}-\pi(x)\}\}\frac{\tilde{a} - \pi(x)}{{\mathrm{var}}(A|X=x)} \label{pseudo_est2}\end{aligned}$$ and hence, letting $$\begin{aligned}
\varphi_\lambda(z) \equiv \{y-\mu(x) - \lambda(x)\{a-\pi(x)\}\}\frac{a - \pi(x)}{{\mathrm{var}}(A|X=x)} + \lambda(x)\end{aligned}$$ then by the IC of ${E}\{\lambda(X)\}$ is, $$\begin{aligned}
\varphi_\lambda(z) - {E}\{\lambda(X)\}\end{aligned}$$ and by , the IC of ${E}\left[{\mathrm{var}}\{\lambda(X)|X_{-s}\}\right]$ is, $$\begin{aligned}
\{\varphi_\lambda(z) - \lambda_s(x)\} - \{\varphi_\lambda(z) - \lambda(x)\} - {E}\left[{\mathrm{var}}\{\lambda(X)|X_{-s}\}\right]\end{aligned}$$ where $\lambda_s(x)={E}\{\lambda(X)|X_{-s}=x_{-s}\}$. The IC for ${\mathrm{var}}\{\lambda(X)\}$ follows as a special case where $s$ includes all the observed covariates. To demonstrate we first note that, by , $$\begin{aligned}
\partial_t {\mathrm{cov}}_{P_t}(A,Y|X=x) &= \partial_t {E}_{P_t}\{[A-{E}_{P_t}(A|X)][Y-{E}_{P_t}(Y|X)]|X=x\} \\
&=  \frac{\tilde{f}(x)}{f(x)} \left[\{\tilde{a}-\pi(x)\}\{\tilde{y}-\mu(x)\} - {\mathrm{cov}}_{P}(A,Y|X=x) \right]\end{aligned}$$ We also obtain $\partial_t {\mathrm{var}}_{P_t}(A|X=x)$ as a special case of the above expression when $Y=A$. By the quotient rule, $$\begin{aligned}
\partial_t \frac{{\mathrm{cov}}_{P_t}(A,Y|X=x)}{{\mathrm{var}}_{P_t}(A|X=x)} &= \frac{\partial_t {\mathrm{cov}}_{P_t}(A,Y|X=x)}{{\mathrm{var}}_{P}(A|X=x)} -  \frac{{\mathrm{cov}}_{P}(A,Y|X=x)}{{\mathrm{var}}_{P}(A|X=x)} \frac{\partial_t {\mathrm{var}}_{P_t}(A,Y|X=x)}{{\mathrm{var}}_{P}(A|X=x)}  \\
&= \frac{\tilde{f}(x)}{f(x)}\{\tilde{y}-\mu(x) - \lambda(x)\{\tilde{a}-\pi(x)\}\}\frac{\tilde{a} - \pi(x)}{{\mathrm{var}}(A|X=x)}\end{aligned}$$ Thus, the desired results follow.

{'source': 'arXiv_out/2204'}