E STIMATING INDIVIDUAL TREATMENT EFFECTS UNDER UNOBSERVED CONFOUNDING USING BINARY INSTRUMENTS
A BSTRACT
Estimating conditional average treatment effects (CATEs) from observational data is relevant in many fields such as personalized medicine.
However, in practice, the treatment assignment is usually confounded by unobserved variables and thus introduces bias.
A remedy to remove the bias is the use of instrumental variables (IVs).
Such settings are widespread in medicine (e.g., trials where the treatment assignment is used as binary IV).
In this paper, we propose a novel, multiply robust machine learning framework, called MRIV, for estimating CATEs using binary IVs and thus yield an unbiased CATE estimator.
Different from previous work for binary
IVs, our framework estimates the CATE directly via a pseudo-outcome regression.
(1) We provide a theoretical analysis where we show that our framework yields multiple robust convergence rates: our CATE estimator achieves fast convergence even if several nuisance estimators converge slowly.
(2) We further show that our framework asymptotically outperforms state-of-the-art plug-in IV methods for
CATE estimation, in the sense that it achieves a faster rate of convergence if the
CATE is smoother than the individual outcome surfaces.
(3) We build upon our theoretical results and propose a tailored deep neural network architecture called
MRIV-Net for CATE estimation using binary IVs.
Across various computational experiments, we demonstrate empirically that our MRIV-Net achieves state-of-theart performance.
To the best of our knowledge, our MRIV is the first multiply robust machine learning framework tailored to estimating CATEs in the binary IV setting.
I NTRODUCTION
Conditional average treatment effects (CATEs) are relevant across many disciplines such as marketing (Varian, 2016) and personalized medicine (Yazdani & Boerwinkle, 2015).
Knowledge about CATEs provides insights into the heterogeneity of treatment effects and thus helps in making potentially better treatment decisions (Frauen et al., 2023).
Many recent works that use machine learning to estimate causal effects, in particular CATEs, are based on the assumption of unconfoundedness (Alaa & van der Schaar, 2017; Lim et al., 2018; Melnychuk et al., 2022a;b).
In practice, however, this assumption is often violated because it is common that some confounders are not reported in the data.
Typical examples are income or the socioeconomic status of patients, which are not stored in medical files.
If the confounding is sufficiently strong, standard methods for estimating CATEs suffer from confounding bias (Pearl, 2009), which may lead to inferior treatment decisions.
To handle unobserved confounders, instrumental variables (IVs) can be leveraged to relax the assumption of unconfoundedness and still compute reliable CATE estimates.
IV methods were originally developed in economics (Wright, 1928), but, only recently, there is a growing interest in combining IV methods with machine learning (see Sec. 3).
Importantly, IV methods outperform classical CATE estimators if a sufficient amount of confounding is not observed (Hartford et al., 2017).
We thus aim at estimating CATEs from observational data under unobserved confounding using IVs.
In this paper, we consider the setting where a single binary instrument is available.
This setting is widespread in personalized medicine (and other applications such as marketing or public policy) (Bloom et al., 1997).
In fact, the setting is encountered in essentially all observational or randomized studies with observed non-compliance (Imbens & Angrist, 1994).
As an example, consider a randomized controlled trial (RCT), where treatments are randomly assigned to patients and their outcomes are observed.
Due to some potentially unobserved confounders (e.g., income, education), some patients refuse to take the treatment initially assigned to them.
Here, the treatment assignment serves as a binary IV.
Moreover, such RCTs have been widely used by public decision-makers, e.g., to analyze the effect of health insurance on health outcome (see the so-called Oregon health insurance experiment ) (Finkelstein et al., 2012) or the effect of military service on lifetime earnings (Angrist, 1990).
We propose a novel machine learning framework (called MRIV) for estimating CATEs using binary
IVs.
Our framework takes an initial CATE estimator and nuisance parameter estimators as input to perform a pseudo-outcome regression.
Different to existing literature, our framework is multiply robust 1 , i.e., we show that it is consistent in the union of three different model specifications.
This is different from existing methods for CATE estimation using IVs such as Okui et al.
(2012), Syrgkanis et al.
(2019), or plug-in estimators (Bargagli-Stoffi et al., 2021;
Imbens & Angrist, 1994).
We provide a theoretical analysis, where we use tools from Kennedy (2022) to show that our framework achieves a multiply robust convergence rate, i.e., our MRIV converges with a fast rate even if several nuisance parameters converge slowly.
We further show that, compared to existing plug-in IV methods, the performance of our framework is asymptotically superior.
Finally, we leverage our framework and, on top of it, build a tailored deep neural network called MRIV-Net.
Contributions: (1) We propose a novel, multiply robust machine learning framework (called MRIV) to learn the CATE using the binary IV setting.
To the best of our knowledge, ours is the first that is shown to be multiply robust, i.e., consistent in the union of three model specifications.
For comparison, existing works for CATE estimation only show double robustness (Wang & Tchetgen Tchetgen, 2018;
Syrgkanis et al., 2019).
(2) We prove that MRIV achieves a multiply robust convergence rate.
This is different to methods for IV settings which do not provide robust convergence rates (Syrgkanis et al., 2019).
We further show that our MRIV is asymptotically superior to existing plug-in estimators.
(3) We propose a tailored deep neural network, called MRIV-Net, which builds upon our framework to estimate CATEs .
We demonstrate that MRIV-Net achieves state-of-the-art performance.
P ROBLEM SETUP
Data generating process
: We observe data D =
( x i , z i , a i , y i ) ni
=1 consisting of n ∈ N observations of the tuple ( X, Z, A, Y ) .
Here, X ∈ X are observed confounders, Z ∈ { 0 , 1 } is a binary instrument,
A ∈ { 0 , 1 } is a binary treatment, and Y ∈ R is an outcome of interest.
Furthermore, we assume the existence of unobserved confounders U ∈ U , which affect both the treatment A and the outcome Y . Note we allow
The causal graph is shown in Fig.
Applicability : Our proposed framework is widely applicable in practice, namely to all settings with the above data generating process.
This includes both (1) observational data and (2) RCTs with non-compliance.
For (1), observational data is commonly encountered in, e.g., personalized medicine.
Here, modeling treatments as binary variables is consistent with previous literature on causal effect estimation and standard in medical practice (Robins et al., 2000).
For (2), our setting is further encountered in RCTs when the instrument Z is a randomized treatment assignment but individuals do not comply with their treatment assignment.
Such RCTs have been extensively used by public decision-makers, e.g., to analyze the effect of health insurance on health outcome (Finkelstein et al., 2012) or the effect of military service on lifetime earnings (Angrist, 1990).
We build upon the potential outcomes framework (Rubin, 1974) for modeling causal effects.
Let Y ( a, z ) denote the potential outcome that would have been observed under A = a and Z
= z .
Following previous literature on IV estimation (Wang & Tchetgen Tchetgen, 2018), we impose the following standard IV assumptions on the data generating process.
Assumption 1 (Standard IV assumptions (Wang & Tchetgen Tchetgen, 2018; Wooldridge, 2013)) .
We assume: (1) Exclusion: Y ( a, z ) =
Y ( a ) for all a, z ∈ { 0 , 1 } , i.e., the instrument has no direct effect on the patient outcome; (2) Independence: Z ⊥⊥ U | X ; (3) Relevance: Z ̸⊥⊥ A | X , (iv) The model includes all A – Y confounder: Y ( a ) ⊥⊥ ( A, Z ) | ( X, U ) for all a ∈ { 0 , 1 } .
Assumption 1 is standard for IV methods and fulfilled in practical settings where IV methods are applied (Angrist, 1990; Angrist & Krueger, 1991; Imbens & Angrist, 1994).
Note that Assumptiondoes not prohibit the existence of unobserved Z – A confounders.
On the contrary, it merely prohibits the existence of unobserved counfounders that affect all Z , A , and Y simultaneously, as it is standard in IV settings (Wooldridge, 2013).
A practical and widespread example where Assumption 1 is satisfied are randomized controlled trials (RCTs) with non-compliance (Imbens & Angrist, 1994).
Here, the treatment assignment Z is randomized, but the actual relationship between treatment A and outcome Y may still be confounded.
For instance, in the Oregon health insurance experiment (Finkelstein et al., 2012), people were given access to health insurance ( Z ) by a lottery with aim to study the effect of health insurance ( A ) on health outcome ( Y ) (Finkelstein et al., 2012).
Here, the lottery winners needed to sign up for health insurance and thus both Z and A are observed.
Objective:
In this paper, we are interested in estimating the conditional average treatment effect (CATE)
[Equation]
If there is no unobserved confounding ( U = ∅ ), the CATE is identifiable from observational data (Shalit et al., 2017).
However, in practice, it is often unlikely that all confounders are observable.
To account for this, we leverage the instrument Z to identify the CATE.
We state the following assumption for identifiability.
Assumption 2 (Identifiability of the CATE (Wang & Tchetgen Tchetgen, 2018)) .
At least one of the following two statements holds true: (1) E
[ A | Z = 1 , X, U ] − E
[ A | Z = 0 , X, U ] =
E [ A | Z = 1 , X ] − E
[ A | Z = 0 , X ] ; or (2) E
[ Y (1) − Y (0) | X, U ] =
E [ Y (1) − Y (0) | X ] .
Example: Assumption 1 holds when the function f
( a, X, U )
=
E [ Y ( a ) | X, U ] is additive with respect to a and U , e.g., f ( a, X, U )
=
g ( a, X ) + h ( U ) for measurable functions h and g .
This implies that no unobserved confounder affects the outcome through a path which is also affected by the treatment.
For example, with patient income as unobserved confounder, the treatment should not affect the (future) patient income.
Under Assumptions 1 and 2, the CATE is identifiable (Wang & Tchetgen Tchetgen, 2018).
It can be written as
[Equation] [Equation]
where µ Yi ( x ) =
E [ Y | Z
= i, X = x ] and µ Ai ( x ) =
E [ A | Z
=
i, X = x ] .
Even if Assumptiondoes not hold, all our results in this paper still hold for the quantity on the right-hand side of Eq.
In certain cases, this quantity still allows for interpretation: If no unobserved Z – A confounders exist, it can be interpreted as conditional version of the local average treatment effect (LATE)
(Imbens & Angrist, 1994) under a monotonicity assumption.
Furthermore, under a no-currenttreatment-value-interaction assumption, it can be interpreted as conditional treatment effect on the treated (ETT) (Wang & Tchetgen Tchetgen, 2018).
This has an important implication for our results:
If Assumption 2 does not hold in practice, our estimates still provide conditional LATE or ETT estimates under the respective assumptions because they are based on Eq.
If Assumption 2 does hold, all three – i.e., CATE, conditional LATE, and ETT – coincide (Wang & Tchetgen Tchetgen, 2018).
R ELATED WORK
Machine learning methods for IV: Only recently, machine learning has been integrated into IV methods.
These are: Singh et al.
(2019) and Xu et al.
(2021a)
generalize 2SLS by learning complex feature maps using kernel methods and deep learning, respectively.
Hartford et al.
(2017) adopts a two-stage neural network architecture that performs the first stage via conditional density estimation.
Bennett et al.
(2019) leverages moment conditions for IV estimation.
However, the aforementioned methods are not specifically designed for the binary IV setting but, rather, for multiple IVs or treatment scenarios.
In particular, they impose stronger assumptions such as additive confounding in order to identify the CATE.
Note that additive confounding is a special case of our Assumption 2.
Moreover, they do not have robustness properties.
In the binary IV setting, current methods proceed by estimating µ Yi ( x ) and µ Ai ( x ) separately, before plugging them in Eq.
(Imbens & Angrist, 1994;
Angrist et al., 1996; Bargagli-Stoffi et al., 2021).
As a result, these suffer from plug-in bias and do not offer robustness properties.
[Table]
Doubly robust IV methods: Recently, doubly robust methods have been proposed for IV settings: Kennedy et al.
(2019) propose a pseudo regression estimator for the local average treatment effect using continuous instruments, which has been extended to conditional effects by Semenova & Chernozhukov (2021).
Furthermore, Singh & Sun (2019) use a doubly robust approach to estimate average compiler parameters.
Finally, Ogburn et al.
(2015) and Syrgkanis et al.
(2019) propose doubly robust CATE estimators in the IV setting which both rely on doubly robust parametrizations of the uncentered efficient influence function (Okui et al., 2012).
However, none of these estimators has been shown to be multiply robust in the sense that they are consistent in the union of more than two model specifications (Wang & Tchetgen Tchetgen, 2018).
Multiply robust IV methods: Multiply robust estimators for IV settings have been proposed only for average treatment effects (ATEs) (Wang & Tchetgen Tchetgen, 2018) and optimal treatment regimes (Cui & Tchetgen, 2021) but not for CATEs .
In particular, Wang & Tchetgen Tchetgen (2018) derive a multiply robust parametrization of the efficient influence function for the ATE.
However, there exists no method that leverages this result for CATE estimation.
We provide a detailed, technical comparison of existing methods and our framework in Appendix G.
Doubly robust rates for CATE estimation: Kennedy (2022) analyzed the doubly robust learner in the standard (non-IV) setting and derived doubly robust convergence rates.
However, Kennedy’s result is not applicable in the IV setting, because we use the multiply robust parametrization of the efficient influence function from Wang & Tchetgen Tchetgen (2018).
In our paper, we rely on certain results from Kennedy, but use these to derive of a multiply robust rate.
In particular, this required the derivation of the bias term for a larger number of nuisance parameters (see Appendix B).
Research gap:
To the best of our knowledge, there exists no method for CATE estimation under unobserved confounding that has been shown to be multiply robust .
To fill this gap, we propose
MRIV: a multiply robust machine learning framework tailored to the binary IV setting.
For this, we build upon the approach by Kennedy (2022) to derive robust convergence rates, yet this approach has not been adapted to IV settings, which is our contribution.
MRIV FOR ESTIMATING CATE S USING BINARY INSTRUMENTS
In the following, we present our MRIV framework for estimating CATEs under unobserved confounding (Sec. 4.1).
We then derive an asymptotic convergence rate for MRIV (Sec. 4.2) and finally use our framework to develop a tailored deep neural network called MRIV-Net (Sec. 4.4).
4. F RAMEWORK
Motivation: A na¨ıve approach to estimate the CATE is to leverage the identification result in Eq.
Assuming that we have estimated the nuisance components ˆ µ Y
i and ˆ µ A i for i ∈ { 0 , 1 } , we can simply plug them into Eq.
(2) to obtain the so-called (plug-in) Wald estimator ˆ τ W ( x ) (Wald, 1940).
However, in practice, the true CATE curve τ ( x ) is often simpler (e.g., smoother, more sparse) than its complements µ Yi ( x ) or µ Ai ( x ) (K¨unzel et al., 2019).
In this case, ˆ τ W ( x ) is inefficient because it models all components separately, and, to address this, our proposed framework estimates τ directly using a pseudo-outcome regression.
Overview: We now propose MRIV.
MRIV is a two-stage meta learner that takes any base method for
CATE estimation as input.
For instance, the base method could be the Wald estimator from Eq.
(2), any other IV method such as 2SLS, or a deep neural network (as we propose in our MRIV-Net later in Sec. 4.4).
In Stage 1, MRIV produces nuisance estimators ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , and
ˆ π ( x ) , where ˆ π ( x ) is an estimator of the propensity score π ( x ) =
P ( Z = 1 | X = x ) .
In Stage 2, MRIV estimates τ ( x ) directly using a pseudo outcome ˆ Y MR as a regression target.
Given an arbitrary initial CATE estimator ˆ τ init ( x ) and nuisance estimates ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , and ˆ π ( x ) , we define the pseudo outcome
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Figure]
[Table]
The pseudo outcome ˆ Y MR in Eq.
(3) is a multiply robust parameterization of the (uncentered) efficient influence function for the average treatment effect E X [ τ ( X )]
(see the derivation in (Wang & Tchetgen Tchetgen, 2018)).
Once we have obtained the pseudo outcome ˆ Y MR , we regress it on X to obtain the Stage 2 MRIV estimator ˆ τ MRIV ( x ) for τ ( x ) .
The pseudocode for
MRIV is given in Algorithm 1.
MRIV can be interpreted as a way to remove plug-in bias from ˆ τ init ( x ) (Curth et al., 2020).
Using the fact that ˆ Y MR is a multiply robust parametrization of the efficient influence function, we derive a multiple robustness property of ˆ τ MRIV ( x ) .
Theorem 1 (multiple robustness property) .
Let ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , ˆ π ( x ) , and ˆ τ init ( x ) denote estimators of µ Y 0 ( x ) , µ A 0 ( x ) , δ A ( x ) , π ( x ) , and τ ( x ) , respectively.
Then, for all x ∈ X , it holds that
E [ ˆ Y MR | X = x ] = τ ( x ) ,if least one of the following conditions is satisfied: (1) ˆ µ Y 0 = µ Y 0 , ˆ µ A 0 = µ A 0 , and ˆ τ init =
τ ; or (2) ˆ π = π
and ˆ δ A = δ A ; or (3) ˆ
π = π
and ˆ τ init =
τ .
The equalities in Theorem 1 are meant to hold almost surely.
Consistency of ˆ τ MRIV ( x ) is a direct consequence: If either the nuisance estimators in (1), (2), or (3) converge to their oracle estimands, ˆ τ MRIV ( x ) will converge to the true CATE.
As a result, our MRIV framework is multiply robust in the sense that our estimator, ˆ τ MRIV ( x ) , is consistent in the union of three different model specifications.
Importantly, this is different from doubly robust estimators which are only consistent in the union of two model specifications (Wang & Tchetgen Tchetgen, 2018).
Our MRIV is directly applicable to
RCTs with non-compliance: Here, the treatment assignment is randomized and the propensity score π ( x ) is known.
Our MRIV framework can be thus adopted by plugging in the known π ( x ) into the pseudo outcome in Eq.
Moreover, ˆ τ MRIV ( x ) is already consistent if either ˆ τ init (x) or ˆ δ A ( x ) are.
4. T HEORETICAL ANALYSIS
We derive the asymptotic bound on the convergence rate of MRIV under smoothness assumptions.
For this, we define s -smooth functions as functions contained in the H¨older class H ( s ) , associated with Stone’s minimax rate (Stone, 1980) of n − 2 s/ (2 s + p ) , where p is the dimension of X .
Assumption 3 (Smoothness) .
We assume that (1) the nuisance component µ Y 0 ( · ) is α -smooth, µ A 0 ( · ) is β -smooth, π ( · ) is γ -smooth, and δ A ( · ) is δ -smooth; (2) all nuisance components are estimated with their respective minimax rate of n − 2 k 2 k + p , where k ∈ { α, β, γ, δ } ; and (3) the oracle CATE τ ( · ) is η -smooth and the initial CATE estimator
ˆ τ init converges with rate r τ ( n ) .
We provide a rigorous definition in Appendix D.
Assumption 3 for smoothness provides us with a way to quantify the difficulty of the underlying nonparametric regression problems.
Similar assumptions have been imposed for asymptotic analysis of previous CATE estimators in (Kennedy, 2022; Curth & van der Schaar, 2021).
They can be replaced with other assumptions such as assumptions on the level of sparsity of the CATE components.
We also provide an asymptotic analysis under sparsity assumptions (see Appendix C).
Assumption 4 (Boundedness) .
We assume that there exist constants C, ρ, (cid:101) ρ, ϵ, K > 0 such that for all x ∈ X it holds that: (1) | µ Yi ( x ) | ≤ C ; (2) | δ A ( x ) | = | µ A 1 ( x ) − µ A
( x ) | ≥ ρ and | ˆ δ A ( x ) | ≥ (cid:101) ρ ; (3) ϵ ≤ ˆ π ( x ) ≤ 1 − ϵ ; and (4) | ˆ τ init ( x ) | ≤ K .
Assumptions 4.1, 4.3, and 4.4 are standard and in line with previous works on theoretical analyses of CATE estimators (Curth & van der Schaar, 2021; Kennedy, 2022).
Assumption 4.2 ensures that both the oracle CATE and the estimator are bounded.
Violations of Assumption 4.2 may occur when working with “weak” instruments, which are IVs that are only weakly correlated with the treatment.
Using IV methods with weak instruments should generally be avoided (Li et al., 2022).
However, in many applications such as RCTs with non-compliance, weak instruments are unlikely to occur as patients’ compliance decisions are generally correlated with the initial treatment assignments.
We state now our main theoretical result: an upper bound on the oracle risk of the MRIV estimator.
To derive our bound, we leverage the sample splitting approach from (Kennedy, 2022).
The approach in (Kennedy, 2022) has been initially used to analyze the DR-learner for CATE estimation under unconfoundedness and allows for the derivation of robust convergence rates.
It has later been adapted to several other meta learners (Curth & van der Schaar, 2021), yet not for IV methods.
Theorem 2 (Oracle upper bound under smoothness) .
Let D ℓ for ℓ ∈ { 1 , 2 , 3 } be independent samples of size n .
Let ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and ˆ µ A
( x ) be trained on D 1 , and let ˆ δ A ( x ) and
ˆ π ( x ) be trained on
D 2 .
We denote ˆ Y MR as the pseudo outcome from Eq.
(3) and ˆ τ MRIV ( x ) =
ˆ E n [ ˆ Y MR | X = x ] as the pseudo-outcome regression on D 3 for some generic estimator ˆ E n [ · | X = x ] of E [ · | X = x ] .
We assume that the second-stage estimator ˆ E n yields the minimax rate n − 2 η 2 η + p and satisfies the stability assumption from Kennedy (2022), Proposition 1 (see Appendix B).
Then, under Assumptions 1-4 the oracle risk is upper bounded by
[Equation]
Proof.
See Appendix B.
The proof provides a more general bound which depends on the pointwise mean squared errors of the nuisance parameters (Lemma 2).
Recall that the first summand of the lower bound in Eq.
(2) is the minimax rate for the oracle CATE τ ( x ) which cannot be improved upon.
Hence, for a fast convergence rate of ˆ τ MRIV ( x ) , it is sufficient if either: (1) r τ ( n ) decreases fast and α , β are large; (2) γ and δ are large; or (3) r τ ( n ) decreases fast and γ is large.
This is in line with the multiply robustness property of MRIV (Theorem 1) and means that MRIV achieves a fast rate even if the initial or several nuisance estimators converge slowly.
Improvement over ˆ τ init ( x ) :
From the bound in Eq.
(2), it follows that ˆ τ MRIV ( x ) improves on the convergence rate of the initial CATE estimator ˆ τ init ( x ) if its rate r τ ( n ) is lower bounded by
[Equation]
Hence, our MRIV estimator is more likely to improve on the initial estimator ˆ τ init ( x ) if either (1) γ is large or (2) α and β are large.
Note that the margin of improvement depends also on the size of γ and δ , i.e., on the smoothness of π ( x ) and δ A ( x ) .
In fact, this is widely fulfilled in practice.
For example, the former is fulfilled for RCTs with non-compliance, where π ( x ) is often some known, fixed number p ∈ (0 , 1) .
4. MRIV VS . W ALD ESTIMATOR
We compare ˆ τ MRIV ( x ) to the Wald estimator ˆ τ W ( x ) .
First, we derive an asymptotic upper bound.
Theorem 3 (Wald oracle upper bound) .
Assume that µ Y 1 ( x ) , µ Y 0 ( x ) are α -smooth, µ A 1 ( x ) , µ A 0 ( x ) are β -smooth, and are estimated with their respective minimax rate.
Let ˆ δ A ( x ) =
ˆ µ A 1 ( x ) − ˆ µ A 0 ( x ) satisfy Assumption 4.
Then, the oracle risk of the Wald estimator ˆ τ W ( x ) is bounded by
[Equation]
Proof.
See Appendix B.
We now consider the MRIV estimator ˆ τ MRIV ( x ) with ˆ τ init =
ˆ τ W ( x ) , i.e., initialized with the Wald estimator (under sample splitting).
Plugging the Wald rate from Eq.
(5) into the Eq.
(2) yields
E (cid:104) (
[Equation]
x
[Equation]
For α = β = γ = δ , the rates of ˆ τ MRIV ( x ) and ˆ τ W ( x ) reduce to E (cid:104) (ˆ τ MRIV ( x ) − τ ( x )) 2 (cid:105) ≲ n − 2 η 2 η + p + n − 4 α 2 α + p and E (cid:104) (ˆ τ W ( x ) − τ ( x )) 2 (cid:105) ≲ n − 2 α 2 α + p .
Hence, ˆ τ MRIV ( x ) outperforms ˆ τ W ( x ) asymptotically for η > α , i.e., when the CATE τ ( x ) is smoother than its components, which is usually the case in practice (K¨unzel et al., 2019).
For η = α , the rates of both estimators coincide.
Hence, we should expect MRIV to improve on the Wald estimator in real-world settings with large sample size.
4. MRIV-N ET
Based on our MRIV framwork, we develop a tailored deep neural network called MRIV-Net for
CATE estimation using IVs.
Our MRIV-Net produces both an initial CATE estimator ˆ τ init ( x ) and nuisance estimators ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , and ˆ π ( x ) .
FF
FF FF
FF FF
FF FF
FF FF
For MRIV-Net, we choose deep neural networks for the nuisance components due to their predictive power and their ability to learn complex shared representations for several nuisance components.
Sharing representations between nuisance components has been exploited previously for CATE estimation, yet only under unconfoundedness (Shalit et al., 2017; Curth & van der Schaar, 2021).
Building shared representations is more efficient in finite sample regimes than estimating all nuisance components separately as they usually share some common structure.
In MRIV-Net, not all nuisance components should share a representation.
Recall that, in Theorem 2, we assumed that (1) ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and ˆ µ A 0 ( x ) ; and (2)
ˆ δ A ( x ) and ˆ π ( x ) are trained on two independent samples in order to derive the upper bound on the oracle risk.
Hence, we propose to build two separate representations Φ 1 and Φ 2 , so that (i) Φ 1 is used to learn the parameters (1), and (ii) Φ 2 is used to learn the parameters (2).
This ensures that the nuisance estimators (1) share minimal information with nuisance estimators (2) even though they are estimated on the same data (cf.
(Curth & van der Schaar, 2021)).
The architecture of MRIV-Net is shown in Fig.
MRIV-Net takes the observed covariates X as input to build the two representations Φ 1 and Φ 2 .
The first representation Φ 1 is used to output estimates ˆ µ Y 1 ( x ) , ˆ µ Y 0 ( x ) , ˆ µ A 1 ( x ) , and ˆ µ A 0 ( x ) of the CATE components.
The second representation Φ 2 is used to output estimates (cid:101) µ
A 1 ( x ) , (cid:101) µ A 0 ( x ) , and ˆ π ( x ) .
MRIV-Net is trained by minimizing an overall loss
L
[Equation]
ˆ µ Yz i ( x i ) − y i (cid:1) 2 + BCE (cid:0) ˆ µ Az i ( x i ) ,a i (cid:1) + BCE (cid:0)(cid:101) µ Az i ( x i ) ,a i (cid:1) + BCE(ˆ π ( x i ) ,z i ) (cid:105) , (6) where θ denotes the neural network parameters and BCE is the binary cross entropy loss.
After training MRIV-Net, we obtain the ˆ τ init ( x ) =
ˆ µ Y 1 ( x ) − ˆ µ Y 0 ( x ) ˆ µ A 1 ( x ) − ˆ µ A 0 ( x ) and obtain the nuisance estimators ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) =
(cid:101) µ A 1 ( x ) − (cid:101) µ A 0 ( x ) and ˆ π ( x ) .
Then, we perform, we perform the pseudo regression (Stage 2) of MRIV to obtain ˆ τ MRIV ( x ) .
Implementation: Details on the implementation, the network architecture and hyperparameter tuning are in Appendix I.
We perform both the training of MRIV-Net and the pseudo-outcome regression on the full training data.
Needless to say, MRIV-Net can be easily adopted for sample splitting or cross-fitting procedures as in Chernozhukov et al.
(2018), namely, by learning separate networks for each representation Φ 1 and Φ 2 .
In our experiments, we do not use sample splitting or cross-fitting, as this can affect the performance in finite sample regimes.
Of note, our choice is consistent with previous work (Curth & van der Schaar, 2021).
In Appendix K we report results using cross-fitting.
C OMPUTATIONAL EXPERIMENTS 5. S IMULATED DATA
In causal inference literature, it is common practice to use simulated data for performance evaluations (Bica et al., 2020a; Curth & van der Schaar, 2021; Hartford et al., 2017).
Simulated data offers the crucial benefit that it provides ground-truth information on the counterfactual outcomes and thus allows for direct benchmarking against the oracle CATE.
Data generation: We generate simulated data by sampling the oracle CATE τ ( x ) and the nuisance components µ Yi ( x ) , µ Ai ( x ) , and π ( x ) from Gaussian process priors.
Using Gaussian processes has the following advantages: (1) It allows for a fair method comparison, as there is no need to explicitly specify the nuisance components, which could lead to unwanted inductive biases favoring a specific method; (2) the sampled nuisance components are non-linear and thus resemble real-world scenarios where machine learning methods would be applied; and, (3) by sampling from the prior induced by the Mat´ern kernel (Rasmussen & Williams, 2008), we can control the smoothness of the nuisance components, which allows us to confirm our theoretical results from Sec. 4.2.
For a detailed description of our data generating process, we refer to Appendix E.
Baselines: We compare our MRIV-Net with state-of-the-art IV baselines.
Details regarding baselines and nuisance parameter estimation are in Appendix G. Note that many of the baselines do not directly aim at CATE estimation but rather at counterfactual outcome prediction.
We nevertheless use these methods as baselines and, for this, obtain the CATE by taking the difference between the predictions of the factual and counterfactual outcomes. [Table] [Figure]
[Table]
Performance evaluation: For all experiments, we use a 80/20 split as training/test set.
We calcalute the root mean squared errors (RMSE) between the
CATE estimates and the oracle CATE on the test set.
We report the mean RMSE and the standard deviation over five data sets generated from random seeds.
Results: Table
shows the results for all baselines.
Here, the DR-learner does not improve the performance of TARNet, which is reasonable as both the DRlearner and TARNet assume unconfoundedness and are thus biased in our setting.
Our MRIV-Net outperforms all baselines.
Our MRIV-Net also achieves a smaller standard deviation.
For additional results, we refer to Appendix J.
We further compare the performance of two different meta-learner frameworks – DRIV (Syrgkanis et al., 2019) and our MRIV– across different base methods.
The results are in Table [Table]
The nuisance parameters are estimated using feed forward neural networks (DRIV) or TAR-
[Table]
Nets with either binary or continuous outputs (MRIV).
Our MRIV improves over the variant without any meta-learner framework across all base methods (both in terms of RMSE and standard deviation).
Furthermore, MRIV is clearly superior over DRIV.
This demonstrates the effectiveness of our MRIV across different base methods (note: MRIV with an arbitrary base model is typically superior to DRIV with our custom network from above).
[Table]
MRIV-Net is overall best.
We also performed additional experiments where we used semi-synthetic data and crossfitting approaches for both meta-learners (see Appendix J and K).
Ablation study: Table 4 compares different variants of our MRIV-Net.
These are: (1) MRIV but network only; (2) MRIV-Net with a single representation for all nuisance estimators; and (3)
our
MRIV-Net from above.
We observe that MRIV-Net is best.
This justifies our proposed network architecture for MRIV-Net.
Hence, combing the result from above, our performance gain must be attributed to both our framework and the architecture of our deep neural network.
[Figure]
Robustness checks for unobserved confounding and smoothness: Here, we demonstrate the importance of handling unobserved confounding (as we do in our
MRIV framework).
For this,
Fig.
3 plots the results for our MRIV-Net vs. standard
CATE without customization for confounding (i.e., TARNet with and without the DR-learner) over over different levels of unobserved confounding.
The RMSE of both TARNet variants increase almost linearly with increasing confounding.
In contrast, the RMSE of our MRIV-Net only marginally.
Even for low confounding regimes, our MRIV-Net performs competitively.
[Figure]
Fig.
4 varies the smoothness level.
This is given by α of µ Yi ( · ) (controlled by the Mat ´ ern kernel prior).
Here, the performance decreases for the baselines, i.e., DeepIV and our network without MRIV framework.
In contrast, the peformance of our
MRIV-Net remains robust and outperforms the baselines.
This confirms our theoretical results from above.
It thus indicates that our MRIV framework works best when the oracle CATE τ ( x ) is smoother than the nuisance parameters µ Yi ( x ) .
5. C ASE STUDY WITH REAL - WORLD DATA
Setting: We demonstrate effectiveness of our framework using a case study with real-world, medical data.
Here, we use medical data from the so-called Oregon health insurance experiment (OHIE)
(Finkelstein et al., 2012).
It provides data for an RCT with non-compliance:
In 2008, ∼ 30, low-income, uninsured adults in Oregon were offered participation in a health insurance program by a lottery.
Individuals whose names were drawn could decide to sign up for health insurance.
After a period of 12 months, in-person interviews took place to evaluate the health condition of the respective participant.
[Figure]
In our analysis, the lottery assignment is the instrument Z , the decision to sign up for health insurance is treatment A , and an overall health score is the outcome Y .
We also include five covariates X , including age and gender.
For details, we refer to Appendix F. We first estimate the CATE function and then report the treatment effect heterogeneity w.r.t.
age and gender, while fixing the other covariates.
The results for MRIV-Net, our neural network architecture without the
MRIV framework, and TARNet are in Fig.
Results: Our MRIV-Net estimates larger causal effects for an older age.
In contrast, TARNet does not estimate positive CATEs even for an older age.
Even though we cannot evaluate the estimation quality on real-world data, our estimates seem reasonable in light of the medical literature: the benefit of health insurance should increase with older age.
This showcases that TARNet may suffer from bias induced by unobserved confounders.
We also report the results for DRIV with DMLIV as base method, and observe that in contrast to MRIV-Net, the corresponding CATE does not vary much between ages.
Interestingly, both our MRIV-Net estimate a somewhat smaller CATE for middle ages (around 30–50 yrs).
In sum, the findings from our case study are of direct relevance for decision-makers in public health (Imbens & Angrist, 1994), and highlight the practical value of our framework.
We performed further experiments on real-world data which are reported in Appendix L.
Reproducibility: The codes for reproducing the experimental results can be found at https: //github.com/DennisFrauen/MRIV-Net .
A E XTENDED RELATED WORK
CATE methods without unconfoundedness: Various machine learning methods for estimating
CATEs without unobserved confounding have been proposed in recent literature (Alaa & van der
Schaar, 2017; Curth & van der Schaar, 2021; K¨unzel et al., 2019; Lim et al., 2018; Shalit et al., 2017;
Wager & Athey, 2018; Yoon et al., 2018; Zhang et al., 2020).
To remove plug-in bias, the DR-learner performs a second stage regression on the uncentered influence function of the average treatment effect (Kennedy, 2022).
However, under unobserved confounding, all of these methods are biased (see Appendix G).
As a result, this hampers their performance in our setting.
Non-IV methods for unobserved confounding: There is a rich literature for causal effect estimation under unobserved confounding, which does not assume the existence of instrumental variables.
Methods include deconfounding methods (Wang & Blei, 2019; Bica et al., 2020b), proxy learning methods (Cui et al., 2020; Xu et al., 2021b), and causal sensitivity analysis (Kallus et al., 2019;
Jesson et al., 2021)..
Classical IV methods: IV methods address the problem of unobserved confounding by exploiting the variance in treatment and outcome induced by the instruments.
Traditionally, two-stage least squares (2SLS) has been used for estimating causal effects (Wright, 1928; Angrist & Krueger, 1991).
2SLS was originally developed in economics, and follows a two-stage procedure: it performs a first stage regression of treatment A on the instrument Z , and then uses the fitted values for a second stage regression to predict the outcome Y .
Several nonparametric methods have been developed in econometric to generalize 2SLS in order to account for non-linearities within the data (Newey &
Powell, 2003; Wang et al., 2021), yet these are limited to low-dimensional settings.
B P ROOFS
We start by deriving an auxiliary Lemma.
That is, we derive an explicit expression for the Stageoracle pseudo-outcome regression E [ ˆ Y MR | X = x ] of MRIV.
Lemma 1.
[Equation]
Proof.
[Equation]
Rearranging the terms yields the desired result.
B. P ROOF OF T HEOREM 1 ( MULTIPLE ROBUSTNESS PROPERTY )
We use Lemma 1 to show that under each of the three conditions it follows that E [ ˆ Y MR | X = x ] = τ ( x )
[Equation] [Equation]
[Equation] [Equation]
B. P ROOF OF T HEOREM 2 (C ONVERGENCE RATE OF MRIV)
To prove Theorem 2, we need an additional assumption on the second stage regression estimator ˆ E n .
We refer to Kennedy (2022)
(Proposition 1) for a detailed discussion on this assumption.
Assumption 5 (From Proposition 1 of Kennedy (2022)) .
Let Y MR be the corresponding oracle to the estimated pseudo-outcome ˆ Y MR .
We assume that the pseudo-regression estimator ˆ E n satisfies
[Equation]
and
[Equation]
19) where r ( x ) =
E [ ˆ Y MR | X = x ] − τ ( x )
To prove Theorem 2, we derive a more general bound the depends on the pointwise mean squared errors of the nuisance estimators.
Theorem 2 follows immediately by applying Assumption 3.
Lemma 2.
Consider the setting described in Theorem 2.
Then,
[Equation]
Proof.
Let Y MR be the corresponding oracle to ˆ Y MR and define (cid:101)
τ MRIV ( x ) =
ˆ E n [ Y MR | X = x ] .
Using Assumption 5, we can apply Proposition 1 of Kennedy (2022) and obtain
[Equation]
where R ( x ) = E (cid:104) ( (cid:101) τ MR ( x ) − τ ( x )) 2 (cid:105) is the oracle risk of the second stage regression.
We can apply
Lemma 1 to obtain
[Equation]
[Equation]
Applying the inequality ( a + b ) 2 ≤ 2( a 2 + b 2 ) together with Assumption 4 and the fact that π ( x ) ≤yields
[Equation]
By setting (cid:101) K = max { K, 1 } , we obtain
[Equation] [Equation]
Applying expectations on both sides yields the results, because (ˆ π ( x ) , ˆ δ [Equation] ( x )
[Equation]
⊥⊥ (ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ τ init ( x )) due to sample splitting.
B. P ROOF OF T HEOREM 3 (C ONVERGENCE RATE OF THE W ALD ESTIMATOR )
Proof.
We define (cid:101) C = max { C, 1 } and obtain the upper bound
[Equation]
where we used the inequality ( a + b ) 2 ≤ 2( a 2 + b 2 ) several times.
Taking expectations and applying the smoothness assumptions yields the result.
C T HEORETICAL ANALYSIS UNDER SPARSITY ASSUMPTIONS
In Sec. 4.2, we analyzed MRIV theoretically by imposing smoothness assumptions on the underlying data generating process.
In particular, we derived a multiple robust convergence rate and showed that MRIV outperforms the Wald estimator if the oracle CATE is smoother than its components.
In this section, we derive similar results by relying on a different set of assumptions.
Instead of using smoothness, we make assumptions on the level of sparsity of the CATE components.
This assumption is often imposed in high-dimensional settings ( n < p ) and is in line with previous literature on analyzing CATE estimators (Curth & van der Schaar, 2021; Kennedy, 2022).
In the following, we say a function f ( x ) is k -sparse, if it is linear in x ∈ R p
and it only depends on k < min { n, p } predictors.
(Yang & Tokdar, 2015) showed, that in this case the minimax rate of f ( x ) is given by k log( p ) n .
The linearity assumption can be relaxed to an additive structural assumption, which we omit here for simplicity.
In the following, we replace the smoothness conditions in
Assumption 3 with sparsity conditions.
Assumption 6 (Sparsity) .
We assume that (1) the nuisance components µ Yi ( · ) are α -sparse, µ Ai ( · ) and δ A ( · ) are β -sparse, and π ( · ) is δ -sparse; (2) all nuisance components are estimated with their respective minimax rate of k log( p ) n , where k ∈ { α, β, δ } ; and (3) the oracle CATE τ ( · ) is γ -sparse and the initial CATE estimator
ˆ τ init converges with rate r τ ( n ) .
We restate now our result from Theorem 3 for MRIV using the sparsity assumption.
Theorem 4 (MRIV upper bound under sparsity) .
We consider the same setting as in Theoremunder the sparsity assumption 6.
If the second-stage estimator ˆ E n yields the minimax rate γ log( p ) n and satisfies Assumption 5, the oracle risk is upper bounded by
[Equation]
Proof.
Follows immediately from Lemma 2 by applying Ass- 6.
Again, we obtain a multiple robust convergence rate for MRIV in the sense that MRIV achieves a fast rate even if the initial estimator or several nuisance estimators converge slowly.
More precisely, for a fast convergence rate of ˆ τ MRIV ( x ) , it is sufficient if either: (1) r τ ( n ) decreases fast and δ is small; (2) r τ ( n ) decreases fast and α and β are small; or (3) all α , β , and δ are small.
We derive now the corresponding rate for the Wald estimator.
Theorem 5 (Wald oracle upper bound) .
Given estimators ˆ µ Yi ( x ) and ˆ µ Ai ( x ) .
Let ˆ δ A ( x ) =
ˆ µ A 1 ( x ) − ˆ µ A 0 ( x ) satisfy Assumption 4.
Then, under Assumption 6 the oracle risk of the Wald estimator ˆ τ W ( x ) is bounded by
[Equation]
Proof.
Follows immediately from the proof of Theorem 3, i.e., from Eq.(30) by applying Ass- 6.
If α = β = δ , we obtain the rates
[Equation]
which means that ˆ τ MRIV ( x ) outperforms ˆ τ W ( x ) for γ < α , i.e., if the oracle CATE is more sparse than its components.
D M ATHEMATICAL DETAILS REGARDING A SSUMPTION
In this section, we briefly state the formal definitions of the convergence ratesin Assumption 3.
We follow Stone (1980).
Let θ be a parameter and T ( θ ) some target functional that we want to estimate.
Definition 1.
A sequence of estimators ˆ T n ( θ ) of a functional T ( θ ) converges with rate r T ( n ) if
[Equation]
Definition 2.
A rate r T ( n ) is called an upper bound to the rate of convergence if for all estimators
ˆ T n ( θ ) it holds for all c > 0 that
[Equation] [Equation]
and
[Equation]
38) r T ( n ) is called optimal if it is both achievable and an upper bound.
Stone (1980) showed that for a nonparametric regression problem with an η -smooth regression function, the optimal rate of convergence is n − 2 η 2 η + p , where p is the dimension of the covariate space.
E S IMULATED DATA
In the following, we describe how we simulate synthetic data for the experiments in Sec.
5.1 from the main paper.
As mentioned therein, we simulate the CATE components from Gaussian processes using the prior induced by the Matern kernel (Rasmussen & Williams, 2008)
[Equation]
where Γ( · ) is the Gamma function and K ν ( · ) is the modified Bessel function of second kind.
Here, ℓ is the length scale of the kernel and ν controls the smoothness of the sampled functions.
We set ℓ = 1 and sample functions δ Y ∼ GP (0 , K ℓ,γ ) , µ Y 0 ∼ GP (0 , K ℓ,α ) , f 1 ∼ GP (0 , K ℓ,β ) , f 0 ∼ GP (0 , K ℓ,β ) and g ∼ GP (0 , K ℓ,β ) .
Then, we define µ Y 1 =
δ Y + µ Y 0 , µ A 1 = 0 .
3 · σ ◦ f 1 + 0 .
7 , µ A 0 = 0 .
3 · σ ◦ f 0 , δ A =
[Equation]
−
[Equation]
, µ Y
= [Equation] 0 δ A , and π = σ ◦ g .
Finally, we set the oracle CATE to
[Equation]
Note that we can create a setup where the CATE τ is smoother than its components by using a small α/β ratio.
An example is shown in Fig.
[Figure]
In the following, we describe how we generate data the ( X, Z, A, Y ) using the CATE components
µ Y i ( x ) , µ A i ( x ) , and π ( x ) .
We begin by sampling n observed confounder X ∼ N (0 , 1) , unobserved confounders U ∼ N (cid:0) 0 , 0 .
2 2 (cid:1) , and instruments Z ∼ Bernoulli( π ( X )) .
Then, we obtain treatments via
[Equation]
with indicator function 1 , noise ϵ A
[Equation]
and
[Equation]
where Φ − 1 denotes the quantile function of the standard normal distribution.
Finally, we generate the outcomes via
[Equation] =
A
[Equation]
where ϵ Y ∼ N (cid:0) 0 , 0 .
3 2 (cid:1) is noise and α U > 0 is a parameter indicating the level of unobserved confounding.
This choice of A and Y in Eq.
(41) and Eq.
(42), respectively, implies that τ ( x ) is indeed the CATE, i. e., it holds that τ ( x ) =
E [ Y (1) − Y (0) | X = x ] .
Lemma 3.
Let ( X, Z, A, Y ) be sampled from the the previously described procedure.
Then, it holds that
[Equation]
Proof.
The first claim follows from
[Equation]
because U + ϵ A ∼ N (0 , √ 0 .
1 2 + 0 .
The second claim follows from
[Equation]
F O REGON HEALTH INSURANCE EXPERIMENT
The so-called Oregon health insurance experiment 3 (OHIE) (Finkelstein et al., 2012) was an important
RCT with non-compliance.
It was intentionally conducted as large-scale effort among public health to assess the effect of health insurance on several outcomes such as health or economic status.
In 2008, a lottery draw offered low-income, uninsured adults in Oregon participation in a Medicaid program, providing health insurance.
Individuals whose names were drawn could decide to sign up for the program.
In our analysis, the lottery assignment is the instrument Z , the decision to sign up for the Medicaid program is the treatment A , and an overall health score is the outcome Y .
The outcome was obtained after a period of 12 months during in-person interviews.
We use the following covariates X : age, gender, language, the number of emergency visits before the experiment, and the number of people the individual signed up with.
The latter is used to control for peer effects, and it is important to include this variable in our analysis as it is the only variable influencing the propensity score (see below).
We extract ∼ 10,000 observations from the OHIE data and plot the histograms of all variables in Fig.
We can clearly observe the presence of non-compliance within the data, because the ratio of treated / untreated individuals is much lower than the corresponding ratio for the treatment assignment.
   < ) U HTXHQ F\   $ ) U HTXHQ F\   = ) U HTXHQ F\     
$JH ) U HTXHQ F\    1XPVLJQHGXS ) U HTXHQ F\     1XPYLVLWV ) U HTXHQ F\ 0DOH )HPDOH *HQGHU ) U HTXHQ F\ 2WKHU (QJOLVK /DQJXDJH ) U HTXHQ F\ Figure
The data collection in the OHIE was done follows:
After excluding individuals below the age of 19, above the age of 64, and individuals with residence outside of Oregon, 74,922 individuals were considered for the lottery.
Among those, 29,834 were selected randomly and were offered participation in the program.
However, the probability of selection depended on the number of household members on the waiting list: for instance, an individual who signed up with another person was twice as likely to be selected.
From the 74,922 individuals, 57,528 signed up alone, 17, signed up with another person, and 158 signed up with two more people on the waiting list.
Thus, the probability of being selected conditional on the number of household members on the waiting list follows the multivariate version of Wallenius’ noncentral hypergeometric distribution (Chesson, 1976).
Propensity score: We computed the propensity score as follows.
To account for the Wallenius’ noncentral hypergeometric distribution, we use the R package BiasedUrn to calculate the propensity score π ( x ) =
P ( Z = 1 | X = x ) .
We obtained π ( x ) =
  0 .
345 , if individual x signed up alone, 0 .
571 , if individual x signed up with one more person, 0 .
719 , if individual x signed up with two more people.
During the training of both MRIV and DRIV, we use the calculated values from Eq.
49) for the propensity score.
G D ETAILS FOR BASELINE METHODS
In this section, we give a brief overview on the baselines which we used in our experiments.
We implemented: (1) CATE methods for unconfoundedness:
TARNet (Shalit et al., 2017) and TARNet combined with the DR-learner (Kennedy, 2022); (2) general IV methods, i.e., IV methods developed for IV settings with multiple or continuous instruments and treatments: 2SLS (Wright, 1928), kernel
IV ( KIV )
(Singh et al., 2019), DFIV (Xu et al., 2021a), DeepIV (Hartford et al., 2017), DeepGMM (Bennett et al., 2019), DMLIV (Syrgkanis et al., 2019), and DMLIV combined with DRIV (as described in (Syrgkanis et al., 2019)); (3) the (plug-in) Wald estimator using linear models and
Bayesian additive regression trees ( BART ) (Chipman et al., 2010).
Of note, the DR-learner assumes unconfoundedness, which is why we only combine it TARNet in our experiments.
In the following, we provide details regarding methods and implementation.
G. CATE METHODS FOR UNCONFOUNDEDNESS
Many CATE methods assume unconfoundedness , i.e., that all confounders are observed in the data.
Formally, the unconfoundedness assumption can be expressed in the potential outcomes framework as
[Equation]
Under unconfoundedness, the CATE is identified as
[Equation]
Methods that assume unconfoundedness proceed by estimating µ i ( x ) =
E [ Y |
A
= i, X = x ] from
Eq.
However, if unobserved confounders U exist, it follows that
[Equation]
which means that estimators that assume unconfoundedness are generally biased.
Nevertheless, we include two baselines that assume unconfoundedness into our experiments:
TARNet (Shalit et al., 2017) and the DR-learner (Kennedy, 2022).
TARNet (Shalit et al., 2017):
TARNet (Shalit et al., 2017) is a neural network that estimates the
CATE components µ i ( x ) from Eq.
by learning a shared representation Φ( x ) and two potential outcome heads
h
i (Φ( x )) .
We train TARNet by minimizing the loss
[Equation]
where θ =
( θ h 1 , θ h 0 , θ Φ ) denotes the model parameters and L denotes squared loss if Y is continuous or binary cross entropy loss if Y is binary.
Note regarding balanced representations: In (Shalit et al., 2017), the authors propose to add an additional regularization term inspired from domain adaptation literature, which forces TARNet to learn a balanced representation Φ( x ) , i.e., that minimizes the distance the treatment and control group in the feature space.
They showed that this approach leads to minimization of a generalization bound on the CATE estimation error if the representation is invertible.
In our experiments, we refrained from learning balanced representations because minimizing the regularized loss from (Shalit et al., 2017) does not necessarily result in an invertible representation and thus may even harm the estimation performance.
For a detailed discussion, we refer to (Curth & van der Schaar, 2021).
Furthermore, by leaving out the regularization, we ensure comparability between the different baselines.
If balanced representations are desired, the balanced representation approach could also be extended to MRIV-Net, as we also build MRIV-Net on learning shared representations.
DR-learner (Kennedy, 2022):
The DR-learner (Kennedy, 2022) is a meta learner that takes arbitrary estimators of the CATE componenets µ i and the propensity score π ( x )
= P ( A = 1 | X = x ) as input and performs a pseudo-outcome regression by using the pseudo outcome
[Equation]
In our experiments, we use TARNet as base method to provide initial estimators ˆ µ i ( X ) .
We further learn propensity score estimates ˆ π ( X ) by adding a seperate representation to TARNet as done in (Shalit et al., 2017).
G. G ENERAL IV METHODS
2SLS (Wright, 1928): 2SLS (Wright, 1928) is a linear two-stage approach.
First, the treatments A are regressed on the instruments Z and fitted values ˆ A are obtained.
In the second stage, the outcome
Y is regressed on ˆ A .
We implement 2SLS using the scikit-learn package.
KIV (Singh et al., 2019): Kernel IV (Singh et al., 2019) generalizes 2SLS to nonlinear settings.
KIV assumes that the data is generated by
[Equation] [Equation]
where U is an additive unobserved confounder and f is some unknown (potentially nonlinear) structural function.
KIV then models the structural function via
[Equation]
and
[Equation]
[Equation]
where ψ and ϕ are feature maps.
Here, kernel ridge regressions instead of linear regressions are used in both stages to estimate µ and V .
Following (Singh et al., 2019) we use the exponential kernel (Rasmussen & Williams, 2008) and set the length scale to the median inter-point distance.
KIV does not provide a direct way to incorporate the observed confounders X .
Hence, we augment both the instrument and the treatment with X , which is consistent with previous work (Bennett et al., 2019; Xu et al., 2021a).
We also use two different samples for each stage as recommended in (Singh et al., 2019).
DFIV (Xu et al., 2021a): DFIV (Xu et al., 2021a) is a similar approach KIV in generalizing 2SLS to nonlinear setting by assuming Eq.
(55) and Eq.
However, instead of using kernel methods, DFIV models the features maps ψ θ A and ϕ θ Z as neural networks with parameters θ A and θ Z , respectively.
DFIV is trained by iteratively updating the parameters θ A and θ Z .
The authors also provide a training algorithm that incorporates observed confounders X , which we implemented for our experiments.
During training,
we DeepIV (Hartford et al., 2017): DeepIV (Hartford et al., 2017) also assumes additive unobserved confounding as in Eq.
(55), but leverages the identification result (Newey & Powell, 2003)
[Equation]
where h ( a, x ) =
f
( a, x ) + E
[ U | X = x ] is the target counterfactual prediction function.
DeepIV estimates F ( a | x, z ) , i.e., the conditional distribution function of the treatment A given observed covariates X and instruments Z , by using neural networks.
Because we consider only binary treatments, we simply implement a (tunable) feed-forward neural network with sigmoid activation function.
Then, DeepIV proceeds by learning a second stage neural network to solve the inverse problem defined by Eq.
DeepGMM (Bennett et al., 2019): DeepGMM (Bennett et al., 2019) adopts neural networks for IV estimation inspired by the (optimally weighted)
Generalized Method of Moments.
The DeepGMM estimator is defined as the solution of the following minimax game:
[Equation]
where f ( z i , · ) and g ( a i , · ) are parameterized by neural networks.
As recommended in (Bennett et al., 2019), we solve this optimization via adversarial training with the Optimistic Adam optimizer (Daskalakis et al., 2018), where we set the parameter (cid:101) θ to the previous value of θ .
DMLIV (Syrgkanis et al., 2019): DMLIV (Syrgkanis et al., 2019) assumes that the data is generated via
[Equation]
where τ is the CATE f
some function of the observed covariates.
First, DMLIV estimates the functions q ( X ) =
E [ Y | X ] , h ( Z, X ) =
E [ A | Z, X ] , and p ( X ) =
E [ A | X ] .
Then, the CATE is learned by minimizing the loss
[Equation]
[Equation]
where ˆ τ ( X, · ) is some model for τ ( X ) .
In our experiments, we use (tunable) feed-forward neural networks for all estimators.
DRIV (Syrgkanis et al., 2019): DRIV (Syrgkanis et al., 2019) is a meta learner, originally proposed in combination with DMLIV.
It requires initial estimators for q ( X ) , p ( X ) , π ( X ) =
E [ Z | X = x ] , and f ( X ) =
E [ A · Z | X = x ] as well as an initial CATE estimatior
ˆ τ init ( X ) (e.g., from DMLIV).
The CATE is then estimated by a pseudo regression on the following doubly robust pseudo outcome:
[Equation]
We implement all regressions using (tunable) feed-forward neural networks.
Comparison between DRIV vs. MRIV: There are two key differences between our paper and (Syrgkanis et al., 2019): (i)
In contrast to DRIV, we showed that our MRIV is multiply robust.
(ii)
We derive a multiple robust convergence rate, while the rate in (Syrgkanis et al., 2019) is not robust with respect to the nuisance rates.
Ad (i): Both MRIV and DRIV perform a pseudo-outcome regression on the efficient influence function (EIF) of the ATE.
The key difference: DRIV uses the doubly robust parametrization of the
EIF from (Okui et al., 2012), whereas we use the multiply robust parametrization of the EIF from (Wang & Tchetgen Tchetgen, 2018) 4 .
Hence, our MRIV frameworks extends DRIV in a non-trivial way to achieve multiple robustness.
Thus, our estimator is consistent in the union of three different model specifications.
Ad (ii): Here, we compare the convergence rates from DRIV and our MRIV and, thereby, show the strengths of our MRIV.
To this end, let us assume that the pseudo regression function is γ -smooth and that we use the same second-stage estimator ˆE n with minimax rate n − 2 γ 2 γ + p for both DRIV and MRIV.
If the nuisance parameters q ( X ) , p ( X ) , f ( X ) , and π ( X ) are α -smooth and further are estimated with minimax rate n −
α 2 α + p , Corollary 4 from (Syrgkanis et al., 2019) states that DRIV converges with rate
[Equation]
In contrast, MRIV assumes estimation of the nuisance parameters µ Y 0 ( x ) with rate n − 2 α 2 α + p , µ A 0 ( x ) and δ A ( x ) with rate n − 2 β 2 β + p , and π ( x ) with rate n − 2 δ 2 δ + p .
If the initial estimator ˆ τ init ( x ) converges with rate r τ ( n ) , our Theorem 2 yields the rate
[Equation]
If all nuisance parameters converge with the same minimax rate of n − 2 α 2 α + p , the rates of DRIV and our MRIV coincide.
However, different to DRIV, our rate is additionally multiple robust in spirit of
Theorem 1.
This presents a crucial strength of our MRIV over DRIV:
For example, if δ is small (slow convergence of ˆ π ( x ) ), our MRIV still with fast rate as long as α and β are large (i.e., if the other nuisance parameters are sufficiently smooth).
G. W ALD ESTIMATOR
Finally, we consider the Wald estimator (Wald, 1940) for the binary IV setting.
More precisely, we estimate the CATE components µ Yi ( x ) and µ Ai ( x ) seperately and plug them into
[Equation]
We consider two versions of the Wald estimator:
Linear: We use linear regressions to estimate the µ Yi ( x ) and logistic regressions to estimate the µ Ai ( x ) .
BART: We use Bayesian additive regression trees (Chipman et al., 2010) trees to estimate the µ Yi ( x ) and random forest classifier to estimate the µ Ai ( x ) .
H V ISUALIZATION OF PREDICTED CATE S
We plot the predicted CATEs for the different baselines and MRIV-Net in Fig.
8 (for n = 3000 ).
As expected, the linear methods (2SLS and linear Wald) are not flexible enough to provide accurate
CATE estimates.
We also observe that the curve of MRIV-Net without MRIV is quite wiggly,
i
[Figure]
e., the estimator has a relatively large variance.
This variance is reduced when the full MRIV-Net is applied.
As a result, curve is much smoother.
This is reasonable because MRIV does not estimate the CATE components individually
[Figure]
but estimates the CATE directly via the Stage 2 pseudo-outcome regression.
Overall, this confirms the superiority of our proposed framework.
[Figure]
I I MPLEMENTATION DETAILS AND HYPERPARAMETER TUNING
Implementation details for deep learning models: To make the performance of the deep learning models comparable, we implemented all feed-forward neural networks (including MRIV-Net) as follows:
We use two hidden layers with RELU activation functions.
We also incorporated a dropout layer for each hidden layer.
We trained all models with the Adam optimizer (Kingma & Ba, 2015) using 100 epochs.
Exceptions are only DFIV and DeepGMM, where we used 200 epochs for training, accounting for slower convergence of the respective (adversarial) training algorithms.
For DeepGMM, we further used Optimistic Adam (Daskalakis et al., 2018) as in the original paper.
Training times: We report the approximate times needed to train the deep learning models on our simulated data with n = 5000 in Table 5.
For training, we used an AMD Ryzen Pro 7 CPU.
Compared to DMLIV and DRIV, the training of MRIV-Net is faster because only a single neural network is trained.
[Table]
Hyperparameter tuning: We performed hyperparameter tuning for all deep learning models (including MRIV-Net), KIV, and the BART Wald estimator on all datasets.
For all methods except
KIV and DFIV, we split the data into a training set (80%) and a validation set (20%).
We then performed 40 random grid search iterations and chose the set of parameters that minimized the respective training loss on the validation set.
In particular, the tuning procedure was the same for all baselines, which ensures that the performance gain of MRIV-Net is due to the method itself and not due to larger flexibility.
Exceptions are only KIV and DFIV, for which we implemented the customized hyperparameter tuning algorithms proposed in (Singh et al., 2019) and (Xu et al., 2021a) to ensure consistency with prior literature.
For the meta learners (DR-learner, DRIV
[Table]
and
MRIV), we first performed hyperparameter tuning for the base methods and nuisance models, before tuning the pseudo-outcome regression neural network by using the input from the tuned models.
The tuning ranges for the hyperparameter are shown in Table 6.
These include both the hyperparameter rangers shared across all neural networks and the model-specific hyperparameters.
For reproducibility purposes, we publish the selected hyperparameters in our GitHub project as .yaml files.
[Table]
Hyperparameter robustness checks: We also investigate the robustness of MRIV-Net with respect to hyperparameter choice.
To to this, we fix the optimal hyperparameter constellation for our simulated data for n = 3000 and perturb the hidden layer sizes, learning rate, dropout probability, and batch size.
The results are shown in Fig.
We observe that the RMSE only changes marginally when perturbing the different hyperparameters, indicating that our method is to a certain degree robust against hyperparameter misspecification.
Furthermore, our results indicate that the performance improvement of MRIV-Net over the baselines observed in our experiments is not due to hyperparameter tuning, but to our method itself.
[Figure]
J R ESULTS FOR SEMI - SYNTHETIC DATA
In the main paper, we evaluated MRIV-Net both on synthetic and real-world data.
Here, we provide additional results by constructing a semi-synthetic dataset on the basis of OHIE.
It is common practice in causal inference literature to use semi-synthetic data for evaluation, because it combines advantages of both synthetic and real-world data.
On the one hand, the real-world data part ensures that the data distribution is realistic and matches those in practice.
On the other hand, the counterfactual ground-truth is still available, which makes it possible to measure the performance of CATE methods.
We construct our semi-synthetic data as follows:
First, we extract the covariates X ∈ R 5 and instruments Z ∈ { 0 , 1 } of our OHIE dataset from Sec.
F. Then, we construct the treatment components µ Ai ( x ) via
[Equation]
where X 1 is the (standardized) age and σ ( · ) is the sigmoid function.
The outcome components are constructed via
[Equation]
We then sample treatments A and outcomes Y as in Eq.
(41) and Eq.
Lemma 3 ensures that µ Yi ( X ) =
E [ Y | Z
= i, X ] and µ Ai ( X ) =
E [ A | Z
=
i, X ] .
Given the above, the oracle CATE becomes
[Equation]
Note that τ ( X ) is sparse in the sense that it only depends on age, while the outcome components depend on all five covariates.
Following our theoretical analysis in Sec.
C, MRIV-Net should thus outperform methods that aim at estimating the components directly.
This is confirmed in Table 7, where we show the results for all baselines and MRIV-Net on the semi-synthetic data.
Indeed, we observe that MRIV-Net outperforms all other baselines, confirming both the superiority of our method as well as our theoretical results under sparsity assumptions from Sec.
C.
[Table]
K R ESULTS FOR CROSS - FITTING
Here, we repeat our experiments from the main paper but now make use of cross-fitting .
Recall that, in Theorem 2, we assume that the nuisance parameter estimation and the pseudo-outcome regression are performed on three independent samples.
We now address this through cross-fitting .
To this end, our aim is to show that our proposed MRIV framework is again superior.
For MRIV, we proceeded as follows:
We split the sample D into three equally sized samples D 1 , D 2 , and D 3 .
We then trained ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and
ˆ µ A
( x ) on D 1 , ˆ δ A ( x ) and ˆ π ( x ) on D 2 , and performed the pseudo-outcome regression on D 3 .
Then, we repeated the same training procedure two times, but performed the pseudo-outcome regression on D 2 and D 1 .
Finally, we averaged the resulting three
CATE estimators.
For DRIV, we implemented the cross-fitting procedure described in (Syrgkanis et al., 2019).
For the DR-learner, we followed (Kennedy, 2022).
The results are in Table 8.
Importantly, the results confirm the effectiveness of our proposed MRIV.
Overall, we find that our proposed MRIV outperforms DRIV for the vast majority of base methods when performing cross-fitting.
Furthermore, MRIV-Net is highly competitive even when comparing it with the cross-fitted estimators.
This shows that our heuristic to learn separate representations instead of performing sample splits works in practice.
In sum, the results confirm empirically that
our
MRIV is superior.
[Figure]
[Table]
L F URTHER EXPERIMENTAL RESULTS ON REAL - WORLD DATA
In Section 5.2, we estimated the ITE on the OHIE data and visualized the treatment heterogeneity with respect to age and gender .
In this section, we provide additional results and also visualize the heterogeneity with respect to age as well as additional covariates .
These are: (1) the number of emergency visits a patient has in its history before signing up for the lottery and (2) the language spoken by the patient (English or other).
We fixed the gender to “female”.
For (1), we plot the estimated ITE for three different age groups over the number of emergency visits.
The results are shown in Fig.
[Figure]
We observe that all methods tend to estimate a larger effect for individuals who had more emergency visits in their patient history.
However, the IV methods (in particular our MRIV-Net) estimate a much larger effect for patients with many visits.
In contrast to the other methods, MRIV-Net also estimates larger effects for older than for younger patients.
The results provided by MRIV-Net seem intuitive, as older patients with a history of emergency visits should be exposed to higher health-related risks, thus benefiting from health insurance.
The fact that TARNet consistently estimates small (and even negative) effects could be an indicator of bias due to unobserved confounding.
For (2), we plot the estimated ITE for three different age groups over the spoken language.
The results are shown in Fig.
[Figure]
For patients of age 50, our MRIV-Net estimates a higher effect for
-speaking patients
Interestingly increases also