E STIMATING INDIVIDUAL TREATMENT EFFECTS UNDER UNOBSERVED CONFOUNDING USING BINARY INSTRUMENTS
Anonymous authors
Paper under double-blind review
A BSTRACT
Estimating individual treatment effects (ITEs) from observational data is relevant in many fields such as personalized medicine.
However, in practice, the treatment assignment is usually confounded by unobserved variables and thus introduces bias.
A remedy to remove the bias is the use of instrumental variables (IVs).
Such settings are widespread in medicine (e g., trials where compliance is used as binary
IV).
In this paper, we propose a novel, multiply robust machine learning framework, called MRIV, for estimating ITEs using binary IVs and thus yield an unbiased ITE estimator.
Different from previous work for binary IVs, our framework estimates the ITE directly via a pseudo outcome regression.
(1) We provide a theoretical analysis where we show that our framework yields multiple robust convergence rates: our ITE estimator achieves fast convergence even if several nuisance estimators converge slowly.
(2) We further show that our framework asymptotically outperforms state-of-the-art plug-in IV methods for ITE estimation.
(3) We build upon our theoretical results and propose a tailored deep neural network architecture called MRIV-Net for ITE estimation using binary IVs.
Across various computational experiments, we demonstrate empirically that our MRIV-Net achieves state-of-the-art performance.
To the best of our knowledge, our MRIV is the first multiply robust machine learning framework tailored to estimating ITEs in the binary IV setting.
I NTRODUCTION
Individual treatment effects (ITEs) are relevant across many disciplines such as marketing (Varian, 2016) and personalized medicine (Yazdani & Boerwinkle, 2015).
Knowledge about ITEs provides insights into the heterogeneity of treatment effects, and thus help in potentially better treatment decisions.
Many recent works that use machine learning to estimate ITEs are based on the assumption of unconfoundedness (Alaa & van der Schaar, 2017; Lim et al., 2018; Shalit et al., 2017; Wager &
Athey, 2018), In practice, however, this assumption is often violated because it is common that some confounders are not reported in the data.
Typical examples are race, income, gender, or the socioeconomic status of patients, which are not stored in medical files.
If the confounding is sufficiently strong, standard methods for estimating ITEs suffer from confounding bias (Pearl, 2009), which may lead to inferior treatment decisions.
To handle unobserved confounders, instrumental variables (IVs) can be leveraged to relax the assumption of unconfoundedness and still compute reliable ITE estimates.
IV methods were originally developed in economics (Wright, 1928), but, only recently, there is a growing interest in combining
IV methods with machine learning (see Sec. 3).
Importantly, IV methods outperform classical ITE estimators if a sufficient amount of confounding is not observed (Hartford et al., 2017).
We thus aim at estimating ITEs from observational data under unobserved confounding using IVs.
In this paper, we consider the setting where a single binary instrument is available.
This setting is widespread in personalized medicine (and other applications such as marketing or public policy) (Bloom et al., 1997).
In fact, the setting is encountered in essentially all observational or randomized studies with observed non-compliance (Imbens & Angrist, 1994).
As an example, consider a randomized controlled trial (RCT), where treatments are randomly assigned to patients and their outcomes are observed.
Due to some potentially unobserved confounders (e.g., income, education), some patients refuse to take the treatment initially assigned to them.
Here, the treatment assignment serves as a binary IV.
Moreover, such RCTs have been widely used by public decision-makers, e.g., to analyze the effect of health insurance on health outcome (see the so-called Oregon health insurance experiment ) (Finkelstein et al., 2012) or the effect of military service on lifetime earnings (Angrist, 1990).
We propose a novel machine learning framework (called MRIV) for estimating ITEs using binary IVs.
Our framework takes an initial ITE estimator and nuisance parameter estimators as input to perform a pseudo-outcome regression.
Importantly, our framework uses a multiply robust parametrization of the efficient influence function as pseudo outcome.
We provide a theoretical analysis, where we use tools from Kennedy (2022) to show that our framework achieves a multiply robust convergence rate, i.e., our MRIV converges with a fast rate even if several nuisance parameters converge slowly.
We further show that, compared to existing plug-in IV methods, the performance of our framework is asymptotically superior.
Finally, we leverage our framework and, on top of it, build a tailored deep neural network called MRIV-Net.
Differences to existing literature : Our framework is multiply robust 1 , i.e., we show that it is consistent in the union of three different model specifications.
This is different from existing methods for ITE estimation using IVs such as Okui et al.
(2012), Syrgkanis et al.
(2019), or plug-in estimators (Bargagli-Stoffi et al., 2021;
Imbens & Angrist, 1994).
Contributions: (1) We propose a novel, multiply robust machine learning framework (called MRIV) to learn the ITE using the binary IV setting.
To the best of our knowledge, ours is the first that is shown to be multiply robust, i.e., consistent in the union of three model specifications.
For comparison, existing works for ITE estimation only show double robustness (Wang & Tchetgen Tchetgen, 2018;
Syrgkanis et al., 2019).
(2) We prove that MRIV achieves a multiply robust convergence rate.
This is different to methods for IV settings which do not provide robust convergence rates (Syrgkanis et al., 2019).
We further show that our MRIV is asymptotically superior to existing plug-in estimators.
(3) We propose a tailored deep neural network, called MRIV-Net, which builds upon our framework to estimate ITEs.
We demonstrate that MRIV-Net achieves state-of-the-art performance.
P ROBLEM SETUP
Data generating process
: We observe data D =
( x i , z i , a i , y i ) ni
=1 consisting of n ∈ N observations of the tuple ( X, Z, A, Y ) .
Here, X ∈ X are observed confounders, Z ∈ { 0 , 1 } is a binary instrument,
A ∈ { 0 , 1 } is a binary treatment, and Y ∈ R is an outcome of interest.
Furthermore, we assume the existence of unobserved confounders U ∈ U , which affect both the treatment A and the outcome Y .
The causal graph is shown in Fig.
Applicability : Our proposed framework is widely applicable in practice, namely to all settings with the above data generating process.
This includes both (1) observational data and (2) RCTs with non-compliance.
For (1), observational data is commonly encountered in, e.g., personalized medicine.
Here, modeling treatments as binary variables is consistent with previous literature on causal effect estimation and standard in medical practice (Robins et al., 2000).
For (2), our setting is further encountered in RCTs when the instrument Z is a randomized treatment assignment but individuals do not comply with their treatment assignment.
Such RCTs have been extensively used by public decision-makers, e.g., to analyze the effect of health insurance on health outcome (Finkelstein et al., 2012) or the effect of military service on lifetime earnings (Angrist, 1990).
We build upon the potential outcomes framework (Rubin, 1974) for modeling causal effects.
Let Y ( a, z ) denote the potential outcome that would have been observed under A = a and Z
= z .
Following previous literature on IV estimation (Wang & Tchetgen Tchetgen, 2018), we impose the following standard IV assumptions on the data generating process.
Assumption 1 (Standard IV assumptions (Wang & Tchetgen Tchetgen, 2018; Wooldridge, 2013)) .
We assume: (1) Exclusion: Y ( a, z ) =
Y ( a ) for all a, z ∈ { 0 , 1 } , i.e., the instrument has no direct effect on the patient outcome; (2) Independence: Z ⊥⊥ U | X ; (3) Relevance: Z ̸⊥⊥ A | X , (iv) The model includes all A – Y confounder: Y ( a ) ⊥⊥ ( A, Z ) | ( X, U ) for all a ∈ { 0 , 1 } .
Assumption 1 is standard for IV methods and fulfilled in practical settings where IV methods are applied (Angrist, 1990; Angrist & Krueger, 1991; Imbens & Angrist, 1994).
Note that Assumptiondoes not prohibit the existence of unobserved Z – A confounders.
On the contrary, it merely prohibits the existence of unobserved counfounders that affect all Z , A , and Y simultaneously, as it is standard in IV settings (Wooldridge, 2013).
A practical and widespread example where Assumption 1 is satisfied are randomized controlled trials (RCTs) with non-compliance (Imbens & Angrist, 1994).
Here, the treatment assignment Z is randomized, but the actual relationship between treatment A and outcome Y may still be confounded.
For instance, in the Oregon health insurance experiment (Finkelstein et al., 2012), people were given access to health insurance ( Z ) by a lottery with aim to study the effect of health insurance ( A ) on health outcome ( Y ) (Finkelstein et al., 2012).
Here, non-compliance information is observed because the lottery winners needed to sign up for health insurance.
Objective: In this paper, we are interested in estimating the individual treatment effect (ITE)
[Equation]
If there is no unobserved confounding ( U = ∅ ), the ITE is identifiable from observational data (Shalit et al., 2017).
However, in practice, it is often unlikely that all confounders are observable.
To account for this, we leverage the instrument Z to identify the ITE.
We state the following assumption for identifiability.
Assumption 2 (Identifiability of the ITE (Wang & Tchetgen Tchetgen, 2018)) .
At least one of the following two statements holds true: (1) E
[ A | Z = 1 , X, U ] − E
[ A | Z = 0 , X, U ] =
E [ A | Z = 1 , X ] − E
[ A | Z = 0 , X ] ; or (2) E
[ Y (1) − Y (0) | X, U ] =
E [ Y (1) − Y (0) | X ] .
Example: Assumption 1 holds is when the function f
( a, X, U )
=
E [ Y ( a ) | X, U ] is additive with respect to a and U , e.g., f ( a, X, U )
=
g ( a, X ) + h ( U ) for measurable functions h and g .
Under Assumptions 1 and 2, the ITE is identifiable (Wang & Tchetgen Tchetgen, 2018).
It can be written as
[Equation]
where µ Yi ( x ) =
E [ Y | Z
= i, X = x ] and µ Ai ( x ) =
E [ A | Z
=
i, X = x ] .
Even if Assumptiondoes not hold, the quantity on the right-hand side of Eq.
(2) still allows for interpretation.
If no unobserved Z – A confounders exist, it can be interpreted as conditional version of the local average treatment effect (LATE) (Imbens & Angrist, 1994) under a monotonicity assumption.
Furthermore, under a no-current-treatment-value-interaction assumption, it can be interpreted as conditional treatment effect on the treated (ETT) (Wang & Tchetgen Tchetgen, 2018).
This has an important implication for our results: If Assumption 2 does not hold in practice, our estimates still provide conditional LATE or ETT estimates under the respective assumptions because they are based on
Eq.
If Assumption 2 does hold, all three – i.e., ITE, conditional LATE, and ETT – coincide (Wang & Tchetgen Tchetgen, 2018).
R ELATED WORK
Machine learning methods for IV: Only recently, machine learning has been integrated into IV methods.
These are: Singh et al.
(2019) and Xu et al.
(2021a)
generalize 2SLS by learning complex feature maps using kernel methods and deep learning, respectively.
Hartford et al.
(2017) adopts a two-stage neural network architecture that performs the first stage via conditional density estimation.
Bennett et al.
(2019) leverages moment conditions for IV estimation.
However, the aforementioned methods are not specifically designed for the binary IV setting but, rather, for multiple IVs or treatment scenarios.
In particular, they impose stronger assumptions such as additive confounding in order to identify the ITE.
Note that additive confounding is a special case of when our Assumption 2 holds.
Moreover, they do not have robustness properties.
In the binary IV setting, current methods proceed by estimating µ Yi ( x ) and µ Ai ( x ) separately, before plugging them in Eq.
(Imbens & Angrist, 1994;
Angrist et al., 1996; Bargagli-Stoffi et al., 2021).
As result, these suffer from plug-in bias and do not offer robustness properties.
[Table]
Doubly robust IV methods: Doubly robust estimators are commonly used in causal inference as they allow for consistent estimation under model misspecification and fast convergence rates (Kennedy, 2022).
Recently, they also have been adopted for IV settings: Kennedy et al.
(2019) propose a pseudo regression estimator for the local average treatment effect using continuous instruments, which has been extended to individual effects by Semenova & Chernozhukov (2021).
Furthermore, Singh & Sun (2019) use a doubly robust approach to estimate average compiler parameters.
Finally, Ogburn et al.
(2015) and Syrgkanis et al.
(2019) propose doubly robust ITE estimators in the IV setting which both rely on doubly robust parametrizations of the uncentered efficient influence function (Okui et al., 2012).
However, none of these estimators has been shown to be multiply robust in the sense that they are consistent in the union of more than two model specifications (Wang & Tchetgen Tchetgen, 2018).
Multiply robust IV methods: Multiply robust estimators for IV settings have been proposed only for average treatment effects (ATEs) (Wang & Tchetgen Tchetgen, 2018) and optimal treatment regimes (Cui & Tchetgen, 2021) but not for ITEs.
In particular, Wang & Tchetgen Tchetgen (2018) derive a multiply robust parametrization of the efficient influence function for the ATE.
However, there exists no method that leverages this result for ITE estimation.
We provide a detailed, technical comparison of existing methods and our framework in Appendix F.
Research gap:
To the best of our knowledge, there exists no method for ITE estimation under unobserved confounding that has been shown to be multiply robust .
To fill this gap, we propose
MRIV: a multiply robust machine learning framework tailored to the binary IV setting.
For this, we build upon the approach by Kennedy (2022) to derive robust convergence rates, yet this approach has not been adapted to IV settings, which is our contribution.
MRIV FOR ESTIMATING ITE S USING BINARY INSTRUMENTS
In the following, we present our MRIV framework for estimating ITEs under unobserved confounding (Sec. 4.1).
We then derive an asymptotic convergence rate for MRIV (Sec. 4.2) and finally use our framework to develop a tailored deep neural network called MRIV-Net (Sec. 4.4).
4. F RAMEWORK
Motivation: A na¨ıve approach to estimate the ITE is to leverage the identification result in Eq.
Assuming that we have estimated the nuisance components ˆ µ Yi and ˆ µ Ai for i ∈ { 0 , 1 } , we can simply plug them into Eq.
(2) to obtain the so-called (plug-in) Wald estimator ˆ τ W ( x ) (Wald, 1940).
However, in practice, the true ITE curve τ ( x ) is often simpler (e.g., smoother, more sparse) than its complements µ Yi ( x ) or µ Ai ( x ) (K¨unzel et al., 2019).
In this case, ˆ τ W ( x ) is inefficient because it models all components separately, and, to address this, our proposed framework estimates τ directly using a pseudo outcome regression.
Overview: We now propose MRIV.
MRIV is a two-stage meta learner that takes any base method for ITE estimation as input.
For instance, the base method could be the Wald estimator from Eq.
(2), any other IV method such as 2SLS, or a deep neural network (as we propose in our MRIV-Net later in Sec. 4.4).
In Stage 1, MRIV produces nuisance estimators ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , and
ˆ π ( x ) , where ˆ π ( x ) is an estimator of the propensity score π ( x ) =
P ( Z = 1 | X = x ) .
In Stage 2, MRIV estimates τ ( x ) directly using a pseudo outcome ˆ Y MR as a regression target.
Given an arbitrary initial ITE estimator ˆ τ init ( x ) and nuisance estimates ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , and ˆ π ( x ) , we define the pseudo outcome
[Equation]
[Table] [Figure] [Table] [Figure]
[Table]
[Equation] [Table] [Equation] [Table]
[Equation] [Table] [Equation] [Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation] [Table] [Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
The pseudo outcome ˆ Y MR in Eq.
(3) is a multiply robust parameterization of the (uncentered) efficient influence function for the average treatment effect E X [ τ ( X )]
(see the derivation in (Wang & Tchetgen Tchetgen, 2018)).
Once we have obtained the pseudo outcome ˆ Y MR , we regress it on X to obtain the Stage 2 MRIV estimator ˆ τ MRIV ( x ) for τ ( x ) .
The pseudocode for
MRIV is given in Algorithm 1.
MRIV can be interpreted as a way to remove plug-in bias from ˆ τ init ( x ) (Curth et al., 2020).
Using the fact that ˆ Y MR is a multiply robust parametrization of the efficient influence function, we derive a multiple robustness property of ˆ τ MRIV ( x ) .
Theorem 1 (multiple robustness property) .
Let ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , ˆ π ( x ) , and ˆ τ init ( x ) denote estimators of µ Y 0 ( x ) , µ A 0 ( x ) , δ A ( x ) , π ( x ) , and τ ( x ) , respectively.
Then, for all x ∈ X , it holds that
E [ ˆ Y MR | X = x ] = τ ( x ) ,if least one of the following conditions is satisfied: (1) ˆ µ Y 0 = µ Y 0 , ˆ µ A 0 = µ A 0 , and ˆ τ init =
τ ; or (2) ˆ π = π
and ˆ δ A = δ A ; or (3) ˆ
π = π
and ˆ τ init =
τ .
Theorem 1 implies that ˆ τ MRIV ( x ) is consistent for τ ( x ) if either condition (1), (2), or (3) holds.
As a result, our MRIV framework is multiply robust in the sense that our estimator, ˆ τ MRIV ( x ) , is consistent in the union of three different model specifications.
Importantly, this is different from doubly robust estimators which are only consistent in the union of two model specifications (Wang &
Tchetgen Tchetgen, 2018).
Our MRIV is directly applicable to RCTs with non-compliance:
Here, the treatment assignment is randomized and the propensity score π ( x ) is known.
Our MRIV framework can be thus adopted by plugging in the known π ( x ) into the pseudo outcome in Eq.
Moreover, ˆ τ MRIV ( x ) is already consistent if either ˆ τ init (x) or ˆ δ A ( x ) are.
4. T HEORETICAL ANALYSIS
In the following, we derive the asymptotic convergence rate of MRIV under smoothness assumptions.
For this, we define s -smooth functions as functions contained in the H¨older class H ( s ) , associated with Stone’s minimax rate (Stone, 1980) of n − 2 s/ (2 s + p ) , where p is the dimension of X .
Assumption 3 (Smoothness) .
We assume that (1) the nuisance component µ Y 0 ( · ) is α -smooth, µ A 0 ( · ) is β -smooth, π ( · ) is γ -smooth, and δ A ( · ) is δ -smooth; (2) all nuisance components are estimated with their respective minimax rate of n − 2 k 2 k + p , where k ∈ { α, β, γ, δ } ; and (3) the oracle ITE τ ( · ) is η -smooth and the initial ITE estimator
ˆ τ init converges with rate r τ ( n ) .
Assumption 3 for smoothness provides us with a way to quantify the difficulty of the underlying nonparametric regression problems.
Similar assumptions have been imposed for asymptotic analysis of previous ITE estimators in (Kennedy, 2022; Curth & van der Schaar, 2021).
They can be replaced with other assumptions such as assumptions on the level of sparsity of the ITE components.
We also provide an asymptotic analysis under sparsity assumptions (see Appendix C).
We additionally impose the following boundedness assumptions on the the underlying data generating process and estimators.
Assumption 4 (Boundedness) .
We assume that there exist constants C, ρ, (cid:101) ρ, ϵ, K > 0 such that for all x ∈ X it holds that: (1) | µ Y i ( x ) | ≤ C ; (2) | δ A ( x ) | = | µ A 1 ( x ) − µ A
( x ) | ≥ ρ and | ˆ δ A ( x ) | ≥ (cid:101) ρ ; (3) ϵ ≤ ˆ π ( x ) ≤ 1 − ϵ ; and (4)
[Equation]
Assumptions 4.1, 4.3, and 4.4 are standard and in line with previous works on theoretical analyses of ITE estimators (Curth & van der Schaar, 2021; Kennedy, 2022).
Assumption 4.2 ensures that both the oracle ITE and the estimator are bounded.
Violations of Assumption 4.2 may occur when working with so-called “weak” instruments, which are IVs that are only weakly correlated with the treatment.
Using IV methods with weak instruments should generally be avoided (Li et al., 2022).
However, in many applications such as RCTs with non-compliance, weak instruments are unlikely to occur as patients’ decisions to follow the treatment are generally correlated with the initial treatment assignments.
We state now our main theoretical result: an upper bound on the oracle risk of the MRIV estimator.
To derive our bound, we leverage the sample splitting approach from (Kennedy, 2022).
The approach in (Kennedy, 2022) has been initially used to analyze the DR-learner for ITE estimation under unconfoundedness and allows for the derivation of robust convergence rates.
It has later been adapted to several other meta learners (Curth & van der Schaar, 2021), yet not for IV methods.
Theorem 2 (Oracle upper bound under smoothness) .
Let D ℓ for ℓ ∈ { 1 , 2 , 3 } be independent samples of size n .
Let ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and ˆ µ A
( x ) be trained on D 1 , and let ˆ δ A ( x ) and
ˆ π ( x ) be trained on
D 2 .
We denote ˆ Y MR as the pseudo outcome from Eq.
(3) and ˆ τ MRIV ( x ) =
ˆ E n [ ˆ Y MR | X = x ] as the pseudo outcome regression on D 3 for some generic estimator ˆ E n [ · | X = x ] of E [ · | X = x ] .
We assume that the second-stage estimator ˆ E n yields the minimax rate n − 2 η 2 η + p and satisfies the stability assumption from Kennedy (2022), Proposition 1 (see Appendix B).
Then, the oracle risk is upper bounded by
[Equation]
Proof.
See Appendix B.
Recall that the first summand of the lower bound in Eq.
(2) is the minimax rate for the oracle
ITE τ ( x ) which cannot be improved upon.
Hence, for a fast convergence rate of ˆ τ MRIV ( x ) , it is sufficient if either: (1) r τ ( n ) decreases fast and α , β are large; (2) γ and δ are large; or (3) r τ ( n ) decreases fast and γ is large.
This is in line with the multiply robustness property of MRIV (Theorem 1) and means that MRIV achieves a fast rate of convergence even if the initial estimator or several nuisance estimators converge slowly.
Improvement over ˆ τ init ( x ) :
From the bound in Eq.
(2), it follows that ˆ τ MRIV ( x ) improves on the convergence rate of the initial ITE estimator ˆ τ init ( x ) if its rate r τ ( n ) is lower bounded by
[Equation]
Hence, our MRIV estimator is more likely to improve on the initial estimator ˆ τ init ( x ) if either (1) γ is large or (2) α and β are large.
Note that the margin of improvement depends also on the size of γ and δ , i.e., on the smoothness of π ( x ) and δ A ( x ) .
In fact, this is widely fulfilled in practice.
For example, the former is fulfilled for RCTs with non-compliance, where π ( x ) is often some known, fixed number p ∈ (0 , 1) .
Hence, for RCTs with non-compliance, MRIV should (at least asymptotically) improve the performance of most estimators.
4. MRIV VS . W ALD ESTIMATOR
In the following, we compare ˆ τ MRIV ( x ) to the Wald estimator ˆ τ W ( x ) .
First, we derive corresponding upper bound under smoothness.
Theorem 3 (Wald oracle upper bound) .
Assume that µ Y 1 ( x ) , µ Y 0 ( x ) are α -smooth, µ A 1 ( x ) , µ A 0 ( x ) are β -smooth, and are estimated with their respective minimax rate.
Let ˆ δ A ( x ) =
ˆ µ A 1 ( x ) − ˆ µ A 0 ( x ) satisfy Assumption 4.
Then, the oracle risk of the Wald estimator ˆ τ W ( x ) is bounded by
[Equation]
Proof.
See Appendix B.
We now consider the MRIV estimator ˆ τ MRIV ( x ) with ˆ τ init =
ˆ τ W ( x ) , i.e., initialized with the Wald estimator (under sample splitting).
Plugging the Wald rate from Eq.
(5) into the Eq.
(2) yields
[Equation]
For α = β = γ = δ , the rates of ˆ τ MRIV ( x ) and ˆ τ W ( x ) reduce to E (cid:104) (ˆ τ MRIV ( x ) − τ ( x )) 2 (cid:105) ≲ n − 2 η 2 η + p + n − 4 α 2 α + p and E (cid:104) (ˆ τ W ( x ) − τ ( x )) 2 (cid:105) ≲ n − 2 α 2 α + p .
Hence, ˆ τ MRIV ( x ) outperforms ˆ τ W ( x ) asymptotically for η > α , i.e., when the ITE τ ( x ) is smoother than its components, which is usually the case in practice (K¨unzel et al., 2019).
For η = α , the rates of both estimators coincide.
Hence, we should expect MRIV to improve on the Wald estimator in real-world settings with large sample size.
4. MRIV-N ET
Based on our MRIV framwork, we develop a tailored deep neural network called MRIV-Net for ITE estimation using IVs.
Our MRIV-Net produces both an initial ITE estimator ˆ τ init ( x ) and nuisance estimators ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) , and ˆ π ( x ) .
FF
FF FF
FF FF
FF FF
FF FF
For MRIV-Net, we choose deep neural networks for the nuisance components due to their predictive power and their ability to learn complex shared representations for several nuisance components.
Sharing representations between nuisance components has been exploited previously for ITE estimation, yet only under unconfoundedness (Shalit et al., 2017;
Curth & van der Schaar, 2021).
Building shared representations is more efficient in finite sample regimes than estimating all nuisance components separately as they usually share some common structure.
In MRIV-Net, not all nuisance components should share a representation.
Recall that, in Theorem 2, we assumed that (1) ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and ˆ µ A 0 ( x ) ; and (2)
ˆ δ A ( x ) and ˆ π ( x ) are trained on two independent samples in order to derive the upper bound on the oracle risk.
Hence, we propose to build two separate representations Φ 1 and Φ 2 , so that (i) Φ 1 is used to learn ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and ˆ µ A 0 ( x ) , and (ii) Φ 2 is used to learn ˆ δ A ( x ) and ˆ π ( x ) .
This ensures that the nuisance estimators (1) share minimal information with nuisance estimators (2) even though they are estimated on the same data.
Intuitively, this should lead to a faster decay of the oracle upper bound (cf.
(Curth & van der Schaar, 2021)).
The architecture of MRIV-Net is shown in Fig.
MRIV-Net takes the observed covariates X as input to build the two representations Φ 1 and Φ 2 .
The first representation Φ 1 is used to output estimates ˆ µ Y 1 ( x ) , ˆ µ Y 0 ( x ) , ˆ µ A 1 ( x ) , and ˆ µ A 0 ( x ) of the ITE components.
The second representation Φ 2 is used to output estimates (cid:101) µ
A 1 ( x ) , (cid:101) µ A 0 ( x ) , and ˆ π ( x ) .
MRIV-Net is trained by minimizing an overall loss
[Equation]
(6) where θ denotes the neural network parameters and BCE is the binary cross entropy loss.
After training MRIV-Net, we obtain the ˆ τ init ( x ) =
ˆ µ Y
[Equation]
− ˆ µ Y 0 ( x ) ˆ µ A 1 ( x ) − ˆ µ A 0 ( x ) and obtain the nuisance estimators ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ δ A ( x ) = (cid:101) µ A 1 ( x ) − (cid:101) µ A 0 ( x ) and ˆ π ( x ) .
Then, we perform, we perform the pseudo regression (Stage 2) of MRIV to obtain ˆ τ MRIV ( x ) .
Implementation: We use PyTorch Lightning for our implementation and train MRIV-Net with the
Adam optimizer (Kingma & Ba, 2015).
Details on the network architecture and hyperparameter tuning are in Appendix H.
We perform both the training of MRIV-Net and the pseudo outcome regression on the full training data.
Needless to say, MRIV-Net can be easily adopted for sample splitting or cross-fitting procedures as in Chernozhukov et al.
(2018), namely, by learning separate networks for each representation Φ 1 and Φ 2 .
In our experiments, we do not use sample splitting or cross-fitting, as this can affect the performance in finite sample regimes.
Of note, our choice is consistent with previous work (Curth & van der Schaar, 2021).
In Appendix J we report results using cross-fitting.
C OMPUTATIONAL EXPERIMENTS 5. S IMULATED DATA
In causal inference literature, it is common practice to use simulated data for performance evaluations (Bica et al., 2020a; Curth & van der Schaar, 2021; Hartford et al., 2017).
Simulated data offers the crucial benefit that it provides ground-truth information on the counterfactual outcomes and thus allows for direct benchmarking against the oracle ITE.
Data generation: We generate simulated data by sampling the oracle ITE τ ( x ) and the nuisance components µ Yi ( x ) , µ Ai ( x ) , and π ( x ) from Gaussian process priors.
Using Gaussian processes has the following advantages: (1) It allows for a fair method comparison, as there is no need to explicitly specify the nuisance components, which could lead to unwanted inductive biases favoring a specific method; (2) the sampled nuisance components are non-linear and thus resemble real-world scenarios where machine learning methods would be applied; and, (3) by sampling from the prior induced by the Mat´ern kernel (Rasmussen & Williams, 2008), we can control the smoothness of the nuisance components, which allows us to confirm our theoretical results from Sec. 4.2.
For a detailed description of our data generating process, we refer to Appendix D.
Baselines: We compare our MRIV-Net with state-of-the-art IV baselines.
Details regarding baselines and nuisance parameter estimation are in Appendix F. Note that many of the baselines do not directly aim at ITE estimation but rather at counterfactual outcome prediction.
We nevertheless use these methods as baselines and, for this, obtain the ITE by taking the difference between the predictions of the factual and counterfactual outcomes.
[Table]
Performance evaluation: For all experiments, we use a 80/20 split as training/test set.
We calcalute the root mean squared errors (RMSE) between the ITE estimates and the oracle ITE on the test set.
We report the mean RMSE and the standard deviation over five data sets generated from random seeds.
Results: Table
shows the results for all baselines.
Here, the DR-learner does not improve the performance of TARNet, which is reasonable as both the DRlearner and TARNet assume unconfoundedness and are thus biased in our setting.
Our MRIV-Net outperforms all baselines.
Our MRIV-Net also achieves a smaller standard deviation.
For additional results, we refer to Appendix I
[Table]
We further compare the performance of two different meta-learner frameworks – DRIV (Syrgkanis et al., 2019) and our MRIV– across different base methods.
The results are in Table
[Table]
The nuisance parameters are estimated using feed forward neural networks (DRIV) or TAR-
[Table]
Nets with either binary or continuous outputs (MRIV).
Our MRIV improves over the variant without any meta-learner framework across all base methods (both in terms of RMSE and standard deviation).
Furthermore, MRIV is clearly superior over DRIV.
This demonstrates the effectiveness of our MRIV across different base methods (note: MRIV with an arbitrary base model is typically superior to DRIV with our custom network from above).
[Table]
MRIV-Net is overall best.
We also performed additional experiments where we used semi-synthetic data and crossfitting approaches for both meta-learners (see Appendix I and J).
Ablation study: Table 4 compares different variants of our MRIV-Net.
These are: (1) MRIV but network only; (2) MRIV-Net with a single representation for all nuisance estimators; and (3)
our
MRIV-Net from above.
We observe that MRIV-Net is best.
This justifies our proposed network architecture for MRIV-Net.
Hence, combing the result from above, our performance gain must be attributed to both our framework and the architecture of our deep neural network.
[Figure]
checks for unobserved confounding and smoothness: Here, we demonstrate the importance of handling unobserved confounding (as we do in our
MRIV framework).
For this,
Fig.
3 plots the results for our MRIV-Net vs. standard
ITE without customization for confounding (i.e., TARNet with and without the DR-learner) over over different levels of unobserved confounding.
The RMSE of both TARNet variants increase almost linearly with increasing confounding.
In contrast, the RMSE of our MRIV-Net only marginally.
Even for low confounding regimes, our MRIV-Net performs competitively.
[Figure]
Fig.
4 varies the smoothness level.
This is given by α of µ Yi ( · ) (controlled by the Mat ´ ern kernel prior).
Here, the performance decreases for the baselines, i.e., DeepIV and our network without MRIV framework.
In contrast, the peformance of our
MRIV-Net remains robust and outperforms the baselines.
This confirms our theoretical results from above.
It thus indicates that our MRIV framework works best when the oracle ITE τ ( x ) is smoother than the nuisance parameters µ Yi ( x ) .
5. C ASE STUDY WITH REAL - WORLD DATA
Setting: We demonstrate effectiveness of our framework using a case study with real-world, medical data.
Here, we use medical data from the so-called Oregon health insurance experiment (OHIE)
(Finkelstein et al., 2012).
It provides data for an RCT with non-compliance:
In 2008, ∼ 30,000 low-income, uninsured adults in Oregon were offered participation in a health insurance program by a lottery.
Individuals whose names were drawn could decide to sign up for health insurance.
After a period of 12 months, in-person interviews took place to evaluate the health condition of the respective participant.
[Figure]
In our analysis, the lottery assignment is the instrument Z , the decision to sign up for health insurance is treatment A , and an overall health score is the outcome Y .
We also include five covariates X , including age and gender.
For details, we refer to
Appendix E. We first estimate the ITE function and then report the treatment effect heterogeneity w.r.t.
age and gender, while fixing the other covariates.
The results for
MRIV-Net, our neural network architecture without the MRIV framework, and TARNet are in Fig.
Results: Our MRIV-Net estimates larger causal effects for an older age.
In contrast, TARNet does not estimate positive ITEs even for an older age.
Even though we cannot evaluate the estimation quality on real-world data, our estimates seem reasonable in light of the medical literature: the benefit of health insurance should increase with older age.
This showcases that TARNet may suffer from bias induced by unobserved confounders.
We also report the results for DRIV with DMLIV as base method, and observe that in contrast to MRIV-Net, the corresponding ITE does not vary much between ages.
Interestingly, both our MRIV-Net estimate a somewhat smaller ITE for middle ages (around 30–50 yrs).
One explanation might be that individual in this age group are more likely to have stable jobs and, thus, are also more likely to be able to afford medical care, decreasing the direct effect of health insurance on individuals health.
In sum, the findings from our case study are of direct relevance for decision-makers in public health (Imbens & Angrist, 1994), and highlight the practical value of our framework.
Reproducibility: Details for reproducing the experiment results can be found in the supplementary materials ( Readme.md ).
A E XTENDED RELATED WORK
ITE methods without unconfoundedness: Various machine learning methods for estimating ITEs without unobserved confounding have been proposed in recent literature (Alaa & van der Schaar, 2017; Curth & van der Schaar, 2021; K¨unzel et al., 2019; Lim et al., 2018; Shalit et al., 2017; Wager & Athey, 2018;
Yoon et al., 2018; Zhang et al., 2020).
To remove plug-in bias, the DR-learner performs a second stage regression on the uncentered influence function of the average treatment effect (Kennedy, 2022).
However, under unobserved confounding, all of these methods are biased (see Appendix F).
As a result, this hampers their performance in our setting.
Non-IV methods for unobserved confounding: There is a rich literature for causal effect estimation under unobserved confounding, which does not assume the existence of instrumental variables.
Methods include deconfounding methods (Wang & Blei, 2019; Bica et al., 2020b), proxy learning methods (Cui et al., 2020; Xu et al., 2021b), and causal sensitivity analysis (Kallus et al., 2019;
Jesson et al., 2021)..
Classical IV methods: IV methods address the problem of unobserved confounding by exploiting the variance in treatment and outcome induced by the instruments.
Traditionally, two-stage least squares (2SLS) has been used for estimating causal effects (Wright, 1928; Angrist & Krueger, 1991).
2SLS was originally developed in economics, and follows a two-stage procedure: it performs a first stage regression of treatment A on the instrument Z , and then uses the fitted values for a second stage regression to predict the outcome Y .
Several nonparametric methods have been developed in econometric to generalize 2SLS in order to account for non-linearities within the data (Newey &
Powell, 2003; Wang et al., 2021), yet these are limited to low-dimensional settings.
B P ROOFS
We start by deriving an auxiliary Lemma.
That is, we derive an explicit expression for the Stageoracle pseudo outcome regression E [ ˆ Y MR | X = x ] of MRIV.
Lemma 1.
[Equation]
Proof.
[Equation]
Rearranging the terms yields the desired result.
B. P ROOF OF T HEOREM 1 ( MULTIPLE ROBUSTNESS PROPERTY )
We use Lemma 1 to show that under each of the three conditions it follows that E [ ˆ Y MR | X = x ] = τ ( x )
[Equation] [Equation]
[Equation] [Equation]
B. P ROOF OF T HEOREM 2 (C ONVERGENCE RATE OF MRIV)
To prove Theorem 2, we need an additional assumption on the second stage regression estimator ˆ E n .
We refer to Kennedy (2022)
(Proposition 1) for a detailed discussion on this assumption.
Assumption 5 (From Proposition 1 of Kennedy (2022)) .
Let Y MR be the corresponding oracle to the estimated pseudo-outcome ˆ Y MR .
We assume that the pseudo-regression estimator ˆ E n satisfies
[Equation]
and
[Equation]
(19) where r ( x ) =
E [ ˆ Y MR | X = x ] − τ ( x )
Proof of Theorem 2.
Let Y MR be the corresponding oracle to ˆ Y MR and define (cid:101)
τ MRIV ( x ) =
ˆ E n [ Y MR |
X = x ] .
Using Assumption 5, we can apply Proposition 1 of Kennedy (2022) and obtain
E (cid:104)
[Equation]
where R ( x ) = E (cid:104) ( (cid:101) τ MR ( x ) − τ ( x )) 2 (cid:105) is the oracle risk of the second stage regression.
We can apply
Lemma 1 to obtain
[Equation]
(cid:18) π ( x ) ˆ π ( x ) − 1 − π ( x ) 1 − ˆ π ( x ) (cid:19) (22) =
δ Y ( x ) π ( x ) ˆ δ A ( x ) ˆ π ( x ) + (cid:0) µ Y 0 ( x ) − ˆ µ Y 0 ( x ) (cid:1) ( π ( x ) − ˆ π ( x ))
ˆ δ A ( x ) ˆ π ( x ) (1 − ˆ π ( x ))
+ (ˆ τ init ( x ) − τ ( x )) − δ A ( x ) π
( x ) ˆ τ init ( x ) ˆ δ A ( x ) ˆ π ( x ) + (cid:0) ˆ µ A 0 ( x ) − µ A 0 ( x ) (cid:1) ˆ τ init ( x ) ( π ( x ) − ˆ π ( x ))
ˆ δ A ( x ) ˆ π ( x ) (1 − ˆ π ( x )) (23) =
( π ( x ) − ˆ π ( x ))
ˆ δ A ( x ) ˆ π ( x ) (1 − ˆ π ( x )) (cid:2)(cid:0) µ Y 0 ( x ) − ˆ µ Y 0 ( x ) (cid:1) + (cid:0) ˆ µ A 0 ( x ) − µ A
( x ) (cid:1) ˆ τ init ( x ) (cid:3) + (ˆ τ init ( x ) − τ ( x ))
+
π ( x ) δ A ( x ) ˆ π ( x ) ˆ δ A ( x ) ( τ ( x ) − ˆ τ init ( x )) (24) =
( π ( x ) − ˆ π ( x ))
ˆ δ A ( x ) ˆ π ( x ) (1 − ˆ π ( x )) (cid:2)(cid:0) µ Y 0 ( x ) − ˆ µ Y 0 ( x ) (cid:1) + (cid:0) ˆ µ A 0 ( x ) − µ A
( x ) (cid:1) ˆ τ init ( x ) (cid:3) + ( τ ( x ) −
[Equation]
Applying the inequality ( a + b ) 2 ≤
[Equation]
2 + b 2 ) together with Assumption 4 and the fact that π ( x ) ≤yields
[Equation]
By setting (cid:101) K = max { K, 1 }
[Equation]
we obtain
[Equation]
Applying expectations on both sides yields
[Equation]
[Equation] [Equation]
because (ˆ π ( x )
[Equation]
ˆ δ A ( x )) ⊥⊥ (ˆ µ Y 0 ( x ) , ˆ µ A 0 ( x ) , ˆ τ init ( x ))
due to sample splitting.
The claim follows now by applying Assumption
[Equation]
B. P ROOF OF T HEOREM 3 (C ONVERGENCE RATE OF THE W ALD ESTIMATOR )
Proof.
We define (cid:101) C = max { C, 1 } and obtain the upper bound (ˆ
[Equation]
[Equation]
where we used the inequality ( a + b ) 2 ≤ 2( a 2 + b 2 ) several times.
Taking expectations and applying the smoothness assumptions yields the result.
C T HEORETICAL ANALYSIS UNDER SPARSITY ASSUMPTIONS
In Sec. 4.2, we analyzed MRIV theoretically by imposing smoothness assumptions on the underlying data generating process.
In particular, we derived a multiple robust convergence rate and showed that MRIV outperforms the Wald estimator if the oracle ITE is smoother than its components.
In this section, we derive similar results by relying on a different set of assumptions.
Instead of using smoothness, we make assumptions on the level of sparsity of the ITE components.
This assumption is often imposed in high-dimensional settings ( n < p ) and is in line with previous literature on analyzing ITE estimators (Curth & van der Schaar, 2021; Kennedy, 2022).
In the following, we say a function f ( x ) is k -sparse, if it is linear in x ∈ R p
and it only depends on k < min { n, p } predictors.
(Yang & Tokdar, 2015) showed, that in this case the minimax rate of f ( x ) is given by k log( p ) n .
The linearity assumption can be relaxed to an additive structural assumption, which we omit here for simplicity.
In the following, we replace the smoothness conditions in
Assumption 3 with sparsity conditions.
Assumption 6 (Sparsity) .
We assume that (1) the nuisance components µ Yi ( · ) are α -sparse, µ Ai ( · ) and δ A ( · ) are β -sparse, and π ( · ) is δ -sparse; (2) all nuisance components are estimated with their respective minimax rate of k log( p ) n , where k ∈ { α, β, δ } ; and (3) the oracle ITE τ ( · ) is γ -sparse and the initial ITE estimator ˆ τ init converges with rate r τ ( n ) .
We restate now our result from Theorem 3 for MRIV using the sparsity assumption.
Theorem 4 (MRIV upper bound under sparsity) .
We consider the same setting as in Theoremunder the sparsity assumption 6.
If the second-stage estimator ˆ E n yields the minimax rate γ log( p ) n and satisfies Assumption 5, the oracle risk is upper bounded by
[Equation]
Proof.
Follows immediately from the proof of Theorem 2, i.e., from Eq.(28) by applying Ass- 6.
Again, we obtain a multiple robust convergence rate for MRIV in the sense that MRIV achieves a fast rate even if the initial estimator or several nuisance estimators converge slowly.
More precisely, for a fast convergence rate of ˆ τ MRIV ( x ) , it is sufficient if either: (1) r τ ( n ) decreases fast and δ is small; (2) r τ ( n ) decreases fast and α and β are small; or (3) all α , β , and δ are small.
We derive now the corresponding rate for the Wald estimator.
Theorem 5 (Wald oracle upper bound) .
Given estimators ˆ µ Yi ( x ) and ˆ µ Ai ( x ) .
Let ˆ δ A ( x ) =
ˆ µ A 1 ( x ) − ˆ µ A 0 ( x ) satisfy Assumption 4.
Then, under Assumption 6 the oracle risk of the Wald estimator ˆ τ W ( x ) is bounded by
[Equation] [Equation]
Proof.
Follows immediately from the proof of Theorem 3, i.e., from Eq.(30) by applying Ass- 6.
If α = β = δ , we obtain the rates
[Equation]
which means that ˆ τ MRIV ( x ) outperforms ˆ τ W ( x ) for γ
D S IMULATED DATA
In the following, we describe how we simulate synthetic data for the experiments in Sec.
5.1 from the main paper.
As mentioned therein, we simulate the ITE components from Gaussian processes using the prior induced by the Matern kernel (Rasmussen & Williams, 2008)
[Equation]
where Γ( · ) is the Gamma function and K ν ( · ) is the modified Bessel function of second kind.
Here, ℓ is the length scale of the kernel and ν controls the smoothness of the sampled functions.
We set ℓ = 1 and sample functions δ Y ∼ GP (0 , K ℓ,γ ) , µ Y 0 ∼ GP (0 , K ℓ,α ) , f 1 ∼ GP (0 , K ℓ,β ) , f 0 ∼ GP (0 , K ℓ,β ) and g ∼ GP (0 , K ℓ,β ) .
Then, we define µ Y 1 =
δ Y + µ Y 0 , µ A 1 = 0 .
3 · σ ◦ f 1 + 0 .
7 , µ A
[Equation]
=
[Equation] [Equation]
σ ◦ f 0 , δ A =
[Equation]
−
[Equation]
, µ Y
= c 0 δ A , and π = σ ◦ g .
Finally, we set the oracle ITE to
[Equation]
Note that we can create a setup where the ITE τ is smoother than its components by using a small α/β ratio.
An example is shown in Fig.
[Figure]
In the following, we describe how we generate data the ( X, Z, A, Y ) using the ITE components µ Y i ( x ) ,
µ A i ( x ) , and π ( x ) .
We begin by sampling n observed confounder X ∼ N (0 , 1) , unobserved confounders U ∼ N (cid:0) 0 , 0 .
2 2 (cid:1) , and instruments Z ∼ Bernoulli( π ( X )) .
Then, we obtain treatments via
[Equation]
with indicator function 1 , noise
[Equation]
where Φ − 1 denotes the quantile function of the standard normal distribution.
Finally, we generate the outcomes via
Y =
A
[Equation]
where ϵ Y ∼ N (cid:0) 0 , 0 .
3 2 (cid:1) is noise and α U > 0 is a parameter indicating the level of unobserved confounding.
This choice of A and Y in Eq.
(38) and Eq.
(39), respectively, implies that τ ( x ) is indeed the ITE, i. e., it holds that τ ( x ) =
E [ Y (1) − Y (0) | X = x ] .
Lemma 2.
Let ( X, Z, A, Y ) be sampled from the the previously described procedure.
Then, it holds that
[Equation]
Proof.
The first claim follows from
[Equation]
because U + ϵ A ∼ N (0 , √ 0 .
1 2 + 0 .
The second claim follows from
[Equation]
E O REGON HEALTH INSURANCE EXPERIMENT
The so-called Oregon health insurance experiment 3 (OHIE) (Finkelstein et al., 2012) was an important
RCT with non-compliance.
It was intentionally conducted as large-scale effort among public health to assess the effect of health insurance on several outcomes such as health or economic status.
In 2008, a lottery draw offered low-income, uninsured adults in Oregon participation in a Medicaid program, providing health insurance.
Individuals whose names were drawn could decide to sign up for the program.
In our analysis, the lottery assignment is the instrument Z , the decision to sign up for the Medicaid program is the treatment A , and an overall health score is the outcome Y .
The outcome was obtained after a period of 12 months during in-person interviews.
We use the following covariates X : age, gender, language, the number of emergency visits before the experiment, and the number of people the individual signed up with.
The latter is used to control for peer effects, and it is important to include this variable in our analysis as it is the only variable influencing the propensity score (see below).
We extract ∼ 10,000 observations from the OHIE data and plot the histograms of all variables in Fig.
We can clearly observe the presence of non-compliance within the data, because the ratio of treated / untreated individuals is much lower than the corresponding ratio for the treatment assignment.
   < ) U HTXHQ F\   $ ) U HTXHQ F\   = ) U HTXHQ F\     
$JH ) U HTXHQ F\    1XPVLJQHGXS ) U HTXHQ F\     1XPYLVLWV ) U HTXHQ F\ 0DOH )HPDOH *HQGHU ) U HTXHQ F\ 2WKHU (QJOLVK /DQJXDJH ) U HTXHQ F\
The data collection in the OHIE was done follows:
After excluding individuals below the age of 19, above the age of 64, and individuals with residence outside of Oregon, 74,922 individuals were considered for the lottery.
Among those, 29,834 were selected randomly and were offered participation in the program.
However, the probability of selection depended on the number of household members on the waiting list: for instance, an individual who signed up with another person was twice as likely to be selected.
From the 74,922 individuals, 57,528 signed up alone, 17, signed up with another person, and 158 signed up with two more people on the waiting list.
Thus, the probability of being selected conditional on the number of household members on the waiting list follows the multivariate version of Wallenius’ noncentral hypergeometric distribution (Chesson, 1976).
Propensity score: We computed the propensity score as follows.
To account for the Wallenius’ noncentral hypergeometric distribution, we use the R package BiasedUrn to calculate the propensity score π ( x ) =
P ( Z = 1 | X = x ) .
We obtained π
[Equation]
  0 .
345 , if individual x signed up alone, 0 .
571 , if individual x signed up with one more person, 0 .
719 , if individual x signed up with two more people.
F D ETAILS FOR BASELINE METHODS
In this section, we give a brief overview on the baselines which we used in our experiments.
We implemented: (1) ITE methods for unconfoundedness:
TARNet (Shalit et al., 2017) and TARNet combined with the DR-learner (Kennedy, 2022); (2) general IV methods, i.e., IV methods developed for IV settings with multiple or continuous instruments and treatments: 2SLS (Wright, 1928), kernel
IV ( KIV )
(Singh et al., 2019), DFIV (Xu et al., 2021a), DeepIV (Hartford et al., 2017), DeepGMM (Bennett et al., 2019), DMLIV (Syrgkanis et al., 2019), and DMLIV combined with DRIV (as described in (Syrgkanis et al., 2019)); (3) the (plug-in) Wald estimator using linear models and
Bayesian additive regression trees ( BART ) (Chipman et al., 2010).
Of note, the DR-learner assumes unconfoundedness, which is why we only combine it TARNet in our experiments.
In the following, we provide details regarding methods and implementation.
F. ITE METHODS FOR UNCONFOUNDEDNESS
Many ITE methods assume unconfoundedness , i.e., that all confounders are observed in the data.
Formally, the unconfoundedness assumption can be expressed in the potential outcomes framework as
[Equation]
Under unconfoundedness, the ITE is identified as
[Equation]
Methods that assume unconfoundedness proceed by estimating µ i ( x ) =
E [ Y |
A
= i, X = x ] from
Eq.
However, if unobserved confounders U exist, it follows that
[Equation]
which means that estimators that assume unconfoundedness are generally biased.
Nevertheless, we include two baselines that assume unconfoundedness into our experiments:
TARNet (Shalit et al., 2017) and the DR-learner (Kennedy, 2022).
TARNet (Shalit et al., 2017):
TARNet (Shalit et al., 2017) is a neural network that estimates the ITE components
µ i ( x ) from Eq.
48 by learning a shared representation Φ( x ) and two potential outcome heads
h i (Φ( x )) .
We train TARNet by minimizing the loss
[Equation]
where θ =
( θ h 1 , θ h 0 , θ Φ ) denotes the model parameters and L denotes squared loss if Y is continuous or binary cross entropy loss if Y is binary.
Note regarding balanced representations: In (Shalit et al., 2017), the authors propose to add an additional regularization term inspired from domain adaptation literature, which forces TARNet to learn a balanced representation Φ( x ) , i.e., that minimizes the distance the treatment and control group in the feature space.
They showed that this approach leads to minimization of a generalization bound on the ITE estimation error if the representation is invertible.
In our experiments, we refrained from learning balanced representations because minimizing the regularized loss from (Shalit et al., 2017) does not necessarily result in an invertible representation and thus may even harm the estimation performance.
For a detailed discussion, we refer to (Curth & van der Schaar, 2021).
Furthermore, by leaving out the regularization, we ensure comparability between the different baselines.
If balanced representations are desired, the balanced representation approach could also be extended to MRIV-Net, as we also build MRIV-Net on learning shared representations.
DR-learner (Kennedy, 2022):
The DR-learner (Kennedy, 2022) is a meta learner that takes arbitrary estimators of the ITE componenets µ i and the propensity score π ( x )
= P ( A = 1 | X = x ) as input and performs a pseudo outcome regression by using the pseudo outcome
[Equation]
In our experiments, we use TARNet as base method to provide initial estimators ˆ µ i ( X ) .
We further learn propensity score estimates ˆ π ( X ) by adding a seperate representation to TARNet as done in (Shalit et al., 2017).
F. G ENERAL IV METHODS
2SLS (Wright, 1928): 2SLS (Wright, 1928) is a linear two-stage approach.
First, the treatments A are regressed on the instruments Z and fitted values ˆ A are obtained.
In the second stage, the outcome
Y is regressed on ˆ A .
We implement 2SLS using the scikit-learn package.
KIV (Singh et al., 2019): Kernel IV (Singh et al., 2019) generalizes 2SLS to nonlinear settings.
KIV assumes that the data is generated by
[Equation]
) where U is an additive unobserved confounder and f is some unknown (potentially nonlinear) structural function.
KIV then models the structural function via
[Equation]
where ψ and ϕ are feature maps.
Here, kernel ridge regressions instead of linear regressions are used in both stages to estimate µ and V .
Following (Singh et al., 2019) we use the exponential kernel (Rasmussen & Williams, 2008) and set the length scale to the median inter-point distance.
KIV does not provide a direct way to incorporate the observed confounders X .
Hence, we augment both the instrument and the treatment with X , which is consistent with previous work (Bennett et al., 2019; Xu et al., 2021a).
We also use two different samples for each stage as recommended in (Singh et al., 2019).
DFIV (Xu et al., 2021a): DFIV (Xu et al., 2021a) is a similar approach KIV in generalizing 2SLS to nonlinear setting by assuming Eq.
(52) and Eq.
However, instead of using kernel methods, DFIV models the features maps ψ θ A and ϕ θ Z as neural networks with parameters θ A and θ Z , respectively.
DFIV is trained by iteratively updating the parameters θ A and θ Z .
The authors also provide a training algorithm that incorporates observed confounders X , which we implemented for our experiments.
During training,
DeepIV (Hartford et al., 2017): DeepIV (Hartford et al., 2017) also assumes additive unobserved confounding as in Eq.
(52), but leverages the identification result (Newey & Powell, 2003)
[Equation]
where h ( a, x ) =
f
( a, x ) + E
[ U | X = x ] is the target counterfactual prediction function.
DeepIV estimates F ( a | x, z ) , i.e., the conditional distribution function of the treatment A given observed covariates X and instruments Z , by using neural networks.
Because we consider only binary treatments, we simply implement a (tunable) feed-forward neural network with sigmoid activation function.
Then, DeepIV proceeds by learning a second stage neural network to solve the inverse problem defined by Eq.
DeepGMM (Bennett et al., 2019): DeepGMM (Bennett et al., 2019) adopts neural networks for IV estimation inspired by the (optimally weighted)
Generalized Method of Moments.
The DeepGMM estimator is defined as the solution of the following minimax game:
[Equation]
where f ( z i , · ) and g ( a i , · ) are parameterized by neural networks.
As recommended in (Bennett et al., 2019), we solve this optimization via adversarial training with the Optimistic Adam optimizer (Daskalakis et al., 2018), where we set the parameter (cid:101) θ to the previous value of θ .
DMLIV (Syrgkanis et al., 2019): DMLIV (Syrgkanis et al., 2019) assumes that the data is generated via
[Equation]
where τ is the ITE f
some function of the observed covariates.
First, DMLIV estimates the functions q
[Equation]
X ) =
E [ Y | X ] , h ( Z, X ) =
E [ A | Z, X ] , and p ( X ) =
E [ A | X ] .
Then, the ITE is learned by minimizing the loss
[Equation]
[Equation]
where ˆ τ ( X, · ) is some model for τ ( X ) .
In our experiments, we use (tunable) feed-forward neural networks for all estimators.
DRIV (Syrgkanis et al., 2019): DRIV (Syrgkanis et al., 2019) is a meta learner, originally proposed in combination with DMLIV.
It requires initial estimators for q ( X ) , p ( X ) , π ( X ) =
E [ Z | X = x ] , and f ( X ) =
E [ A · Z | X = x ] as well as an initial ITE estimatior ˆ τ init ( X ) (e.g., from DMLIV).
The
ITE is then estimated by a pseudo regression on the following doubly robust pseudo outcome:
[Equation]
We implement all regressions using (tunable) feed-forward neural networks.
Comparison between DRIV vs. MRIV: There are two key differences between our paper and (Syrgkanis et al., 2019): (i)
In contrast to DRIV, we showed that our MRIV is multiply robust.
(ii)
We derive a multiple robust convergence rate, while the rate in (Syrgkanis et al., 2019) is not robust with respect to the nuisance rates.
Ad (i): Both MRIV and DRIV perform a pseudo-outcome regression on the efficient influence function (EIF) of the ATE.
The key difference: DRIV uses the doubly robust parametrization of the
EIF from (Okui et al., 2012), whereas we use the multiply robust parametrization of the EIF from (Wang & Tchetgen Tchetgen, 2018) 4 .
Hence, our MRIV frameworks extends DRIV in a non-trivial way to achieve multiple robustness.
Thus, our estimator is consistent in the union of three different model specifications.
Ad (ii): Here, we compare the convergence rates from DRIV and our MRIV and, thereby, show the strengths of our MRIV.
To this end, let us assume that the pseudo regression function is γ -smooth and that we use the same second-stage estimator ˆE n with minimax rate n − 2 γ 2 γ + p for both DRIV and MRIV.
If the nuisance parameters q ( X ) , p ( X ) , f ( X ) , and π ( X ) are α -smooth and further are estimated with minimax rate n −
α 2 α + p , Corollary 4 from (Syrgkanis et al., 2019) states that DRIV converges with rate
[Equation]
In contrast, MRIV assumes estimation of the nuisance parameters µ Y 0 ( x ) with rate n − 2 α 2 α + p , µ A 0 ( x ) and δ A ( x ) with rate n − 2 β 2 β + p , and π ( x ) with rate n − 2 δ 2 δ + p .
If the initial estimator ˆ τ init ( x ) converges with rate r τ ( n ) , our Theorem 2 yields the rate
[Equation]
If all nuisance parameters converge with the same minimax rate of n − 2 α 2 α + p , the rates of DRIV and our MRIV coincide.
However, different to DRIV, our rate is additionally multiple robust in spirit of
Theorem 1.
This presents a crucial strength of our MRIV over DRIV:
For example, if δ is small (slow convergence of ˆ π ( x ) ), our MRIV still with fast rate as long as α and β are large (i.e., if the other nuisance parameters are sufficiently smooth).
F. W ALD ESTIMATOR
Finally, we consider the Wald estimator (Wald, 1940) for the binary IV setting.
More precisely, we estimate the ITE components µ Yi ( x ) and µ Ai ( x ) seperately and plug them into
[Equation]
We consider two versions of the Wald estimator:
Linear: We use linear regressions to estimate the µ Yi ( x ) and logistic regressions to estimate the µ Ai ( x ) .
BART: We use Bayesian additive regression trees (Chipman et al., 2010) trees to estimate the µ Yi ( x ) and random forest classifier to estimate the µ Ai ( x ) .
G V ISUALIZATION OF PREDICTED ITE S
We plot the predicted ITEs for the different baselines and MRIV-Net in Fig.
8 (for n = 3000 ).
As expected, the linear methods (2SLS and linear Wald) are not flexible enough to provide accurate
ITE estimates.
We also observe that the curve of MRIV-Net without MRIV is quite wiggly,
i
[Figure]
e., the estimator has a relatively large variance.
This variance is reduced when the full MRIV-Net is applied.
As a result, curve is much smoother.
This is reasonable because MRIV does not estimate the ITE components individually
[Figure]
but estimates the ITE directly via the Stage 2 pseudo outcome regression.
Overall, this confirms the superiority of our proposed framework.
[Figure]
H I MPLEMENTATION DETAILS AND HYPERPARAMETER TUNING
Implementation details for deep learning models: To make the performance of the deep learning models comparable, we implemented all feed-forward neural networks (including MRIV-Net) as follows:
We use two hidden layers with RELU activation functions.
We also incorporated a dropout layer for each hidden layer.
We trained all models with the Adam optimizer (Kingma & Ba, 2015) using 100 epochs.
Exceptions are only DFIV and DeepGMM, where we used 200 epochs for training, accounting for slower convergence of the respective (adversarial) training algorithms.
For DeepGMM, we further used Optimistic Adam (Daskalakis et al., 2018) as in the original paper.
Training times: We report the approximate times needed to train the deep learning models on our simulated data with n = 5000 in Table 5.
For training, we used an AMD Ryzen Pro 7 CPU.
Compared to DMLIV and DRIV, the training of MRIV-Net is faster because only a single neural network is trained.
[Table]
Hyperparameter tuning: We performed hyperparameter tuning for all deep learning models (including MRIV-Net), KIV, and the BART Wald estimator on all datasets.
For all methods except
KIV and DFIV, we split the data into a training set (80%) and a validation set (20%).
We then performed 40 random grid search iterations and chose the set of parameters that minimized the respective training loss on the validation set.
In particular, the tuning procedure was the same for all baselines, which ensures that the performance gain of MRIV-Net is due to the method itself and not due to larger flexibility.
Exceptions are only KIV and DFIV, for which we implemented the customized hyperparameter tuning algorithms proposed in (Singh et al., 2019) and (Xu et al., 2021a) to ensure consistency with prior literature.
For the meta learners (DR-learner, DRIV
[Table]
and
MRIV), we first performed hyperparameter tuning for the base methods and nuisance models, before tuning the pseudo outcome regression neural network by using the input from the tuned models.
The tuning ranges for the hyperparameter are shown in Table 6.
These include both the hyperparameter rangers shared across all neural networks and the model-specific hyperparameters.
For reproducibility purposes, we publish the selected hyperparameters in our GitHub project as .yaml files.
[Table]
Hyperparameter robustness checks: We also investigate the robustness of MRIV-Net with respect to hyperparameter choice.
To to this, we fix the optimal hyperparameter constellation for our simulated data for n = 3000 and perturb the hidden layer sizes, learning rate, dropout probability, and batch size.
The results are shown in Fig.
We observe that the RMSE only changes marginally when perturbing the different hyperparameters, indicating that our method is to a certain degree robust against hyperparameter misspecification.
Furthermore, our results indicate that the performance improvement of MRIV-Net over the baselines observed in our experiments is not due to hyperparameter tuning, but to our method itself.
[Figure]
I R ESULTS FOR SEMI - SYNTHETIC DATA
In the main paper, we evaluated MRIV-Net both on synthetic and real-world data.
Here, we provide additional results by constructing a semi-synthetic dataset on the basis of OHIE.
It is common practice in causal inference literature to use semi-synthetic data for evaluation, because it combines advantages of both synthetic and real-world data.
On the one hand, the real-world data part ensures that the data distribution is realistic and matches those in practice.
On the other hand, the counterfactual ground-truth is still available, which makes it possible to measure the performance of ITE methods.
We construct our semi-synthetic data as follows:
First, we extract the covariates X ∈ R 5 and instruments Z ∈ { 0 , 1 } of our OHIE dataset from Sec.
E. Then, we construct the treatment components µ Ai ( x ) via
[Equation]
[Equation]
where X 1 is the (standardized) age and σ ( · ) is the sigmoid function.
The outcome components are constructed via
[Equation]
We then sample treatments A and outcomes Y as in Eq.
(38) and Eq.
Lemma 2 ensures that µ Yi ( X ) =
E [ Y | Z
= i, X ] and µ Ai ( X ) =
E [ A | Z
=
i, X ] .
Given the above, the oracle ITE becomes
[Equation]
Note that τ ( X ) is sparse in the sense that it only depends on age, while the outcome components depend on all five covariates.
Following our theoretical analysis in Sec.
C, MRIV-Net should thus outperform methods that aim at estimating the components directly.
This is confirmed in Table 7, where we show the results for all baselines and MRIV-Net on the semi-synthetic data.
Indeed, we observe that MRIV-Net outperforms all other baselines, confirming both the superiority of our method as well as our theoretical results under sparsity assumptions from Sec.
C.
[Table]
J R ESULTS FOR CROSS - FITTING
Here, we repeat our experiments from the main paper but now make use of cross-fitting .
Recall that, in Theorem 2, we assume that the nuisance parameter estimation and the pseudo-outcome regression are performed on three independent samples.
We now address this through cross-fitting .
To this end, our aim is to show that our proposed MRIV framework is again superior.
For MRIV, we proceeded as follows:
We split the sample D into three equally sized samples D 1 , D 2 , and D 3 .
We then trained ˆ τ init ( x ) , ˆ µ Y 0 ( x ) , and
ˆ µ A
( x ) on D 1 , ˆ δ A ( x ) and ˆ π ( x ) on D 2 , and performed the pseudo-outcome regression on D 3 .
Then, we repeated the same training procedure two times, but performed the pseudo-outcome regression on D 2 and D 1 .
Finally, we averaged the resulting three
ITE estimators.
For DRIV, we implemented the cross-fitting procedure described in (Syrgkanis et al., 2019).
For the DR-learner, we followed (Kennedy, 2022).
The results are in Table 8.
Importantly, the results confirm the effectiveness of our proposed MRIV.
Overall, we find that our proposed MRIV outperforms DRIV for the vast majority of base methods when performing cross-fitting.
Furthermore, MRIV-Net is highly competitive even when comparing it with the cross-fitted estimators.
This shows that our heuristic to learn separate representations instead of performing sample splits works in practice.
In sum, the results confirm empirically that
our
MRIV is superior.
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]