Title: A Non-parametric Direct Learning Approach to Heterogeneous Treatment Effect Estimation under Unmeasured Confounding
Venue: NeurIPS 2024
Accepted: yes
Keywords: direct learning

heterogeneous treatment effect

instrumental variable

multiply robustness

unmeasured confounding
Abstract: In many social, behavioral, and biomedical sciences, treatment effect estimation is a crucial step in understanding the impact of an intervention, policy, or treatment. In recent years, an increasing emphasis has been placed on heterogeneity in treatment effects, leading to the development of various methods for estimating Conditional Average Treatment Effects (CATE). These approaches hinge on a crucial identifying condition of no unmeasured confounding, an assumption that is not always guaranteed in observational studies or randomized control trials with non-compliance. In this paper, we proposed a general framework for estimating CATE with a possible unmeasured confounder using Instrumental Variables. We also construct estimators that exhibit greater efficiency and robustness against various scenarios of model misspecification. The efficacy of the proposed framework is demonstrated through simulation studies and a real data example.

A Non-parametric Direct Learning Approach to Heterogeneous Treatment Effect Estimation under Unmeasured Confounding

Abstract

In many social, behavioral, and biomedical sciences, treatment effect estimation is a crucial step in understanding the impact of an intervention, policy, or treatment. In recent years, an increasing emphasis has been placed on heterogeneity in treatment effects, leading to the development of various methods for estimating Conditional Average Treatment Effects (CATE). These approaches hinge on a crucial identifying condition of no unmeasured confounding, an assumption that is not always guaranteed in observational studies or randomized control trials with non-compliance. In this paper, we proposed a general framework for estimating CATE with a possible unmeasured confounder using Instrumental Variables. We also construct estimators that exhibit greater efficiency and robustness against various scenarios of model misspecification. The efficacy of the proposed framework is demonstrated through simulation studies and a real data example.

1 Introduction

In various domains, different subjects may exhibit different responses to the same set of treatments. The exploration of this heterogeneity in the effects resulting from exposure has gained substantial interest in recent years. For instance, inferring the heterogeneous effect of a medical treatment on clinical outcome can contribute to the development of personalized treatment (Cai et al., 2011). A similar concept has found application in personalized marketing as well (Chandra et al., 2022). The heterogeneity among subjects can be measured by the disparity in conditional mean outcomes given other covariates, typically referred to as the Conditional Average Treatment Effect (CATE). Another problem closely related to the heterogeneity in treatment effects is the optimal Individualized Treatment Regime (ITR), which is a decision rule that selects treatments for individuals to maximize the expected outcome.

There has been significant development in the literature regarding the estimation of CATE and the optimal ITR in the case of no unmeasured confounding. For example, Q-learning (Qian and Murphy, 2011) models the conditional mean outcome under each treatment separately and the estimated CATE is constructed using the difference between the estimated conditional mean outcomes. The success of this method relies on the correct specification of the outcome models. To address this issue, direct learning (DL) (Tian et al., 2014; Qi and Liu, 2018) and robust direct learning (RD) (Meng and Qiao, 2022) models the conditional contrast between treatments directly, which has been shown to be more robust to model misspecification. Another strand of work approaches with tree-based or forest-based methods. Hill (2011) and Green and Kern (2012) extended the Bayesian Additive Regression Tree (BART) method of Chipman et al. (2010) for estimating heterogeneous treatment effect. Athey and Imbens (2016) proposed Causal Trees with an 'honest' splitting approach, wherein the partitioning is

constructed in one sample, and the treatment effects within each node are estimated using another sample. This methodology is subsequently adopted in Causal Forest (Wager and Athey, 2018), which extends the random forest algorithm to estimate heterogeneous treatment effects. On the other hand, optimal ITR estimation aims to determine the optimal decision rule for treatment assignment based on subjects' covariates to maximize the mean outcome. A significant line of work in the field involves transforming ITR estimation into a classification problem through the use of Inverse Probability Weighting (IPW). Notable contributions include Outcome Weighted Learning (OWL) (Zhang et al., 2012; Zhao et al., 2012) and Residual Weighted Learning (RWL) (Zhou et al., 2017).

The aforementioned methods all rely on the key assumption of no unmeasured confounding to identify the heterogeneous treatment effect or the optimal ITR. However, this assumption is in most cases unverifiable (if not untrue) in observational studies or randomized controlled trials (RCT) with noncompliance. A well-known approach that takes into account the unmeasured confounding is the use of an instrumental variable (IV). A proper IV is usually a pre-treatment variable that is independent of any possible unmeasured confounder while correlated with the treatment. For example, in RCT with non-compliance, the random treatment assignment can be considered as an IV while the treatment received is considered the treatment variable. Here these two are clearly correlated since a subject will not receive the treatment if they are not assigned one, though the strength of the correlation may depend on other characteristics such as the education level of the subject.

There is a growing literature on estimating heterogeneous treatment effects or optimal ITR under unmeasured confounding using IV. Imbens and Angrist (1994) identified and estimated the socalled Local Average Treatment Effect (LATE), restricted to the subgroup of the always-compliant population, with the help of an IV. More recently, machine learning methods like Doubly Robust IV (Syrgkanis et al., 2019) and Generalized Random Forest (Athey et al., 2019) have shown their applicability and effectiveness in various settings including unmeasured confounding, particularly when used in conjunction with an IV. Wang and Tchetgen Tchetgen (2018) introduced two alternative assumptions on the unobserved confounders and the IV, which enable the identification of the Average Treatment Effect (ATE). They proposed an estimator that has the so-called multiply robustness property, which guarantees consistent estimate under three observed data models. These findings were incorporated into Cui and Tchetgen Tchetgen (2021) to obtain an optimal ITR estimation while accounting for unmeasured confounding. On the other hand, Frauen and Feuerriegel (2022) utilized these findings for CATE estimation.

In this paper, we propose a new framework for estimating CATE using IV when there exist unmeasured confounders. This framework can be viewed as an extension of the Direct Learning method under unconfoundedness to the case that allows the existence of unmeasured confounding. We call the proposed method Direct Learning using Instrumental Variables (IV-DL). The proposed framework is easy to implement under many flexible learning methods. Additionally, we introduce several efficient and robust estimators by residualizing the outcome. These estimators have been demonstrated to be robust to multiple model misspecification scenarios.

The rest of this paper is organized as follows. The notations and some related preliminaries are introduced in Section 2. The proposed framework IV-DL is formally introduced in Section 3. In Section 4 and 5, we proposed efficient and robust estimators. In Section 6, we conduct simulation studies and compare the performance with existing methods in the literature. A real data example is included in Section 7. Section 8 concludes the paper with a discussion on possible future work. Proofs and additional simulations are provided in the Appendix.

2 Notations and Preliminaries

Denote A ∈ A = { +1 , -1 } as the binary treatment, and X ∈ X ⊆ R p the pre-treatment covariates. We adapt the potential outcome framework (Rubin, 1974) in causal inference and denote by Y ( a ) ∈ R the potential outcome that the subject would have obtained if the received treatment was a ∈ A . The observed outcome is then given by Y = Y ( A ) = Y (1) ✶ [ A = 1] + Y ( -1) ✶ [ A = -1] . Denote by U the unobserved confounder of the effect of A on Y . Suppose we have access to a pre-treatment binary IV denoted by Z ∈ Z = { +1 , -1 } . Then the complete data consists of independent and identically distributed copies of ( Y, X, A, U, Z ) , even though only copies of ( Y, X, A, Z ) are observed.

Our goal is to estimate the Conditional Average Treatment Effect (CATE), defined as ∆( x ) ≜ E [ Y (1) -Y ( -1) | X = x ] . As mentioned in Section 1, most of the prior works are based on the core assumption of no unmeasured confounding:

Assumption 1 (Unconfoundedness) . Y ( a ) ⊥ ⊥ A | X for a = ± 1 .

This assumption essentially implies that the observed covariates X would suffice to account for the confounding of the effect of A on Y , thereby excluding the presence of U . Under the above assumption of unconfoundedness, it can be easily verified that CATE is identified by ∆( x ) = E [ Y | A = 1 , X = x ] -E [ Y | A = -1 , X = x ] . Q-learning (Qian and Murphy, 2011) models the two conditional mean outcomes separately and estimates the CATE by taking the difference between these estimates. Consequently, its effectiveness depends on correctly specifying the models for the conditional mean outcomes. Denote the propensity score for the treatment as π A ( a, x ) = P[ A = a | X = x ] for a = ± 1 . Direct Learning (Qi and Liu, 2018; Tian et al., 2014) propose to directly model for the heterogeneous treatment effect, based on the observation that ∆( x ) = E [ AY/π A ( A,X ) | X = x ] . In other words, one can obtain an estimate of CATE by regressing the modified outcome AY /π A ( A,X ) on X . Robust Direct Learning (RD) Meng and Qiao (2022) further extends this framework by residualizing the outcome using an estimate of the main effect, which is the average of the two conditional mean outcomes. This method demonstrates double robustness in the sense that it yields consistent estimation of CATE if either the propensity score or the main effect is correctly specified. Despite the success in RCT or observational studies, all the methods mentioned above rely on the unconfoundedness Assumption 1. In the next section, we will introduce a general framework that directly models CATE using an IV approach when there exists unmeasured confounding.

3 Direct Learning with Instrumental Variable Approach

In this paper, we look beyond Assumption 1, and consider the existence of an unmeasured confounder U . To establish the identification of CATE in this setting, we approach with the use of a proper IV. We will start with the following assumptions seen in Cui and Tchetgen Tchetgen (2021).

Assumption 2. This assumption consists of five parts as follows:

Here, Y ( z, a ) represents the potential outcome that would be observed if a subject were exposed to treatment a ∈ A , and the IV takes a value of z ∈ Z . Assumption 2.a rules out the existence of any other confounder, except for X and U , for the joint effect of Z and A on the outcome Y . However, this unconfoundedness is hidden from the data collected, since U is never observed. Assumptions 2.b-2.e provides us with a well-defined IV. Assumption 2.b requires a correlation between the IV and the treatment given observed covariates. In many applications, a strong correlation is often necessary to ensure accurate inference in the estimation process. Assumption 2.c guarantees that the causal effect of Z on Y is not confounded given X ; otherwise Z suffers the same issue as A . Additionally, required by Assumption 2.d, the causal effect of Z on Y can only be mediated by the treatment A . In light of this assumption, we omit the argument z in the potential outcome and denote the common value as Y ( a ) . Assumption 2.e implies that each subject has a positive chance of having either value of the IV. An example of the relationships between variables that satisfy Assumption 2 is presented in a directed acyclic graph in Figure 1. In order to identify the CATE, we also need the following assumption on the unmeasured confounder.

Assumption 3. At least one of the following is true:

Z

A

Y

X

U

Assumption 3 states that, conditional on the measured covariates, either the additive effect of Z on A is independent of U , or the additive effect of A on Y is independent of U . Now, we finally have identification of the CATE.

Proposition 1. Under Assumptions 2-3, the CATE can be identified by

where π Z ( z, x ) = P[ Z = z | X = x ] and δ ( x ) = P[ A = 1 | Z = 1 , X = x ] -P[ A = 1 | Z = -1 , X = x ] for any x ∈ X .

The first equality (1) was shown in Wang and Tchetgen Tchetgen (2018), which means that the CATE is identified by the conditional Wald estimand. Equation (2) reveals an interesting observation that we do not need the realized treatment A as long as we have δ ( x ) , which can be viewed as the conditional effect of the IV on the treatment given observed covariates. Hereafter, we denote the conditional means of Y and A by µ Y z ( x ) = E [ Y | Z = z, X = x ] and µ A z ( x ) = E [ A | Z = z, X = x ] , respectively, for any z ∈ {-1 , +1 } and x ∈ X .

3.1 Conditional Average Treatment Effect Estimation

In this section, we will introduce the IV-DL framework. Motivated by Equation (2), the next lemma offers a way to estimate ∆( x ) using inverse propensity score of IV as weight.

Lemma 1. Under Assumptions 2-3,

Based on Lemma 1, we can adopt many existing regression methods to obtain an estimate on CATE by regressing the modified outcome on the covariates, weighted by the propensity score for Z . Specifically, given the data { y i , x i , a i , z i } n i =1 , an estimator ˆ π Z of the propensity score function and an estimator ˆ δ of the effect of Z on A ' the IV-DL estimate for ∆ is given by

where F is a function space with norm ∥·∥ F , and λ ≥ 0 is the tuning parameter for the regularization term ∥ f ∥ F . To obtain ˆ π Z , we can fit a logistic regression of Z on X or a non-parametric model such as random forest. Since δ is the treatment effect of Z on A , it is noteworthy that, under Assumption 3, estimation of δ can be viewed as a CATE estimation problem with unconfoundedness. In this case, A may be viewed as a binary 'outcome' and Z a binary 'treatment'. Thus, we can adopt many existing CATE estimation methods such as Q-learning, DL, and Causal Forest.

The proposed framework allows a variety of learning methods to model the treatment effect ∆( x ) . For example, under the linear model, we may model f ( x ) = ˜ x T β where the regression coefficients

are β and ˜ x i ≜ (1 , x T i ) T . Then IV-DL estimator for β is

and the CATE ∆( x ) is estimated by ˆ f ( x ) = ˜ x T ˆ β .

In high dimensional setting where p is large, sparse regularization can be easily applied here because the optimization is essentially a weighted least square problem. For example, we can use Least Absolute Shrinkage and Selection Operator (LASSO) and the estimator of β is given by

where λ > 0 is the tuning parameter for the l 1 penalty.

In practice, there is no guarantee that the true treatment effect follows a linear model. For a more complex model, we can adopt nonlinear methods such as Kernel Ridge Regression (KRR) and solve

where K i is the i -th column of the kernel matrix K = ( K ( x i , x j )) n × n and K ( · , · ) is a kernel function. KRR might be computationally expensive when dealing with large datasets. In such cases, other machine learning methods capable of solving a weighted least squares problem can be considered. Examples include local regression, regression trees, random forests, and neural networks.

3.2 Optimal Individualized Treatment Regime Estimation

In some domains, the optimal Individualized Treatment Regime (ITR) can be of interest. The goal here is to find a mapping d : X → A from a specific class D to maximizes the expected outcome: d ∗ ≜ argmax d ∈D E [ Y ( d ( X ))] , where Y ( d ( X )) is the potential outcome that the subject X obtained after receiving treatment d ( X ) , and E [ Y ( d ( X ))] is also known as the Value of the regime d .

ITR and CATE are closely related. For example, in the binary treatment setting, the CATE ∆ is the difference between two conditional mean outcomes. Assuming greater values of outcome is preferred, then the sign of ∆ will determine which treatment is optimal. It can be verified that d ∗ ( x ) = sign (∆( x )) . Therefore, we define the estimated optimal ITR using IV-DL as ˆ d ( x ) = sign ( ˆ f ( x )) , where ˆ f ( x ) may be any CATE estimator introduced in the last subsection.

4 Efficient Estimators by Residualization

In the literature, considerable advancements have been made to enhance the efficiency and robustness of the CATE and optimal ITR estimation. To this end, residualization and augmentation are two common strategies. For example, in the IPW framework for optimal ITR estimation, Zhou et al. (2017) and Zhou and Kosorok (2017) proposed to replace the outcome by its residual Y -ˆ g ( x ) in estimation of the optimal regime, where ˆ g ( x ) is an estimate of the weighted average of the conditional mean outcomes. For the estimation of CATE, Meng and Qiao (2022) residualized the outcome by an estimate of the average of conditional mean outcomes. Frauen and Feuerriegel (2022) proposed augmenting a preliminary estimate of CATE to enhance the robustness of the estimator.

In this section, we present the Robust Direct Learning using IV approach (IV-RDL), which involves residualizing the outcome in IV-DL to enhance both efficiency and robustness. We propose two ways of residualization, referred to as IV-RDL1 and IV-RDL2, respectively. They are shown to reduce the variance when estimating CATE. In Section 5, we show that they have robustness properties when confronted with model misspecification for nuisance variables.

4.1 Residualization using a Function of Covariates

We first consider residualizing the outcome by a function of the observed covariates only. Ideally, we would like to find a function g : X → R that can improve the efficiency of the estimation on CATE,

while keeping it consistent. As shown in the following lemma, the consistency of the estimator is in fact preserved under a shift of Y by any function of the observed covariates.

Lemma 2. For any measurable g : X → R and any probability distribution for ( Y, X, A, Z )

Asymptotically, the variance of the estimator is related to the variance of the derivative of [ π Z ( Z, X )] -1 [2( Y -g ( X )) Z ) Zδ -1 ( X ) -f ( X )] 2 , the weighted loss for each individual. Hence, it is natural to choose g that minimize the variance of [ π Z ( Z, X )] -1 [2( Y -g ( X )) Zδ -1 ( X ) -f ( X )] . See Appendix B for a more detailed discussion using the linear model as an example. The following theorem gives us the minimizer.

Theorem 1. Among all measurable g : X → R , the following function minimize the variance of 1 π Z ( Z,X ) ( 2( Y -g ( X )) Z δ ( X ) -f ( X ) ) :

There is an interesting interpretation of the optimal function g ∗ , which equals the average of µ Y 1 ( x ) and µ Y -1 ( x ) . Recall that Eq. (1) states that CATE under unmeasured confounding is identified by the ratio of two contrasts, where the numerator happens to be µ Y 1 ( x ) -µ Y -1 ( x ) . The residualization strategy amounts to shifting the outcome Y , and hence µ Y 1 ( x ) and µ Y -1 ( x ) as well. Naturally, shifting both by their average will not affect their difference, but it will reduce the variance. A similar residualization was incorporated in RD under unconfoundedness (Meng and Qiao, 2022), where the goal was to learn the contrast between conditional mean outcomes given the two treatments.

In practice, g ∗ needs to be estimated before we can estimate the CATE. There are several approaches to obtain the estimate of g ∗ , denoted by ˆ g ∗ . For example, we can take the average of estimated conditional mean outcomes, i.e., ˆ g ∗ ( x ) = (ˆ µ Y 1 ( x )+ˆ µ Y -1 ( x )) / 2 . One can also regress Y/ (2 π Z ( Z, X )) on X , inspired by Eq. (3). Given ˆ g ∗ ( x ) , the IV-RDL1 estimator for ∆ is obtained by

In Section 5, we will show that this estimator is robust against misspecification of either g ∗ or π Z , given that δ is correctly specified.

4.2 Residualization using Covariates, Treatment, and IV

In this paper, we also consider an alternative way of residualizing the outcome by a function h : ( X , A , Z ) → R . Like IV-RDL1, the optimal choice is the function that minimizes the variance while maintaining the consistency of CATE estimation. Among all functions that still convey consistent CATE estimation, the following three equivalent functions minimize the variance of [ π Z ( Z, X )] -1 [2( Y -h ( X,A,Z )) Zδ -1 ( X ) -f ( X )] .

where m Y ( x ) ≜ ( µ Y 1 ( x ) + µ Y -1 ( x )) / 2 and m A ( x ) ≜ ( µ A 1 ( x ) + µ A -1 ( x )) / 2 . The technical details are provided in the Appendix C. In practice, all these conditional means ( µ Y -1 , µ Y 1 , µ A -1 and µ A 1 ) need to be estimated, together with estimations of π Z and δ . Additionally, we need to obtain a preliminary estimate of CATE. The IV-RDL2 estimator is constructed by,

where ˆ h ∗ is an estimator for one of h ∗ 1 , h ∗ 2 and h ∗ 3 .

5 Robustness Properties

In this section, we investigate the robustness properties of IV-RDL1 and IV-RDL2. We start with the following theorem to demonstrate the double robustness property of the IV-RDL1 that residualizes the outcome by using g ( x ) .

Theorem 2. Suppose Assumption 2-3 holds, and we have a consistent estimator of δ , denoted by ˆ δ . Let ˜ π Z be a working model for π Z , and ˜ g be a working model for g ∗ . Then we have

if either ˜ π Z ( z, x ) = π Z ( z, x ) or ˜ g ( x ) = g ∗ 1 ( x ) almost surely.

Theorem 2 indicates that we will have a doubly robust estimator for ∆ if either π Z or g ∗ is correctly specified when δ is known or correctly specified. However, the requirement of a consistent estimate of δ would not pose a significant issue in practical application, since it is essentially a CATE estimation problem under no unmeasured confounding. A consistent estimator for δ can be found by implementing any state-of-the-art CATE estimation method in the literature.

For the IV-RDL2, there are more nuisance variables that need to be estimated. The next theorem shows that IV-RDL2 is robust to various scenarios of misspecified nuisance variables.

Theorem 3. Suppose Assumption 2-3 holds. Let ˜ π Z , ˜ δ , ˜ µ Y 1 , ˜ µ Y -1 , ˜ µ A Z , ˜ µ A -1 , ˜ m Y , ˜ m A and ˜ ∆ be working models for π Z , δ , µ Y 1 , µ Y -1 , µ A Z , µ A -1 , m Y , m A and ∆ , respectively. Denote ˜ h 1 , ˜ h 2 and ˜ h 3 as chosen augmentation formulated according to h ∗ 1 , h ∗ 2 and h ∗ 3 using working estimates. Then we have

if any one of the following condition is satisfied: (1) ˜ π Z = π Z and ˜ ∆ = ∆ almost surely, and ˜ h can be any one of ˜ h 1 , ˜ h 2 and ˜ h 3 . (2) ˜ π Z = π Z and ˜ δ = δ almost surely, and ˜ h can be any one of ˜ h 1 , ˜ h 2 and ˜ h 3 . (3) ˜ µ Y 1 = µ Y 1 , ˜ µ a 1 = µ A 1 and ˜ ∆ = ∆ almost surely, and ˜ h = ˜ h 1 . (4) ˜ µ Y -1 = µ Y -1 , ˜ µ a -1 = µ A -1 and ˜ ∆ = ∆ almost surely, and ˜ h = ˜ h 2 . (5) ˜ m Y = m Y , m A = m A and ˜ ∆ = ∆ almost surely, and ˜ h = ˜ h 3 . (6) ˜ m Y = m Y , m A = m A and ˜ δ = δ almost surely, and ˜ h = ˜ h 3 .

Theorem 3 summarizes in total six cases of the minimal combination of correctly specified nuisance variables in order to have a consistent estimate of CATE. The three choices of residualization functions possess robustness against different scenarios. In the first two scenarios, obtaining a consistent estimate of CATE is guaranteed as long as we correctly specify π Z and either ∆ or δ . This consistency holds irrespective of the choice of the three ˜ h functions. In practice, the second scenario may be particularly accessible, especially when π Z is known. The other scenarios are less likely to be verified in practice and therefore requires more domain knowledge of the data structure. Specifically, scenarios (3)-(5) requires the corresponding set of conditional means to be correctly specified as well as the preliminary ∆ . Lastly, in scenario (6), when δ and the averages of conditional means are correctly specified, the IV-RDL2 will also provide a consistent estimate of CATE.

While working on this paper, we encountered unpublished work by Frauen and Feuerriegel (2022) that is similar to our IV-RDL2 estimator. Inspired by Wang and Tchetgen Tchetgen (2018), Frauen and Feuerriegel introduced the MRIV framework, which is a two-step process. First, a preliminary estimator of CATE and nuisance estimators of δ , π Z , µ Y -1 and µ A -1 are obtained. Then, a pseudooutcome is created by augmenting the preliminary CATE with the nuisance estimates, and the final CATE estimator is obtained by regressing the pseudo-outcome on the covariates. As shown in Wang and Tchetgen Tchetgen (2018), this estimator is robust against model misspecification of the nuisance variables in three of the six scenarios in Theorem 3 (scenarios (1), (2), and (4)). Our numerical studies have shown that our proposed IV-DL framework performs better than the MRIV method.

6 Simulation Study

In this section, we present the results of the simulation study conducted to assess the performance of the proposed IV-DL framework. We compared the proposed method with Bayesian additive regression trees (BART; Chipman et al., 2010), robust direct learning (RD; Meng and Qiao, 2022), causal forest with IV approach (CF; Athey et al., 2019), MRIV method (Frauen and Feuerriegel, 2022), and weighted learning with IV approach (IPW-MR; Cui and Tchetgen Tchetgen, 2021).

6.1 Simulation Settings

We begin by introducing the data-generating mechanism. The covariates, denoted as X = ( X 1 , X 2 , X 3 , X 4 , X 5 ) , were generated from uniform distribution with X i ∼ Unif ( -1 , 1) for i = 1 , . . . , 5 . We followed Cui and Tchetgen Tchetgen (2021) and generated the treatment A under logistic model with probability for success: P ( A = 1 | X,Z,U ) = expit { 2 X 1 +2 . 5 Z -0 . 5 U } , where the instrumental variable Z was a Bernoulli random variable with probability 1 / 2 and U was the unobserved confounder that followed Bridge distribution with parameter ϕ = 1 / 2 . By the results from Wang and Louis (2003), the above usage of Bridge distribution will guarantee that the marginal distribution f ( A | X,Z ) can be modeled directly by logistic regression. In other words, there exists some vector α such that logit { P ( A = 1 | X,Z ) } = α T (1 , X, Z ) .

The outcome Y was generated in two different settings corresponding to linear and non-linear models of the true CATE:

where the error term ϵ follows N (0 , 1) . Functions h ( X ) and q ( X ) are defined as follows:

In Setting 1, the true CATE is 2 q ( x ) , which is linear in x . In Setting 2, the true CATE is 2( exp ( q ( x )) -1) , which is nonlinear. The sample size for each setting was 500 and the simulation was repeated 100 times. An independent sample of size 5000 was used to evaluate the performance of different methods. The proposed methods were implemented according to Sections 3 and 4 with ˆ δ ( X ) estimated by causal forest ('grf' package) and the other nuisance variables estimated by random forest. For methods that require to estimate the same nuisance variable, they shared the same copies of nuisance estimates.

6.2 Numerical Results

We compared all methods based on three performance metrics in the testing sample: the correct classification rate by the estimated ITR (AR); the value function evaluated at the estimated ITR (Value); the mean squared error of the estimated CATE (MSE). Table 1 reports the mean and standard error of these three evaluation metrics over 100 replications for different methods in the two settings.

Table 1: Simulation results: mean × 10 -2 (SE × 10 -2 ). IPW-MR: the multiply robust weighted learning; BART: Bayesian additive regression trees; RD: robust direct learning; CF: causal forest. The empirical maximum value is 0.998 for setting 1 and 1.01 for setting 2.

Among the methods implemented, BART and RD rely on the unconfoundedness assumption and therefore fail to identify CATE when there is unobserved confounding. Both IPW-MR and CF make use of the IV to take unmeasured confounding into account. IPW-MR had fine performances on estimating ITR and maximizing the value. However, it was not designed to estimate CATE. CF performs slightly worse than IPW-MR in terms of AR and value in Setting 1, despite offering a CATE estimation. Its performance is more competitive compared to IPW-MR in Setting 2. Our proposed methods showed superior performances on all the metrics. In particular, IV-RDL1, which residualized the outcome using averages of the estimated conditional means, outperformed all the methods in both settings. IV-RDL2 had a more complicated residualization, and achieved the secondbest performance (but still fairly close to IV-RDL1). Even the unresidualized IV-DL performed better than other methods in most of the metrics. Additional simulation results on testing the robustness of the proposed framework is reported in Appendix D.

7 Data Analysis

In this section, following Angrist and Evans (1998), we study the causal effect of child-rearing on a mother's labor-force participation, using a sample of married mothers with two or more children from the U.S. 1980 census data (80PUMS). Assuming the sex of children is random, 'first two children mixed sex or not' becomes a suitable instrumental variable for the causal effect of having a third child on a mother's labor force participation. Angrist and Evans showed that having a third child reduces women's labor force participation on average. Our goal is to investigate heterogeneity among families, offering personalized insights on the decision to have a third child and its impact on employment opportunities. We used a dataset of 478,005 subjects with at least two children. The outcome, Y , represents whether the mother was employed in the year preceding the census. The treatment, A , indicates whether the mother had three or more children at the census time, and the instrumental variable, Z , indicates whether the first two children were of the same sex. We considered five covariates, X : mother's age at first birth, age at census time, years of education, race, and the father's income.

-0.22

100%

yesAGEM<33-no

-0.3

53%

AGEMFB<23-

-0.38

38%

INCOMED>=2.1-

-0.4

36%

INCOMED<26

-0.12

-0.043

-0.12

-0.11

32%

4%

3%

15%

47%

40000-

sdnoubqns

count

5

20000-

0

Estimated CATE

We used the random forest algorithm for both the implementation of the proposed method and the estimation of the nuisance variables µ Y z , µ A z , δ , and π Z . The preliminary CATE estimator was formulated according to Eq. 1 with plug-in estimates on the conditional means. To identify subgroups with distinct treatment effects, we used the estimated CATE as the response to construct a regression tree, shown in Figure 2. The splits occurred at the mother's age at census (33), age at first birth (23), and father's income ($2.1k/year and $26k/year). By investigating the five subgroups (32%, 4%, 3%, 15%, and 47% of the sample), labeled as groups 1-5, we have made the following observations. First, older mothers are more likely to work after having a third child (subgroups 4 and 5 show a larger estimated treatment effect). Second, younger mothers with very low-income fathers (subgroup 3) tend to stay in the labor force after the third child. Lastly, younger mothers are more likely to stop working if their husband's income is between $2.1k and $26k/year (subgroups 1-3). Figure 3 displays the histogram of estimated CATE for the three majority groups (1, 4, and 5). The estimated CATE for group 1 is overall smaller than for groups 4 and 5. We also constructed 3-dimensional scatter

plots based on the three splitting variables for a more detailed look at the heterogeneity (shown in Appendix E).

8 Conclusions

In this paper, we proposed a new framework to estimate CATE under unmeasured confounding by using an instrumental variable. Under the proposed framework, the estimation procedure boils down to solving a weighted least square problem, which can be tackled with any modern statistical or machine learning method. We also constructed two robust estimators by residualizing the outcome, which are shown to be more efficient and robust to model misspecification on nuisance variables. Numerical studies have shown very competitive performance for our proposed methods.

A potential extension of our work involves using IV to estimate treatment effects for multi-arm and continuous treatments, with the challenge lying in the generalization of Assumption 3. Another avenue is to incorporate deep neural networks to make use of their rich expressiveness for data distribution. However, the empirical performance and theoretical properties need to be formally studied. One notable limitation is the issue of extreme weights, which can arise during the estimation process and potentially lead to instability and biased results. Addressing this limitation is crucial for improving the reliability and accuracy of our method.

References

Angrist, J. D. and Evans, W. N. (1998), 'Children and Their Parents' Labor Supply: Evidence from Exogenous Variation in Family Size,' The American Economic Review , 88, 450-477.

Athey, S. and Imbens, G. (2016), 'Recursive partitioning for heterogeneous causal effects,' Proceedings of the National Academy of Sciences , 113, 7353-7360.

Athey, S., Tibshirani, J., and Wager, S. (2019), 'Generalized Random Forests,' The Annals of Statistics , 47, 1148-1178.

Cai, T., Tian, L., Wong, P. H., and Wei, L. (2011), 'Analysis of randomized comparative clinical trial data for personalized treatment selections,' Biostatistics , 12, 270-282.

Chandra, S., Verma, S., Lim, W. M., Kumar, S., and Donthu, N. (2022), 'Personalization in personalized marketing: Trends and ways forward,' Psychology & Marketing , 39, 1529-1562.

Chipman, H. A., George, E. I., and McCulloch, R. E. (2010), 'BART: Bayesian additive regression trees,' The Annals of Applied Statistics , 4, 266 - 298.

Cui, Y. and Tchetgen Tchetgen, E. (2021), 'A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity,' Journal of the American Statistical Association , 116, 162-173.

Frauen, D. and Feuerriegel, S. (2022), 'Estimating individual treatment effects under unobserved confounding using binary instruments,' arXiv preprint arXiv:2208.08544 .

Green, D. P. and Kern, H. L. (2012), 'Modeling heterogeneous treatment effects in survey experiments with Bayesian additive regression trees,' Public opinion quarterly , 76, 491-511.

Hill, J. L. (2011), 'Bayesian nonparametric modeling for causal inference,' Journal of Computational and Graphical Statistics , 20, 217-240.

Imbens, G. W. and Angrist, J. D. (1994), 'Identification and Estimation of Local Average Treatment Effects,' Econometrica , 62, 467-475.

Meng, H. and Qiao, X. (2022), 'Augmented direct learning for conditional average treatment effect estimation with double robustness,' Electronic Journal of Statistics , 16, 3523-3560.

Qi, Z. and Liu, Y. (2018), 'D-learning to estimate optimal individual treatment rules,' Electronic Journal of Statistics , 12, 3601-3638.

Qian, M. and Murphy, S. A. (2011), 'Performance guarantees for individualized treatment rules,' Annals of statistics , 39, 1180.

Rubin, D. B. (1974), 'Estimating causal effects of treatments in randomized and nonrandomized studies.' Journal of educational Psychology , 66, 688.

Syrgkanis, V., Lei, V., Oprescu, M., Hei, M., Battocchi, K., and Lewis, G. (2019), 'Machine learning estimation of heterogeneous treatment effects with instruments,' Advances in Neural Information Processing Systems , 32.

Tian, L., Alizadeh, A. A., Gentles, A. J., and Tibshirani, R. (2014), 'A simple method for estimating interactions between a treatment and a large number of covariates,' Journal of the American Statistical Association , 109, 1517-1532.

Wager, S. and Athey, S. (2018), 'Estimation and inference of heterogeneous treatment effects using random forests,' Journal of the American Statistical Association , 113, 1228-1242.

Wang, L. and Tchetgen Tchetgen, E. (2018), 'Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables,' Journal of the Royal Statistical Society Series B: Statistical Methodology , 80, 531-550.

Wang, Z. and Louis, T. A. (2003), 'Matching conditional and marginal shapes in binary random intercept models using a bridge distribution function,' Biometrika , 90, 765-775.

Zhang, B., Tsiatis, A. A., Davidian, M., Zhang, M., and Laber, E. (2012), 'Estimating optimal treatment regimes from a classification perspective,' Stat , 1, 103-114.

Zhao, Y., Zeng, D., Rush, A. J., and Kosorok, M. R. (2012), 'Estimating individualized treatment rules using outcome weighted learning,' Journal of the American Statistical Association , 107, 1106-1118.

Zhou, X. and Kosorok, M. R. (2017), 'Augmented outcome-weighted learning for optimal treatment regimes,' arXiv preprint arXiv:1711.10654 .

Zhou, X., Mayer-Hamblett, N., Khan, U., and Kosorok, M. R. (2017), 'Residual weighted learning for estimating individualized treatment rules,' Journal of the American Statistical Association , 112, 169-187.

A Proofs

Proof of Proposition 1. For any z ∈ {-1 , +1 } , we have

Evaluate the above equality at z = 1 and z = -1 , and take the difference. Then we have

Based on Assumption 3, we have

Combining the above two, we have

The above equality is equivalent to Equation (1). On the other hand, for any x ∈ X ,

=

E

[

Y

|

Z

= 1

, X

=

x

]

-

E

[

Y

|

Z

=

-

1

, X

=

x

]

Dividing both sides of the above equality by δ ( x ) = P[ A | Z = 1 , X = x ] -P[ A | Z = -1 , X = x ] yields Equation (2).

Lemma 3. Let ℓ ( X,f ) = E [ Q ( X,f ) | X ] and L ( f ) = E ℓ ( X,f ) . Denote f ∗ ∈ argmin f ℓ ( X,f ) . Then f ∗ ∈ argmin f L ( f ) .

Proof. Denote f + ∈ argmin f L ( f ) . Then by definition, we have the following two inequalities:

Therefore, L ( f ) = L ( f ) and f ∈ argmin L ( f ) .

Proof of Lemma 1. Let ℓ ( X,f ) = E [ 1 π Z ( Z,X ) ( 2 Y Z δ ( X ) -f ( X ) ) 2 ∣ ∣ ∣ ∣ X ] . By Lemma 3, it suffices to show ∆ ∈ argmin f ℓ ( X,f ) .

The gradient of ℓ ( X,f ) with respect to f is given by

Since ℓ ( X,f ) is convex, we have ∆ ∈ argmin f ℓ ( X,f ) .

Proof of Lemma 2. Let ℓ g ( f ) = E [ 1 π Z ( Z,X ) ( 2( Y -g ( X )) Z δ ( X ) -f ( X ) ) 2 ∣ ∣ ∣ ∣ X ] . By Lemma 3, it suffices to show that for any g , argmin f ℓ g ( X,f ) = argmin f ℓ ( X,f ) . For any g ( x ) ,

Here the second term and the fourth term don't depend on f , and the third term is 0 because E [ Z π Z ( Z,X ) ∣ ∣ ∣ X = x ] = 0 . Therefore, argmin f L g ( f ) = argmin f L ( f ) .

Proof of Theorem 1. For any g ( x ) , the variance of the derivative of the weighted loss at f = ∆ is given by

Set the gradient of S with respect to g equal to 0. Then for any x ∈ X , we have

Additionally, S is convex since ∂ 2 S ∂g 2 = 8 ( δ ( x )) 2 π Z (1 ,x ) π Z ( -1 ,x ) . By Lemma 3, g ∗ ∈ argmin g E [ S ( X,g )] .

Proof of Theorem 2. Let ˜ ℓ g ( X,f ) = E [ 1 ˜ π Z ( Z,X ) ( 2( Y -˜ g ( X )) Z ˆ δ ( X ) -f ( X ) ) 2 ∣ ∣ ∣ X ] . By Lemma 3, it suffices to show ∆ ∈ argmin f ˜ ℓ g ( X,f ) , if either ˜ π Z = π Z almost surely or ˜ g = g almost surely.

The gradient of ˜ ℓ g ( x, f ) with respect to f is given by

If ˜ π Z = π Z almost surely, then

If ˜ g = g almost surely, then ˜ g ( x ) = [ µ Y 1 ( x ) + µ Y -1 ( x )] / 2 . Thus, we have

Check that π Z (1 ,x ) ˜ π Z (1 ,x ) + π Z ( -1 ,x ) ˜ π Z ( -1 ,x ) > 0 . By convexity of ˜ ℓ g ( x, f ) , ∆ ∈ argmin f ˜ ℓ g ( X,f ) in both cases.

Proof of Theorem 3. Let ˜ ℓ h ( X,f ) = [ 1 ˜ π Z ( Z,X ) ( 2( Y -˜ h ( X,A,Z )) Z ˜ δ ( X ) -f ( X ) ) 2 ∣ ∣ ∣ X ] . By convexity of ˜ ℓ h and Lemma 3, it suffices to show that the gradient of ˜ ℓ h ( x, f ) with respect to f is 0 at f = ∆ in all cases. The gradient is given by

If ˜ π Z = π Z almost surely, then we have the unweighted difference E [ ˜ h ( X,A,Z ) | Z = 1 , X = x ] -E [ ˜ h ( X,A,Z ) | Z = -1 , X = x ] = 0 by Equation (4). The resulting gradient is

It will yield 4( f ( x ) -∆( x )) if either ˜ ∆ = ∆ or ˜ δ = δ almost surely.

If ˜ µ Y 1 = µ Y 1 , ˜ µ A 1 = µ A 1 and ˜ ∆ = ∆ almost surely, and the choice of residualization function is ˜ h 1 , then we have

If ˜ µ Y -1 = µ Y -1 , ˜ µ A -1 = µ A -1 and ˜ ∆ = ∆ almost surely, and the choice of residualization function is ˜ h 2 , then we have

If (˜ µ Y 1 +˜ µ Y -1 ) / 2 = ( µ Y 1 + µ Y -1 ) / 2 and (˜ µ A 1 +˜ µ A -1 ) / 2 = ( µ A 1 + µ A -1 ) / 2 almost surely, and the choice of residualization function is ˜ h 3 , then the gradient is

It will yield 2 ( f ( x ) -∆( x )) )( π Z (1 ,x ) ˜ π Z (1 ,x ) + π Z ( -1 ,x ) ˜ π Z ( -1 ,x ) ) if either ˜ ∆ = ∆ or ˜ δ = δ almost surely.

B Optimal residualization (linear model example)

Consider linear model for ∆( x ) with coefficients denoted by β . The objective function with outcome residualized by a function g ( x ) is defined as follows:

Let β ∗ be the unique minimizer of Q ( β ) ≜ E [ L g ( Y, Z, X, β )] . Let ℓ g ( y, z, x, β ) be the derivative of L g ( y, z, x, β ) with respect to β . Denote by ˆ β the root of the estimating equation n -1 ∑ n i =1 ℓ g ( Y i , X i , β ) = 0 . By Bahadur representation, we have

where H is the second derivative of Q ( β ) with respect to β at β = β ∗ . Therefore, selecting the optimal g is equivalent to minimizing the variance of ℓ g ( Y i , X i , β ∗ ) .

C Technical details for IV-RDL2

Unlike IV-RDL1, we need additional constraints on h ( x, a, z ) to make sure the estimation for CATE remains consistent after the residualization.

Lemma 4. For any measurable h : ( X , A , Z ) → R satisfying

or equivalently, E [ h ( X,A,Z ) | Z = 1 , X = x ] = E [ h ( X,A,Z ) | Z = -1 , X = x ] , we have

Proof of Lemma 4. Let ℓ h ( X,f ) = E [ 1 π Z ( Z,X ) ( 2( Y -h ( X,A,Z )) Z δ ( X ) -f ( X ) ) 2 ∣ ∣ ∣ ∣ X ] . By Lemma 3, it suffices to show argmin f ℓ h ( X,f ) = argmin f ℓ ( X,f ) . For any h ( x, a, z ) satisfying Equation (4), we have

Here the second term and the fourth term don't depend on f , and the third term is 0. Therefore, argmin f ℓ h ( X,f ) = argmin f ℓ ( X,f )

As shown in Lemma 4, the minimizer is invariant of a shift on outcome by a function h that satisfies Eq. (4). Similar to the way of finding ˆ g ∗ , we would like to find the function h with the smallest variance of [ π Z ( Z, X )] -1 [2( Y -h ( X,A,Z )) Zδ -1 ( X ) -f ( X )] among all h that satisfies Eq. (4).

Theorem 4. Among all measurable h : ( X , A, Z ) → R satisfying Eq. (4) , the following function minimizes Var [ 1 π Z ( Z,X ) ( 2( Y -h ( X,A,Z )) Z δ ( X ) -f ( X ) )] :

if the conditional means µ Y ( x ) and µ A ( x ) is one of these three pairs: (1) µ Y 1 ( x ) and µ A 1 ( x ) ; (2) µ Y -1 ( x ) and µ A -1 ( x ) ; (3) m Y ( x ) ≜ ( µ Y 1 ( x ) + µ Y -1 ( x )) / 2 and m A ( x ) ≜ ( µ A 1 ( x ) + µ A -1 ( x )) / 2 .

Proof of Theorem 4. For any h ( x, a, z ) satisfying Equation (4), the variance of the derivative of the weighted loss L h ( f ) at f = ∆ is given by

where

Now we seek to minimize E [ S ( X,Z,h )] . By convexity of S ( X,Z,h ) and Lemma 3, it suffices to show that the gradient of S ( X,Z,h ) with respect to h is 0 at f = ∆ , if h is one of the three equivalent forms. To this end, set the gradient of S with respect to h to be 0. Then for any ( x, z ) ∈ ( X , Z ) , we have

which leads to the following condition on the optimal h :

Since δ ( x )∆( x ) = µ Y 1 ( x ) -µ Y -1 ( x ) , it can be verified that Equation (5) implies E [ h ( X,A,Z ) | Z = 1 , X = x ] = E [ h ( X,A,Z ) | Z = -1 , X = x ] , which is equivalent to Equation (4). We will then verify that the following three equivalent functions satisfy Equation (5).

To see their equivalence, notice that ∆( x ) = 2[ µ Y 1 ( x ) -µ Y -1 ( x )] / [ µ A 1 ( x ) -µ A -1 ( x )] . Then we have

and h ∗ 3 is simply the average of h ∗ 1 and h ∗ 2 . It suffices to show h ∗ 1 satisfies (5). We have

which completes the proof.

D Additional Simulations

In this section, we conducted simulations that evaluate the performance of the proposed framework against model mispecification on the nuisance variables. The data is generated by the same model as Setting 1 in Section 6, except that π Z (1 , X ) = expit { 2 X 1 } . Based on the true model, the conditional mean outcome is non-linear on X . However, in Setting 3, we will use its OLS estimate as a case of misspecification. In Setting 4, we deliberately used a wrong propensity ˆ π Z (1 , x ) = 1 / 2 . We keep all the other procedures the same as Setting 1. The results are summarized in Table D. We can observe that the residualized version have superior performance, and have significant lower MSE compared to the original version.

E 3D plots for the data analysis

In the data analysis, we construct a 3-dimensional plot for the estimated CATE based on the three splitting variables (age of mom at census, age of mom at first birth, and income of father). The plot is presented in two rotations in Figure 4. The points in the plots are color-coded by the estimated CATE with red indicating more likely to work and blue indicating more likely to not work. We can see that, overall, blue points are at the bottom of the plots, with a majority of them below $25k/year. Subgroup 3 of young mothers with extremely low fathers' income only accounts for 3% of the data and hence is hard to see here.

Estimated CATE

120

100

Income of Dad

100

1

0.5

Income of Dad

80

80

60

0

60

40

0.5

20

40

1.5

40

35

20

30

atFirst Birth

25

50

55

2

5

20

Age of Mom at First Birth

25

Age of Mom

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: They provide a summary of the research objectives, methodologies, and findings, which are consistently supported by the detailed content of the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We included discussion on the accessibility of the assumptions in practice.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The proof for each theoretical result can be found in Appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: The code provided in the supplemental materials can be used to reproduce results in both simulation study and real data analysis.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code is included in the supplementary materials, with instructions to reproduce the results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: All steps for data generation, model fitting, evaluation, etc., are discussed in the paper. Full details provided with the code.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: For each evaluation metric, we provided standard error as a measure of the error bar.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [No]

Justification: Each experiment takes a few hours on an old Macbook. Detailed information on the computer resources and an estimate of time needed is included in the code instructions.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conform with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The paper proposed an estimation framework and technical details within a specific field, without offering practical applications or solutions that could be implemented in broader societal contexts.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: Our data analysis used public data.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification:

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
In this paper, the authors proposed a general framework for estimating CATE with a possible unmeasured confounder using instrumental variables. They construct estimators that exhibit efficiency and robustness against various scenarios of model misspecification. The efficacy of the proposed framework is demonstrated through simulation studies and a real data example.

STRENGTHS:
Strengths: The robustness and efficacy of the proposed method are shown in theory and simulations.

WEAKNESSES:
Please refer to Questions.

QUESTIONS:
The authors might consider scenarios where one might be interested in heterogeneous treatment effect on $X'$, where $X'$ is a subset of $X$.

LIMITATIONS:
Please refer to Questions.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe author mainly introduces a method of Directly Learning using Instrumental Variables (IV-DL) to estimate the conditional average treatment effect (CATE) $\\Delta(x)$ and optimal Individualized Treatment Regime (ITR) $\\hat{d(x)}$ in the presence of unobserved confounding. They propose two efficient and robust estimators, IV-RDL1 and IV-RDL2, by residualizing the outcome. The authors conduct two simulation settings and use real-world data to demonstrate the efficiency of the approach.\n\nSTRENGTHS:\nThis paper primarily focuses on estimating conditional average treatment effects and determining the optimal individualized treatment regime using the Direct Learning with Instrumental Variable Approach. It presents a well-structured logical framework to discuss this concept.\n\nWEAKNESSES:\nI think overall the authors did interesting research, but my main concerns are listed below.\n\nThe variables A, Y, and Z in this paper are all binary variables. It would be beneficial to discuss how the framework of IV-DL can be extended or adapted to handle a continuous instrumental variable 𝑍. The current focus on binary variables may limit the generalizability of the findings.\n\nIn Section 5, the authors do not provide a detailed introduction to work similar to IV-RDL2. A more thorough explanation of the differences and connections between this work and related research would enhance the reader's understanding of the unique contributions and context of the presented study.\n\nQUESTIONS:\n1. In section 6, why is the performance of IV-RDL1 than IV-RDL2 in three metrics?\n\n2. The authors should conduct some experiments to show whether IV-RDL1 and IV-RDL-2 are robust compared to other methods in the presence of misestimation.\n\n3. The authors miswrite “$f(x)=\\tilde{x}^{T}\\mathbf{\\beta}$” as “$\\Delta(x)=\\tilde{x}^{T}\\mathbf{\\beta}$” in line 152.\n\nLIMITATIONS:\nThe authors discuss some of the algorithm's shortcomings.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the problem of estimating the conditional average treatment effect (CATE) under the assumption of unmeasured confounding. The authors focus on the specific scenario where some observed variable acts as instrument w.r.t. unmeasured confounder but might be confounded by some other observed confounder, so that standard IV methods may fail. They derive a method which extends Direct Learning (DL) by an additional scaling factor of the outcomes. This scaling factor is the CATE of the instrument on the treatment, which can be estimated with standard methods (e.g., DL). In a simulation study, the authors compare the proposed method to a set of baseline methods.\n\nSTRENGTHS:\nThe problem and method are well-presented. The resulting method is simple but elegant. It extends Direct Learning by, first estimating the CATE of the instrument on the treatment, and estimates the CATE of the treatment on the outcome through Direct Learning leveraging the result of the first step.\n\nThe authors prove identifiability under a provided set of assumptions. They propose two ways to residualize the outcomes in order to reduce the variance of the estimator, and provide a set of sufficient conditions (in terms of correctly specified nuisance functions) under which the estimator yields consist CATE estimates.\n\nWEAKNESSES:\nAssumption 2.f seems rather strong as the unobserved confounder can only additively affect the treatment. The data generating process in the experimental section violates this assumption. As the proposed method still outperforms the baselines, this may suggest that the method is less sensitive to the assumption. Though, that should be studied empirically in more detail.\n\nCompared to other recent publications focusing on estimating the (conditional) treatment effect, the assumed data generating process in the simulation seems overly simple. Other methods involving GPs, normalizing flows, and other highly non-linear models allow for high-dimensional confounders. They are typically assessed using semi-artificial data (e.g., with images as confounders and image labels as confounding mechanism). Without such hard problems and the corresponding baseline methods, it is hard to assess the overall practical value of the proposed method.\n\nQUESTIONS:\nThe paper is clear; no questions but some minor comment:\n- The term instrumental variable for Z might be a bit confusing as Z and Y are confounded; Z is an IV w.r.t. unmeasured confounder. It may help to clarify that in the very beginning.\n- Figure 1 might be improved in terms of order and size of the nodes; having the instrument on the right, the treatment at the top, and the outcome on the left is rather unconventional.\n- lines 151-164 have likely little value as this should be known to the Neurips audience\n\nLIMITATIONS:\nNeurIPS Paper Checklist is provided; no concerns.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new type of CATE estimator using instrumental variables. The proposed method employs the direct learning approach.\n\nSTRENGTHS:\n1. The paper is self-contained and comprehensible.\n2. Besides developing the CATE estimator, the paper also proposes an estimator for finding the optimal treatment regimes.\n\nWEAKNESSES:\n__Missing literature review__ \n\nThe paper most closely related to this work is [Machine Learning Estimation of Heterogeneous Treatment Effects with Instruments[(https://proceedings.neurips.cc/paper/2019/file/3b2acfe2e38102074656ed938abf4ac3-Paper.pdf). It develops a fast-converging CATE estimator for local average treatment effects using instrumental variables. However, this paper is not cited in the literature review. Please consider including a discussion of this paper for richer context. More importantly, please compare your work with this paper to highlight the novelty of the current work.\n\n__Weak motivation on directed learning__ \n\nThe introduction section lacks plausible reasons for proposing directed learning. What are the alternative methods and their pros and cons? Why should we specifically consider the directed learning approach?\n\n__Validity of Assumption 2__\n\nAssumption 2f is a weaker version of the following assumption: \"$U$ is noninformative to $A$ given $Z$ and $X$ (i.e., $A \\perp U \\mid X,Z$).\" Given that there are no practical settings where Assumption 2f holds while $A \\perp U \\mid X,Z$ doesn't (except some peculiar parametrization), and both assumptions are non-testable, I don't see any practical distinction between $A \\perp U \\mid X,Z$ and Assumption 2f. In other words, Assumption 2f is just another representation of $A \\perp U \\mid X,Z$ tailored for identification.\n\nCombining $A \\perp U \\mid X,Z$ with Assumption 2c ($Z \\perp U \\mid X$) results in $(U \\perp A \\cup Z \\mid X)$ by the contraction property of conditional independence. This means $U$ does not influence $(A,Z)$ given $X$. Consequently, in any related causal graph, there should be no edges from $U$ to $A$. This leads to the ignorability condition that $Y(a) \\perp A \\mid X$.\n\nIn summary, interpreting Assumption 2f as $A \\perp U \\mid X,Z$ means Assumption 2 is essentially an ignitability assumption. Therefore, it is important to discuss the validity of Assumption 2 in practical settings, to disprove that Assumption 2f is merely another representation of $A \\perp U \\mid X,Z$ designed for identification. Have you considered the LATE setting, given that the estimand in Proposition 1 will remain unchanged?\n\n__More analysis is required__\n\nMultiple robustness properties provided in Theorem 3 imply that the proposed estimator converges to the optimal estimator faster. For example, if nuisances converge at an $n^{-1/4}$ rate, where $n$ is the number of samples, then the estimator converges at an $n^{-1/2}$ rate. These results are beneficial since they guarantee fast convergence. While Theorem 3 is attractive, it is somewhat impractical because, in practice, the working model is rarely considered a true model. Please provide more analysis on the rate of convergence concerning the convergence rate of nuisance parameters.\n\n__Fair comparison with other estimators__\n\nEven if the empirical evidence in Table 1 is strong, the discussion on why the proposed estimator converges faster than other multiply-robust estimators, such as MRIV, is missing. Asymptotically, there are no reasons to believe the proposed estimator converges faster than the MRIV estimator. Can you provide a discussion on why the proposed estimator converges faster than its competitors?\n\nQUESTIONS:\n1. $\\Delta(x)$ in line 89 and $\\Delta(x)$ in line 94 are the same?\n\n2. What are the practical examples where Assumption 2 holds? \n\n3. Is there a reason to choose Assumption 2 other than the LATE assumption? Both assumptions yield the same target parameter (in Proposition 1).\n\nLIMITATIONS:\n1. The paper assumes discrete/binary $Z$. \n\n2. The paper is relying on Assumption 2.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-29"}