A Non-parametric Direct Learning Approach to Heterogeneous Treatment Effect Estimation under Unmeasured Confounding

Authors: xinhai  zhang
xingye  qiao
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 00:42:42

Abstract: In many social, behavioral, and biomedical sciences, treatment effect estimation is a crucial step in understanding the impact of an intervention, policy, or treatment. In recent years, an increasing emphasis has been placed on heterogeneity in treatment effects, leading to the development of various methods for estimating Conditional Average Treatment Effects (CATE). These approaches hinge on a crucial identifying condition of no unmeasured confounding, an assumption that is not always guaranteed in observational studies or randomized control trials with non-compliance. In this paper, we proposed a general framework for estimating CATE with a possible unmeasured confounder using Instrumental Variables. We also construct estimators that exhibit greater efficiency and robustness against various scenarios of model misspecification. The efficacy of the proposed framework is demonstrated through simulation studies and a real data example.

Main paper: 
 introduction: 
 In various domains, different subjects may exhibit different responses to the same set of treatments. The exploration of this heterogeneity in the effects resulting from exposure has gained substantial interest in recent years. For instance, inferring the heterogeneous effect of a medical treatment on clinical outcome can contribute to the development of personalized treatment (Cai et al., 2011) . A similar concept has found application in personalized marketing as well (Chandra et al., 2022) . The heterogeneity among subjects can be measured by the disparity in conditional mean outcomes given other covariates, typically referred to as the Conditional Average Treatment Effect (CATE). Another problem closely related to the heterogeneity in treatment effects is the optimal Individualized Treatment Regime (ITR), which is a decision rule that selects treatments for individuals to maximize the expected outcome.There has been significant development in the literature regarding the estimation of CATE and the optimal ITR in the case of no unmeasured confounding. For example, Q-learning (Qian and Murphy, 2011) models the conditional mean outcome under each treatment separately and the estimated CATE is constructed using the difference between the estimated conditional mean outcomes. The success of this method relies on the correct specification of the outcome models. To address this issue, direct learning (DL) (Tian et al., 2014; Qi and Liu, 2018) and robust direct learning (RD) (Meng and Qiao, 2022) models the conditional contrast between treatments directly, which has been shown to be more robust to model misspecification. Another strand of work approaches with tree-based or forest-based methods. Hill (2011) and Green and Kern (2012) extended the Bayesian Additive Regression Tree (BART) method of Chipman et al. (2010) for estimating heterogeneous treatment effect. Athey and Imbens (2016) proposed Causal Trees with an "honest" splitting approach, wherein the partitioning is constructed in one sample, and the treatment effects within each node are estimated using another sample. This methodology is subsequently adopted in Causal Forest (Wager and Athey, 2018) , which extends the random forest algorithm to estimate heterogeneous treatment effects. On the other hand, optimal ITR estimation aims to determine the optimal decision rule for treatment assignment based on subjects' covariates to maximize the mean outcome. A significant line of work in the field involves transforming ITR estimation into a classification problem through the use of Inverse Probability Weighting (IPW). Notable contributions include Outcome Weighted Learning (OWL) (Zhang et al., 2012; Zhao et al., 2012) and Residual Weighted Learning (RWL) (Zhou et al., 2017) .The aforementioned methods all rely on the key assumption of no unmeasured confounding to identify the heterogeneous treatment effect or the optimal ITR. However, this assumption is in most cases unverifiable (if not untrue) in observational studies or randomized controlled trials (RCT) with noncompliance. A well-known approach that takes into account the unmeasured confounding is the use of an instrumental variable (IV). A proper IV is usually a pre-treatment variable that is independent of any possible unmeasured confounder while correlated with the treatment. For example, in RCT with non-compliance, the random treatment assignment can be considered as an IV while the treatment received is considered the treatment variable. Here these two are clearly correlated since a subject will not receive the treatment if they are not assigned one, though the strength of the correlation may depend on other characteristics such as the education level of the subject.There is a growing literature on estimating heterogeneous treatment effects or optimal ITR under unmeasured confounding using IV. Imbens and Angrist (1994) identified and estimated the socalled Local Average Treatment Effect (LATE), restricted to the subgroup of the always-compliant population, with the help of an IV. More recently, machine learning methods like Doubly Robust IV (Syrgkanis et al., 2019) and Generalized Random Forest (Athey et al., 2019) have shown their applicability and effectiveness in various settings including unmeasured confounding, particularly when used in conjunction with an IV. Wang and Tchetgen Tchetgen (2018) introduced two alternative assumptions on the unobserved confounders and the IV, which enable the identification of the Average Treatment Effect (ATE). They proposed an estimator that has the so-called multiply robustness property, which guarantees consistent estimate under three observed data models. These findings were incorporated into Cui and Tchetgen Tchetgen (2021) to obtain an optimal ITR estimation while accounting for unmeasured confounding. On the other hand, Frauen and Feuerriegel (2022) utilized these findings for CATE estimation.In this paper, we propose a new framework for estimating CATE using IV when there exist unmeasured confounders. This framework can be viewed as an extension of the Direct Learning method under unconfoundedness to the case that allows the existence of unmeasured confounding. We call the proposed method Direct Learning using Instrumental Variables (IV-DL). The proposed framework is easy to implement under many flexible learning methods. Additionally, we introduce several efficient and robust estimators by residualizing the outcome. These estimators have been demonstrated to be robust to multiple model misspecification scenarios.The rest of this paper is organized as follows. The notations and some related preliminaries are introduced in Section 2. The proposed framework IV-DL is formally introduced in Section 3. In Section 4 and 5, we proposed efficient and robust estimators. In Section 6, we conduct simulation studies and compare the performance with existing methods in the literature. A real data example is included in Section 7. Section 8 concludes the paper with a discussion on possible future work. Proofs and additional simulations are provided in the Appendix.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: They provide a summary of the research objectives, methodologies, and findings, which are consistently supported by the detailed content of the paper. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
notations and preliminaries: 
 Denote A ∈ A = {+1, -1} as the binary treatment, and X ∈ X ⊆ R p the pre-treatment covariates. We adapt the potential outcome framework (Rubin, 1974) in causal inference and denote by Y (a) ∈ R the potential outcome that the subject would have obtained if the received treatment was a ∈ A. The observed outcome is then given by Y= Y (A) = Y (1)1[A = 1] + Y (-1)1[A = -1].Denote by U the unobserved confounder of the effect of A on Y . Suppose we have access to a pre-treatment binary IV denoted by Z ∈ Z = {+1, -1}. Then the complete data consists of independent and identically distributed copies of (Y, X, A, U, Z), even though only copies of (Y, X, A, Z) are observed.Our goal is to estimate the Conditional Average Treatment Effect (CATE), defined as∆(x) ≜ E[Y (1) -Y (-1)|X = x].As mentioned in Section 1, most of the prior works are based on the core assumption of no unmeasured confounding: Assumption 1 (Unconfoundedness). Y (a) ⊥ ⊥ A|X for a = ±1.This assumption essentially implies that the observed covariates X would suffice to account for the confounding of the effect of A on Y , thereby excluding the presence of U . Under the above assumption of unconfoundedness, it can be easily verified that CATE is identified by (Qian and Murphy, 2011 ) models the two conditional mean outcomes separately and estimates the CATE by taking the difference between these estimates. Consequently, its effectiveness depends on correctly specifying the models for the conditional mean outcomes. Denote the propensity score for the treatment as π A (a, x) = P[A = a|X = x] for a = ±1. Direct Learning (Qi and Liu, 2018; Tian et al., 2014) propose to directly model for the heterogeneous treatment effect, based on the observation that ∆∆(x) = E[Y |A = 1, X = x] -E[Y |A = -1, X = x]. Q-learning(x) = E[AY /π A (A, X)|X = x].In other words, one can obtain an estimate of CATE by regressing the modified outcome AY /π A (A, X) on X. Robust Direct Learning (RD) Meng and Qiao (2022) further extends this framework by residualizing the outcome using an estimate of the main effect, which is the average of the two conditional mean outcomes. This method demonstrates double robustness in the sense that it yields consistent estimation of CATE if either the propensity score or the main effect is correctly specified. Despite the success in RCT or observational studies, all the methods mentioned above rely on the unconfoundedness Assumption 1. In the next section, we will introduce a general framework that directly models CATE using an IV approach when there exists unmeasured confounding.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We included discussion on the accessibility of the assumptions in practice.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
direct learning with instrumental variable approach: 
 In this paper, we look beyond Assumption 1, and consider the existence of an unmeasured confounder U . To establish the identification of CATE in this setting, we approach with the use of a proper IV. We will start with the following assumptions seen in Cui and Tchetgen Tchetgen (2021) . Assumption 2. This assumption consists of five parts as follows:a. Y (z, a) ⊥ ⊥ (Z, A)|X, U for z, a = ±1. b. Z ̸ ⊥ ⊥ A|X. c. Z ⊥ ⊥ U |X. d. Y (z, a) = Y (z ′ , a) for z, z ′ , a = ±1. e. 0 < π Z (1, X) < 1 almost surely, where π Z (z, x) = P[Z = z|X = x] for z = ±1.Here, Y (z, a) represents the potential outcome that would be observed if a subject were exposed to treatment a ∈ A, and the IV takes a value of z ∈ Z. Assumption 2.a rules out the existence of any other confounder, except for X and U , for the joint effect of Z and A on the outcome Y . However, this unconfoundedness is hidden from the data collected, since U is never observed. Assumptions 2.b-2.e provides us with a well-defined IV. Assumption 2.b requires a correlation between the IV and the treatment given observed covariates. In many applications, a strong correlation is often necessary to ensure accurate inference in the estimation process. Assumption 2.c guarantees that the causal effect of Z on Y is not confounded given X; otherwise Z suffers the same issue as A. Additionally, required by Assumption 2.d, the causal effect of Z on Y can only be mediated by the treatment A. In light of this assumption, we omit the argument z in the potential outcome and denote the common value as Y (a). Assumption 2.e implies that each subject has a positive chance of having either value of the IV. An example of the relationships between variables that satisfy Assumption 2 is presented in a directed acyclic graph in Figure 1 . In order to identify the CATE, we also need the following assumption on the unmeasured confounder. Assumption 3. At least one of the following is true:a. E[A|Z = 1, X, U ] -E[A|Z = -1, X, U ] = E[A|Z = 1, X] -E[A|Z = -1, X] b. E[Y (1) -Y (-1)|X, U ] = E[Y (1) -Y (-1)|X] Z A Y X UFigure 1 : A directed acyclic graph with unmeasured confounding and an IV Assumption 3 states that, conditional on the measured covariates, either the additive effect of Z on A is independent of U , or the additive effect of A on Y is independent of U . Now, we finally have identification of the CATE. Proposition 1. Under Assumptions 2-3, the CATE can be identified by∆(x) = E[Y |Z = 1, X = x] -E[Y |Z = -1, X = x] P[A = 1|Z = 1, X = x] -P[A = 1|Z = -1, X = x](1)EQUATIONwhere π Z (z, x) = P[Z = z|X = x] and δ(x) = P[A = 1|Z = 1, X = x] -P[A = 1|Z = -1, X = x] for any x ∈ X .The first equality (1) was shown in Wang and Tchetgen Tchetgen (2018) , which means that the CATE is identified by the conditional Wald estimand. Equation ( 2) reveals an interesting observation that we do not need the realized treatment A as long as we have δ(x), which can be viewed as the conditional effect of the IV on the treatment given observed covariates. Hereafter, we denote the conditional means of Y and A by µY z (x) = E[Y |Z = z, X = x] and µ A z (x) = E[A|Z = z, X = x], respectively, for any z ∈ {-1, +1} and x ∈ X .In this section, we will introduce the IV-DL framework. Motivated by Equation (2), the next lemma offers a way to estimate ∆(x) using inverse propensity score of IV as weight. Lemma 1. Under Assumptions 2-3,∆ ∈ argmin f E 1 π Z (Z, X) 2ZY δ(X) -f (X) 2 .Based on Lemma 1, we can adopt many existing regression methods to obtain an estimate on CATE by regressing the modified outcome on the covariates, weighted by the propensity score for Z. Specifically, given the data {y i , x i , a i , z i } n i=1 , an estimator πZ of the propensity score function and an estimator δ of the effect of Z on A" the IV-DL estimate for ∆ is given byf (x) = argmin f ∈F 1 n n i=1 1 πZ (z i , x i ) 2z i y i δ(x i ) -f (x i ) 2 + λ∥f ∥ F ,where F is a function space with norm ∥ • ∥ F , and λ ≥ 0 is the tuning parameter for the regularization term ∥f ∥ F . To obtain πZ , we can fit a logistic regression of Z on X or a non-parametric model such as random forest. Since δ is the treatment effect of Z on A, it is noteworthy that, under Assumption 3, estimation of δ can be viewed as a CATE estimation problem with unconfoundedness. In this case, A may be viewed as a binary "outcome" and Z a binary "treatment". Thus, we can adopt many existing CATE estimation methods such as Q-learning, DL, and Causal Forest.The proposed framework allows a variety of learning methods to model the treatment effect ∆(x).For example, under the linear model, we may model f (x) = xT β where the regression coefficients are β and xi ≜ (1, x T i ) T . Then IV-DL estimator for β isβ = argmin β∈R p+1 1 n n i=1 1 πZ (z i , x i ) 2z i y i δ(x i ) -xT i β 2 and the CATE ∆(x) is estimated by f (x) = xT β.In high dimensional setting where p is large, sparse regularization can be easily applied here because the optimization is essentially a weighted least square problem. For example, we can use Least Absolute Shrinkage and Selection Operator (LASSO) and the estimator of β is given by βlasso = argmin β∈R p+11 n n i=1 1 πZ (z i , x i ) 2z i y i δ(x i ) -xT i β 2 + λ∥β∥ 1 ,where λ > 0 is the tuning parameter for the l 1 penalty.In practice, there is no guarantee that the true treatment effect follows a linear model. For a more complex model, we can adopt nonlinear methods such as Kernel Ridge Regression (KRR) and solveargmin β∈R n ,β0∈R 1 n n i=1 1 πZ (z i , x i ) 2z i y i δ(x i ) -(K T i β + β 0 ) 2 + λβ T Kβ,where K i is the i-th column of the kernel matrix K = (K(x i , x j )) n×n and K(•, •) is a kernel function. KRR might be computationally expensive when dealing with large datasets. In such cases, other machine learning methods capable of solving a weighted least squares problem can be considered. Examples include local regression, regression trees, random forests, and neural networks.In some domains, the optimal Individualized Treatment Regime (ITR) can be of interest. The goal here is to find a mapping d : X → A from a specific class D to maximizes the expected outcome:d * ≜ argmax d∈D E[Y (d(X))],where Y (d(X)) is the potential outcome that the subject X obtained after receiving treatment d(X), and E[Y (d(X))] is also known as the Value of the regime d.ITR and CATE are closely related. For example, in the binary treatment setting, the CATE ∆ is the difference between two conditional mean outcomes. Assuming greater values of outcome is preferred, then the sign of ∆ will determine which treatment is optimal. It can be verified that d * (x) = sign(∆(x)). Therefore, we define the estimated optimal ITR using IV-DL as d(x) = sign( f (x)), where f (x) may be any CATE estimator introduced in the last subsection.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) Justification: Each experiment takes a few hours on an old Macbook. Detailed information on the computer resources and an estimate of time needed is included in the code instructions.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
efficient estimators by residualization: 
 In the literature, considerable advancements have been made to enhance the efficiency and robustness of the CATE and optimal ITR estimation. To this end, residualization and augmentation are two common strategies. For example, in the IPW framework for optimal ITR estimation, Zhou et al. (2017) and Zhou and Kosorok (2017) proposed to replace the outcome by its residual Y -ĝ(x) in estimation of the optimal regime, where ĝ(x) is an estimate of the weighted average of the conditional mean outcomes. For the estimation of CATE, Meng and Qiao (2022) residualized the outcome by an estimate of the average of conditional mean outcomes. Frauen and Feuerriegel (2022) proposed augmenting a preliminary estimate of CATE to enhance the robustness of the estimator.In this section, we present the Robust Direct Learning using IV approach (IV-RDL), which involves residualizing the outcome in IV-DL to enhance both efficiency and robustness. We propose two ways of residualization, referred to as IV-RDL1 and IV-RDL2, respectively. They are shown to reduce the variance when estimating CATE. In Section 5, we show that they have robustness properties when confronted with model misspecification for nuisance variables.We first consider residualizing the outcome by a function of the observed covariates only. Ideally, we would like to find a function g : X → R that can improve the efficiency of the estimation on CATE, while keeping it consistent. As shown in the following lemma, the consistency of the estimator is in fact preserved under a shift of Y by any function of the observed covariates. Lemma 2. For any measurable g : X → R and any probability distribution for (Y, X, A, Z)∆ ∈ argmin f E 1 π Z (Z, X) 2(Y -g(X))Z δ(X) -f (X) 2Asymptotically, the variance of the estimator is related to the variance of the derivative of[π Z (Z, X)] -1 [2(Y -g(X))Z)Zδ -1 (X) -f (X)] 2, the weighted loss for each individual. Hence, it is natural to choose g that minimize the variance of[π Z (Z, X)] -1 [2(Y -g(X))Zδ -1 (X) -f (X)].See Appendix B for a more detailed discussion using the linear model as an example. The following theorem gives us the minimizer.Theorem 1. Among all measurable g : X → R, the following function minimize the variance of1 π Z (Z,X) 2(Y -g(X))Z δ(X)-f (X) :EQUATION)There is an interesting interpretation of the optimal function g * , which equals the average of µ Y 1 (x) and µ Y -1 (x). Recall that Eq. ( 1) states that CATE under unmeasured confounding is identified by the ratio of two contrasts, where the numerator happens to be µY 1 (x) -µ Y -1 (x).The residualization strategy amounts to shifting the outcome Y , and hence µ Y 1 (x) and µ Y -1 (x) as well. Naturally, shifting both by their average will not affect their difference, but it will reduce the variance. A similar residualization was incorporated in RD under unconfoundedness (Meng and Qiao, 2022) , where the goal was to learn the contrast between conditional mean outcomes given the two treatments.In practice, g * needs to be estimated before we can estimate the CATE. There are several approaches to obtain the estimate of g * , denoted by ĝ * . For example, we can take the average of estimated conditional mean outcomes, i.e., ĝ *(x) = (μ Y 1 (x)+ μY -1 (x))/2.One can also regress Y /(2π Z (Z, X)) on X, inspired by Eq. (3). Given ĝ * (x), the IV-RDL1 estimator for ∆ is obtained byfg (x i ) = argmin f ∈F 1 n n i=1 1 πZ (z i , x i ) 2(y i -ĝ * (x i ))z i δ(x i ) -f (x i ) 2 + λ∥f ∥ F .In Section 5, we will show that this estimator is robust against misspecification of either g * or π Z , given that δ is correctly specified.In this paper, we also consider an alternative way of residualizing the outcome by a function h : (X , A, Z) → R. Like IV-RDL1, the optimal choice is the function that minimizes the variance while maintaining the consistency of CATE estimation. Among all functions that still convey consistent CATE estimation, the following three equivalent functions minimize the variance of[π Z (Z, X)] -1 [2(Y -h(X, A, Z))Zδ -1 (X) -f (X)]. h * 1 (x, a, z) = µ Y 1 (x) + ∆(x) a -µ A 1 (x) -zδ(x) /2 h * 2 (x, a, z) = µ Y -1 (x) + ∆(x) a -µ A -1 (x) -zδ(x) /2 h * 3 (x, a, z) = m Y (x) + ∆(x) a -m A (x) -zδ(x) /2 where m Y (x) ≜ (µ Y 1 (x) + µ Y -1 (x))/2 and m A (x) ≜ (µ A 1 (x) + µ A -1 (x))/2.The technical details are provided in the Appendix C. In practice, all these conditional means (µ Y -1 , µ Y 1 , µ A -1 and µ A 1 ) need to be estimated, together with estimations of π Z and δ. Additionally, we need to obtain a preliminary estimate of CATE. The IV-RDL2 estimator is constructed by,fh (x i ) = argmin f ∈F 1 n n i=1 1 πZ (z i , x i ) 2(y i -ĥ * (x i , a i , z i ))z i δ(x i ) -f (x i ) 2 + λ∥f ∥ F ,where ĥ * is an estimator for one of h * 1 , h * 2 and h * 3 . 
robustness properties: 
 In this section, we investigate the robustness properties of IV-RDL1 and IV-RDL2. We start with the following theorem to demonstrate the double robustness property of the IV-RDL1 that residualizes the outcome by using g(x).Theorem 2. Suppose Assumption 2-3 holds, and we have a consistent estimator of δ, denoted by δ.Let πZ be a working model for π Z , and g be a working model for g * . Then we have∆ ∈ argmin f ∈{X →R} E   1 πZ (Z, X) (Y -g(X))Z δ(X) -f (X) 2   if either πZ (z, x) = π Z (z, x) or g(x) = g * 1 (x) almost surely.Theorem 2 indicates that we will have a doubly robust estimator for ∆ if either π Z or g * is correctly specified when δ is known or correctly specified. However, the requirement of a consistent estimate of δ would not pose a significant issue in practical application, since it is essentially a CATE estimation problem under no unmeasured confounding. A consistent estimator for δ can be found by implementing any state-of-the-art CATE estimation method in the literature.For the IV-RDL2, there are more nuisance variables that need to be estimated. The next theorem shows that IV-RDL2 is robust to various scenarios of misspecified nuisance variables.Theorem 3. Suppose Assumption 2-3 holds. Let πZ , δ, μY 1 , μY -1 , μA Z , μA -1 , mY , mA and ∆ be working models for π Z , δ, µ Y 1 , µ Y -1 , µ A Z , µ A -1 , m Y , mA and ∆, respectively. Denote h1 , h2 and h3 as chosen augmentation formulated according to h * 1 , h * 2 and h * 3 using working estimates. Then we have Theorem 3 summarizes in total six cases of the minimal combination of correctly specified nuisance variables in order to have a consistent estimate of CATE. The three choices of residualization functions possess robustness against different scenarios. In the first two scenarios, obtaining a consistent estimate of CATE is guaranteed as long as we correctly specify π Z and either ∆ or δ. This consistency holds irrespective of the choice of the three h functions. In practice, the second scenario may be particularly accessible, especially when π Z is known. The other scenarios are less likely to be verified in practice and therefore requires more domain knowledge of the data structure. Specifically, scenarios (3)-( 5) requires the corresponding set of conditional means to be correctly specified as well as the preliminary ∆. Lastly, in scenario (6), when δ and the averages of conditional means are correctly specified, the IV-RDL2 will also provide a consistent estimate of CATE.∆ ∈ argmin f ∈{X →R} E   1 πZ (Z, X) (Y -h(X, A, Z))Z δ(X) -f (X)While working on this paper, we encountered unpublished work by Frauen and Feuerriegel (2022) that is similar to our IV-RDL2 estimator. Inspired by Wang and Tchetgen Tchetgen (2018) , Frauen and Feuerriegel introduced the MRIV framework, which is a two-step process. First, a preliminary estimator of CATE and nuisance estimators of δ, π Z , µ Y -1 and µ A -1 are obtained. Then, a pseudooutcome is created by augmenting the preliminary CATE with the nuisance estimates, and the final CATE estimator is obtained by regressing the pseudo-outcome on the covariates. As shown in Wang and Tchetgen Tchetgen (2018) , this estimator is robust against model misspecification of the nuisance variables in three of the six scenarios in Theorem 3 (scenarios (1), (2), and (4)). Our numerical studies have shown that our proposed IV-DL framework performs better than the MRIV method. 
simulation study: 
 In this section, we present the results of the simulation study conducted to assess the performance of the proposed IV-DL framework. We compared the proposed method with Bayesian additive regression trees (BART; Chipman et al., 2010) , robust direct learning (RD; Meng and Qiao, 2022) , causal forest with IV approach (CF; Athey et al., 2019) , MRIV method (Frauen and Feuerriegel, 2022) , and weighted learning with IV approach (IPW-MR; Cui and Tchetgen Tchetgen, 2021).We begin by introducing the data-generating mechanism. The covariates, denoted as X = (X 1 , X 2 , X 3 , X 4 , X 5 ), were generated from uniform distribution with X i ∼ U nif (-1, 1) for i = 1, . . . , 5. We followed Cui and Tchetgen Tchetgen (2021) and generated the treatment A under logistic model with probability for success:P(A = 1|X, Z, U ) = expit{2X 1 + 2.5Z -0.5U },where the instrumental variable Z was a Bernoulli random variable with probability 1/2 and U was the unobserved confounder that followed Bridge distribution with parameter ϕ = 1/2. By the results from Wang and Louis (2003) , the above usage of Bridge distribution will guarantee that the marginal distribution f (A|X, Z) can be modeled directly by logistic regression. In other words, there exists some vector α such that logit{P(A = 1|X, Z)} = α T (1, X, Z).The outcome Y was generated in two different settings corresponding to linear and non-linear models of the true CATE:1. Y = h(X) + q(X)A + 0.5U + ϵ 2. Y = h(X) + {exp(q(X)) -1}A + U + ϵwhere the error term ϵ follows N (0, 1). Functions h(X) and q(X) are defined as follows:h(X) = 0.5 + 0.5X 1 + 0.8X 2 + 0.3X 3 -0.5X 4 + 0.7X 5 q(X) = 0.2 -0.6X 1 -0.8X 2 .In Setting 1, the true CATE is 2q(x), which is linear in x. In Setting 2, the true CATE is 2(exp(q(x))-1), which is nonlinear. The sample size for each setting was 500 and the simulation was repeated 100 times. An independent sample of size 5000 was used to evaluate the performance of different methods. The proposed methods were implemented according to Sections 3 and 4 with δ(X) estimated by causal forest ("grf" package) and the other nuisance variables estimated by random forest. For methods that require to estimate the same nuisance variable, they shared the same copies of nuisance estimates.We compared all methods based on three performance metrics in the testing sample: the correct classification rate by the estimated ITR (AR); the value function evaluated at the estimated ITR (Value); the mean squared error of the estimated CATE (MSE). Table 1 reports the mean and standard error of these three evaluation metrics over 100 replications for different methods in the two settings. Among the methods implemented, BART and RD rely on the unconfoundedness assumption and therefore fail to identify CATE when there is unobserved confounding. Both IPW-MR and CF make use of the IV to take unmeasured confounding into account. IPW-MR had fine performances on estimating ITR and maximizing the value. However, it was not designed to estimate CATE. CF performs slightly worse than IPW-MR in terms of AR and value in Setting 1, despite offering a CATE estimation. Its performance is more competitive compared to IPW-MR in Setting 2. Our proposed methods showed superior performances on all the metrics. In particular, IV-RDL1, which residualized the outcome using averages of the estimated conditional means, outperformed all the methods in both settings. IV-RDL2 had a more complicated residualization, and achieved the secondbest performance (but still fairly close to IV-RDL1). Even the unresidualized IV-DL performed better than other methods in most of the metrics. Additional simulation results on testing the robustness of the proposed framework is reported in Appendix D. 
data analysis: 
 In this section, following Angrist and Evans (1998), we study the causal effect of child-rearing on a mother's labor-force participation, using a sample of married mothers with two or more children from the U.S. 1980 census data (80PUMS). Assuming the sex of children is random, "first two children mixed sex or not" becomes a suitable instrumental variable for the causal effect of having a third child on a mother's labor force participation. Angrist and Evans showed that having a third child reduces women's labor force participation on average. Our goal is to investigate heterogeneity among families, offering personalized insights on the decision to have a third child and its impact on employment opportunities. We used a dataset of 478,005 subjects with at least two children. The outcome, Y , represents whether the mother was employed in the year preceding the census. The treatment, A, indicates whether the mother had three or more children at the census time, and the instrumental variable, Z, indicates whether the first two children were of the same sex. We considered five covariates, X: mother's age at first birth, age at census time, years of education, race, and the father's income. We used the random forest algorithm for both the implementation of the proposed method and the estimation of the nuisance variables µ Y z , µ A z , δ, and π Z . The preliminary CATE estimator was formulated according to Eq. 1 with plug-in estimates on the conditional means. To identify subgroups with distinct treatment effects, we used the estimated CATE as the response to construct a regression tree, shown in Figure 2 . The splits occurred at the mother's age at census (33), age at first birth (23), and father's income ($2.1k/year and $26k/year). By investigating the five subgroups (32%, 4%, 3%, 15%, and 47% of the sample), labeled as groups 1-5, we have made the following observations. First, older mothers are more likely to work after having a third child (subgroups 4 and 5 show a larger estimated treatment effect). Second, younger mothers with very low-income fathers (subgroup 3) tend to stay in the labor force after the third child. Lastly, younger mothers are more likely to stop working if their husband's income is between $2.1k and $26k/year (subgroups 1-3). Figure 3 displays the histogram of estimated CATE for the three majority groups (1, 4, and 5). The estimated CATE for group 1 is overall smaller than for groups 4 and 5. We also constructed 3-dimensional scatter plots based on the three splitting variables for a more detailed look at the heterogeneity (shown in Appendix E). 
conclusions: 
 In this paper, we proposed a new framework to estimate CATE under unmeasured confounding by using an instrumental variable. Under the proposed framework, the estimation procedure boils down to solving a weighted least square problem, which can be tackled with any modern statistical or machine learning method. We also constructed two robust estimators by residualizing the outcome, which are shown to be more efficient and robust to model misspecification on nuisance variables. Numerical studies have shown very competitive performance for our proposed methods.A potential extension of our work involves using IV to estimate treatment effects for multi-arm and continuous treatments, with the challenge lying in the generalization of Assumption 3. Another avenue is to incorporate deep neural networks to make use of their rich expressiveness for data distribution. However, the empirical performance and theoretical properties need to be formally studied. One notable limitation is the issue of extreme weights, which can arise during the estimation process and potentially lead to instability and biased results. Addressing this limitation is crucial for improving the reliability and accuracy of our method. 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The research conform with the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: The paper proposed an estimation framework and technical details within a specific field, without offering practical applications or solutions that could be implemented in broader societal contexts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [NA] Justification: Our data analysis used public data. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Proof of Proposition 1. For any z ∈ {-1, +1}, we haveE[2Y |Z = z, X] = E U E[2Y |Z = z, X, U ] = E U E[Y (1 + A)|Z = z, X, U ] + E U E[Y (1 -A)|Z = z, X, U ] = E U E[Y (1)(1 + A)|Z = z, X, U ] + E U E[Y (-1)(1 -A)|Z = z, X, U ] = E U E[Y (1) + Y (-1)|Z = z, X, U ] + E U E[AY (1) -AY (-1)|Z = z, X, U ] = E U E[Y (1) + Y (-1)|X, U ] + E U E[Y (1) -Y (-1)|X, U ]E[A|Z = z, X, U ]Evaluate the above equality at z = 1 and z = -1, and take the difference. Then we have2E[Y |Z = 1, X] -2E[Y |Z = -1, X] = E U E[Y (1) -Y (-1)|X, U ] E[A|Z = 1, X, U ] -E[A|Z = -1, X, U ]Based on Assumption 3, we haveE[A|Z = 1, X, U ] -E[A|Z = -1, X, U ] = E[A|Z = 1, X] -E[A|Z = -1, X]. Combining the above two, we have E[Y |Z = 1, X] -E[Y |Z = -1, X] = 1 2 E U E[Y (1) -Y (-1)|X, U ] E[A|Z = 1, X] -E[A|Z = -1, X] = E[Y (1) -Y (-1)|X] P[A|Z = 1, X] -P[A|Z = -1, X]The above equality is equivalent to Equation (1). On the other hand, for any x ∈ X ,E ZY π Z (Z, X) X = x = π Z (1, x)E Y π Z (1, X) Z = 1, X = x + π Z (-1, x)E -Y π Z (-1, X) Z = -1, X = x = E[Y |Z = 1, X = x] -E[Y |Z = -1, X = x]Dividing both sides of the above equality by δ(x) = P[A|Z = 1, X = x] -P[A|Z = -1, X = x] yields Equation (2). Lemma 3. Let ℓ(X, f ) = E[Q(X, f )|X] and L(f ) = Eℓ(X, f ). Denote f * ∈ argmin f ℓ(X, f ). Then f * ∈ argmin f L(f ). Proof. Denote f + ∈ argmin f L(f ).Then by definition, we have the following two inequalities:L(f + ) ≤ L(f * ) = Eℓ(X, f * ) L(f * ) = Eℓ(X, f * ) ≤ Eℓ(X, f + ) = L(f + ) Therefore, L(f * ) = L(f + ) and f * ∈ argmin f L(f ). Proof of Lemma 1. Let ℓ(X, f ) = E 1 π Z (Z,X) 2Y Z δ(X) -f (X) 2 X . By Lemma 3, it suffices to show ∆ ∈ argmin f ℓ(X, f ).The gradient of ℓ(X, f ) with respect to f is given by∂ ∂f ℓ(X, f ) = E ∂ ∂f 1 π Z (Z, X) 2Y Z δ(X) -f (X) 2 X = -2E 1 π Z (Z, X) 2Y Z δ(x) -f (X) X = 2E f (X) π Z (Z, X) X -2E 2Y Z δ(X)π Z (Z, X) X = 4f (X) -4∆(X) Since ℓ(X, f ) is convex, we have ∆ ∈ argmin f ℓ(X, f ). Proof of Lemma 2. Let ℓ g (f ) = E 1 π Z (Z,X) 2(Y -g(X))Z δ(X) -f (X) 2 X .By Lemma 3, it suffices to show that for any g, argmin f ℓ g (X, f ) = argmin f ℓ(X, f ). For any g(x),ℓ g (X, f ) = E 1 π Z (Z, X) 2Y Z δ(X) -f (X) - 2g(X)Z δ(X) 2 X = ℓ(X, f ) + 2E 1 π Z (Z, X) 2Y Z δ(X) -f (X) 2g(X)Z δ(X) X + E 1 π Z (Z, X) 2g(X)Z δ(X) 2 X = ℓ(X, f ) + 8 g(X) (δ(X)) 2 E Y π Z (Z, X) X -4 g(X)f (X) δ(X) E Z π Z (Z, X) X + 4 g(X) δ(X) 2 E 1 π Z (Z, X) XHere the second term and the fourth term don't depend on f , and the third term is 0 becauseE Z π Z (Z,X) X = x = 0. Therefore, argmin f L g (f ) = argmin f L(f ).Proof of Theorem 1. For any g(x), the variance of the derivative of the weighted loss at f = ∆ is given byVar 1 π Z (Z, X) 2 Y -g(X) Z δ(X) -∆(X) = E E 1 (π Z (Z, X)) 2 2 Y -g(X) δ(X) -Z∆(X) 2 X ≜ E[S(X, g)]Set the gradient of S with respect to g equal to 0. Then for any x ∈ X , we have The gradient of lg (x, f ) with respect to f is given by0 = E - 4 δ(x)(π Z (Z, x)) 2 2(Y -g(x)) δ(x) -Z∆(x) X = x 2g(x)E 1 (π Z (Z, x)) 2 X = x = 2E Y (π Z (Z, x)) 2 X = x -E Zδ(x)∆(x) (π Z (Z, x)) 2 X = x 2g(x) π -1 Z (1, x) + π -1 Z (-1, x) = 2µ Y 1 (x) π Z (1, x) + 2µ Y -1 (x) π Z (-1, x) -µ Y 1 (x) -µ Y -1 (x) π -1 Z (1, x) -π -1 Z (-1, x) 2g(x) π -1 Z (1, x) + π -1 Z (-1, x) = µ Y 1 (x) + µ Y -1 (x) π -1 Z (1, x) + π -1 Z (-1, x) g(x) = 1 2 µ Y 1 (x) + µ Y -1 (x) = 1 2 E Y π Z (Z, X) X = x Additionally, S is convex since ∂ 2 S ∂g 2 = 8 (δ(x)) 2 π Z (1,x)π Z (-1,x) . By Lemma 3, g * ∈ argmin g E[S(X, g)]. Proof of Theorem 2. Let lg (X, f ) = E 1 πZ (Z,X) 2(Y -g(X))Z δ(X) -f (X)∂ lg (x, f ) ∂f = 2E 1 πZ (Z, X) f (X) X = x -2E 1 πZ (Z, X) 2(Y -g(X))Z δ(X) X = x = 2f (x) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - 4 δ(x) π Z (1, x) πZ (1, x) µ Y 1 (x) -g(x) - π Z (-1, x) πZ (-1, x) µ Y -1 (x) -g(x)If πZ = π Z almost surely, then∂ lg (x, f ) ∂f = 4f (x) - 4 δ(x) µ Y 1 (x) -µ Y -1 (x) = 4 f (x) -∆(X) If g = g almost surely, then g(x) = [µ Y 1 (x) + µ Y -1 (x)]/2. Thus, we have∂ lg (x, f ) ∂f = 2f (x) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - 2 δ(x) π Z (1, X) πZ (1, x) µ Y 1 (x) -µ Y -1 (x) - 2 δ(x) π Z (-1, x) πZ (-1, x) µ Y 1 (x) -µ Y -1 (x) = 2 (f (x) -∆(x)) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) Check that π Z (1,x) πZ (1,x) + π Z (-1,x) πZ (-1,x) > 0. By convexity of lg (x, f ), ∆ ∈ argmin f lg (X, f ) in both cases. Proof of Theorem 3. Let lh (X, f ) = 1 πZ (Z,X) 2(Y -h(X,A,Z))Z δ(X) -f (X) 2 X. By convexity of lh and Lemma 3, it suffices to show that the gradient of lh (x, f ) with respect to f is 0 at f = ∆ in all cases. The gradient is given by 4). The resulting gradient is∂ lh (x, f ) ∂f = 2E 1 πZ (Z, X) f (X) X = x -2E 2Z πZ (Z, X) Y -h(X, A, Z) δ(X) X = x = 2f (x) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - 4 δ(x) π Z (1, x) πZ (1, x) µ Y 1 (x) -E[ h(X, A, Z)|Z = 1, X = x] + 4 δ(x) π Z (-1, x) πZ (-1, x) µ Y -1 (x) -E[ h(X, A, Z)|Z = -1, X = x] • If πZ = π Z almost surely, then we have the unweighted difference E[ h(X, A, Z)|Z = 1, X = x] -E[ h(X, A, Z)|Z = -1, X = x] = 0 by Equation (∂ lh (x, f ) ∂f = 4f (x) - 2(µ A 1 (x) -µ A -1 (x)) δ(x) ∆(x) -∆(x) -4 ∆(x) = 4 f (x) -∆(x) - δ(x) δ(x) ∆(x) -∆(x) It will yield 4(f (x) -∆(x)) if either ∆ = ∆ or δ = δ almost surely. • If μY 1 = µ Y 1 , μA 1 = µ A1 and ∆ = ∆ almost surely, and the choice of residualization function is h1 , then we have∂ lh (x, f ) ∂f = 2f (x) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - π Z (1, x) πZ (1, x) 2 ∆(x) + 4 δ(x) π Z (-1, x) πZ (-1, x) ∆(x)δ(x) -[2δ(x) -δ(x)] ∆(x)/2 = 2(f (x) -∆(x)) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) + π Z (-1, x) πZ (-1, x) δ(x) δ(x) 4(∆(x) -∆(x)) = 2(f (x) -∆(x)) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) • If μY -1 = µ Y -1 , μA -1 = µ A -1and ∆ = ∆ almost surely, and the choice of residualization function is h2 , then we have∂ lh (x, f ) ∂f = 2f (x) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - 4 δ(x) π Z (1, x) πZ (1, x) ∆(x)δ(x) -[2δ(x) -δ(x)] ∆(x)/2 + π Z (-1, x) πZ (-1, x) 2 ∆(x) = 2(f (x) -∆(x)) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - π Z (1, x) πZ (1, x) δ(x) δ(x) 4(∆(x) -∆(x)) = 2(f (x) -∆(x)) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) • If (μ Y 1 + μY -1 )/2 = (µ Y 1 + µ Y -1 )/2 and (μ A 1 + μA -1 )/2 = (µ A 1 + µ A -1)/2 almost surely, and the choice of residualization function is h3 , then the gradient is∂ lh (x, f ) ∂f = 2f (x) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) - 4 δ(x) π Z (1, x) πZ (1, x) ∆(x)δ(x) 2 -(δ(x) -δ(x)) ∆(x)/2 + 4 δ(x) π Z (-1, x) πZ (-1, x) - ∆(x)δ(x) 2 -(-δ(x) + δ(x)) ∆(x)/2 = 2 f (x) -∆(x) - δ(x) δ(x) (∆(x) -∆(x)) π Z (1, x) πZ (1, x) + π Z (-1, x) πZ (-1, x) It will yield 2 f (x) -∆(x)) π Z (1,x) πZ (1,x) + π Z (-1,x) πZ (-1,x) if either ∆ = ∆ or δ = δ almost surely.Consider linear model for ∆(x) with coefficients denoted by β. The objective function with outcome residualized by a function g(x) is defined as follows:L g (y, z, x, β) = 1 π Z (z, x) 2(y -g(x))z δ(x) -x T β 2 Let β * be the unique minimizer of Q(β) ≜ E[L g (Y, Z, X, β)].Let ℓ g (y, z, x, β) be the derivative of L g (y, z, x, β) with respect to β. Denote by β the root of the estimating equation n -1 n i=1 ℓ g (Y i , X i , β) = 0. By Bahadur representation, we haveβ -β * = n -1 H -1 n i=1 ℓ g (Y i , X i , β * ) + o P (n -1 )where H is the second derivative of Q(β) with respect to β at β = β * . Therefore, selecting the optimal g is equivalent to minimizing the variance of ℓ g (Y i , X i , β * ).Unlike IV-RDL1, we need additional constraints on h(x, a, z) to make sure the estimation for CATE remains consistent after the residualization. Lemma 4. For any measurable h : (X , A, Z) → R satisfying E Zh(X, A, Z) π Z (Z, X) X = x = 0 (4) or equivalently, E[h(X, A, Z)|Z = 1, X = x] = E[h(X, A, Z)|Z = -1, X = x], we have ∆ ∈ argmin f E 1 π Z (Z, X) 2(Y -h(X, A, Z))Z δ(X) -f (X) 2 . Proof of Lemma 4. Let ℓ h (X, f ) = E 1 π Z (Z,X) 2(Y -h(X,A,Z))Z δ(X)-f (X) 2 X . By Lemma 3, it suffices to show argmin f ℓ h (X, f ) = argmin f ℓ(X, f ). For any h(x, a, z) satisfying Equation ( 4), we haveℓ h (X, f ) = E 1 π Z (Z, X) 2Y Z δ(X) -f (X) - 2h(X, A, Z)Z δ(X) 2 X = ℓ(X, f ) + 2E 1 π Z (Z, X) 2Y Z δ(X) -f (X) 2h(X, A, Z)Z δ(X) X + E 1 π Z (Z, X) 2h(X, A, Z)Z δ(X) 2 X = ℓ(X, f ) + 8 (δ(X)) 2 E Y h(X, A, Z) π Z (Z, X) X - 4f (X) δ(X) E Zh(X, A, Z) π Z (Z, X) X + 4 (δ(X)) 2 E (h(X, A, Z)) 2 π Z (Z, X) XHere the second term and the fourth term don't depend on f , and the third term is 0. Therefore,argmin f ℓ h (X, f ) = argmin f ℓ(X, f )As shown in Lemma 4, the minimizer is invariant of a shift on outcome by a function h that satisfies Eq. ( 4). Similar to the way of finding ĝ * , we would like to find the function h with the smallest variance of[π Z (Z, X)] -1 [2(Y -h(X, A, Z))Zδ -1 (X) -f (X)] among all h that satisfies Eq. ( 4).Theorem 4. Among all measurable h : (X , A, Z) → R satisfying Eq. (4), the following function minimizes Var1 π Z (Z,X) 2(Y -h(X,A,Z))Z δ(X) -f (X) : h * (x, a, z) = µ Y (x) + ∆(x) 2 a -µ A (x) -zδ(x)if the conditional means µ Y (x) and µ A (x) is one of these three pairs:(1) µ Y 1 (x) and µ A 1 (x); (2) µ Y -1 (x) and µ A -1 (x); (3) m Y (x) ≜ (µ Y 1 (x) + µ Y -1 (x))/2 and m A (x) ≜ (µ A 1 (x) + µ A -1 (x))/2.Proof of Theorem 4. For any h(x, a, z) satisfying Equation ( 4), the variance of the derivative of the weighted loss L h (f ) at f = ∆ is given byVar 1 π Z (Z, X) 2 Y -h(X, A, Z) Z δ(X) -∆(X) = E E 1 (π Z (Z, X)) 2 2 Y -h(X, A, Z) δ(X) -Z∆(X) 2 Z, X ≜ E[S(X, Z, h)]whereS(x, z, h) = E 1 (π Z (Z, X)) 2 2 Y -h(X, A, Z) δ(X) -Z∆(X) 2 Z = z, X = xNow we seek to minimize E[S(X, Z, h)]. By convexity of S(X, Z, h) and Lemma 3, it suffices to show that the gradient of S(X, Z, h) with respect to h is 0 at f = ∆, if h is one of the three equivalent forms. To this end, set the gradient of S with respect to h to be 0. Then for any (x, z) ∈ (X , Z), we haveE - 4 δ(X) 1 (π Z (Z, X)) 2 2(Y -h(X, A, Z)) δ(X) -Z∆(X) Z = z, X = x = 0,which leads to the following condition on the optimal h:E[h(X, A, Z)|Z = z, X = x] = E[Y |Z = z, X = x] -z∆(x)δ(x)/2 (5) Since δ(x)∆(x) = µ Y 1 (x) -µ Y -1 (x), it can be verified that Equation (5) implies E[h(X, A, Z)|Z = 1, X = x] = E[h(X, A, Z)|Z = -1, X = x],which is equivalent to Equation (4). We will then verify that the following three equivalent functions satisfy Equation (5).h * 1 (x, a, z) = µ Y 1 (x) + ∆(x) 2 a -µ A 1 (x) -zδ(x) h * 2 (x, a, z) = µ Y -1 (x) + ∆(x) 2 a -µ A -1 (x) -zδ(x) h * 3 (x, a, z) = µ Y 1 (x) + µ Y -1 (x) 2 + ∆(x) 2 a - µ A 1 (x) + µ A -1 (x) 2 -zδ(x)To see their equivalence, notice that∆(x) = 2[µ Y 1 (x) -µ Y -1 (x)]/[µ A 1 (x) -µ A -1 (x)]. Then we have µ Y -1 (x) -µ A -1 (x)∆(x)/2 = µ Y -1 (x)µ A 1 (x) -µ Y 1 (x)µ A -1 (x) µ A 1 (x) -µ A -1 (x) = µ Y 1 (x) -µ A 1 (x)∆(x)/2,and h * 3 is simply the average of h * 1 and h * 2 . It suffices to show h * 1 satisfies (5). We haveE[h * 1 (X, A, Z)|Z = 1, X = x] = µ Y 1 (x) - ∆(x) 2 δ(x) E[h * 1 (X, A, Z)|Z = -1, X = x] = µ Y 1 (x) + ∆(x) 2 µ A -1 (x) -µ A 1 (x) + δ(x) = µ Y -1 (x) + ∆(x) 2 δ(x),which completes the proof.In this section, we conducted simulations that evaluate the performance of the proposed framework against model mispecification on the nuisance variables. The data is generated by the same model as Setting 1 in Section 6, except that π Z (1, X) = expit{2X 1 }. Based on the true model, the conditional mean outcome is non-linear on X. However, in Setting 3, we will use its OLS estimate as a case of misspecification. In Setting 4, we deliberately used a wrong propensity πZ (1, x) = 1/2. We keep all the other procedures the same as Setting 1. The results are summarized in Table D . We can observe that the residualized version have superior performance, and have significant lower MSE compared to the original version. .5 (0.9) Value 74.5 (1.2) 84.9 (0.6) 87.3 (0.8) 86.3 (0.8) 84.8 (0.7) 83.5 (0.9) 92.6 (0.7) 87.9 (0.9)In the data analysis, we construct a 3-dimensional plot for the estimated CATE based on the three splitting variables (age of mom at census, age of mom at first birth, and income of father). The plot is presented in two rotations in Figure 4 . The points in the plots are color-coded by the estimated CATE with red indicating more likely to work and blue indicating more likely to not work. We can see that, overall, blue points are at the bottom of the plots, with a majority of them below $25k/year. Subgroup 3 of young mothers with extremely low fathers' income only accounts for 3% of the data and hence is hard to see here.  


Decision: This paper proposes a new CATE estimator (variants) in the instrumental variable setting using direct learning, under the setting when the instrument may be confounded by an observed variable. The paper provides identification conditions, a multiply robust estimator and conduct empirical evaluation on synthetic simulations and real world data. Theoretical results provide standard convergence guarantees of the doubly/multiply-robust estimator.

Overall all reviewers found the paper clear, comprehensible. Deriving optimal treatment regimes in addition to CATE was considered a strength. 

Multiple reviewers raised some concerns about Assumption 2f, particularly in the context of existing literature of Wang and Tchetgen Tchetgen (2018) (as well as its justification of additive effect of unobserved confounders on treatment --- or outcome as was clarified). Impact of high-dimensional confounding was raised, and addressed to some extent (doubly robust estimators still have empirical limits in practice, but this is sufficiently addressed for the paper for now). 

Overall I have ignored the uninformative review from iUwm and recommended accept in this case.

Reviews: review: paper_summary: paper_summary: This paper introduces a new type of CATE estimator using instrumental variables. The proposed method employs the direct learning approach.
strength_weakness: strength_weakness: __Missing literature review__ 

The paper most closely related to this work is [Machine Learning Estimation of Heterogeneous Treatment Effects with Instruments[(https://proceedings.neurips.cc/paper/2019/file/3b2acfe2e38102074656ed938abf4ac3-Paper.pdf). It develops a fast-converging CATE estimator for local average treatment effects using instrumental variables. However, this paper is not cited in the literature review. Please consider including a discussion of this paper for richer context. More importantly, please compare your work with this paper to highlight the novelty of the current work.

__Weak motivation on directed learning__ 

The introduction section lacks plausible reasons for proposing directed learning. What are the alternative methods and their pros and cons? Why should we specifically consider the directed learning approach?

__Validity of Assumption 2__

Assumption 2f is a weaker version of the following assumption: "$U$ is noninformative to $A$ given $Z$ and $X$ (i.e., $A \perp U \mid X,Z$)." Given that there are no practical settings where Assumption 2f holds while $A \perp U \mid X,Z$ doesn't (except some peculiar parametrization), and both assumptions are non-testable, I don't see any practical distinction between $A \perp U \mid X,Z$ and Assumption 2f. In other words, Assumption 2f is just another representation of $A \perp U \mid X,Z$ tailored for identification.

Combining $A \perp U \mid X,Z$ with Assumption 2c ($Z \perp U \mid X$) results in $(U \perp A \cup Z \mid X)$ by the contraction property of conditional independence. This means $U$ does not influence $(A,Z)$ given $X$. Consequently, in any related causal graph, there should be no edges from $U$ to $A$. This leads to the ignorability condition that $Y(a) \perp A \mid X$.

In summary, interpreting Assumption 2f as $A \perp U \mid X,Z$ means Assumption 2 is essentially an ignitability assumption. Therefore, it is important to discuss the validity of Assumption 2 in practical settings, to disprove that Assumption 2f is merely another representation of $A \perp U \mid X,Z$ designed for identification. Have you considered the LATE setting, given that the estimand in Proposition 1 will remain unchanged?

__More analysis is required__

Multiple robustness properties provided in Theorem 3 imply that the proposed estimator converges to the optimal estimator faster. For example, if nuisances converge at an $n^{-1/4}$ rate, where $n$ is the number of samples, then the estimator converges at an $n^{-1/2}$ rate. These results are beneficial since they guarantee fast convergence. While Theorem 3 is attractive, it is somewhat impractical because, in practice, the working model is rarely considered a true model. Please provide more analysis on the rate of convergence concerning the convergence rate of nuisance parameters.

__Fair comparison with other estimators__

Even if the empirical evidence in Table 1 is strong, the discussion on why the proposed estimator converges faster than other multiply-robust estimators, such as MRIV, is missing. Asymptotically, there are no reasons to believe the proposed estimator converges faster than the MRIV estimator. Can you provide a discussion on why the proposed estimator converges faster than its competitors?
questions: questions: 1. $\Delta(x)$ in line 89 and $\Delta(x)$ in line 94 are the same?

2. What are the practical examples where Assumption 2 holds? 

3. Is there a reason to choose Assumption 2 other than the LATE assumption? Both assumptions yield the same target parameter (in Proposition 1).
limitations: limitations: 1. The paper assumes discrete/binary $Z$. 

2. The paper is relying on Assumption 2.
paper_summary: paper_summary: The authors study the problem of estimating the conditional average treatment effect (CATE) under the assumption of unmeasured confounding. The authors focus on the specific scenario where some observed variable acts as instrument w.r.t. unmeasured confounder but might be confounded by some other observed confounder, so that standard IV methods may fail. They derive a method which extends Direct Learning (DL) by an additional scaling factor of the outcomes. This scaling factor is the CATE of the instrument on the treatment, which can be estimated with standard methods (e.g., DL). In a simulation study, the authors compare the proposed method to a set of baseline methods.
strength_weakness: strength_weakness: Assumption 2.f seems rather strong as the unobserved confounder can only additively affect the treatment. The data generating process in the experimental section violates this assumption. As the proposed method still outperforms the baselines, this may suggest that the method is less sensitive to the assumption. Though, that should be studied empirically in more detail.

Compared to other recent publications focusing on estimating the (conditional) treatment effect, the assumed data generating process in the simulation seems overly simple. Other methods involving GPs, normalizing flows, and other highly non-linear models allow for high-dimensional confounders. They are typically assessed using semi-artificial data (e.g., with images as confounders and image labels as confounding mechanism). Without such hard problems and the corresponding baseline methods, it is hard to assess the overall practical value of the proposed method.
questions: questions: The paper is clear; no questions but some minor comment:
- The term instrumental variable for Z might be a bit confusing as Z and Y are confounded; Z is an IV w.r.t. unmeasured confounder. It may help to clarify that in the very beginning.
- Figure 1 might be improved in terms of order and size of the nodes; having the instrument on the right, the treatment at the top, and the outcome on the left is rather unconventional.
- lines 151-164 have likely little value as this should be known to the Neurips audience
limitations: limitations: NeurIPS Paper Checklist is provided; no concerns.
paper_summary: paper_summary: The author mainly introduces a method of Directly Learning using Instrumental Variables (IV-DL) to estimate the conditional average treatment effect (CATE) $\Delta(x)$ and optimal Individualized Treatment Regime (ITR) $\hat{d(x)}$ in the presence of unobserved confounding. They propose two efficient and robust estimators, IV-RDL1 and IV-RDL2, by residualizing the outcome. The authors conduct two simulation settings and use real-world data to demonstrate the efficiency of the approach.
strength_weakness: strength_weakness: I think overall the authors did interesting research, but my main concerns are listed below.

The variables A, Y, and Z in this paper are all binary variables. It would be beneficial to discuss how the framework of IV-DL can be extended or adapted to handle a continuous instrumental variable 𝑍. The current focus on binary variables may limit the generalizability of the findings.

In Section 5, the authors do not provide a detailed introduction to work similar to IV-RDL2. A more thorough explanation of the differences and connections between this work and related research would enhance the reader's understanding of the unique contributions and context of the presented study.
questions: questions: 1. In section 6, why is the performance of IV-RDL1 than IV-RDL2 in three metrics?

2. The authors should conduct some experiments to show whether IV-RDL1 and IV-RDL-2 are robust compared to other methods in the presence of misestimation.

3. The authors miswrite “$f(x)=\tilde{x}^{T}\mathbf{\beta}$” as “$\Delta(x)=\tilde{x}^{T}\mathbf{\beta}$” in line 152.
limitations: limitations: The authors discuss some of the algorithm's shortcomings.
paper_summary: paper_summary: In this paper, the authors proposed a general framework for estimating CATE with a possible unmeasured confounder using instrumental variables. They construct estimators that exhibit efficiency and robustness against various scenarios of model misspecification. The efficacy of the proposed framework is demonstrated through simulation studies and a real data example.
strength_weakness: strength_weakness: Please refer to Questions.
questions: questions: The authors might consider scenarios where one might be interested in heterogeneous treatment effect on $X'$, where $X'$ is a subset of $X$.
limitations: limitations: Please refer to Questions.

Comments: title: Response to Question
Response
Response to Question
Response to Comment
One quick question
Response
comment: Thank you for your feedback and for raising your score.

We understand that you would like some explanation for the better performance of the proposed method for the finite samples in the simulation. One possible reason is that our proposed IV-RDL1 method requires fewer nuisance estimates than MRIV. Another reason may be that in all our proposed methods, we use inverse propensity score $1/\pi_Z(Z,X)$ as the weights in a weighted least square framework. This helps to balance different IV groups and helps to reduce bias and variance (see Seaman and White, 2013).
Thank you for addressing my concern. I will raise my point from 5 to 6, given that the end of the discussion period is coming. 

My final question is this: Can you discuss the fast convergence shown in the experiment? In theory, MR-IV and the proposed estimators both converge fast, but the proposed estimator outperforms in the simulation. Can you justify this simulation result?
Thank you for your follow-up question. We appreciate your interest in understanding the challenges associated with extending our framework to accommodate other types of treatments. We will briefly highlight the key aspects of identification in the binary case and then explain the challenges involved in generalizing to other types of treatments.

To begin with, we will need the following notations:
- $\Delta(X)=E[Y(1)-Y(-1)\vert X]$
- $\delta_Y(X)=E[Y\vert Z=1,X]-E[Y\vert Z=-1,X]$
- $\delta_A(X)=P[A\vert Z=1,X]-P[A\vert Z=-1,X]$
- $\tilde\delta_Y(X,U)=E[Y(1)-Y(-1)\vert X, U]$
- $\tilde\delta_A(X,U)=P[A\vert Z=1,X, U]-P[A\vert Z=-1,X, U]$

The proof of Proposition 1 demonstrates that, in the binary case, the identification on the Conditional Average Treatment Effect (CATE), denoted $\Delta(x)$, hinges on the following relationship:
$$\delta_Y(X)=E_U[\tilde\delta_Y(X,U)\tilde\delta_A(X,U)]=\Delta(X)\delta_A(X)$$
Here, Assumption 2f provides a sufficient condition for the validity of the second equation. Then we have identified the CATE: $\Delta(X)=\delta_Y(X)/\delta_A(X)$.

For a $k$-arm treatment scenario, a natural approach involves selecting one treatment arm as the baseline and defining the CATE as the difference between each of the other arms and this baseline. This results in a CATE vector of dimension $k-1$. Extending Assumption 2f to accommodate this setup and maintain the equality $E_U[\delta_Y(X,U) \delta_A(X,U)] = \Delta(X) \delta(X)$ is not straightforward. In particular, this identification equation will become a system of linear equations, whose solution requires the inversion of a $(k-1)$ dimensional square matrix. Hence, we feel that this would be too complicated to incorporate into the current papers as an additional section; rather, it deserves a separate paper.

The challenge increases with the generalization to continuous treatments, as the existing identification relies on differences between conditional means given two discrete IV levels. This suggests the need for novel theoretical frameworks or assumptions tailored to these more complex scenarios.
Thank you for informing us that your concerns about the motivation and justification of Assumption 2f have been resolved. We value your continued engagement and feedback. Below are our responses to the remaining questions and suggestions:

## Remaining Questions on Convergence Rate

We would like to clarify that we did not claim the proposed estimator has a faster convergence rate. Our theoretical results only establish consistency. We will review our text to ensure it does not suggest that we provide convergence rate results. While we are not aware of similar findings in the existing literature, especially related to Wang and Tchetgen Tchetgen (2018), this could be an intriguing direction for future research.

## Suggestions to Strengthen the Paper

1. **Motivation for Direct Learning in Introduction:**  
  We will move the explanation about the benefits of direct learning over traditional methods like Q-learning to the introduction section as suggested to improve the motivation.

2. **Including Rate of Convergence:**  
  We agree. As mentioned, while we are not aware of similar findings in the existing literature, especially related to Wang and Tchetgen Tchetgen (2018), this could be an intriguing direction for future research.
Thank you for your response. Regarding the binary case, you mentioned, "During our research, we discovered that extending our framework to accommodate other types of treatment is a non-trivial task." Could you please provide more insight or intuition on why this is the case?
Thank you for your response. My concerns about the motivation and the justification of Assumption 2f have been addressed. However, the following questions remain:  
1. Can you provide a discussion on why the proposed estimator converges faster than its competitors?  
2. What is the rate of convergence in relation to the convergence rate of nuisance parameters?  

I believe the paper could be stronger if 
1. the motivation of the direct learning is more clearly explained in the introduction section, and 
2. the rate of convergence is added. 

By the way, the current answer misses the response for my question: Can you provide a discussion on why the proposed estimator converges faster than its competitors?