Double Machine Learning Density Estimation for Local Treatment Effects with Instruments
Authors: Yonghan Jung
Jin Tian
Elias Bareinboim
Venue: NeurIPS 2021 Spotlight (2021)
Keywords: instrumental variables
local treatment effect
complier average causal effects
density estimation
double/debiased machine learning
semiparametric inference

Abstract: Local treatment effects are a common quantity found throughout the empirical sciences that measure the treatment effect among those who comply with what they are assigned. Most of the literature is focused on estimating the average of such quantity, which is called the ``local average treatment effect (LATE)'' [Imbens and Angrist, 1994]). In this work, we study how to estimate the density of the local treatment effect, which is naturally more informative than its average. Specifically, we develop two families of methods for this task, namely, kernel-smoothing and model-based approaches. The kernel-smoothing-based approach estimates the density through some smooth kernel functions. The model-based approach estimates the density by projecting it onto a finite-dimensional density class. For both approaches, we derive the corresponding double/debiased machine learning-based estimators [Chernozhukov et al., 2018]. We further study the asymptotic convergence rates of the estimators and show that they are robust to the biases in nuisance function estimation. The use of the proposed methods is illustrated through both synthetic and a real dataset called 401(k).

Double Machine Learning Density Estimation for Local Treatment Effects with Instruments
Yonghan Jung Purdue University j jung2220purdue edu
Jin Tian Iowa State University jtian@iastate. edu
Elias Bareinboim Columbia University eb0cs columbia a edu
Abstract
Local treatment effects are common quantity found throughout the empirical sciences that measure the treatment effect among those who comply with what they are assigned. Most of the literature focused on estimating the average such quantity, which is called the "local average trealment effect (LATE)" (31). In this work, we study how to estimate the density of the local treatment effect, which is naturally more informative than its average. Specifically, we develop two families of methods for this task, namely, sernel-smoothing and model-batt based approaches. The seateell-mooohoobaaaseeee approach estimates the density through some smooth kernel functions. The model-based approach estimates the density by projecting it onto derive the corresponding double/debiased machine learning-bast based estimators [13]. We further study the asymptotic convergence rates of the estimators and show that they are robust to the biases in nuisance function estimation. The use of the proposed methods is illustrated through both synthetic and real dataset called 401(k).
density class. For both approaches, we
Introduction
Controlled experimentation is one powerful tool used throughout the empirical sciences infer the effect of a certain treatment on given outcome. The idea to randomize the treatment assignment so neutralize the effect of unobserved confounders. However, in some practical settings, may be challenging to ascertain that individuals who are selected for treatment will follow their recommendations, Issues of non-compliance and unmeasured confounding are quite common and
lead to the non con-identification of treatment effects in many real-world cases 29 50 32, [6].
An approach known as instrumental variables (IVs) has been proposed to try cireumvent this issue [68]. The idea is to find a set of variables (possibly singleton) that are not the target of the analysis by itself but that will help control for the unobserved confounding between the treatment and the outcome. In particular, IVs are special variables that (i) are correlated with the treatment, (ii) do not directly influence the outcome, and (iii) For concreteness, consider a study of the effect of 4K1(K) participation (X) on the distribution of financial assets (Y) (2]. This setting is represented in the causal graph in Fig. [ Note that dashed-bidirected arrow exists between X and Y, which in graphical language represents unobserved confounding affecting both X and Y. The variable Z in this model represents the eligibility of (0).)). We note that Z qualifies as an instrument in this case (i) it does affect the participation of (01 (X) and (ii) has no direct influence on the net financial asset (Y), (ii) is not affected by unmeasured confounders between X and Y. The variable W represents observed covariates (e.g., gender, age, ethnicity, income, family size).
affected by certain unmeasured confounders.
We are interested the particular setting where only individuals who were offered the treatment may have access (3]] For instance, in the case of 401 participation (X 1), only eligible
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Figure 2:
Densities of outcome Y among compliers under the treatment X 1. All densities have mean O and a variance 2.
individuals (Z = 1) would be allowed to join the program. This assumption known in the literature monotonicity, which rules out the possibilityy that any units would respond contrary to the instrument. Under monotonicity, the causal effect in the subpopulation whose actual treatment X coincides with the assigned treatment Z (called compliers) is identifiable [3]] 2]. The average treatment effect (ATE) for the compliers is called 'Lcca ATE (LATE) (or Complier average causal
effects, CACE) 31
The most common quantification of these effects in settings found in practice is the average (e.g.. LATE). The average is certainly an informative summary; however, may fail to capture significant differences in the causal distributions the outcome. For instance, consider Fig. that shows the densities of outcomes Y under treatments X among compliers which are generated from samples drawn from four synthetic data generating processes represented by the IV graph in Fig. (further discussed in Sec. 2e..5..om All of the four distributions have the same mean 0 and variance 2. However, the difference in the LTE distributions self-evident.
Most of the prior work on quantifying distributions of treat- ment effects focuses on estimating cumulative distribution functions (CDFs) or quantiles, and little attention has been given to estimating densities (refer to Sec. [.I for further comparison). As a complement to CDFs, densities have vari- ous advantages, including a more interpretable visualization of the distribution and generative capability of producing samples. One challenge with estimating densities that while CDFs netbiise-differenttable and enjoy Vn-rate estimators (n the size of data), densities are not (i.e., they non-regular), and therefore possess influence functions nor /n-rate estimators without approximations 7 Ch. 3].
W
Z
Figure 1: A causal graph for the IV setting. Bidirected arrows encode measured confounders.
In this paper, our goal to provide methods to estimate densities of local treatment effects in IV settings under the monotonicity assumption. We develop two families of methods for this task based on ternel-smoothing and model-based approximations. The former smooths the density by convolution with a kernel function: the latter projects the density onto a -iite-dimensional density class based on a distributional distance measure. For both approaches, we construct doublc/debiased machine learning (DML) style density estimators 4g convergence properties of the estimators, showing that they can converge fast (i.e., vūi-rate) even when nuisance estimates converge slowly (e.g., n 1/4 rate) (a property called 'debiasedness' D. We illustrate the proposed methods on synthetic and real data.
54 52
70 13]. We analyze the asymptotic
1.1 Related work
Our work touches different areas, which we will discuss next.
Double/Debiased Machine Learning (DML) [3] based causal effeet estimators. The DML framework has been adapted for estimating the average causal effect under the setting where the back-door criterion [50] Sec. 3.3.1] (also known as ignorability (57]) holds (e.g., [1] Recently, DML-based causal effect estimators have been developed for any identifiable causal functionals in
given causal graph and equivalence class thereof B3 34
Also known 'nomparametric doubly robust
37 'rate doubly robust'
599

---

Local average & quantile treatment effect. The formal identification results for LATE under the monotonicity assumption in IV settings were developed by [3T Building on these results,
semiparametrie estimation for LATE has received remarkable attention 12
60 23
62
48 including
robust LATE estimators that achieve debiasedness 47
64 As shown in Fig. however,
the average is sometimes insufficient to capture the effects of the treatment on the distributions of outcomes. To address this issue, the problem of estimating quantiles or CDFs has taken attention. A common approach to estimate quantiles or CDFs is based on the LATE estimation. Since the expectation of yy (Y), an indicator that outcome falls short of threshold y, reduces to the CDF (i.e., replacinn Y in LATE with y (Y)), estimators for the LATE can be used to estimate quantiles or CDFs [5 24 16
30
45 18, 69].
Non-regular target estimand. An estimand that possesses no influence functions nor v /n-rate estimators called 'non-regular' Densities are an example of non-regular target estimands 7 Chap. 3]. One can approximate a non-regular target with smooth one such that an influence function and vn-rate estimators can be derived. Two broadly used approaches are sernet-smoothinggboe -based (e.g..
52
19 35))
and model-based (e.g.. 46 52 21
40 39]
Causal density estimation. There limited literature on estimating the density of treatment effects.
22 used the sernel-smoothin technique to estimate the density of a treatment effect, and 42 provided kernel- smoothing based density estimator that achieves doubly robustness and debiasedness building on top of the work in [53]. Recently, (39] investigated a model-based approach and developed estimators that
Most of the results assume that the ignorabilityybackkoo
admissibility holds [55
achieve debiasedness properties. Under the IV setting, [T0] provided local polynomial regression- based density estimator for local treatment effects; We are not aware of any work studying debiased density estimators. As mentioned, this paper investigates both keroel-s and model- model-based approaches for estimating local treatment effects under IV settings and develops DML-style density estimators for both.
2 LTE Estimation Problem setup
In our analysis, each variable is represented with a capital letter (X) and its realized value with a small letter (z). For a discrete (e.g.. binary) random variable X, we use (x) to represent the indicator function such that (xX) ) = if = x; Ix(X) = otherwise. For a continuous variable X with probability density p(z) of a distribution P' and a function fx)) Ep [f(X)] Sxx (z))((z) d[æ] where X' is the domain for X, and [x] = v [Er(([())) f is said to converge to f at rate rn if [1((x)) [xx] (p(//r.) For a dataset D (V:) we use ED [S(V) = (1/n) E (VV) to denote the empirical mean of ((V) with D.
Structural Causal Models (SCMs). We use the language of SCMs as our basic semantic and inferential framework (50 4). An SCM M is a quadruple M = (U, V. P(U), F") where U is a set of exogenous (latent) variables following joint distribution P(u). and V is set of endogenous (observable) variables whose values are determined by functions F = =fvv Jv,ev such that V + fv, (pa,, u,) where PA S V and U, C U. Each SCM M induces distribution P(u) and causal graph G = ((M) over V in which there exists a directed edge from every variable in PA, to Vi and dashed- bidirected arrows encode common latent variables (e.g., Fig. [.. Within the structural semantics, performing an intervention and setting X = r is represented through the op-oraao, do(X = æ), which encodes the operation of replacing the original equations of X (i.e., fx (paz, uz))) by the constant x and induces a submodel M,, and an interventional distribution P(e do(x)). For any variable Y E V. the potential response Yz (1) is defined as the solution of Y in the submodel My given U = u, which induces a counterfactual variable Y,-
Local Treatment Effect (LTE) with IV. We consider the IV setting represented by the causal graph G in Fig. ff where Z is a binary instrument with domain (0. 1}. X is a binary treatment with domain (0, 1). and Y is a (set of) continuous outcomes with bounded domain y C Rd, and W is a set of covariates (continuous, discrete, or mixed). G satisfies the IV assumption that Z has no direct influence on outcome Y and is not affected by unmeasured confounders between X and Y.
Iit is common in the literature to define IV assumptions in terms of conditional independences among whose connection with the causal graph in Fig. [ is discussed in
counterfactual [5 9 Assumption A.]
g
60 47
3

---

The causal density p(y=) is not identifiable from the observed density p(z, 2, 2,2,2)))))) w) due to the unobserved confounders between X and Y. However, the effect is possibly recovered for certain subpopulation under additional assumptions. Formally, a unit in the population is an always-taker if Xz=1= = Xz=0 = 1, a never-taker X == Xz=0 = O, complier if Xz=1 =1 Xz=o = O, and a defier if Xz=1 = 0, === = 1 23.
We will make the following assumptions based on literature.
Assumption (Monotonicity). There are no defters: X 2 Xz=0-
Assumption (Positivity).. P(z|z, w) 0. P(zlw) > 0 for any x, 2, w.
Let C' denote the event that unit is complier (i.e.. a unit such that Xz=u 0 and Xz=1 1). For a given constant and variable X. let x denote the event X = a. The LTE p(y=(C) is identifiable under monotonicity and is given by 21:
Ep [p(y|r, z", W) W) p(yla, 2 W)P(sk W)] Er [( W) P(x W)]
plu CC
(I)
where the expectation is over W. In this paper, we aim to estimate the LTE density p(uz (C) in Eq. (T). We will make the following mild assumption some densities, popularly employed in density estimation literature (e.g.. [44 25 27
61 26 42)).
Assumption 3. For any z, 2, w. y, densities p(ylu; w. 2, x)) plylz, x) and P(y=(C) are bounded, and C() is twice differentiable.
DML method. Let v = vhp' denote a functional of an arbitrary distribution P'. We use P to denote the true distribution such that 'D ~ P. Let vh = 1hp denote the true parameter to be estimated. To estimate vo DML-based estimators use Neyman Orthogonal score <pVV; tho 1) (where 1) = TJPr is set of nuisance parameters and 1/0 = T)p denotes the true nuisances), a function such that Ep (<PVV; vo, 710] = 0, (0/ar)).. Ep [VVV;; t/o 17)] = O. Given sp an DML estimator is constructed using the cross-fitting technique as follows: For randomly split halves of DD denoted {Do, D1). let np for p E {0, 1} denote the estimates for 77 from Pp. Let Tp denote a solution such that Ep,-, (e p. ]p = OpNNN Then, T = (To + /22/2 is an DML. estimator [3] Def. 3.1]. In addition to being consistent, the estimator T exhibits robustness property called debiasedness: T converges to bhy in the root-N rate even when i) converges to no in slower N rate [3] Thm. 3.1].
Neyman Orthogonal Score can be derived by adding v to its influence function 6 [14] Thm. 1]. An influence function of the functional vp is defined as a solution satisfying Ep [Ø] 0, Ep [6²)
oc,
and /attttt)rrr le= Ep [o(V; th, SV; bounded mean-zero functions g(-) over V. and S, = 0) = (0/at) log P.(u)le=o [63 Chap. 25].
0)] where P: = (vv) (1 + (v)) for t E IR and any
Due to space constraints, all the proofs are provided in Appendix B in suppl. material
3 Kernel-smoothing-baseee approach
In this section, we develop a soanell-moothing-baeee based approach for estimating the LTE density, The kernel-smootine technique approximates a non non-pathwise pathwise-difffreniiid differrnniialle target estimand with a differentiable estimand by convoluting the density with a kernel function K(y) Properties of
the kernel function includes symmetry about the origin (i.e., Jy yK(y)) (5) d'y] = 0). non-negativity (0 ((y) o0, Vy E )), and integrates to (i.e., Jy K(y) d'yl = 1) [66] Chap. 4.2].
We consider a product kernel (n,()')) = = -I K (v; -,)//)) with given bandwidth fa E RR and fixed point y { Rdd We assume that the kernel of interest has a bounded second moment and norm: i.e., (K) = ly Kw)) dly] < oo and [[(())] < oc following 27 61). Example of kernels include Gaussian kernel: K(u) = (L/V2T)) exp (-u²/2) for E R. Epanechnikov kernel: K(u)= = (3/4)(1 u²) 1m1s (u), Quadratic kernel: K(u) (15/16)(1
u 2 ( (u). Cosine kernel: kuu)= (T/4) cos (Tu/2) 1 |uls] (1), etc.
For convenience, we denote the target estimand by (1()) p(v= IC). In the smootl-- basedd approach, we will aim to estimate smoothee approximation for (b)) defined as follows:
(n(2)) = J, v0(1)) ((W)KKn,) Kn. ( db/ K(Kn,y(Y)
(2)

---

where v[f()) is an expectation of a function f(Y) w.r.t. vyy)). which is specified as
+[f(Y]] = Ep [ [ (X)]= W] Ep ( W]] Ep P ((>1222 | W) - P(=1|20 W)l
(3)
The second equality in Eq. is by Eq. For target estimand ())f()), we will denote nuisances by Tz (w) = P(zlw), Eor (z, w) = P(r 2, w) and (zz, 2, ()[[( = I2p (x)) w), shortly (,, 0).
We aim construct a DML estimator for the estimand vhg Toward this goal, we will first derive Neyman orthogonal score for 4h- Since a Neyman orthogonal score can be constructed based on moment score functions function of parameters such that expectation is the true parameters) 14 Thm. 1], we start by defining the moment score function. Let
d = Ep ( W) - (2, w)],
(4)
1 (Z) (Z) (1,) Est (Z, + (Ex1 W) - (x", w)}- (5)
Vx {
#2 (W)
Then, the following moment score function for vh=
m(d/:
&x (1h v/ Vx.
(6)
where th is given in Eq. (2 and vf is estimate of thn-
Next, we derive an influence function for the moment score function :(V/;ta) We first define the following function: for bounded function S(Y) < oc, let
XX [f(()] Ep [0(x, 2", W)(f(Y)) - -, W)
(7)
L.- (Z) 121- (Z) (X) - ((),)))))
Vyx((=, ((II())
Tz(W)
+ 10(2, (W)(f(Y) 0(x, 21=,2"-1 ()][(()))
(8)
and
601= (m, ()((() Φx (
-
where Vx is defined in Eq. ( Then, the influence function for the expectation of the moment score function m (1// (h)) in Eq. 66 is given follows:
Lemma (Influence function for m(w w)))). Les m(w whn) be the score defined in Eq. o Then, the influence function for Ep (m((k"; 1hh)), denoted @m given by
Om {,, v) oln, +) (n,
(10)
where o is in Eq. 9
For any score function (c.g., m in Eq. (6) ) its addition to the influence function of the expected score (c.g., Qm) is Neyman orthogonal score (14 Thm.1], [13] Sec. 2.2.5]). Specifically.
Lemma 2 (Neyman orthogonal score for th.). Let mill vh)) be the score function in Eq. (6). and dm (n {,,E, tn) be the influence function for Ep [m(1"' rhh)] given in Eq. (IO). Then, a Neyman orthogonal score for th is given as plv n = {,,,, 0}) = m + (m, v);; Specifically,
1
"(W': (T. = vx (rx((m,0)[(K ( (Y)) & (x((
(11)
Given the Neyman orthogonal score ((v/:n) an estimate dn. satisfying
ED (p(sin: n (*, { {}) op(n-1/2 gives DML estimator. Specifically, we propose the following sernel-smoothingg based estimator for the LTE density, named 'KLTE' (kernel-based estimator for LTE):
AA Neyman orthogonal score is function á satisfying [p((( mp) 0 and where yo denotes the true nuisance [ Def.2.2). In words, a score function that is not sensitive to local errors in nuisance models.
@ Ep [o( m)]l 0.
5y,
5

---

Definition 1 (KLTE estimator for th), Let p(p::-"=-=----: { be the Neyman orthogonal score
for 4h given in Eq. 77.. Let {D, }'} denote the randomly split halves of the samples, where
1 = n. Let n = { {) denote the estimates for the nuisance yj using D'. Then, the KLTE estimator for (y) for all E V denoted vi (y), is given by
(n ED (vx, (Y)) /Ed (xx(tx,())
(12)
where Vx and Vyx x are given in Eqs. 5
respectively.
We will show that the KLTE is DML estimator exhibiting debiasedness property. Detailed asymp- totic properties discussed next.
3.1
Asymptotic convergence
Now, we study the convergence rate of the estimator th (v)- For any fixed V, the error di (v) (t)) will be analyzed in two folds: we will first analyze the error between the estimator Eq. 12 and the smoothed estimand in Eq. (2) (i.e., tha (v) - 4/ (v)). and then analyze the error between the smoothed estimand and the true estimand (i.e., 44 (1)).
The following result gives the error analysis for v. (n(:):
Lemma 3 (Convergence rate of vij to th). For any fixed y y, suppose the estimators for muisances are consistent; Le., || - Al| = or(1) E = { E, 0) for all (w, 2,z). Suppose
0o, and nh + oo as n + co. Then,
wn(y) ()) = =p (1/Vnhé + RS + 1/vmi)
where
R Σ | Tell {10= - 0g 02|+ -Ea E=
(13)
where n, = Tw),,, E. (z, W) and 02 0(xx 2, [)(Kn,
The eтror analysis in Lemma. 3 iimplies the following:
forany Corollary 1 (Debiasedness property of oh to vhn). Ifall nuisances { (,,5,)) for any given (w,, 2, v) converge at rate {nh43-1/4, then the target estimator (y((y) achieves Vnhe-rate convergence thh
We now analyze the gap between the smoothed estimand thy and the true estimand th; i.c., th v:. Lemma 4 (166) Thm. 6.28)). The following holds:
2 (v) (y) = By 0.5h² ((P)))())))))))) 10%y
+ O(h²).
(14)
Combining the results of Lemma.
we have the following result:
Theorem 1 (Convergence rate of So to e.), For any fixed E V, suppose the estimators for
muisances are consistent; Le.,, ||V- 2|| (r(1) v E = E, for all (w, 2,)). Suppose
0o, and nh d + 00 as n + 0o. Then
(n(y)) (1yy = Or (1/Vnhe + RS +1//m) + By:
(15)
where By is defined in Eq. 14), and R2 is defined in Eq.
13
Thm. Iimplis that (2) converges fast (see Corol. (vy)
By. A natural question is then how The following provides a guideline in
to choose the bandwidth h that minimizes the gap in Eq. 15
choosing the bandwidth h:
Lemma 5 (Data-adaptive bandwidth selection). The bandwidth h that minimizes the error in Eq. (15) h O(n (n-11/(-11)) This choice of satisfies the assumption in Lemma L nhe oo).
Recall that Corol. states the debiasedness property of ti th, for any bandwidth h satisfying nhd > sc. With the choice of in Lemma dh converges to d with the debiasedness property preserved.

---

Corollary 2 (Debiasedness property of Sn to v) Let = 000 0(n-1/(444)) If muisances { 0. converge at 1-14 rate for any (w, 2, v). then the target estimator (y) achieves vnhe-ratt convergence to v/
So far, we have analyzed the error (v)(v) vi(1) pointwise for the fixed E V. To analyze the difference between the two densities (1) and (v) for all y V, consider the following divergence function of two densities:
Definition 2 (f-Divergence D, [20). Let f denote a convex function with f(1) = 0. (y(p,)) Jy f((()) (1)) d'y]. is f-divergence function between two densities p, q-
f-divergence covers many well-known divergences. For example, Dj reduces to KL divergence with f q) = (p/9) log(p/q). We will assume that the function f(p ạ) in Dj is differentiable w.r.t. and q.
We now analyze the distance between 4i and de w.r.t. Dy. The following result provides upper bound for Dy-
Lemma (Upper bound of the divergence Dy). Suppose D is a f-divergence such that f(p, q) 0 p Then,
Dy(b, Fn) & //1 w(y) (dx(t) -(v)) d'y).
where wwy) f( (v)(,,) (v)))s f((, (0/04) q) and Gn = tv)(())+ (1-t)((w) for some fixed t [0. 1].
By invoking Thm. we derive an upper bound for (,( (/)) as follows:
Theorem 2 (Convergence rate of dh)) Suppose the estimators for nuisances consistent; i.e. ||V 2I| (1) E { E. 08 for all (w, 2, Suppose Dj is f-divergence such that g(p, 0 if q. Suppose w(y) in Lemma iis finite. Then,
Dy(0. vn) S Op sup vEY
R + B,) + 1/Vnnd + +1/57),
(16)
where Rh is defined in Eq. 13 and By is defined in Eq. 14).
The following result asserts that the debiasedness property is exhibited w.r.t. Dy:
Corollary 3 (Debiasedness property of dn w.r.t. Dr). Let h = ((n1/1(+))) Suppose D satisfies [(p q) 0 if Suppose w(y) in Lemma (nhd)-114 -1/4 rate for any (w, 2, x. y), then D, (b) 1h) converges to 0 at Vnhe-rate.
6 is finite. If nuisances (4, converges at
Model-based approach
In this section, we develop model-based approach for estimating the LTE density vyy)) p(ya CC. We will upproximate di with class of distributions density model G {g(y: 3) 3 R*) where ( 3) E G is differentiable w.r.t. B. Example density models include exponential family (e.g.. Gaussian distribution), mixture of Gaussians,
generally, mixture of exponential families. The choice of the density model may depend on domain knowledge. Alternatively, one may choose among set of candidate density families using separate validation data applying cross-validation. We adapt the model-based approach developed in [39] for estimating the causal density under the no unmeasured confounders assumption.
Given a density model G, the best approximation for (yy) is defined as g(v; 36) E that achieves the minimum f-divergence to th:
8o arg min (y(v()) 8)),
(17)
BER
where Dy is the f-divergence defined in Def. 2 Our goal is estimating Sp.
7

---

Consider m(3: u) = ( g(v; 8)). Definition of Po given in Eq. (17) implies that m(3; Φ) at 5.. We note that m(3: (6) serves as moment score function. The closed-form
expression of the score is given by [9
/ g' (((::)) 3) (E(eW)) 9)) 8) + f(v(v)) dly),
m(3;v)=
(18)
where (' (0/03)g(v: B) and [2 4) = (v.00)
To construct DML estimator based on the score function m13: t), we first derive an influence function for the score:
Lemma 7 (Influence Function for mBB 1)). An functioo for m(3; v) Eq. 18) denoted
&m is given by
@m (B: r7 { E, () v)) = o(n, 4) (Y,
(19)
where ((n, + is defined in Eq.
and
Ry(Y: 3, = 3) ( ((W))
where g'(y: B) (0/03)g(u: 8), fi q) (0/ 9) and fh (p. = (a/0p)
We derive Neyman orthogonal score based on the moment score m(3, 1)) and its influence function 6m (3, 77, v)::
Lemma 8 (Neyman orthogonal score for 3). A Neyman orthogonal score for estimating B, denoted (0 (1) = [ E. 0), v)), is given by
l (1 (,, 4)) m(3,, .b) (8, 17, 2),
(20)
where om (3, 17, w is defined in Eq. 19
Given the orthogonal score (0(8"; (17, 2)) in Eq. named 'MLTE' (model-based estimator for LTE):
20 we propose the following estimator for B,
Definition 3 (MLTE estimator for (). Let ( 1b) be the Neyman orthogonal
score for 3 given in Eq. 20) Let D') denote the randomly split halves of the samples, where
[D] = I = n. Let f = {, denote the estimators for the nuisance using D'. Then, the MLTE estimator for B, denoted B, is given as a solution satisfying ED (3: b) =op(n-1) 1/2))
To illustrate, we exemplify Eq. (18) and Lemma 8 for the case where D, is a KL--divereence m (0² 1/, 1)), where ma(u; 1, 0) = (1/a²) (v- p) and (a sh μ) = (0.5/o*) (o² ( 2)) We note that Pm = ( and 2? = o p)) are estimators for 3g = {/o, for the score m(3;
and g(v: 3 = (μ, o²)) is normal distribution. First, m v = (p 1/), my
Also, (Y; = -(0/05) log ())
{R, :1, (b) Ry RY; })), where (Y; v) = (p 0)/0² and (Y 16) = 0.5(o² (Y -
/// Then, the Neyman or- thogonal score is given as slu; 0², 1/) = (1/02) {n (V- Φ(n, w)) [Y}} and ppo² n, n, (0.5/o") {o" wI n - o(n, () - 1) Finally, solutions for plu; n. v) and :p(0,, 147, v) are given by (A. 0²), where, for Φi+] in Eq. (9) A = t| + ED p(n. D]M] and 6² = ( - . + Ep (( 3) a)
The MLTE estimator in Def. is consistent provided that nuisances estimates v are consistent 14 Thm.4]. Such 9 is known achieve debiasedness [13, since 3 is DML estimator. Specifically,
Theorem (Convergence rate of 3). Lel (1 = (,,,, 4) be given in Eq. 22 no. th denote the true parameters. Let B
Let 6m (3, ) be given in Eq. (19). Let Bo.no,
be the MLTE estimator for 3 defined in Def. 3 Suppose (I) Ryly,
is bounded and R2,,,,8 3, +b) = (/ Ry < oo; (2) There exists function H < 0c s.f.
supa. max (Ry(,, B, v,, R,,,,,,
(H())) (3) (s((; (n, +))} is Donske W.r.t. for
A function class where complexities are restricted. See Def. S.1 in the Appendix for the definition. Donsker class include Sobolev, Bounded monotone, Lipschitz class, etc.
8

---

the fixed ny: (3) The estimators are consistent: B - Bo =op(1) and ||-b = (1(1) for {T=(w),&n (2, w) (xx, 2, (Y) for all (w, 2, x.y); y); and Ep (p(3: (1, +)) is differen- tiable w.r.t. B at B = 90 with non-singular matrix (3o. (1), xb) (0/03)] Ep (p: v))]
for all (n, ), where M (3o. (ń. x)) Ls M M(6. (70, 20)) Then.
B 30 -M 'ED Iehm (19; (1h0, ro))] op(n 1/2) + (P())
where
R Σ (1 x= [1][0=1 -0.
11l0--o.11: -|:13 {: &= 1|" *+10== --00--011
0.1' + +|{E: -5: [||l|0= 6.
where = ==((W), &. = (2, W), and 02 = 0l0, (Y)
Corollary 4 (Debiasedness property for B). nuisanccs { {, converges at 1114 rate, then the target estimator converges to Bo vn-rate.
For the above example where D is the KL divergence and g (3 is normal distribution, H(Y) Y for Ry(2: pt, v) and H(Y) Y for Ry (ır; o², v)).
Empirical applications
In this section, we apply the proposed methods synthetic and real datasets. For the kernel-smoothing based approach, we compare KLTE with a baseline plug-in in estimator ( "kerel--smoothen ), where estimates of nuisances f { E. 0) are plugged in the estimand Eq. ( We use the Gaussian kernel. The bandwidth is set to / 0.5n 1/5 In estimating the density, we choose 200 equi-spaced points { in y and evaluate both estimators at Kh, for = 1.. 200. For the model-based approach, we compare MLTE (c.g.. (,,03 with a moment-score-batt based estimator (called 'moment'), ) defined as fm, satisfying m(3m 1)) (ppppnn/// 1/2) (e.g., fAm: a }). We use KL divergence for D and the normal distribution for ( 3). For both approaches, nuisances are estimated through gradient boosting model XGBoost [1] which is known to be flexible.
5.1 Synthetic dataset
We applied the proposed estimators estimate the LTE p(y (C) where the densities are given as in the 4th plot in Fig. 2 As shown in the ground-truth in Fig. 3a true densities p( p(yz=/C) |C),, P(V. (C) given mixture of four Gaussians. Estimated densities for Moment and MLTE are given in Fig. 3b 3c). We note that model- based approaches fail to capture important characteristics (such as the number of modes) of the true density ( ('ground-truthe in Fig. Baj because the assumed density class is misspecified. The "kemel- highest densities, and this leads misinterpretation of the true densities. KLTE (Fig. 3e is able to capture the number, location, and scales of modes correctly.
(Fig. (3d captures only one of the modes having the
(a) Ground-truth
(b) Moment
MLTE
(d) Kernel-smoothing
(e) KLTE
Figure 3: LTE estimation with synthetic dataset. The ground-truth density is in (a). Red and Green for - and wt, respectively.
5.2 Application to 401(k) data
We applied the proposed estimators (KLTE and MLTE) 401(k) data, where the data generating processes corroborate with Fig. [ Monotonicity assumption holds naturally, since ineligible units
(Z 0) cannot participate (X 1) in 4K1(k). In our analysis, we used the dataset introduced

---

by [2] containing 9275 individuals, which has been studied in 2 17
47
58
64 to cite few.
Model-based approaches (Moment Fig. 4a and MLTE Fig. 4b) and semel-smoothing based approaches (kernel-smoothing in Fig. 4c and KLTE in Fig. 4db are implemented to analyze the data.
The model-based (Fig. (4a 4b)) and kernel-smoothing based (Fig. (4c 4d estimates both capture important characteristics of the distribution, such as mode, location, and scale parameters. The results of proposed estimators (MLTE and KLTE in Fig. (4b 4 are consistent with findings from previous analyses 2 177
5 58] The effects of the (k)(k) participation (i.e., X = 1) on net financial assets are positive over the whole range of asset distributions. To connect CDF method, we provide in Fig. 4e the CDF estimate induced by KLTE density estimation (Fig. 4ap. We note that the CDF in Fig. 4e captures the nonconstant impact trend of the (k) participation on the net financial assets, 58
which has been also described in the previous analyses 2
Moment
(b) MLTE
(c) Kerne--mmoohhngg
KLTE
(c) KLTE CDF
Figure 4: LTE of 40)(k) participation (X) net financial asset (Y). Red and Green for x0 and xt, respectively.
6 Conclusion
In this paper, we develop kernel-smooothinggbaeeee and model-based approaches for estimating the LTE density in the presence of instruments. For each approach, we give Neyman orthogonal scores (Lemma (2 and constructed corresponding DML estimators (KLTE in Def. [nd MLTE in Def. We demonstrated our work through synthetic
that exhibit debiasedness property (Corol. and real datasets. The performance of model--ased estimators depends critically on the choice of the density class. Kernel-based estimators do not have to make assumptions about the true density class but will suffer from the curse of dimensionality. This work is limited to settings where the monotonicity assumption holds, i.c., there are no defiers. One could perform sensitivity analyses on the impact of potential defiers to the estimates as conducted in [65 36).
Acknowledgements
We thank the reviewers for their feedback helping to improve this manuscript. Elias Bareinboim and Yonghan Jung were partially supported by grants from NSF IS-1750807 (CAREER). Jin Tian was partially supported by ONR grant N000141712140.
References
[1] A. Abadie. Bootstrap tests for distributional treatment effects in instrumental variable models. Journal of the American statistical Association, 97(457):284 292, 2002.
[2] A. Abadie. Semiparametric instrumental variable estimation treatment response models. Journal of econometrics, 13(2):231-266 2003.
[3] D. Angrist, Imbens, and Rubin. Identification of causal effects using instrumental variables. Journal of the American statistical Association, 91(434):444 455, 1996.
[4] E. Bareinboim, J. D. Correa, D. Ibeling, and T. Ieard, On pearl's hierarchy and the foundations of causal inference. Technical Report R-60. Causal Artificial Intelligence Laboratory, Columbia University, 2020.
[5] A. Belloni, Chernozhukov, and Hansen. Inference on treatment effects after selection among high-dimensional controls. The Review of Economic Studies, 8 (1(2):608-650) 2014.
10

---

[6] A. F. Bibaut and M. J. van der Laan. Data-adaptive smoothing for optimal-rat -rate estimation of possibly non-regular parameters. arXiv preprint arXiv:1706.0744 07408. 2017.
[7] Bickel, C. Klaassen, P. J. Bickel, Y. Ritov, Klaassen, A. Wellner, and Ritov. Effi- cient and adaptive estimation for semiparametric models, volume 4. Johns Hopkins University Press Baltimore, 1993.
[8] C. Brito. Instrumental sets. In R. Dechter, H. Geffner, and J. Y. Halpern, editors, Heuristics, Probability and Causality, A Tribute to Judea Pearl. College Publications, 2010.
[9] C. Brito and J. Pearl. Generalized instrumental variables. In Proceedings of the Eighteenth conference on Uncertainty in artificial intelligence, pages 85-93, 2002.
[10] M. D. Cattaneo, M. Jansson, and X. Ma. Local regression distribution estimators. Journal of Econometrics, 2021.
[11] T. Chen and C. Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785-794, 2016.
[12] V. Chernozhukov, D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, and W. Newey. Dou- ble/debiased/neymann machine learning of treatment effects. American Economic Review, 107(5):261-65, 2017.
[13] V. Chernozhukov. D. Chetverikov. M. Demirer, E. Duflo, Hansen, W. Newey, and J. Robins. Double/debiased machine learning for treatment and structural parameters: Double/debiased machine learning. The Econometrics Journal, 21(1), 2018.
[14] V. Chernozhukov, J. Escanciano, H. Ichimura, W. K. Newey, and J. M. Robins. Locally robust semiparametric estimation. arXiv preprint arXiv: 1608..00333 2016.
[15] V. Chernozhukov, I. Fernández-Val, and A. Galichon. Quantile and probability curves without crossing. Econometricu, 1833) 2010.
[16] V. Chermozhukov, I. Fernnddez Val, and B. Melly. Inference on counterfactual distributions. Econometrica, 8 (16)):2055)2268 2013.
[17] V. Chemozhukov and C. Hansen. The effects of 401 (k) participation the wealth distribution: an instrumental quantile regression analysis. Review of Economics and statistics, 86(3):735-751, 2004.
[18] V. Chemozhukov, C. Hansen, and K. Wuthrich. Instrumental variable quantile regression. arXiv preprint arXiv:2009.00436, 2020.
[19] K. Colangelo and Y.-Y. Lee. Double debiased machine learning nonparametric inference with continuous treatments. arXiv preprint arXiv:2004.0300 2020.
[20] I. Csiszár. Information-type measures of difference of probability distributions and indirect observation. studia scientiarum Mathematicarum Hungarica, 2:229-318, 1967.
[21] I. Díaz and M. J. van der Laan. Targeted data adaptive estimation of the causal dose-response curve. Journal of Causal Inference, 1(2):171-192, 2013.
[22] DiNardo, N. M. Fortin, and T. Lemieux. Labor market institutions and the distribution of wages, 1973-1992 A semiparametric approach. Econometrica: Journal of the Econometric Society, pages 1001-1044, 1996.
[23] M. Frölich.
Nonparametric iv estimation of local average treatment effects with covariates. Journal of Econometrics, 139(1):35-75, 2007.
[24] M. Frölich and B. Melly. Unconditional quantile treatment effects under endogeneity. Journal of Business & Economic Statisties, 31(3):346-33 -376663577250 2013.
[25] S. Ghosal al. Convergence rates for density estimation with bernstein polynomials. The Annals of Statistics, 1264 2001.

---

[26] E. Giné, R. Nickl, et al. Confidence bands in density estimation. The Annals of Statistics, 38(2):1122-11777 2010.
[27] L. Györfi, M. Kohler, A. Krzyzak, and H. Walk. A distribution-free theory of nomparametric regression. Springer Science Business Media, 2006.
[28] J. Hahn. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica, pages 315-331, 1998.
[29] J. Heckman. Randomization as an instrumental variable, 1995.
[30] Y.-C. Hsu, T.-C. Lai, and R. P. Lieli. Estimation and inference for distribution functions and quantile functions in endogenous treatment effect models. Technical report, Institute of Economics, Academia Sinica, Taipei, Taiwan, 2015.
[31] G. W. Imbens and J. D. Angrist. Identification and estimation local average treatment effects. Econometrica, 62(2))467-447 1994.
[32] G. W. Imbens and D. B. Rubin. Causal inference statisties, social, and biomedical sciences. Cambridge University Press, 2015.
[33] Y. Jung, J. Tian, and Bareinboim. Estimating identifiable causal effects on markov equivalence class through double machine learning. In Proceedings of the 38th International Conference on Machine Learning (ICML), 2021.
[34] Y. Jung, J. Tian, and E. Bareinboim. Estimating identifiable causal effects through double machine learning. In Proceedings the 35th AAAI Conference on Artificial Intelligence, 2021.
[35] N. Kallus and M. Uehara. Doubly robust off-policy value and gradient estimation for determin- istic policies. Advances in Neural Information Processing Systems, 33, 2020.
[36] H. Kang, Y. Jiang, Q. Zhao, and D. S. Small. Ivmodel: an r package for inference and sensitivity analysis of instrumental variables models with one endogenous variable. Observational Studies, 1-2):1224, 2021.
[37] E. H. Kennedy, Semiparametric theory and empirical processes in causal inference. In Statistical causal inferences and their applications in public health research, pages 141-167. Springer, 2016.
[38] E. H. Kennedy, S. Balakrishnan, M. G Sell, et al. Sharp instruments for classifying compliers and generalizing causal effects. Annals of Statistics, (8(4):2008-2030 2020.
[39] E. H. Kennedy, S. Balakrishnan, and L. Wasserman. Semiparametric counterfactual density estimation. arXiv preprint 102. 2021.
[40] E. H. Kennedy, Lorch, and D. Small. Robust causal inference with continuous instruments using the local instrumental variable curve. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 81(1):121 143, 2019.
[41] E. H. Kennedy, Z. Ma, M. D. McHugh, and D. S. Small. Nonparametrie methods for doubly robust estimation of continuous treatment effects. Journal of the Royal Statistical Society. Series B, Statistical Methodology, 79(4):1229, 2017.
[42] K. Kim, J. Kim, and H. Kennedy, Causal effects based on distributional distances. arXiv preprint
2018.
[43] C. A. Klaassen. Consistent estimation of the influence function of locally asymptotically linear estimators. The Annals of Statistics, pages 1548-1562, 1987.
[44] T. Leonard. Density estimation, stochastic processes and prior information. Journal of the Royal Statistical Society: Series (Methodological),
1978.
[45] B. Melly and K. Wüthrich. Local quantile treatment effects. 2016.
12

---

[46] R. Neugebauer and M. van der Laan. Nonparametric causal effects based on marginal structural models, Journal of Statistical Planning and Inference, 137(2):419-3444 2007.
[47] E. L. Ogburn, A. Rotnitzky, and J. M. Robins. Doubly robust estimation of the local average treatment effect curve. Journal of the Royal Statistical Society, Series B, Statistical methodology. 77(2):373, 2015.
[48] R. Okui, D. S. Small, Z. Tan, and J. M. Robins. Doubly robust instrumental variable regression. Statistica Sinica, pages 173-205, 2012.
[49] J. Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669-715 1995.
[50] J. Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, 2000. 2nd edition, 2009.
[51] J. Pearl. Parameter identification: A new perspective. Technical Report R-276, 2001.
[52] Robins, L. Li, Tchetgen, van der Vaart, et al. Higher order influence functions and minimax estimation of nonlinear functionals. In Probubility and statistics: essays in honor of Duvid A. Freedman, pages 335-421. Institute of Mathematical Statistics, 2008.
[53] J. Robins and A. Rotnitzky. Comment on "inference for semiparametrrc models: Some questions and an answer," by pj bickel and kwon. Statistica Sinica, 1 11:920-936, 00000-96,,100 2001.
[54] M. Robins and Y. Ritov. Toward curse of dimensionality appropriate (coda) asymptotic theory for semi parameereetric models. Statistics in medicine, 16(3):285 1997.
[55] R. Rosenbaum and D. B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1):41-55, 1983.
[56] K. J. Rothman and S. Greenland. Causation and causal inference in epidemiology. American journal of public health, 95(S)):S144 S100--5114-1100,,,0--10- 2005.
[57] D. B. Rubin. Bayesian inference for causal effects: The role of randomization. The Annals of statistics, pages 34-58, 1978.
[58] R. Singh and L. Sun. De-biased machine learning for compliers. arXiv preprint MXXiv:990.00222 2019.
[59] R. Smucler, A. Rotnitzky, and J. M. Robins. A unifying approach for doubly & regular- ized estimation of causal contrasts. arXiv preprint arXiv: 2019.
[60] Z. Tan. Regression and weighting methods for causal inference using instrumental variables. Journal of the American Statistical Association, 101(476):100-16118 2006.
[61] A. B. Tsybakov. Introduction to nonparametric estimation. Springer Science & Business Media, 2008.
[62] S. D. Uysal. Doubly robust iv estimation of the local average treatment effects, 2011.
[63] A. W. Van der Vaart, Asymptotic statistics, volume 3. Cambridge university press, 2000.
[64] L. Wang, Y. Zhang, T. S. Richardson, and J. M. Robins. Estimation of local treatment effects under the binary instrumental variable model. Biometrika, 2021.
[65] X. Wang, Y. Jiang, N. R. Zhang, and D. Small. Sensitivity analysis and power for instrumental variable studies. Biometrics, 74(4): 150-1 2018
[66] L. Wasserman. All of nonparametric statistics. Springer Science & Business Media, 2006.
[67] J. M. Wooldridge. Econometric analysis of cross section and panel data. MIT press, 2010.
[68] P. G. Wright. Tariff on animal and vegetable oils. Macmillan Company, New York, 1928.
[69] K. Wüthrich. A comparison of two quantile models with endogeneity, Journal of Business & Economic Statisties, 38(2):443 456, 2020.
[70] W. Zheng and M. van der Laan. valddtedl targeted minimum-loss- estimation. In Targeted Learning, pages 474. Springer, 2011.
13


Reviews:
[{"review_id": "wzOdiBakPOa", "reviewer": "Reviewer_A9r3", "summary": "The most common problem in causal effect estimation is that of estimating average effects over populations. However, to get to individual level causal effects, one has to estimate the full *distribution* or densities in the continuous outcome case. The authors are estimating such a density for compliers in the IV setting. Two density estimation strategies are provided i) kernel-based, ii) model-based. Essentially the core contribution is that in the identifiable setting (under monotonicity), authors have extended density estimation methods to the IV setting for estimating density of causal effects for compliers. The key methodology used is double machine learning which has extensive precedent for designing robust estimators for causal inference problems. I think the contribution is valuable to the community and toward making causality realistic and using it for real-world applications. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The technical novelty of model-based part is sort of unclear (authors are proposing a robust estimator that has previously only been derived in the fully observed confounding case). However the contributions in the kernel smoothing part is unclear in comparison to cited work. Is it also that previously kernel based approaches have only been proposed for say, the bow-graph and common settings and not the IV setting considered here?\n\nThe authors have not discussed any details or insights about model selection. Atleast this is less of an issue for kernel based methods, maybe one can tweak kernel parameters till estimation errors look reasonable. However the parametrizations in the model based estimation seems to need significant amount of domain knowledge. How should one choose the family of g. I understand it might be hard but please include a few comments reagarding practicality of proposed methods.\n\n\nThe estimand in line 149 should be more clearly derived. \n\nLemma (S.1) appendix. Define Donsker (Link shows up only later so best move to S.1)\n\nLine 598 in appendix: Can the authors please expand the derivation they have used for asymptotic analysis. It has taken a long time to actually verify whether this is correct. If the authors can update the appendix, I am happy to review again.\n\nI have only made cursory checks to proofs in Theorem 1 and 3 but suggests they are reasonable.\nMinor:\n1. Line 77: confusing line - As shown in Fig. 2, however, the average is sometimes insufficient to capture the treatment effects on distributions. -> to capture the distribution of treatment effects(?)\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Major limitations I think are lack of clarity in places where certain details are hidden and leaves the reader to fill gaps. I recommend fixing this. There are no negative societal impacts of this work.", "review_text": "", "extra_scores": {"ethical_concerns": "I do not see ethical concerns with the paper. ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3-4 hours"}, "extra_text": {}}, {"review_id": "aFB8BcM-yQX", "reviewer": "Reviewer_PVR7", "summary": "The authors propose kernel-smoothing-based and model-based approaches for estimating the LTE density in the presence of instruments. They derive Neyman orthogonal scores and construct the corresponding DML estimators (KLTE and MLTE), that exhibit debiasedness property. They experiment on synthetic and real datasets.", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the paper is well-written and easy to follow. \n\nThe theoretical results seem thorough and address multiple aspects of bias and consistency of the proposed algorithms. Derivations of the Neyman orthogonality scores for the two cases are interesting.\n\nIn lines 32-33, it is unclear if the argument in (ii) holds in reality. It is possible that the availability of 401k depends on the confounders too.\n\nThe experiments are inadequate. The authors only report plots without any measure of uncertainty. I encourage the authors to report the actual accuracy numbers with a measure of uncertainty so that we can measure the significance of the improvements.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors should further discuss the impact of potential defiers (Assumption 1) in the algorithm.\n", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "_i35h9ejNBY", "reviewer": "Reviewer_86MR", "summary": "This paper proposes two methods for estimating the density of the outcome variable for the compliers in the potential outcomes framework. The first is a kernel-smoothing based approach, while the second is a model-based approach, and in both cases, the double machine learning approach based on data splitting is employed. The nature of the contribution seems mostly theoretical, with a lot of effort into proving the convergence rates. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I thank the authors for their work put into this submission. I tried to do it justice with my review; however, given time constraints, and my lack of prior knowledge on some of the previous work, there are likely to be misunderstandings on my part, and I do apologise in advance for those. \n\nI believe looking at distributional aspects of treatment effect beyond the mean is an important issue, and looking at the density is one interesting way of doing it. Since considering distributional aspects of treatment effects have received a lot of attention (by looking at cumulative distribution functions or quantiles, as the authors mention, but also by looking at kernel mean embeddings; see [Counterfactual mean embeddings, Muandet et al., 2020] or [Kernel methods for policy evaluation, Singh et al., 2020], for example), it is quite surprising that this paper is among the first to look at densities. This paper also applies ideas of double machine learning, in which I see value. Overall, I think this paper should be seen as a natural follow-up to existing ideas, rather than a really original piece of work. \n\nThe paper is generally fairly well-written, although I think the clarity of the development could be improved. For example, in Section 3, I think it would be helpful to postulate explicitly what “influence function”, “moment condition” and “Neyman orthogonal score” refer to, and why they are used, in clear mathematical terms (rather than in brackets and in footnotes, as currently is written). \n\nThe following are some specific points that arose during my review. Most of them are very minor typos, but I think some of the points could be significant, but it is also likely that I am mistaken about them. I will give an overall score of 6 for now, although if all the points are adequately addressed, I would be happy to raise the score. \n\n1) line 25: it will help -> will help\n2) line 70: ‘debaisedness’ -> ‘debiasedness’ \n3) lines 101-104: Just a matter of notation, but I would prefer the distinction between the function and its evaluation to be clear. So I would write “… density p and a function f, … || f || = sqrt{E_P[(f(X))^2]}, … if || hat{f} - f ||=O_P(1/r_n). \n4) footnote 2: which -> whose\n5) lines 142-147: The kernel K takes y in R^d as argument, so in the examples of kernels given, instead of u^2, I think you would need || u ||^2. Also, For the same reason as above, I think it would be better to write || K || on line 144 rather than || K(y) ||; we’re considering the norm of the function K, rather than the norm of the real number K(y). \n6) displayed equation after line 569: on the first line, shouldn’t phi_X[f(Y)]  be just phi_X? And on the second line, shouldn’t psi^X[f(Y)] be just psi^X? On the second, third and fourth lines, shouldn’t V_X[f(Y)] just V_X, or V_X({pi,xi})?\nAlso, I have a question on this proof. It seems that the influence function of a quotient A/B is given as influence(A)/B - influence(B)A/B^2, just like the quotient rule for differentiation. Is this obvious? I would have preferred it if there was some explanation / lemma as to why this is true. \n7) line 577: is is -> is\n8) The condition on line 595 is presumably to satisfy the condition on line 554 in Lemma S.1, but there, the nuisance is also the estimate hat{eta}, whereas here the true eta is used. Is the condition still satisfied for hat{eta}, which I presume is what is required?\n9) Is M=-1 obvious? Could you please write this out? \n10) displayed equations after line 606: the 0 expression added could be written so that the + and the - are swapped; this would be consistent with the next line. \n11) displayed equations after line 607: in the last term on the third line, isn’t psi_h missing? Also, throughout, I think I would have preferred psi__h being written as psi_h(y), since it is dependent on a particular choice of y, and K_{h,y} is always written with explicit dependence on y. But it might clutter up the notation, so I’ll leave the decision to the authors. \n12) line 616: psi_h seems to be missing here too, and the displayed equations that follow. This actually seems like it will make a difference to the rate?\n13) displayed equations after line 620, first line: phi should be phi_m.\n14) displayed equations after line 630: presumably, psi_{YX}+R_{YX}=hat{V}_{YX}, which means (ignoring the fact that the right-hand side is random and the left-hand side is random, since expectation is taken in the end anyway) the expectation of V_{YX} is psi_{YX}? and psi_X+R_X=hat{V}_X? \n14) line 632: shouldn’t hat{V}_{YX}, V_{YX}, hat{V}_X and V_X be their expectations? Otherwise a random element is equal to a non-random element, since R_{YX} and R_X are not random. \n15) line 152: in the definition of theta, there is no dependence on z on the right-hand side. Shouldn’t x in the conditioning be z? \n16) line 153: there is a redundant ). ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The nature of the work is mostly theoretical, and therefore I deem it not necessary to consider societal impact. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "15"}, "extra_text": {}}, {"review_id": "SAWkzEOC0TW", "reviewer": "Reviewer_GUHn", "summary": "This paper deals with the problem of the estimation of the local treatment effect (LTE) which is measure of the causal impact of a treatment. To this extent the authors develop kernel-smoothing and model-based methodologies to recover this quantity from the data. \nThe theoretical properties of the machinery are developed and the authors show that the estimator of the LTE is consistent.\nTwo applications of the proposed methodology showcase the technique.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a great paper which deals with causal inference based on machine learning methods. The authors know the literature very well. The paper is well written. The paper has both interesting theoretical results as well as a nice empirical application. This paper would make a great contribution to the program.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "this paper does not raise any concerns.", "review_text": "", "extra_scores": {"ethical_concerns": "this paper does not raise any concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers unanimously found your paper to be compelling. I expect you will make the changes discussed during the review process.
Decision: Accept (Spotlight)