DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning
Authors: Siqi Xu
Lin Liu
Zhonghua Liu
Venue: NeurIPS 2022 Accept (2022)
Keywords: Causal Inference
Causal Mediation Analysis
Causal Machine Learning
Deep Learning
Semiparametric Statistics
Fairness

Abstract: Causal mediation analysis can unpack the black box of causality and is therefore a powerful tool for disentangling causal pathways in biomedical and social sciences, and also for evaluating machine learning fairness. To reduce bias for estimating Natural Direct and Indirect Effects in mediation analysis, we propose a new method called DeepMed that uses deep neural networks (DNNs) to cross-fit the infinite-dimensional nuisance functions in the efficient influence functions. We obtain novel theoretical results that our DeepMed method (1) can achieve semiparametric efficiency bound without imposing sparsity constraints on the DNN architecture and (2) can adapt to certain low dimensional structures of the nuisance functions, significantly advancing the existing literature on DNN-based semiparametric causal inference. Extensive synthetic experiments are conducted to support our findings and also expose the gap between theory and practice. As a proof of concept, we apply DeepMed to analyze two real datasets on machine learning fairness and reach conclusions consistent with previous findings.

DeepMed: Semiparametric Causal Mediation Analysis with Debiased Deep Learning
Siqi Xu
Department of Statistics and Actuarial Sciences University of Hong Kong Hong Kong SAR, China sqxu hku. hk
Lin Liu'
Institute of Natural Sciences, MOE-LSC, School of Mathematical Sciences, CMA-Shanghai, and SJTU-YaL Joint Center for Biostatistics and Data Science
Shanghai Jiao Tong University and Shanghai Artificial Intelligence Laboratory Shanghai, China
1 iinliussjen jt..e.....uu.cccc edu.cn
Zhonghua Liu" Department of Biostatistics Columbia University New York, NY, USA 2.2250 96 cumc. co umb a edu
Abstract
Causal mediation analysis can unpack the black box of causality and is therefore a powerful tool for disentangling causal pathways in biomedical and social sciences, and also for evaluating machine learning fairness. To reduce bias for estimating Natural Direct and Indirect Effects mediation analysis, we propose a new method called DeepMed that uses deep neural networks (DNNs) to cross-fit fit the infinite- dimensional nuisance functions in the efficient influence functions. We obtain novel theoretical results that our DeepMed method (1) can achieve semiparametric efficiency bound without imposing sparsity constraints on the DNN architecture and (2) can adapt certain low-dimensional structures of the nuisance functions, significantly advancing the existing literature on -NNebased semiparametric causal inference. Extensive synthetic experiments conducted to support findings and also expose the gap between theory and practice. As a proof of concept, we apply DeepMed analyze two real datasets machine learning faimesss and reach conclusions consistent with previous findings.
1 Introduction
Tremendous progress has been made in this decade on deploying deep neural networks (DNNs) in real-world problems (Krizhevsky et al., 2012; Wolf et al., 2019; Jumper al., 2021; Brown al., 2022). Causal inference is no exception. In semiparametric causal inference, series of seminal works (Chen al., 2020; Chemoozuuovv al., 2020; Farrell al., 2021) initiated the investigation of
"Co-corresponding authors, alphabetical order
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

nuisance functions, a statistical properties of causal effect estimators when the nuisance functions (the outcome regressions and propensity scores) are estimated by DNNs. However, there are a few limitations in the current literature that need to be addressed before the theoretical results can be used to guide practice:
(1) Most recent works mainly focus on total effect (Chen al., 2020; Farrell al., 2021). In many settings, however, more intricate causal parameters are often of greater interests. In biomedical and social sciences, one is often interested in "mediation analysis" to decompose the total effect into direct and indirect effect to unpack the underlying black-box causal mechanism (Baron and Kenny, 1986). More recently, mediation analysis also percolated into machine learning faimess, For instance, in the context of predicting the recidivism risk, Nabi and Shpitser (2018) 8) argued that, for a "fair" algorithm, sensitive features such as race should have no direct effect on the predicted recidivism risk. If such direct effects can be accurately estimated, one can detect the potential unfairness of a machine learning algorithm. We will revisit such applications in Section 5 and Appendix G.
(2) Statistical properties of DNN-based causal estimators in recent works mostly follow from several (recent) results on the convergence rates of DNN-based nonparametric regression estimators (Suzuki, 2019; Schmidt-Hieber, 2020; Tsuji and Suzuki, 2021), with the limitation of relying on sparse DNN architectures. The theoretical properties are in turn evaluated by relatively simple synthetic experiments not designed to generate nearly setting considered by almost all the above related works.
The above limitations raise the tantalizing question whether the available statistical guarantees for DNN-based causal inference have practical relevance. In this work, we plan to partially fill these gaps by developing a new method called DeepMed for semiparametric mediation analysis with DNNs. We focus on the Natural Direct/nndirect Effects (NDE/NIE) (Robins and Greenland, 1992; Pearl, 2001) (defined in Section 2.1), but our results can also be applied to more general settings; see Remark 2. The DeepMed estimators leverage the "multiply-rouust" property of the efficient influence function (EIF) of NDE/NIE (Tchetgen Tchetgen and Shpitser, 2012; Farbmacher et al., 2022) (see Proposition 1 in Section 2.2), together with the flexibility and superior predictive power of DNNs (see Section 3.1 and Algorithm 1). In particular, we also make the following novel contributions to deepen our understanding of DNN-based semiparametric causal inference:
On the theoretical side, we obtain new results that our DeepMed method can achieve semi- parametric efficiency bound without imposing sparsity constraints the DNN architecture and can adapt to certain low-dimensional structures of the nuisance functions (see Section 3.2), thus significantly advancing the existing literature on DNN-based semiparametric causal inference. Non-sparse DNN architecture is more commonly employed in practice (Farrell et al., 2021), and the low-dimensional structures of nuisance functions can help avoid curse-of-dingg off-oo--eiiiionlityyy.. These two points, taken together, significantly advance our understanding of the statistical guarantee of DNN-based causal inference.
More importantly, on the empirical side, in Section 4, we designed sophisticated synthetic experiments to simulate nearly dimeesoodi functions, which are much more complex than those in previous related works (Chen et al., 2020; Farrell et al., 2021; Adcock and Dexter, 2021). We emphasize that these nontrivial experiments could be of independent interest to the theory of deep learning beyond causal inference, to further expose the gap between deep learning theory and practice (Adcock and Dexter, 2021; Gottschling et al., 2020); see Remark 9 for extended discussion. As proof of concept, in Section 5 and Appendix G. we also apply DeepMed to re-analyze two real-world datasets on algorithmic fairness and reach similar conclusions to related works.
Finally, a user-friendly R package can be found at Making such resources available helps enhance reproducibility, a highly recognized problem in all scientific disciplines, including (causal) machine learning (Pineau et al., 2021; Kaddour et al., 2022).
2 Definition, identification, and estimation of NDE and NIE
2.1 Definition of NDE and NIE
Throughout this paper, we denote Y as the primary outcome of interest, D as a binary treatment variable, M as the mediator on the causal pathway from D to Y. and X E [0, 1]2 (or more generally,
2

---

compactly supported in IR") as baseline covariates including all potential confounders. We denote the observed data vector as O= (X, D. M, Y). Let M(d) denote the potential outcome for the mediator when setting D = d and Y(d, m) be the potential outcome of Y under D d and M = m. where d E {0. 1 and m is in the support . of M. We define the average total (treatment) effect as Trot == E (1, (1)) Y YO, (())), the average NDE of the treatment D on the outcome Y when the mediator takes the natural potential outcome when = d as TNDE (d) == E[Y (1, M (d) (O, . (d))], and the average NIE of the treatment D on the outcome Y via the mediator M as TNIE (d) = E[y (d, M()) Y (d, (0)]] We have the trivial decomposition Trot = TNDE (d) - TNIE d') for d + d'. In causal mediation analysis, the parameters of interest are TNDE (d) and (d))
2.2 Semiparametric multiply-robus estimators of NDE/NIE
Estimating (d) and NIEE()) (d) can be reduced to estimating o(d, d') = ElY for d, d' E {0. 13. We make the following standard identification assumptions:
Consistency: if D d, then M = (d) for all d E {0, 1]; while if D = d and M m, then Y = (d, m) for all E {0, 1] and all m in the support of M.
ii Ignorability: ((d, m) 1 D|X, (dd, m) 1 M D. (dd) 1 DDX, and Y m) 1 MX' almost surely for all d, {0,1] and all m M. The first three conditions are, respectively, no unmeasured treatment-outcons. mediator-outcome and treatment-mediate confounding, whereas the fourth condition is often referred to the "cross-world" condition. We provide more detailed comments on these four conditions in Appendix A.
ii.. Positivity: The propensity score al a(d[x) = Pr(D = d(x) E (c, for some constants 0 < c C 1, almost surely for all d {0, 13: f(m] X, d). the conditional density (mass) function of M m (when M is discrete) given X and D = d, is strictly bounded between [e. p) for some constants 0 < e S p < oo almost surely for all m in M and all d {0. 1}.
Under the above assumptions, the causal parameter old, d') for d, d' E {0, 1} can be identified as either of the following three observed-data functionals:
(d,, d') E 11D (1.(MIX,''')) = [D== =d u(x,a,M) x(
a(d')x)
(I)
pla, d, f d' (2))) drade,
where I { denotes the indicator function, p(z) denotes the marginal density of X, and ulzz d, m) E|Y [X = x, D = d, .M = m] is the outcome regression model, for which we also make the following standard boundedness assumption:
iv. u(x, d, m) is also strictly bounded between [-R, .t] for some constant R > 0.
Following the convention in the semiparametric causal inference literature, call a, S, "nuisance functions".. Tehetgen Tchetgen and Shpitser (2012) derived the EIF of ((d, d'): a, = 1/d.a (0) o(d, d'), where
1(D= =d d). f(MIX,
aa (0) (0))
(Y (y-x(X,d,MM)
a(d)X) f(M(X,d)
(1- +1- 11--(((x)) a(d'x) MEM m(X, ()) + 1(D= 1== d) a(d'lX) u(xx M)
110--11))
(2)
The nuisance functions ja(o,, d, m), a(d]:) and f(m]æ d) appeared in (o) are unknown and generally high-dimensional. But with a sample D = } one can construct the following generic sample-splitting multiply-robust estimator of o(d, d'):
1 of the observed data, based on (b
((d,d)
Σ (O,), IED
n
where D,, = (m is a subset of all N data, and Va (0) replaces the unknown nuisance functions a, f, 2t in Vasa" (o) by some generic estimators f, computed using the remaining N nuisance
3

---

sample data, denoted as Dy. Cross-fit is then needed to recover the information lost due to sample splitting; see Algorithm 1. It is clear from (2) that é(d, d') is consistent estimator of p(d, d' as long any two of a. fü are consistent estimators of the corresponding true nuisance functions, hence the name "multiply-oouus"" Throughout this paper, we take = N and assume:
V. Any nuisance function estimators are strictly bounded within the respective lower and upper bounds of a. f.
To further ease notation, we define: for any E {0, 11, ta.d = (f ssd
(f6 dF m 0)} 1/2 and Fa,d = {s m) mld 0)} 1/2 where (1,(()) a(d(x) (d) 6 ,,, m) f(m)2z, f(m)ar d) and 5y (,, m) == jū(a, d, m)
p(z, m) are point-wise estimation errors of the estimated nuisance functions. In defining the above Lz-estimation errors, we choose take expectation with respect to (W.r..)) the law F(m, a 0) only for convenience, with no loss of generality by Assumptions iii and v.
To show the cross-fit version of o(d, d') is semiparametric efficient for o(d, d'), we shall demonstrate under what conditions vollŌdd, d') - o(d, d')) 4 (( 2(EIF2,ar)) (Newey, 1990). The following proposition on the statistical properties of (dd, ) is key step towards this objective.
Proposition 1. Denote (ias( d = Elo(d, -(d, d') [Pu)
] as the bias of o(d, d') conditional
on the musancce sample Dv. Under Assumptions i Bias ((d, d, d') is of second-order
(iiss@((ddd')) S max (ad ry.d:
max rf.d" Tp,ds Fa,d T Tudd d'E(0,1)
(4)
Furthermore, if the RHS of (4) is o(n-1/2), then
vā (old, -
- d') 1
Σ "
(O:) + +0 S, N/ (0.E (EIF(E)
(5)
Although the above result is a direct consequence of the EIF &a (0), we prove Proposition 1 in Appendix B for completeness.
Remark 2. The total effect Ttot = +[1, - 0(0 0) can be viewed as a special case, for which d d' for o(d, d'). Then ElF ElFa corresponds to the nonparametrie EIF of ((d, ((d) =
(d, ((d)):
EIF = wal(0) - ((d) with vb(()) = =(D= 10-++11111
(1-1(D=a) u ((x,)), a(d[X) -1-1111000 ald]x)
where p1x, d) = E[V[X = x. D d. Hence all the theoretical results in this paper are applicable total effect estimation. Our framework can also be applied to all the statistical functionals that satisfy so-called "mixed-bias" property, characterized recently in Rotnitzky et al. (2021). This class includes the quadratic functional, which is important for uncertainty quantification in machine learning.
3 Estimation and inference of NDE/NIE using DeepMed
We now introduce DeepMed, method for mediation analysis with nuisance functions estimated by DNNs. By leveraging the second-orde, bias property of the multiply-robust estimators of NDE/NIE (Proposition 1), we will derive statistical properties of DeepMed in this section. The nuisance function estimators by DNNs are denoted as a, f.p.
3.1 Details on DeepMed
First, we introduce the fully-connected feed-forward neural network with the rectified linear units (ReLU) as the activation function for the hidden layer neurons (FNN-ReLLU), which will be used to estimate the nuisance functions. Then, will introduce an estimation procedure using V-fold

---

cross-fitting with sample-splitinn to avoid the Donsker-tyre empirical-proeess assumption on the nuisance functions, which, in general, is violated in high-dimensional setup. Finally, we provide the asymptotic statistical properties of the DNN-based estimators of Ttot, TNDE(d) and (NIE
We denote the ReLJU activation function as (u) == max(u, 0) for any R. Given vectors Jr b, denote (x(2)) = (x b), with a acting on the vector r b component-wise.
Let Fun denote the class of the FNN-ReLU functions
Fu = { R R" R: f(x) W ..... W W (x))
where о is the composition operator, is the number of layers (i.e. depth) of the network, and for 1 = 1,... L. ( is a K1+1 x Ki-dimensional weight matrix with K2 being the number of neurons in the 1-th layer (i.e. width) of the network, with K₃ = P and KL+1 1, and b()) is a K7-dimensional vector. To avoid notation clutter, concatenate all the network parameters as @ = (W(), (,))),))))1
1, ................................ , L) and simply take K₂ = ... Kg K. We also assume θ to be bounded: |[O]|co S B for some universal constant B > We may let the dependence on L, K, B explicit by writing Fun as Fm B).
DeepMed estimates Thot, TNDE (d), (d) by (3), with the nuisance functions a, f l estimated using Fan with the V-fold cross-fitting strategy, summarized in Algorithm I below; also see Farbmacher al. (2022). DeepMed inputs the observed data D {O:) and outputs the estimated total effect Fror, NDE (d) and NIE (d)), together with their variance estimators oi (DDE() and NIE(d).
Algorithm 1 DeepMed with V-fold cross-fitting
1: Choose some integer V (usually V E (2.3.... 10})
2: Split the N observations into V subsamples Io C (L, .N = [N] with equal size n N/V:
3: for 1.... V: do
4: Fit the nuisance functions by DNNs using observations in [N] \
5: Compute the nuisance functions in the subsample Lo using the estimated DNNs in step 4
6: Obtain (O.), (O.) her. for the subsample I, based on (2), respectively, with the nuisance
functions replaced by their estimates in step
7: end for
8: Estimate average potential outcomes by @(d) = + o(d, = I
9: Estimate causal effects by Frots id) and (NiE(d) with ((d) and H(d, d')
10: Estimate the variances of Froes (N) and (d)) by:
N
Fiat N2 Σ (O.) (0.))² N Fror ONDE (d)
1 E E (0.)) - (d))
(N)E(d) = No Σ (O.) - (o,) N THIE (d)
Output: Foats (d). PNIE (d), Poo.. RDD (d) and (d)
Remark 3 (Continuous or multi-dimennional mediators). For binary trealment D and continuous or conditional density estimation, we can rewrite a(d]z) [m]] (m)])))) 2.d) as ald]a.m)[ (1-((((() by the Bayes' rule and the integral w.rt. f 2, d')
multi-dimensional M, to avoid pomparrmetrichhig--lieennsi
f(m (m))..d))
1-a(d)a,m)
in (2) as Elj((X, d, M) = =x, D d'). Then we can first estimate p(æ, d, m) by plæz d, m) and in turn estimate Elp(X, d, =x, D d'] by regressing paX,, d, M) against (X, D) using the FNN-ReLU class. We mainly consider binary M to avoid unnecessary complications; but see Appendix G for an example which this strategy is used. Finally, the potential incompatibility between models posited for a(d]z) and a(d]æ, m) and the joint distribution of (X, A, M. Y) is not of great concern under the semiparametric frumework because all nuisance functions are estimated nonpurametrically; again, see Appendix G for an extended discussion.
3.2 Statistical properties of DeepMed: Non-sparse DNN architecture and low-dimensional structures of the nuisance functions
According to Proposition 1, to analyze the statistical properties DeepMed, it is sufficient to control the Lssstmmatio errors of nuisance function estimates à, f. fa fit by DNNs. To case presentation,
5

---

we first study the theoretical guarantees on the L2 esttmmatti error for generic nuisance function I. for which we assume:
g: W [0, 1|"
vi. Z - ((W + 5, with & sub-Gaussian with mean zero and independent of W.
Note that when 9 corresponds to a, f, ZW corresponds to (X, 1(D 1)), ((X, D), 1(M 1))
and ((X, D, ), Y), respectively.
We denote the DNN output from the nuisance sample Dy as 9. For theoretical results, we consider as the following empirical risk minimizer (ERM):
arg min (L. IED
Σ (Z - -(W)))
(6)
To avoid model misspecification, one often assumes G. where G is some infinite- seffiite-dimensionall function space. A common choice is 9 = (a C), the Hölder ball on the input domain [0, 1P, with smoothness exponent and radius C. Hölder space is of the most well-studied function spaces in statistics and it is convenient to quantify its complexity by a single smoothness parameter o; see Appendix C for review. It is well-known that estimating Hölder functions suffers from curse-of-dinensionalty (Stone, 1982). One remedy is to consider the following generalized Hölder space, by imposing certain low-dimensional structures on g:
Ha C) = (9(w)= = h(Tw): : h E Hs(a; C), E Rkxp unknown, S pp}.
Remark 4. The above definition contains g(w) - h(wr), where I C {1,...,,0) ,P). as a special case, in which g is assumed only depend on a subsel of the feature vector w. One can easily generalize the above definition to additive models g(w) مニ hy(10)) where h E Ha, (aiji Cj), allowing even more modeling flexibility. To avoid complications, we only consider the above simpler model.
We can show that the ERM estimator ÿ (6) from the FNN-ReLU class FL, K, B) attains the optimal estimation rate over Hi (a; C') up log factors, by choosing the depth and width appropriately without assuming sparse neural nets.
Lemma 5. Under Assumptions iii vi. ifg E C) k S p. with LK n - we have sup (EH)(( [E[((()- - ((W)) 1/2 z = (logn)².
Lemma 5, together with Proposition 1, implies the main theoretical result of the paper.
Theorem 6. Under Assumptions i- vi and the following condition on o, f.pe a E (aaaa,
K[(ayyC) C) Hapap C)2, with
min
aa +
ar
0
+ 0g aa
Ou } 2
te,
(7)
2ag +
2a + K 2ay
20 + k 2ar
k
2ay +
for le S. and some arbitrarily small 6 > 0, if a, f. Tt are respectively the ERM (6) from FNN-ReLU classes Frn(Inn Ka; B), F.nnnLL K, B), Frn (Ly, K,, B), of which the product of the depth and
width satisfies Lg x n 2(4 for g {a, f,14, then the DeepMed estimators Feat. (N) and (NIE(d) computed by Algorithm are semiparametric efficient:
Coo(Froo - Teoe), OND (W) (dN(EEE)) TNDE(d) " ( (d) - (NIE(d)) -4 (O, 1), with
No. B E((EIF, - EIF NONDE(d) (d) & E((EIF EIFu.a) and (ON & E[(EIF2,1
EIF (.F..))). i.e. otot NDDE and OHE are consistent variance estimators.
Remark 7. To unload notation in the above theorem, consider the special case where the smoothness
of all the nuisance functions coincides, i.e. Oa = aa = ag = a. Then Condition (7) reduces to k/2 + for some arbitrarily small 0. For example, if the covariates X have dimension p = 2 and no low-dimensional structures are imposed on the nuisance functions (i.e. k = p). one needs 1 ensure semiparametric efficiency of the DeepMed estimators.
We emphasize that Lemma 5 and Theorem 6 do not constrain the network sparsity S, better reflecting how DNNs are usually used in practice. Theorem 6 advances results on total and decomposition effect

---

estimation with non-sparse DNNs (Farrell et al., 2021, Theorem 1) in terms of (1) weaker smoothness conditions and (2) adapting to certain -ewdiimesioonal structures of the nuisance functions. The proof of Lemma 5 follows from combination of the improved DNN approximation rate obtained Lu al. (2021); Jiao et (2021) and standard DNN metrie entropy bound (Suzuki, 2019). We prove Lemma 5 and Theorem 6 in Appendix C for completeness. One weakness of Lemma 5 and Theorem 6, as well as in other contemporary works (Chen et al., 2020; Farrell et al., 2021), is the lack of algorithmic/raaningg process considerations (Chen et al., 2022); see Remark 10 and Appendix for extended discussions.
Remark 8 (Explicit input-layer regularization). Training DNNs in practice involves hyperparameter tuning, including the depth L and width K in Theorem 6 and others like epochs. the synthetic experiments, we consider the nuisance functions only depending imput. A reasonable heuristic is to add L -eggulaaaizzionnn the input-layer of the DNN. Then the regularization weight À is also hyperparameter. In practice, simply use cross-validation to select the hyperparameeerss that minimize the validation loss. We leave its theoretical justification and the performance of other alternative approaches such the minimax criterion (Robins et al., 2020; Cui and Tchetgen Tchetgen, 2019) to future works.
k-subset of p-dimensional
4 Synthetic experiments
In this section and Appendix E, we showcase five synthetic experiments. Since ground truth is rarely known in real data, we believe synthetic experiments play an equally. not more, important role as real data. Before describing the experimental setups, we garner the following key take-home message:
(a) Compared with the other competing methods, DeepMed exhibits better finite-sample perfor- mance in most of our experiments;
(b) Cross-validation for DNN hyperparameter tuning works reasonably well in our experiments;
(c) We find DeepMed with explicit regularization in the input layer improves performance (see Table A2) when the true nuisance functions have certain low-dimensional structures in their dependence regularization in DNNs for total effect estimation. Our observation does not contradict Farrell al. (2021) as (1) the purpose of the input-layer regularization is sparsity of the DNN architecture and (2) we do not further regularize hidden layers;
the covariates. Farrell et al. (2021) warned against blind explicit
control the
(d) Experimental setups for Cases 3 to 5 generate nuisance functions that are nearly infinite- dimensional and close to the boundary of a Hölder ball with given smoothness exponent (Liu et al., 2020; Li et al., 2005). Thus these synthetic experiments should be better benchmarks than Cases 1 and settings in other related works such as Farrell et al. (2021). We hope that these highly nontrivial synthetic experiments are helpful to researchers beyond mediation analysis or causal inference. We share the code for generating these functions as a part of the DeepMed package.
We consider a sample with 10,000 i.i.d. observations. The covariates X = (X,, XT) are independently drawn from uniform distribution Uniform([11. 1)). The outcome Y. treatment 1) and mediator M are generated as follows:
D Bermouli(s(d(()))) M 0.2D + mXX) + +N((0, 1), Y 0.2D + M + xX)) + AV 1),
where ((r) = (1 +e-=)-1, and we consider the following three cases to generate the nonlinear functions d(x), m(x) and y(x) in the main text:
Case 1 (simple functions):
d(x) = 22 + Iy + sin F1, m(x) = 4 Σ sin 3x, y(x) = (21 + +x2)² 3 + 5 sin
Zi.
km

---

Case 2 (composition of simple functions): we simulate more complex interactions among covariates by composing simple functions as follow:
2
d(x) = d2 do o du(211. ;as),
2s), with do( ,3a)
sin
sin 32.
<-8
1=1
2
di(01, ... ..) = (sin(a1 + 02), sin 02, 03, as) and dz(b1,"
ba) = 0.5 0.5 (b) + ba) + 0.5(bs + ba);
m(x) = my mo Is), with mo( ... ,x5) (sin
sin ars) mi (a1, as) = 5sin
and ((r) y2 y1 (01 22), with yo (00) sin
24 sin
32 sin
22
im1 2m3
yu (11, 02, 0a) (sin(a) + 02), aa) and y2(bi, b2) 10 + b2).
Case (Hölder functions): we consider more complex nonlinear functions as follows:
d(x)= = 21 + 2y244 + 0.55g(0.221: m(x) Σ 17 (0.32,;; a) u(x) = 21 + +37 0.2 >
x45 a
Ejentez 2 2. wyy (1) with J {0, 3,,,, 10, 16) and wy,(-) is the D6 father wavelet functions dilated at resolution j shifted by 1. By construction, n(2; a) E (a; B) for some known constant B > 0 following Härdle et al. (1998, Theorem 9.6). Here we set a = 1.2 and 1. Thus we expect the DeepMed estimators are semiparametric efficient. It is indeed the case based on the columns corresponding to Case 3 in Table 1, suggesting that DNNs can be adaptive certain lwwddimensional structures.
where n(2;; a:)
the intrinsic dimension
Remark 9. The nuisance functions in Cases 3 5 (see Appendix E) are less smooth than what have been considered elsewhere, including Farrell et al. (2021). Chen et al. (2020), and even Adcock and Dexter (2021), a paper dedicated exposing the gap between theoretical approximation rates and DNN practice. These nuisance functions are designed to be near the boundary of a Hölder ball with given smoothness exponent assumption under which most of the known statistical properties of DNNs are developed.
add wavelets at very high resolution in nf(2;; oz). This is the
Table 1: The biases, empirical standard errors (SE) and root mean squared errors (RMSE) of the estimated Teats TNDE (1) and TNIE(1), and the coverage probabilities (CP) of their corresponding 95% confidence intervals. = 5 (no irrelevant covariates). The simulation is based on 200 replicates. The full table including NDE (0) and İNE (0) can be found in Table A1 in the Appendix.
Case 1
Case 2
Case 3
Method
Bias
SE
RMSE
CP
Bias
SE
RMSE
CP
Bias
SE
RMSE
CP
Tea
DeepMed
-0.001
0.032
0.032
0.945
-0.004
0.032
0.032
0.955
0.008
0.037
0.038
0.920
Lasso
0.192
0.089
0,212
0.460
-0.304
0.116
0.325
0,215
0.346
0.079
0.355
0.010
RF
0.067
0.042
0.079
0.775
-0.078
0.056
0.096
0,950
-0.009
0.042
0.043
0.985
GBM
-0.015
0.036
0.039
0.940
-0.044
0.055
0.070
0,850
0,019
0.041
0.045
0.930
Orccle
-.0011
0.029
0.029
0.955
-0.003
0.029
0.029
0,925
-0.001
0.032
0.032
0.935
(TDe(1) ))
DeepMed
0.000
0.027
0.027
0.945
-0.007
0.023
0.024
0.955
0.000
0.026
0.026
0.965
Lasso
0.130
0.043
0,137
0.220
-0.375
0.059
0.380
0.000
0,226
0.064
0,235
0.050
RF
0.048
0.029
0,056
0.700
-0,188
0.044
0,193
0,005
0.030
0.038
0,048
0.980
GBM
-0.040
0.031
0,051
0.770
-0.164
0.046
0.170
0,040
0.011
0.042
0,043
0.920
Oracle
0.000
0.022
0.022
0.945
-0.002
0.020
0.020
0.985
0.001
0.022
0.022
0.955
(1)
DeepMed
40.001
0.025
0.025
0.960
0.005
0.019
0.029
0.915
0.008
0.031
0.032
0.905
Lasso
0.058
0.077
0.096
0.875
0.069
0.094
0.117
0,905
0.120
0.045
0.128
0.220
RF
0.066
0.037
0.076
0.665
0.108
0.059
0.123
0.860
-0.045
0.038
0.059
0.765
GBM
0.023
0.031
0.039
0.890
0.120
0.064
0.136
0.485
-0.001
0.037
0.037
0.935
Oracle
-0.001
0.020
0.020
0.975
0.000
0.021
0.021
0.930
-0.002
0.022
0.022
0.920
In all the above cases, Tsat = 0.4 and (ND = TNNIEEE (d) = 0.2 for d E (0, 13. We also consider the cases where the total number of covariates p 20 and 100 but only the first five covariates are relevant to Y, M and D. All simulation results are based on 200 replicates. The sigmoid function used in the final layer when the response variable is binary. For comparison, we also use the Lasso.
8

---

random forest (RF) and gradient boosted machine (GBM) to estimate the nuisance functions, and use the true nuisance functions (Oracle) as the benchmark. The Lasso is implemented using the R package "hdm" with data-driven penalty. The DNN, RF and GBM are implemented using the packages "keras", "randomForest" and "ghm". respectively. We adopt a 3-fold cross-validation to choose the hyperparameters for DNNs (depth I,, width K, -ereulalaiizztion parameter À and epochs), RF (number of trees and maximum number of nodes) and GBM (numbers of trees and depth). We use completely independent sample for the hyperparameter selection. In this paper, we only use one extra dataset to conduct the cross- rross-validation for hyperparameter selection, so our simulation results are conditional on this extra dataset. We use the cross-entropy loss for the binary response and the mean-squared loss for the continuous response. We fix the batch-size as 100 and the other hyperparameters for the other methods are set to the default values in their R packages. See Appendix E. for more details.
We compare the performances of different methods in terms of the biases, empirical standard errors (SE) and root mean squared errors (RMSE) of the estimates as well as the coverage probabilities (CP) of their 95% confidence intervals. When p k- 5 (all covariates are relevant no low-dimensional structures), DeepMed has smaller bias and RMSE than the other competing methods, and is only slightly worse than Oracle. Lasso has the largest bias and poor CP expected since it does not capture the nonlinearity of the nuisance functions. RF and GBM also have substantial biases, especially in Case 2 with compositions of simple functions. Overall, DeepMed performs better than the competing methods (Table 1). From the empirical distributions, we can also see that they are nearly unbiased and normally distributed in Cases 1-3 (Figures Al-A3). When p 20 or 100 but only the first five covariates are relevant (k- = 5), L -egraagulaizzaionn in the input-layer drastically improves the performance of DeepMed (Table A2). DeepMed with L Iæ-regularization in the input-layer also has smaller bias and RMSE than the other competing methods (Tables A3 and A4).
As expected, more precise nuisance function estimates (i.e., smaller validation loss) generally lead to more precise causal effect estimates. The validation losses of nuisance function estimates from DeepMed are generally much smaller than those using Lasso, RF and GBM (Tables A5-A7).
Remark 10. Due to space limitations, we defer Cases 4, 5 to Appendix E, in which DeepMed fails to be semiparametric efficient, compared to the Oracle; see an extended discussion in Appendix E. We conjecture this may be due to the implicit regularization of wradient-based training algorithm such as SGD (Table All) or adam (Kingma and Ba, 2015) (all simulation results except Table All), which is used to train the DNNs to estimate the nuisance parameters, instead of actually solving the ERM (6). Most previous works focus on the benefit of implicit regularization (Neyshabur, 2017: Bartlett et al., 2020) on generalization. Yet, implicit regularization might inject implicit bias into causal effect estimates, which could make statistical inference invalid. Such a potential curse of implicit regularization has not been documented in the DNN-based causal inference literature before and exemplify the value of our synthetic experiments. We believe this is important open research direction for theoretical results to better capture the empirical performance of DNN-based causal inference methods such as DeepMed.
Real data analysis on fairness
As proof of concept, we use DeepMed and other competing methods to re-analyze the COMPAS algorithm (Dressel and Farid, 2018). In particular, we are interested in the NDE of race D on the recidivism risk (or the COMPAS score) Y with the number of prior convictions as the mediator M. For race, we mainly focus on the Caucasians population (D = 0) and the African-Americam population (D= = 1), and exclude the individuals of other ethnicity groups. The COMPAS score (Y) is ordinal, ranging from 1 to 10 (1: lowest risk; 10: highest risk). We also include the demographic information (age and gender) as covariates X.
All the methods find significant positive NDE of race on the COMPAS score at allevel 0.005 (Table 2; all p-values < 10 " consistent with previous findings (Nabi and Shpitser, 2018). Thus the COMPAS algorithm tends to assign higher recidivism risks to African-Americans than to Caucasians, even when they have the same number of prior convictions. The validation losses of nuisance function estimates by DeepMed are smaller than the other competing methods (Table A8), possibly suggesting smaller biases of the corresponding NDE/NIE estimators.
9

---

We emphasize that research in machine learning faimesss should be held accountable (Bao et al., 2021). Our data analysis is merely a proof-of-onneey that DeepMed works in practice and the conclusion from our data analysis should not be treated as definitive. We defer the comments potential issues of unmeasured confounding to Appendix F and another real data analysis to Appendix G.
Table 2: Results for real data application to COMPAS algorithm faimess.
Method
Effect
Estimate
SE
Method
Effect
Estimate
SE
Thot
1.136
0.069
That
1.083
0.111
TNDE
0.564
0,068
TNDE (1)
0.589
0.070
DeepMed
TNDE (0)
0.524
0.062
RF
TNDE (0)
0.569
0.103
TNIE (1)
0.612
0.042
THIE (1)
0.514
0.049
TNIE (0)
0.572
0.051
THIE
0.494
0.065
Thot
1.150
0,068
That
1.180
0.068
TNDE (1)
0.575
0.063
TNDE (1)
0.550
0.063
Lasso
TNDE (0)
0.587
0.062
GBM
TNDE (0)
0.526
0.061
TNIE
0.563
0.032
THIE
0.654
0.041
TNIE
0.575
0.040
THIE
0.630
0.044
Conclusion and Discussion
In this paper, we proposed DeepMed for semiparametrie mediation analysis with DNNs. We estab- lished novel statistical properties for DNN-based causal effect estimation that can (1) circumvent sparse DNN architectures and (2) leverage certain low-dimensional structures of the nuisance func- tions. These results significantly advance our current understanding of DNN-based causal inference including mediation analysis.
Evaluated by our extensive synthetic experiments, DeepMed mostly exhibits improved sampe- performance over the other competing machine learning methods. But as mentioned in Remark 10, there is still a large gap between statistical guarantees and empirical observations. Therefore an important future direction to incorporate the training process while investigating the statistical properties to have deeper theoretical understanding of DNN-based causal inference. It is also of future research interests to enable DeepMed handle unmeasured confounding and more complex path-specific effects (Malinsky al., 2019; Miles et al., 2020), and incorporate other hyperparameter tuning strategies that leverage the multiply-rouusnnes. property, such as the minimax criterion (Robins et al., 2020; Cui and Tchetgen Tchetgen, 2019).
Finally, we warn readers that all causal inference methods, including DeepMed, may have negative societal impact if they are used without carefully checking their working assumptions.
Acknowledgement and Disclosure of Funding
The authors thank four anonymous reviewers and one anonymous area chair for helpful comments, Fengnan Gao for some initial discussion on how incorporate low-dimensional manifold assump- tions using DNNs and Ling Guo for discussion on DNN training. The authors would also like to thank Department of Statistics and Actuarial Sciences The University of Hong Kong for providing high- performance computing servers that supported the numerical experiments in this paper. L. Liu grate- fully acknowledges funding support by Natural Science Foundation of China Grant No. 12101977 and No. 12090042, Pujiang National Lab Grant No. P22KN00524, Natural Science Foundation of Shang- hai Grant No.2 1400000, Shanghai Science and Technology Commission Grant No.21JC1402900 Shanghai Municipal Science and Technology Major Project No. No.20211HH2ZZ2X0000, and Shanghai Pujiang Program Research Grant No.20PJ140890.
References
Ben Adcock and Nick Dexter. The gap between theory and practice function approximation with deep neural networks. SIAM Journal Mathematics of Data Science, 3(2):624-655, 2021.
10

---

Michelle Bao, Angela Zhou, Samantha A Zottola, Brian Brubach, Sarah Desmarais, Aaron Seth Horowitz, Kristian Lum, and Suresh Venkatasubramanian. It's COMPASlicated: The messy relationship between RAI datasets and algorithmic fairness benchmarks. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021.
Reuben M Baron and David A Kenny. The moderator-meddiator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6):1173 1986.
Peter I. Bartlett, Philip M Long, Gábor Lugosi, and Alexander Tsigler. Benign overfitting in linear regression. Proceedings of the National Academy of Sciences, 117(48):30063-30070 2020.
Jeremy Glissen Brown, Nabil M Mansour, Pu Wang, Maria Aguilera Chuchuca, Scott Minchen- berg. Madhuri Chandnani, Lin Liu, Seth A Gross, Neil Sengupta, and Tyler M Berzin. Deep learning computer-aided polyp detection reduces adenoma miss rate; a United States multi-center randomized tandem colonoscopy study (CADeT-CS Trial). Clinical Gastroenterology and Hepa- tology, 20(7):1499-1550 1597-1577, 2022.
Sitan Chen, Adam R Klivans, and Raghu Meka. Learning deep ReLU networks is fixed-parameter tractable. In 2021 IEEE 62nd Annual Symposium on Foundations of Computer Science (FOCS), pages 696-707. IEEE, 2022.
Xiaohong Chen, Ying Liu, Shujie Ma, and Zheng Zhang. Casual inference general treatment effects using neural networks with diverging number of confounders. arXiv preprint arXiv:2009.07055 2020.
Victor Chernozhukov, Whitney Newey, Rahul Singh, and Vasilis Syrgkanis. Adversarial estimation of Riesz representers. arXiv preprint 1XXv:2101100009, 2020.
Yifan Cui and Eric Tchetgen Tchetgen. Selective machine learning of doubly robust functionals. 2019.
arXiv preprint arXiv:
Julia Dressel and Hany Farid. The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1 4())xea005580, 2018
Helmut Farbmacher, Martin Huber, Lukáš Lafférs, Henrika Langen, and Martin Spindler. Causal mediation analysis with double machine learning. The Econometrics Journal, 25(2)2277-300, 2022.
Max H Farrell, Tengyuan Liang, and Sanjog Misra. Deep neural networks for estimation and inference. Econometrica, 89(1):181-213, 2021.
Nina M Gottschling, Vegard Antun, Ben Adcock, and Anders C Hansen. The troublesome kernel: why deep learning for inverse problems is typically unstable. arXiv preprint arXiv:2001.01258. 2020.
Wolfgang Härdle, Gerard Kerkyacharian, Dominique Picard, and Alexander Tsybakov. Wavelets, approximation, and statistical applications, volume 129. Springer Science & Business Media, 1998.
Yuling Jiao, Guohao Shen, Yuanyuan Lin, and Jian Huang. Deep nonparametric regression on approximately low-dimensional manifolds. arXiv preprint 20X1v22104.06708, 2021.
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger,
Kathryn Tunyasuvunakool, Russ Bates, Augustin Zidek, Anna Potapenko, et al. Highly accurate protein structure prediction with AlphaFold. Nature, 596(7873):583-58,, 2021.
Jean Kaddour, Aengus Lynch, Qi Liu, Matt Kusner, and Ricardo Silva. Causal machine learning: survey and open problems. arXiv preprint 220.::2200..4475. 2022.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In International Conference on Learning Representations, 2015.

---

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In Advances in Neural Information Processing Systems, volume 25, pages 1097-1105, 2012.
Lingling Li, Eric Tchetgen Tehetgen, Aad van der Vaart, and James Robins. Robust inference with higher order influence functions: Parts I and II. In Joint Statistical Meetings, Minneapolis, Minnesota, 2005.
Lin Liu, Rajarshi Mukherjee, and James Robins. On nearly assumption-free free tests of nominal confidence interval coverage for causal parameters estimated by machine learning. Statistical Science, 35(3):518-533) 2020.
Jianfeng Lu, Zuowei Shen, Haizhao Yang, and Shijun Zhang. Deep network approximation for smooth functions. SIAM Journal on Mathematical Analyvis, 5365):5455-556,, 2021.
Daniel Malinsky, Ilya Shpitser, and Thomas Richardson. A potential outcomes calculus for identifying conditional path-specific effects. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 3080-3088. PMLR, 2019.
Caleb H Miles, Ilya Shpitser, Phyllis Kanki, Seema Meloni, and Bric Tchetgen Tchetgen. On semi- parametric estimation of a path-specific effect in the presence of mediator-outcome confounding. Biometrika, 107(1):159-11)) 2020.
Razieh Nabi and Ilya Shpitser. Fair inference on outcomes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
Whitney K Newey. Semiparametric efficiency bounds. Journal of Applied Econometrics, 5(2):
99-135, 1990.
Behnam Neyshabur. Implicit regularization in deep learning. PhD thesis, Toyota Technological Institute at Chicago (TTIC), 2017.
Judea Pearl. Direct and indirect effects. In Proceedings of the Seventeenth Conference on Uncertainty and Artificial Intelligence, pages 41 120.. Morgan Kaufman, 2001.
Joelle Pineau, Philippe Vincent-Lamarre Koustuv Sinha, Vincent Larivière, Alina Beygelzimer, Florence d' Alché Buc, Emily Fox, and Hugo Larochelle. Improving reproducibility in machine learning research: report from the NeurlPS 2019 reproducibility program. Journal of Machine Learning Research, 22, 2021.
James Robins, Mariela Sued, Quanhong Lei-Gomez, and Andrea Rotnitzky. Double-robust and efficient methods for estimating the causal effects of binary treatment. arXiv preprint trXiv:2008.005000 2020.
James M Robins and Sander Greenland. Identifiability and exchangeability for direct and indirect effects. Epidemiology, 332):143-155, 1992.
Andrea Rotnitzky, Ezequiel Smucler, and James M Robins. Characterization of parameters with mixed bias property. Biometrika, 108(():231-233, 2021.
Johannes Schmidt-Hieher. Nonparametric regression using deep neural networks with ReLU activa- tion function. Annals of Statistics, 48(4):1875-187, 2020.
Charles J Stone. Optimal global rates of convergence for nonparametric regression. The Annals of Statistics, 10(4):1400-1053, 1982.
Taiji Suzuki. Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality. In International Conference Learning Representations, 2019.
Eric J Tchetgen Tchetgen and Ilya Shpitser. Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness and sensitivity analysis. The Annals of Statistics, 40(3): 1845,,845, 2012.
12

---

Kazuma Tsuji and Taiji Suzuki. Estimation enror analysis of deep learning on the regression problem on the variable exponent Besov space. Electronic Journal of Statistics, (5())1189--1988 2021.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. Huggingface's transformers: State-of- the-art natural language processing. arXiv preprint 1rXiv:1910.03771, 2019.
13

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 5 and 6.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendices B and C.
3. If you ran experiments...
(a) Did you include the code. data, and instructions needed reproduce the main ex- perimental results (either the supplemental material or as a URL)? [Yes] See the supplementary materials.
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] See Section 4.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] In all experiments, we provide the Monte Carlo variances in the tables.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See the word document in the supplementary materials.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "koUugODFDM_", "reviewer": "Reviewer_gGUT", "summary": "The authors propose a new method called DeepMed for performing semiparametric mediation analysis with DNNs. \n", "questions": "\n- In remark 3, authors have suggested to use a(d | x, m) and a(d | x) to avoid curse of dimensionality in fitting f(m | x, d) if m is multidimensional and or continuous. However, this alternative solution can lead to new complications such as incompatibility in posing models for a(d | x, m) and a(d | x) while having a coherent joint over X, D, M, Y. That would be great if authors could comment on this and possibly provide remedies to resolve the issue. \n\n- In section 2.2., It would be more clear if the “cross-world” ignobility assumption is written down more explicitly, i.e.,  Y(d, m) \\indep M(d’) \\mid X\n\n- The identifying functional on top of page 3, should have an integration over dPx as well. \n\n- Why is X defined as {0, 1}^p in Section 2.1? The arguments don't seem to rely on restricted state space on X. ", "limitations": "NA", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Their main theoretical contribution is using the multiply-robust behavior of IF-based estimators for (in)direct effect in order to relax the sparsity constraints in training nuisance functions with DNN architectures. \n\nHowever, this work https://academic.oup.com/ectj/article/25/2/277/6517682 (and possibly others) discusses most of what hacve been proposed in this draft regarding the second order bias and properties of the estimator within the sample splitting scheme (theorem 1). \n\nI’m less familiar with the DNN literature, and can’t comment on the novelty of the theoretical claims specific to using DNNs to fit the nuisances. But I find a full-fledged procedure to fit nuisances via DNNs to compute mediated effects worthwhile and not immediately straightforward. \n\nI also think the authors have done a good job with the synthetic experiments.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "h8J8ElJFXro", "reviewer": "Reviewer_NkWa", "summary": "This paper proposes a semi-parametric method for estimating direct and indirect effects from the observational data with deep neural networks.", "questions": "I think the paper doesn't raise too many questions ", "limitations": "- somewhat unclear what the limitations are \n- societal impact has been addressed ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- The paper is somewhat hard to follow with a lot of Mathematical notation that would benefit from a bit better explanation \n- Quality of theoretical and experimental results appears sound - even though I did not go through any derivations\n- The reviewer is unclear on the level of novelty/originality it appears to be ok but can't say for certain\n ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "bOvWsMN6_9", "reviewer": "Reviewer_UV1a", "summary": "This work proposes DeepMed, which reduces bias in semiparametric mediation analysis using the power of neural networks. It relaxes sparsity assumptions of the prior work on the theoretical side which gives more flexibility and expressivity to neural networks and validates it with synthetic and real-world data experiments.", "questions": "In remark 10, the authors conjecture that the reason DeepMed is not semiparametric efficient in cases 4 and 5 is due to implicit regularization of Adam. It would be interesting to test this conjecture by running these experiments with normal SGD as the optimizer, and checking if DeepMed becomes efficient.", "limitations": "yes", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The Paper is well written.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "E0Gr_JPka2b", "reviewer": "Reviewer_7FFK", "summary": "This work proposes DeepMed, a semiparametric causal medication analysis framework, which focuses on natural direct/indirect effects (NDE/NIE) in the causal analysis domain. By leveraging the second-order bias property of the multiply-robust estimators of NDE/NIE, DeepMed is able to outperform baseline methods in both simulated datasets and real datasets.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe authors provide extensive theoretical analysis as to the semiparametric multiply-robust estimators of NDE/NIE, as well as the statistical properties of DeepMed\n\nWeaknesses: \nThe authors rely on synthetic experiments to provide in-depth analysis of how DeepMed is able to outperform baselines as well as perform better causal mediation analysis.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept