Estimating Causal Effects Identifiable from a Combination of Observations and Experiments
Authors: Yonghan Jung
Ivan Diaz
Jin Tian
Elias Bareinboim
Venue: NeurIPS 2023 poster (2023)
Keywords: Causal Effect Estimation
Causal Effect Identification
Data Fusion
Double Machine Learning
Doubly Robust Estimator
TL;DR: We develop estimators for identifying causal effects by combining multiple observational and experimental samples.

Abstract: Learning cause and effect relations is arguably one of the central challenges found throughout the data sciences.
Formally, determining whether a collection of observational and interventional distributions can be combined to learn a target causal relation is known as the problem of generalized identification (or g-identification) [Lee et al., 2019]. 
Although g-identification has been well understood and solved in theory, it turns out to be challenging to apply these results in practice, in particular when considering the estimation of the target distribution from finite samples. 
In this paper, we develop a new, general estimator that exhibits multiply robustness properties for g-identifiable causal functionals. 
Specifically, we show that any g-identifiable causal effect can be expressed as a function of generalized multi-outcome sequential back-door adjustments that are amenable to estimation. 
We then construct a corresponding estimator for the g-identification expression that exhibits robustness properties to bias. We analyze the asymptotic convergence properties of the estimator. Finally, we illustrate the use of the proposed estimator in experimental studies. Simulation results corroborate the theory.

Estimating Causal Effects Identifiable from a Combination of Observations and Experiments
Yonghan Jung', Iván Díaz", Jin Tian', and Elias Bareinboim"
Purdue University
jung2228purdue. edu ivan diaz@nyu. edu jtian@iastate edu ebccs www.woia.......oooooo.o......... edu
New York University
3 lowa State University
CCouubbia University
Abstract
Learning cause and effect relations is arguably one of the central challenges found throughout the data sciences. Formally, determining whether a collection of obser- vational and interventional distributions can be combined lear a target causal relation is known as the problem of generalized identification (or g g-identifiiation) [Lee et al.. 2019]. Although g-identification has been well understood and solved in theory, it turns out to be challenging to apply these results in practice, in partic- ular when considering the estimation of the target distribution from finite samples. In this paper, we develop a new, general estimator that exhibits multiply robust- ness properties for g-identifiable causal functionals. Specifically, we show that any g-identifiable causal effect can be expressed as a function of generalized multi- outcome sequential back-door adjustments that are amenable to estimation. We then construct corresponding estimator for the g-identification expression that cxhibits robustness properties to bias. We analyze the asymptotic convergence properties of the estimator. Finally, we illustrate the use of the proposed estimator in experimental studies. Simulation results corroborate the theory.
Introduction
Performing causal inferences is a crucial aspect of scientific research with broad applications ranging from the social sciences economics, biology medicine. provides set of principles and tools to draw causal conclusions from a combination of observations and experiments. Two significant tasks in the realization of these inferences causal effect identification and estimation. Causal effect identification concerns determining the conditions under which one can infer the causal effect P(Y = y)do(X æ)) (shortly, P(y ()) of the treatment X I the outcome Y y from combination of available data distributions and a causal graph depicting the data-generating process [Pearl, 2000, Bareinboim and Pearl, 2016]. Causal effect estimation aims to develop an estimator for the identified causal effect expression using set of finite samples.
Recent advances in the literature generalized causal effect identification (g (g-identification) have developed algorithms that can identify causal effects by using a set of observational and experimental distributions and a causal graph. The result is expression of the causal effect function of available observational and experimental distributions [Bareinboim and Pearl, 2012, Lee et al., 2019]. For concreteness, consider some practical scenarios that exemplify g-identification.
Example 1. Many studies have investigated how training program's eligibility (X) affects future salary (Y) (e.g..[Gll g.Glln and Kashin, 2017J). Actual registration in the program (2) determines the salary, and experimental studies have looked into how Z affects Y (e.g., [LaLonde, 1986)). Eligibility determined by past average income (W), which is associated with both Z and Y. The causal
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

X1
W
X2
R
W
Z
Y
(a)
(b)
Figure I: Causal graphs of examples and 2. The nodes representing the treatment and the outcome are marked in blue and red, respectively.
graph in Fig. Ia shows the data-generating process, with bidirected edges indicating unmeasured confounders affecting the variables. According to Lee et al. [2019], the causal effect P(y do(x)) can be identified by combining the experimental distribution on Z (denoted P(-Ido(=)) with the observational distribution P. It's given as P(y P(y)d)(>)) = Σx... ((y)do(=)) ()//:w)))p. P(w).
Example 2. There have been many experimental studies on the effect of an antibypertensive drug (X1) on blood pressure (W) (e.g.. Hansson et al. [1999]) and on the effect of using an anti-diabetic drug (Xx2) on cardiovascular disease (Y) (c.g.. Ajjan and Grant [2006], Kumar et al. [2016]). R a set of mediators. Their relations are depicted in Fig. 1b. Recent stud- ies report that simultaneously taking antihypertensive and anti-diabetic drugs may be harmful [Ferrannini and Cushman, 2012]. This motivates the study of the combined causal effect of both treatments (i.e., P(yldo(z1, by combining the two experimental studies (i.e., from P(-Ido(x1)) )) and P(-[do(12)) )). According to Lee et al. [2019], turns out that P(y]do(z1, = Enw P(yir, rr, w, d)(=z))P(( (r|122, (o(x1))()) En; P(w|r, 222, do(x)) ( ((x)ldo((x) the joint treatment effects can be computed using the two experimental studies on X1 and X2.
do((x))) 1)), which means that
On the other hand, causal effect estimation has mainly focused on limited identification scenarios, relying on stringent assumptions such as the no unmeasured confounder assumption. Beyond these restrictions, recent progress has been made in developing statistically appealing estimators from observational data for any identification functional given by the complete identification algorithms [Jung et al., 2020a,b, lba, Bhattacharya et al., 2022, Xia et al., 2021]. While these estimators are capable of estimating any identification expression from observational data, they are not yet suffi- ciently advanced to estimate g-identification,, which involves multiple observations and experiments.
Recently, Jung al. [2023] generalized existing doubly robust estimators [Mises. 1947, Bickel et al., 1993, Robins and Rotnitzky, 1995, Bang and Robins, 2005, Robins et al., 2009, van der Laan and Gruber, 2012, Luedtke et al., 2017, Chernozhukov et al., 2018, Rotnitzky et al.. 2021] to estimate covariate adjustments (e.g.. back-door adjustment [Pearl, 1995], sequential back-door (SBD) adjustment [Pearl and Robins, 1995] or multi-outcome SBD (mSBD) [Jung et al., 2021bl) in the g-identification setting, where the expression is in the form of covariate adjustment involving multiple experimental distributions. However, the covariate adjustments only cover a limited por- tion of all g-identifiability scenarios in Examples (1,2). On a different thread, Xia et al. [2023] developed a neural network-based estimation framework capable of taking a combination of observa- tional/experimentu data. Still, the derived estimators do not possess the doubly robustness property. In other words, there is still a gap between g-identification and causal effect estimation.
In this paper, our goal is to bridge the gap between g-identification and causal effect estimation. Specifically, this paper presents framework for estimating identification expressions using multi- ple sets of samples from both observational and interventional distributions. This framework is a generalization of the results in Jung et al. [2021b] since our results reduce to theirs when only ob- servational data is available. Furthermore, our work subsumes the results in Jung et al. [2023] when the identification functional takes the form covariate adjustments.
The contributions of our paper are as follows:
We show that any causal effects identifiable by g-identification can be expressed as a function of generalized mSBD adjustments. We provide systematic procedure for specifying the function.

---

2. After developing doubly robust estimator for generalized mSBD adjustments, we construct an estimation framework for any g-identifiable causal effects, which enjoys multiply robustness against model misspecification and bias. Experimental studies corroborate our results.
1.1 Preliminaries
We use bold letters (X) to denote a random vector and X random value. Each random vector is represented with capital letter (X) and its realized value with small letter (x). Given a set X
{X1,,.. } aligned by an order such that X; x X, for i < j. we denote X == {X1,... XX,
(X) to represent the indicator function such that 1x(X)= 1 if X = x; [x( = otherwise. We use [m]] := {1...... .n) a collection of index. For discrete vector V. we use P(v) == P(V =v) where P is a distribution. We use Ep [S(V]] = Eveey ((v) for a function f. where Gy denote the support of V. We will use Dv to denote the domain of V. For a sample set D = (Vo)) where V(a) denotes the We use ||/ = /Ep [EP((.(V))T. If function f is a consistent estimator of f having rate rn, we will use f f = Op(ra). We will say f is Lz-consistent if f fll P op(1). We will use f-f (p(1 if f is bounded in probability. Also, / - I is said to be bounded in probability at rate rn if ʃ - use the typical graph terminology cC()) de(C)) an(C), (CC) to represent the union of C with its parents, children, descendants, ancestors in the graph G. We use pre C; G) to denote the union of the predecessors of C2 E C given topological order <g over graph G. We use C(C) to denote the subgraph of G' over C. Throughout the paper, we will assume a fixed topological order <o over V on G.
and X {X,,... ..,....))........ .x,)- For a discrete vector X,
ith samples, we use ED [s(V] = (1/n) Ei=/
Op(rn). We
ch,
Structural Causal Models (SCMs). We use Structural Causal Models (SCMs) as our framework [Pearl, 2000, Bareinboim et al., 2022]. An SCM .M is quadruple M = (U, V, P(U), F), U is a set of exogenous (latent) variables following a joint distribution P(U). V is set of endogenous (observable) variables whose values are determined by functions F = (ʃv; Jv,ev such that V; < fv; (po,, u.) where PA, S V and U. C U. Each SCM M induces a distribution P(V) and a causal graph G G(M) over V in which there cxists a directed edge from every variable in PA, to VÚ and dashed-bidirected arrows encode common latent variables (c.g., see Fig. Ia). Performing an intervention fixing X x is represented through the do-operator, do(X x), which encodes the operation of replacing the original equations of X (i.e., fx (paz, Uz)) by the constant x for all X X and induces interventional distribution P(V]do(x)).
Experimental Distributions and Samples To clarify the connection between the experimental sam- ples where the randomization is applied to Z C V and the distribution ()) we introduce the notation Po(z)( (V) where (Z) denotes that Z is randomized. The distribution (z) ( (V) is dis- tribution induced by the SCM in which the original equation Z + fzz fp uz) for Z E Z is replaced to the function assigning the value to Z
at random without depending on other en- and 0 at probability 0.5 for each. We note that P = Pa(o) when observational. For any set A, B, Z C V, the interventional distribution can be represented as P(A]do(z, (do(z). , B) = (Z(z) (A I = z, B) by the definition of the do-operator and Pa(z) distribu- tion. We use Pe (A = Zz) (A|Z
dogenous variables PAz; e.g., Z
z, B) to highlight that the distribution is induced from the randomization and conditioning on Z z. The experimental samples from randomization a (Z) induces samples Dz(2) following Pa(z) (V). We use Dz to denote the subsample of Dz fixing
Z = z, which follows (v)V..
g-identification. Let Z := {Z, ) _ denote a collection of variables where Z, can be an empty set. Let P = (P) (V: Z, E Z),, a collection of distributions inducing experimental samples from trials randomizing Z, E Z. A causal effect P ((y]do(x) is said to be g-identifiable from IP in a causal graph G if P(y]do(x) dd(x)) is uniquely computable from the combination of distributions in P in any SCM that induces G [Lee et al., 2019. Def. 4]. The complete g-identification algorithm developed by Lee et al. [2019] identifies the causal effect by decomposing so-called confounded components (c-component). A c-component is a maximal set of variables where every pair is connected by a bidirectional path composed of bidirectional edges (V: <> V₁). For example, graphs in Figs. (la, 1b) form a single c-component since bidirectional paths connect any pairs of variables. For any sets C C V, the quantity Q == P(c) is called c-factor. To identify the causal effect
3

---

P(y do(x)) from P and G, the g- g-identification algorithm in [Lee et al., 2019, Algo. 1] (and rewrote in Algo. 1) rewrites the causal effect as a marginalizaaiin over a product of c-factors, P(y|do(x)) = Dayye fa Q[D:], where D == an (a(V))( and D, are c-components in GDD, and identifies each Q D from P.
1.2 Problem Statement
This paper aims to develop an estimation framework for the g-identifiable causal effect ((y]do(x)) identified as function of distributions in IP from experimental samples ID = (Dz, Po(za) (V) P). We impose the following regularity assumptions:
Assumption 1 (Regularity). For variables V and distributions Pa(z) E P, the following conditions hold: (1) All variables in V are discrete; (2) (v) c, cv Dy for some (0,1). 1).
We discuss the relaxation of the regularity assumption in Appendix C. This relaxation allows some subset of variables in V can be mixture of continuous and discrete random variables. Due to space constraints, all proofs are provided in Appendix B.
2 Expressing Causal Effects as Combination of mSBD Adjustments
In this section, we present an algorithm that expresses any g-identifiable causal effects as a combina- tion of wwr.giaillzaioommmmlllliiittttti
of adjustment functionals defined in the following.
We begin by formally defining the generalized multi-outcome sequential back-door adjustment (g- mSBD) functional, which strictly generalizes the mSBD adjustment proposed by Jung et al. [2021b]:
Definition 1 (generalized-ig mSBD adjustment (g-mSBD)) Let (W, R) be a disjoint pair in V topologically ordered as (W.R) R) = RRg W,, ... R,,. W m. Rm } by <a. where R, can be and R == 1 for Vi E [m]] Let C S W. Let Zu S Z be some set such that 4Z E Zo. Wnzz n = 0. Let seq(Zo) Zo) denote a sequence (Zi,... 2m) where 2; denotes some realization of Z; Zo (same z, could appear multiple times in the sequence). Then, the g-mSBD adjustment expressed as an operator Ao(W,
empty. Let W == {W
[W C, R; Zo, seq, G (ww,,, r) defined by
W C, R: Zo, see (w/c.
Σ II CEêc &.WEW
Pz. (u,
zz).
(1)
The g-mSBD adjustment specializes to the mSBD adjustment [Jung et al., 2021b] when Zo - 0. The g-mSBD adjustment can be viewed as variant of the g- g-formula [Robins, 1986) involving multiple distributions. The power of the g g-mSBD adjustment lies in its ability to express the e-factor:
Lemma 1 (c-component Identification [Jung et al., 2021b]). Ler S denote c-component in G. == Z..) for some Z; E Z. Let = pa(S)a, \S. Let (S.R. be ordered as (Ro, Si, Rm Rm-11, by Lg. Let A C S denote set satisfying A an(A)),(s) Let C = (S) Let Zo = {Z,} and seq(Zo) be a sequence of repeating m times. Then, the
c-factor ({A} g-identifiable as follows:
Q[A]= AosS, C, R. {Z,) segl (a,r)
Σ II cEE J:V,ES
Pa. (1. 8 , Zz).
We propose an identification algorithm, Algo. 1, which expresses any causal effect as a combination of marginalizations, multiplications, and divisions of g-mSBD operators. Here are some results used for the g-mSBD operation. An example of using these results is provided in Appendix A.
Lemma 2 (Marginalization), Let Ao Ao C, R. Zo, seq] (w)l(w) c, r) denote the g-mSBD operator in Def. 1. Let Wo C. W C. Let W mar C (W 0: C} denote the vector formed by the following proce- dure: Starting from W = 0. for j = m.
W
mar W U (W;} if (I) W, E {Wo. C}
and (2) 3 E i...... .m). such that Rj..... .,..,,,,. RR-I- = 0. W
k+1m
C W and Z₁ _... Z₁
and 2ạ =.. Zj. Let W W Wman R' = pre( W
o. C/ NR and C' == (
W mar Let
Z' Zo denote the collection of Z, corresponding to the variable in W', and seq' the correspond- ing sequence. Then,
Σ WoE
Ao A, C, seq[(w)(c.)) A, C','' ",,, (eg
(3)

---

Algorithm 1: GID (x.y.
Input: y, 2 (Z:), (Potz) (V), 4Z, E Z
Output: Expression of P(y]do(x)) w.r.t distributions in P
If EZ2 Z such that P Pz. (y) for some 2; 2. then return Pa. (y).
2 Let V & an(Y); P(v) + P(an Y)); and G + G(an(Y)).
Let D an( ( G(V())
Find the C-component of G(D): D1, DR.-
5 foreach D;
M do
foreach Z, E Z do
Find the c-component Sj in G(V ( such that Dj Sjj.
Q Ao 0. R 2 (Z.) seq)] (rr,,))) rj), where R = pa(())avvvi)))s) //
By Lemma Run QDD,] SUBID(D,, Sj. QIS G(S))).
If Q.P,] # FAIL,, then break.
end
12
If Q[D,] = FAIL, then return FAIL.
13 end
P(y]do(x)) Earyee I1, Q[D,]. // Apply Lemmas (2,3,4) if viable
15 return P(y (()))
Procedure SUBID(C, T, (TT, G(T)
a.2
Let A = an (C) {A1, A2, An,) such that A1 <a ... <a An, in GT).
u.3
Let [[A] Ee Ectasrr Q[T]. // Apply Lemma if viable
If A C, then return [[A..
u.5
If A = T, then return FAIL.
else
a.7
Let be the c-component in G(A) such that C C S.
Eb; 1c0 Q[A]
Let Q[S] = II (RAAES) Eneen, [(A] for :; : AAA // Apply Lemmas 32,3,4) if viable return SUBID (C G(S))
a.8
a.10
end
W, C, R: Zo seq](w') (w',,, given This lemma provides graphical criterion where Emocow W, as g-mSBD operator.
Lemma 3 (Multiplication). Let Ab
Ao W, ,, 0. R. Z, seq'i(w..
I Psj (wogl W F. Zz)) for {1,2) where seq' = ym Let W W1 U W2. Let
R = (R, U ( W. Let (W, R. be ordered by <g. Let Z, Zz. Assume the following: (1) (22)
W, W, V: and (2) VW, E W, W,WWi,k E W, such that (w -1 R R (.. R.
Let seq = (2,)).Weew where z 2 for all j. Then,
A x A3 W. 0. R; Z, seq) w.r)
II Pe w/ F
(4)
j.SEW
This lemma provides graphical criterion where product A A is given as a gBD operator.
Lemma
4 (Division). Let Ai = Ao 0. R4; E, seq') rw;,
=
I P
(wig w
z2) for E {1,2} where seq' = -z)) Let W = W,
Let R
(R U W2) pre( G).
Assume the following: (1) W₂ C W1: and (2)
VW, E W. AW W, such that (W"
= (W1 RR ) Z Zj and Zi = Zj.
Then,
Ab/A3 = Ao W, 0 R: , seq (w,r)
II j:WeW
Pe, (w, 1zy)
(5)
5

---

This lemma provides a graphical criterion where product Ab/A3 is given as g-mSBD operator.
We have rewritten the identification algorithm proposed by Lee et al. [2019] as Algo. express the g-identifiable causal effect as a combination of marginalizations, multiplications, and divisions of g-mSBD. It's worth of noting that the identification algorithm proposed by Lee al. [2019] and Algo. are equivalent.
Theorem 1 (Expression of g-Identifiable Causal Effects). Algo.
returns any g-identifiable causal effects as a function of a set {A0} of g-mSBDD adjustment operators in the form
P(y]do(x)) = f((4bTm)))
(6)
where the function f(-) applies marginalization, mulriplication, or division over g-mSBD operators in { J as specified by Algo. 1.
For concreteness, demonstrate the application of Algo. 1 for Figs. (la,Ib), where the effeets P(y [do(x)) are g-identifiable. Detailed and visually friendly demonstrations are described in Ap- pendix A.
Example 3 (Application of Algo. 1 to Example 1). Note Z = =(0, 2}. Line 3-4: D {2, Y
where D {Z} and D [Y}. Line 5-13: Identify DD1 from Z, = 0 as follow. Note D2 C S" where S == V where Q[87] = Ao 0, 0: 2n == 0, (s 0) = P(v). Run @ = SUBID(D1, So Q[ G) and obtain Q[D,] = Ag == Aol(W, 2), W, W.
2 P(z|2r, P(w). Lemmas (2,3,4) are used in running the sub-procedure. Now, identify
[W, XW, the c-component in Z). Note Q Ao(S" 0: 2 = ( (s 0) = =P,(w, x. y). where seq' = (,, We run Q = SUBIDD St, "(S),, G(S))) and obtain Q[ = A Ao 0, 0: Z seq (v,
@[D₂] from Z₂ =[2] follow. Note D2 C S'
Pv()). Lemma is used in the sub- procedure. Line 14-15: P(y]do(z)) = Exeo, Ab18-
Example 4 (Application of Algo. 1 to Example 2). Note E = [2). Line 3-4: D =
{R. Y} where D1 == {R). D2- {W). and D3 = [Y]. Line 5-13: In G(V ) D1 = [DD]=Q[S]]
S = [R).
= A = R. x2; = {[x1).sq] x2) P(r d)())))))))
where seq'
(z1). In G(V X1). D2 S.S {X,, W.Y.. Q
Ag 0, R: =
(X,). s r) = (x) Pa,
where seq = (1). Run S) Q[[) ((S)) G(S)) = =A = Aol(X2, W), X2, Rt " seq") =
P,,
QDaa = SUBID(D2,
Lxse6x, x, Pa, (w|r, 2/2 Pr, (12) where seq* = (x), Lemma 2 is used in the sub-procedure.
Since SUBID(D2, S ([S) G(S) retum FAIL. we find the e-component S = (Y} where D3 St. Note @D2, Q[Si A = An 0, (R, W); = (X2). seq fr, w
Pz (iww.)). r), where seq" = (I2). Line 14-15: Applying Lemma 3, A = A x Ag
Aol R, YJ.0 0, (X2, 213 = (X,, X2), seq" 13) ((r,),, (x2, w)) Pa, (r(og) P (ulr,w)
where seq 13 = (2). Then, Ply (2)))) = EnWEBR A A3
3 Estimating g-Identifiable Causal Effects
In this section, we develop estimator for P(y]do(x)) using samples TD) (Do(z.) (o(z)) (V) E P) obtained from randomized experiments and observations (where Z: = 0). We use Pa(z) instead of P& to highlight the distribution Do(2.) E D follows.
We first introduce an estimator for the g-mSBD adjustment that exhibits the doubly robust property. The nuisance parameters for the g-mSBD adjustment are defined as follows:
Definition 2 (Nuisances for g-mSBD). Nuisances for g-mSBD Ao in Eq. (1) are {ut+1 -
defined as follows.
Let 10
m++1 = m== == Wwo(WW' (W.C). For i = m 1, 1,
1b (W,, R = Eroc,,,)) 11 (W' +1
R 1:8 ro, Zi+1
.1t1.
Paiza (W R
1r, (R.)
Tód (W', R Poiz. R
(8)
1241, ro) Pa(Z. (R, W. R 1i-1 Zo)
Remark 1 (Simplitication of Nuisances). Although the nuisances TI may seem compli- cated, they can be simplified in several important special cases. For example, zij
1r (R. R' R .z,ro) ifz (Z) for any Z C V where Z is possibly empty.

---

In general, employing off-the-shelf classification methods for density ratio estimation is feasible, leveraging the techniques outlined Section 5.4 of Díaz al. [2021].
We now introduce a g-mSBD estimator exhibiting the robustness properties using these nui- sances. This estimator is motivated by the double/debiased machine-learning style estimators [Cher- nozhukov et al., 2018, 2022]:
Definition 3 (DR-g-mSBD Estimators). Let Da(Z,) for Z₁ denote the experimental sam- ples from randomizing the variable Z, Let De, for Z, E Dz. denote the subsamples of Dr(z,) fixing R, Z; = ro 2 and Z, Z;. The DR-g-mSBD estimator Å for the g-mSBD adjustment Aol W. C, R: Zo = [z h seq == (z, (w (. r) is defined as follows:
1. Randomly partition Dx, into
i.e., Dz: = UE Punes VZ, E 2 and E Zz.
2. For each fold l E [L], let p i+1 denote learned Ho i+1 using De4,1 (D De for i = m. ... 2; and
TH learned Tú for i = 1, ". m 1. Define júg 1i++1 == 11 (W" Fú, R 1: and मंे == -1 m.
3. Estimate Á = A(tuttt (/Deemm-1,eeE1)) = (1/L) Ae(tmgt T/ (eeje/mm- where
m-1
A = Aelluttt bebemm-
E Destrie
-
- plttt] +ED,, [p]]] (9)
where E. Day.e H is an empirical average over samples Dzy.t-
We now analyze the doubly robustness property of this estimator.
Proposition 1 (Asymptotic Analysis of g-mSBD Estimators). Assume that the nuisance estimates
He and 7 are Læ-consistent; i.e., || i+1 140 | ( (1), 11,5+ - -1b Pr:z.41)
( (1) and || Toll Pa1z = 4112 (1)for 1,... m 1, and ||ا p Po(Z1>
OPo(z)) (1). Let n4 = Dz. for i E {1, m). Then,
m
L m
Å A
Σ R
Opeiznr (lln 140 1 11l =bll);
(10)
where RG is random variable such that n22 R₁ converges distribution to a mean-zero normal random variable.
This estimator possesses a doubly robustness property since the estimator is bounded in probabil-
ity at rate n 112 (for n == min(n ;mm) whenever Op. .. (Ilu 1+1 Po ||| =fll) =
6+1
Op(n7713 1/22 for all i.
We now construct an estimator for the g-identification expression using the DR-g-mSBD estimator defined in Def. 3. The resulting estimator called the MR-gID estimator:
Definition 4 (MR-gID Estimator). The MR-gID estimator & for the identification expression of the causal effect du = f({AbT [mm () in Theorem is given as follows: For each AB composing f((A)) let A = A (f T/eeeeemm -1),E(()) denote the DR-g-mSBD estimator with nuisance estimates [ Thee for the true nuisances [u _ - Then,
= f(tA* 1k=1).
(11)
We impose assumptions on the identification expression and its nuisances for further analysis.
Assumption 2 (Analysis of MR-gID), ). The identification function I S(AAK (k=1) in Thm. and each nuisances (u D for A satisfy the following properties:
1. Twice differentiabilliy: J(AA] { is twice continuously Frechet differentiable w.r.t. { Jk=1 w.r.t. {Ak) 1k=1.
7

---

2. Boundedness: Ak E [K] vZ, E Σ, V VA A+ 00) 11 - -AA = OP P(1)) (AA -A()
3. Lz-Consistency: ll - M2,0 P. (1 = ox (1), linkt - -1/0011 Pact
i+1
= 00 112g (1),
| Ti.oll mottta) or, ( (I), and || P ((2))) =OP 22 (1).
Assumption is imposed to limit the error of the MR-gID, which is linear function of the errors of each DR-g- m-D estimator.
Theorem 2 (Asymptotic Analysis of MR-gID). Suppose Assumption 2 holds. Let meei = DD for Z E E and z E Dzz. Let & denote the MR-gID estimator in Def. 4 for the causal effect 40 = [((A0)
K k=1= in Theorem 1. Then, the error of to is given as
m*
L m
o (m 1/2) 1
3 v0 Σ>
Op
Op
Pizz)) (llate 1k.0 IlI #k,ll))
(12)
We highlight that the gRD v exhibits robustness property since & - 2/ for d/o = P(y)do(x)) is bounded at rate n 11/2 (for n = min(nx,} and E IP) even when all nuisances {Mk,e o are bounded slower 1144 rate. Furthermore, the MR-gID estimator exhibits multiply robustness, as the error of the MR-gID a linear function of the error of DR-g-mSBD, which demonstrates the doubly robustness property. Formally,
Corollary 2 (Multiply Robustness (Corollary of Thm. 2)). Suppose (1) Assumption 2 holds; (2) Either Thet Tkk0 or F.g. 12 for = i 1,...,,"" for all i, E, k; and (3) all nuisances (+1 (Tk ME,E JE, are bounded by some constant. Then, the MR-gID & (Def. 4) is consistent to tho.
For concreteness, we illustrate the application of Thm. 2 for Examples (1, 2). Detailed procedures provided in Appendix A.
Example 5 (Application of Thm. 2 to Example 1). Recall that ((y)))())) f((Ah. 45) = E.Es, AbA The nuisance set for Ag is Mi.0 (X, X, W) Ep [1, (Z) X, W and _i,o(X, (X, W) = 1x()//(((WW) Then, the estimator for Ab is A = (11 ( defined in Def. 3. The nuisance set for AB is p2ã0 = Pa (Y)). Then, the estimator for A is d = (122 (eee)))) Then, the estimator is constructed as Def. 4, as S([A), (((A),A)) (²)). By Thm. the error of the estimator is Op(mo 11/2) + Op(n=112
(1/L) Et_ 11=1 Op(lIn ri -t,ol)), where no D and n, = [D2] where D ~ P and Dj P..
Example 6 (Application of Thm. 2 to Example 2). Recall that P(yldo(21, x2)) = f((A), (b3)) Er wr.ww...w ABAD3. The nuisance set for AB 12,0 Ep,, [w) X2] and L.o (R)(RR ( (R| Then, the estimator for AB is A = (2(1220 JeEe))) The mui- sance set for A is 1I3.0 = Ep., (R, Y)JR, W1 and T|3.0 = (o(x)((leaa 1.(W)
(M( (o
Then, the estimator is Á13 = (13(fuf T tl,,oee Then, the estimator is constructed as Def. 4, as f( A2)). By Thm. 2. the error of the estimator is OP.Xx,) (n, (1/ + Op.nn)) 1/2) (1/L) 2 (Orexx (1122 - 1
=22ol) + OPeOy (INLae - Piaolll - (.,))))) where = Dil and n2 = D2l where D₁ ~ Pa, and D2 ~ Pay
4 Experiments
In this section, we demonstrate the MR-gID estimator from Definition (4) through Examples (1,2) and Project STAR dataset[Krueger and Whitmore, 2001, Schanzenbach, 2006). For each example,
the proposed estimator is constructed using a dataset ID := (Dz.: Z. E Z} simulated from an underlying SCM. Our goal is to provide empirical evidence of the fast convergence behavior and the robustness property of the proposed estimator compared to competing baseline estimators. We consider two standard baselines in the literature: the 'regression-basel estimator (reg)' only uses the

---

Scenario
Scenario 2
Scenario
Scenario
Modi
Mosi 11
Fig. 1a (Example 1)
Voda
Vocd
Fig. 1b (Example 2)
2
Figure 2: AAE Plots for Examples (1,2) for Scenarios (1,2,3.4) depicted in the Experimental Setup section. The æ-axis and y-axis are the number of samples and AAE, respectively.
regression nuisance parameters μ² for شá defined Def. 2, and the 'probability weighting-based estimator (pw)' only uses the probability weighting parameters x=-1 for mo def-ndeefindd Def. 2, while our MR-gID uses both in estimating the g-mSBD operators A composing [omposing_([[44) f([A*)) in Thm. 1. Details of the regression- based ('reg') ) and the probability weighting-basee ("pw") estimators are provided in Appendix A. The details of the simulation are in Appendix (D, E).
4.1 Synthetic Dataset Analysis
Accuracy Measure. We compare the proposed estimator ('mr') in Def. to the regression-based es- timator ('reg') and the probability weighting-based estimator ('pw"). In particular, we use 7 (x) for est E (reg, pw, mr} to denote the g-ID estimators that leverage regression-based ('reg'), probability weighting-based ('pw'), ) and MR-gID in estimating each operator A in the identification expression s((A*)) of the causal effect P(y do(x)). We assess the quality of the estimators by computing the average absolute error AAE Fxx ΣxeDx
[x [do]]] where |Dx] is the cardi- estimated using gradient boosting models called XGBoost [Chen and Guestrin, 2016]. We ran 100 simulations for each n = {200, 400, 600, 800, 1000} for Dz for vZ C Z. We label the box-plot for these AAES as 'AAE-plot'.
nality of Dx. Nuisance functions
=
Experimental Setup. We evaluate the AAESt for Examples (1,2) in four scenarios:
(Scenario 1) There were noises in estimating nuisances.
(Scenario 2) We introduced converging noise in estimating the nuisance, decaying at rate (i.e., 6 ~ Normal(n - )) for = 1/4 to emphasize the errors induced by the finiteness of samples. This scenario is inspired by the experimental design discussed in [Kennedy, 2020].
(Scenario 3) Nuisance {ukt Je, are estimated incorrectly simulated by training the model with random matrix having the same dimension as the input matrix.
(Scenario 4) Nuisance { t are estimated incorrectly in Scenario
In Scenario 1, we aim to show that all estimators 7 TPW T are converging to the true causal quantity Ply]do(x)). In Scenario 2, we aim to show that the MR-gID estimator exhibits fast con- vergence behavior compared competing estimators. Scenario (3,4), our goal is to highlight the multiply robustness property of the MR-gID estimator.

---

X1
X2
05
Model
65
Model
0.5
Model mi
04
C4
0.20
R
03
63
0.15
022
010
W
011
005
00
20
830
1300
(a) Graph
(b) Scenario 2
(c) Scenario
(d) Scenario 4
Figure 3: A graph and the AlE- for Project STAR.
Experimental Results. The AAE plots for all scenarios are presented in Fig. 2. All the estimators ('reg', 'pw', 'mr') converge in Scenario I as the sample size grows. In Scenario 2, where the estimated nuisances are controlled to be bounded in probability at n 1-44 rate, the proposed MR- glD & outperforms the other two estimators by achieving fast convergence. This result corroborates the robustness property in Thm. 2. Scenarios (3.4), where the estimated nuisances for {u² )=22 or {= are wrongly specified, the MR-gID estimator converges while other estimators fail to converge. This result corroborates the multiply robustness property in Coro. 2.
4.2 Project STAR Dataset
This section provides an overview the analysis using Project STAR dataset [Krueger and Whit- more, 2001, Schanzenbach, 2006]. Project STAR investigated the impact of teacher/student ratios on academic achievement for students in kindergarten through third grade. The dataset D includes class size (XX,), the academic outcome in kindergarten (W) for kindergarten, the academic outcome in sec- ond grade (R), class size (X2), and the academic outcome for the third grade (Y). We assume that the SCM M underlying Project STAR dataset D can be depicted in Figure 3a. The target quantity is E[ [d(((1 where Ps =s (v) = ErED, Pz, (r) Ext,wEDX,, Pzg (yl=4, w., v, 222 Pa, (x1, w). The detailed procedures are in Appendix E.
Experimental Setup. We generate datasets Dj and D2 from the original dataset D to demon- strate the gID estimation. D2 is a random subsample of D with only (X1, W, R} and follows (XX, W, R). D2 constructed by resampling from D between X1 and W, and X, and Y presents, following Po(xa)(Xi (X1, W, X2, R. R.Y). We conducted 100 simulations by generating new instances of D1 and D2 to create the AAE plot. Estimators were constructed solely from D, and D2, with D used exclusively to construct the ground-truth estimate.
way that the confounding bias
Experimental Results. We evaluated the AAE of estimators T for est E {reg, pw, mr). The AAE plots for scenarios (2,3.4) are in Figs. (3b,3c,3d). Our findings indicate that the MR-gID estimator mr consistently provided reliable estimates for the ground quantity.
5
Conclusions
We present a framework for estimating the causal effect P(y]do(x)) by combining multiple observa- tional and experimental datasets and causal graph G. We introduce the generalized multi-outcome sequential back-door adjustment (g-mSBD) operator (Def. and its operations. We show that any g g-identifiable causal effects can be expressed as function of the g-mSBD operators specified Algo. 1 (Thm. 1). We then develop an estimator called DR-g OR-g-mSBD (Def. 3) for the g-mSBD operator and analyze its statistical properties in Prop. 1. Based on the DR-g-mSBD estimator, we develop the MR-gID estimator (Def. 4) and analyze its statistical properties (Thm. 2 and Coro. 2) which exhibits fast convergence and multiply-robustnes, Our experimental results demonstrate that the -R-gID estimator is a consistent and robust estimator of P(y do(x)) against model misspecifi- cation and slow convergence.
10

---

Acknowledgement
This research was supported in part by the NSF, ONR, AFOSR, DoE, Amazon, JP Morgan, and The Alfred P. Sloan Foundation.
References
Ramzi A Ajjan and Peter J Grant. Cardiovascular disease prevention in patients with type 2 diabetes: The role of oral anti-diabetic agents. Diabetes and Vascular Disease Research, 3(3):147-158, 2006.
Heejung Bang and James M Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 644)9962,-77,, 2005.
Elias Bareinboim and Judea Pearl. Causal inference by surrogate experiments: z-identifiability. In In Proceedings of the 28th Conference Uncertainty in Artificial Intelligence, pages 113-120. AUAI Press, 2012.
Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113277)7345-7352 2016.
Elias Bareinboim, Juan Correa, Duligur lbeling, and Thomas Ieard. On pearls hierarchy and the foundations of causal inference. In Probabilistic and causal inference: the works of judea pearl, pages 507-556. 2022.
Rohit Bhattacharya, Razieh Nabi, and Ilya Shpitser. Semiparametric inference for causal effects in graphical models with hidden variables. Journal of Machine Learning Research, 23:1-76, 2022.
Peter J Bickel, Chris AJ Klaassen, Peter Bickel, Yaacov Ritov, Klaassen, Jon Wellner, and Acov Ritov. Efficient and adaptive estimation for semiparametric models, volume 4. Johns Hopkins University Press Baltimore, 1993.
Philippe Blanchard and Erwin Brüning. Mathematical methods in Physics: Distributions, Hilbert space operators, variational methods, and applications in quantum physics, volume 69. Birkhääseer 2015.
Tianqi Chen and Carlos Guestrin. Xghoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785-794, 2016.
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural param- eters: Double/debiased machine learning. The Econometrics Journal, (1). 2018.
Victor Chernozhukov, Juan Carlos Escanciano, Hidehiko Ichimura, Whitney K Newey, and James M Robins. Locally robust semiparametric estimation. Econometrica, 90(4):1501-1555, 2022.
Richard K Crump, V Joseph Hotz, Guido W Imbens, and Oscar Mitnik. Dealing with limited overlap in estimation of average treatment effects. Biometrika, 96(1): 187-199, 2009.
Iván Díaz, Nicholas Williams, Katherine Hoffman, and Edward J Schenck. Nonparametric causal effects based longitudinal modified treatment policies. Journal of the American Statistical Association, pages 1-16, 2021.
Ele Ferrannini and William C Cushman. Diabetes and hypertension: the bad companions. The Lancet, (3009941):601-6100 2012.
Amanda M Gentzel, Purva Pruthi, and David Jensen. How and why to use experimental data to evaluate methods for observational causal inference. In International Conference on Machine Learning, pages 3660-3671. PMLR, 2021.
Adam N Glynn and Konstantin Kashin. Front-door differeeceeinnddiffeennes estimators, American Journal of Political Science, 61(4))989-1002, 2017.

---

Lennart Hansson, Lars H Lindholm, Tord Ekbom, Bjöm Dahlöf, Jan Lanke, Bengt Scherstén, PO Wester, Thomas Hedner, Ulf Faire, STOP-Hypertensionn Study Group, al. Randomised trial of old and new antihypertensive drugs in elderly patients: cardiovascular mortality and mor- bidity the swedish trial in old patients with hypertension-2 study. The Lancet, 354(9192):7551- 1756, 1999.
Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20( 11:27722, 2011.
Yimin Huang and Marco Valtorta. Identifiability in causal bayesian networks: A sound and complete algorithm. In Proceedings of the national conference on artificial intelligence, volume 21, page 1149. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2006.
Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating causal effects using weighting-base estimators. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020a.
Yonghan Jung, Jin Tian, and Elias Bareinboim. Learning causal effects via weighted empirical risk minimization. Advances in Neural Information Processing Systems, 33, 2020b.
Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating identifiable causal effects on markov equivalence class through double machine learning. In Proceedings of the 38th International Conference on Machine Learning, 2021a.
Yonghan Jung, Jin Tian, and Elias Bareinhoim. Estimating identifiable causal effects through double machine learning. In Proceedings of the 35th AAAI Conference Artificial Intelligence, 2021b.
Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating joint treatment effects by combining multiple experiments. In Proceedings of the 40th International Conference on Machine Learning. 2023. URL https::////
h/ttc::s//proocednngss mlr pwwpsereee00 jung23c. html.
Edward H Kennedy. Optimal doubly robust estimation of heterogeneous causal effects. arXiv preprint arXiv:2004.14497, 2020.
Edward H Kennedy, Sivaraman Balakrishnan, Max GSell, al. Sharp instruments for classifying compliers and generalizing causal effects. Annals of Statistics, 48(4):2008-2030, 2020.
Alan B Krueger and Diane M Whitmore. The effect of attending small class in the early grades on college-test taking and middle school test results: Evidence from project star. The Economic Journal, (11(44)): 1-28, 2001.
R Kumar, DM Kerins, and T Walther. Cardiovascular safety of anti-diabetic drugs. European Heart Journal- Cardiovascllar Pharmacotherapy, 2(1))32-43, 2016.
Robert LaLonde. Evaluating the econometric evaluations of training programs with experimental data. The American economic review, pages 604-620, 1986.
Sanghack Lee, Juan D Correa, and Elias Bareinboim. General identifiability with arbitrary surrogate experiments. In Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 2019.
Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. Advances neural information pro- cessing systems, 30, 2017.
Alexander R Luedtke, Oleg Sofrygin, Mark van der Laan, and Marco Carone. Sequential double robustness in right-censored longitudinal models. arXiv preprint arXiv: 1r05 2017.
v Mises. On the asymptotic distribution of differentiable statistical functions. The annals of
mathematical statistics, 18(3):309-343 1947.
Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669-710, 1995.
Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, 2000. 2nd edition, 2009.
12

---

Judea Pearl and James Robins. Probabilistic evaluation of sequential plans from causal models with hidden variables. Proceedings of the Eleventh conference on Uncertainty in artificial intelligence, pages 444 444-553, 1995.
James Robins. A new approach to causal inference in mortality studies with sustained exposure periodapplication to control of the healthy worker survivor effect. Mathematical modelling, (9-12):1333-1512, 1512, 1986.
James Robins, Lingling Li, Eric Tchetgen, and Aad W van der Vaart. Quadratic semiparametric von mises calculus. Metrika, 69:227-247, 2009.
James M Robins and Andrea Rotnitzky, Semiparametric efficiency in multivariate regression models with missing data. Journal of the American Statistical Association, 90(429):122-123) 1995.
Andrea Rotnitzky, James Robins, and Lucia Babino. On the multiply robust estimation of the mean of the g-functional. arXiv preprint arXiv: 777.005822 2017.
Andrea Rotmitzky, Ezequiel Smucler, and James M Robins. Characterization of parameters with mixed bias property. Biometrika, 108(1))231-233) 2021.
Diane Whitmore Schanzenbach. What have researchers learned from project star? Brookings papers on education policy, (9):205-228, 2006.
James H Stock, Mark W Watson, et al. Introduction econometrics, volume 104. Addison Wesley Boston, 2003.
Jin Tian and Judea Pearl. On the identification of causal effects. Technical Report R-290-1, 2003.
Mark J van der Laan and Susan Gruber. Targeted minimum loss based estimation of causal effects of multiple time point interventions. The international journal of biostatistics, 8(1), 2012.
Aad W Van der Vaart. Asymptotic statistics, volume Cambridge university press, 2000.
Stijn Vansteelandt, Andrea Rotnitzky, and James Robins. Estimation of regression models for the mean of repeated outcomes under nonignorable nonmonotone nonresponse. Biometrika, 94(4): 841-860, 2007.
K. Xia, Y. Pan, and E. Bareinboim. Neural causal models for counterfactual identification and estimation. The 11th International Conference on Learning Representations, Feb 2023.
Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural connection: Expressiveness, learnability, and inference. Advances in Neural Information Processing Systems, 34, 2021.
Junzhe Zhang and Elias Bareinboim. Near-optimal reinforcement learning in dynamic treatment regimes. Advances in Neural Information Processing Systems, 32, 2019.
13

---

Supplement to "Estimating Causal Effects Identifiable from a Combination of Observations and Experiments"
Contents
Introduction
1.1 Preliminaries
1.2 Problem Statement
2 Expressing Causal Effects as Combination of mSBD Adjustments
3
Estimating g-Identifiable Causal Effects
Experiments
4.1 Synthetic Dataset Analysis
4.2 Project STAR Dataset
10
5
Conclusions
10
Further Details
16
A.1
Example 3
16
A.2
Example 4
17
A.3
Example
19
A.3.1
Specification of Nuisances
19
A.3.2
Construction of Estimators
19
A.4
Example 6
20
A.4.1
Specification of Nuisances
20
A.4.2 Construction of Estimators
20
A.5
Details on Regression-based (REG) and Probability Weighting-band based (PW) estimators.
20
A.5.1
Regression-based Estimator
A.5.2
Probabiliy-weiihting based Estimator
22
Proofs
23
B.1
Proof of Lemma
23
B.2
Proof of Lemma
................................
24
B.3
Proof of Lemma
24
B.4
Proof of Lemma 4
25
B.5
Proof of Theorem
26
B.6
Proof of Proposition
26
B.7 Proof of Theorem
35
B.8 Proof of Corollary 2
36
Discussion
38
C.I Relaxation of Discreteness Assumption
38
C.2 Sequential Doubly Robustness: 2 robustness versus m-robustness
39
14

---

D Details of Experiments
39
D.1 Designs of Simulations
40
D.1.1 Example 1
40
D.1.2
Example 2
40
E Project STAR: Estimating Joint Effects of Class Sizes to Academic Outcomes
41
15

---

A Further Details
We restate the notation here. To clarify the relationship between the experimental samples, where randomization applied to C V, and the distribution Pz (V)z), we introduce the notation Pe(z) (V), where (Z) indicates that Z has been randomized. 'he distribution P. (z( (V) is de- rived from the Structural Causal Model (SCM), where the original equation Z < fp0,, Uz) for
Z E Z replaced by a function that assigns a value to = 2 randomly, independent of other endogenous variables. For example, assigning Z 1 and 0 with a probability of 0.5 each.
It should be noted that when considering observational data, P = Pa())- For any sets A, B, and V, the interventional distribution can be represented as P(A do(z), B) Pozz)(A|z z, B) according the definition of the do-operator and the Po(z) distribution. To emphasize that the distribution is induced from randomization and conditioning on Z z, we use (2(A = Pa(z) (A ZZ = z, B). The experimental samples obtained from randomization o (Z) lead to samples Dz(z) that follow Pa(z)(V (V). We denote the subsample of Dz(Z). where Z fixed, as Dz. which follows (V).
A.1 Example 3
We provide detailed illustration of Example 3, demonstrating the application of Lemmas (2,3,4). Input: x = {z², y == {v). Z = {0, 2}. The goal is to identify ([)) from P which contains P and Pa(z) (V). the identification, P(V) and ZV) = Pa(z) (V|z) for 2 E Dz are used.
Line 3-4: Since V {Z,Y}. D an(Y G(Z,Y) {Z,Y}. Let D1 = {2} and D2 (Y).
We now run Line 5-13. We first run D₁ = {Z} and Z = 0. Then,
1. Line 7: The c-component S = V ( X, } includes Dj.
2. Line 8: The e-factor Q[S}] is identified as
Qi Ao 0, 0: 2 = 0, (s 0) P(rk)) P(yx))
3. Line 9: Run Q[Di] SUBID(D,, G(S))).
(a) Line a.(33)) A an( = (W, X, Z). Then, by Lemma 2,
[[A] = Σ Q[S] Aol x, 2),0, 0,0:0 (f x, 0) = P(w)P(clu)) yεБY
(b) Line a.(7-8): Note S = {W, is c-component in G(A) containing D1 {Z}. Then,
Q[A]
Q[S] = Σ
Q[AI)
Exerz Q
x
By Lemma 2,
Σ
QIA] Ao [W000000000000 0:0. 0, (w, 0) = P(w),
x,
Σ
QIA] Ao [[W x3, 0,0,0 x).0,0) = P(rl))
SE6Z
By Lemma 4,
Q(A
Aol(W: X, 2,,,X, 2).0)
Exe62 Q
(w,
Ao(Z, 0, (,,70,,W,,)::0 (W,X):0,0|6 (w,
P())w.x).
16

---

By Lemma 3,
Q[S] = Ao [ 0, 0, (1 0) x Ao|Z.0, 0, ( :: (2 {w, x))
-, (fw,,
((w))
because the order is W G X < Z.
(c) Line a.9: Run Q S Q()))))(((()
(d) Line a.(33): A an ( (s()) = = Then, by Lemma 2,
@[D,] Σ Q[S]
wEW
= A = Aol(W, 2), X;0, (3
Σ
((2,)(2(w,z))
WETW
We now run Line 5-13. first run D2 (Y} and Z = 0. We note that it fails since the sub-procedure subID(D2, S ((S G(S))) fails. Specifically, A = an((2))((s) V St. Therefore, by Line a.5, the procedure fails.
We now run D2 with Z2 {
1. Line 7: The c-component S = V = (W,
2. Line 8: Q = Ao 2 0, O. = (Z). = P.(w))P((clu)PP(((le w). where seqa (W, )) = (z, 2, z)
3. Line 9: We run D = SUBID S Q())) G(S)))
4. Line a.(233): A = an D2)c(s)) = (Y) = D2. Then, by Lemma 2
QD22] Σ
Q[S
(A.1)
M.
Σ
[2), (3:))
(A.2)
w.ZEGw.x
AolY, 0, Z3 = (2), ((x)))
(A.3)
Pz(y).
(A.4)
Let
QDD. = A = Wo Z}, 0,0,0 x)
(A.5)
D = A = Ao 0: seg (fv):0).
(A.6)
By Line 14,
Ply]do(z)) Σ QD]+== Σ 44
(A.7)
zE6x
266y
A.2 Example
We provide a detailed illustration of Example 4, demonstrating the application of Lemmas (2,3,4).
Input: ( y {y), Z (X,, X2). The goal is identify Plyldo x2)) from F. = (Px(x.) Pa(x)) (V)). Specifically, two distributions Pz, (V\X)) and Pa, (V) will be used in the identification task.
Line 3-4: D an (Y (R, W. YJ. De (R), D2 = (W) and D, (Y).
17

---

Line 5-13: Consider D; { | and 2 == (X1}. Note S) = {R} D1 is c-component in (i)) Therefore, Q = Q[S]]. where
QDI = A == Ao 0. X2; = (X1 seq x2) Pa, (r(22),
(A.8)
where sass = (x1).
Line 5-13: We now consider D3 - {Y). Note that Q[Da] is not identifiable from Pa. (1). To witness, consider the c-component $ (W, X2, YJ in X1). Then, the sub- uub-procedure SubID(D,, S3 [[)) G(S))) fails because failure condition in line a.5 is triggered. Specifically, an()) = S.
Therefore, we consider D3 = (Y) with Z3 == x2- Note S == (Y) = D3 a e-component in G(V) Therefore, [D} = Q which given by line 8:
@D2a] Ao[Y, 0, {R. : 23 = (X2), sas = (x (v: (r,w)}) P.. (ylr; w)
(A.9)
(A.10)
Line 5-13: Consider D2 { and Z = X1. Note that Q[2²] is not identifiable from (V2 (2). To witness, consider the c-component $ == [X,,W] W) in G(V) Then, the sub-procedure subID(D2, S3 Q[S2], G(S)) fails because failure condition in line a.5 triggered. Specifically, an(W) ((())) S2.
Therefore, consider D2 {W} with Z = X1.
1. Note S} = (X2, W, Y is a c-component in G(V) containing D2. Then,
Q = Aul(X2, W, Y 0, R; = {X1]. segg = (1,,21, w, v.,rr) (A.11)
= Pei (1)))))))))) (v/ w..
(A.12)
2. Run Q[Da] = SUBID )(S)
3. Line a2(33)) = an(W an(W)a(s)) liss = (W) D2. Then,
@D2 = Σ
Q
(A.13)
M.I2E6y.X,
Σ
Aol(X,, W,Y),,, R; seqbl(1x2, w. (),r)
(A.14)
= Aol(X2, W). X,, R; segs = (21,z))((wrr)
(A.15)
= Σ
Ps, (x2) (w/r,
(A.16)
Also, by Lemma 3,
Q[D,]Q[[s]
(A.17)
A
(A.18)
= Ao[R, 0, Z| = (X1), (eg))(r,22) AolY, (R, (X2), seq fr. w])
(A.19)
Au((R, 13;2 X2), seq (2), G((r) w)) (x2,
(A.20)
Pz, (n. (v)r,
(A.21)
Finally,
Ply 2(y) =
AG 13
(A.22)
18

---

A.3
Example 5
A.3.1 Specification of Nuisances
Recall that the topological order of the variable W <a <a 2 Y. Also, P(yldo(x))
E1e52 Ag where
Aj WoleW, W;0, = E
P(zlx,w)P(U)
(A.23)
WESW
A Ao 0, 0; [2), 0) Pε(1) = Po(z)(Ulz).
(A.24)
That is,
P(yldo(x)) = s(Ag. AB) = Σ A
(A.25)
#E
By leveraging the definition of the nuisance in Def. 2, the nuisance composing A is (uf.or which are defined follow:
xX, W) Ep (1, W], x 1X, W) == (xXXXX)((([))
The nuisance composing Aă is {12,0} which is م3,0 = Ep. [Y,(
A.3.2 Construction of Estimators
We apply the procedure in Def. 3 to construct estimators Á and A for A and Ag- We choose L = 2. We first construct A' for the fixed {2,0} E Dz.x. We note that A} for & E [1,2} is given as follow: For a fixed z, x,
A = ED, (X, - (2, + (2, W)]
(A.26)
and
A- 1/L Σ A
(A.27)
where T ,2² are nuisances estimated using DV Specifically, (X,,(X, W) is obtained by using the XGBoost [Chen and Guestrin, 2016] regression model which regresses 11 onto the {X, W} using Dy 2².e (ar, W) evaluated from D, after fixing a column for X to æ. similar, n² as follow: we first model P P(X(W) by regressing X onto W from the data D Dr using the XGBoost [Chen and Guestrin, 2016]. Then, we evaluate #1,0 W) by plugging the trained P(X
We now construct d for the fixed and y. We first take the subsamples D2 from the experimental samples Dz(2) E D, where D, the sample where Z Then, we compute the following:
- ف = Ep, [1,
(A.28)
Then, following Def. the MR-gID is constructed as follow:
5(11, Σ A
(A.29)
19

---

A.4 Example 6
A.4.1
Specification of Nuisances
Recall that the topological order of the variable is X, 30 X2 <a R Ro W ig Y. Also,
Ag = 2ol W) (x2) R: = (X1}. seq" = (21,21)((wrr)
(A.30)
Σ
Pey (w)), (a/))
(A.31)
dG
A = R, Y), (X,, W (X1, seq = (xI, (2)
(A.32)
Pz, (r) 222) Pay (y/r,w)
(A.33)
Then,
Pyy do(
Σ
(A.34)
The nuisance composing A3 is {1ă00, 22 which are defined as follow:
1 R, X2) = Ep,, (W)) X2). T2,0 R2 X2) (R)) (R
(A.35)
(A.36)
The nuisance composing Ab is {u3,0- x2,03 which are defined as follow:
vi3.0 R, W) Ep,, ni R, Y) W] =Bro [1 ryy ( R, Y) RR W,
(A.37)
(A.38)
= -R)(R)Ppo(x)) (Y) R. W, xz).
(A.39)
and
1 1X,, W) (R(2)((|12,311)))
(A.40)
P((x))(((22) (W]
A.4.2 Construction of Estimators
We apply the procedure in Def. 3 to construct estimators A and Â for AB and A We choose
2. We first construct A for the fixed (w,r,v1}. We note that A (L/L) A? for f E {1,2} where A? is given as follow:
A = E . (R, (1) (2, 12,e(1, X2)]
(A.41)
where Dr, is a subsample of Da(x,) fixing X1 = x1, and T Hå,r are nuisances trained using D2, &D We note that (2,,(R, X2) is constructed by regressing I (W) onto X2). Also, T22 (R, X2) is constructed by regressing R onto X2.
We now construct Á (1/L) Al3 for f (1,2) where A! is given follow:
A 3 = DD (1,. (1, (R,Y) - (R, W))] + E (R [)]] (A.42)
where Dr, is subsample of Do(x,) fixing Xi F1. and Dz, is subsample of Dr, fixing X₂ 22. Th 12,5 are nuisances trained using D,, DD,,, and Dz, DDz.e
A.5 Details on Regression- (REG) and Probability Weighting-based (PW) estimators.
In this section, we provide details two alternative g-ID estimators used in Sec. 4: 7 = f(LAbm8) } ("regression-basee based estimators') where 4 denotes the regression-based estima-
20

---

tor for the g-mSBD operator, and T = f((AAtpm (==))) ('probability weighting-basee estimators)
where A denotes the probability weighting- bassed estimator for the g-mSBD operator.
A.5.1 Regression Estimator
The regression-basee g-mSBD estimator is defined as follows:
Definition A.1 (Regression-based g-mSBD Estimator). Let Docz.) for Z; E & denote the exper- imental samples from randomizing the variable Z, Let Dz, for E Dz, denote the subsamples of Do(z.) fixing Ro Zz R adjustment A W, R; Zo = [ seq = (z, ((w)cr) is given as follows:
ro 2; and Z; Zi. A regression-based estimator Å for the g-mSBD
1. Randomly partition D2, into ( i.e. De. U D2nee 7Z, & and 2, Zz
For each fold & [L]. let u denote learned jo+ using Danm. Dauti for i = m, ... ,
Define ph 1++1 = He (W re, 1:1-1
3. Estimate Á A (1nt "e Je(m- ().e-))))))))))))))))))))))))))) = (1/D) A (1rt De(m-1)) ((p(t")e(m-))
where
A =
= ED [
(A.43)
The егrоr of the regression-based estimator is given as follows:
Proposition A.1 (Error Analysis of the regression-based g-mSBD estimator). Suppose 1| Moll PocZ, Op,c, (1). Then,
1 T t=1
A Ao OPiZz 01 +
O OPocy, nεll).
(A.44)
Proof of Proposition A.I. We note that
Ao EpID [pōlz], ro]
(A.45)
by the analysis Lemma S.2. Therefore, by Lemma S.7,
A
Ag = EU,, (Pol
[10]]
(A.46)
+ ED, PPocs,)eee
[1] [3]
(A.47)
+ E P.,,,
1i8. - 12 ro. 21
(A.48)
By the central limit theorem.
Eq. (A.46) = O (n 1/2
(A.49)
where n1,8 = DDz.el- By [Kennedy al., 2020, Lemma 2] and the given assumption that ||ا vell Pa(a)) OPr(2 (1),
Eq. (A.47) Or.,, (1/n) (1/1 1/33. Finally, by applying Cauchy-Schward inequality,
(A.50)
Eq. (A.48) Orezz (11 - 1Bl1).
(A.51)
21

---

Finally,
1
a Ag
(A Ao)
(A.52)
L
t=1
Oooiz (n 1/211))))++ + +
Paiz, P. ( - -Poll)
(A.53)
1 Σ OP.ii On 1/211+
OpOIz -1ô1)-
(A.54)
1=1
A.5.2 Probability-weightinn based Estimator
In this section, we define and analyze the probability weighting-based g-mSBD estimator. The probablity-weiightin--mee estimator defined as follows:
Definition A.2
g-mSBD Estimator). Let (z.).) for Z. E Z denote the experimental samples from randomizing the variable Z, Let Dz, for 2, E Dz. denote the subsamples of Dz(,.) fixing Ro' ZZ = ro (2 and Z; Z₁. probability weighting-based estimator ÁP" for the g-mSBD adjustment Ao W, C, R: = (Z [Z seq = (z.) ( [w|c,, (c, r) is given as follows:
1. Randomly partition Dz, into {D2, JEELL i... Dr, Uf VZ, E E and z, Dz.
For each fold & E [L], let my denote learned TÓ using Dz, \ Den. for = m 1, ,, 1. Let m- Tz-
3. Estimate Anw = Ánw (1 hje(m 1).(m)))) ().e(())) = (L/L) (1/L) A!T (1 (em()e where
APT = APT ((T/)se(m-1)) = D. 1 ((C)
(A.55)
Lemma S.1 (Representation of the g-mSBD operator using Probability Weighting). The mSBD adjustment Ao in Def. can be represented as
Ao = EPs
To Tarn-wwiee "ww'e (C(cccWWCC)C))) [C))) ro]
(A.56)
Proof of Lemma S.1. It suffices show that, for k: 1,.... 1. A k-
E E 1)
k+1 (2441.0)) ro] EProne 1oip
zR. rol
(A.57)
If this holds, Lemma S.1 can be shown as follows:
Ao
Ep
12, ro]
Epsing)
T 22, ro]
Eroiom)
[To Mo mm+1
2m ro] (CC)
Ep,
T [w)e W
Im ro]
Eq. (A.57) holds as follows:
Ep Epottss)
T 2 12x+1,
Ep +1) 10
#+1 2x+1,,
Er
b p- |z,, ro]
This completes the proof.
22

---

Equipped with Lemma S.1, we analyze the enror of the grobablity-weighiingbbaa estimator as follow:
Proposition A.2 (Error Analysis of the probability weighting-based g-mSBD estimator). Sup- pose [ - mbr0) Po(2z) = (22)2)) (1). Then,
\pw Ao = ( +
Ore(am) (I|== - T0 II)
(A.58)
L 1=1
Proof of Proposition A.2. By Lemma S.7 and Assumption
1,
A
Ao
E "Dam e- (o(22m)lmm.r
#0 wle (WC))
(A.59)
E D if Polzm 1o0m
[ To I W
(A.60)
To Lww (W(]) Ep Po 1m 1o (
(A.61)
By the central limit theorem,
1/2 Eq. (A.59) = OPoczm) (m,
(A.62)
By [Kennedy et al., 2020, Lemma 2] and the given assumption,
Eq. (A.60) = Or.czm) (mm,)
1/2
(A.63)
Finally, by applying Cauchy-Schwarz inequality,
Eq. (A.61) = OPocn. (11 T0 HI).
(A.64)
This completes the proof.
B Proofs
B.1 Proof of Lemma
Lemma 1 (e-component Identification [Jung et al., 2021b]). Let S denote c-component in
G: = Zz) for some Z; E Z. Let R = pa(S)a, \S. Let () be ordered as (Ra, Si... Sm, by La- Let A C denote a set satisfying A an(A)).(s) Let C = AS) Let Zn = (Z,) and seq(Z) be sequence of n, repeating m times. Then, the c-factor [AA] is g-identifiable as follows:
Q A|| = Ao[S, C. R; = (2,), seq (a,
Σ JVVEE
II
Pz. (r. 8 10-1_--11
p (2))
(2)
Proof of Lemia 1. Let Co := al., 2021b, Lemma 1].
n C. Let Ci = -C,C We first note that, by (Jung
Q = Σ II
Pa (u (1. (1)))22))
(B.1)
C,Ec, J:VY EAUC,
Therefore, it suffices to show that
Eq. (B.1) Ao SS R:Z, = seq),,e
(B.2)
23

---

It holds as follows:
Ao R: Zo == se])
Σ
II
Pa. (1gls 1,,
(25))
(B.3)
cEEa VjES
E
Σ
II
(1/ls/ (2)))
(B.4)
co c,6ca ViES
E
II
.F-1 (2)) (z))
(B.5)
Co VGES(C,
= Σ
II
Pa (usls F (2))
(B.6)
CpEEco VGEACC
= Eq. (B.1).
(B.7)
We note that the third equation holds since the all Pz. (tyls) s -1)1)))))))))))) will be marginalized
if V3 E Ct. The fourth equation holds since S = A U Co W C₁, which implies that S)C, A U A.C.
B.2 Proof of Lemma 2
Lemma 2 (Marginalization) Let Aol W. C, C, seql(w)cc,r) denote the g-mSBD operator in
Def. 1. Let Wo S. C. C. Let W mar C (Wo, 0. C} denote the vector formed by the following proce-
dure: Starting from W mar 0. for j = m,
1. W mar W mar U [W if (1) W {Wo, C}
and (2) 3k ii.... m) such that R. RR-1
W k+1mm
C W mar, and Ze
Z
and =... Zj. Let W' == W Wman R' =
and C' = (W, CJ mar Let
z' C LLdo denote the collection of Z₁ corresponding to the variable in W', and seq' the correspond- ing sequence. Then,
Σ WoE
C, = R',''''' seq
(3)
Proof of Lemma 2. Let W mar = (Wo, CJ W
mar- We note that
Σ Ww
Aul AolW, C, R: R; seg((w,..r
(B.8)
Σ
II
Pa, (wy) (
(B.9)
Wg.coEGww Wo. Co j:WEEW
Σ Σ
II
Pa [w: W 1- F
(z P2, (Wi,
W6
W
11
Ps,
(w))
w
f-1
Wc. SWe Waze E Waw 1=
Em1
(B.10)
j-1
Pz (LWg.... wk) Wman 1/ 1 zz)
Pay (wel -f
(B.11)
d'eee
Σ
II
Pe (wy (wy -11-)
(B.12)
d'ERG W.WWW.... W
= Ao C² R'; seq
(B.13)
B.3 Proof of Lemma 3
Lemma 3 (Multiplication). Let Ab = AolW 0, R. Z, seq'((,,..))
I Pg (w., w FF (z) for {1,2} where seq' = (z)) Let W = Let
24

---

R = (R. U (.) W. Let (W, R) be ordered by <g. Let = Z1 U Uz. Assume the following: (1)
Wi n W2 = 1): and (2) wW; E W, W, E W; such that (W ,R' = -W: k-1
R
Let seq = (zj),:W,ew where 2 z for all j. Then,
Ab x A3 = A, 0, R, seq] (w,r)
II JW,
Pe, (u)(w//w'-1,r 12))
(4)
Proof of Lemma 3. Ao[ 0, R; 2, seq[(w,
(B.14)
II
Paj (wyll// r (2))
(B.15)
J.,
(u) W . 12))
II
Pay(w2, W
-1-1_ , 1
x3))
WWW.EEEE W W.WISW W.W,
2: E W. W2,=W,
(B.16)
m III j=1
1
Paj (ay 22)
Pap (w2,)
W2 -1 -12- 123))
(B.17)
= Aj x Ag.
(B.18)
B.4 Proof of Lemma 4
Lemma 4 (Division). Let Ab
= Aol AolW,, R,; sw,"
I1 Paj Pay(ur,, (wij/w? .I. (2)) for E {1,2} where seq' = (x)) Let W = W2
Let R = (R1 U W2) n (re(W;)) Assume the following: (1) W2 C W1; and (2)
VW, E W, 3. E Wi such that (W"-1 R )² ( k-1 R k-1,
) Zi, Zi and Zi = Zj.
Then,
A Ao W. 0. R: 2,, seq" (w,r)
II (us(wsl
15 1 1zz). 1zz).
(5)
SW,
Proof of Lemma 4.
I1 Pgj (W1,) w . (z,
A
(B.19)
I1 Pa (W2 W. F/ - 122))
II kWW.WW
(wi, /w, k-1 r, k- 1 (22
(B.20)
Paj (1), k-1 n nw, wT k- (,, (w, x1 W.W.WT
II
(B.21)
We note that W n W = W for some W E W s.t. W, = Wi, Also
=
W₂ U R,. Therefore, UwW ( W. R k-1 Therefore,
} R. = (R, U n pre((GG)
Ab/A3 =
II
Pa WT nw, k- /w, ( b- (2
(B.22)
W.W.EWAW.
=II Pa (we (1,,-11
(B.23)
&.W.W
= A,/WW 0 R; (w,
(B.24)
25

---

B.5 Proof of Theorem
Theorem 1 (Expression of g-Identifible Causal Effects). Algo. returns any g-identifiable causal effects as a function of a set {Aa} of g-mSBD adjustment operators in the form
P(y]do(x)) = [(((A))=))
where the function f() applies marginalization, multiplication, or division over g-mSBD operators in {Ay } as specified by Algo. I.
Proof of Theorem 1. Throughout the proof, we refer the algorithm developed in [Lee et al., 2019, Algo. 1] as the "standard gID" algorithm, in comparison to our gID) algorithm presented in Algo. 1. is established that the standard gID algorithm is sound, as stated in [Lee et al., 2019, Theorem 2]. This means that if the algorithm returns an identification expression. it must be correct. Furthermore, the standard glD algorithm is proven to be complete [Lee et al., 2019, Theorem 3]. In other words, the causal effect Ply do(x) x)) is identifiable from TP and the causal graph G if and only if the standard glD algorithm does not return FAIL.
In our proof, we will show the soundness and completeness of Algo. based on the foundation provided by the standard gID algorithm.
Algo. is sound If Algo. returns expression (((A5)/)) then it holds that f({4)) ((y) The soundness of Algo. 1 is derived from the soundness of Tian's c-factor operation, as demonstrated in (Tian and Pearl, 2003, Lemmas (3,4) and Lemma 1.
We will now show that Algo. is complete. Suppose there exists input (x, y, v,Z,E P, G) for which the standard gID algorithm does not return FAIL while Algo. does return FAIL. This implics the existence of D, such that [[D,] is not identifiable from all Q[S]] where D, c-component in GDD), and Sj is the c-component in G(V that contains Dy. This observation is a consequence of the soundness and completeness of the SUBID procedure, as established in [Huang and Valtorta, 2006, Theorem 1].
should be noted that Q[D,] is not identifiable from SJ for all {t: Z, E Z) only when there exists a c-component T in G(S)) that serves as an ancestral set of D, and includes Dy. However, in such a scenario, the standard gID algorithm fails due to lines 12 and 13 of the algorithm. This contradicts the initial assumption that the standard gID algorithm does not return FAIL Consequently, Algo. returns FAIL whenever the standard gID algorithm does so. The completeness of the standard gID algorithm implies that Algo. is complete in the g-identification task.
The fact that f()) is a function involving marginalization, multiplications, and divisions of g-mSBI> (generalized modified single back-door) operators is consequence of applying Lemmas (2, 3, 4) within the algorithm. These lemmas establish the properties and operations of the g-mSBD operators, which are then utilized in the construction of f-)) in Algo. 1.
B.6 Proof of Proposition
We first restate the g-mSBD adjustment, its nuisances, and the g-mSBD estimator here:
Definition 1 (generalized-mSBBD adjustment (g-mSBD)). Let (W, R) be disjoint pair in V topologically ordered as (W. R) = (Ro W ... R, Rm) by <G. where R, can be
empty. Let w = [W 1i and R = R for Vi E [m]] Let C C W. Let Zo C Z be some set such that ZZ E Zo. Wnzz Z = 0. Let seq(Zo) denote a sequence (Zi, 2m) where z; denotes some realization of Z, Zo (same z, could appear multiple times in the sequence). Then, the g-mSBD adjustment is expressed as an operator W, C, R; Zo, seq, (w),c, r) defined by
Ao (, C, Zo seq (w) = Σ II
Pz. (w. W F (1)))
(1)
cE:C d.W.W
26

---

Definition 2 (Nuisances for g-mSBD). Nuisances for 8-mSBD Au in Eq. (1) are {uo²! mmm-
defined as follows. Let 140 10+1 u = w (W) (W.C). For i = m 1,
1b "W" 13 EPo(24)))
44 1 '11
W R 1:
rit1;
ro, Zi+1
To( ( R Pa(za)
W. R "RRRR"LZ Z0)
1r (R.)
Po(z..) R (441, ro) Po(Z. (R, W. W.,WW. R la-1 ro)
Definition 3 (DR-g-mSBD Estimators). Let Da(z,) for Z, E denote the experimental sam- ples from randomizing the variable Z, Let Dz, for z, E Dz denote the subsamples of Do(z.) fixing Ro Z; ro 2; and Z;
The DR-g-mSBD estimator Å for the g-mSBD adjustment A, C, R: Zo = | ,,,,,, seq = (z.) (wlc,, (,, r') defined as follows:
1. Randomly partition Dz, into {Dx, i.e., Da: Uf=1 AZ, Z and 2, Zz
2. For each fold é E [L], let p #+1 denote learned Mo i+1 using D2,,1 D
2. 1.e for i = m. ... 2: and
गे learned T for i 1, 1. Define f ii+1 = 14 i+1 (W" r,, R
l: and गे. = 11 m/.
3. Estimate Å = A(Luet =.) = (1/L) Et Aettme m jemmmmm-1) 1n) where
m-1
A = Aeltuttt bebemm-1)
- m/tt1) -
+ED., [p8] (9)
where E. DDy.
| is an empirical average over samples D Dzy.t-
We analyze the bias of the g-mSBD estimator using the following results: Lemma S.2 (Representation of g-mSBD). The g-mSBD adjustment Ao in Def. I can be repre- sented as
m-
Ao
Ep Erazznn
1 Po Hzi+1,ro]] BrazN [pblz,,
(B.25)
where fit (W, R
1:i-1
rí, R 1 and = IT at as defined in Def. 3.
Proof of Lemma S.2. Throughout the proof, we will use w' as some realization of (\C. Recall that 1 w (ooe(w")))) ('') when w'|c'' w''c and zero otherwise. We first recall that
Ag Σ II (wie(""'cc)Paaa
,F 122),
(B.26)
W'EG sWWWW
by the definition of the experimental distribution Po(z.).
For all = 1,... m----
To W. R
W
R
10 (W', R 1:
2441, ro]
(B.27)
Fo ( (rizz
Mo (W
i+1
R 1si
w'
2241, - 10 (w", (R")))) 2,41,
ro
(B.28)
Epaez 0,
To
1:t (1b
+1
(
R 1:
140
4+1
12
(B.29)
(B.30)
where the equation = holds by the total law of expectation, and 2 holds by the definition of p. xi+1
27

---

It suffices to show that
Ho
F [21 ru
= Ao
Σ lwia(w')
(w') (') II
Po Pazz (w)) (u) W (w/
, zz).
w'E
j=1
(B.31)
To prove the equation, we show that, for all & m, m
.2
Iwlo (c) II (ozz)) w w
lik-1 kij-1 .k: -1
Mo
R
R
w
ro. 2z)
w kim
Wrim
(B.32)
This equation holds when I m, because
PO
W
R 1am
[wicc("(c')) W
R lim-1
ro, 2m
(B.33)
1wie(w" (w.
W Wn
R
1.m
No. Zm)
(B.34)
m E@wm
For = m - 1,
(W" m- 1
40
(B.35)
= =Epoczm-il
A0 W
1m- R...
1:m-2 w R 1.mm-2
1m- Zm 1, ro
(B.36)
172
Twle(W'(c) (c))
II
(w)
F pm-1zj-1 m -1 w
1zj-1
W
R
lom
Zz. ro)
f=m-1
mm -im
(B.37)
Based this observation, we make the following induction hypothesis: .2.... m), the following holds:
Suppose, for fixed k
k+1 (w", R 1sk induction
Σ
Iwic(w'ld) II
(o' W R.l. lsk w &tliill phtl:j11 2zz ro).
Mo
W
EB
1im
j=k+1
(B.38)
Then, the induction hypothesis holds for & - as follows:
Po (W" k-1
.R 1:k-1)
(B.39)
Broizess)
[ (""
R 1:k-1
rk
W
R 1:k-1
r0
(B.40)
Σ
Imic(w'loo) II Pa Pa(z))
R 1:k
phoj1
2j, Po12.)
(w/)
W
R 11:-1
2k ro)
w kim EGpolim
$=
(B.41)
Iwie(w'lco) II m
Puzzy ,R lsk-
w
boj-1
- Zzj ro).
(B.42)
w
E
pplim
Also, we already checked that the induction hypothesis holds for m. Therefore, the hypothesis holds for all &s = 2, m.
1:k
Σ
Iwle(w'(c)
II
(u W
R 1:k-1 w
kis- -k:s
22. ro).
EG
EGpricm
(B.43)
28

---

Then.
(6(W1,R))
Σ
l (c)) (ut R1, w ,2:i511 2j. ro).
wa
(B.44)
=
Σ
Twle(w'(c) (')
II
Pozz)(uflW 2 _,--122 2z))
(B.45)
w pim
EB
j
Then,
Epoza [M0 W1, 11),,
(B.46)
1 Iwlo(w'k Σ
(w'www) (c')
II
Po Pozz, 2 _r-r111 , Po (u/]2,,)))))
(B.47)
w cOw
f=2
Σ
(W((("((c))
II
Po Po(z, W w ,,0-1,2)) ,2))
(B.48)
w" E6wm
_=1
Ag-
(B.49)
Lemma S.3 (Bias Analysis of g-mSBD Estimators (1)). Let A be the quantity defined
A
Ep Po(z)))) [ nt H2s4,,
EPI(Z [13]22,
(B.50)
For i -,....,m m,,
Bpo(z.e))
( (1bb i+2 1x |1,+1, ru] + Erorz,) [T [ 1 |2 ro]
(B.51)
(PP(zzer) [ (
2+1 H 1z14,,1, + EPSiZA ( 1--1 n Hz,,
Proof of Lemma S.3. We first rewrite Eq, (B.51) as follows:
Eq. (B.51) (p.izan)
- liitz 11 zi+1. rol
(B.52)
+ EP,
i [ p H Hz,,
(B.53)
+ Errrz
( ( - n² Hz,, ro]
(B.54)
Also,
Eq. (B.53)
W
R
1:1-1
( R R -
++1 (W'
R 1: H2 ro
W. R 1:i-1
= =Epaza1)
R
P
Po(z.)
25 ro)
(1
W
R
1it-1 1----/tt"" .R. l:f-1
1122441,
Po(z.,) (W',
2i+1,
Pa(z.) (W'
R
(2,
1r. (R,)
=EPoiZA11)
110t1 - mbt1
21
Pa(zur) W
R
ro) Po (Z(Z.) (R, W
R 1:i-1
ru. Zit1)
1( R
Jar' ( R R') (W", ,R.)
i+1 (W'; R
12
130
(B.55)
29

---

Therefore,
Eq. (B.52) + Eq. (B.53)
Eq. (B.52) + Eq. (B.55)
(1)
T [1bt x²
++ |zit1, To)
+ Eq. (B.55)
(PP(Z.41)
[ (1btt ++1 - 11" 1+11 Hzit1, Fo
+ Eq. (B.55)
11
( {ubtt 11 - (/+11iz+1, No)
+ E
[ m mo(uut1
- 11+11 |zit1, Fo
-EPoia4))
1-1 (16 1-1 (
1 HT² (Z
(B.56)
Finally,
Eq. (B.51) Eq. (B.56) + Eq. (B.54).
Lemma S.4 (Bias Analysis of g-mSBD Estimators (2)). Let A be the quantity defined as
A =
Ep Pa(z
( p H2/41, ro] + EP ro]
(B.57)
Then, for h = 3,...,,,, 1,
Ā - Ao =
Σ
E R (
ा [ 110t1 1 ( ToHlrrt1.o) Errcz ( (0-1(10t
|2x, ro]
k-2
Ep. (2
| (//ht22 1 (/t12//241,xx))) Broiz [+2- 118 z1. z1.
Proof of Lemma S.4. The equation holds for = m - -1. It can shown as follows:
A Ao
Br Broiz
[T (itt2 - - 1z:41, + E r,;z, [i - -A
Ep (Z (Zm41)
m (16 (mgmit -==+1 |2m+1, ro]
E Poczmm) iT [ ( 1 2m, ro]
(B.58)
m
E B
[ fittt (1:11) + Brozz)) 112 r
(B.59)
Then,
Eq. (B.58)
Lemma 5.3
Epe
1 (110 (1 m Mm " To 12m ro] + E Payzan
= {10
11"
}
1m Nu]
(B.60)
Therefore,
A - Ao
m-2
Eq. (B.60) +
Σ
Ep B.izi
(1//t22 - n² + 12411 ro] + Epozz [p z1 (B.61)
30

---

For any fixed &t + E {m - -,, ,4,, suppose the following holds:
A - Ao
Σ
Brai 1 T (P6 --=" [x="-=11__ 1
11,
(B.62)
r=k+1
Epoet 1)
T (1
k+2 1
(B.63)
k-1
Σ Epo(z441) 1ii 10*01
|2
(B.64)
+ -PPo(z))
[ir 22 ro]
(B.65)
We note that this holds when k = -mmmm2 2, as shown in Eq. (B.61). We will now show that it will hold for k, too. First,
Eq. (B.63) + Eq. (B.64)
(B.66)
E Breze
(Tr lngtt k+2 1 [2
Σ
E P
[ ( - p wt+1 1 ro]
1=1
[Tt (1 k+ 122+11224,,,
E E..,)
[ [ n 2x. ro]
k-2
Ero(2.11)
[T [ 1111,1212,,00
Lemma S.3
EPo [TT- k+1 |=* 20]24t1,, + Erocz [T k- (1it1 Har, ro]
Er
k-2
Er.z, 11) [T(1 -1 12it1,
(B.67)
Therefore,
A Ao Eq. (B.62) + Eq. (B.63) Bq. (B.64) + Eq. (B.65)
Eq. (B.62) + Eq. (B.67) + Eq. (B.65)
m
Σ
EPa [ 1 r1 - n HT - 2r+1, + E b ( k+t 11 M
a00 22 ro
r=k+1
[ (iitt] -1 ro]
k-2
+ Ep EPoiz. 1=1
J (i/h22 - 1 H1141,,00
+ Eq. (B.65)
Therefore, the equation holds for k too. This completes the proof.
Lemma S.5 (Bias Analysis of g-BSDD Estimators (3)).
Epsc, [ {10) 12² 12)) ro] EEaz, [A² A Z1, ro EP,e, [1ug 1²) mbf 1z2 rol
31

---

Proof of Lemma S.5. Note that
EPo(Z) [12
[W1, Ro) - 1 (1. R2) ro]
Pa(z,) (W1, Ro R1, ro)
=BP,(2)
{1r (W1, Ro) - 16 Wi, Ro)} 22, ro
Po(z)) W1, Ro X2, ro)
Po(z,) [., Ro 2,, ro)
n (R.)
{ (,W,,RR) - R1)) 22,
Pa Po(zz) (W1, Ro Z2, ro) Po(Za)
R1 Z2, ro
Ao. (1) (W1, 1 W 1. X1) Z2,
Therefore,
Ep Braizi
a (16 1r |22,
/22 ro] E..,z 112 1 21 rol
-.Poiz)) (1) - /²}122 ro] -EP EPo(Z) [1t 15 z1, ro]
EPoizz)
[T (15 p²) p²}/Zz, |22, ro] E..,z (Tb μ Jz2, ro]
Ep.(z))
[Qg 2² Hx² 12)
Lemma S.6 (Bias Analysis g-mSBD Estimators). Let A be the quantity defined as
m
A BoIz [1 n Het1l1z4,,rr) + R. P.(Z) [0]
(B.68)
where n², ne are urbitrary nuisances for true nuisances Tà and pà defined in Def. 2. Let Ao denote the g-mSBD in Def. 1. Then,
- Ao = Σ Opoznn1 (1 -
- Tolllau*+1 - -16 -1tt11)
(B.69)
Proof of Lemma S.6. By Lemmas (5.3,8.4,..55..
We also use the following results which are used by Kennedy et al. [2020].
Lemma S.7 (Decomposition). Let D P denote finite sample set following a distribution P. Let I(V:n) denote an arbitrary random function taking yj as a nuisance. For any y,, qoo
Ep [&<V: m)
Ep [N[V: mm))
(B.70)
ED-P [VV + Ep- (h(V:n) n(V: + Ep (p(hVV:) -(V: no)]
(B.71)
Proof of Lemma S.7.
ED [h(V:n)] Ep (V: no)]
(B.72)
ED_P P [h(V; mm) + Ep [k(V: :1) - k(V:n)))
(B.73)
Ep-p [h(V; m)) + Ep-p (<(V:)) A(V:no)) + Ep (b(V:)) - A(V:
(B.74)
Lemma S.8 (Continuous Mapping Theorem for (P)). Let X X denote random sequence defined on metric space S. Suppose function g S S' (where S' is another metric space) is bounded and continuous almost everywhere. Then,
Xn
L2(P) +
X = g(Xn) 4 9(X).
Lx(P)
(B.75)
32

---

Proof of Lemma S.8. We first note that Xn (2( X implies X2 4 X. Then, by continuous
mapping theorem, ((X)) & 9(X). Then.
lim 1l((X,) - -xx))) lim / 1o(X))) d(P) / lim lo(Xn) - 2(x) d = 0, n-
n-+00
(B.76)
where the equation holds by the dominated convergence theorem, which is applicable since g(Xn), (XX) are bounded functions (from the given condition) and Xn 4 X.
Proposition 1 (Asymptotic Analysis of g-mSBD Estimators). Assume that the muisance estimates
Mg and are Læ-consistent; i.e., l| -Mo 2--m |I (Z),))))) (1), l1 p1o" Pl Opo(zi41) (1) and || Toll Pall = (1) (1) = ,,,,,, ,m 1, and | - p PM(Z,) =
(poiz)) (1). Let n, = DDz for f1, m). Then,
m
L m
A Ao
+
O OPOZZR (llpk 140 1+1 ||| =fll)
(10)
where R₃ is a random variable such that n. R3 converges distribution to a mean-zero normal random variable.
Proof of Proposition 1. We start the proof by noting that
A Ao E Ao}
(B.77)
where d defined in Def. This proof focuses on analyzing A Ag-
Also, we recall that De,, for all z; follows the distribution Pa(z,) (Viro, zi). Throughout the proof, we will denote
P) (V) = Pa(tzr (V/1,,,,
(B.78)
Then, each As Ao in Eq. (B.77) is given as follow:
A Ao
E Da Pa(Z41)) 44t100 [Toiet2 1i [ + ET.,, -P
[6]]]
(B.79)
Σ
ET 11.0 10 11:50
[xt(tt(22 11 11111 mb(16b2 [bt11]]
==1
+ E E Paczi))
[it - Kin]
(B.80)
m-1
+
Ep
zir ue
i+1 Fo 222
16 1+1 1 + Ep.,Z 1n3 p pi rol
(B.81)
Define
R4 = ED, Pa
[60]]
(B.82)
and for i = 1,. m
R = ET E 1 100
Foll i42 .++
-D [1]
(B.83)
33

---

By the central limit theorem, we note that R for = 1,,,,, m is random variable such that 1/2 ii Ri, converges in distribution to a mean-zero normal random variable. Therefore,
Eq. (B.79)
R
(B.84)
also behaves the same.
We now analyze the second term. Define
R2 = ED,,, - Po [1 - -
(B.85)
and for i = 1,2, .., 1,
14+1. E Po(2,41)1
[rliih ++1 -
(B.86)
By [Kennedy et al., 2020, Lemma 2] and the continuous mapping theorem in Lemma S.8,
|1, | A5ll
R2. OPoiz.)
(B.87)
V
and for i = 1....,,... m 1,
||| 1
11 1+1
To (
PD 3+1
H|l
R RoLLE Opeiz.
(B.88)
14++1.
Under the given assumption, for 1, .., m.
Rh = 0p.122) (1);
(B.89)
and
m Σ
Eq. (B.80)
Ppiz (1). 1=1
(B.90)
Define
RAg R. + R 1/2 Then, R5 is also random variable such that nie R4 converges in distribution to a mean-zero normal random variable, by Slutsky's 's theorem.
(B.91)
We now analyze the third term. By Lemma S.6, the third term can be analyzed follow:
Eq. (B.81)
Σ O (1ltt1
n + ||| 7
(B.92)
Therefore,
Ag Ao Eq. (B.79) + Eq. (B.80) Eq. (B.81)
(B.93)
m
m-1
Rit +
Σ O (1lu [ - 10 |||| m =/ll)
(B.94)
34

---

Define R2 = (1/L) Ric. We note that R2 is a random variable such that n1/² R. converges distribution to a mean-zero normal random variable. Then,
1
Á - Ao
Σ 8=1
(A - Ao
(B.95)
L
Σ
R₂ +
Opezan (11 molllimt 16t111)
(B.96)
==1
1=1
i=1
B.7 Proof of Theorem 2
We first restate the definition of the MR-gID estimator, the theorem, and corresponding assump- tions.
Definition 4 (MR-gID Estimator).
The MR- estimator & for the identification expression of in Theorem is given as follows: For each A composing f((AA)R=1) let A A (frgee Th. (helse(me-1),eetta denote the DR-g-mSBD estimator with nuisance estimates { 3+1 Thee) for the true nuisances {uk,0 7.,0 Then,
the causal effect to f(AA) (5k=))
$ = s((A+)
(11)
Assumption (Analysis of MR-gID). The identification function f([A*) in Thm. I and each i+1
nuisances (uke ehee for A satisfy the following properties:
1. Twice differentiabilltyy ((1A))=))) is twice continuously Frechet differentiable w.E.. {Ak Jk=1 w.r.t {A* [A])== k==
Boundedness: Vk E [K] and VZ; E Z. A f( AA 1B] OP (A* (A4- -Ag).
Ly-Consistency: l1 - 1k,00 Parzt = P (1), 11 - 1 Ploaa
= op OPOcRt (1).
| Ti.oll = (1), and || pll Po(z)) OP.c2 (1).
Theorem 2 (Asymptotic Analysis of MR-gID). Suppose Assumption 2 holds. Let nhg.i (Dzt for Z E E and z Dzy. Let & denote the MR-gID estimator in Def. 4 for the causal effect vo s((A() in Theorem I. Then, the error of à is given as
m
m
- v0
Op
(N
-1/2 )
+
1 Σ
Σ
Op
++1 M2.0 IlI Tklll))
k=1 1=1
==
t=1
(12)
Proof of Theorem 2. We first define the following notation. For map 9(x), will use (ag(z))l1) [h] = lom;++(lg(z) + th) ((zo))/t. We first note that by the definition of Fréchet Derivative-bel Taylor expansion [Blanchard and Brüning, 2015, Def. 34.1] and the given assump- tion ('Twice differentiability), ) the error e - vp can be represented as follow:
Vi
A f((45) -A + -48),
(B.97)
where, by Prop. 1,
m
m
A
A
Σ
Op
Openg
(1i..
-1/2
1/2
+
1
Op Opoez Po
Clliel lllll =koll). (B.98)
T,
1=1
i=1
1=1
35

---

Therefore, by the big O in probability calculus [Van der Vaart, 2000, Chap. 2},
n
L mi
o(A4 - -
Σ
Op
(n
1/2)
+
OF
(llkke
M2.0 I1 ||| =k,ll))
14-1
L
Em
(B.99)
m
L m -1
(
+
L
op
P 1
1+1 I11 (llek,
(B.100)
By applying the given assumption ("Bonndedness' ) we have the following:
m
V At [(AA)))))-
A
Σ
Op
m02 (n
1/2)
Opoctt..) (llutte m 111 Tkull).
(B.101)
Therefore,
Vu
N (((4)) 1|
-AA +OAA -A8)
(B.102)
1=1
m
m
(nk.8
1/2
Op 12 (lluate
1k.0 IlI
rooll)
L
(B.103)
m
k
OP oiztt
(1
-1/2
1/2
OP Paizt (1l 1+1 p
=
(B.104)
m mm=1
Op
(1
1/2
Op o122) (llmutt)
-Kk00 IlI Tklll).
0(2)))
T,
(B.105)
B.8 Proof of Corollary 2
Corollary (Multiply Robustness (Corollary of Thm. 2)). Suppose (1) Assumption 2 holds; (2) Either The = Tg.0 or ph, v for = + ,m for all i, E, k; and (3) all nuisances 1 J1, are bounded by some constant. Then, the MRD & (Def. 4) consistent to voo
( Hk,
Proof of Corollary 2. We first note that f a continuous function under the twice differentiability condition Assumption 2. Suppose each A a consistent estimator of AB under given conditions. Then, by the continuous mapping theorem, f{{Á*) Jk=1) is consistent to Px(y) = [((A)) )) Therefore, it suffices to show that each A is consistent estimator of AK_
36

---

We first recall that A = (1/1) A by Def. 4. By applying Lemma S.7, A rewritten as follow:
A can be
A Au
D
1/11
ozzt
1ll 1.00
F 1xx00
ET
sf. Do,t
- Pa
[k
(B.106)
==1
D
Pa Paret
11 141.00
Fhellihte 1422 -M mb,e) -
1t2 ++1
1k.0
1=1
D
[ikee 12
(B.107)
Pe 127))
m-1
(rh Tk,elthte - Mb.e
ok -ett 2
+ Ep )(z)) [ ro. 141]
(B.108)
We first note that all term in Eq. (B.106) converges in the mean-zero normal distribution and is bounded in probability at 1,11, rate. Therefore,
1122
m
Eq. (B.106) = Σ Or (2,)) (Ni,e,) 1/2)
(B.109)
We now analyze the second term. We note that, by [Kennedy et al., 2020, Lemma 2],
F at
F Thelep - 1k,t 1+1 } T 1/.00 12 (1+11
(B.110)
110
112 - 11,,1 - ko 12422 1xtt01111
=OPa
(B.111)
++2 We note that || eiktte given condition. Therefore, it is bounded in probability at n,41e rate. By the same analysis,
1+1 Ma,t
i+2
1 is bounded by some constant by the -1/2
[ike 12 bounded probability at 1/ /T,,,.-raats.. Therefore,
ED tte -P -((
m
Eq. (B.
Σ Or (z
(m
1/2
(B.112)
Finally, under the given assumption, the third term can be analyzed by Lemma S.6 and is zero:
m
Eq. (B.108)
Op Opoizt
(luEti
- witt || - = = 0.
(B.113)
Therefore,
AF Ag Eq. (0. + Eq. (B. + Eq. (B.
Op
-1/2 Opocz
(T )
(B.114)
1=1 1=
37

---

and, finally.
À Ao
Σ
(A
Ao}
(B.115)
L {=1
m
Op (((25) (14,
11/2)
(B.116)
L 1=1 1=1
m
(Tigk 1/2
Op OP (()))
(B.117)
op (1),
(B.118)
oiz
==1
where the last equation holds since Op(n ()) 0p(1) when 0.
C Discussion
C.1 Relaxation of Discreteness Assumption
In this paper, we made the strong assumption that all variables are discrete. However, this assump- tion does not hold in general. In this section, relax the assumption to certain degree while ensuring that the proposed estimators and corresponding error analysis remain applicable without sacrificing generality.
First, we define set of variables, denoted as disc, which must be discrete in order to apply the proposed estimators and leverage the error analyses presented in the paper.
Definition (Discreteness set disc). For some given inputs (x, y, Z. IP, G), suppose
(C.1)
where each A is specified as
A = Aol Ck R² seq beq"](w"")e rk),
(C.2)
for W R V and z Z. Then, the discreteness set disc (1 K
( defined as follow:
K
disc((A()) =
(C.3)
k=1
For Example 1, the discreteness set is given as
disc(fA). 4 (2 -((W).
(C.4)
For Example 2, the discreteness set is given as
disc({A], A = R, ( W,X, = X R,W,Y = V
(C.5)
Equipped with the discreteness set, we relax the assumption as follows: Assumption 1.1 (Relaxed Regularity). For variables V and the Radon-Nikodyan derivative Po(Z) of Po(z)) for Z E Z, the following conditions hold:
1. All variables in disc(fAA)) are discrete;
2. (v) > c, Vv E Dy for some E (0, 1).
We note that the proposed estimator is well-defined and corresponding enror analyses Theorem and Corollary 2 hold true under the relaxed assumption:
38

---

Lemma S.9 (Well-defined MR-gID Estimator under Relaxed Regularity). The MR-gID estima- for in Definition patnvise-differentabbl under Assumption 1.1 and Assumption 2.
Proof of Lemma S.9. For k 1,... K, define
A
Ep
Thk lit ui 1
12t41, + Er Er..z (122) [iglet, rō]
(C.6)
To establish the nuthwise-difffrenttabllyy of the MR-gID estimator f((A*) }k=1)
[ as defined in Def- inition 4, it is sufficient to ensure the nathwise-diffeeetiabllty of individual A Under Assump- tion 1.1, M +1 = w et (W C*) is well-defined since (W C*) ) E disc ( ( are dis- crete. Also, each 1 (R.) in each Tं are well-defined since R E disc({Ab) [Ab) are discrete. Fi- nally, the conditional expectation Er Br [||,,, r5] is well-defined since Z disc(1A))
discrete. Also, under the positivity condition stated Assumption 1.1, 7 Eq. (C.6) pathwise- differentiable. By combining this with Assumption 2, we conclude that the MR-gID estimator is pathwise-diiferenttable
L
C.2 Sequential Doubly Robustness: 2 robustness versus m-robustness
In this section, we discuss the practical properties of the proposed doubly robust g-mSBD estimator in Def. 3. We recall that the estimator is doubly robust by the analysis in Lemma S.6 as follows:
Lemma S.6 (Bias Analysis of g-mSBD Estimators). Let A be the quantity defined as
A
[ n²+ H(*"'H2+1,,r) Pa(z) [i3 ro]
(B.68)
where x²,, ye are arbitrary nuisances for true nuisances Th and uá defined in Def. 2. Let Ag denote the g-mSBD in Def. Then,
A Ao Pooters (1 T ++ 140 11)+11)
(B.69)
The term in Eq. (B.69) exhibits doubly robustness, becoming zero when either n" = T or j²" 16+ hold for all r 1,2,.... m This phenomenon referred to as the sequential doubly robustness [Luedtke et al., 2017], or 2 robustness in the sense that there are 2 ways make Eq. (B.69) zero |Vansteelandt et al., 2007, Rotnitzky et al., 2017].
While the proposed doubly robust g-MSBD estimator defined in Definition 3 exhibits doubly robust- ness, as shown in Proposition 1, it does not satisfy 2 robustness. This is due to the dependencies between plg and fr for E {r 1, 1.... .m). Specifically, if l is misspecified for some > r, renders the case 1u 1á impossible. Consequently, instead of having 2 possibilities, there are only ways to make Eq. (B.69) equal to zero. For each r 1,...
1, this requires either TH = To or μ 1á for s > r. This condition is referred to as m--robustness. In summary, the dou- bly robust g-mSBD estimator achieves m-rrobustness instead of 2 robustness. We acknowledge that interesting open direction is explore ways to enhance the doubly robust g-mSBD estimator to attain 2 robustness, building upon the findings presented in Luedtke et al. [2017].
D Details of Experiments
As described in Sec. 4, we used the XGBoost [Chen and Guestrin, 2016] as model for estimating nuisances μ., ग. {xr) )m 2 { Ji=1- We implemented the model using Python. In modeling nuisance using the XGBoost, we used the command
39

---

to use the XGBoost with the default pa- rameter settings. In estimating the weight, we set the weight nł. = 10 whenever the estimated weight over 10 [Crump et al., 2009]. For Example I, the dimension of W is set as [W] 10. We chose L 2. All variables are set to be binary. We compute the effect of P(Y 1[do(x)).
xgboost. XCBClassifier
D.1 Designs of Simulations
In this section, we present the structural causal models (SCMs) utilized for generating the dataset. Furthermore, we include segment of the code employed generate the dataset.
D.1.1 Example
We define the following structural causal models for Example 1:
Uw. Uw Uwy. UXy normal (0, 1);
W = fw(Uwz, UwY, Uw),
X [x(W, (xr)),),
Z
Y = Uwr,,
where
1
(wY)
exp(Uw Uwz + Uwr)
1
fx(W,Uxy)
explc W + +UY)
fz(W,X,Uw2)
1+ exp(UUu c7 W 4X -2+ Uwz).
2
(,,,Uwy,UxV)
+ c42 - 2 0.50wy Uxy)
where the coefficient vector cx, Cz are such that their ith element is if is even number and 1 otherwise.
D.1.2 Example
We define the following structural causal models for Example 2:
Ux, Ux,, Ux U. Ux,YY v ~ normal (0, 1);
X1 = fx, (Ux,..));
X₂ Sx, (Ux,,X));
R = fn(Ux,, X2), Xxx,,
W fw(Ux, W, Ux.. X1, RI,
Y fy X2, R. W),
'Detailed parametrization of parameters including learning rates, maximum depth of the trees. etc. are explained in https://xgbon http:////cobootrtrathcoosssss http://hhww.cc..o///////aaaallaa httpn//taale//ythootoeeoo wwwpppttttttooeeeoopopppppppptet api html# XcBClassifier.
readthedocs.
40

---

X1
X1
X1
X2
X2
C
R
C
R
C
R
W
W
W
Y
(a) SCM M Project STAR
(b) ((Ma)
(c) G(Mzz)
Figure E.4: Example causal graphs for Section E. Nodes representing the treatment and outcome marked in blue and red, respectively.
where
fx, xUx,xx))
1 + exp 22 X, -
fxx =
+ exp((3U X1,Xo
fR(Ux,, X2) =
1
2X, + UX 4)
Uo(Ux,,,UXx,l X1, R) =
+ exp(Uxx x1w (2X; (2X1 1) + (4R - 2) + Uxg.
R2
I exp
2X2 2W Ux 2)
E
Project STAR: Estimating Joint Effects of Class Sizes to Academic Outcomes
We applied the proposed estimators to the Project STAR dataset [Krueger and Whitmore, 2001, Schanzenbach, 2006]. Project STAR is an experimental study investigating teacher/student ratios' impact on academic achievement for kindergarten through third-grade students. In the study, stu- dents were randomly assigned to three different class sizes: small-size classes, regular classes, and large-size classes. The objective was evaluate how class size affects academic outcomes [Schanzenbach, 2006]. In analysis, used the dataset introduced in the online complement of Stock et al. [2003].
Project STAR Dataset. We denote the Project STAR dataset D. The dataset D) includes the following information: class size for kindergarten (X1), the academic outcome in kindergarten (W), the academic outcome in second grade (R),, class size for third grade (X2), the academic outcome in the third grade (Y), free lunch receiving for kindergarten (C1), gender (C2), ethnicity (Cz) and free lunch receiving for the third grade (C4). We will use C₁ = (C1, C2) and C2 == (C3, Ca).
Assumption on Dataset. We assume that the SCM M generating the variables (X,, W, R, X2, Y. C2, C2) induces a causal graph depicted in Figure E.4a. Here, we mark C₁ C2 as gray to denote that these variables will be considered latent; i.e., these variables will not be used in the data analysis.
Project STAR dataset D is longitudinal experimental study randomizing X1 and X2; i.e., the dataset is induced by the submodel Mr, .no for T1,22 E Dx,,x,, represented in Fig. E.4b. The samples for variables {X,, follow a distribution (X,, W, = Po(X.,,,)XX) W, R), and the samples for variables {X,, W, R. X2, Y} follow distribution Po(x, (X1, W, X2, Y). To demonstrate, will describe how to generate the sample fol-
41

---

lowing a distribution (XI,
by creating unmeasured confounding bias between X1 and W, and X1 and Y, which is depicted by Fig. E.4c.
Creation of Datasets from Marginal Experiments. In this empirical study, we create two datasets from this dataset: D1 and D2. The dataset D1 is a random subsample of only including (X1, W.. Then, Dj follows (x(x,) (X1, W, R).
To construct the dataset D2 following the marginal experimental distribution Px(x))(X1,, W R, X2, Y), the confounding bias between X1 and W and X1 and Y should be introduced. To do so, we follow a standard procedure for introducing confounding bias from experimental studies used in Hill [2011], Louizos et al. [2017]. Zhang and Bareinboim [2019], Gentzel al. [2021].
A setting for the standard procedure is the following. For any arbitrary random variable X, 2, W such that X + Y, Z + Y, and there are no arrows into X, the dataset D = {(X(n) Y(a): Z(o)Wos i = 1, ... : is given. In D, Z is not a confounding variable since Z X. The goal to generate a new dataset D' {())) Y(n) Zin : -. .n'} where Z serves confounding variable between X and Y. The procedure named antroduceConfounding X, Y; Z, D) is given as follows: Initialize D' = () For = ..... n, do the followings:
1. Generate the Bernoulli Random (()) with parameter ((X())Z()).
2. If (1) 1, include (X))), Y(u, Z0o W()) in D'.
Finally, we exclude Z is removed from D'. By doing so, we introduce unmeasured confounding bias between and in D'; i.e. D' IntroduceConf won.oongoooniingggo.. X, Y; Z, D).
To generate the dataset D2 Po(X).))) W, R, CI, C2). we have to introduce the unmeasured confounding bias between X1 and W, and X, and Y. We do this by D2 = IntroduceConf woweooonoooninggn..... W: W.C, Then, D2 is variable containing (X1, W, R, X2, C22 Y) and there is confounding bias between X1 and W. Then, we set D2 = IntroduceConf woncceCCoooooniiiii inni,,
Patx,) (X1, W, R, X2, from D
D2). Then, D2 is variable containing (X1, W, R, X2, Y) and there a confounding bias between X1 and Y, and X1 and W. A causal graph Fig. E.4c depicted the dependencies in D2.
Goal. In this empirical study, we aim to study the joint effect of the class size for kindergarten (X1) and the third grade (X2) on the third grade's 's academic outcome (Y); i.e., E [Y [{)((x,,)))] Since D is longitudinal experimental dataset following Pramdt X, xx) X1, W,, X2, Y), the ground-truth [Y [d((x1, 42)] is estimated ED [Y 1 1i (X1, X2)] /ED [1a (X1, X2)].
Causal Effect Identification.
We identify (oy]do(z), 22)) through Algo.
1. Line 3: Set D an(Y) (M))))) (Y, R, W).
2. Line 4: Set D = (R), D2 = (W). = (Y).
Each Q[D,] Q[W]. @D2 Q [D3] Q are identified as follows: For Q(D.),
1. Line 7: For D,, Z, == {X1] 2 = [.1). S {W).
2. Line 8: Set [[S]] A R = seq] = =(x:)) (w,)) where R =
3. Line a.4: Since S = D1, we set
Q = Q[S
A/
NM/W,6,;20 (X1). seas (a) (wr 0) (u(x))(w(21) = Pr, (w).
For Q[D2] Q[R],
42

---

1. Line 7: For 22, Z2 = (X1). 22 = (m1). 8 = (R).
2. Line 8: Set [ Au(S) R, (Xi), = (11))((, where R = (W).
3. Line a.4: Since $ 22 set
D2 Q
A
= Aols 0, = seq (x1)[(w,r)
(1) = Par (rlw).
For Q[Da] = Q[Y,.
1. Line 7: For D3, Z3 == (X2). 23 = (x2). S = (Y, WJ.
2. Line 8: Set Q[$7] = Ao|S7, 9, R7 = {X2]. segs = (x2))(U,r) where R = (W):
i.e.,
Q[ = Ao((X, W. Y), 0, (R): (X2), seau (x2))((F,,
(y 21 w, w,r, (x2) ( wl222
3. Line a.2: Set A == am DD (Da)a(s3) = (V).
4. Line a.3: Q[A] Ao((Y), (X1, W/, [3 = (X2). (x2))(1,r).
5. Line a.4: Since A = D2, we set
@ Q[A]
A/
AollY), (X, (R): [X2), (22))(u,r)
(E.1)
E Pa(x)) (xx))) w/x2).
si,wEDx,,,
Then, by Line 14,
Ply)do(z1,
Σ Q!1QQ[N[Q1Y]
r n,W
Σ
A AB AY
By Lemma 3,
A AWR AWAR AB
Aol(R, WJ,0,0; [X1}, sbas = (I1,,)))((
(l,)))
= Pr. P., (w.
Therefore,
22)) = Σ
A AB
T n.W
Σ
AWR A
r.w n.W
A
WR
R
AA
rED wEDW
where the last equation holds since AA is not a function of W. By Lemma 2
A
Σ
WW A, (X1), (z) [r. 0) = Pa Pacx))r(a1))= = Pa (r).
WE Dw w
43

---

Scenario
Scenario 2
Scenario
Scenario
Mode
Vodk
Model
Fig. E.3a (Project STAR)
Figure E.5: AAE Plot for Project STAR dataset analysis for Scenarios (1,2,3,4).
Therefore,
P(y)
A
r
WEOW
Σ
A A
r
Here, we only describe the nuisance for A in Eq. (E.1), since esti- mating A5" Pay (r) is trivial. We define the nuisance as follows: For the fixed 22 E xx,,
Causal Effect Estimation.
Po(X,, W, R) = Eppg [Y W1.
(E.2)
(R)
To ( W) P2g (R| W)
(E.3)
Then, AI in Eq. (E.1) can be expressed as follows:
Eq. (E.1)
(E.4)
(R) Ta R R2 W)
EPn2
or,
(E.5)
Ep,g
[oo(X (x1, W,r),, or,
(E.6)
(R)
-Er,, To(R) W) (Y - Mo W, R)) + (X1, (X,,W, W.r)
(E.7)
We then construct the regression-bast basedd, probability weighting-based, and MR-gID (MR) Tes Tow mr using the following procedure.
For each fixed 22 E x2 and sample set Day for E [1,2), randomly split the sample Dag, a and Dag.e-
2. Use Day,t train the model for learning nuisances in Eq. (E.2) and Eq. (E.3). Let μ(X1, W, R) and ( W) denote the learnt models. We use the XGBoost [Chen and Guestrin, 2016] to learn the model.
3. Then, each estimator is defined as follows:
T ED (e( (X,, W.r)]
(E.8)
1,(R)
Taw E.
(F.9)
x [R|X, W)
n = EDzg.C
T [R|X, W) (Y - n(X1,
(R)
W, R)) ((X,(X,, + ED [n(X,, W,r)]-
(E.
Experimental Results As described in the Experimental Setup section (Sec. 4), we evaluated the AAE of estimators est for est E {reg, pw, mr) in Scenarios {1, 2, 3, 4). The AAE plots for all
44

---

cases can be seen in Fig. E.5. In this particular scenario, the sample size was not varied since the sample itself was externally given.
In Case 2, introduced variation by adjusting the size of the converging noise which follows normal distribution Normal(n 2a) for E {200, 400, 600, 800, 1000]. It was observed that the MR-gID estimator 7 outperformed the other two estimators by achieving fast convergence, as demonstrated in Theorem 2. For Scenarios [3, 4), the DML estimator THe exhibited doubly robust properties, as illustrated in Corollary 2.
45


Reviews:
[{"review_id": "WsPekkFWcE", "reviewer": "Reviewer_T9Rz", "summary": "This paper builds on prior work in causal effect estimation, from observational as well as experimental data, by providing a more general framework with desirable properties. This more generalized framework is shown to be robust and consistent in its estimation and overcomes the limitations of some of the prior work in this area. The new, general framework, operating in the realm of generalized identification or g-identification, exhibits multiply robust properties, which is experimentally validated. ", "questions": "1. Is there any scenarios where Jung et al 2023 is still useful -- i.e., are there complementary notions of use to that framework and the one proposed here, or does the framework here subsumes Jung 2023 completely? Is it possible to compare the two frameworks in a particular setting to empirically demonstrate the relationship between the two frameworks?\n\n2. What practical challenges remain for g-identification when using the proposed estimator? \n\nSuggestions:\n\n- Remove citation from abstract, just let the 'in the literature' stand, and cite instead when first mentioned in the introduction.\n- Claim in Line 286 -- \"We consider two standard baselines in the literature\" --  must be backed by citations, especially the notion of being \"standard\". ", "limitations": "It is not clear what are the limitations of this work and the proposed new framework, and a lack of such an elucidation might hurt the uptake of what seems to be a promising new framework. There does not seem to be any discussion of any kind of limitations or potential broader impacts. ", "rating": 7, "confidence": 1, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The new proposed estimator could be now the go-to framework for any g-identifiable causal functionals, potentially representing a significant theoretical advance in the field of learning causal effects.  \nThe paper is well-scoped and specific in what is being advanced and corroborates the desirable properties of the estimator via simulations.  \nThe exposition is quite clear and well-written. ", "weaknesses": "It is not clear what challenges remain for g-identification in practice with the proposed new estimator. Some guidance for practitioners and other researchers as to the context of problems and types of data they can consider using the new estimator would be highly useful.  ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for sharing your thoughts and feedback!\n\n---\n\n> W1. It is not clear what challenges remain for g-identification in practice with the proposed new estimator. Some guidance for practitioners and other researchers as to the context of problems and types of data they can consider using the new estimator would be highly useful.\n\n> Q2. What practical challenges remain for g-identification when using the proposed estimator?\n\n> L1. It is not clear what are the limitations of this work and the proposed new framework, and a lack of such an elucidation might hurt the uptake of what seems to be a promising new framework. There does not seem to be any discussion of any kind of limitations or potential broader impacts.\n\nWe appreciate you for raising this issue and thank you for the opportunity to provide further clarification. We stay by the generality of our results and are genuinely excited about them. However, we do not claim that our paper has resolved every issue; there are several challenges that remain for the community to address. First, a critical assumption in our work is that the endogenous (observed) variables are discrete. This assumption limits the generality of our claims. While we offer some preliminary ideas on how to relax this limitation in Section C1 of the Appendix, the issue largely remains unresolved. Second, our proposed estimator is not shown to be semiparametric efficient due to the constraints implied by causal graphs. This is despite its doubly robust nature and fast convergence rates. Addressing these challenges would be highly non-trivial but important extensions of the work proposed in this submission.\n\n---\n\n> Q1. Is there any scenarios where Jung et al 2023 is still useful -- i.e., are there complementary notions of use to that framework and the one proposed here, or does the framework here subsumes Jung 2023 completely? Is it possible to compare the two frameworks in a particular setting to empirically demonstrate the relationship between the two frameworks?\n\nThis work encompasses all the results presented in Jung et al., 2023. To clarify their relationship, note that under the graphical settings and assumptions imposed in Jung et al., 2023 (for example, no unmeasured confounders between outcomes in multiple experiments when input distributions are multiple experimental distributions), the proposed g-mSBD method reduces to Jung et al., 2023.\n\n---\n\n> Suggestion 1. Remove citation from abstract, just let the 'in the literature' stand, and cite instead when first mentioned in the introduction.\n\nWe will revise it accordingly, thank you for the suggestion.\n\n---\n\n> Suggestion 2. Claim in Line 286 -- \"We consider two standard baselines in the literature\" -- must be backed by citations, especially the notion of being \"standard\".\n\nSince the gID estimation problem has not been addressed in prior literature, no existing estimators were available. To enable comparison with our proposed estimator and emphasize its doubly robust properties, we developed ‘baseline estimators’ using regression and probability-weighting methods. That is why this sentence was not backed by the citations. These methods are intuitively conceivable and frequently employed in standard settings where the sole input distribution is observational (hence our reference to them as \"standard in the literature\"). We appreciate you raising the issue. We would like to note that some discussions explaining the baseline estimators are in Section A5 in Appendix. Thank you for raising this point."}, {"review_id": "ZuBLGnnRIg", "reviewer": "Reviewer_PJpp", "summary": "The paper aims to fuse the recent literature on causal effect identification and estimation. It expresses any identification (in the general sense) of a causal effect in terms of generalized mSBD adjustments, and then develops a corresponding estimator. For the latter robustness properties and empirical studies are shown.", "questions": "Can you include a future work section? My perception of the significance of the work would be improved if I could see some directions for how to build on this.", "limitations": "Yes, they have (there is ample discussion of e.g. the assumptions relied on).", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The contribution made by this paper is well-situated within the problem space and it's a natural generalization of other recent papers. All the results appear to be sound.", "weaknesses": "I believe the contribution to be valuable but the methods are not novel; they seem to be iterative applications of existing ones.\nIn many places it's too dense: for example, Examples 3, 4 are not illuminating, and readers may just skip them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the time and valuable feedback, and appreciate the positive assessment of our work.\n\n---\n\n> W1. I believe the contribution to be valuable but the methods are not novel; they seem to be iterative applications of existing ones. In many places it's too dense: for example, Examples 3, 4 are not illuminating, and readers may just skip them.\n\n\nThank you for the opportunity to clarify this issue. One of our primary research findings (as detailed in Theorem 1) reveals that the gID function can be expressed as a function of g-mSBDs. We believe this realization is non-trivial. This insight paved the way for the development of our proposed estimator, into which each DR-g-mSBD estimator is plugged in. It represents a natural and effective estimator built upon our main findings.\n\nGiven the intricate nature of this topic, which spans multiple disciplines, and the tight constraints in terms of the number of pages,  presenting the necessary background in a gentle manner is non-trivial.  We apologize for that. Still, detailed explanations of the examples can be found in the Appendix; a step-by-step discussion of Example 3 is presented in Appendix A.1, and Example 4 is elaborated in Appendix A.2.\n\n---\n\n> Q1. Can you include a future work section? My perception of the significance of the work would be improved if I could see some directions for how to build on this.\n\nSure. We will discuss potential future directions in the updated manuscript. For now, a summary of possible research directions is as follows: \n1. Investigate more generalized scenarios that account for the presence of selection bias or the challenges of transportability/generalizability across changing conditions; \n\n2. Relax the restrictions of this paper, such as the assumption of discrete variables; \n\n3. Develop a semiparametric efficient estimator; \n\n4. Construct computationally efficient estimation procedures; \n\n5. Develop approximated estimators for whenever the effect is not identifiable."}, {"review_id": "xfdWtXXTba", "reviewer": "Reviewer_t8rU", "summary": "The authors study a class of causal effect that are called “g-identifiable” (gID) following Lee et al. (UAI 2019). There are two main results.\n1. Expressing these causal effects as transformations of generalized multi outcome sequential back door adjustments (g-mSBDs).\n2. Proposing doubly robust estimators for the adjustments, which then imply estimators for the causal effects if the transformations are smooth.\n\nIt appears that Jung et al (ICML 2023) previously provide these types of results for a subset of gID causal effects. The contribution is to extend the previous results for the full class of gID causal effects.", "questions": "I would be willing to improve the score if these questions are addressed by the authors.\n\n1. Algorithm 1 is a modification of Lee et al. (2019)’s complete algorithm. Is it the case that Algorithm 1 is precisely the previous algorithm but with Lemmas 2, 3 and 4? Does it remain complete?\n\n2. Since the transformation f is shown to be marginalization, multiplication, or division, wouldn’t we already know that aspects of Assumption 2 are automatically satisfied? Phrased another way, when is each condition in Assumption 2 binding?\n\n3. Are the proposed estimators semiparametrically efficient?\n\n4. What does scenario 2 mean? Is this measurement error, or is it the usual error of a regression problem? \n\n5. Suggested corrections:\nline 8: multiply --> multiple.\nline 86: specify norm when writing convergence of f_hat-f when they are functions.\nline 127: “(and rewrote in Algo. 1) rewrites” is awkward phrasing.\nline 248: Fretchet --> Frechet.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Originality: The first contribution is, in my view, the stronger one. Its connection to prior work is clearly stated. It appears that the workhorse is Lemmas 2, 3, and 4, which demonstrate the good properties of Definition 1. The second contribution follows quite closely from prior work.\n\n2. Quality: The introduction was generally well written, though it would make more sense to choose Examples 1 and 2 to be examples that are not in the class of Jung et al (ICML 2023).\n\n3. Clarity: Overall, the organization of the paper was very logical.\n\n4. Significance: The contribution seems sound but possibly incremental.", "weaknesses": "I would be willing to improve the score if these weaknesses are addressed by the authors.\n\n1. I find the title and framing too broad. There are many papers that could have this title, and it does not clearly convey what is in this paper but not in other papers.\n\n2. Line 56 gives a list of doubly robust estimators. At the very least, the sequential regression paper by Bang and Robins (Biometrics 2005), the targeted learning in data science textbook (van der Laan and Rose 2018), and the debiased machine learning paper by Chernozhukov et al. (2018 Econometrics Journal) should be cited. \n\n3. More relevant are the sequential doubly robust estimators such as Luedtke et al. (arXiv:1705.02459, 2017) and Chernozhukov et al. (arXiv:2203.13887, 2022) and the references in those papers. Definition 3 and Proposition 1, are very closely related to the prior works listed above, so explicit comparisons would be helpful here.\n\n4. The delineation between Lee et al. (2019)’s algorithm and this algorithm should be clearer. The sentences in lines 127 and 156 seem to clash in explaining the relationship to the prior work. \n\n5. The notation in Definition 1, Lemma 1, Lemma 2, and Lemma 3 is very heavy, and no interpretation is assigned to any of the variable symbols. It would help to revisit Examples 1 and 2 here.\n\n6. In the appendix, the data generating processes are not described analytically; instead, python code is pasted. Please describe them. Please explicitly write what are the functions mu and pi.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback and for the opportunity to provide further elaboration. We respectfully ask the reviewer to reconsider our paper based on the clarifications provided below. If there are any points that remain unclear, we would be glad to offer additional explanations.\n\n---\n\n> Weakness 1\n\nThe current title, \"Estimating Causal Effects Identifiable from Combination of Observations and Experiments,\" reflects what we believe is the broad scope of the paper. In particular, we introduce the first family of estimators designed for (1) any mix of observational and experimental distributions and (2) any given causal diagram, extending beyond the back-door case. Our aim was to convey comprehensiveness. However, based on your feedback, we are contemplating titles like \"Estimating G-identifiable Causal Effects.\" The term \"G-identifiable\" highlights the paper's focus on the flexibility of combining observational and experimental distributions. To the best of our knowledge, no other paper can estimate such a diverse coverage of effects, input distributions, and diagrams. Technically, our results match the completeness results on the identification side, which means that any effect that is in fact 'g-identifiable' is also 'estimable' via our proposed method (Theorems 1 and 2). While there may be other papers that claim generality, we are not aware of any research covering all g-identifiable effects. Having said that, we are certainly open to any suggestions you might have regarding a more apt title for our paper.\n\n---\n\n> Weakness {2,3} \n\nIndeed, this was an oversight, and while one citation is in the appendix, we will consolidate them and cite all the referred papers. Nice catch, thank you.\n\n--- \n\n> Question 1 and Weakness 4\n\nLee's algorithm and our Algorithm 1 are equivalent. This implies that Algorithm 1 also provides a sound and complete procedure for identifying causal effects from a causal graph and the corresponding mix of observational and interventional distributions. The formal proof of this equivalence can be found in the proof of Theorem 1.\n\nThe motivation behind Algorithm 1's formulation is to express the g-identifiable causal effect as a combination of marginalizations, multiplications, and divisions of g-mSBD. Unlike Lee's algorithm, Algorithm 1 is formulated in terms of c-factors, which can be expressed as a function of g-mSBDs, as shown in Lemma 1. Since each c-factor is expressible as a g-mSBD, our algorithm has an amenable form that allows for the application of marginalizations, multiplications, and divisions, as required by Lemmas 2, 3, and 4. To align the sentences in lines 127 and 177, we will clearly state this point by replacing the sentence in line 157 as follows:\n\n\"We have rewritten the identification algorithm proposed by Lee et al. (2019) as Algorithm 1 to express the g-identifiable causal effect as a combination of marginalizations, multiplications, and divisions of g-mSBD.\"\n\n---\n\n> Weakness 5\n\nThank you for the feedback. We will add an explanation and detailed illustrations with Examples 1 and 2 in the Appendix.\n\n--- \n\n> Weakness 6\n\nWe have updated the data generation process analytically based on your feedback. This update will be available in future revisions. Additionally, the exact and explicit representations of mu and pi can be found in Sections A.5 and A.6 of the submitted Appendix.\n\n---\n\n**We now consider the \"Questions\". The first question was answered above.**\n\n> Question 2\n\nThank you for the good question. The estimator $f(\\{\\hat{A}^{k}\\})$ can be infinite if the estimated nuisance $\\pi$ for the weight is zero. For example, if $f(\\{\\hat{A}^{1}, \\hat{A}^2\\}) := \\hat{A}^1 / \\hat{A}^2$ and $\\hat{A}^2$ is zero, the function $f$ may not be differentiable, and the boundedness in Assumption 2 may not hold. Assumption 2 is a set of conditions excluding such cases and a sufficient condition for representing an upper bound of the error in terms of products of the two sets of nuisances for g-mSBD estimators. \n\n---\n\n> Question 3 \n\nWe appreciate the question. The answer is no; the proposed estimators have not been shown to be semiparametric efficient. Let us elaborate further. A semi-Markovian causal graph, characterized by directed and bidirected edges, implies two types of constraints in the probability distribution: conditional independence and Verma constraints (also called Nested Markov factorization). To the best of our knowledge, constructing the tangent space for a semiparametric statistical model that takes these constraints into account remains an unsolved problem. In other words, while we can construct doubly robust estimators if the distribution is compatible with a semi-Markovian model, building a semiparametric efficient estimator is still an open research question.\n\n---\n\n> Question 4 \n\nThis simulates sampling error induced by the finiteness of samples, which we believe is what you meant by the usual error in regression. In Scenario 2, the noise was added to the regression estimates, which decay as the sample size grows, to emphasize the errors induced by the finiteness of samples, following the technique outlined in Kennedy's 2020 paper. Specifically, we assumed that the error converges at a rate of N^{-1/4} to highlight the fast convergence of the proposed estimator. Another usual error in a regression problem is a specification error, which is caused by choosing the wrong regression models in estimating nuisances. Scenarios 3 and 4 simulate cases of specification errors. We have incorporated this discussion into the updated manuscript. Thank you.\n\n[Kennedy20] Kennedy, Edward H. \"Towards optimal doubly robust estimation of heterogeneous causal effects.\" arXiv preprint arXiv:2004.14497 (2020).\n\n\n---\n\n> Question 5\n\nGood catch, thank you! We revised the paper accordingly."}, {"review_id": "oI8yWBiQhi", "reviewer": "Reviewer_sTQB", "summary": "This paper presents a new framework for estimating the causal effect when multiple observational and experimental datasets are available. The authors proved that any g-identifiable causal effects can be written as a functional of \"g-mSBD\" operators, based on which they further developed a doubly robust estimator. Asymptotic properties of this estimator are then analyzed, and the performance of this estimator is confirmed in simulation studies.", "questions": "Please see the \"Weakness\" section.", "limitations": "Please see the \"Weakness\" section.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper tackles a hard problem with theoretical soundness and practical applicability.", "weaknesses": "I have two questions/comments. \n\nThe first regards the assumption imposed in their Proposition 2 as well as Assumption 2 -- the authors assumed the nuance estimates are L2-consistent. While I understand that such a property would make the theoretical analysis more amenable, I am curious and would appreciate it if the authors can provide a concrete (and maybe idealized) example, under which the assumption can be verified.\n\nMy second question is about practical performance of the proposed estimator. While I am convinced about its utility in the simulation study, it would be great if the authors can apply their methodology in a real dataset and verify it still outperforms its competitors.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback provided and hope that the answers below help to address the raised issues. Still, we would be happy to provide further clarification if you find that this was not the case.\n\n---\n\n> W1. The first regards the assumption imposed in their Proposition 2 as well as Assumption 2 -- the authors assumed the nuance estimates are L2-consistent. While I understand that such a property would make the theoretical analysis more amenable, I am curious and would appreciate it if the authors can provide a concrete (and maybe idealized) example, under which the assumption can be verified.\n\nGood point. This assumption is plausible in practice in the era of modern machine learning and deep neural networks. Thanks to the flexibility of these models (e.g., the universal approximation theorem of neural networks), it is plausible that the estimates will converge to the true nuisance whenever the sample size grows. Specifically, a recent study reveals that if the true nuisance function satisfies some mild smoothness assumptions, then the deep neural network model-based estimators will be L2-consistent [1].\n\n[1] Farrell, Max H., Tengyuan Liang, and Sanjog Misra. \"Deep neural networks for estimation and inference.\" Econometrica 89.1 (2021): 181-213.\n\n---\n\n> W2. My second question is about practical performance of the proposed estimator. While I am convinced about its utility in the simulation study, it would be great if the authors can apply their methodology in a real dataset and verify it still outperforms its competitors.\n\nSure. We will prepare a real-world study in the revised revision.\n"}, {"review_id": "wLXGxsJLCa", "reviewer": "Reviewer_8jnJ", "summary": "The paper presents a method for causal effect estimation through a combination of experimental and observational data. ", "questions": "No real questions just a request that the paper is rewritten to increase clarity and improve presentation ", "limitations": "Little discussion is included about the limitations of the method", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 1, "contribution": 4, "strengths": "- The paper addresses a very important problem \n- The solution appears to be mathematically sound\n- Experimental results appear to show improvement over other methods", "weaknesses": "- The paper is incredibly  hard to parse. Pages 4-8 are just a wall of mathematical notation with little to no explanation. \n- The paper would improve significantly if after each definition / lemma / proof a plain word intuition was given. \n- Running an example through an algorithm in the manner its done in page 6 is borderline incomprehensible. \n- Figure 2 is too small to read even when pdf is zoomed \n\n  Honestly this is a great paper that is severely limited by the way its presented. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your feedback, thank you.\n\n---\n\n> W1. The paper is incredibly hard to parse. Pages 4-8 are just a wall of mathematical notation with little to no explanation.\n\n> W2. The paper would improve significantly if after each definition / lemma / proof a plain word intuition was given.\n\nGiven the intricate and technical nature of this topic that spans multiple disciplines (theories of causal effect identification and semiparametric inference), presenting the necessary background in a digestible manner is quite challenging. Given the constraints on the length of NeurIPS papers, we have strived to strike a balance, emphasizing our novel contribution and demonstrating it in a condensed manner. We recognize the density of our presentation and apologize for it. We plan to enhance the clarity of our results by adding more verbal explanations for our technical results.\n\n---\n\n> W3. Running an example through an algorithm in the manner its done in page 6 is borderline incomprehensible.\n\nWe would like to note that a detailed step-by-step procedure for following Algorithm 1 is provided in Appendix A: example 3 in Appendix A.1 and Example 4 in Appendix A.2.\n\n---\n\n> W4. Figure 2 is too small to read even when pdf is zoomed\n\nIt was an oversight and we apologize for that. We included the enlarged figure in the 1-page attached pdf in the general response above. We will incorporate the same in the updated version of the manuscript, thank you for bringing this up.\n"}]

Author rebuttal:
Thank you for your valuable feedback. We responded to each review individually. Here, we uploaded an enlarged plot for our simulation result (Figure 2). 

Meta-review: The paper introduces a new framework for estimating causal effects using both experimental and observational data. It extends the class of g-identifiable causal effects and provides a doubly robust estimator for these effects. The theoretical foundations are solid, and the simulation results suggest the effectiveness of the proposed method. However, there are concerns about the presentation and clarity of the paper, and there's a need for more practical application and discussion of limitations. The paper would benefit from addressing these comments.
Decision: Accept (poster)