Title: Estimating Causal Effects Identifiable from a Combination of Observations and Experiments
Venue: NeurIPS 2023
Accepted: yes
Keywords: Causal Effect Estimation

Causal Effect Identification

Data Fusion

Double Machine Learning

Doubly Robust Estimator
Abstract: Learning cause and effect relations is arguably one of the central challenges found throughout the data sciences.
Formally, determining whether a collection of observational and interventional distributions can be combined to learn a target causal relation is known as the problem of generalized identification (or g-identification) [Lee et al., 2019]. 
Although g-identification has been well understood and solved in theory, it turns out to be challenging to apply these results in practice, in particular when considering the estimation of the target distribution from finite samples. 
In this paper, we develop a new, general estimator that exhibits multiply robustness properties for g-identifiable causal functionals. 
Specifically, we show that any g-identifiable causal effect can be expressed as a function of generalized multi-outcome sequential back-door adjustments that are amenable to estimation. 
We then construct a corresponding estimator for the g-identification expression that exhibits robustness properties to bias. We analyze the asymptotic convergence properties of the estimator. Finally, we illustrate the use of the proposed estimator in experimental studies. Simulation results corroborate the theory.

Estimating Causal Effects Identifiable from a Combination of Observations and Experiments

Abstract

Learning cause and effect relations is arguably one of the central challenges found throughout the data sciences. Formally, determining whether a collection of observational and interventional distributions can be combined to learn a target causal relation is known as the problem of generalized identification (or g-identification ) [Lee et al., 2019]. Although g-identification has been well understood and solved in theory, it turns out to be challenging to apply these results in practice, in particular when considering the estimation of the target distribution from finite samples. In this paper, we develop a new, general estimator that exhibits multiply robustness properties for g-identifiable causal functionals. Specifically, we show that any g-identifiable causal effect can be expressed as a function of generalized multioutcome sequential back-door adjustments that are amenable to estimation. We then construct a corresponding estimator for the g-identification expression that exhibits robustness properties to bias. We analyze the asymptotic convergence properties of the estimator. Finally, we illustrate the use of the proposed estimator in experimental studies. Simulation results corroborate the theory.

1 Introduction

Performing causal inferences is a crucial aspect of scientific research with broad applications ranging from the social sciences to economics, biology to medicine. It provides a set of principles and tools to draw causal conclusions from a combination of observations and experiments. Two significant tasks in the realization of these inferences are causal effect identification and estimation. Causal effect identification concerns determining the conditions under which one can infer the causal effect P ( Y = y | do ( X = x )) (shortly, P ( y | do ( x )) ) of the treatment X = x on the outcome Y = y from a combination of available data distributions and a causal graph depicting the data-generating process [Pearl, 2000, Bareinboim and Pearl, 2016]. Causal effect estimation aims to develop an estimator for the identified causal effect expression using a set of finite samples.

Recent advances in the literature on generalized causal effect identification (g-identification) have developed algorithms that can identify causal effects by using a set of observational and experimental distributions and a causal graph. The result is an expression of the causal effect as a function of available observational and experimental distributions [Bareinboim and Pearl, 2012, Lee et al., 2019]. For concreteness, consider some practical scenarios that exemplify g-identification.

Example 1. Many studies have investigated how a training program's eligibility ( X ) affects future salary ( Y ) (e.g.,[Glynn and Kashin, 2017]). Actual registration in the program ( Z ) determines the salary, and experimental studies have looked into how Z affects Y (e.g., [LaLonde, 1986]). Eligibility is determined by past average income ( W ), which is associated with both Z and Y . The causal

W

X

Z

Y

(a)

X

1

X

2

R

W

Y

(b)

graph in Fig. 1a shows the data-generating process, with bidirected edges indicating unmeasured confounders affecting the variables. According to Lee et al. [2019], the causal effect P ( y | do ( x )) can be identified by combining the experimental distribution on Z (denoted P ( ·| do ( z )) ) with the observational distribution P . It's given as P ( y | do ( x )) = ∑ z,w P ( y | do ( z )) P ( z | w,x ) P ( w ) . ■

Example 2. There have been many experimental studies on the effect of an antihypertensive drug ( X 1 ) on blood pressure ( W ) (e.g., Hansson et al. [1999]) and on the effect of using an anti-diabetic drug ( X 2 ) on cardiovascular disease ( Y ) (e.g., Ajjan and Grant [2006], Kumar et al. [2016]). R is a set of mediators. Their relations are depicted in Fig. 1b. Recent studies report that simultaneously taking antihypertensive and anti-diabetic drugs may be harmful [Ferrannini and Cushman, 2012]. This motivates the study of the combined causal effect of both treatments (i.e., P ( y | do ( x 1 , x 2 )) ) by combining the two experimental studies (i.e., from P ( ·| do ( x 1 )) and P ( ·| do ( x 2 )) ). According to Lee et al. [2019], it turns out that P ( y | do ( x 1 , x 2 )) = ∑ r,w P ( y | r, w, do ( x 2 )) P ( r | x 2 , do ( x 1 )) ∑ x ′ 2 P ( w | r, x ′ 2 , do ( x 1 )) P ( x ′ 2 | do ( x 1 )) , which means that the joint treatment effects can be computed using the two experimental studies on X 1 and X 2 . ■

On the other hand, causal effect estimation has mainly focused on limited identification scenarios, relying on stringent assumptions such as the no unmeasured confounder assumption. Beyond these restrictions, recent progress has been made in developing statistically appealing estimators from observational data for any identification functional given by the complete identification algorithms [Jung et al., 2020a,b, 2021b,a, Bhattacharya et al., 2022, Xia et al., 2021]. While these estimators are capable of estimating any identification expression from observational data, they are not yet sufficiently advanced to estimate g-identification, which involves multiple observations and experiments.

Recently, Jung et al. [2023] generalized existing doubly robust estimators [Mises, 1947, Bickel et al., 1993, Robins and Rotnitzky, 1995, Bang and Robins, 2005, Robins et al., 2009, van der Laan and Gruber, 2012, Luedtke et al., 2017, Chernozhukov et al., 2018, Rotnitzky et al., 2021] to estimate covariate adjustments (e.g., back-door adjustment [Pearl, 1995], sequential back-door (SBD) adjustment [Pearl and Robins, 1995] or multi-outcome SBD (mSBD) [Jung et al., 2021b]) in the g-identification setting, where the expression is in the form of covariate adjustment involving multiple experimental distributions. However, the covariate adjustments only cover a limited portion of all g-identifiability scenarios as in Examples (1,2). On a different thread, Xia et al. [2023] developed a neural network-based estimation framework capable of taking a combination of observational/experimental data. Still, the derived estimators do not possess the doubly robustness property. In other words, there is still a gap between g-identification and causal effect estimation.

In this paper, our goal is to bridge the gap between g-identification and causal effect estimation. Specifically, this paper presents a framework for estimating identification expressions using multiple sets of samples from both observational and interventional distributions. This framework is a generalization of the results in Jung et al. [2021b] since our results reduce to theirs when only observational data is available. Furthermore, our work subsumes the results in Jung et al. [2023] when the identification functional takes the form of covariate adjustments.

The contributions of our paper are as follows:

We show that any causal effects identifiable by g-identification can be expressed as a function of generalized mSBD adjustments. We provide a systematic procedure for specifying the function.

After developing a doubly robust estimator for generalized mSBD adjustments, we construct an estimation framework for any g-identifiable causal effects, which enjoys multiply robustness against model misspecification and bias. Experimental studies corroborate our results.

1.1 Preliminaries

We use bold letters ( X ) to denote a random vector and X a random value. Each random vector is represented with a capital letter ( X ) and its realized value with a small letter ( x ). Given a set X = { X 1 , · · · , X n } aligned by an order ≺ such that X i ≺ X j for i < j , we denote X i := { X 1 , · · · , X i } and X i : j := { X i , · · · , X j } . For a discrete vector X , we use /x31 x ( X ) to represent the indicator function such that /x31 x ( X ) = 1 if X = x ; /x31 x ( X ) = 0 otherwise. We use [ n ] := { 1 , · · · , n } a collection of index. For a discrete vector V , we use P ( v ) := P ( V = v ) where P is a distribution. We use E P [ f ( V )] := ∑ v ∈ S V f ( v ) P ( v ) for a function f , where S V denote the support of V . We will use D V to denote the domain of V . For a sample set D := { V ( i ) } n i =1 where V ( i ) denotes the i th samples, we use E D [ f ( V )] := (1 /n ) ∑ n i =1 f ( V ( i ) ) . We use ∥ f ∥ P := √ E P [ { f ( V ) } 2 ] . If a function ̂ f is a consistent estimator of f having a rate r n , we will use ̂ f -f = o P ( r n ) . We will say ˆ f is L 2 -consistent if ∥ ˆ f -f ∥ P = o P (1) . We will use ̂ f -f = O P (1) if ̂ f -f is bounded in probability. Also, ̂ f -f is said to be bounded in probability at rate r n if ̂ f -f = O P ( r n ) . We use the typical graph terminology pa ( C ) G , ch ( C ) G , de ( C ) G , an ( C ) G to represent the union of C with its parents, children, descendants, ancestors in the graph G . We use pre ( C ; G ) to denote the union of the predecessors of C i ∈ C given a topological order ≺ G over a graph G . We use G ( C ) to denote the subgraph of G over C . Throughout the paper, we will assume a fixed topological order ≺ G over V on G . ■

Structural Causal Models (SCMs). We use Structural Causal Models (SCMs) as our framework [Pearl, 2000, Bareinboim et al., 2022]. An SCM M is a quadruple M = ⟨ U , V , P ( U ) , F ⟩ . U is a set of exogenous (latent) variables following a joint distribution P ( U ) . V is a set of endogenous (observable) variables whose values are determined by functions F = { f V i } V i ∈ V such that V i ← f V i ( pa i , u i ) where PA i ⊆ V and U i ⊆ U . Each SCM M induces a distribution P ( V ) and a causal graph G = G ( M ) over V in which there exists a directed edge from every variable in PA i to V i and dashed-bidirected arrows encode common latent variables (e.g., see Fig. 1a). Performing an intervention fixing X = x is represented through the do-operator, do ( X = x ) , which encodes the operation of replacing the original equations of X (i.e., f X ( pa x , u x ) ) by the constant x for all X ∈ X and induces an interventional distribution P ( V | do ( x )) . ■

Experimental Distributions and Samples To clarify the connection between the experimental samples where the randomization is applied to Z ⊆ V and the distribution P z ( V \ z ) , we introduce the notation P σ ( Z ) ( V ) where σ ( Z ) denotes that Z is randomized. The distribution P σ ( Z ) ( V ) is a distribution induced by the SCM in which the original equation Z ← f Z ( pa z , u z ) for Z ∈ Z is replaced to the function assigning the value to Z = z at random without depending on other endogenous variables PA Z ; e.g., Z = 1 and 0 at probability 0 . 5 for each. We note that P := P σ ( ∅ ) when observational. For any set A , B , Z ⊆ V , the interventional distribution can be represented as P ( A | do ( z ) , B ) = P σ ( Z ) ( A | Z = z , B ) by the definition of the do -operator and P σ ( Z ) distribution. We use P z ( A | B ) := P σ ( Z ) ( A | Z = z , B ) to highlight that the distribution is induced from the randomization and conditioning on Z = z . The experimental samples from randomization σ ( Z ) induces samples D σ ( Z ) following P σ ( Z ) ( V ) . We use D z to denote the subsample of D σ ( Z ) fixing Z = z , which follows P z ( V ) . ■

g-identification. Let Z := { Z i } i m =1 denote a collection of variables where Z i can be an empty set. Let P := { P σ ( Z i ) ( V ) , Z i ∈ Z } , a collection of distributions inducing experimental samples from trials randomizing Z i ∈ Z . Acausal effect P ( y | do ( x )) is said to be g-identifiable from P in a causal graph G if P ( y | do ( x )) is uniquely computable from the combination of distributions in P in any SCM that induces G [Lee et al., 2019, Def. 4]. The complete g-identification algorithm developed by Lee et al. [2019] identifies the causal effect by decomposing so-called confounded components (c-component). A c-component is a maximal set of variables where every pair is connected by a bidirectional path composed of bidirectional edges ( V i ↔ V j ). For example, graphs in Figs. (1a, 1b) form a single c-component since bidirectional paths connect any pairs of variables. For any sets C ⊆ V , the quantity Q [ C ] := P ( c | do ( v \ c )) is called a c-factor . To identify the causal effect

P ( y | do ( x )) from P and G , the g-identification algorithm in [Lee et al., 2019, Algo. 1] (and rewrote in Algo. 1) rewrites the causal effect as a marginalization over a product of c-factors, P ( y | do ( x )) = ∑ d \ y ∈ S D \ Y ∏ k d i =1 Q [ D i ] , where D := an ( Y ) G ( V \ X ) and D i are c-components in G ( D ) , and identifies each Q [ D i ] from P . ■

1.2 Problem Statement

This paper aims to develop an estimation framework for the g-identifiable causal effect P ( y | do ( x )) identified as a function of distributions in P from experimental samples D := { D Z i ∼ P σ ( Z i ) ( V ) ∈ P } . We impose the following regularity assumptions:

Assumption 1 ( Regularity ) . For variables V and distributions P σ ( Z ) ∈ P , the following conditions hold: (1) All variables in V are discrete; (2) P σ ( Z ) ( v ) > c, ∀ v ∈ D V for some c ∈ (0 , 1) .

We discuss the relaxation of the regularity assumption in Appendix C. This relaxation allows some subset of variables in V can be a mixture of continuous and discrete random variables. Due to space constraints, all proofs are provided in Appendix B.

2 Expressing Causal Effects as a Combination of mSBD Adjustments

In this section, we present an algorithm that expresses any g-identifiable causal effects as a combination of marginalization/multiplication/divisions of adjustment functionals defined in the following. We begin by formally defining the generalized multi-outcome sequential back-door adjustment (gmSBD) functional, which strictly generalizes the mSBD adjustment proposed by Jung et al. [2021b]: Definition 1 ( generalized-mSBD adjustment (g-mSBD) ) . Let ( W , R ) be a disjoint pair in V topologically ordered as ( W , R ) = { R 0 , W 1 , · · · , R m -1 , W m , R m } by ≺ G , where R i can be empty. Let W i -1 := { W j } i -1 j =1 and R i -1 := { R j } i -1 j =0 for ∀ i ∈ [ m ] . Let C ⊆ W . Let Z 0 ⊆ Z be some set such that ∀ Z ∈ Z 0 , W ∩ Z = ∅ . Let seq ( Z 0 ) denote a sequence ( z 1 , · · · , z m ) where z i denotes some realization of Z i ∈ Z 0 (same z i could appear multiple times in the sequence). Then, the g-mSBD adjustment is expressed as an operator A 0 [ W , C , R ; Z 0 , seq , G ]( w \ c , r ) defined by

The g-mSBD adjustment specializes to the mSBD adjustment [Jung et al., 2021b] when Z 0 = ∅ . The g-mSBD adjustment can be viewed as a variant of the g-formula [Robins, 1986] involving multiple distributions. The power of the g-mSBD adjustment lies in its ability to express the c-factor:

Lemma 1 ( c-component Identification [Jung et al., 2021b]) . Let S denote a c-component in G i := G ( V \ Z i ) for some Z i ∈ Z . Let R := pa ( S ) G i \ S . Let ( S , R ) be ordered as ( R 0 , S 1 , · · · , R m -1 , S m ) by ≺ G . Let A ⊆ S denote a set satisfying A = an ( A ) G i ( S ) . Let C := ( S \ A ) . Let Z 0 := { Z i } and seq ( Z 0 ) be a sequence of z i repeating m times. Then, the c-factor Q [ A ] is g-identifiable as follows:

We propose an identification algorithm, Algo. 1, which expresses any causal effect as a combination of marginalizations, multiplications, and divisions of g-mSBD operators. Here are some results used for the g-mSBD operation. An example of using these results is provided in Appendix A.

Lemma 2 ( Marginalization ) . Let A 0 [ W , C , R ; Z 0 , seq ]( w \ c , r ) denote the g-mSBD operator in Def. 1. Let W 0 ⊆ W \ C . Let W mar ⊆ { W 0 , C } denote the vector formed by the following procedure: Starting from W mar = ∅ , for j = m, · · · , 1 , W mar = W mar ∪ { W j } if (1) W j ∈ { W 0 , C } and (2) ∃ k ∈ { j, · · · , m } such that R j , · · · , R k -1 = ∅ , W k +1: m ⊆ W mar, and Z k = · · · = Z j and z k = · · · = z j . Let W ′ := W \ W mar, R ′ := pre ( W ′ ; G ) ∩ R and C ′ := { W 0 , C }\ W mar. Let Z ′ ⊆ Z 0 denote the collection of Z i corresponding to the variable in W ′ , and seq ′ the corresponding sequence. Then,

̸

This lemma provides a graphical criterion where ∑ w 0 ∈ S W 0 A 0 [ W , C , R ; Z 0 , seq ]( w \ c , r ) is given as a g-mSBD operator.

Lemma 3 ( Multiplication ) . Let A i 0 := A 0 [ W i , ∅ , R i ; Z i , seq i ]( w i , r i ) := ∏ m i j =1 P z i j ( w i,j | w j -1 i , r j -1 i \ z i j ) for i ∈ { 1 , 2 } where seq i := ( z i j ) m i j =1 . Let W := W 1 ∪ W 2 . Let R := ( R 1 ∪ R 2 ) \ W . Let ( W , R ) be ordered by ≺ G . Let Z := Z 1 ∪ Z 2 . Assume the following: (1) W 1 ∩ W 2 = ∅ ; and (2) ∀ W j ∈ W , ∃ W i,k ∈ W i such that ( W j -1 , R j -1 ) = ( W i k -1 , R i k -1 ) . Let seq := ( z j ) j : W j ∈ W where z j = z i k for all j . Then,

This lemma provides a graphical criterion where a product A 1 0 × A 2 0 is given as a g-mSBD operator. Lemma 4 ( Division ) . Let A i 0 := A 0 [ W i , ∅ , R i ; Z i , seq i ]( w i , r i ) := ∏ m i j =1 P z i j ( w i,j | w j -1 i , r j -1 i \ z i j ) for i ∈ { 1 , 2 } where seq i := ( z i j ) m i j =1 . Let W := W 1 \ W 2 . Let R := ( R 1 ∪ W 2 ) ∩ pre ( W ; G ) . Assume the following: (1) W 2 ⊆ W 1 ; and (2) ∀ W j ∈ W , ∃ W 1 ,k ∈ W 1 such that ( W j -1 , R j -1 ) = ( W 1 k -1 , R 1 k -1 ) , Z i,k = Z j and z i,k = z j . Then,

This lemma provides a graphical criterion where a product A 1 0 /A 2 0 is given as a g-mSBD operator.

We have rewritten the identification algorithm proposed by Lee et al. [2019] as Algo. 1 to express the g-identifiable causal effect as a combination of marginalizations, multiplications, and divisions of g-mSBD. It's worth of noting that the identification algorithm proposed by Lee et al. [2019] and Algo. 1 are equivalent.

Theorem 1 ( Expression of g-Identifiable Causal Effects ) . Algo. 1 returns any g-identifiable causal effects as a function of a set { A k 0 } of g-mSBD adjustment operators in the form

where the function f ( · ) applies marginalization, multiplication, or division over g-mSBD operators in { A k 0 } as specified by Algo. 1.

For concreteness, we demonstrate the application of Algo. 1 for Figs. (1a,1b), where the effects P ( y | do ( x )) are g-identifiable. Detailed and visually friendly demonstrations are described in Appendix A.

Example 3 ( Application of Algo. 1 to Example 1 ) . Note Z = {∅ , Z } . Line 3-4: D = { Z, Y } where D 1 := { Z } and D 2 := { Y } . Line 5-13: Identify Q [ D 1 ] from Z 1 = ∅ as follow. Note D 1 ⊆ S 0 , where S 0 := V where Q [ S 0 ] = A 0 [ S 0 , ∅ , ∅ ; Z 0 := ∅ , ∅ ]( s 0 , ∅ ) = P ( v ) . Run Q [ D 1 ] = SUBID ( D 1 , S 0 , Q [ S 0 ] , G ) and obtain Q [ D 1 ] = A 1 0 := A 0 [ { W,Z } , W, X ; ∅ , ∅ ]( z, x ) = ∑ w ∈ S W P ( z | x, w ) P ( w ) . Lemmas (2,3,4) are used in running the sub-procedure. Now, identify Q [ D 2 ] from Z 2 = { Z } as follow. Note D 2 ⊆ S 1 := { W,X,Y } , the c-component in G ( V \ Z ) . Note Q [ S 1 ] = A 0 [ S 1 , ∅ , ∅ ; Z 1 := { Z } , seq 1 ]( s 1 , ∅ ) = P z ( w,x,y ) , where seq 1 = ( z, z, z ) . We run Q [ D 2 ] = SUBID ( D 2 , S 1 , Q [ S 1 ] , G ( S 1 )) , and obtain Q [ D 2 ] = A 2 0 := A 0 [ Y, ∅ , ∅ ; Z 1 , seq 1 ]( y, ∅ ) = P z ( y ) . Lemma 2 is used in the sub-procedure. Line 14-15: P ( y | do ( x )) = ∑ z ∈ S Z A 1 0 A 2 0 . ■

3 Estimating g-Identifiable Causal Effects

Example 4 ( Application of Algo. 1 to Example 2 ) . Note Z = { X 1 , X 2 } . Line 3-4: D = { R,W,Y } where D 1 := { R } , D 2 := { W } , and D 3 = { Y } . Line 5-13: In G ( V \ X 1 ) , D 1 = S 1 1 := { R } . Q [ D 1 ] = Q [ S 1 1 ] = A 1 0 := A 0 [ R, ∅ , X 2 ; Z 1 := { X 1 } , seq 1 ]( r, x 2 ) = P ( r | do ( x 1 ) , x 2 ) , where seq 1 = ( x 1 ) . In G ( V \ X 1 ) , D 2 ⊆ S 1 2 := { X 2 , W, Y } . Q [ S 1 2 ] = A 0 [ S 1 2 , ∅ , R ; Z 2 := { X 1 } , seq 2 ]( s 1 2 , r ) = P x 1 ( x 2 ) P x 1 ( w | x 2 , r, ) P x 1 ( y | x 2 , w, r ) where seq 2 = ( x 1 , x 1 , x 1 ) . Run Q [ D 2 ] = SUBID ( D 2 , S 1 2 , Q [ S 1 2 ] , G ( S 1 2 )) = A 2 0 := A 0 [ { X 2 , W } , X 2 , R ; Z 2 , seq 2 ]( w,r ) = ∑ x ′ 2 ∈ S X 2 P x 1 ( w | r, x ′ 2 , ) P x 1 ( x ′ 2 ) where seq 2 = ( x 1 , x 1 ) . Lemma 2 is used in the sub-procedure. Since SUBID ( D 3 , S 1 2 , Q [ S 1 2 ] , G ( S 1 2 )) return FAIL, we find the c-component S 2 1 := { Y } where D 3 = S 2 1 . Note Q [ D 3 ] = Q [ S 2 1 ] = A 3 0 := A 0 [ Y, ∅ , { R,W } ; Z 3 := { X 2 } , seq 3 ]( y, { r, w } ) = P x 2 ( y | w,r ) , where seq 3 = ( x 2 ) . Line 14-15: Applying Lemma 3, A 13 0 := A 1 0 × A 3 0 = A 0 [ { R,Y } , ∅ , { X 2 , W } ; Z 13 := { X 1 , X 2 } , seq 13 ]( { r, y } , { x 2 , w } ) = P x 1 ( r | x 2 , ) P x 2 ( y | r, w ) , where seq 13 = ( x 1 , x 2 ) . Then, P ( y | do ( x 1 , x 2 )) = ∑ r,w ∈ S R,W A 13 0 A 2 0 . ■

In this section, we develop an estimator for P ( y | do ( x )) using samples D := { D σ ( Z i ) ∼ P σ ( Z i ) ( V ) ∈ P } obtained from randomized experiments and observations (where Z i = ∅ ). We use P σ ( Z ) instead of P z to highlight the distribution D σ ( Z i ) ∈ D follows.

We first introduce an estimator for the g-mSBD adjustment that exhibits the doubly robust property. The nuisance parameters for the g-mSBD adjustment are defined as follows:

Definition 2 ( Nuisances for g-mSBD ) . Nuisances for g-mSBD A 0 in Eq. (1) are { µ i +1 0 , π i 0 } m -1 i =1 defined as follows. Let µ m +1 0 = µ m +1 := /x31 w \ c ( W \ C ) . For i = m -1 , · · · , 1 ,

Remark 1 ( Simplification of Nuisances ) . Although the nuisances π i 0 may seem complicated, they can be simplified in several important special cases. For example, π i 0 = /x31 r i ( R i ) /P σ ( Z ) ( R i | W i , R i -1 , z , r 0 ) if Z = { Z } for any Z ⊆ V where Z is possibly empty.

In general, employing off-the-shelf classification methods for density ratio estimation is feasible, leveraging the techniques outlined in Section 5.4 of Díaz et al. [2021].

We now introduce a g-mSBD estimator exhibiting the robustness properties using these nuisances. This estimator is motivated by the double/debiased machine-learning style estimators [Chernozhukov et al., 2018, 2022]:

Definition 3 ( DR-g-mSBD Estimators ) . Let D σ ( Z i ) for Z i ∈ Z denote the experimental samples from randomizing the variable Z i . Let D z i for z i ∈ D Z i denote the subsamples of D σ ( Z i ) fixing R 0 \ Z i = r 0 \ z i and Z i = z i . The DR-g-mSBD estimator ˆ A for the g-mSBD adjustment A 0 [ W , C , R ; Z 0 := { Z i } i m =1 , seq := ( z i ) i m =1 ]( w \ c , r ) is defined as follows:

Randomly partition D z i into { D z i ,ℓ } ℓ ∈ [ L ] ; i.e., D z i = ∪ L ℓ =1 D z i ,ℓ , ∀ Z i ∈ Z and z i ∈ D Z i .

For each fold ℓ ∈ [ L ] , let µ i +1 ℓ denote learned µ i +1 0 using D z i +1 \ D z i +1 ,ℓ for i = m, · · · , 2 ; and π i ℓ learned π i 0 for i = 1 , · · · , m -1 . Define ˇ µ i +1 ℓ := µ i +1 ℓ ( W i , r i , R 1: i -1 ) and π i ℓ := ∏ i j =1 π j ℓ .

Estimate ˆ A := ˆ A ( { µ j +1 ℓ , π j ℓ } j ∈ [ m -1] ,ℓ ∈ [ L ] ) := (1 /L ) ∑ L ℓ =1 ˆ A ℓ ( { µ j +1 ℓ , π j ℓ } j ∈ [ m -1] ) where

where E D z j ,ℓ [ · ] is an empirical average over samples D z j ,ℓ .

We now analyze the doubly robustness property of this estimator.

Proposition 1 ( Asymptotic Analysis of g-mSBD Estimators ) . Assume that the nuisance estimates µ i ℓ and π i ℓ are L 2 -consistent; i.e., ∥ µ i +1 ℓ -µ i +1 0 ∥ P σ ( Z i +1 ) = o P σ ( Z i +1 ) (1) , ∥ ˇ µ i +2 ℓ -ˇ µ i +2 0 ∥ P σ ( Z i +1 ) = o P σ ( Z i +1 ) (1) and ∥ π i ℓ -π i 0 ∥ P σ ( Z i +1 ) = o P σ ( Z i +1 ) (1) for i = 1 , · · · , m -1 , and ∥ ˇ µ 2 ℓ -ˇ µ 2 0 ∥ P σ ( Z 1 ) = o P σ ( Z 1 ) (1) . Let n i := ∣ D z i ∣ for i ∈ { 1 , · · · , m } . Then,

where R i is a random variable such that n 1 / 2 i R i converges in distribution to a mean-zero normal random variable.

This estimator possesses a doubly robustness property since the estimator is bounded in probability at rate n -1 / 2 (for n := min { n 1 , · · · , n m } , whenever O P σ ( Z i +1 ) ( ∥ µ i +1 ℓ -µ i +1 0 ∥∥ π i ℓ -π i 0 ∥ ) = O P ( n -1 / 2 i +1 ) for all i .

We now construct an estimator for the g-identification expression using the DR-g-mSBD estimator defined in Def. 3. The resulting estimator is called the MR-gID estimator:

Definition 4 ( MR-gID Estimator ) . The MR-gID estimator ˆ ψ for the identification expression of the causal effect ψ 0 := f ( { A k 0 } K k =1 ) in Theorem 1 is given as follows: For each A k 0 composing f ( { A k 0 } K k =1 ) , let ˆ A k := ˆ A k ( { µ j +1 k,ℓ , π j k,ℓ } j ∈ [ m k -1] ,ℓ ∈ [ L ] ) denote the DR-g-mSBD estimator with nuisance estimates { µ j +1 k,ℓ , π j k,ℓ } for the true nuisances { µ j +1 k, 0 , π j k, 0 } . Then,

We impose assumptions on the identification expression and its nuisances for further analysis.

Assumption 2 ( Analysis of MR-gID ) . The identification function f ( { A k } K k =1 ) in Thm. 1 and each nuisances { µ i +1 k,ℓ , π i k,ℓ } k,ℓ for ˆ A k satisfy the following properties:

Twice differentiability : f ( { A k } K k =1 ) is twice continuously Frechet differentiable w.r.t. { A k } K k =1 w.r.t. { A k } K k =1 .

Boundedness : ∀ k ∈ [ K ] and ∀ Z i ∈ Z , ∇ A k f ( { A j 0 } K j =1 )[ ˆ A k -A k 0 ] = O P σ ( Z i ) ( ˆ A k -A k 0 ) .

Assumption 2 is imposed to limit the error of the MR-gID, which is a linear function of the errors of each DR-g-mSBD estimator.

Theorem 2 ( Asymptotic Analysis of MR-gID ) . Suppose Assumption 2 holds. Let n k,i := | D z k i | for Z k i ∈ Z and z k i ∈ D Z k i . Let ˆ ψ denote the MR-gID estimator in Def. 4 for the causal effect ψ 0 := f ( { A k 0 } K k =1 ) in Theorem 1. Then, the error of ˆ ψ is given as

We highlight that the MR-gID ˆ ψ exhibits robustness property since ˆ ψ -ψ 0 for ψ 0 = P ( y | do ( x )) is bounded at rate n -1 / 2 (for n = min { n k,i } and P ∈ P ) even when all nuisances { µ i +1 k,ℓ , π i k,ℓ } are bounded at slower n -1 / 4 rate. Furthermore, the MR-gID estimator exhibits multiply robustness, as the error of the MR-gID is a linear function of the error of DR-g-mSBD, which demonstrates the doubly robustness property. Formally,

Corollary 2 ( Multiply Robustness (Corollary of Thm. 2)) . Suppose (1) Assumption 2 holds; (2) Either π i k,ℓ = π i k, 0 or µ j k,ℓ = µ j k, 0 for j = i + 1 , · · · , m k for all i, ℓ, k ; and (3) all nuisances { π i k,ℓ , µ i +1 k,ℓ } i,ℓ,k are bounded by some constant. Then, the MR-gID ˆ ψ (Def. 4) is consistent to ψ 0 .

For concreteness, we illustrate the application of Thm. 2 for Examples (1, 2). Detailed procedures are provided in Appendix A.

Example 5 ( Application of Thm. 2 to Example 1 ) . Recall that P ( y | do ( x )) = f ( { A 1 0 , A 2 0 } ) := ∑ z ∈ S Z A 1 0 A 2 0 . The nuisance set for A 1 0 is µ 2 1 , 0 ( X,W ) := E P [ /x31 z ( Z ) | X,W ] and π 1 1 , 0 ( X,W ) := /x31 x ( X ) /P ( X | W ) . Then, the estimator for A 1 0 is ˆ A 1 := ˆ A 1 ( { µ 2 1 ,ℓ , π 1 1 ,ℓ } ℓ ∈ [ L ] ) defined in Def. 3. The nuisance set for A 2 0 is µ 2 2 , 0 := E P σ ( Z ) [ /x31 y ( Y )] . Then, the estimator for A 2 0 is ˆ A 2 := ˆ A 2 ( { µ 2 2 ,ℓ } ℓ ∈ [ L ] ) . Then, the estimator is constructed as Def. 4, as f ( { ˆ A 1 , ˆ A 2 } ) . By Thm. 2, the error of the estimator is O P ( n -1 / 2 0 ) + O P ( n -1 / 2 z ) + (1 /L ) ∑ L ℓ =1 O P ( ∥ µ 2 1 ,ℓ -µ 2 1 , 0 ∥∥ π 1 1 ,ℓ -π 1 1 , 0 ∥ ) , where n 0 := | D | and n z := | D z | where D ∼ P and D z ∼ P z .

Example 6 ( Application of Thm. 2 to Example 2 ) . Recall that P ( y | do ( x 1 , x 2 )) = f ( { A 2 0 , A 13 0 } ) = ∑ r,w ∈ S R,W A 2 0 A 13 0 . The nuisance set for A 2 0 is µ 2 2 , 0 := E P x 1 [ /x31 w ( W ) | R,X 2 ] and π 1 1 , 0 := /x31 r ( R ) /P x 1 ( R | X 2 ) . Then, the estimator for A 2 0 is ˆ A 2 := ˆ A 2 ( { µ 2 2 ,ℓ , π 1 1 ,ℓ } ℓ ∈ [ L ] ) . The nuisance set for A 13 0 is µ 2 13 , 0 := E P x 2 [ /x31 r,y ( R,Y ) | R,W ] and π 1 13 , 0 := P σ ( X 1 ) ( R | x 2 ,x 1 ) P σ ( X 2 ) ( R | x 2 ) /x31 w ( W ) P σ ( X 2 ) ( W | R,x 2 ) . Then, the estimator is ˆ A 13 = ˆ A 13 ( { µ 2 13 , 0 , π 1 13 , 0 } ℓ ∈ [ L ] ) . Then, the estimator is constructed as Def. 4, as f ( { ˆ A 13 , ˆ A 2 } ) . By Thm. 2, the error of the estimator is O P σ ( X 1 ) ( n -1 / 2 1 ) + O P σ ( X 2 ) ( n -1 / 2 2 ) + (1 /L ) ∑ L ℓ =1 { O P σ ( X 1 ) ( ∥ µ 2 2 ,ℓ -µ 2 2 , 0 ∥∥ π 1 2 ,ℓ -π 1 2 , 0 ∥ ) + O P σ ( X 2 ) ( ∥ µ 2 13 ,ℓ -µ 2 13 , 0 ∥∥ π 1 13 ,ℓ -π 1 13 , 0 ∥ ) } , where n 1 := | D 1 | and n 2 := | D 2 | where D 1 ∼ P x 1 and D 2 ∼ P x 2 .

4 Experiments

In this section, we demonstrate the MR-gID estimator from Definition (4) through Examples (1,2) and Project STAR dataset[Krueger and Whitmore, 2001, Schanzenbach, 2006]. For each example, the proposed estimator is constructed using a dataset D := { D Z i , Z i ∈ Z } simulated from an underlying SCM. Our goal is to provide empirical evidence of the fast convergence behavior and the robustness property of the proposed estimator compared to competing baseline estimators. We consider two standard baselines in the literature: the 'regression-based estimator (reg)' only uses the

Fig. 1a

(Example 1)

Scenario 1

Fig. 1b

(Example 2)

Scenario 2

Scenario 3

Scenario 4

Model

Model

Model

Model

PW

REG

MR

0.5

REG

PW

MR

0.5

REG

MR

PW

0.5

REG

MR

PW

0.4

0.4

0.4

0.4

'0

0.3

0.3

0.3

0.2

0.1

200

400

600

800

1000

0.2

0.1

200

400

600

800

1000

0.2

0.1

200

400

009

800

1000

0.2

0.1

+

200

400

+

600

800

1000

Model

Model

Model

Model

0.25

REG

PW

MR

1.0

REG

PW

MR

REG

PW

MR

REG

PW

MR

0.3

0.3

0.2

0.8

0.24

0.24

0.15

0.6

0.18

0.18

S0'0

0.1

200

400

600

800

1000

0.2

0.4

200

白

400

600

800

1000

白

0.12

0.06

200

400

009

800

1000

0.12

0.06

200

400

600

800

1000

regression nuisance parameters µ 2 for µ 2 0 defined in Def. 2, and the 'probability weighting-based estimator (pw)' only uses the probability weighting parameters π m -1 for π m -1 0 defined in Def. 2, while our MR-gID uses both in estimating the g-mSBD operators A k composing f ( { A k } ) in Thm. 1. Details of the regression-based ('reg') and the probability weighting-based ('pw') estimators are provided in Appendix A. The details of the simulation are in Appendix (D, E).

4.1 Synthetic Dataset Analysis

Accuracy Measure. We compare the proposed estimator ('mr') in Def. 4 to the regression-based estimator ('reg') and the probability weighting-based estimator ('pw'). In particular, we use T est ( x ) for est ∈ { reg , pw , mr } to denote the g-ID estimators that leverage regression-based ('reg'), probability weighting-based ('pw'), and MR-gID in estimating each operator A k in the identification expression f ( { A k } ) of the causal effect P ( y | do ( x )) . We assess the quality of the estimators by computing the average absolute error AAE est := 1 | D X | ∑ x ∈ D X | T est ( x ) -P ( y | do ( x )) | where | D X | is the cardinality of D X . Nuisance functions are estimated using gradient boosting models called XGBoost [Chen and Guestrin, 2016]. We ran 100 simulations for each n = { 200 , 400 , 600 , 800 , 1000 } for n := | D Z | for ∀ Z ⊆ Z . We label the box-plot for these AAEs as 'AAE-plot'.

Experimental Setup. We evaluate the AAE est for Examples (1,2) in four scenarios:

(Scenario 1) There were no noises in estimating nuisances.

(Scenario 2) We introduced a converging noise ϵ in estimating the nuisance, decaying at a n -α rate (i.e., ϵ ∼ Normal ( n -α , N -2 α ) ) for α = 1 / 4 to emphasize the errors induced by the finiteness of samples. This scenario is inspired by the experimental design discussed in [Kennedy, 2020].

(Scenario 3) Nuisance { µ i +1 k,ℓ } ℓ,k,i are estimated incorrectly - simulated by training the model with a random matrix having the same dimension as the input matrix.

(Scenario 4) Nuisance { π i k,ℓ } ℓ,k,i are estimated incorrectly as in Scenario 3.

In Scenario 1, we aim to show that all estimators T reg , T pw , T mr are converging to the true causal quantity P ( y | do ( x )) . In Scenario 2, we aim to show that the MR-gID estimator exhibits fast convergence behavior compared to competing estimators. In Scenario (3,4), our goal is to highlight the multiply robustness property of the MR-gID estimator.

X

1

X

2

W

Y

R

(a) Graph

(b) Scenario 2

(c) Scenario 3

(d) Scenario 4

0.5

Model

0.5

Model

0.25

Model

REG

REG

REG

0.4

PW

0.4

PW

0.20

PW

MR

MR

MR

0.3

0.3

0.15

0.2

0.2

0.10

0.1

0.1

0.05

0.0

200

400

600

800

1000

0.0

0.00

Experimental Results. The AAE plots for all scenarios are presented in Fig. 2. All the estimators ('reg', 'pw', 'mr') converge in Scenario 1 as the sample size grows. In Scenario 2, where the estimated nuisances are controlled to be bounded in probability at n -1 / 4 rate, the proposed MRgID ˆ ψ outperforms the other two estimators by achieving fast convergence. This result corroborates the robustness property in Thm. 2. In Scenarios (3,4), where the estimated nuisances for { µ i } i m =2 or { π i } m -1 i =1 are wrongly specified, the MR-gID estimator converges while other estimators fail to converge. This result corroborates the multiply robustness property in Coro. 2.

4.2 Project STAR Dataset

This section provides an overview of the analysis using Project STAR dataset [Krueger and Whitmore, 2001, Schanzenbach, 2006]. Project STAR investigated the impact of teacher/student ratios on academic achievement for students in kindergarten through third grade. The dataset D includes class size ( X 1 ), the academic outcome in kindergarten ( W ) for kindergarten, the academic outcome in second grade ( R ), class size ( X 2 ), and the academic outcome for the third grade ( Y ). We assume that the SCM M underlying Project STAR dataset D can be depicted in Figure 3a. The target quantity is E [ Y | do ( x 1 , x 2 )] where P x 1 ,x 2 ( y ) = ∑ r ∈ D R P x 1 ( r ) ∑ x ′ 1 ,w ∈ D X 1 ,W P x 2 ( y | x ′ 1 , w, r, x 2 ) P x 2 ( x ′ 1 , w ) . The detailed procedures are in Appendix E.

Experimental Setup. We generate two datasets D 1 and D 2 from the original dataset D to demonstrate the gID estimation. D 1 is a random subsample of D with only { X 1 , W, R } and follows P σ ( X 1 ) ( X 1 , W, R ) . D 2 is constructed by resampling from D in a way that the confounding bias between X 1 and W , and X 1 and Y presents, following P σ ( X 2 ) ( X 1 , W, X 2 , R, Y ) . We conducted 100 simulations by generating new instances of D 1 and D 2 to create the AAE plot. Estimators were constructed solely from D 1 and D 2 , with D used exclusively to construct the ground-truth estimate.

Experimental Results. We evaluated the AAE est of estimators T est for est ∈ { reg , pw , mr } . The AAE plots for scenarios (2,3,4) are in Figs. (3b,3c,3d). Our findings indicate that the MR-gID estimator T mr consistently provided reliable estimates for the ground-truth quantity.

5 Conclusions

Wepresent a framework for estimating the causal effect P ( y | do ( x )) by combining multiple observational and experimental datasets and a causal graph G . We introduce the generalized multi-outcome sequential back-door adjustment (g-mSBD) operator (Def. 1) and its operations. We show that any g-identifiable causal effects can be expressed as a function of the g-mSBD operators as specified in Algo. 1 (Thm. 1). We then develop an estimator called DR-g-mSBD (Def. 3) for the g-mSBD operator and analyze its statistical properties in Prop. 1. Based on the DR-g-mSBD estimator, we develop the MR-gID estimator (Def. 4) and analyze its statistical properties (Thm. 2 and Coro. 2) which exhibits fast convergence and multiply-robustness. Our experimental results demonstrate that the MR-gID estimator is a consistent and robust estimator of P ( y | do ( x )) against model misspecification and slow convergence.

References

Ramzi A Ajjan and Peter J Grant. Cardiovascular disease prevention in patients with type 2 diabetes: The role of oral anti-diabetic agents. Diabetes and Vascular Disease Research , 3(3):147-158, 2006.

Heejung Bang and James M Robins. Doubly robust estimation in missing data and causal inference models. Biometrics , 61(4):962-973, 2005.

Elias Bareinboim and Judea Pearl. Causal inference by surrogate experiments: z-identifiability. In In Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence , pages 113-120. AUAI Press, 2012.

Elias Bareinboim and Judea Pearl. Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences , 113(27):7345-7352, 2016.

Elias Bareinboim, Juan D Correa, Duligur Ibeling, and Thomas Icard. On pearls hierarchy and the foundations of causal inference. In Probabilistic and causal inference: the works of judea pearl , pages 507-556. 2022.

Rohit Bhattacharya, Razieh Nabi, and Ilya Shpitser. Semiparametric inference for causal effects in graphical models with hidden variables. Journal of Machine Learning Research , 23:1-76, 2022.

Peter J Bickel, Chris AJ Klaassen, Peter J Bickel, Yaacov Ritov, J Klaassen, Jon A Wellner, and YA'Acov Ritov. Efficient and adaptive estimation for semiparametric models , volume 4. Johns Hopkins University Press Baltimore, 1993.

Philippe Blanchard and Erwin Brüning. Mathematical methods in Physics: Distributions, Hilbert space operators, variational methods, and applications in quantum physics , volume 69. Birkhäuser, 2015.

Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 785-794, 2016.

Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters: Double/debiased machine learning. The Econometrics Journal , 21(1), 2018.

Victor Chernozhukov, Juan Carlos Escanciano, Hidehiko Ichimura, Whitney K Newey, and James M Robins. Locally robust semiparametric estimation. Econometrica , 90(4):1501-1535, 2022.

Richard K Crump, V Joseph Hotz, Guido W Imbens, and Oscar A Mitnik. Dealing with limited overlap in estimation of average treatment effects. Biometrika , 96(1):187-199, 2009.

Iván Díaz, Nicholas Williams, Katherine L Hoffman, and Edward J Schenck. Nonparametric causal effects based on longitudinal modified treatment policies. Journal of the American Statistical Association , pages 1-16, 2021.

Ele Ferrannini and William C Cushman. Diabetes and hypertension: the bad companions. The Lancet , 380(9841):601-610, 2012.

Amanda M Gentzel, Purva Pruthi, and David Jensen. How and why to use experimental data to evaluate methods for observational causal inference. In International Conference on Machine Learning , pages 3660-3671. PMLR, 2021.

Adam N Glynn and Konstantin Kashin. Front-door difference-in-differences estimators. American Journal of Political Science , 61(4):989-1002, 2017.

Lennart Hansson, Lars H Lindholm, Tord Ekbom, Björn Dahlöf, Jan Lanke, Bengt Scherstén, PO Wester, Thomas Hedner, Ulf de Faire, STOP-Hypertension-2 Study Group, et al. Randomised trial of old and new antihypertensive drugs in elderly patients: cardiovascular mortality and morbidity the swedish trial in old patients with hypertension-2 study. The Lancet , 354(9192):17511756, 1999.

Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics , 20(1):217-240, 2011.

Yimin Huang and Marco Valtorta. Identifiability in causal bayesian networks: A sound and complete algorithm. In Proceedings of the national conference on artificial intelligence , volume 21, page 1149. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2006.

Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating causal effects using weighting-based estimators. In Proceedings of the 34th AAAI Conference on Artificial Intelligence , 2020a.

Yonghan Jung, Jin Tian, and Elias Bareinboim. Learning causal effects via weighted empirical risk minimization. Advances in Neural Information Processing Systems , 33, 2020b.

Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating identifiable causal effects on markov equivalence class through double machine learning. In Proceedings of the 38th International Conference on Machine Learning , 2021a.

Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating identifiable causal effects through double machine learning. In Proceedings of the 35th AAAI Conference on Artificial Intelligence , 2021b.

Yonghan Jung, Jin Tian, and Elias Bareinboim. Estimating joint treatment effects by combining multiple experiments. In Proceedings of the 40th International Conference on Machine Learning , 2023. URL https://proceedings.mlr.press/v202/jung23c.html .

Edward H Kennedy. Optimal doubly robust estimation of heterogeneous causal effects. arXiv preprint arXiv:2004.14497 , 2020.

Edward H Kennedy, Sivaraman Balakrishnan, Max GSell, et al. Sharp instruments for classifying compliers and generalizing causal effects. Annals of Statistics , 48(4):2008-2030, 2020.

Alan B Krueger and Diane M Whitmore. The effect of attending a small class in the early grades on college-test taking and middle school test results: Evidence from project star. The Economic Journal , 111(468):1-28, 2001.

R Kumar, DM Kerins, and T Walther. Cardiovascular safety of anti-diabetic drugs. European Heart Journal-Cardiovascular Pharmacotherapy , 2(1):32-43, 2016.

Robert J LaLonde. Evaluating the econometric evaluations of training programs with experimental data. The American economic review , pages 604-620, 1986.

Sanghack Lee, Juan D Correa, and Elias Bareinboim. General identifiability with arbitrary surrogate experiments. In Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence . AUAI Press, 2019.

Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. Advances in neural information processing systems , 30, 2017.

Alexander R Luedtke, Oleg Sofrygin, Mark J van der Laan, and Marco Carone. Sequential double robustness in right-censored longitudinal models. arXiv preprint arXiv:1705.02459 , 2017.

R v Mises. On the asymptotic distribution of differentiable statistical functions. The annals of mathematical statistics , 18(3):309-348, 1947.

Pearl. Causal diagrams for empirical research. Biometrika , 82(4):669-710, 1995.

Judea Pearl. Causality: Models, Reasoning, and Inference . Cambridge University Press, New York, 2000. 2nd edition, 2009.

Judea Pearl and James Robins. Probabilistic evaluation of sequential plans from causal models with hidden variables. In Proceedings of the Eleventh conference on Uncertainty in artificial intelligence , pages 444-453, 1995.

James Robins. A new approach to causal inference in mortality studies with a sustained exposure periodapplication to control of the healthy worker survivor effect. Mathematical modelling , 7 (9-12):1393-1512, 1986.

James Robins, Lingling Li, Eric Tchetgen, and Aad W van der Vaart. Quadratic semiparametric von mises calculus. Metrika , 69:227-247, 2009.

James M Robins and Andrea Rotnitzky. Semiparametric efficiency in multivariate regression models with missing data. Journal of the American Statistical Association , 90(429):122-129, 1995.

Andrea Rotnitzky, James Robins, and Lucia Babino. On the multiply robust estimation of the mean of the g-functional. arXiv preprint arXiv:1705.08582 , 2017.

Andrea Rotnitzky, Ezequiel Smucler, and James M Robins. Characterization of parameters with a mixed bias property. Biometrika , 108(1):231-238, 2021.

Diane Whitmore Schanzenbach. What have researchers learned from project star? Brookings papers on education policy , (9):205-228, 2006.

James H Stock, Mark W Watson, et al. Introduction to econometrics , volume 104. Addison Wesley Boston, 2003.

Jin Tian and Judea Pearl. On the identification of causal effects. Technical Report R-290-L, 2003.

Mark J van der Laan and Susan Gruber. Targeted minimum loss based estimation of causal effects of multiple time point interventions. The international journal of biostatistics , 8(1), 2012.

Aad W Van der Vaart. Asymptotic statistics , volume 3. Cambridge university press, 2000.

Stijn Vansteelandt, Andrea Rotnitzky, and James Robins. Estimation of regression models for the mean of repeated outcomes under nonignorable nonmonotone nonresponse. Biometrika , 94(4): 841-860, 2007.

Xia, Y. Pan, and E. Bareinboim. Neural causal models for counterfactual identification and estimation. In The 11th International Conference on Learning Representations , Feb 2023.

Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural connection: Expressiveness, learnability, and inference. Advances in Neural Information Processing Systems , 34, 2021.

Junzhe Zhang and Elias Bareinboim. Near-optimal reinforcement learning in dynamic treatment regimes. Advances in Neural Information Processing Systems , 32, 2019.

Supplement to 'Estimating Causal Effects Identifiable from a Combination of Observations and Experiments'

Contents

A Further Details

We restate the notation here. To clarify the relationship between the experimental samples, where randomization is applied to Z ⊆ V , and the distribution P z ( V \ z ) , we introduce the notation P σ ( Z ) ( V ) , where σ ( Z ) indicates that Z has been randomized. The distribution P σ ( Z ) ( V ) is derived from the Structural Causal Model (SCM), where the original equation Z ← f Z ( pa z , u z ) for Z ∈ Z is replaced by a function that assigns a value to Z = z randomly, independent of other endogenous variables. For example, assigning Z = 1 and 0 with a probability of 0.5 each.

It should be noted that when considering observational data, P := P σ ( ∅ ) . For any sets A , B , and Z ⊆ V , the interventional distribution can be represented as P ( A | do ( z ) , B ) = P σ ( Z ) ( A | Z = z , B ) according to the definition of the do-operator and the P σ ( Z ) distribution. To emphasize that the distribution is induced from randomization and conditioning on Z = z , we use P z ( A | B ) := P σ ( Z ) ( A | Z = z , B ) . The experimental samples obtained from randomization σ ( Z ) lead to samples D σ ( Z ) that follow P σ ( Z ) ( V ) . We denote the subsample of D σ ( Z ) , where Z = z is fixed, as D z , which follows P z ( V ) . ■

A.1 Example 3

We provide a detailed illustration of Example 3, demonstrating the application of Lemmas (2,3,4).

Input : x = { x } , y := { y } , Z := {∅ , Z } . The goal is to identify P ( y | do ( x )) from P which contains P and P σ ( Z ) ( V ) . In the identification, P ( V ) and P z ( V \ Z ) := P σ ( Z ) ( V | z ) for z ∈ D Z are used.

We now run Line 5-13 . We first run D 1 = Z and Z 1 =

\ { } { } { } { } { } ∅ . Then,

Line 7 : The c-component S 1 1 = V = { W,X,Z,Y } includes D 1 .

Line 8 : The c-factor Q [ S 1 1 ] is identified as

Line 9 : Run Q [ D 1 ] = SUBID ( D 1 , S 1 1 , Q [ S 1 1 ] , G ( S 1 1 )) .

(a) Line a.(2-3): A = an ( Z ) G ( V ) = { W,X,Z } . Then, by Lemma 2,

(b) Line a.(7-8): Note S = { W,Z } is a c-component in G ( A ) containing D 1 = { Z } . Then,

because the order is W ≺ G X ≺ G Z .

(c) Line a.9: Run Q [ D 1 ] = SUBID ( D 1 , S , Q [ S ] , G ( S )) .

(d) Line a.(2-3): A = an ( D 1 ) G ( S ) = { Z } = D 1 . Then, by Lemma 2,

We now run Line 5-13 . We first run D 2 = { Y } and Z 1 = ∅ . We note that it fails since the sub-procedure subID ( D 2 , S 1 1 , Q [ S 1 1 ] , G ( S 1 1 )) fails. Specifically, A := an ( D 2 ) G ( S 1 1 ) = V = S 1 1 . Therefore, by Line a.5 , the procedure fails.

We now run D 2 with Z 2 = { Z } .

Line 7 : The c-component S 1 2 = V \ Z = { W,X,Y } .

Line 8 : Q [ S 1 2 ] = A 0 [ S 1 2 , ∅ , ∅ , ; Z 1 2 = { Z } , seq 1 2 ]( s 1 2 , ∅ ) = P z ( w ) P z ( x | w ) P z ( y | x, w ) , where seq 1 2 ( W j ) = ( z, z, z ) .

Line 9 : We run Q [ D 2 ] = SUBID ( D 2 , S 1 2 , Q [ S 1 2 ] , G ( S 1 2 )) .

Line a.(2-3): A = an ( D 2 ) G ( S 1 2 ) = { Y } = D 2 . Then, by Lemma 2

Let

By Line 14 ,

A.2 Example 4

We provide a detailed illustration of Example 4, demonstrating the application of Lemmas (2,3,4).

Input : x = { x 1 , x 2 } , y := { y } , Z := { X 1 , X 2 } . The goal is to identify P ( y | do ( x 1 , x 2 )) from P := { P σ ( X 1 ) ( V ) , P σ ( X 2 ) ( V ) } . Specifically, two distributions P x 1 ( V \ X 1 ) and P x 2 ( V \ X 2 ) will be used in the identification task.

Line 5-13 : Consider D 1 = { R } and Z 1 := { X 1 } . Note S 1 1 := { R } = D 1 is a c-component in G ( V \ X 1 ) . Therefore, Q [ D 1 ] = Q [ S 1 1 ] , where

where seq 1 1 := ( x 1 ) .

Line 5-13 : We now consider D 3 = { Y } . Note that Q [ D 3 ] is not identifiable from P x 1 ( V \ X 1 ) . To witness, consider the c-component S 1 3 := { W,X 2 , Y } in G ( V \ X 1 ) . Then, the sub-procedure subID ( D 3 , S 1 3 , Q [ S 1 3 ] , G ( S 1 3 )) fails because failure condition in line a.5 is triggered. Specifically, an ( Y ) G ( S 1 3 ) = S 1 3 .

Therefore, we consider D 3 = { Y } with Z 3 := x 2 . Note S 2 3 := { Y } = D 3 is a c-component in G ( V \ X 2 ) . Therefore, Q [ D 3 ] = Q [ S 2 3 ] which is given by line 8:

Line 5-13 : Consider D 2 = { W } and Z 1 := X 1 . Note that Q [ D 2 ] is not identifiable from P x 2 ( V \ X 2 ) . To witness, consider the c-component S 2 2 := { X 1 , W } in G ( V \ X 2 ) . Then, the sub-procedure subID ( D 2 , S 2 2 , Q [ S 2 2 ] , G ( S 2 2 )) fails because failure condition in line a.5 is triggered. Specifically, an ( W ) G ( S 2 2 ) = S 2 2 .

Therefore, we consider D 2 = { W } with Z 1 := X 1 .

Run Q [ D 2 ] = SUBID ( D 2 , S 1 2 , Q [ S 1 2 ] , G ( S 1 2 )) .

Line a.(2-3): A = an ( W ) G ( S 1 2 ) = { W } = D 2 . Then,

Finally,

A.3 Example 5

A.3.1 Specification of Nuisances

Recall that the topological order of the variable is W ≺ G X ≺ G Z ≺ G Y . Also, P ( y | do ( x )) = ∑ z ∈ S Z A 1 0 A 2 0 where

That is,

By leveraging the definition of the nuisance in Def. 2, the nuisance composing A 1 0 is { µ 2 1 , 0 , π 1 1 , 0 } which are defined as follow:

The nuisance composing A 2 0 is { µ 2 2 , 0 } which is µ 2 2 , 0 := E P z [ /x31 y ( Y )] .

A.3.2 Construction of Estimators

We apply the procedure in Def. 3 to construct estimators ˆ A 1 and ˆ A 2 for A 1 0 and A 2 0 . We choose L = 2 . We first construct ˆ A 1 for the fixed { z, x } ∈ D Z,X . We note that ˆ A 1 ℓ for ℓ ∈ { 1 , 2 } is given as follow: For a fixed z, x , and

where π 1 , µ 2 are nuisances estimated using D \ D ℓ . Specifically, µ 2 1 ,ℓ ( X,W ) is obtained by using the XGBoost [Chen and Guestrin, 2016] regression model which regresses /x31 z ( Z ) onto the { X,W } using D \ D ℓ . µ 2 1 ,ℓ ( x, W ) is evaluated from D ℓ after fixing a column for X to x . In similar, π 1 as follow: we first model P ( X | W ) by regressing X onto W from the data D \ D ℓ using the XGBoost [Chen and Guestrin, 2016]. Then, we evaluate π 1 1 , 0 ( X,W ) by plugging in the trained P ( X | W ) .

We now construct ˆ A 2 for the fixed z and y . We first take the subsamples D z from the experimental samples D σ ( Z ) ∈ D , where D z is the sample where Z = z . Then, we compute the following:

Then, following Def. 4, the MR-gID is constructed as follow:

A.4 Example 6

A.4.1 Specification of Nuisances

Recall that the topological order of the variable is X 1 ≺ G X 2 ≺ G R ≺ G W ≺ G Y . Also,

Then,

The nuisance composing A 2 0 is { µ 2 2 , 0 , π 1 2 , 0 } which are defined as follow:

The nuisance composing A 13 0 is { µ 2 2 , 0 , π 1 2 , 0 } which are defined as follow:

and

A.4.2 Construction of Estimators

We apply the procedure in Def. 3 to construct estimators ˆ A 2 and ˆ A 13 for A 2 0 and A 13 0 . We choose L = 2 . We first construct ˆ A 2 for the fixed { w,r, x 1 } . We note that ˆ A 2 := (1 /L ) ∑ L ℓ =1 ˆ A 2 ℓ for ℓ ∈ { 1 , 2 } where ˆ A 2 ℓ is given as follow:

where D x 1 is a subsample of D σ ( X 1 ) fixing X 1 = x 1 , and π 1 2 ,ℓ , µ 2 2 ,ℓ are nuisances trained using D x 1 \ D x 1 ,ℓ . We note that µ 2 2 ,ℓ ( R,X 2 ) is constructed by regressing /x31 w ( W ) onto { R,X 2 } . Also, π 1 2 ,ℓ ( R,X 2 ) is constructed by regressing R onto X 2 .

We now construct ˆ A 13 := (1 /L ) ∑ L ℓ =1 ˆ A 13 ℓ for ℓ ∈ { 1 , 2 } where ˆ A 13 ℓ is given as follow:

where D x 1 is a subsample of D σ ( X 1 ) fixing X 1 = x 1 , and D x 1 is a subsample of D x 1 fixing X 2 = x 2 . π 1 2 ,ℓ , µ 2 2 ,ℓ are nuisances trained using D x 1 \ D x 1 ,ℓ and D x 2 \ D x 2 ,ℓ .

A.5 Details on Regression-based (REG) and Probability Weighting-based (PW) estimators.

In this section, we provide details on two alternative g-ID estimators used in Sec. 4: T reg := f ( { ˆ A k, reg } K k =1 } ) ('regression-based estimators') where ˆ A k, reg denotes the regression-based estima-

tor for the g-mSBD operator, and T pw = f ( { ˆ A k, pw } K k =1 } ) ('probability weighting-based estimators) where ˆ A k, reg denotes the probability weighting-based estimator for the g-mSBD operator.

A.5.1 Regression-based Estimator

The regression-based g-mSBD estimator is defined as follows:

Definition A.1 ( Regression-based g-mSBD Estimator ) . Let D σ ( Z i ) for Z i ∈ Z denote the experimental samples from randomizing the variable Z i . Let D z i for z i ∈ D Z i denote the subsamples of D σ ( Z i ) fixing R 0 \ Z i = r 0 \ z i and Z i = z i . A regression-based estimator ˆ A reg for the g-mSBD adjustment A 0 [ W , C , R ; Z 0 := { Z i } i m =1 , seq := ( z i ) i m =1 ]( w \ c , r ) is given as follows:

Randomly partition D z i into { D z i ,ℓ } ℓ ∈ [ L ] ; i.e., D z i = ∪ L ℓ =1 D z i ,ℓ , ∀ Z i ∈ Z and z i ∈ D Z i .

For each fold ℓ ∈ [ L ] , let µ i +1 ℓ denote learned µ i +1 0 using D z i +1 \ D z i +1 ,ℓ for i = m, · · · , 2 . Define ˇ µ i +1 ℓ := µ i +1 ℓ ( W i , r i , R 1: i -1 ) .

The error of the regression-based estimator is given as follows:

Proposition A.1 ( Error Analysis of the regression-based g-mSBD estimator ) . Suppose ∥ µ 2 ℓ -µ 2 0 ∥ P σ ( Z 1 ) = o P σ ( Z 1 ) (1) . Then,

Proof of Proposition A.1 . We note that by the analysis in Lemma S.2. Therefore, by Lemma S.7,

By the central limit theorem,

where n 1 ,ℓ := | D z 1 ,ℓ | .

By [Kennedy et al., 2020, Lemma 2] and the given assumption that ∥ µ 2 ℓ -µ 2 0 ∥ P σ ( Z 1 ) = o P σ ( Z 1 ) (1) ,

Finally, by applying Cauchy-Schwarz inequality,

Finally,

A.5.2 Probability-weighting based Estimator

In this section, we define and analyze the probability weighting-based g-mSBD estimator. The probability-weighting-based estimator is defined as follows:

Definition A.2 ( Probability-weighting-based g-mSBD Estimator ) . Let D σ ( Z i ) for Z i ∈ Z denote the experimental samples from randomizing the variable Z i . Let D z i for z i ∈ D Z i denote the subsamples of D σ ( Z i ) fixing R 0 \ Z i = r 0 \ z i and Z i = z i . Aprobability weighting-based estimator ˆ A pw for the g-mSBD adjustment A 0 [ W , C , R ; Z 0 := { Z i } i m =1 , seq := ( z i ) i m =1 ]( w \ c , r ) is given as follows:

Randomly partition D z i into { D z i ,ℓ } ℓ ∈ [ L ] ; i.e., D z i = ∪ L ℓ =1 D z i ,ℓ , ∀ Z i ∈ Z and z i ∈ D Z i .

For each fold ℓ ∈ [ L ] , let π i ℓ denote learned π i 0 using D z i \ D z i ,ℓ for i = m -1 , · · · , 1 . Let π m -1 ℓ := ∏ m -1 i =1 π i ℓ .

Lemma S.1 ( Representation of the g-mSBD operator using Probability Weighting ) . The gmSBD adjustment A 0 in Def. 1 can be represented as

Proof of Lemma S.1 . It suffices to show that, for k = m -1 , · · · , 1 ,

If this holds, Lemma S.1 can be shown as follows:

Eq. (A.57) holds as follows:

This completes the proof.

Equipped with Lemma S.1, we analyze the error of the probability-weighting-based estimator as follow:

Proposition A.2 ( Error Analysis of the probability weighting-based g-mSBD estimator ) . Suppose ∥{ π 1 ℓ τ 2 ℓ -π 1 0 τ 2 0 }∥ P σ ( Z 2 ) = o P σ ( Z 2 ) (1) . Then,

Proof of Proposition A.2 . By Lemma S.7 and Assumption 1,

By the central limit theorem,

By [Kennedy et al., 2020, Lemma 2] and the given assumption,

Finally, by applying Cauchy-Schwarz inequality,

This completes the proof.

B Proofs

B.1 Proof of Lemma 1

Lemma 1 ( c-component Identification [Jung et al., 2021b]) . Let S denote a c-component in G i := G ( V \ Z i ) for some Z i ∈ Z . Let R := pa ( S ) G i \ S . Let ( S , R ) be ordered as ( R 0 , S 1 , · · · , R m -1 , S m ) by ≺ G . Let A ⊆ S denote a set satisfying A = an ( A ) G i ( S ) . Let C := ( S \ A ) . Let Z 0 := { Z i } and seq ( Z 0 ) be a sequence of z i repeating m times. Then, the c-factor Q [ A ] is g-identifiable as follows:

Proof of Lemma 1 . Let C 0 := pre ( A ; G ( S )) ∩ C . Let C 1 := C \ C 0 . We first note that, by [Jung et al., 2021b, Lemma 1],

Therefore, it suffices to show that

It holds as follows:

We note that the third equation holds since the all P z i ( v j | s j -1 , r j -1 \ z i ) will be marginalized out if V j ∈ C 1 . The fourth equation holds since S := A · ∪ C 0 · ∪ C 1 , which implies that S \ C 1 = A ∪ C 0 .

B.2 Proof of Lemma 2

Lemma 2 ( Marginalization ) . Let A 0 [ W , C , R ; Z 0 , seq ]( w \ c , r ) denote the g-mSBD operator in Def. 1. Let W 0 ⊆ W \ C . Let W mar ⊆ { W 0 , C } denote the vector formed by the following procedure: Starting from W mar = ∅ , for j = m, · · · , 1 , W mar = W mar ∪ { W j } if (1) W j ∈ { W 0 , C } and (2) ∃ k ∈ { j, · · · , m } such that R j , · · · , R k -1 = ∅ , W k +1: m ⊆ W mar, and Z k = · · · = Z j and z k = · · · = z j . Let W ′ := W \ W mar, R ′ := pre ( W ′ ; G ) ∩ R and C ′ := { W 0 , C }\ W mar. Let Z ′ ⊆ Z 0 denote the collection of Z i corresponding to the variable in W ′ , and seq ′ the corresponding sequence. Then,

B.3 Proof of Lemma 3

Lemma 3 ( Multiplication ) . Let A i 0 := A 0 [ W i , ∅ , R i ; Z i , seq i ]( w i , r i ) := ∏ m i j =1 P z i j ( w i,j | w j -1 i , r j -1 i \ z i j ) for i ∈ { 1 , 2 } where seq i := ( z i j ) m i j =1 . Let W := W 1 ∪ W 2 . Let

R := ( R 1 ∪ R 2 ) \ W . Let ( W , R ) be ordered by ≺ G . Let Z := Z 1 ∪ Z 2 . Assume the following: (1) W 1 ∩ W 2 = ∅ ; and (2) ∀ W j ∈ W , ∃ W i,k ∈ W i such that ( W j -1 , R j -1 ) = ( W i k -1 , R i k -1 ) . Let seq := ( z j ) j : W j ∈ W where z j = z i k for all j . Then,

B.4 Proof of Lemma 4

Lemma 4 ( Division ) . Let A i 0 := A 0 [ W i , ∅ , R i ; Z i , seq i ]( w i , r i ) := ∏ m i j =1 P z i j ( w i,j | w j -1 i , r j -1 i \ z i j ) for i ∈ { 1 , 2 } where seq i := ( z i j ) m i j =1 . Let W := W 1 \ W 2 . Let R := ( R 1 ∪ W 2 ) ∩ pre ( W ; G ) . Assume the following: (1) W 2 ⊆ W 1 ; and (2) ∀ W j ∈ W , ∃ W 1 ,k ∈ W 1 such that ( W j -1 , R j -1 ) = ( W 1 k -1 , R 1 k -1 ) , Z i,k = Z j and z i,k = z j . Then,

Proof of Lemma 4 .

We note that W 1 k -1 ∩ W = W j -1 for some W j ∈ W s.t. W j = W 1 ,k . Also, ( W 1 \ W ) ∪ R 1 = W 2 ∪ R 1 . Therefore, ∪ k : W k ∈ W 1 \ W 2 { W 1 k -1 \ W , R 1 k -1 } = R := ( R 1 ∪ W 2 ) ∩ pre ( W ; G ) . Therefore,

B.5 Proof of Theorem 1

Theorem 1 ( Expression of g-Identifiable Causal Effects ) . Algo. 1 returns any g-identifiable causal effects as a function of a set { A k 0 } of g-mSBD adjustment operators in the form

where the function f ( · ) applies marginalization, multiplication, or division over g-mSBD operators in { A k 0 } as specified by Algo. 1.

Proof of Theorem 1 . Throughout the proof, we refer to the algorithm developed in [Lee et al., 2019, Algo. 1] as the 'standard gID' algorithm, in comparison to our gID algorithm presented in Algo. 1. It is established that the standard gID algorithm is sound, as stated in [Lee et al., 2019, Theorem 2]. This means that if the algorithm returns an identification expression, it must be correct. Furthermore, the standard gID algorithm is proven to be complete [Lee et al., 2019, Theorem 3]. In other words, the causal effect P ( y | do ( x )) is identifiable from P and the causal graph G if and only if the standard gID algorithm does not return FAIL.

In our proof, we will show the soundness and completeness of Algo. 1 based on the foundation provided by the standard gID algorithm.

Algo. 1 is sound - If Algo. 1 returns an expression f ( { A k 0 } K k =1 ) , then it holds that f ( { A k 0 } K k =1 ) = P ( y | do ( x )) . The soundness of Algo. 1 is derived from the soundness of Tian's c-factor operation, as demonstrated in [Tian and Pearl, 2003, Lemmas (3,4)] and Lemma 1.

We will now show that Algo. 1 is complete. Suppose there exists an input ( x , y , Z , P , G ) for which the standard gID algorithm does not return FAIL while Algo. 1 does return FAIL. This implies the existence of D j such that Q [ D j ] is not identifiable from all Q [ S i j ] where D j is a c-component in G ( D ) , and S i j is the c-component in G ( V \ Z i ) that contains D j . This observation is a consequence of the soundness and completeness of the SUBID procedure, as established in [Huang and Valtorta, 2006, Theorem 1].

It should be noted that Q [ D j ] is not identifiable from S i j for all { i : Z i ∈ Z } only when there exists a c-component T i j in G ( S i j ) that serves as an ancestral set of D j and includes D j . However, in such a scenario, the standard gID algorithm fails due to lines 12 and 13 of the algorithm. This contradicts the initial assumption that the standard gID algorithm does not return FAIL. Consequently, Algo. 1 returns FAIL whenever the standard gID algorithm does so. The completeness of the standard gID algorithm implies that Algo. 1 is complete in the g-identification task.

The fact that f ( · ) is a function involving marginalization, multiplications, and divisions of g-mSBD (generalized modified single back-door) operators is a consequence of applying Lemmas (2, 3, 4) within the algorithm. These lemmas establish the properties and operations of the g-mSBD operators, which are then utilized in the construction of f ( · ) in Algo. 1.

B.6 Proof of Proposition 1

We first restate the g-mSBD adjustment, its nuisances, and the g-mSBD estimator here:

Definition 1 ( generalized-mSBD adjustment (g-mSBD) ) . Let ( W , R ) be a disjoint pair in V topologically ordered as ( W , R ) = { R 0 , W 1 , · · · , R m -1 , W m , R m } by ≺ G , where R i can be empty. Let W i -1 := { W j } i -1 j =1 and R i -1 := { R j } i -1 j =0 for ∀ i ∈ [ m ] . Let C ⊆ W . Let Z 0 ⊆ Z be some set such that ∀ Z ∈ Z 0 , W ∩ Z = ∅ . Let seq ( Z 0 ) denote a sequence ( z 1 , · · · , z m ) where z i denotes some realization of Z i ∈ Z 0 (same z i could appear multiple times in the sequence). Then, the g-mSBD adjustment is expressed as an operator A 0 [ W , C , R ; Z 0 , seq , G ]( w \ c , r ) defined by

Definition 2 ( Nuisances for g-mSBD ) . Nuisances for g-mSBD A 0 in Eq. (1) are { µ i +1 0 , π i 0 } m -1 i =1 defined as follows. Let µ m +1 0 = µ m +1 := /x31 w \ c ( W \ C ) . For i = m -1 , · · · , 1 ,

Definition 3 ( DR-g-mSBD Estimators ) . Let D σ ( Z i ) for Z i ∈ Z denote the experimental samples from randomizing the variable Z i . Let D z i for z i ∈ D Z i denote the subsamples of D σ ( Z i ) fixing R 0 \ Z i = r 0 \ z i and Z i = z i . The DR-g-mSBD estimator ˆ A for the g-mSBD adjustment A 0 [ W , C , R ; Z 0 := { Z i } i m =1 , seq := ( z i ) i m =1 ]( w \ c , r ) is defined as follows:

Randomly partition D z i into { D z i ,ℓ } ℓ ∈ [ L ] ; i.e., D z i = ∪ L ℓ =1 D z i ,ℓ , ∀ Z i ∈ Z and z i ∈ D Z i .

/L

For each fold ℓ ∈ [ L ] , let µ i +1 ℓ denote learned µ i +1 0 using D z i +1 \ D z i +1 ,ℓ for i = m, · · · , 2 ; and π i ℓ learned π i 0 for i = 1 , · · · , m -1 . Define ˇ µ i +1 ℓ := µ i +1 ℓ ( W i , r i , R 1: i -1 ) and π i ℓ := ∏ i j =1 π j ℓ . 3. Estimate ˆ A := ˆ A ( { µ j +1 ℓ , π j ℓ } j ∈ [ m -1] ,ℓ ∈ [ L ] ) := (1 ∑ ˆ A ℓ := ˆ A ℓ ( { µ j +1 ℓ , π j ℓ } j ∈ [ m -1] ) := m -1 ∑ j =1 E D z j +1 ,ℓ [ π j ℓ { ˇ µ j +2 ℓ -µ j +1 ℓ } ] + E D z 1 ,ℓ [ ˇ µ 2 ℓ ] , (9)

)

L

ℓ

=1

where E D z j ,ℓ [ · ] is an empirical average over samples D z j ,ℓ .

We analyze the bias of the g-mSBD estimator using the following results:

Lemma S.2 ( Representation of g-mSBD ) . The g-mSBD adjustment A 0 in Def. 1 can be represented as

where ˇ µ i +1 ℓ ( W i , R 1: i -1 ) := µ i +1 ℓ ( W i , r i , R 1: i -1 ) and π i := ∏ i j =1 π j as defined in Def. 3.

Proof of Lemma S.2 . Throughout the proof, we will use w ′ \ c ′ as some realization of W \ C . Recall that /x31 w \ c ( w ′ \ c ′ ) = 1 when w ′ \ c ′ = w \ c and zero otherwise. We first recall that

by the definition of the experimental distribution P σ ( Z i ) .

For all i = 1 , · · · , m -1 ,

where the equation 1 = holds by the total law of expectation, and 2 = holds by the definition of ˇ µ i +1 0 .

ˆ

A

(

{

µ

j

ℓ

+1

, π

j

ℓ

}

)

where

ℓ

j

∈

[

m

-

1]

It suffices to show that

To prove the equation, we show that, for all k = m,m -1 , · · · , 2 ,

This equation holds when k = m , because

For k = m -1 ,

Based on this observation, we make the following induction hypothesis: Suppose, for a fixed k ∈ { 2 , · · · , m } , the following holds:

Then, the induction hypothesis holds for k -1 as follows:

:=

E

P

σ

(

Z

k

+1 )

ˇ

µ

(

W

,

R

,

r

k

)

|

W

,

R

,

z

k

+1

,

r

0

(B.40)

[

]

Also, we already checked that the induction hypothesis holds for k = m . Therefore, the hypothesis holds for all k = 2 , · · · , m :

k

+1

k

1:

k

-

1

k

-

1

1:

k

-

1

Then,

Then,

Lemma S.3 ( Bias Analysis of g-mSBD Estimators (1) ) . Let A be the quantity defined as

Proof of Lemma S.3 . We first rewrite Eq. (B.51) as follows:

Also,

Therefore,

Lemma S.4 ( Bias Analysis of g-mSBD Estimators (2) ) . Let A be the quantity defined as

Then, for k = 3 , · · · , m -1 ,

Proof of Lemma S.4 . The equation holds for k = m -1 . It can be shown as follows:

Then,

Eq. (B.58)

Therefore,

Finally,

For any fixed k +1 ∈ { m -1 , · · · , 4 } , suppose the following holds:

We note that this holds when k = m -2 , as shown in Eq. (B.61). We will now show that it will hold for k , too. First,

Therefore,

Therefore, the equation holds for k , too. This completes the proof.

Proof of Lemma S.5 . Note that

Lemma S.6 ( Bias Analysis of g-mSBD Estimators ) . Let A be the quantity defined as

where π i , µ i are arbitrary nuisances for true nuisances π i 0 and µ i 0 defined in Def. 2. Let A 0 denote the g-mSBD in Def. 1. Then,

Proof of Lemma S.6 . By Lemmas (S.3,S.4,S.5).

We also use the following results which are used by Kennedy et al. [2020].

Lemma S.7 ( Decomposition ) . Let D ∼ P denote a finite sample set following a distribution P . Let h ( V ; η ) denote an arbitrary random function taking η as a nuisance. For any η, η 0 ,

Proof of Lemma S.7 .

Lemma S.8 ( Continuous Mapping Theorem for L 2 ( P ) ) . Let X n , X denote a random sequence defined on a metric space S . Suppose a function g : S → S ′ (where S ′ is another metric space) is bounded and continuous almost everywhere. Then,

Proof of Lemma S.8 . We first note that X n L 2 ( P ) → X implies X n p → X . Then, by continuous mapping theorem, g ( X n ) p → g ( X ) . Then,

where the equation ∗ = holds by the dominated convergence theorem, which is applicable since g ( X n ) , g ( X ) are bounded functions (from the given condition) and X n p → X .

Proposition 1 ( Asymptotic Analysis of g-mSBD Estimators ) . Assume that the nuisance estimates µ i ℓ and π i ℓ are L 2 -consistent; i.e., ∥ µ i +1 ℓ -µ i +1 0 ∥ P σ ( Z i +1 ) = o P σ ( Z i +1 ) (1) , ∥ ˇ µ i +2 ℓ -ˇ µ i +2 0 ∥ P σ ( Z i +1 ) = o P σ ( Z i +1 ) (1) and ∥ π i ℓ -π i 0 ∥ P σ ( Z i +1 ) = o P σ ( Z i +1 ) (1) for i = 1 , · · · , m -1 , and ∥ ˇ µ 2 ℓ -ˇ µ 2 0 ∥ P σ ( Z 1 ) = o P σ ( Z 1 ) (1) . Let n i := ∣ ∣ D z i ∣ ∣ for i ∈ { 1 , · · · , m } . Then,

where R i is a random variable such that n 1 / 2 i R i converges in distribution to a mean-zero normal random variable.

Proof of Proposition 1 . We start the proof by noting that

where ˆ A ℓ is defined in Def. 3. This proof focuses on analyzing ˆ A ℓ -A 0 .

Also, we recall that D z i ,ℓ for all z i follows the distribution P σ ( Z i ) ( V | r 0 , z i ) . Throughout the proof, we will denote

Then, each ˆ A ℓ -A 0 in Eq. (B.77) is given as follow:

Define and for i = 1 , · · · , m -1 ,

By the central limit theorem, we note that R a i,ℓ for i = 1 , · · · , m is a random variable such that n 1 / 2 i,ℓ R a i,ℓ converges in distribution to a mean-zero normal random variable. Therefore,

also behaves the same.

We now analyze the second term. Define and for i = 1 , 2 , · · · , m -1 ,

By [Kennedy et al., 2020, Lemma 2] and the continuous mapping theorem in Lemma S.8,

and for i = 1 , · · · , m -1 ,

Under the given assumption, for i = 1 , 2 , · · · , m ,

Then, R i,ℓ is also a random variable such that n 1 / 2 i,ℓ R i,ℓ converges in distribution to a mean-zero normal random variable, by Slutsky's theorem.

We now analyze the third term. By Lemma S.6, the third term can be analyzed as follow:

Therefore,

and

Define

Define R i := (1 /L ) ∑ L ℓ =1 R i,ℓ . We note that R i is a random variable such that n 1 / 2 i R i converges in distribution to a mean-zero normal random variable. Then,

B.7 Proof of Theorem 2

We first restate the definition of the MR-gID estimator, the theorem, and its corresponding assumptions.

Definition 4 ( MR-gID Estimator ) . The MR-gID estimator ˆ ψ for the identification expression of the causal effect ψ 0 := f ( { A k 0 } K k =1 ) in Theorem 1 is given as follows: For each A k 0 composing f ( { A k 0 } K k =1 ) , let ˆ A k := ˆ A k ( { µ j +1 k,ℓ , π j k,ℓ } j ∈ [ m k -1] ,ℓ ∈ [ L ] ) denote the DR-g-mSBD estimator with nuisance estimates { µ j +1 k,ℓ , π j k,ℓ } for the true nuisances { µ j +1 k, 0 , π j k, 0 } . Then,

Assumption 2 ( Analysis of MR-gID ) . The identification function f ( { A k } K k =1 ) in Thm. 1 and each nuisances { µ i +1 k,ℓ , π i k,ℓ } k,ℓ for ˆ A k satisfy the following properties:

Twice differentiability : f ( { A k } K k =1 ) is twice continuously Frechet differentiable w.r.t. { A k } K k =1 w.r.t. { A k } K k =1 .

Boundedness : ∀ k ∈ [ K ] and ∀ Z i ∈ Z , ∇ A k f ( { A j 0 } K j =1 )[ ˆ A k -A k 0 ] = O P σ ( Z i ) ( ˆ A k -A k 0 ) .

Theorem 2 ( Asymptotic Analysis of MR-gID ) . Suppose Assumption 2 holds. Let n k,i := | D z k i | for Z k i ∈ Z and z k i ∈ D Z k i . Let ˆ ψ denote the MR-gID estimator in Def. 4 for the causal effect ψ 0 := f ( { A k 0 } K k =1 ) in Theorem 1. Then, the error of ˆ ψ is given as

Proof of Theorem 2 . We first define the following notation. For a map g ( x ) , we will use ∇ x g ( x 0 )[ h ] := lim t → 0 ( g ( x 0 + th ) -g ( x 0 )) /t . We first note that by the definition of Fréchet Derivative-based Taylor expansion [Blanchard and Brüning, 2015, Def. 34.1] and the given assumption ('Twice differentiability'), the error ˆ ψ -ψ 0 can be represented as follow:

where, by Prop. 1,

Therefore, by the big O in probability calculus [Van der Vaart, 2000, Chap. 2],

By applying the given assumption ('Boundedness'), we have the following:

Therefore,

B.8 Proof of Corollary 2

Corollary 2 ( Multiply Robustness (Corollary of Thm. 2)) . Suppose (1) Assumption 2 holds; (2) Either π i k,ℓ = π i k, 0 or µ j k,ℓ = µ j k, 0 for j = i + 1 , · · · , m k for all i, ℓ, k ; and (3) all nuisances { π i k,ℓ , µ i +1 k,ℓ } i,ℓ,k are bounded by some constant. Then, the MR-gID ˆ ψ (Def. 4) is consistent to ψ 0 .

Proof of Corollary 2 . We first note that f is a continuous function under the twice differentiability condition in Assumption 2. Suppose each ˆ A k is a consistent estimator of A k 0 under given conditions. Then, by the continuous mapping theorem, f ( { ˆ A k } K k =1 ) is consistent to P x ( y ) = f ( { A k 0 } K k =1 ) . Therefore, it suffices to show that each ˆ A k is a consistent estimator of A k 0 .

We first recall that ˆ A k := (1 /L ) ∑ L ℓ =1 ˆ A k ℓ by Def. 4. By applying Lemma S.7, ˆ A k ℓ -A k 0 can be rewritten as follow:

We first note that all term in Eq. (B.106) converges in the mean-zero normal distribution and is bounded in probability at n -1 / 2 i +1 ,ℓ,k rate. Therefore,

We now analyze the second term. We note that, by [Kennedy et al., 2020, Lemma 2],

We note that ∥ π i k,ℓ { ˇ µ i +2 k,ℓ -µ i +1 k,ℓ } -π i k, 0 { ˇ µ i +2 k, 0 -µ i +1 k, 0 }∥ is bounded by some constant by the given condition. Therefore, it is bounded in probability at n -1 / 2 i +1 ,ℓ,k rate. By the same analysis, E D z k 1 ,ℓ -P σ ( Z k 1 ) | z k 1 , r 0 [ ˇ µ 2 k,ℓ -ˇ µ 2 k, 0 ] is bounded in probability at 1 / √ n 1 ,ℓ,k -rates. Therefore,

Finally, under the given assumption, the third term can be analyzed by Lemma S.6 and is zero:

Therefore,

and, finally, where each A k 0 is specified as

for W k , R k ⊆ V and Z k ⊆ Z . Then, the discreteness set disc ( { A k 0 } K k =1 ) is defined as follow:

For Example 1, the discreteness set is given as

For Example 2, the discreteness set is given as

Equipped with the discreteness set, we relax the assumption as follows:

Assumption 1.1 ( Relaxed Regularity ) . For variables V and the Radon-Nikodym derivative p σ ( Z ) of P σ ( Z ) for Z ∈ Z , the following conditions hold:

1. All variables in disc ( { A k 0 } ) are discrete;

We note that the proposed estimator is well-defined and corresponding error analyses Theorem 2 and Corollary 2 hold true under the relaxed assumption:

where the last equation holds since O P ( n -α ) = o P (1) when α > 0 .

C Discussion

C.1 Relaxation of Discreteness Assumption

In this paper, we made the strong assumption that all variables are discrete. However, this assumption does not hold in general. In this section, we relax the assumption to a certain degree while ensuring that the proposed estimators and corresponding error analysis remain applicable without sacrificing generality.

First, we define a set of variables, denoted as disc , which must be discrete in order to apply the proposed estimators and leverage the error analyses presented in the paper.

Definition 4 ( Discreteness set disc ) . For some given inputs ( x , y , Z , P , G ) , suppose

Lemma S.9 ( Well-defined MR-gID Estimator under Relaxed Regularity ) . The MR-gID estimator in Definition 4 is pathwise-differentiable under Assumption 1.1 and Assumption 2.

Proof of Lemma S.9 . For k = 1 , · · · , K , define

discrete. Also, under the positivity condition stated in Assumption 1.1, A k in Eq. (C.6) is pathwisedifferentiable. By combining this with Assumption 2, we conclude that the MR-gID estimator is pathwise-differentiable.

To establish the pathwise-differentiability of the MR-gID estimator f ( { ˆ A k } K k =1 ) as defined in Definition 4, it is sufficient to ensure the pathwise-differentiability of individual A k . Under Assumption 1.1, µ m k +1 k := /x31 w k \ c k ( W k \ C k ) is well-defined since ( W k \ C k ) ∈ disc ( { A k 0 } K k =1 ) are discrete. Also, each /x31 r k i ( R k i ) in each π i k are well-defined since R k i ∈ disc ( { A k 0 } K k =1 ) are discrete. Finally, the conditional expectation E P σ ( Z k i ) [ ·| z k i , r k 0 ] is well-defined since Z k i ∈ disc ( { A k 0 } K k =1 ) are

C.2 Sequential Doubly Robustness: 2 m -1 robustness versus m -robustness

In this section, we discuss the practical properties of the proposed doubly robust g-mSBD estimator in Def. 3. We recall that the estimator is doubly robust by the analysis in Lemma S.6 as follows:

Lemma S.6 ( Bias Analysis of g-mSBD Estimators ) . Let A be the quantity defined as

where π i , µ i are arbitrary nuisances for true nuisances π i 0 and µ i 0 defined in Def. 2. Let A 0 denote the g-mSBD in Def. 1. Then,

The term in Eq. (B.69) exhibits doubly robustness, becoming zero when either π r = π r 0 or µ r +1 = µ r +1 0 hold for all r = 1 , 2 , . . . , m -1 . This phenomenon is referred to as the sequential doubly robustness [Luedtke et al., 2017], or 2 m -1 robustness in the sense that there are 2 m -1 ways to make Eq. (B.69) zero [Vansteelandt et al., 2007, Rotnitzky et al., 2017].

While the proposed doubly robust g-mSBD estimator defined in Definition 3 exhibits doubly robustness, as shown in Proposition 1, it does not satisfy 2 m -1 robustness. This is due to the dependencies between µ r ℓ and µ s ℓ for s ∈ { r +1 , · · · , m } . Specifically, if µ s ℓ is misspecified for some s > r , it renders the case µ r ℓ = µ r 0 impossible. Consequently, instead of having 2 m -1 possibilities, there are only m ways to make Eq. (B.69) equal to zero. For each r = 1 , · · · , m -1 , this requires either π r ℓ = π r 0 or µ s ℓ = µ s 0 for s > r . This condition is referred to as m -robustness. In summary, the doubly robust g-mSBD estimator achieves m -robustness instead of 2 m -1 robustness. We acknowledge that an interesting open direction is to explore ways to enhance the doubly robust g-mSBD estimator to attain 2 m -1 robustness, building upon the findings presented in Luedtke et al. [2017].

D Details of Experiments

As described in Sec. 4, we used the XGBoost [Chen and Guestrin, 2016] as a model for estimating nuisances µ, π, { µ i } i m =2 , { π i } i m =1 . We implemented the model using Python. In modeling nuisance using the XGBoost, we used the command

xgboost.XGBClassifier(eval_metric='logloss') 1 to use the XGBoost with the default parameter settings. In estimating the weight, we set the weight π i ℓ = 10 whenever the estimated weight is over 10 [Crump et al., 2009]. For Example 1, the dimension of W is set as | W | = 10 . We chose L = 2 . All variables are set to be binary. We compute the effect of P ( Y = 1 | do ( x )) .

D.1 Designs of Simulations

In this section, we present the structural causal models (SCMs) utilized for generating the dataset. Furthermore, we include a segment of the code employed to generate the dataset.

D.1.1 Example 1

We define the following structural causal models for Example 1:

where where the coefficient vector c X , c Z are such that their i th element is 0 if i is an even number and 1 otherwise.

D.1.2 Example 2

We define the following structural causal models for Example 2:

X

1

X

2

W

Y

R

C

1

C

2

(a) An SCM

M

for

Project

STAR

X

1

X

2

W

Y

R

C

1

C

2

(b)

G

(

M

x

1

,x

2

)

X

1

X

2

W

Y

R

C

1

C

2

(c)

G

(

M

x

2

)

where

E Project STAR: Estimating Joint Effects of Class Sizes to Academic Outcomes

We applied the proposed estimators to the Project STAR dataset [Krueger and Whitmore, 2001, Schanzenbach, 2006]. Project STAR is an experimental study investigating teacher/student ratios' impact on academic achievement for kindergarten through third-grade students. In the study, students were randomly assigned to three different class sizes: small-size classes, regular classes, and large-size classes. The objective was to evaluate how class size affects academic outcomes [Schanzenbach, 2006]. In our analysis, we used the dataset introduced in the online complement of Stock et al. [2003].

Project STAR Dataset. We denote the Project STAR dataset as D . The dataset D includes the following information: class size for kindergarten ( X 1 ), the academic outcome in kindergarten ( W ), the academic outcome in second grade ( R ), class size for third grade ( X 2 ), the academic outcome in the third grade ( Y ), free lunch receiving for kindergarten ( C 1 ), gender ( C 2 ), ethnicity ( C 3 ) and free lunch receiving for the third grade ( C 4 ). We will use C 1 := ( C 1 , C 2 ) and C 2 := ( C 3 , C 4 ) .

Assumption on Dataset. We assume that the SCM M generating the variables ( X 1 , W, R, X 2 , Y, C 1 , C 2 ) induces a causal graph depicted in Figure E.4a. Here, we mark C 1 , C 2 as gray to denote that these variables will be considered latent; i.e., these variables will not be used in the data analysis.

Project STAR dataset D is a longitudinal experimental study randomizing X 1 and X 2 ; i.e., the dataset is induced by the submodel M x 1 ,x 2 for x 1 , x 2 ∈ D X 1 ,X 2 , represented in Fig. E.4b. The samples for variables { X 1 , W } follow a distribution P σ ( X 1 ) ( X 1 , W, R ) = P σ ( X 1 ,X 2 ) ( X 1 , W, R ) , and the samples for variables { X 1 , W, R, X 2 , Y } follow a distribution P σ ( X 1 ,X 2 ) ( X 1 , W, R, X 2 , Y ) . To demonstrate, we will describe how to generate the sample fol-

lowing a distribution P σ ( X 2 ) ( X 1 , W, R, X 2 , Y ) by creating unmeasured confounding bias between X 1 and W , and X 1 and Y , which is depicted by Fig. E.4c.

Creation of Datasets from Marginal Experiments. In this empirical study, we create two datasets from this dataset: D 1 and D 2 . The dataset D 1 is a random subsample of D only including { X 1 , W } . Then, D 1 follows P σ ( X 1 ) ( X 1 , W, R ) .

To construct the dataset D 2 following the marginal experimental distribution P σ ( X 2 ) ( X 1 , W, R, X 2 , Y ) , the confounding bias between X 1 and W and X 1 and Y should be introduced. To do so, we follow a standard procedure for introducing confounding bias from experimental studies used in Hill [2011], Louizos et al. [2017], Zhang and Bareinboim [2019], Gentzel et al. [2021].

A setting for the standard procedure is the following. For any arbitrary random variable X,Y,Z,W such that X → Y , Z → Y , and there are no arrows into X , the dataset D := { ( X ( i ) , Y ( i ) , Z ( i ) ,W ( i ) : i = 1 , · · · , n } is given. In D , Z is not a confounding variable since Z ⊥ ⊥ X . The goal is to generate a new dataset D ′ := { ( X ( j ) , Y ( j ) , Z ( j ) : j = 1 , · · · , n ′ } where Z serves as a confounding variable between X and Y . The procedure named IntroduceConfounding ( X,Y ; Z, D ) is given as follows: Initialize D ′ = {} . For i = 1 , · · · , n , do the followings:

Generate the Bernoulli Random B ( i ) with parameter P ( X ( i ) | Z ( i ) ) .

Finally, we exclude Z is removed from D ′ . By doing so, we introduce unmeasured confounding bias between X and Y in D ′ ; i.e., D ′ = IntroduceConfounding ( X,Y ; Z, D ) .

To generate the dataset D 2 ∼ P σ ( X 2 ) ( X 1 , W, R, X 2 , Y ) from D ∼ P σ ( X 1 ,X 2 ) ( X 1 , W, R, X 2 , Y, C 1 , C 2 ) , we have to introduce the unmeasured confounding bias between X 1 and W , and X 1 and Y . Wedo this by D ′ 2 = IntroduceConfounding ( X 1 , W ; C 1 , D ) . Then, D ′ 2 is a variable containing ( X 1 , W, R, X 2 , C 2 , Y ) and there is a confounding bias between X 1 and W . Then, we set D 2 = IntroduceConfounding ( X 1 , Y ; C 2 , D ′ 2 ) . Then, D 2 is a variable containing ( X 1 , W, R, X 2 , Y ) and there is a confounding bias between X 1 and Y , and X 1 and W . A causal graph Fig. E.4c depicted the dependencies in D 2 .

Goal. In this empirical study, we aim to study the joint effect of the class size for kindergarten ( X 1 ) and the third grade ( X 2 ) on the third grade's academic outcome ( Y ); i.e., E [ Y | do ( x 1 , x 2 )] . Since D is a longitudinal experimental dataset following P rand ( X 1 ,X 2 ) ( C, X 1 , W, X 2 , Y ) , the ground-truth E [ Y | do ( x 1 , x 2 )] is estimated as E D [ Y /x31 x 1 ,x 2 ( X 1 , X 2 )] / E D [ /x31 x 1 ,x 2 ( X 1 , X 2 )] .

Causal Effect Identification. We identify P ( y | do ( x 1 , x 2 )) through Algo. 1.

Line 3 : Set D := an ( Y ) G ( V \{ X 1 ,X 2 } ) = { Y, R, W } .

Each Q [ D 1 ] = Q [ W ] , Q [ D 2 ] = Q [ R ] , Q [ D 3 ] = Q [ Y ] are identified as follows: For Q [ D 1 ] ,

Line a.4 : Since S 1 = D 1 , we set

Line 8 : Set Q [ S 1 1 ] = A 0 [ S 1 1 , ∅ , R 1 1 ; Z 1 1 = { X 1 } , seq 1 1 = ( x 1 )]( w,x 1 ) where R 1 1 := ∅ .

Line 7 : For D 2 , Z 2 := { X 1 } , z 2 := { x 1 } , S 1 2 := { R } .

Line a.4 : Since S 1 = D 2 , we set

For Q [ D 3 ] = Q [ Y ] ,

Line 7 : For D 3 , Z 3 := { X 2 } , z 3 := { x 2 } , S 2 1 := { Y, X 1 , W } .

Line 8 : Set Q [ S 2 1 ] = A 0 [ S 2 1 , ∅ , R 2 1 ; Z 2 1 = { X 2 } , seq 2 1 = ( x 2 )]( y, r ) where R 2 1 := { W } ; i.e.,

Line a.2 : Set A := an ( D 3 ) G ( S 2 1 ) = { Y } .

Line a.4 : Since A = D 2 , we set

Line a.3 : Q [ A ] = A 0 [ { Y } , { X 1 , W } , { R } ; Z 2 1 = { X 2 } , seq 2 1 = ( x 2 )]( y, r ) .

Therefore,

where the last equation holds since A Y 0 is not a function of W . By Lemma 2

Fig. E.3a

(Project STAR)

Scenario 1

Scenario 2

Scenario 3

Scenario 4

0.25

Model

0.5

Model

0.5

Model

0.25

Model

0.20

REG

PW

MR

0.4

REG

= PW

= MR

0.4

REG

PW

MR

0.20

PW

REG

MR

0.15

0.3

0.3

0.15

0.10

0.2

0.2

0.10

0.05

0.1

0.1

0.05

土

0.00

0.0

200

400

600

008

1000

0.0

0.00

Therefore,

Causal Effect Estimation. Here, we only describe the nuisance for A Y 0 in Eq. (E.1), since estimating A R ′ 0 = P x 1 ( r ) is trivial. We define the nuisance as follows: For the fixed x 2 ∈ D X 2 ,

Then, A Y 0 in Eq. (E.1) can be expressed as follows:

We then construct the regression-based, probability weighting-based, and MR-gID (MR) T reg , T pw , T mr using the following procedure.

For each fixed x 2 ∈ D X 2 and a sample set D x 2 for i ∈ { 1 , 2 } , randomly split the sample as D x 2 ,t and D x 2 ,e .

Use D x 2 ,t to train the model for learning nuisances in Eq. (E.2) and Eq. (E.3). Let µ ( X 1 , W, R ) and π ( R | X 1 , W ) denote the learnt models. We use the XGBoost [Chen and Guestrin, 2016] to learn the model.

Then, each estimator is defined as follows:

Experimental Results As described in the Experimental Setup section (Sec. 4), we evaluated the AAE est of estimators T est for est ∈ { reg , pw , mr } in Scenarios {1, 2, 3, 4}. The AAE plots for all

cases can be seen in Fig. E.5. In this particular scenario, the sample size was not varied since the sample itself was externally given.

In Case 2, we introduced variation by adjusting the size of the converging noise ϵ , which follows a normal distribution Normal ( n -α , n -2 α ) for n ∈ { 200 , 400 , 600 , 800 , 1000 } . It was observed that the MR-gID estimator T mr outperformed the other two estimators by achieving fast convergence, as demonstrated in Theorem 2. For Scenarios {3, 4}, the DML estimator T mr exhibited doubly robust properties, as illustrated in Corollary 2.

Peer reviews:
comments: SUMMARY:
The paper presents a method for causal effect estimation through a combination of experimental and observational data.

STRENGTHS:
- The paper addresses a very important problem 
- The solution appears to be mathematically sound
- Experimental results appear to show improvement over other methods

WEAKNESSES:
- The paper is incredibly  hard to parse. Pages 4-8 are just a wall of mathematical notation with little to no explanation. 
- The paper would improve significantly if after each definition / lemma / proof a plain word intuition was given. 
- Running an example through an algorithm in the manner its done in page 6 is borderline incomprehensible. 
- Figure 2 is too small to read even when pdf is zoomed 

  Honestly this is a great paper that is severely limited by the way its presented.

QUESTIONS:
No real questions just a request that the paper is rewritten to increase clarity and improve presentation

LIMITATIONS:
Little discussion is included about the limitations of the method
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-04

comments: SUMMARY:
This paper presents a new framework for estimating the causal effect when multiple observational and experimental datasets are available. The authors proved that any g-identifiable causal effects can be written as a functional of "g-mSBD" operators, based on which they further developed a doubly robust estimator. Asymptotic properties of this estimator are then analyzed, and the performance of this estimator is confirmed in simulation studies.

STRENGTHS:
This paper tackles a hard problem with theoretical soundness and practical applicability.

WEAKNESSES:
I have two questions/comments. 

The first regards the assumption imposed in their Proposition 2 as well as Assumption 2 -- the authors assumed the nuance estimates are L2-consistent. While I understand that such a property would make the theoretical analysis more amenable, I am curious and would appreciate it if the authors can provide a concrete (and maybe idealized) example, under which the assumption can be verified.

My second question is about practical performance of the proposed estimator. While I am convinced about its utility in the simulation study, it would be great if the authors can apply their methodology in a real dataset and verify it still outperforms its competitors.

QUESTIONS:
Please see the "Weakness" section.

LIMITATIONS:
Please see the "Weakness" section.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study a class of causal effect that are called “g-identifiable” (gID) following Lee et al. (UAI 2019). There are two main results.\n1. Expressing these causal effects as transformations of generalized multi outcome sequential back door adjustments (g-mSBDs).\n2. Proposing doubly robust estimators for the adjustments, which then imply estimators for the causal effects if the transformations are smooth.\n\nIt appears that Jung et al (ICML 2023) previously provide these types of results for a subset of gID causal effects. The contribution is to extend the previous results for the full class of gID causal effects.\n\nSTRENGTHS:\n1. Originality: The first contribution is, in my view, the stronger one. Its connection to prior work is clearly stated. It appears that the workhorse is Lemmas 2, 3, and 4, which demonstrate the good properties of Definition 1. The second contribution follows quite closely from prior work.\n\n2. Quality: The introduction was generally well written, though it would make more sense to choose Examples 1 and 2 to be examples that are not in the class of Jung et al (ICML 2023).\n\n3. Clarity: Overall, the organization of the paper was very logical.\n\n4. Significance: The contribution seems sound but possibly incremental.\n\nWEAKNESSES:\nI would be willing to improve the score if these weaknesses are addressed by the authors.\n\n1. I find the title and framing too broad. There are many papers that could have this title, and it does not clearly convey what is in this paper but not in other papers.\n\n2. Line 56 gives a list of doubly robust estimators. At the very least, the sequential regression paper by Bang and Robins (Biometrics 2005), the targeted learning in data science textbook (van der Laan and Rose 2018), and the debiased machine learning paper by Chernozhukov et al. (2018 Econometrics Journal) should be cited. \n\n3. More relevant are the sequential doubly robust estimators such as Luedtke et al. (arXiv:1705.02459, 2017) and Chernozhukov et al. (arXiv:2203.13887, 2022) and the references in those papers. Definition 3 and Proposition 1, are very closely related to the prior works listed above, so explicit comparisons would be helpful here.\n\n4. The delineation between Lee et al. (2019)’s algorithm and this algorithm should be clearer. The sentences in lines 127 and 156 seem to clash in explaining the relationship to the prior work. \n\n5. The notation in Definition 1, Lemma 1, Lemma 2, and Lemma 3 is very heavy, and no interpretation is assigned to any of the variable symbols. It would help to revisit Examples 1 and 2 here.\n\n6. In the appendix, the data generating processes are not described analytically; instead, python code is pasted. Please describe them. Please explicitly write what are the functions mu and pi.\n\nQUESTIONS:\nI would be willing to improve the score if these questions are addressed by the authors.\n\n1. Algorithm 1 is a modification of Lee et al. (2019)’s complete algorithm. Is it the case that Algorithm 1 is precisely the previous algorithm but with Lemmas 2, 3 and 4? Does it remain complete?\n\n2. Since the transformation f is shown to be marginalization, multiplication, or division, wouldn’t we already know that aspects of Assumption 2 are automatically satisfied? Phrased another way, when is each condition in Assumption 2 binding?\n\n3. Are the proposed estimators semiparametrically efficient?\n\n4. What does scenario 2 mean? Is this measurement error, or is it the usual error of a regression problem? \n\n5. Suggested corrections:\nline 8: multiply --> multiple.\nline 86: specify norm when writing convergence of f_hat-f when they are functions.\nline 127: “(and rewrote in Algo. 1) rewrites” is awkward phrasing.\nline 248: Fretchet --> Frechet.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

comments: SUMMARY:
The paper aims to fuse the recent literature on causal effect identification and estimation. It expresses any identification (in the general sense) of a causal effect in terms of generalized mSBD adjustments, and then develops a corresponding estimator. For the latter robustness properties and empirical studies are shown.

STRENGTHS:
The contribution made by this paper is well-situated within the problem space and it's a natural generalization of other recent papers. All the results appear to be sound.

WEAKNESSES:
I believe the contribution to be valuable but the methods are not novel; they seem to be iterative applications of existing ones.
In many places it's too dense: for example, Examples 3, 4 are not illuminating, and readers may just skip them.

QUESTIONS:
Can you include a future work section? My perception of the significance of the work would be improved if I could see some directions for how to build on this.

LIMITATIONS:
Yes, they have (there is ample discussion of e.g. the assumptions relied on).
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper builds on prior work in causal effect estimation, from observational as well as experimental data, by providing a more general framework with desirable properties. This more generalized framework is shown to be robust and consistent in its estimation and overcomes the limitations of some of the prior work in this area. The new, general framework, operating in the realm of generalized identification or g-identification, exhibits multiply robust properties, which is experimentally validated.\n\nSTRENGTHS:\nThe new proposed estimator could be now the go-to framework for any g-identifiable causal functionals, potentially representing a significant theoretical advance in the field of learning causal effects.  \nThe paper is well-scoped and specific in what is being advanced and corroborates the desirable properties of the estimator via simulations.  \nThe exposition is quite clear and well-written.\n\nWEAKNESSES:\nIt is not clear what challenges remain for g-identification in practice with the proposed new estimator. Some guidance for practitioners and other researchers as to the context of problems and types of data they can consider using the new estimator would be highly useful.\n\nQUESTIONS:\n1. Is there any scenarios where Jung et al 2023 is still useful -- i.e., are there complementary notions of use to that framework and the one proposed here, or does the framework here subsumes Jung 2023 completely? Is it possible to compare the two frameworks in a particular setting to empirically demonstrate the relationship between the two frameworks?\n\n2. What practical challenges remain for g-identification when using the proposed estimator? \n\nSuggestions:\n\n- Remove citation from abstract, just let the 'in the literature' stand, and cite instead when first mentioned in the introduction.\n- Claim in Line 286 -- \"We consider two standard baselines in the literature\" --  must be backed by citations, especially the notion of being \"standard\".\n\nLIMITATIONS:\nIt is not clear what are the limitations of this work and the proposed new framework, and a lack of such an elucidation might hurt the uptake of what seems to be a promising new framework. There does not seem to be any discussion of any kind of limitations or potential broader impacts.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 1, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-23"}