Original: Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved Confounders
Revised: Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved Confounders

Original: Anonymous Author(s)

Original: AfﬁliationAddress

Original: Abstract
Revised: Abstract

Original: The ability to answer causal questions is crucial in many domains, as causal inference allows one to understand the impact of interventions.
Revised: The ability to answer causal questions is crucial in many domains, as causal inference allows one to understand the impact of interventions.

Original: In many applications,only a single intervention is possible at a given time.
Revised: In many applications, only a single intervention is possible at a given time.

Original: However, in some importantareas, multiple interventions are concurrently applied.
Revised: However, in some important areas, multiple interventions are concurrently applied.

Original: Disentangling the effectsof single interventions from jointly applied interventions is a challenging task—especially as simultaneously applied interventions can interact.
Revised: Disentangling the effects of single interventions from jointly applied interventions is a challenging task— especially as simultaneously applied interventions can interact.

Original: This problem ismade harder still by unobserved confounders, which inﬂuence both treatmentsand outcome.
Revised: This problem is made harder still by unobserved confounders, which influence both treatments and outcome.

Original: We address this challenge by aiming to learn the effect of a singleintervention from both observational data and sets of interventions.
Revised: We address this challenge by aiming to learn the effect of a singleintervention from both observational data and sets of interventions.

Original: We provethat this is not generally possible, but provide identiﬁcation proofs demonstratingthat it can be achieved under non-linear continuous structural causal models withadditive, multivariate Gaussian noise—even when unobserved confounders arepresent.
Revised: We prove that this is not generally possible, but provide identification proofs demonstrating that it can be achieved under non-linear continuous structural causal models with additive, multivariate Gaussian noise—even when unobserved confounders are present.

Original: Importantly, we show how to incorporate observed covariates and learnheterogeneous treatment effects.
Revised: Importantly, we show how to incorporate observed covariates and learn heterogeneous treatment effects.

Original: Based on the identiﬁability proofs, we provide analgorithm that learns the causal model parameters by pooling data from differentregimes and jointly maximizing the combined likelihood.
Revised: Based on the identifiability proofs, we provide an algorithm that learns the causal model parameters by pooling data from different regimes and jointly maximizing the combined likelihood.

Original: The effectiveness of ourmethod is empirically demonstrated on both synthetic and real-world data.
Revised: The effectiveness of our method is empirically demonstrated on both synthetic and real-world data.

Original:  Introduction
Revised:  Introduction

Original: The ability to answer causal questions is crucial in science, medicine, economics, and beyond,see Gilligan-Lee (2020) for a high-level overview.
Revised: The ability to answer causal questions is crucial in science, medicine, economics and beyond, see [Gilligan-Lee, 2020] for a high-level overview.

Original: This is because causal inference allows one tounderstand the impact of interventions.
Revised: This is because causal inference allows one to understand the impact of interventions.

Original: In many applications, only a single intervention is possible ata given time, or interventions are applied one after another in a sequential manner.
Revised: In many applications, only a single intervention is possible at a given time or interventions are applied one after another in a sequential manner.

Original: However, in someimportant areas, multiple interventions are concurrently applied.
Revised: However, in some important areas, multiple interventions are concurrently applied.

Original: For instance, in medicine, patientsthat possess many commodities may have to be simultaneously treated with multiple prescriptions; incomputational advertising, people may be targeted by multiple concurrent campaigns; and in dietetics,the nutritional content of meals can be considered a joint intervention from which we wish to learnthe effects of individual nutritional components
Revised: For instance, in medicine, patients that possess many commodities may have to be simultaneously treated with multiple prescriptions; in computational advertising, people may be targeted by multiple concurrent campaigns; and in dietetics, the nutritional content of meals can be considered a joint intervention from which we wish to learn the effects of individual nutritional components.

Original: Disentangling the effects of single interventions from jointly applied interventions is a challenging task—especially as simultaneously applied interventions can interact, leading to consequences not seen when considering single interventions separately.
Revised: Disentangling the effects of single interventions from jointly applied interventions is a challenging task—especially as simultaneously applied interventions can interact, leading to consequences not seen when considering single interventions separately.

Original: This problem is made harder still by the possible presence of unobserved confounders, which inﬂuence both treatments and outcome.
Revised: This problem is made harder still by the possible presence of unobserved confounders, which influence both treatments and outcome.

Original: This paper addresses this challenge, by aiming to learn the effect of a single-intervention from both observational data and sets of interventions.
Revised: This paper addresses this challenge, by aiming to learn the effect of a single-intervention from both observational data and sets of interventions.

Original: We prove that this is not generally possible, but provide identiﬁcation proofs demonstrating it can be achieved in certain classes of non-linear continuous causal models with additive multivariate Gaussian noise—even in the presence of unobserved confounders.
Revised: We prove that this is not generally possible, but provide identification proofs demonstrating it can be achieved in certain classes of non-linear continuous causal models with additive multivariate Gaussian noise—even in the presence of unobserved confounders.

Original: This reasonably weak additive noise assumption is prevalent in the causal inferenceand discovery literature (Rolland et al., 2022; Saengkyongam and Silva, 2020; Kilbertus et al., 2020).
Revised: This reasonably weak additive noise assumption is prevalent in the causal inference and discovery literature [Rolland et al., 2022, Saengkyongam and Silva, 2020, Kilbertus et al., 2020].

Original: Importantly, we show how to incorporate observed covariates, which can be high-dimensional, andhence learn heterogeneous treatment effects for single-interventions.
Revised: Importantly, we show how to incorporate observed covariates, which can be high-dimensional, and hence learn heterogeneous treatment effects for single-interventions.

Original: Our main contributions are:
Revised: Our main contributions are:

Original: A proof that without restrictions on the causal model, single-intervention effects cannot beidentiﬁed from observations and joint-interventions.
Revised: A proof that without restrictions on the causal model, single-intervention effects cannot be identified from observations and joint-interventions.

Original: Proofs that single-interventions can be identiﬁed from observations and joint-interventionswhen the causal model belongs to certain  classes of non-linear continuous structural causalmodels with additive, multivariate Gaussian noise.
Revised: Proofs that single-interventions can be identified from observations and joint-interventions when the causal model belongs to certain (but not all) classes of non-linear continuous structural causal models with additive, multivariate Gaussian noise.

Original: An algorithm that learns the parameters of the proposed causal model and disentanglessingle interventions from joint interventions.
Revised: An algorithm that learns the parameters of the proposed causal model and disentangles single interventions from joint interventions.

Original: An empirical validation of our method on both synthetic and real data.
Revised: An empirical validation of our method on both synthetic and real data.

Original:  Related Work
Revised:  Related Work

Original: Disentangling multiple concurrent interventions: Parbhoo et al. (2021) study the question ofdisentangling multiple, simultaneously applied interventions from observational data.
Revised: Disentangling multiple concurrent interventions: [Parbhoo et al., 2021] study the question of disentangling multiple, simultaneously applied interventions from observational data.

Original: They proposea specially designed neural network for the problem and show good empirical performance on somedatasets.
Revised: They propose a specially designed neural network for the problem and show good empirical performance on some datasets.

Original: However, they do not address the formal identiﬁcation problem, nor do they address possiblepresence of unobserved confounders.
Revised: However, they do not address the formal identification problem, nor do they address possible presence of unobserved confounders.

Original: By contrast our work derives the conditions under whichidentiﬁability holds.
Revised: By contrast our work derives the conditions under which identifiability holds.

Original: We moreover propose an algorithm that can disentangle multiple interventionseven in the presence of unobserved confounders—as long as both observational and interventionaldata is available.
Revised: We moreover propose an algorithm that can disentangle multiple interventions even in the presence of unobserved confounders—as long as both observational and interventional data is available.

Original: Related work by Parbhoo et al. (2020) investigated the intervention-disentanglementproblem from a reinforcement learning perspective, where each intervention combination constitutesa different action that a reinforcement learning agent can take.
Revised: Related work by [Parbhoo et al., 2020] investigated the intervention-disentanglement problem from a reinforcement learning perspective, where each intervention combination constitutes a different action that a reinforcement learning agent can take.

Original: Unlike this approach, our workexplicitly focuses on modelling the interactions between interventions to learn their individualeffects.
Revised: Unlike this approach, our work explicitly focuses on modelling the interactions between interventions to learn their individual effects.

Original: Closer to our work is Saengkyongam and Silva (2020), who investigate identiﬁability ofjoint effects from observations and single-intervention data.
Revised: Closer to our work is [Saengkyongam and Silva, 2020], who investigate identifiability of joint effects from observations and single-intervention data.

Original: They prove this is not generally possible,but provide an identiﬁcation proof for non-linear causal models with additive Gaussian noise.
Revised: They prove this is not generally possible, but provide an identification proof for non-linear causal models with additive Gaussian noise.

Original: Ourwork addresses a complementary question; we want to learn the effect of a single-intervention fromobservational data and sets of interventions.
Revised: Our work addresses a complementary question; we want to learn the effect of a single-intervention from observational data and sets of interventions.

Original: Additionally, another difference between our work andthat of Saengkyongam and Silva (2020) is that they do not consider identiﬁcation of individual-levelcausal effects given observed covariates.
Revised: Additionally, another difference between our work and that of [Saengkyongam and Silva, 2020] is that they do not consider identification of individual-level causal effects given observed covariates.

Original: In a precursor to the work by Saengkyongam and Silva(2020), Nandy et al. (2017) developed a method to estimate the effect of joint interventions fromobservational data when the causal structure is unknown.
Revised: In a precursor to the work by [Saengkyongam and Silva, 2020], [Nandy et al., 2017] developed a method to estimate the effect of joint interventions from observational data when the causal structure is unknown.

Original: This approach assumed linear causal modelswith Gaussian noise, and only proved identiﬁability in this case under a sparsity constraint.
Revised: This approach assumed linear causal models with Gaussian noise, and only proved identifiability in this case under a sparsity constraint.

Original: However,like Saengkyongam and Silva (2020), our result does not need the linearity assumption, and nosparsity constraints are required in our identiﬁcation proof.
Revised: However, like [Saengkyongam and Silva, 2020], our result does not need the linearity assumption, and no sparsity constraints are required in our identification proof.

Original: Finally, others including Schwab et al.(2020); Egami and Imai (2018); Lopez and Gutman (2017); Ghassami et al. (2021) explored howto estimate causal effects of a single categorical-, or continuous-valued treatment, where differentintervention values can produce different outcomes.
Revised: Finally, others including [Schwab et al., 2020, Egami and Imai, 2018, Lopez and Gutman, 2017, Ghassami et al., 2021] explored how to estimate causal effects of a single categorical-, or continuous-valued treatment, where different intervention values can produce different outcomes.

Original: Unlike our work, they do not consider multipleconcurrent interventions that can interact.
Revised: Unlike our work, they do not consider multiple concurrent interventions that can interact.

Original: Combining observations and interventions: Bareinboim and Pearl (2016) have investigated nonparametric identiﬁability of causal effects using both observational and interventional data, in aparadigm they call “data fusion.”
Revised: Combining observations and interventions: [Bareinboim and Pearl, 2016] have investigated nonparametric identifiability of causal effects using both observational and interventional data, in a paradigm they call “data fusion.”

Original: More general results were studied by Lee et al. (2020), whoprovided necessary and sufﬁcient graphical conditions for identifying causal effects from arbitrarycombinations of observations and interventions.
Revised: More general results were studied by [Lee et al., 2020], who provided necessary and sufficient graphical conditions for identifying causal effects from arbitrary combinations of observations and interventions.

Original: Recent work in Correa et al. (2021) exploredidentiﬁcation of counterfactual—as opposed to interventional—distributions from combinations ofobservational and interventional data.
Revised: Recent work in [Correa et al., 2021] explored identification of counterfactual—as opposed to interventional—distributions from combinations of observational and interventional data.

Original: Finally, Ilse et al. (2021) investigated the most efﬁcient way tocombine observational and interventional data to estimate causal effects.
Revised: Finally, [Ilse et al., 2021] investigated the most efficient way to combine observational and interventional data to estimate causal effects.

Original: They demonstrated theycould signiﬁcantly reduce the number of interventional samples required to achieve a certain ﬁt whenadding sufﬁcient observational training samples.
Revised: They demonstrated they could significantly reduce the number of interventional samples required to achieve a certain fit when adding sufficient observational training samples.

Original: However, they only prove their method theoreticallyin the linear-Gaussian case.
Revised: However, they only prove their method theoretically in the linear-Gaussian case.

Original: In the non-linear case, they parameterise their model using normalisingﬂows and demonstrate their method empirically.
Revised: In the non-linear case, they parameterise their model using normalising flows and demonstrate their method empirically.

Original: They only consider estimating single-interventions,and do not deal with multiple, interacting interventions.
Revised: They only consider estimating single-interventions, and do not deal with multiple, interacting interventions.

Original: Additive noise models: While certain causal quantities may not be generally identiﬁable fromobservational and interventional data, by imposing restrictions on the structural functions underlyingcausal models, one can obtain semi-parametric identiﬁability results.
Revised: Additive noise models: While certain causal quantities may not be generally identifiable from observational and interventional data, by imposing restrictions on the structural functions underlying causal models, one can obtain semi-parametric identifiability results.

Original: One of the most common weakrestrictions used in the causal inference community are additive noise models (ANMs), ﬁrst studiedin the context of causal discovery by Hoyer et al. (2009)—and still widely used today (Rolland et al.,2022).
Revised: One of the most common weak restrictions used in the causal inference community are additive noise models (ANMs), first studied in the context of causal discovery by [Hoyer et al., 2009]—and still widely used today [Rolland et al., 2022].

Original: ANMs limit the form of the structural equations to be additive with respect to latent noisevariables—but allow nonlinear interactions between causes.
Revised: ANMs limit the form of the structural equations to be additive with respect to latent noise variables— but allow nonlinear interactions between causes.

Original: Janzing et al. (2009) used ANMs todevise a method for inferring a latent confounder between two observed variables.
Revised: [Janzing et al., 2009] used ANMs to devise a method for inferring a latent confounder between two observed variables.

Original: This is otherwisenot possible without additional assumptions on the underlying causal model.
Revised: This is otherwise not possible without additional assumptions on the underlying causal model.

Original: See Lee and Spekkens(2017) and Lee et al. for an extension of this approach beyond ANMs.
Revised: See [Lee and Spekkens, 2017], [Lee et al., 2019], and [Dhir and Lee, 2020] for an extension of this approach beyond ANMs.

Original: ANMs have also beenemployed by Kilbertus et al. (2020) to investigate the sensitivity of counterfactual notions of fairnessto the presence of unobserved confounding.
Revised: ANMs have also been employed by [Kilbertus et al., 2020] to investigate the sensitivity of counterfactual notions of fairness to the presence of unobserved confounding.

Original: Our work proves that in certain classes of ANMs, theeffect of a single-intervention can be identiﬁed from observational data and sets of interventions—even in the presence of unobserved confounders.
Revised: Our work proves that in certain classes of ANMs, the effect of a single-intervention can be identified from observational data and sets of interventions—even in the presence of unobserved confounders.

Original: Moreover, we show how to incorporate observedcovariates in these ANMs to learn the heterogeneous effects of single-interventions.
Revised: Moreover, we show how to incorporate observed covariates in these ANMs to learn the heterogeneous effects of single-interventions.

Original:  Model Identiﬁability
Revised:  Model Identifiability

Original: Identiﬁability is a fundamental concept in parametric statistics that relates to the quantities that can, orcan not, be learned from data Rothenberg (1971).
Revised: Identifiability is a fundamental concept in parametric statistics that relates to the quantities that can, or can not, be learned from data [Rothenberg, 1971].

Original: An estimand is said to be identiﬁable from data if itis theoretically possible to learn this estimand, given inﬁnite samples.
Revised: An estimand is said to be identifiable from data if it is theoretically possible to learn this estimand, given infinite samples.

Original: If two causal models coincideon said data then they must coincide on the value of the estimand in question.
Revised: If two causal models coincide on said data then they must coincide on the value of the estimand in question.

Original: Hence if one ﬁndstwo causal models which agree on said data, but disagree on the estimand, then the estimand is notidentiﬁable unless further restrictions are imposed.
Revised: Hence if one finds two causal models which agree on said data, but disagree on the estimand, then the estimand is not identifiable unless further restrictions are imposed.

Original: In this section, we provide identiﬁcation proofsfor single-variable interventional effects from observational data and joint interventions, for severalmodel classes.
Revised: In this section, we provide identification proofs for single-variable interventional effects from observational data and joint interventions, for several model classes.

Original: Our theoretical analysis provides insights into the fundamental limitations of causalinference—and the assumptions that are required for identiﬁcation.
Revised: Our theoretical analysis provides insights into the fundamental limitations of causal inference—and the assumptions that are required for identification.

Original: Problem Deﬁnition.
Revised: Problem Definition.

Original: We adopt the Structural Causal Model (SCM) framework as introduced by Pearl (2009).
Revised: We adopt the Structural Causal Model (SCM) framework as introduced by [Pearl, 2009].

Original: An SCM M is deﬁned by h { C , X , Y } , U , f , P U i , where { C , X , Y } are endogenous115 variables separated into covariates C , treatments X , and the outcome Y , U are exogenous variables(possibly confounders), f are structural equations, and P U deﬁnes a joint probability distributionover the exogenous variables.
Revised: An SCM M is defined by ⟨{ C , X , Y } , U , f , P U ⟩ , where { C , X , Y } are endogenous variables separated into covariates C , treatments X , and the outcome Y , U are exogenous variables (possibly confounders), f are structural equations, and P U defines a joint probability distribution over the exogenous variables.

Original: The SCM M also induces a causal graph—where vertices represent endogenous variables, andedges represent structural equations.
Revised: The SCM M also induces a causal graph—where vertices represent endogenous variables, and edges represent structural equations.

Original: Vertices with outgoing edges to an endogenous variable X i aredenoted as the parent set of this variable, or PA ( X i ) .
Revised: Vertices with outgoing edges to an endogenous variable X i are denoted as the parent set of this variable, or PA ( X i ) .

Original: Typically, the observed covariates C causallyinﬂuence the treatments as well as the outcome, and are a part of this set.
Revised: Typically, the observed covariates C causally influence the treatments as well as the outcome, and are a part of this set.

Original: Every endogenous variable
Revised: Every endogenous variable

Original: X i (including Y ) is then a function of its parents in the graph PA ( X i ) and a latent noise term
Revised: X i (including Y ) is then a function of its parents in the graph PA ( X i ) and a latent noise term

Original: U i ,denoting the inﬂuence of factors external to the model:
Revised: U i , denoting the influence of factors external to the model:

Original: [Equation]
Revised: [Equation]

Original: In Markovian SCMs, these latent noise terms are all mutually independent.
Revised: In Markovian SCMs, these latent noise terms are all mutually independent.

Original: However, in general,distinct noise terms can be correlated according to some global distribution P U .
Revised: However, in general, distinct noise terms can be correlated according to some global distribution P U .

Original: In this case, suchcorrelation is due to the presence of unobserved confounders .
Revised: In this case, such correlation is due to the presence of unobserved confounders .

Original: An intervention on variable X i is denoted by do ( X i = x i ) , and it corresponds to replacing itsstructural equation with a constant, or removing all incoming edges in the causal graph.
Revised: An intervention on variable X i is denoted by do ( X i = x i ) , and it corresponds to replacing its structural equation with a constant, or removing all incoming edges in the causal graph.

Original: The corequestion we wish to answer in this work, is under which conditions the treatment effect of a singleintervention can be disentangled from joint interventions and observational data.
Revised: The core question we wish to answer in this work, is under which conditions the treatment effect of a single intervention can be disentangled from joint interventions and observational data.

Original: That is, givensamples from the data regimes that induce
Revised: That is, given samples from the data regimes that induce

Original: E [ Y | X i
Revised: E [ Y | X i

Original: = x
Revised: = x

Original: i , X j = x j , C = c ] , and E [ Y | do ( X i = x i , X j = x j ) , C = c ] , when can we learn conditional average causal effects:
Revised: i , X j = x j , C = c ] , and E [ Y | do ( X i = x i , X j = x j ) , C = c ] , when can we learn conditional average causal effects:

Original: [Equation]
Revised: E [ Y | do ( X i

Original: In what follows, we ﬁrst show that this quantity is not identiﬁable without restrictions on the causalmodel—a proof by counterexample.
Revised: = x i ) , X j = x j , C = c ] , or E [ Y | X i

Original: We then go on to prove, again by counterexample, that simplyassuming ANMs without restrictions on the structure of the causal graph—the core assumption
Revised: = x i , do ( X j = x j ) , C = c ]?

Original: P M ( Y, X 2 | do ( X 1 ))
Revised: P M ( Y, X 2 | do ( X 1 ))

Original: Y =
Revised: Y =

Original: Y =do ( X 1 = 0)
Revised: Y =do ( X 1 = 0)

Original: X 2 =
Revised: X 2 =

Original: X 2 =  do ( X 1 = 1)
Revised: X 2 =  do ( X 1 = 1)

Original: X 2 =1  p
Revised: X 2 =1 − p

Original: X 2 = p
Revised: X 2 = p

Original: P M 0 ( Y, X 2 | do ( X 1 ))
Revised: P M ′ ( Y, X 2 | do ( X 1 ))

Original: Y =
Revised: Y =

Original: Y =do ( X 1 = 0)
Revised: Y =do ( X 1 = 0)

Original: X 2 =1  p
Revised: X 2 =1 − p

Original: X 2 =
Revised: X 2 =

Original: p do ( X 1 = 1)
Revised: p do ( X 1 = 1)

Original: X 2 =1  p X 2 = 
Revised: X 2 =1 − p X 2 = p

Revised: In what follows, we first show that this quantity is not identifiable without restrictions on the causal model—a proof by counterexample.

Original: p made by Saengkyongam and Silva (2020)—is not enough for this complementary research question.
Revised: We then go on to prove, again by counterexample, that simply assuming ANMs without restrictions on the structure of the causal graph—the core assumption made by [Saengkyongam and Silva, 2020]—is not enough for this complementary research question.

Original: Finally, we prove identiﬁability of the treatment effect for ANMs without causal interactions betweentreatments.
Revised: Finally, we prove identifiability of the treatment effect for ANMs without causal interactions between treatments.

Original: Note that this latter case does not mean treatments are independent: treatments can beinﬂuenced by observed covariates and unobserved confounders, and their interactions on the outcomeare deﬁned by the unrestricted function f Y .
Revised: Note that this latter case does not mean treatments are independent: treatments can be influenced by observed covariates and unobserved confounders, and their interactions on the outcome are defined by the unrestricted function f Y .

Original: 3.1 Unidentiﬁability for general SCMs
Revised: 3.1 Unidentifiability for general SCMs

Original: [Table]
Revised: [Table]

Original: [Table] [Table] For simplicity, but without loss of generality, we consider two treatment variables { X 1 , X 2 } and no covari-144 ates.
Revised: For simplicity, but without loss of generality, we consider two treatment variables { X 1 , X 2 } and no covari- ates.

Original: We will show that two different SCMs
Revised: We will show that two different SCMs

Original: M and
Revised: M and

Original: M 0 can yield identical observational distributions, as146 well as joint interventional distributions.
Revised: M ′ can yield identical observational distributions, as well as joint interventional distributions.

Original: They will alsoagree on the single-variable interventional distributionfor treatment X 2 , but disagree on the single-variable effect of treatment X 1 .
Revised: They will also agree on the single-variable interventional distribution for treatment X 2 , but disagree on the single-variable effect of treatment X 1 .

Original: As such, given data fromsets of interventions, this example shows the treatment effect of single-variable interventions is notgenerally identiﬁable.
Revised: As such, given data from sets of interventions, this example shows the treatment effect of single-variable interventions is not generally identifiable.

Original: Our SCMs are deﬁned in Table 1, where U 1 ={+
+}
Revised: Our SCMs are defined in Table 1, where U {+
+}

Original:  U 2 =
Revised: = U 2 =

Original: U Y ⇠ Bernoulli ( p ) .
Revised: U Y ∼ Bernoulli ( p ) .

Original: Theobservational, joint, and single-variable interventional distributions on X 2 are identical under M and
Revised: The observational, joint, and single-variable interventional distributions on X 2 are identical under M and

Original: M 0 ; we defer them to the supplemental material.
Revised: M ′ ; we defer them to the supplemental material.

Original: The interventional distribution on X 1 differs forMand M 0 , as shown in Table 2.
Revised: The interventional distribution on X 1 differs for M and M ′ , as shown in Table 2.

Original: This counterexample shows that single-variable interventional effectsare not identiﬁable for general SCMs.
Revised: This counterexample shows that single-variable interventional effects are not identifiable for general SCMs, given observational data and joint interventions.

Original: 3.2 Unidentiﬁability for general ANMs
Revised: 3.2 Unidentifiability for general ANMs

Original: An Additive Noise Model is an SCM where the inﬂuence of the latent noise variables is restricted tobe additive in the structural equations.
Revised: An Additive Noise Model is an SCM where the influence of the latent noise variables is restricted to be additive in the structural equations.

Original: That is, Eq. 1 is restricted to the form:
Revised: That is, Eq. 1 is restricted to the form:

Original: [Equation]
Revised: [Equation]

Original: Following Saengkyongam and Silva (2020), we additionally assume the noise distribution to be azero-centered Gaussian with an arbitrary covariance matrix: P U ⇠ N (0 , ⌃ ) .
Revised: Following [Saengkyongam and Silva, 2020], we additionally assume the noise distribution to be a zero-centered Gaussian with an arbitrary covariance matrix: P U ∼ N (0 , Σ) .

Original: Table 3 deﬁnes two
Revised: Table 3 defines two

Original: SCMs that satisfy these assumptions.
Revised: SCMs that satisfy these assumptions.

Original: M and M 0 yield identical observational, joint interventional,and single-interventional distributions on X 2 .
Revised: M and M ′ yield identical observational, joint interventional, and single-interventional distributions on X 2 .

Original: However—they disagree on the causal effect ofintervening on X 1 .
Revised: However—they disagree on the causal effect of intervening on X 1 .

Original: An intuitive underlying reason for this, is that we can identify the expression 
Revised: An intuitive underlying reason for this, is that we can identify the expression E

Original: E [ X 2 | X 1 = x 1 ] = f 2 ( x 1 ) + E
Revised: [ X 2 | X 1 = x 1 ] = f 2 ( x 1 ) + E

Original: [ U 2 | X 1 = x 1 ] , but have no tools to disentangle the effects comingfrom the structural equation, f 2 ( x 1 ) , from those stemming from the additive noise, E [ U 2 | X 1 = x 1 ] .
Revised: [ U 2 | X 1 = x 1 ] , but have no tools to disentangle the effects coming from the structural equation, f 2 ( x 1 ) , from those stemming from the additive noise, E [ U 2 | X 1 = x 1 ] .

Original: As a result, E [ X 2 | do ( X 1 = x 1 )]
Revised: As a result, E [ X 2 | do ( X 1 = x 1 )]

Original: = f 2 ( x 1 ) is not identiﬁable, proving that general ANMs withunrestricted causal structures are insufﬁcient for disentangling treatment effects.
Revised: = f 2 ( x 1 ) is not identifiable, proving that general ANMs with unrestricted causal structures are insufficient for disentangling treatment effects.

Original: In this example,and in general SCMs of this structure, joint interventions can be disentangled for the consequencetreatment X 2 , but not for the causing treatment X 1 :
Revised: In this example, and in general SCMs of this structure, joint interventions can be disentangled for the consequence treatment X 2 , but not for the causing treatment X 1 :

Original: Theorem 1 (Identiﬁability of disentangled conditional average treatment effects in additive noisemodels with a causal dependency between treatments) .
Revised: Theorem 1 (Identifiability of disentangled conditional average treatment effects in additive noise models with a causal dependency between treatments) .

Original: Let M = h { C , X , Y } , U , f , P U i be an SCM X i = f i ( C )
Revised: Let M = ⟨{ C , X , Y } , U , f , P U ⟩ be an SCM, where

Original: + U i ,

Original: X j = f j ( C , X i ) + U j ,

Original: [Equation]
Revised: [Equation]

Original: = [Equation] Y [Equation]
Revised: and P U ∼ N (0 , Σ) .

Original: , X  )[Equation] U Y , ⇠ N (0 , ⌃
Revised: The estimand E [ Y | do ( X j ) , C ] is identifiable from the conjunction of two data regimes: (1) the observational distribution, and (2) the joint interventional distribution on ( X i , X j ) .

Original: [Equation]
Revised: [Equation]

Original: where ⌃ = " 1 11 11 1 1 # , and ⌃ 0 = " 1 00 01 0 1 # .
Revised: where Σ = (cid:34) 1 11 11 1 1 (cid:35) , and Σ ′ = (cid:34) 1 00 01 0 1 (cid:35) .

Original: We prove this by showing that the structural equations are identiﬁable from the joint interventionalregime—whereas the necessary (co-)variances are identiﬁable from observational data.
Revised: We prove this by showing that the structural equations are identifiable from the joint interventional regime—whereas the necessary (co-)variances are identifiable from observational data.

Original: Formal proofsare deferred to the supplementary material.
Revised: Formal proofs are deferred to the supplementary material.

Original: 3.3 Identiﬁability for Symmetric ANMs
Revised: 3.3 Identifiability for Symmetric ANMs

Original: The crux of the problem of non-identiﬁability in Section 3.2 comes from the fact that treatment X 1has a direct causal effect on treatment X 2 .
Revised: The crux of the problem of non-identifiability in Section 3.2 comes from the fact that treatment Xhas a direct causal effect on treatment X 2 .

Original: In many realistic applications, this might never occur.
Revised: In many realistic applications, this might never occur.

Original: Treatments will decidedly be correlated, but this can be encoded in the SCM via either the observedcovariates or the unobserved confounders.
Revised: Treatments will decidedly be correlated, but this can be encoded in the SCM via either the observed covariates or the unobserved confounders.

Original: Then still, as we have no restrictions on the functionalform of the structural equations with respect to the treatments (i.e. f Y ), treatments can still yieldhighly nonlinear interaction effects on the outcome.
Revised: Then still, as we have no restrictions on the functional form of the structural equations with respect to the treatments (i.e. f Y ), treatments can still yield highly nonlinear interaction effects on the outcome.

Original: As such, we now focus on the case wherethe SCM is devoid of causal links between treatments, but includes observed covariates as well asunobserved confounders.
Revised: As such, we now focus on the case where the SCM is devoid of causal links between treatments, but includes observed covariates as well as unobserved confounders.

Original: In this case, the structural equations take the following form:
Revised: In this case, the structural equations take the following form:

Original: [Equation]
Revised: [Equation]

Original: Identiﬁability in this setting requires additional assumptions about the correlation between thecovariates and confounders.
Revised: Identifiability in this setting requires additional assumptions about the correlation between the covariates and confounders.

Original: That is, without any assumptions on the nature of the relation between
Revised: That is, without any assumptions on the nature of the relation between

Original: C and U , our estimand is not identiﬁable.
Revised: C and U , our estimand is not identifiable.

Original: We include a proof (by counterexample) of this result inthe supplemental material.
Revised: We include a proof (by counterexample) of this result in the supplemental material.

Original: Intuitively, the problem is that allowing E [ U Y | C ] to be non-zero impactsthe identiﬁability of f Y under joint interventions—as f Y also depends on C —whilst this is crucialto allow for disentanglement.
Revised: Intuitively, the problem is that allowing E [ U Y | C ] to be non-zero impacts the identifiability of f Y under joint interventions—as f Y also depends on C —whilst this is crucial to allow for disentanglement.

Original: As we have shown additional restrictions on the class of SCMs arenecessary, we now assume independence between C and U .
Revised: As we have shown additional restrictions on the class of SCMs are necessary, we now assume independence between C and U .

Original: Future work will explore whether thiscan be relaxed to allow for linear relationships, among others.
Revised: Future work can explore whether this can be relaxed to allow for linear relationships, among others.

Original: Fig.
Revised: Fig.

Original: 1 visualises the structure of the causal graph we assume, when we have K treatment variables.
Revised: 1 visualises the structure of the causal graph we assume, when we have K treatment variables.

Original: We call such causal graphs “symmetric”.
Revised: We call such causal graphs “symmetric”.

Original: SCMs with this structure occur in many application areasfrom economics to online advertising and beyond.
Revised: SCMs with this structure occur in many application areas from economics to online advertising and beyond.

Original: In SCMs of this form, the causal effect of allsingle-variable interventions can be disentangled from observational and joint interventional data:
Revised: In SCMs of this form, the causal effect of all single-variable interventions can be disentangled from observational and joint interventional data:

Original: [Equation]
Revised: [Equation]

Original: U Y C Theorem 2 (Identiﬁability of disentangled conditionalaverage treatment effects in additive noise models withsymmetric structure) 
Revised: Theorem 2 (Identifiability of disentangled conditional average treatment effects in additive noise models with symmetric structure) .

Original: Let M = h { C , X , Y } , U , f , P U i be an SCM, where
Revised: Let M = ⟨{ C , X , Y } , U , f , P U ⟩ be an SCM, where

Original: X i =
Revised: [Equation]

Original: f i ( C ) + U
Revised: C ⊥⊥ U

Original: i , 8 i = 1 , . . , K, Y = f Y ( C , X ) + U Y ,
Revised: [Equation]

Original: C ?? U , and P U⇠ N (0 , ⌃ ) .
Revised: and P U

Original: The estimand
Revised: [Equation]

Original:  E [ Y | do ( X i ) , C ] is identiﬁable from the conjunction oftwo data regimes: (1) the observational distribution, and(2) any interventional distribution on a set of treatments
Revised: The estimand E [ Y | do ( X i ) , C ] is identifiable from the conjunction of two data regimes: (1) the observational distribution, and (2) any interventional distribution on a set of treatments

Original: X int ✓ X that holds X i : X i 2 X int .
Revised: X int ⊆ X that holds X i : X i ∈ X int .

Original: In this section, we have studied for several classes of SCMs whether causal effects can be disentangled.
Revised: In this section, we have studied for several classes of SCMs whether causal effects can be disentangled.

Original: Our results provide insights into the fundamental limits of learning and inference, and help crystallisewhich assumptions are necessary and sufﬁcient to make disentanglement of effects from sets ofinterventions feasible.
Revised: Our results provide insights into the fundamental limits of learning and inference, and help crystallise which assumptions are necessary and sufficient to make disentanglement of effects from sets of interventions feasible.

Original: Figure 2 visualises and summarises our key results denoting which singlevariable causal effects can be disentangled in ANMs with zero-mean Gaussian noise.
Revised: Figure 2 visualises and summarises our key results denoting which singlevariable causal effects can be disentangled in ANMs with zero-mean Gaussian noise.

Original: In what follows,we present a learning methodology and validate our theoretical identiﬁability results with empiricalobservations on synthetic and real data.[Figure] [Equation] [Figure]
Revised: In what follows, we present a learning methodology and validate our theoretical identifiability results with empirical observations on synthetic and real data.

Original: U

Original: [Figure]
Revised: [Figure]

Original:  Methodology
Revised:  Methodology

Original: In this section, we introduce our methodology for estimating SCMs and providing estimates for treatment effects under any set of interventions.
Revised: In this section, we introduce our methodology for estimating SCMs and providing estimates for treatment effects under any set of interventions.

Original: Estimating an SCM from a combination of observationaland interventional data boils down to (1) estimating the structural equations, and (2) estimating thenoise distribution.
Revised: Estimating an SCM from a combination of observational and interventional data boils down to (1) estimating the structural equations, and (2) estimating the noise distribution.

Original: We extend the Expectation-Maximisation-style iterative algorithm proposed by Saengkyongam and Silva (2020) to handle observed covariates, and to disentangle causal effectsinstead of combining them.
Revised: We extend the Expectation-Maximisation-style iterative algorithm proposed by [Saengkyongam and Silva, 2020] to handle observed covariates, and to disentangle causal effects instead of combining them.

Original: The resulting method is not limited to learning single-variable causaleffects, as the results from Saengkyongam and Silva (2020) allow us to extend these newly learnedsingle-variable effects to sets of interventions.
Revised: The resulting method is not limited to learning single-variable causal effects, as the results from [Saengkyongam and Silva, 2020] allow us to extend these newly learned single-variable effects to sets of interventions.

Original: As a result, our method is able to generalise to setsof interventions that were never observed in the training data, with the only restriction that everyvariable that makes up the set was part of some intervention set in the training data.
Revised: As a result, our method is able to generalise to sets of interventions that were never observed in the training data, with the only restriction that every variable that makes up the set was part of some intervention set in the training data.

Original: Say we intervene on a subset of treatments X int ✓ X , and X obs ⌘ X \ X int .
Revised: Say we intervene on a subset of treatments X int ⊆ X , and X obs ≡ X \ X int .

Original: In general, we can228 decompose a causal query with interventions on X int as follows:
Revised: In general, we can decompose a causal query with interventions on X int as follows:

Original: [Equation]
Revised: [Equation]

Original: Samples consist of observed values for all endogenous variables.
Revised: Samples consist of observed values for all endogenous variables.

Original: For convenience, we denote asample by x .
Revised: For convenience, we denote a sample by x .

Original: Suppose we have data from d different data regimes, corresponding to d different setsof interventions (the empty set ; corresponds to the observational regime).
Revised: Suppose we have data from d different data regimes, corresponding to d different sets of interventions (the empty set ∅ corresponds to the observational regime).

Original: The full dataset consistsof samples and their corresponding interventions D = { ( X int ; x ) } .
Revised: The full dataset consists of samples and their corresponding interventions D = { ( X int ; x ) } .

Original: Algorithm
Revised: Algorithm

Original: 1: Input: Dataset D 2: Output: Parameter estimates b ✓ , b ⌃ 3: Initialise b ✓ and b ⌃ 4: while not converged do 5: //
Revised: 1: Input: Dataset D 2: Output: Parameter estimates (cid:98) θ , (cid:98) Σ 3: Initialise (cid:98) θ and (cid:98) Σ 4: while not converged do 5: //

Original: Solve for ✓ with ﬁxed b ⌃ 6:
Revised: Solve for θ with fixed (cid:98) Σ 6:

Original: Optimise log-likelihood in Eq.7: // Solve for ⌃ with ﬁxed b ✓ 8:
Revised: Optimise log-likelihood in Eq.7: // Solve for Σ with fixed (cid:98) θ 8:

Original: Estimate b ⌃ from b
Revised: Estimate (cid:98) Σ from (cid:98)

Original: U = x  f ( x ; b ✓ ) return b ✓ , b ⌃
Revised: U = x − f ( x ; (cid:98) θ ) return (cid:98) θ, (cid:98) Σ

Original: Estimating the structural equations.
Revised: Estimating the structural equations.

Original: We parameterise the structural equations with ✓ , denoted as f ( · ; ✓ ) .
Revised: We parameterise the structural equations with θ , denoted as f ( · ; θ ) .

Original: The Gaussian likelihood withcovariance matrix ⌃ is denoted as P U ( · ; ⌃ ) .
Revised: The Gaussian likelihood with covariance matrix Σ is denoted as P U ( · ; Σ) .

Original: The likelihood for a single endogenous variable x
Revised: The likelihood for a single endogenous variable x

Original: i is deﬁned as L ( x i ; ✓ , ⌃ ) =
Revised: i is defined as L ( x i ; θ, Σ) =

Original: P U ( x i f i ( PA ( x i ); ✓ ); ⌃ ) .
Revised: P U ( x i − f i ( PA ( x i ); θ ); Σ) .

Original: The likelihood for a samplex is deﬁned as the product of the likelihoods forevery endogenous variable that was not intervened on in that sample:
Revised: The likelihood for a sample x is defined as the product of the likelihoods for every endogenous variable that was not intervened on in that sample:

Original: [Equation]
Revised: [Equation]

Original: Naturally, the log-likelihood of the dataset is then:
Revised: Naturally, the log-likelihood of the dataset is then:

Original: [Equation]
Revised: [Equation]

Original: In principle, any iterative optimisation procedure can be used to maximise Eq.
Revised: In principle, any iterative optimisation procedure can be used to maximise Eq.

Original:  when we 
Revised: 6 when we fix the covariance Σ . parameterisation.

Original: Estimating the noise distribution.
Revised: Estimating the noise distribution.

Original: Following Saengkyongam and Silva (2020), we note that themaximum likelihood estimate for the covariance matrix of a multivariate normal can be directly computed from the sample.
Revised: Following [Saengkyongam and Silva, 2020], we note that the maximum likelihood estimate for the covariance matrix of a multivariate normal can be directly computed from the sample.

Original: This allows for efﬁcient closed-form computation of this step.
Revised: This allows for efficient closed-form computation of this step.

Original: Additionally,note that the assumption of zero-mean Gaussian noise simpliﬁes the proof and yields an analyticalsolution for this step, but this is not a general limitation.
Revised: Additionally, note that the assumption of zero-mean Gaussian noise simplifies the proof and yields an analytical solution for this step, but this is not a general limitation.

Original: Indeed—what matters is that we canestimate the conditional noise E [ U Y | X obs ] , which can in principle be estimated via a different model.
Revised: Indeed—what matters is that we can estimate the conditional noise E [ U Y | X obs ] , which can in principle be estimated via a different model.

Original: Algorithm 1 shows the full procedure to estimate the parameters of a symmetric ANM.
Revised: Algorithm 1 shows the full procedure to estimate the parameters of a symmetric ANM.

Original: At inferencetime, Eq. 4 allows us to estimate the outcome under any—even unseen—set of interventions.
Revised: At inference time, Eq. 4 allows us to estimate the outcome under any—even unseen—set of interventions.

Original:  Empirical Validation
Revised:  Empirical Validation

Original: We now empirically validate the effectiveness of our method in estimating SCMs from observationaland joint-interventional data, and assess the accuracy of the inferred outcomes under varying sets ofinterventions.
Revised: We now empirically validate the effectiveness of our method in estimating SCMs from observational and joint-interventional data, and assess the accuracy of the inferred outcomes under varying sets of interventions.

Original: We do this both on synthetic data and semi-synthetic data based on real data from amedical study.
Revised: We do this both on synthetic data and semi-synthetic data based on real data from a medical study.

Original: The research questions we wish to answer from empirical results, are the following:
Revised: The research questions we wish to answer from empirical results, are the following:

Original: RQ1: Are we able to accurately estimate the outcome under previously unseen sets of interventions?RQ2: Are we able to accurately estimate the structural equations and the noise distribution?RQ3: How does our method handle varying levels of unobserved confounding?
Revised: RQ1: Are we able to accurately estimate the outcome under previously unseen sets of interventions? RQ2: Are we able to accurately estimate the structural equations and the noise distribution? RQ3: How does our method handle varying levels of unobserved confounding?

Original: 5.1 Synthetic experiment
Revised: 5.1 Synthetic experiment

Original: First we adopt a simulation setup, which gives us the freedom to vary the true underlying SCMand observe performance differences among competing methods.
Revised: First we adopt a simulation setup, which gives us the freedom to vary the true underlying SCM and observe performance differences among competing methods.

Original: The structural equations f i , f Y in
Revised: The structural equations f i , f Y in

Original: Eq.
Revised: Eq.

Original: 3 are polynomials with second-order interactions to illustrate the effectiveness of the learningmethod even when treatment effects are highly non-linear, and the optimisation surface is highlynon-convex.
Revised: 3 are polynomials with second-order interactions to illustrate the effectiveness of the learning method even when treatment effects are highly non-linear, and the optimisation surface is highly non-convex.

Original: We use the well-known Adam optimiser in our experiments (Kingma and Ba, 2014).
Revised: We use the well-known Adam optimiser in our experiments [Kingma and Ba, 2014].

Original: Asa baseline learning method we adopt a regression model that estimates the outcome from its directtreatments using pooled data from different regimes.
Revised: As a baseline learning method we adopt a regression model that estimates the outcome from its direct treatments using pooled data from different regimes.

Original: Here, the presence of unobserved confoundersseverely hinders the method to provide accurate estimates under varying sets of interventions, furthermotivating the need for causal models.
Revised: Here, the presence of unobserved confounders severely hinders the method to provide accurate estimates under varying sets of interventions, further motivating the need for causal models.

Original: We let | C | = | X | = 4 , yielding ✓ i 2 R8 f i , and ✓ y 2 R 37 for the parameterisation of f y .[-
-]
Revised: We let | C | = | X | = 4 , yielding θ i ∈ R∀ f i , and θ y ∈ R 37 for the parameterisation of f y .[-
-]

Original: The parameters for the structural equations are randomly sampled from a uniform distribution over[  2 , +2] .
Revised: The parameters for the structural equations are randomly sampled from a uniform distribution over [ − 2 , +2] .

Original: The covariance matrix ⌃ 2 R 5 ⇥ 5 is uniformly sampled from a uniform distribution over[  1 , +1] , and then ensured to be positive semi-deﬁnite through Ledoit-Wolf shrinkage (Ledoit and Wolf, 2004).
Revised: The covariance matrix Σ ∈ R 5 × 5 is uniformly sampled from a uniform distribution over [ − 1 , +1] , and then ensured to be positive semi-definite through Ledoit-Wolf shrinkage [Ledoit and Wolf, 2004].

Original: We repeat this process 10 times with varying random seeds and report conﬁdenceintervals over obtained measurements.
Revised: We repeat this process 10 times with varying random seeds and report confidence intervals over obtained measurements.

Original: We vary the size of the training sample |D| 2 { 2 i 8 i = 5 , . . .
Revised: We vary the size of the training sample |D| ∈ { 2 i ∀ i = 5 , . . .

Original: , 13 } , where the set of intervened treat-279 ments X int is one of { ; ; { X 0 , X 1 } ; { X 1 , X 2 } ; { X 2 , X 3 } ; } .
Revised: , 13 } , where the set of intervened treat- ments X int is one of {∅ ; { X 0 , X 1 } ; { X 1 , X 2 } ; { X 2 , X 3 } ; } .

Original: As such, no single-variable interventions280 and the majority of the possible sets of interventions are never observed.
Revised: As such, no single-variable interventions and the majority of the possible sets of interventions are never observed.

Original: From this data, we estimatean SCM using the procedure laid out in Algorithm 1.
Revised: From this data, we estimate an SCM using the procedure laid out in Algorithm 1.

Original: Then, for every possible ( 2 | X | ) set of interventions, we simulate 10 000 samples and estimate theoutcome based on our estimated SCM using Eq. 4.
Revised: Then, for every possible ( 2 | X | ) set of interventions, we simulate 10 000 samples and estimate the outcome based on our estimated SCM using Eq. 4.

Original: We report the Mean Absolute Error (MAE)between our estimated outcome and the true outcome.
Revised: We report the Mean Absolute Error (MAE) between our estimated outcome and the true outcome.

Original: Note that, as the true outcome is Gaussian, theoptimal estimate is the location of that Gaussian, which will have an expected deviation of q 2 ⇡ where  is the standard deviation on the noise parameter U Y .
Revised: Note that, as the true outcome is Gaussian, the optimal estimate is the location of that Gaussian, which will have an expected deviation of (cid:113) 2 π σ where σ is the standard deviation on the noise parameter U Y .

Original: Estimating outcomes (RQ1).
Revised: Estimating outcomes (RQ1).

Original: Figure 3 visualises the results from the procedure laid out above,increasing the size of the training sample over the x-axis and reporting the MAE on the y-axis.
Revised: Figure 3 visualises the results from the procedure laid out above, increasing the size of the training sample over the x-axis and reporting the MAE on the y-axis.

Original: Notethat both axes are logarithmically scaled, and the shaded regions indicate 95% conﬁdence intervals.
Revised: Note that both axes are logarithmically scaled, and the shaded regions indicate 95% confidence intervals.

Original: The plots clearly indicate that our method is able to provide accurate and close to optimal estimatesfor the outcome, under any possible set of interventions.
Revised: The plots clearly indicate that our method is able to provide accurate and close to optimal estimates for the outcome, under any possible set of interventions.

Original: Indeed, the plots labelled observational ,do ( X 0 , X 1 ) , do ( X 1 , X 2 ) and do ( X 2 , X 3 ) show the accuracy of our method to estimate the outcomeunder interventions that occurred in the training data—but the remaining twelve plots relate to 
Revised: Indeed, the plots labelled observational , do ( X 0 , X 1 ) , do ( X 1 , X 2 ) and do ( X 2 , X 3 ) show the accuracy of our method to estimate the outcome under interventions that occurred in the training data—but the remaining twelve plots relate to previously unseen data regimes. While our results allow us to disentangle joint interventions,

Original:  [Figure] [Figure]
Revised: the [Figure] [Figure]

Original: previously unseen data regimes. While our resultsallow us to disentangle joint interventions, theresults of Saengkyongam and Silva (2020) allow us to combine interventions.
Revised: We validate is results of [Saengkyongam and Silva, 2020] allow us to combine interventions.

Original: As such, our methodincorporates (and subsumes) theirs, in order to generalise to arbitrary sets of interventions.
Revised: As such, our method incorporates (and subsumes) theirs, in order to generalise to arbitrary sets of interventions.

Original: In contrast,the regression method that is oblivious to confounders fails to accurately estimate the outcome underany data regime—even those that are seen in the training data, or those where confounders yield noinﬂuence on the outcome (i.e. all treatments are intervened on).
Revised: In contrast, the regression method that is oblivious to confounders fails to accurately estimate the outcome under any data regime—even those that are seen in the training data, or those where confounders yield no influence on the outcome (i.e. all treatments are intervened on).

Original: As such, the reported results validatethat the proposed method provides an unbiased and consistent estimator for the disentangled causaleffect, learned from sets of interventions in the presence of unobserved confounders.
Revised: As such, the reported results validate that the proposed method provides an unbiased and consistent estimator for the disentangled causal effect, learned from sets of interventions in the presence of unobserved confounders.

Original: Estimating SCMs (RQ2).
Revised: Estimating SCMs (RQ2).

Original: We visualise the results with respect to the accuracy of the parameterestimates in Figure 4.
Revised: We visualise the results with respect to the accuracy of the parameter estimates in Figure 4.

Original: These measurements are obtained using the same procedure and runs asfor RQ1.
Revised: These measurements are obtained using the same procedure and runs as for RQ1.

Original: Increasing the size of the training sample over the x-axis, the leftmost plot shows theaccuracy of the estimated parameters for f Y .
Revised: Increasing the size of the training sample over the x-axis, the leftmost plot shows the accuracy of the estimated parameters for f Y .

Original: The rightmost plot shows the accuracy of the estimatedcovariance matrix for the multivariate normal that deﬁnes the noise distribution.
Revised: The rightmost plot shows the accuracy of the estimated covariance matrix for the multivariate normal that defines the noise distribution.

Original: As the baselineregression method is oblivious to the noise distribution, it is not included in the latter plot.
Revised: As the baseline regression method is oblivious to the noise distribution, it is not included in the latter plot.

Original: Both axesare logarithmically scaled, and the shaded regions indicate 95% conﬁdence intervals.
Revised: Both axes are logarithmically scaled, and the shaded regions indicate 95% confidence intervals.

Original: We observe thatour method is able to efﬁciently and effectively estimate the underlying causal model—empiricallyvalidating the identiﬁability results presented in this work.
Revised: We observe that our method is able to efficiently and effectively estimate the underlying causal model—empirically validating the identifiability results presented in this work.

Original: 5.2 Semi-synthetic experiment based on real medical data
Revised: 5.2 Semi-synthetic experiment based on real medical data

Original: We now show our method can accurately estimate unseen sets of interventions ( RQ1 ) and demonstrateit is consistent under varying levels of unobserved confounding ( RQ3 ) using a semi-synthetic setupon real-world data from the International Stroke Trial database (Carolei, 1997).
Revised: We now show our method can accurately estimate unseen sets of interventions ( RQ1 ) and demonstrate it is consistent under varying levels of unobserved confounding ( RQ3 ) using a semi-synthetic setup on real-world data from the International Stroke Trial database [Carolei, 1997].

Original: This was a large,randomized trial of up to 14 days of antithrombotic therapy after stroke onset.
Revised: This was a large, randomized trial of up to 14 days of antithrombotic therapy after stroke onset.

Original: There are two possibletreatments: the aspirin allocation dosage, and the heparin allocation dosage.
Revised: There are two possible treatments: the aspirin allocation dosage α a , and the heparin allocation dosage α h .

Original: The goal is to understandthe effects of these treatments on a composite outcome, a continuous value in [0 , 1] predicting thelikelihood of patients’ recovery.
Revised: The goal is to understand the effects of these treatments on a composite outcome, a continuous value in [0 , 1] predicting the likelihood of patients’ recovery.

Original: [Figure]
Revised: [Figure]

Original: We partially follow the semi-syntheticsetup laid out in Appendix 3 of Zhang and Bareinboim (2021).
Revised: We partially follow the semi-synthetic setup laid out in Appendix 3 of [Zhang and Bareinboim, 2021].

Original: Speciﬁcally, we adopttheir probability table for the joint observational distribution of the covariates: thegender, age and conscious state of a pa tient.
Revised: Specifically, we adopt their probability table for the joint observational distribution of the covariates: the gender, age and conscious state of a patient.

Original: This table was computed from thedataset to reﬂect a real-world observationaldistribution.
Revised: This table was computed from the dataset to reflect a real-world observational distribution.

Original: While Zhang and Bareinboim(2021) deal with single discrete treatments,we extend to the continuous and multitreatment case where treatments can be interpreted as varying dosages of aspirin andheparin.
Revised: While [Zhang and Bareinboim, 2021] deal with single discrete treatments, we extend to the continuous and multi-treatment case where treatments can be interpreted as varying dosages of aspirin and heparin.

Original: This procedure is described in detail in the Supplementary Material.
Revised: This procedure is described in detail in the Supplementary Material.

Original: We add zero-mean Gaussian noise to the treatments and the outcome, where we randomly generatepositive semi-deﬁnite covariance matrices with bounded non-diagonal entries—varying the limit onthe size of the covariances in order to assess the effect of varying confounding on our method.
Revised: We add zero-mean Gaussian noise to the treatments and the outcome, where we randomly generate positive semi-definite covariance matrices with bounded non-diagonal entries—varying the limit on the size of the covariances in order to assess the effect of varying confounding on our method.

Original: Werepeat this process 5 times, and sample 512 observational and 512 joint-interventional samples (both↵ a and ↵ h to learn the SCM from).
Revised: We repeat this process 5 times, and sample 512 observational and 512 joint-interventional samples (both α a and α h to learn the SCM from).

Original: We evaluate our learning methods on 5 000 evaluation samples topredict the outcome under a single-variable intervention on the aspirin dose ↵ a .
Revised: We evaluate our learning methods on 5 000 evaluation samples to predict the outcome under a single-variable intervention on the aspirin dose α a .

Original: Figure 5 shows results with respect to the accuracy of the learnt model for making predictions onthe outcome under varying single-variable interventions on the aspirin dosage ↵ a .
Revised: Figure 5 shows results with respect to the accuracy of the learnt model for making predictions on the outcome under varying single-variable interventions on the aspirin dosage α a .

Original: Naturally, as theregression method has no way of accounting for the confounding effect, it fails dramatically.
Revised: Naturally, as the regression method has no way of accounting for the confounding effect, it fails dramatically.

Original: Ourproposed method, is able to consistently approximate the optimal performance—which is attainedby predicting the expectation of the outcome distribution and limited by its variance.
Revised: Our proposed method is able to consistently approximate the optimal performance—which is attained by predicting the expectation of the outcome distribution and limited by its variance.

Original: For increasinglevels of confounding, the gap between the optimal performing oracle and our learnt model increasesslightly.
Revised: For increasing levels of confounding, the gap between the optimal performing oracle and our learnt model increases slightly.

Original: As such, this semi-synthetic experiment validates the effectiveness of our proposed learningmethod, which can predict outcomes under unseen sets of interventions ( RQ1 ) and is robust tovarying levels of confounding ( RQ3 ), for a use-case based on real-world medical data.
Revised: As such, this semi-synthetic experiment validates the effectiveness of our proposed learning method, which can predict outcomes under unseen sets of interventions ( RQ1 ) and is robust to varying levels of confounding ( RQ3 ), for a use-case based on real-world medical data.

Original:  Conclusions & Future Work
Revised:  Conclusions & Future Work

Original: In this work, we motivated the need for methods that can disentangle the effects of single interventionsfrom jointly applied interventions.
Revised: In this work, we motivated the need for methods that can disentangle the effects of single interventions from jointly applied interventions.

Original: As multiple interventions can interact in possibly complex ways,this is a challenging task; even more so in the presence of unobserved confounders.
Revised: As multiple interventions can interact in possibly complex ways, this is a challenging task; even more so in the presence of unobserved confounders.

Original: First, we provedthat such disentanglement is not possible in the general setting, even when we restrict the inﬂuenceof the unobserved confounders to be additive in nature.
Revised: First, we proved that such disentanglement is not possible in the general setting, even when we restrict the influence of the unobserved confounders to be additive in nature.

Original: By restricting the structure of the causalgraph to be symmetric we showed identiﬁability can be acquired.
Revised: By restricting the structure of the causal graph to be symmetric we showed identifiability can be acquired.

Original: Additionally, we showed how toincorporate observed covariates, and have empirically demonstrated our method.
Revised: Additionally, we showed how to incorporate observed covariates, and have empirically demonstrated our method.

Original: Future work will tackle the case where the noise distribution is not restricted to a zero-mean multivariate Gaussian.
Revised: Future work will tackle the case where the noise distribution is not restricted to a zero-mean multivariate Gaussian.

Original: As mentioned in Section 4, this assumption provides a closed-form expression for theconditional expectation of the outcome noise given the observed treatments—but universal functionapproximators could be used to obtain similar guarantees in more general model classes.
Revised: As mentioned in Section 4, this assumption provides a closed-form expression for the conditional expectation of the outcome noise given the observed treatments—but universal function approximators could be used to obtain similar guarantees in more general model classes.

Original: Investigatingwhether the assumption of causal independence between the unobserved confounders and covariatescan be relaxed, to allow for linear relationships say, provides another avenue to explore. 
Revised: Investigating whether the assumption of causal independence between the unobserved confounders and covariates can be relaxed, to allow for linear relationships say, provides another avenue to explore. Appendix

Revised: In what follows, we include complete and formal mathematical proofs for the theorems presented in the paper, as well as more information about our experimental setup for the semi-synthetic experiment performed to answer RQ3.

Revised: A Proof of Theorem 2: Identifiability in Symmetric Additive Noise Models

Revised: This section provides a proof for identifiabiltiy of single-variable causal effects in symmetric additive noise models ( Theorem 2 ).

Revised: Proof.

Revised: Our causal estimand is the effect of intervening on X i .

Revised: For notational convenience, we assume k

Revised: = i , i.e. the intervention is on the last treatment.

Revised: From the observational data regime, we can trivially obtain the joint P ( C , X 1 , . . .

Revised: , X k , Y ) .

Revised: As such, we can condition on the covariates and remaining treatment variables and then marginalise to obtain E

Revised: [ Y | do ( X k = x k ) , C = c ] .

Revised: We can rewrite our causal estimand as follows:

Revised: [Equation]

Revised: From the joint interventional data regime, we have access to the following expectation:

Revised: [Equation]

Revised: Subtracting Eq.

Revised: 8 from Eq.

Revised: 7 shows that we only need to provide identifiability for the conditional expectation on the outcome noise, given the remaining treatment variables and the observed covariates:

Revised: [Equation]

Revised: where Σ u y = (cid:2)

Revised: [Equation]

Revised: Y. . .

Revised: σ

Revised: [Equation]

Revised: [Equation]

Revised: Σ u x =

Revised: [Equation]

Revised: u

Revised: [Equation]

Revised: Here, σ ij denotes the covariance between noise variables

Revised: U i and U j , and σ Y i denotes the covariance between the outcome U Y and U i .

Revised: There are two types of unidentified factors in these expressions: the structural equations , that is, the f i ’s encapsulated in u x from Eq. 9, and the parameters of the noise distribution , which are encapsulated in Σ u x and Σ u y .

Revised: We tackle these in what follows:

Revised: Identifying the structural equations.

Revised: As a direct result of our model definition, we obtain E [ X i | C

Revised: = c ]

Revised: = f i ( c ) from the observational data regime.

Revised: This follows because in the DAG of Figure 1, C and U i are independent for all i .

Revised: As such, the structural equations (and thus u x from

Revised: Eq. 9) are identifiable.

Revised: Identifying the noise distribution.

Revised: For any pair of treatment variables, we can obtain E [ X i | C

Revised: = c, X j = x j ] =

Revised: f i ( c ) + E

Revised: [ U i |

Revised: X j = x j ] .

Revised: The latter term in this expression can then be rewritten as E

Revised: [ U i |

Revised: U j = x j − f j ( c )] , for fixed values of x j and c .

Revised: Because we have shown the structural equations ( f i , f j ) to be identifiable, we have that E

Revised: [ U i |

Revised: U j ] is identifiable, which gives us the covariance σ ij .

Revised: Hence, the entirety of Σ u x is identifiable.

Revised: The same procedure can be used for every entry of Σ u y , and the covariances σ Y i are identifiable as a result.

Revised: It follows naturally that E [ Y | C =

Revised: c, X 1 = x 1 , . . .

Revised: , X k − 1 = x k − 1 , do ( X k = x k )] is identifiable.

Revised: As we have data from the observational regime, any marginalisation of this query is identifiable as well, concluding the proof.

Revised: B Unidentifiability under Unconstrained SCMs

Revised: Tables B hold the full distributions for the counterexample in Section 3.1, showing that singlevariable interventional effects are unidentifiable from observational and joint interventional data for unconstrained SCMs.

Revised: [Table]

Revised: [Table]

Revised: Y )

Revised: [Table]

Revised: X

Revised: [Table]

Revised: p

Revised: [Table]

Revised: X

Revised: [Table]

Revised: X

Revised: [Table]

Revised: [Table]

Revised: X

Revised: [Table]

Revised: X

Revised: [Table]

Revised: M

Revised: M ′

Revised: C =

Revised: U C

Revised: C =

Revised: U C

Revised: X 1 = C

Revised: + U

Revised: X 1 = 2 C

Revised: + U

Revised: X 2 = C

Revised: + U

Revised: X 2 = 2 C

Revised: + U

Original: {+Y = CX 1 X 
+}
Revised: {+Y = CX 1 X 
+}

Revised: + U Y

Revised: Y = CX 1 X

Revised: + U Y where Σ =   1 1 11 1 11 1 11 1 1  , Σ ′ =

Revised:   1 0 00 0 00 0 01 0 0  .

Revised: C Unidentifiability when C is not independent of U

Revised: Table 5 provides the counterexample mentioned in Section 3.3 (just after Equation 3) showing that single-variable interventional effects are unidentifiable from observational and joint interventional data when the covariates C are not independent of the unobserved confounders U in the ANMs we consider.

Revised: Indeed, in model M , C is correlated with the latent U 1 , U 2 , telling us this model is non-Markovian: that there is an unobserved confounder correlating C and U 1 , U 2 .

Revised: C is thus not independent of this unobserved confounder.

Revised: This is not the case in model M ′

Revised: Moreover, we can see that M and M ′ are identical under interchange of X 1 and X 2 , and have the same joint and marginal distributions, as well as the same joint interventional distribution.

Revised: As the two SCMs coincide on observational and joint interventional distributions, this proves that the confounding distribution is not identifiable from these data regimes alone, and single-variable interventions as well as the full SCM are unidentifiable.

Revised: This demonstrates the need for the additional restrictions between C and U in Theorem 3.2.

Revised: D Proof of Theorem 1: Identifiability with Causally Dependent Treatments

Revised: This section provides a proof for identifiability of single-variable causal effects when there is a causal dependency among treatments ( Theorem 1 ).

Revised: Observational and joint interventional data are not sufficient in this case to identify causal effects on all treatments – but we can identify the causal effect of intervening on the consequence treatment instead of the causing treatment.

Revised: Proof.

Revised: Our causal estimand is the effect of intervening on X j .

Revised: We can rewrite our causal estimand as follows:

Revised: [Equation]

Revised: From the joint interventional data regime, we have access to the following expectation:

Revised: [Equation]

Revised: Subtracting Eq.

Revised: 11 from Eq.

Revised: 10 shows that we only need to provide identifiability for the conditional expectation on the outcome noise, given the observed value for treatment X i and the observed covariates C :

Revised: [Equation]

Revised: Here, the first step comes from our SCM definition, and the second step comes from the fact that we assume the noise distribution to be a zero-centered multivariate Gaussian.

Revised: As such, we need to identify the function f

Revised: i , the variance on the noise variable U i , and the covariance between U i and

Revised: U Y .

Revised: We obtain E [ X i | C

Revised: =

Revised: c ]

Revised: = f i ( c ) directly from the observational data regime.

Revised: This makes the noise variable U

Revised: i

Revised: =

Revised: X i − f i ( C ) identifiable.

Revised: Now, as a result, we can identify its variance σ ii and covariance σ Y i from the observational data regime, which concludes the proof.

Revised: E Consistency under Varying Levels of Confounding (RQ3)

Revised: In order to validate the consistency of our learning method under varying levels of unobserved confounding ( RQ3 ), we adopt data from the International Stroke Trial database [Carolei, 1997].

Revised: We partially follow the semi-synthetic setup laid out in Appendix 3 of [Zhang and Bareinboim, 2021].

Revised: Specifically, we adopt their probability table for the joint observational distribution of the covariates: the gender, age and conscious state of a patient.

Revised: This table was computed from the dataset to reflect a real-world observational distribution.

Revised: They deal with discrete treatments—which we extend to the continuous case where treatments can be interpreted as varying dosages of aspirin and heparin.

Revised: We model the structural equation on the outcome as:

Revised: [Equation]

Revised: This loosely reflects the same intuitions as laid out in [Zhang and Bareinboim, 2021], where S is the gender, A the age, C the conscious state, α a the aspirin dose and α h the heparin dose.

Revised: Note that this does not reflect correct medical knowledge or insights—the goal is merely to have a polynomial with second-order interactions that we can learn to model.

Revised: As described in the main text, we add zero-mean Gaussian noise to the treatments and the outcome, where we randomly generate positive semi-definite covariance matrices with bounded non-diagonal entries—varying the limit on the size of the covariances in order to assess the effect of varying confounding on our method.

Revised: We repeat this process 5 times, and sample 512 observational and 512 joint-interventional samples (both α a and α h ) to learn the SCM from.

Revised: We evaluate our learning methods on 5 000 evaluation samples to predict the outcome under a single-variable intervention on the aspirin dose α a .

Revised: All experiments ran in a notebook on a laptop.