DeCaFlow: A deconfounding causal generative model
Authors: Alejandro Almodóvar
Adrián Javaloy
Juan Parras
Santiago Zazo
Isabel Valera
Venue: NeurIPS 2025 spotlight ()
Keywords: hidden confounders
causal identifiability
causal query
latent variable model
counterfactuals
Primary area: probabilistic_methods
TL;DR: We propose a causal generative model that accurately estimates a broad class of causal queries, including counterfactuals, in the presence of hidden confounders.

Abstract: We introduce DeCaFlow, a deconfounding causal generative model. Training once per dataset using just observational data and the underlying causal graph, DeCaFlow enables accurate causal inference on continuous variables under the presence of hidden confounders. Specifically, we extend previous results on causal estimation under hidden confounding to show that a single instance of DeCaFlow provides correct estimates for all causal queries identifiable with do-calculus, leveraging proxy variables to adjust for the causal effects when do-calculus alone is insufficient. Moreover, we show that counterfactual queries are identifiable as long as their interventional counterparts are identifiable, and thus are also correctly estimated by DeCaFlow. Our empirical results on diverse settings—including the Ecoli70 dataset, with 3 independent hidden confounders, tens of observed variables and hundreds of causal queries—show that DeCaFlow outperforms existing approaches, while demonstrating its out-of-the-box applicability to any given causal graph.

DeCaFlow: A deconfounding causal generative model
Alejandro Almodóvar';"
Adrián Javaloy?*
Juan Parras'
Santiago Zazo Isabel Valera²
Information Processing and Telecommunications Center, Universidad Politécnica de Madrid, ES
Schooo of Informatics, University of Edinburgh, UK
3 Department of Computer science, Saarland University, DE
Abstract
We introduce DeCaFlow, deconfounding causal generative model. Training once per dataset using just observational data and the underlying causal graph, DeCaFlow enables accurate causal inference on continuous variables under the presence hidden confounders. Specifically, we extend previous results on causal estimation under hidden confounding to show that a single instance of DeCaFlow provides correct estimates for all causal queries identifiable with do-calculus, leveraging proxy variables to adjust for the causal effects when do-calculus alone is insufficient. Moreover, we show that counterfactual queries are identifiable as long their interventional counterparts are identifiable, and thus are also correctly estimated by DeCaFlow. Our empirical results diverse dataset, with independent hidden confounders, tens observed variables and hundreds of causal queries- show that DeCaFlow outperforms existing approaches, while demonstrating its out-of-the- box applicability to any given causal graph.
the Ecoli70
1 Causal generative models and hidden confounding
Causal inference seeks to determine how changes one variable affect others, which is crucial to evaluate the effects of interventions in fields such as healthcare [15], marketing policing [74] or education [85]. In real-world scenarios, where empirical trials often are infeasible due to ethical, financial, or practical constraints, answering causal queries from observational data becomes essential. However, this is challenging task, especially in the presence of unmeasured or hidden confounders affecting subset of the observed variables [1, 20].
In this work, we aim to propose practical approach for accurate causal inference on continuous variables under the presence of hidden confounders. To this end, we build on two key concepts: i) causal generative models (CGMs) [9, 27, 31], class of generative models that can generate samples not only from the observational distribution, but also from interventional and (in some cases) counterfactual distributions; and ii) proxy variables, i.e., conditionally independent variables that yield information about the hidden confounders [45, 46, 46, 76). Consequently, we introduce the deconfounding causal normalizing flow (DeCaFlow). a CGM which provides correct estimates of a broad class of interventional and counterfactual queries under hidden confounding, requiring only observational data, the causal graph, and training once per dataset. Architecturally, DeCaFlow resembles variational autoencoders [33] as it is trained with the ELBO and comprises: i) causal normalizing flow (CNF) [27] "decoder". adapted to be conditioned hidden confounders; and ii) conditional normalizing flow [79] as "encoder". computing the modeled posterior distribution of the hidden confounders given the observations.
the (potentially many)
Equal contribution. Correspondence alejandro almodovar@upm. and ajavaloy@ed. ac. uk.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

We proved theoretically that DeCaFlow yields correct estimates for both interventional and counter- factual queries that are identifiable with do-calculus, levenaging the information of proxy variables when do-calculus alone is insufficient. To that end, we first extend recent advances in proximal causal inference by Miao et al. [45] and Wang and Blei [76] to include counterfactual causal queries. Then, with do-calculus, expanding the number of identifiable queries
we integrate of which DeCaFlow is shown to provide correct estimates.
As proof of the flexibility that DeCaFlow offers, Fig. 1 illustrates the causal graph of the Ecoli70 dataset [66], comprising 43 observed variables and 3 hidden confounders, showing that De- CaFlow can effortlessly scale complex set- tings and accurately recover diverse causal ef- fects after single training process. Remark- ably, green edges in the figure represent direct causal effects that DeCaFlow can identify, des- pite the presence of hidden confounders. We additionally, provide algorithms to help prac- titioners easily check in the given the causal graph whether a particular query of interest can be correctly estimated by DeCaFlow.
abp5
atpo
pupa
Juik
pip4
Figure 1: DeCaFlow can be effortlessly ap- plied to highly complex causal graphs, as that of the Ecoli70 dataset [66], with multiple hidden confounders and dozens of variables. We dash hidden confounders, and highlight direct hidden- confounded effects as identifiable (and thus cor- rectly estimated by DeCaFlow), or unidentifiable.
Finally, we empirically validate all our claims on semi-synthetic and real-world experiments, demonstrating that DeCaFlow outperforms ex- isting alternatives while being widely applic- able. An implementation of DeCaFlow can be found in github. con httpa//mmmmmorrree/eaaaaa
1.1 Related works
We briefly discuss relevant works in the literature, and defer the reader to SE for further details.
Causal generative models. As mentioned above, we refer as CGMs to the class of generative mod- els that can generate samples from the observational, interventional and, in some cases, counterfac- tual distributions. A common recipe to build causally consistent CGMs consists of modeling each variable function of its causal parents with an independent model. In terms of the choice for mod- cling these functions, prior works range from simple yet well-established additive noise models [24]. more complex but powerful diffusion-based causal models [9], among others [35, 51, 52, 59, 83]. Due to their sequential nature, these approaches can overfit and propagate errors to downstream vari- ables. Alternatively, recent works have explored using single (structurally- constrriie)))))) network to model the SCM at once, e.g., using normalizing flows [27, 31], or graph neural networks [65, 84]. However, all the aforementioned approaches assume causal sufficiency, i.e. the absence of hidden confounders, limiting their applicability in settings with hidden confounding.
Causal inference hidden confounding. When dealing with hidden confounding, many approaches handle only interventional queries and are tailored specific causal graph and single treatment- outcome pair, requiring us to train one model for each query we want to answer. Prior works exploit instrumental variables [4], mediators [55], and, more recently, proxy variables account for hidden confounding [2, 28, 37, 39, 44-46, 76]. from which we build upon later in $4. Recent works have aimed to unify causal inference and generative modeling under hidden confounding [48, 80. 81]. In particular, Neural Causal Models perform causal identifiability and estimation under hidden confounding on discrete variables [81, 82] by, given a causal query, training two "adversarial" models and returning their estimation if they coincide. The model by Wasr-Esfahany et al. [48], instead, focuses on counterfactual queries for simple causal graphs where adjustment sets or instrumental variables are available. Our work thus aims to complement this line of work by providing practical and scalable GCM to solve a broad class of causal queries, interventional or counterfactual, on continuous variables and large causal graphs with single end-to-end training.

---

2 Confounded structural causal models
Definition 1. A (confounded) Structural Causal Model (SCM) is a triplet M = Pu. P2) describ- ing a -lttrrnnng process over D observed (endogenous) variables x = (x1, X2...., (p)) E X:
X (pa((i), uz, for i= 1,,,,....... ...,, D, with u (u1, U2 "....,,upp up) Pa, z Ps. (1)
where fi represents the structural equation to compute the i-th endogenous variable, x,, from its observed causal parents, pa(i), the i-th exogenous variable, u, and the hidden confounders, z E Z.
While we make the dependence on the hidden confounders explicit for all observed variables in Eq. we assume w.l.o.g. that a subset of them may not be directly affected by the hidden confounders. Furthermore, given SCM M, we denote by G the faithful causal graph that induces, representing a direct causal relationship between pairs of endogenous and hidden variables only if it exists.
A core idea of causal inference is the do operator [56], denoted by do(t), which formalizes the action of externally intervening on the variable i, i.e. to fix t to a value independently its parents. The do operator enables the computation of interventional and counterfactual queries in SCMs [58]:
A causal query (MM) = (y do(t), c) is a distribution over E x (the outcome variable), as a result of intervening upon the variable x (the treatment variable). Additionally, Q(M) denotes an interventional or counterfactual query if the variable e is, respectively, the empty set or the vector of observed factual values, x².
Definition 2.
We call a causal query identifiable if it can be expressed as function of the observational distribution, (x)(x), and the causal graph 9 [55]. observational distribution produces correct estimates of that causal query. Moreover, any identifiable query can be rewritten this way using a set of three rules, the do-calculus [54]. Yet, in the presence of hidden confounders, this may not be possible and even applying the do-operator to evaluate causal queries would produce incorrect estimates, as unaccounted confounders would bias the results.
result, any SCM inducing the same graph and matching the
3 Deconfounding causal normalizing flows
In this work, we assume the existence of an underlying confounded SCM, .,, as Def. 1, of which we have access to N i.i.d. observations as well as to the faithful causal graph, G. Our objective is to design and learn a CGM that can accurately estimate as many causal queries from the original SCM as possible, despite the presence of unobserved hidden confounding. In other words, we seek substitute model of M that we can use to accurately perform causal inference.
Assumptions. In addition, we assume all variables to be continuous, and the SCM M to: have C² --ddiffeomorphic causal equations conditioned on z, and ii) induce an acyclic causal graph G.
Note that assumption i) implies that f: U x Z + X' is invertible from x to u, given z. This is not limiting assumption, since we never observe u and we can always find an invertible mapping by merging all producing the same observations and taking their while remaining causally equivalent to the original SCM assuming all other assumptions hold.
transport [34, 62),
3.1 (Unconfounded) Causal normalizing flows
Causal normalizing flows (CNFs) [27] play important role in this work, they form the foundations of DeCaFlow, given their identifiability guarantees despite a mild set of assumptions. Given a causal graph G, a CNF, To, is masked autoregressive normalizing flow [50] built such that, paired with distribution Pu. defines an unconfounded SCM Mo = (To. P.) that induces graph g by design.
As demonstrated by Javaloy et al. [27], CNFs represent a remarkable family of CGMs, as they not only form a parametric family of identifiable SCMs, but they can provably approximate the underlying SCM in the three rungs of Pearl's ladder of causation [55] simply by maximizing the observed joint evidence, i.e., maxy log po (x). Furthermore, CNFs are also equipped with an exact do-operator for efficient sampling of any causal query, enabling their use for complex causal- -nffe-en tasks.
Their main downside, as discussed in $1, is that CNFs need to assume causal sufficiency on top of the assumptions made above-to to guarantee the aforementioned capabilities, thus limiting their application. Next, we attempt to address this limitation without losing theoretical guarantees.
3

---

Deconfounding network
Generative network
2 ~ p(z)
Intervened generative network
n
T
W
Un
n
~ (s xx)
Un n
b
Uw
uw 0
W
Uh
u ~ p(u)
ub
b
Y
u1
11 ===o((())
U
t=
Uy
y
Uy
y
(a) DeCaFlow training computations.
(b) Computation of causal queries.
Figure 2: Example of DeCaFlow computations for the causal graph G in Fig. A.2. Circles represent input/output variables of the masked conditional normalizing flows, and black dots conditional inputs. (a) Steps performed during training (Eq. 4), where e is non-causal random variable needed to model z with T.. (b) Steps performed to compute an interventional or counterfactual query with CNFs [27]. see $D, where uf the value for which = a always, i.e, u' = Ta, (T (u)x).
3.2 Deconfounding causal normalizing flows
We now introduce the deconfounding causal normalizing flow (DeCaFlow), a family of models which extend CNFs [27] to account for hidden confounding while retaining all their theoretical properties. To achieve this, DeCaFlow follows the structure of variational autoencoder [33], i.e., DeCaFlow comprises two main components: i) generative network that exploits structural constraints to accurately model the underlying SCM, given a substitute z; and ii) an inference network which approximates the intractable posterior distribution of z modeled by the generative network, given the observed endogenous variables. In the following, we provide further details on both networks.
Generative network. We adapt CNFs [27] to take the hidden confounders as conditional inputs by using conditional masked autoregressive normalizing flows [79], instead of unconditional ones. The resulting model, To, is thus an invertible transformation, conditioned on z, describing a data- generating process that maps a set of exogenous variables to endogenous ones and vice versa, i.e., (x) = u ~ Pu and x = Tø (u), where we further exploit the given causal graph G to ensure that the generative process is faithful, i.e., that
(x x 2) II x=1
(x pa()),
defining data-generating process similar to that of Def. and, just as in that definition, in Eq. 2 only the children of are actually conditioned on
Deconfounding network. To model the posterior distribution of given observations as modeled by To. i.e., the abduction step needed to compute counterfactuals [55], we use another masked autoregressive conditional normalizing flow [79], it can approximate this distribution arbitrarily well. Once again, we exploit knowledge of G and mask the resulting network, T such that it models using only the strictly necessary variables to ease learning:
(z x) = go (z ch(z) U pa(ch(z)))
(3)
We provide in $C more general version of Eq. 3 that accounts for several independent hidden confounders, and empirically validate the architecture and factorization choices in $$B.2 and B.3.
Training process. We jointly train both networks as typically done in deep latent-variable models, ie., we maximize the evidence lower bound (ELBO) [33]:
C(O, Φ) Egg log (x, z)] + H(q (z x) = E.g, log (x z)] - (z x)] p(z)),
(4)
where KL is the Biilbback-Leibles divergence [38], H the differential entropy [36] and p(z) is the prior distribution of z which we set as a standard Gaussian. The motivation for this loss is three-fold: i) we want the generative network to explain the observations given samples from do (first term of Eq. 4); ii) as we do not know the optimal size for z, we need to prevent the deconfounding network from allocating information exclusive of in (entropy term in Eq. 4); and iii) all the results introduced

---

rely on DeCaFlow matching the observational distribution, P.M. (x). which we encourage since
max ( 0) = min (x)] + [T x) 6,0
po x)]
(5)
0,0
To avoid posterior collapse, i.c., that the approximate posterior matches the prior and thus having uninformative latent variables [77], we incorporate KL balancing terms [73] to prevent the KL in Eq. 4 from vanishing, ensuring that the latent representation remains informative during training. Other implementation details, e.g.. the way masking both encoder and decoder with via the causal adjacency matrix, how to adapt the do-operator of CNFs can be found in SSC and D, respectively.
3.3 Inherited causal properties
As consequence of leveraging (causal) normalizing flows, DeCaFlow inherits many of the great properties of this family of models. For once, both components of DeCaFlow are universal dens- ity approximators [50] meaning that, given enough resources, the generative network can perfectly match the observational distribution and the deconfounding network can perfectly learn the modeled posterior. In other words, we can perfectly minimize the two KI. terms that appear in Eq. 5. Further- more, note that the generative network, combined with two base distributions for u and z, defines a confounded SCM as in Def. 1, ie.. MMg- = (To P2).
Causal consistency. As we leverage the causal graph 9 to appropriately mask the conditional CNF of the generative network, have that MM respects all causal connections described by G. In other words, Mg induces the same causal graph 9 as .M. As a result, we can ensure which variable affects which when generate observations with Tg. Fig. 2 depicts these structural constraints relating and z with x, and we provide a detailed description in SC.
Moreover, we prove in $A.1 that DeCaFlow preserves one of the most crucial aspects of CNFs: Identifying (in the sense of Xi and Bloem-Reddy [80]) the underlying SCM concerning those variables that are not directly caused by 2:
Proposition 3.1 (Informal). If DeCaFlow induces the same causal graph and observational distribu- tion as the underlying (confounded) SCM generating the data. Then, DeCaFlow recovers the SCM for every variable not in ch( ch(z). up to element-wis transformation of their exogenous distributions.
DeCaFlow do-operator. While the above result ensures the causal equivalence of Mg and M for unconfounded variables, is still unclear how to intervene on MM.. To this end, we adapt the do- operator of CNFs [27], represented in Fig. 2b and detailed in $D, which provides efficient and exact way of sampling from any interventional and counterfactual distribution. Namely, to sample from an interventional distribution p(x do == ai) over Mo i) sample z Pa and Pu; ii) find the value of u, that yields t = a given u, which we can easily do as To is invertible given z; and iii) return the sample xdo() = Tø.z (uxir. u.) The counterfactual case is quite similar, as the bijectivity of To implies that every counterfactual distribution a delta distribution given z, and we can simply follow the process above but using z ~ 4 (z) x²) and u Te (x') for step one. As result, we can guarantee the correctness of DeCaFlow estimations on a number of causal queries:
Corollary 3.2 (Informal). DeCaFlow provides correct estimates of any causal, interventional or counterfactual, query for which both the treatment and outcome variables not direct children of hidden confounder, i.e., l,y & ch(z).
Thus far, we have shown that for causal queries over non-children of z, DeCaFlow inherits the theoretical guarantees of CNFs. In the following section, investigate under which conditions DeCaFlow can also provide correct estimates of causal queries defined over children of z.
Estimation of causal queries under hidden confounding
By leveraging recent results in preserves the properties of CNFs, but expand them. Namely, we characterize queries which DeCaFlow correctly estimates despite the hidden confounding. While we present here an intuitive summary of our main theoretical results, formal statements and derivations can be found in SA. Throughout this section, we define praxy variables relative to the causal query to estimate: if we are interested in the causal effect of t over y, which are confounded by z, a proxy is an observed variable related to z and conditionally independent of t or (see Prop. 4.1 next for precise definitions). Intuitively, proxy variables contain exploitable information about the hidden confounders, which we leverage this section to provide accurate estimates of the causal queries of interest.
we next show that DeCaFlow not only
5

---

4.1 Interventional queries
First, we consider the identifiability of hidden-confounded interventional queries, i.e., queries of the form Q(M) [y] do(t)), where y, E ch(z) are any two children of the hidden confounder. We summarize our findings in the following proposition, which we properly formalize in $A.2:
Proposition 4.1 (Informal). An interventional query of the form Q(M) PMa do(t)), where y. E ch(z) are two different children of z, identifiable if there exists (potentially empty) subser of blocking variables b C. x {t.y), and two other variables n, w E x {t. yt,y,b) b) such that:
1. (b, z) forms a valid adjustment set, i.e., p(y| do(t)) = If p(yl 1, b, ()p z) cb dz,
2. is proxy variable given b, i.e., w 11 (t, n) b,z,
3. n is a null proxy variable given b, i.e., y 11 n| t, b, z, and
4. both w and n yield enough information about the hidden confounder z.
Prop. 4.1 extends the results from Miao et al. [45] and Wang and Blei [76] to prove identifiable of queries under hidden confounding even if treatment and outcome have observed parents in common. In turn, these results render causal queries identifiable in the infinite-data regime by leveraging proxy information, complementing classical do-calculus [39]. Intuitively, w serves the purpose of building a function which "substitutes" the hidden confounder for that query, and n is used to ensure that this substitute yields the correct estimate. From this result, one natural step is then to extend the class of causal queries which are identifiable using do-calculus, where we introduce the queries identifiable with Prop. 4.1 as additional base case for the do-calculus recursive steps:
Corollary 4.2. An interventional query is identifiable if. using do-cal colccllulus, it can be reduced to combination of observational queries and identifiable interventional queries in the sense of Prop. 4.1.
To understand the implications of Prop. 4.1 and Cor. 4.2, consider the causal graph in Fig. 3, and suppose we want compute Q(M) I do())) Then, we can proceed as usual and apply the rules of probability theory and do-calculus to rewrite (MM) as
n
m
Y1
- f ply, (,)2)((y2) do(t)) dy. (6)
Q(M) ply, do(()) -
As result, the identifiability of p(y₂l do(t)) implies that of Q(M).
Figure 3: Illustrative causal graph where the presence or absence of some parts render p(y, do(t)) identifiable using do-calculus. Else. Prop. 4.1 yields identifiability if w and n are informative proxies.
We can then devise a few different scenarios:
1. If there is no edge from to t, i.e., & ch(z), then the backdoor
criterion holds for (n, b) = pa(t) C x and both p(y, dt)()) and p(yz dt) are identifiable.
2. If there exists a mediator variable between t and y2. m, we can apply the front-door adjustment and both ply, dt) and p(yal do( are identifiable.
3. If y22 is not caused by t, then p(y. do())) = ply2) and both queries are identifiable.
4. Otherwise, we can still render p(ya (t) (and thus p(y, do(t)) identifiable if w and n yield sufficient information about z (intuitively, this means that the posterior of z changes enough as we change w and n; we formalize this notion in Def. 4) and we can hence apply Prop. 4.
The example above nicely illustrates how Prop. 4.1 complements do-calculus: if we find a query unidentifiable due to reaching a dead end with do-calculus- in this case, p(yz then Prop. 4.1 provides an additional case for which the query can still be made identifiable. Moreover, this case clearly shows how Prop. 4.1 extends prior results these did not allow for common observable ancestors between outcome and treatment [45, 76]. Nevertheless, note that Prop. 4.1 provides only sufficient conditions for identifiability, and there could exist identifiable queries which do not comply with the requirements of the proposition.
Finally, recall from $3.3 that, similar to CNFs [27], we can readily interpret the generative network of DeCaFlow as parametric confounded SCM (Def. 1) of the form M0 == (T Pa, Px). This SCM induces the same causal graph as the underlying M by design, G. and since the family of normalizing flows are universal density approximators, Ma can match the observational distribution (x) given enough resources. We can then leverage the previous results to prove the following:

---

Corollary 4.3. If DeCaFlow induces the same causal graph G as M and PM - DeCaFlow provides correct estimates of any query identifiable in the sense of Cor. 4.2.
po px(x), then
In other words, Cor. 3.2 guarantees that, if we match the observational distribution with DeCaFlow, then the do-operator presented in $3.3 provides a correct estimate of the identifiable query of interest.
4.2 Counterfactual queries
Next, we focus on counterfactual queries of the form Q(M) = =MM( do(("), x²), where xf is the observed factual. Intuitively, this query represents the distribution the outcome would have had, had we intervened on the treatment variable. We demonstrate, for the first time to our knowledge, a one- to-one correspondence between proxy-identie proxy-deetiffible interventional and counterfactual queries. More specifically, we show that (all formal derivations can be found in $A.3):
Proposition 4.4 (Informal). If an interventional query plyl do(t)) is identifiable in the sense of Prop. 4.1, then its counterfuctual counterpart, p do(t"), x²), is also identifiable.
The proof of Prop. 4.4 exploits the notion of twin SCM [5], which duplicates the structural equations for the factual and counterfactual worlds while sharing the exogenous variables, and the fact that Prop. A.2 (the formal version of Prop. 4.1) allows for queries with additional covariates as long as they do not form colliders, which is always the case with x' in (y do(f"),x')
as we show in the illustrative twin network of Fig. 4. We can then follow the same derivations from the previous section to show that:
Corollary 4.5. De low induces the same causal graph 9 as M and p. (x) = p)) then DeCaFlow provides correct estimates of any counterfactual query which can be decomposed in combination of (proxy- jidentifiable queries using do-calculus.
Figure 4: Twin SCM with ob- served factual nodes grayed out and edges severed to com- pute p(y=l do(r"), x²) in red.
Cor. 4.5 implies that, if DeCaFlow correctly estimates an interven- tional query, then also does for its counterfactual counterpart. While the above results can look surprising at first, recall that we assume continuous endogenous variables and diffeomorphic causal generators ($3). Moreover, the correct estimation of counterfactual queries does not come without challenges: i) we need to accur- ately estimate (z x). which is why it is crucial to correctly design and train js: and ii) given z and x, we need to accurately perform the abduction step. Fortunately, the latter step is trivialized using CNFs as generative networks [27], since they are bijective given z.
Remarks. Whilst DeCaFlow can estimate any causal query, this estimation can be incorrect for unidentifiable queries. Therefore, we must verify the identifiability for each query of interest, which we aim ease by providing algorithms to check the graphical requirements of Prop. 4.1 in $F. Namely, Alg. 6 checks if a query that involves specific caatment-outcong pair, which includes average treatment effects and counterfactuals, is identifiable. If we were interested in query on all variables, e.g., as samples from an interventional distribution, we should evaluate the identifiability of the causal query for all descendants of the treatment, as proposed Alg. 7. Similarly, note that all results above rely on the assumption that DeCaFlow matches the observational distribution, and thus it is crucial to ensure that the training completed successfully.
5 Empirical evaluation
In this section, we assess the performance of DeCaFlow relative to existing methods. Namely, we show that DeCaFlow accurately estimates interventional and counterfactual queries when the requirements of Corols. 3.2 and 4.3 are met. All experimental details are described in $B.
Common evaluation. For all experiments, we measure the estimation quality for interventional and counterfactual queries using the mean absolute error (MAE) of, respectively. the average treatment effect (A (ATE) and the counterfactual (CF) samples, as we have access to the ground-trui values. We use as reference (or oracle) a CNF [27] that does obxerve the hidden confounders. We also account for differences across observed variables by computing all errors over the standardized variables. Note that ATE and CF errors provide complementary measures of estimation quality, and their interpretation is best understood relative to the oracle performance in each metric.
7

---

1-3
1.0
1.0
0.5
0.5
0.0
ATE cIIE
CF cxix
ATE enor
CP enor
ATE
CF cio
ATE onor
cnoy
Addliiive
Nomadditive
Nonadditive
(a) Sachs' dataset.
Ecoli70 dataset.
Figure 7: ATE and CF еrror boxenplots [23] of different CGMs on the (a) Sachs and (b) Ecoli70 datasets, aggregating over all identifiable direct effects (see Figs. and 8) after intervening on their 25th, 50th, and 75th percentiles over 5 random initializations.
Hyperparameter selection. We choose hyperparameters based on the MMD [21] over validation observational data, following our theoretical premise that DeCaFlow correctly estimates causal queries when (x) (xx); see $B.6.3 for details on the hyperparameter grid.
5.1 Ablation study and practical considerations
In order to provide insights into practical limitations of DeCaFlow, we first conduct an ablation study understand the extent for which misspecifying the size of z affects DeCaFlow, well as its sensitivity to the number of available proxies. For additional details and results, refer to $B.1.
Experimental setup. We consider two synthetic SCMs with linear and non-linear causal equations that follow the causal graph Ç depicted in Fig. comprising variable, and S null proxies. We evaluate how well DeCaFlow estimates plyl (t) as we change the number of proxy variables, S, and the specified latent dimensionality. Dz.
independent hidden confounders affecting every
The completeness condition (Prop. A.2), i.e., that proxies yield "enough information" (Prop. 4.1), is difficult to verify. Figure 5: Ablation graph.
Proxy informativeness.
Fortunately, Fig. shows that using more proxies consistently improves the estimation of confounded causal queries in practice, as it is more likely to satisfy completeness. Thus, practitioners should aim to collect as many informative proxies as possible to ensure correct causal estimates.
Latent dimensionality. When the dimension- ality of the hidden confounders is unknown, we expect the entropy term in Eq. 4 to prevent modeling exogenous variables with z, dis- cussed in $3.2. Fig. 6 corroborates our intuition, showing that DeCaFlow remains robust to specification of the latent dimension, Dz, while error increases as we underestimate This sug- gests that, in practice, choosing large latent space preferable.
Lincar
Noulincar
Cieerrrr
D7
D4
Ouale
Figure 6: CF eiror as we increase the number of proxies, S, and the latent dimensionality, Dz. Plots show mean and 95 % CI over 5 realizations, inter- vening on the 25th, 50th, and 75th percentiles of
Other ablations. We summarize here other experiments of practical interest that can be found in the appendix. $$B.2 and B.3 corroborate our design choices for the deconfounding network, namely, the use of conditional normalizing flows and the posterior factorization in Eq. 3. Then, $B.4 assess the sample efficiency of DeCaFlow, showing that both DeCaFlow and the oracle monotonically improve their estimations the training size increases, supporting that correct causal estimates are obtained when the model accurately fits the observational distribution.
5.2 Semi-synthetic experiments
Next, we evaluate how DeCaFlow performs relative to existing approaches. To this end, we consider semi-synthetic datasets which we have access to the ground-truth. Additional details, results, and justification the use of semi-synthetic data can be found in $$B.5 and B.6.

---

Baselines. We consider three CGMs which assume causal sufficiency and are thus wawarre of the hidden confounders: CNFs [27]; ANMs [24]; and DCMs [9]; as well as the Deconfounder [75], which uses proxies to provide unbiased ATE estimates under hidden confounding, yet it requires to train one model per outcome. We take the oracle model as a lower bound of the error.
5.2.1 Protein-signaling networks
Following Chao et al. [9], we first experiment with the protein-signaling network dataset [63]. Namely, we randomly generate a non-linear SCM inducing the same causal graph as the original dataset, see Fig. 8, except for the root nodes, for which we use the original data. As a result, we have a bidimensional hidden confounder, PKC and PKA, and three treatment variables to intervene upon, Raf, Mek, and Erk. We consider additive and non-additive causal equations, measure the effect of interventions on the downstream nodes and, more importantly, ensure when generating the SCM that the randomized effect of the hidden confounder is perceptible.
Bat
Jnk
Mak
Erk
P88
Flat
PIPS
PIP2
Figure 8: Sachs' graph. Green edges mark proxy- identifiable effects.
Results. We present visualization of the results in Fig. 7a, where we can observe that DeCaFlow outperforms every considered approach in all cases, for both ATE and counterfactual errors, slaying on par with the oracle model. Moreover, we appreciate great difference in performance between DeCaFlow and CNFs, which corroborates the importance of the architecture and variational training employed by DeCaFlow, since CNF is equivalent to DeCaFlow with D₂ = 0.
5.2.2 Gene networks
We next conduct similar experiment in the previous section, considering this time the causal graph of the Ecoli70 dataset [66], depicted in Fig. 1, which represents gene network extracted from E. coli data. This time, we replace root nodes with Gaussian variables. See $B.6.
Results. Similar the previous case, the results presented in Fig. 7b demonstrate that DeCaFlow is indeed able to closely match the performance of the oracle model, outperforming existing approaches. is worth-noting, however, that the non-additive case shows long-tailed error distributions for all models, including the oracle, highlighting unavoidable issues of any data-centric approach, also DeCaFlow, and that have to be considered during evaluation and deployment.
We feel compelled to explain that the striking performance of the Deconfounder is an artifact of evaluating on causal queries that it cannot correctly estimate. As we discuss in $B, the Deconfounder guarantees correct ATE estimation under more stringent assumptions than those from $4. If we plot the ATE error evaluated on only those queries that meet Deconfounder's ' assumptions, we indeed observe that it achieves significantly lower error, as shown in the inset figure.
ATT-- emor
6.6
This experiment highlights every strength of DeCaFlow, as it needs to: i) model several hidden confounders affecting different sets of variables; ii) correctly estimate all causal queries for which we have proxy information; and iii) achieve the above in an agnostic manner, i.e., training the model out-of-the-box and one single time, despite the graph G having 43 observed variables.
5.3 Fairness real-world use case
Taking inspiration from the experiments by Kusner al. [40] and Javaloy et al. [27] we test whether, by modeling the confounded SCM with DeCaFlow, we can leverage it for more than causal-query estimation and, in particular, for counterfactual
prediction. See $B.7 for further details.
Dataset and objective. Our aim is to train a gradient-boostte decision tree [17] on the law school dataset [78], which comprises of 21 790 law students, that remains accurate while being fair toward the sensitive attributes of the students. In particular, we aim predict the decile of a student in its 3rd year of university, given their undergraduate and 1st year grades, family income, race, and sex.
Bax
GPA
Dec 1lmB
F20
Figure 9: Assumed causal graph in $5.3. Only the classi- fiers consider Decile3.
Experimental setup. First, we train DeCaFlow assuming the causal graph in Fig. 9, excluding Decile3, where all grades are af- fected by a common "knowledge" hidden confounder [40]. Then, we train a predictor using as input the hidden confounder and non-sensitive exogenous variables extracted from DeCaFlow. If, as dis- cussed in $3.3, DeCaFlow successfully recovers the exogenous variables, we expect the predictor to be fair yet slightly less accurate, since Decile3 is directly affected by the sensitive attributes.

---

Results. Tab provides the prediction error (RMSE) and the difference between group dis- tributions (MMD) for the proposed DeCaFlow- based predictor, comparing with an Unfair pre- dictor that uses sensitive attributes; an Unaware predictor that excludes sensitive attributes, and two fair predictors. Fair K and Fair Add, as ini- tially proposed by Kusner et al. [40]. We see that the proposed predictor slightly increases the error, while significantly reducing the MMD between the predicted distributions between sensitive attributes. Moreover, the other fair classifiers behave just as a baseline always predicting the average prediction.
Table 1: Test RMSE on Decile3 prediction and
MMD of inter-group predictive distributions.
Unfair
Unaware
DeCaFlow
Fair K
Fair Add
Mean
RMSE 1.413
1.419
1.604
2,817
2.826
2.83
MMD
0.163
0.147
0.0054
10
10
0
To provide a better intuition on the differences between predictors. we plot in Fig. 10 the dis- tributions predicted by the gradient-booste -booste de- cision tree stratified by the sensitive classes, for the Unfair and DeCaFlow-basee classifiers. Fig. 10 shows that, while both classifiers provide predictions for both sex classes, female and male, we find a qualitatively significant difference between the race classes, white and non-white, with the predictions of the Unfair classifier clearly skewed, predicting much lower deciles for the non-white population. In contrast, the DeCaFlow-basse classifier provides much similar predictions for all sensitive classes at the expense of a slightly higher prediction error.
Female
S exc
Male
White
Race
white
DeCaFlow Unfair
10
15
3rd year decile
3rd year decile
Figure 10: Distribution of predicted Decile3. A fair predictor yields similar distributions across the considered groups per attribute (Sex and Race).
Concluding remarks
In this work we introduced DeCaFlow, a CGM that can enable accurate estimation of interventional and counterfactual queries under hidden confounding. DeCaFFlow expands CNFs, preserving their properties while offering several key advantages over prior works. Namely, DeCaFlow can be applied out-of-the-box to any given causal graph and, training once per dataset, it correctly estimates broad class of (potentially hidden confoudee)) interventional and counterfactual queries over continuous endogenous variables. Moreover, estimates as those for which do-calculus yields observational or proximal-idees -ddentiiiibieee queries, ex- tending prior results [45, 76) to include counterfactuals and observed common ancestors. Finally, showed that DeCaFlow outperforms existing methods across a variety of settings, accurately recover- ing more hidden-confounded causal effects and enabling fair downstream predictions.
theoretically characterized all queries that DeCaFlow correctly
Limitations. While DeCaFlow relaxes causal sufficiency, it still relies on the existence of auffeiiently-inoorattvee proxies. This condition is untestable since we do not observe the hidden confounder, but collecting additional proxies can help satisfy it [3], as shown in $5.1. Similarly, De- CaFlow works with continuous random variables by assumption (see $3), although $5.3, Javaloy et al. [27] and de Vassimon Manela et al. [13] show that CNFs effectively approximate discrete distribu- tions in practice. Another limitation is assuming perfect knowledge of the causal graph G. In practice, G may be partially available or noisy. When graph misspecification does not involve children of the hidden confounders, DeCaFlow inherit from CNFs the ability to operate with a known causal order- ing or with partially specified graphs where variables are grouped, see [27, App. A.2.2]. However, if the assumed graph incorrectly specifies the relations involving hidden confounders -nd thus violat- ing the assumptions in Prop. 4.1-our identifiability results under hidden confounding no longer hold, and estimates for confounded causal queries may become inaccurate. Alternatively, DeCaFlow could be combined with methods that jointly perform causal identification and estimation for individual queries [82], which handle cases identifiable beyond our theory but trade scalability for flexibility.
Future work. We believe DeCaFlow opens many intriguing venues we are excited to explore, such as expanding the range of queries it can estimate using instrumental variables [22], applying it to settings with time-varying treatments or where multiple interventions take place, investigating the empirical sensitivity of DeCaFlow to noisy or misspecified causal graphs [49], or extending our framework to support soft (stochastic) interventions with dedicated evaluation protocols [10]. We are also excited to see DeCaFlow applied to real-world problems such as decision support systems [64], educational analysis [47], or policy making [16], yet always validating them with interventional data.
10

---

Acknowledgments and Disclosure of Funding
The authors would like to thank Luigi Gresele for useful discussions and comments which helped improving the quality of this work. This work has been supported by the project "Society-Aware Machine Leurning: The paradigm shift demanded by society to trust machine learning," funded by the European Union and led by IV (ERC-2021-STG, SAML, 101040177); and the Deutsche Forschungxgemeinscaaff (DFG) grant number 389792660 as part of the Transregional Collaborative Research Centre TRR 248: Center for Perspicuous Computing (CPEC) (TRR 248 CPEC). Members of Universidad Politécnica de Madrid (AA,, JP and SZ) have received the funding from the SYNTHIA project. SYNTHIA (Synthetic Data Generation framework for integrated validation of use cases and Al healthcare applications) is supported by the Innovative Health Initiative Joint Undertaking (IHI JU) under grant agreement No 101172872. Funded by the European Union, the private members, and those contributing partners of the IHI JU. In addition, Programa Propio UPM funded the stay of AA at Saarland University. Moreover, AJ has received funding from the "UNREAL: a Unified Reasoning Layer for Trustworthy ML" project (EP/Y023838/1) selected by the ERC and funded by UKRI EPSR. Views and opinions expressed are, however, those of the author(s) only and do not necessarily reflect those of the aforementioned funding agencies. Neither of the aforementioned parties can be held responsible for them.
Bibliography
[1] Jeffrey Adams, Niels Hansen, and Kun Zhang. Identification of Partially Observed Linear Causal Models: Graphical Conditions for the Non-Gaussian and Heterogeneous Cases. In Marc' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NewrlPS 2021, December 6-14, 2021, virtual, pages 22822-22833, 2021. B (Cited in page 1.)
[2] Elizabeth S. Allman, Catherine Matias, and John A. Rhodes. Identifiability Of Parameters In Latent Structure Models With Many Observed Variables. The Annals of Statistics, 37(6))33099-
3132, 2009. ISSN 00905364, 21688966. & (Cited in pages 2, 50, 51, and 52.)
[3] Donald WK Andrews. Examples of 12-complete and foundedly-compledd distributions. 2011
(Cited in pages 10 and 27.)
[4] Joshua D Angrist and Jörn-Steffen Pischke. Mostly harmless econometrics: An empiricist's companion. Princeton university press, 2009 (Cited in pages 2 and 51.)
[5] Alexander Balke and Judea Pearl. Probabilistic Evaluation of Counterfactual Queries. Probabil- istic and Causal Inference, 1994, & (Cited in page 7.)
[6] Eli Bingham, Jonathan P Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D Goodman. Pyro: Deep universal probabilistic programming. Journal of machine learning research, 20(28):1 2019 (Cited in page 45.)
[7] Patrick Blöbaum, Peter Götz, Kailash Budhathoki, Atalanti A. Mastakouri, and Dominik Janzing. DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models. ArXiv preprint, abs/2206.0682) 2022. & (Cited in page 43.)
[8] Marine Carrasco, econometrics estimation based on spectral decomposition and regularization. Handbook of econometrics, 6:5633-5751, 2007 (Cited page 27.)
Florens, and Eric Renault. Linear inverse problems in structural
[9] Patrick Chao, Patrick Blöbaum, and Shiva Prasad Kasiviswanathan. Interventional and counter- factual inference with diffusion models, ArXiv preprint, abs/2302.00860) 2023. & (Cited in pages 1, 2, 9, 38, and 39.)
[10] Juan D. Correa and Elias Bareinboim. A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate Experiments. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Shirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances Artificial Intelligence. EAAI 2020, New York, NY, USA, February 7-12, 2020, pages 10093-10100. AAAI Press, 2020. & (Cited in page 10.)
[11] Yifan Cui, Hongming Pu, Xu Shi, Wang Miao, and Eric Tchetgen Tchetgen. Semiparametric proximal causal inference. Journal of the American Statistical Association, 119(546):1348- 1359, 2024 (Cited in page 51.)
11

---

[12] Alexander D' Amour. On Multi-Cause Approaches to Causal Inference with Unobserved Counfounding: Two Cautionary Failure Cases and A Promising Alternative. In Kamalika Chaudhuri and Masashi Sugiyama, editors, The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, volume 89 of Proceedings of Machine Learning Research, pages
PMLR, 2019. G (Cited in
page 51.)
[13] Daniel de Vassimon Manela, Laura Battaglia, and Robin J. Evans. Marginal Causal Flows for Validation and Inference. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, UIrich Paquet, Jakub M. Tomezak, and Cheng Zhang, editors, Advances Neural Information Processing Systems 38: Anmual Conference on Neural Information Processing Systems 2024, NewrIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024. B (Cited in page 10.)
[14] Conor Durkan, Artur Bekasov, Jain Murray, and George Papamakarios. Neural Spline Flows. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 7509-7520, 2019. & (Cited in page 43.)
[15] Stefan Feuerriegel, Dennis Frauen, Valentyn Melnychuk, Jonas Schweisthal, Konstantin Hess, Alicia Curth, Stefan Bauer, Niki Kilbertus, Isaac S Kohane, and Mihaela van der Schaar. Causal machine learning for predicting treatment outcomes. Nature Medicine, 30(4):958-99) 2024 (Cited in page 1.)
[16] Denis Fougère and Nicolas Jacquemet. Policy evaluation using causal inference methods. In Handbook of Research Methods and Applications in Empirical Microeconomics, pages 294- 324. Edward Elgar Publishing, 2021 (Cited in page 10.)
[17] Jerome H Friedman. Greedy function approximation: a gradient boosting machine. Annals of statistics, pages 1189-1232, 2001 (Cited in pages 9 and 45.)
[18) Mathieu Germain, Karol Gregor, lain Murray, and Hugo Larochelle. MADE: Masked Autoen- coder for Distribution Estimation. In Francis R. Bach and David M. Blei, editors, Proceed- ings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 881-889. JMLR.org, 2015. & (Cited in page 46.)
[19] Olivier Goudet, Diviyan Kalainathan, Philippe Caillou, Isabelle Guyon, David Lopez-Pau Paz, and Michele Sebag. Learning functional causal models with generative neural networks. Explainable and interpretable models in computer vision and machine learning, pages 39-80, 2018 (Cited
in page 53.)
[20] Sander Greenland. Basic methods for sensitivity analysis of biases. International journal of epidemiology, 107-1166 1996 (Cited in page 1.)
[21] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander J. Smola. A Kernel Method for the Saa---aamppplllleee
In Bernhard Schölkopf, John C. Platt, and Thomas Hofmann, editors, Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4-7, 2006, pages 513-520. MIT Press, 2006. B (Cited in pages 8 and 44.)
[22] Jason S. Hartford, Greg Lewis, Kevin Leyton-Brown, and Matt Taddy. Deep IV: A Flexible Approach for Counterfactual Prediction. In Doina Precup and Yee Whye Teh, editors, Proceed- ings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pages 1414 1423. PMLR, 2017. t (Cited in pages 10 and 51.)
[23] Heike Hofmann, Karen Kafadar, and Hadley Wickham. Letter-value plots: Boxplots for large data. Technical report, had.co.nz, 2011 (Cited in page 8.)
[24] Patrik O. Hoyer, Dominik Janzing, Joris M. Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. In Daphne Koller, Dale Schuurmans, Yoshua Bengio, and Léon Bottou, editors, Advances in Neural Information Processing Systems
21, Proceedings of the SScnod Annual Conference on Neural Information Processing Systems, Vancouver, Brütihh Columbia, Canada, December 8-1 2008, pages 689-696. Curran Associates, Inc., 2008, C (Cited in pages 2 and 9.)
12

---

[25] Yimin Huang and Marco Valtorta. Pearl's calculus of intervention is complete. arXiv preprint arXiv:/206.68.. 2012 (Cited in page 30.)
[26] Amin Jaber, Adèle H. Ribeiro, Jiji Zhang, and Elias Bareinboim. Causal Identification under Markov equivalence: Calculus, Algorithm, and Completeness. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022. B page 53.) [27] Adrián Javaloy, Pablo Sánchez-Martin, and Isabel Valera. Causal normalizing flows: from theory practice. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. & (Cited in pages 1, 2, 3, 4, 5, 6, 7, 9, 10, 26, 30, 31, 44, 46, 47, 48, and 53.)
(Cited in
[28] Nathan Kallus, Xiaojie Mao, and Madeleine Udell. Causal Inference with Noisy and Missing Covariates via Matrix Factorization. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 6921-6932, 2018. (Cited in pages 2, 51, and 52.)
C
[29] Nathan Kallus, Xiaojie Mao, and Angela Zhou. Interval Estimation of Individual-LLved Causal Effects Under Unobserved Confounding. In Kamalika Chaudhuri and Masashi Sugiyama, editors, The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, volume 89 of Proceedings of Machine Learning Research, pages 2281 22900. PMLR, 2019. & (Cited in pages 51 and 52.)
[30] David Kaltenpoth and Jilles Vreeken. Nonlinear Causal Discovery with Latent Confounders. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning. ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 15639-15654, PMLR, 2023. ? (Cited in page 38.)
[31] lyess Khemakhem, Ricardo Pio Monti, Robert Leech, and Aapo Hyvärinen. Causal Autore- gressive Flows. In Arindam Banerjee and Kenji Fukumizu, editors, The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event, volume 130 of Proceedings of Machine Learning Research, pages 3520-3528. PMLR. (Cited in pages and 2.)
2021. Ø3
[32] Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. G (Cited in page 42.)
[33] Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes. In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. ? (Cited in pages and 4.)
[34] Herbert Knothe. Contributions the theory of convex bodies. Michigan Mathematical Journal, 4:39-52, 1957 (Cited in pages 3 and 37.)
[35] Murat Kocaoglu, Christopher Snyder, Alexandros G. Dimakis, and Sriram Vishwanath. Causal- GAN: Learning Causal Implicit Generative Models with Adversarial Training. In 6th Interna- tional Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. & (Cited in page 2.)
[36] Andrey Kolmogorov. On the Shannon theory of information transmission in the case of continuous signals. IRE Transactions on Information Theory, 2(4):102-108, 1956 (Cited in page 4.)
[37] Benjamin Kompa, David R. Bellamy, Thomas Kolokotrones, James M. Robins, and Andrew Beam. Deep Learning Methods for Proximal Inference via Maximum Moment Restriction. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors,
13

---

Advances in Neural Information Processing Systems 35: Annual Conference on Neural Informa- tion Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022. B (Cited in pages 2 and 51.)
[38] Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathematical statistics, 22(1))79-86, 1951 (Cited in page 4.)
[39] Manabu Kuroki and Judea Pearl. Measurement bias and effect restoration in causal inference. Biometrika, 101(2):423433) 437, 2014 (Cited in pages 2, 6, 50, 51, and 52.)
[40] Matt J. Kusner, Joshua R. Loftus. Chris Russell, and Ricardo Silva. Counterfactual Faimness. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors,Avvance Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 4066 4076, 2017. 3 (Cited in pages 9, 10, 44, and 45.)
[41] Christopher P Long and Maciek R Antoniewicz. Metabolic flux analysis of Escherichia coli knockouts: lessons from the Keio collection and future outlook. Current opinion in biotechnology, 18:1113 2014 (Cited in page 39.)
[42] Christos Louizos, Uri Shalit, Joris M. Mooij. David A. Sontag, Richard S. Zemel, and Max Welling. Causal Effect Inference with Deep Latent-Variable Models, In Isabelle Guyon, Ulrike von Luxburg. Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6446 6666,,6666266 2017. & (Cited in pages 51 and 52.)
[43] Ruiyan Luo and Hongyu Zhao. Bayesian hierarchical modeling for signaling pathway inference from single cell interventional data. The annals of applied statistics, 5:725-745, 2011. 1. doi:
10.1214/10-AB A0AS422 (Cited in page 38.)
[44] Afsanch Mastouri, Yuchen Zhu, Limor Gultchin, Anna Korba, Ricardo Silva, Matt J. Kusner, Arthur Gretton, and Krikamol Muandet. Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction. In Marina Meila and Tong Zhang. editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021. Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 7512-7523 PMLR. 2021. K (Cited in pages 2 and 51.)
[45] Wang Miao, Zhi Geng, and Fric J Tchetgen Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4))987-993, 2018 (Cited in pages 1,
2, 6, 10, 27, 29, 51, and 52.)
[46] Wang Miao, Wenjie Hu, Elizabeth I. Oghurn, and Xiao Zhou. Identifying effects of multiple treatments in the presence of unmeasured confounding. Journal of the American Statistical Association, 1953-19) 2023 (Cited in pages 1, 2, 27, 51, and 52.)
[47] RJ Murnane. Methods matter: Improving causal inference in educational and social science research. Oxford University Press, 2010 (Cited in page 10.)
[48] Arash Nasr-Esfahany, Mohammad Alizadeh, and Devavrat Shah. Counterfactual Identifiability of Bijective Causal Models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honollul,, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 25733-25754. PMLR, 2023. G and 53.)
(Cited in pages 2, 27,
[49] Chris J Oates, Jessica Kasza, Julie A Simpson, and Andrew B Forbes. Repair of partly misspecified causal diagrams. Epidemiology, 28(4):548- 552,, 2017 (Cited in page 10.)
[50] George Papamakarios, Eric T. Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing Flows for Probabilistic Modeling and Inference. J. Mach. Learn. Res., 22:57:1-57:64, 2021. E (Cited in pages 5, 30, and 38.)
[51] Álvaro Parafita and Jordi Vitrià. Estimand -gnossicc causal query estimation with deep causal graphs. IEEE Access, 10:71370-71386, 2022 (Cited in page 2.)
[52] Nick Pawlowski, Daniel Coelho de Castro, and Ben Glocker. Deep Structural Causal Models for Tractable Counterfactual Inference. Hugo Larochelle, Mare Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and EssunnTTien Lin, editors, Advances in Neural Information P Processing
14

---

Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. & (Cited in page 2.)
[53] Pearl and D. Mackenzie. The Book of Why: The New Science of Cause and Effect. Penguin Books Limited, 2018. ISBN 9780241242643, & (Cited in page 33.)
[54] Judea Pearl. Causal Diagrams for Empirical Research. Biometrika, 82(4):669-688, 1995. ISSN 00063444, 14643510. 3 (Cited in page 3.)
[55] Judea Pearl. Causality. Cambridge university press, 2009 (Cited in pages 2, 3, 4, 34, and 51.)
[56] Judea Pearl. The Do-Calculus Revisited. In Nando de Freitas and Kevin P. Murphy, editors, Proceedings of the Twenty-Fighth Conference on Uncertainty in Artificial Intelligence, Catalina Island, CA, USA, August 14-18, 2012, pages 3-11, AUAI Press, 2012. B (Cited in page 3.)
[57]
Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016 (Cited in page 48.)
[58] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: founda- tions and learning algorithms. The MIT Press, 2017 (Cited in pages 3, 29, and 34.)
[59] Md. Musfiqur Rahman and Murat Kocaoglu. Modular Learning of Deep Causal Generative Models for High-dimensional Causal Inference. In Forty-first International Conference Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. G
(Cited in pages 2 and 53.)
[60] Rajesh Ranganath and Adler Perotte. Multiple causal inference with latent confounding. ArXiv preprint, abs//805 0011180508820 08273,, 2018. Ø (Cited in pages 51 and 52.)
[61] severi Rissanen and Pekka Marttinen. A Critical Look at the Consistency of Causal Estimation with Deep Latent Variable Models. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Anmual Conference Neural Information P Processing Systems 2021, NeurlPS 1021, December 6-14, 2021, virtual, pages 4207-4217, 2021. 8 (Cited in page 51.)
[62] Murray Rosenblatt. Remarks on a Multivariate Transformation. The Annals of Mathematical Statistics, 23(3):47--472 1952. ISSN 00034851. B (Cited in pages 3 and 37.)
[63] Karen Sachs, Omar Perez, Dana Pe' Douglas A. Lauffenburger, and Garry P. Nolan. Causal Protein- Protein-Sinnalin Networks Derived from Multiparameter Single-Cell Data. Science, 308 doi: 10.1126/scienee. 10586, t (Cited in pages 9, 38, and 39.)
(5721):523-529) 2005.
[64] Pedro Sanchez, Jeremy P Voisey, Tian Xia, Hannah Watson, Alison Q O'Neil and Sotirios A Tsaftaris. Causal machine learning for healthcare and precision medicine. Royal Society Open Science, (8):22063 2022 (Cited in page 10.)
[65] Pablo Sánchez-Martin, Miriam Rateike, and Isabel Valera. VACA: Designing Variational Graph Autoencoders for Causal Queries. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22- March 1, 2022, pages 8159-8168. AAAI Press, 2022. & (Cited in page 2.)
[66] Juliane Schäfer and Korbinian Strimmer. A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical applications in genetics and molecular biology, 4(1), 2005 (Cited in pages 2, 9, and 39.)
[67] Marco Scutari. Learning Bayesian Networks with the bnlearn R Package. Journal of Statistical Software, 35(3):1 22, 2010. doi: 10.18637/jss.v00 wit.ssss.....3...... (Cited in pages 38 and 40.)
[68] Xu Shi, Wang Miao, Jennifer C Nelson, and Eric Tchetgen Tchetgen. Multiply robust causal inference with double-negative control adjustment for categorical unmeasured confounding. Journal of the Royal Statistical Suciety Series Statistical Methodology, 82(2))521-540, 2020 (Cited in page 51.)
[69] Ilya Shpitser and Judea Pearl. Identification of joint interventional distributions in recursive semi-Markovian causal models. In AAAI, pages 1219-1226, 2006 (Cited pages 29 and 30.)
[70] Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2001 (Cited in page 29.)
15

---

[71] Eric J Tchetgen Tchetgen, Andrew Ying, Yifan Cui, Xu Shi. and Wang Miao. An introduction to proximal causal learning. ArXiv preprint, Abs/2009.10982, 2020. t (Cited in page 51.)
[72] Santtu Tikka and Juha Karvanen. Identifying causal effects with the R package causaleffect. Journal of Statistical Software, 76:1 160 2017 (Cited in page 30.)
[73] Arash Vahdat and Jan Kautz. NVAE: A Deep Hierarchical Variational Autoencoder. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Flonin Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. & (Cited in pages 5, 37, and 46.)
[74] Hal R Varian. Causal inference in economics and marketing. Proceedings of the National Academy of Sciences, 1 113(7)):310--3115 2016 (Cited in page 1.)
[75] Yixin Wang and David M Blei. The blessings of multiple causes. Journal of the American Statistical Association, 114458):1574-15589 2019 (Cited in pages 40, 51, and 52.)
[76] Yixin Wang and David M. Blei. A Proxy Variable View of Shared Confounding. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML. 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 10697-10707. PMLR, 2021. t (Cited in pages 1, 2, 6, 10, 27, 29, 40, 51, and 52.)
[77] Yixin Wang, David M. Blei, and John P. Cunningham. Posterior Collapse and Latent Variable Non-identifiabiiit.
Marc Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages
2021. & (Cited in pages 5 and 52.)
[78] Linda F Wightman. LSAC National Longitudinal Bar Passage Study. LSAC Research Report Series. 1998 (Cited in page 9.)
[79] Christina Winkler, Daniel Worrall, Emiel Hoogeboom, and Max Welling, Learning Likelihoods with Conditional Normalizing Flows. ArXiv preprint, abs/ 1b1//992,00 2019. C (Cited in pages and 4.)
[80] Quanhan Xi and Benjamin Bloem- Reddy. Indeterminacy in Generative Models: Characteriza- tion and Strong Identifiability. In Francisco J. R. Ruiz, Jennifer G. Dy, and Jan-Willem van de Meent, editors, International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning Research, pages 6912-6939. PMLR, 2023. & (Cited in pages 2, 5, 26, and 53.)
[81] Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The Gausal-Neure Connec- tion: Expressiveness, Learnability, and Inference. In Mare' Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan, editors, Advances in Neural Information Processing Systems 34: Anmual Conference on Neural Information Processing Sys- tems 2021, NeurlPS 2021, December 6-14, 2021, virtual, pages 10823-10836, 2021. Ø (Cited in pages 2 and 52.)
[82] Kevin Muyuan Xia, Yushu Pan, and Elias Bareinboim. Neural Causal Models for Counterfac- tual Identification and Estimation. In The Eleventh International Conference on Learning Rep- resentations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. B (Cited in pages 2, 10. and 52.)
[83] Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. VAE: Structured causal disentanglement in variational autoencoder. ArXiv preprint, abs/2004.08697,
2020. C (Cited in page 2.)
[84] Matej Zežević, Devendra Singh Dhami, Petar Velivcković, and Kristian Kersting. Relating graph neural networks structural causal models. ArXiv preprint, abs/2109.04173, 2021. C (Cited in page 2.)
[85] Siyuan Zhao and Neil Heffernan. Estimating Individual Treatment Effect from Educational Studies with Residual Counterfactual Networks, International Educational Data Mining Society, 2017 (Cited in page 1.)
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction assert that DeCaFlow provides correct estimates for every do- calculus and proximal identifiable causal query, including counterfactuals, under hidden confounding and outperforms prior models. These points are rigorously proven in the theory sections ($4) and demonstrated by comprehensive empirical results ($5). Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: dedicated "Limitations" paragraph at the end of $6 explicitly states that DeCaFlow depends sufficiently informative variables-an untestable assumption confounded SCM, thereby clarifying when the method may fail and why.
Ci-diffeomorphic
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-speeification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or seeech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
17

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: Every theorem, proposition and corollary in $4 is stated with explicit numbered assumptions and the correspoding formal proofs given in $A and cross referenced. As example, Prop. A.2 is the formal version of Prop. 4.1, which includes a list independence, completeness and regularity conditions and then supplies step-by-step proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof's in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
The main paper explicitly refers to "all experiments details" to $3.2 and $C. $B and ŞD, where it specified the data-generation pipelines, the causal graphs, the architecture, the ELBO training objective with its regularization process, the do-operator, the metrics and the evaluation protocol In addition, the authors commit releasing the full codebase, together with the hyper-parameter search and seeds.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general.
18

---

releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (c.g.. .. in the case of large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution a new model (c.g., a large language model), then there should either be a way to access this model for reproducing the results or way to reproduce the model (c.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in cases, in which case authors welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access the model is limited in some way (e-g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: he datasets employed are publicly available or fully described, a user-friendly implementation of DeCaFlow is available in the link of the introduction of this paper (github /github.com/aat comm /ataa//aoooorrreeeeaaaaooooooo
as well as examples of use, identifiability check algorithms and visualizations. In addition, we offer the whole training, hp tuning and validation pipeline, for reproducib- ility, under request to correspondence authors.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https: http://pppp.cccpbbcc/uug/e////// CodeSubmissionPoliccy for more details.
While we encourage the release code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed results. See the NeurIPS code and data submission guidelines (https: / /ntps/iipp/.ao/// cc/ cc/publicc.ccccccc c/guide s/ s/CodeSubmie wwwbbbmmiiiiiiioooommmmmmmmmimmm onPolicy) for more details.
to reproduce the
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
19

---

Providing much information as possible in supplemental material (appended the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: $B entirely devoted to experimental details, describing for each dataset the generation pipeline, train-test splits, the number of runs, the interventions percentiles and exact metries. In addition, $C supplements with implementation hyper-parameters, warm-up schedule, posterior factorization and masking strategy.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Every key figure and table includes statistical uncertainty. Fig. plots boxenplots (with all percentiles), and tables provide mean and standard deviation all metries across all seeds and all evaluated causal effects. Tables also include significantly better results related with statistical tests and intervals included in the captions.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experi- mental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK report I-sigma error bars, but should state it. The authors should preferably report 2-sigma eтror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are of range (e.g. negative enror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
20

---

For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Details about the execution resources and times are provided in &C, Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experi- ments reported in the paper (e.g.. preliminary or failed experiments that didn 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https htop:://neurippe hetpr//p..ccpplllcccchccccceeeed
Answer: [Yes]
Justification: All experiments rely on publicly available or semi-synthetic benchmark data (Sachs protein signalling. Ecoli70 gene network, and the anonymised LSAC law-school dataset) and introduce no new personal data collection, human-subject interaction, or high-risk model release; the work explicitly addresses fairness ($5) and reports moderate compute usage, thereby avoiding the privacy, discrimination, security, or environmental concerns enumerated in the NeurIPS Code of Ethics. Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We explain how DeCaFlow can enable better decision-making in domains such as health-
and education while cautioning that causal assumptions must be validated, especially in sensitive applications, and state that the method introduces no additional ethical risks beyond those already known for causal-inference models.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinform- ation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, is legitimate to point out that improvement the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The release comprises only the DeCaFlow training and evaluation code plus lightweight models trained on publicly available or semi-synthetic benchmarks; because no large pretrained generative models or scraped datasets with dual-use potential are distributed, special safeguards are unnecessary.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The licenses and all the copyright information will be included in every asset the code.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
22

---

If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com /at/aatseess has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The paper introduces a new, fully self-contained DeCaFlow repository released under the General Public License. No new datasets personal data created, consent and privacy disclosures are unnecessary.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details of the dataset/code/mooel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants screenshots, applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The study does not involve any crowdsourcing prospective research with human par- ticipants; it relies exclusively on pre-existing, publicly available or synthetic datasets (Sachs, Ecoli70, LSAC), participant instructions, screenshots, and compensation details not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The work uses only pre-existing benchmark datasets-Saae protein-signalling semi- synthetic Ecoli70 gene network, and the publicly released LSACC law-school dataset-without recruiting new participants collecting personal data, human-subjects review and IRB disclosure are not applicable.
23

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, non-standard component of the core methods in this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The authors have not used LLMs for important tasks of the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, or non-standard components.
Please refer our LLM (tttgsss/earrrpp.cc////teeereeee or should not be described.
for what should
24

---

Appendix
Table of Contents
A Causal identifiability
26
A.1 Model identifiability
Query identifiability
................................
A.3 Counterfactual query identifiability
................................
34
B Experimental details and additional results
35
B.1 Ablation study on latent dimension and number of proxies
36
B.2 Ablation study for encoder selection
36
B.3 Ablation on encoder factorization
................................
Ablation on train size
B.5 Semi-synthetic Sachs' dataset
................................
B.6 Semi-synthetic Ecoli70 dataset
................................
39
B.7 Law school fairness use-case
44
C Implementation details
46
C.I Posterior factorization of the deconfounding network
iec.............................
46
C.2 Regularization of Kullback-Leible term in ELBO
................................
Structural inductive bias
D Do-operator
48
D.I. Do-operator in causal normalizing flows
48
D.2 Do-operator in interventional distributions with DeCaFlow
48
D.3 Do-operator in counterfactuals with DeCaFlow
49
Additional details on related work causal inference with hidden confounders
E.1 Methods tailored graph and query
E.2 CGM with unobserved confounders
52
F Algorithms for causal query identification
53
E1 Pipeline for using DeCaFlow
55
25

---

A Causal identifiability
A.1 Model identifiability
We briefly discuss the identifiability (in the sense of Xi and Bloem-Reddy [80]) of those variables that are indirectly confounded by not confounded at all, i.e., of those variables that not children of any hidden confounder. As we discuss now, we can reduce our SCM (Def. 1) to a conditional one that only models these aforementioned variables, recovering the identifiability guarantees from Javaloy et al. [27]. To prove model identifiability. we resort to what we call the induced conditional SCM, which intuitively represents the original SCM where we restrict our view to a subset of variables, and assume the rest of the variables are given.
Pu, P2). and subset of observed variables x' C X, we define the induced conditional SCM of M given x', denoted by M to the SCM result of having observed x', and where causal generators and exogenous variables are restricted to only those associated with the unconditioned variables, i.e., x x'.
Definition 3 (Induced conditional SCM). Given SCM M
X2
X1
Z
X2
X3
X5
(a) Confounded SCM.
(b) Conditional unconfounded SCM.
Figure A:1: Example of: (a) a confounded SCM M; and (b) its induced conditional counterpart, Mix where the children of the hidden confounder are observed and fixed, x' ch(z) = {x1, X7}. Note that Mpx does not exhibit hidden confounding.
We provide a visual depiction of this idea in Fig. A.1. Using this definition, we can observe that, if we were to condition on the children of the hidden confounder, we would be left with (conditional) unconfounded SCM, as the influence of the hidden confounder has been completely blocked by conditioning on its children. Now, if we have two models that perfectly match their marginal distributions, this means that they perfectly match their induced conditional SCM, no matter which value we observed for ch(z). and we can thus leverage existing results from Javaloy et [27] for unconfounded SCMs. More specifically:
Corollary A.1. Assume that we have two SCMs M = (f., Pu. P2) and M == (f, P2) that are Markov- equivalent i.e., they induce the same causal graph and which coincide in their marginal distributions, p(x) aec. p(x). Then, both SCMs, restricted to every variable other than ch(z), are equal up an element- wise transformation of the exogenous distributions.
Proof. The proof follows almost directly from [27, Theorem 1]. First, note that the two induced conditional SCMs are no longer influenced by z once that we have observed a specific realization of ch(z), so that we can drop z from their structure, i.e., we can rewrite them instead unconfounded SCMs, Mich(ze)
(fichtz): Pu Pujch(z)) and M = (fich(z), Pajch(z)) To ease notation, let us call x = x cz) the variables that are not children of
Next, note that for almost every realization of ch(z), we have that p(x² ch(z)) 0.c. ch()) since p(x) a.c. p(x) by assumption and p(x) = ch(z))p(ch(z)) As result, for each realization of ch(z) we can apply Theorem of Javaloy et al. [27], which yields that the two induced conditional SCMs are equal up an element-wist transformation of the exogenous distribution.
26

---

Finally, since the causal generators and exogenous distributions of the induced SCMs are, for almost every ch(z), identical to their counterparts in the original SCMs (as we have just discarded those components associated with ch(z)). we get that, those elements in both SCMs associated with x², are identical up said (possibly ch(z) (deeeeddd)))))) component-wise transformation.
A.2 Query identifiability
We now prove the identifiability of the causal queries considered in the main text. To this end, one key property that we will use the following is that of completeness (as, e.g.. in the work of Wang and Blei [76]). Intuitively. we say that a random variable z is complete given another random variable n if "any infinitesimal change is accompanied by variability in n" [46], yielding enough information to recover the posterior distribution of z. This concept is similar in spirit to that of variability in the case of discrete random variables [48]. In practice, completeness is more likely to be achieved the more proxies we measure [3].
Definition 4 (Completeness), We say that a random variable is complete given for almost all if, for any square-integrable function g()) and almost all c. g(z, c)p(z) c, n) dz = O for almost all n, if and only if g(z, c) 0 for almost all z.
The following proposition (informally simplified in Prop. 4.1) is a generalization of the results previously presented by Miao et al. [45] and Wang and Blei [76], where we include an additional covariate to the causal query, and make no implicit assumptions on the causal graph allowing, e.g., for the treatment and outcome variables to hame some observed parents in common. However, note that c cannot be a collider (e.g., forming subgraph of the form n - C 4 y). Otherwise, conditioning on would make independent variables dependent (in the example, y and n), and the causal effect of t on y would not be identifiable.
Proposition A.2 (Query identifiability). Given two SCMs M == Pu, Pa) and M == Pa, Pa), assume that they are Markov-equivllent-ie they induce the same causal graph-and and which coincide in their mar- ginal distributions, p(x) ae p(x). Then, they compute the same causal query, p(yl do(t), c) = plyy do(t), c), where y.t.c C x, if there exists two proxies w, n C x and b C none of them overlapping nor containing variables from the previous subsets, s.t.:
i) is conditionally independent of (t, given b, z and That is, 1L (t, b,
ii n is conditionally independent of y given t, b. z and c. That is, y 11 t, b, c.
iii) (b, 2) forms valid adjustment set for the query ply do(t), c). That is, given c, they are independent of after severing any incoming edges to it, t til (b, z)l c, and they block every backdoor path from t to
iv) z is complete given n for almost all b, and
v) ă is complete given w for almost all b and c, and the following regularity conditions also hold:
vi) [J p(ž) w, b, c) ă. b, c) dž dw < do for all b, c, and
vii) f f(yl b, 2, p( b, c) dă < oo for all t, b, and c.
Proof. First, note that the first three independence assumptions hold for both models, M and M, as they induce the same causal graph. Following the same arguments as Miao et [45, Proposition 1], we have that assumptions v), vi), and vii) guarantee the existence of function such that solves the integral equation over M,
p t, b, 2, c) = f hly 1, b, w, c)p(w b, ž, c) dw,
since assumption vi) ensures that the conditional expectation operator is compact [8], assumption v) that all square-integrable functions are in the image of the operator (i.e., the operator is surjective), and assumption vii) that p(y) t, b, ă, c) is indeed part of the image.
27

---

We can show that f also solves similar integral equation, this time over the other SCM, M, as follows:
ply [t, n, c) = p(y t. b, c)
[equal marginals]
(8)
/ ply b. ž. c) n.). dž
[augment with z]
(9)
2,,1,,,2,,0)78 (t,,b,,,)/(/(bbbnn,))). c,
[assumption ii)
(10)
1, b, n, dădw S/ b, w, b, b,
[plug Eg. 7]
(11)
If bly,t,b,w. b.,,t,b.,ccpp( b. b, n, dž dw = / h(y,t, w, w.c)p(w |t, (,b,
[assumption i)]
(12)
[equal marginals]
(13)
Note that Eq. 13 a Fredholm equation of the first kind that is implicitly solved by modeling the observational data. Similarly, we can relate the expression for the interventional distribution of both models:
ply do(()) c) / ply do(t), b, ž, c)p z db
[augment and ass. iii)]
(14)
/ ply tt,,, 2, cb, c) db
[backdoor criterion]
(15)
/ʃ My,,t, c. b. a c)p(bb c) db dă
[plug Eq. 7]
(16)
/ hyy,,, t, b, w, (b, w dbdw
[equal marginals)
(17)
p(y c))
(18)
where the last equality is consequence of Eq. 13 will show now. More specifically, we have that
ply c) = / h(y, b, w. c)p(w t, c)
[Eq. 13]
(19)
/I h(y, b, w. c)p(w b,z.t,,,)p(z t, n, c) dw dz,
[augment with z]
(20)
// h(y, b, w, c)p(w b,z,c)p(z t, b, c) dw dz.
[assumption i)]
(21)
Similarly, we have that
ply dy, / ply 11, I,,, b,,is,/,,,,,,,,, d)
[augment with z]
(22)
f ply (t,,,,,c)p(z b,,,
[assumption ii)
(23)
Now, equating both expressions we have that
0 0= // ply t.b. / hly,,,
c)p(w b,,y,t,.w,.))(ww b.2,c)dw p(z 1, b, n,
(24)
28

---

which, due assumption iv), implies that
p(y bt,,,,,,) ue. f b,, b, b,
(25)
Finally, putting all together we see that we can write the interventional distribution of the original model using h,
ply do(().c) = /ʃ ply do(t), 2, c)p(b, cc db
Laugment and assumption iii)
(26)
// ply 11 (,,,t,,,,, 2 dbdz
[backdoor criterion]
(27)
// b,,,t, b, c)p(((w c)p(,, db dw [Eg. 25]
(28)
h, w, c)p(b, w dbdw,
lequal marginals]
(29)
which justifies the last equality in Eq. 18.
In Prop. A.2, assumptions i)-iit) regard the conditional independence of different variables in the causal graph, which can be directly verified given faithful causal graph. Assumptions iv) and v) regard the information that w and n contain of the hidden confounders which, intuitively. means that the posterior of the hidden confounder varies enough This assumption is harder to verify but, as we show in $B.1, the more proxy variables we have, the better the estimation of the hidden confounder's 's effect and the more accurate the causal query estimation is. Finally, assumptions ví) and vii) are standard regularity conditions [45, 76] that are (almost surely) fulfilled in practice, long the random variables well behaved, e.g., having finite moments. Such conditions are typically satisfied by most continuous and discrete distributions used in probabilistic modeling, including Gaussian, exponential family, and bounded-support distributions, making them mild and non-restrietive assumptions.
we change the values of w and n, in order to properly perform inference on it.
Using causal graph similar to the presented by Miao et al. [45]. now provide some intuition on the semantics of each random variable in Prop. A.2. More specifically, consider the causal graph that depict in Fig. A.2, and say that we want to check if the causal query p(yl do(t)) is identifiable (note that this the same query as in Prop. A.2 but with causal inference literature [58, 70]. t and y represent the treatment and outcome random variables. More specific to Prop. A.2 are and Here, w is a proxy variable whose role is that of distinguishing the information from z and other variables, to reconstruct the information of and block the backdoor path that z would usually block. Similarly, the variable n is another proxy variable which, in this case, serves the purpose of verifying that the substitute formed with w is indeed a good one. Finally, the variable b serves the purpose of blocking all the remaining backdoor paths that may not block, so that we can apply the backdoor criterion.
0). As it is common in the
Figure A.2: Example for which Prop. A.2 applies, and where b + V.
Moreover, note that for all interventional queries we let e be the empty set, similar the results proved by Miao et al. [45] and Wang and Blei [76]. We will consider cases when c is not empty later in $A.3 to prove counterfactual identifiability. Note also that Prop. A.2 reduces to previous results when c b - 0.
We turn attention towards proving Cor. 4.2, i.e., towards broadening the concept of query identifiability by introducing Prop. A.2 as base case of do-calculus. To this end, we introduce the concept of a hedge which will be use later, but still strongly recommend reading the work by Shpitser and Pearl [69].
29

---

Definition 5 (Hedge, [69, Def. 6D. y,t C. x be disjoint sets of variables in G. Let F, F" be r-rooted C-forests (see [69, Def. 5]) such that F nt o 0, Fx nt= 0, F' C F, and r is a subset of the ancestors of y after severing the incoming edges of Then F and F' form a hedge for p(y) do(t)) in G.
Corollary 4.2. An interventional query is identifiable if. using do-calculus, it can be reduced to a combination of observational queries and identifiable interventional queries in the sense of Prop. 4.1.
Proof. With the additional notion of proxy-identifiabiiiy provided by Prop. A.2 (informally presented in Prop. 4.1), the result is just a consequence of applying the identifiability algorithm provided by Shpitser and Pearl [69]. See also [25, 72] for other references.
Since the do-calculus rules are complete in the classical sense of identifiability, a query is not identifiable if the aforementioned algorithm yields a FAIL status (i.e., it executes line 5 of Figure 3 in [69]. If that is the case, then it means that, at the specific recursive call for which the algorithm failed, the local graph G contains hedge and the interventional query p(yl do())) is not identifiable in the classical sense.
Crucially, this hedge (F, F') expresses the inability of identifying an interventional query of the form p(rl do())) where the root r is subset of ancestors of y' S and t' - t. Then, this local query can still be proxy- mrxxyidenttifable if Prop. A.2 can be applied, and thus we can continue running the identification algorithm.
The stated result is then a consequence of successfully applying the logic above each time we find FAIL status, yielding a final FAIL status otherwise.
To be even more explicit regarding the identifiability of the queries proven in corollary above, let us call M the original SCM as usual, and M another SCM inducing the same causal graph as M and which matches the observational marginal distribution of M. i.c., p(x) a.c. p(x). Then, the output of the identifiability algorithm from the corollary above for both SCMs will be two identical expressions EXP composed of sum, integrals, and products of observational quantities (i.e., marginals and conditionals of subsets of x) well as proxy- identifiable queries of the form p(yl do(t)) as in Prop. A.2. Therefore.
Q(M) = (X) = (M) Q(M),
(30)
where the second equality is a consequence of both SCMs having equal observational distributions (and thus any other quantity than can derived exclusively from p(x)) and of applying Prop. A.2 for any interventional query that appears in the expression.
Corollary 4.3. If DeCaFlow induces the same causal graph G as M and (M = (x(x), then DeCaFlow provides correct estimates of any query identifiable in the sense of Cor. 4.2.
Proof. The proof is a direct consequence of the corollary above and the fact that can interpret DeCaFlow as a dense parametric family of confounded SCMs inducing the same causal graph as M (similar to the interpretation of Javaloy et al. [27] as bijective SCMs) by considering the triplet MMa- == (TØ Pu. Pz), where T is the inverse of the generative network that transforms u into x given z. This family being dense is consequence of the generative networks forming family of universal density approximators [27, 50). 11
To be completely exhaustive, in the following we explore the general proposition Prop. A.2 on all scenarios where and may or may not be directly caused by the hidden confounder, as we show in the following subsections.
A.2.1 Fully hidden-confounded case
In the case where both variables are children of z, we must see whether we can apply do-calculus with Prop. A.2 as additional base case, described in Cor. 4.2.
30

---

A.2.2 Hiddden-unconfounded case
Assume the case where neither nor are children of the hidden confounder, i.e., y, & ch(z). In this case, the proof of Prop. A.2 can be simplified and drop the requirement of finding valid proxy variables.
Corollary A.3. Given two SCMs M == Pu: Pa) and M == (f, Pa, Pa), assume that they are Markov- equivalent-i.c i.e., they induce the same causal graph and coincide in their marginal distributions, p(x) as p(x). If $ ch(z), then, p(y) do(t), c) = plyl do(t), c), where y, 1, C x.
Proof. The proof follows directly by applying Prop. A.2 with the minimal subset b C pa(t) all the backdoor paths. and by noticing that this case there is no need to use the variables z and z. That is, we can go from Eq. 14 to Eq. 18 directly by using only b and the equal-marginak assumption:
\ {c} that blocks
/ ply do((), b, c)p(b c) db
ply | do((), c)
(31)
/ p(y It, c) c) db
(32)
/ ply 1t, cbp(b cb
(33)
p(y c))
(34)
Even though can leverage and simplify Prop. A.2 shown above, it is worth remarking that, for this particular case, the model identifiability results described in $A.1 are stronger, as it provides results on the identifiability of the causal generators and exogenous distributions, and therefore of any causal query derived from them.
A.2.3 Confounded outcome case
For the case where only the outcome variable is a child of the hidden confounder, we can apply a similar reasoning as did in the previous case, although this time we cannot leverage the stronger results from Javaloy et al. [27]. More specifically:
== Pu; P2) and M (f, Pū, Pa), assume thut they are Markov- equivalent-i.e. they induce the same causal graph-an and coincide in their marginal distributions, p(x) ac p(x). Assume that t & ch(z). Then, p(yl do(t), c) = p(y] do((), where y, c, C. x.
Corollary A.4. Given two SCMs =
Proof. The proof is identical that of Cor. A.3.
Front-door example. While the proof above is trivial given the previous results, it is worth stressing the importance of modeling the hidden confounder as we do in this work with DeCaFlow. As an example, consider the SCM depicted in Fig, A.3, where we have that the outcome is directly confounded by z, while effects of b and using à to model the influence b onto that is not explained through t. Other models that do not model c.g., unaware CNF would be able to match the observed marginal distribution (as they are universal density approximators) and therefore to estimate p(yl db))) (as is identifiable through the mediator using the front-door criterion), yet they would necessarily fail to estimate p(yl (t),),, since they assume that Il. bi t yet we know that y y- bJ in the true model. In other words, an unaware CNF would hold that p(yl do(t)) p(ylt) t) which is clearly false by looking at Fig. A.3.
not. In this case, DeCaFllww can correctly estimate the causal y, i.e.. to correctly estimate p(yl do(t)) and p(yl do(b)),
Figure A.3: Example of a front-door causal.
31

---

To be even more explicit, in this case would have data-generating process that factorizes
p(b. y, p( 2)p(t
b)p(y [t,
(35)
and hence the estimated interventional distribution from DeCaFlow matches the true one:
ply do(()) = / ply t, (bb db
[b forms a valid adjustment set]
(36)
/ / ply b, 2)p(2 b) dž p(b) db
[Factorization and eq. marginals]
(37)
// ply (,2)((2 b)
[Factorization Eq. 35]
(38)
/ ö(y I, (1)) ply do(t)).
(marginalize b]
(39)
(40)
A.2.4 Hidden-confounded treatment case
When only the treatment variable is a child of z. we can face two different scenarios: we find valid adjustment set b blocking all backdoor paths, which case we can reason just as in the other partially hidden-confounded case, and ii) we cannot, and then rely on do-calculus and the identifiability w.r.t. b. For example, if b happens to be parent of which is directly caused by the treatment variable and the hidden confounder as in Fig. A.4, we cannot find a valid adjustment set for the causal query, but it may still serve us if we can identify the same query with the adjustment set as outcome variable.
X
Figure A.4: Case with no valid adjustment set.
Corollary A.5. Given two SCMs M = Pf, P2) and M = Pa, Pa), assume that they are Markov- and coincide in their marginal distributions, p(x) as p(x). y & ch(z) then, plyl do(t), c) p(y| do(t). c). where y,t, C x if there exists x not containing variables from the previous subsets. such that one of the following two conditions true:
equivalent-ic i.e., they induce the same cuusal
b forms valid adjustment set for the query p(yl do(t), c).
ii b blocks all backdoor paths and the query p(bl do(t), c) is identifiabl.
Proof. If condition holds, then we have valid adjustment set, and the proof is identical that of Cor. A.3.
Otherwise, if condition holds, have that the interventional query on y equals the observational query when conditioned on b, but that now b is not independent of do(t), i.e.
/ ply do(t), b, c)p(bb do(t), c) db
ply do(t),
(41)
/ ply bt,b, c) do(t), c) db
(42)
ply tt, cb do(t), c) db
(43)
ply do(t).c).
(44)
where needed that the query p(bl do(t), c) is identifiable in the third equality.
32

---

A.2.5 Napkin example
Finally, want to show last illustrative example where DeCa- Flow provides correct estimates of a causal query that is identifiable by the do-calculus, but neither the backdoor nor the front-door cri- teria are applicable. While redundant (as the query is identifiable in the classical sense, and then Cor. 4.2 applies), believe it can be good exercise to convince the reader. Namely, the graph of Fig. A.5 appears the napkin graph in Pearl and Mackenzie [53, Fig. 7.5]. What is particularly interesting in this graph is that w is not a valid adjustment set since, despite blocking the backdoor path from through b, it forms a collider of Zj and Z2.
21
Z2
w
b
Figure A.5: Napkin causal graph [53].
However, 21 only affects the outcome and 22 only affects the treatment, Following from our previous results, the causal effect from First, let us express the causal query of interest another form applying do-caleulus:
should be correctly estimated by DeCaFlow. Here, we show that this is the case.
ply do(()) = ply do()),do()) =
[Rule of do-calculus since y 1T G. b [1]
(45)
= p(y t, do(())
=
[Rule of do-calculus y 1L 96." t b]
(46)
p(y, do()))
[Conditional probability]
(47)
p(t dob))
Once we have this expression, let us work on the numerator, considering that DeCaFlow is Markov-equivalent with the graph in Fig. A.5:
ply, do(b)) = / p(y.t) b, w))(w) dw
[Backdoor criterion]
(48)
ply, 21, 82 b, w)p(w) dw dži dē2
[Eq. marginals]
(49)
= -/ʃ/ p(ylt, 21, 22, b, (.))((tii, 22, b, w) / htppp)pwww (() dw dā, dē2 [Factorization]
(50)
//ʃ/ ply It, (22)))(( 22. b)p 21. 22 w)p(w) dw dēz dī2
[Do-calculus rule I]
(51)
/ / ply | (1,z))((11 ž2. b) 222 dž dē
[Marginalize w]
(52)
/ / ply (1,2))((1 2, (7 dē dā,
[2] 11 g22]
(53)
ply t, (2)) dz p(22))(tt 22, b) d22
[Separate integrals]
(54)
= ply do(()) (t do(b))
[DeCaFlow estimate)
(55)
Note also that, as shown in Eq. 40, DeCaFlow correctly estimates p(t) do(b)). Therefore, we substitute Eq. 55 in Eq. 47, we have that
ply do(t)) p(yl do(t)) p(t] do(b)) ply do(t))
(56)
p(t do()))
That is, have explicitly shown that DeCaFlow correctly estimates the true causal query p(yl do(t)).
33

---

(wll
Uu
m
2
m
W
(a)
(b)
(c)
Figure A.6: Example of the transition from (a) the regular depiction of (confounded) SCM, to (b) an explicit SCM where the exogenous variables are drawn, and (c) counterfactual twin SCM where the data-generating process is replicated in the "factual and counterfactual worlds". Figure (c) also depicts which nodes are observed and which are severed in order to compute counterfactual query of the type p(y do(f"),,
A.3 Counterfactual query identifiability
In this section, we show that counterfactual query identifiablliyy is a direct result of the interventional query identifiability from the previous section.
order to formally define counterfactuals, this section we introduce the concept of counterfactual SCMs in rather untraditional fashion. Namely, combine the concepts of twin networks from Pearl [55] (which replicates the data-generating process) and that of counterfactual SCMs from Peters et al. [58] (which defines a counterfactual prior the intervention) as follows:
Definition (Counterfactual twin SCM). Given a SCM M (f, P2). we define its counterfactual twin SCM as SCM M where all structural equations duplicated, and the exogenous noise shared across replications, and where additionally one of the halves observed ("the factual world"), and the other half unobserved ("the counterfactual world").
We provide in Fig. A.6 a more intuitive depiction on the construction of these counterfactual twin networks. From this definition, one recover the counterfactual SCM defined by Peters et al. [58] by just focusing on the replicated part of the counterfactual twin network, and conditioning the exogenous noise and hidden confounder the observed half, i.e., (f, Pul x P2 x²) Similarly, one compute the usual counterfactual query by performing an intervention on the counterfactual twin network, i.c., by replacing the intervened equations by the constant intervened value, and computing the query conditioned the factual variables, p(ydf do(t"f) x This visually represented in Fig. A.6c.
In order to prove query identifiability in the counterfactual setting, need to the following technical random variable:
result regarding the completeness
Lemma A.6. If a random variable z is complete given for almost all b, as given by Def. 4, then it is complete given n for almost all and c, where с is another continuous random variable.
Proof. We prove this result by contradiction. Assume that the result does not hold, then there must exist non-zero measure subset of the space of b x cfor which there exists a square-integrable function g(-) such that J g(z, b, c)p(z| b. c, n) dz = 0 for almost all but g(z, b. c) + 0 for almost all z.
Since this subset has positive measure, there must contain an g-ball within. we now focus on the projection of this ball where fix its value the center, have that is subset of non-zero measure the space of b (as otherwise it would be zero-measure the cartesian-prodiee measure), where the function g(-, c) breaks our initial assumption of the completeness of Thus, we reach contradiction. O
34

---

Given Def. 6, it rather intuitive that, if causal query is identifiable in SCM M. then it has to be identifiable in both halves of its induced counterfactual twin SCM M as they are identical. More importantly, we can now leverage again Prop. A.2, this time with xf, to prove counterfactual query identifiability whenever we have interventional query identifiability.
Proposition A.7 (Counterfactuaa identifiability). Given two SCMs M (f, Pa) and M (P, Pa), assume that they are Markor-equivalent--ie, they induce the same causal graph and that they coincide their marginal distributions, p(x) a.e j(x). Then, if a query p(yl do(t)) identifiable the sense of Prop. A.2, where y, C x, the query ply
do((")) x²) is also identifiable in the induced counterfactual twin SCM as long as the regularity conditions still hold, i.e. if:
i) [ʃ p(z) W. b, c) z, b, c) dž dw oo for almost all b, c, and i) [ 2)) p(Z b, c) dž oo for almost all and
Proof. We essentially need to prove that the independence and completeness assumptions keep holding when we add the factual covariate, x'.
For the independence, we need show that, if we have a set of variables that fulfill the independence conditions from Prop. A.2, then this set of variables keeps holding them if we include = x This is, however, easy to show since factual and counterfactual variables only have "tail-to-tail" dependencies, i.e., they are connected only through the shared exogenous variables. As result, if two variables from the same half are conditionally independent given a third set of variables, conditioning on the other half cannot change this independence.
For the completeness, we need to show that introducing the factual variable retains the completeness assumed in Prop. A.2, which is direct to show using Lemma A.6. Specifically, holds that
i) z complete given n for almost all b, and c, and ii) Ż is complete given w for almost all b and c. Therefore, the requirements of Prop. A.2 hold when we append factual variable to the twin network, and thus we can reapply all the results from the previous sections the counterfactual cases.
Once proven the result above, proving Cor. 4.3 is direct by following the exact same steps did in $A.2 to the counterfactual twin network instead the original network.
It is important to note that, while the results above provide counterfactual identifiability whenever we have interventional identifiability, we still rely how much of good approximation the encoder is to the inverse the decoder in the proposed DeCaFlow model. That is, the quality of the encoder determines how well we can perform the abduction step compute counterfactuals. This consideration is unique to counterfactuals, as we just have to sample from the prior of z in the case of interventional queries.
B Experimental details and additional results
This section presents series of ablation studies designed to answer practical questions about the behavior of DeCaFlow and to justify key design choices. These analyses provide empirical guidance for practitioners, clarifying how model performance depends on factors such training data size, latent dimensionality, and proxy quality. Beyond validating theoretical claims, the results offer concrete recommendations for effectively applying DeCaFlow in realistic scenarios.
Finally, we include complementary experimental details and extended comparisons with baseline methods, covering dataset deseriptions, lata-generating processes, and additional quantitative results and visualizations that extend those presented in $5.
35

---

B.1 Ablation study on latent dimension and number of proxies We include here additional results the ATE er- ror, complementary to those of $5.1. . If observe Fig. B.I, we extract the same conclusion as ob- serving counterfactual error, the causal effect is not recoverable with less than two proxies, and more proxies result in better estimates. On the other hand, the selection of the dimension of the latent space big- ger than the true dimension of the latent confounders does not affect the performance negatively.
Lincar
Nonlinear
1.0
0.5
0,00
D2
Ds
Orate
Figure B.I: ATE absolute error ber of proxy variables, S, and the latent dimensionality, Dz. We plot mean and 95 % CI over 5 realizations, in- tervening on the 25th, 50th, and 75th percentile value of Oracle represents a causal normalizing flow that observes z.
change the num-
Overall, these findings indicate that DeCaFlow is ro- bust to latent space over-specificalle speciffciii,,,inn, thanks to KL regularization, and that, in practice, providing more and better proxies leads to more accurate estimation of causal effects even when confounding structure is unknown.
Details of the generative process. We show the equations that we have used for the ablation study. There exist unobserved confounders, 21 and Zz. Note that the proxies available in the nonlinear experiment are bounded or periodic, especially sigmoids and hyperbolic tangents saturate and max(0, x) loses all the information about the confounder for negative values and sines and cosines periodic functions. In other words, the distributions p(z n.) are not complete, we lose information about z when in the transformations each n. However, if we add more proxies of the confounders, the information that the proxies contain about the confounder is higher, and the causal effect of X₁ on X2 becomes recoverable.
Linear
Nonlinear
21 Par
22
Paz
21 Pai
z
72
sin
21 + 0.6 Ur
22 ~ Pzz
2
[ 1.5 2i + 0.5 22 + 0.4
0.8
22
+
0.5 1 1+ X1
0.3 + 0.2 Uy
4
y = -0.75.21 2.9 -- 0.3 Uy
n1 -0.5.% +033 +05
n: = 0.6 z
+
72
4
+ 0.3
sin
22
+ 0.5 U.
n2 0.75 0. 0.4
sin
+
008
0.44
2
3
n1 0.85 2ı + 0.6 0.6 Us
Z1
22
tanh
0.6 Us
n4 = 0.6 21 + 0.6 2, U44
2
3
ny -0.8 x +044 +2 0.4
n4 tanh
21
a
22
0.55 U4
2
ng 0.9 0.7
n1 0.72 2: + 0.5. 0.56 Us
n6 = a
2
+ max (0, - 22) + 0.4 Us
ns 0.78 2 + 0.58 us
nG = max
21) 0.5 max (0, 22) 0.6
ng 0.55 2ι + 0.7 22 0.6 Uo
n7 max(0, 21) 0.3 ma -zz) + 0.5 zi U7
nuu = 0.88 21 + 0.3 zu + 1..40uup
ng 0.8 m0 21) + 0.3 max(0. 22) 0.58 us
11 = 0.75 max (0, 21 + 0.5 max (0. 22) + 0.6 Up
n = - 2 0.5 |z2| + 0.4 u10
B.2 Ablation study for encoder selection
We have performed an ablation study for selecting the encoder in the Sachs' dataset, where we evaluate the emors in the estimations of causal queries using conditional normalizing flow (Flow) and multilayer
36

---

perceptron (MLP) as encoders. We also evaluate the impact of using the warm-up regularization [73] in the KL term. We can observe in Fig. B.2 that we achieve lower emrors when applying a regularized flow. This is able to model dependent latent variables and provides more flexible representation. addition, we can appreciate that applying the warm-up regularization term is useful and does not produce negative effects.
The improvement achieved by the flow is explained by the following practical aspects of the conditional normalizing flows. First, we can efficiently intro-
Oracle
Flow
MI
No enovler
Regulationd
Unveglarizeal
0.6
duce the factorization proposed in Eq. 3, taking ad- vantage of the structure of the causal graph (see Fig. C.I for an example), while this factorization im- plies the of several MLP. Second, normalizing flows are universal density approximators and do not need assume specific posterior distributions (i.e. Gaussians). Note that every continuous distribution be modeled by conditional normalizing flow, following the Knöthe-Rosenble transport [34, 62].
0.4
0.0
0.0
arnr
ence
crmor
Additive
Nonadditive
Figure B.2: Ablation for encoder selection in Sachs' dataset. Metries and 95% over 5 realization and all confounded identifiable effects, intervening percent- iles 25, 50 and each intervened variable. Oracle represents a causal normalizing flow that observes all the confounders.
B.3
Ablation encoder factorization
Using conditional normalizing flow the encoder allows us to model the dependencies between the observations and the posterior of the latent variables desired.
We propose in Eq. 3 (extended in Eq. 60) factoriz- ation in which each hidden confounder depends on its parents (other hidden confounders), its children and the parents of children, avoiding cycles. If child of unobserved confounder, has other par- ents, then that child is collider between the hidden confounders and the other parents of Therefore, conditioned on c, the hidden confounder is depend- ent of the other parents of c, given c. That is the reason because we consider sensible to include the other parents of in the factorization of the hidden confounder, z.
Oracle
DeCaflowsch
DeCaFlow-all
0.6
0.0
eeeer
Additive
Nonadditive
Figure B.3: Ablation for posterior factorization in Ecoli70 dataset. Boxenplots error metrics in the iden- tifiable edges of Fig. 1. DeCaFlow-ch uses Eq. 57 and DeCaFlow-all uses Eq. for posterior factorization.
However, we also provide an ablation study on the Ecoli70 dataset, where we show that this factorization indeed helps the estimation causal queries. Note that in the Ecoli70 dataset, lacY is collider between eut& and cspG. Therefore, conditioned on lacY, the two hidden confounders eutG and csp& become dependent. The factorization Eq. 60 implies that the posterior of capG is modeled employing all the children of capC and also the parents of its children, with eut& among them. This dependency can be modeled by our encoder an autoregressive manner.
This factorization incorporates more variables approximate the posterior of the hidden confounders, compared with simpler approach that consist in modeling only children dependencies:
De x) = II k=
4 (z
ds (zx ch(2x))
(57)
As shown in Fig. B.3, leveraging the factorization of Eq. 60 reduces the errors estimating causal queries in complex graphs, where colliders and dependent hidden confounders present.
37

---

B.4 Ablation on train size
We have proven theoretically that DeCaFlow is able produce correct estimates of the identifiable causal queries, having that DeCaFlow achieves a perfect matching of the observational distribution, P.M (x).
Although normalizing flows are universal density approximators [50], as a machine learning method, its performance improves
increase the size of the dataset.
Therefore, to further investigate the behavior of De- CaFlow under varying data availability, we conduc- ted ablation study the training data size. This analysis allows us to assess how the model's ability estimate causal queries evolves as the number of observed samples increases. Since the objective of DeCaFlow is recover the underlying causal mech- anisms by matching the observational distributions, is crucial to understand how data scarcity affects this matching process and, consequently, the accur- acy of downstream tasks such ATE and counter- factual estimation.
Ocacle
0.6
ATE Lrver
Cf Er
0.4
10a
10a
10
10a
10a
10
Train Size
Train Size
Figure B.4: ATE and Counterfactual error in Sachs' additive dataset, varying the number of train samples. Test size are the same for all realizations. Metrics and 95% CI over 10 realizations and all confounded identifiable effects, intervening in percentiles 25 and 75 of each intervened variable. Oracle represents CNF that observes the confounders.
Fig. B.4 reports both ATE and CF estimation errors function of the training set size. As expected, the errors systematically decrease when more data are available, since the model obtains a more accur- ate approximation of the data distribution. Notably, DeCaFlow exhibits a similar trend to the oracle, with both ATE and counterfactual errors monotonically decreasing as the number of training samples grows.
In contrast, the CNF (unaware confounders) also benefits from larger datasets, but shows a slower improvement rate and an earlier plateau, since it does not have guarantees of correct causal estimation even if it matches the observational distribution. These results empirically validate our theoretical claims: as the training distribution approaches to the true observational distribution, the guarantees of DeCaFlow hold, leading to vanishing estimation errors.
B.5 Semi-synthetic Sachs' dataset
This dataset represents a network of protein-signaling in human T lymphocytes. Every variable, except PKA and Pleg be intervened upon; therefore, there is not only one causal query of interest, but tens of possible causal queries can arise in this setting. This highlights one the strengths of DeCaFlow, because we only need single trained model answer all identifiable causal queries.
The original data contains total of 853 observational samples; however, have decided evaluate model on semi-synthetic data because of the following reasons:
The original network of Sachs et al. [63] contains cycles, which is violation of one of our assumptions. However, we have found different versions of the causal graph [30, 43] that do not contain cycles. Therefore, we have decided to employ the causal graph that appears in the library bnlearn [67]-a recognized library for Bayesian Network learning-aa as ground truth causal graph. The best way to ensure that the causal graph used is the ground truth is by generating samples according to the causal graph. In addition, that causal graph is the one used by Chao al. [9].
We can compare model with one of the baseline models, DCM, with the same dataset as Chao et al. [9] used.
Semi- Semi-synthetiic data allow to compute all metrics evaluate causal queries, having the ground truth.
38

---

Table B.I: Performance metrics on Sachs datasets. Me over five runs and all causal queries of interest. Interventions Raf, Mek and Akt and evaluating confounded identifiable effects. Bold indicates signific- antly better results (95% CI from a Mann- Whitney test). Lower error values indicate better performance.
Additive
Non-
Model
MMD obs x10*
MMD int X10*
AATB enl x10*
|CF cnr
MMD ebs x10*
MMD x1u*
JATE cm x10
[CF etr
x10*
x10
Oraclo
CNF
1.841.5 84
7.506.17
6.056.8a
10.03:0
5.962. 87
6.712.07
2.34g.02
4.843.48
DeCaFlow Deconfounder
2.150.54
8.04
4.49G. 76
92.95amm on
6.122.12
7.58 92
61
1.83
Aware
31.34 31.3433.45
71. 13.1..8.9 98
8.14g0.
63. 03.15y9.12
CNF
5.80 c.
44.4939.12 73.94s8.78
56.00ss 80 5.11
12.7920.
9.74g6.71
15..51
Unaware ANM
83, 8613.11
110. 110.28
12.443
22. 12.0.
29. 40,2.22 22
$1.90
60.40 00.40144.08
23. 23.8813.94
28. 17.9472.44
DCM
2..80
125.59: 18.20
21.2111.34
28.256..8
14.234.37
69.74gp0.s1
8.447.96
27.5023.71
The interventions made in the real world dataset are soft interventions, i.c., an external factor used that modifies one of the variables, changing- On the other hand, DeCaFlow performs hard interventions, making it unclear how to compare the two causal queries.
For generating the data in this experiment, we have followed the procedure proposed by Chao et al. [9], where they take the causal graph of Sachs et al. [63] and the empirical distribution of the root nodes, and generate the rest of the variables with random non-linear mechanisms. In addition, exogenous variables have been included in an additive and non-additive manner, respectively.
In the following, we complement the figures presented $5 with a table that summarizes all the interesting metrics, evaluated on the confounded identifiable causal queries shown in Fig. 8. Interventional distributions and counterfactuals have been computed intervening in percentiles 25, 50 and 75 of the intervened variable.
Since observational MMD is computed only once, the statistics given in Tab B.1 calculated only over 5 runs. On the other hand, we have as many interventional MMDs per run as interventions have been made. However, the statistics of interventional MMD) computed over all the interventions of all intervened variables and 5 runs (5 runs X 3 intervened variables = 15 samples). Finally, statistics over counterfactual error and ate error aggregate all the intervention-outccmee pairs over the five runs. For example, in this case we intervene in 3 variables, performing 3 different interventions and evaluate in 3, 2, and I variable, respectively, for each intervened variable, and we have total of (3+2+1)x3x5 90 different measurements compute the statistics.
The metrics in Tab B.1 indicate that DeCaFlow outperforms all basclines across all interventional and counterfactual causal queries in both settings of the semi-synthetic datasets. However, discussed in $6, a limitation of our empirical approach that the differences in observational MMD, the selection criterion for CGMs, marginal between the oracle, DeCaFlow, and CNF Notably, DeCaFlow even achieves lower MMD than the oracle. This discrepancy arises because the number of variables is large, and the MMD differences are on the order of 10-4
B.6 Semi-synthetic Ecoli70 dataset
The Ecoli70 dataset represent the gene expression of 46 genes of the RNA sequence of the Excherichia coli bacteria. The assumed causal graph comes from the study of [66]. which provides insight into the regulatory mechanisms governing E. coli gene expression. Examples of interventions in these networks are gene knockout and gene over-expression [41]. A priori, there could be several variables in which intervening can be interesting in evaluating the effects in the cell.
For this experiment, have generated the data in the same way done with Sachs' dataset with random mechanisms, but in this case, since we do not have enough samples, root nodes follow standard Gaussian distributions. We have included additive and non-additive ways of including exogenous variables. In this
39

---

Table B.2: Performance metrics on Ecoli70 dataset. ATE and CF error statistics computed aggregating all causal queries and 5 runs. Intervened and evaluated on the direct confounded identifiable causal effects of Fig. 1. Bold indicates significantly better results (95% CI from a Mann- Hoin-Whinney U test). Lower error values indicate better performance.
Additive
Non-additive
Model
MMD obs x10*
MMD int
JATE carl x10"
|CF crrl x10"
MMD obs
MMD int x10*
JATE cn) x10*
|CF ctrl
x10*
x10*
x10*
Ornele
CNF
2.340.82
6.05s.28
5.047.42
9.91
1.49..77
4.05g.22
3.514.84
1.67,.04
DeCaFlow
2.420.52
7.04g.89
4.490.70
12.95g.o
1.58p.05
9.222a
8. 8.7917.00 01
2. 2.152.10
Aware
Deconfounder
27. 27.30 17
12.15
30.00sa.24
9.90p.47
CNF
2.98
23.91 10.2512.18
34.0223.00
1.95p.77
10. 10.2020.57
12.7219.21
2.45g.00
Unaware
ANM
32.802.81
14.331r.e2
21. 8828 89
31.3320 64
13.17s.95
27. 27.56a..57
15.041s.18 2.7
DCM
31.65p.27
49.503n.x3
24. 4533.3 31
30 30.222s.66
18. 786.01
33.3736.14 15.0T22.37 2.362.08
case, we have used a semi-synthetic dataset because the real dataset available in bulearn [67] contains only 9 samples.
In Fig. 1 is presented the causal graph of this setting. In addition, note that Fig. 1 has been extracted from our Alg. 6 of causal effect identifiability. That is, we have specified the causal graph and the variables that are unmeasured, and our Algorithm returns (in green) all the paths that identifiable by DeCaFlow. Consider that black arrows are also identifiable, not only by DeCaFlow, but also for any CGM that approximates the observed data. In red, arrows that are not identifiable by DeCaFlow because there are not enough proxies infer an unbiased causal effect.
A table summarizing the results obtained in the estimation confounded identifiable causal queries are presented in Tab B.2. The statistics have been computed in the same way as in Sachs' dataset. In the case of ATE and CF error, they have been computed only on the direct confounded identifiable paths, i.e., the green paths in Fig. 1.
DeCaFlow significantly outperforms the basclines in ATE and counterfactual estimation in the additive setting and in ATE estimation in the non-additive setting. The MMD differences, both observational and interventional, are negligible between the oracle, DeCaFlow, and CNF, likely due to the high number of variables diluting estimation bias. Counterfactual differences in the non-additive setting are also insignificant. However, compared to the oracle, the gap between the oracle and unaware CGMs is smaller than in the additive case. While DeCaFlow reaches an intermediate point, the difference remains insignificant.
B.6.1 Comment the deconfounder results
One may realize that the errors committed by the deconfounder of [75, 76] are greater than those from unaware models. First, we want underline that, although the deconfounder allows predict counterfactual queries, the algorithm does not present any guarantees of a correct counterfactual estimation since it does not model the exogenous variables of the SCM. We hypothesize this be the reason behind its performance in counterfactual estimation.
Moreover, let us explain some of the other paths where the errors of the deconfounder are greater than for unaware models. In Sachs' dataset, to model the causal effect Ekt->Akt, the factorization model of the deconfounder uses Raf, Mek, Jnk and P38 to extract the substitute confounder; the factorization model assumes that all those variables are independent conditioned to غ, while that is not the case in the true SCM and, therefore, this SCM violates the independence assumption of [75]. The same argument is valid for the paths ycaD-, lacA yceP-y ydeE- and pspB->psp4
On the other hand, the paths JacZ-yyae,, -annA are frontdoor paths that DeCaFlow can identify because it models the hidden confounder following the true causal graph. However, the deconfounder is not designed to model this paths. To evaluate its performance for frontdoor paths, deconfounder uses the same variables as DeCaFlow to extract the substitute of the confounder. However, the deconfounder assumes independence
40

---

Table B.3; Performance metrics on Ecoli70 dataset. Statistics computed an all samples over 5 runs, intervening and evaluating only in the causal effects that deconfounder should solve. Bold indicates significantly better U test). Lower enror values indicate better performance.
results (95% CI from a
Model
ATE emr x10²
CCF emr x10
Oracle
CNF
10.9
1.491.86
DeCaFlow
7.787.30
1.871.50
Aware
Deconfounder
14.3515.24
12.0315.81
CNF
27. 27.82x0.17
4.01a.62
Unaware
ANM
27. 27.6329.74
3.644.15
DCM
42.4554.23
4.084.12
conditioned to the substitute confounder and that is not the case; therefore, are violating the independence assumption again.
The only two paths that meet the deconfounder as- sumptions in Fig. are lacA->ladh and yedE -pppB.. In consequence, we can observe in Fig. B.5 that in those paths, the deconfounder performs least as well unaware methods. On the other hand, all the factor models used for the deconfounder implement- ation (PPCA, Deep exponential families and Vari- ational autoencoder) assume additive noise. There- fore, interventional distributions in non-additive set- tings are not computable theoretically with these case.
Deoefrenky
0.8
0.6
CP ome
Figure B.5: ATE and CF error evaluating only links where the deconfounder should work in the additive
models.
B.6.2 Metrics the other paths
this subsection we include a comparison between all the models in the unconfounded and the unidenti- fiable effects. For unconfounded effects, our expect- ation is to observe that all the CGMs achieve a per- formance comparable with the oracle. On the other hand, we expect have higher errors in unidentifi- able effects, since we do not have theoretical guar- antees.
Orccl
DeCaFlow
CNF
ANM
DCM
Additive
Nonadditive
Unconfounded Effects. The results for uncon- founded effects are summarized in Fig. B.6 and Tab B.4, considering only direct effects ATE and counterfactual етror computations. As expected, De- CaFlow and CNF achieve metrics comparable to the oracle in both ATE and counterfactual estimations, particularly evident in Fig. B.6, where error distributions are nearly identical. B.4 does not show statistically significative differences between DeCaFlow and CNF. Notably, architectures based causal normalizing flows outperform ANM and DCM, which model each causal mechanism, ft. with separate networks. This difference is crucial in settings with many variables and complex relations, where scalability is essential. Un- like ANM and DCM, which suffer from error propagation and limited scalability, causal normalizing flows leverage single amortized model, making them more efficient in high-dimensional scenarios.
Figure B.6: Error boxenplots on the Ecoli70 dataset for different CGMs, averaged over all unconfounded direct effects (see Fig. 1) after intervening in their 25th, 50th, and 75th percentiles and 5 random realizations of the experiment.
41

---

Table B.4: Performance metrics on Ecoli70 dataset. Statistics computed on all inconfounded direct effects and runs. Bold indicates significantly better results (95% CI from a Mann-Whitner test). Lower error values indicate better performance.
Additive
Non-additive
Model
MMD int
|ATE emr
|CF em
MMD int
|ATE eml
|CF em
x10
x10²
x10
x10*
x10²
x10
Oracle
CNF
3.723.73
2.002.27
1.273.49
1.942.00
1.92
1.764.10
Aware DoCaFlow
4.53
2.002.07
1.31
2.83c.36
1.931.95
1.62
CNF
4.776.u9
2.022.21
1.22s.18
2.977.04
1.951.92
1.713.93
Unaware ANM
34.72s.06
3.57a.u2
2.024.09
1312.7
3.533.15
2.6 2.645.34
DCM
36.2314.29
3.482.75
2.692.30
21. 21.2211.0s
3. 3.422.63
3.00a.42
Orazle
DeCaffow
CNF
ANM
DCM
0.4
0.2
0.0
Additive
Nonadditive
Figure B.7: Error boxenplots on the Ecoli70 dataset for different CGMs, averaged over all unidentifiable direct effects (see Fig. 1) after intervening in their 25th, 50th, and 75th percentiles and 5 random realizations of the experiment.
Finally, note that the deconfounder has not been included in these metrics because it is not designed for unconfounded queries and there are many queries, while one deconfounder model is needed for each query.
Unidentifiable Effects. The results for unidentifiable effects- causal queries that violate the assumptions in $4 are summarized in Fig. B.7 and Tab B.5. Notably, the oracle performs significantly better than the other CGMs. As seen in Fig. B.7, error distributions are highly skewed, with ATE and counterfactual errors reaching extreme considerin that metrics are computed on the standardized variables. Tab B.5 shows no significant differences between the metries achieved by DeCaFlow and CNF.
B.6.3 Hyper-parameters and splits
We have performed hyperparameter grid search over validation data in both experiments semi-synthetic datasets, exploring a large combination of hyperparameters for each model and dataset.
These are the parameters that were modified for each model:
CNF: the number of neurons and hidden layers of the single-layer flow, the type of flow (MAF, NSF). LR scheduler reducing on plateau and early stopping were applied with Adam optimizer [32].
DeCaFlow: number of neurons and hidden layers of the single-layer causal flow (generative network), type of generative network architecture (MAF, NSF), number of neurons and hidden layers of the single- layer encoder flow (inference network). type of encoder architecture (MAF, NSF), KL regularization (True, False). LR scheduler reducing on plateau and early stopping was applied with the Adam optimizer [32].
Deconfounder: type of factorization model (PPCA, VAF, Deep Exponential Families), number of neurons and hidden layers (in case of deep models), type of outcome model (MLP, random forest, linear regression), number of neurons and hidden layers of the outcome model (in case of deep models).
42

---

Table B.5: Performance metrics on Ecoli70 dataset. Statistics computed on all unidentifiable direct effects and runs. Bold indicates significantly better results (95% CI from a Kann-Whitney test), Lower error values indicate better performance
Additive
Model
MMD int
ATT eml
|CF etrl
MMD int x10
|ATE emr
CCF etr
x10
x10²
x10
x10²
x10²
Oracle
CNF
3.713.52
1.791.36
5.8815.16
16.98a.87
1.751.59
1.624.57
Aware
DcCaFlow
3.80g.01
3.957.89
33.62g0.37
23.0221.96
1.751.es ..
1.88
CNF
4.54
4.7510.60
44. 76126.8
20.22 20.226.os
2.323.
2.13g.20
Unaware ANM
34.38s.17 7.4312.64
52. 7017799
130.7L41.64
4.013.82
2.937.21
DCM
35.494.96
7.6713.93
67. 67.46132.21
198. 198.23ps.62
3.432.78
3.29s.92
DCM: number of neurons and hidden layers of each network, learning rate and number of iterations (we have not introduced carly stopping or learning rate scheduler). The rest of hyperparameters were selected to the default value in the original code.
ANM: an automatic search was performed across several models in the original DCM code. This search is performed with the DoWhy package [7].
The selection was based on the matching of the observational for the causal generative models, using MMD. In the deconfounder, the factorization networks were selected by the likelihood of the observed variables and the outcome models with maximum likelihood.
Although including all hyperparameters would be very extensive, we give here a sample of the hyperparameters selected for DeCaFlow in the Ecoli70 additive dataset:
Hidden neurons of causal flow (generative network): x 128
Type of causal flow (generative network): neural spline flow (NSF) [14].
Hidden neurons of encoder flow (inference network): x 64
Type of normalizing flow (inference network): neural spline flow (NSF) [14].
Regularize: True (warm-up: 30 epochs)
Total number of parameters: 182k.
Both experiments were performed with 25,000 data, split into 80%, 10%, 10% (train, validation, and test). All metrics are given over the test dataset, and hyperparameter search was performed over the validation dataset.
B.6.4 Processing times
All the experiments were conducted on CPU. Although the experiments were carried out on cluster of different CPU, we include here two tables for the two semi-syntie memi-synthetic datasets (Tab B.6 and Tab B.7) with the processing times measured in CPU Intel(R) Core(TM) i7- 17-13650HX laptop, just to show that even in a laptop CPU, the training and inference times are sensible even for large datasets as the Ecoli70 dataset.
Note that DeCaFlow takes more time in training. This is because the network is complex, due the inference network, and that we have to sample from the posterior distribution. However, the difference in inference is not that relevant. In fact, DeCaFlow takes less time than the oracle in inference, even when they are sampling the same number of variables (hidden confounders observed variables). The unaware causal normalizing flow (CNF) only samples from the observed variables. That is why the inference time is lower.
43

---

Table B.6: Computation times per model across training and evaluation regimes for Ecoli70 additive dataset. Mean and standard deviation of the training and inference time over 100 epochs in training and over 7 interventions in inference.
Model
Epoch Tr. [s] (20000 samples)
Interventional [s] (2500 samples)
CF [s] (2500 samples)
Oracle
0.64g.06
0.30
0.36g.03
DeCaFlow
1.
0.28p.02
0.35o.04
CNF
0.60o.07
0.260.01
0.32o.05
Table B.7: Computation times on the Sachs' Additive Dataset. Mean and standard deviation of the training and inference time over 100 epochs in training and over interventions in inference.
Model
Epoch Tr. [s] (20000 samples)
Interventional [s] (2500 samples)
CF [s] (2500 samples)
Oracle
0.32p.06
0.08p.001
0. 0.1022.011
DeCaFlow
0.75p.12
0.05p.004
0. 0869.005
CNF
0.33p.06
0.048p.003
0. 0.065g.006
B.7 Law school fairness use-case
The experiment with real-world data was inspired by Kusner al. [40] and Javaloy al. [27]. The goal is find a fair estimator of the decile of the grades each student will occupy in their third year of university.
The dataset contains information 27 000 law students who admitted by the Law School Admissions Council (LSAC) from 1991 to 1997. We have performed an experiment similar to that carried out by Kusner al. [40], where race and sex treated sensitive attributes. We have considered the following variables to include in our study:
Race: binary indicator of the race that distinguish between white and non-white.
Sex: binary indicator of the sex that distinguish between male and female.
Fam: family income.
LSAT: the grade achieved in the Law School Admission Test (LSAT).
UGPA: the undergraduate grade point average (GPA) of the student previous to the admission.
FYA: ferst-year average grade.
Decile3: the decile of the grades in the third year of university. This is the variable to predict.
We consider that an estimator ý is fair if it meets Demographic parity. defined as follows [40, Def. 3]: A predictor ý satisfies demographic parity the predicted distributions for different values of a sensitive attribute are equal: p( Maximum Mean Discrepancy (MMD) [21], where lower distance between the predictions of two sensitive groups denotes a fairer predictor.
0) p(ý = 1). We evaluate the difference between predicted distributions using
The assumed causal graph is slightly different from that of Kusner et al. [40], since their purpose is to make a fair prediction FYA accounting only for Race, Sex, LSAT and UGPA. However, include Fam and FYA predictors and the task is to predict Decile3 and the assumed causal graph is the one of Fig. 9.
Proposed fair predictor with DeCaFlow. We propose to model the confounded SCM presented in Fig. B.8, where are explicitly shown the exogenous variables, that are independent the other variables the graph except of their associated endogenous variable.
Afterwards, we predict the outcome, Decile3 from the extracted latent variable that acts as substitute the kmowledge and the exogenous variables of FYA and Fam, following the causal graph of Fig. 9, using gradient-
44

---

ug
Sex
GPA
UG
UR
Race
LSAT
UFI
Fam
FYA
UFYA
Figure B.8: Confounded SCM modeled by DeCaFlow.
boosted decision tree [17]: DDDecile3 upI, uPyA. z). DeCaFlow models z and the exogenous variables as independent from Race and Sex. Therefore, the prediction of Decile3 should be fair.
Baselines. We consider as baselines the methods Fair K and Fair add proposed by Kusner et al. [40].
Fair K is a fair predictor categorized in Level in Kusner et al. [40], which postulates that the student's knowledge, know affects GPA, LSAT, FYA and Decile following the distributions described below.
Fam N (bram + w Fame,
GPA N (ba + w know + w@Race + wosex + w Fam, oG)
LSAT Poisson (exp(bL + W know + Race + wfsex + wL Fam Fam))
(58)
FYA N (w know + w Rfce + w + of fammamm 1),
Decile3 Poisson (exp(w)) know know Race wHsex (amf)
know (0,
Then, the posterior distribution know is inferred using Monte Carlo with the probabilistic programming language Pyro [6]. The outcome predicted using the inferred know using a gradient-booster decision tree [17]: ((Decile3 know).
On the other hand, Fair Add predicts the outcome from the residuals of predicting each variable with each parent, which guarantees that these residuals are independents of Race and Sex. That is, the predictor estimates the distribution p(Decile3 FFam FUGPA: ILSAT, reYA). where these residuals are computed as:
FFam Fam
E Faam
Sex, Race]
TUGPA UGPA
[GP
Sex, Race, Fam]
(59)
ILSAT
LSAT
E[LSAT
Sex, Race, Fam]
rFYA =
FYA
[[FYA
Sex, Race, Fam]
All predictors used are gradient-boosted decision trees [17].
Discussion of Results. Although the fair methods proposed by Kusner et al. [40] achieve significantly better demographic parity than approach using DeCaFlow (as indicated by much lower MMD), their predictive performance is substantially inferior. Specifically, their performance is comparable to predicting the outcome using only the mean of the distribution, which serves baseline in Tab 1. In contrast, DeCaFlow achieves a 98% reduction in MMD while incurring only an 11% increase in RMSE, as illustrated in Fig. 10.
These experiments demonstrate that leveraging DeCaFlow to model confounded Structural Causal Models beneficial beyond causal query estimation, leading improved overall performance.
45

---

Algorithm 1 KL. regularization term in the training loop
1: function ELBO COMPUTATION(epoch, warmup, ())
2: if' epoch < warmup:
3: [( 0) Eg [log po x z)] 3 KL x) p(z)]
4: else:
5: C(o, 0) ee po 2)] (L]go()/x))pp(a
6: return L
7: end function
Implementation details
Posterior factorization of the deconfounding network
DeCaFlow is capable of modeling confounded SCMs that contain several hidden confounders, {z+} )= as in the Sachs' dataset (Fig. 8), Ecoli70 dataset (Fig. 1) or the Napkin graph (Fig. A.5). In such cases, the posterior over latent variables factorizes. We propose factorized posterior in which each hidden confounder is conditioned on its children and on the parents of its children.
17x II k=1
(z x)
To % pa(z)) Uch(zz) U (pa(c) (z) = k)) cEch(s;))
(60)
Since we propose use conditional normalizing flow the encoder, the dependencies between hidden confounders are modeled in autoregressive manner. The rightmost part of the conditioning set accounts for collider-induced associations: conditioning on a child of 2t. e, makes Zę dependent on other parents of c. Other parents of c can also be hidden confounders. To model this, a causal ordering of the components is assumed to avoid cycles in factorization, but it does not affect estimation, as collider associations have no inherent causal direction.
C.2 Regularization of the Kullback-L.eblle term in ELBO
We propose the implementation of a warm-up adaptive regularization term that weights the contribution of the Kullback-Leiile epoch is lower than the predefined warm-up parameter, the KI. term is weighted by B, which we define as 3 = min(1, KL (z) x)] p(z))). as shown in Alg. 1.
term in the ELBO, to avoid posterior collapse [73]. During training, if the current
this way, encourage the model to focus on data reconstruction on the first epochs, ignoring the KL term if the posterior is very similar to the prior, i.c., if KL x= O, then 3 z 0 and C(ó, 0) ss Ey [log pe(xl z)]. After the warm-up epoch, the loss is equivalent to the usual expression for the ELBO. We have tested in the ablation study of $B.2 that the inclusion of the regularization term is useful in the Sachs' dataset. On the other hand, when posterior collapse does not occur, the term will be upper bounded by 1, therefore, not affecting the training process.
C.3 Structural inductive bias
As presented in the original paper by Javaloy et al. [27], the adjacency matrix that represents the causal graph is used build the normalizing flow. In practice, this is implemented following the usual implementation of autoregressive normalizing flows using a Masked Autoencoder for Distribution Estimation (MADE) hypernetwork [18] that uses the causal graph for masking. this case, we introduce the structural constrains between i) exogenous and endogenous variables and ii) conditional variables and endogenous variables.
As a result, our deconfounding network factorizes the posterior distribution as shown in Eq. 3, modeling each hidden confounder as a function of its children, its parents and the parents of its children. Similarly, the
46

---

Causal Graph
Adjacency matrix
g
6
1
Doagonal
21
R2
W
Deconfounding network
T
Generative network Tp
u/a
81 c (pa(e) ( AP)
Jo(w, w.w.bbbe (.))
~ pv(w(ks)
braau(biw
Ry #s
12:
(ci,es) (s),
Adjacency
Adjacency
Conditional adjacency
Conditional adjocency
Ethas May My
(w,b,t,y) - [s),2s]
(Rt, [w. b.,,)
Figure C.I: Complete illustration DeCaFlow architecture, expanding Fig. 2 applied to the specific graph of Fig. A.5. Both the deconfounding and the generative networks are conditional normalizing flows that factorize the distributions the posterior and endogenous variables following Eq. and Eq. 2, respectively. Within these networks, functional dependencies are represented following the compacted version from Javaloy et al. [27, Fig. 4(c)). The orange edges of the encoder corresponds to the collider association in the posterior factorization, and encodes that associations.
structural information in the generative network allows to model each endogenous variable exclusively from its parents, whether these are other endogenous variables or hidden confounders, following Eq. 2.
We include in Fig. C.1 expanded version of Fig. for the Napkin causal graph (Fig. A.5), where it is shown detail how its structural constraint is introduced in each conditional normalizing flow. Finally, note that the do do-operator is inherited from the CNFs [27], and details its extension for DeCaFlow can be found in $D.
47

---

D Do-operator
We introduce now the algorithms that DeCaFlow employ to generate interventional and counterfactual samples. First, include those of Javaloy al. [27]. Note that the notation applied for DeCaFlow is slightly different from the that used for CNFs by Javaloy et al. [27], naming the intervened variable as t, instead x,, in order be consistent with the notation used in $$2 and 4.
D.1 Do-operator in causal normalizing flows
Algorithm 2 Algorithm to sample from P(x do(x; = a)). From Javaloy et al. [27].
1: function SAMPLEINTRRVENDDDIS((...1
2:
u Pu
Sample value from the observational distribution.
3:
x Tg (u)
4:
x; 1---0 a
> Set z; to the intervened value a. Change the i-th value of u.
5:
u; (x);
6:
x Tg (u)
7: return x
b Return the intervened sample.
8: end function
Traditionally the computation of counterfactual samples follows the abduction, action and prediction steps postulated by Pearl et [57]. The abduction step consists of using the observations to determine the value of the exogenous variables. Then, the action step computes the intervention, modifying the causal mechanism of the intervened variable and prediction consist of using the exogenous variables and the modified SCM compute the counterfactual. The computation of interventional samples follows similar pattern, yet the exogenous values are directly sampled, i.c., skipping the abduction step. Javaloy et al. [27] proposed alternative implementation where, instead of modifying the causal mechanisms in the action step, the distribution of the exogenous variable associated with the intervened variable changed instead, as described in Algorithms 2 and 3.
Algorithm 3 Algorithm to sample from P(x" do(x; a), x²). From Javaloy et [27].
1: function GETCCOUNTERFACTUUN.../// i.
2:
u Tx(x))
Abbduction: Get u from the factual sample.
3: x; <-
Action: Set æ; to the intervened value
4: u, + (x(x"): )
AAction: Change the i-th value of u.
5:
x + To (u)
Prediction: Get counterfactual
6: return x
Return the counterfactual value.
7: end function
D.2 Do-operator in interventional distributions with DeCaFlow
The sampling process consists of first sampling from the prior distribution of the latent variables and from the exogenous distribution. Then, one can use the generative network (To) to generate interventional sampling, changing the components of u associated with described in the previous section for CNFs. Note that not an input of the normalizing flow, but a condition (or context). Therefore, z is transformed neither the forward nor reverse pass of the normalizing flow.
48

---

Algorithm 4 Algorithm to sample from the interventional distribution, P(x do(( a:) with DeCaFlow.
1: function SAMPLELNTERVEEEDDDISII,,, a))
2:
Pz
Sample value from the prior of b Sample value from the observational distribution.
3: u~ Pu
4: x + (u)
5:
t a
b Set t to the intervened value a.
6: 4, + (X),
Change the component of associated with t.
7: x Tø (u)
8:
return
P Return the intervened sample.
9: end function
(m) px(x)
4- (n, w. b,t a,y),
x T To (u)
p(2)
To
8
To
To
No ~ p(u))
D. 2v(n8)
11
1h
po 8)
p(..)
~ (u(w))
11g
13
p(u))
b pu(b/8)
11
U1
b b~ po(b/k)
~ p(u)
paftla, b.2)
t=a
p(u))
polyl,
Uy
p, do(t),
Figure D.I: Schematic of the sampling process for an interventional distribution using the graph from Fig. 3 and intervening in By sampling from the prior of the hidden confounders, p(z). and the exogenous distribution, p(u), we obtain samples of the interventional distribution by appropriately setting u, i.e. samples from po Pyl dt)... Note that sampling from the interventional distribution only requires the generative network, Tg. Dashed gray arrows represent the cancellation of causal effects due to the intervention.
As we can easily sample from interventional distributions, we compute the average treatment effect (ATE) via Monte Carlo. For example, compute the ATE comparing two interventions (01, az) in the variable t, we would generate samples of both interventional distributions, p(x do(t a1)),p(x do(t = a1)), and approximate their expectations by taking the sample average:
NTEx(an 0:2) = E[x do(t = a2)l
E[x do = a1)]
(61)
Σ
x
Σ
x
(62)
N
N
P do((=)2))
x P(x do((=o.))
If we were interested in the ATE of a subset of variables, e.g., y, would simply need generate samples of x and take only those from the variable of interest, y.
D.3 Do-operator in counterfactuals with DeCaFlow
As part of the abduction step, model estimates the posterior distribution of hidden confounders given factual datapoint, go (z x"). Therefore, we can sample from the inferred posterior the hidden confounders, and use those samples as the context for the generative network.
49

---

Algorithm 5 Algorithm to sample from the counterfactual distribution, P(x do(t a)) with DeCaFlow.
1: function GETCOUNTERFACTUALE (x', a,
2: as(z x!) < Deconfounding network(x*)
Abduction: Get z from the factual sample. >Abduction: Sample the posterior distribution. DAbduction: Get u from the factual sample.
3: a ~~ lo(2 )
4:
u (xf)
5:
i' +ta a
>Action: Set t the intervened value a.
6:
UP f (x)
>Action: Change the component of u associated with t.
7: xa t -Top (u)
Prediction: compute the counterfactual
8: return xf
Return the counterfactual value.
9: end function
Abduction
Action
Prediction
z Te)
11 Tox(x²)
11. 4- (na w bb,t = a,y")
x of
Te (u)
n
T +
To
Te
Te
Un
Un
11
12
W
w
16
U1
Up
bd b
y
6
12
Iima
Figure D.2: Schematic of the process of performing counterfactual inference with the causal graph from Fig. 3 intervening in t. Both the deconfounding network, Tg. and the generative network, Ta. are needed to generate counterfactual samples. Dashed gray arrows represent the cancellation of causal effect due to the intervention.
Additional details on related work of causal inference with hidden confounders
E.1
Methods tailored to graph and query
First, we want to remark that all the following methods are designed to address causal inference queries in specific causal graphs (or sub-graphs) therefore they can be used when these causal relationships hold. We summarize the causal graphs assumed by each work in Fig. E. the following, we assume the notation introduced in $2, where z is the hidden confounder, is the intervened variable (or treatment) and y is the outcome, i.e., variable where we want to evaluate the causal effects.
We have classified the different approaches depending on the graph that they are designed to address. However, there are two considerations that are common for all these approaches. First, these methods follow a two-stage process: i) extracting substitute of the unobserved confounder using variables affected by the confounder or instrumental variables, ž, and ii) estimating the outcome given this substitute, ý p(y t). In this case, one predictor must be trained per outcome, as well as one extractor per independent confounder. Second, none of these methods estimate counterfactual distributions, since they do not model exogenous variables.
Presence of null proxies independent of (Fig. E.la). We say n to be a null proxy of z if it is a child of z independent of the outcome, y. given z, i.e., Il z. When null proxies of the confounder they are also independent of the intervened variable, n I1 z, these proxies can be used to infer a substitute of the hidden confounder. Among these works, Allman et al. [2] and Kuroki and Pearl [39] study the case in
available and
50

---

which the confounder categorical, and use matrix factorization to extract substitute when, either, there exist three Gaussian proxies [2], when the conditional distribution of the confounder given the proxy is known [39], or when other proxies are available [39]. Kallus et al. [28] also employ matrix factorization for cases where the confounder is continuous and the relation with the covariates and treatment (but not with the outcome) is linear. Similarly, Kallus et al. [29] uses kernel functions to extract the substitute confounder when the generators are nonlinear. The most relevant method based on deep generative methods is the one proposed by Louizos et al. [42], where a variational autoencoder (VAE) used to extract the substitute confounder when several null proxies are available, although no theoretical guarantees were provided and it was later shown to struggle in practice with complex distributions [61]. Finally, Miao et al. [46] offer a regression- based approach to estimate the unobserved confounder under equivalence, which assumes that any model of the joint achieves element-wis wise transformations of the latent variables, something that is not feasible to check: p(t,z z n) p(t, V(z) n).
Presence of two proxies: null and not null (Fig. E.Ib). When the null proxies affect treatment (notice that in Fig. Eb the proxy n affects the treatment 1), Miao et al. [45] offer theoretic guarantees of causal identifiability in the presence of another proxy, w, and completeness conditions. The proxy w can be active, that is, it can directly affect y. Then, Tehetgen et al. [71] introduced the two-stage proximal least squares (P2SLS), which infers the substitute confounder from p(w t. P2SLS can be implemented using neural networks to achieve greater flexibility. Several works have followed-up the ideas introduced by Miao et al. [45], aiming to estimate the bridge function, i.e., finding explicit form for the function h shown in Eq. 13. For example, Cui et al. [11] designed a doubly-robust estimator of the ATE by estimating the bridge function semiparametrically. and Mastouri et al. [44] and Kompa et al. [37] applied moment restrictions to estimate the bridge function using deep neural networks. Other works have proposed multiple-robust methods when the confounders are categorical [68].
Instrumental variable (Fig. F.1c). Another condition that enables causal inference is the presence of instrumental variables (IVs), i.e. variables that affect only the treatment and are independent of both the unobserved confounder and the outcome, given the treatment (in Fig. E. is an IV). In the linear case, Pearl [55] and Angrist and Pischke [4] demonstrated how a two-stage regression process can mitigate the confounding bias, as the only effect that occurs from the IV the outcome is through the treatment variable. A substitute of the confounder is then extracted by computing the conditional distribution of the treatment given the IV, i.e., 2 ~ p(t n). Furthermore, Hartford et al. [22] extended this idea to include arbitrarily complex nonlinear uutaa-enerating processes, designing swo-step deep approach based on neural networks.
Multitreatment affected by a common confounder (Fig. E.1d). Finally, the multitreatment scenario has been studied by Wang and Blei [75] and Ranganath and Perotte [60], where it is called multitreatment since all covariates can be seen as treatments over the outcome variable, y. Here, it is assumed that in the true causal model there exist several covariates that are independent given the unobserved confounder. Therefore, Wang and Blei [75] proposed to a factorization model to infer the substitute confounder, such as probabilistic PCA or Poisson matrix factorization. In short, a factorization model assumes that the distribution of all the
treatments factorizes as follows: p(t, z) = p(z) II. =1 p(t// z), which should allow to construct a substitute of the confounder from the posterior of z: showing that the deconfounder does not achieve nonparametric identification without additional assumptions and, notably, one of the alternatives proposed by D Amour [12) highlights the use of proxy variables, which is the approach adopted by DeCaFlow.
p(z t). Later, D' Amour [12] provided counterexamples
Similar to Wang and Blei [75]. Ranganath and Perotte [60] proposed to VAE as the factorization model, adding a regularization term to reduce the additional mutual information between the estimated confounder and the treatment ly, given the rest of treatments, t-j. However, the theoretical guarantees of this approach require an infinite number of treatments to achieve unbiased estimates of the causal effects. Wang and Blei [76] connect the ideas of Miao al. [45] and Wang and Blei [75] ensuring causal identification in the multitreatment setting when we know that some of the treatments can act as null proxies, that is, when they do not affect the outcome. This assumption allows them provide theoretical guarantees when the number
51

---

of treatments does tend to be infinite. In despite of that, factorization model such the one Wang et al. [77] propose can only model independent treatments given the hidden confounder, which greatly limits its practical utility.
What is the relation of the deconfounder Wang and Blei [75, 76] with DeCaFlow? Similar to DeCaFlow, the deconfounder infers the posterior distribution of the confounder substitute from observational data using a generative model. However, the application of factorization model restricts the structural dependencies that the it can model. For example, the deconfounder cannot model the structural dependencies of Fig. E. since the factorization model assumes n IL t I1L. w z. In contrast, DeCaFlow leverages CNFs which can model these dependencies since the causal graph is encoded in the normalizing flow architecture. It is also important to stress that DeCaFlow models the whole confounded SCM, including the exogenous variables. This allows us to compute counterfactuals and train a query-agnostic manner. In contrast, the deconfounder cannot compute counterfactuals and needs of separate model per causal query.
n
w
L2
L
44
(a) One proxy.
(b) Two proxies.
(c) Instrumental variable.
(d) Multitreatment
Figure E.1: Graphs assumed by prior works. (a) Allman et al. [2], Kallus al. [28, 29], Kuroki and Pearl
[39], Louizos et al. [42], Miao et al. [46] address the case where is independent of t. (b) Miao et al. [45] assumes the case where there exist two proxies. (c) Graph with an instrumental variable. (d) Ranganath and Perotte [60], Wang and Blei [75, 76] work with the multitreatment setting.
E.2
CGM with unobserved confounders
There exist several works that employ causal generative models (CGMs) in the presence of hidden confounders. We explain here the differences with our proposal, highlighting the practical advantages of DeCaFlow.
Neural Causal Models (NCMs). Xia et [81] proposed a class of sequential causal generative models where each structural equation-i.e., the functional relationship between a variable and its parents in the causal graph-is modeled by a different neural network. The model is trained end-to-end to jointly learn all structural mechanisms. Beyond estimation, NCMs aim to determine whether a given causal query is identifiable from the data-gnnerating process. To assess identifiability, two NCMs are trained: one that maximizes the causal query, subject perfect observational fitting, and one that minimizes it. If both NCMs yield the same outcome, the query is deemed identifiable. Interestingly, this approach formalizes identifiability empirical condition based optimization agreement.
However, the framework presents significant practical constraints: i) it only supports finite discrete variables, typically binary and dow-dimeniional, due to tractability constraints; ii) it assumes that the true observational distribution is available for training: iii) two NCMs are trained per query, leading to high computational cost; and iv) identifiability is only revealed post-training, offering guidance before the model is trained. To perform counterfactual inference, Xia et al. [82] extended NCMs to estimate queries involving latent exogenous variables. However, their approach relies rejection sampling to perform the abduction step, which is inefficient and unsuitable for continuous or high-dimensional settings, thus limiting its applicability in real-world scenarios.
In contrast, DeCaFlow addresses these limitations. First, we provide a principled criterion to estimate the identifiability of query prior to model training. Second, our framework supports continuous variables and scales to high-dimensional settings. Third, we train a single model that jointly estimates all causal mechanisms and enables efficient inference of counterfactual queries. Fourth, we use variational inference
52

---

approximate the posterior of hidden confounders, avoiding the inefficiency of rejection- baaed methods. Finally, we guarantee the identifiability of unconfounded exogenous variables (in the sense of Xi and Bloem- Reddy [80] by leveraging the theoretical framework of CNFs [27]. As a result, DeCaFlow is substantially more efficient and suited for real-world applications.
Modular Causal Generative Models. Rahman and Kocaoglu [59] introduce a modular framework for high- dimensional causal inference. where variables influenced by the same hidden confounder are modeled jointly in end-to-end submodules, A key advantage of this approach is the ability to incorporate pretrained models into submodules, enabling flexible modeling of complex or structured variables when the modular criterion holds. The method supports continuous and discrete variables and uses adversarial training to match observational distributions. Symbolic identifiability computed using the algorithm of Jaber et al. [26], and they prove that identifiable queries remain estimable under their modular decomposition. However, the framework does not support counterfactual inference nor proximal learning, and relies on adversarial optimization.
In comparison, DeCaFlow trains single end-to-en model, estimates both observational and counterfactual distributions also in proximal settings, and enables efficient inference with broad applicability to real-world settings.
Counterfactual Identifiability of Bijective Causal Models. Nasr-Esfahany et al. [48] propose sequential causal model using conditional normalizing flows to map exogenous to endogenous variables. The model focuses on counterfactual inference under backdoor and instrumental variable (IV) settings, with identifiability proven only for discrete variables. Proxy variables are not considered, and the use of invertible mappings over discrete domains makes theoretical claims less robust. Although the model claims support for continuous data, guarantees are restricted to discrete IV scenarios. Moreover, it does not model observational nor interventional distributions, and lacks parameter amortization due to its sequential structure.
In contrast, DeCaFlow supports continuous variables, models both observational and interventional distribu- tions, and enables counterfactual inference under general confounding and proxy settings. It also requiresa single end-to-end model and scales efficiently to real-world data.
Learning Functional Causal Models with Generative Neural Networks. Goudet et al. [19] propose a method for causal discovery rather than causal inference under unobserved confounding. Given a Markov equivalence class (or graph skeleton), their approach uses generative neural networks to model each causal direction, selecting the graph that best matches the observational distribution evaluated via maximum mean discrepancy (MMD). The model is trained sequentially and assumes no hidden confounders. While not directly comparable our work, such causal discovery tools may serve as a preprocessing step when the causal graph is unknown, enabling downstream application of models-suce as ours that assume known and correct structure.
F Algorithms for causal query identification
As explained in $4.2, we can ask DeCaFlow to estimate any causal query, but we do not have the guarantee that the estimation DeCaFlow does is correct unless the query is identifiable. Therefore, we provide the practitioner with algorithms to check the identifiability of causal queries.
Specific treatment-outeome pair. We start presenting in Alg. 6 an algorithm to identify causal query, given a pair of treatment and outcome variables, which is valid for estimating the interventional distribution of the outcome, p(y] do(t), c), and the counterfactual one, p(ydf dt))), x"), as we postulated $4 that the latter is identifiable if the former is.
We have employed Alg. 6 on all direct paths of the Sachs and Ecoli70 datasets to check their identifiability. in order to get visual representation of the queries that DeCaFlow can estimate in such complex graphs. If one is interested in evaluating a query which involves several outcomes, (Y1, ..oo) one causal query per outcome variable should be evaluated.
53

---

Algorithm 6 Identification of causal queries that include intervention and outcome (t, y)
Require: Graph 9, intervention variable outcome variable y, covariates hidden variables
Ensure: Boolean indicating if query is identifiable
1: f- hidden variables that are parents of both and
2: return True if z is 0
Unconfounded is identifiable
3: for all Zy: E do
4: Comment: Each 2₁ is an independent component of
5: n-proxies t- children of zy d-separated from given (z. c)
6: w-proxies + children of 2μ d-separated from given (z,
7: if there exist n & n-proxies and w w-proxies such that d-separated from given (z, c) then
8:
is deconfounded
9: end if
10: end for
11: return all 2y are deconfounded
Evaluation on all the variables. Although Alg. consists of iteratively applying Alg. 6, we also find it interesting to include the extension to identify causal queries evaluated on variables in the dataset, which useful for the case where DeCaFlow generative model for the joint interventional distribution, p(x do(t)), or to generate joint counterfactual samples intervening in a specific variable, C x, p(x" do((), x)).
Algorithm 7 Identification of causal queries, intervening and evaluating in all variables
Require: Graph G, intervention variable t, hidden variables z Ensure: Boolean indicating if the interventional distribution is identifiable
1: t hidden variables that are parents of
2: for all x; descendants of do
3: Comment: Evaluate only on descendants of the intervention
4: Check (t, x,) identifiability with Alg. 6
5: end for
6: return all (t, x.) are identifiable
54

---

E1 Pipeline for using DeCaFlow
Our framework provides a systematic approach to estimating causal queries by integrating DeCaFlow, model trained on observational data, with algorithms designed for query identi- fiability analysis.
Graph
Queries (Q:
As depicted in the pipeline, the framework takes as input dataset D, a causal graph G, and of N interesting queries (Q:) -==- The process begins by training DeCaFlow on 'D and 9, enabling learn the confounded M..
Simultaneously, the identifiability of each causal query Qi is assessed using dedicated algorithms (Alg. 6 and Alg. 7). If Qi is identifiable, the trained DeCaFlow is used to estimate (M)
DeCaFlow trained
Check Query idenitaauliiy Alg. and Alp.
(Alg. 4 and Alg. 5), yielding the estimated causal effect (M If Q; is not identifiable, the framework indicates that answering the query is not feasible given the available data and causal structure. Other causal queries can be answered by the model without retraining, provided that their identifiability is verified beforehand.
Eatimate Q.(M) Alp. 4 and Alg.
Yos
Q. aleutiialle?
(RM)
This workflow ensures principled approach to causal infer- ence, leveraging both data-driven modeling and theoretical guarantees on identifiability.
Figure F.l: Block diagram of our pipeline.
Validation with interventional data. As a final step in the pipeline for real-world scenarios, especially in sensitive applications, we encourage practitioners validate the framework with interventional data. Causal queries such as the average treatment effects (ATEs) can be validated if a randomized experiment is available in which interventions are carried out on the treatment variable.
55


Reviews:
[{"review_id": "EcHVCJDtJv", "reviewer": "Reviewer_4Y4D", "summary": "This paper introduces DeCaFlow, a method for causal inference in the presence of latent confounders. \nThe primary contribution is extending causal normalizing flows to the latent confounded case.\n\nGiven a SCM with endogenous variables X and latent confounders Z, this work assumes:\n- Underlying causal graph G is known and acyclic\n- Structural equations f are invertible given Z, and both f and its inverse are continuously differentiable\n- Perfect interventions (do-calculus) over X\n- Access to proxy variables (CI variables informative of hidden confounders) for identifiability beyond what do-calculus provides\n\nThe method is implemented via a latent variable model, where the encoder (X to Z) and decoder (Z to X) are based on normalizing flows.\n\nEvaluations consider 3 settings: two semi-synthetic (real graphs, simulated structural equations) and one real (law school dataset).", "questions": "1. In 5.2.1, new SCMs were sampled based on the Sachs graph to ensure that \"the randomized effect of the hidden confounder is perceptible\" [337]. Since the effect size of the original interventions can be quantified (based on the change in target protein activity), is it possible to evaluate the method on (some of) the actual data?\n\n2. The paper is somewhat dense. It would be nice if the main paper included more intuitive explanations throughout. For example: what are proxy variables, how the CNFs are implemented, what do-calculus implies in the context of the illustrative example in Fig 1.\n\n3. Is it necessary to include the ablation study as the first part of results? Moving it to the appendix + citing that the model is not sensitive to increasing $D_z$ could free up space for expanded discussion of [above].\n\n4. Overall, the writing can be made more concise to improve readability. Some limited examples:\n- 103-106 can be split into 2 sentences. \n- 136-137 \"we take [....] and adapt them [...]\" -> \"we adapt CNFs to [...]\"\n- 159-161 -> \"To avoid posterior collapse, we implement [...]\"", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths:\n- Experiments show strong empirical performance, approaching the oracle baseline in both semi-synthetic experiments.\n- The proposed method seems efficient and reasonable -- requiring one training run per dataset (vs. one per query).\n\n# Weaknesses:\n\nA primary weakness of this work is whether the setting is indeed realistic (inherited from causal normalizing flows, upon which this work is based).\n\n1. Invertibility\n\nIn biological datasets (like those used in semi-synthetic experiments), confounders take the form of, e.g. which lab technician conducted the experiment, or the batch index of the cells when being sequenced (later batches are allowed to grow for longer). It is easy to train an inference model to predict these confounders. However, it is not always possible for a generative model to generalize: there may be no predictable pattern to how a new lab technician would act, compared to previous ones.\n\nCould you provide some examples in which invertibility does make sense?\n\n2. Knowledge of G\n\nG is often \"known\" only to a noisy extent. For example, there are multiple versions of the Sachs graph, and feedback loops play a central role in biological systems.\n\nCan you comment on the robustness of your method to errors in G?", "was_revised": true, "final_justification": "The author's rebuttal has resolved the concerns that I raised during the review. Therefore I am adjusting the score.", "rebuttal": "Thanks to the reviewer for their questions and suggestions, which will help us improve the readability and understandability of our manuscript. We will address all raised concerns in the next revision, and would appreciate it if the reviewer could reconsider their rating, had we properly addressed their concerns. \nNext, let us discuss the points raised during the review: \n\n> Invertibility (and biological datasets example)\n\nThere are two different questions to address here. As for the example provided, the reviewer is right. DeCaFlow, just like Causal NFs and any other ML model, relies on data and generalization is always a challenge: if a new technician arrives with highly unpredictable behavior (relative to the previous ones) the model could struggle to generalize. However, existing ML techniques (e.g. a small amount of fine-tuning) should help in this scenario. \n\nThen, note that we assume invertibility from $U$ to $X$ given $Z$, which we argue is not a real limitation in terms of the SCMs we can model. The reasoning is that, as we do not observe the exogenous variables and we do not work directly with them (i.e. we do not give them semantics), given a non-invertible SCM we could always use the Knothe-Rosenblatt transport to find an invertible one (again, from u to x given z) which is causally equivalent in all three rungs of Pearl’s causal hierarchy, as argued in the original CNF paper.\n\n> Can you comment on the robustness of your method to errors in G?\n\nThis is an important question to discuss in the manuscript, which we will make sure to add. It is important to note that DeCaFlow is robust to some misspecifications of the causal graph and, to clarify this, consider the following two cases: \n\n1. If the variables are not directly affected by the hidden confounders, we inherit the theory from CNFs (see App. A.1), where it is sufficient to know the causal ordering (rather than the causal graph), and where we can use partial knowledge of the causal graph to group variables together and work with blocks of variables (see App. A.2.2 of the original CNF paper).\n2. If some variables are children of the hidden confounders, our theory relies on the conditions in Prop. 4.1. Thus, if there is a mismatch between the true and assumed causal graphs regarding these assumptions (e.g., proxy variables not being actual proxies), our identifiability results will no longer hold and hence our estimates can be inaccurate. We thus believe that a sensitivity analysis studying causal-graph misspecification is an interesting venue for future work, which we will add to our discussion. \n\n\n> Since the effect size of the original interventions can be quantified (based on the change in target protein activity), is it possible to evaluate the method on (some of) the actual data?\n\nBesides line 337, we explain why we resort to semi-synthetic data in App. B.4. In short, the true causal graph potentially contains cycles, and we do not have access to counterfactuals. Semi-synthetic data solve these two problems. While we could attempt to use the original interventional data to evaluate our model, we could not find a reliable way of disentangling the model prediction error with the error due to the potential causal-graph misspecification.\n\nMoreover, interventions in Sachs’ dataset are soft interventions, while DeCaFlow performs hard ones, making it unclear how to compare them. We find it interesting for future work to extend causal generative models to allow for soft interventions, or to design metrics to compare different types of interventions, but this is out of the scope of this paper and thus deferred to future work. More generally, a real-world evaluation of causal inference approaches remains an open challenge (especially in the context of counterfactuals, since they are not observable by definition [1, 2]) in the literature [3, 4], which like us relies on (semi-)synthetic datasets. Thus, it is an important but challenging venue for future work. \n\n> The paper is somewhat dense. It would be nice if the main paper included more intuitive explanations throughout. For example: what are proxy variables, how the CNFs are implemented, what do-calculus implies in the context of the illustrative example in Fig 1.\n\nWe appreciate the feedback and agree with the reviewer that extra explanations and intuitions can make the paper more accessible. We will thus add to the main paper intuitions on what are proxy variables, a more detailed explanation of what do-calculus implies, as well as how CNFs are implemented in practice, as well as any other explanation that we find missing while revisiting the manuscript, such as a “limitations” section to discuss noisy causal graphs and the validity of our assumptions.\n\n> Is it necessary to include the ablation study as the first part of results? Moving it to the appendix + citing that the model is not sensitive to increasing could free up space for expanded discussion of [above].\n\nAt submission time, we believed it important to validate the sensitivity of DeCaFlow to the choice of $D_z$. However, we can see how in its current state we can defer the ablation to the appendix and free up more space for the changes promised above. We will thus apply the changes suggested by the reviewer in an attempt to make the paper as accessible as possible. We truly appreciate the feedback.\n\n> References\n\n[1] Holland, P. W. Statistics and causal inference. Journal of the American Statistical Association, 81(396):945–960, 1986.\n\n[2] Bareinboim, E., Correa, J. D., Ibeling, D., & Icard, T. (2022). On Pearl’s hierarchy and the foundations of causal inference. In Probabilistic and causal inference: the works of judea pearl (pp. 507-556).\n\n[3] Hill, J. L. (2011). Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics, 20(1), 217-240.\n\n[4] Curth, A., & Van Der Schaar, M. (2023, July). In search of insights, not magic bullets: Towards demystification of the model selection dilemma in heterogeneous treatment effect estimation. In International conference on machine learning (pp. 6623-6642). PMLR."}, {"review_id": "AueKcWmMoQ", "reviewer": "Reviewer_eBDx", "summary": "In this paper, the authors extend existing work on Causal Normalizing Flow and introduce DeCaFlow, for \"deconfounding causal normalizing flow\". The focus of the authors is on scenarios where not all variables of a system are observed and, in particular, where there exists a non-empty set of hidden confounders $\\mathcal{Z}$. The goal of DeCaFlow is to be able to answer interventional and counterfactual queries over observed variables $\\mathcal{X}$. To this end, they propose to train two models: a deconfounding network (from $\\mathcal{X}$ to $\\mathcal{Z}$) and a generative network (from the exogenous noise and $\\mathcal{Z}$ to $\\mathcal{X}$). The authors prove that DeCaFlow correctly solves causal queries between non-confounded variables and provide sufficient conditions for queries between confounded variables.", "questions": "Do the authors have any intuition on how DeCaFlow could be integrated with existing causal discovery algorithms for confounded data?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The problem of estimating causal effects between confounded variables is known, interesting, and well-motivated.\n- As with Causal Normalizing Flows, DeCaFlow also represents a class of models identifiable from the observational distribution.\n- The authors provide in the supplementary materials an algorithm to verify whether a causal query can be identified or not.\n\nWeaknesses:\n- The method requires the causal graph between observed variables, which is not trivial to recover in general, in particular under hidden confounding. This problem is not discussed in the paper.", "was_revised": true, "final_justification": "The authors acknowledged the reported weakness on causal discovery under confounding. Their discussion in the rebuttal, to be integrated in the paper, is satisfactory. This was a minor point that does not change my overall positive assessment on the acceptance of the paper.", "rebuttal": "We thank the reviewer for their positive review of DeCaFlow and acknowledge that there are important but also very challenging venues for future work (e.g., integration of causal discovery and causal inference methods) that could build on our work. Let us respond to the comments raised by the reviewer below:\n\n> The method requires the causal graph between observed variables, which is not trivial to recover in general, in particular under hidden confounding. This problem is not discussed in the paper. Do the authors have any intuition on how DeCaFlow could be integrated with existing causal discovery algorithms for confounded data?\n\nWe agree that this is an important point that should be discussed in the paper, and thus will discuss it in a new section in the next revision of the manuscript. \n\nFor readability, we consider a fully-specified causal graph throughout the paper. However, DeCaFlow is robust to some misspecifications of the causal graph. For example, for all variables not directly affected by hidden confounders, DeCaFlow should be able to capture their causal relationships given only a causal ordering, which is a feature inherited from Causal Normalizing Flows (CNFs). Similarly, we can work with partially specified causal graphs and consider groups of variables, as in App. A.2.2 of the original CNF paper. However, if the given causal graph misspecifies the relations with the hidden confounders, resulting in violations of the assumptions in Prop. 4.1, then our estimations for confounded causal queries may be inaccurate. \n\nMoreover, while there have been some attempts to use CNFs for causal discovery (see [1]), these are still preliminary works. Hence, for causal queries in which there is a high uncertainty about the variables affected by the hidden confounders, the practitioner may instead rely on approaches that jointly perform causal identification and estimation for those individual queries such as [2]. While they also assume knowledge on the causal graph, they perform causal identification for each causal query based on the available data. Such an approach would, however, lose the scalability advantage of DeCaFlow, which with a single model provides estimates for many causal queries in large graphs and handles continuous variables. Consequently, we encourage practitioners to combine both approaches based on the reliability of their assumptions. We will make sure to clarify this in the discussion of the limitations of our work.\n\nWe remain at the disposal of the reviewer, in case they have further questions, and we would appreciate it if they could consider re-evaluating our work after the rebuttal period.\n\n> References\n\n[1] Xi, Q., Gonzalez, S., & Bloem-Reddy, B. (2023, October). Triangular monotonic generative models can perform causal discovery. In Causal Representation Learning Workshop at NeurIPS 2023.\n\n[2] Kevin Muyuan Xia, Yushu Pan, and Elias Bareinboim. Neural Causal Models for Counterfactual Identification and Estimation. In The Eleventh International Conference on Learning Representations, ICLR 2023."}, {"review_id": "tBb82MOQCS", "reviewer": "Reviewer_HMeD", "summary": "This work presents deconfounding causal normalizing flow (DeCaFlow), a framework that extends causal normalizing flows (CNFs) to the case when hidden confounders are present. When a causal graph is given, CNFs are autoregressive normalizing flows that parametrically model the generative mechanisms and allow us to answer causal queries among other applications. DeCaFlows are similar, however they are equipped to handle hidden confounders as well, under some assumptions. The framework assumes access to the underlying causal graph and iid observations, and essentially builds a generative model and a deconfounding model, which are jointly trained using a variant of the standard ELBO loss.\n\nThe paper shows that DeCaFlows satisfy many desiderata, such as identifiability results for causal queries such as do operations or interventional queries (even though we have hidden confounders). However, these results apply under various assumptions on the causal structure of the query and infinite data regime. Finally the technique is validated via experiments on semi-synthetic data and the Ecoli70 gene dataset, and it outperforms prior techniques on standard metrics such as MAE. The counterfactual fairness-prediction application is also interesting. The target audience are people interested in causal inference.", "questions": "None", "limitations": "Limitations have been discussed.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "## Strengths:\n\n- The problem of building normalizing flows in the presence of hidden confounders is important and relatively less studied, so this work is a useful step towards this direction.\n\n- The paper is well-written, and related work seems to be cited well. The proof techniques are very interesting, generalize prior works and the experiments probe different aspects such as misspecification.\n\n## Weaknesses:\n\n- While the identifiability results are interested, I'm unable to gauge how restrictive the assumptions are, e.g. in proposition A.2. It'd be nice if the authors could comment on this.", "was_revised": true, "final_justification": "I read the other reviews and the authors' responses. They clarified my questions.", "rebuttal": "Thanks to the reviewer for their positive feedback of DeCaFlow and for the positive comments about the practical significance of the paper, and its presentation. Let us try to help the reviewer better understand the assumptions made in our work and, if appropriate, re-evaluate our work after the rebuttal period.\n\nRegarding the __assumptions in section 3__ (lines 113-114), these are the same assumptions made in the original Causal Normalizing Flow paper [1]. Namely, we assume:\n\n1. An acyclic causal graph, as it allows us to write the data-generating process in a finite number of steps (i.e. to “unroll” the equations). This is as typical of an assumption that some authors directly define SCMs as inducing acyclic graphs, see [3] for an example.\n2. Diffeomorphic causal structures. This subsumes three conditions: continuity, bijectivity, and differentiability. The first two we argue are not restrictive since, as we do not observe the exogenous variables, we can always reduce a given SCM to a causally equivalent one which is continuous and bijective with respect to the exogenous variables. Differentiability was assumed in the original CNF paper to relate the model Jacobian and causal dependencies of the SCM.\n3. Continuous random variables. While it can sound restrictive at first, the original paper (see their App. A.2.1) argue how to adapt CNFs to allow for discrete random variables too.\n\n\nRegarding the __assumptions in Proposition A.2__, these are quite similar to those previously made by Miao et al. [4] and Wang and Blei [5]:\n\n1. Assumptions i)-iii) regard the conditional independence of different variables in the causal graph, which can be automatically verifiable given a faithful causal graph. An example of such structural assumptions is actually provided in our real-world use case in Section 5.3,  where it is an accepted assumption in the literature (see [7]) to consider knowledge (the hidden confounder)  to be independent of the socio-demographic features (the adjustment set in Assumption 1), and the exam scores GPA and LSAT to be conditional independent among them and with FYA (the outcome), conditioned on knowledge and the adjustment set. Similar examples can be found in other contexts such as healthcare, where it is common to leverage data on other diseases or comorbidities—such as records of routine check-ups or hospital admissions for causes other than the treatment of interest—as proxies for unmeasured confounders. For example, as shown in [4], when measuring the effect of _influenza vaccination_ on _influenza‐related hospitalization_, _health-seeking behaviour_ is an unobserved confounder, and two proxies can be employed: the _number of annual wellness visits_, which correlates with _health-seeking behaviour_ but does not affect the risk of _influenza hospitalization_, and _trauma-related hospital admissions_, which, conditional on _health-seeking behaviour_, is independent of _influenza hospitalization_. Other well known examples can be found in [8].\n\n2. Assumption iv) and v) regards the _quality_ of the variables $\\mathbf{w}$ and $\\mathbf{n}$ as proxies of the hidden confounder. Intuitively, they mean that, as we change their values, the posterior of the hidden confounder varies enough as to properly perform inference on it. Perhaps it helps to know that this notion of completeness has been adopted from that of statistics (cf. “Completeness (statistics)” in Wikipedia). This assumption is harder to verify but, as we show in Figure 6, the greater the number of proxy variables, the better the estimation of the hidden confounder's effect and the more accurate the causal query estimation is.\n\n3. Finally, assumptions vi) and vii) are standard regularity conditions [5,6] that are (almost surely) fulfilled in practice, as long as our random variables are well-behaved (e.g. having finite moments). \n\n\nFinally, for causal queries in which there is a high uncertainty around these assumptions, one could consider alternative approaches that jointly perform causal identification and estimation for those individual queries such as [9]. While they also assume knowledge on the causal graph, they perform causal identification for each causal query based on the available data. Such an approach would, however, lose the scalability advantage of DeCaFlow, which with a single model provides estimates for many causal queries in large graphs and handles continuous variables. Consequently, we would encourage practitioners to combine both approaches based on the reliability of their assumptions. We will make sure to clarify this in the discussion of the limitations of our work.\n\n> References\n\n[1] Javaloy, A., Sánchez-Martín, P., & Valera, I. (2023). Causal normalizing flows: from theory to practice. Advances in Neural Information Processing Systems, 36, 58833-58864.\n\n[2] Reizinger, P., Sharma, Y., Bethge, M., Schölkopf, B., Huszár, F., & Brendel, W. (2023). Jacobian-based causal discovery with nonlinear ICA. Transactions on Machine Learning Research.\n\n[3] Schölkopf, B., & von Kügelgen, J. (2022, April). From statistical to causal learning. In Proceedings of the International Congress of Mathematicians (Vol. 1). EMS Press.\n\n[4] Shi, Xu, Wang Miao, and Eric Tchetgen Tchetgen. \"A selective review of negative control methods in epidemiology.\" Current epidemiology reports 7.4 (2020): 190-202.\n\n[5] Wang Miao, Zhi Geng, and Eric J Tchetgen Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika.\n\n[6] Wang, Y., & Blei, D. (2021, July). A proxy variable view of shared confounding. In International Conference on Machine Learning (pp. 10697-10707). PMLR.\n\n[7] Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2017). Counterfactual fairness. Advances in neural information processing systems, 30.\n\n[8] Lipsitch, M., Tchetgen, E. T., & Cohen, T. (2010). Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology, 21(3), 383-388.\n\n[9] Kevin Muyuan Xia, Yushu Pan, and Elias Bareinboim. Neural Causal Models for Counterfactual Identification and Estimation. In The Eleventh International Conference on Learning Representations, ICLR 2023."}, {"review_id": "nh75ardaHJ", "reviewer": "Reviewer_7wep", "summary": "The paper introduces deconfounding causal flows an extension to causal normalising flows that uses and VAE-style approach to estimate causal queries under hidden confounding. The paper introduces novel indentifiability results that open promise correct estimates on a wide range of causal queries. The experiments support the performance of the proposed method compared to relevant baselines making this a promising method for causal practitioners.", "questions": "- Can this deal with differences of which variables are available during training and inference?\n- A strong requirement here is to perfectly fit the data distribution. How hard is this in practice? How hard is it to estimate if you correctly capture the distribution while only having access to finite samples?\n- How do you chose a prior over z? What’s the impact of this?\n- L 235 - 4. How do you go about measuring “enough information”?\n- How are the hyper parameters chosen? B.3 almost sounds like it’s been chosen given test set performances.\n- How come you can achieve almost 0 ATE error but significantly higher CF error if the identifiability of ATEs mean identifiability of CFs?\n- Why are you replacing the data generating processes rather than simply taken real world causal datasets and drop confounders to estimate the correctness under hidden confounding?", "limitations": "mostly", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "+ The paper tackles the interesting problem of causal inference under hidden confounding providing novel theoretical insights as well as a practical method that should be applicable to a wide range of real-world tasks.\n+ The paper is well written and thoroughly explains the approach and significance. (I did however, not check the correctness of the proofs or appendix)\n\n- The empirical results could be stronger by testing this method on more real-world datasets without replacing the ground-truth functional relationships. The paper could easily produce additional causal benchmarks by hiding confounders for evaluation, turning a fully observed causal benchmark into one with hidden confounders.\n- Could be a bit clearer on the functional limits on CNFs before the conclusion.\n- It’s unclear how the hyper parameters of the models are chosen and how a practitioner should go about using this method.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful comments and positive feedback. Below, we clarify the concerns raised and, once they have been satisfactorily addressed, would appreciate your reconsideration of our evaluation.\n\n> results could be stronger by testing this method on more real-world datasets.\n\nWe agree and would truly appreciate it if the reviewer could provide some specific examples of these datasets. \nUnfortunately, we could not find existing datasets that we could easily use to test DeCaFlow. At the end, we opted for a similar approach as described by the reviewer, where the oracle has access to the hidden confounders and we simply hide them for the rest of the models. Then, to properly measure the error committed by DeCaFlow (among other reasons, see App. B.4 and B.5), we decided to use semi-synthetic data where we have access to the ground truth SCM which is, to the best of our knowledge, the only way to assess counterfactual estimations.\n\nNote, however, that we already test DeCaFlow in real-world data and application in section 5.3. Nevertheless, we find the evaluation of DeCaFlow (and any other CGM) in real-world scenarios with accessible interventional data at scale an exciting research direction, which recent developments such as causal chambers [1] start making this goal more viable.\n\n> Could be a bit clearer on the functional limits on CNFs before the conclusion.\n\nWe may not fully understand what the reviewer refers to as “functional limits” and would appreciate further clarifications. If the reviewer refers to the assumptions on invertibility and smoothness in line 114, these are not too restrictive assumptions since, as we do not observe the hidden confounders, we can reduce the real SCM to a causally-equivalent one which fulfills these conditions, as argued in the CNF paper. Moreover, we will add a limitations section discussing other limitations derived from the assumptions made in our work.\n\n> It’s unclear how the hyper parameters of the models are chosen and how a practitioner should go about using this method.\n\nWe apologize it is not clear, and would appreciate more detailed feedback in this regard. We tried to be explicit, and in App. B.5.3, “Hyper-parameters and splits”, we explain that hyperparameters were selected according to the MMD distance between the validation split and samples generated with DeCaFlow (both using observational data). That is, the best model is chosen by best matching the original observational distribution. All metrics are reported on the test split.\n\nWhile we tried to report all details, e.g., how we regularized the ELBO loss (App. C.2), how we implement the do-operator (App. D.1), or how to use DeCaFlow at test time (App. F.1), we will revise the manuscript and address what remains unclear.\n\n> Can this deal with differences of which variables are available during training and inference?\n\nWe assumed that all endogenous variables in the causal graph are observed in both training and inference (except the hidden confounders). This is, however, an interesting line of future work if we assume that the test dataset has a different set of observed variables than the training set, e.g., using tractable probabilistic models that allow for the marginalization of observed variables. This could lead to semi-supervised settings that can be very useful for real-world problems where we have related datasets extracted from different sources. We will add this in our future work discussion.\n\n> A strong requirement here is to perfectly fit the data distribution. How hard is this in practice? How hard is it to estimate if you correctly capture the distribution while only having access to finite samples?\n\nNormalizing flows are universal density approximators, and if the causal graph is well specified, a causal normalizing flow should be able to fit the data distribution. However, as an ML method, its performance improves as we increase the size of the dataset. Based on the insightful suggestion by the reviewer, we have conducted an ablation study to compare performance across different data regimes which confirms this trend and will be included in the revised paper. This will help practitioners better understand the expected performance of our approach.\n\n> How do you select a prior over z? What’s the impact of this?\n\nIn all experiments we set $p(z)$ as a standard multivariate Gaussian, and we will make sure to add this information in the next revision. Theoretically, the prior of z should not impact the model. To see why, note that we can always rewrite $p(z)$ to a standard Gaussian using cumulative functions without affecting the observed causal model. This is a similar argument as that of the choice of $p(u)$ in the original CNF paper, since their role in the data-generating process is almost identical, see Eq. (1). However, in practice the choice of $p(z)$ can make the problem easier/harder to learn, and any prior information can be leveraged. To see this effect in practice, we refer the reviewer to App. D.3 of the original CNF paper, where the authors ablate the choice of $p(u)$.\n\n> L 235 - 4. How do you go about measuring “enough information”?\n\nWhat we refer to as “enough information” in the main text to ease readability, it is actually the same completeness conditions used by [2] and [3]. Paraphrasing [3]: “Roughly, completeness requires that the distributions of $z$ corresponding to different values of $w$ are distinct”. Refering to [3], while measuring completeness is non-trivial, many causal models satisfy it by construction such as exponential families, location-scale families, and nonparametric regression models.\n\nThis completeness condition is also employed in nonparametric causal identification and, generally, in statistics (cf. Completeness (statistics) in Wikipedia). We refer the reviewer to the works [4] and [5] for further pointers on sufficient conditions for completeness.\n\n> How are the hyper parameters chosen? B.3 almost sounds like it’s been chosen given test set performances.\n\nPlease refer to the answer above. In short, we use the MMD between validation split and generated data to select the hyperparameters (observational distribution matching). All metrics were reported on a held-out test set after hyperparameters selection.\n\n> How come you can achieve almost 0 ATE error but significantly higher CF error if the identifiability of ATEs mean identifiability of CFs\n\nWe assume that the reviewer refers to Figure 7. Here, the difference proceeds from the fact that the metrics are different, and, therefore, not comparable. _ATEerror_’ compares the means of the true and estimated distribution, while _CFerror_ measures the error sample by sample and then averages them. These differences are also observed in the oracle, highlighting why we should compare the performance of the models with the oracle.\n\n> Why are you replacing the data generating processes rather than simply taken real world causal datasets and drop confounders to estimate the correctness under hidden confounding?\n\nWe did cover this question in the App. B.4, but let us expand further. As mentioned above, we could not find suitable datasets to use and, regarding Sach’s and Ecoli datasets, we found that: i) there were several proposed causal graphs, so the true causal graphs could potentially contain cycles; and ii) we did not have access to counterfactuals to assess DeCaFlow. Moreover, we found that the Ecoli dataset has 9 observational samples. Therefore, we followed the approach by Chao et. al. [6] and resorted to semi-synthetic data. \n\nBesides, the interventional data in Sachs’ dataset correspond to soft interventions, while DeCaFlow performs hard ones, making it unclear how to compare them. We find it interesting for future work to extend causal generative models to allow for soft interventions, but out of the scope of this paper and thus deferred to future work. More generally, a real-world evaluation of causal inference approaches remains an open challenge (especially in the context of counterfactuals [7, 8]) in the literature [9,10], which like us relies on (semi-)synthetic datasets. Thus, it is an important but challenging venue for future work. . However, note once again that the experiment in section 5.3 is based on real-world data.\n\nWe hope to have clarified all the concerns raised during the review, and remain at the disposal of the reviewer if further questions come up during the rebuttal period.\n\n> References\n\n[1] Gamella, J.L., Peters, J. & Bühlmann, P. Causal chambers as a real-world physical testbed for AI methodology. Nat Mach Intell.\n\n[2] W Miao, Z Geng, and Eric JT Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika.\n\n[3] Wang, Y., & Blei, D. (2021). A proxy variable view of shared confounding. In International Conference on Machine Learning. PMLR.\n\n [4] D’Haultfoeuille, X. (2011). On the completeness condition in nonparametric instrumental problems. Econometric Theory.\n\n[5] Hu, Y., & Shiu, J. L. (2018). Nonparametric identification using instrumental variables: sufficient conditions for completeness. Econometric Theory.\n\n[6] P Chao, P Blöbaum, and S Kasiviswanathan. Interventional and coun-\nterfactual inference with diffusion models. ArXiv preprint, abs/2302.00860, 2023. \n\n[7] Holland, P. W. Statistics and causal inference. Journal of the American Statistical Association.\n\n[8] Bareinboim, E., Correa, J. D., Ibeling, D., & Icard, T. , 2022. On Pearl’s hierarchy and the foundations of causal inference. In Probabilistic and causal inference: the works of judea pearl.\n\n[9] Hill, J. L. (2011). Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphical Statistics.\n\n[10] Curth, A., & Van Der Schaar, M. (2023). In search of insights, not magic bullets: Towards demystification of the model selection dilemma in heterogeneous treatment effect estimation. In International conference on machine learning. PMLR."}]

Author rebuttal:

Meta-review: The paper introduces DeCaFlow, extending causal normalizing flows to hidden-confounder settings via proxy variables and proximal identifiability, and provides a practical encoder/decoder flow architecture that answers many causal queries from a single trained model. Reviewers acknowledged the importance of the problem, clear theoretical contributions (identifiability conditions and a query-check algorithm), efficiency relative to query-specific estimators, and competitive results on semi-synthetic benchmarks and a real application. The discussion improved the paper with clearer intuitions (proxies, do-calculus, CNF implementation), an explicit limitations section, and a more detailed treatment of assumptions and robustness to graph misspecification. The authors should (i) consolidate the new explanations and limitations, including sensitivity to noisy graphs and proxy quality; (ii) add practical guidance on model selection (e.g., MMD-based validation) and priors; and (iii) outline pathways for broader real-world evaluation (soft interventions, additional datasets) and for integrating with causal discovery.
Decision: Accept (spotlight)