Causal Effect Estimation with Mixed Latent Confounders and Post-treatment Variables
Authors: Yaochen Zhu
Jing Ma
Liang Wu
Qi Guo
Liangjie Hong
Jundong Li
Venue: ICLR 2025 Poster (2025)
Keywords: causal effect estimation
latent post-treatment bias
identifiable VAE
Primary area: causal reasoning

Abstract: Causal inference from observational data has attracted considerable attention among researchers. One main obstacle is the handling of confounders. As direct measurement of confounders may not be feasible, recent methods seek to address the confounding bias via proxy variables, i.e., covariates postulated to be conducive to the inference of latent confounders. However, the selected proxies may scramble both confounders and post-treatment variables in practice, which risks biasing the estimation by controlling for variables affected by the treatment. In this paper, we systematically investigate the bias due to latent post-treatment variables, i.e., latent post-treatment bias, in causal effect estimation. Specifically, we first derive the bias when selected proxies scramble both latent confounders and post-treatment variables, which we demonstrate can be arbitrarily bad. We then propose a Confounder-identifiable VAE (CiVAE) to address the bias. Based on a mild assumption that the prior of latent variables that generate the proxy belongs to a general exponential family with at least one invertible sufficient statistic in the factorized part, CiVAE individually identifies latent confounders and latent post-treatment variables up to bijective transformations. We then prove that with individual identification, the intractable disentanglement problem of latent confounders and post-treatment variables can be transformed into a tractable independence test problem despite arbitrary dependence may exist among them. Finally, we prove that the true causal effects can be unbiasedly estimated with transformed confounders inferred by CiVAE. Experiments on both simulated and real-world datasets demonstrate significantly improved robustness of CiVAE.

Published as a conference paper at ICLR 2025
CAUSAL EFFECT ESTIMATION WITH MIXED LATENT CONFOUNDERS AND TREATMENT VARIABLES
Yaochen Zhu', Jing Ma², Liang Wu², Qi Guo Liangjie Hong", Jundongg Li' Uniivessity of Virginia, "Case Western Reserve University, LLinkedIn Inc.
{uqp4qh, bundong)evirginia.. jing. manecaaaeecaee.. edu [liawu, qguo, liahong] (etegg/lliiiiiiiie inkedin.com
ABSTRACT
Causal inference from observational data has attracted considerable attention among researchers. One main obstacle is the handling of confounders. As direct mea- surement of confounders may not be feasible, recent methods seek address the confounding bias via proxy variables, i.e., covariates postulated to be con- ducive the inference of latent confounders. However, the selected proxies may scramble both confounders and post-treatment variables in practice, which risks biasing the estimation by controlling for variables affected by the treatment. In this paper, we systematically investigate the bias due to latent post-treatment vari- ables, i.e., latent post-treatment bias, in causal effect estimation. Specifically, we first derive the bias when selected proxies scramble both latent confounders and post-treatment variables, which we demonstrate can be arbitrarily bad. We then propose Confounder-identifanl on a mild assumption that the prior of latent variables that generate the proxy belongs to general exponential family with at least one invertible sufficient statis- tic in the factorized part, CIVAE individually identifies latent confounders and latent post-treatment variables up to bijective transformations. We then prove that with individual identification, the intractable disentanglement problem of latent confounders and post-treatment variables can be transformed into tractable inde- pendence test problem despite arbitrary dependence may exist among them. Finally, we prove that the true causal effects can be unbiasedly estimated with transformed confounders inferred by CiVAE. Experiments on both simulated and real-world datasets demonstrate significantly improved robustness of CiVAE.
VAE (CiVAE) to address the bias. Based
INTRODUCTION
Causal inference, which aims infer cause-and-effect relations from data, has gained increasing prominence in various fields, such as social science, economics, and public health (Glass et al.
2013 Johansson al., 2016 Prosperi et al. 2020). Traditional methods rely the gold standard of randomized control trials (RCT) to draw causal conclusions via experimentation (Cook et al., 2002). Recently, more attention has been dedicated to causal inference from observational data, where treatments, outcomes, and unit covariates are passively observed, and researchers have no control over the treatment assignment mechanism (Shalit al. 2017 Shi et al. 2019 Wager & Athey 2018).
One main obstacle to inferring causal relations from observational data is confounding bias. which occurs when we fail account for the systematic difference between treatment and trettretttent groups due to variables that causally influence treatment and outcome, i.c., unobserved confounders (Zhu et al., 2024 Jager et al. 2008). If confounders can be measured, we can address the bias by controlling them via covariate adjustment (Pocock et al., (002) or propensity score re- weighting (Li) cet al.) 2018). However, confounders are not always measurable (Kuroki & Pearl 2014). Therefore, recent efforts have been devoted to adjusting for unobserved confounders based on their proxies, which are observed covariates postulated to be causally related with the unobserved confounders (Miao et al. 2018 Yao et al.] 2018 Madras et al. 2019). One exemplar work is the causal effect variational auto-encoder (CEVAE) (Louizos et al. (017). which has demonstrated that confounding bias can be mitigated by controlling latent variables inferred from the proxies of confounders.

---

Published as a conference paper at ICLR 2025
Insrrmncsnal Fariabies
X
comroaa
Adjurter
Confonddrr
prekatte Jaraaly
par f
Confondder
Post-crentineet
Harisbies
SCM Assumed CEVAE
SCM Assumed TEDVAE
(c) SCMAssumed by the Proposed CIVAE
Figure 2: Comparison between the causal models assumed by CEVAE, TEDVAE, and CiVAE. Bi-directional dashed arrow means we allow arbitrary correlation between the two variables
Although proxy-based methods have achieved substantial progress in recent years, they may risk controlling latent post-treatment variables scrambled in the proxies, where latent post-treatment
bias can be introduced in the estimation. The negative effects of controlling observed post-treatment
variables have been investigated in prior research (Acharya et al. 2016 Elwert & Winship 2014 King & Zeng 2006). For example, MMntggomery al. (2018) found that more than 50% of the papers published in top journals of politics inadvertently controlled post-trealment variables in the experimental setting, even though the researchers had complete control over which covariates to control for. On this basis, we postulate that the post-treatment bias could be even worse for proxy-based methods in the setting of observational study where available covariates are passively recorded. In addition, the post-treatment variables can be latent and scrambled into the observed covariates together with the latent confounders, which makes them difficult to disentangle.
Consider real-world example from LinkedIn. We found that changing a job from onsite on- line mode causes applicants to make different de- cisions, and we want to estimate the causal effects of switching job from onsite to online mode on the decisions of the applicants (Here, the units under study are the jobs instead of applicants). To study the case, we collected two groups of online (treated) and onsite (control) jobs, where (the average) application decisions of job seekers are treated the outcome. Clearly, job type is confounder as certain jobs (e.g., IT) are more likely to permit online work and are more difficult to apply. When job type is not provided in the posting, the required skills of the job (which is mandatory) can be used as the proxy of the confounder "job type". However, a caveat is that switching to an online work mode may also alter the required skills of a job, thereby affecting the qualification and, therefore, the decision of the applicants. Consequently, directly using the "required skills" as the proxy of the confounder "job type" for adjustment could unintentionally control latent mediators (changed skills), which introduces latent post-treatment bias in the causal effect estimation.
mode-related skills (egs, onlinr conttaation
M
C
job type (Ag. 27)
work mode
application decisions
X
mode-inv invariant skills (eg. joston pegrassingg
required skills
Figure 1:
Example of latent post-treatment bias.
Similar examples are ubiquitous observational studies in other fields where data are collected post hoc and pre-treatment and post-treatment components entangle in the observed covariates. For example, for medical treatment effect estimation, a common circumstance is that only post-treatment test results of the patients are available (Liu et al.) 2012). Some results, such as the changes in blood pressure, could be influenced by the treatment, while others (depending on the specific treatment) could remain invariant the treatment but are indicative of patient's 's pre-treatment health (i.c., an important confounder for treatment effect estimation). The problem can be exacerbated in politics and economy (Klar al. 2020). For example, when estimating the causal effect of historical policies, usually only vearly-aggregated data on the covariates, such as GDP, employment rates, or social welfare are available. These variables are often a mixture of pre-policy conditions (e.g.. GDP from the beginning of the year to the date the policy took effect) reflecting the latent pre-policy state of the area (i.e., confounders) and post-policy outcomes (e.g., GDP after the policy), where latent post-treatment bias can be introduced if these covariates are treated confounder proxies (Homola et al. 2024).
Addressing the latent post-treatment bias faces multi-faceted challenges. First, there lacks a theoretical formulation of the bias when proxies scramble latent post-treatment variables for existing proxy-based methods. In addition, it is difficult to distinguish confounders and post- trst-treatment variables in the latent space due their similar effects observed covariates. Existing covariate disentanglement- based methods, e.g., TEDVAE (Zhang et al.) 202T), focus on an easier task of disentangling latent confounders with adjusters and instrumental variables, which can be achieved by
2

---

Published as a conference paper at ICLR 2025
leveraging their different predictive abilities w.r.t. the treatment and outcome. However, since both latent confounders and post- traatteettmntt variables correlate with the treatment and the outcome, they cannot be disentangled by these methods. Finally, even if latent confounders can be distinguished from post-treatment variables, since most existing latent variable models have no identifiability guarantee (Khemakhem et al. 2020), it is unclear whether controlling the inferred latent variables, which may be arbitrary transformations of the true confounders, can provide unbiased estimations.
To address the challenges, we first analyze existing proxy-based methods when selected proxies scramble both latent confounders and latent post-treatment variables, where we show the estimation can be arbitrarily biased. We then propose a confounder-identifalle latent test-treatment bias. Specifically, we prove that based on a mild assumption that the prior of latent variables that generate the observed proxy belongs to a general exponential family with at least one invertible sufficient statistic in the factorized part, latent confounders and latent post-treatment variables can be individually identified up to bijective transformations and permutation. With such an identifiability guarantee, based the causal relations among confounders, post-treatment variables, and treatment, we further demonstrate that the inferred confounders could be properly disentangles from the latent post-treatment variables with pair-wise conditional independence tests, even if they may have arbitrary dependence with each other. Finally, we prove that the true causal effects can be unbiasedly estimated based on the confounders inferred by CiVAE. Experiments on both simulated and real-world datasets demonstrate the robustness of CiVAE to latent post-treatment bias.
VAE (CiVAE) address the
2 PROBLEM FORMULATION
In this paper, assume the causal model in Fig. (c). We use a binary random variable T to denote the treatment, a random vector X E Rxx to denote the observed covariates (i.e., the proxy), and a random scalar Y E IR to denote the outcome. Furthermore, the observed covariates X assumed to be generated from K independent latent confounders C 2 [C1, C2.., Ckel causally influencing both T and Y, and KM latent post- pest-treatment variables M influence of the treatment (where the relation between M and Y can be arbitrary). We use the random vector Z A [ E RR K to denote all relevant latent factors. We assume that beyond C, there are no latent confounders that confound the latent variables Z. Our aim is to estimate the average causal effects of treatment T on outcome Y with the auxiliary confounder information in X, where the estimation should be devoid of both confounding bias and post-treatment bias.
MKxxl under the causal
THEORETICAL ANALYSIS OF LATENT TREATMET BIAS
3.1 PRELIMINARIES AND ASSUMPTIONS
To achieve such purpose, we first define the (conditional) average treatment effects (C/ATE) when covariates X scramble both latent confounders C' and (mixed-in) post-treatment variables M. We then define the post-treatment bias when covariates X directly used as the proxy of confounders. To facilitate the analysis, we make the following assumption regarding the causal generative process.
Assumption 1. We assume X = f(C M) + e, where f is a deterministic function that combines latent confounders C and some mixed-in latent post-treatment variables M into the observations X. and e is a random noise. In addition, we assume that the function f is injective beyond injectivity, f can be arbitrarily nonlinear. We use ft X [C|[M] to denote its left inverse. We use fb. X > C and fMz X > M to denote the mapping from X to C, M, respectively.
Similar assumptions between X and C' are commonly made either explicitly or implicitly in most existing proxy-of-ooo croxy-of-confounde confounder-
methods. For example, if both X and C are categorical, Kuroki & Pearl (2014) assume that X has at least the same number of categories as C and the conditional distribution p( X CC) is non-degenerate, whereas the effect restoration algorithm (Miao et al. 2018) assumes that Kx Kc and the matrix [p(X, .ec to be full-rank. Although CEVAE (Louizos et al., 2017) makes no explicit injectivity assumption between C' and X, it requires that the joint distribution P(C,,,T,3 can be fully recovered from the observations (X,T,)).
Note that when Kx > Kz, non-injectives have measure zero in functional space (f : (C, M) X).
3

---

Published as a conference paper at ICLR 2025
Anandkumar al. (2014) show that some of the possible identification criteria for the recovery of plC, I,,,,,,YY include (i) X having multiple independent views of C' (Edwards et al. 2015), or (ii) C being categorical and X being mixture of Gaussian with mixture components determined by C' (that is, X is generated by bijective mapping of C' to the mean of the corresponding mixture component with added Gaussian noise).
In the following part, we will temporarily omit the noise to gain a better intuition of latent post- treatment bias (but all the conclusions will still hold in the posterior sense). In Section 4 assume noise exists and demonstrate our method can still properly identify the latent confounders.
3.2 CAUSAL ESTIMAND AND THE TRUE ATE
Based on Assumption II we are ready to define the estimand of average treatment effect (ATE) through controlling the covariates X', as well as the true (conditional) average treatment effects. Definition 1. (DCEV & DEV). We define the Difference in Conditional Expected Values (DCEV) as:
DCEV((') - 1, 20) E|Y|T X'
X. =x'];
which is the difference of the expected value of Y for units with variable X' = x' in the treatment group and the con-treatment group. Based on DC ICCEV(*') (z'), we define the Difference in Expected Value (DEV) as DEV(X') = Ep (DCEV as the expectation of DCEV w.r.t p(X').
DEV(X') denotes the estimand of ATE when X' is the covariates that we choose to control (i.e., calculate the expected difference in each stratum of X' ='). If X' = 0, DEV(0) represents the naive estimator that directly calculates the expected difference of the outcome Y between the treatment group and the non-treatment group. With the causal estimand D DEV(X' defined, we then derive the causal effects with the covariates X' when it serambles both latent confounders and post-treatment variables according to the generative process described in Assumption |
Definition 2.
Under Assumption 7 we define the Conditional Average Treatment Effect (CATE) for individuals with observed covariates X- x by controlling only the confounder part in as:
CATE(x)) ET C = [x)) [ O,C fb
(2)
with the Average Treatment Effect (ATE) of treatment T defined as:
ATE = E[Y]do(T = 1)] E do = = BB -,
(3)
Please note that we only consider the latent confounder component of the observed features X in the definition of CATE due to the indeterminate causal relation between the post-treatment variables M and the outcome Y. However, if the specific relationship between M and Y can be further established by the researcher (e.g., all elements of M are latent mediators), more precise forms of CATE can be derived with path-specific counterfactual analysis (Cheng et al.) 2022 Imai et al. 2010).
3.3
LATENT POST-TREATMENT BIAS
With DEV (X') (the ATE estimator that controls for the covariates X'), CATE, and ATE defined in Section 3.2 in this section, we analyze the latent post-treatment bias of existing proxy-of-conee -f-cooune--f based causal inference methods, such as CEVAE, that control for latent variables inferred from the covariates X to estimate the ATE of T' on Y. when X scrambles both latent confounders and post-treatment variables as Assumption |
In our analysis, Lemma 3.1 will be frequently used.
Lemma 3.1. For an injective function E = x'1 = E
g(x')] holds.
The proof when is differentiable a.e. can be referred to in Appendix A.
Although the latent variable models used in existing methods (such as VAE with factorized Gaussian prior) lack identifiability guarantee (i.e., the recovery of the exact latent variables), we assume that these models can recover the true latent space Z C M up to invertible transformations f. where the inference process can be represented as 2 = f(x) = f ft)).. With such an assumption, we have the following theorem regarding the latent post-treatment bias when X mixes post-treatment variables.
4

---

Published as a conference paper at ICLR 2025
Theorem 3.2. If the observed covariates X are generated from latent confounders C and latent post- cast-treatment variables M according to Assumption 1 the latent post- post-treatment bias of a proxy- based causal inference algorithm that controls latent variables 2 inferre X via f f RKx +B K estimate the ATE can be formulated follows:
Bias ATE DEV (J()) - ATE ET 1, f(X)] = [(X]]]
ATT - BIE ( o
(4)
-EEE -E CJ] -EIE[Y,,, - C,
which can be arbitrarily bad. Therefore, the estimator of existing proxyoff-confounde--bee meth ods, ie., DEV (f))), is an arbitrarily biased estimator of the ATE, when the selected proxy of confounders X accidentally mixes in latent post-treann post-treatment variables M.
The final step of Eq. 44) can be proved since injective and f bijective, the composite fo f: M] + 2 bijective, so we can use Lemma 3.1 to remove ʃ o ft f in the condition.
3.4 EXAMPLES IN THE LINEAR CASE
Generally, the latent post-treatment bias defined in Eq. cannot be simplified, because (i) the causal relationship between M and Y are indeterminate, and (ii) the causal influence of C, M, and 7 on Y can be arbitrary. However, for linear structural causal models with determined causal relationships between M and Y (e.g., M are mediators, which are post-treatment variables that have causal influences on the outcomes), stronger conclusions can be drawn as follows:
Corollary 3.3. (Latent Mediator). For the linear Structural Causal Model (SCM) defined as:
(1) + 1(T + Σ B. C, > a), (it) Mj + aM + +73 TT
(5)
(iii) X ax + MC (iv) Y t ax + r T Σ9,
0 M, + Σ Kg Ci,
where the mixture function f = A k is a full colunn-rank matrix, the CATE, ATE, and the bias of mroxy-of-confoundee based causal inference model that controls the latent variables Ż inferred via 2 = f(X) = B can be formulated as follows:
ATE CATE + Σ 23 0j, and DEV E(DCEV (C)
Bias 2) = ATE - = En. 03.
where B E ERRxxx(c is another full column-rank matrix. Since Σng 0j is arbitrary, the estimator (EV EBDCEV x)] is arbitrarily biased for ATE estimation.
The proof of Eq. (6) is provided in Appendix A.2 In addition, show that post-treatment variables M do NOT necessarily need to have direct causal effects on the outcome Y to incur arbitrary bias in ATE estimation. Appendix [A.3] we provide another example (i.e.. Latent Correlator) in the linear case where M is correlated with through unobserved confounders U in Corollary A.I
4 METHODOLOGY
In this section, we introduce the proposed cenfouunderiidenifaall Variational Auto-Encoder (CiVAE) in detail. Specifically, we first prove that if the prior distribution of the true latent variables Z = M] satisfies certain weak assumptions, CiVAE can individually identify up to element-wise bijective transformations and permutations. Then, utilizing the invariant causal relations between C, M, and T. we transform the challenging confounder disentanglement problem into a tractable wair conditional independence test problem, which can be effectively solved with kernel-based methods, despite the potential arbitrary interactions among C and M. Finally, we prove that controlling confounders inferred by CiVAR provides unbiased estimate of ATE.
5

---

Published as a conference paper at ICLR 2025
4.1 GENERATIVE PROCESS
We first introduce the generative process of X and Z in CiVAE that leads to the identification. The fundamental work on identifiability of deep variational inference, i.e., the identifiable VAE (iVAE) (Khemakhem et al.) 2020), makes strict assumption that the prior of true latent variables Z (i.c., M is conditionally factorized given observed covariates other than X. However, since C form fork structures with Y, and M could have arbitrary relations with Y (see Fig. ((c)). C, Cj. Mr, and M, are not independent given Y. Recently, Non-Factorized iVAE (NF-iVAE) (Lu et al. 2021) allows arbitrary dependence among the true latent variables Z in the conditional priors, where Z can be identified up to arbitrary non-linear transformations. However, the transformation is not necessarily invertible, which is risky multiple values of confounders may collapse, leading to biased ATE estimation when averaging DCEV calculated in each stratum of the inferred confounders.
In contrast, CiVAE guarantees the elementwise bijective identifiability of Z up to permutations. Specifically, we put a general exponential family with at least one invertible sufficient statistic in the factorized part the prior when conditioning on treatment T and outcome Y as follows:
Assumption 2. Let Z [C be the random vector for latent variables that causally gen- erate the observed covariates X according to Assumption | We assume that the conditional prior of Z given the outcome Y and the treatment T belongs to general exponential family with parameter vector A(Y,T) and sufficient statistics S = " (Z) Specif- ically, S(Z) is composed of (i) the sufficient statistics of a factorized exponential family, i.e. where all components Sy,(Z,) (Z,) have dimension larger
= (z) than or equal and each Sys y.s has least one invertible dimension, and (ii) S (Z), where Snd is a neural network with Rel.U activation. The density of the conditional prior can be formulated as:
=S)) ..,.)"...........
(Zxz)" (
PS. P (Z)( Q(Z)/C(YTT) exp(S(Z) ()Y,T))
where Q(Z) is the base measure, and C(Y, T) is the normalizing constant independent of Z.
Intuition. We justify that assumption 2 is practical as follows. (i) Neural networks with ReLU activation have universal approximation ability of distributions (Lu & Lu 2020). Therefore, Eq. (7) can model arbitrary dependence between true latent confounders C' and treatteettent variables M conditional on T and Y. (ii) Although CiVAE makes an extra assumption that Vi, at least one dimension of Syz is invertible, this can be easily satisfied as most commonly used exponential family distributions, such as Gaussian, Bernoulli, etc., have at least one invertible sufficient statistic.
The reason why we use ReLU as the activation is that, the identifiability of iVAE relies on the condition that the sufficient statistics have zero second-order cross-derivative. The factorized part, i.e. Sy. satisfies it trivially as all cross-derivatives of Sy are zero. In addition, since the ReLU neural networks are linear a.e., all second-order derivatives of Sn are zero. Therefore, identifiability holds after adding Say in the prior that allows the capturing of arbitrary dependence among Z.
4.2 OPTIMIZATION OBJECTIVE
Combining Assumptions I and 2 the generative process assumed by CiVAE can be formulated as:
(o Z Y, Y,T) (X Z) Ps.x (Z Y,T)), (ii) (X Z) Pe f(2)).
(8)
where 0 (f θ are the parameters of the generative distribution. Y.T are omitted from Po as Z form the Markov Blanket of X in Fig. (c). Since the generative process of CiVAE is parameterized by deep neural networks, the posterior distribution of Z, i.e., Po Z X, XYY,)). is intractable. Therefore, we resort to variational inference (Blei et al., 2017). where we introduce an approximate posterior (Z X, Y")") parameterized by a deep neural network with trainable parameters Φ, and in l (2)-) finds the one closest to pe ( Z|-) measured by KL divergence. The minimization of KL is equivalent to the maximization of the evidence lower bound:
C(O, Φ) := Eg lo (X
Z) + logps.> (Z Y,T) - log go Z 1.))
KL. of posterior with prior
Since the normalization constant C in Eq. 7 is generally intractable, it is infeasible to directly learn S, A by optimizing Eq.. (9). Therefore, substitute the KI. term in Eq. (9) with the widely-used

---

Published as a conference paper at ICLR 2025
score matching (Hyvärinen & Dayan
2005 to learn unnormalized densities instead follows:
C(S, Φ)= = (z)
V log ( 1-) - Vz log P Y,T)I
K
0 Ps (Z
Y.T)
( (.T))
21 I
(10)
= E (21-)
+ 2 azz
+ const.
3z3
4.3 IDENTIFIABILITY OF CIVAE
With the generative process and optimization objective of CiVAE discussed in previous sub-sections, we introduce the final assumption of CiVAE on the regularity of the data, which, combined with Assumptions I and 2 leads the main Theorem that states the identifiability of CiVAE.
Assumption 3. Assume the following: (i) The set (X E ( 0} has measure zero, where the characteristic function of the density py in Eq. (8). (ii) The sufficient statistics, SJ in Sj are all twice differentiable. (iii) The mixture function f in Eq. (8) has all second-order cross derivatives. (iv) There exist & + distinct points (Y,T)o "...,.YY (Y.T)s s.t. the matrix L [A((Y)T)) (T)) A((Y,T)x)
A((Y,))o) size l x k invertible, where k. Dim(S).
The identifiability theorem of CiVAE can be formulated as follows.
Theorem 4.1. If Assumptions / and hold, hold, and if E θ po = the true latent variables Z are identifiale up to permutation and element-wise bijective transformation. Furthermore, in the case of variational inference, if we denote the true parameter that generates the contains the posterior Pa Z X, Y, T), and X, X, Y. T) 0, (ii) optimize Eq. (4) w.r.t. both O, Φ. then in the limit of infinite data, true 96 parameters 6* can be learned up to a permutation and bijective transformation of Z.
data as 0", (i) the distribution family gsl Z Y,
Intuition. We justify that Assumption 3 weak and practical follows: ( is commonly-used
denote that the data generative distribution should not be degenerative. 3 (i.. 3 can be trivially
satisfied by neural networks. For (iv), Section B.2.3 of (Khemakhem et al. 2020) shows that
the functional for the factorized part of the exponential family parameters ); (Y, 7') are independent
(which is very weak), (iv) can be satisfied with arbitrary ls + different (Y; T) points.
The proof of Theorem 4.1 extends NF-iVAE (Lu et al. 2021 by incorporating the new assump- tion introduced in CiVAE (i.e., each Sy,, has at least one invertible dimension) to ensure that the transformation of each Z, is bijective. The detailed proof is provided in Appendix A.4
4.4 IDENTIFICATION OF LATENT CONFOUNDERS. No INTERACTION CASE
Intuitively, theorem 4.1 ensures the latent variables 2 inferred by CiVAE cannot (i) mix confounders and post-treatment variablss in each dimension, or (ii) collapse different values of the latent con- founders into the same value. To further determine the dimensions of confounders and post-treatment variables in Z, we rely on the invariant causal relations between latent variables Ž and the treatment T. We first discuss the simplified cases with no latent interactions to gain intuitions:
Case I. Intra Coira--onfounders. Latent confounders C., Cj and the treatment T form the V-structure C, + T & Cj. Therefore, C₁ and Cj are marginally independent, whereas they become dependent when conditioning the assigned treatment T.
Case 2. Intra-Post Treatment Variables. Latent post-treatment variables M,, Mj and the treatment become independent after conditioning on the assigned treatment T.
T form a Fork-structure M, + - My. where M,, M, are marginally dependent, but they
Case 3. Cross- Coos-Conf and Post-Treatmnn Treatment Variables. Latent confounder C,, latent post- treatment variable M, and the treatment T forms Chain structure C, + T + M₁, where C,, M, are marginally dependent, and they become independent after conditioning on T.
Since only the case of intra-confounddes does the dependence between two latent variables 2, and 2, increase after conditioning on the treatment T. if more than one latent confounder exists, which is highly probable when covariates X are high-dimensional, can conduct independence
7

---

Published as a conference paper at ICLR 2025
test Ind (2,, 2,) and CInd TT) for all pairs of inferred latent variables in Z and select the pairs where the p-value of CInd is larger than that Ind as latent confounders. Here, we note that the kernel-based (conditional) independence test incurs N2 x K2 complexity in the training phase (Zhang al. 2012). However, once the dimensions of the confounders in 2 are determined, CiVAE has the same complexity as CEVAE for the estimation of CATE and ATE in the test phase.
4.5 GENERALIZATION TO INTERACTED LATENT VARIABLES
We further generalize CiVAE to address interactions among latent variables [C][ Since Assumption allows arbitrary dependence in the latent space, CiVAE still individually identify M] in Z by optimizing Eq. (00. However, in scenarios where latent interactions exist, cases 1-3 Section 4.4 may not hold, which precludes us from further disentangling from 2. For example. if C' confounds M, case 2 may not hold, M; and M, are still dependent after conditioning on T.
The generalization still leverages the V-structure between latent confounders C and treatment T, i.e.
C. + + Cj and involves more advanced pairwise independence test for every Z, 2, E Z:
Case 1* Confo-Confouddrs. If there exist Zc 2/12,, 2, 2,1cc
Case 2', Intra-Post Treat. Variables. If there exist 2M C 2/(2, s.t. 2: 1 2 T). Case 3* Pross-Variables. If there exist ZC, C 2/(2,, s.t. 2, + 2,112c,
The proof that, for Case 2" and Case 3", since M., M, form fork structure with 7 and M,, Cj form chain structure with T, we cannot find Zo that leads to independence without conditioning on T (as with Case 1* If no latent interactions exist, 2c. 2M. Ze.M = 0. which degenerates to the three cases discussed in Section 4.4 If interaction exists, start with Zc. 2M ZC,M 0 and gradually include more variables in into the condition sets. We provide examples of ìc. 2M, Zc.m for different types latent interactions Appendix C Similarly, once the dimensions of the confounders Ć' in Ż are determined, retrieving C' from Z takes ((1) time in the test phase.
4.6 ATE ESTIMATOR WITH TRANSFORMED CONFOUNDERS
Finally, we demonstrate that controlling the transformed confounders Ò inferred by CiVAE provides an unbiased estimation of ATE. Specifically, we have the final Theorem show the unbiasedness.
Theorem 4.2. Controlling bijective of confounders is equivalent to original confounders in ATE estimation, i.e., (EV(C) = DEV (g(C)) = ATE, if the transformation function is bijective.
The proof of Theorem 4.2 for discrete C is trivial (where Ć = gCC represents simple relabeling of the stratum that we calculate the DCEV and take the expectation). With Theorem 4.2 we can control the identified latent confounders true confounders, providing an unbiased estimate of ATE,
EMPIRICAL STUDY
5.1 DATASETS
Simulated Datasets. We first establish two simulated datasets, i.c., LatentMediator and LatentCorrelast
C
applicont Rasures
that consider two types of post-treatment variables, i.e., (i) medi- ators and (ii) correlators, i.e., variables that are correlated with the outcome Y via latent con- founders U. The generative processes of the two datasets are provided in Corollary 3.3 and Corollary A.I In experiments, C' are gen- erated from Gaussian distribution as C Goussian (0, Ixc). For LatentMediator. 7 is set 1, 1], and is set 2, which results in an AT E. -1. For the 11,,-,,- -1], 0 is set 11
work mode"
spplication decivsions
C-l., mode invatian skills
required
Figure 3: Generative process of the real-world datasets.
8

---

Published as a conference paper at ICLR 2025
(a) Case 1: Intra- Conr-Conffuuder
(b) Case 2: Intra-Mediator
(c) Case 3: Innfounder-Mediatee
Figure 4: Visualization of p-value independence test before and after conditioning on treatment T..
LatentCorrelatow dataset, we set the same and O as the LatentMediatoz dataset, where parameters and are set to I, which results in ATE of 1.
Real-world Datasets. In addition, we build real-world datasets from Linkedln to estimate the ATE of switching a job from onsite to online work mode on the decision of applicants. The decision can be measured by the
rate of applicants to the job posting (i.e., Y2 in Fig. B}. which is too coarse in granularity. More practically, we consider
average age and the variance of gender of the applicants as two outcomes of interest, which are determined by the direct outcome, i.e decision to apply Ya. and applicants' age and gender Xa. where X₂ are pre- trtettteeeetttt variables. The causal graph is illustrated in Fig. 3 which extends the example in Fig. [ Covariates X E {0, 1 include the required skills of the job. Specifically, we establish a cohort of 3 3,228 jobs from the Bay Area in the US, where preliminary study shows that DEV (8) years (i.e., online job applicants two years younger than onsite job applicants
the collected data), and DEV(0) = -.0015 (i.e.,
online jobs exhibit 0.015 more gender variance than onsite jobs in the collected data). To simulate C and M, we first learn a generative model as follows:
Z ~ Gaussian (0, Ixz), x X ~ Multi(NN,(CC (Z)), Gaussian (w @ Z,1),
(1I)
where M Multi represents multinomial distribution, N is neural network with softmax activation, Z, w E K2, Kz = 8, and @ represents the element-wise product operator, respectively. We then treat the first Kc 5 dimensions of Z the latent confounders C' and the remaining KM Kz - Kc dimensions as the latent mediators M. After learning N N and w according to Eq. we draw latent confounders C' E Gaussian (0, I), latent mediators M -7, and set the
outcome Y = w 3 M + T,, where the true ATE can be calculated as sum O w- KMr)) +
5.1.1 DISENTANGLE CONFOUNDERS AND POST-TREATMENT VARIABLES
We first show the p-value of the kernel-based pairwise independence test of the true latent variables before and after conditioning on the assigned treatment T. From Fig. we can find that the distinction of the intra-confounder case from the other two cases discussed in Subsection 4.4 is significant. Here, we should note this relies on the assumption that latent variables are independent. Experiments generalized CiVAE address interactions among latent variables are in Section C
5.2 BASELINES
The baselines include be categorized into three classes. (i) Unawareness, where informa- tion X is used for ATE estimation. We implement the naive LRO estimator, which regresses Y on T and the coefficient to estimate the ATE (Imbens & Rubin 2015) and is equivalent to DEV(0), (ii) Control-X, which directly controls the covariates X. In this class, LRI regresses Y on T and X. whereas TarNet uses a two-branch neural network to estimate DEV(XX (iii) Control-Z, Z. which controls latent variables Z learned from the covariates X. Methods from this class include CEVAE (Louizos et al. 2017) and covariate disentanglement methods, such as DR-CER (Hassanpour
Greiner 2020 TEDVAE (Zhang et al. 2021). NICE (Shi et al. 2021). and AFS (Wang et al. 2023)
5.2.1 RESULTS AND ANALYSIS
find that for all four datasets, CEVAE is worse than the naive LRO estimator. In addition, for the LatentMedi or and LinkedIn (Age) datasets, all methods except CiVAE fail to predict the negativity of the ATE. Covariates disentanglementtbassee methods, i.e., DR-CFR and TEDVAE, inherit the latent post-treatment bias of CEVAR. The reason is that, these methods
From Table
which leads 0.178 and -0.105 after standardization of the outcome.

---

Published as a conference paper at ICLR 2025
Table I: Comparison of CiVAE with baselines under latent post-treatment bias on various datasets.
Dataset
MatentMediator
lateettorrelltor
Linkedin n (Age)
Linked In (Gender)
Method
ATE.
Err.
ATE.
Er.
ATE.
Err.
ATE.
Err.
LRO
0.975 + 0.022
1.975
2.977 + 0.032
1.977
0.131 0.015
0.399
-0.105 0.099
-0.213
LRI
1.457 = 0.167
2.457
3,400 + 0.130
2,400
0.093 0.029
0.361
-0.175 0.014
-0.256
TarNet
1.461 +:: 0.172
2.461
3.414 1: 0.146
2.414
0.112
0.380
-0.167
-0.248
CEVAE
1.550 = 0.292
2.550
3.323 + 0.167
2.323
0.106
0.374
-0.180 0.028
-0.261
DR-CFR
1.239 i 0.324
2.239
3.185 1 0.319
2.185
0.094 0.089
0.362
-0.159 0.030
-0.240
NICE
1,868 = 0.530
2.868
1.942 + 0.524
0.942
0.149 = 0.126
0.417
-0.186 + 0.041
-0.267
TEDVAE
1,042 + 0.315
2.042
3.138 + 0.281
2.138
0.097 + 0.093
0.365
-0.143 + 0.027
-0.224
AFS
1.496 +: 0.825
2.496
3.251
1: 0.398
2.251
0.105 1: 0.102
0.373
-0.163
0.045
-0.244
CiVAB
-0,822 + 0.753
0.178
1.199
+ 0.765
0.199
-0.140
+0.137
0.128
-0.106 + 0.064
-0.187
True ATE
-1,000 0.000
0.000
1.000 +: 0.000
0.000
-0.268 +: 0.000
0.000
-0.081 + 0.000
0.000
disentangle latent confounders C' from latent instrumental variables I and latent adjusters A by utilizing their causal relations with and Y. i.e., is predictive only for T, is predictive only for Y, whereas C' is predictive for both T and Y. For example, TEDVAE includes three encoders to infer three sets of latent variables I, Á. Ö from X and adds classification losses p(T) Ć) and p p(Y|T, č, A) on the CEVAE loss. However, since both C and M correlated with both T' and Y. these methods cannot disentangle C. from M. An exception is NICE (Shi et al. 2021). which uses invariant risk minimization (IRM) (Arjovsky et al. 2019) to find all causal parents of the outcome Y as the confounders, which makes it more robust in the LatentCorrelle or case. However, since mediators M are also the causal parent of Y. the performance degrades substantially on the Latent post-treatment variables M in the proxy X, assumes that they can separated from other variables in X in the observational space, and relationship exists between the post-treatment variables and the outcome, so still has poor performance in our setting since both assumptions are violated.
ator dataset. Although AFS (Wang et al. (023) considers the existence of
5.3 SENSITIVITY ANALYSIS
In this part, we vary the number of confounders and post-treatment variables that generate proxy X in the LinkedIn (Age) and LinkedIn (Gender) datasets and compare CiVAE with the baseline TEDVAE in Fig. 5 Fig. 5 shows that the eтror consistently lower for CiVAE. In addition, the eтrоr comparatively higher when the number of confounders is low since the mis-identificatton of latent post- treatment variables confounders can have a comparatively larger influence on the ATE estimation, In addition, when the number of confounders becomes larger, the performance gap between CiVAE and TEDVAE gracefully shrinks.
CVVEE
TEDVAR
TEDVAB
Birce
Biroe
Company (Age)
(b) Company (Gender)
Figure 5: Error with different ratio of latent confounders and latent post-treatment variable in the latent space.
CONCLUSIONS
In this paper, we systematically investigate the latent post-treatment bias in causal inference from observational data. We first prove that unresolved latent post-treatment variables scrambled in the proxy confounders can arbitrarily bias the ATE estimation. To address the bias, we proposed the Confounder-identffabbe VAE (CiVAE), which, utilizing mild assumption regarding the prior of latent factors, guarantees the identifiability of latent confounders up to bijective transformations. Finally, show that controlling the latent confounders inferred by CiVAE provide unbiased estimation of the ATE. Experiments on both simulated and real-world datasets demonstrate that CiVAE has superior robustness latent post-treatment bias compared to tate-of-theear methods.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work is supported in part by the National Science Foundation (NSF) under grants IIS-2006844, 0S--144209, IIS-2223769, CNS-2154962, BCS-2228534, and CMMI-2411248 the Office of Naval Research (ONR) under grant N000142412636; and the Commonwealth Cyber Initiative (CCI) under grant VV-1Q24-011.
REFERENCES
Avidit Acharya, Matthew Blackwell, and Maya Sen. Explaining causal findings without bias: Detecting and assessing direct effects. American Political Science Review, 110(3):512-529 2016.
Animashree Anandkumar, Rong Ge, Daniel Hsu, Sham M Kakade, and Matus Telgarsky. Tensor decompositions for learning latent variable models. Journal of Machine Learning Research, 15: 2773-2832, 2014.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:190..0289) 2019.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the Americun Statistical Association, 11((518)855-8877 2017.
Lu Cheng, Ruocheng Guo, and Huan Liu. Causal mediation analysis with hidden confounders. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp- 113-122, 2022.
Thomas Cook, Donald Thomas Campbell, and William Shadish. Experimental and quasi- experimental designs for generalized causal inference. Houghton Mifilin Boston, MA, 2002.
Peng Ding and Luke W Miratrix. To adjust not adjust? sensitivity analysis of m-bias and butterfly-bias Journal of Causal Inference, 3(1):41-57, 2015.
Jessie K Edwards, Stephen R Cole, and Daniel Westreich. All your data are always missing: incorporating bias due to measurement error into the potential outcomes framework. International Journal of Epidemiology,
11D1::1552-14533 2015.
Felix Elwert and Christopher Winship. Endogenous selection bias: The problem of conditioning on collider variable. Annual review of sociology, 40:31-53, 2014.
Thomas Glass, Steven Goodman, Miguel Hernán, and Jonathan M Samet. Causal inference in public health. Annual Review of Public Health, 34:61-75, 2013.
Negar Hassanpour and Russell Greiner. Counterfactual regression with importance sampling weights. In IJCAI, pp. 5880-5887, 2019.
Negar Hassanpour and Russell Greiner. Learning disentangled representations for counterfactual regression. In International Conference on Learning Representationss 2020.
Jonathan Homola, Miguel M Percira, and Margit Tavits. Fixed effects and post-treatment bias in legacy studies. American Political Science Review, 118(1)5537-544 2024.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Kosuke Imai, Luke Keele, and Dustin Tingley. A general approach to causal mediation analysis. Psychological Methods, 15(4):309, 2010.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press, 2015.
KJ Jager, C Zoccali, A Macleod, and FW Dekker. Confounding: what is and how to deal with Kidney international, 266, 2008.
11

---

Published as a conference paper at ICLR 2025
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In International Conference on Machine Learning, pp. 3020-3029, 2016.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ICA: unifying framework. In International Conference on Artificial Intelligence and Statistics, pp. 2207-2217. PMLR, 2020.
Gary King. A hard unsolved problem? post-treatment bias in big social science questions. In Hard Problems in Social Science" Symposium, April, volume 10, 2010.
Gary King and Langche Zeng. The dangers of extreme counterfactuals. Political Analysis, 14(2): 131-159, 2006.
Samara Klar, Thomas Leeper, and Joshua Robison. Studying identities with experiments: Weighing the risk of posttreatment bias against priming effects. Journal of Experimental Political Science, 7 (1):56-60,, 2020.
Manabu Kuroki and Judea Pearl. Measurement bias and effect restoration in causal inference. Biometrika, (01(2):233-431 2014.
Fan Li, Kari Lock Morgan, and Alan M Zaslavsky. Balancing covariates via propensity score weighting. Journal of the American Statistical Association, 113(521):390-400 2018.
Wei Liu, M Alan Brookhart, Sebastian Schneeweiss, Xiaojuan Mi, and Soko Setoguchi. Implications of m bias in epidemiologic studies: simulation study. American journal of epidemiology, 176 (10):938-99) 948, 2012.
Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling. Causal effect inference with deep latent-variable models. Advances in Neural Information Processing Systems, 30, 2017.
Chaochao Lu, Yuhuai Wu, José Miguel Lormández-Lobato, and Bernhard Schölkopf. Invariant causal representation learning for -iitr-idiitritttion generalization. In International Conference on Learning Representations, 2021.
Yulong Lu and Jianfeng Lu. A universal approximation theorem of deep neural networks for expressing probability distributions. In Advances in Neural Information Processing Systems, pp. 3094-3105, 2020.
David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. Faimesss through causal awareness: Learning causal latent-variable models for biased data. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 349-358, 2019.
Wang Miao, Zhi Geng, and Eric Tchetgen Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4):987-993. 2018.
Jacob M Montgomery, Brendan Nyhan, and Michelle Torres, How conditioning on posttreatment variables can ruin your experiment and what to do about it. American Journal of Political Science, 62(3):760-775, 2018.
Judea Pearl. Conditioning on post-treatment variables. Journal of Causal Inference, 3(1):311-137, 2015.
Stuart Pocock, Susan E Assmann, Laura E Enos, and Linda E Kasten. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practiceand problems. Statistics in Medicine, 21119)9917-22930 2002.
Mattia Prosperi, Yi Guo, Matt Sperrin, James S Koopman, Jae S Min, Xing He, Shannan Rich, Mo Wang, lain E Buchan, and Jiang Bian. Causal inference and counterfactual prediction in machine learning for actionable healthcare. Nature Machine Intelligence, 2(7):369-375 2020.
Uri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: general- ization bounds and algorithms. In International Conference on Machine Learning, pp. 3076-3088 2017.
12

---

Published as a conference paper at ICLR 2025
Claudia Shi, David Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. Advances in Neural Information Processing Systems, 2019.
Claudia Shi, Victor Veitch, and David M Blei. Invariant representation learning for treatment effect estimation. In Uncertainty in Artificial Intelligence, pp. 1546-1555. PMLR, 2021.
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 1136622):128-124,, 2018
Haotian Wang, Kun Kuang, Haoang Chi, Longgi Yang. Mingyang Geng, Wanrong Huang, and Wenjing Yang. Treatment effect estimation with adjustment feature selection. In Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 2290-2301 2023.
Ziqi Xu, Debo Cheng, Jiuyong Li, Jixue Liu, Lin Liu, and Kui Yu. Causal inference with conditional front-door adjustment and identifiable variational autoencoder. The Twelfth International Conference on Learning Representations.
Liuyi Yao, Sheng Li, Yaliang Li, Mengdi Huai, Jing Gao, and Aidong Zhang. Representation learning for treatment effect estimation from observational data. In Advances in Neural Information Processing Systems, volume 31, 2018.
Kun Zhang. Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kemne-bbssed conditional independence test and application in causal discovery. arXiv preprint urXiv:1202.337 2012.
Weijia Zhang, Lin Liu, and Jiuyong Li. Treatment effect estimation with disentangled latent factors. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 2021.
Yaochen Zhu, Yinhan He, Jing Ma, Mengxuan Hu, Sheng Li, and Jundong Li. Causal inference with latent variables: Recent advances and future prospectives. In Proceedings of the 30th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 6677 -687, 2024.
13

---

Published as a conference paper at ICLR 2025
Appendix
A THEORETICAL ANALYSIS
A.1 PROOF OF LEMMA 3.1
Proof. Let Z = g(X) and z = g(2x). g is injective and differentiable a.e., and g the left-inverse,
we have:
frig(x)(vlo(z)) = = = fy.2
[ (z)l
fy.X (v,x)
frix (y)z))
(z(2)
[x(g(
fx(x)
(12)
where f. and f.j. represent the marginal and conditional density functions, respectively, and Jo (z) is the Jacobian matrix of function gو evaluated Based Eq. 12 we have:
X| = f y ( = - El z = ( g(x)]. (13)
A.2 PROOF OF COROLLARY 3.3
Proof. For X = æ. let m =
ft) =
A ---xx- axx),, where At is the left
inverse of the full column-rank matrix A in Eq. we have:
CATE(()) = E| = 1, - - [x()]
= -
Elar T E 0g- M₁ Σ k C. 1, =c
Elay + Σ 0.. + Σ R4 0,
ay + -..ETT/T-- 1,C= Σ
Σ
R; E[
- ay T - = c|+ Σ
E[M, 0, 0, - c|+ Σ Ki =
= 11 - 0) + 01 03
-,, (1g (1- 0)) + Σ ki (a-G)
= + Σ 0, ' Fl[ Σ 03 75 ATE,,
(14)
where the first equality is due the definition of CATE in Eq. 2 In addition, the causal estimand and bias of a proxy-of-comfoundee based causal inference model that controls the latent variable Z inferred via = f(x)= B" X (where B is also a full column-rank matrix) can be formulated as:
DCEV (BT x) E|Y T 1, Z Bd xF E
Z BTx]
EEY T 1, 2 Bd ox + B A
cl m]] - B| = 0, = BT ax + B Ac|[c||
"1 = c, = m|
B| 0, c, m)
= +r Σ
M = m] + Σ Kig E[C. = 1, 1, = = e, M = m
ay + Σ
8j E[M, T 0,C M =m] + Σ
Kg E[C, 0, C c, M m
= =T (1 - 0) + Σ 0, (m, - m2) + Σ Ni (c: ci)
= T = [[]] = (BDC X)),
(15)
where step (a) is due to the fact that, since both and B are full column-rank matrices, B A is an invertible matrix, and the mapping f B"aayaaaa B", A is bijective. Therefore, Lemma 3.1 and apply the left-inverse of f. i..., fi = (B"A)~ B axx to the condition of the expectation. The rest steps based on the structural causal equations defined in Eq. (2) n
invoke
14

---

Published as a conference paper at ICLR 2025
A.3 ANOTHER CASE OF LINEAR SCM WITH LATENT CORRELATORS
Corollary A.I. (Latent Correlator). For another Linear Structural Causal Model defined as:
T 1(TT + 3.. 3,,C;>a)
M, + aM + 72 T + 0o Uy.U,
(16)
X + ax + A[M]]C]
Y + ay + T o, + Ex-
Kc Cs.
where f= A (RKKx((K+KK)) full column-rank matrix, the CATE, ATE, and the bias of proxy-o-coorfounder-aaa basse causal inference model that controls the latent variable Z inferred via Z f(X) B X be formulated as follows:
ATE CATE-
EIDCEV(Z B)X] (Z = B X) X) - Σ
0.-7%
Φ
(17)
Bias = ATE B|DCEV(B" X)] = Σ 8g Φ
where B E (K) is another full column-rank matrix. Since Σ
9,
is arbitrary, the
9,
estimator EEDCEV(B" X)] is arbitrarily biased for the estimation of ATE.
Proof. The proof of the CATE and ATE is trivial. The causal estimand and the bias of proxy- of-confounder-base causal inference model that controls the latent variables Z inferred via f(X) B" X (where B also a full column-rank matrix) be formulated as follows:
DCEV x) EE 7 1, Z B x] E = 0, Z BT
= =E 1, Z ax B [c] B 0, Z ax + B A[c[m]]]
@ 1, M m] -
M m
=ay +r 1 9, E T = -
ml Σ K4 E 1, M m
ay + Σ
0j BLU = 0, e, M = m] + Σ Ki E.T. = 0, = c, M = m
= T (1 0) + Σ
0j (07
(m, a. - -1)) 05 (m, am)) + Σ Ki (G c:)
0, 7
0j 7j
7
E
t Σ
= -EDDCEV(" X)).
Φ
Φj
(18)
where step (a) and the rest of the proof follow the same logic as the proof in Section 3.3
A.4 PROOF OF THEOREM
The strict definitions of the exponential family, strong exponential (which is assumed for the factorized part of the conditional prior), and identifiability follow the definitions in (Khemakhem et al.. 2020 Lu et al. 2021), and can be referred to in Appendix E, F of (Lu et al. 2021) which we omit to avoid redundancy. The proof Theorem 4. is largely based on the NF-IVAE paper (Lu et al. 2021). where most of the details can be found, with the assumption introduced in CiVAE that each S has at least one invertible dimension incorporated to ensure that each dimension of the inferred latent variables is bijective transformation of the corresponding true latent variable.
15

---

Published as a conference paper at ICLR 2025
A.4.1 PART
Step I. In this step, we transform the equality of noisy conditional marginal distribution of X given Y, T of two models with parameter 0, 0 θ into the equality of noise-free distributions.
po ( X Y.T)
P X Y.T)
P Z) Z Y.T)dZ = Pj(( Z)ps,x |Y,T)dZ
2
Pc(x (-(X-I(ZZ))) ([()))ss ( Y,T)dZ (p((x-(z))ss, YYT)
(a)
Pe X)PSA (X) Y,T) vol (J)1 (x) dx=
Pe(X XIPPAA (f(X) X,T) vol (J/((X)) dx
(19)
Px ( / Po(X X
JRG
+ (PI.S.A...I* * Pe
(x)
PII.S.AY = [
(w)PE(w)
F F(61,SAXYT)()) [...A.P.] (w)
(I..ANT(X (X) = -j..A....(().
Step (a) is based on the rule of change-of-variable, where vol(A) = V det (ATA). In step (b),
we define PJ.S..YY.TXX) X) - Ps.a (st(X) IY,T) vol (x)(()) (X) In step (c), we use F to
denote the Fourier transform. In step (d), we drop (e((w) as it is non-zero a.e. (see Assumption 3
Step II. In this step, we transform the equality of the noise-free distributions into the relationship of the sufficient statisties S and S. By taking logarithm of both sides of Eq. 19 we have:
log vol (X)) + log Q (ft (x)) - log C(Y, T) + (S (x)) A(Y,T))
(20)
= log vol (J (x) + log @ (fl - log C(Y,) ($ (f()) A(yTT)
Let (Y,T), (Y, & be the A: + distinct points defined in Assumption 3- (iv). We obtain k + 1 equations by evaluating the Eq. (20) these points, where the first equation is subtracted from the remaining ones, which leads to the tollowing equation system:
C ((Y,T)o
(S (f*())) , -> ((X,T))) + log ((Y,T))
(21)
C ((Y,T)))
= -(s ( + log Č ((YTT)))
1= 1,
Let L be the invertible matrix defined in Assumption 3 (iv) and L be the counterpart for X, if summarize all terms irrelevant to X into constant b, we have:
La' (f)()) = - (f((x) +b
(22)
= -S((())) = As (fl(x) + cc
where A E TR and c TR
Step III. Ideally, prove the element-wise bijective identifiability of the latent variables Z, the transformation of the sufficient statisties S derived in Eq. (22) should be bijective. We claim that the conditional prior P (Z YYT is strongly exponential and L is invertible, L and A must also be invertible. The proof is omitted, and can be referred to in Appendix H.1.1 of et al. (2021).
16

---

Published as a conference paper at ICLR 2025
A.4.2 PART II
In this part, we prove that, if Assumptions I and 3 hold, we can identify the factorized part of the sufficient statistics ( i.e., S,(2), up to permutation and element-wiss transformation.
Specifically, if we use to denote the composite map f f Z + Z, Eq. 22 can be rewritten into:
S ( + c.
(23)
We aim to prove that A in Eq. 23 a block permutation matrix.
Step I. We start by showing that v is component-wise function. If we differentiate both sides of Eq.
(3 with respect to Z, and Z,, where + I, we have:
aS(
K2
a5 (Z)
(Z)
A
82.
bv. Z
Oz.
-=1
(24)
S2) OZ,0%,
8 S 2
(2y(Z) (v;(Z) oz,
A (S(v Pu, i=1 (z) az,0Z.
A
oc, Z )z)
oz.
Note that for the factorized part of the sufficient statistics S, i.e., Sj, all cross cross-derivatives zero, and for the non-factorized part of S, i.e., Saf. which is a neural network with ReLU activation (i.e.. linear a.e.), all second-order derivatives are zero. Therefore, the second order cross-derivalives the LHS. of Eq. (24) are zero, which leads to the following equality:
Kz 02 (p5((()))
dv:(
du: (
Kz OS( (Z) ())
0
A
Σ
(25)
(v,(Z)² az.
az.
dv. (Z) Oz,0Z.
Eq. 25 can be written into the matrix- product form as follows:
0 = AS (Z)'",
())",,,,
(26)
where
Sz) = (2((((2))
82 (S((((Z))
E CRAXKz
Ou (2)²
OUKz (Z)
[uv 10v((2) (Z)
do (Z) 0z.
dukz (Z)
Oukz(Z) dzg
(Z)
E ERKx,
OZ,
az,
and
(Z))
05(0 2))
Š'(Z) =
E ERexKz
(2 OuKz (Z)
Sor
Pukr (Z)
T
(Z)
E R2
02,02.
oz,0%,
If we denote the concatenation (z)
[5"(
Rkx2Kz and (2)
(Z), "", E R2Ks we have:
0= AS" Z)u (Z).
(27)
Finally, if we denote the rows of (Z) that correspond to the factorized part of S by 5yx( (Z). according to Lemma 5 of (Khemakhem et al. 2020) and the assumption that k: > 2K 2, we have that the rank of Sy (Z) is 2 2Kz. Since k > 2Kz, the rank of S (Z) is also 2Kg. Since the rank of A kt, the rank of A Z) is 2Kz, which implies that v"! (Z) E R 222 is zero vector. Therefore, we have v',, ZZ) = 0, Vs + t, and we have demonstrated that is a component function.
Step II. Based on Step I. we demonstrate that A is a block permutation matrix. Without loss of gen- erality, we assume that the permutation in is Identity, where v Z)) [Va (Zz), UKK (Zxz)l ) and each is is nonlinear univariate scalar function. Since and f are injective, e is bijective and
17

---

Published as a conference paper at ICLR 2025
の (Z) = ( ,..., ,KK, (Zkg) )]² we denote ()) S(o(Z)) + c- Eq. 23
( 2)). We then apply wー to Z on both sides, which gives can be reformulated as ( = (
S (z) - Z
(28)
Let t be the index of an entry in S that corresponds to the factorized part Sy. For all s o t, we have:
0 = as (u (V))
k Σ ==1
0S(
at
(29)
OZ.
OZ.
Since the entries of S are linearly independent, Otj is zero for any j such that (5 includes the entries Sj that correspond to (1) the factorized part that does not depend on Z; and (2) the fantorctorieed part Sng- Therefore, when t the index of an entry in the sufficient statistics S that corresponds to factor i in the factorized part Sy. i.e., Sy.5, the only non-zero dej are the ones that map between S, (Z:) and S f.t (ve (Z:)). Therefore, we can construct an invertible submatrix A with all non-zero elements aty for all that corresponds to factor i, such that
az. * 0. This
S/ (Z.) = 5r (vs (Z)) 5,a (my (Z.)) + Ci. i= K2:
(30)
where c; denotes the corresponding elements of c. Eq. (30) means that for each 1, 1,... Kz. the matrix block A/ of A affinely transforms the i-specific sufficient statistics vector S (7i) into
S, (o (Z.)). In addition, there is also an additional block A' that affinely transforms Sn (Z) in
into S (u( ()). This completes the proof that A is block permutation matrix.
A.4.3 PART III
Let 2 = U/ (Z.) f be the ith inferred latent variable. Assume again that the permutation in is Identity. In this part, we prove that if Assumption 2 holds, each inferred latent variable 2, the bijective transformation of the true latent variable. The proof is as follows.
Proof. Plugging 2, into Eq. 30). we have:
Sz,(ZZ) (2,)
(31)
According to Assumption 2 there exists one dimension of S,,, i.c., j. such that Sy, is bijective. This implies that Sy,, is injective, and therefore it has a left-inverse S we apply f,; to both sides of Eq. 31). which gives:
Z, S S
(32)
Since A/ is a block of an invertible block permutation matrix, A is also an invertible matrix, and therefore A/ is a bijective mapping. In addition, since $, is injective, S is also injective, and therefore the composite map A/, R + R that applies on 2, is a bijective. This completes f.:
the proof that each inferred latent variable Z, is the bijective transformation of the true latent variable in the case of no noise, where Z ftXX) are the true latent variables. If noise & exists, the posterior distribution of the latent variables can be identified up to analogous bijective indeterminacy.
A.4.4 CONSISTENCY
Proof. If the family of the variational posterior go (Z),,, contains the true posterior Po( X Y. T), then by optimizing the loss of Eq. (9) (with the KL term replaced by the score match- ing loss defined in Eq. (10) over its parameter Φb the score matching term will eventually vanish. Therefore, the ELBO term in Eq. (9) will be equal to the log-likelihood. Under this circumstance, CiVAE inherits all the properties of maximum likelihood estimation (MLE). Since the identifiability of CiVAE is guaranteed up to permutation and component sispo bijective transformation of the latent variables, the consistency property MLE means that the model will converge to the true parameter 0" up to such mild indeterminacy of the latent variables in the limit of infinite data.
18

---

Published as a conference paper at ICLR 2025
A.5 PROOF OF THEOREM 4.2
Proof. Let C be the true latent confounders and C' be the transformed confounders, where the transformation function is bijective and differentiable a.e. Let f- denote its inverse. The ATE estimator that controls transformed confounders C can be formulated
DEV(C) = E PPcc 1, -
a]].
(33)
Specifically, for the continuous case where density functions exist, for each term, have:
E MC) BIVIT C, = =]] - lo(@) f SYIT,OlUlt, ē)dydč.
(34)
For the marginal density fe(e), the following equality holds:
fa(C) fc(ff (() = fo(c)),, (c)
(35)
As for the conditional density fyir,clylt, é), since f is bijective, according to Eq. 12 we have:
f f c) friT.c(Ult.e)
(36)
Combining Eqs.
35 and 36). and given that dē |Jf(c)|de, we have:
/ [o(c)),, f (ddyJ,
34
J (0)- sc(e) f (yl,
(37)
/ solc) / fyir,clult, c)dyde
c]
where the term (3)] - (c) vanishes in step (a) as the two factors have the product of one. Therefore, if we plug Eq. (37) into Eq. (33), it leads to the following equality:
DEV(C) P(C) E
Y č 2| E 0, 2]]
(38)
= Pp [ 1, = E = 0, = DFV(C) = ATE,
where the last step due Eq. 2 in Definition 2 which completes our proof that controlling bijectively transformed confounders provides an unbiased estimation of ATE.
B RELATED WORK
B.1 POST-TREATMENT BIAS IN CAUSAL INFERENCE
Bias due to accidentally controlling post-treatment variables, i.c., post-treatment bias, has long been recognized as dangerous in causal effect estimation (King 2010). Back 2005, Pearl (0015) cautioned that controlling more is not better, and uses the collider bias (Elwert & Winship 2014 examples show that bias be increased when
and M-Bias DDng & Miratrix 2015) controlling the post-treatment variables. Furthermore, Montgomery et al.) (2018) show that indirect correlations between post-treatment variable M and outcome can still cause bias. Recent works prove that even if M has no causal relationship with Y, controlling it can still increase the variance of estimand (Hassanpour & Greiner 2020). However, most of these works study the post-treaties treatment bias in the observational space, where latent post- post-treatment variables that are mixed with confounders generate the observed covariates can be easily ignored by the researcher. Therefore, it motivates us to develop CiVAE, which is robust to the latent post-treatment bias under mild assumptions.
B.2 COVARIATE DISENTANGLEMENT
Recently, researchers have realized that directly controlling the proxy confounders X may not be safe, variables other than confounders could lurk in the confounder proxy and ruin the ATE
19

---

Published as a conference paper at ICLR 2025
estimation (Hassanpour & Greiner 2020 Traditional methods assume that the variables that generate the observed covariates X are a mixture of confounders, adjusters, and influencers (Shalit et al. 2017), where adjusters should not be controlled as it can increase the estimation variance (Hassanpour & Greiner 2019). Most methods rely on the fact that adjusters are correlated only with the treatment separate them from other variables (Hassanpour & Greiner 2020 Zhang et al. (02T) (see Fig. (2). This can also be used to remove post-treatment variables that are not correlated with the outcome, which have similar statistical properties with adjusters (Wang et al. 2023). Here, a different work is NICE (Shi et al., 2021), which uses the fact that confounders and influencers are direct causal parents of the outcome to find these variables with invariant learning as the control set (Arjovsky) et al. 2019). However, since mediators are also direct parents of the outcome, NICE ís still not robust to general post-treatment bias. Given that all the above methods cannot satisfactorily address the latent post-treatment in general cases, it is imperative to design the CiVAE, where confounders can be identified and disentangled with latent post-treatment variables for unbiased adjustment. We note that Xu et al. propose CFDiVAE that identifies sufficient mediators from covariates with iVAE and estimates the ATE via front-door adjustment, which is an interesting alternative to CiVAE provided that the covariates X contain all mediators between T and Y where front-door criterion satisfies.
C EXTENDING CIVAE TO ADDRESS LATENT INTERACTIONS
C.I EXAMPLES FOR THE IDENTIFICATION CRITERIA
In this section, we provide concrete examples to more intuitively demonstrate the ability of CiVAE to address (non-cyclic) interactions among the latent confounders and latent post-treatment variables M introduced in Section 4.3 which are discussed as follows:
Example 1. Intra- Corr-Coofouuder Interactions.
In this example, latent confounders are allowed to interact with each other, i.e., for arbitrary i', Cr could be the causal parent of Cyr. In this example, if two inferred latent variables 2,, 2, E 2 (which individually identify the true latent variables according to Theorem I.I) are latent confounders, i.c., Cz, Cj (case 1"), we have åc = PA(Cx) n PA(Cp) C Č, where PA denotes the parent set of the node in the true causal graph. For cases 2" ,", we have 2M 2C. = 0.
Example 2.
Interactions.
In this example, post-treatment variables are allowed to sequentially (i.e., non non-cyclic) influence one another, i.e., for arbitrary i', j', M2 could be the causal parents of My no circle is formed. In this case, if two inferred latent variables Z, 2, E 2 (which individually identify the true latent variables according to Theorem 4.1 are latent post-treatment variables, i.e., My, My (ease 2"), we have ZM PA(M,) n (A( C M. For cases 1', 13,3", we have Zc Zc, 0.
Example 3. Cross Confounder- Senenddrr--oo--eeeee Post-treatment Variable Interactions.
In this example, confounders are allowed influence the post-treatment variables (note that since confounders are pre-treatment, they cannot be influenced by the post- testtrrattmett variables). In this example, similar to Example 2, if two inferred latent variables 2,, 2, E 2 (which individually identify the true latent variable) latent post-treatment variables, i.e., Me. My (case 2*), we have 2M = PA(M.) n (annAAPAA PA(M;) C Ć. For cases 1',3", we have GG Zc. C = 0.
More complicated cases can be viewed as combinations of Examples 1, 2, and 3, where the condition sets Żc, ZM, Zc,M = @ can be directly derived by utilizing the above conclusions.
C.2 ADDITIONAL EXPERIMENTS
C.2.1 INTRA-INTERACTIONS AMONG LATENT MEDIATORS
In this subsection, we empirically analyze the case where latent post-treatment variables M interact with each other. Specifically, we extend the simulated datasets described in Section 5.1 where we
Code available httptp////gggtuuu...o////cccce//
20

---

Published as conference paper at ICLR 2025
Table 2: Comparison of CiVAE with basclines when intra-interactions among M exist.
Dataset
IatentMediator
LatentCorreLator
LinkedIn (Age)
LinkedIn (Gender)
Method
ATE.
Emr.
ATE.
ATE.
ATE.
CEVAR
1,627 +: 0.549
2.627
2.659 1: 0.302
1.353
0.152 0.027
0,420
-0.225
0,044
-0.144
TEDVAE
1,653 1: 0.511
2,042
2.827 1: 0.259
1.521
0.180 0.477
0.448
-0.189
00122
-0.108
CiVAE
-0.350 + 0.695
1.785
1.785 + 0.481
0.479
-0.073 40.101
0.195
-0.136
0.087
-0,055
True ATE
-1.000 0.000
0.000
1.306 + 0.000
0.000
-0.268 + 0.000
0.000
-0.081
0.000
0.000
Table 3: Comparison of CiVAE with baselines when inter-interactions between C' and M exist.
Dataset
LatentMediator
Latenttoorrelator
Linkedln (Age)
Linked (Gender)
Method
ATE.
Err
ATE.
ATE.
ATE.
CEVAE
2.070 + 0.279
3.070
2.831 + 0.398
1.831
0.094 0.01
0.362
-0.192
0.015
-0.111
TEDVAE
1.743 à 0.307
2.743
2.954
1. 0,763
1.954
0.109
0.116
0.377
-0.212
0.019
-0.131
CiVAB
-0.716 0.523
0.284
1.385
+ 0.660
0.385
-0.041
10.144
0.227
-0.129
0.044
-0.048
True ATE
-1.000 0.000
0.000
1,000
+: 0.000
0.000
-0.268 +: 0.000
0.000
-0.081 0.000
0.000
make (i) T' directly affects M1. (ii) M1 affects M2, and (iii) M1. M2 affect M3. The coefficients are randomly sampled from (0, 1/3). The results Table demonstrate that the adapted CiVAF still more robust to latent post-treatment bias compared to CEVAE and TEDVAE.
C.2.2 INTER-INTERACION BETWEEN LATENT MEDIATORS AND LATENT CONFOUNDERS
In this subsection, we empirically analyze another case where inter-interactions exist between latent confounders C' and latent post-treatment variables M. Specifically, we extend the simulated datasets described Section 5.1 to allow each latent confounder C, E R determine M R$. The coefficients randomly sampled from (N 1/3). The results in Table demonstrate that the adapted CiVAE is still significantly more robust to latent post-treatee cost-treatment bias compared to CEVAE and TEDVAE, which empirically verify claim that the adapted Ci can address the case where inter-interactions exist among latent confounders and post-treatment variables.
C.3 ROBUSTNESS ANALYSIS OF CIVAE
C.3 Robustness Analysis of CiVAE
C.3.1 SENSITIVITY W.R.T. ASSUMPTION
In this section, we evaluate the sen- sitivity of CiVAE to the injectiv- ity assumption stated in Assumption I For the latentMediator and latentCorrelator cases out- lined in Section 5.1] the mixture func- tion corresponds the matrix A described in Corollaries and A. respectively. When Kx Ke +
Laiset Medivior
Labent Comwlator
ATE Eximmtion
65
0.25
0.5
0.55
TEDVAE
-
and
Figure 6: Sensitivity CiVAE to injectivity assumption.
KM. directly random sampling of A results in A being almost surely injective.
To make A approach non-injectivity, we first perform singular value decomposition (SVD) on A yielding U V,, where U and V are orthogonal matrices of left/right eigenvectors, and A is the diagonal matrix of singular values. We then dampen the largest singular value in by a factor
of 1 5, resulting in the modified singular value matrix A1-5. The dampened reconstruction A = U A V is used to generate the covariates X from C. M instead of A. When 6 = 0.
A maintains injectivity. As 8 approaches 1, A accordingly approaches non-injectivity.
21

---

Published as a conference paper at ICLR 2025
The evaluation of CiVAE, CEVAE, and TEDVAE data generated under different 5 are illustrated in Fig. 6 From Fig. we observe that violation of injectivity assumption indeed has a negative influence the accuracy of ATE estimation. However, even with the decrease of 6, which indicates approaching non-injectivity, CiVAE is still consistently more robust to latent post-treatment bias and outperforms both CEVAE and TEDVAE in terms of maintaining accurate ATE.
C.3.2 SENSITIVITY W.R.T. LATENT DIMENSION
In this section, we explore the sen- sitivity of CiVAE to model mis- specification. Specifically, the dimension of latent variables as-
Neditor
Conretttr (Wl it))
vary
sumed by CiVAE, i.e., Dim 2).
to values in [4, 80 10] for both the latentMediator and the Correltoo cases out- lined in Section 5.1 (with the correct latent dimension being 6) and com- pare its performance with CEVAE and TEDVAE. The results are illus- trated in Fig. 7 In Fig. 7 the upper figures show the cases where inter- actions exist among latent variables, and the lower figures show the cases when latent confounders also confound the latent post-treatment variables (see Section C.2.2). From Fig. 7 find that underestimating the latent dimension does have an evident negative influence on CiVAE. However, CiVAE demonstrates good robustness to the overestimate in the latent dimension, showing more robustness of latent post-treatment bias CEVAE and TEDVAE even when the dimension is mis- mis-specified. This highlights CiVAE's ability to adapt to model specifications and underscores its enhanced robustness to latent post-treatment bias compared to the baseline methods.
Join
Edimated ATE
Dim(2)
Din(2)
-0-
GEVAE
TEDVAE
-
QIAE
Ground Trith
Figure Sensitivity of CiVAE to model misspecification.
22


Reviews:
[{"review_id": "rRuiRoIOBS", "reviewer": "Reviewer_7C4u", "summary": "This paper studies the estimation of treatment effects under the presence of latent confounders and latent post-treatment variables. This is a challenging scenario where conventional methods may yield biased estimates of treatment effects. To correct the bias, this paper proposes a novel method, namely Confounder-identifiable VAE (CiVAE), to estimate treatment effects. The paper provides some theoretical guarantees for the method. Moreover, the paper demonstrates that the proposed method performs well in empirical applications.", "questions": "See Weaknesses above.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The problem is well-motivated.\n- The method seems to work well in empirical applications.\n- Figures 1 and 2 are very helpful for understanding the problem setup.", "weaknesses": "- The real-world dataset used in the empirical study does not seem to be publicly available.\n- The paper uses many notations without definition. For example, the definitions of $K_Z$ in footnote 3, $p(X\\mid C)$, and the matrix of $p(C, X)$ are missing.\n- Section 4 is a bit challenging to follow -- notations are heavy, and the exposition is a bit detail-oriented. Providing some high-level intuitions in this section could be helpful. \n- I am not sure why there is always a clear separation between latent confounders and post-treatment variables. It is possible that the value of latent confounders (e.g., health indicators) can be changed by the treatment, and thus, these latent confounders also seem to be post-treatment variables.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PEmTXirPkW", "reviewer": "Reviewer_PTBp", "summary": "The manuscript proposes CiVAE method to address the risk of conditioning on post-treatment (latent) variables in estimating causal effects, by disentangling the latent unobserved confounders from the latent post-treatment variables. The identification result of CiVAE and identification of latent confounders are shown. The authors also show the empirical results by a simulated data and a real data.", "questions": "See weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The authors study a important issue of latent post-treatment bias.\n- Pratical algorithm and sound theoretical results are provided under certain conditions. \n- Empirical performance is strong comparing with existing ones.", "weaknesses": "- Assumption 1 requires $f$ to be injective, which is much stronger than common conditions in the literature, even the dimension of the observation space is larger than dim(C) + dim(M). In many cases, there might be a  latent confounder or post-treatment variable not correlated with $X$. It is suggested to check if the proposed method sensitive to the injective condition.\n\n- The interactions among latent variables might come from other unobserved variables that confound $Z_i$ and $Z_j$, not belonging to three cases in section 4.5.\n\n- How to determine the dimension of Z? How's the different selections of dim(Z) affect your ATE and identification of latent confounders with or without interactions?\n\n- Please fix the typo in equation (8).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uyQPcmiuIg", "reviewer": "Reviewer_22EG", "summary": "This paper addresses the challenge of handling latent confounders and post-treatment variables. Recent research has attempted to solve this problem by using proxy variables. However, selected proxies may inadvertently combine both confounders and post-treatment variables, potentially introducing bias into estimates. To tackle this, the authors propose a novel method called the Confounder-identifiable Variational Autoencoder (CiVAE) to mitigate this bias.", "questions": "See Above.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.I find the problem they studied intriguing, specifically how proxy attributes may cover both confounders and post-treatment variables. It is crucial to separate these biases throughout the causal effect estimation process.\n\n2. The paper is well-written and theoretically sound, with proofs for all statements provided in the appendix.\n\n3.The core idea is clearly explained and easy to follow. Including a code link improves reproducibility.", "weaknesses": "1. In Figure 2, there is a bi-directional path between M and Y. However, in the following example, M is treated as a strict mediator. Please ensure consistency in the directions of M. Or add more examples to show if the case T->M<-Y happened. In line 269, it states, \"However, since both C and M form fork structures with the outcome Y (see Fig. 2-(c)).\" I am wondering why M forms fork structures.\n\n2. Z is the true latent space (Z = [C, M]). According to the paper, there exists a bi-directional path between M and Y. Could you explain why only p(Z|Y,T) is considered? If T → M → Y, I agree with using p(Z|T); if T → M ← Y, I agree with p(Z|T,Y). I suggest considering different cases using different forms of p(·).\n\n3. More interestingly, if Z also includes C, then is C identifiable? I am wondering if C is treated as a general case in CEVAE without considering identifiability. Even with iVAE, additional information may be missing to guide identifiability.\n\n4. A related work may need to be included. [1] discusses the iVAE learning process of the front-door adjustment set, which is a very similar setting to this paper. I suggest adding a discussion of this in the related work section.\n\n[1] Causal Inference with Conditional Front-Door Adjustment and Identifiable Variational Autoencoder in ICLR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bTzjpjZPHt", "reviewer": "Reviewer_JSgY", "summary": "The paper introduces the Confounder-identifiable Variational Autoencoder (CiVAE) to estimate treatment effects in settings where the observed covariates are a function of latent confounders and latent post-treatment variables. The proposed method is designed to disentangle these latent components, and it is provably unbiased under some specific assumptions. Through experiments on both simulated and real-world datasets, the method demonstrates superior performance compared to existing baselines.", "questions": "- Could the author provide some examples in the causal inference literature that involve the use of post-treatment proxy variables?  \n- Under what circumstances do you expect the injectivity assumption to hold, given that even some simple linear models may fail to satisfy this?\n- Can the author share experimental results showing the impact of violating injectivity in Assumption 1?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The issue of post-treatment bias is understudied, making this paper a valuable contribution to the field.\n\n- The paper is well-written, which makes it easy to understand the core ideas.\n\n- The empirical validation shows that the method outperforms existing baselines.", "weaknesses": "- **Main concern**. The method depends on strong and untestable assumptions for identifiability, e.g. the injective mapping described in Assumption 1. Such assumptions may be difficult to verify, even with domain knowledge, and fall short of the causal inference field's emphasis on interpretable and justifiable identification assumptions. \n- **Other concerns**. While post-treatment variables are recognized as an issue, I am skeptic regarding the significance of *latent* post-treatment variables as a problem. The example provided does not clarify why these latent variables pose a significant issue, and I'm uncertain about its practical relevance.\n- **Other concerns**. The assumption of independence between latent confounders and latent post-treatment variables (see Fig. 1c) is quite strong and unlikely to hold in any realistic application.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)