Causal Dynamic Variational Autoencoder for Counterfactual Regression in Longitudinal Data
Authors: Mouad El Bouchattaoui
Myriam Tami
BENOIT LEPETIT
Paul-Henry Cournède
Venue: Accepted by TMLR ()

Abstract: Accurately estimating treatment effects over time is crucial in fields such as precision medicine, epidemiology, economics, and marketing. Many current methods for estimating treatment effects over time assume that all confounders are observed or attempt to infer unobserved ones. In contrast, our approach focuses on unobserved adjustment variables—variables that specifically have a causal effect on the outcome sequence. Under the assumption of unconfoundedness, we address estimating Conditional Average Treatment Effects (CATEs) while accounting for unobserved heterogeneity in response to treatment due to these unobserved adjustment variables. Our proposed Causal Dynamic Variational Autoencoder (CDVAE) is grounded in theoretical guarantees concerning the validity of latent adjustment variables and generalization bounds on CATEs estimation error. Extensive evaluations on synthetic and real-world datasets show that CDVAE outperforms existing baselines. Moreover, we demonstrate that state-of-the-art models significantly improve their CATE estimates when augmented with the latent substitutes learned by CDVAE—approaching oracle-level performance without direct access to the true adjustment variables.

Published in Transactions on Machine Learning Research (10/2025)
Causal Dynamic Variational Autoencoder for Counterfactual Regression in Longitudinal Data
Mouad EI Bouchattaouj
Myriam Tami"
Benoit Lepetit
Paul-Henry Cournède!
Parri-Saclay Universily, CentralaSupéler, MICS Lab, Gif-sur- Yucite, France
SSant- Gobain Pronce
Reviewed on OpenReview: https: //
// openreveew.eee.... net/ net/ forum? id at/9g49De
Abstract
Accurately estimating treatment effects over time is crucial fields such as precision modicine, epidemiology, economics, and markcting. Many current methods for estimat- ing treatment effects over time assume that all confounders are observed or attempt to infer unobserved ones. In contrast, approach focuses unobserved adjustment vari- abbia--varaales that specifically have causal effect the outcome sequence. Under the
ables variables assumption of unconfoundedness, we address estimating Conditional Average Treatment Effects (CATEs) while accounting for unobserved helerogeneity in response to treatment due to these unobserved adjustment variables. Our proposed Causal Dynamic Variational Autoencoder (CDVAE) grounded in thcoretical guarantocs concerning the validity of latent adjustment variables and generalization bounds on CATEs estimation error. Ex- tensive evaluations synthetic and real-world datasets show that CDVAE outperforms existing baselines. Moreover, we demonstrate that state-of-the-art models significantly im- prove their CATE estimates when augmented with the latent substitutes learned by CD- VAF approaching oracle-level performance without direct access to the true adjustment variables.
1 Introduction
Estimating Conditional Average Trealment Effects (CATEs) helps us undersland how individuals uniquely respond to the same treatment, thereby enabling more personalized and effective decision- decision-making, For ex- ample, healthcare, two patients receiving the same drug might experience considerably different outcomes due to underlying genetic lifestyle differences (Atan al., 2018 Shalit 2020: Mueller & Pearl 2023 The exact curriculum might yield more remarkable improvement for one student than another in education, de-
pending background factors like socioeconomic status (Morgan 2013 Imbens & Rubin 2015 Likewise, in marketing, identical promotions might drive purchases in one customer segment but not another (Hair Jr
& Sarstedt 2021 Fang et al. 2023).
Longitudinal data arisc naturally in these domains. Consider, for instance, a medical datasct recording blood pressure, treatments (e.g., vasopressors), and vitals for each patient al regular intervals. Or a retail dataset tracking weekly customer purchases following commercial campaigns. Each of these longitudinal data describes sequence of treatments, covariates, and responses causally interacting through time. However, this setling brings unique challenges: (1) Time-dependent confounding: Confounders influenced by past treatment can impact subsequent treat and responses (Platt et al. 2009;; (2) Selection bias: Time- varying covariates exhibit imbalanced distributions across treatment regimes which should be accounted
for to estimate treatment effects accurately [Robins et al. 2000 Schisterman et al. 2009 Lim 2018
The implementation is available at https://github cww/noad Lit.ooo/dd.ihccoo//ccvaaaa

---

Published in Transactions on Machine Learning Research (10/2025)
(3) Long-term dependencies: A treatment effect may unfold over extended periods, requiring models to capture complex long-range interactions between covariates, treatments, and responses (Choi et al.) 2016: Pham et al., 2017); (4) Missing covariates: Some variables crucial for predicting outcomes like genetic predispositions or environmental maassurcs-iitrrouue bias and less personalized treatment effect estimation unless properly accounted for.
Assumptions over Confounders and Existing Approaches The existing literature primarily ad- dresses the first three challenges under the assumption of sequential igmorability, where all confounders, whether static or timo-varying, are fully observed. Methods such as Marginal Structural Models (MSMs) [Robins& & Hernán 2009a). Recurrent Marginal Structural Models (RMSM) (Lim 2018p, Counterfactual Re- current Networks (CRN) [Bia et al. 2020a): G-Net (Li et al. (021). ) Causal Transformers (CT) (Melnychuk let al. 2022), and Causal CPC (Bouchattaoui et al. 2024] have been developed based on this assumption. When the fourth challenge-missing covariates- is addressed, it is often in the form of missing confounders, leading to the violation of sequential ignorability. Existing approaches either condition on observed prories
of confounders to infer a latent representation of the unobserved confounders (Kuroki & Pearl 2014 Miao
et al. 2016 Louizos et al.) [0177] Cheng el al., (021]) or apply the deconfounding lechnique, which involves imposing a factor model over the treatment assignment to mitigate hidden confounding (Lopez & Gutman
2017 Ranganath & Perotto 2018 Wang & Blei 2019a: Zhang et al., 2019 Bica et al. 2020b Hatt
Feuerriegel 2024).
Our Focus In contrast to missing confounders, we study for the first time the presence of unobserved slatic adjustment variables- factors that affect only the outcome sequence and remain iineeinvariant. In a medical context, these could include genetic factors, environmental conditions, or lifestyle attributes that impact treatment response but are not directly observed (Sadowski et al., 2024). The absence of such variables can lead to loss of heterogeneity in the estimated treatment. effect. This phenomenon can also be understood through the lens of population structure, which arises from distinct subgroups within a population that share characteristics such as geography, sociocconomic status, or cultural practices. Having knowledge of, or being able to accurately infer, population structure allows for a more precise estimation of CATEs by capturing variations in treatment responses across subgroups. In genomics (Laird & Lange 2011 Peter et al. 2020). for instance, such structure arises due to evolutionary or migration histories. This concept extends beyond geneties to fields such as economics, healthcare, and education, where differences among subgroups stem from factors such as socioeconomie conditions or environmental influences. Moreover, population structure often acts effect modifier (Hernán & Robins, 2020), altcring treatment effects across subgroups without directly affecting trealment assignment (Hyun et al. 2024). By accounting for such elfect modiliers through stratified or interaction analyses, treatment effects can be estimated more accurately, even in randomized trials (Schochet 2024).
Adjustment Variables vs. Confounders. We distinguish confounders, pre-treatment variables that affect both treatment and outcome, whose omission biases estimates, from adjustment variables, pre-treatment vari- ables that affect the outcome and may modify treatment effects but not treatment assignment (Hernán MA 2020). Adjusting for the latter clarifies further treatment-effect heterogeneity, even when confounding variables are fully observed. For example, in healthcare, pharmacogenetic variants (VKORC1, CYP2C9) markedly alter warfarin (a blood thinner) response yet typically do not determine treatment assignment. Modeling these variants (observed or represented) improves precision and personalization of effects (McClain et al. 2008 Schwarz et al., 2008). In education, Baseline test scores and socioeconomic indicators are im- portant adjustment variables and often are effect modifiers. Covariate adjustment in school-level RCTs thus substantially increases precision and clarifies subgroup treatment effects (Bloom et al., 2007 Steingrimsson let al. 2017). In online experiments/ marketing, pre-period behavior (preceding the A/B test) consists of important adjustment variables that do not affect the randomized assignment, yet adjusting for them leads to faster detection of heterogeneity in treatment effect and even a reduction of the required sample size to achieve a given statistical power (Deng et al. 2013 Benkeser et al. 2021
In this work, we specifically address the challenge of missing covariates by focusing on estimating the contem- poraneous treatment effect- the effect of the current treatment on the subsequent response, given the history of confounding processes (see Remark 1] We consider the estimation of an Augmented Conditional Average
2

---

Published in Transactions on Machine Learning Research (10/2025)
Treatment Effect (ACATE) that depends not only on the confounding variables but also adjustment vari- ables. We aim to achieve near-orucle performance in treatment effect estimation by leveraging the learned representation of unobserved adjustment variables. Specifically, we prove that for all baselines satisfying se- quential ignorability, treatment effect cstimation is substantially improved when these models are augmented with the learned representation of unobserved adjustment variables produced by our model- ultimately de- livering performance that approaches the oracle scenario, where the true unobserved adjustment variables are directly fed into the baselines. Extensive experiments on synthetic data and semi-synthetic data derived from real-world datasets such as MIMIC-III (Johnson et al. 2016) validate our approach.
Our Approach We treat the unobserved adjustment variables as latent variables and leverage a proba- bilistic modeling approach based on the Dynamic Variational Autoencoder (DVAE) framework (Girin et al, 2021) to estimate the ACATE, where we augment the CATE with the learned representation of unobserved adjustment variables, which we term substitutes. We address the covariate imbalance induced by selection bias in longitudinal data using a weighted empirical risk minimization strategy, where the weights are a function of the propensity scores. Additionally, we derive a generalization bound for the error in estimating ACATE and use its approximation as a loss function for our model, which we refer to as Causal DVAE (CDVAE). To account for potential population structure induced by unobserved adjustment variables, we define a flexible prior over the latent variables using learnable Gaussian Mixture Model. We ensure the causal validity of the learned substitutes by imposing finito-order Conditional Markov Model (CMM) on the response series. The causal validity follows from our proof that, once the finite-order CMM holds, the learned substitutes account for all relevant adjustment variables (Theorem 6 and that the ACATE leveraging the substitutes is identifiable. To experimentally support the theoretical analysis, we discuss the relevance of all CDVAE components through an ablation study.
Furthermore, since we adopt a probabilistic approach, we represent the substitute for unobserved adjust- ment variables as a stochastic latent variable. Consequently, the choice of the substitute given individual longitudinal data is not unique. As a novel contribution, we study Causal DVAE in the near-deterministic regime, where we reduce the variance of the responses (outputs of the Causal DVAE) to near zero. This process indircetly pushes the covariance matrix of the substitutes toward zero, preventing their distribution from becoming overly diffuse. An intuitive consequence of this approach is that any sampled substitute of the adjustment variables yields the same treatment effect as the mean substitute. We demonstrate that as the response variance approaches zero, the treatment effect remains consistent regardless of the specific instance of the substitutes. The neae-deteriinistic regime of VAEs has not been previously explored in the context of causal inference, making this a novel contribution. To draw an analogy, in the deconfounding literature (Lopez & Gutman 2017; Ranganath & Perotte 2018 Wang & Blei 2019a| Zhang et al.) 2019) Bica et al. 2020b Hatt & Feuerriegel 2024), substitutes reeeerieeeeegggeee to learned representations of missing confounders are theoretically assumed to follow Dirac posterior. This enables deterministic selection and facilitates treatment effect identifiability. However, the training of these models is based on a non- degenerate posterior, creating a disconnect between theoretical assumptions and practical implementation. Our approach bridges this gap by achieving near-deterministic behavior while maintaining probabilistic framework.
Contributions Our contributions can be summarized as follows: (1) We propose a principled approach to infer a latent representation of unobserved adjustment variables (S (2) We prove the theoretical validity of the learned adjustment variables by modeling the response series as a finite-order conditional Markov process, ensuring that ACATEs conditioned on learned substitutes are identifiable (S 4.1. (3) We study for
the first time the near-deterministic regime of VAEs for causal inference and show that reducing the variance of response predictions enforces consistency in treatment effect estimation under stochastic modeling of adjustment variables ($ 4.3). (4) Numerical experiments confirm the effectiveness of CDVAE in est estimating (SOTA) baselines (S 5). as
ACATEs, showing that our method consistently outperforms state-of-the further analyzed through an ablation study. (5) We show that for SOTA basclines, augmenting their input covariates with the inferred representation of adjustment variables given by CDVAE substantially enhances their accuracy in estimating ACATEs 5) and even provides near-oracle performances.
3

---

Published in Transactions on Machine Learning Research (10/2025)
2
Related Work
Causal Inference in Timo-Varying Varying Scttings Assuming sequential ignorability, MSMs (Robins et al., 2000) are widely used to adjust for time-varying confounders in causal inference, employing inverse probability of treatment weighting (Robins& & Hernán 2009a) to balance covariates by estimating treatment probabilities based on past treatment and confounders. However, MSMs often yield high-variance estimates, and their effectiveness heavily relies on the accurate specification of the treatment assignment mechanism - challenge in complex, high-dimensional settings. Recent neural network-basee models address some of these challenges. The RMSM (Lim 2018) uses Recurrent Neural Networks (RNNs) in propensity and outcome modeling for multi-step oulcome forecasting and oulperforms traditional MSMs. Furthermore, the CRN (Bica el al. 2020a) uses adversarial domain training to learn prratment-invariine representation space to mitigate bias from time-varying confounders. G-Net (Li et al. 2021) combines g-computation (Vansteclandt. & Joffe, 2014) with sequential deep models for multi-linestep counterfactual prediction. Causal Transformer (Melnychuk et al. 2022) employs self-attention to capture temporal dynamics and to learn treatment-invariint repre- sentations using an adversarial approach, similar to Bica et al. [2020a]. G-Transformer (Xiong et al. 2024)
extends g-computation by using Transformer to model covariate dynamics and Monto- Carlo rollouts for counterfactuals under dynamic regimes and is thus conceptually a combination of Causal Transformer and G- Net. Additionally, Causal CPC (Bouchattaoui et al. 2024 leverages temporal dynamics through contrastive predictive coding and information maximization and learns a balanced representation using an adversarial training strategy. Our approach enhances these SOTA baselines by augmenting their input covariates with our inferred substitute of adjustment variables, achieving a more accurate estimation of ACATEs.
Combining Weighting and Representation Learning Representation learning in causal inference helps correct covariate imbalance in high-dimensional settings by balancing learned features and reducing covariate shift. When combined with weighting mothods (Zubizarreta] 2015 Li al. 2018 Johansson et al.
2018 Hassanpour & Greiner 2019b), these approaches become more effective showing that carefully chosen weights can improve causal estimation. However, there a tradc-off between covariate balance and predictive performance: overly strict balancing can lead to the loss of valuable features, increasing bias in treatment effect estimates. Zhang et al. (2020) and JJohansson et al. (2019) noted that such regularization might lead to a loss of ignorability in the representation and suggested learning representations in which the context information remains preserved but where treatment groups overlap. To this end, Assaad et al. 2021 intro- duced the Balancing Weights Counterfactual Regression (BWCFR) method, which aims to achieve ba within the reweighted covariate representations instead of directly balancing the covariates representations. Assaad et al.] 2202)) argues that BWCFR provides bounds on the dogree of imbalance as a function of the propensity model and offers theoretical guarantees for estimating the CATE using the overlapping weight. Johansson al. 2022) built on the previous work on sampling weighting for counterfactual regression and
representation learning (Johansson et al. 2016 Shalit et al. 2017 Kallus 2020 Jung et al. 2020 Assaad ct al. 2021), and provide a comprehensive theory for weighted risk minimization for CATE for a learned representation from the data. In this paper, we adapt the weighted empirical risk minimization (WERM) framework traditionally developed for static settings to the longitudinal context. Our approach empha- sizes contemporaneeus trealments effects over sequences of interventions (see Remark D], allowing a natural adaptation of WERM to vare-varying data.
Probabilistic Modeling in Causal Inference In scenarios where confounders are unobserved but proxy variables are available, probabilistic models are used to infer a representation for the unobserved confounding given the proxy variables (Kuroki & Pearl 2014 Miao al. 2016
Louizos al. 2017, Cheng et al.) 2021). it draws inspiration from the theory of
While our work assumes the presence of observed confounding.
deconfounding (Lopez & Gutman| 2017 Ranganath & Perotte
2018
Wang & Blei 2019a), and its extensions
to time-varying settings (Bica et al. 2020b Hatt & Feuerriegel 2024. The dcconfounding involves applying a
factor model over the treatment assignment, where each treatment becomes conditionally independent given latent variables that serve as substitutes for the unobserved confounders. [ica et al. (2020b) extended the application of the confounding method to soquential settings with multiple treatments to infer time-varying confounders. They assumed that the joint distribution of causes at each time step, conditioned on latent variables and observed confounders, could be decomposed into the product of the conditional distribution
4

---

Published in Transactions on Machine Learning Research (10/2025)
of each treatment. For single treatment per time step, Hatt & Feuerriegel (2024] assumed conditional Markov model over the trealment. sequences given the sequence of confounders and the latent. variables. In this work, we demonstrate that the core idea of the factor model can be applied to learn valid substitutes for unobserved adjustment variables in the time-varying domain. Unlike previous deconfounder works, we show that assumiig a higher-order conditional Markov model for the sequence of responses sufficient to infer a valid representation of the unobserved adjustment variables.
3 Problem Definition
Following the Potential Outcome (PO) framework (Robins Hernán 2009b), we consider cohort of individuals indexed E {1, 2.....,n,, observed over T time steps. At each time point E (1, T), we define: (1) A binary treatment Wa W {0, 1}, such as whether cancer patient receives radiotherapy or not; (2) An outcome Y E C R, which represents the response to the treatment (e.g., tumor volume); (3) A context X, E X' C Rd,, time-varying dz-dimensional vector capturing confounders, such as health records and patient measurements; (4) Partially observed potential outcomes (xx(1) and (o) E y C IR, which represent the outcomes that would be observed under treatments Wit and W,r = 0, respectively; (5) Unobserved adjustment variables U, l C Rdo, which are static variables affecting the response series Y11, Ye,....,
Remark 1
We could have defined the PO for an individual given their treatment history up to time t as E W Here, we focus on the contemporoneous treatment effect,
Yx) where wi,gt == Wip.............. that relates to the current treatment Wa- We assume the treatment history up to 1, Wi to be consistent with the observed history, that is, Wi,c Wi,st-
History H,
Figure 1: A simplified representation of the DGP time Edges between Yee, Web, and X are omitted for simplicity.
We define the confounding history process Ha = [X, Wi,C, Yi,co), capturing information up to the as- sigument of Wa depicted in Figure I which also compactly represents the Data Generating Process (DGP) for which we can estimate the CATE, that is
T (h.) E(Y(1) (,) H. h.).
(1)
We assume the CATE identifiability from the observed data distribution using sequential ignorability Robins & Hernán
2009b
Lim
2018
Bica et al. 2020a) represented in Assumptions 2
Assumption 2 (CATE Identifiability) We make three key identifiability assumptions at each time step i, for any potential treatment w, and given the realization ha of the confounding history Hµ;
Consistency. If a unit i receives treatment w, then the observed outcome corresponds to the potential outcome under w. Formally, W., = w = Y,=Y² :.: Unconfoundedness/ IIgnorabbiit... The potential outcomes Yu(ww) are independent of the treat- ment assignment W., given the confounding history Hü hg- Thus, Yu(ww) 11 W, H ha-
5

---

Published in Transactions on Machine Learning Research (10/2025)
3. Cttrlap/Positicnn probabilily of observing each trealment regime. Formally, p(W, = w h,) >> 0.
For any confourding contezt h, where p(he) * O, there is non-zero
With Assumptions 2 ensure that the CATE is identifiable. Specifically, have:
Tr (h. B H, h,, W 1) B(Y, H. h., W. - =0).
(2)
So lar, we expressed the CATE as a function of the confounding history H,, Now, assuming we observe adjustment variables and aim estimate a heterogeneous treatment effect depending on both H, and U. We define the Augmented CATE (ACATE) follows:
Ty (h,, u) = (LY (,) H. = h, u..
Since the adjustment variables U influence only the response series and do not act as confounders in the treatment-responer relationship at any time step, the ignorability of treatment given He suffices for identi- fying the ACATE, as expressed below:
To (h,, u) = E(Y H. = h, = u, W, = 1) B(Y, H. = h, = u, W, = 0).
(4)
Inference Problem Deriving the ACATE becomes straightforward when both confounding and adjustment variables are fully observed. However, when U is missing, unobserved heterogencity in the treatment elfect arises, leading to incomplete or biased conclusions about treatment effects. This raises the following key inference question: Under what conditions can a substätute: Z for the unobserved adjustment variables U ensure the identifiability of the ACATE by replacing U with Z, specifically guaranteeing that the following equality (i) holds?
Te z) = B(Y((1) Y((0) H. = h., Z = B(Y H. = h., Z = 2, W. 1) B(Y, H. = h., Z = W. 0).
4 Causal DVAE
4.1
When does Latent Representation Act as Valid Substitute for Unobserved Adjustment Variables?
To address the causal inference problem defined in Section we treat Z as latent variable to be learned from the observed data distribution p(Xer, W Ysr) within a probabilistic model defined over the conditional responses as follows:
Assumption 3 (CMM(p): Conditional Markov Model of order p) We say that latent variable Z follows a Conditional Markov Model of order p. written Z ~ CMM(p), if there crists a fixed order p E N and a parameler vector such that the response distribution factorizes as:
T
polusT, z (gT,wer) = p(z) II
(ye 9t-let-p Wgr,)
For < 1, we set y = 0 by convention.
The fact that Z is set only with prior in Assumption 3 mimics the role of U because U consists of static adjustment variables that are parentless, as illustrated in the causal graph (Figure D.. The bounded memory assumption over the sequence of responses, that is, the direct causal effect of past. responses on future ones stops at an arbitrary order p. is a technical condition ensuring the causal validity of as in the Theorem where we demonstrate the sequential ignorability property when augmenting the history process with Z, replicating the same result as if the true adjustment variables U were available. All the proofs are deferred to Appendix A

---

Published in Transactions on Machine Learning Research (10/2025)
Z
Figure 2: A simplified causal graph for the sketch of the proof for Theorem 6 We do not represent Wgt, Xet for simplicity.
Theorem (Sequential Ignorability with Augmented History) Let Z be latent variable verifying CMM(p). Assume the response domain y is Borel subset of compact interval. Therefore, sequential igmorability holds when augmenting the history process with Z:
Ye(w) IL W. - h. 2 ((,,e, 2):
where H. represents the history process up to time t.
The ignorability result of Theorem | is the first step into answering the inference problem of as it allows us establish the identifiability the ACATE when Z replaces the true unobscrved variable Eq. summarized in the following corollary:
Corollary 5 (Identifiability of ACATE with Z) Let Z be a latent variable salisfying CMM(p). The Augmented CATE, when augmented with Z instead of U defined in Equation 4 is identifiable Specifically:
To (b, z) = E(Y, h. Z = z, W
B(G = h, Z = We N==10
(5)
As a result, Corollary 5 addresses the identifiability problem and conditions as raised in Section 3+ CMM of arbitrary order over the conditional distribution of responses, along with mild regularity condition the response domain as stated in Theorem 4 constitutes sufficient condition to ensure the identifiability of the ACATE. To further emphasize the relevance of Z ~ CMM((p, we show that any two substitutes satisfying (p) must necessarily be related through a measurable map given the entire process history Hr:
Theorem Let be latent variable such that Z ~ CMM(p), and assume the domain y a Borel subset of compact interval. Then, any static adjustment variable that influences the entire series of responses the panel must be measurable with respect to (Z, Hy).
Intuition behind CMM(p) We provide an intuition behind the CMM(p) assumption by sketching a proof of Theorem t using d-separation properties, assuming the data generation process follows the graph depicted in Figure ! Let Z' be an unobserved risk variable independent of Z, as illus- trated in Figure 2 Given that Z CMM and for
p, Y, and Y-- d-separated given (Y1-1:0-+11. Z. W W.,.. Xgr) implying that Y, IL Y,-m Y--x--m41 Z,
Now, suppose Z' influences the entire series responses similarly to Z. In this case, Z' acts as a common parent for both Y, and Y-m,, creating a path Y,m + + Y, that cannot be blocked by conditioning on (Y.-l:t-m+1+ Z. W W X Consequently, Y, K Y,m 1t-1ttm+11 Z, W St. X which contradicts the implications of the conditional Markov assumption.
4.2 Definition of the Probabilistic Model
We define in the section the probabilistic model learn substitute building on the DVAE approach. We break down the approach into three steps, where we first deal with selection bias and then incorporate the

---

Published in Transactions on Machine Learning Research (10/2025)
inductive bias of population structure, defining GMM over the substitutes and finally define the suitable Evidence Lower Bound (ELBO) to be maximized.
Step 1: Handling Selection Bias maximizing:
In a purely factual setting, modeling data with a DVAE involves
T
log logpo(UsT XgT,weT)
Σ log (ole Vee, Xen.x) log (3 (y,
based on the observed data distribution. However, this model cannot perform counterfactull regres- sion-switching treatment. assignment during inference to estimate counterfact.al responses does it allow for causal inference. This limitation arises in observational studies where treatment assignment is typically non-random and depends on observed covariates, resulting in sclection bias. We adopt the im- portance sampling strategy to reduce the selection bias by reweighing the likelihood of the factual model (Shimodaira 2000). The key idea to assign weights (a(h,,4)))? to units in the population such that the resulting distribution a given treatment regime matches that of the entire population. Therefore, we seek to maximize the likelihood defined as:
T
B.. [a(H,, W,) H.,,, log He, W.)]
A more convenient formulation expresses L expectation all repeated cross-sectional data of population:
T
L
Σ
Ed, [a(H, Wx) Po H4, W;)l
(6)
with Dr (Ygr, X5T, Wsr). ditional generative model po polyTT, I (oo(ye Xge,,, p()..
The formulation of a conditional DVAE model specifics the con- xgT, which factorizes po XgT,Wer)
Step Incorporating Population Structure To account for an eventual population structure induced by unobserved adjustment variables, we place Gaussian mixture prior over the latent variable Z, with K components:
K p(2) Σ (xN Ee).
(7)
This prior corresponds to first generating a cluster index Cat(r), followed by generating Z N(Jc, Ec). Here, π represents the probability distribution over cluster assigmments, verifying Σ = 1. with (T) denoting the categorical distribution parameterized by æ. Each cluster e is associated with a learnable Gaussian distribution of mean He and covariance Σg. The complete generative model becomes:
Po e XgT,Wer) II Polv yen Xgn. p)) c)p(c).
(8)
We approximate the true posterior over (z, c) using a factorized variational distribution
(z(z,c c Dr) = 90. (z Dr) (c Dr);
(9)
where Φ = &z Ug. collects the variational parameters. The following theorem characterizes the corresponding variational bound.
Theorem (Weighted ELBO Decomposition) Under the generative model defined in Eq. and the approzimate posterior factorization (Eq. ()), the weighted likelihood in Eq. [6) can be decomposed as:
I EDr ELBOO, Dr; 0, ED, (Do +)),
(10)
8

---

Published in Transactions on Machine Learning Research (10/2025)
where the individual gap term Ao is given by:
T
Do(Dr; Φ) Σ
a(hi, wi) Dxr, Dr) polz, c| [T)))
(11)
and the individual ELBO termed ELBOg is expressed as:
Ez Ez (1Dr) (arhe, wi) log has we, Z)]
(12)
Σ w() (DKIlGo. Dr) p(z)) + Ex (KD)) (go Dr) |l p(c Z))
The individual gap An is a positive term that quantifies the inaceuracy of the variational approximation of L using ELBO, (Eq. (12). The ELBO term, ELBO,, comprises three components: (1) A weighted
sum of the conditional log- likelihoods of the responses (reconstruction term). (2) A KL divergence between the approximate postcrior of the continuous latent variables 96. Dr), which serve as substitutes for adjust variables, and the unconditional prior p(z). (3) A KT. divergence between the approximate posterior of the discrete latents 4s.(- (- Dr) and the true conditional posterior p(C Z), averaged over go. (- Dr). The decoupling of these two approximate posteriors ariscs from the assumed factorization over the joint approximate posterior as in Eq. 9
Step Addressing the Discrcte Latent Variable The approximate posterior qs(z, e (D)) involves both continuous and discrete latent variables, z and e, respectively. Training deep generative models with discrete latent variables presents challenges due to difficulties in reparameterization and handling high car- dinality, even when using classical approaches like Gumbel-Softmax (Jang et al. 2017 Tucker et al. 2017 Huijben et al. 2022). To avoid designing an additional inference network for go. (c Dr)) and the associated computational cost, we follow Jiang et al. (2017); Falck et al. (2021) and define a Baaess posterior
for the discrete latent variable. Specifically, we leverage the decomposition of Eq. (9] and choose go Dr)
in such a way that the third torm the ELBO of Eq. 12] minimizod by dofinition:
min Ez~as, (11r)) D (9o. | DT) || p(c Z)). (-Pr)
(13)
The minimal value of this optimization problem is
min (1p,) (s.(1Pr))
((1,) Dri( (T)) || p(: Z)) - log Const(qe. (- [T)),
(14)
where Const(gs. (-1 Dr)) is a constant that depends only on the approximate posterior over the continuous latents:
Const(as.() (1 exp Rz (Dr) logp(c Z)).
The minimizer is given by:
exp 2 go, (112,) log Z))
(ec go. (.1
(15)
Cst(1s. (: (T))
Moreover, the minimal value is:
min E Ez (1Dr) D Drr(gs. DT) p(c Z) log Z(gs. (. (-rr)
(16)
Thus, we update the original ELBO, in Eq. 12 to obtain a modified ELBO by replacing (: Dr) with the minimizer TCc] |g. (D)) and plugging in the minimal value from Eq. (16):
T
BL: Ezzge, (1Dr)) (a log Hr, W., Z)
(17)
Σ (())
( (Killeo ( Dr) p(z)) - log Z(gs. (- DT))) (
9

---

Published in Transactions on Machine Learning Research (10/2025)
As result, the modified ELBO is computationally more efficient than the original one, it bypasses the need for applying a reparameterization trick to discrete latents and eliminates the necessity of training over specific parameters de. all while remaining a valid lower bound for the weighted likelihood L. This validity holds since Theorem 7 applics to any arbitrary approximate posterior qo. Dr), and in particular, to (e Uo. (- Pr)); which the minimizer of Eq. 13 We henceforth use & to denote d..
4.3 CDVAE in the Near- Dear-eetemiiiitic Regime
In this section, we explore the behavior of Causal DVAE in the near-delerministic regime, where the variance of the generative model approaches zero, We examine how this regime affects both inference and causal identifiability, demonstrating that it leads to a non-diffuse posterior suitable for treatment effect estimation.
Model Setup For the remainder of our discussion, we define the DVAE model parameters as Maae {8, } and assume the approximate posterior to be Gaussian gp(z) Dr) N(zz Σz), where the mean is defined as Pl fre (Dr, σ), and the covariance matrix is given by Σg S2S, with S2 (Dr, Φ). Morcover, we assume the conditional generative model to be Gaussian such that:
Po(ee Mecs Mct, Xgr, z) = (fi we, z; 0;
(18)
with a > 0, spatially and temporally uniform scale parameter.
Motivation In probabilistic modeling, our inference model for the substitute adjustment variable is stochastic. However, to calculate an individualized ACATE for each individual i, that is
Tha 2.) = B(Y, Hu -ha Z = z, W 1) - B(Y, H. = hg. Z = 2, We = 0),
we need to choose a unique instance of the latent variable representation. Otherwise, infinitely many ACATEs for the same individual i could be generated by sampling repeatedly from (. Dir). A natural choice for this deterministic representation is the mean of the posterior. Yet, because we sample from the approx- imate posterior during training rather than using the mean, is essential to prevent the posterior from becoming overly diffuse-cnsuring that the mean remains valid representation during causal inference. Previous studies (Dai Wipf, 2019 Takida et al. 2022) show that operating in the near-delerminislic regime of the VAE decoder where a + 0+ leads the encoder's covariance matrix toward zero, resulting in near-deterministic behavior. We, therefore, explore the near-deterministcc regime in our Causal DVAE by controlling the decoder's variance to induce a non-diffuse, stable posterior distribution suitable for causal inference in the latent space.
Side Benefit: Preventing Posterior Collapse A well-known challenge in training VAEs is posterior collapse, where the approximate posterior converges to the uninformative prior, making the latent. space irrelevant (Bowman al. 2015 Sonderby al.) 2016 Higgins et al. 2017 Dai & Wipf 2019 Fu et al. 2019 Lucas et al. 2019: Wang et al. 2021 Takida et al. 2022h. A notable advantage of the near-determiiistcc regime is its ability to avoid posterior collapse within the latent space, benefit. demonstrated in static VA (Lucas et al. 2019 Takida et al.) 2022). Specifically, Takida et al. (2022) showed that the decoder's output variance and covariance could influence the latent space by causing over-smoothing by affecting the gradient regularization strength, which in turn leads to posterior collapse. By controlling variance in the near-deterministic regime, we mitigate this collapse and thereby presorve meaningful latent representations.
We advance the study of dynamic VAE models by investigating their behavior in the near-deterministic regime within the context of causal inference. This work is the first to explore dynamic VAEs in this regime for causal applications, addressing gap in the literature, as such behavior has yet to be thoroughly examined even in non-causal, factual settings. For the remainder, and unless otherwise stated, we omit dependence on the individual i for notational ease.
We prove that in our framework, replacing the original lower bound ELBO, 0 Φ) with the modified ELBO(Dr;; ø) still ensures the existence of parameterization under which the approximate likelihood of responses converges to the true likelihood. Specifically, Theorem demonstrates that, for a sequence of
10

---

Published in Transactions on Machine Learning Research (10/2025)
inference and generative models parameterized by o, the gap between the modified and true importance- weighted likelihoods asymptotically approaches zero.
Theorem 8 (Asymptotic Likelihood Recovery) Suppose d2 S T. There exists family of aulvencoders {6o, Og Joso such that
lim po, (yST xgn, p(ySr xgr,Wer), a-0+
and when operating under the Bayes-optimal posterior over the discrete latents, i.C.,
go(e) Dr) = [<e go( Dr));
the gop A(DriOn 0a) between the modified ELBO (Eq. 17 and the true weighted likelihood (Eg. satisfies
6
(DDr 0.) = ELBO 0o) L(Dr;0o)
and converges to zero:
lim A(DriDa,0a =0. a=
This result generalizes prior findings for static VAFs DDai Wipl 2019 to the more complex dynamic setting with GMM priors and weighted objectives. also addresses an oversight in Dai & Wipf (2019) regarding the proof of gap convergence.
More importantly, we establish a critical property in our setting within the near-deterministic regime: the causal effect remains asymptotically consistent whether one uses an arbitrary realization of the latent variable & or the posterior mean fxx- Theorem g formalizes this insight as follows:
Theorem (Realization-Invarinn Causal Consistency) For a fixed 0, let parametrization of
Causal DVAE M (a) {#", 6% be an optimal solution to the ELBO defined in Eq. [17]. Then, for any
substilute realization z Rd: the conditional expected outcome satisfies the following consistency in the dear-deteriiiisticc regime:
lim (M) [Y Hr, W, w, w.Z = lim EM [Yg H2, We w. Z [2] a
(19)
o-0
As a result of Corollary the Augmented CATE satisfies the same consistency in the near-deterministic regime:
lim TM (z) (H, = lim TM (o) Hz).
(20)
0
o-0
Theorem g demonstrates that as the variance parameter of the generative model approaches zero, both the expected outcome and the estimated causal effect remain consistent, regardless of whether one relies on an arbitrary realization of the latent variable or its posterior mean μ2. Such consistency ensures that, in the near-deteriinistcc regime, the causal effect is robust the specific realization of the latent variable.
4.4 Generalization Bound over Treatment Effect Estimation and Derivation of Model Loss
We establish in this section theoretical generalization bound for the error in estimating ACATE, which serves as foundation for deriving the training loss of CDVAE. We begin by defining a suitable represen- tation function to address the high dimensionality of the history H,, Next, we introduce a weighted risk minimization framework that ensures covariate balance between treatment groups. We then derive a gen- cralization bound for treatment effect estimation, relating the error in estimating ACATE to the model's weighted risk. Finally, we construct the CDVAE training objective by integrating these theoretical insights into structured loss function.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Step 1: Representation Learning To address the high dimensionality of the history H4, define representation function & that reduces dimensionality while preserving essential information in latent space. Assuming & is invertible, we ensure sequential ignorability holds in the latent space if and only if it holds in the original space. In practice, the outcome and treatment models share the same representation ((H,), capturing information predictive both treatment and response. While invertibility theoretically not strictly enforced in implementation. Instead, we prioritize ensuring (H,)) remains predictive across treatment regimes, applying regularization techniques balance representations
assumed for identifiability,
in Shalit et al. (2017): Lim 2018 Bica et al. (020a Assaad et al. (0211 Melnychuk et al. (2022)
Johansson et al. 2022
Remark 10 Following Eq. 18). the generative model for the responses is defined Y, Se W,, Z + Es,
where 4 (0, is the error lerm. A common modeling [Johansson el al. 2016 Shalit el al. 2017
Shi
et al. 2019 Johansson et al. 2022) approach for ft is to assume
fHH W., 2) = Wefe, Z) + +1- (H,,
where fro and fe,1 denote the mappings to the response under each treatment regime W, 0,
The representation function & over (H,) produces domer-dimensional vectors (r:)ī=1> living in a space
R C R,, Instead of defining deeendepnndent generative functions ft, we define a single hypothesis function
XxRxWRR< + +
such that:
Y = f({(H), WT, + Wifi (..,( Z) + (1 - (.)) Ec.
Step 2:
Weighted Risk Minimization To integrate causal inference into the DVAE framework, we
employ weighted risk minimization techniques (Kallus 2020 Johansson et al. 2022 First, we justify the choice of weights aHH,, W2) used in the definition the weighted ELBO in Section 4.2 Eq. general, these weights can be any arbitrary mapping a H, x W - I&+ such that for every w E W, |H, [w] 1. The weights a a(h,,,) w) induce, therefore, a weighted probability target distribution g over the population s.t g(h, W = w) (phh, We w). Choosing an appropriate weighting strategy a(h;, w) is essential for achieving covariate balance between treatment groups, specifically g(h,
117
W; = 1) = g(b, W, = 0). We use overlap weights [Li et al. 2018 Assaad et al. 2021). defined as:
(b)(h))) e(o(h:)) (1 (1-((:))) w w + +1 w) (1 (1-e(((h))))
w( x
where (b,))) = p(W, = 1 is the propensity score within the representation space. Overlap weights emphasize units with propensity scores near 0.5, concentrating on regions where treated and control groups have the most overlap, thereby enhancing comparability and improving covariate balance (Li et al.) 2018).
We now define the weighted population risk for given treatment regime We = w, measuring the expected risk for given (f, b) at time & and over the weighted population distributed according to g, as:
Ry (f. 2) = EH,,,, [aH,, Wx) f,o (H,, W.) | W. = w],
where f (h,, w) is the expected pointwise loss associated with the hypothesis f:
(hi, w) = (2 Ex, ( ( W.)) H, H; 11, h, Z).
(21)
In the definition of the expected pointwise loss, we respect the temporal order by conditioning the approx- imate posterior of the continuous latents only on the longitudinal data up to time step t. In line with our probabilistic modcling in Scction 4.2 and the distributional assumptions in Scction 4.3 we define the loss
the negative log-likelihood:
L (Yi(w), (((H,, Z, W.). log.V (Yr(w):/ (HHH), Z, Z.).) ,0
12

---

Published in Transactions on Machine Learning Research (10/2025)
Step 3: Generalization Bound Our aim during training is to minimize the factual quantities [Ri (J, +))== to reduce the error in estimating treatment effects. To formalize and theoretically justify this objective, we establish a generalization bound for the Precision in Estimation of Heterogeneous Effectt (PEHE) Hill 2011), which accounts for our weighted population risks { (f. ff 1) PEHE mcasures the mean squared error (MSE) between the true and estimated ACATE at time as
H. Z) - - (He 2)) СPEНE, = EH, Bz (Z|Pg() (He
where (H, Z) is the true ACATE and (H,, Z) is the estimated ACATE. Similarly, we define the weighted PEHE relative to the target distribution g:
b = BeHing---oo 2o Z g)-)) (H, 2) (H,,
2))²
We provide an upper bound for the weighted PEHE for CDVAE similar to Assaad et al. (2021). This bound consists of three key components: the weighted factual prediction error, term capturing the discrepancy be- tween treatment and control distributions in the confounders representation space, and term that accounts for the variance of the generative model over responses.
Theorem 11 (Generalization Bound for Weighted PEHE) Let MvAE(O) (0,.0o) be the VAE model defined in Section 4.4 For given class of functions G, assume there erists constant Ba such that 4.a Bo E G. Assume the representation & is invertible. Then, the error in eslimating the treatment effect at time for weighted population is upper bounded by:
EPEHE 200² { (f. Rig (f,0 + B& IPMo (go (.1 1), g W, 0)) log((2T0))) (22)
where go (r) is the distribulion in the representation space R. The Integral Probabilily Metric (IPM) Müller 1997 Sriperumbudur et al. 2009) mensures the dissimilarity between distributions.
Assuming strict overlap in treatment assignment, where E (0,).5) such that 5 < e(h,) < 1 - then there exists constants At.g: Br,g such that the unweighted PEHE εPEНE, is bounded:
Atg EPEHE.. (f) S EPEHE, (f) S Br,g EPEHE,.. (f).
The representation discrepancy in 22 quantifies the imbalance between treatment groups. measured using TPMs like the Wasserstein distance Maximum Mean Discrepancy [Gretton et al. 2012). The connection between ÉPEHE, and εpEHE,,, indicates that minimizing the weighted PEHE can also minimize εPEНE, thereby enhancing the reliability of CATEs estimation across the original population.
To justify the pertinence of our probabilistic model defined in Section 4.2 we will show how maximizing the likelihood term the ELBO of Eq. implies minimization of the risks 1 (f. 2) and R (f, (f.
Proposition 12 (ELBO-Risk Connection) Assume stationarity of the approzimated posterior of given sub-longitudinal data; that is, there crists to such that for t 2 to, go De) x qo(Z Dr). Given finite sample B = (Der = (Wa, Yu, X =1,........ 1.....|[]] [B}, we have the following approzimation:
E Dr) (a (He, Wi) log (Y(: H,, W, Z)]
1
Σ
(8) Ri. (f. + no R (f.4)}. (23)
B
with
Ry (f, +)
Σ
Ez~go (2)pa) (a. w) log Poggo Ha,,, w, Z)],
The
iEB, WiTE
where 1 represents the number of instances in the batch B for which Wa
Note that po (He, 2) p(Z 1 (H,) because is d-separated from Z given Di = VY X X Ws
13

---

Published in Transactions on Machine Learning Research (10/2025)
As result, maximizing the ELBO in Eq. [17 not only maximizes the weighted conditional likelihood over responses (the 'reconstruction term") but also, shown in Eq. [23 minimizes the weighted population risk. This, in turn, decreases the generalization bound in Theorem effect estimation. To further reduce this error, we include the IPM term from Theorem regularizing the WELBO with the IPM term each time slep:
reducing the error in treatment
LIPM (O,, 1) IPMa (g9 ( W. = 1), 90., 00,
W. = -)).
where Ow are the parameters of the trealment network.
Intuition: Stationarity of the inferred latent variables We further justify the core assump- tion made in Proposition 12 The representation learning for the adjustment variables is achieved by approximating the posterior po(z)Dr) with gs (z))) Here, the missing baseline covariates are in- ferred by analyzing all longitudinal data {VST, Xgr. However, since these covariates are static and pre-response variables, we should, ideally, be able to infer the exact substitute from a shorter longitudinal data with /2 41 Thus, ensuring form of stationarity in our posterior approximation is crucial. To understand the importance of this property, consider estimating the Average Treatment Effect (ATE) for panel data at time t:
XST", where T" < T, or from any temporal slice (yt,:ta Xtata Wt
T = (()(1) - Y()) Ez,H, By, w. D Z, H4 W. 1] By, [Y, H, W, o]]
To compute 7r, need marginalize the conditional response over the joint distribution of covariates and latent adjustment variables, p(h,, z) p(z Dst)p(h,). Because we model pa (z (D), which depends on the entire history, there is, generally, no guarantee that p(z h₂) = (o
As a consequence of the stationarity assumption in Proposition 12 we introduce a penalty term in the form of the Wasserstein distance between consecutive posterior distributions qo (z/D,) and (z(2DD,,),, penalizing significant variations the data history grows. To preserve the model's capacity to capture time-varying dependencies, we begin regularization from to > 0, typically to = T.
T
( = Σ Ep. W (go(z) Dz), 90(z) Dz-1)).
Assuming Gaussian approximate posterior, the Wasserstein distance simplifies to
T
(D) = Σ Ep.llu((D,)
Ep,1 ( (D (D -
1=to
where the penalty regularizes the first and second moments of the posterior distribution as the data history grows.
Step 4: Total Loss Function and Training Strategy The overall loss for the CDVAE model combines the ELBO from Eq. (17 with two additional terms: LIrM, which reduces covariate imbalance at cach time step and addresses residual imbalance not corrected by weighting, and LDISM, which captures the global, static nature of adjustment variables. To push the Causal DVAE toward a near-deterministc regime, we treat the variance parameter learnable and fit it using the following loss function:
Lior fw, á, 4b a) = ELBO Ow, o, T..a) + AurM +) + ADistM (DiaM(
(24)
Since the loss Loot depends on the treatment parameters 6., as the weighted ELBO also function of the propensity scores, introduce an additional loss function Lw, binary cross-entropy loss to predict
14

---

Published in Transactions on Machine Learning Research (10/2025)
treatment from confounders representation. To optimize CDVAE, we adopt an adversarial training strategy, simultaneously minimizing Liot with respect to 0, 20000 t and 0, while minimixing Lw with respect to Ow (Algorithm 3 Appendix
ming 0,0 0 Liot (0, Ow, 2 P,0), mino. Lw (0w, D).
The adversarial nature of the training arises because, in one optimization step, we learn the representation & to predict the response while applying an IPM regularization to ensure balance between the weighted covariates. In the subsequent optimization step, for fixed representation, we train the treatment classifier to predict the treatment. We use the Wasserstein distance as specific case of IPM, with details its computation provided in Algorithm 2 Appendix E.1 A computational complexity analysis of CDVAR is also provided in Appendix E.2
CDVAE: Model Specification We use three-head neural network architecture for CDVAE to learn a shared representation between outcome and treatment models inspired by Shi et al. (2019). This architecture consists of two prediction heads for potential outcomes and one for treatment (Appendix G We use a GRU Cho et al., 2014) to learn a representation (()+1) of the context history ha+1- The input of the outcome model fe, is the shared representation ((b+1)) and a sampled substitute from g0(z Dr)). The outcome model Ja, is represented by two non-linear functions Sez and Sey for treatment. assignments 1 and 0, respectively. The third head is a binary classifier fa,, estimating the propensity score. The inference model (encoder) 2o (z VST, XgT, (gT) is modeled by GRU, producing a hidden state that represents the sequence. Two non-linean mappings, Péz and Eds: learn the mean and diagonal covariance matrix in the latent space.
5 Experiments
Baselines In all our experiments, we compare CDVAE against the relevant baselines: RMSN, CRN, G.eet, Causal Transformer, and Causal CPC. Each baseline is evaluated in three different settings: 1. The 'base approach," * where models are trained using only static and time-varying confounders. 2. The "substitute approach," where models are augmented with substitutes for the unobserved adjustment variables, obtained from CDVAE and represented by the mean of the approximated postcrior. 3. The "oracle approach," " where models are trained with the true adjustment variables.
Hyperparameter Selection All models are fine-tuned using grid search over hyperparameters, in- cluding architecture and optimizer settings. Model selection is based the mean squared error (MSE) of factual outcomes on a validation set, which is also used as the criterion for early stopping. More details are in Appendix F
Adapting Baselines to Our Causal Settings All the considered baselines do not assume the obser- vation of adjustment variables; however, they only consider time-varying and static confounders. is not straightforward to incorporate adjustment variables into these models when observed. Naively augmenting their input with adjustment variables and treating them as static confounders often results in suboptimal performance or even worse outcomes compared to excluding the adjustment variables altogether. This is primarily due to two reasons: 1. Most baselines concatenate static confounders with time-varying ones cach time step. When high-dimensional adjustment variables are included, this dramatically increases the number of parameters, leading to overfitting. 2. When comparing different versions of the same baseline, it is important to ensure comparable complexity represented by the number of learnable parameters (Table 3. Models such as CRN, Cansal Transformer, and Causal CPC learn a balanced representation. Including adjustment variables naively as input may bias the learned representation towards these variables, as they are inherently balanced.
15

---

Published in Transactions on Machine Learning Research (10/2025)
Model
Params (R).
Pazamu (k).
To ensure fair experimentation, we adapt the baselines as follows:
syathetic deta
MIMIC-II
dala
1. CRN, Causal Transformer, and Causal CPC: Instead of solely feeding the balancod representation the outcome network, we augment it with static adjustment variables, similar to the CDVAE design, where the representation ((H)) is augmented with substitutes Z.
Causol CPC (base)
6.7
5.3
CPC (with subeti-
tule)
Cansol CPC (oracle)
14.3
CT [bume]
18.7
16.3
CT (with subotitute)
16.7
CT (oraele)
18.7
G-Ner (hase)
21.2
G-Net (with subattatuee)
21.5
4.3
2. GNet: Adjustment variables are only used to aug- ment the representation fed to the outcome model, not the represenlation used to reconstruct the current co- variates X..
G-Net (oracle)
28.5
5.7
CRN (Exeee
3.5
CRN (with aubstiture))
8.5
3.7
CRN foracle)
RASN (fuse))
14
6.7
RMSN (with aubanituse)
15.3
RMSN (ocacle)
20.9
8.0
3. RMSN: Stabilized weights are learned solcly function of He, while the adjustment variables catenated with the representation of H,
Trainable parameters in thousands (k) for basclines in all configurations for the syn- thetic and semi-synthetic MIMIC-III data.
Table 1:
5.1 Synthetic Data Sets
Generation We simulate longitudinal data with a time length of T = 75 using an autoregressive model for both confounders and treatment assignments. The confounders X, have dimensionality de = 100. The outcome model includes unobserved adjustment variables U, with dimensionality d, 100, generated from Gaussian mixture model. A detailed description is provided in the Appendix C.1.1 The experiment is conducted with 5000 samples for training, 500 for validation, and 1000 for testing.
Controlling the Effect of Unobserved Adjustment Variables To assess the impact of unobserved variables U, we vary the parameter (1)) (Appendix C.1.1 which determines the contribution of U in generating (1)). By design, Y1(1) increases with YY ((1) and the parameter ((1 is varied within the range [0,2.5] with a step size of 0.25, and we generate a longitudinal dataset at each level of YX We report the mcan and standard deviation over 10 different seeds of PEHE in estimating the onc-step-aleead ACATE, i.c., EPEHErT1: over the test individuals.
The Figure 3 shows the performances of all models given the three possible configurations. First, the error estimating the ACATE increases substantially as the coefficient YYU increases because the contribution of the unobserved static variables becomes considerably more important in the writing of the potential outcomes. However, CDVAE notably displays slower incrcase in crror and superior performance in all data configurations. Second, the errors decrease across all levels of unobserved heterogeneity when the baselines are provided with the substitutes learned features by CDVAF, highlighting the superiority of the substitute approach assisted by CDVAE over the base one. As expected, a baseline in the oracle approach generally performs better than with substitutes, especially at higher values of YU while the substitute 7() approach provides near-oracle performance. Extended results are provided in Appendix C.2 which we complement by stressing performances by varying soquence length T and number of simulated vitals dz A Appendix C.3
Results
5.2
Semi-Synthetic MIMIC-III Data
We adapt to our setting semi-syuthetic dataset constructed by Melnychuk et al. 2022 based the MIMIC-III datasct (Johnson et al. 2016). The covariates are high-dimensional and include several vital measurements recorded in intensive care units. We investigate the effect of vasopressor trealments on blood pressure (response) for patients. The static covariates the dataset (gender, ethnicity, and age) are included in the data construction as adjustment variables for the response. Detailed descriptions are provided Appendix [.I] We conduct our experiments with 1400 patients for training, 200 patients for validation, and 400 patients for testing.
16

---

Published in Transactions on Machine Learning Research (10/2025)
2010
PEHE (Moan Std)
40
PEHE (Mean Std)
17.5
CTAME
15.0
CENWE
10.0
7.5
25-
0.5
1.0
1.25 values
valuess
(a)
Performance of Causal Transformer with CDVAE
(b) Performance of Causal CPC with CDVAE Compari-
Comparison
som
base S.betBuuee cracie CUNAE
bast 4BotTue
PEHE (Mean 5td)
PEHE (Mean # (tdd
orack: COVRE
42-
32-
2:
0.25
0.5
3.25
values
Via values
(c) Performance of CRN with CDVAE Comparison
(d) Performance of G-Net with CDVAE Comparison
subidiuute
PEHE (Mean Std)
placke CDvl
1.25
1.5
1.75
2.0
2.25
2.5
Vil
(e) Performance of RMSN with CDVAE Comparison
Figure 3: Evolution of PEHE in estimating ACATE for synthetic data across increasing levels of heterogeneily induced by adjustment variables U.
Similar to the synthetic data experiment, we evaluate all baselines using three approaches (base, with substi- tutes, and oracle), and report the PEHE in estimating the one-step-ahead ACATE for a patient trajectory.
Results Figure 4 reports the mean and standard deviation of PEHE for CDVAE and all basclines across all configurations. Similar to the synthetic data results, the base approach always performs worse than the substitute and oracle approaches. However, the substitute approach substantially enhances performance compared to the base approach, highlighting the relovance of the substitutes learned by CDVAE. Across all configurations, CDVAE achieves the best performance compared all baselines. Extended results are provided in Appendix D.2
17

---

Published in Transactions on Machine Learning Research (10/2025)
Model
20.0
Gausd Trandformer
PEHE (Mean * Std)
19.5
G-Net
CRU
RMEN
19.0
17.5
base
substitute Configuration
oracle
Figure Results on the MIMIC-III data reported by PEHE and organized following the three possible configurations. Smaller better.
6 Discussion
Ablation Study To experimentally validate the design CDVAE, we perform an ablation study
by exploring several model configurations: the full CDVAE model with all components; CD-
VAE without the IPM term (Λди 0): CDVAE without the distribution matching term
(DDistM 0); CDVAE without both the IPM term and distribution matching term (AırM
O, ADistM 0); CDVAE without the IPM term and weighting (ΔrРм 0, w/o weighting); CDVAE
(ΛIРM = ADISMM = 0, w/o weighting); and CDVAE with a fixed variance parameter (a 1).
We follow the same experimental protocol as in prior experiments, conducting the ablation for the synthetic dataset across heterogeneity levels in- duced by YUU and for the semi-syuthetic MIMIC-III dataset. Across all configurations, the full CDVAE consistently outperforms the other models. Fur- thermore, removing individual components results in progressively higher errors, as shown in Tables and 2 Interestingly, the configurations (ΛιРм = 0, DistM = 0, without weighting) and (a = 1) either perform comparably or worse than the second-
CDVAE Coufigurasion
PEIE
Ours
17.63:2.25
App
18.3810.26
Auiess
18.21
=
20.312 B4
ALFM D, Apunt
18.45=0.23
AHnm w/o weighting
18.97:0.28
(Aips 0, Apias
O, w/o weighting)
19.23+0.28
Table 2: CDVAE ablation study with semi-synthetic MIMIC-III data reported by PEHE. Smaller better.
best model, Causal CPC, on both the synthetic dataset and the MIMIC-III dataset.
Table Results on the synthetic data reported by PEHE, Smaller is better.
CDVAR Consggaation
10
7(1
10
10
100
1.25
700
1.75 100
1(0)
Y20
700
Ours
0.43-0.02
0.50+0.00
0.96++0.09
1.57-0.08
1.90+0.10
2.35++0.18
3.57+0.17
4.80+10.20
6.84+0.20
7.64-0.48
9.03+0.50
0.06:0.00
2.5310.15
Ausin -
0.96.10.02
0.87.0003
1.101.0.222
1.78.00.13
2.1810.13
26610.18
3.95.10.15
5:20.10.12
7.1410.113
S.08.00.14
9.25.10.19
0.91+0.40
0.66+0.07
2,17+0.53
5.73+1.51
10.67+2.02
12.06++2.16
14.89-1.25
15,03+4.04
18.23-6.71
17.1945.11
Aps 0, Apuess
0.68+0.02
0.57:0003
1.16+0.07
1.84:0.11 17
2.53+0.12
2.66+0.14
4.05=0.100
5.36-40113
7.48+0.12
8.20:0.11
9.80+0.21
Appa w/o weighting
2.6.10.15
5.30.10.00
Xpp 0, Amwas w/o weighting 0.30.00.2
0.62.10002
1.2710000
1.98.10.10
2,3910.09
2.8510.14
4.1911.133
5.59110.13
7.6110.15
$:1110.14
9.78..1.1
How CDVAE handles autoregressive order in practice. We require CMM(p) ensure identifia- bility of ACATE and validity of learned substitutes. In practice, do not fix lag p but rely two
18

---

Published in Transactions on Machine Learning Research (10/2025)
mechanisms: 1. Fading memory. Recurrent predictors and oading-memoll filters down-weight distant history exponentially. Classical results show that fading-memory maps can be uniformly approximated by fimite-memory models: for any E > 0, there exists an order P. such that conditioning on the last lags is 6-close to conditioning on the full history (Funahashi & Nakamura, 1993; Miller & Hardt 2019 Gonon &
Orlega 2021). Bias from training. Backpropagation through time biases RNNs Loward short meeiium context, with little gain from longer lags (Khandelwal et al. 2018).
To empirically substantiate these claims, train CDVAE synthetic data generated with Purue 10. On test data, we compute counterfactual responses while masking outcomes to simulate CMM(p) with P {4,6, 6,6,8 8, 10, 12, 14, 16]. One-step residuals (conditioned on H, and Z) are tested for autocorrelation using Ljung Box. Persistent autocorrelation or steadily improving accuracy with higher would suggest violation of CMM(p). In all cases, Ljung Box p-vauues up to lag 20 remain well above 0.05 (Table 5 e.g., p = 4: 0.085; p= 10: 0.109): so residual autocorrelation not rejected. Meanwhile, one-step RMSE (~ 1.129-1.131) and PEHE (~ 0.975-0.986) are nearly flat across p (Table 4 Thus, conditioning on short chort/medium windows sulfices to remove serial dependence: increasing p beyond 4 6 yields no material improvement. This agrees with (i) portmanteau tests showing no residual autocorrelation up to lag h, and (ii) the fading-memory memory approximation theorem together with the empirical bias of RNN training toward recent history (Table 0
Metric
p = 4
6
p
8
p
10
p
12
p
14
p 16
RMSE 1-step
1.131
1.130
1.130
1.129
1.129
1.130
1.130
PEHE sssep
0.977
0.984
0.986
0.986
0.982
0.975
0.982
Table 4: Ono-step accuracy vs. masked autoregressive order p.
Metric
4
6
p=
8
10
p=
12
14
16
mi p-value # lags 20 with p < 0.05
0.085
0.083
0.097
0.109
0.095
0.079
0.111
0
0
0
0
0
0
0
Reject any?
No
No
No
No
No
No
No
Table 5: Residual whiteness vs. masked order p (Tjung- Baagg-Box, joint up to h = 20).
p
Q(1)
P(1)
Q(5)
P(5)
Q(10)
p(10)
Q(20)
p(20)
4
2.963
0.085
7.442
0.190
10.812
0.372
12.111
0.912
2.996 6
0.083
7.442
0.190
10.725
0.379
12.067
0.914
8
2.755
0.097
6.669
0.246
9.892
0.450
11.483
0.933
10
2.566
0.109
7.012
0.220
9.927
0.447
11.213
0.941
12
2.788
0.095
6.825
0.234
10.289
0.416
11.653
0.927
14
3.077
0.079
8.189
0.146
11.694
0.306
13.236
0.867
16
2.533
0.111
7.135
0.211
10.652
0.385
11.930
0.918
Table 6: Ljung-Bor Box statistics (Q) and p-values at selected lags l E {1,5, 10, 20).
On the Near-Deterministic Behavior of CDVAE For the theoretical results Section 4.3 to hold, it is necessary to verify whether the variance parameter 0, as a Icarnable paramcter, indeed decreases toward zero during training. In all experiments, a is initialized at one, and we show in Figure 5a its behavior during training on the synthetic dataset across all levels of YYU well as on the MIMIC-III dataset in Figure 5b| In all cases, the variance parameter stably decreases toward zero, validating the near-deterministic regime at the end of training. This supports the consistency result related to the treatment effect established in Theorem g
CDVAE Sensitivity to unobserved confounding To assess the sensitivity of CDVAE when sequential ignorability is violated and when adjustment variables U are unobserved, we mask four confounders from
19

---

Published in Transactions on Machine Learning Research (10/2025)
1.0
-0
1.0
MIMIC III
54
V1-15
11--1355
V2-22
Y12 -2
52
Iteration
Iteration
(a) Synthetic data.
(b) MIMIC III.
Figure 5: Evolution of variance parameter update during training for synthetic data (left) for cach level of YU and MIMIC-II (right) averaged over 10 random initializations ((
the mimic experiment during model training, which are sodium, Glasgow Coma Scale total, cholesterol, and hemoglobin. We repeat the same experimental protocol as in the experiment Figure | and show Figure the results of all models under the three configurations (base, substitute, and oracle). As expected, PEHE increases for all models across the three configurations, with average increase of 2 points in PEHE, which is at least an 11% increase. CDVAE still outperforms bascline in base and substitutes configurations; however, baselines provided with true adjustment variables either provide comparable performances to CDVAE or slightly outperform (eg, CRN).
For synthetic data, we perform sensitivity analysis in the following way: We exlend the simulator by partition- ing covariates X; = [Xq. X into observed ones Xς and hidden ones X and summarizing the contribution of X into hidden score. Sensitivity to unobserved confounding is controlled the selection side by adding S, to the treatment logit (Rosenbaun-I style (Rosenbaum & Rubin 1983 and the outcome side by arm-specific loadings S, in the notential-outcome equations. Setting
S0 = 3 = 0 recovers the baseline DGP. Varying yields families of datasets with controlled degrees of hidden selection bias and hid- den effect modification by unobscrved confoundrr (cf. Appendix C.1.2). Figure shows how CDVAE, along with baselines the "base" approach, degrades performance (PEHE) as confounding level T increases in (1,1........ 5]. CDVAE still provides globally lower PEHE, but the performance margin narrows as I increases.
CEVAE
ERN
Gettt
Cau
RMSN
Caunal Tharefeemer
PEHE (Mean = Stdi
3.5
2.5
3.5
4.0
4.5
53
Figure 6: Sensitivity analysis on the synthetic data. We report the variation of PEHE under different levels
of confounding T for CDVAE and baselines under the "base" approach. Mean and standard deviation computed for 10 different seeds. Smaller is better.
20

---

Published in Transactions on Machine Learning Research (10/2025)
Modal Old for ghall
Reb (oummet)
Cottial CPC
CRN IMCN
PENE (Meann Std)
20
I Ffffi
suboitute Configuration
Figure 7: Results on the MIMIC-III data under violation of sequential ignorability. For comparison, results under sequential ignorability of Figure are included in shadow. We report PEHE, Smaller better.
CDVAE Robustness to the Number of Components in the Prior We demonstrate that the number of components K in the Gaussian mixture prior does not substantially affect the PEHE. The chosen value, selected via random search,
= 7, while the ground truth is K = 8. To test robustness, we evaluate CDVAE with values below and above this baseline, namely K 2, 5, 8, 11. For the semi-synthetic MIMIC-III dataset, the true number of components is unknown, and the value selected via grid search is K 5. To test robustness, also evaluate K 2, 8, 11, 14. Figure 9a shows the evolution of errors for the synthetic data across levels of ((1) for different values of K. While there are slight differences in performance means for CDVAE with varying K, almost all error bars overlap across all levels of 7 The same observations hold for MIMIC-ITI, as depicted in Figure 9b This highlights the robustness of CDVAE to the choice of K for the prior and its low sensitivity compared to the components analyzed in the ablation study.
Do the learned substitutes Z better capture the clustering structure in U (population struc- ture)? We assess this by computing the mutual information (MI) between learned representations and the true U-cluster labels across the sensitivity sweep I' E 11,1.,.... 5} (Figure S First, we notice that the MI for the baselines positive. This is expected given the causal structure: the unobscrved U affects the entire response series, and observed outcomes serve therefore as proxies for U, so encoders that use past responses inevitably preserve some cluster signal. CDVAE more efficient because learns dedicated substitute (the variational posterior) and, under CMM(p), explicitly exploits outcomes as high-signal proxies for U, thereby better capturing response hetcrogeneity driven by latent variables. The downward trend in baselines with larger I is expected: we hold the contribution of U to Y, fixed while increasing r, which amplifies the influence of missing time-varying confounders, making outcomes less diagnostic about U. CDVAE nevertheless retains more MI because it learns a dedicated, sequence-level substitute Z (near- deterministic variational posterior) that globally modulates the trajectory and is fed only to the outcome head while baselines basolines-though they inherit some MI via past outcomes acting as proxies -lcck lack this explicit latent channel and therefore capture less of the population structure.
Bayesian Model Assessment of CDVAE We assess the quality of the conditional response fitting in variational framework through posterior predictive check (Rubin) 1984 Meng 1994 Gelman et al. 1995) and similar to Bica et al. (2020b); Hatt & Feuerriegel (2024). For cach time step f, let y = (Vi Fiz represent the observed responses in the validation dataset. We generate S replicated datasets of responses y (s) = [,, rep (ay for = ,,...,S, such that for each individual in the validation dataset, we draw S samples of latent substitutes (1),,. ...,. (S) ~ q-(- (D) The approximate posterior serves as proxy for the true (inaccessible) posterior p(z D). Replicated responses are then generated from the fitted conditional distribution: (s) yet, X Uge, 2, (s).
21

---

Published in Transactions on Machine Learning Research (10/2025)
(Mean Std
Transforerr
CRN
0.6
Figure 8: Mutual Information between learned representations and true cluster labels in under a sensitivity analysis to missing con foundingg controlled by Г. We report the mean and standard deviation over 10 different seeds. Higher is better.
10
CDVAE (ours, K=7]
EOVAE (K=2)
PEHE (Mean = Std)
COVAE (K-5)
28
(gronnd buth,
COVAE (--1)
37
CDNAL
16
CONAE (+5)
CONAE (K-B)
COVAE (-141
0.0
05
1.0
15
2.0
255
15
11
14
Kvalues in CDVAE
(a) Synthetic data.
(b) MIMIC data.
Figure 9: Results of CDVAE when varying the number of components K of the prior for the synthetic data
(left) and MIMIC-III (right) reported by PEHE. Smaller is better.
To compare the observed and replicated data, define statistic T based on the conditional log- liliikiiihodd:
Nea
T(y:bs) =
log (v ha, 2i(s)),
Nval
and for replicated responses:
(()))((s)) Neai
MMat log Logge(viit (P).)(s)) baywa,, z.()). We then define
posterior predictive p-value as:
p Pr(T(y) (s)( ))
which is approximated as:
I[T(y (s) > T)
p 5
We compare these p-values over time to assess how closely the distribution of the conditional responses matches the distribution of the replicated responses. If the model accurately captures the conditional response distribution given the substitutes, the test statistics for the replicated data should be close those for the observed data, ideally resulting in valuee equal to 0.5. Figures T0al and 10b the temporal evolution of
22

---

Published in Transactions on Machine Learning Research (10/2025)
0,9
0.8
0.7
0.6
0.5
0.4
0.3
-20
6 150
0.2
67 122
0.1
0.0
Time Steps
(a) Synthetic data
20
30
Time Steps
(b) MIMIC III
Figure 10: Evolution of CDVAE posterior predictive p-values for synthetic (top) and semi-syuthetic MIMIC- III data (bottom). We report the average for 10 random initializations at each time step.
the p-values for both the synthetic data and semi-synthetic MIMIC-III In general, all the p-values fluctuate inside the interval [0.41, 0.59] except for the synthetic dataset with 7/1 0.0 because U no longer modifies the potential outcomes (1))))) Interestingly, the bias in the p-values (Figure 10a decreases as Y() increases because CDVAE is designed to capture substantial heterogeneity in the responses due to the unobserved adjustment variables.
YU
Connection to the Deconfounder Theory Our approach can also provide remedy for key in- consistency in the deconfounder theory (Lopez & Gutman 2017 Ranganath & Perotte 2018 Wang & Blei 2019a b). Crucially, while the inference model for the latent confounder is probabilistic, leading to a distribution over the substitute confounder, the theory relies on the assumption that the posterior distri- bution collapses into a Dirac delta distribution, implying that the confounder is estimated with certainty. This assumption simplifies the theoretical framework but introduces fundamental incoherence between the stochastic nature of the inference models and the deterministic assumption in the theory (D'Amour 2019b) Imai & Jiang 2019 D'Amour 2019a). There is significant theoretical-appliattio gap: In theory, the deconfounder assurnes that the substitule confounder estimated with certainly, while in practice, infer- ence models are stochastic (Zhang et al. 2019 Bica et al.) 2020b Hatt & Feuerriegel 2024). and the latent confounder described by distribution, introducing uncertainty. Our use of near-determinisic VAEs indi- rectly controls the posterior variance, finding a compromise where the posterior remains less diffuse yet not fully deterministic. This approach enables a more precise estimation of the substitute confounder without
23

---

Published in Transactions on Machine Learning Research (10/2025)
assuming perfect certainty, addressing inherent inconsistency. Applying this method within deconfounder theory holds independent interest. beyond this paper's scope. Future research could explore integrating these improvements into the deconfounder framework, with near-deterministic VAEs for substitute confounders inference to bridge theory and practice.
Comparing our approach to the deconfounder Unlike the doconfounder, our method does not require the consistency assumption for identifying treatment effects. Whereas the deconfounder framework models unobserved covariates as confounders affecting both treatment and outcome, we treat these unobserved covariates adjustment variables, thus bypassing the need for the consistency assumption. A theoretical discussion of this distinction is in Appendix B.1. Furthermore, aligned with critiques of the deconfounder the validity of inferred substitutes (D'Amour 2019b] [mai & Jiang, 2019) D'Amour 2019a; Ogburn et al. 2019), we investigate in Appendix B.2 whether the substitute adjustment variables Z might inadvertently capture "bad variables" that could bias causal effect cstimation.
On failure modes of CDVAE We now present the principal theoretical failure modes of CDVAE that may caution against its use in practice:
1. Hidden confounding beyond He, We assume sequential ignorability; only missing adjustment variables U affect outcomes, but not treatments. If confounders are missing, identifiability fails. In a knife-odge case where these confounders are (i) time-invariant and (ii) satisfy the same CMM(p)
structure as U, identifiability results (e.g.. Theorem ! Corollary 5 Theorem G might still hold. But this is implausible in practice, and crucially, our propensity model omits the latent variable Z, introducing bias in treatment weighting.
2. Time variation in the unobserved variables. CDVAE treats U as static adjustment variables. If the true unobserved effect modifiers are time-varying, or if there are time-varying unobserved confounders, the latent substitute can no longer absorb all the relevant heterogeneity with a time- invariant representation, and both the identifiability argument and the sequential ignorability with augmented history may fail.
3. Very long memory or infinitc-order dependence. Our identifiability results rely on finite conditional Markov order p. If outcomes exhibit very long distributed-lag effects, or more strongly, if there is no finite order m for which responses more than m steps back are conditionally independent given current history Z, then the process is effectively infinite-order. This violates CMM(p) and undermines our ability to infer reliable substitutes for U from the outcome sequence, and ACATE may not be ideltifiable.
4. Outcome support and heavy tails. As technical requirement, Theorem 6 assumes y lies in a Borel subset of compact interval. Heavy-tailed outcomes stretch this regularity condition, but it can often be mitigated by varianco-stabillzingg transforms or winsorization.
Parameter nonstationarity/ree /regime switching. Even if a CMM(p) holds structurally, our anal- ysis presumes fired parameters 8. If the conditional response dynamics switch across regimes (e.g.. latent or exogenous regime changes) so that, even given Z, the effective parameters drift over time, the fixed-0 factorization in Assumption 3 breaks. A regine-switching or time-varying parameter extension would be needed; otherwise, causal validity and estimation stability can degrade.
Conclusion In this paper, we proposed Causal DVAE, novel framework for estimating treatment effects in high-dimens,ooal,, time-narying settings. By leveraging variational inference with robust regularization tech- niques, we introduced principled approach to infer latent adjustment variables while ensuring identifiability and mitigating covariate imbalance. Our framework demonstrated theoretical guarantees for generalization and treatment effect consistency in the near- near-deterministic regime of VAEs an overlooked property in the causal inference literature. Extensive empirical evaluations support the effectiveness of CDVAE on both synthetic and semi-synthetic datasets. Future work could focus on extending this framework to: 1. dynamic treatment. regimes, for example, by incorporating the g-calculus (Robins 1997 Vansteelandt & Joffe 2014]
24

---

Published in Transactions on Machine Learning Research (10/2025)
2. providing generalization bounds for treatment effects when they depend sequence of treatments LLewis
& Syrgkanis 2021 Vankadara et al. 2022 Oh et al. 2022 Csillag et al. 2024): 3. exploring theoretically
the bias-varies variance trade-off in the estimation of treatment effects in the near-deterministic regime.
25

---

Published in Transactions on Machine Learning Research (10/2025)
References
Ahmed Alaa and Mihaela Van Der Schaar. Validating causal inference models via influence functions. In International Conference on Machine Learning, pp. 191-201. PMLR, 2019.
Serge Assaad, Shuxi Zeng, Chenyang Tao, Shounak Datta, Nikhil Mehta, Ricardo Henao, Fan Li, and Lawrence Carin. Counterfactual representation learning with balancing weights. In International Confer- ence on Artificial Intelligence and Statistics, pp. 1972-1980 PMT.R, 2021.
Onur Atan, James Jordon, and Mihaela Van der Schaar. Deep-treat: Learning optimal personalized treat- ments from observational data using neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
David Benkeser, Iván Diaz, Alex Luedtke, Jodi Segal, Daniel Scharfstein, and Michael Rosenblum. Improving precision and power in randomized trials for covid-19 treatments using covariale adjustment, for binary, ordinal, and time-to-event outcomes. Biometrics, 77(4):1467 1481, 2021.
Joana Bica, Ahmed M Alaa, James Jordon, and Mihaela van der Schaar. Estimating counterfactual treat- ment outcomes over time through adversarially balanced representations. In International Conference on Learning Representations, 2020a. URL
Ioana Bica, Ahmed M. Alaa, and Mihaela van der Schaar. Time series deconfounder: Estimating treatment effects over time in the presence hidden confounders. In International Conference on Machine Learning, 2020b.
Howard S. Bloom, Lashawn Richburg-Hayes, and Alison Rebeck Black. Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29(1):30-5m 59, 2007. doi: 10.3102//1622333777295533 URL ottp:://ddi.oor/10...
0162373707299550
Mouad El Bouchattaoui, Myriam Tami, BENOIT LEPETIT, and Paul- Henry Cournède, Causal contrastive learning for counterfactual regression over time. In The Thirty-eighth Annual Conference Neural In- formation Processing Systems, 2024. URL https://opeereviie
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. Gener- ating sentences from continuous space. arXiv preprint arXiv:1511.063/ 2015.
Lu Cheng, Ruocheng Guo, and Huan Liu. Causal mediation analysis with hidden confounders. Pro- ceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, null: mull, 2021. doi: 10.1145/3488560.34999 11770110066440222666555777717117
URL
hatpaatiiicccooaaooooppapaaapppp org/paper/
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Damitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rmn encoder-decoder for statisti- cal machine translation. arXiv preprint orXiv:1406.1077 2014.
Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Advances in neural information processing systems, 29, 2016.
Daniel Csillag, Claudio Jose Struchiner, and Guilherme Tegoni Goedert. Generalization bounds for causal re- gression: Insights, guarantees and sonsitivity analysis. In Forty-first International Conference on Machine Learning, 2024. URL http::///pppaaaeees.ee.eeeerrrrp wwt..e/ooormmmiieeeeerrrgg/////
Marco Cuturi and Arnaud Doucct. Fast computation of wasserstein barycenters. In International conference on machine learning, pp. 685 693. PMLR, 2014.
Bin Dai and David Wipf. Diagnosing and enhancing VAE models. In International Conference on Learning Representations, 2019. URL httpss///pparreeeee..aa////rrtap
26

---

Published in Transactions on Machine Learning Research (10/2025)
Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker. Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. In Proceedings of the sirth A ACM international conference on Web search and data mining, pp. 123 132, 2013.
Alexander D'Amu.r. On multi-cause approaches to causal inference with unobserved counfounding: Two cautionary failure cases and a promising alternative. In The 22nd International Conference on Artéficial Intelligence and Statistics, pp. 3478 3486. PMLR, 2019a.
Alexander D'Amour. Comment: Reflections on the deconfounder. Journal of the American Statistical Association, 111(628):1597-1161, 2019b.
Fabian Falck, Haoting Zhang, Matthew Willetts, George Nicholson, Christopher Yau, and Chris C Holmes. Multi-facet clustering variational autoencoders. Advances in Neural Information Processing Systems, 34: 8676 8690, 2021.
William Falcon and The PyTorch Lightning team. Pytorch lightning.
https://github. com/
2019. Version 1.0.
Jumpeng Fang, Qing Cui, Gongduo Zhang, Caizhi Tang, Lihong Gu, Longfei Li, Jinjie Gu, Jun Zhou, and Fei Wu. Alleviating matching bias in marketing recommendations. In Proceedings of the 46th International ACM SIGIR Conference Rescarch and Development in Information Retricual, pp. 3359-3360 2023.
Hao Fu, Chunyuan Li, Xiaodong Liu, Jianfeng Gao, Asli Celikyilmaz, and Lawrence Carin. Cyclical anncaling schedule: A simple approach to miligaling kl vanishing. arXiv preprinl or xrv::11900 10145, 2019.
Ken-ichi Funahashi and Yuichi Nakamura. Approsimation of dynamical systems by continuous time recurrent neural networks. Neural networks, 6(6):801-80)) 1993.
Androw Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesion data analysis. Chapman and Hall/CRC, 1995.
Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier Alameda-Pineda. Dynamical variational autoencoders: A comprehensive review. ArXiv, abs/2008.115588 2021.
Lukas Gonon and Juan-Pablo Ortega. Fading memory echo state networks are universal. Newral Networks, 138:10 13, 2021.
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-777 773, 2012.
Joseph F Hair Jr and Marko Sarstedt. Data, measurement, and cansal inferences in machine learning: opportunities and challenges for marketing. Journal of Marketing Theory and Practice, 29):666 77, 2021.
Negar Hassanpour and Russell Greiner. Learning disentangled representations for counterfactual regression. In International Conference on Learning Representations, 2019a.
Negar Hassanpour and Russell Greiner. Counterfactual regression with importance sampling weights. Tn
IJCAI, pp. 5880 5887, 2019b.
Tobias Hatt and Stefan Feuerriegel. Sequential deconfounding for causal inference with unobserved con- founders. In Francesco Locatello and Vanessa Didelez (eds.), Proceedings of the Third Conference on Causal Learning and Reasoning, volume 236 Proceedings of Machine Learning Research, pp. 934-956. PMLR, 01 03 Apr 2024. URL htttps://proceeddngg/ mlr. wwtpss/v26//att222222 html
Miguel A. Hernán and James M. Robins. Causal Inference: What If. Chapman & Hall/CRC, 2020. URL https:/ h/wp:::///ww..// hsph harvard.
book/
Robins JM Hernán MA. Causal Inference: What If. Boca Raton: Chapman & Hall/ 2020.
27

---

Published in Transactions on Machine Learning Research (10/2025)
Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. ICLR (Poster), 3, 2017.
Jennifer L Hill. Bayesian nonparametric modeling for causal inference. Journal of Computational and Graphicul Stalistics, 20(1):217 240, 2011.
Iris AM Huijben, Wouter Kool, Max B Paulus, and Ruud JG Van Sloun. A review of the gumbel-max trick and its extensions for discrete stochasticity in machine learning. IEEE transactions on pattern analysis and machine intelligence, 45(2):1353- 1371, 2022.
Noorie Hyun, Abisola E Idu, Andrea J Cook, and Jennifer Bobb. Increased risk of type errors for detecting heterogeneity of treatment effects in musterrrandomizerr trials using mixed-effect models. arXiv preprint arXfv:2407.06466 2024.
Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429- (499-499,, 1999.
Kosuke Imai and Zhichao Jiang. Comment: The challenges of multiple canses. Journal of the American Slatistical Associalion, 114((28):1605 1610, 2019.
Guido W Imbens and Donald B Rubin. Causal inference in statistics, social, and biomedical sciences.
Cambridge University Press, 2015.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax In In- lernalional Conference Learning Represenlalions, 2017. URL httpp:s//oppenreriiieeettt///// wttp://rwwwe.eeeoommmmmiei. rkE3y85ee
Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou Variational deep embedding: unsupervised and generative approach to clustering. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, IJCAI11, pp. 1972-1972. AAAI Press, 2017. ISBN 9780999241103.
Fredrik Johansson, Uri Shalit, and David Sontag. Learning representations for counterfactual inference. In
International conference on machine learning, pp. 3020 3029. PMLR, 2016.
Fredrik D Johansson, Nathan Kallus, Uri Shalit, and David Sontag. Learning weighted representations for generalization across designs. orXiv preprint arXiv: 180.. 880.00858, 2018.
Fredrik D Johansson, David Sontag, and Rajesh Ranganath. Support and invertibility in domain-invani representations. In The 22nd International Conference on Artificial Intelligncce and Statistics, pp. 527 536. PMLR, 2019.
Fredrik D Johansson, Uri Shalit, Nathan Kallus, and David Sontag. Generalization bounds and representa- tion learning for estimation of potential outcomes and causal effects. The Journal of Machine Learning Research, 23(1):7489-735) -538, 2022.
Alistair FW Johnson, Tom Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-ili, freely accessible critical care database. Scientific data, 3(1):1 9, 2016.
Youghan Jung, Jin Tian, and Elias Bareinboim. Learning causal effects via weighted empirical risk mini- mization. Advances in neural information processing systems, 33:12697-12709, 2020.
Olav Kallenberg, Foundations of modern probability. Probobility Theory and Stochastic Modelling, 2021.
Nathan Kallus. Deepmatch: Balancing deep covariate representations for causal inference using adversarial training. In International Conference Machine Learning, pp. 5067-5077. PMI.R 2020.
28

---

Published in Transactions on Machine Learning Research (10/2025)
Urvashi Khandelwal, He He, Peng Qi, and Dan Jurafsky. Sharp nearby, fuzzy far away: How neural language models use context. In Tryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pp. 284 294, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P18- 1027. URL https: Sagclarbbology.crr//111-1112
Manabu Kuroki and J. Pearl. Measurement bias and effect restoration in causal inference. Biomelrika, 101:423-437, 2014. doi: BAA.B////OOE//STTOOOOO URL attpp::/ww..ennniiccchoooll www..orgggpppp..... /apper///
Nan M Laird and Christoph Lange. The fundamentals of modern statistical genetics. Springer, 2011.
Greg Lewis and Vasilis Syrgkanis. Double/ Double/debiased debiased machine learning for dynamic treatment effects. In
A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances Neural Information Processing Systems, 2021. URL httppp//ppaaaneeeee..eeee/ooottp wtt.:///oorrrriesttsooo.........
Fan Li, Kari Lock Morgan, and Alan M Zaslavsky. Balancing covariates via propensity score weighting, Journal of the Americon Statistical Association, 113((21):390-445) 2018.
Rui Li, Stephanie Hu, Mingyu Lu, Yuria Utsumi, Prithwish Chakraborty, Daby M. Sow, Piyush Madan, Jun
Li, Mohamed F. Ghalwash, Zach Shahn, and Li wei H. Lehman. G-net: recurrent network approach to g-computation for counterfactual prediction under a dynamic treatment regime. In MLAH@NeurIPS,, 2021.
Bryan Lim. Forccasting treatment responses time using recurrent marginal structural networks. advances in neural informalion processing systems, 31, 2018.
Michael J Lopez and Roce Gutman. Estimation of causal effects with multiple treatments: review and new ideas. Slatistical Science, pp. 432-454, 2017.
Christos Louizos, Uri Shalit, Joris M Mooij, David Sontag, Richard Zemel, and Max Welling, Causal effect inference with deep latent-variable models. Advances in neural information processing systems, 30, 2017.
Danni Lu, Chenyang Tao, Junya Chen, Fan Li, Feng Guo, and Lawrence Carin. Reconsidering generative objectives for counterfactual reasoning. Advances in Neural Information Processing Systems, 33:21539- 21553, 2020.
James Lucas, George Tucker, Roger B Grosso, and Mohammad Norouzi. Don't blame the elbo! lincar vac perspective posterior collapse. Advances in Neural Information Processing Systems, 32, 2019.
Monica R McClain, Glenn Palomaki, Margaret Piper, and James E Haddow. rapid-acce review of cyp2c9 and vkorel alleles testing to inform warfarin dosing in adults elevated risk for thrombotic events to avoid serious bleeding. Genetics in Medicine, 10(2):89-98 98, 2008.
Valentyn Melnychuk, Dennis Franen, and Stefan Fenerriegel. Causal transformer for estimating counterfac- tual outcomes. In International Conference on Machine Learning, pp. 15293 15329. PMLR, 2022.
Xiao-Li Meng. Posterior predictive p-values. The annals of statistics, 22 1994.
W. Miao, Z. Geng, and E. T. Tehetgen Tchetgen. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105 4:987 993, 2016. doi: 10.100//BOOMTT/ASYSS URL https:
John Miller and Moritz Hardt. Stable recurrent models. In International Conference on Learning Represen- tations, 2019. URL http::////pppprrreeeeeeeee//eeee
Stephen L Morgan. Handbook of causal analysis for social research. Springer, 2013.
Scott Mueller and Judea Pearl. Personalized decision making-a conceptual introduction. Journal of Causal Inference, (002200000 2023.
29

---

Published in Transactions on Machine Learning Research (10/2025)
Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in applied probabilily, 29(2):429-44) 443,, 1997.
Xinkun Nie and Stefan Wager. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, 108 (2):299-314 2021.
Elizabeth L. Ogburn, I. Shpitser, and F. Tchetgen. Comment on "blessings of multiple causes". Journal of the Americon Statistical Association, 114:1611 1615, 2019. doi: 00.10000000621459.201...66 URL
wtt.://.ematiischola.r.... / htg.aa/cc......ooooooooooofoooaa
Eun Jeong Oh, Min Qian, and Ying Kuen Cheung. Gencralization error bounds of dynamic treatment regimes in penalized regression-based learning. The Annals of Slatistics, 50(4):2047-2) 2071, 2022.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkuthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Newral Information Processing Systems, 2019.
Benjamin M Peter, Desislava Petkova, and John Novembre. Genetic landscapes reveal how human genetic diversity aligns with geography. Molecular biology and evolution, 37(4):943-955 9519 2020.
Kaare Brandt Petersen, Michael Syskind Pedersen, et al. The matrix cookbook. Technical University of Denmark, 7(15):510, 2008
Trang Pham, Truyen Tran, Dinh Phung, and Svetha Venkatesh. Predicting healthcare trajectories from medical records: deep learning approach. Journal of biomedical informatics, 69:218 229, 2017.
Robert W Platt, Enrique F Schisterman, and Stephen R Cole. Time-modified confounding. American journal of epidemiology. 170(6):687-694, 2009.
Rajesh Ranganath and Adler Perotte. Multiple causal inference with latent confounding. arXiv preprint arXiv:1805.0820 2018.
James M Robins. Causal inference from complex longitudinal data. In Latent variable modeling and appli- calions causalily, pp. 69-117. Springer, 1997.
James M Robins and Miguel A Hernán. Estimation of the causal effects of time-varying exposures. Longi- tudinal data analysis, 553:599, 2009a.
James M Robins and Miguel A Hernán. Estimation of the causal effects of time-varying exposures. Longi- tudinal data analysis, 553:599, 2009b.
James M Robins, Miguel Angel Hernan, and Babette Brumback. Marginal structural models and causal inference in epidemiology. 2000.
Paul R Rosenbaum and Donald B Rubin. Assessing sensitivity to an unobserved binary covariate in an ob- servational study with binary outcome. Journal of the Royal Statistical Society: Series B (Methodological), 15(2):212-215 1983.
Donald B. Rubin. Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician. The Annals of Statistics, 12(4):1151 - 1172, 1984. doi: 10.
URL https://doi....... org/
10. 18.1214/aos. 10.1214/ass/176346755 1176346785
Michal Sadowski, Mike Thompson, Joel Mefford, Tanushree Haldar, Akinyemi Oni-Orisan, Richard Border, Ali Pazokitoroudi, Julien Ayroles, Sriram Sankararaman, Andy Dahl, et al. Characterizing the genetic architecture of drug response using gene-context interaction methods. medRziv, pp. 2024-03, 2024.
Enrique F Schisterman, Stephen Cole, and Robert W Platt. Overadjustment bias and unnecessary ad- justment in epidemiologic studies. Epidemiology (Cambridge, Mass.), 20(4):488, 2009.
30

---

Published in Transactions on Machine Learning Research (10/2025)
Peter Z Schochet. Design- Design-based ret estimators and central limit theorems for baseline subgroup and related analyses. Jourmal of Causal Inference, 12(1):2023006, 2024.
Peter Schulam and Suchi Saria. Reliable decision support using counterfactual models. Advances in neural information processing systems, 30, 2017.
Alejandro Schuler, Michael Baiocchi, Robert Tibshirani, and Nigam Shah. A comparison of methods for model selection when estimating individual treatment effects. arXiv preprint 1804 05146, 2018.
Ute I. Schwarz, Marylyn D. Ritchie, Yuki Bradford, Chun Li, Scott M. Dudek, Amy Frye- Andersnnesonn, Richard B. Kim, Dan M. Roden, and C. Michael Stein. Genetic determinants of response to war- farin during initial anticoagulation. New England Journal of Medicine, 358 (58119999991108 2008. doi: 10. (0056///NM000000007 URL https: /wtp://wwwwwwc wwwww.wwj..... wweja.oor/do/ffll///0 101.:///0....0006000000007000770
Uri Shalit. Can we learn individual-level treatment policies from clinical data? Biostatistics, 21( 362, 2020.
Cri Shalit, Fredrik D Johansson, and David Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning, pp. 3076 3085. PMLR, 2017.
Claudia Shi, David Blei, and Victor Veitch. Adapting neural networks for the estimation of treatment effects. Advances in neural information processing systems, 32, 2019.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
Richard Sinkhorn. Diagonal equivalence to matrices with prescribed row and column sums. The American Mathematical Monthly, 74(4):402-44) 405, 1967.
Casper Kaae Sonderby, Tapani Raíko, Lars Maalee, Søren Kaae Sønderby, and Ole Winther. Ladder varia- tional autoencoders. Advances in newral information processing systems, 29, 2016.
Bharath K Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, and Gert RG Lanckrict. On integral probability metrics, :phi-divergences and binary classification. aiviv preprinl ar arXi::0901.20: 2009.
Jon Arni Steingrimsson, Daniel F Hanley, and Michael Rosenblum. Improving precision by adjusting for prognostic baseline variables in randomized trials with binary outcomes, without regression model assump- tions. Contemporrry clivical trials, 54:18 24, 2017.
Yuhla Takida, Wei-Hsiang Liao, Chieh-Hsin Lai, Toshimitsu Uesaka, Shusuke Takahashi, and Yuki Mitsufuji. Preventing oversmoothing in vae via generalized variance parameterization. Neurocomputing, 509:137-1566 2022.
George Tucker, Andriy Mnih, Chris J Maddison, John Lawson, and Jascha Sohl-Dickstein. Rebar: Low- variance, unbiased gradient estimates for discrete latent variable models. Advances Neural Information Processing Systems, 30, 2017.
Leena Chennuru Vankadara, Philipp Michael Faller, Michaela Hardt, Lenon Minorics, Debarghya Ghosh- dastidar, and Dominik Janzing, Causal forecasting: generalization bounds for autoregressive models. In Uncertuinly in Artificial Intelligence, pp.
PMLR, 2022.
Stijn Vansteelandt and Marshall Joffe. Structural Nested Models and G-estimation: The Partially Realized Promise. Statistical Science, 29 (4)) - 731, 2014. doi: 10. 10.1214/14-STS493 URL https:://dl.orgg//
1214/14-879493
Yixin Wang and David M Blei. The blessings of multiple causes. Journal of the American Statistical
Association, 1144528):1574-1599 2019a.
31

---

Published in Transactions on Machine Learning Research (10/2025)
Yixin Wang and David M Blei. The blessings of multiple causes: A reply to ogburn et (2019...). arXiv preprint Xiv: 1910. 19100073000 2019b.
Yixin Wang, David Blei, and John P Cunningham. Posterior collapse and latent variable non-identifiabliity Advances neural information processing systems, 34:5443 $4:5443-5455, 2021.
Hong Xiong, Feng Wu, Leon Deng, Megan Su, Zach Shahn, and Li-wei H Lehman. G-transformer: Counter- factual outcome prediction under dynamic and time-varying treatment regimes. Proceedings of machine learning research, 252:ht(ps proceedings, 2024.
Linying Zhang, Yixin Wang, Anna Ostropolets, Jami J Mulgrave, David M Blei, and George Hripesak. The medical deconfounder: assessing treatment effects with electronic health records. In Machine Learning for Healthcare Conference, pp. 490-512. PMLR, 2019.
Yao Zhang, Alexis Bellot, and Mihaela Schaar. Learning overlapping representations for the estimation of individualized treatment effects. In International Conference on Artificial Intelligence and Statistics, pp. 1004-1014. PMLR, 2020.
José R. Zubizarreta. Stable weights that balance covariates for estimation with incomplete outcome data. Journal of the Americon Statistical Association, 110(511):910-922) 2015.
A Proofs
A.1 Identifiability: Treatment Effects and Unobserved Adjustment Variables
Proof [CATE Identifiability) Assuming the consistency, overlap, and ignorability assumptions
1
2
we demonstrate that the CATE, as defined in Eq. 2 is identifiable from the observed data distribut
n H. h, W2 1) HH, h, W, 0).
(25)
Establishing the identifiability of CATE requires showing that the two potential outcome expectations are identifiable. By the ignorability assumption as:
( the potential outcome under treatment can be expressed
(h.)) (yj(,,, (Y₁(1) H. = h.) By,a,, (Y,(1) H, = h., W, =).
Using the consistency assumption the observed response can identify Y((1) when conditioned on W; 1:
m Hy,,.w. (Y/ H. h, We 1).
Similarly, we identify the expected potential outcome under no treatment:
m (h.) H, (Yr H. h, W 0).
The existence these expectations is guaranteed by the overlap assumption.
Proof [Augmented CATE Identifiability) Assuming that the adjustment variables U are observed, we demonstrate the identifiability of the Augmented CATE:
7i (h, u) E(Y h, U u. W, = 1) H, h, U u, W, = 0).
The assumptions of consistency, overlap, and ignorability ensure that CATE identifiable. Since U does not affect the treatment, the overlap assumption (3) remains valid. Further, U being independent of the treatment implies that the ignorability assumption holds when conditioning on U:
Ya(w) II W. h, Ya(w) I1 H. h, U=n u..
32

---

Published in Transactions on Machine Learning Research (10/2025)
The remainder of the proof follows the CATE identifiability argument.
Proof [Theorem Let a latent variable such that ~ CMM Any static adjustment variables alfecting all response series the panel must be measurable with respect (Z, Hr). We assume weak regularity conditions on treatment and response domains.
Assumption 13 (Regularity) The response domain y is a Borel subset of compact interval.
The treatment domain W {0, } is Borel subset of [0, 1]. To prove the theorem, we need the following lemma:
Lemma 14 (Kernels and Randomization (Kallenberg. 2021) Let ia be probability kernel from measurable space S to Borel space S2. There crists measurable function f Si x [0, + S2 such that if 0 is uniform on [0, 1], then f() is distributed as p(s1.)).
Suppose by contradiction the existence of Zr that is not measurable with respect to (Z, H7) and such that:
(w) + Ha, Z; Ww,
Let be an arbitrary time step in the panel data. By lemma H4 x W x Z [0, y such that:
14 there exists a measurable function fr
Yu = f( Wa, Zí Re), Ta (H Wa, Z.).
The conditional Markov property implies the independence of the following conditional distributions:
(Yu H., Z, Wtt w) (Yiv Har, Zi, Was w/).
Such that verifies 1r| p. We can thus conclude that:
(Yi((w) Ha, Z.) IL (Yar(w)) H Z.).
Because
(Ya(w) Ha, Z.) (Yu(w) Has Z,, W = w)
(Yu Hyg Z₁ Ws. w).
The first oquality follows from the sequential ignorability of the theorem 4 and the second equality follows the consistency assumption. On the other hand, we also have from the CMM(p) that:
Yu 11 Yi. H Z,
From which follows using the fact that nh 11 (He, Wse Z,) and (u((w) H Z, Yn Ho, Z. Wa. wt
Yu TT Yue(w) Hic. Z.
By using twice the lemma 14 write:
Nuu h4 (Zl,,,,
Matr 11 Zzi
(26)
And:
Yu(ww) =ga(Zn gz Ei), ea 11 Ztz.
(27)
Since Z/; is not measurable with respect to (Z, Ha), then by equation 26 and equation 27
Yar JL Yu Hap Z;
We have thus a contradiction.
33

---

Published in Transactions on Machine Learning Research (10/2025)
A.2 Derivation of CDVAE Loss
Proof [EBBO] We provide proof that the bound in Eq. for the weighted conditional logliikelihood.
12 is indeed the Evidence Lower Bound (ELBO)
By the concavity of the logarithm, for every t E
we have:
log Po (vr, Z, C hr,
log (ye h4, wi) 2 (zz.0oo(..-PP)
(28)
go C Dr)
We use the identity po(3e. z, h, = Po(ve posterior go(z.c c Dr)= = qo. (z Dr (c) Dr), to write:
z, he, c), and the factorization of the approximate
po Pv(: Z, w)) p(C (Z)
(+) Cv.gaaal
log
4o. (Z Dr) to. 40.(C) (C) Dr)
FZz,Cr Poog Z, h wx)] (KI (Z Dr)aa. (C
(C Dr) p(C ())(Z)
(**)
The KL divergence term (**) between the joint approximate posterior and the prior can be decomposed into the sum of two KL divergences:
p (Zp(Z)
(x*)
(z log
9s. (Z Dr.q (C [Dr))
K
((Z)
K
Ez-as(sdpa log p(e 2) 20 Dr)
Ez (.o)Dr) log
+
96. (Z Dr))
c=
0=1
= DeL (4o, (z Dr) || p(z)) + Ez (/Dr) DEL (go (c Dr) || p(c Z))
We can now define the individual ELBO by performing a weighted sum over the likelikldoodd terms and marginalizing over the full longitudinal data distribution:
L Edr (a, W;) Pog H,, Wo] 2 EDr Σ Ez, E2.coggoi:1P4 a(b,, wx) log Po (yr, Z, C ((,W))
T
Σ t=1
az C' (
Given the developed expressions for (+) and (*+), we can express (LBO0(DT;;,0)) as:
ELBO, Z~ (1Dr) (ar wi) log h wr, Z)]
(29)
Σ arr) ( (DKLl0o (T) Dr) (())) + Ez (ID,) DKI(O. Dr) || p(c Z))}
The gap in our variational approximation is defined as the difference between the true weighted log-likelihood and the ELBO:
Ep, Ao,ro(Dr:0.0)) = L ED, ELBO((Dr))))
The per-time-step gap Eq. 28 can be rewritten as:
log Pe (yr. Wt) (Z, C Dr) Po (y,, Z, C h4 wr)
log po (ye he,w)) (*) = C2..0ro(--1Pr)
(Z, C. Dr)
-Bz,C-qs (T-ID))
log Po (Z, C D.))
= (gs C'
Dr) pe (Z, C' (D))
34

---

Published in Transactions on Machine Learning Research (10/2025)
This last equation holds because and (ye, hy, wi} Wsn Dr.
po (y,, Z, C we, po (Z, yt, wi, po (yr. we,
The individual gap thus:
r
A0(Dr;0,0) = Σ
o(hy wi) DKL (9o (r) || po (Z, C (2))
t=1
A.3 Transfer of Ignorability under Invertible Maps
Proposition 15 Let & an invertible representation function. Then, Ye(w) W (H) holds if and only if ((W)) II W. Moreover, p(W, w|b(b.)) > 0 holds if and only if p(W, w(h.)) 0.
Proof Assume Yi(w) IL H4 For a non-invertible @, we have, letting tb Φ-'(r) = (b. ((h)) = r):
p((((W) wr,r) = Sh,60-10)) p(Y((w) wi, he)p(hs | wi wy)dh,
Snea 10e) p(he wi)dh,
[h.e0-1()) ((Y((w) he)p(h, ws
(30)
(m p(he we wi)dh,
where ignorability implies that, for general 4, p(Y((w) wr, & p(Y((w) p(Y((w) we, +(h.)) p(Y((w) ((h))). Similarly, p(W, (h.().)) p(W, h.).
|r). For an invertible 4, however,
A.4 CDVAE in the Near-Deterministic Regime
Proof Theorem
Part 1: lome- (yST xgT,Wer) pLSTT XgT,
Define the conditional cumulative distribution function (CDF) of the response given covariates, for each 88:
Folye he,we) = f p(u/ he wry/
Define the mapping F ym + [0, 11 as:
F(un,... yr xgr, = [F1 (yn h1,w)). .... Fr(yT [hr,wr)]
The differential of the mapping is given by:
........ .rr xgT,WeT) p(yeT xxg,
Now, define the following mapping for the latent variables using the Darmois construction (Hyvärinen &
Pajunen 1999):
G2(z: 2 21...., = S p(z4 21, ,1. 21, 2412
We then define the mapping G d [0, 11TH such that:
G(z) = (2)) 21),..., Gilza (3y...,#d,-1)
35

---

Published in Transactions on Machine Learning Research (10/2025)
Since d2 T by assumption, can trivially augment the mapping G to G so that the image is defined [0,1] [,1],,,in i.e., G: & [0, 1)3 such that: ((z) == [G(z), 0....,... Ol. The differential is given by: dG(z) p (2)
T-d
We now define the mean and variance of the encoder as follows:
1
fe we; 2: 0;) [F- (G xgn,wer)ee of Vs
There is a consistency issue to address with this definition. First, observe that the function ft(:; takes as input only data up to time step t, but the inverse of the cumulative CDF is defined given the whole sequence of (XgT, WgT). We therefore need to verify the following lemma which holds for our definition of fel:; 07).
Lemma 16 Let (yST, &ST,WET) and (WeT. Xer, w'sr) be two distinct realizations of repeated measurements such that there exists to for which there enist two identical sub-trajectories Xsto = X'sto and Weto = a'sto' Then, for every S lo, we have:
(G(z) we)) (G ('))l
Now, we decompose the marginal probabilistic model:
/ Rdd
Pe, (UST XgT, WgT)
Pe, Po, (UET, 2 Kgr,
Pe, (yr, (eo
Rd
N (y
(F-)(
xcT, (o;)) p(z)
T
II
W (y.
(exr dE
10.17
t=1
T
II
N (u [Vgale (a:) plU'r Xgr,werr
yr
Finally, we have:
( II N (ve Werke (oi)²) PlU'T (er.Wer)ddss / JyT II 5(ye sillsr deysweerddier +00 y
T
lim
=P(VTT (gs.
Part 2: Proof of lom (+)
First, we give an explicit writing of the modified individual gap:
A(Dr: 0.) L(Dr; 0,, 6.) ELBOO,(P;;0,... o.) (D, - ELBO(Dr;
(Dr 0s, )2) + (Dr; Φ.) 0,,
a wx)
Ez DKI Dr) || p[c| Z))
Σ a. (h,, wi)
log Z(go, (:1) (rr)
Σ
aabe DKI Dr) || pa, (z,c D.)).
1
36

---

Published in Transactions on Machine Learning Research (10/2025)
The last equality holds because assume 9s, (c Dr) .T. Dz) and (cl (c|(r) Dr) is minimizer of min E En~g, (1Dx) DKL (c Pr) p(c Z)) log ((9 (- Pr)). (c(Dr)
To show that lim A(Dr; B. 0x) = 0, we will define o, such that for every c E fI,..., K} and t e
(T)...,T):
lim (us Dr) po, (2, D.)) = o. 800000
Let us define 2s. (z Dr) such that
( xgT,Wer))
and
fsx (Dr. 0oy / (D)
with Σz(Dr, being the inverse of:
Jac(F- (. xgT, G(z))Jac(FF+( (
Using Bayes' rule, we can write the true posterior as:
pa, (MST. c Xgr,we)) I N | (G XxT] (xT) p(z c)p(c)
pa, (z, e D:)
Po, (yst Wie)
po, (yst Xge,
By the GMM assumption over the prior, we have p(z c) =N(2 Me; Ee). The approximate posterior is of the form:
96. (z,cl Dr) = qs. (c) Dr) = N(z (Dr, oo), (i o) (c Dr).
We now show convergence by performing a change of variables. Define z' = = - -z (z- 2"). We analyze the behavior of the distributions ph, (z',c) D.) and d', (z',c) (r).
We prove that
pb. (z',e De) 1/ (z', e Dr)
converges to a constant independent of z' as 8 + xc. Since both do (z',c Dr) and ph, (z',c D:) are probability distributions, the constant must be 1. Therefore, the KL divergence between them converges to as - 0o.
We have:
Pb. e Dr)
N(zz + + 1i, ofE TT, (c Po, (yse xen. WeT)
(31)
Ls (x'C Dr)
Ii N( (ihh, x² + a. z'; 0.). o W2) + t
* x Men Ec)p(c)
Now, let A, be matrix such that its Moore Penrose inverse At satisfies:
+ feelhgg + z'; = At fsT(heT, WeT, t + +os + Z':0), yst At UST
That is, projecting onto + selects the first / responses. The matrix A+ has the form:
4 O
At = [Te 0ex(T-9)> A
37

---

Published in Transactions on Machine Learning Research (10/2025)
By noticing that [AT(o2Ir)-AA]" = oل We can therefore apply the mean rearranging formula
Petersen
et al. 2008
N A yST
At fgr Wgn x' + +o, - z':0.), [AT(O)TTAA1]
(d N UST (sT(her.WET. 2' +o, n 2':0,),
/doo(2
N 2" + +o, z' 1 (Dr 0)) To. (c Dr.po, (USe (gg)
P/. (z',c 1 Di))
92 (z',c c Dr)
y /det(2no3 .V ddec(2
VST SeT(heT, 20 + 000 z'; 0r;0,,,311 03
N(z" + +o, tr x' Mos Ec)p(c)
(2T) 7/2 det(Lz))
1 2
T
(h, x² + - zz,,''²
det(2io) - exp
08
[n
2
20
1=1
1 (z" +os t z' Mo) No)T2i
+ z' - mo)
+ 2
2e11 (z" to,
(z"
To. (c
Dr)po, (UST (gs)wg))
x
plc)
By Noticing that
T Σ
(y - Wi, z" + 0 ====z'000 0²)02))² = lluer - - F + +o.- x==z² z') (xrr
We apply a first-order Taylor expansion
Ver V-T- F- + x xgT, s -Jac(F FJac(F-1((())) [G)
54000
which implics by norm continuity that
T
E (un fihh,,,,, wi, x² + + 22)02)) stc [
0 ブ-
5
Now, define the constant Cst(Dr; u. 0s) with respect to z' as:
det(Ex) To, (c Dr)po, (USe x Wg)) det(F.x)
(c, = (2Th) :ي
p(c)
Thus,
Pb. (z',c) Dr)
ž _ - -Ho)2 (z" - -mo) - 0s) exp
1
{ 2(2, (1--1)]
-Os
= -
+ox
20t 11) TT
%6 (z',cl
Dr)
2+00
2
2
#000 5 5(2" u.) (z" - C,s
To conclude the proof, we still need to show that lim Cst(D,;o,, 0,, Φ.) exists and finite because the model parameters still depend the chosen variance. This the main misapprehension of Dai & Wipf (2019h for VAE in the static setting with an unimodal prior over the continuous latents Z.
We have already shown that
lim pa, (UST xgT,WeT) p(UST xgT,weT), 0
38

---

Published in Transactions on Machine Learning Research (10/2025)
and therefore:
lim Cst(Drio 0.) (2T) -
(E.)
p(yeT xgT,WgT)
lim
(c) (c) Dr)
det(Ex)
p(c)
212000
We verify that 1 To, Dr) exists and is finite and not zero for all e (I....,K]
exp(Ez~ga, (DDr) log Z))
no. (e Dr)
Σxn exp(Ez (-Dr) log p(c Z))
expl.2 log 2)W z z², 70 x)) dz)
E exp(Iz p(c z)NV (z z², of Dr 3) dz)
explJz log z)0(z zz)
Ee exp(_2 log p(c 2)0(z )zz))
K
p(c l()
E p(c 2²))
16. (x.c(Dr) 4, [D) converges into a nonzero constant which finishes the
We have therefore proved that the ratio proof of the theorem.
Proof of theorem
We start by considering the ELBO:
L(Dr;0,0)
Z Z- (Dr) wo
log log He, W,, Z)]
a(h; wi)
DxL. (ao(z (r) p(z))
Σ ath,, wi)
log Z(gol- [T)).
By the positivity of the KL divergence and using the fact that log Z(gol Dr)) minimizes a positive functional (as explained in Eq. equation 16 obtain:
C(DT: 02, 03) 2
(l (a(be, ux) log (po (ye H., W., Z))]
t=1
(br, we W2)
2 log (2Ta²)
+
1
(yr f (H,, Z, W:))²
202
1
Σ a(b), o² 02log (2TT²)
Σ
a(h; wr) Ez (Ve - (H, Z, W)))
2
(x)
Suppose there exists to 21, T3 such that lim da(t)) > 0. Then we have lim L(Dr; 0- (5) +oo, which is impossible since we assume L(Dr; 0z Φt) to be minimized. Therefore, conclude that for all t e (1,2...., T}, lim (t) = 0.
Consoquently, we have
lim BO,N((,,)) (v. f (H₁ We, (Pr: (Dri) fs, (D)) 111
39

---

Published in Transactions on Machine Learning Research (10/2025)
which implies that
(Dr: lim f (H,, We, (Pr: + fs, (Dr; = Ut, almost surely. 8-
In particular, we also have
lim (H, (Prro( - a-
A.5 Upper bound weighted PEHE
Proof [theorem 11 To prove the theorem, we rely on the following key lemma, but first, we need to define further mathematical objects. We define the weighted population risk over the whole population as
Re,g (f, 2) = WH,,, [H, W,) (,0 (H, Wol.
and the weighted population counterfactual risk as (f, 2)CF= BH.1-w, [a(H,, W3) (,, Wo).
Lemma 17
Re +) + Rt, D)r S (5, (J.00 + B& IPMg (go(- W, = 1), go W. = =))
Proof The proof immediately follows from lemma. in Shalit et al. (2017)
Next, to complete the proof, we define the expected potential outcome at time given the context. history H. = h and latent z as
m (h, 2) = 2Y, H, = h,, Z = z) WEW,
and we show the following lemma:
Lemma 18 Denote (he,, w, z) == p(h, w) (z DII- We can decompose Re 2) and Re (f, 2)
using our distribution assumptions as follows:
B) (a(be, w) (f(h, w. z) m, (h., 2))1 20² (Rt,( 1) - log((Too)) and,
Σ
Var (he (Yi),
WE (0.1)
(p)(hem))) [a(b;, 1-1, w) (f(h, 1 - w. w.z) -m, 1-20 (0,--((,, ((;,z))] = 20² (R 1 log((2oo))=
Σ
Var (he (m.,
wE(0.1}
Proof We have:
Rt, (S, 1) EH.., (H, W,) Wa)]
=> [
-a(be, log b), hi, 2) w) as(a Pee-i)dyidke
wE(0,1) HexExy
Σ
w)) { log (2mT²)) + 20 (y - (bs, (I,, w,)) poln hi p(hi, w) go(2) Ddy-i)dyi
wE(0.1)
Hi 2
a(bi, p(he, w)dh 20 H. f Je albersey (y (uc- mmm mi (b, 2))
log (2TO²)
Σ
wE(D.1) Me
-((hh, mi (h), } p(ve h w) p(he, (o Des-i)dyydzcll
log (2TT²) + 1
J / a(brrw))( whr,,, "mi (h., p/be, w) ga(2 D ddddhe
2
20 (0(),1)
40

---

Published in Transactions on Machine Learning Research (10/2025)
In similar way, show the decomposition related to Re, ()
The remainder idea of the proof is to decompose the PEHE such a way we can upper bound it with an expression including (J, b) and (J, f)CF and then use lemma 17 We actually have:
< = H Ri- ge-)) (H, 2) (H, [])
BH Ba ge-))
(mt (be:2) 2) mhs 2) (h. 1,x) + (h,, 0.2))
2B B2 (2)2g
[(/ (be 1, m/ (b., x))²] } + 2EH ~2 B2 1) [C (h,, 0.2) -mi 2))²]
BH, Bz - Ps 1)
[a(be, 1)( (he, 1.
m. (he, z)) (o(h).)((/ (he, L, mi (Ir, =
+ Fn. (2)2s1
+ EH.,. (112
()(b,,0( (he, 0, - my (h,, =))²
+ EH. E (1
[a(be, Lf (he, 0, my (hr. =))²
B2, (HH, (f (H, Wi, Z. m W: (H,, Z])]
+ EH. 1- Bos (2 HH, (f (H,, We Z)
m, (H, 2))]
20
Re.(J. 1) (S. (.. D)r log (2ma²)
Σ
(Varp, (t. + Var (w))
wE(O,1)
(8)
20
Re.(S, (S, ()F log (2no²)
(4)
(L) (5, Ba IPMc (gol: W: 1), go(: Wi 0)) log (2TA²) }
20
The inequality follows from the property (a (a- S 2a² + 262. The equation (2) follows from plugging
(1)
(3) in the equations from Lemin 18 the inequality follows from the positivity the variance terms, and follows from Lemma 17
(4)
A.6 Proof of Proposition 12
Using Monte Carlo approximation, we can express the approximate factual risk as:
1
By (S.
(2 [a(ha, w) (wva ((b())) 2, w.,)
W
By the stationarity assumption, for 12 lo, this approximation holds:
R (1
Ezzqs [o(ha, w) log (yue ((F(ha) 2,w), o2)]
On the other hand, the reconstruction term in the ELBO can be written as:
Ez-as (o,H, Wog H, We Z)
Ez~as (Z))r) [a w) log f(b(h)) f((ch)) Z, Z,w), o²)]
t=
t=
ieB
Ezras Dar) [a log N (Bva; ((F(a)) Z, w)
WEW iEB, W.
n2 (8) R. (f.
m (2) R (f, (f
41

---

Published in Transactions on Machine Learning Research (10/2025)
B Discussion of the validity of the Risk Factor Substitute
B.1 Estimating (()((W))
In causal inference, when unobserved confounders are present, the conditional distribution of potential outcomes plays critical role. If the adjustment variables U are observed, the polential oulcome can be represented as:
p(Y((W)) = p ha We =w) = /piu h,, W. = w, = u)p(u h, W2 = w)du.
Due to the structural assumptions over the causal graph, the treatments W2 and covariates X, are d-separated from U given H,, implying:
p(u he, W, w) = p(u Dz-1),
where the conditional potential outcome identifiable, but estimating it requires sampling from the postcrior distribution of the adjus(ment variables, p(u D.-1).
When U is not observed, let Z be latent variable satisfying the CMM(p). Under what conditions does the following equality hold?
f plu h, W. w. Z z)p(z Dz-1) dz (2) / plv h4, W, w. U u)p(u Dc-1) du
(32)
= p((((w) h.).
This equation addresses the potential risk of non-uniqueness in the CMM assumption, which could lead varying values on the left-hand side (LHS) of Eq. (32). However, our framework differs from the traditional dcconfounder framework. Here, the identification of the conditional potential outcome p(Y, h2) occurs indeperdently of the latent variables, as sequential ignorability holds without requiring conditioning on them. The CMM assumption, under the condition Z 1l. X WST, ensures that no confounding backdoor paths are introduced by conditioning on Z. Thus, using Z as a substitute for U presents no identifiability or consistency issues. This conclusion also extends to the marginal distribution potential outcomes, which can be formally shown through copula argument, as in D'Amour (2099a).
p(Y((w)) / P(Y((w) H. = h.)p(b.)dhh.
p(Y((w)) / p(Y. H. h, We w(h
p(Y((w)) / p(ysnxgnWes) p(he,wr)
p(hee) dhi,
(33)
p(Y((w)) / Plusuz /
p(h,) dh,dz, p(he, wi)
= / c(venz) x h wi )pz x (ee: dh,dz
The copula c(Ust, Xge, is restricted by the CMM(p) assumption. This constraint differs significantly from the deconfounder theory, where factor models often separate multiple treatments or treatment sequences and impose no such restrictions on the outcome model. Therefore, identifying potential outcomes in the deconfounder framework requires additional assumptions, as discussed in Wang & Blei (2019a)) b). However, our framework enforces conditional separation of responses given past sequences treatments and covariates, implying independence between the latent variables and treatment sequences. Finally, in practice, how
42

---

Published in Transactions on Machine Learning Research (10/2025)
estimate (wY? We express this as:
/ plue H. - hr, W; - w)p(he)ddh
p(Yi(w))
/ / plue w,,,2z)p(z h ()) ddh,d,
(34)
/ / p(ye h,, z)p(2 D (h dh,
~ / po(Be 2) Pr)p(hh)
Here, Po(ve h,, 2) is modeled by the outcome model (decoder), while qo(z | Dr) is the approximate posterior distribution of the latent variables, estimated using the observed sequence of data.
B.2 Capturing "bad variables"
In this section, we examine whether the substitute adjustment variables Z can inadvertently capture 'bad variables," that is, variables that may introduce bias into the estimation of causal effects.
M-Colliders One potential source of bias arises from M-colliders. If the substitute adjustment variables Z caplure information not only about the true adjustment variables U, but also about collider variable M, this could result in Z y Wer-
Figure 11: An example of an M-collider structure, where the variable influences both the treatment and the response, as indicated by the dashed arrows.
The key concern is whether Z could "blindly' capture information from the sequence of responses treat- ments, Such cases may lead to open collider paths like U - Y, + Wst or U + Y, + X These scenarios would violate the assumption that Z Wst and X
In essence, the inclusion of colliders in the substitule adjustment variables can open "backdoor" paths, leading biased estimates of the treatment effect. To prevent this, it is essential ensure that the substitute variables Z do not inadvertently encode information about colliders that affect both the treatments and the responses.
C Experiments on synthetic data: Details
C.1 Description of the Simulation Model
C.1.1 Simulation under sequential ignorability
We simulate a longitudinal data set lime-length T 75 by generating the time-varying variables autore- gressively. Specifically, the confounders X, each time-step are generated in R with d, 100, and the dynamics specified through autoregression of order p plus regression over the past treatment trajectory. Specifically, each dimension E .....,d.) of X, is defined as:
P
x
x.(1.) x 1- f
+
xW.(t.) W.+ + cx,
k=
43

---

Published in Transactions on Machine Learning Research (10/2025)
X((i)) Tk
~ (0,11)
XW.(t)) ~N(0,1) ;= ==
%
To ensure dependence between confounders, the vector error ot is generated by a Gaussian distribution with a non-diagonal covariance matrix
DS = plu + - 11-
where p = 0.5 and o² = 0.3.
The treatment Wt generated using a Bernoulli distribution with a probability ( defined with a logistic model to simulate the assignment mechanism. a() refers the sigmoid function:
11
1 P
W 1 k=1
We-k d&P
Wx,t 2
X,-k)+
1k
WY YY= + cW.
To ensure the variation of imbalance between treatment and control groups through time, simulate the regression parameters as follows:
Wit. WY, Tk -kk
Wx,t ~ (N(in(( 0.001) ew ((0,000011) ((sin L 0.001
((sin(())..0r
W B(o(T))
To specify the outcome model, we first generate the unobserved adjustment variables in RR with d, 100 using a Gaussian mixture of three distributions:
U
K Σ Nuu, Em.)
where /i,-...,KK; U{[-10, U[[1100 10) and Σ, 0.2 diag (I,,), and K 8. Finally, write the expression of the two potential outcomes function of the context history. The chosen specification is motivated by the classic mixed-effect approach. We introduce the random effects to have an individual reaction to the change in a covariate Xty, and we model by the Hadamard product between the covariate vector and the unobsorved adjustment voctor. The observed response is defined following the consistency assumption
Y. = Y((IW,, + Y2(0)1 - W.)
P
Y((w)= = _
(12 YU
U
W + d,P k=1
YX.t X._ + Niw
New t- +
Yst
dy
P
PA=
YX, Nkw
YX ,0.013) ~ ~N(1
YU ~ ~N 00.013)
Ya (k((w)
N 0.12) ey N(O0,0011)
With xY M = 0.0, and No YX n 0.2, 0.1 and vo YU = 0.1 while > is varied generate multiple datasets with different modification levels of treatment effect induced by U.
C.1.2 Simulation under hidden confounding
We partition the covariates into observed and hidden blocks:
X, (X; x!),, X E Rdo x E Rdh, do + da dr-
We keep the block-diagonal covariance
Ers 0
Σ
Σnh
44

---

Published in Transactions on Machine Learning Research (10/2025)
that X is not proxy for X; at baseline.
Define the hidden score aggregated over the current steps:
dn p-1 St
= dp. j= 1m
We inject the hidden score S, into the treatment logit with a selection-sensiivvity parameter y
Sy
W₁ ~ (ernoull(o((:)))
baseline under ignorability
Interpretation (Rosenbaum-T): If St is standardizod, y log I where I 2 1 is the maximum treatment odds ratio due to unobserved confounding.
We allow St to load differently on the two potential outcomes via coefficients So, B1:
Yı(w) = 1 + B. E! w E 10.
The ITE is then
Tt = -X, (X," (1) - (0)) + - -B)- (E(") - =
If B1 # fo, the hidden block changes the ITE, making PEHE sensitive hidden confounding.
C.2 Additional results
Results of baselines on the synthetic datasets The following Table i provides the detailed results responsible for Figures related basclines with the three different approaches across levels of YYU
(1)
Table 7: Results the synthetic data reported by PEHE. Smaller is better.
T.l.
10
70
700
CIVAR (anm)
0.43-0.02
0.50-100X
0.966-0U0 0.00
1.57-0.00
1.90-0.10
2.35-0.18
1.57
1.0-0.20
6.000020
7.64:0.38
9.03-0.50
Canoal CFU
0:13:001
DIELOS
2.09-0.1 1.
23
150510.6
Causal CPC (with subetitute)
0.662002
0.0910.01
10250.05
2.58-0.06 0.08
2.556.000
4.51:0.1
5.38-00.2
8:
10.11:041
12.0:4054
16085:001
Consal CPC (oracle)
0,450002
0.340.00
0966-00
1.00-000
21450.08
4.41-010
5.630.30 30
101540047
19918955
Transforucce (with substiture)
0,620002
06728.02
7:7240118
11
Tronsformer (oracle)
0,062002
5.11:0009
7.349.13
115560.2
G-Net
0825000
039300.0
9000
18236-13
5-70:013
10:30:125
15.17
21
32
G-Net (with substitute)
0.362004
0.7940.01
36112202
4.0020.30 20
42766.18
5.50
5.9123064
13.6520.07
17
19.25422.4
20:21:2110
G-Net (arocle)
0.880002
0.92340000
3. 1002005
4.36
4.6660112 12
5.2458117
2.25.0.00 21
13.1060.00 50
17.4720066
15.22200.66
35.16E1.77
CRN
0.530002
0.8530.00 54
1622004
2.00200 31
5.1456-17
6.00
5.8300.3 36
1190300077
16
18.0840..7
20.662
CRN (with substitute)
0.482001
0.6510.01
1511:002
2.56:0.J8 08
2.886.21
6.3620.15
6.8:20.66
8,50=1115
18.515076
15.2310.75
26.115:141
CRN (oracle)
0:53-001
0.0310.00
10:00
67516.13
0.10-011
59219.22
M66 035
13.19-033
15,0140.61
25.96
15.96-1177
RMSN
0.57:0022
0.6740.02
2.07:0000
4:110015
11:250007
26.0720077
RMSN suloditute)
0.16:001
0.0740.02
1511:004
2.51
5.156.20
4.61=020
6.1429.50
0.58=0822
18.6341.50
21.08:1.00
Robustness Number of Prior Components The Table g gives detailed results summarized in Figure 9a that assess the sensitivity of CDVAE the variation of the prior cluster numbers K.
Table 8: Results of CDVAE when varying the number of components K of the prior. The study is conducted the synthetic data and is reported by PEHE. Smaller is better.
3.
0.25
0.5
0.75
70
1.26
70
10
70
10
CDVAE (ours)
0,4820.02
0.50%:000
0.96:0009
1.5720.08
1,9020.10
0.36-0.18
5.57-0117
0.80=0.20
6.84:0.20
7.64:0.48
9.00
COVAE (K 2)
0.BLL0.11
0.5010001
01060000
1301001
2071007
0.140.14
1.10..1
144.045
10.5
7.9010.55
5.88.00.5
CDVAE (K = 5)
0.4240.01
0.48:0.02
0.00:0002
14140.11
2005000
2.52-0066
3.27-022
4.250045
6.52:19:25
7.97
8.12:0048
CDVAR (N
0.40+0.01
0.46+0.01
00060006
1,57+003
13130004
0.50-0007
3.30-0225
4.15+045
6434044
6.3340.25
5.9140.19
CDVAE (IC 11)
0.4249.122
0.4746.00
1.406006
2002000
2.47:0222
3,76m022
54850132
6.4820114
7.7881 232
9.01261.22
45

---

Published in Transactions on Machine Learning Research (10/2025)
C.3 Computational Scalability
We include additional experiments varying both the sequence length T and the covariate dimension dz. Empirically, CDVAE's one-step PEНE remains essentially flat when T more than doubles (75 + 175), while it increases modostly (about +57%) when d2 quintuples (100 - 500). This behavior matches the model's complexity profile: GRU unrolling makes both run-time and memory scale linearly in T, and the IPM term cerm-computer on a fixed-size batch of 16-dimensional representations- costs O(b²) in batch size b but does not depend on dg. Only the first dense projection and GRU input weights scale with dæs explaining the
near-lincar slowdown.
Metric
T 75
T = 100
T 125
T 150
= 175
PEНE
1.09 + 0.09
0.98 + 0.15
1.00 + 0.20
0.98 + 0.21
0.97 + 0.09
Wall-clock (min)
35 + 3
57 +
82 + 2
95 3
105 3
Table 9: Performance of CDVAE as sequence length increases.
Metric
d, = 100
d, 200
d₂ 300
d 400
d = 500
PEHE
1.09 + 0.09
1.51 + 0.03
1.60 + 0.04
1.71 + 0.05
1.71 + 0.05
Wall-clock (min)
35 + 3
33 + 2
34 + 2
32 2
33 + 2
Table 10: Performance of CDVAE as covariate dimension dg increases.
D Experiments MIMIC-III data
D.1 Description of the MIMIC-III semi-synthetic model Model
A cohort of 2,000 patients is extracted from the MIMIC-III dataset, with the simulation setup proposed
by Mclnychuk et al. (022)) extending the model introduced in (Schulam & Saria (2017). Let dy denote the dimension of the outcome variable. For multiple outcomes, untreated outcomes, denoted as zz((0) for I,.... dy. are generated for cach patient in the cohort. The generation process is defined as:
zz B-splinnnee + ofgh(( + aff
X
+
(35)
endogenous
noise
exogenoos
where the B-spline B-spline(t) models the endogenous component, ()) is sampled independently for each patient from a Gaussian process with a Matérn kernel, and f (.) is sampled from Random Fourier Features (RFF) approximation of a Gaussian process.
To introduce confounding into the assignment mechanism, current time-varying covariates are incorporated via a random function ff (X,) and the average of subset of the previous Tj treated outcomes, Ar, (Y--1). For dg binary treatments A/ where I I...., day the treatment assignment mechanism modeled as:
PAj An (Y--1) + A' (X+)+ (1)+0 bi)
A Bernoulli (PA:).
where of-) denotes the sigmoid function.
Static fcatures U, such as gender and cthnicity, which are categorical, are onc-hot encoded. random Singu- lar Value Decomposition (SVD) transformation (E(+ is then applied to U, retaining all singular components. Subsequently, treatments applied to the untreated outcomes using the following expression:
min H A =1 PAy Big + EtE fE(U)
[A/1]]PA] (w
(1) Σ
(36)
i)²
46

---

Published in Transactions on Machine Learning Research (10/2025)
where w' is the treatment window.
The final outcome combines the treatment effect and the untreated simulated outcome:
Y Z3 Et)
(37)
D.2 Additional results
Results of baselines on the semi-synthetic MIMIC-III The following Table 11 provides the detailed results responsible for Figures related to baselines with the three different approaches.
Table 11: Results on the MIMIC-III data reported by PEHE. Smaller is better.
Model
PEITE
CDVAE (ours)
17.63:0.22
Causal CPC
19:27.10.25
Causal CPC (with substitute)
18.15:0.26
Causal CPC (oracle)
17.98+0.21
Causal Transformer
19.68=0.22 20 +
Causal Transformer (with substitute)
18.58+
Causal Transformer (oracle)
17,91 0.20
G-Net
19.75:00221
G-Net (with substituto)
18.00+0.225
G-Net (oracle)
17.95.1 0.23
CRN
19.80+00.23
CRN (with substitute)
18.5810.22
CRN (oracle)
17.91 0.21
RMSN
19.85=00225
RMSN (with substitute)
18. 18.06=00.2 23
RMSN (oracle)
0.19
E
Algorithmic Details
E.1
Algorithms Description
In this appendix, we provide the algorithmic details for CDVAE. First, we describe the approximation of the Integral Probability Metric (IPM) term used Eq. (24). The IPM term aims to reduce covariate imbalance between the treatment and control groups by quantifying the dissimilarity between their distributions. To calculate the IPM, we use the Sinkhorn-Knopp algorithm (Sinkhorn 1967) and the Wasserstein distance computation algorithm (Algorithm in Cuturi & Doucet (2014)). The Wasserstein distance computation (Algorithm calculates the pairwise distances between data points and constructs a kernel matrix using a regularization parameter. It then computes the row and column marginals based on the weights assigned to each data point. The Metchorn-KKopp algorithm (Algorithm D integrated within the Wasserstein distance computation, used to compute the optimal transport matrix. Finally, the Wasserstein distance is obtained by summinng the products of the optimal transport matrix and the pairwise distances. Computing the Wasserstein distance at each time step and for every batch computationally expensive. To accelerate training, we compute it for subsample of the time indices, sampled randomly at cach batch. Empirically, this approach is sufficient for maintaining model performance. Specifically, we sample 10% of the time indices at each batch, corresponding to 7 time steps, as shown in Algorithm 3
2
10
E.2 Computational complexity analysis of CDVAE
We provide both theorctical and practical computational complexity analysis. The training complexity of CDVAE can be summarized in the following proposition.
Proposition 19 (Per-cpoch complexity) For mini-batch of size sequence length T, raw covariate dimension de, GRU hidden size H, and fixed representation width do(= 16), let e denote the entropic regularization strength in the Sinkhorn algorithm for the IPM term. Then, one training epoch of CDVAF
47

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm Sinkhorn-Knopp Algorithm
Require: Kernel matrix E R
Require: Row marginal vector g", Column marginal vector E R
Ensure: Optimal transport matrix T E R"
1: Initialize transport matrix 7()) with all entries set to 1
2: Set iteration counter k t- 0
3: while not converged do
4: Update row scaling vector u E R
ti
Update column scaling vector v R"
bj
Bj
Ene
Update transport matrix ((++1) (k+1)
a;K,y"
S uy Kay
10:
Increment iteration counter k +-
11: if convergence criterion met then
12: Return (1++1)
13: end if
14: end while
Algorithm 2 Weighted Wasserstein Distance Computation
Require: Batch = {[wa, ya, x x 1,....|8) [
Require: Representation learner +
Require: Weights vectors ag (h,,w)
Require: Regularization parameter X
1: for E (1.2..... T} do Compute n, (t) EieB Wa, n1
Eies(1 Wat) Compute pairwise distances matrix M R 19 xn
M H, H Vi,je B; W,, 1 and
Wjr 0
Initialize kernel matrix K R xn(" such that K(()
t -
Compute row marginal vector (!)) E R. such that a;
as (he,)))
Ex-t, ( (bia)!))
Compute column marginal vector ((0) E R such that 154)
(b....0)
Eesr (bhha.))
7: Compute optimal transport matrix ((t) E R xn
7(0) Sinkhorn-Knopp( (1t1)
((t))a())) (1)) b(1)
Compute Wasserstein distance D, < En. E the ( M
10: end for
11: Return Σ D.
48

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm Pseudo-code for training CDVAE
Require: Training Data Dr = ([wis Yie, Xa 1
,,....
Require: CDVAE parameters 6, Oy, Own Φ, a, Optimizer parameters
1: for e (lo.hh, do for batch = [[Ws,, Yir. xx -
do
Compute approximate posterior go(2
Sample latent variables from 90(2 Ver Vgr. XgT, WgT) Compute representation ((H,) for = I...,T Compute ELBO(0, Φ, D)
6:
7:
Compute LDiSM( ())
Choose f tm, U(I,T]) compute IPM term as
LIPM Σ
IPMG (go .l We, 1), 90 We: =
Compute total loss Liot ELBO(0, o, Φ, σ) + (IPM 4) + (DaMM)) Update parameters related to the total loss
10:
( (0. Bw, Φ. +
00 d. 2 o] < 10,0
00, $
11:
Compute binary cross-entropy loss for the propensity network Lw (Ow, 2) and update parameters
(0,:(Ow.
(Ow. ②)
Ow + Ow - pw
20
12:
end for end for
13:
14: Return Trained CDVAE model
49

---

Published in Transactions on Machine Learning Research (10/2025)
requires
HnT de))
+
On do) + Om b² log b/=²)
GRU forward back- prop
D-projection
Sinkhorn IPM m= 00.17"
and stores O(T H nb) activations.
Discussion. The first two terms are linear in the sequence length T and (near-)lineur in the covariate dimension dæ because the GRU unrolls once per time step and the project ion layer is dense. The third term corresponds to the entropically regularized Sinkhorn solver; its cost is quadratic only in the batch size b and independent of dæ once the feature map has been compressed to dg.
Since we evaluate the IPM on only m [0.17] randomly chosen time points, the overall runtime grows linearly with I' and almost linearly with dy for realistic batches (b < 1024). The sole quadratic factor arises from the 8 dependence inside Sinkhorn.
Practical results. On the simulation datasct, Table 12 reports the mean training time (in minutos) of all models, averaged over 10 random seeds. All experiments were single NVIDIA Tesla T4 GPU.
Model
Training time (min)
CDVAE (ours)
35 + 3
Causal CPC
8
CT
27 2
G-Net
12
CRN
7
RMSN
6 +
Table 12: Mean training times of different models on the simulation datasct (10 soeds, 1x NVIDIA Tesla T4).
Interpretation. CRN, RMSN, and CPC lack the IPM term and thus train faster, while the Causal Transformer incurs higher cost due transfer blocks. CDVAE's additional complexity comes from explicit distributional balancing over (HH)), which we found necessary for stable ACATE estimation. The random- time subsampling of representations keeps the added cost manageable in practice.
F Models hyperparameters
We use Pytorch (Paszke et al. 2019) and Pytorch Lightening (Falcon & team 2019) implement CDVAE and all basclincs. For the selection of hyperparameters, we fine-tuned our models using random grid search. We use the weighted reconstruction error the selection criterion for CDVAE. We do not use metric related to the quality of estimating ACATEs as selection criterion; choice cannot be used in a real scenario because ACATEs are usually unavailable. It is still rather an open problem how to design a criterion for causal cross- validation or hyperparameters tuning for a causal model. Many proxies are used in the literature, including loss over the factual outcomes (Lim 2018 Hassanpour & Greiner 2019a Bica al. 2020a b), one nearest-ncighhor imputation (Shalit al. 2017 Johansson et al.) (022), for the counterfactual outcome, influence functions (Alaa & Van Der Schaar 2019). rank- pank-pres causal cross-vals validation (Schuler
et al. 2018), Robinson residual decomposition (Nie & Wager 2021 Lu et al 2020 In this work, and since we intend to search for the best regularization parameters (ΛΙPм: AMAS) among other parameters related to CDVAE architecture and optimization, we chose not to consider the total loss as this may bias the choice of (ΛrРм, AMM) loward small values. We, therefore, use the weighted reconstruction error as an objectivv function for fine-tuning. Similarly, we fine-tune Causal CPC, Causal Transformer, CRN, and RMSM using the loss over the factual response as a critcrion.
We report in the following tables the search space of hyperparameters for all baselines.
50

---

Published in Transactions on Machine Learning Research (10/2025)
Table 13: Hyper-parameters search range for RMSN
Model
Sub-model
Hyperparameter
Synthetic data
MIMIC
LSTM layens
Learning
0.00, 1.001,0.000.
0.01, 0.005, 0.001,
Batch size
32,
64,
RMSNs
Propensity Treatment Network
LSTM hidden units
6,8, 12,
6.
LSTM dropout rate
Max gradient norm
0.5, 1,2
0.5, 1,2
Early Stopping (min delta)
0,001
0.001
Early Stopping (patience)
10
30
LSTM layers
Learning rate
0.01,0.005, 0.001, 0.0001
0.01, 0.005, 0.001, 0.0001
Batch size
32, 64, 128
64, 128, 256
Propensity History Network
LSTM hidden units
6,8, 12, 14
4, 30
LSTM dropout rate
Early Stopping (min delta)
0.001
0.0001
Early Stopping (patience)
10
30
LSTM layers
Learning rate
0.01. 0.01,0.005, 0.005, 0.001, 0.0001
0.01, 0.005, 0.001, 0.0001
Bazch size
64, 128,
32, 64.
Encoder
LSTM hidleen
18,20
8,
LSTM dropout
Early Stopping (min delta)
0.001
0.001
Early Stopping (patience)
10
30
LSTM
Learning rate
0.01, 0.005, 0.001, 0.0001
0.01, 0.005, 0.001, 0.0001
Batch size
32, 64, 125, 256
128. 512, 1024
Decoder
LSTM hidden units
6,8,
18,20
6,8, 18, 20
LSTM dropout rate
Max gradient norm
0.5. 1,2
0.5, 1,5,1,2
Early Stopping (min delta)
0.001
0,0001
Early Stopping (patience)
10
30
Table 14:
search range for CRN
Model
Sub-model
Hyperparameter
Syuthetic data
MIMIC IIII
LSTM layers
Learuing rate
0.01, 0.005, 0.001 0,0001
0.01, D.I05, 0.001, 0.0001
Batch size
32,61, 128, 260
32, 64, 128
CRN
Encoder
ISTM
units
6,8,.., 18,20
5,5, 18,20
LSTM dropont
BR size
6,8. 18, 20
6,8,-. 18, 20
Early Stopping (miin delta)
0.001
0.001
Early Stopping (patience]
30
LSTM layers
Learning rate
0.01,0.00,, 0.003, 0.001, 0.0001
0.01, 0.005, 0.001, 0.0001
128,
512,
Decoder
LSTM liddlen units
6,8. 18,
6.8.
38,20
LSTM dropont rase
6,8,... 18,
6,8,... 18,20
Early Stopping (miin delta)
0,001
(.000
Early Stopping (patience)
10
30
Table 15: Hyper-parameten search range for G-Net
Hyperparameter
Cancer simulation
MIMIC III (SS)
LSTM layers
Learning rate
0.01, 0.005, 0.001, 0.0001
0.01, 0.005, 0.001, 0.0001
Batch size
32, 64, 128 6, 8, 18, 20
32, 64, 128 6,8,
LSTM hidden units
18, 20
FC hidden units
6, 6,8.....
18, 20
6, 18, 20
LSTM dropout rate
R size
6, S,..., 50
18, 20
4, 6 ......53 30
MC samples
50
Early Stopping (min delta)
0.001
0.001
Early Stopping (patience)
10
30
51

---

Published in Transactions on Machine Learning Research (10/2025)
Table 16: Hyper-parameters search range for Causal Transformer
Hyperparaneter
Cancer simulation
MIMIC III (SS)
Transformeer blocks
Learning rate
0.01, 0.005, 0,001, 0.0001
0.01, 0.005, 0,001, 0.0001
Batch size
32, 64, 128
32, 64, 128 2 4, .......
Attention heads
Transformer units
4, 6, ..... 20
LSTM dropout rate
BR size
6, 8, ..... 18, 20
......... 4, 6, 20
FC hidden units
6, 6,8, 18, 20
Sequential dropout rate
0.1,0.2,0.3
0.1, 0.2, 0.3
Max positional encoding
15 0.001
20
Early Stopping (min delta)
0.001
Early Stopping (patience)
10
30
Table 17: Hyper- pareerr-rammeerrss search range for Causal CPC
Model
Sub-model
Iyperparameter
Cancer simulation
MIMIC III (SS)
GHU layers
Learning
0.01 0.005, 0.001, 0.0001
0.01, 0.005, 0.001, 0,0000
Batch size
32, 64, 128
64, 122% 256
CRU liddlen
18,20
20,
Cansal CPC
Enonder
GRU dropont nate
Local lestlures (LF)
6,8,
6. 20
Context Represettation (CR) size
6,8, 18,20
4. G........ 20
Early Stopping (min dolta)
0.001
0:001
Early Stopping (pataence)
10
30
layors
Leaaning rale (deconles w/o tresiment sub-neswork)
0.01, 0.005, 0.001, 0.0001
O.DI, 0,005, 0,901, 0,0001
Learning rate (encoder line teeintunngg
0.001, 0.0005, 0.000L, 0,00005
0.001, D.0005, 0,0001, 0.00005
Lestaing tale (Trew-ment sub-nelwerk)
0.05, 0.01, 4.000
0.05, 0.015, 0,0001
Batch size
32, 628
32, 64, 128
CRR hiddlen
CR size
CR size
Decoder
GRU dropont nate
CR size
CR sizee
GRU layers (Theat Encoder)
GRU litkdeu (Trest Encuder)
FC hidden units
6,8, 18,20
4. 6........ 20
Randow time indices (mn).
10%
Early Stopping (min delin)
0.001
0X001
Early Stopping (pasience)
Table 18: Hyper-parameters search range for CDVAE
Model
Sub-madel
Hyperparamctor
MIMIC
GRU layers
Infurence Network
CRU hidden unite
6,8.... 14
4,6, 20
GRU dropout rate
CDVAE
Latent dun z
0,001
D.001 4, 6,......
Propuusity Network
hidden units
6,8,
LSTM layers
GRU hidden units
6,8. 18, 20
6, 8.8. 18,20
Representation Learner
GRU dropout rate
Dimension of representation
6.8.... ...
1,6. 20
bidden units
(dim(x) + dimi ()/2
(Aimizz) + dimm (HH)))
Learning (w/o propensity network)
0.01, 0:005, 0. 0.001, 0.0001
0.01, 0.005, 0.001, 0.000)
Learoing rate [propensity network)
0.01, 0.000, 0,001, 0,0001
0.01, 0:005, 0.001, 0.0001
Global
Eatch size
32, 64, 128, 256
128, 512, 1024
Max gradient norm
0.5. 1,2
1,2
Number of components Prior
2,4.... 18,20
2,
52

---

Published in Transactions on Machine Learning Research (10/2025)
G
The Neural Architecture of CDVAE
The extended neural architecture of CDVAE comprises multiple components which we did not explicit in We first begin by detailing neural network functions related to the generative model. The
Section
4.4
Table
20 outlines the architecture for the Representation Learner + which encodes the context history, Table presents the identical architecture for both foy and fez responsible for generating the two potential
oulcomes. Meanwhile, Table 21 ralustrates the design of propensily network (.) built on the top of the shared representation. Lastly, Tables 22 depict the architecture used to learn both the mean and covariance matrix for the approximate posterior assumed to Gaussian.
Inputs: {4(he))1ss7) Z
Concatenate: (4( 1<iST
Linear Layer
Weight Normalization
ELU
Linear Layer
Weight Normalization
Linear Layer
Output:
Table 19: Architecture of the outcome model prediction, i.e, the decoder.
Inputs: (1/, X1, WITESES
Inputs: {
Inputs: {yh, Xe, Wi, 1S1ST
Concat: [V, X WIlSSTT
Linear Layer
Concal: We. X1, 1515T
GRU layer
ELU
GRU layer
Linear Layer
Sigmoid
Linear Layer
Linear Layer
Tanh
Output: (Wi+(()))ssS)
Egs (gr)
Hbz (gr
Outputs: hishsT
Table 21: Architocture: propensity Table 22:
Inference
network
Table 20: Architecture: representa- network ea.
9 (z VgT, xgr,
tion learner & of CDVAE
53


Reviews:
[{"review_id": "Mq0JGuhOwd", "reviewer": "Reviewer_axn3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Some of the key strong points of the paper are as follows:\n1. The paper provides rigorous theoretical grounding through the Conditional Markov Model assumption and proves that learned substitutes satisfy sequential ignorability and enable ACATE identifiability. The near-deterministic regime analysis is particularly practical, addressing a fundamental gap in existing deconfounder theory where probabilistic inference models conflict with deterministic theoretical assumptions. \n2. CDVAE integrates multiple sophisticated components: weighted empirical risk minimization with overlap weights for selection bias, Gaussian Mixture Model priors for population structure, adversarial training for representation balance, and distribution matching penalties for latent variable stationarity. The technical handling of discrete latents through Bayes-optimal posteriors demonstrates computational awareness. The experimental design thoughtfully adapts baselines to ensure fair comparison while maintaining parameter complexity balance.\n3. The experimental results are strong and demonstrate CDVAE's effectiveness across synthetic and semi-synthetic MIMIC-III datasets. Most importantly, the \"substitute approach\" shows that existing state-of-the-art methods achieve near-oracle performance when augmented with CDVAE's learned substitutes. This transferability represents significant practical value, as practitioners can enhance existing methods without complete framework replacement.\n\nGiven these strengths and the key contributions towards causal learning, there are a few ways the paper can be improved upon as follows:\n1. The abstract and introduction could better motivate the specific focus on adjustment variables versus confounders. The distinction is crucial but may not be immediately clear to readers familiar with traditional causal inference literature. The introduction would benefit from concrete examples illustrating scenarios where adjustment variables are present but confounders are adequately observed, and why this setting is practically relevant across different domains.\n2. The paper lacks detailed computational complexity analysis, which is critical given the sophisticated multi-component architecture. While individual algorithmic details appear in appendices, a comprehensive analysis of training time, memory requirements, and scalability compared to baselines is needed. The adversarial training procedure, IPM computation, and multi-objective optimization likely impose significant computational overhead that should be characterized.\n3. The paper would benefit from more thorough discussion of when CDVAE might fail or perform poorly. The CMM assumption, while theoretically justified, may be restrictive in practice. The method's sensitivity to hyperparameter selection (λ_IPM, λ_DistM) and robustness to violation of the adjustment variable assumption deserve more attention. The connection to critiques of deconfounder theory suggests potential vulnerability to \"bad variables\" that could bias estimates.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes the Causal Dynamic Variational Autoencoder (CDVAE) for estimating Individual Treatment Effects (ITEs) in longitudinal data with unobserved adjustment variables. The key innovation is distinguishing between unobserved adjustment variables (affecting only outcomes) and traditional confounders (affecting both treatment and outcomes). CDVAE leverages a Dynamic VAE framework operating in a \"near-deterministic regime\" to learn latent representations of these adjustment variables, termed \"substitutes.\"\n\nThe paper made several key contributions as follows:\n(1) A principled approach to infer latent adjustment variables through Conditional Markov Models (CMM); \n(2) Theoretical validity proofs ensuring treatment effect identifiability when conditioning on learned substitutes; \n(3) Novel exploration of near-deterministic VAE regimes for causal inference, proving treatment effect consistency regardless of specific latent realizations; \n(4) Empirical demonstrations showing CDVAE outperforms baselines and enables near-oracle performance when augmenting existing methods with learned substitutes; \n(5) Comprehensive theoretical framework including generalization bounds and weighted risk minimization for covariate balance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "There are a few aspects that might benefit from clarification as follows:\n- Regarding the Adversarial Setting, could you provide more intuitive explanation of the adversarial training dynamics? The paper mentions simultaneously minimizing L_tot with respect to {θ, φ, Φ, σ} while minimizing L_W with respect to θ_ω, but the adversarial relationship isn't entirely clear. Is the adversarial aspect primarily about learning representations that predict outcomes well while being balanced across treatment groups? How does this differ from standard domain adaptation, and what prevents the representation learner from simply ignoring the treatment classifier?\n- What are the primary failure modes or limitations of CDVAE? Under what conditions might the CMM(p) assumption be violated, and how would this manifest in practice? How sensitive is the method to the choice of the finite order p in the Conditional Markov Model? \n- Additionally, could you elaborate on computational scalability  - how does CDVAE performance degrade with longer time series or higher-dimensional covariate spaces?\n\nSome other minor questions:\n- How does the choice of overlap weights affect performance compared to other weighting schemes like entropy weights or ATE weights?\n- Could you provide more guidance on hyperparameter selection, particularly for λ_IPM and λ_DistM, in real-world applications where oracle performance is unknown?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qjDYl0q4jd", "reviewer": "Reviewer_Z88p", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper investigates its key problem in a mostly comprehensive way: Clear writing, fair comparisons with respect to the existing literature and good explanations on how the considered setting differs from those.\n- The key points and possible question marks about the framework are answered preemptively, with proper justifications and theoretical insights. For instance, Section 4.2 clearly explains how the proposed approach handles the common issues (e.g., selection bias and imbalance), Theorem 9 (realization invariant causal consistency) in Section 4.3. I haven’t gone through the proofs in the Appendix, but the treatment in the main paper seems solid to me.\n- Albeit in a limited number of settings, CDVAE outperforms the baselines comfortably. Ablation studies also support the usefulness of each sub-component of the method.\n\n\nWeaknesses:\n- Uncounfedness is a significant assumption (even though it’s understandable). Exploring the sensitivity of the method to violations of this assumption via experiments is a missed opportunity. \n- Besides the first point, I don’t see another striking weakness of the paper. Though I should admit that I’m not too familiar with the recent literature on this subject, so it’s not likely, but possible that I am missing some points.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Causal Dynamic VAE (CDVAE) for estimating individual treatment effects (ITE) when there are unobserved static adjustment variables that affect the outcome sequence. Critical assumption is that these unobserved adjustment varibles do not confound the treatment.\nThe static (time-invariant) adjustment variables are somewhat justified through examples, such as genetic factors and environmen conditions.\n\nThe paper lays out the main challenges in working with longitudinal data and points out that the existing literature mostly addresses them under ``sequential ignorability’’ where all confounders are observed. For the missing covariates though, this common assumption is violated.\nTo address the challenge of missing covariates, the paper starts by answering the question: Under what conditions can a latent representation be a valid substitute for the unobserved adjustment variable such that the identifiability of CATE holds? After answering the question positively for a finite-order conditional Markov Process,  the paper addresses additional modeling points (e.g., using a Gaussian mixture model prior to account for the population structure caused by the unobserved adjustment variables).\n\nAn interesting component of the paper is studying VAE in the near-deterministic regime, and demonstrating that the treatment effect remains consistent regardless of the substitute. Studying this in causal inference is a nice contribution of this work, which has further implications regarding the related literature on deconfounders.\n\nFinally,  the proposed method CDVAE is shown to outperform the baselines (adapted to the setting of the paper) for both synthetic and a semi-synthetic dataset. This includes showing that augmenting the input covariates with the inferred latent substitutes (of the unobserved adjustments) enhances the performance of the baseline approaches.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "One mid-level request (I’d prefer to see this added):\n- Regarding the first weakness point. I suggest the authors to add some experiments with a small confounding effect. I wouldn’t require the method to be very robust or maintain great performance, but seeing how it fares in such assumption violations can be useful to guide practitioners.\n\nSome minor questions and requests (none of them is too critical):\n- Performance gap between CDVAE and the baselines: This gap seems to be much smaller in the semi-synthetic data (Figure 4) as opposed to synthetic data (Figure 3). Do you have an interpretation for this?\n- How sensitive is your method to hyperparameters? I see from the tables in the appendix that grid search for each baseline and CDVAE seems comparable, so it’s a fair comparison. That being said, it’d be nice to see some interpretation on the effect of the grid search, e.g., is there a big drop in the performance when using sub-optimal hyperparameters?\n- Some of the tables are too small, almost impossible to read without zooming (e.g., Table 3). I suggest enlarging such tables.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JIGX7kbXhh", "reviewer": "Reviewer_CXvK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1) The paper is very well written and easy to follow. \n\n2) The idea is intuitive and the approach to proofs is thorough (although I am unable to comment on their correctness).\n\n3) The latent substitutes/representations for the unobserved adjustment variables can be used in combination with existing methods. I think leveraging representations from one method to augment another is an obvious but often overlooked opportunity in many related works.\n\n4) The results indicate consistent improvement of performance when utilising the additional representation. \n\n5) The authors take a comprehensive approach to evaluations, including synthetic data, modified real-data, ablations, and a series of other checks. The comparisons include an accounting for the parameter counts associated with the modified comparison methods.\n\n6) The authors include code attached as supplementary, although I have not evaluated/reproduced any of their experiments with it.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Its been at least a couple of years since working in this particular space, so I defer to the other reviewers in terms of some of the more technically challenging components of this contribution. Nonetheless, I recall that much work at the intersection between causal inference and variational approaches concerned unobserved confounders. Whilst confounders present more problematically (in terms of bias), it is interesting to see the current authors tackling something a bit different: unobserved adjustment variables or, if I'm not mistaken, these are also referred to as unobserved risk factors. i.e. these are factors which affect the outcome sequence, but not the treatment assignment. If successful, such a method would be able to estimate the effect of treatment over time with more precision, as the method would be accounting for further sources of outcome heterogeneity. \n\nThe authors approach this problem with a dynamic variational autoencoder CDVAE. In addition, they prove theoretical validity under certain Markov assumptions, and demonstrate the effectiveness of the method empirically.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "*Questions:*\n\n1) I see a lot of switching between ITE and CATE/ACATE. Whilst I found Section 3 very clear in terms of its definition of ACATE on page 5 (eqs. 2-4), I found the switch to ITE in 4.3 abrupt. I remember that Shalit et al.'s work on CFR net seemed to confuse CATE with ITE, and this co-occurred with a flurry of causal-ML papers which did the same thing. Can the authors clarify (a) the difference , and (b) perhaps reference the ITE in Section 3 if an ITE is indeed the quantity they seek to estimate? Without wanting to be patrionizing, by my understanding,  the expected value of an outcome for an individual E[E[Y_i|W=0, X_i]- E[Y_i|W=1, X_i]] is a CATE, where the condition achieves a specific stratification over an observed set of covariates, rather than an ITE. I assume the authors will explain that the transition from CATE to ITE is achieved via the near-deterministic regime, but I'm not 100% sure this is exactly what is happening.\n\n2) Can the authors comment on their intuition about Figure 3 - why does CDVAE perform so much better than other methods in the oracle and substitute conditions?\n\n3) To what extent is the method sensitive to consistency between the distribution of U and the choice of the prior for Z?\n\n\nMinor:\n- I wonder if the authors should include G-transformer https://arxiv.org/abs/2406.05504 as another comparative method in their related work section.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)