Generative Causal Structure Learning with Dual Latent Spaces and Annealing
Authors: Soma Bandyopadhyay
Sudeshna Sarkar
Venue: Accepted by TMLR ()

Abstract: In this work, we address causal structure learning in the presence of unobserved confounders. Such causal structures can be represented by Acyclic Directed Mixed Graphs (ADMGs), where observed cause-effect relations are depicted by directed edges and unobserved confounded relations by bidirected edges. Prior methods for causal structure learning with unobserved common causes have primarily focused on search-based approaches, and more recently on flow-based generative models. We propose a novel generative method based on a variant of the Variational Autoencoder (VAE) with dual latent spaces to represent the directed cause-effect relations and the bidirected unobserved confounded relations, associating two trainable adjacency matrices. To enhance the learning process, we introduce a causality constraint combined with the concept of a causal annealing strategy during training, guiding the learning toward meaningful causal structures. Experimental results show that our method achieves competitive performance in identifying both observed and latent causal relationships on synthetic datasets. Furthermore, we demonstrate that the learned causal structure significantly improves downstream causal inference performance on real-world data.

Published in Transactions on Machine Learning Research (annuryy/0200
Generative Causal Structure Learning with Spaces and Annealing
Dual Latent
Soma Bandyopadhyay
cowa.bbndyoppdlhyyy@tte..cmm
Deportment of Artificial Intelligence
Indian Institute of Technology Kharagpur TCS Reseorch, TATA Consultancy Services Limited
Sudeshna Sarkar
sudeshna &csss Sitkesssss...kk..c in
Department of Computer Science and Engineering Indian Institute of Technology Kharagpur
Reviewed on OpenReview: https: openrevieee wet/pdffiddul5rf/yp
Abstract
In this work, we address causal structure learning in the presence of unobserved confounders. Such causal structures can be represented by Acyclic Directed Mixed Graphs (ADMGs), where observed cause-effect relations are depicted by directed edges and unobserved con- founded relations by bidirected edges. Prior methods for causal structure learning with unobserved common causes have primarily focused search-based approaches, and more recently on flow-based generative models. We propose novel generative method based on a variant of the Variational Autoencoder (VAE) with dual latent spaces to represent the directed cause-effect relations and the bidirected unobserved confounded relations, as- sociating two trainable adjacency matrices. To enhance the learning process, we introduce a causality constraint combined with the concept of a causal annealing strategy during training, guiding the learning toward meaningful causal structures. Experimental results show that our method achieves competitive performance in identifying both observed and latent causal relationships on synthetic datascts. Furthermore, we demonstrate that the learned causal structure significantly improves downstream causal inference performance on real-world data.
1 Introduction
Learning cause-effect relationships without known causal structures is fundamental to causal reasoning. In many roal-world scenarios, unobserved common causes distort statistical associations among features, making structure learning more challenging. For example, in human mobility, external factors such weather can jointly influence departure time and destination, acting as latent confounders that bias inferred relations.
This work aims to learn causal structures among features the presence of unobserved (latent) confounders, without relying on prior structural knowledge. Further, we demonstrate the applicability of the proposed method for the causal inference task real-workd dataset, leveraging the learned causal relationships.
Prior studies causal structure identification under unobserved confounding have predominantly employed score- based approaches, using criteria such as the Bayesian Information Criterion (BIC), and constraint- based methods relying on conditional independence tests.
Subsequently, the causal structure discovery has been reformulated as a continuous optimization problem enforcing acyclicity constraints (Zheng al. 2018 via differentiable function with gradient based techniques,
Code available OSF repository

---

Published in Transactions on Machine Learning Research (January /202)
excluding the need for combinatorial search. However, these continuous optimization based methods assume causal sufficiency and do not consider unobserved variables.
Recently, the bow-free constraint (Bhattacharya et al. 2021] applied to Acyelic Directed Mixed Graphs (ADMGs) (Richardson & Spirtes 2002]. ADMG has been utilized to represent both direct causal relation- ships, and unobserved confounding relations. The observed cause-effect relations are represented by directed edges. The unobserved confounders are represented by bidirected edges between the variables which have unobserved common cause that influences both variables. The bow-free constraint prevents overlapping directed and bidirected edges between the same pair of variables. Specifically, if there is a directed edge Vi + Vj, then there cannot also be bidirected edge VÚ + Vj. This concept, integrated into Structural Causal Models (SCMs) [Pearl 2009), has recently been explored in neural ADMG contexts.
We follow ADMG structure estimation assumptions with latent confounding from prior work (Ashman et al. 2023). The framework assumes a nonlinear additive-noise SCMs, where cach variable modelled as nonlin- ear funetion of its direct causes combined with independent noise and exogenous latent confounders. These assumptions support estimation of directed and bidirected edges from observational data, however observa- tionally equivalent structures may still arise. Haise.Wiil While these assumptions enable plausible causal interpretations, they do not guarantee exact recovery of the ground-truth SCM, and discussion on causal guarantees is provided in Appendix D
We represent the directed cause effect relationship by the directed adjacency matrix A (Obeervd)) or AD, which captures directed, asymmetric, observed cause-effect relationships. The unobserved common cause association with different pair of variables is represented by the bidirected adjacency matrix A (Unobserved) Ag which captures bidirected, symmetric, unobserved confounded associations.
Our approach employs Variational Auto Encoder (VAE) (Doersch 2021 Kingma & Welling
2013 based causally constrained generative framework designed to estimate unobserved causal structures within the ADMG formalism. We denote this model as G-ADMG-CL (Generative Acyclic Directed Mixed Graph based Causal Structure Learning). Detection of unobserved confounders enabled under the bow-free ADMG assumption (Ashman et al., 2023), using a nonlinear additive-noise SCM. The framework captures both nonlinear effect relationships and latent confounding through trainable adjacency matrices AD and Ag. Overall, the learned adjacency matrices provide interpretable causal representations that jointly encode observed and latent relationships, enabling structure discovery under confounding. The proposed framework is characterized by the following components:
VAF-based architecture that disentangles observed and unobserved causal relations through dual latent spaces.
Separate latent spaces are dedicated to directed cause effect relationships and unobserved con- founded relations, governed by AD and Ag, respectively.
A causally aware objective function that enforces acyclicity for observed relations, bow-free constraints for confounded edges, and entropy-sparsity regularization to balance structural variability while preserving the asymmetric/symmeett nature of Ap and Ag.
A causal anncaling strategy that gradually increases the rousal-regularization weight from 0 to 1 until the causal transition epoch (CTE), prioritizing reconstruction and KI-divergence opti- mization during early training before activating full causal constraints.
We evaluate G-ADMG-CL on synthetic data generated from nonlinear SCMs and woal-world data demon- strating effective causal structure learning under latent confounding.
2 Related Work
Causal structure learning is a fundamental problem with wide applications across scientific and real-world domains. Estimating causal relationships in the presence of unobserved confounders is both more realistic and more challenging. We review key existing approaches, with a particular focus on methods that attempt to model latent confounding and capture both directed and bidirected dependencies.
2

---

Published in Transactions on Machine Learning Research (January /2026)
Score-based and constraint- methods: Classical approaches causal discovery (Hasan et al, 2023 Zanga et. al. 2023) include constraint-based and score-based methods. Constraint-based methods, such as the PC and FCI algorithms (Spirtes et al. 2000), rely on conditional independence testing to infer causal graphs while score-based based methods search for graph structures that optimize predefined scoring criteria such as the BIC. However, these methods typically assumne access to a fully observed variable set or struggle with indistinguishabiiiyy in equivalence classes under hidden confounding.
The NOTEARS framework (Zheng et al 2018) introduced differentiable acyclicity constraint based on the trace of a matrix exponential, enabling gradient- based optimization for DAG structure learning. This approach has been extended by neural methods such as DAG-GNN (Yu et al.) 2019b) and N-DAG-G (Geffner et al. 2022), which integrate deep architectures for end-to-end causal discovery. While effective for fully observed systems, these methods assume causal sufficiency and cannot represent latent confounding or bidirected edges. (Zecevic et al. [021] relate graph neural networks to structural causal models on graph-structured data, offering causal interpretation of message passing but without explicit modelling of latent confounders.
Differentiable DAG learning (no latent confounding):
Handling unobserved confounders: Several methods explicitly attempt to model latent confounding, Fast Causal Inference (FCI) (Spirtes et al. 2000) is a constraint-based method capable of detecting latent variables through conditional independence tests, but it cannot distinguish between equivalence classes with identical independencies. Repetitive Causal Discovery (RCD) (Maeda & Shimizu 2020) handles linear non- Gaussian models and introduces bidirected edges to represent unobserved common causes. CAM-UV (Maeda & Shimizu 2021) extends the Causal Additive Model (CAM) to latent- varrable settings by using HSIC- based independence tests (Gretton et al. 2007) combined with a scoring procedure. More recently, Gonzales & Valizadeh (2024) proposed score-based global search over augmeente DAGs that can handle latent confounding, but is restricted in scalability.
Ancestral graphs and bow-free constraints: To more explicitly model both directed and bidirected edges under latent confounding, recent works have extended the causal discovery framework to ADMGs. The approach in [Bhattacharya et al., (202)) introduces differentiable objective incorporating acyelicity, ancestral constraints, c-tree penalty, and the bow-free constraint, targeting linear Gaussian additive noise models. These constraints help ensure identifiability and interpretability in the learned graphs. None of these methods utilizes a generative model.
Flow-based methods: Despite these advances, most existing methods are either search-based or rely on structural constraints but do not leverage a generative modelling perspective. A notable exception is (Ash- man et al. 2023), which introduces flow-based aulo-regressive model for estimating ADMG structures with nonlinear relationships. Their method supports both bow-free (N-BF-ADMG-G) and general ADMG settings (N-ADMG-G), and is capable of modelling latent confounders. However, it lacks latent-variable disentanglement and does not optimize for structure recovery through ceneration-basee objectives.
3 Methodology
Task overview: In this section, we elaborate the two tasks. First, causal structure learning in the presence of both observed cause-effect and unobserved confounded relations, using the proposed G- G-ADMG- CL model. Second, we extend this framework to prediction and causal inference (denoted G-ADMG- CL+P), where the learned causal graph supports downstream estimation of treatment effects, such as the Average Treatment Effect (ATE)). This extension integrates the inferred structural knowledge into the decoder to enable structuro-aware prediction and causal inference.
We propose G-ADMG-CL, cousality-come constrained VAE to jointly learn both directed and bidirected causal relationships in the presence of latent confounding. The key innovations in our method are the introduction of dual latent spaces, causality constraint optimization, and a causal anncaling strategy to guide causality learning. Our approach disentangles observed cause effect interactions from unobserved confounded associations using these dual latent spaces. To guide structure learning, we propose a causality- awarc objective that incorporates acyelicity, bow-free constraints, and structural regularization. We introduce a novel cousal annealing strategy that gradually enforces causal constraints during training, enabling more stable and accurate graph recovery. The goal of our method is to learn both edge types simultaneously
Method overview:
3

---

Published in Transactions on Machine Learning Research (Jamuar/
while ensuring that the resulting structure supports robust ADMG structure estimation, is interpretable, and aligned with the true underlying causal mechanisms.
Causal structure learning: VAE is a popular generative model for learning disentangled latent represen- tation (iggins et al. 2017). Its probabilistic formulation allows incorporating constraints in the latent space while learning and optimizing the loss function. The functional components of this method are illustrated in Figure]
Learning
Aa D-Adjacency Matrix
Latent Space for observed causal
relationd
Enceder
Cousal constraints
Decoder/ Generator
effect variables and causal graph
Latent for unobserved
Input Data
Generated/ Predictad Data
(bi-directed), 2g
Ag
B-Adjacency Matrix
Figure 1: Functional components of the proposed model, featuring dual latent spaces Zp and ZB guided by the trainable adjacency matrices Ad and Ag for causal structure learning.
Functional components: The encoder maps the input data inlo two disentangled latent spaces: the directed latent space ZD representing observed causal relations, and the bidirected latent space ZB repre- senting unobserved confounded relations. These latent representations trainable adjacency matrices, Ap and Ag- Causal constraints are incorporated during training to guide the learning process to ensure meaningful structural recovery. The learned adjacency matrices are fed into the (eecoder/generttoo to reconstruct or generate the predicted data, and to obtain the inferred causal structure, integrating both observed causo-effect and unobserved confounded relations. The predicted data is further utilized for causal inference tasks. Additionally, the proposed causal annealing approach further guides the causal structure learning to enforce causal constraints during training.
used to learn two corresponding
Detailed methodology:
The encoder uses dual latent spaces to learn the observed and unobserved cause- relation- ships. The directed latent space Adirected (= =D) learns the observed causal relationship, and the bidirected latent space Zbidivented (= zn) addresses unobserved or confounded relationship. Latent space represen- tations for both latent spaces are obtained by computing mean Pdirected (= HD),, Pbidirected (= (B), and log- Darvaariance log 2 airected: log Ridiretted respectively. Latent variables are sampled using the reparameteriza- tion trick: first is element-wise sampled from (0, and then are computed using Eq. 2 We compute Adirected _uiiedg_dde____eeeeeee Adgiadddddjceeeeeeae (= (DAD) (=)DA) and Mbidirected _uuided Adjacency (= PBAR) using Eq. to guide 2D and 2B using the directed cause effect relationships encoded in Ad and the unobserved confounded relationships encoded in Ag. The aim is to obtain the causally structured latent variables, improving reconstruction and making the trained model more interpretable by following the cause- effect. relationships.
Encoder:
The encoder comprises two fully connected layers (128 and 64 units) with ELU activations; dropout is applied for regularization.
PDAs = Adirected A D ('BAB Mbidirected Ag-
(1)
ZD UDAD + EED @ (0.5 log ))
PBAs + Eg C exp log 1og
(2)
Decoder: The decoder combines the causally guided dual latent spaces [ æn] from the encoder and reconstructs the given input from the combined latent spaces, retaining the causal and unobserved confounded relationship among the variables. This reconstructed outcome is further exploited for the prediction task. The decoder consists of two fully connected layers with 64 and 128 units, respectively, using LeakyReLU activations and dropout for regularization.
4

---

Published in Transactions on Machine Learning Research (January /202)
Generator: The decoder is employed to generate new data samples by passing the latent samples sampled from Gaussian distribution.
Loss function (Stotal) (Eq. 33 This comprises three components. The first two components are already well-studied in VAE-based models. We introduce a novel third component, the causal mixed graph loss (CCausul_ADMC) weighted by the Acausal-
The loss function follows the structure of ADMG. The adjacency matrices Ap and AB are learned end- to-end and are parameterized via trainable weight matrices W1 and W2, respectively. Using this causality constraint, (Ecausal (CCa) the training process enforces the structural regularization aligned properties of ADMGs, and learn causal relationships in the presence of unobserved confounders.
Total Loss:
Liotal Creconstruction AKL ( directed + CKL bidirected)
+ Acausal (C (CCaual-ADMC))
(3)
Reconstruction loss: Creconstruction ||x * 212 the mean-squared error (MSE) between input (x)
and reconstrueted data (8), standard for Gaussian VAEs.
KL divergence loss: This is for latent space regularization, where, LKL directed (Eq. 4 for the directed, and LKL_ bidirected (Eq. (5) for the bidirected KT. divergence. We apply KT. annealing (Ful et al. 2019), kl weight (AKL) (a gradually increasing weight) to multiply the KL divergence term to counter KL-vanishing during the initial training phase and focus primarily minimizing the reconstruction error.
ÀKL LKL directed = AKL KL (dddireeeted X) |l po (ddieeceed X)),
(4)
AKL MKL_reidieetted = 7KL KL (go (biddreectdd
X) pa (Ziidieected (X)
(5)
3. Causal mixed graph loss (Lcausal :D This is for causal structure regularization, with the causal regularization weight Acausal as shown in Eq. (6) comprises the following components:
LCausal ADMG = Aeyeleheyclo (Ap) Abow (AD, AB)
Xentropy (Ap) Lentropy (Ap)
Aentropy (An Centropy (AB)
+ Xasymmetry (Ap) (Ap)Cas J (Ap) Asymmetry (Ag) Saymmetry (AB)
+ Asparsity (Ap) Caparsity (Ap)) + Asparsity (A Csparsity (Ab). (6)
Acyclicity:
Leyele (Eq. 77) applies acyclic consts on Ap to prevent cyelic causal structures
among directed latent variables. Minimizing Leycle enforces acyclicity by penalizing non-zero tr)
hhow (Eq. penalizes anmultaneous directed and bidirected edges, restricting vari- able pairs from sharing both cause-effect. and unobserved confounded relations. Cnow(Ap Ab) ensures the bow-free property by suppressing such dual connections.
Bow-free:
Entropy: Centropy (Ap) and Centropy (Ab) (Eq. maintain variability in latent spaces enforce steady learning.
Symmetry Asymmetry:
Lusymmettry (Ap) ensures non-existence of both Ap and Ap i] while Laymmetry ensures sinullaneous existence of [] and Anlj. i), as Ap is asymmetric and bidirected edges in Ag are symmetric (Eq. (10)).
Sparsity: Roduces noise in both directed Ap and bidirocted Ag adjacency matrices (Eq.
Laycle -|trace(eA - d) , (d dimensionAp. AB)
(7)
Lhow Σ
Anli.
B
Σ
Anli,,]
(8)
n
n
1

---

Published in Transactions on Machine Learning Research (annuryy/0200
(ontropp(An) - -E (AD) logpiAd) (AD)) Centrop(An)) Σ p (AD) logp (AB)
(9)
(AD)
|AD
(Aa)
AA
Pi ΣS ApDi]
Pi
ΣS |A
Lusymmetry (Ap) 1lAD ① AbblI,
Lsymmetry (AB) = ||AB Abll
(10)
Csparsity (Ap) ||blli Csparsity (AB) ||A
(11)
Causal annealing: We introduce causal annealing, a learning strategy designed to systematically control the influence of the causal regularisation within the total loss during training, where we keep Arausal a gradually increasing between to until the causal transition epoch (CTE). At CTE Acausal becomes and remains for the rest of the learning cycle. It helps to learn the data characteristics associated with the reconstruction and KL divergence components, and then apply the causality constraints. Algorithm 2 describes the causal annealing. The proposed causal annealing has two schedules, one is the default hard mode, where Acausal remains 0 for e < CTE, and switches to I at CTE. To avoid abrupt changes in the gradient behavior and prevent abrupt instability at transition a relaxed schedule is introduced. In this relaxed schedule, a relaxed transition epoch e, defined, during which Acausal starts to linearly increase from 0 E, to the CTE.
Algorithm Causal Relationships Learning: G-ADMG-CL
1: Input: Data: X E Raxd with unobserved confounder
2: Output:
Reconstructed data X, Directed (Ap), Bidirected (Ab) adjacency matrices Encoder Block
Initialize trainable adjacency matrices: (Once at start of training)
Directed adjacency matrix: W1 N 1)axd
5: Bidirected adjacency matrix: W2 ~ (O, 1)axd
6: Compute directed latent statistics:
7: PD. log o < FC layers on X UDAD < MD W1
Compute bidirected latent statistics:
10: MB, log o +- FC layers X
11:
MAAn MB W2
Reparameterization
12: Sample structure aware latent variables using reparameterization
13: zD + ADAd + eD O cxp(0.5 log d)) cp ~ (0,
14: 2B +- μBAA + Eg @ exp(0.5 log ) ég ~ 100,1)
15: Concatenate latent representations:
2D, ZB
Decoder Block
16:
X + (ecoder(z)
Adjacency Matrix Estimation (Per-epoch latent-to-adjaacnncy decoding via W,, W₂)
17: AD + f(ZD) = WDW,, Ag 1 f(zz) = ZgW,
Learning and Optimization
18: Optimize with causal constraints: as defined in Eq. 13 during training
19: Liotal = Ereconstruction + Á directed + LKL. bidirected) Acausal L Caausl ADMG
Learned Causal Structure
20:
Return Ap, Ag for evaluation and X
Algorithm D presents G-ADMG-CL, which learns causal structures with unobserved confounders based on ADMG, without assuming a known causal graph. The objective function enforces acyclicity and the bow-free property, ensuring Ap remains asymmetric and Ag symmetric, while balancing randomness and sparsity. This method supports applications such conditional and counterfactual data generation, providing inter- pretable results through its dual latent spaces and learned adjacency matrices.

---

Published in Transactions on Machine Learning Research (annuayy/20000
Algorithm 2 Causal Annealing During Training
1: Input: Total epochs E, causal transition epoch CTE, lincar transition start epoch e,, anneal mode ("hard" or "linear")
2: Output: Causal regularization schedule Acausal for each epoch
3: Initialize Acausal f-
for epoch e to E do if anneal mode = "hard" then
if CTE then
Acausal +-
8:
else
9:
Acausal +
10:
end if
11: else
12:
if < ez then
13:
Acausal
14:
else if < CTF then
Et
15:
Acausal CTE et
16:
else
17:
Acausal +-1
18:
end if
19:
end if
20: Update model parameters using Arausal
21:
end for
In contrast, other VAE-based causal models, such as CausalVAE (Yang el al. 2021]; D, learn disentangled causal representations consistent with a prespecified causal graph under the assumption of causal sufficiency, but they do not perform causal structure discovery model latent confounding via bidirected edges. (Leeb)
et al. 2021 study how to probe causal relations in the latent space of auto-encoders using interventional assays, focusing on analyzing and manipulating learned representations rather than recovering an ADMG. Notably, none of these approaches employ a cousal annealing strategy like ours.
Causal Inference: We further employ our method to the prediction task as G-ADMG-CL+P to conduct the causal inference using learned dual latent spaces, [æD, 2B]. The learned directed cause-effect and unobserved confounded relationships from the latent spaces reduce the variability in the average treatment effect (ATE) (Eq. 12 the difference in the expected effect when the treatment cause is applied depending both observed and unobserved confounders.
ATF = ((()) EY())),
(12)
where (1), (0) represent the outcomes with and without applying the treatment (or cause), respectively,
4 Data
To evaluate our method under varying levels of structural complexity and confounding we consider two synthetic datasets: Fork Collider (FC) and Erdös-Rényi (ER) and one real-world dataset IHDP for causal structure identification and causal inference tasks. Additionally, conduct an early evaluation the Sachs protein-signaling data (Sachs et al. 2005) claborated in Appendix F
Fork Collider (FC):
We synthesize 4000 samples of this data using the Eq. 13 of the SCM following (Ashman al. 2023 The causal graph and adjacency matrices are depicted Figure 2a Variables X2, X3 and X3, X4 have bidirected edges, where ul and u2 are the respective confounders influencing these pair of variables. The bidirected

---

Published in Transactions on Machine Learning Research (January /2026
edges represented by the dotted line, and the directed edges are shown by the solid line.
T [ui 62, E,, 6s ~ N.O,I),
E1
42 16exp(-1i) 0.1e2,
(13)
= 16epp--if + /6exp(-1i + 0.2cs,
44 66epp( u3) c6
-z+) + 0.1c4,
(6exp(-aj 0.1e5.
We evaluate method on two ER based synthetic graph configurations: (R(4,6,4) and 10,),,50,10), with unobserved confounders, where (d, e, m) denote observed nodes, directed, and bidi- rected edges, respectively, each simulated with 4000 samples. The first configuration, ER(4,6,4), contains 6 directed edges, and m = 4 bidirected edges representing & compact graph with
Erdös Rényi (ER):
d = variables, moderate confounding. The second, 10,12,50,10), inchudes = 12 variables, e = 50 directed edges, and m bidirected edges, modelling a more complex and densely connected structure. These settings are designed to evaluate the scalability and robustness of our model in recovering both observed and latent causal relationships under increasing graph density and confounding.
18 (Appendix following the ER-based ADMG simulation procedure described in (Ashman et al. 2023 The corresponding causal graphs and adjacency matrices are illustrated in Figure 3a Notably, in these ER settings, directed (solid) and bidirected (dotted) edges often overlap reflecting realistic scenarios where a pair of variables possessing direct cause effect relationship may influenced by the unobscrved common cause.
Data are generated using the Eq-
IHDP: THDP (Infant Health and Development Program) is a real-world dataset originally introduced by (Hill 2011) comprising measurements of infants and their mothers, collected during randomized experiment. We adopt the IHDP dataset (Louizos et al. 2017), for the causal inference task, where the goal is to estimate the effect of home visits by specialists (cause) on the cognitive development of infants (effect). The dataset is used with 10 replicates, each consisting of 70% train [HHDP_train) and 30% test [THDP_te)) split, with fixed size of 747 individuals per replicate. It includes 19 binary covariates and 6 continuous covariates (e.g., birth weight, head circumference), all normalized before training. To incorporate unobserved confounding, treated individuals with non-white mothers excluded. Outcomes are simulated based Setting B llng-lincar response surfaces), and Gaussian exogenous noise is applicd.
5 Results
We evaluate the proposed G-ADMG-CH model across both synthetic and real-world datasets to assess its ability to recover causal structure in the presence of latent confounders and to support causal inference.
5.1 Performance Measure
We evaluate model performance using the following metrics:
F1 Score:
The harmonie mean of precision and recall, computed over predicted vs. ground-truth edges (directed and bidirected separately). It is defined as:
=1 = 2 Precision Recall
(14)
Precision + Recall
RMSE-ATE ATE: Measures the error in causal inference by computing the root mean squared error between the predicted average treatment effect (ATE P) and ground-truth ATE over samples:
1 (ATE. P. ATE.)2
RMSE-ATE
(15)
8

---

Published in Transactions on Machine Learning Research (January /2026)
5.2 Evaluation of G-ADMG-CL
We evaluate the performance of the learned causal structure across different datasets by comparing it with various established state-of-the-art methods. Fl scores are used to measure the performance. The FlD, b, FlB represent the F1 scores obtained for directed cause-effect relationships and for the bidirected unobserved confounded relations among the variables, respectively.
We compare our method against the following otate-of-the-art approaches:
FCI: Constraint-based method using independence tests to detect confounders.
CAM-UV: Additive noise model with HSIC tests for hidden variables.
RCD: Learns linear non-Gaussian models with bidirected edges.
DCD: Additive causal discovery with structural regularization.
N-DAG-G: Neural DAG learner assuming no hidden confounding.
N-ADMG-G: Flow-based ADMG learner capturing latent confounders.
N-BF-ADMG-G: Flow-based model for ADMGs with bow-free constraints.
Unlike these approaches, our method leverages dual latent space learning with causality constraints to jointly model both directed and bidirected causal relationships. Additionally, we implement a causal annealing strategy that enhances performance and training stability, leading to more reliable estimation of latent con- founding structures. Together, these innovations promote more effective disentanglement of causal directions and latent confounders, leading to consistent performance gains in both FID and Flg scores.
We determine the significant edges by applying a best (optimal) thresholding technique presented in Ap- pendix B (Algorithm 3 to binarize the learned adjacency matrices An and An. Appendix B also provides sensitivity analysis of the best threshold (threshold-F7 score sensitivity curves) and a comparison between optimal and adaptive thresholding (Algorithm 4 supporting robust and fully data-driven binarization of both Ap and Ag. When ground truth absent, adaptive thresholding serves as a fully data-driven alterna- tive.
Ag_ [[o. 0),
1,
X2
X1 Ao- 0,
0,
AaL- [[0, 0, 1,0, 0J.
00,0,000.00
x2
X1
AoL= [[0, 0, 1}.
[0 0. 0).
1,,1,,01,0, 1,0), U1
[o. 0, 0],
[1,0,0,1,0] U1
[0 0 0 0
[0, 0, 0,0,0,
00,00000.0
[0, 0,
00, 0 0 0
[0, 0,
0, 0,
[0, 0,, 0, 01]
[0,
X3
X4
X5
U2
(a) Ground truth causal graph and adjacency matrices.
(b) Learned causal graph and adjacency matrices.
Figure 2: Comparison ground-truth (left) and learned (right) causal graphs for FC. Solid: directed; dashed:
bidirected; red: incorrect edges. Adjacency matrices: ground-truth: Ap, AB; learned: ApL, ABL-
In case of FC, we train the model with 5000 epochs using ADAM optimizer with ExponentialDecay (Good- fellow et al. 2016) learning rate scheduling. We apply KL annealing with kl weight (a gradually increasing weight) until epoch 50 to multiply the KL divergence term to counter KL-vanishing during the initial training phase and focus only on reconstruction error minimization. The causal transition epoch (CTE) is set to 150 to learn the data distribution and input characteristics well before applying the causal regularization. The full configuration details are available in the Appendix (Table 9h, along with their sensitivity analysis.
The learned causal graph and adjacency matrices are shown Figure All directed causc-effoct relations are correctly identified by the proposed method. For bidirected confounding, the model correctly detects the edge between X3 and X4, the more complex confounding pattern, since X4 participates in multiple causal

---

Published in Transactions on Machine Learning Research (January
dependencies, having both directed and bidirected inftuence, but misses the ground-truth bidirected edge X3 <+ X2 and instead introduces spurious edge X3 <> X1, representing a close confounding approximation and demonstrating the model's ability to infer approximate latent structures even in challenging scenarios. A red edge significs a relationship absent in the ground truth.
For ER, each dataset FR((4,6,4) and (R(,2,,0, we train the proposed model using the ADAM opti- mizer with appropriate learning rate scheduling (either ExponentialDecay or CosineDecay), KL annealing to gradually increase the kl sl_weight to 1.5, and dataset-specific regularization coefficients. The complete configuration details, including learning schedules, latent dimensions, and regularization parameters, pro- vided in the Appendix E (Table 9h Figure 3a shows the ground-truth ER(4,6,4) causal graph and Figure 3b
Ag- [[O, 1, 0,1, 0,,0, 0},
1], Ap=
[1, 11,
ASL: [[0, 1, 0,1] [1,0, 1}, [o, 1, 1J, 11,1,100]
[0. 1. 1].
01,0, 0),
Apu [1001]
[[O 101
[1,0,1. 1, 0]]
X2
X1
[o, 1]
X2
[1000]
[0 0]]
X3
X4
X3
X4
(a) Ground truth causal graph and adjacency matrices.
(b) Learned causal graph and adjacency matrices.
Figure 3: Comparison of ground-truti (left) and learned (right) causal graphs for ER(4,6,4). Solid: directed;
dashed: bidirected; red: incorrect edges. Adjacency matricos: ground-truth: Ap, AB; learned: ApL, ABL-
shows the corresponding learned structure. CTE plays important role in the training process.
The comparative results are summarized in Table | demonstrating that our method delivers consistently competitive performance and, in certain scenarios, outperforms state-of-the-an approaches across multiple datasets. To assess robustness, we further evaluate our method on multiple independently sampled ER graphs (Appendix C reporting mean + std performance over five independent runs in Table [ detailed analysis of worst-case bidirected performance (including 12)12,50,0)) logether with limitations and failure modes, is provided in Appendix For fairness and reproducibility, we retain the best-reported values for all bascline methods.
Table F1 score comparison (FID: directod, FlB: bidirocted) across FC and ER
FC
ER(4,6,4)
ER(12, 50, 10)
Method
Flp
FlB
Flp
FlB
Flp
FlB
FCI
0.00
0.75
0.50
0.40
0.25
0.33
CAM-UV
0.80
0.67
0.30
0.25
0.38
0.36
RCD
0.00
0.54
0.35
0.35
0.45
0.20
DCD
0.00
0.67
0.25
0.20
0.32
0.18
N-DAG-G
0.50
0.00
0.60
0.00
0.55
0.00
N-ADMG-G
0.49
0.75
0.60
0.60
0.38
N-BF-ADMG-G
0.64
0.93
0.78
0.80
0.60
0.40
Proposed (G-ADMG-CL)
1.0 (0.00)
0.50 (0.00)
0.84 (0.05)
0.89 (0.03)
0.53 (0.05)
0.41 (0.05)
For the FC dataset, the proposed model achieves perfectly stable recovery with Flp = 1.0(0.00) and FI 0.50(0.00) across all runs. The bidirected score of 0.50 corresponds close confounding approximation (X3 paired with X, instead of X2), reflecting subtle latent-dependency ambiguity rather than model instability.
10

---

Published in Transactions on Machine Learning Research (Januar/
For the ER dataset, our method achieves the highest FID score of 0.92 and the highest Flg score of 0.89 the (4,64, graph (Figure 3), effectively recovering both directed and bidirected causal relationships. Av- eraged across multiple runs, the method attains an FID score of 0.84 and an FlB score of 0.89 on (R)4, On the more complex ER 52,),000,)0) graph, the model attains a competitive average FID score of 0.53 (best: 0.58), and an FI score of 0.41 (best: 0.45), outperforming all baseline methods detecting unobserved confounded relationships. These findings demonstrate the robustness of the proposed methodology in dense and high-dimensional confounding scenarios.
For the siveiniiggggling dataset (Sachs), Appendix F presents preliminary struchure-estimation results showing the applicability of G-ADMG-CL in complex biochemical system.
Ablation study: We perform ablation study for the causal relationships learning task.
Caausll ADMG: The ablation study on causal mixed graph loss LCausal ADMG, presents the significance of the proposed causal constraint applied to the loss function of G-ADMG-CL.
FC: We observe that removing causal constraint LCausal_ADD ADMG causes significant reduction in the perfor- mance in FC. As next step to establish the importance of the different components of causal mixed graph loss like bow-free constraint Abow, symmetry/ wymmetrr/ssymmetry /asymmetry constraints of adjacency matrices, we set their value to zero to remove their influence. We alter the value of Acyele to a high value to judge its sensitivity, as depicted in Table 2 This study establishes that all these components siguificantly impact performance and play significant role in causal structure learning.
ER: We obscrve that removing CCausal ADMG causes a significant reduction in the performance depicted in Table 2 As next step to establish the importance of the different components of this causal mixed graph loss we perform the same steps as performed in FC. We notice that the symmetry constraint of An plays an important role. ER data has overlapping directed and bidirected edges where bow-free constraint aims to ensure non overlapping directed and bidirected edges which reduces the impact of the bow effect. This study also reflects that our method performs well in the presence of overlapping directed and bidirected edges, that is, when variables have both a direct cause-effect relationship and are influenced by a common unobscrved confounder.
The additional ablation study in Appendix F.2 shows that strong bow-free regularisation and moderate sparsity improve recovery of both directed and bidirected structures, while excessive regularisation can degrade performance.
Table 2: Ablation study of LCausal_ADMa ADMG and its components using Fl score
FC
ER
(4,6,4)
Method
FID
FI
FID
Fln
Proposed with Ltotal
1.0
0.50
0.92
0.89
Liotal CCausal_ADDi
0.50
0.31
0.86
0.78
LCausul ADMG Lbow
0.44
0.5
0.86
0.88
LCansal ADMG Laymmetry (A
0.66
0.25
0.86
0.82
LCausal ADMG Lasyumeetry (Ap)
0.66
0.5
0.92
0.89
LCansal ADMG Acyole 10
0.57
0.40
0.92
0.89
Ablation study on causal annealing: This study presents the sensitivity of the proposed causal an- nealing applied to the G-ADMG-CL, the causally constrained VAE. We compare the bascline G-ADMG-CL. without annealing, (Acausal = 0) against G-ADMG-CL with hard causal annealing (Acausal =1). Table 3 presents an ablation study to evaluate the impact of causal annealing.
FC: In this datasct, we observe that using bascline G-ADMG-CL. reduces the performance for directed relationships on the FC dataset (F1p 0.50) significantly. In contrast, G-ADMG-CL with annealing achieves the highest directed FIp 1.00 performance.
11

---

Published in Transactions on Machine Learning Research (January /202)
Table 3: Ablation study of causal annealing using Fl score
FC
ER
(4,6,44
Method
FlD
FlB
FlD
FlB
G-ADMG-CL
(with Causal Annealing)
1.0
0.50
0.92
0.89
G-ADMG-L
(without Causal Annealing)
0.50
0.50
0.75
0.80
ER: For ER, notice that using baseline G-ADMG-CL reduces the performance for directed relationships the ER dataset (FID 0.75) significantly, and (F1B = 0.80). In contrast, G-ADMG-CL. with annealing achieves the highest directed FlD on ER (0.92) and maintains strong bidirected performance (0.89). All ablation experiments are conducted using the best-performing configuration.
In addition to employing hard causal annealing, early evaluations are conducted using linear causal- annealing schedule on the ER(12,50,10) graph, as detailed in Appendix E.2.1 Overall, lincar schoduling demonst,rtes comparable effectiveness to hard causal annealing.
5.3 Causal Inference
We demonstrate the applicability of our method to a causal inference task using the IHDP dataset. While we refer the causal inference extension of our model as G-ADMG-CL++P, we report results and comparisons using the core model name G-ADMG-CL, consistent with baseline naming conventions. We first train the model using IHDP train to obtain the learned directed and bidirected latent representations. The model is trained for 5000 epochs using the ADAM optimizer, with ExponentialDecay as the learning scheduler. The initial lcarning rate is set at 0.001, with decay_steps=1000 and decay_ rate = 0.90. To address KL vanishing, we apply KL annealing, where the KL divergence term is multiplied by a gradually increasing slweight until it reaches 1.5 at epoch 20, allowing the model to focus on reconstruction error minimization during carly training. The dimensionality of both latent spaces 2D and 2/ set to 50, as this data is relatively complex. We consider Aeycle 1, Abow 5, Aeutropy (Ap) 0.01, Aentropy (Ab) 0.001, Aasymmetry ( (An) 0.05, Asymmetry (AB) 4.75. The causal transition epoch (CTE) is set to 1000 ensure that the G-ADMG-LL learns the data distribution and input characteristics well before applying the causal regularization.
We apply the trained model, G-ADMG-CL, to the IHDP test for obtaining the predicted outcome Prreated
Hoontrol_P as shown Eq. 16), Eq. 17 using the decoder reconstructed output as described in the methodology.
model G-ADMG-CL(IHDP train)
(16)
yrreated__P Py Mcontroi p model,predic(([H])) test)
(17)
Table 4 shows that the proposed method outperforms existing stale-of-the-ar methods by achieving the Table 4: Causal inference results using the IHDP dataset
Method
RMSE- ATE
FCI
0.13
CAM-UV
0.15
RCD
0.14
DCD
0.16
N-DAG-G
0.12
N-BF-ADMG-G
0.10
G-ADMG-CL
0.031
lowest RMSE-ATE, Since the IHDP does not provide a ground-truth causal graph, the ATE serves as the most reliable metric for comparing causal structure learning performance.
12

---

Published in Transactions on Machine Learning Research (January /2026)
For completeness, Appendix provides consolidated overview of the supplementary experimental analyses, including threshold selection, ER reproducibility, hyperparameter sensitivity, causal annealing behaviour, and early evaluation on the Sachs protein signaling dataset.
6 Conclusion
We propose a novel VAE-baser generative method for learning causal relationships under latent confound- ing. A key novelty of architeclure is the use of dual latent. spaces, which separately encode directed cause cause-effect relationships and bidirected latent confounded influences an ability not present in existing VAE- based causal discovery methods. These latent reprosentations associate with the trainable adjacency ma- trices AD and Ag, enabling ADMG-based structure learning. We introduce the causal mixeddgraph loss LCausal ADMG to enforce acyclicity and capture bidirected dependencies. Another contribution is the causal annealing mechanism, which progressively activates causal constraints during training and is absent in prior causal structure learning approaches. Ablation studies confirm the importance of both components. Our method, G-ADMG-CI, achieves competitive improved performance established baselines, and the learned structure improves causal inference real data.
Future work includes exploring annealing schedules (CTE effects) and analyzing coexistence in complex graphs Another direction is modelling compler and hierorchical confounders to improve bidirocted recovery under densely confounded graphs. We aim to extend the evaluation of our method to physics-basee basie simulators such as SriFrnger (Wüthrich el al. 2021 and Beta-WWrld (Yu el al. 2019a), which exhibit rich nonlinear dynamics and multi-object dependencies, providing a challenging testbed for causal structure learning.
edge
References
M. Ashman, C. Ma, A. Hilmkil, J. Jennings, and C. Zhang. Causal reasoning in the presence of latent confounders via neural admg learning. In Proceedings of the 11th International Conference on Learning Representations (ICLR), Kigali, Rwanda, 2023.
R. Bhattacharya, T. Nagarajan, D. Malinsky, and I. Shpitser. Differentiable causal discovery under unmea- sured confounding. In International Conference Artificial Intelligence and Statistics, pp. 2314 2322. PMLR, 2021.
C. Doersch. Tutorial on variational autoencoders. arXie preprint arxi meter166600000 2021.
Fu, C. Li, X. Liu, J. Gao, A. Celikyilmaz, and L. Carin. Cyclical annealing schedule: A simple approach mitigating kl vanishing. 2019. URL http://arxiv.. argg/bs//iat 10145.
T. Geffner, J. Antoran, A. Foster, W. Gong, C. Ma, E. Kiciman, A. Sharma, A. Lamb, M. Kukla, N. Pawlowski, et al. Deep end-to-end causal inference. ai preprint ar 2rXiv:2202.02195, 2022.
C. Gonzales and A. Valizadeh. A full dag score- based algorithm for learning causal bayesian networks with latent confounders. In Proceedings of the 27th European Conference on Artificial Intelligence (ECAT 2024). 2024.
Goodfellow, Y. Bengio, and Courville. Deep Learning. MIT Press, 2016.
A. Gretton, K. Fukumizu, C. H. Teo, L. Song, B. Schölkopf, and A. J. Smola. A kerrel statistical test of independence. In Advances in Newral Information Processing Systems, volume 20, pp. 2007.
C. Hasan, E. Hossain, and M. O. Gani, A survey on causal discovery methods for i.idd and time series data. Technical report, 2023.
Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamod, and A. Lerehner. Beta- vae: Learning basic visual concepts with a constrained variational framework. In Proceedings of the International Conference on Learning Representations (TCT.R, 2017.
13

---

Published in Transactions on Machine Learning Research (January /202)
J. L. Hill. Bayesian nonparametric modelling for causal inference. Journal of Computational and Graphical Statistics, 20(1):217-240, 2011.
D. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint orXiv:1312.6114, 2013.
F. Leeb, S. Bauer, M. Besserve, and B. Schölkopf. Exploring the latent space of autoencoders with interven- tional assays. arXiv:2106. 0er:2106.16091, 16091, 2021.
C. Louizos, U. Shalit, J. M. Mooij, D. Sontag, R. Zemel, and M. Welling, Causal effect inference with deep latent-variable models. In Advances in Newral Information Processing Systems, volume 30, 2017. https://githube www.itttw..ooo///tooooeeee com http:///t.....ttteera.//a/t/t/ta
T. N. Maeda and S. Shimizu. Red: Repetitive causal discovery of linear non-gaussian acyelic models with latent confounders. In Proc. Int. Conf. Artificial Intelligence and Statistics (AISTATS), pp. 735-745. PMLR, 2020.
T. N. Maeda and S. Shimizu. Causal additive models with unobserved variables. In Proc. Uncerlainly in Artificial Intelligence (UAI), pp. 97-106. PMLR, 2021.
J. Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge, 2nd edition, 2009.
J. Pearl and E. Bareinboim. External validity: From do-caleulus to transportability across populations. Statistical Science, 29(4):579-595, 2014.
J. Peters, D. Janzing, and B. Schölkopf. Elements of Causal Inference: Foundations and Learning Algorithmms. MIT Press, 2017.
T. Richardson and P. Spirtes. Ancestral graph markov models. The Annals of Statistics, 30(4):962-103), 2002.
K. Sachs, O. Perez, D. Pe'er, D. A. Lauffenburger, and G. P. Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721):523-527) 529,, 2005.
P. Spirtes, C. Glymour, and R. Scheines. Causalion, Prediction, and Search. MIT Press, 2000.
M. Wüthrich, F. Widmaicr, F. Grimminger, S. Joshi, V. Agrawal, B. Hammoud, M. Khadiv, M. Bogdanovic, V. Berenz, J. Viereck, M. Naveau, L. Righetti, Schölkopf, and S. Bauer. Trifinger: An open-source robot for learning dexterity. In Proceedings of the 2020 Conference on Robot Learning (CoRL), pp. 1871-1882, 2021.
M. Yang, F. Liu, Z. Chen, X. Shon, J. Hao, and J. Wang. Cansalvac: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVP Conference on Computer Vision and Pattern Recognátion (CVPR), pp. 9593 9602, 2021.
T. Yu, D. Quillen, Z. He, R. Julian, A. Narayan, H. Shively, A. Bellathur, K. Hausman, C. Finn, and S. Levine. Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. Proceedings of the Conference on Robot Learning (CoRL), 2019a.
Yue Yu, Jie Chen, Tian Gao, and Mo Yu. Dag-gnn: Dag structure learning with graph neural networks. In Proceedings of the 36th International Conference on Machine Learning. 2019b.
A. Zanga, E. Ozkirimli, and F. Stella. A survey on causal discovery: Theory and practice. arXiv:2305.10032 2023.
M. Zecevic, D. S. Dhami, P. Velickovic, and K. Kersting. Relating graph neural networks to structural causal models. ar Xix::2109.00 04. 04179, 2021.
X. Zheng, B. Aragam, P. K. Ravikumar, and E. P. Xing. Dags with no tears: Continuous optimization for structure learning. In Advances in Neural Information Processing Systems, volume 31, 2018.
14

---

Published in Transactions on Machine Learning Research (January (Jamuary2226)) 2026)
Appendix
Overview
This appendix provides supplementary analyses and experimental details that support and extend the find- ings presented in the main paper, organized as follows:
This section describes the ER graph construction, sampling protocol, and the nonlinear SCM used for training and evaluation.
ER data synthesis (Appendix A):
This section first presents the thresholding algorithms used to binarize the learned adjacency matrices, followed by sensitivity analysis showing that directed recovery is robust to threshold variation and that adaptive thresh- olding provides fully data-driven alternative when ground truth is unavailable.
Threshold selection and sensitivity analysis (Appendix B):
This section reports reproducibility analysis for the ER(4,6,4) and ER(12,50,10) settings, presenting performance across multiple independently sampled graphs.
Additional results on ER (Appendix CD:
ADMG estimation and causality disclaimer (Appendix D): This section clarifies the as-
sumptions underlying ADMG estimation and the limitations of recovering true latent confounding structures.
Hyperparameter sensitivity and training configurations (Appendix E This section com- prises the following: Appendix [E.1 reports the hyperparameter settings used for the synthetic datasets. Appendix E.2 presents an additional ablation study on the regularisation coefficients, while Appendix E.2.1 analyses the effect of linear causal annealing, highlighting the impact of bow- free constraints, sparsity, and annealing schedules.
Additional experiment: Sachs
dataset (early results) (Appendix F):
This section reports preliminary struccure-sstimatiion results on the Sachs dataset, illustrating the applicability of the method to real biochemical data with nonlinear dependencics.
Discussion on limitations (Appendix ditions of the method.
G This section analyses the scope and boundary con-
This section provides consolidated overview of the additional experimental studies reported in the Appendix, including threshold anal- yses, ER reproducibility experiments, hyperparameter ablations, causal annealing schedules, and real-world evaluation.
Summary of experimental results (Appendix H):
A
ER Data Synthesis
ER Configuration:
The ER-based graphs are generated using three parameters: d, e, and m.
d: the number of observed variables or nodes in the causal graph,
the number of directed edges representing direct cause-effect relationships,
m: the number of bidirected edges modeling unobserved confounding between variables.
For example, ER(4,6,4) denotes 4-variable graph with directed edges and 4 bidirected edges. Similarly, ER(12,50,10) corresponds to larger graph with 12 variables, 50 directed edges, and 10 bidirected edges.
15

---

Published in Transactions on Machine Learning Research (January /002)
These settings allow to evaluate how the model performs under increasing structural complexity and the presence of latent confounders. We use the Eq. 18 to generate ER data.
AD ~ ER d, d(d 1)
(iag(AA)) =
{ if there is a directed edge from i to j.
[1
Apli.j]
00 otherwise.
AB(i,j] ~ Bernoulli
m d) diag(A (Ap) =
An triu(Ap, 1) + triu(AA, triu extracts the elements above the diagonal,
~ 0.1,0.13), u~ (0,00.13)
X,
Σ
+
Σ
g(un) + Eq.
(18)
pEPad(()
(EPag(1)
B Threshold Selection and Sensitivity Analysis
In this section, we present the threshold selection method. We also describe the procedure for identifying the optimal threshold to obtain the best score for the learned adjacency matrices.
B.1 Algorithm for Optimal Threshold Selection
To obtain the best-directed (F1p) and bidirected (F1B) adjacency Fl scores, we employ the method described in Algorithm
Figure illustrates the variation of F1 scores with respect to the threshold values to obtain
Algorithm Optimal Threshold Selection for F1 score
Input: Ground truth adjacency matrix A, learned adjacency matrix W, set of thresholds T Output: Best threshold t", maximum Fl score Fl Flmax
Fl Famax + O, tx t 0
4: for each threshold t E T do
5:
Whin + I([W] 2 t)
Binarize W at threshold Convert matrix to vector Convert W bin to vector
a + flatten(A)
+ ClattenWWus
F1, + F1 score(a, w)
if F1, > Flaax then
10:
Flmax + Fl, t-
11:
12: end if
13: end for
14:
return 1", Fl max
the optimal threshold selection for ER(12, 50, 10). Figure 4a shows the best-obtained Fl score for the directed adjacency matrix Ap, while Figure depicts the best-obtained Fl score for the bidirected adjacency matrix Ag. The optimal (best) threshold, corresponding to the peak F1 value in each curve, is selected described in Algorithm 3 The optimal threshold selection plots for the directed and bidirected adjacency matrices the FC dataset are shown in Figure 5 highlighting the threshold values that yield the highest Fl score for each edge type.
16

---

Published in Transactions on Machine Learning Research (January
Directed F1 Score Threshold
Bi-directed Score Threshold
0.5
0.450
0.425
0.4
0.400
F1 Score
FI 5core
0.375
0.3
0.350
0.325
0.2
0.300
0.1
Threshold
Threshold
(a) Obtained best Fl score for learned Ap
(b) Obtained best F1 score for learned Ap.
Figure 4: Optimal threshold selection for ER(12, 50, 10).
Directed Score vs Threshold
Bi-directed F1 Score vs Threshold
1.0
0.500
0.9
0.475
0.450
0.8
F1 Score
FI Score
0.425
0.400
0.6
0.375
0.5
0.350
0.325
01
0.2
0.3
0.5
0.1
0.2
0.3
0.4
0.5
Threshold
Threshold
(a) Obtained best F1 score for learned for Ap.
(b) Obtained best F1 score for Ag.
Figure 5: Optimal threshold selection for FC.
Sensitivity analysis of thresholding: This sensitivity analysis conducted on a random ER graph ER(12, 50, 10). Once trained, the threshold parameter varied analyze F1 score stability.
The model evaluated across multiple threshold ranges, shown in Table Reported the best-directed (Ap) and bidirected (Aø) F1 scores with their corresponding optimal thresholds (TD, TB). The final row shows mean + standard deviation across all settings.
In this example, across all threshold ranges, the directed-edge (Ap) Fl scores exhibit a mean of 0.511710.00 confirming stable directed strueture learning with only minor numerical fluctuation. Although their standard deviation is slightly higher than that of the bidirected edges, the corresponding optimal thresholds (Tp 0.04 0.01) remain narrowly concentrated, demonstrating strong robustness to threshold perturbation.
For the bidirected edges (An), the mean F1 score of 0.346 + 0.005 shows slightly lower numerical variation but wider threshold spread (тв 0.34 + 0.11), reflecting moderate sensitivity due to latent confounding.
Overall, these results confirm that the learned adjacency matrices remain quantitatively stable under thresh- old variation, with directed relations showing consistent robustness and bidirected ones exhibiting moderate sensitivity linked to confounding effects.
In Figure D each pair of subplots shows the variation of F1 scores with changing binarization thresholds for (top) directed edges (Ap) and (bottom) bidirected edges (An)- Directed F1 score remains consistently
17

---

Published in Transactions on Machine Learning Research (January
Table 5: Threshold sensitivity analysis on ER(12, 50, 10).
Count
Threshold Range
Best FlD
TD
Best Flg
TB
0.01 0.50
0.507
0.05
0.341
0.33
2
0.52-0.50
0.519
0.02
0.348
0.57
3
0.03-0.50
0.519
0.03
0.352
0.25
4
0.04-0.50
0.519
0.04
0.341
0.33
5
0.01 0.60
0.522
0.04
0.348
0.24
Mean
SD
0.517 + 0.006
0.04 + 0.01
0.346 + 0.005
0.34
0.11
(a) 1: Ap
(b) 2: Ap
(c) 3: Ap
(d) 4: Ad
(e) 5: AD
dest Coree
(f) 1: AB
(g) 2:
AB
(h) 3: AB
(i) Ab
(j) 5: AB
Figure 6: Optimal threshold selection for threshold sensitivity analysis ER(12, 50, 10).
around 0.5 across small threshold shifts (0.02-0.05), while bidirected F1 score peaks near 0.3 for thresholds between 0.25 0.33, confirming stable causal discovery under latent confounding. In all experiments, the best threshold is selected by varying over fixed grid of candidate values shared across datasets, without any manual per-dataset tuning.
B.2 Adaptive Thresholding
In realistic scenarios where the ground-truth causal graph is unavailable, the adaptive threshold mechanism (Algorithm 4 offers principled alternative for threshold selection. However, since state-of-the-ard methods typically report their best-obtained performance, we also compare against their best-achieved results to ensure fairness. As shown in Table G adaptive thresholding yields consistent Fll scores across runs, that are broadly consistent with best-throshold selection. The directed structure (Ap) demonstrates high stability (mean Fl, 0.508 vs. 0.536), while the bidirected component (An) shows moderate variability due sensitivity latent confounding density.
Table 6: Comparison of adaptive best threshold F1 scores for directed (Ap) and bidirected (Ap) for different FR(12,)0,10 graphs.
Rum
Adaptive Flp
Adaptive Flg
Best Flp
Threshold (TD)
Best Fl
Threshold (TB)
1
0.508
0.304
0.516
0.12
0.333
0.29
0.492
0.348
0.507
0.05
0.341
0.33
3
0.525
0.304
0.586
0.05
0.308
0.26
Mean
0.508
0.319
0.536
0.07
0.327
0.29
18

---

Published in Transactions on Machine Learning Research (annuryy/20000
Algorithm 4 Adaptive Thresholding for Unsupervised Binarization
1: Input: Learned adjacency matrix W, spread threshold scaling factor
2: Output: Binary adjacency matrix Wbin
3: Wabs [
Take element-wise absolute value
4: Wmax f max (nax(aabe))) (abs))
Wmed f- median )
Wmax
Compute spread ratio
(Wmed =)
6: if S then
7:
0 < Wmod
8: else
8 T T. Wmax
10: end if
11: Whin 1-((W] 20
Binarize adjacency matrix
12: return Win
Reliability verification thresholding methods: To robustness of the proposed structure learning, we further performed reliability testing the FC dataset, which has fixed causal graph. We compared three thresholding strategies: absolute value cutoff (JW]mas 2 0.45), adaptive thresholding, and best threshold search and observed consistent results across all settings. This confirms that the learned adjacency patterns remain stable under different thresholding criteria and that the proposed G-ADMG-CL framework yields reproducible causal struclures across multiple evaluation protocols.
C Additional Results on ER
In this section we provide reproducibility analysis for the ER(4, 6, and ER(12, 50, 10) configurations. Each run corresponds new random graph sampled under the respective ER settings. Table and Table S report F1 scores across five such runs. Notably, the bidirected recovery remains relatively stable across
and
Table 7: Additional results for ER(4,6,4) dataset with five independent graphs.
Run
FlD
FiB
0.933 0.800
0.800
0.842
0.857
0.880
0.800
0.889
0.920
0.889
Mean + Std
0.835 +: 0.051
0.887 + 0.031
Table 8: Additional results for ER(12, 50, 10) with five independent graphs.
Run
F1n
FlB
0.576
0.316
2
0.589
0.450
3
0.538
0.414
4
0.510
0.450
5
0.452
0.424
Mean + Std 0.533 + 0.049 0.411 + 0.052
runs, highlighting the robustness of our model in consistently capturing latent confounding even in dense, high-dimensional graphs such 12)
19

---

Published in Transactions on Machine Learning Research (January /2026
D ADMG Estimation and Causality Disclaimer
ADMGs provide principled framework for representing causal structure under latent confounding; how- ever, learning a causal graph from observational data alone is fundamentally limited (Peters et al. 2017 approaches cannot guarantee recovery of the exact ground-truth SCM, since multiple ADMGs may induce the same observational distribution (Markov equiv- alence) (Spirtes et al. 2000), and the placement of latent confounders may remain ambiguous.
Pearl & Bareinboim 2014]. Contimouss-opimizaaie
In alignment with prior work (Ashman et al. 2023 we follow ADMG structure estimation assumptions that allow for latent confounding, offering identifiability criteria for ADMGs in nonlinear additivc-noiso SCMs, although it does not ensure the structural identifiability of the underlying magnified SCM. Within this framework, each variable is represented as a nonlinear function of its direct causes, combined with independent noise and exogenous latent confounders, with no interaction effects between the observed and latent.
While these assumptions provide theoretical grounding, practical recoverabilityy may still be limited when real data violate the additive-noise or independence conditions. Accordingly, the proposed method learns one ADMG structure that is consistent with the observed data; however, it does not constitute definitive causal proof, rather, the learned graph should be interpreted as plausible causal hypothesis obtained through robust ADMG estimation, supported by the data and the adopted modelling assumptions.
E Hyperparameter Sensitivity and Training Configurations
E.1 Training Configurations for Synthetic Datasets
We present the training configuration table for the synthetic data FC and ER below Table
Table 9: Training configurations for FC, ER(4,6,4), and ER(1,,50,10) experiments. Hyperparameters include KL annealing, learning rate schedules, and regularization weights (A) for entropy, symmetry, cycle, and bow- free constraints.
Setting
FC
(4,6,444
ER(12, 50, 10)
Epochs
5000
3000
2000
Optimizer
ADAM
ADAM
ADAM
Learning Rate Schedule
ExponentialDecay
CosineDecay
ExponentialDecay
Initial Learning Rate
0.001
0.001
0.001
Decay Steps
1000
1000
64
Decay Rate
0.90
0.98
Minimum Learning Rate (ax)
x 10-5
KL Annealing End Epoch
50
100
800
Final kl weight
1.5
1.5
1.5
Latent Dimension
Zdir: 24, Abidiri 24
24
36
Acycle
5
Abow
5
5
5
Xentropy (Ap)
0.005
0.001
0.001
Xentropy (AB)
0.01
0.01
0.001
Aasymmetry (Ap)
0.05
0.05
0.05
Asymmetr (AB)
0.5
1.5
1.75
sppariity (Ad)
0.001
0.001
0.001
ssparsity (AB)
0.02
0.001
0.001
Causal Transition Epoch (CTE)
150
150
1000
20

---

Published in Transactions on Machine Learning Research (Jamuary/226)) 2026)
E.2 Additional Ablation Study
Abow sensitivity: We first lune Abow to enforce bow-freeness.
We performed an ablation over the bow-free coefficient Abow on the FC data, varying it in [1.0, 5.0], depicted in Figure 7 The Flp increased with Abow and peaked at Abow E f4.0, 5.0} (F1p = 1.0), confirming that a sulficiently strong bow-free constraint is beneficial. We therefore fix Abow 5.0 in all main experiments and report the sensitivity curve.
Sensitivity of Flp to Bow-Free Coefficient (Abow)
1.00
0.95
Flo
0.85
0.80
0.70
2
Abow
Figure 7: Sensitivity of Fl to the bow-free regularization coefficient. (Atow) on the FC dataset. The model achieves the highest FIn at Anow {4,5), confirming that moderate penalization yields optimal separation between directed and bidirected dependencics.
Sparsity sensitivity: We further analyze the effect of the sparsity coefficient Asparse on the bidirected weight matrix W2 using the FC dataset. As shown in Table 10 the directed-edge accuracy (F1 remains consistently perfect across all values, while the bidirected Flg improves slightly with higher sparsity, peaking at Auparse E (0.02, 0.03) (Flg = 0.5). Higher sparsity (0.04) begins to prune truc directed edges. This indi- cates that moderate sparsity eucourages better disentanglement of confounded relations without degrading directed structure recovery.
Table 10: Effect of sparsity coefficient structure recovery (FC dataset).
Apparse
F1p
F1 B
0.005
1.0
0.4
0.01
1.0
0.4
0.02
1.0
0.5
0.03
1.0
0.5
0.04
0.8
0.5
Guidelines for setting structural hyperparameters:: In practice, we tune Abow first to ensure bow- freeness, then adjust Aeparse and Aentropy- For datasets with structured and sparse confounding (c.g.. FC), a moderate increase in (11) helps suppress weak spurious links while keeping Flp stable. Excessively large Acycle should be avoided, as it remove genuine directed dependencies.
For the directed adjacency Ap, we keep the entropy weight low to avoid randomization of edges, and use sparsity with a balanced ratio between sparsity and entropy:
PD 4
Asparse
enttrpy
21

---

Published in Transactions on Machine Learning Research (January /202)
This keeps AD parsimonious without injecting noise; excessively high sparsity or entropy can hamper the recovery of true directed relations.
For the bidirected adjacency AB, the choice of sparsity depends on the nature of the latent confounding,
When confounding is sparse and structured (e.g., FC), higher (B) improves disentanglement of true bidirected edges. However, for dense and randomly distributed confounding patterns (as in 50,11,00,)0), a smaller sparsity weight is preferred, because strong sparsity prunes true confounding edges and destabilizes symmetry, leading to degraded F1 performance. Thus, sparsity must be tuned per dataset rather than using global value.
Symmetry asymmetry balance: To preserve the structural properties of ADMGs, the symmetry regu- larization on AB must. dominate the asymmetry penalty on AD:
A
(B) symmetrr
(D) Aasymmetry'
This enforces clean separation between directed and bidirected components, preventing leakage of asym- metric patterns into AB and avoiding spurious symmetric patterns in AD- In practice, we choose ( Aeymmetry E [1.0, and X E [0.05, 0.1), maintaining stable ratio that empirically improves identifiability.
Assmmeetr
In practice, we first tune \bow to enforce bow- now-freeness, then choose (Asparse, enntropy) with PD and a small absolute entropy weight, and finally adjust (A (BB) entropy to control the density of AB when graphs are potentially complex.
A more exhaustive pearsty--ensitivvee analysis across wider range of graph families and real-world datasets is beneficial, and we identify this an important direction for future work.
Over-sparsiffiation effect: While moderate sparsity (Aeparse [0.01,0.03) preserves perfect directed recovery on FC 1.0) and slightly improves FlB, higher value (Neparse = 0.04) reduces Fl, to 0.8, indicating sprrspaasiiiiatiin This confirms a tradc-off: excessive sparsity can prune true directed edges. Accordingly, X(parse S 0.03 and recommend separate sparsity terms for directed and bidirected components, luning Asparse independently when confounding is complex.
Practical observation during training: In real-world scenarios, the true causal structure is unknown, and direct F1-based evaluation is infeasible. In such cases, monitoring the evolution of the learned adjacency matrices AD and AB during training provides useful qualitative guidance. Stable directed adjacency patterns with low entropy indicate convergence toward interpretable causal relations, whereas excessive sparsity or rapidly fluctuating weights suggest over-regularization or noisy learning. Hence, users can rely on the relative stability and sparsity of Ap and AB as proxies to tune Asparse and Xentropy when ground truth graphs are unavailable.
E.2.1 Linear Causal Annealing (Initial Study on ER112,50,10
In addition to hard causal annealing (used in all main experiments). we conducted preliminary tests with a linear causal-annealing schedule on the more complex (11,5,00,10 graph, where the cansal-regularization weight is kept at for the first 500 epochs, then increased linearly over 1000 epochs, and finally fixed at 1. Across two independent batches of experiments linear causal annealing yiclds directed-odge F1 values the range 0.45 0.57 and bidirected-edgee F1 in the range 0.26 0.40. These values are comparable to those obtained under hard annealing, indicating that the linear schedule does not provide systematic performance benefit under the current configuration.
Overall, the linear schedule yields performance very similar to that of hard causal annealing, with no significant or systematic improvement. This suggests that the proposed hard annealing scheme is already sufficiently stable for structure learning in practice. However a more cxhanstive exploration of noulincar annealing schedules (e.g., cosine or exponential warm-up) is a promising direction for future work but remains outside the scope of the current revision.
22

---

Published in Transactions on Machine Learning Research (Januar/
F Additional Experiment: Sachs Protein-Signaligg Dataset (Early Results)
We further conducted preliminary experiments on the well-known Sachs protein-signaling datasct Sachs et al. 2005), which models biochemical interactions among 11 observed variables (PKC, P38, Jnk, PKA, Rof, Mek, Erk, Plog, PIP2, PIP3, Akt). The original dataset does not explicitly include latent confounders, but in practice, unmeasured experimental factors may introduce confounding effects. The dataset remains widely used real-world benchmark for real-world causal structure discovery.
Sachs provides curated directed graph but does not define ground-truth bidirected edges. Following prior causal-discovery practice, we construct minimal hcuristic reference set capturing known co-regulation patterns purely for qualitative assessment. Thus, Flg should be interpreted as an indicator of latent- dependency capture, not as strict accuracy.
Ground-truth edges: The directed edges used for evaluation are:
PKC + (P38, Jnk, Raf, Mck), PKA + {Jnk, Raf, Mck, Erk, Akt]. Mek - Erk, Pleg + PIP2, PIP2 + PIP3, PIP3 + Akt.
Raf + Mck,
Heuristic shallow bidirected reference edges (used only for qualitative comparison):
{PKC <> Jnk, PKC ++ PIP3, Raf t Erk, Mek ++ Akt].
As an early evaluation result on the Sachs dataset, the proposed G-ADMG- CL model achieves a directed-edge
Table 11: Recovered directed and bidirected edges on the protein signalling network using G-ADMG-CL (evaluated against the ground-truth adjacency An used in our implementation).
Edge Type
Recovered Edge
Directed Edges
Correct
PKC Jnk
Correct
PKA + Mek
Correct
PKA - Erk
Correct
PKA + Akt
Correct
PIP3 + Akt
Spurious
Jnk + Erk
Spurious
Mek + PIP3
Spurious
Pleg + Raf
Spurious
PIP3 + P38
Spurious
Mek + PIP3
Spurious
Akt + Pleg
Bidirected Edges
Correct (heuristic)
Raí <+ Erk
Spurious
P38 <+ Jnk
Spurious
PTP2 <> P38
Spurious
PKC <> Akt
Spurious
PKC <> Pleg
Spurious
Mek ++ Pleg
Spurious
Multiple weak cross-links
recovery of Flp 0.387 at threshold 0.41 and bidirected-odge recovery of Flg 0.117 at threshold 0.05 after training for 150 epochs with CTE = 50. For these experiments, we set the structural regularisation weights to Acycle = and Abow = 5. stronger sparsity weight is applied to the bidirected component (Asparsity (Ap) = 0.1) because increased sparsification helps suppress spurious bidirected links while prescrving the directed structure, whereas a lower weight (Aeparsity (Ap) 0.001) is used for the directed component, following the hyperparameter guidelines in Appendix F. To assess binarisation real data. Figure presents
23

---

Published in Transactions on Machine Learning Research (January
Directed (W1 D) F1 Score Threshold
Bidirected (W2 B) F1 Score Threshold
0.40
0.10
0.30
Score
0.08
0.25
0.06
0.02
0.15
0.00
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Threshold
Threshold
(a) Directed (W1 vs AD) Fl score vs threshold.
(b) Bidirected (W2 vs AB) Fl score vs threshold.
Figure 8: Optimal threshold analysis on the Sachs protein signaling dataset
the optimal threshold curves for the Sachs dataset, and Table 11 reports the corresponding structure- metrics.
These preliminary findings suggest that the proposed G-ADMG-CL framework is generalizable to real- world biochemical systems.
G Discussion on Limitations
Shallow vs. complex confounded associations: The model reliably captures structured shallow con- founding, including nodes that simultaneously participate in both directed causal and latent confounded relations, exemplified by the FC variable r4. Recovering such mixed-role variables demonstrates the capac- ity of G-ADMG-CL capture observed and unobserved causal relations within a unified ADMG framework. The method further maintains robust directed recovery in dense, high-dimensional scttings, evidenced by the ER(12,50,10) graph where it achieves average Fln 0.53, outperforming existing ADMG learners under comparable conditions. Performance degradation may arise under extremely dense overlapping con- founding where many variables share entangled latent causes, while directed edges remain stable, indicating graceful degradation Ag estimation.
Case study on (11,,000,)) To examine latent confounding recovery in detail, we analyze one rep- resentative ER(12,50,10) graph, where the model achieves Fln 0.584 and Flg 0.308. Shallow con- founding denotes pairwise hidden causes (e-g., X₂ < Xs), whereas complex confounding arises when multiple variables share overlapping latent dependencies. As shown in Table 12 the proposed method recov- ers most shallow confounding relations (c.g., X1 ++ Xg, X, + X6). In multi-variable scttings (c.g., among Xs, X1o, Xn1,, additional weak bidirected links appear, explaining the moderate Fl 0.308 despile high recall.
Worst-case bidirected performance: Across multiple independently sampled ER graphs, when con- founding signals are weak, diffuse, or overlap with directed edges, bidirected recovery enters worst-case regime. The ER(12,50,10) setting (Flg 0.308) is representative of this bchaviour, where densely entangled latent structures make it difficult lo distinguish true shared confounders from residual dependence. Directed- edge recovery remains stable, while bidirected estimation degrades gradually, providing an empirical lower bound on the ability to recover complex latent confounding from observational data alone.
Limitations and failure modes: As with other continuous optimization- based causal discovery ap- proaches, such as NOTEARS and DAG-GNN, our framework does not ensure unique recovery of the truc underlying structural causal model across all possible lata-generating processes. Rather, its objective is achieve precise estimation of ADMGs.
24

---

Published in Transactions on Machine Learning Research (January /202)
Table 12: Comparison of ground truth vs. learned confounding relations at тB = 0.26
Pair (X; + Xj)
GT
Learned
Match
X1 X9
X1 X12
1
1
X, + X6
1
1
X5 + X,
1
1
Xs + X
1
O
X
X6 + X2
1
1
N6 X3
X7 ++ Xs
X7 X8
X7 ++ X12
Xg + X1
1
1
X10 + X,
0
X10 + X11
1
X11 + X10
1
1
X12 X1
1
1
X12 X7
1
1
Total Correct
16
14
87.5%
As outlined below following practical failure modes may affect the recoverability of both the directed adja- cency matrix Ap and the bidirected adjacency matrix As-
Weak causal effects: When directed causal effects are small relative to noise, the induced statistical dependencies become insufficient for reliable detection, reducing the recoverability edges in Ap.
Near-deterministic relationships: Extremely strong or near-deterministic functional relation- ships can collapse latent variability, leading to edge misclassification or spurious bidirected connec- tions in Ag.
3. Overlapping latent confounders: When multiple latent confounders induce similar covariance patterns, the model may attribute confounding to incorrect variable pairs. In such cases, true bidirected edges in Ag may be misinterpreted as weak directed edges in Ap, producing ambiguity between the two edge types.
Highly correlated parents: If several parents of a child variable exhibit strong correlation, the resulting dependence patterns may be statistically indistinguishable without interventions, creating unavoidable ambiguity in both Ap and AB.
These limitations reflect inherent challenges of obscrvational causal discovery.
H Summary of Experimental Results
addition to the robustness demonstrated the main paper, this section provides a consolidated summary of the experimental studies and analyses reported in the Appendix.
Threshold sensitivity and adaptive thresholding (Appendix B): We conduct sensitivity analysis on optimal (best) threshold selection to analyze F1 score stability. Directed structure recovery is robust to threshold variation, whereas bidirected recovery is more sensitive. Adaptive thresholding provides a fully data-driven alternative that performs comparably to best-threshold selection when ground truth is unavailable. We further verify threshold reliability using absolute- value cutoff, adaptive thresholding, and best-threshold selection, observing consistent results across all settings the FC dataset.
25

---

Published in Transactions on Machine Learning Research (January
Additional results ER graphs (Appendix C Reproducibility analysis for the ER settings is described in this section, illustrating the consistent capture of latent confounding even in dense, high-dimensional settings, such as 10,)
Additional ablation study (Appendix E.2 We conduct additional ablation study the regularisation coefficients to analyse hyperparameter sensitivity. Increasing the bow-free regu- larisation coefficient Ahow improves directed edge recovery, with optimal results at Abow E {4.0, 5.0]; Abow = 5.0 is used in subsequent experiments. For sparsity (Asparse), moderate values E {0.02, 0.03) enhance bidirected F1 without harming directed accuracy, while higher sparsity (0.04) removes true edges, revealing a trade-off between interpretability and robustness. Separate sparsity terms for di- rected and bidirected matrices are recommended, particularly under complex confounding. Practical guidelines include tuning Xbow first, balancing sparsity and entropy for directed edges, and setting bidirected sparsity according to dataset-specific confounding. Symmetry regularisation should dom- inate asymmelry penalisation to ensure clear separation of dependency types. Monitoring adjacency matrix evolution during training provides useful diagnostics when ground truth is unavailable.
Overall, strong bow-free regularisation and moderate sparsity support accurate graph recovery, whereas excessive regularisation degrades performance.
Causal annealing schedules (Appendix E.2.1): ): Linear causal annealing achieves performance comparable to hard annealing across ER settings, indicating robustness w.r.t the annealing strategy.
Additional experiment on Sachs protein-signalinnn dataset (early (esults(Apenddi)) F ):
An early evaluation performed on the Sachs protein-signalinn signaling dataset, the model recovers mean- ingful directed relationships consistent with known biological pathways, while bidirected edges are interpreted qualitatively due to the absence of ground-truth confounding annotations.
The analysis in- dicates that the model captures shallow and structured confounding, including variables with both directed and bidirected roles (e.g.. FC x4). Performance degradation may arise under extremely dense overlapping confounding, while directed edge recovery remains robust and bidirected estima- tion degrades gracefully.
Shallow vs. complex confounding and worst-case analysis (Appendix G):
26


Reviews:
[{"review_id": "jLoBOczl5R", "reviewer": "Reviewer_2qMB", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Thank you for working towards a solution to the hard problem of causality within AI.\nIn the following, I try to provide an honest, respectful and most importantly helpful review such that this work can grow to be a great contribution to our community.\n \nThe present manuscript provides an effort in using a VAE approach to handle the task of learning a causal structure from data.\nThe data is generated by a Pearlian SCM and we are only given a causally insufficient set of variables i.e., hidden confounders are present in the actual data generating process.\nThe authors model the structural relationships between any pair of variables by splitting the task at hand into a directed and bi-directed graph respectively.\nThe authors model this split via two latent layers in the VAE that are being optimized in tandem.\nThe authors make use of various learning strategies (mainly, structural constraints and annealing).\nThe authors present experiments two-fold: 1) how well the method is able to recover the correct structure(s), 2) how well the learned structure(s) perform in downstream causal inference.\n\nThe strengths of the paper IMHO are:\n* A great research problem\n* Ablating across various learning techniques\n* Detailled technical description for reproduction\n\nThe weakness of the paper IMHO are:\n* No guarantee for causality in the learned model\n* Triviality: \"Shallow\" hidden confounder\n* Missing scholarly input (insufficient related work)\n* Weak experimental evidence\n* Error-prone / Unpolished presentation\n* Missing code base for reproduction", "claims_and_evidence_yn": "No", "claims_explanation": "Regarding the strengths:\n* A great research problem: since causality for AI is far from solved, yet its inferences are undeniably necessary for any sort of actual AI, any progress towards the matter, however miniscule, would be important.\n* Ablating across various learning techniques: since any of the given techniques can play a crucial role in promoting any given method to outperform the competition in any given recovery metric.\n* Detailled technical description: since reproducibility is a pillar of science and especially nowadays with growing complexities in systems it is in particular important to know how to reproduce a given result (I've not reproduced any of the results and also I could not find any code that was provided, however, the descriptions at least do seem precise enough for reproduction).\n\nRegarding the weaknesses:\n* No guarantee for causality: as with other continuous-optimization approaches to structure learning (e.g. NOTEARS) we focus on learning DAGs, and in the case of the present manuscript, ADMGs. While these structures are used in causality and also the data is assumed to stem from a corresponding SCM, this family of methods does not guarantee learning the structure of said SCM. No term in the optimization can actually enforce that and the model itself does not maintain an ability to proclaim said result.\n* \"Shallow\" hidden confounder: as per Reichenbach's common cause principle, we know that if there is correlation between two variables, then there must be causation present either in the form of direct causation between said two variables (in which case correlation implies causation) or in the form of a third variable, a hidden confounder, causing both variables (in which case correlation does not imply causation). The authors use a \"bow-free\" constraint. The bow referring to the idea that the structure of three variables resembles a bow, when there is a hidden confounder but also a direct causation between the two named variables. Now, if we have a bow-free assumption, then necessarily it must be true that the correlation that we observe between the two given variables stems from a hidden confounder, therefore, rendering this result trivial.\n* Missing scholarly input: the referenced related work is insufficient in covering the existing literature, specifically also with similar or adjacent VAE approaches in consideration. What comes to mind immediately are for instance following three works on causality and VAE (some also do CSL, whereas others focus on inference - the authors downstream task): (1) [Yang et al. 2021](https://openaccess.thecvf.com/content/CVPR2021/papers/Yang_CausalVAE_Disentangled_Representation_Learning_via_Neural_Structural_Causal_Models_CVPR_2021_paper.pdf), (2) [Leeb et al. 2023](https://arxiv.org/abs/2106.16091), and (3) [Matej Zečević et al. 2021](https://arxiv.org/abs/2109.04173)\n* Insufficient experimental evaluation: the deployed testing ground is well-known in the causal community and IMHO generally considered more as a sanity-check rather than an actual real world relevant assessment. Given that the presented work is in particular keen on beating any state of the art approach, this point actually becomes a worthwhile weakness to discuss.\n* Error-prone / Unpolished presentation: the present manuscript suffers from several language and typesetting mistakes. Furthermore, the complete style of presentation is rather reminiscent of a \"technical report\" rather than a scientific paper.\n* Missing code base for reproduction: it is crucial for a scientific community to be able to independently evaluate any given result of technical form, in particular, when the key contribution is of technical nature. Ideally, the VAE approach should be plug-and-play available to anyone (for now, the reviewers)", "audience_yn": "Yes", "audience_explanation": "Since the TMLR community is concerned with not just causality and causality for AI but more importantly general learning approaches, in particular also for structure learning, this work speaks to a broader audience.", "requested_changes": "Going over the weaknesses one by one:\n* No guarantee for causality: this is a fundamental thing and I cannot expect the authors to resolve this within a revision. However, we need to provide a disclaimer on this and more than just a single sentence since this is a sensitive and important topic.\n* \"Shallow\" hidden confounder: this is my biggest concern since to me the \"learning about unobserved confounders\" aspect of this structure learning approach is proclaimed (or should be) its biggest selling point. This is the actual core research problem that is supposed to be solved, I cannot give any proposed answer to this.\n* Missing scholarly input: please once again do a thorough scholarly search, make use of the references I've provided and dangle within their citations trees to find more relevant work more easily and extend the existing discussion, especially w.r.t. (dis-)similarities.\n* Insufficient experimental evaluation: within the causality community we make use of more complex settings e.g. robot environments (for example the MPI tri-finger environment: https://webdav.tuebingen.mpg.de/trifinger/), test within these simulators.\n* Error-prone / Unpolished presentation: ideally let internal, independent reviewers pass over the paper to find mistakes that you as authors are unable to spot anymore since you have been working so much on this. I'd much more prefer this over just an AI pass, however, even that should've resolved the mistakes that I found within the present version.\n* Missing code base for reproduction: prepare the code for sufficient online presentation (i.e., instruction on how to run and dependency lists) and post it online (anonymously) and provide the URL.", "broader_impact_concerns": "I've no concerns of any ethical nature regarding this work, whereas the potential impact of a success in this particular domain of research would have far reaching implications in all of science and industry.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yTbjoixB5i", "reviewer": "Reviewer_dbKT", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a VAE-based method G-ADMG-CL for disentangles unobserved confounders. The core contribution lies in learning dual latent spaces, which seperately model the directed adjacency matrix, and the bidirected adjacency matrix. The method stabilizes training through a set of causally aware objective functions and a causal-annealing training strategy.\n\nStrengths:\n1) The authors ground their assumptions and training constraints in established identifiability results, ensuring a solid theoretical foundation.\n2) The ablation studies and thorough comparative experiments enhance the credibility of the results.\n\nWeakness：\n1) The implementation is limited to small-scale variables and semi-synthetic data. Although the authors emphasize that the method benefits downstream causal-effect estimation tasks, the experimental conclusions do not fully demonstrate this advantage.\n2) There is no sufficient theoretical proof establishing the consistency between the dual latent spaces disentangled from data and the observed/unobserved causal graphs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The supporting evidence is strong, and the experiments are thorough. The ablation studies and thorough comparative experiments enhance the credibility of the results.", "audience_yn": "Yes", "audience_explanation": "Causal structure learning in the presence of latent confounders is an important research problem. The authors introduce a variety of causally-aware objective functions and, by combining them with a VAE framework, greatly enhance the flexibility of the work, offering rich inspiration to the causal community.", "requested_changes": "1) Add identifiability guarantees and a comprehensive failure-mode analysis: clarify the conditions under which A_D and A_B are recoverable (addressing Weakness 2), identify which edge types are most error-prone, and specify when A_B is likely to be misclassified as A_D. Such supplements will strengthen theoretical confidence and guide practical deployment.\n\n2) Thresholding sensitivity analysis required: Report how A_D and A_B F1-scores vary with different binarization thresholds, clarify whether per-dataset tuning is necessary, and compare a single global threshold against an adaptive one. This ensures that reported performance is reliable.\n\n3) The current experimental results can demonstrate the effectiveness of the work; however, the evidence regarding its impact in real-world scenarios, as claimed in the paper, remains somewhat weak.Expand real-world and down-stream validation: Beyond IHDP, include at least one additional real-world dataset with latent confounding and provide structure-level metrics. For down-stream validation, synthetic or real-world causal effect estimation is necessary.", "broader_impact_concerns": "I do not see any concern on the ethical implication.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "KpOVSAeUTq", "reviewer": "Reviewer_Bhj7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Authors of this paper proposed a novel generative method for causal structure learning in the presence of unobserved confounders. The proposed model is built on variational autoencoder with dual latent spaces to represent directed cause-effect relations and the bidirected unobserved confounded relations. A causal mixed graph loss and a novel causal annealing strategy are introduced to learn meaningful causal structures. Experiments on synthetic and real-world datasets demonstrating the effectiveness of the proposed model for causal structure learning in the presence of unobserved confounders. However, the proposed causal mixed graph loss is not well defined and explained, so it is difficult to justify the correctness of the proposed model. The experiments could be significantly improved by reporting robust convincing results and providing proper guidelines to setup model hyperparameters.", "claims_and_evidence_yn": "No", "claims_explanation": "There are several critical concerns. First, the proposed model is not well presented, especially the proposed causal mixed graph loss. This makes the justification of the proposed model difficult. Second, the experimental results are not reported in a convincing way, such as the best result of 5 runs is reported, and the analyses are performed on the best results. Third, the proposed model is a combination of many factors, but their contributions are not properly explored. Since there lacks the guidelines to setup so many hyperparameters in the proposed model, the usability of the model may be limited.", "audience_yn": "Yes", "audience_explanation": "The disentangle of two different graphs based on VAE may be interested for a small group of individuals.", "requested_changes": "The loss function in (3) is a combination of many items. However, these loss functions are not properly explained or even defined. For example, what is the reconstruction loss in VAE? Authors need to explain why minimizing (7)-(8) can achieve acyclicity and bow-free. Notice that the losses of symmetry-asymmetry and sparsity are not defined. And the sparsity on A_B is missing. It is recommended to clarify these loss terms clearly and accurately.\n\nWhat is the adjacency matrix estimation in step 17 of Algorithm 1?\n\nIn section 4, 4000 samples for FC are sampled, but it is unclear for ER.\n\nThe results in section 5 are reported using the best run over 5 repeated runs. According to Table 5 and Table 6, the best result is much better than the mean of 5 runs. In addition, the repeated experiments for ER are done for randomly sampled graph structure, while the varied number of samples can also have impact on the quality of the learned causal structure. In real application, the ground truth graph is unknown, so it is impossible to select the best run. The recommendation is to report the mean+std for all compared methods in Table 1. The case study could be done for the best and the worst runs to see how different they are.\n\nAblation study in Table 2 shows that all items of the proposed causal loss contribute to the final output. However, the study was only performed by excluding one item each time. This is not helpful to set so many hyperparameters in (3) as did in Table 7. Notice that the hyperparameters are not constant for all three datasets, and some hyperparameters are not included in the table such as $\\lambda_{KL}$ and $\\lambda_{sparse}$. It seems that the sparsity hyperparameter should be more important to the final graph structures, but it is not investigated. Authors need to provide proper parameter sensitivity analysis to provide guidelines for users.\n\nCausal annealing experiment shown in Table 3 was done for comparing hard causal annealing and without annealing. However, the experiments are done for linear interpolation. Authors are recommended to add one with linear case. In addition, the CTE and linear transition start epoch $e_t$ will have impact on the final output. So, the ablation study over these hyperparameters is interested.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)