THE ROBUSTNESS OF DIFFERENTIABLE CAUSAL DISCOVERY IN MISSPECIFIED SCENARIOS
Authors: Huiyang Yi
Yanyan He
Duxin Chen
Mingyu Kang
He Wang
Wenwu Yu
Venue: ICLR 2025 Poster (2025)
Keywords: Differentiable causal discovery
model assumption violations
benchmark
Primary area: datasets and benchmarks
TL;DR: Causal discovery algorithms are typically based on untestable causal assumptions. We study the performance of several mainstream algorithms under model assumption violations and find that differentiable causal discovery exhibits robust performance.

Abstract: Causal discovery aims to learn causal relationships between variables from targeted data, making it a fundamental task in machine learning. However, causal discovery algorithms often rely on unverifiable causal assumptions, which are usually difficult to satisfy in real-world data, thereby limiting the broad application of causal discovery in practical scenarios. Inspired by these considerations, this work extensively benchmarks the empirical performance of various mainstream causal discovery algorithms, which assume i.i.d. data, under eight model assumption violations. Our experimental results show that differentiable causal discovery methods exhibit robustness under the metrics of Structural Hamming Distance and Structural Intervention Distance of the inferred graphs in commonly used challenging scenarios, except for scale variation. We also provide the theoretical explanations for the performance of differentiable causal discovery methods. Finally, our work aims to comprehensively benchmark the performance of recent differentiable causal discovery methods under model assumption violations, and provide the standard for reasonable evaluation of causal discovery, as well as to further promote its application in real-world scenarios.

Published as a conference paper at ICLR 2025
THE ROBUSTNESS OF DIFFERENTIABLE CAUSAL DISCOVERY IN MISSPECIFIED SCENARIOS
Huiyang ;i;; Yanyan He?; Duxin Chen' Mingyu Kang², He Wang', Wenwu Yu'
1 School of Mathematics, Southeast University, Nanjing 210096, China Schooo of Cyber Science and Engineering. Southeast University, Nanjing 210096, China yihuiyang@ edu. en, heyy@seu. edu. en, chendx@seu edu. cn kangmingyu, chi @ com, wanghe 91 @seuu. edu. en, wwyuêseu, edu. cn
ABSTRACT
Causal discovery aims to learn causal relationships between variables from targeted data, making it fundamental task in machine learning. However, causal discovery algorithms often rely unverifiable causal assumptions, which are usually difficult to satisfy in real-world data, thereby limiting the broad application causal discov- ery in practical scenarios. Inspired by these considerations, this work extensively benchmarks the empirical performance of various mainstream causal discovery algorithmss, which assume i.i.d. data, under eight model assumption violations. Our experimental results show that differentiable causal discovery methods exhibit robustness under the metrics of Structural Hamming Distance and Structural Inter- vention Distance of the inferred graphs in commonly used challenging scenarios, except for scale variation. We also provide the theoretical explanations for the performance of differentiable causal discovery methods. Finally, our work aims to comprehensively benchmark the performance of recent differentiable causal discovery methods under model assumption violations, and provide the standard for reasonable evaluation of causal discovery. as well as further promote its application in real-world scenarios.
INTRODUCTION
In the realm of modern science, numerous endeavors hinge upon the elucidation of underlying causal mechanisms. However, owing to practical constraints including costs, risks, and ethical implications, the execution of randomized experiments frequently proves unviable. Consequently, mining causal
relationships from purely observational data, known as causal discovery. plays a crucial role in
addressing causal questions such as intervention and counterfactual (Peters al., 2017 Spirtes et 2001 Pearl 2009
Pearl & Mackenzie 2018).
Causal discovery encompasses a comprehensive suite of methodologies, primarily categorized into constraint-based score-based, functional causal model-driven, and gradient-based approaches. These methods often rely on unverifiable causal assumptions as their foundation (Peters et al. 2017 Vowels
et al. 2022). Constraint-based methods, notably PC (Spirtes & Glymour] 1991) and FCI Spirtes
ct al. 1995). meticulously reconstruct causal graphs to the Markov equivalence class (MEC) through rigorous statistical independence tests, guided by the faithfulness assumption. Score-based techniques, such as GES (Chickering) 2002), employ a scoring function to quantify the congruence between an equivalence class and observed data, optimally searching the vast landscape of directed acyclic graphs (DAGs) to identify the MEC.
To transcend the limitation of solely identifying MECs from observational data, functional causal model-based methods, exemplified by LiNGAM (Shimizu et al. (2006). leverage precise assumptions regarding the functional class and noise distribution within structural equation models (SEMs), enabling the unambiguous identification of a unique DAG. Recently, Zheng et al. (2018) introduced gradient-based techniques (c.g. NOTEARS (Zheng al. 2018)), which convert combinatorial acyclic
Equaa Contribution.
1 Duxin Chen the corresponding author.

---

Published as a conference paper at ICLR 2025
constraints into smooth equality constraints and solve the optimization by transforming equality- constrained optimization into unconstrained optimization through the augmented Lagrangian method
(ALM) (Nemirovsky 1999). In some literature (Zhang et al. 2023 Liu et al. 2023), gradient-based methods are also referred to as differentiable causal discovery.
Apart from the various assumptions of the methods above, traditional approaches typically rely on causal sufficiency and no measurement error assumptions to simplify the problem (Peters et al. [011)] Zhang et al. 20118) Real-world data often fail to meet all of these assumptions, and these are also impossible to verify adequately (Peters et al. 2017). Although some studies have considered the
complexity of real data and developed algorithms targeted latent confounders (Spirtes et al. 1995
Xie et al.] 2020: (Salehkaleyban et al. 2020; Cai et al. 2019 Kong et al., |023] Cai ct al. 2023).
measurement error (Zhang et all] 2018 Dai et al. 2022), , heterogeneity (Huang et al., 2020 Cai et al.
2020: Ghassami et al. 2017 7 2018), scale variation (Shimizu et al. 1 et al. 2023 Deng
et al. 2024), and missing data (Tu et al. 2009aa Gao et | (022), the true mechanisms remain unclear when the causal discovery algorithms applied to real data. These specifically designed algorithms also cannot be effectively employed for real data. Therefore, the robustness of causal discovery algorithms in scenarios where model hypotheses are violated is of great importance.
Previous research (Heinze- Demnzee-emllae et al. 2018)) mainly evaluated various constraint- based and score- based algorithms under different scenarios, only focusing on linear SEM. The work (Mooij et al.) 2016) benchmarked causal discovery for nonlinear additive noise models and iefoormationggommettti approaches, limiting to bivariate scenarios. The previous study (Singh al. 2017) primarily assessed algorithms that use only observational data, a mix of observational and interventional data, and active learning, but their algorithm outputs were restricted to MEC. Also, those works (Glymour et al. 2019
Vowels et al. 2022) reviewed the advancements in traditional causal discovery (constraint- based, score-based, and functional causal model-based and differentiable causal discovery, respectively, but lacked experimental support. The recent work (Ng et al. (0224) conducted an experimental assessment of the advancements in differentiable causal discovery, illuminating the shorteomings of current methods. However, their evaluation overlooked the ubiquitous violation of model assumptions that characterize real-world applications. Conversely, another work (Montagna et al., 2023) benchmarked the efficacy of traditional causal discovery algorithms, encompassing score-mater matching techniques, under scenarios where model assumptions were violated. Nevertheless, their analysis did not encompass the recent strides made in differentiable causal discovery, and the misspecified conditions they evaluated were constrained in scope. Given that the application of causal discovery methods to real data inevitably entails the violation of one more unidentified assumptions, and that algorithms premised on specific assumptions may falter in practical use, the robustness of causal discovery in such misspecified contexts assumes paramount importance.
Our study undertakes an exhaustive empirical evaluation of both established and cutting-edge causal discovery methodologies, comprehensively examining their performance under diverse scenarios with assumption violations. The misspecified scenarios encompassed in our analysis represent the most extensive coverage in the current research landscape. We meticulously evaluate mainstream causal discovery approaches, spanning constraint-based, score-based, functional causal model-based, gradient-based methodologies, among others, ensuring a holistic view of the field. Notably, our work fills a crucial gap in the literature by being the first to assess the performance of gradient-based methods across a wide array of misspecified scenarios. Considering their practical implementation potential, it is important to evaluate their performance. Our contributions can be summarized as follows:
We conduct extensive large-scale experimental evaluations of twelve prominent causal discovery algorithms across eight pivotal model assumption violation scenarios. Our rigorous research endeavor involves executing over 70,000 experiments on more than 2,400 synthetic datasets, ensuring comprehensive assessment of the algorithm capabilities.
We delve into challenging scenarios such as heterogeneity, scale variation, missing data, and mechanism violation, thereby enriching the benchmark data landscape for model assumption violations. This also aims to foster more comprehensive benchmark testing and foster a more rational evaluation framework for future causal discovery algorithms.
Our analysis of the experimental outcomes offers theoretical insights into the performance of linear differentiable causal discovery methods under certain misspecified scenarios. Recognizing the robustness demonstrated by differentiable methods in these commonly used challenging settings,
2

---

Published as a conference paper at ICLR 2025
we underscore the significant value of further in-depth research into differentiable causal discovery, as holds the promise of advancing the field in novel and impactful directions.
BACKGROUND
In this section, introduce the definition of causal discovery (Section 2.1 functional causal model-based (Section 2.2). score-based (Section 2.3p and gradient-based (Section 2.3 methods. For
constraint-basse based methods, see Appendix A.2 and B.1
2.1 TASK FORMULATION
A structural causal model M Pearl 2009)
consists of the set of endogenous variables X (X....., Xa) Rd, exogenous variables (UI..... Ud) R and functional mechanisms
F= (fi...., fa). Each variable X, is defined by structural equation:
X; = fi(X U.), Vi 1, , d,
where X; is the i-th node variable. (x,), denote the parents of X; fi IR X + IR is the causal structure function, and covariance matrix Ω = cov(U) = diag(o) (.. .3).
..., Ud) are jointly independent noise variables with
The task of causal discovery is infer a DAG that describes the causal relationships among variables from independent and identically distributed (i.i.d.) observational data X E RRnxd, which are drawn from the joint probability distribution P(X).
2.2 STRUCTURE IDENTIFIABILITY
To uniquely identify DAG G from purely observational data X sampled from P(X), we need to make assumptions about the SEM ( Considering set of assumptions A on a structural causal model (SCM) MA (P(X), G). the graph G is identifiable from P(X) if there is other SCM MA (P(X). satisfying the same A such that & G and P(X) = P(X). Existing identifiable causal models include: linear non-Gaussian acyclic models (Shimizu et al. (0066). linear Gaussian models with equal noise variances (Peters & Bühlmann, 2014), post-nonlinear models (Zhang &
Hyvarinen 2012) and nonlinear additive noise models (Hoyer et al. 2008 Peters et al. 2014).
2.3 DIFFERENTIABLE SCORE- BASSED CAUSAL DISCOVERY
Traditional score-based causal discovery defines a combinatorial optimization problem: min FG;: X) Lree (G; X) AC (9) s.t. DAG,
(2)
where F' is a score function, Lice (9; X) represents the goodness-of-fit between the estimated DAG g and the true DAG, Lyparse (9) denotes the sparsity regularization term and X is hyperparameter that controls the strength of regularization.
As the number of nodes rises, the total count of possible DAGs expands super-exponentially (Robin)
son 1973). Consequently, most conventional score-based approaches utilize local heuristic search techniques, including greedy search (Chickering) 2002) Hauser & Bühlmann 2012) and hill-
climbing (Gámez et al. 2011 Tsamardinos et al. 2006).
In addition search strategies, the design of score functions is also crucial. Commonly, score functions are classified into two categories: Bayesian-based scores and information-ttheoreiic scores. Bayesian-based scores emphasize goodness-of-fit and enable the integration of prior knowledge, such as the Bayesian Dirichlet equivalent (Heckerman et al. 1995) and the K2 score (Kayaalp & Cooper scores, on the other hand, account for both model goodness-of-fi and complexity, including the Bayesian information criterion (Neath & Cavanaugh 2012) and the Akaike information criterion (Akaike) 1998).
2012). information-theoree
To overcome the challenges of combinatorial optimization, NOTEARS (Zheng et al. 2018 formulates
the DAG structure learning task as:
min F
s.t. h(W(G)) = 0,
9
3

---

Published as a conference paper at ICLR 2025
where (9) E Rd is weighted adjacency matrix, d is the number of nodes, hG)(G) 0 differentiable equality DAG constraint.
h(W(G)) = 0 if and only if (G) is DAG. Commonly used DAG constraints include h(W ( Tr(e W WoW) - (Zheng et al. 2018). h (G) = Tr[ + aW W (a 0) Yu ct al. 2019)
and h (W) log det(ss W W) dlog (s 0) (Bello et al. 2022). Furthermore,
can transform the equality-constrainen optimization into unconstrained optimization (4) using the
ALM (Nemirovsky }
min F(g:X) + ogh( + 1t (M(W()))
(4)
where og and ut are the Lagrange multiplier and penalty parameter at the f-th iteration, respectively.
EXPERIMENTAL. DESIGN
In this section, we introduce the generation of synthetic datasets with violated model assumptions, the tested causal discovery algorithms, the algorithm hyperparameters, and the evaluation metrics.
3.1 SYNTIETIC DATASETS
Many prevalent causal discovery approaches hinge upon unverifiable assumptions. Our study pri- marily scrutinizes the efficacy of these methodologies in cireumstances where their underlying assumptions are breached. To achieve this, we commence by elucidating the baseline model under both linear and nonlinear frameworks, subsequently delving into scenarios where these fundamental assumptions fail to hold.
Table 1: Summary of the algorithm assumptions and their corresponding output graph types. The content within the cells indicates whether algorithm supports (V) or does support (X) the specific condition in the corresponding row. The table style is adjusted from Montagna et al. 2023
DirsaLiNGAM
1et F-SootRRereess RorrReen
NOTEARS
NOTEARS
GaN-DAG
PC
GES
CAM
GOLEM
NoCuri
DAGMA
Gaussian
Non-Gaussian noise
X
Linear mechanisms
Nonlincar mechanisms
/
X
Unfaithful distrihution
X
Confounding effects
Measurement errors
X
Autoregressive effects
X
X
Heterogeneous effects
X
Scale-variant effects
X
X
Missing mechanisms
X
X
X
X
Output
CPDAG
CPDAG
DAG
DAG
DAG
DAG
DAG
DAG
DAG
DAG
DAG
DAG
Linear vanilla model. In linear SCM, following the settings of Zheng et al. (2018), coefficients are sampled from U--2, -055) U (0.5, 2) with additive standard Gaussian noise. refer to this model as the linear vanilla model, which satisfies both identifiability and the assumptions of most linear benchmark methods (see Table [
Nonlinear vanilla model. In nonlinear settings, following the settings of Zheng et al. (222)), the SEM in equation (I) is generated under the Gaussian process with radial basis function kernel of bandwidth one, where fi is additive noise models with U; as standard Gaussian noise. We refer to this model as the nonlinear vanilla model, which satisfies both identifiability and the assumptions of nonlinear benchmark methods (see Table
To eliminate the impact of Gaussian noise in the vanilla model on experimental results, we also consider cases where the vanilla model uses non-Gaussian noise (see Appendix G).
4

---

Published as a conference paper at ICLR 2025
3.1.1 MODEL ASSUMPTION VIOLATION SCENARIOS
Four scenarios of model assumption violations are defined below. The other four cases, i.e., con- founded, measurement error, unfaithful and autoregressive model, follow the same settings as Mon lagna al, (202)). These eight misspecified scenarios can applied both the linear vanilla and nonlinear vanilla model to generate datasets.
Heterogeneous model. Existing causal discovery algorithms typically rely on the assumption of i.i.d. data. However, real data often exhibit distribution shifts (Huang et al., 2020h. The heterogeneous multi-domain data considered in this paper primarily refers to scenarios where the underlying causal generation process remains unchanged, but the distribution of noise terms varies (Huang et al. 2020 Zhang et al., 2023) Wang et al., 2022). Specifically, we consider data from two domains é and
62. The proportion of data from is Pi E {0.1, 0.3, 0.5, 0.9, and the proportion from eg is 1 1 P1- The noise variance in é1 is set the same as the vanilla model, while variance in eg is set to y E {0.01, 0.05, 0.53.
Scale-variant model. Reisach et al. (222 observed a significant performance decline in linear gradient-based methods, such as NOTEARS (Zheng et al. 2018) and GOLEM (Ng et al., 2020), when applied to data with scale variation. However, there has been notable absence of research investigating the performance of nonlinear methods in the context of scale variation. Therefore, we also consider scale variation as misspecified scenario. The structural equations considered are:
X;
X,
U4
Vi 1i=1,...,44
Var (X.)
where u, and Var(X,) are the mean and variance of X,, respectively. The input data are standardized, while the ground truth graph remains consistent with the causal graph that generates the original data.
Missing model. Missing data is a prevalent challenge in real-world datasets, necessitating that causal discovery algorithms effectively address this issue (Tu et al., 20196). In our study, we adopt the Missing Completely At Random (MCAR) mechanism (Tu et al. 2019a), where the occurrence of missing values follows a Bernoulli distribution with a missingness probability of e {0.005, 0.01, 0.05, 0.13. Given that the algorithms under consideration are incapable of directly processing datasets with missing values, we eliminate any records containing such gaps. To mitigate the influence of data quantity on the experimental outcomes, we ensure the volume of data remains consistent before and after the removal of incomplete records.
Mechanism violation. Most current causal discovery algorithms presuppose either linear nonlinear mechanism, especially methods based on functional causal models (Shimizu el al. 2006; Peters & Bühlmann| [0144] /hhang& Hyvarinen 2012) Hoyer et al.) (2008] |Pteers et al. 22014). These methods necessitate specific assumptions about the SEM mechanism to guarantee identinability. Given that the SEM in real-world data typically unknown, the robustness of algorithms
the face of mechanism violation becomes critically important. In mechanism violation, the input data for algorithms designed for lincar SEM will adhere to the nonlinear vanilla model, whereas the input data for algorithms tailored to nonlinear SEM will conform the linear vanilla model.
3.1.2 DATA GENERATION
Following the data generation of Zheng et al. 2018 2020) and Liu et al. (2023). different datasets generated for both linear and nonlinear vanilla model. We simulate ER and SF graphs based on the number of nodes E {10, 20, 50), average degree of nodes & E {2, 4]. In addition, we consider scenarios with Gaussian Random Partitions (GRP) (Brandes al.. 2003) graph and average node degree of 6. For each experimental configuration and scenario, 10 datasets of 2000 samples are generated. The mean and standard deviations of the evaluation metries (Section 3.4) is reported to ensure a fair comparison.
3.2 METHOSS
We select 12 mainstream causal discovery algorithms, including masttrint-basee score- baoeeb functional causal model--aaeed gradient- and other representative methods. For a more detailed introduction to the various methods, see the Appendix
5

---

Published as a conference paper at ICLR 2025
3.3 HYPERPARAMETER SETTINGS
PC (Spirtes Glymour 1991), CAM (Bühlmann et al. 2014). and GraN-DAG require adjustment of the significance level in the statistical independence tests. NOTEARS, GOLEM, NOTEARS- MLP, and DAGMA need adjustment of the sparsity coefficient X for the 4-norm regularization term. Typically, the ground truth of real data is unknown, making it difficult to effectively select hyperparameters for various algorithms. Thus, to ensure a fair comparison various methods, we tune λ in {0.005, 0.01, 0.05, 0.5, 2, 5) and tune in {0.001, 0.01, 0.05, 0.1).
3.4 EVALUATION METRICS
We employ Structural Hamming Distance (SHD) and Structural Intervention Distance (SID) (Peters Bühlmann, 2015) to evaluate performance. SHD counts the number of edge insertions, deletions, and reversals necessary to transform the estimated graph into the true graph. SID is used to assess the distinctions intervention distribution between the estimated and the true graph. Intuitively, SHD focuses differences in graph structure, while SID focuses differences in causal ordering. Lower SHD and SID values indicate better estimation of the target causal graph by the algorithm. For cases where the output is MEC, follow the same approach as Zheng et al. (2018), evaluating them favorably by assuming the undirected edges in the MEC are in the correct direction.
CRITICAL EXPERIMENTAL RESULTS AND INSIGHTS
In this section, we first present the experimental results of the misspecified datasets generated accord- ing to Section 3.1.I comparing them with the findings from the vanilla scenario to draw conclusions. Finally, we provide a more in-depth discussion on the performance of CAM (Section 4.1.1) and offer theoretical insights into the performance of differentiable causal discovery (Section 4.1.2). Due to space limitations, the main text focuses on the experimental results for ER-2 graphs of 10 nodes (Table 2.1 2.2 3.1 3.2 4.1 4.2). whereas similar conclusions apply to different nodes, graph
types and graph densities (see Appendix B and (T) To visually and concisely present the results, the outcomes of the 10 nodes graph under linear, nonlinear, and MLP settings (Section 4.1.1
summarized in Figure ! We also consider the real-world Sachs (Sachs et al. 2005) dataset (sce Appendix [). combined misspecified scenarios (see Appendix FF), vanilla model with non-Gaussian noise (see Appendix[Gi. semi-synthetic data (see Appendix [ and runtime of the benchmark methods (see Appendix DD. For each scenario, we generate datasets with 10 different random seeds, each time drawing 2000 samples. We report the mean and standard deviations of the metrics over 10 trials. To guarantee fair comparison of various methods, the hyperparameters for each method determined as the optimal values relative to the specific dataset.
4.1 CURRENT METHODS' PERFORMANCE IN MISSPECIFIED SCENARIOS
Our experiments show that differentiable causal discovery algorithms almost always achieve optimal or competitive performance commonly used misspecified scenarios other than scale variation. In this paper, robustness refers to the ability of the model to perform well in misspecified scenarios, consistent with the understanding of Montagna et al., (2023).
Confounded, measurement error, autoregressive and heterogeneous model. Table 2.1 2.2
and 3 indicate that under the commonly used confounded. measurement error, autoregressive and heterogeneous (Pi = 0.5, 7 0.1) scenarios, differentiable causal discovery achieves optimal or competitive performance compared to other methods. For the nonlinear Gaussian process mechanism, although CAM (Bühlmann et al. 2014] demonstrates better performance, the discussion in Section 4.1.I reveals that CAM still has limitations compared to differentiable causal discovery.
Missing model. We generate missing data that are MCAR with the missingness probability 0.01. Table 2.1 2.2 3.1 and 3.2 indicate that under MCAR, the result various algorithms is close to that in the vanilla model. Our experiments show that the performance of differentiable causal discovery under missing data is consistent with traditional methods considered by Tu et al. (2019b/a), a), including PC and GES.

---

Published as a conference paper at ICLR 2025
Varilla
Vanila
Mechanism violation
Confounders
Mechanism violetion
Confounders
Missing
Measurementeno Missing
Messurement emor
Scale-vrrant
Autoregressive
Scale-varint
Autoregressive
Wifrrrndd e-based Random
Random
Unfaltnful
Hoterogenoous
DIRALINGAM Va-Kortillegins
Unfathriu
Heterogeneous
GES
DNRCILINGAM
SHD with linear ER-2 graphs of 10 nodes
SID with lincar ER-2 graphs of 10 nodes
Vanilb
Vanila
Mechanism viatation
Contounders
Mochanism vislation
Confounders
Missing
Measurement Missing
Measurement
Scalevariant
Autoregressive
Scale-variant
Autoregressive
Random
Random
PC
PC
Unfolthfll
Heterogeneous
GES
Heterogeneous
GES
(c) SHD with nonlinear ER-2 graphs of 10 nodes
(d) SID with nonlinear ER-2 graphs of 10 nodes
Vanilla
Mechanism viclation
Confounders
Mechanism vtotation
Confounders
Missing
Measurement error Missing
Measurement enror
Scale-verlant
Autoregressive
Scaleverrint
Autoregressive
Deferentable based
Rerentable
CAM
CAM
Unfalthll
Heterogeneous
Unfaithful
Heterogeneous
(c) SHD with MIPP ER-2 graphs of 10 nodes
(f) SID with MIP ER-2 graphs of 10 nodes
Figure I: Experimental results under the lincar, nonlincar, and mlp settings for both the vanilla scenario and the eight misspecified scenarios. SHD (the lower better) SID (the lower the better) are evaluated over 10 trials on the 10 nodes ER-2 graphs. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure Inland Figure Idlare more favorable to CAM,, we conduct a more reasonable evaluation of CAM and differentiable causal discovery under the MI.P setting (Section 4.1.l)
Mechanism violation. For mechanism violation the linear setting of Table 2.1 and 2.2 despite PC and GES's ability to handle nonlinear mechanisms, we are surprised to observe that linear differeeniaale causal discovery algorithms achieve competitive performance. In the nonlinear setting of Table 3.1 and 3.2] we discover that nonlinear differentiable causal discovery algorithms, such as NOTEARS-ME MLPPRS-MMPP (Zheng et al. 2020). GraN-DAG (Lachapelle et al. 2019), and DAGMA (Bello al. 2022), outperform other types of algorithms. From Table 3.1 and 3.2 also that CAM does not perform well under mechanism violation, although excels in other scenarios. We speculate that this is because the Gaussian process mechanism used in the nonlinear vanilla model aligns well

---

Published as a conference paper at ICLR 2025
with CAM's assumptions about SEM. For more detailed discussion CAM performance, see
Section 4.1.1
Table 2.1: Linear Setting, for ER-2 graphs of 10 nodes (Part D).
Vanilla model
Latent confounders SHD
Measurement error STD
Autoregressive
10 nodes
SHD
STD
STD
SHD
SHD
SID,
Random
25.6#3.1
57.919.5
27.9-2.3
67.817.8
25.9:3.5
60.4111.3
27.943.2
62.018.1
PC
12,4=3.1
40.9=13.4
18.1=4.
58.1=15.6
19,4#4.1
18.0=13.1
14.5+2.00
44,8=9.5
GES
13.827.8
32.0=13.6
25.9-77
42.6214.0
20.2:4.8
16.2
20.8:55.5
49.7
DirM
19.6#3.3
16.1
20.4=5.0
42.0:6.0
17.6+224
48.8+12.4
19.7+442
50.418.4
Var- Regenns
11.2+3.55
8.418.55
17.6=58
12.649.9
19.6:28
11.4:87
18.8:224
16.5:110.6
R RerrnRe
20.2:44.8
32.4=14.0
25.7=4.1
37.6=13.0
25.6=660
39.2416.00
25.6:1499
38.8+19.00
NOTEARS
1.54+.6
1.844.2
8,5439
9.548.1
12.5+200
19.6+886
12.2443.6
27.5+14.2
GOLEM
1.4144
0,41:12
6.7:2.88
14.24998
17.8:12.5
43.1111.3
16.61440
34,911699
NoCurl
2.0
5.145588
9.14442
5,4::3.9
1188:18
17,918.4
14.8:22.5
17.5
DAGMA
1.2-122
3,345.3
8,443.9
8,8+7.7
12.642.5
18.548.6
12.2:13.6
28.4+15..
Table 2.2: Linear Setting, for ER-2 graphs of 10 nodes (Part II).
Heterogeneous SHD,
Unfaithfll SHDL
Scale-variant SHD]
Missing
Mechanism violation
10 nodes
SIDJ
SID
SID
SHD
SID
SHDL
SIDJ
Random
26.3+335
57.7+7.6
26.1+3.7
60.7+1299
26.7+3.0
64.0=8.7
27,5+3.
63.1+6.0
28.3+3.00
63.6478
PC
13.8426
47.5=10.3
13.9
40.6:10.4
12.4
40.9=13.4
13.0a4.3
44.8+:6.0
17.1425
64.9=100
GES
15,5+6611
15.2+1222
17.8-6.4
39.0+15.7
1388+78
32.0-1336
10.1+52
25.4+12.6
16.2+2.2
57.8+10.7
DirecLiNGAM
34.729.
44.7+139
62.6299
49.8=11.1
63.5+7.7
Var-SottnRegress
17.9133
8.619.3
12.412.1
13.518.0
30.715.11
$4.51100
7,313.5
9.318.44
15.61333
39.016.7
R Re-SorRRerress
26.0.154
37.01144
29.8148
51.0111.
20.2.14.8
32.411400
20.5.6.7
32.018.8
20.3133.7
66.119.7
NOTEARS
5.5-2.7
5.415.11
2.7.3.11
3.145.1
18.0
60.5:733
2.3
6.428.5
19.0:0.9
58.318.0
GOLEM
6.5.1.5
9.818.1
2.1122
0.6.18
17.511.2
64,4168
1.7117
6.2.10088
18.6.1.6
52.7.14.3
5.5+57
2.24233
2.0244
27.245.1
09.9=79
3.1+32
4.7:58
19.1#1.0
58.9:995
DAGMA
5.5-13
12.01822
21122
0.6.118
17,9114
58.7.66
1.5114
4,5.7.11
19.0100.9
58.3.18.0
Scale-variant model. In Table 2.1 and 2.2 we observe that the results of linear differentiable causal discovery algorithms, such as NOTEARS (Zheng et al., 2018). GOLEM (Ng et al.) 2020). NoCurl (uu et al. 202T), and DAGMA, significantly decline under scale-variant data, performing worse than PC and GES, which consistent with the observations nonlinear differentiable methods, performance under scale-variant data has not been explored in previous research. Table 3.1 3.2 and 4.2 indicate that nonlinear differentiable methods also show performance degradation under scale variation scenarios, and their results almost always lower than CAM. However, unlike the linear scenarios, the result of nonlinear differentiable algorithms is almost always superior to PC and GES.
Reisach et al.] (2021). For
and aand2 Unfaithful model. In the linear setting of Table and 2.2
we see a significant performance drop for Rer-Ssss (Reisach al. 2021) and R SertSortRegrress (Reisach et al. 2023) under unfaithful distributions. The explanation for this is that for each triplet X; + Xj + Xx + X; in the graph, after the causal direct effect of X, + X₁ cancels out, the variance of node X₁ changes significantly. This reduces the har-Sortability, further leading to a performance decline in the two SortnRegress algorithms and linear differentiable methods. In the nonlinear setting of Table 3.1 3.2] the SHD of various algorithms generally decline to some extent under unfaithful path cancellations. This is consistent with the experimental results of Montagna et al. (2023), which indicate that cancellation of causal effects in unfaithful nonlinear scenarios makes structural inference of sparse graphs casier.
4.1.1 DISCUSSION ON CAM PERFORMANCE
Motivations. The nonlinear vanilla model adopts a Gaussian process that is consistent with the assumptions of the CAM. To provide a fair benchmark for CAM, we consider the nonlinear vanilla model following different functional mechanism. We compare CAM with the representative differen- tiable causal discovery method: NOTEARS-MLP
We simulate ER-2 graphs based on the number of nodes d E {10, 50). Following the data generation mechanisms of Zheng al. (2020), consider ft in nonlinear vanilla model (Section 3.I is modified to be parameterized by a neural network with one hidden layer size 100.
Simulations.
Results. From Table and under almost all model assumption violations. Considering that the functional mechanisms of data in scenarios are usually unknown, believe that differentiable causal discovery has
observe that NOTEARS-MLP achieves better performance
8

---

Published as a conference paper at ICLR 2025
significant advantage CAM in all types of assumption violation scenarios except for scale variation.
Table 3.1: Nonlinear Setting, for ER-2 graphs of 10 nodes (Part D).
Vanilla model
Latent confounders SHID,
Measurement error
Autoregressive
10 nodes
STID
SID,
SID
SHDD
SIDI
SIID
SID
Random
27.74332
63.6+11.2
27.4+2..5
59.3-9.4
37.4+3.6
63.247.7
28.5+2.3
65.8+8.6
PC
17.142.5
64.91100
18.5+19
62.8=9.1
18.2=1.1
61.6412.2
18.4+1.
61.249.77
GES
16,2422
57,8+10.7
17.1
55.2+15.5
17.0=1.0
52.8+9.1
17.2+220
54,94788
CAM
4.7+119
16.3=9,5
10,4422.8
28.2=5,3
13,9=1,9
48.2+7.4
12.5=33.0
33,9:16.3
NOTEARS-MI.
12.4+22.2
36.3+7.11
17.0++17
49.2-8.6
16,5=
48.9+44.8
17.0+3.77
47.74+1..0
GraN-DAG
12.7422
33.2:110.
15.6+12.1
42.4=888
20.0=1.1
13.8::11.
16.2:22.3
44.2:110.0
DAGMA
13.5+2.0
40.7=8.1
18.6+2.00
62.0+13.3
17.3=1.
54.948.3
19.0+2.0
16,6+10.55
Table 3.2: Nonlinear Setting, for ER-2 graphs of 10 nodes (Part II).
Heterogeneous
Unfaithful SHD1
Scale-variant SHD]
Missing SHD
Mechanism violation
nodes
SHDI
SID
SIDI
SIDJ
SID
SHDD
SIDI
Random
24.913.2
62.1
28.742.1
69.8279
27.5+2.
69.9
27.01:22
68.118.0
28.3
64.5
PC
21.31322
56.5110.4
15.41144
57.3.955
17,1125
64.911000
17.81311
63.5110.5
12.413.1
40.91134
GES
19.8+22
55.7+8.4
15.0445
50.2=13.1
16,2422
$7.8=
16.6=226
56.249.9
13.8
32,0=136
21.71128
24.71130
6.51211
15.417.3
178114
45.911700
NOTEARS MLP
16.4337
42.6210.6
11.442.1
43.8-292
16.1=25
48.3+9.7
12.3+21
33.647.00
5.9425
19.7+8.7
GraN-DAG
1481111
40.5:1008
11.243.1
37.7110.0
10.0
26.1::12.
10.4134
25.718.1
16.312.1
54,715.1
DAGMA
16414.3
36.611577
8.6.3.1
29.6.1488
15.712.
53.31129
13.7127
41.417.8
3.313.1
9,51116
4.1.2 THEORY ON DIFFERENTIABLE CAUSAL DISCOVERY IN MISSPECIFIED SCENARIOS
We analyze the performance of linear differentiable causal discovery under measurement error, unfaithful and missing scenarios by introducing the theories (Theorem 7 and Theorem 9) from Loh & Bühlmann (2014). Theorem 7 in Loh & Bühlmann 201 4) states that for a linear model with equal noise variance, minimizing the least squares score will return the true DAG in the large sample limit. For linear model with non-equal noise variances, we define the noise ratio
max maxx.of....... 2
r min (o....... .
Theorem in Loh & Bühlmann 2014) states that if < + š, where & > 0 is the gap between the score of the true DAG and the next best DAG, minimizing the least squares score will return the true DAG in the large sample limit.
Measurement error. In measurement error model considered by Montagma et al. (2023) the observed variables are:
X, = X; + 6i Vi 1,...,d, where X; = (x(m)())))+ + U. fi a linear mechanism, U, ~ NO, 1), G, ~ NNO, 5* Var(X,)) with å E [0.2, 0.4,0.6,00 0.83. the vanilla model, r = However, the measurement error model, the noise ratio becomes
max(1 + 8 Var (1) K.)...... 1 8 Var(Xa)) min + 8 Var(X,),
(8)
+ Var(X.))
Due to the increasing trend of marginal variances of nodes along the causal direction (Reisach et al., 2021), we infer that F > = 1. In this scenario, there is no guarantee that F < 1 + & and linear differentiable causal discovery based on least squares cannot guarantce obtaining the true DAG, which can explain their performance decline in Table 2. and 2.2
Unfaithful model. In unfaithful model considered by Montagna et al. 2023), for each triplet X; + X, X2 + X, in the graph, the causal mechanisms adjusted such that the direct effect of X, on X cancels out. To illustrate the change in noise ratio after path cancellation, we consider a DAG g
with variable set V(G) (X1, X3) and edge set E(g) (X1 X2, X2 - Xa. X1 X3). The structural equations is defined as:
X, U1,
X₂ fi(Xi) + U2,
(9)
X3 f((1)) - X2 + Usi
9

---

Published as a conference paper at ICLR 2025
where fi is a linear mechanism. After the direct causal effect of X1 X3 cancels out, the noise term of Xg is U3- - U2 with the distribution of NO,,2) 2). Similarly, in the unfaithful datasets with nodes d E {10, 20, 50} considered by our experiments, for each triplet X; + X, + Xx t X; in the graph, once unfaithful path cancellation occurs, the noise term of X is Ux - Uj with the distribution of N (0, 2). In this case, the noise ratio becomes 2 > r (vanilla model). Due
to the increasing of the noise ratio, there is no guarantee that < 1 + & and linear differentiable causal discovery based on least squares cannot guarantee obtaining the true DAG, which can also explain their performance decline in Table 2.1 and 2.2
Missing model. Under the MCAR case, we deleted the rows with missing values and regenerated the data under the i.i.d. assumption to ensure the unchanged sample size. The noise ratio r = 1 remains constant before and after data imputation. This explains the superior performance of differentiable
causal discovery methods observed in Table 2.1 and 2.2
Table 4.1: MLP Setting, for ER-2 graphs of 10 nodes (Part I).
Vanilla model
Latent confounders SHD,
Measurement ertor
Autoregressive
10 nodes
SHD1
SID,
SIDI
SHD SIDI
SHD
SID,
CAM
12.4:3.6
39.31:16.5
16.61-422 11.7+5.5
42.4 33.3+17.0
19.7=4.7 18,5=377
59.6#12.0
16.043.4
16.0:116.6
NOTEARS-MLP
8.1+2.7
22.2++10.6
47.1
15.7+4.3
39.8=9.6
Table 4.2: MLP Setting, for ER-2 graphs of 10 nodes (Part II).
Heterogenoous SHDI
Unfaiitdul
Scalo-variant SHDI
Missing
Mechanism violation SHDD
nodes
SID,
SHDI
SIDI
SIDJ
SHD,
SIDJ
SIDI
CAM NOTEARS-MLP
49.1+7.2 14.5122
19.9+166
1300:32 14.8139
36.0=14.0
12.4-36 18.212
39.3=1665
13.2436
44.7+16.7
17.8+4.4 5.912.5
45.9=170
34.3110.4
32.31128
61.01110
90133
22.011000
19.7.18.7
4.2 SUMMARY AND IMPLICATIONS FOR PRACTICE
In Appendix H we summarize the results of the most competitive methods under misspecified scenarios. Differentiable causal discovery methods demonstrate optimal or competitive performance in commonly used scenarios other than scale variation. Notably, the recent work by Deng et al. (2024) shows that for linear diffcrentiable methods, scale invariance can be achieved by appropriately choosing the loss function. This further reinforces our conclusion regarding the robustness of differentiable methods. In our benchmarks, the results in Table [1 Table 21 and Table 25 indicate that the performance of nonlinear differentiable methods under scale variation remains challenging and warrants further investigation. In practice, the misspecified scenarios are inevitably encountered, making the robustness of algorithms critically important. Based on the summarized results on eight misspecified synthetic datasets (see Appendix H.. runtime results of benchmark methods (see Appendix D). real-world (see Appendix [ and semi-synthetic data (see Appendix D results, we observe that differentiable causal discovery methods have the potential to achieve optimal or competitive performance on real-world data with an almost negligible time cost. The fast and robust characteristies of differentiable methods enable them to better address the challenges of applying causal discovery algorithms to real-world data, demonstrating their practical implementation potential.
5 CONCLUSION
This work assesses the efficacy of twelve preeminent causal discovery methods across eight scenarios involving violations of model assumptions. These methods encompass approaches grounded in independence constraints, scoring criteria, functional causal models, and differentiable causal discov- cry. Our experimental results show that differentiable causal discovery methods exhibit remarkable resilience in commonly used scenarios of model assumption violations, except for scale variation. is not our intention to assert that differentiable causal discovery will achieve optimal performance across all circumstances, rather, we aim to underscore its substantial potential within the benchmarks we have evaluated, thereby emphasizing the necessity for further exploration in this direction. In future work, causal discovery methods for more semi-synthetic data and real-world scenarios will be explored. Finally, our study confines itself to non- -emporl causal discovery algorithms. Equally eru- cial is the conduct of benchmark assessments for causal discovery in time series and event sequences under model assumption violations.
10

---

Published as a conference paper at ICLR 2025
6 ACKNOWLEDGMENTS
This work was supported in part by the National Key R&D Program of China under Grant 2022ZD0120004 in part by the Zhishan Youth Scholar Program, in part by the National Natu- ral Science Foundation of China under Grant 62233004, Grant 62273090, and Grant 62073076, and part by the Jiangsu Provincial Scientific Research Center of Applied Mathematics under Grant BK20233002.
REFERENCES
Hirotogu Akaike. Information theory and an extension of the maximum likelihood principle. In Selected papers of hirotugu akaike, pp. 199-213. Springer, 1998.
Kevin Bello, Bryon Aragam, and Pradeep Ravikumar. DAGMA: Learning DAGs via M-matrices and a log-determinant acyclicity characterization. In Advances Neural Information Processing Systems, volume 35, pp. 8226-8239, 2022.
Ulrik Brandes, Marco Gaertler, and Dorothea Wagner. Experiments on graph clustering algorithms. In European symposium on algorithms, pp. 568-579. Springer, 2003.
Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexan- dre Drouin. Differentiable causal discovery from interventional data. Advances in Neural Information Processing Systems, volume 33, pp. 21865-21877, 2020.
Peter Bühlmann, Jonas Peters, and Jan Ernest. CAM: Causal additive models, high-dimensional order search and penalized regression. The Annals of Statistics, 42(6):2526-2556 2014. doi: 10.1214/14-ASSD20
Ruichu Cai, Feng Xie, Clark Glymour, Zhifeng Hao, and Kun Zhang. Triad constraints for learning causal structure of latent variables. In Advances in Neural Information Processing Systems, volume 32, 2019.
Ruichu Cai, Jincheng Ye, Jie Qiao, Huiyuan Fu, and Zhifeng Hao. FOM: Fourth-order moment based causal direction identification on the heteroscedastic data. Neural Networks, 124:193-201, 2020.
Ruichu Cai, Zhiyi Huang, Wei Chen, Zhifeng Hao, and Kun Zhang. Causal discovery with latent confounders based on higher-order cumulants. In International Conference on Machine Learning, pp. 1380-3407, PMLR, 2023.
David Maxwell Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3NN0v)50075554 2002.
Haoyue Dai, Peter Spirtes, and Kun Zhang. Independence testing-based approach to causal discovery under measurement eπrоr and linear Gau-Gassian models. Advances in Neural Information Processing Systems, 35:27524-27538 2022.
Chang Deng. Kevin Bello, Pradeep Ravikumar, and Bryon Aragam. Likelihood-baan differentiable structure learning, arXiv preprint arXiv:2410.06163. 2024.
José A Gámez, Juan L Mateo, and José M Puerta. Learning Bayesian networks by hill climbing: Effi- cient methods based on progressive restriction of the neighborhood. Data Mining and Knowledge Discovery, 22: 108,168,811111111111111111111181 2011.
Erdun Gao, Junjia Chen, Li Shen, Tongliang Liu, Mingming Gong, and Howard Bondell. FedDAG: Federated DAG structure learning. arXiv preprint (rXiv:2112.03555 2021.
Erdun Gao, Ignavier Ng, Mingming Gong, Li Shen, Wei Huang, Tongliang Liu, Kun Zhang, and Howard Bondell. MissDAG: Causal discovery in the presence of missing data with continuous additive noise models. In Advances in Neural Information Processing Systems, volume 35, pp. 5024-5038, 2022.
11

---

Published as a conference paper at ICLR 2025
AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash, and Kun Zhang. Learning causal structures using regression invariance. In Advances in Neural Information Processing Systems, volume 30, 2017.
AmirEmad Ghassami, Negar Kiyavash, Biwei Huang, and Kun Zhang. Multi-domain causal structure learning linear systems. In Advances in Neural Information Processing Systems, volume 31, 2018.
Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graphical models. Frontiers in Genetics, 10:524, 2019.
Alain Hauser and Peter Bühlmann. Characterization and greedy learning of interventional Markov equivalence classes of directed acyclic graphs. Journal of Machine Learning Research, 13(1): 1409-24644 2012.
Yue He, Peng Cui, Zheyan Shen, Renzhe Xu, Furui Liu, and Yong Jiang. DARING: Differentiable the 27th ACM SIGKDD Conference
causal discovery with residual independence. In Proceedings the on Knowledge Discovery and Data Mining, pp. 596-605, 2021.
David Heckerman, Dan Geiger, and David M Chickering. Learning Bayesian networks: The combination of knowledge and statistical data. Machine learning, 20:197-243, 1995.
Christina Heinze-Deml, Marloes H Maathuis, and Nicolai Meinshausen. Causal structure learning. Annual Review of Statistics and Its Application. 5(1):371-391, 2018.
Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. In Advances in Neural Information Processing Systems, volume 21, 2008
Biwei Huang, Kun Zhang, Jiji Zhang, Joseph Ramsey, Ruben Sanchez-Romero, Clark Glymour, and Bernhard Schölkopf. Causal discovery from beterogeneouu//oosttaiinnnr data. Journal of Machine Learning Research, 21(89):1-553 2020.
Diviyan Kalainathan and Olivier Goudet. Causal discovery toolbox: Uncover causal relationships in Python. arXiv preprint arXiv:1903.02278: 2019.
Diviyan Kalainathan, Olivier Goudet, Isabelle Guyon, David Lopez-Paz, and Michèle Sebag. Struc- tural agnostic modeling: Adversarial learning of causal graphs. Journal of Machine Learning Research, 23(219):1 2022.
Mehmet Kayaalp and Gregory Cooper. A Bayesian network scoring metric that based on globally uniform parameter priors. arXiv preprint arXiv:1301.0577 2012.
Lingjing Kong, Biwci Huang, Feng Xie, Eric Xing, Yuejie Chi, and Kun Zhang, Identification of nonlinear latent hierarchical models. In Advances in Neural Information Processing Systems, volume 36, pp. 2010-2032, 2023.
Trent Kyono, Yao Zhang, and Mihaela van der Schaar. CASTLE: Regularization via auxiliary causal graph discovery. In Advances in Neural Information Processing Systems, volume 33, pp. 1501-1512, 2020.
Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, and Simon Lacoste-Julien. Gradient-based neural DAG learning. arXiv preprint arXiv:1906.022266 2019.
Fangfu Liu, Wenchang Ma, An Zhang, Xiang Wang. Yueqi Duan, and Tat-Seng Chua. Discovering dynamic causal space for DAG structure learning. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1429-1440, 2023.
Po-Ling Loh and Peter Bühlmann. High-dimensional learning of linear causal networks via inverse covariance estimation. Journal of Machine Learning Research, (5(1):3055-1105 2014.
Francesco Montagna, Atalanti Mastakouri, Elias Eulig, Nicoletta Noceti, Lorenzo Rosasco, Dominik Janzing, Bryon Aragam, and Francesco Locatello. Assumption violations in causal discovery and the robustness of score matching. In Advances in Neural Information Processing Systems, volume 36, 2023.
12

---

Published as a conference paper at ICLR 2025
Joris M Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bernhard Schölkopf. Dis- tinguishing cause from effect using observational data: Methods and benchmarks. Journal of Machine Learning Research, 17(32):1-102, 2016.
Joris M Mooij, Sara Magliacane, and Tom Claassen. Joint causal inference from multiple contexts. Journal of Machine Learning Research, 21(99):1-108, 2020.
Andrew A Neath and Joseph E Cavanaugh. The Bayesian information criterion: background, derivation, and applications. Wiley Interdisciplinary Reviews: Computational Statistics, 4(2): 199-203, 2012.
AS Nemirovsky. Optimization II. Numerical Methods for Nonlinear Continuous Optimization. 1999.
Ignavier Ng. Shengyu Zhu, Zhitang Chen, and Zhuangyan Fang, graph autoencoder approach to causal structure learning. arXiv preprint arXiv:1911.07420 2019.
Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and DAG constraints for learning linear DAGs. In Advances in Neural Information Processing Systems, volume 33, pp. 17943-17954, 2020.
Ignavier Ng. Biwei Huang, and Kun Zhang. Structure learning with continuous optimization: A sober look and beyond. In Causal Learning and Reasoning, pp. 71-105. PMLR, 2024.
Roxana Pamfil, Nisara Stiwattanaworachai, Shaan Desai, Philip Pilgerstorfer, Konstantinos Geor- gatzis, Paul Beaumont, and Bryon Aragam. DYNOTEARS: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics, pp. 1595-1605. PMLR, 2020.
Judea Pearl. Causality. Cambridge university press, 2009.
Judea Pearl and Dana Mackenzie. The Book of Why: The New Science of Cause and Effect. Basic books, 2018
Jonas Peters and Peter Bühlmann. Identifiability of Gaussian structural equation models with equal error variances. Biometrika, 101(1):219-22) 2014.
Jonas Peters and Peter Bühlmann. Structural intervention distance for evaluating causal graphs. Neural Computation, 27(3):771-799, 2015.
Jonas Peters, Joris M. Mooij, Dominik Janzing, and Bernhard Schölkopf. Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 15(58):2009-2000 2053, 2014. httt:://m/m http:////ww.rrr/ppppe/////////// http::///wwleettteeeeeeemmm
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press, 2017.
Alexander Reisach, Christof Seiler, and Sebastian Weichwald. Beware of the simulated DAG! causal discovery benchmarks may be easy game. In Advances in Neural Information Processing Systems, volume 34, pp. 27772-27784, 2021.
Alexander Reisach, Myriam Tami, Christof Seiler, Antoine Chambaz, and Sehastian Weichwald, A scale-invariant sorting criterion to find a causal order in additive noise models. In Advances in Neural Information Processing Systems, volume 36, 2023.
Robert W Robinson. Counting labeled acyclic digraphs. New directions in the theory of graphs, pp- 239-273, 1973.
Karen Sachs, Omar Perez, Dana Pe' er, Douglas A Lauffenburger, and Garry P Nolan. Causal protein- signaling networks derived from multiparameter single-cell data. Science, 308(5721) (15572))::23-52, 2005.
Saber Salehkaleybar, AmirEmad Ghassami, Negar Kiyavash, and Kun Zhang. Learning linear non-Gate Con-Gassiian causal models the presence latent variables. Journal of Machine Learning Research, 2123 2020.
13

---

Published as a conference paper at ICLR 2025
Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non- Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Shohei Shimizu, Takanori Inazumi, Yasuhiro Sogawa, Aapo Hyvarinen, Yoshinobu Kawahara, Takashi Washio, Patrik O Hoyer, Kenneth Bollen, and Patrik Hoyer. DirectLiNGAM: A direct method for learning linear Gaussaani structural equation model. Journal of Machine Learning Research, 12(Apr):1255-1248, 2011.
Karamjit Singh, Garima Gupta, Vartika Tewari, and Gautam Shroff. Comparative benchmarking of causal discovery techniques. arXiv preprint arXiv: 06200000066,... 2017.
Peter Spirtes. Introduction causal inference. Journal of Machine Learning Research, 1 11(5). 2010.
Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs. Social science computer review, 9(1::22-7,, 1991.
Peter Spirtes, Christopher Meek, and Thomas Richardson. Causal inference in the presence of latent variables and selection bias. In Proceedings the the Eleventh Conference on Uncertainty in Artificial
Intelligence, pp. 499-506, 1995.
Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction, and Search. MIT press, 2001.
Ioannis Tsamardinos, Laura E Brown, and Constantin F Aliferis. The max-min hill-climbing Bayesian network structure learning algorithm. Machine learning, 65:31-78, 2006.
Ruibo Tu, Cheng Zhang, Paul Ackermann, Karthika Mohan, Hedvig Kjellström, and Kun Zhang Causal discovery in the presence of missing data. In The 22nd International Conference Artificial Intelligence and Statistics, pp. 1762-1770. PMLR, 2019a.
Ruibo Tu, Kun Zhang, Bo Bertilson, Hedvig Kjellstrom, and Cheng Zhang. Neuropathic pain diagnosis simulator for causal discovery algorithm evaluation. In Advances Neural Information Processing Systems, volume 32, 2019b.
Matthew Vowels, Necati Cihan Cangoz, and Richard Bowden. D'ya like DAGs? a survey on structure learning and causal discovery. ACM Computing Surveys, 55(4):1-36, 2022.
Yu Wang, An Zhang, Xiang Wang, Yancheng Yuan, Xiangnan He, and Tat- Chua. Differentiable invariant causal discovery. arXiv preprint arXiv:2205.15633, 2022.
Dennis Wei, Tian Gao, and Yue Yu. Dags with no fears: closer look at continuous optimization for learning Bayesian networks. Advances in Neural Information Processing Systems, 33:3895-3906. 2020.
Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang. Generalized independent noise condition for estimating latent variable causal graphs. In Advances in Neural Information Processing Systems, volume 33, pp. 14891-14902, 2020.
Shuai Yang, Kui Yu, Fuyuan Cao, Lin Liu, Hao Wang, and Jiuyong Li. Learning causal representations for robust domain adaptation. IEEE Transactions Knowledge and Data Engineering, 35(3): 2750-2764, 2021.
Yue Yu, Jie Chen, Tian Gao, and Mo Yu. DAG-GNN: DAG structure learning with graph neural networks. In International Conference on Machine Learning, pp. 7154-7163. PMLR, 2019.
Yue Yu, Tian Gao, Naiyu Yin, and Qiang Ji. DAGS with curl: An efficient DAG structure learning approach. In International Conference on Machine Learning, pp. 12156-12166. PMLR, 2021.
An Zhang, Fangfu Liu, Wenchang Ma. Zhibo Cai, Xiang Wang, and Tat-Seng Chua. Boosting differentiable causal discovery via adaptive sample reweighting. arXiv preprint arX1v:2303.03117, 2023.
Keli Zhang, Shengyu Zhu, Marcus Kalander, Ignavier Ng, Junjian Ye, Zhitang Chen, and Lujia Pan. gCastle: A Python toolbox for causal discovery. arXiv preprint arXiv:2111.15155, 2021.
14

---

Published as a conference paper at ICLR 2025
Kun Zhang and Aapo Hyvarinen. On the identifiability of the post-nonlinear causal model. arXiv preprint arXiv: 2599, 2012.
Kun Zhang, Mingming Gong, Joseph Ramsey, K. Batmanghelich, Peter Spirtes, and Clark Gly- mour. Causal discovery with linear non-Gaussian models under measurement error: Struc- tural identifiability results. In Conference on Uncertainty in Artificial Intelligence, 2018. URL
https://apps semanticccooolla...... wrt/CorpusIi:: ID 54058644
Muhan Zhang, Shali Jiang, Zhicheng Cui, Roman Gamett, and Yixin Chen. D-VAE: variational
autoencoder for directed acyclic graphs. In Advances in Neural Information Processing Systems, volume 32, 2019.
Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. DAGs with NO TEARS: Continuous optimization for structure learning. In Advances in Neural Information Processing Systems, volume 31, 2018.
Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, and Eric Xing. Learning sparse nonparametric DAGs. International Conference on Artificial Intelligence and Statistics, pp. 1414-3425. PMLR, 2020.
Yujia Zheng, Biwei Huang. Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang. Causal-learn: Causal discovery Python. Journal of Machine Learning Research, 25(60):1-8, 2024.
Rong Zhu, Andreas Pfadler, Ziniu Wu, Yuxing Han, Xiaoke Yang. Feng Ye, Zhemping Qian, Jingren Zhou, and Bin Cui. Efficient and scalable structure learning for Bayesian networks: Algorithms and applications. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pp. 2613-2624. IEEE, 2021.
Shengyu Zhu, Ignavier Ng, and Zhitang Chen. Causal discovery with reinforcement learning. arXiv preprint arXiv:I906. 144,04779004,77,,.0 2019.
15

---

Published as a conference paper at ICLR 2025
CONTENTS
Introduction
Background
Task formulation
2.2
Structure identifiability
2.3
Differentiable score-based causal discovery
3
Experimental design
Synthetic datasets
3.1.1
Model assumption violation scenarios
3.1.2
Data generation
3.2
Methods
3.3
Hyperparameter settings
3.4
Evaluation metrics
Critical experimental results and insights
Current methods performance in misspecified scenarios
6
4.1.1
Discussion on CAM performance
4.1.2
Theory differentiable causal discovery in misspecified scenarios
Summary and implications for practice
10
5
Conclusion
10
Acknowledgments
11
Causal Assumptions
18
A.I
Causal Markov Property
18
A.2
Faithfulness
18
A.3
Causal Sufficiency
18
A.4
Independent and identically distributed
18
A.5
Equal Noise Variances
18
Benchmark methods
19
PC
19
B.2 GES
19
B.3
DirM
19
B.4 4
CAM
19
B.5
SortnRegress
19
B.6 NOTEARS
20
16

---

Published as a conference paper at ICLR 2025
B.7 GOLEM
20
B.8 NOT
B9 GraN-DAG
22
10 NOCURL
22
B111 DAGMA
23
Related work
23
D Table results for runtime of the benchmark methods
24
E Table results across nodes, graph types, and graph densities
25
Table results for combined misspecified scenarios)
28
Table results for non-Gaussian noise
28
H Summary of the most competitive methods
29
Table Results on real-world data
30
J Figure results across nodes, graph types, and graph densities
K Table results for extreme measurement error
38
Table results on semi-synthetic data
39
17

---

Published as a conference paper at ICLR 2025
A CAUSAL ASSUMPTIONS
In this section, we introduce the assumptions frequently used in the causal discovery literature.
A.1 CAUSAL MARKOV PROPERTY
The joint probability distribution P(X) satisfies the global Markov property (Peters et al. 2017 with respect to the DAG G if
XA g Xc XA 1T P(X) Xc,
(10)
where XA. XB and Xc are the disjoint subsets of = i, X,) E Rd, و. denotes separation in the causal graph G, and IL r(x) represents independence in the joint probability distribution P(X).
In the causal graph of SCM, each variable is independent of its non-descendant nodes when its parents are known, which is referred to as the local Markov property (Peters et al. 2017). Causal Markov property also implies that the joint probability distribution P(X) can factorized in the following form:
d II P P pa(X:))
P(X)
(11)
A.2 FAITHFULNESS
The joint probability distribution P(X) is faithful (Peters et al. 2017
DAG G if
X PB X = XA eX Xc-
(12)
Faithfulness assumption implies that the conditional independence in the P(X) be used infer the graph structure. Constraint-based and traditional score-based causal discovery are typically founded the faithfulness assumption. In causal graph, the cancellation of effects along multiple causal paths can lead to violation of faithfulness assumption.
A.3 CAUSAL SUFFICIENCY
Causal sufficiency assumption also referred as latent confounder. A of variables X said to satisfy the causal sufficiency if there is no unobserved common cause variable C' that influences more than one variable X (Spirtes) 2010). This assumption is also frequently considered in causal discovery literature. However, since we cannot always observe all variables in the real world, causal sufficiency assumption is inevitably violated.
A.4 INDEPENDENT AND IDENTICALLY DISTRIBUTED
Non-temporal causal discovery algorithms typically also require the i.i.d. assumption. In the main text, heterogeneous multi-domain data and autoregressive scenarios are two special cases where i..d. assumption is violated. Heterogeneous multi-domani data is closely related to the stationnyy time series data considered the literature on temporal causal discovery (Huang et al.) (2020). Below, we introduce the connection between them. We consider the distribution of X, changing with domain or time index, where the mechanism for the /-th data point as follows:
X fú. (pa(X,,)). eq,e)
(13)
where εj is the noise term of X In heterogeneous multi-doman data, / represents the domain index, whereas non-stationary time series data, denotes the time index.
A.5 EQUAL NOISE VARIANCES
Ng et al. (2024) observe that the performance of linear differentiable causal discovery methods significantly declines data with non-equal noise variances. They hypothesize that this may be due to the optimization problem becoming severely non-convex under non-equal noise variances. leading to local optimal solutions. Although differentiable methods do not explicitly assume equal noise variance, the performance decline suggests treating equal noise variance causal assumption.
18

---

Published as a conference paper at ICLR 2025
B BENCHMARK METHODS
B. PC
PC (Spirtes & Glymour 1991) algorithm is a representative constraint- causal discovery method. In the first step, under the faithfulness assumption. the global causal skeleton is determined based on conditional independence tests. In the second step, some edge directions in the skeleton are determined by identifying collider structures. Finally, the remaining edge directions are determined using orientation rules, resulting in the MEC. We use the implementation of the PC algorithm in causal-learn (Zheng al. 2024) python package, available
thub. ccomppy-wiiii
Sausal-leane
B.2 GES
GES (Chickering 2002) algorithm is classical score-based causal discovery method. GES mainly includes two stages. In the first stage, starting from an empty graph, edges are added through greedy equivalence search, and the structures the equivalence class the new graph are scored. The graph with the highest score is selected, and the edge-adding process is repeated until the score reaches a local maximum. stage, edges are removed through greedy equivalence search, and the structures in the equivalence class of the new graph are scored. The graph with the highest score is selected, and the edge- removal process is repeated until the score reaches local maximum. We use the implementation of the GES algorithm in causal-learn (Zheng et al. 2024) python package, available at https 7/github. com/ py why/ causal learn
the second stage, starting from the graph obtained in the first
B.3 DIRECTLINGAM
DirectLiNGAM (Shimizu et al. 2011) a classical linear method based on the functional causal model. To address the issues of slow convergence and large errors in the ICA-based LiNGAM (Shimizu et al., 2006) algorithm,Shimizu (Shimizu et al. (201 proposed DirectLiNGAN based the principle of residual independence. Although the solving speed became slower, the accuracy and convergence improved. We use the implementation of Direct LNGAM algorithm in gCas- lle (Zhang et al. 2021) python package, available at atthtps:: / com/ http:hhwwwwaanaaaeeeee t ErustwortyyAA/teeeee /tt.///reee/aettee/gaattt////
B.4 CAM
CAM (Bühlmann et al. 2014) is method used for nigh-dimensional additive structural equation models. CAM separates the search for the order of variables from the selection of edges. It performs the variable order search through nonregularized maximum likelihood estimation and sparse regression techniques for edge selection. We use the implementation of CAM algorithm in Causal Discovery Toolbox (Kalainathan & Goudet 2019) python package, available at htttpp
B.5 SORTNREGRESS
SortnRegress algorithm includes Var-SortnReeress SortnRegress RReisach et al. 2023).
(Reisach
al.
2021 and R²_
Reisach al. 2202 emphasized that in the bivariate linear case, causal direction inferred by minimizing mean squared error (MSE) loss is from the variable with smaller variance to the variable with larger variance. They further hypothesized that, the multivariate case, there is a consistency between the underlying causal direction of the data and the increasing order of the marginal variances of the variables. They provided general definition of sortability:
Vx(X9) Σi= E(--toe (x) ((X,)) where incr(a, b)
1
a < b
1/2
a b (14)
Σd 1 ((-))))A
0
a >
19

---

Published as a conference paper at ICLR 2025
represents function with 7(X) [0, matrix power, (s + t) E A(g)* if and only if at least one directed path of length from X, to Xt.
A(g) is the adjacency matrix of 9, A(g)' is the i-th
In Var-Sortabillty, s) denotes the variance of X,. Var-Sortability measures the consistency between the causal structure order and the increasing order of marginal variances of nodes. Intuitively, the greater Var-Sortabillity of the data, the better performance of methods based MSE loss. Subsequently, they proposed the War-Sort algorithm to discover causality using only variance. In the first step, the nodes are ranked according the increasing order of marginal variances. In the second step, linear and Lasso regression are used for estimation. We use the implementation of Sur-SortnRegress algorithm in CausalDisco python package provided by the authors, available at https:// //t::://tthtb
Reisach et al. (2021) demonstrated that the performance linear differentiable causal discovery algorithms is greatly affected by Var-Sortability. Building previous (eik, et al. (2023) pointed out that the coefficient of determination R² remains unchanged after scaling the data, and proposed the R22 Rernnneegsssss algorithm, which achieves better performance on scale-variant data. The definition of It::
Var X2 E Xe ( ((.0.l))01)))))
R² =
(15)
Var (X,)
In --Sortability, r(X, s) denotes the coeffieient of determination R of X,. R'-Sortability measures the consistency between the causal structure order and the increasing order of R2. If (f = 1, the causal order be fully identified by the increasing order of R². If v = 0, the causal order can be fully identified by the decreasing order of Ii2 The only difference between (e=-SortnRegress and Rer-ess lies in the definition of We use the implementation of R² Re-SortnRegress algorithm in CausalDisco python package provided by the authors, available at /ttps:://ithuu.comm// http:///hhw.oooooooacaaaaa/aaaaa
B.6 NOTEARS
Based on 3 the NOTEARS score function is:
1 X XW + XIIW( 2n
min F(G:X)
h(W(g))
(16)
where || ||r the Frobenius norm, || II is the sum of absolute values of all elements in the matrix.
The unconstrained objective function obtained through the ALM is:
min (W)), = -
2n X XW-XXGG)IIF (| +g)(((()))) 14 ((W()))) (17)
9 The update rule for the parameters is:
W W()),, 02 arg min (W, 0, W(G).9
0k+1 as + Hgh (W (G)))
(18)
Mk+1 nua, if Ih (W y 1/h (W())-11)
μis, otherwise
where 9 represents the parameters of neural network used to fit nonlinear function, and can be ignored for linear model. The hyperparameters are usually set as 17 10 and 7
In practice, the optimization stopping criterion is h (W())) < E {le-6, 1e- le 1e10), which does not guarantee the output be a DAG. Finally, for values in W()) with absolute values smaller than threshold we set them to 0 in order to obtain a DAG as closely as possible. We use the implementation of NOTEARS algorithm provided by authors, available https 7/qithub.c com httmm///xxnnnnnnnnttteaee
B.7 GOLEM
GOLEM Ng et al. 2020 proposed an improved loss function to address the numerical and ill- conditioned issues that often arise during the multiple iterations of optimization in NOTEARS. The
20

---

Published as a conference paper at ICLR 2025
unconstrained optimization problem formulated by GOLEM is:
W (G)) (G): X) = L9) X) + A \ + (2))
min
(19)
where A and A2 are hyperparameters, i {1,
When assuming linear Gaussian with non-equal variances, that is, GOLEM-NV:
n
(W X)
log
Σ
(x!
W(9), x()) - -log
/log/det(I det
W(G)),,
(20)
where x. denotes k-thh data point of X,.
When assuming linear Gaussian with equal variances, that is, GOLEM-EV:
d
L2(W(G);; X) = log ΣΣ (x!* (xx!* W(9)) [ x -log log det W(G))
(21)
The authors proved that in the case of linear Gaussian with equal variances, when the hard DAG constraint is not satisfied, the least-squares optimal solution of NOTEARS returns cyclic graph, whereas the optimal solution of GOLEM-EV corresponds to the ground-truthh GOLEM combines the maximum likelihood objective function with soft DAG constraint, replacing the least-squares objective function and hard DAG constraint, making the optimization casier to solve and the results better. We use the implementation of GOLEM-EV algorithm provided by the authors, available at https: /ttp::://tttttt thub. wwtp://aww//ennga...oo.oom em
B.8 NOTEARS-MLPP
NOTEARS-MLP (Zheng et 2020) extends the differentiable causal discovery framework the nonlinear case. Each variable X; is defined as follows:
X, (X U2), Vj =, 1,..... d.
(22)
The authors proved that fy is independent Xg if and only if [|@xffllz, = 0, where || 1|za is the L norm. Next, they define nonlinear causal effects through partial derivatives:
(W)))ss |susslle>
(23)
where [W))))k is the causal effects from X₂ to Xj. and W()) = (fi.... .a) Rddd In practice, neural networks are used to fitting nonlinear functional relationships fz: 1( (A (...
MLP, A0))
(... (2) (A())))
(24)
where A xm--1, is the activation function.
For the convenience of derivative computation, the authors proved that MLP, are independent of X if and only if the k-tth column of the first-layer weight matrix (1) is entirely zero. The parameters of MLP, are 8, [A!"...,,A) AAA)) The authors ultimately obtain a weighted adjacency matrix representation that is independent of the depth of the neural network:
[ (0)kss kth column (A.))L
(25)
The objective function of NOTEARS-MLPis
d
min Σ LX,, (X;0,) + + s.t. A(W(0)) = 0.
(26)
NOTEARS-MLE trains d neural networks and represents the acyclicity constraint only through the first-layer parameters of neural networks. We use the implementation of NOTEARS-MLP algorithm provided by the authors, available at hhtp:s:///gitubb....///uuuu/
21

---

Published as a conference paper at ICLR 2025
B.9 GRAN-DAG
GraN-DAG also extends NOTEARS to the nonlinear case and considers the ANM data generation
mechanism:
X, = [X px)(x))) Uj, = -,.... .d.
(27)
The authors state that the parameters of j-th neural network (NN) are:
ww G). (6) w((h"))
(
(28)
where w is the é-th weight matrix of the NN.
They define the j-th connection matrix:
Cu (w(++)) )
w
(2)
(29)
Based the above definition, the authors construct weighted adjacency matrix related the depth of the NN:
(Wa) E (CG))
if otherwise
(30)
10
where m is the output dimension of the NN.
The objective function of GraN-DAG is:
max P(X) Σ logp, (Xy (p)(
s.t. h(o) Tre d=
(31)
The unconstrained objective function of GraN-DAG is:
max L (Φ) 04t, (4, (+) xxxP(X)
log Pj (X₃ X (pa(X,): oxh(())
Pr h(d) 2
(32)
When the data generation mechanism follows the nonlinear Gaussian additive noise model, it can be proven that the optimal solution of GraN-DAG corresponds to the ground-truth. We use the implementation of GraN-DAG algorithm provided by the authors, available at htl/tttss//gtttut
BaeiiannGrra--DAA GrAN-DAG
B.10 NOCURL
Since a DAG is related to curl-free functions on its edge set, NoCurl proposed new representation of DAG:
A = <W, p), where W is skew-symmetric matrix with W -WT IRd is the potential function on the vertices of the graph.
(33)
The authors further proved that:
(W.p) W (grad(()))))
(34)
where grad is the gradient operator.
The optimization problem established by NoCurl is:
(W* ",',,"))= argmin p),, W.p
(35)
with the optimal DAG A' = W' o ReLU (grad (p*)). NoCurl implicitly ensures the acyclicity constraint, overcoming the shortcomings of the ALM, avoiding multiple iterations, and improving computational efficiency. We use the implementation NoCurl algorithm provided by the authors, available at /tttps://qittb.... com wht.://conn/iinmnnnnmmm 234 DAG-NoCur)
22

---

Published as a conference paper at ICLR 2025
B.11 DAGMA
DAGMA (Bello et al.
2022 proposed a log-determinant form of acyclicity representation W) + dllgg s (s 0), which has three advantages compared
Mide (W) log del W
to the exponential acyclicity constraints hexpm (W) Tr(eWow) d (Zhnng al. 2018 and polynomial acyelicity constraints Apoly [r[( aW wy4 d (a > 0) (Yu al. 2019). The authors proved that:
Rexpa (W)
Tr ((W wy+) d,
k!
(36)
d
Apoly (W)
Tr ((W o w)) -
d
where Tr (WW w)) represents the information of cycles of length k. The information of cycles of (*)
length & in hexpm (W) and (Pool((W) is weakened by kt and respectively. It can be theoretically d' proven that Mact (W) is an upper bound of hexpm (W) and hoply retaining more information about the cycles.
The authors also proved that:
V Vhexpm (W) = 2 (c oW 1
Vhioty (W) = ((+++) W
d-1 Hoowow wW)
W.
(37)
Vhidet (W) 2 ((sl-Wo W) W
Viteppm (W) and V/ipoty (W) are prone to the vanishing gradient problem. It can be theoretically proven that Whfaes (W) is an upper bound of V Vexxpm (W) and V/ (W), retaining more informa- tion about the cycles.
The third advantage is that, in practice, Midet (W) and Vhijett (W) are faster to compute. Because the computation of a (W) and Vhi (W) involves matrix log-determinant and matrix inverse, both of which have been extensively studied and solved. In contrast, other acyclicity constraints and their partial derivatives involve multiple matrix-to-matie multiplications, which are slower. We use the implementation of DAGMA algorithm provided by the authors, available at https 7/ htt.ub..oo//eviieeeee 11 /dagaa
C RELATED WORK
Differentiable causal discovery methods. Building on traditional score-based causal discovery algorithms, NOTEARS (Zheng el. (2088) transformed discrete constrained optimization into smooth cquality-constrained optimization. This formulation has been extended to various settings, including more efficient linear models (GOLEM (Ng et 2020), NoCurl YYu al. (022). NOFEARS (Wei
et al. 2020). LEAST (Zhu et al. 2021)), neural networks MOTEARRS-LA. (Zheng et al. 2020).
GraN-DAG [Lachapelle al. 2019). DAGMA (Bello et al., 2022), DARING (He et al. 2021).
CASTLE (Kyono et al.] (0200). generative adversarial networks (SAM (Kalainathan et al. 2022)).
variational autoencoders (D- VAE Zhang et al.) 2019)), graph neural network (GAE (Ng et al.
2019), DAG-GNN (Yu et al.] 2019)). federated learning (FedDAG (Gao et al. (00T)) reinforcement learning (RL-BIC Zhu et al. 2019).. interventional data (DCDI (Brouillar et al.) 2020 ) time series data (DYNOTEARS (Pamfil et al. 2020)), multi-domain data (DICD (Wang et al. 2022).
ReScore (Zhang et al. 2023), CASPER (Liu et al. ()23)), and domain adaptation (CAE (Yang et al. 2021b). Although differentiable causal discovery has made significant progress, it is also affected by Var-Sortability (Reisach et al. [021] 2023) and highly non-convex optimization problems (Ng et al.) 2024). Recent research by Deng al. (2024) shows that differentiable causal discovery methods can achieve scale invariance and global optimization when the correct loss function is used.
23

---

Published as a conference paper at ICLR 2025
D TABLE RESULTS FOR RUNTIME OF THE BENCHMARK METHODS
The results in Table 8 and [[ssww that differentiable causal discovery, exemplified by DAGMA, NOTEARS-MLP, and NoCurl, achieve superior performance with almost negligible runtime cost.
Table Results for runtime (in seconds) degree graphs of 10 and 20 nodes. The reported results are the mean and standard deviation of the runtime over 10 repetitions across different graph types, vanilla and model assumption violation scenarios.
Method
d
Runtime (seconds)
10
1.29:10.24
PC
20
1.91
10
1.64:0.53
GES
7.82+2.03 20
10
1.25::0.28
DirectLiNGAM
2.06++0.43 20
10
1.11
Regress
1.26::.229 20
10
1.13:0.16
Re-SortnRegress
1.24 20
10
9.35+:2.63
NOTEARS
20
35.57+4.77
10
130.23+2.54
GOLEM
178.52+3.41 20
10
5.68++0.49
NoCurl
20
1..29
10
45.73+22.67
CAM
20
113.37
10
4.70-H0.82
NOTEARS-MLP
5.84 20
10
119.28+
GraN-DAG
20
2.3.5
10
2.41+-0.22
DAGMA
20
3.19:10.48
24

---

Published as a conference paper at ICLR 2025
E
TABLE RESULTS ACROSS NODES, GRAPH TYPES, AND GRAPH DENSITIES
More experimental results for linear, nonlinear and MLP settings are reported in the Appendix as
Table 9
10
12
13
14
15
16
17
18
19
20
21 shows.
Table 9: Linear Setting, for ER-2 graphs of 10, 20, 50 nodes.
Toaden
104:51
11114
1000100
323.33
201812
26-41
13:43
T23077
348-22
1746-22
134458
11.0
14
773114
081
80
601
45
021
70L
96
6%
Tel Sart
1131144
065401
3431 416
$13.34
348-14
155
132/17
Table 10: Nonlinear Setting, for ER-2 graphs of 10, 20, 50 nodes.
0284
114-44
483-48
187657
STALTO
$41.07
178
M1:11
$13.44
16
1251-214
1944
43L
ECL
90L
ECL
411
114400
15/3-100
01 01251111
13856116
21 11-45
200050
074.74
1941/077
736
Table 11: MLP Setting, for ER-2 graphs of 10, 20, 50 nodes.
0811
4013
-083]
(011
51157
194
166115
14317
12.0
4.
90%
590
59%
0101
40%
590
90%
90
900
59V
15
38 6106
204
201420
300
500
331
100
300
200
60
301
CM
MI
100 34000
A00 mis
453
1111 100
25

---

Published as a conference paper at ICLR 2025
Table 12: Linear Setting, for 4R-4 graphs of 10, 20, 50 nodes.
2iv02
305-10
2344813
136.33
3301
N4:
10:
3031
IMBA
611-22
811
XEL
3001
3EL
DEL
821
3XL
20 1404
163-00
02000
Table 13: Nonlinear Setting, for ER-4 graphs of 10, 20, 50 nodes.
062100
10:2471
21
1902-55
3664
141154
002-41
(028-10)
158
190 110
Table 14: Linear Setting, for SF-2 graphs of 10, 20, 50 nodes.
Scrsunds
13412
T04130
1662
255-144
363-12
32418
SESTEY
TREETT
122644
(044011
$1.114
Stev33
1155
1811
196
Table 15: Nonlinear Setting, for SF-2 graphs of 10, 20, 50 nodes.
120
200
TRA4TE
275-400
1R2-001
25611
113000
28.2-114
$2443
26

---

Published as a conference paper at ICLR 2025
Table 16: Linear Setting, for SF-4 graphs of 10, 20, 50 nodes.
183443
24:12
176r13
153.55
134.55
13.50
64.04
Thears
400L
401
107
611
PC
T52RCE
152644
1203
13.00
Fanden
TRESUSS
OU PM
90091144
36
091
46.5 14
201
104
1044 144
Table 17: Nonlinear Setting, for SF-4 graphs of 10, 20, 50 nodes.
21
145-44
017-454
238144
055-114
178148
5185
P
9555 413
114
100 1121
115
15-143
183
632427
CAPACO
30/2603
156% 427
45/133
305
25
J1.5
311
3405
CMM
M05110
15
1611 111
1434
100 1115
Table 18: Linear Setting, for ER-6 graphs of 20 nodes.
302-154
1164-1
1164404
1154434
113
1121643
1.11
100.80
120024
284 1441
143.99
546/18
16-50 06-3145
086
122
018111
294-155
228-83
154-14
Table 19: MLP Setting, for ER-6 graphs of 20 nodes.
11011
S11
101L
1981-184
063-10
74713
223
1164166
104377
112 1020
4134
1145112
4144
3152-194
3113
130 5404
Table 20: Linear Setting, for GRP-2 graphs of 20 nodes.
6.11
28L
150000
3021
158176
162055
04.54
28.44
$3613
3M6-2
4360
30114
03100
245UX0
18:35
20113
19815
174c4
Table 21: MLP Setting, for GRP-2 graphs of 20 nodes.
sods
S1b.
Vall
SD.
1EL
SIEL
40%
SID.
SICS
SEEC
SEL
2015
264-212
125127
441/3
1974
198994
110
DAUVEA
$5115
IM441
337144
135104
345-19
345
63107
155
48/23
35i1
38:03
[66M
27

---

Published as a conference paper at ICLR 2025
F TABLE RESULTS FOR COMBINED MISSPECIFIED SCENARIOS
We consider two (confounding and heterogeneity), three (confounding, measurement error, and het- erogeneity) and four (confounding, measurement error, heterogeneity, and autoregression) combined misspecified scenarios. The results in Table 22 and Table 20 show that under combined misspecified scenarios, the performance of various methods worse compared to single misspecified scenario. However, differentiable causal discovery still achieves optimal or competitive performance.
Table 22:
Linear Setting with two (confounding and heterogeneity), three (confounding, measurement error, and heterogeneity) and four (confounding, measurement error, heterogeneity, and autoregres- sion) combined misspecified scenarios, for ER-2 graphs of 10 nodes.
aides
SHD,
StD:
S1D)
SED:
SEDD
SID,
SHD.
SDD
S1D
SHD.
SED)
SIED]
SED)
SIDD
SED)
Basion
288411
084-.1
2.144
00.1.57
543130
362.43
271413
22-7442
012413
56:1423
00.2335
Bk1-4
010-43
218644
253:180
238477
33.0.:3
4354149
232
3524123
261454
020
363450
175624
120111
$4488:
[T
125.00
31.5us
1155.00
130-43
337
302143
3241540
230411
23 333300
301-300
201344
3191144
33-100
283116
253153
DOLESI
1A:14
6A113
13128
142135
118124
201333
63144
18161
184440
363 159
263.43
IBA137
-62151
133143
123163
RAUMA
12:12
33155
14130
55.13
120125
25135
12AL82
113:35
254.153
-2015
125.89
1451
304165
135131
312.K4
Table 23: MLP Setting with two (confounding and heterogeneity). three (confounding, measurement error, and heterogeneity) and four (confounding, measurement error, heterogeneity, and autoregres- sion) combined misspecified scenarios, for ER-2 graphs of 10 nodes.
Vadiaa model
Thrse caalllllii SPED)
GHDL
401)
SUF))
SID)
SHD.
480
SHD,
S1D)
SHD)
SID)
Raden
63 60
25 25-4811
63.8
33.3013
TE.422
65:2445
24 2824411
01.21-33
32,4033
20::44
2822
14.2544
38 383223
00391:03
548:33
MOTEARSABL DAGMA
8.122
22:3185
113185
353.150
6111115
15:145
715158
114111
198137
323153
BC K.B133
83117
1321162
93143
275.315
141114
362083
127113
31.172
118121
41.1156
1538133
48:11044
184
516..304
358125
688173
G TABLE RESULTS FOR NON-GAUSSIAN NOISE
We consider the vanilla model with exponential noise. The results in Table 24 and Table 25 show that differentiable causal discovery still achieve optimal or competitive performance when model assumptions are violated.
Table 24: Linear Setting with exponential noise, for ER-2 graphs of 10 nodes.
SRD)
S4D)
SE1
1H00)
stn
stn
S00)
S00)
S0b)
1E..
N15114
3:30
1313
17.173
68.63
35AL55
150333
12034
13.3
63.00
#1 Sum Regame
19681
30017
D0E 134
13537
30035
133885
09217
43360
165544
365151
63404
4044
64114
Table 25: MLP Setting with exponential noise, for ER-2 graphs of 10 nodes.
Vadila
$180)
SHO0
1101
SH01
100.
2101
$E5.
Kanke
115335
94.
12284N
11:2333
$250117
11::24
115:35
54 4:00
138053
50
Jazara
BCT253
CAM
11308%
48 3mm
86 Swats
316
(52445
135e22
15
113
41 1150m
312477
4254881
DNAMA
45.23
135
123
362133
122154
155050
135.56
153/m1
38:1144
28

---

Published as a conference paper at ICLR 2025
H SUMMARY OF THE MOST COMPETITIVE METHODS
Table 26: Summary of performances of the most competitive methods under linear setting. The reported results the and standard deviation of the metrics 10 repetitions across different graph types, vanilla and model assumption violation scenarios.
Method
d
SHD
SID
10
8.51=5.92
23.88:123322
NOTEARS
20
22.66+14.22
109,96+
50
61.07++39.27
945.01++774.12
10
9.02=+6.90
30.67+227.39
GOLEM
20
14.19
155.60+112.441
50
55.57=35.00
1000.95+842.800
9.39
23.59-123.92
NoCurl
20
22.22
99.43:199.13
50
128.97=113.22
914.22:1699.51
10
8.12
22.42-122.72
DAGMA
20
20.98 14.10
127.31+112255
50
=37897
883.76-883.40
Table 27: Summary of performances of the most competitive methods under nonlinear setting. The reported results are the mean and standard deviation of the metrics over 10 repetitions across different graph types, vanilla and model assumption violation scenarios.
Method
d
SHD
SID
10
8.2115.17
22.661116.98
CAM
20
22.33++2..66
117.37=79.80
50
61.92:135.50
696.471463.94
10
12.23+3.63
14.68+110.27
NOTEARS-MLP
20
28.86-7.74
217.09+78.45
50
13.40+118.41
1271.244560.81
10
10.44+-5.11
38.19
GraN-DAG
20
30.46+18.23
214.86++75.62
50
85.34
1453.35
10
12.6414.66
46.99117.88
DAGMA
20
288.2818.88
211.26 85.84
50
72.07 1201.5
1254.571583.11
Table 28: Summary of performances of the most competitive methods under MLP setting. The reported results are the mean and standard deviation of the metrics over 10 repetitions across different graph types, vanilla and model assumption violation scenarios.
Method
SHD
SID
10
15.67+2.87
44.70-16.53
CAM
20
34.88+9.23
179.14+29.388
50
84.89-226226
904.99+134.22
10
12.9314.24
34.63+12.57
NOTEARS-MLP
20
26.97-8.02
132.86:152.22
50
70.08+22.880
740.91+239.08
29

---

Published as a conference paper at ICLR 2025
I TABLE RESULTS ON
DATA
We test the performance of 12 benchmark methods on the real-world Sachs (Sachs et al. 2005 dataset. Sachs bioinformatics dataset used study the expression levels of various proteins and phospholipids in human cells, and it is a commonly used benchmark in the causal discovery field. We conduct experiments based on 7466 samples. The true graph structure of the Sachs dataset contains 11 nodes and 17 edges, and it is widely accepted by the biological research community.
Table 29: Results on Sachs dataset.
Method
SHD
SID
Random
33
56
PC
22
49
GES
30
47
DirectLiNGAM
14
50
Reg-Sortnnegress
19
49
Re-SortnRegress
22
51
NOTEARS
17
48
GOLEM
15
58
NoCurl
16
50
CAM
15
51
NOTEARS-MLA.
14
46
GraN-DAG
15
45
DAGMA
12
42
The results in Table 29 show that, represented by DAGMA, differentiable causal discovery achieves optimal performance on the real-world Sachs dataset. Considering that Sachs is also regarded as real-world heterogeneous dataset (Mooij et al., 2020). the results on both Sachs and synthetic datasets further indicate that differentiable causal discovery performs better under model assumption violations.
30

---

Published as a conference paper at ICLR 2025
J FIGURE RESULTS ACROSS NODES, GRAPH TYPES, AND GRAPH DENSITIES
This section presents comprehensive analysis of the figure results across varying numbers nodes, graph types, and graph densities, in Figure 3
and
Varilla
Varila
Mechenism violation
Confounders
Mechanism violetion
Confounders
100
Missing
Measurement enorr Missing
Messurement
Scale-vrrant
sutoregressive
Scale-varant
Autoregressive
Difernntacl based
Diforentab based
Unfaitriul
Hoterugeneous
NGAM Sartiliils
Unfathru
Heterogeneous
GES
(a) SHD with linear ER-2 graphs of 20 nodes
SID with linear ER-2 graphs of 20 nodes
Confounders
Mochanism vislation
Confoundors
Missing
10
Measurementtero Missing
Measurement
Scale-eriant
Autoregressive
Scale-vrrant
Autoregressive
Random
PC
PC GES.
Heterogeneous
GES
Heterogeneous
(c) SHD with nonlinear F.R-2 graphs of 20 nodes
(d) SID with nonlinear ER-2 graphs of 20 nodes
Figure 2: Experimental results under the linear and nonlinear ER-2 graphs 20 nodes. SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, present only the optimal results. As the nonlinear settings in Figure 2c and Figure more favorable to CAM, conduct more reasonable evaluation of CAM and differentiable causal discovery under the MI.P setting (Section 4.1.1
31

---

Published as a conference paper at ICLR 2025
Varilla
Varila
Mechenism violation
Confounders
Mechanism violetion
Confounders
10
Missing
Measurement enror Missing
Messurement emor
Autregressive
Autoregressive
Orfernniacle based Random
Diforentat: basod Random
Unfaltftll
Heterogeneous
DiMM Var Sorthoo
UnfaltYLi
Heterogeneous
MeCctLINGAA Var SommRogrss
(a) SHD linear ER-4 graphs of nodes
(b) SID with linear ER-4 graphs 10 nodes
Vanills
Vanilla
Mochanism vialation
Confounders
Mochanism vialation
Confounders
Measurement Missing
Measurement
Scale-arrant
Autoregressive
Scale-varfant
Autoregrescive
Unfaithful
Heterogeneous
GES
Unfaltri
Heterogeneous
GES
CAM
CAM
(c) SHD with nonlinear ER-4 graphs of 10 nodes
(d) SID with nonlinear ER-4 graphs of 10 nodes
Figure 3: Experimental results under the lincar and nonlinear ER-4 graphs of 10 nodes. SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure 3elureeecna and Figure ad are more favorable CAM, we conduct more reasonable evaluation CAM and differentiable causal discovery under the MI.P setting (Section
32

---

Published as a conference paper at ICLR 2025
Varilla
Varila
Mecherism violation
Confounders
Mechanism violetion
Confounders
Missing
Measurement enror Missing
Messurement emor
Autregressive
Scale
Autoregressive
Difernniacle based Random
Diforentat: basod Random
Unfaltntll
Heterogeneous
DIMM
UnfaltYLi
Heterogeneous
WeeCtLINGAA Var SommRogrss
Var Sorthoo
(a) SHD linear ER-4 graphs of nodes
(b) SID with linear ER-4 graphs nodes
Vanills
Vanilla
Mochanism vialation
Confounders
Mochanism vialation
Confounders
Laca
Measurementenog Missing
Measurement
Scale-arrant
Autoregressive
Scale-varfant
Autoregrescive
Unfaithful
Heterogeneous
GES
Unfaltri
Heterogeneous
GES
CAM
CAM
(c) SHD with nonlinear ER-4 graphs of 20 nodes
(d) SID with nonlinear ER-4 graphs of 20 nodes
Figure 4: Experimental results under the lincar and nonlinear ER-4 graphs of 20 nodes. SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure 4c and Figure Adgure/dlaa 4d more favorable CAM, we conduct more reasonable evaluation CAM and differentiable causal discovery under the MI.P setting (Section
33

---

Published as a conference paper at ICLR 2025
Varilla
Vanila
Mechenism violation
Confounders
Mechanism violetion
Confounders
Missing
Measurement enror Missing
Messsrement emor
Autregressive
Autoregressive
Difernniacle based Random
Diforentat: basod Random
Unfaltftll
Heterogeneous
DiMM
UnfaltYLi
Heterogeneous
MeCctLINGAA Var SommRegrss
Val Sorthoo
(a) SHD with linear SF-2 graphs of nodes
(b) with linear SF-2 graphs of nodes
Vanills
Vanilla
Mochanism viatation
Confounders
Mochanism vialation
Confounders
Measurementeemo Missing
Measurement
Scale-errant
Autoregressive
Scale-varfant
Autoregrescive
Unfaithful
Heterogeneous
GES
Unfaltri
Heterogeneous
GES
CAM
CAM
(c) SHD with nonlinear SF-2 graphs of 10 nodes
SID with nonlinear SF-2 graphs of 10 nodes
Figure 5: Experimental results under the linear and nonlinear SF-2 graphs of 10 nodes, SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure[5caan and Figure are more favorable CAM, we conduct more reasonable evaluation CAM and differentiable causal discovery under the MI.P setting (Section
34

---

Published as a conference paper at ICLR 2025
Varilla
Varila
Mechenism violation
Confounders
Mechanism violetion
Confounders
LOO
Missing
Measurement enror Missing
Messurement emor
Autregressive
Scale
Autoregressive
Difernniacle based Random
Diforentat: basod Random
Unfaltffll
Heterogeneous
DiMM
UnfaltYLi
Heterogeneous
NGGN Var SommRegrss
Va SorthRo
(a) SHD with linear SF-2 graphs of nodes
(b) with linear SF-2 graphs of nodes
Vanills
Vanilla
Mochanism vialation
Confounders
Mochanism vialation
Confounders
G0
80
100
Measurementenco Missing
Measurement
Scale-arrant
Autoregressive
Scale-varfant
Autoregrescive
Unfaithful
Heterogeneous
GES
Unfaltri
Heterogeneous
GES
CAM
CAM
(c) SHD with nonlinear SF-2 graphs of 20 nodes
SID with nonlinear SF-2 graphs of 20 nodes
Figure 6: Experimental results under the linear and nonlinear SF -2- graphs of 20 nodes, SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure pe and Figure adlare are more favorable CAM, we conduct more reasonable evaluation CAM and differentiable causal discovery under the MI.P setting (Section
35

---

Published as a conference paper at ICLR 2025
Varill
Varila
Mechenism violation
Confounders
Mechanism violetion
Confounders
Missing
Measurement enror Missing
Messsrement eror
Autregressive
Autoregressive
Difernniacle based Random
Diforentat: basod Random
Unfaltntll
Heterogeneous
DiMM
UnfaltY
Heterogeneous
MeCctLINGAA Var SommRegrss
Val Sorthoo
(a) SHD with linear SF-4 graphs of nodes
(b) with linear SF-4 graphs of nodes
Vanills
Vanill
Mochanism viatation
Confounders
Mochanism vialation
Confounders
Measurementeemo Missing
Measurement
Scale-arrant
Autoregressive
Scale-varfant
Autoregrescive
Unfaithful
Heterogeneous
GES
Unfaltri
Heterogeneous
GES
CAM
CAM
(c) SHD with nonlinear SF-4 graphs of 10 nodes
SID with nonlinear SF-4 graphs of 10 nodes
Figure 7: Experimental results under the linear and nonlinear SF-4 graphs of 10 nodes, SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure 7c and Figure ad are more favorable CAM, we conduct more reasonable evaluation CAM and differentiable causal discovery under the MI. setting (Section
36

---

Published as a conference paper at ICLR 2025
Varilla
Varila
Mechenism violation
Confounders
Mechanism violetion
Confounders
150
20-
100g
Missing
Measurement enror Missing
Messsrement emor
Autregressive
Scale
Autoregressive
Difernniacle based Random
Diforentat: basod Random
Unfaltntll
Heterogeneous
DiMM
UnfaltYii
Heterogeneous
WeCctLINGAA Var SommRegrss
Va SorthRo
(a) SHD with linear SF-4 graphs of nodes
(b) with linear SF-4 graphs of nodes
Vanills
Vanill
Mochanism vialation
Confounders
Mochanism vialation
Confounders
200
Missing
Measurementenog Missing
Measurement
Scale-errant
Autoregressive
Scale-varfant
Autoregrescive
Unfaithful
Heterogeneous
GES
Unfaltri
Heterogeneous
GES
CAM
CAM
(c) SHD with nonlinear SF-4 graphs of 20 nodes
SID with nonlinear SF-4 graphs of 20 nodes
Figure 8: Experimental results under the linear and nonlinear SF-4 graphs of 20 nodes, SHD (the lower the better) and SID (the lower the better) are evaluated over 10 trials. For the differentiable causal discovery method, we present only the optimal results. As the nonlinear settings in Figure and Figure 3d are more favorable CAM, we conduct more reasonable evaluation CAM and differentiable causal discovery under the MI.P setting (Section
37

---

Published as a conference paper at ICLR 2025
K
TABLE RESULTS FOR EXTREME MEASUREMENT ERROR
Table 30 presents the results in the linear setting under measurement error with ă = 10. The results Table indicate that when ò takes larger values, differentiable causal discovery methods fail demonstrate robust performance. 8 is used to control the variance of in (7). As ó increases, the noise ratio correspondingly increases, which leads the loss of robustness differentiable methods. Tables 2. and 9 present the results in the linear setting under measurement error with á = 0.8. The results in Tables 2. and indicate that when 0.8 takes a smaller value, the noise ratio correspondingly lower, allowing differentiable methods to demonstrate robust performance.
Table 30: Linear Setting under measurement error with å = 10, for ER-2 graphs of 10, 20 nodes.
Vanilla model
Measurement error (ō = 10)
10 nodes
SHD
SID
SHD4
SID
Random PC
25.613.1
57.9
23.1
61.2+7.5
12.4.33.1
40.9
19.241.1
56.919.3
GES
13.8+7.8
13.6++3.6
20.2+4.5
54.1+11.6
DirectLiNGAM
19.6133.
10.1
20.0
61.248.2
DAGMA
1.2+1.2
3.3+5.3
20.7+1.2
58.2+7.6
20 nodes
SHD
SID
SHD
SID
Random
107.947.0
253.2426.3
92.7+6.7
243.6+19.4
PC
31.616.55
168.5427.6
44.514.6
213.8+24.5
GES
34.3+24.6
104.5+51.1
51.2+7.1
220.9+27.6
DirectLiNGAM
55.7+9.1 166.4431.0
41.8+1..
210.3+24.3
DAGMA
5.4139
14.2+10.3
49.414.6
227.5+24.9
38

---

Published as a conference paper at ICLR 2025
L TABLE RESULTS ON SEMI-SYNTHETIC DATA
The semi-synthetic data is generated based on the network structure of the real-world Sachs dataset, using linear and nonlinear vanilla models create eight datasets with model assumption violations. The results in Table 31 and 32 indicate that differentiable causal discovery methods still achieve optimal or competitive performance in scenarios other than scale variation.
Table 31: Linear Setting, for semi- eemi-synthetic data of 11 nodes.
Monc Amm
Sacke S1D1
Madia
SE3
30101
$33
51001
SIDO
SEDI
SD.
188-113
CLS
300u33
3384150
178.35
$45.29
225
155.35
123.24
383
52163
263.42
18.33
365183
182.03
28.51144
88:14
18993
135293
133253
129077
113583
17a-15
11354
83553
1920
34:39
13954
183:26
NOTHARS
22203
300083
183
11.-M4
1:3184
14814
355-1
04629
13
483.82
13n13
33803
14.523
317618
35:85
18153
113:39
18.12
MAD23
183184
112-44
93-32
13133
A1151
05:39
46.31
123134
488.64
15:00
431.54
225:35
232:43
Table 32: MLP Setting, for semi- data of 11 nodes.
11 Fede
SID.
SHO0
58%
SIEN
SED4
SEG81
2L15
S1E
515
SIDU
$5N
3100
5001
$E5
Recker
BLONTA
118-21
6. 1-0-0>
11.5044
55 Sunt
342803
57 $7.00a0
CES
102113
2241114
211-36
4296113
154122
48 23.00
ITALI
MANLERS
25 $.40
211153
22-54113
195.20
2700-00
844137
265/63
081472
GUNIDNG
IM8-24
30.6-38
261443
0.8621
15.3.00
111551
121:49
39


Reviews:
[{"review_id": "054sZszbz2", "reviewer": "Reviewer_KWup", "summary": "This work aims to empirically study the behavior of causal discovery algorithms, with a particular focus on optimization based methods. The authors approach this problem by generating a set of graph structures using erdos renyi and scale free graph generation algorithms, and then considering a set of scenarios of misspecification with respect to the generated data. These results are compared across a large representative set of causal discovery algorithms. Results are mixed overall, but generally show preferable performance for optimization and additive noise models.", "questions": "Following up from above:\n* Are the presented results the mean over 10 trials? \n* Is significance being computed? If so how? \n* It seems strange that many algorithms seem to have the same value for SHD and SID in the latent confounder setting. Can the authors provide some intuition for this? \n* The degree of these graphs is fairly limited. Did the authors try denser settings? If so can these results be described as well?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper is addressing a very important, and generally understudied, problem within causal discovery. Because causal discovery relies on a set of untestable assumptions in order to provide the accompanying theoretical guarantees on structure recovery (and as a result entailed causal estimands), it's critical to have an understanding of the relative merit of these algorithms under violations of assumptions. The authors do a nice job of compiling a large set of candidate algorithms, and considering multiple misspecification scenarios. The paper also presents a substantial number of empirical results both in the main text and the supplement. While the paper contains little theoretical content, its direction represents work that is valuable to the community, allowing practitioners to reason over the behavior of current state of the art outside of the context of work that is advocating for a specific approach.", "weaknesses": "* Erdos Renyi and scale free networks are both fairly unrepresentative of the structures encountered in real world settings. It would have been preferable to either have a much wider range of graph generation algorithms, or used a semi-synthetic approach where the structure of known, real world networks, are used and synthetic data drawn using those implied conditional independencies to augment the current results. \n* Overall, I would have liked to have seen more discussions on the implications of the findings of the experiments in practice. The authors do a commendable job of trying to assess a number of scenarios, but the accompanying discussion is limited and there, to my reading, isn't a discussion summarizing the findings or broader implications for practice. Including discussion and analysis along these lines would substantially improve the value to the community. \n* It isn't clear how significance is being computed here. The simulations appear to only use 10 iterations per setting. There also aren't any uncertainty bounds given. See below for more questions on this front.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HU3tH6thqb", "reviewer": "Reviewer_VU3C", "summary": "Causal discovery algorithms frequently depend on unverifiable causal assumptions, which are often challenging to fulfill in real-world data. This study conducts an extensive empirical evaluation of the performance of various mainstream causal discovery algorithms across eight types of model assumption violations. The work finds that differentiable causal discovery methods demonstrate unexpected robustness in challenging conditions. Furthermore, they offer theoretical insights into the observed performance of differentiable causal discovery methods.", "questions": "(See weaknesses above)", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper addresses one of the most critical challenges in causal discovery: the applicability of causal discovery methods to real-world data. Overall, the paper is well-structured and clearly written. \n\n2. The experiments are extensive, covering eight types of model assumption violations, twelve causal discovery methods, and varying graph sizes.", "weaknesses": "1. In the synthetic experiments, each type of model assumption violation is considered in isolation. However, in real-world datasets, multiple issues may arise simultaneously. How do the causal discovery methods perform when these issues are combined, e.g., confounding effects + measurement errors + heterogeneity? \n\n2. Additionally, if you were to recommend the single most reliable causal discovery method for real-world datasets, what would it be? \n\n3. Although this paper aims to address real-world problems, no real-world datasets can be found in the study.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MncdRCrk6p", "reviewer": "Reviewer_gPsY", "summary": "This paper provides a comprehensive study of the performance of various causal discovery algorithms under eight types of model assumption violations. Notably, the differentiable causal discovery algorithm demonstrates the greatest robustness across different metrics. The paper also includes a degree of theoretical discussion regarding the performance of differentiable causal discovery methods.", "questions": "* Recent work [1] has shown that methods like NOTEARS and DAGMA can achieve scale invariance if the correct loss function is applied and optimization reaches the global optimum. In this paper, differentiable causal discovery performs poorly on scaled datasets; however, this recent work [1] suggests that such issues can be addressed. I believe this insight adds valuable perspective on differentiable causal discovery and warrants an appropriate discussion in the paper.\n\n\n\n[1] Deng, C., Bello, K., Ravikumar, P. and Aragam, B., 2024. Likelihood-based Differentiable Structure Learning. arXiv preprint arXiv:2410.06163.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This paper presents an extensive empirical study of twelve leading causal discovery methods across eight scenarios involving model assumption violations. The benchmark provides strong evidence for the robustness of differentiable DAG learning algorithms.", "weaknesses": "* The paper does not include experiments on real-world data. Since the focus is on evaluating various DAG learning algorithms under assumption violations, real-world data would be a natural and valuable addition. Including real-data applications could strengthen the study’s relevance.\n\n* Some algorithms are challenging to train and may require longer training times, which could partly explain their strong performance. A comparison of running times across algorithms would help clarify this aspect.\n\n* Including the standard error of each result would improve the analysis by indicating the stability of each algorithm across different datasets.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9aIyIxfFJn", "reviewer": "Reviewer_KM7a", "summary": "The present paper provides an extensive benchmark on causal discovery with a focus on differentiable algorithms in misspecified scenarios. This is an interesting effort to better understand the behavior of these methods from an empirical perspective, as the previous benchmark in similar misspecified settings (Montagna et al., 2023) almost did not cover any of the differentiable methods tested in this paper.", "questions": "1. Why did the authors limit their theoretical analysis of section 4.2.1 to measurement error and unfaithful datasets? I believe that in the light of the theoretical analysis in Ng, 2024 (Structure Learning with Continuous Optimization: A Sober Look and Beyond), it would be of great interest if this theoretical study was carried over all the scenarios. This is not a criticism to the paper, but a suggestion of what I believe would be a good contribution in connection to this work.\n2. L64: the authors say *true mechanisms remain unclear when applied to real data*. This sentence is obscure to me, could they clarify?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 1, "presentation": 3, "contribution": 2, "strengths": "The misspecifications and the differentiable algorithms subject to testing are well chosen, and the experimental setting is clearly explained. Section 4.1.2. provides interesting theoretical explanations about the empirical performance of differentiable-based methods in the case of a few misspecified scenarios.", "weaknesses": "## Main points\n1. I don’t understand why the linear vanilla model is defined with Gaussian noise terms, which are notably non-identifiable. This may be an interesting choice in the light that non-standardized SCMs may still be identified in fact by differentiable causal discovery methods, even in these setting - e.g. exploiting equal variance effects, as studied in Peters et al., 2012 (Identifiability of Gaussian structural equation models with equal error variances). Yet, it is limiting that the only linear setting is that with Gaussian noise, due to the very well-known identifiability limitations affecting almost every algorithm in the list (with the exception of PC and GES).\n2. An additional, strong limitation of this work, is that it limits the benchmarking to non-standardized data: this is problematic specifically when testing occurs on differentiable-based algorithms, which notoriously degrade their performance when variance does not increase in the causal order. The authors’ experimental design choice, then, undermines most of the conclusion that one might draw from the experimental outcomes: looking at the linear Gaussian experiments (non-identifiable), the competitive performance of differential-based methods is clearly symptomatic that they are using the variance-induced *shortcuts* in non-standardized data (which is not surprising, as the authors seem to be aware given that they cite Reisach et al., 2021 and Ng et al., 2024 as references) (see the discussion on noise ratio in section 4.2.1 to understand what I mean by *shortcuts*). In light of the good performance of differentiable-based methods on vanilla linear data, all the conclusions that one can derive from these experiments boil down to the following question: given a misspecified scenario, does the noise ratio (discussed in section 4.2.1) ensure that the optimal solution for a differentiable based methods is the correct one? But this is not testing these algorithms’ robustness, it is more testing the properties of the misspecified datasets. Hence, conclusions of *counter-intuitive robustness* (L19) of differentiable based methods in the misspecified settings are wrong (in the sense that they are not surprising if the noise ratio allows it, as in the vanilla linear case). Moreover, they are partial, given that benchmarking occurs on a very limited class of SCMs, those with cumulating variance along the causal order, which we know to enable causal discovery with differentiable based methods. Testing should be performed on the class of standardized causal models, to gain meaningful conclusions about robustness. \n3. The SHD outcomes of Tables 2 and 3 are odd/off: e.g. in reference to the R2-sortnregress line of table 2, 50 nodes, you have SHD values of 388, 100, 346, 552, 549, 442, … How is this possible? In SHD, each error (edge addition, removal, or direction flip w.r.t. ground truth) counts as one, or counts as two if the direction flip is counted twice (because we have both a false positive and a false negative): in the latter case, for a fully connected graph with 50 nodes and 2 edges per nodes, the maximum achievable SHD is 200. Hence, I believe something fishy is going on with the SHD values proposed in the table (SHD beyond the possible bound appears all over Table 2 and Table 3). Could you explain this?\n4. In Table 2 the latent confounding columns SHD values have a weird behavior: it looks like all methods simply output an empty graph, i.e. for ER-2 graphs with 10 nodes, an empty graph gives you SHD 20, for 20 nodes you get SHD 40, and 50 nodes you get SHD 100: these are exactly the values you get for almost any run, which is suspicious, also in the light of results from Montagna et al., 2023, which already tested the nonlinear confounded setting, and whose result disagree with yours (in their paper there is some robustness displayed by some methods under confounded setting, while no algorithms seem to give the results agreeing with what the authors show). Comments on this point are needed.\n5. L343: authors say that on nonlinear data differentiable CD achieves optimal (what do you mean by optimal?) or competitive performance, whereas this is very bad assuming that SHD is bounded by the number of edges, which should be the case according to the description of SHD in section 3.4. One example (which generalizes to other nonlinear cases): on the vanilla model, 10 nodes, the differentiable methods achieve SHD around 10, on 20 nodes SHD around 30, on 50 nodes SHD between 70 and 85, which are all very bad. Hence, the analysis of the authors is incorrect.\n6. Authors only present pointwise SHD, without any deviation from multiple experiments or confidence interval, which should instead be provided\n\n## Minor points\n1. Reading Tables 2 and 3 is hard, as there are many lines packed in a small space, in a small font. Consider moving it to the appendix?\n2. In the introduction, I believe that authors are unnecessarily overselling the value of differentiable causal discovery methods, or at least they don’t seem objective: they define differentiable causal discovery a *ground breaking* *advancement* (L47), whereas in my direct and undirect experience (readings) they are not groundbreaking in the sense they don’t perform generally better than traditional methods or many other approaches to causal discovery. Later, they talk about their *transformative potential in the realm of causal inference* (L53), which is also unsupported by evidence, to the best of my knowledge.\n3. Again in the introduction, they say that causal sufficiency and absence of measurement error are *generally imperative* (L54) and they are *fundamental prerequisites for guaranteeing validity and reliablility of the causal inference* (L56-57). They are not imperative, nor fundamental prerequisites: these assumptions are not necessary for identifiability, as methods for causal discovery under confounding effects and measurement error exist, as the authors themselves mention. \n\n   In reference to this and the previous point: using expressions like *immense potential, imperative need, etc.,* makes unnecessarily strong statements, I would advise against it.\n4. L134: the citation to Lachapelle, 2019 (GRAN-DAG paper) when introducing identifiability is wrong. They just propose an algorithm and no identifiability statement. I would remove it, the citations coming after are those that are on point.\n5. The paper claims that Montagna et al., 2023 misspecifications are limited in scope, yet 6/9 scenarios covered here (e.g. those in the graphs of Figure 1) are covered there.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)