Title: Differentiable Constraint-Based Causal Discovery
Venue: NeurIPS 2025
Accepted: yes
Keywords: Causal Discovery

Constraint-based

d-Separation

Continuous optimization

Structure learning
Abstract: Causal discovery from observational data is a fundamental task in artificial intelligence, with far-reaching implications for decision-making, predictions, and interventions. Despite significant advances, existing methods can be broadly categorized as constraint-based or score-based approaches. Constraint-based methods offer rigorous causal discovery but are often hindered by small sample sizes, while score-based methods provide flexible optimization but typically forgo explicit conditional independence testing. This work explores a third avenue: developing differentiable $d$-separation scores, obtained through a percolation theory using soft logic. This enables the implementation of a new type of causal discovery method: gradient-based optimization of conditional independence constraints. Empirical evaluations demonstrate the robust performance of our approach in low-sample regimes, surpassing traditional constraint-based and score-based baselines on a real-world dataset. Code implementing the proposed method is publicly available at [https://github.com/PurdueMINDS/DAGPA](https://github.com/PurdueMINDS/DAGPA).

Differentiable Constraint-Based Causal Discovery

Abstract

Causal discovery from observational data is a fundamental task in artificial intelligence, with far-reaching implications for decision-making, predictions, and interventions. Despite significant advances, existing methods can be broadly categorized as constraint-based or score-based approaches. Constraint-based methods offer rigorous causal discovery but are often hindered by small sample sizes, while score-based methods provide flexible optimization but typically forgo explicit conditional independence testing. This work explores a third avenue: developing differentiable d -separation scores, obtained through a percolation theory using soft logic. This enables the implementation of a new type of causal discovery method: gradient-based optimization of conditional independence constraints. Empirical evaluations demonstrate the robust performance of our approach in low-sample regimes, surpassing traditional constraint-based and score-based baselines on a real-world dataset. Code and data of the proposed method are publicly available at https://github.com/PurdueMINDS/DAGPA .

1 Introduction

Causal discovery-the task of inferring causal relationships from observational data-is a fundamental problem in machine learning and statistics with far-reaching implications across multiple disciplines, including biology, economics, social sciences, and medicine [19, 25, 28, 33, 45]. Directed Acyclic Graphs (DAGs) provide a powerful framework for representing causal relationships, enabling the estimation of the effects of interventions or actions. However, in many complex systems, the underlying causal graphs remain unknown, and conducting randomized controlled trials (RCTs) to establish causal relationships can be prohibitively expensive or impractical. This has significant implications in various domains, including planning, explainability, and fairness [26, 47, 49], highlighting the need for reliable methods to infer causal structures from observational data alone.

The task of causal discovery is inherently challenging, specifically under small sample sizes. Considering Markov, acyclicity, and faithfulness assumptions [25, 34], some recent works have focused on two popular avenues [13, 39, 44]: differentiable score-based methods, which leverage differentiability to optimize objective functions, and constraint-based approaches, which rely on conditional independence tests to infer causal structures. Common methods for constraint-based causal discovery, such as the PC algorithm [34], are vulnerable to errors in datasets with small sample sizes, due to uncertainty in conditional independence tests (CI tests). More recent constraint-based methods, such as LOCI and k -PC [18, 42] among others, have ameliorated these challenges with smaller conditioning sets.

In recent years, significant progress has been made on addressing the challenges of small sample sizes in causal discovery via differentiable score-based methods, casting the combinatorial graph search problem as a continuous optimization problem on weighted adjacency matrices representing directed

graphs [4, 36, 51, 52]. These differentiable methods rely either on linear models or increasingly complex neural networks to model the underlying functional relationship between variables.

Contributions. A promising avenue for advancing causal discovery, complementing existing approaches, is the development of hybrid methods [13, 37, 39]. Our work tackles a key challenge in this domain by introducing differentiable functions of d -separation and d -connection , capable of scoring conditional independencies within a probabilistic causal graph, bridging constraint-based and gradient-based methods. To achieve this, we propose a novel framework grounded in a percolation measure applied to a continuous relaxation of the causal graph structure. This new measure effectively captures the inherent dependencies among paths in a causal graph, addressing a significant limitation of conventional diffusion-based methods, such as matrix powers, which would be unable to adequately model d -separation.

The distinction between diffusion and percolation lies at the heart of deriving a differentiable metric for d -separation, which essentially boils down to measuring reachability over random graphs [16]. To illustrate this concept, consider a typical setup in gradient-based graph optimization, where a matrix A ∈ [0 , 1] n × n represents edge probabilities [41, 51], with each entry A XY encoding the likelihood of an edge from node X to node Y . Astraightforward approach to assess connectivity between X and Y might involve computing A n XY , which represents the sum of all path probabilities of length n from X to Y . However, this method implicitly assumes independence between paths, akin to a diffusion process. For instance, in the paths X → Z → Y and X → Z → W → Y , the simultaneous absence of the sampled X → Z edge would invalidate both paths, introducing a probabilistic dependence that is not captured by matrix powers or diffusion in general.

In contrast, a d -separation measure over A is a type of percolation measure, which bounds the probability of sampling valid paths between X and Y , accounting for edge dependencies. Although characterizing percolation involves combinatorial complexity, we derive a differentiable bound for d -separation using soft logic, enabling gradient-based optimization of causal structures. We instantiate this differentiable d -separation framework in an algorithm (DAGPA, for DAG Percolation Apartness), where we combine CI-based scores, state-of-the-art techniques in multi-task learning, and Bayesian sampling. While DAGPA instantiates this novel framework, we believe differentiable d -separation is of independent interest and can be integrated into existing gradient-based methods.

Empirically, DAGPA shows that differentiable d -separation yields strong results in the small-sample regime, demonstrating good robustness and accuracy compared to popular constraint-based and differentiable score-based baselines. Moreover, on the real-world Sachs dataset [28], DAGPA shows that differentiable d -separation offers accurate modeling of the independence patterns in the data, outperforming baselines in our metrics.

2 Notation and background

Notation. We denote the integer set { 1 , . . . , d } as [ d ] and a dataset of n samples and d variables as D . We represent a weighted directed graph with d nodes and edge weights in the range [0 , 1] by a real square matrix W ∈ [0 , 1] d × d . Given a node z ∈ [ d ] , we denote W -z the submatrix obtained by removing the z -th row and column of W , which is equivalent to removing the node z and their connecting edges from the graph. When the square matrix is binary, e.g. A ∈ { 0 , 1 } d × d , we interpret it as an unweighted directed graph. Here, we write x → A y to say ' y is connected from x via a directed edge in A , ' and x ⇝ A y to say ' y is reachable from x via a directed path (a path where every edge has direction from x to y ) in A .' Conversely, x ̸→ A y and x ⇝ A y stand for negations of these statements. Finally, we use ⊥ ⊥ D for (conditional) independence statements in the dataset D 1 , and if A is acyclic, ⊥ ⊥ A for d-separation statements in the DAG A .

̸

Constraint-based causal discovery. Methods in this category aim to identify a collection of DAGs from a dataset D such that each DAG A in it is an I-Map of D , meaning all d-separations in A imply conditional independencies in D with a sufficiently large dataset. Namely, ∀ x, y ∈ [ d ] , Z ⊆ [ d ] ,

The collection of all DAGs satisfying this criterion constitutes the Markov Equivalence Class (MEC). Our method, as most methods for learning DAGs, relies on the Causal Faithfulness Assumption, which

posits the unique existence of a DAG A ∗ such that the observed data D exhibits only the conditional independencies represented by the d-separations in A . This assumption is equivalent to the converse direction of Equation (1). However, challenges emerge when dealing with large conditioning sets Z , as statistical tests for conditional independence become increasingly unreliable. To mitigate this issue, recent approaches like LOCI [42] and k -PC [18] restrict their search to DAGs with low-order conditioning sets (typically limited to a cardinality of 2 or less). As a result, their solution space is not the entire MEC, but rather the k -equivalence class (or k -essential graphs) [18], which comprises DAGs whose low-order d-separation statements are reflected as low-order conditional independencies in D . In a similar vein, our approach focuses on conditioning sets with a cardinality of 1 or less ( |Z| ≤ 1 ), allowing for more reliable and efficient causal discovery.

Continuous DAG Acyclicity Constraint. Pioneered by NOTEARS [36], recent score-based methods provide an interesting alternative in DAG search by reformulating it as a continuous optimization problem over weighted adjacency matrices. This transformation relies on the development of DAG acyclicity regularizers, which quantify the degree of cyclicity in weighted adjacency matrices. Our work follows this practice and uses the log-determinant DAG regularizer introduced by DAGMA [4], defined as:

where s is a hyperparameter that controls the size of the valid domain of this function, denoted as W s = { W ∈ R d × d | s ≥ ρ ( W ) } , with ρ ( W ) representing the spectral radius of W . As established by Bello et al. [4], for W ∈ W s , the condition L DAG ( W , s ) = 0 implies that W maps to a DAG.

3 Differentiable d-Separation Framework

Our core contribution lies in the transformation of d -separation, which is discrete and binary computed via combinatorial algorithms [8, 19, 32], into a continuously differentiable function over W ∈ [0 , 1] d × d . This transformation satisfies three central properties: (1) it exactly recovers conventional d-separation statements when evaluated on discrete DAGs A ∈ { 0 , 1 } d × d , (2) it provides a principled probabilistic interpretation when W is viewed as a parameterization of a distribution over graphs, and (3) it is a differentiable function admitting gradient-based optimization on W .

The key insight is grounding the score in a percolation-based graph connectivity measure. Specifically, we quantify how well a probabilistic graph structure (parameterized by W ) aligns with conditional independence (CI) patterns observed in data, such as those measured through statistical tests yielding p-values, which pave the way to a comprehensive set of differentiable objective functions that measure violations of low-order I-Mapness, Faithfulness, and Acyclicity constraints.

We achieve this differentiable framework through three key steps: (1) recasting d -separation as first-order logic (FOL) formulae operating on graph reachability (Section 3.1), (2) applying soft logic operators to transform these discrete statements into continuously differentiable functions with valid probabilistic interpretations (Section 3.2), and (3) combining these soft d-separation measures from the graph with conditional independence measures from data to construct objective functions that enforce low-order I-Mapness and Faithfulness constraints while maintaining acyclicity (Section 4).

3.1 FOL Formulation of d-Separation

Conventionally, d-separations are checked by discrete, combinatorial algorithms using data structures such as hash sets and double-ended queues [8, 19, 32]. The inherently sequential and combinatorial nature of these algorithms presents significant challenges in developing continuous relaxations and enabling gradient-based optimization. To address this limitation, we introduce a First-Order Logic (FOL) framework that provides an equivalent characterization of d-separation computations in discrete settings while maintaining differentiability properties. This reformulation bridges discrete reasoning and continuous optimization paradigms, as it allows us to leverage established techniques from probabilistic soft logic and fuzzy logic systems for end-to-end differentiation of the computation.

Specifically, we observe that the conventional notion of (low-order) d-separation and d-connection on any given discrete DAG A ∈ { 0 , 1 } d × d can be precisely captured by FOL formulæ based solely on graph reachability information. That is, given the graph reachability logical predicate

backdoor path

fr ontdoor path

(a) 0th-order d-connection pattern: checking existence of common ancestor (b) 1st-order d-connection pattern A: checking existence of backdoor path that does not involve z

(c) 1st-order d-connection pattern B: checking existence of frontdoor path that involves z

Figure 1: Illustration of the graph connectivity patterns checked by the low-order d-separation/dconnection FOL formulæ (Definition 3.1) to determine whether the given query ( ( x, y ) or ( x, y | z ) ) is d-separated or d-connected. The double-dashed line == · · · == refers to a 0th-order d-connecting path, and the arrowed line →··· → refers to a directed path. In either pattern, the variable nodes a and b could be the query node x or y themselves.

R A : [ d ] 2 →{ 0 , 1 } defined as

We can define the FOL formulæ expressing the 0th-order (i.e. unconditional) and 1st-order (i.e. conditioning on one variable) d-separation and d-connection statements as follows:

Definition 3.1. (Low-order d-separation and d-connection FOL formulæ) Given a discrete directed graph A ∈ { 0 , 1 } d × d , the 0th-order d-separation formula S (0) A : [ d ] 2 → { 0 , 1 } and the 1st-order d-separation formula S (1) A : [ d ] 3 →{ 0 , 1 } are defined as:

Equivalently, the 0-th and 1-st order d-connection statements C (0) A and C (1) A are:

̸

To understand why these formulæ correctly capture d-separation and d-connection when A is a DAG, consider first the 0th-order formulae S (0) A ( x, y ) and C (0) A ( x, y ) . These formulæ essentially determine whether any node (including possibly x or y themselves) serves as a common ancestor to both x and y (e.g. the node a ? in Figure 1a). If such a common ancestor exists, it creates either a chain structure (when a = x or a = y , meaning x is the ancestor of y or vice versa) or a fork structure (when a = x and a = y , and a third node is the common ancestor). In either case, x and y are unconditionally d-connected, yielding C (0) A ( x, y ) = 1 as expressed in Equation (4).

̸

The 1st-order formulæ S (1) A ( x, y | z ) and C (1) A ( x, y | z ) involve more intricate logic but follow an intuitive two-step process. Take C (1) A ( x, y | z ) for instance. It first examines, via the first term C (0) A -z ( x, y ) , whether x and y remain d -connected in the subgraph A -z where conditioning node z is removed - effectively checking for any backdoor path that bypasses z (Figure 1b). If no such backdoor path exists, the formulæ then evaluate whether there exists a valid frontdoor path through z . For this frontdoor path to d-connect x and y when conditioning on z , z must be a collider or a descendant of a collider. This translates to checking whether there is any 0th-order d-connecting path that does not involve z and that connects x to any ancestor of z or z itself, and similarly whether there is such a path connecting y to any ancestor of z or z itself. As illustrated in Figure 1c, this is checking

if the 0th-order d-connecting path x == · · · == a ? and the directed path a ? →··· → z both exist at the same time. The following theorem formalizes the correctness of these logical characterizations.

Theorem 3.2. For any DAG A with d nodes and any three nodes x, y, z ∈ [ d ] that are distinct, x ⊥ ⊥ A y if and only if S (0) A ( x, y ) = 1 , and x ⊥ ⊥ A y | z if and only if S (1) A ( x, y | z ) = 1 . Similarly, x ̸ ⊥ ⊥ A y if and only if C (0) A ( x, y ) = 1 , and x ̸ ⊥ ⊥ A y | z if and only if C (1) A ( x, y | z ) = 1 . 2

The d-separation and d-connection formulæ presented in Theorem 3.2 rely fundamentally on the graph reachability predicate R A . Crucially, we observe that R A itself can be expressed through FOL formulæ that operate directly on the adjacency matrix A , by taking a recursive form analogous to the generalized Bellman-Ford algorithm [3].

Definition 3.3. (Graph Reachability FOL formulæ) Given a discrete directed graph A ∈ { 0 , 1 } d × d , we define the reachability formulæ R ( l ) A ( x, y ) : [ d ] 2 → { 0 , 1 } for path lengths no greater than l recursively as follows:

In practice, we take R A ( x, y ) := R ( d -1) A ( x, y ) where d is the number of nodes, since the longest directed path in a DAG with d nodes has length d -1 . The correctness of these formulæ is articulated in the following lemma:

Lemma 3.4. For any discrete graph A ∈ { 0 , 1 } d × d with maximum directed path length l and for all pair of nodes x, y ∈ [ d ] , x ⇝ A y if and only if R ( l ) A ( x, y ) = 1 .

By integrating the formulæ in Definition 3.1 with those in Definition 3.3, we establish a complete framework that can systematically determine d-separation and d-connection relationships in DAGs using purely FOL operations, laying the foundation for the differentiable relaxation that follows.

3.2 Continuous Relaxation via Soft Logic

Having established d-separation and d-connection as FOL formulæ, we can now leverage the rich literature on probabilistic soft logic (PSL) and fuzzy logic to transform these discrete statements into continuously differentiable functions. These soft logic frameworks systematically relax Boolean operations (conjunction, disjunction, existential and universal quantification) into continuous functions that operate over the interval [0,1] via various t-norms and t-conorms-continuous analogs of logical AND and OR operations [2, 5, 17, 38]. In doing so, the logical semantics necessary to defined reachability (a percolation metric) are preserved while enabling gradient-based optimization and sampling.

In this work, we adopt the LogLTN framework proposed by Badreddine et al. [2], which implements the product t-norm (for logical AND) and max t-conorm (for logical OR) in the logarithmic space. Given a set of m values { x i } i ∈ [ m ] , each x i ∈ [0 , 1] , and their logarithmic representations { x ′ i } i ∈ [ m ] , each x ′ i = log( x i ) , the standard product t-norm and max t-conorm T m , O m : [0 , 1] m → [0 , 1] , and LogLTN's logarithmic versions ˜ T m , ˜ O m : R m -→ R -, where R -:= { x ∈ R | x ≤ 0 } , are:

where C = max( x ′ 1 /α, . . . , x ′ m /α ) and α ∈ (0 , 1] is the temperature controlling the approximation accuracy (the closer α is to 0 the more accurate). Notably, the logarithmic t-norm ˜ T m exactly

represents its standard counterpart, i.e., ˜ T m ( · ) = log( T m ( · )) , whereas the logarithmic t-conorm ˜ O m implements the Log-Mean-Exponential operation that provides a lower bound approximation of log( O m ( · )) with bounded error [2]:

Our choice of the LogLTN operators is motivated by two key considerations. First, a comparative analysis by van Krieken et al. [38] shows that the product t-norm provides the most stable and informative gradients for differentiable learning, while Badreddine et al. [2] shows that the LogMeanSum approximation of the max t-conorm in the logarithmic space further enhances gradient stability. Second, as we will demonstrate next, this specific combination of max-product operators consistently yields principled lower bounds on the quantities being estimated and allows us to derive continuously differentiable functions, which we name the differentiable d-separation/d-connection , that compute lower bounds on expected d-separation/d-connection statements over a distribution of DAGs parameters by the weighted adjacency matrix W . Specifically, we now transform the FOL formulæ in Definition 3.1 and Definition 3.3 into their continuous counterparts that operate on the weighted adjacency matrices W ∈ [0 , 1] d × d , using the LogLTN logical operators.

Definition 3.5. (Differentiable Graph Reachability and Unreachability Percolations) Given a weighted adjacency matrix W ∈ [0 , 1] d × d , we define the relaxed reachability and unreachability functions, ˜ R ( l ) W ( x, y ) , ˜ U ( l ) W ( x, y ) : [ d ] 2 → R -, for path lengths no greater than l recursively as follows:

̸

where we take log(0) = -∞ .

Definition 3.6. (Differentiable d-Separation and d-Connection) Given a weighted adjacency matrix W ∈ [0 , 1] d × d , we define the 0th-order differentiable d-separation and d-connection function ˜ S (0) W ( x, y ) , ˜ C (0) W ( x, y ) : [ d ] 2 → R -and the 1st-order differentiable d-separation and d-connection function ˜ S (1) W ( x, y | z ) , ˜ C (1) W ( x, y | z ) : [ d ] 3 → R -as follows:

Notably, for the d-separation functions relying on graph un reachability, we directly derive unreachability from the negation of reachability FOL formulæ rather than computing log(1 -exp( ˜ R ( l ) W ( x, y ))) . This is essential for preserving the correct probabilistic interpretation, as we now demonstrate.

Consider W as representing a parametric distribution over discrete graphs A . Namely, to sample A , we sample each edge x → y independently via A xy ∼ Bern ( W xy ) (i.e. Bernoulli distribution). We denote the entire graph's sampling as A ∼ Bern ( W ) . We first notice that the differentiable graph reachability and graph unreachability functions on W given in Definition 3.5 always yield a lower bound to the expected reachability and unreachability on A respectively, when A ∼ Bern ( W ) . Intuitively, this is because the max operator provides a lower bound on the probability of a union of events (logical OR), while the product operator correctly computes the probability of an intersection of events (logical AND) when said events are mutually independent or provides a lower bound when they are positively correlated, which is precisely the case for overlapping paths in graphs needed in reachability (a percolation metric). As these operators compose recursively in our reachability computation, they preserve the lower bound properties, leading to the following lemma.

Lemma 3.7 (Reachability Percolation Lower Bound) . Given a weighted adjacency matrix W ∈ [0 , 1] d × d , for any 0 ≤ l < d , and for any pair of nodes x, y ∈ [ d ] , we have

The lower bound property of both our reachability and unreachability functions is critical-note that naively computing unreachability as log(1 -exp( ˜ R ( l ) W ( x, y ))) would yield an upper bound instead. This consistency in providing lower bounds is essential for extending our theoretical guarantees to the differentiable d-separation and d-connection functions. As the following lemma demonstrates, these functions also preserve the lower bound property relative to their expected values:

Theorem 3.8 (Lower Bound on Expected d -Separation Statements) . Given a weighted adjacency matrix W ∈ [0 , 1] d × d , for any three nodes x, y, z ∈ [ d ] ,

These lemmas establish that our differentiable functions provide lower bounds on the logarithm of expected d-connection and d-separation statements when viewing W as parameterizing a distribution over discrete graphs. This mathematical guarantee opens a principled pathway for gradient-based optimization by maximizing these lower bounds.

4 DAGPA (DAG Percolation Apartness)

We now introduce DAGPA, a practical instantiation to demonstrate the validity of our differentiable d -separation scores (Section 3.2). This implementation employs certain heuristics and simplifications to facilitate optimization. The primary goal is not to claim DAGPA's optimality, but rather to provide empirical validation of the framework's potential for causal discovery and open avenues for future research exploring more sophisticated realizations of the core principles.

In DAGPA, we parameterize the weighted adjacency matrix W with parameters θ ∈ R d × d via the sigmoid transformation W = σ ( θ ) . We develop objective functions that encourage alignment between the d-separation/d-connection statements implied by the graph structure W and conditional independence patterns in data. Rather than using binary decisions based on thresholded p-values, DAGPA directly incorporates raw p-values from statistical independence tests as soft CI labels from data. This approach makes the model naturally robust to uncertainty in CI testing, as gradient-based optimization can prioritize clear cases (p-values near 0 indicating dependence or near 1 indicating independence) over borderline ones.

Our objective functions consists of three components: Acyclicity, "True Positive," and "True Negative" losses 3 . A true positive occurs when the model predicts a high probability of d-separation for variable pairs showing high independence p-values in the data. Conversely, a true negative occurs when the model predicts a high probability of d-connection for pairs showing low p-values (strong dependence).

̸

Definition 4.1 (Multi-task Constraint-based CI Statement Losses of DAGPA) . Given a dataset D with n samples and d variables, and the p-values p D ( · ) of some statistical conditional independence tests on all the 0th-order and 1st-order statements. Let I 0 = { ( x, y ) | x, y ∈ [ d ] , x > y } and I 1 = { ( x, y, z ) | x, y, z ∈ [ d ] , x > y, x = z, y = z } . The CI Statement 'true positive (TP)' losses L TP-0 and L TP-1, the 'true negative (TN)' losses L TN-0 and L TN-1, as well as the log-determinant DAG acyclicity loss L DAG from DAGMA [4], defined on the model parameters θ ∈ R d × d , are as follows:

̸

where M 0 = max I 0 p D ( x, y ) , M 1 = max I 1 p D ( x, y | z ) , σ is the sigmoid function, and s is the hyperparameter controlling the domain of the log-determinant DAG loss of DAGMA [4].

We note that, assuming the p-values accurately reflect the true CI patterns, minimizing the 'TP' and 'TN' losses maximizes ˜ S and ˜ C on the correct queries. Furthermore, since ˜ S and ˜ C are lower bounds of the expected d-separation and d-connection statements over Bern ( W ) (Theorem 3.8), we are maximizing lower bounds - the correct optimization direction. The following lemmas show the consistency of the losses and the validity of taking sum aggregation over individual CI queries.

Match?

Model's Predicted DAG

Ground Truth DAG

0th- and 1st-order

d-Separation Statements

Number of

A

B

C

D

A

B

C

D

Better

Better

1.0

0.8

0.6

0.4

0.2

0.0

0.2

0.3

0.9

1.0

0.4

0.5

0.6

0.7

0.8

Low-order CI Statement MCC (CI-MCC)

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

(k=1) kPC

(k=2) kPC

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

Lemma 4.2 (Consistency of Multi-Task CI Losses) . Given a faithful data D , as the sample size n →∞ and the LogMeanExp temperature α → 0 + (Equation (8) ), the optimal DAG A ∗ achieves the minimum value on each of L TP-0 , L TP-1 , L TN-0 , L TN-1, and L DAG.

̸

Lemma4.3 (Mutual Independence of Low-order CI Statements) . All 0th- and 1st-order CI statements over the variable sets I 0 = { ( x, y ) | x, y ∈ [ d ] , x > y } and I 1 = { ( x, y, z ) | x, y, z ∈ [ d ] , x > y, x = z, y = z } are mutually independent. In other words, none of these CI statements can be implied from any other of these CI statements via the graphoid axioms [24].

̸

In Appendix C, we provide the complete algorithm for DAGPA along with detailed descriptions of additional heuristics employed to address optimization challenges, including: PCGrad [43] for resolving conflicting gradients in multi-task learning, Discrete Langevin Proposal (DLP) [50] for efficient exploration of the weight space, and an optimization-time DAG selection score based on CI statement measures from the data.

5 Experiments

This section presents an empirical evaluation of DAGPA's ability to discover DAGs whose dseparation statements are consistent with those derived from the underlying causal structure that generated the data. Our objective is not to test superiority over baselines in terms of exact structural recovery (that is left to Appendix F), but rather to demonstrate that DAGPA successfully achieves its intended purpose: learning DAGs that entail the same low-order conditional independence statements as the ground truth. The experimental results indicate that our differentiable d-separation framework offers a promising direction for causal discovery.

Evaluation. To evaluate causal discovery from a CI statement alignment perspective, we propose a metric that aligns with this objective: the Low-order CI Statement Matthews Correlation Coefficient (CI-MCC) , illustrated in Figure 2. Given a model's predicted causal graph (which may be a DAG, PDAG, CPDAG, or k-essential graph), we derive the set of all 0th- and 1st-order d-separation statements implied by the structure. We then compare these against the corresponding d-separation statements derived from the ground-truth DAG that generated the data. Treating this as a binary classification task, we compute the Matthews Correlation Coefficient, which ranges from [ -1 , 1] with higher values indicating better alignment between model-predicted and groundtruth CI statements. We select MCC over alternatives like F1 score due to its robustness to class imbalance-a critical consideration since the true ratio of independence to dependence relationships varies widely across different causal structures and is typically unknown in real-world settings. For completeness, results using conventional causal discovery metrics, such as graph structure F1 and Structural Hamming Distance (SHD) scores, are provided in Appendix F.

Baselines. We compare DAGPA with representative causal discovery methods spanning different paradigms: constraint-based approaches including PC [34] and k-PC [18]; score-based methods such

Better

Better

1.0

0.8

0.6

0.4

0.2

0.0

0.2

0.7

0.3

0.4

0.5

0.6

Low-order CI Statement MCC (CI-MCC)

Empirical CDF

DAGMA (Linear)

DAGMA (Nonlinear)

GES

(k=1) kPC

(k=2) kPC

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(a) ER

d

= 10

, r

= 2

, n

= 100

Better

Better

1.0

0.8

0.6

0.4

0.2

0.0

0.2

1.0

0.4

0.6

0.8

Low-order CI Statement MCC (CI-MCC)

Empirical CDF

DAGMA (Linear)

DAGMA (Nonlinear)

GES

(k=1) kPC

(k=2) kPC

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(b) ER

d

= 10

, r

= 2

, n

= 10000

as GES [6]; and continuous optimization approaches including NOTEARS (linear and nonlinear) [36] and DAGMA (linear and nonlinear) [4]. This selection provides a comprehensive comparison landscape across methodological families. Detailed baseline configurations are provided in Appendix D.5.

Experimental Settings. We evaluate methods on both synthetic and real-world datasets. For synthetic data, we generate binary data requiring no normalization. We follow a protocol similar to Kocaoglu [18], generating binary datasets from randomly constructed DAGs. We use both Erd˝ os-Rényi (ER) and Scale-Free (SF) graph models via the pyAgrum package [10], with varying complexity parameters: nodes d ∈ { 10 , 50 } , edge-to-node ratio r ∈ { 2 , 4 } , and sample sizes n ∈ { 100 , 1000 , 10000 , 100000 } . For each configuration, we generate 10 datasets. All constraintbased methods employ Fisher-z tests [11] (with potential of other CI tests as alternative; details in appendix), for computing conditional independence p-values, ensuring a fair comparison. For real-world validation, we use the Sachs dataset [28], a benchmark protein signaling network with 11 variables and a known ground-truth causal structure derived from experimental interventions. This dataset represents a challenging real-world case where the underlying causal mechanisms are likely nonlinear and complex. We employ the standard discretized version preprocessed via the Hartemink discretization method, which converts continuous protein concentrations into 3-level categorical variables (low, average, high) while preserving dependence structure.

We note that our CI-constraint-based approach offers inherent robustness to preprocessing artifacts that can affect score-based methods: standard CI tests (e.g. Chi-squared for discrete data, Fisher-Z for continuous data) are invariant to scaling transformations, whereas likelihood-based objectives can exploit variance differences as spurious causal signals [27]. This makes our framework particularly robust across different preprocessing pipelines.

Results on Sachs. To evaluate performance on real-world data with potentially complex underlying mechanisms, we test DAGPA on the Sachs dataset [28], a benchmark protein signaling network with 11 variables and 853 samples. Figure 3 displays the empirical CDF of CI-MCC scores across methods. DAGPA demonstrates dramatically superior performance, achieving CI-MCC scores of 0.75-0.98, while all baseline methods cluster between 0.3-0.65. For our method, we select the 10 DAGs with highest DAG selection scores to reflect the uncertainty inherent in causal discovery from real data. For baselines that depend on random seeds, we run each method with 10 different initializations to ensure fair comparison. The substantial performance gap highlights DAGPA's ability to effectively discover causal structures in real-world data where the underlying data-generating process is complex and potentially nonlinear. This confirms that our differentiable d-separation framework, combined with gradient-informed discrete sampling, provides robust causal discovery capabilities that generalize well beyond synthetic settings to real-world applications where ground truth is available but unknown during learning.

Results on synthetic binary data. Figure 4 presents the empirical cumulative distribution functions (CDFs) of CI-MCC scores across methods for different sample sizes. For each baseline, we include

one result per data instance in the batch (10 results total), while for DAGPA, we select the 5 discovered DAGs with the highest DAG selection score (Appendix C.3) per instance, yielding 50 total evaluations.

In low-sample settings (n=100, Figure 4a), DAGPA demonstrates superior performance compared to most baselines and performs on-par with DAGMA-Nonlinear [4], with CDF curves positioned toward the bottom-right indicating consistently higher CI-MCC scores. The advantage of DAGPA is particularly pronounced when compared to constraint-based methods (PC, k-PC), which struggle in low-sample regimes due to their reliance on hard conditional independence statements derived from uncertain p-values. Most score-based methods, including DAGMA-linear and both NOTEARS variants [36], also underperform in this setting, likely due to overfitting to limited data.

As sample size increases to n=10000 (Figure 4b), most methods improve as expected, with DAGPA maintaining competitive performance alongside constraint-based approaches. This pattern holds consistently across different graph structures and dimensions (see Appendix F), confirming that DAGPA effectively addresses RQ1 by exhibiting strong robustness in low-sample settings while maintaining competitive performance when data is abundant.

6 Related work

Constraint-based Methods. These methods discover causal structures through conditional independence (CI) testing. The PC algorithm [33, 34] produces CPDAGs representing Markov equivalence classes but struggles with small samples due to unreliable high-order CI tests. Recent advances like LOCI [42] and k-PC [18] demonstrate that low-order CI statements can sufficiently identify causal structures in many practical settings. Our work builds on these insights but uniquely transforms discrete CI decisions into a differentiable learning framework, further enhancing robustness in the small sample regime.

Score-based Methods. These methods optimize a score function over the DAG space. Traditional approaches like GES [6] use greedy search with information-theoretic criteria but face scalability challenges due to the discrete DAG space. NOTEARS [51] introduced a breakthrough with differentiable acyclicity characterization, enabling gradient-based optimization on weighted adjacency matrices and spurring extensions like nonlinear variants [52] and improved acyclicity constraints in DAGMA [4]. Our framework builds upon these continuous optimization advances but redirects the objective toward maximizing agreement with CI constraints rather than fitting functional models. This creates a hybrid approach combining strengths from both paradigms and opening a new avenue for causal discovery.

A more comprehensive discussion on related work can be found in Appendix E.

7 Conclusions

We introduced a novel causal discovery framework that bridges constraint-based and gradient-based continuous-optimization methods through differentiable d-separation. By proposing percolationbased differentiable d-separation scores, we enabled gradient-based learning of causal DAG structures from conditional independence patterns in finite datasets. Our model instantiation, DAGPA, demonstrates competitive performance across synthetic and real-world settings, with strong robustness in low-sample regimes where traditional methods often struggle, while maintaining competitive performance as data becomes abundant.

Limitations and Future Work. Several limitations in our current approach present opportunities for future research. The reliance of DAGPA on p-values to measure both conditional dependence and independence from data can be improved, as using a constant minus the p-value to assess dependence strength is a simple heuristic. Fortunately, our framework is measurement-agnostic and supports alternatives, which we recommend future work to investigate. In addition, our method focuses on only low-order conditioning sets and assumes no confounding variables in the data. Future directions include extending the framework to handle latent confounders, efficiently incorporating higher-order CI statements, and simultaneously improving scalability. Finally, as suggested by a reviewer, a particularly promising avenue is integrating our differentiable d-separation framework with score-based methods like NOTEARS and DAGMA, enabling causal structure learning from both data likelihood signals and CI statement fitness signals (case studies in Appendix G.1). These advancements would further bridge constraint-based and score-based paradigms in causal discovery.

Acknowledgment

The authors thank Ruqi Zhang and Patrick Pynadath for the insightful discussions and valuable feedback on Bayesian sampling methods. This work was funded in part by the Ford Motor Company under the Purdue-Ford Alliance, the National Science Foundation (NSF) awards CAREER IIS-1943364, CNS-2212160 and CAREER 2239375, IIS-2348717, Amazon Research Award, AnalytiXIN, Adobe Research, Intuit, the Wabash Heartland Innovation Network (WHIN), Walther Cancer Foundation and Purdue Institute for Cancer Research (PICR) (Grant P30CA023168). Computing infrastructure supported in part by AMD under the AMD HPC Fund program and PICR. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.

References

Yashas Annadani, Nick Pawlowski, Joel Jennings, Stefan Bauer, Cheng Zhang, and Wenbo Gong. Bayesdag: Gradient-based posterior inference for causal discovery. Advances in Neural Information Processing Systems , 36:1738-1763, 2023.

Samy Badreddine, Luciano Serafini, and Michael Spranger. logltn: Differentiable fuzzy logic in the logarithm space. ArXiv , abs/2306.14546, 2023.

John S. Baras and George Theodorakopoulos. Path problems in networks. Synthesis Lectures on Communication Networks , 3:1-77, 2010.

Kevin Bello, Bryon Aragam, and Pradeep Ravikumar. Dagma: Learning dags via m-matrices and a log-determinant acyclicity characterization. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 8226-8239. Curran Associates, Inc., 2022.

Matthias Bröcheler, Lilyana Mihalkova, and Lise Getoor. Probabilistic similarity logic. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence , pages 73-82, 2010.

David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research , 3(Nov):507-554, 2002.

Gabor Csardi and Tamas Nepusz. The igraph software package for complex network research. InterJournal , Complex Systems:1695, 11 2005.

Adnan Darwiche. Modeling and reasoning with Bayesian networks . Cambridge university press, 2009.

Iddo Drori, Anant Kharkar, William R Sickinger, Brandon Kates, Qiang Ma, Suwen Ge, Eden Dolev, Brenda Dietrich, David P Williamson, and Madeleine Udell. Learning to solve combinatorial optimization problems on real-world graphs in linear time. In 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA) , pages 19-24. IEEE, 2020.

Gaspard Ducamp, Christophe Gonzales, and Pierre-Henri Wuillemin. aGrUM/pyAgrum : a Toolbox to Build Models and Algorithms for Probabilistic Graphical Models in Python. In 10th International Conference on Probabilistic Graphical Models , volume 138 of Proceedings of Machine Learning Research , pages 609-612, Skørping, Denmark, September 2020.

Ronald A Fisher. Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika , 10(4):507-521, 1915.

HL Frisch and JM Hammersley. Percolation processes and related topics. Journal of the society for industrial and applied mathematics , 11(4):894-918, 1963.

Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graphical models. Frontiers in genetics , 10:524, 2019.

Isabelle Guyon, Constantin Aliferis, Gregory Cooper, André Elisseeff, Jean Philippe Pellet, Peter Spirtes, and Alexander Statnikov. Causality workbench. In Causality in the sciences . Oxford University Press, 2011.

WKeith Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika , 57(1):97-109, 1970.

Barry D Hughes. Random walks and random environments . Oxford University Press, 1996.

George Klir and Bo Yuan. Fuzzy sets and fuzzy logic , volume 4. Prentice hall New Jersey, 1995.

Murat Kocaoglu. Characterization and learning of causal graphs with small conditioning sets. Advances in Neural Information Processing Systems , 36, 2023.

Daphne Koller and Nir Friedman. Probabilistic Graphical Models: Principles and Techniques . The MIT Press, 2009. ISBN 0262013193.

Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems , 34:18878-18890, 2021.

Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. Equation of state calculations by fast computing machines. The journal of chemical physics , 21(6):1087-1092, 1953.

Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi-task learning as a bargaining game. In International Conference on Machine Learning , pages 16428-16446. PMLR, 2022.

Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and dag constraints for learning linear dags. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 17943-17954. Curran Associates, Inc., 2020.

Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference . Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1988. ISBN 1558604790.

Judea Pearl. Causality: Models, Reasoning and Inference . Cambridge University Press, USA, 2nd edition, 2009. ISBN 052189560X.

Judea Pearl and James M. Robins. Probabilistic evaluation of sequential plans from causal models with hidden variables. In Conference on Uncertainty in Artificial Intelligence , 1995.

Alexander Reisach, Christof Seiler, and Sebastian Weichwald. Beware of the simulated dag! causal discovery benchmarks may be easy to game. Advances in Neural Information Processing Systems , 34:27772-27784, 2021.

Karen Sachs, Omar Perez, Dana Pe'er, Douglas A Lauffenburger, and Garry P Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science , 308(5721): 523-529, 2005.

Gideon Schwarz. Estimating the dimension of a model. The Annals of Statistics , 6(2):461-464, 1978.

Marco Scutari and Jean-Baptiste Denis. Bayesian Networks with Examples in R . Chapman and Hall, Boca Raton, 2nd edition, 2021. ISBN 978-0367366513.

Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural information processing systems , 31, 2018.

Ross D Shachter. Bayes-ball: Rational pastime (for determining irrelevance and requisite information in belief networks and influence diagrams). In Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence , pages 480-487, 1998.

Peter Spirtes and Clark Glymour. An algorithm for fast recovery of sparse causal graphs. Social science computer review , 9(1):62-72, 1991.

Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search . MIT press, 2000.

Eric V Strobl, Kun Zhang, and Shyam Visweswaran. Approximate kernel-based conditional independence tests for fast non-parametric causal discovery. Journal of Causal Inference , 7(1): 20180017, 2019.

Xiangyuan Sun, Oliver Schulte, Guiliang Liu, and Pascal Poupart. Nts-notears: Learning nonparametric dbns with prior knowledge. In International Conference on Artificial Intelligence and Statistics , 2021.

Ioannis Tsamardinos, Laura E Brown, and Constantin F Aliferis. The max-min hill-climbing bayesian network structure learning algorithm. Machine learning , 65:31-78, 2006.

Emile van Krieken, Erman Acar, and Frank van Harmelen. Analyzing differentiable fuzzy logic operators. Artificial Intelligence , 302:103602, 2022.

Matthew J Vowels, Necati Cihan Camgoz, and Richard Bowden. D'ya like dags? a survey on structure learning and causal discovery. ACM Computing Surveys , 55(4):1-36, 2022.

Zirui Wang and Yulia Tsvetkov. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In Proceedings of the International Conference on Learning Representations (ICLR) , 2021.

Dennis Wei, Tian Gao, and Yue Yu. Dags with no fears: A closer look at continuous optimization for learning bayesian networks. Advances in Neural Information Processing Systems , 33:38953906, 2020.

Marcel Wienöbst and Maciej Liskiewicz. Recovering causal structures from low-order conditional independencies. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 10302-10309, 2020.

Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems , 33:5824-5836, 2020.

Alessio Zanga, Elif Ozkirimli, and Fabio Stella. A survey on causal discovery: Theory and practice. International Journal of Approximate Reasoning , 151:101-129, 2022.

Bin Zhang, Chris Gaiteri, Liviu-Gabriel Bodea, Zhi Wang, Joshua McElwee, Alexei A Podtelezhnikov, Chunsheng Zhang, Tao Xie, Linh Tran, Radu Dobrin, et al. Integrated systems approach identifies genetic nodes and networks in late-onset alzheimer's disease. Cell , 153(3):707-720, 2013.

Jiji Zhang. Causal inference and reasoning in causally insufficient systems. Philosophy of Science , 75(5):960-970, 2008.

Junzhe Zhang and Elias Bareinboim. Fairness in decision-making - the causal explanation formula. In AAAI Conference on Artificial Intelligence , 2018.

Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. In Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI) , pages 804-813, 2012.

Kun Zhang, Bernhard Schölkopf, Krikamol Muandet, and Zhikun Wang. Domain adaptation under target and conditional shift. In International conference on machine learning , pages 819-827. Pmlr, 2013.

Ruqi Zhang, Xingchao Liu, and Qiang Liu. A langevin-like sampler for discrete distributions. In International Conference on Machine Learning , pages 26375-26396. PMLR, 2022.

Xun Zheng, Bryon Aragam, Pradeep K Ravikumar, and Eric P Xing. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems , 31, 2018.

Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, and Eric Xing. Learning sparse nonparametric dags. In International Conference on Artificial Intelligence and Statistics , pages 3414-3425. PMLR, 2020.

Yujia Zheng, Biwei Huang, Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang. Causal-learn: Causal discovery in python. Journal of Machine Learning Research , 25(60):1-8, 2024.

A Proofs

Definition 3.1. (Low-order d-separation and d-connection FOL formulæ) Given a discrete directed graph A ∈ { 0 , 1 } d × d , the 0th-order d-separation formula S (0) A : [ d ] 2 → { 0 , 1 } and the 1st-order d-separation formula S (1) A : [ d ] 3 →{ 0 , 1 } are defined as:

Equivalently, the 0-th and 1-st order d-connection statements C (0) A and C (1) A are:

Theorem 3.2. For any DAG A with d nodes and any three nodes x, y, z ∈ [ d ] that are distinct, x ⊥ ⊥ A y if and only if S (0) A ( x, y ) = 1 , and x ⊥ ⊥ A y | z if and only if S (1) A ( x, y | z ) = 1 . Similarly, x ̸ ⊥ ⊥ A y if and only if C (0) A ( x, y ) = 1 , and x ̸ ⊥ ⊥ A y | z if and only if C (1) A ( x, y | z ) = 1 . 4

Proof. First, we note that the d-separation formulæ S (0) A , S (1) A and the d-connection formulæ C (0) A , C (1) A are the negation of each other and can be obtained from each other via De Morgan's law. Thus, the statement ' x ⊥ ⊥ A y if and only if S (0) A ( x, y ) = 1 ' is equivalent to ' x ̸ ⊥ ⊥ A y if and only if C (0) A ( x, y ) = 1 ', and vice versa, the statement ' x ⊥ ⊥ A y | z if and only if S (1) A ( x, y | z ) = 1 ' is equivalent to ' x ̸ ⊥ ⊥ A y | z if and only if C (1) A ( x, y | z ) = 1 .' Thus, we proceed to prove the statements involving the d-connection formulæ C (0) A , C (1) A .

Part 1: x ̸ ⊥ ⊥ A y if and only if C (0) A ( x, y ) = 1 .

We show the direction x ̸ ⊥ ⊥ A y = ⇒ C (0) A ( x, y ) = 1 .

If x ̸ ⊥ ⊥ A y , i.e., x is d-connected to y with an empty conditioning set, then there exists a path between x and y that does not contain any colliders, because otherwise the path would be blocked. Denote the sequence of nodes in this path p = ( p 1 , . . . , p m ) with p 1 = x and p m = y . In other words, for any three consecutive nodes a, b, c ∈ p , we have either the chain structure a → b → c or a ← b ← c , or the fork structure a ← b → c .

Because there is no collider structure in p , we will show that there must be at most one fork structure in p . To see this by contradiction, consider if p contains more than one fork, and let p l -2 ← p l -1 → p l and p r ← p r +1 → p r +2 be two forks ( l < r ) in p such that there is no other fork between p l and p r . Then, either there are some colliders between p l and p r , or p l and p r are connected by some directed paths, i.e., either p l ⇝ p r or p r ⇝ p l . If p l ⇝ p r , then p r is a collider, and vice versa if p r ⇝ p l , then p l is a collider. Hence, in all scenarios, p will have at least one collider, contradicting the fact that p should not have any collider.

Now, we discuss case by case based on whether p has no fork, or it has one fork. Figure 1a illustrates the structure such a path p .

Case 1: Suppose there is no fork on p : If there is no fork at all, then p is a directed path. Without loss of generality, assume p = x →··· → y . Then, let a = x , we have a ⇝ x (a node is always

naively reachable from itself) and a ⇝ y . Hence, ( R A ( a, x ) ∧ R A ( a, y )) = 1 , and therefore the formulæ C (0) A ( x, y ) = ∃ a ∈ [ d ] , R A ( a, x ) ∧ R A ( a, y ) is 1.

Case 2: Suppose there is one fork on p When there is one and only one fork, it means the center node of the fork is the common ancestor of both x and y . In other words, let a be this center node, then we have a ⇝ x and a ⇝ y . Thus, similarly, we have ( R A ( a, x ) ∧ R A ( a, y )) = 1 , and therefore the formulæ C (0) A ( x, y ) = ∃ a ∈ [ d ] , R A ( a, x ) ∧ R A ( a, y ) is 1.

Thus, we have shown that x ̸ ⊥ ⊥ A y = ⇒ C (0) A ( x, y ) = 1

Now we show the other direction C (0) A ( x, y ) = 1 = ⇒ x ̸ ⊥ ⊥ A y .

Given C (0) A ( x, y ) = 1 , we have ∃ a ∈ [ d ] such that a ⇝ x and a ⇝ y . Let a be such a node. Thus, a ∗ is a common ancestor of x and y . Let p x = a →··· → x be the directed path that starts at a and ends at x , and similarly let p y = a →··· → y . Then, the path form by the sequence of nodes in p x and p y , i.e., p = ( x, . . . , a, . . . , y ) , is clearly a path consisting of at most one fork structure with other wise chain structures. When a = x or a = y , p is a directed path without fork, and otherwise has one fork. In either case, p is a d-connecting path between x and y , making x ̸ ⊥ ⊥ A y .

We show the direction x ̸ ⊥ ⊥ A y | z = ⇒ C (1) A ( x, y | z ) = 1 .

We are given x ̸ ⊥ ⊥ A y | z , i.e., x and y are d-connecting when conditioning on z . Thus, there must exist a non-empty set of d-connecting paths P between x and y that is not blocked given z . We proceed with a case-by-case discussion depending on whether all of these paths contain colliders.

First, we consider the case where there is some path in P that does not contain any collider. Let p ∈ P be such a path. Then, z must not be on p , because otherwise, since z is not a collider and it is being conditioned, p would be blocked and therefore p ̸∈ P .

Since p does not include z , it remains unchanged in the subgraph A -z , in which node z is removed from the graph A , and thus it is still a d-connecting path in A -z . Hence, because of p , we have x and y are d-connecting in the subgraph A -z . In other words, we have C (0) A -z ( x, y ) =

1 . Since C (0) A -z ( x, y ) is the one of two terms from the outmost disjunction operator ∨ in the formula for C (1) A ( x, y | z ) , we have C (0) A -z ( x, y ) = 1 renders C (1) A ( x, y | z ) = 1 . In Figure 1b, this pattern is illustrated via the 'backdoor path' structure.

Otherwise, all paths in P contain colliders. We first show that this implies that we can find a path p ∗ in P that has only one collider .

To see this, pick any path p ∈ P . If p indeed has only one collider, then we are done. Hence, we consider the non-trivial case where p has more than one collider. We further divide the cases depending on whether node z is included in path p .

(a) If z is included in p , then z must itself be a collider, because otherwise p would be blocked by z .

Now, let z ′ be another collider in p . Then, z must be a descendant of z ′ , because otherwise p would be blocked by z ′ even though z is in the conditioning set. In other words, z is reachable from z ′ with a directed path, i.e., z ′ ⇝ z .

Using this knowledge, we can construct the following alternative path p z ′ that still has z as a collider but converts z ′ to a non-collider. Specifically, without loss of generality, we assume the path p consists of the following sequence of nodes: p = ( x, . . . , p l ′ , z ′ , p r ′ , . . . , p l , z, p r , . . . , y ) for some indices l, r, l ′ , r ′ with collider structures p l ′ → z ′ ← p r ′ and p l → z ← p r . We have shown that z ′ ⇝ z , and denote the corresponding directed path z ′ → q 1 → ··· → q m → z . We can obtain the new path as follows:

where we replace the middle section sub-path ( z ′ , p r ′ , . . . , p l , z ) in p to ( z ′ , q 1 , . . . , q m , z ) to form p z ′ .

Now, note that in p z ′ , z ′ is no longer a collider but forms a chain structure, because p l ′ → z ′ → q 1 . Nevertheless, z is still a collider, because q m → z ← p r . Furthermore, p z ′ is still a d-connecting path. This is because the sub-paths ( x, . . . , p l ′ , z ′ ) and ( z, p r , . . . , y ) are d-connecting conditioning on z because they are parts of the original p which is a d-connecting path, and the 'new' sub-path ( z ′ , q 1 , . . . , q m , z ) by definition is d-connecting conditioning on z because it is a directed path. Hence, taken together, p z ′ is d-connecting given z , so p z ′ ∈ P .

Thus, by iteratively constructing these alternative paths p z ′ for any colliders z ′ in p that is not z , we can arrive at a path p ∗ ∈ P that has z as the only collider.

(b) Now, suppose z is not included in p and p has more than one collider. Let z ′ and z ′′ be any two such colliders, and without loss of generality, let the path consists of the sequence of nodes p = ( x, . . . , p l ′ , z ′ , p r ′ , . . . , p l ′′ , z ′′ , p r ′′ , . . . , y ) . Similarly, both z ′ and z ′′ must be ancestors of z , because otherwise one of them would block the path. Hence, we know that z ′ ⇝ z and z ′′ ⇝ z . Let the directed paths be z ′ → q 1 → ··· → q m → z and z ′′ → o 1 →··· → o s → z respectively. We can construct the following new path:

Here, note that both z ′ and z ′′ are no longer a collider, because p l ′ → z ′ → q 1 and o 1 ← z ′′ ← p r . Nevertheless, z is a collider since q m → z ← o s . Furthermore, p z ′ ,z ′′ is still a d-connecting path conditioned on z . As we have established in the discussion of the previous case, that the sub-paths ( x, . . . , p l ′ , z ′ ) and ( z ′′ , p r , . . . , y ) are d-connecting. Since ( z ′ , q 1 , . . . , q m , z ) and ( z, o s , . . . , o 1 , z ′′ ) are directed paths, we have that the subpaths ( x, . . . , q m , z ) and ( z, o s , . . . , y ) are also d-connecting. Taken together, the whole path is d-connecting given z , i.e., p z ′ ,z ′′ ∈ P .

In other words, we have converted both colliders z ′ and z ′′ in the original p into noncolliders while introducing z as a new collider. This means that the resulting path p z ′ ,z ′′ satisfies the initial condition of a path p discussed in the previous case. Hence, by further following the procedure described in the previous case, we can eventually arrive at a path p ∗ ∈ P that has z as the only collider.

Now that we know P will necessarily include a path p ∗ with only one collider, denote this collider z ∗ , and we know either z ∗ = z or z ∗ is an ancestor of z . In other words, we have z ∗ ⇝ z . Hence, we can denote p ∗ as a sequence of nodes: p ∗ = ( x, . . . , p ∗ l , z ∗ , p ∗ r . . . , y ) for some indices l, r , where p ∗ l → z ∗ ← p ∗ r .

Now, let a = x if there is no fork structure in the sub-path ( x, . . . , z ∗ ) of p ∗ , otherwise let a be the rightmost fork in ( x, . . . , z ∗ ) (i.e. there is no other fork in the sub-path ( a, . . . , z ∗ ) of p ∗ ). Then, we have a ⇝ z ∗ because there is neither any fork nor any collider in the sub-path ( a, . . . , z ∗ ) , and we know it has a constituent edge p ∗ l → z ∗ with an edge direction pointing towards z ∗ . Moreover, because z ∗ ⇝ z , we have a ⇝ z , and equivalently R A ( a, z ) .

On the other hand, the sub-path ( x, . . . , a ) is a d-connecting path conditioning on z because its super-path p ∗ ∈ P is a d-connecting path conditioning on z , does not have any collider because z ∗ is the only collider in p ∗ , and does not include z . Hence, in the subgraph A -z where z is removed from the graph A , the sub-path ( x, . . . , a ) remains unchanged and is still a d-connecting path, and in this case without conditioning on z . Therefore, x and a is d-connecting in A -z , i.e., x ̸ ⊥ ⊥ A -z a . Hence, we have found a node a such that ( x ̸ ⊥ ⊥ A -z a ) ∧ ( a ⇝ z ) , or equivalently, we have the following:

Similarly, let b = y if there is no fork structure in the sub-path ( z, . . . , y ) of p ∗ , otherwise let b ∗ be the leftmost fork in ( z, . . . , y ) . We then have also have the following:

Combining everything, we fulfill second term in the outermost disjunction operator in the FOL formula and thus have C (1) A ( x, y | z ) = 1 .

In Figure 1c, this pattern is illustrated as the 'frontdoor path' structure.

Now we show the other direction C (1) A ( x, y | z ) = 1 = ⇒ x ̸ ⊥ ⊥ A y | z .

If C (1) A ( x, y | z ) = 1 , then we have or we have

We discuss it case by case.

If it is the first case, C (0) A -z ( x, y ) = 1 , then there exists a d-connecting path p between x and y in the subgraph A -z . p then does not include z , and must not include any colliders. Hence, back in the original graph A , p is a 'backdoor' path that does not involve z that renders x and y d-connecting. Thus, we have x ̸ ⊥ ⊥ A y and also x ̸ ⊥ ⊥ A y | z .

We now consider the other case. Since the left-hand-side term is a conjunction ( ∧ ), both term in the conjunction must be true, meaning there exists some nodes a and b satisfying either term respectively. Taking a step further, this means that all of the four terms C (0) A -z ( x, a ) , R A ( a, z ) , C (0) A -z ( y, b ) , and R A ( b, z ) - are true.

Using a similar reasoning as mentioned in the previous case, C (0) A -z ( x, a ) implies that we have a d-connecting path between x and a and that x ̸ ⊥ ⊥ A a | z . Similarly, C (0) A -z ( y, b ) implies that y ̸ ⊥ ⊥ A b | z . Furthermore, because R A ( a, z ) = 1 and R A ( b, z ) = 1 , that is, a ⇝ z and b ⇝ z , the path between a and b , formed by the corresponding directed paths that renders a ⇝ z and b ⇝ z respectively, is a d-connecting path conditioning on z , as z serves as the only collider. Hence, we have a ̸ ⊥ ⊥ A b | z .

Hence, by transitivity of d-connection statements, we have that x ̸ ⊥ ⊥ A y | z , finishing the entire proof.

Lemma 3.4. For any discrete graph A ∈ { 0 , 1 } d × d with maximum directed path length l and for all pair of nodes x, y ∈ [ d ] , x ⇝ A y if and only if R ( l ) A ( x, y ) = 1 .

Proof. We prove this lemma by induction on the maximum directed path length l .

̸

Basecase : Suppose the maximum directed path length in A is 0. This means there is no path and A is an empty graph. Thus, for all x ∈ [ d ] , x ⇝ A x and for all x, y ∈ [ d ] with x = z , x ⇝ A y .

̸

Induction : Suppose the statement in the lemma holds true with a maximum directed path length of l -1 . We now prove the statement for l .

We prove the direction x ⇝ A y = ⇒ R ( l ) A ( x, y ) = 1 .

Given that x ⇝ A y , let p be such a directed path starting from x and ending at y . Let u be the second to last node in p with an edge u → y , or equivalently, A u,y = 1 . Then, we also have x ⇝ A u .

Since the path from x to u will have a length smaller than or equal to l -1 , using the induction hypothesis, we have R ( l -1) A ( x, u ) = 1 .

Thus, we have R ( l -1) A ( x, u ) ∧ A u,y = 1 , and consequently ∨ u ∈ d R ( l -1) A ( x, u ) ∧ A u,y = 1 , satisfying the first term in the outmost disjunction in the recursive formula for R ( l ) A , rendering R ( l ) A ( x, y ) = 1 .

Now we prove the other direction R ( l ) ( x, y ) = 1 = ⇒ x ⇝ A y

Given R ( l ) A ( x, y ) = 1 , there are possible scenarios: that ∨ u ∈ d R ( l -1) A ( x, u ) ∧ A u,y = 1 or that R ( l -1) ( x, y ) = 1 . we discuss case by case.

A . A

Suppose ∨ u ∈ d R ( l -1) A ( x, u ) ∧ A u,y = 1 , then there exists at least one node u such that R ( l -1) A ( x, u ) ∧ A u,y = 1 , meaning both R ( l -1) A ( x, u ) = 1 and A u,y = 1 . This means that, by the induction hypothesis, x ⇝ A u with maximum path length l -1 . Furthermore, we have u → y is an directed edge. Thus, there exists a directed path from x to y , i.e. x ⇝ A y with maximum path length l .

Suppose R ( l -1) A ( x, y ) = 1 , then by the inductive hypothesis, x ⇝ A y with a maximum path length l -1 , which also makes the statement true under the setting of a maximum path length of l .

Thus, in either case, we have x ⇝ A y .

The proof of Lemma 3.7 and subsequent Theorem 3.8 requires the following lemma showing the lower-bound nature of LogLTN's [2] t-norm and t-conorm logical operators. We state this lemma and its proof as follows.

Lemma A.1. Given the t-norm and t-conorm operators of LogLTN [2], which we restate as follows,

Let x 1 , x 2 , . . . , x m be m Bernoulli random variables that are mutually independent or positively correlated. Denote their probabilities as p 1 , p 2 , . . . , p m , respectively. Then, we have the following:

Proof. Wefirst show the inequality involving the t-norm ˜ T m . Wefirst observe that ˜ T m is the logarithm of the product t-norm, meaning

Now, since x i 's are mutually independent or correlated, we naturally have ∏ m i =1 p i ≤ P ( x 1 ∧ · · · ∧ x m ) . This is because for any two random variables x i and x j ,

where the last inequality sign holds because Cov ( x i , x j ) ≥ 0 for mutually positively correlated or independent random variables.

Thus, we have

For the t-conorm ˜ O m , the inequality holds because ˜ O m is a lower bound of the logarithm of max, and max is then also a lower bound of the union of random variables. Specifically, as shown in the main text, ˜ O m as the LogMeanExp operation has the following property:

On the other hand, max is in general the lower bound of the probability of the union of random variables,

Thus, taken together, we have

Lemma 3.7 (Reachability Percolation Lower Bound) . Given a weighted adjacency matrix W ∈ [0 , 1] d × d , for any 0 ≤ l < d , and for any pair of nodes x, y ∈ [ d ] , we have

Intuitively, this inequality holds because in a probabilistic graph W , where each edge x → y is a Bernoulli random variable parameterized by W x,y , the paths as random variables are either mutually independent, when they consist of distinct edges, or mutually positively correlated, when they share some common edges. Thus, using the results of Lemma A.1, the t-norm of the path probabilities yields a lower bound to their joint probability, and the t-conorm yields a lower bound to their union probability. Thus, the continuous reachability score ˜ R ( l ) W , consisting of disjunctions and conjunctions of the path and edge probabilities, also gives lower bounds.

More specifically, we can observe this fact via induction. In the base case, we obviously have

In addition, we can obviously see that the random variable R (0) A ( x, u ) is independent of the random variable A u,y for any nodes u ∈ [ d ] .

Then, assuming the inequality holds for a graph with a maximum path length of l -1 , and that the random variable R ( l -1) A ( x, u ) is either independent of or positively correlated with the random variable A u,y for any nodes u ∈ [ d ] , we now aim to show that these two statements also hold for l .

First, we can see that, for any node u ∈ [ d ] ,

from results in Lemma A.1 regarding the t-norm ˜ T 2 and the induction hypothesis that R ( l -1) A ( x, u ) is either independent of or positively correlated with A u,y .

Then, again using the result in Lemma A.1 regarding the t-conorm ˜ O m , we have

Regarding the relationship between R ( l ) A ( x, u ) and A u,y , we can see that the probability of R ( l ) A ( x, u ) increases if the probability of A u,y also increases. This is because in the formula, there is no negation and A u,y makes a possible contribution to the value of R ( l ) A ( x, u ) . Thus, R ( l ) A ( x, u ) and A u,y are positively correlated.

Now, we show for the unreachability inequality, ˜ U ( l ) W ( x, y ) ≤ log E A ∼ Bern ( W ) [ U ( l ) A ( x, y ) ] .

Similarly, we show by induction. In the base case, we have

̸

In addition, we can see that the random variable U (0) A ( x, u ) is independent of the random variable ¬ R (0) A ( x, y ) .

Then, assuming the inequality holds for a graph with a maximum path length of l -1 , and that the random variable U ( l -1) A ( x, u ) is either independent of or positively correlated with the random variable ¬ R (0) A ( x, y ) . We aim to show that these two properties still hold for l .

First, using the result in Lemma A.1 regarding the t-conorm ˜ O m , we have, for all nodes u ∈ [ d ] ,

Then, we note that the random variable ( U ( l -1) A ( x, u ) ∨ ¬ R (0) A ( u, y )) for different u are mutually independent or positively correlated. They are also independent of or positively correlated with U ( l -1) A ( x, y ) . This is similar to the mutual independence or positive correlation among paths, since for these 'non-paths', if they consist of distinct 'non-edges', then they are independent. Otherwise, they are positively correlated, since increasing the probability of the shared 'non-edge' (or decreasing the probability of the shared edge) would simultaneously increase the probabilities of said 'non-paths.' Thus, again using the result in Lemma A.1 regarding the t-norm ˜ T d +1 , we have

Theorem 3.8 (Lower Bound on Expected d -Separation Statements) . Given a weighted adjacency matrix W ∈ [0 , 1] d × d , for any three nodes x, y, z ∈ [ d ] ,

Proof. First, we reiterate that, in the computation of differentiable d-separation ˜ S (0) W and ˜ S (1) W , we use the continuous unreachability score ˜ U ( d ) W directly, rather than taking the negation of the continuous reachability score.

Using a similar reasoning as in the proof of the previous lemma, we first observe that, when treating S (0) A , S (1) A , C (0) A , and C (1) A as random variables when A ∼ Bern ( W ) , all terms in the d-separation/dconnection formulæ (Definition 3.1) are either mutually independent or positively correlated. In particular, this holds true for S (0) A -z ( x, a ) and ¬ R A ( a, z ) for any a ∈ [ d ] , as well as for C (0) A -z ( y, b ) and R A ( b, z ) for any b ∈ [ d ] . Thus, using the results from Lemma A.1, we have the lower bounds as stated in the theorem.

Lemma 4.2 (Consistency of Multi-Task CI Losses) . Given a faithful data D , as the sample size n →∞ and the LogMeanExp temperature α → 0 + (Equation (8) ), the optimal DAG A ∗ achieves the minimum value on each of L TP-0 , L TP-1 , L TN-0 , L TN-1, and L DAG.

Proof. Let θ ∗ ∈ R d × d be the parameter that approximates the optimal binary DAG A ∗ of the given data D via σ ( θ ∗ ) → A ∗ .

When the sample size n → ∞ in a faithful data D , we have p D ( x, y ) → 0 and p D ( x, y | z ) → 0 when x ̸ ⊥ ⊥ A ∗ y and x ̸ ⊥ ⊥ A ∗ y | z , and conversely p D ( x, y ) → 1 and p D ( x, y | z ) → 1 when x ⊥ ⊥ A ∗ y and x ⊥ ⊥ A ∗ y | z for any x, y, z ∈ [ d ] , where A ∗ is the optimal causal graph of D . When the LogMeanExp temperature α → 0 + , we have log max { x i } i m =1 -˜ O m ( { log( x i ) } i m =1 ) → 0 + .

In addition, we note that when given a sequence of binary random variables x 1 , . . . , x m , whose Bernoulli probabilities approach in the limit to either 0 or 1, the product t-norm T m and max tconorm O m over x i 's probabilities approach in the limit to the probability of P ( x 1 ∧ · · · ∧ x m ) and P ( x 1 ∨ · · · ∨ x m ) respectively. This is because in the limit case, the random variables reduce to constant 0's and constant 1's, in which case the product t-norm and t-conorm reduce to the vanilla Boolean operations of conjunction and disjunction. Thus, combining the fact that LogLTN's [2] t-norm ˜ T m is equivalent to T m in the logarithm, and its t-conorm ˜ O m approaches in limit to O m in the logarithm, we have the following:

Using this property, we can see that for the continuous reachability and unreachability, for any given maximum path length l and any nodes x, y ∈ [ d ] , they satisfy

Consequently, for the differentiable d-separation and d-connection scores, for any nodes x, y, z ∈ [ d ] , they satisfy

Furthermore, recall from Theorem 3.2, since A ∗ is a discrete DAG, S (0) A ∗ , S (1) A ∗ , C (0) A ∗ , and C (1) A ∗ reduces exactly to the ground-truth d-separation and d-connection statement values in A ∗ . Thus, combining with the result above, we have that

̸

Therefore, combining with the values of the p-values p D , we have the following value matching between the model's predicted d-separation and d-connection score and the p-values. For all ( x, y ) ∈ I 0 ( I 0 = { ( x, y ) | x, y ∈ [ d ] , x > y } ) and for all ( x, y, z ) ∈ I 1 ( I 1 = { ( x, y, z ) | x, y, z ∈ [ d ] , x > y, x = z, y = z } ),

̸

where M 0 = max I 0 p D ( x, y ) → 1 , and similarly M 1 = max I 1 p D ( x, y | z ) → 1 . In other words, the differentiable d-separation/d-connection scores over the optimal parameter θ ∗ ) and the p-values from the data have zero mismatch.

Thus, the TP and TN losses have the following values:

which are the lowest possible values that these loss functions can achieve, because any other configuration of the values for the differentiable d-separation/d-connection scores would result in a larger loss value due to possible mismatches with the p-values. Thus, the TP and TN losses are consistent.

Finally, since A ∗ is a DAG, the log-det acyclicity loss L DAG also achieves the minimum value, for any value of the hyperparameter s . This property is proved in Bello et al. [4]. Thus, all loss functions in Definition 4.1 are consistent.

̸

Lemma4.3 (Mutual Independence of Low-order CI Statements) . All 0th- and 1st-order CI statements over the variable sets I 0 = { ( x, y ) | x, y ∈ [ d ] , x > y } and I 1 = { ( x, y, z ) | x, y, z ∈ [ d ] , x > y, x = z, y = z } are mutually independent. In other words, none of these CI statements can be implied from any other of these CI statements via the graphoid axioms [24].

̸

Proof. Let X , Y ⊆ [ d ] be any non-empty node sets, and Z , W ⊆ [ d ] be node sets that could be empty. The graphoid axiom [24] states the following five rules describing the relationship and dependencies between different CI statements in a graphoid dependency model:

Symmetry: X ⊥ ⊥ Y | Z ⇐⇒ Y ⊥ ⊥ X | Z

Decomposition: X ⊥ ⊥ Y ∪ W | Z = ⇒ ( X ⊥ ⊥ Y | Z ) ∧ ( X ⊥ ⊥ W | Z )

Weak Union: X ⊥ ⊥ Y ∪ W | Z = ⇒ ( X ⊥ ⊥ Y | Z ∪ W ) ∧ ( X ⊥ ⊥ W | Z ∪ Y )

Contraction: ( X ⊥ ⊥ Y | Z ) ∧ ( X ⊥ ⊥ W | Z ∪ Y ) = ⇒ X ⊥ ⊥ Y ∪ W | Z

Intersection: ( X ⊥ ⊥ Y | Z ∪ W ) ∧ ( X ⊥ ⊥ W | Z ∪ Y ) = ⇒ X ⊥ ⊥ Y ∪ W | Z

We check rule by rule whether any of the low-order CI statements considered in I 0 and I 1 can be implied by any others.

Symmetry: No CI statement in I 0 and I 1 can be inferred from others via the symmetry rule. This is because I 0 and I 1 consists of only asymmetric statements where always x > y .

Decomposition: In the non-trivial case when |Y| ≥ 1 and |W| ≥ 1 , this rule does not apply because its left-hand side (LHS) statement involves nodes set Y ∪ W with cardinality of at least 2. This type of statement is not considered in I 0 or I 1 .

Weak Union: Similarly, in the non-trivial case when |Y| ≥ 1 and |W| ≥ 1 , this rule does not apply because its LHS statement is not included in I 0 or I 1 .

In the non-trivial case when |Y| ≥ 1 and |W| ≥ 1 , the LHS applies, but the right-hand-side (RHS) involve node set Y ∪ W with cardinality of at least 2, which is not included in I 0 or I 1 .

Intersection: Similarly, in the non-trivial case when |Y| ≥ 1 and |W| ≥ 1 , the RHS involve node set Y ∪ W with cardinality of at least 2, which is not included in I 0 or I 1 .

Thus, no CI statements in I 0 and I 1 can be inferred from other statements in I 0 and I 1 via the graphoid axioms. Thus, the CI statements we considered in our approach are mutually independent from the perspective of the graphoid dependency structure, justifying the use of the sum aggregation over these CI statements in the TP and TN losses.

B Further Discussions on Percolation versus Diffusion

The distinction between graph percolation and graph diffusion is fundamental to understanding the probabilistic interpretation of our differentiable d-separation framework. Percolation theory studies graph properties (such as connectivity or reachability) when the graph structure itself is randomized [12, 16]. In contrast, diffusion theory typically assumes a fixed graph structure, with randomness arising from particles making probabilistic traversal decisions across this deterministic environment.

In the context of our weighted adjacency matrix W ∈ R d × d , these perspectives offer different interpretations. From a percolation standpoint, each entry W x,y represents the probability that edge x → y exists in the graph. The reachability question then becomes: what is the probability that a directed path exists from one node to another, considering all possible graph configurations? From a diffusion perspective, W would instead represent transition probabilities on a complete graph, where

0.4

0.2

0.7

0.6

0.5

True Reachability Probability (Percolation):

Max-Product Bellman-Ford is Lower Bound:

at node x , the weights { W x,u } u ∈ [ d ] (possibly normalized) determine the probability distribution of a particle's next position. The key distinction is that percolation randomizes the environment (the graph structure), while diffusion randomizes the trajectory through a fixed environment.

It is therefore most appropriate to frame our problem as a graph percolation problem. Given a weighted adjacency matrix W , we interpret it as parametrizing a distribution Bern ( W ) from which random discrete graphs are sampled, A ∼ Bern ( W ) . Our goal is to estimate the expected reachability (with path lengths up to d ), E A ∼ Bern ( W ) [ R ( d ) A ( x, y ) ] , where the randomness comes from the graph structure itself.

This insight guides our estimation approach for the percolation-based reachability. We observe that our generalized Bellman-Ford-based differentiable reachability score (Definition 3.5) provides a lower bound on this expectation, while graph diffusion methods like Random Walk algorithms typically yield upper bounds. This distinction becomes clear when considering two potential paths p 1 and p 2 from node x to node y that share some edges. In percolation theory, reachability requires only that at least one path forms completely. Our Bellman-Ford approach with max operators captures this intuition-when one path's formation probability approaches 1, additional paths contribute minimally to the overall reachability probability. The lower bound property stems from the approximation gap between product-max t-norms/t-conorms and the true intersection/union of events, combined with the approximation in LogLTN's [2] LogMeanExp operator (Equation (8)). Formal proof of this lower bound property can be found in the proof for Lemma 3.7 in Appendix A. Consequently, our method will systematically underestimate true percolation-based reachability.

Conversely, in diffusion models, particles make independent decisions at each node. When two paths overlap, the probability of a particle reaching the target via either path is treated additively rather than as a union probability. This independence assumption fails to account for the correlation between overlapping paths in the percolation setting, causing diffusion methods to overestimate the true percolation-based reachability.

Figure 5 provides a concrete example illustrating the distinction between our max-product BellmanFord reachability and graph diffusion scores. The example shows a weighted adjacency matrix W with 5 nodes, where edge weights are as indicated in the figure (with zeros elsewhere). For the diffusion-based method, we choose the typical Random Walk algorithm, where we assume self-loops at each node so that the sum of self-loop probability and outgoing edge probabilities equals 1 (e.g., node x has a self-loop probability of 0.6).

Consider two possible paths from x to y : path p 1 = x → z → v → y and path p 2 = x → z → w → y . p 1 and p 2 are two random variables under the graph distribution Bern ( W ) . The true percolation-based reachability probability, denoted by P ( x ⇝ y ) , is computed as the union of the

events of these two paths forming: P ( p 1 ∨ p 2 ) . Since these paths share the edge x → z but diverge afterward, we must carefully apply probability theory on union of events. For the divergent segments, the probability is:

Because these two sub-paths share no edges, they are independent, giving P (( z → v → y ) ∧ ( z → w → y )) = P ( z → v → y ) · P ( z → w → y ) . Calculating the full expression yields P ( x ⇝ y ) = 0 . 1592 .

Our max-product Bellman-Ford approach approximates this union using the max operator: for the divergent sub-paths from z to y , we compute R (2) W ( z, y ) = max { P ( z → v → y ) , P ( z → w → y ) } . Since the max operator always provides a lower bound for the union of events, our approach yields the final value R (3) W ( x, y ) = 0 . 12 , which underestimates the true reachability probability. In contrast, the diffusion-based Random Walk method, denoted by P ( RW ) ( x ⇝ y ) , adds probabilities additively across possible paths: P ( RW ) ( x ⇝ y ) = P ( p 1 ) + P ( p 2 ) . This leads to an overestimation of the true reachability probability, giving P ( RW ) ( x ⇝ y ) = 0 . 176 . This example clearly demonstrates why our max-product approach provides a principled lower bound on percolation-based reachability, while diffusion-based methods typically yield upper bounds.

Our max-product Bellman-Ford approach is thus particularly well-suited for our proposed model instantiation, DAGPA, as it provides consistent lower bounds throughout computation, ensuring a coherent probabilistic interpretation. While we focus on lower-bound estimation, an alternative approach could theoretically estimate upper bounds on expected d-separation/d-connection statements. For such a method, diffusion-based approaches like Random Walk would be more appropriate for estimating graph reachability. However, this would require identifying a different pair of t-norm/tconorm differentiable logical operators that yield upper bounds instead of lower bounds. Based on van Krieken et al. [38]'s comprehensive analysis of t-norm and t-conorm suitability for differentiable learning, we have not identified a configuration that simultaneously provides upper bounds and maintains gradient stability comparable to our max-product operators. This represents an interesting direction for future research, potentially expanding the theoretical foundations of differentiable d-separation frameworks, and we encourage future work to explore this avenue.

C Full Algorithm and Implementation Details of DAGPA

DAGPA, our proposed instantiation of the differentiable d-separation framework, employs multiple techniques to address optimization challenges and result validation. This section provides comprehensive implementation details for all techniques used in our approach. We first discuss the multi-task gradient conflict resolution technique (Section C.1) and the Bayesian sampling method for enhanced weight space exploration (Section C.2). Since DAGPA uses Bayesian sampling, each optimization step generates a new candidate causal graph. Consequently, we require a principled procedure to select the best DAGs from those sampled, using only information available from the input data. We present this DAG selection heuristic in Section C.3. Finally, we provide the complete algorithm pseudocode in Section C.4.

C.1 Multitask optimizer to address conflicting gradients

The five distinct loss functions-0th and 1storder TP/TN losses plus the DAG acyclicity constraint (Definition 4.1)-render our optimization a multi-objective problem. Recent research in multi-task learning has demonstrated that naively summing these loss functions leads to theoretically suboptimal convergence, as contradictory gradient directions among tasks can impede optimization progress [20, 22, 31, 40, 43]. This challenge is particularly pronounced in our framework, where the TP losses encouraging d-separation

Algorithm 1 PCGrad [43] Gradient Projection

naturally conflict with the TN losses that require d-connection. To address these inherent gradient conflicts, we adopt PCGrad [43], a gradient projection technique that resolves conflicting directions by projecting one gradient onto the normal plane of the other when they conflict. We reproduce the PCGrad algorithm [43] adopted to our setting in Algorithm 1.

C.2 Gradient-informed discrete Bayesian sampling of W

Rather than using conventional stochastic gradient descent (SGD), we adopt Discrete Langevin Proposal (DLP) [50], a gradient-informed discrete Bayesian sampling technique, to sample parameters θ that yield low-loss configurations 5 . Our choice of DLP is motivated by several key factors. First, our empirical observations show that SGD often becomes trapped in local minima, possibly due to the highly non-convex loss landscape, in which case Bayesian sampling can naturally escape local minima and explores the weight space more effectively. Second, we favor discrete over continuous sampling because the most likely DAG A from a weighted adjacency matrix W is obtained via thresholding at 0.5. This threshold divides the parameter space into discrete regions, and continuous optimization or sampling approaches spend significant computational effort exploring within a single region, repeatedly yielding the same causal graph structure. In contrast, DLP's discrete support enables efficient exploration across different regions while leveraging gradient information from the continuous space to guide proposals.

To adopt DLP for DAGPA, we first restrict the parameter space from the real space R d × d to a space of discrete supports, D d × d , where D = { D 1 , . . . , D m | D i < D j , ∀ i < j } is a finite set of values. Furthermore, we require D 1 < 0 and D m > 0 , so that for any pair of nodes ( x, y ) , if θ x,y = D 1 , then W x,y = σ ( θ x,y ) < 0 . 5 , which allows us to interpret x, y as more likely to not having an edge. Similarly, if θ x,y = D m , then W x,y = σ ( θ x,y ) > 0 . 5 , which allows us to interpret x, y as more likely to have an edge x → y .

In reality through empirical experiments, we found that D = {-2 . 0 , 0 . 0 , 2 . 0 } works quite well. We hypothesize this could be due to -2 . 0 and 2 . 0 as sigmoid logits are not too extreme so that they enable smooth and meaningful gradients, while also sufficiently far apart for the differentiable d-separation formulæ. Furthermore, the 'middle point' 0 . 0 allows DAGPA to model uncertain edges, whose final values can only to be decided after other edges are settled.

The next step to adopt DLP [50] is to define the proposal sampling distribution and the Metrpolis-Hastings(MH) acceptance-rejection step [15, 21]. This is the discrete Metropolisadjusted Langevin algorithm (DMALA) variant of DLP algorithm [50]. One of DLP's novel innovations is the parallel sampling of the parameters θ when its proposal distribution can be factorized along the dimensions, meaning at every step, multiple θ i for different i 's may have their values updated. This technique significantly enhances DLP's efficiency compared to other discrete sampling methods. DAGPA fits this requirement as we can formulate a factorizable proposal distribution. Specifically, we adopt Equation (2) from Zhang et al. [50], but change the gradients to those obtained via PCGrad [43] from Algo-

Algorithm 2 DLP (DMALA) [50] Sampling Step

Require: Model parameters θ , step size β , gradient ∆ θ ( PC ) , current energy U ( θ )

1: // Proposal Step

2: for i = 1 : d do ▷ Can be done in parallel

3: construct q i ( · | θ ) as in Equation (9)

4: sample θ ′ i ∼ q i ( · | θ )

5: // MH Acceptance Step

6: compute U ( θ ′ )

7: compute ∆ θ ′ ( PC ) via PCGrad (Algorithm 1)

8: compute q ( θ ′ | θ ) = ∏ i q i ( θ ′ i | θ )

10: set θ ← θ ′ with probability in Equation (10)

9: compute q ( θ | θ ′ ) = ∏ i q i ( θ i | θ ′ )

11: return new sample θ ′

rithm 1. That is, let q ( θ ′ | θ ) be the proposal distribution. It can be factorized along the d dimensions, q ( θ ′ | θ ) = ∏ d i =1 q i ( θ ′ i | θ ) , where q i ( θ ′ i | θ ) is a categorical distribution of the form:

where ∆ θ ( PC ) is the projected gradients given by the PCGrad algorithm (Algorithm 1), and β is a hyperparameter controlling the DLP step size.

After a proposal θ ′ is sampled from θ according to Equation (9), we perform an MH acceptancerejection step. Intuitively, this step is to ensure that we are not taking a step too far and landing into a 'bad region' in the parameter space. Mathematically, this step is to ensure the Markov chain is reversible. Here, we adopt Equation (3) from Zhang et al. [50], but for the (negative) energy function, we simply adopt the heuristic of summing the multi-task losses (Definition 4.1). Thus, for DAGPA, we accepts the proposal θ ′ with probability

where U ( θ ) = L TP-0 ( θ, D ) + L TP-1 ( θ, D ) + L TN-0 ( θ, D ) + L TN-1 ( θ, D ) + L DAG ( θ, s ) with D the dataset.

We reproduce the DLP sampling algorithm adopted to DAGPA in Algorithm 2. We note, however, that the combination of PCGrad[43] projected gradients ∆ θ ( PC ) and the energy function U ( θ ) as the sum of multi-task losses may not form a mathematically well-defined and reversible Markov chain for the sampling, because the gradients are not directly derived from U ( θ ) but have been post-hoc modified via gradient projections. Nevertheless, we argue both theoretically and from empirical observations that the gradient modification step with either PCGrad or some alternative multi-task learning method is essential, as it addresses gradient conflicts and navigates the gradient landscape much more efficiently. Furthermore, we empirically found that the MH acceptance step using the energy function U ( θ ) obtained via summing the multi-task losses is sufficiently performative and, more importantly, provides the acceptance rate as an important indicator for adjusting the DLP step size β , which is one of the most important hyperparameters in DAGPA. We detail the hyperparameter choice in Appendix D.4. We invite future research to tackle this challenging of integrating multi-task gradients with MH acceptance step in a more principled way.

C.3 Training-time DAG Selection

Finally, DAGPA employs a heuristic score to evaluate the quality of causal graphs obtained from sampled model parameters. This evaluation step is essential because DAGPA adopts DLP [50], and each DLP sampling iteration generates new model parameters and corresponding causal graphs, making it necessary to identify the highest-quality DAGs for final output. Importantly, this score is not a traditional "validation score" since it requires neither a separate validation dataset nor knowledge of the ground-truth causal structure. Instead, it operates solely on the input dataset D that is already used during optimization and sampling. This approach offers a practical advantage over model-based methods like NOTEARS [51] and DAGMA [4], which typically require splitting the dataset to create separate validation sets, thereby reducing the amount of data available for model training. Our score makes full use of all available data while providing a principled mechanism for DAG selection.

We name this score the 'TPTN Ratio score', as it essentially checks the ratio of weighted true positive (TP) and true negative (TN) d-separation / CI statements predicted by the model, while treating the p-values from the dataset as the soft ground-truth labels. Specifically, given the current sampled model parameter θ , we obtain the weighted adjacency matrix via W = σ ( θ ) , and then threshold it to convert it to a binary graph ˆ A , ˆ A x,y = 1 [ W x,y > 0 . 5] , ∀ x, y ∈ [ d ] . Additionally, we would like to ensure that at every step of sampling, we are evaluating a DAG, as the graph ˆ A converted from the model parameters θ found by the DLP sampling algorithm may not fully satisfy the acyclicity constraint. This requirement can be achieved by pruning ˆ A to form an acyclic A . In DAGPA, we proceed by finding a feedback arc set (ARC) from ˆ A and remove all the edges in it to construct A . We use the ARC method provided in the igraph package [7].

Then, setting the LogMeanExp temperature α (Equation (8)) to a very small value (e.g. α = 1 e -5 ), we use the differentiable d-separation scores (Definition 3.6) on A to obtain ˜ S (0) A and ˜ S (1) A . Since α is small but not exactly 0, exp( ˜ S (0) A ) and exp( ˜ S (0) A ) are close to but not exactly 0's or 1's. Thus, we threshold again to obtain the binary d-separation statements, S (0) A ( x, y ) = 1 [exp( ˜ S (0) A )( x, y ) > 0 . 5] and S (1) A ( x, y | z ) = 1 [exp( ˜ S (1) A )( x, y | z ) > 0 . 5] . We treat these as the binary d-separation statements predicted by the model.

Then, given the p-values p D from the data, we compute the true positives (TP), true negatives (TN), false positives (FP), and false negative (FN) scores as:

And then the TPTN Ratio score is computed as

which ranges in [0 , 1] and the higher the score, the better the causal graph in matching the low-order CI statements found in the data.

C.4 Full algorithm of DAGPA

Combining all techniques together, DAGPA's full algorithm is given in Algorithm 3.

C.5 Computation Complexity Analysis

The computational bottleneck of DAGPA is the p-values computation for all low-order CI statements and the reachability computation required by the differentiable d-separation formulae.

The complexity of p-values computation (which we GPU-accelerate) depends on the specific choice of statistical independence test. Take the Chi-squared test for example. Given n data points, computing each single Chi-squared p-value takes O ( n ) as this is the time required for iterating over all data points and building the contingency. Thus, the overall time required is O ( d 3 n ) as we obtain a total of d 2 + d 3 number of p-values for the unconditional and first-order conditional statements.

For the differentiable d-separation formulae: the reachability subroutine (Definition 3.3) involves a maximum of d recursive steps, and each all-pairs Bellman-Ford update step operates on O ( d 3 ) matrix entries (Equation (7)). Thus, the total computation time is O ( d 4 ) . The resulting all-pairs reachability matrix will be cached and accessed by the d-connection/d-separation score computation. Now, since in the 1st-order d-connection/d-separation scores (Equations (3) and (5)), we need reachability on

the node-deleted subgraph A -z for all conditioning node z , this requires a total of d +1 all-pairs reachability matrices, thus rendering the total runtime of this part O ( d 5 ) .

For the 0th-order d-separation/d-connection formulae (Equations (2) and (4)), computing S (0) _ A ( x, y ) or C (0) _ A ( x, y ) for each (x, y) takes O ( d ) time, since it iterates over d possible common ancestors and accessing the reachability cache takes O (1) . Thus, computing all-pairs 0thorder d-separation/d-connection scores take O ( d 3 ) time. Now, since later in the 1th-order formulae, we need all-pairs 0th-order d-separation/d-connection scores for the node-deleted subgraphs A -z for all conditioning node z , this adds another d dimension. Thus, the total time is O ( d 4 ) .

For the 1st-order d-separation/d-connection formulae (Equations (3) and (5)), similarly, computing the result for each query triple ( x, y | z ) takes O ( d ) time. Thus, getting the results for all triple of nodes takes a total of O ( d 4 ) computations.

Thus in summary, computing all-pairs p-values takes O ( d 3 n ) computations, computing reachability matrix (and cache for later use) takes O ( d 5 ) computations, and computing 0th-order and 1st-order d-separation/d-connection scores each take O ( d 4 ) computations. The total will be dominated by the reachability computation, which is O ( d 5 ) .

Acceleration in Practice The total O ( d 5 ) time complexity notwithstanding, we seize many promising opportunities for acceleration in the code implementation of DAGPA.

First, we note that all computations of reachability and d-separation/d-connection scores are matrixvector operations, which can benefit from GPU vectorization . In practice, we leverage PyTorch GPU tensor library for all such computations, avoiding any explicit for-loops and significantly improving the speed of optimization.

Second, in practice one can limit the reachability computation to only consider paths of a constant maximum length k , reducing the time complexity from O ( d 5 ) to O ( d 4 k ) , rendering the entire pipeline O ( d 4 ) . In the experiments for larger graphs with d = 50 number of nodes (Appendix F.2), we limit the path length to k = 25 . Observations during earlier method development suggest negligible performance degradation with significantly improved computational efficiency.

Third, the all-pairs p-values computation can be massively parallelized and then cached and reused. This is particularly useful in the case where one wishes to obtain multiple output causal graphs for the same input dataset to enhance solution diversity and exploration of the Markov equivalence class. In this case, the computation time for the all-pairs p-values step can be amortized as it can be reused in subsequent runs.

Finally, we expect future work to further examine how to incorporate sparsity assumptions into the graph to further reduce the time complexity. For example, instead of assuming each node can connect to all d -1 other nodes, one reasonable assumption is to restrict to a maximum of constant k degree. In that case, the reachability computation subroutine computation can drastically speed up.

D Experiment Details

D.1 Code and dataset release

We release our code and data in https://github.com/PurdueMINDS/DAGPA

D.2 Dataset creation and conversion

We generated synthetic binary datasets following the approach introduced in the k -PC codebase [18]. We simulated DAGs using both Erd˝ os-Rényi (ER) and Scale-Free (SF) graph models with d ∈ { 10 , 50 } nodes and expected edge-to-node ratios r ∈ { 2 , 4 } . For ER graphs, we set edge probability p = r/d to achieve the desired arc ratio. For SF graphs, we used the Barabási-Albert preferential attachment model with attachment parameter m = ⌊ r/ 2 ⌋ to generate undirected graphs, then randomly oriented edges to form DAGs.

For each generated DAG adjacency matrix A , we constructed a binary Bayesian network using pyAgrum [10]. Each variable X i takes values in { 0 , 1 } , and we randomly generated conditional

probability tables (CPTs) for all nodes. For root nodes (variables with no parents), we sampled marginal probabilities P ( X i = 1) ∼ Uniform (0 . 2 , 0 . 8) to avoid extreme probabilities. For non-root nodes with parent set PA ( i ) , we sampled conditional probabilities P ( X i = 1 | PA ( i ) = c ) ∼ Uniform (0 . 2 , 0 . 8) independently for each parent configuration c ∈ { 0 , 1 } | PA ( i ) | . From these fully specified Bayesian networks, we drew n ∈ { 100 , 1000 , 10000 , 100000 } independent samples via ancestral sampling (forward sampling from the topological order). Since the data is binary, no normalization or scaling preprocessing was needed.

We also generated continuous data with simulated ER and SF DAG following the approach introduced in [4]. Specifically, for each DAG adjacency matrix A , we generated structural equation models of the form X i = ∑ j ∈ PA ( i ) w ji f j ( X j ) + ϵ i , where PA ( i ) denotes the parent set of node i , edge weights w ji are sampled uniformly from [ -2 . 0 , -0 . 5] ∪ [0 . 5 , 2 . 0] , and noise terms ϵ i ∼ N (0 , 1) are independent Gaussian. For linear continuous data, we set f j ( x ) = x . For nonlinear continuous data, we randomly assigned each f j to one of four nonlinear functions: f j ( x ) ∈ { x 2 , x 3 , sin( x ) , cos( x ) } with equal probability. We generated datasets with the same sample sizes n ∈ { 100 , 1000 , 10000 , 100000 } and graph structures (ER and SF with d ∈ { 10 , 50 } nodes and arc ratios r ∈ { 2 , 4 } ) as the binary setting to enable direct comparison across data types.

For experiments on real-world dataset, we benchmarked our method and baselines over the Sachs dataset [28] and the LUCAS (LUng CAncer Simple set) dataset [14]. The Sachs dataset is a widely recognized benchmark in causal discovery research, consisting of protein signaling pathway data collected through flow cytometry experiments. It contains measurements of 11 phosphorylated proteins and phospholipids derived from thousands of individual primary immune system cells, gathered under various experimental conditions. What makes this dataset particularly valuable for benchmarking causal learning methods is that the pathways between these proteins are wellestablished in scientific literature, providing a reliable ground truth causal graph against which algorithms can be evaluated. The dataset has been extensively used in numerous studies to assess the performance of causal discovery algorithms, including recent work that demonstrates how less restrictive modeling approaches can capture complex causal relationships in this data that traditional methods assuming additive noise often fail to identify. We obtained and did benchmark on the subset of Sachs data containing approximately 800 samples with no perturbation. We used the standard discretized version preprocessed using the Hartemink discretization method, which converts the original continuous protein concentration measurements into 3-level categorical variables (low, average, high) while preserving the underlying dependence structure. The LUCAS dataset is an artificially generated benchmark specifically designed to evaluate causal discovery algorithms under different conditions. It features binary variables in a causal Bayesian network structure and comes in several variants that present increasing levels of challenge. Among all experiments introduced in LUCAS, we used LUCAS0 (baseline unmanipulated data) of size 2000.

D.3 Evaluation metrics

The primary metric we adopt in this work is the Conditional Independence Matthews Correlation Coefficient (CI-MCC), which we describe in detail in Section 5 with a visual illustration in Figure 2.

Additionally, we evaluate our method and baselines using the following standard graph structure-based metric:

CPDAG Arrowhead F1: We compare predicted causal graphs with the ground-truth CPDAG derived from the ground-truth DAG. Since different methods produce varying graph types, we standardize all outputs by converting them to CPDAGs. For methods like NOTEARS [51] and DAGMA [4] that return DAGs, we convert their outputs to CPDAGs using the utility method from the causal-learn package [53]. For k-PC [18], which returns k-essential graphs containing circle-marked edges (in addition to directed and undirected edges), we treat circle-marked edges as undirected edges in the CPDAG conversion. This treatment is appropriate because circle-marked edges in k-essential graphs denote edges whose directions cannot be determined from low-order CI statements alone. Once both predicted and ground-truth graphs are converted to CPDAGs, we compute the arrowhead F1 score exclusively over the directed edges in the CPDAGs.

CPDAG Skeleton F1: Following the same standardization process as CPDAG Arrowhead F1, we convert all predicted causal graphs to CPDAGs. We then further convert all directed edges in both

predicted and ground-truth CPDAGs to undirected edges, creating graph skeletons. The F1 score is computed on these undirected skeleton graphs.

CPDAG Structural Hamming Distance (SHD): After standardizing predicted causal graphs by converting them to CPDAGs, we compute the structural Hamming distance (SHD) between predicted and ground-truth CPDAGs. Each unit of SHD corresponds to a single edge difference between the two CPDAGs, including: missing edges, extra edges, incorrect edge orientations (directed vs. undirected), or incorrect edge directions.

DAG F1: We also evaluate methods that directly output DAGs by comparing them with the ground-truth DAG. Since constraint-based methods like GES [6], PC [33], and k-PC [18] return CPDAGs or k-essential graphs rather than DAGs, this metric only applies to score-based methods: our approach, NOTEARS (linear and nonlinear) [51, 52], and DAGMA [4].

We note that these graph structure-based metrics may pose an inherent disadvantage to our method compared to CI-MCC. Through empirical observation, we have identified cases where DAGPA produces DAGs with similarly high CI-MCC scores but vastly different performance on structurebased metrics. For instance, one sampled DAG may achieve both high CI-MCC and high structurebased scores, while another DAG with comparable CI-MCC performance may score poorly on structure-based metrics. This suggests that while DAGPA consistently samples DAGs that align well with conditional independence patterns (as measured by CI-MCC), this alignment does not necessarily translate to high performance on traditional graph structure metrics. We provide detailed analysis of this phenomenon in Appendix F.1.

D.4 DAGPA model details and hyperparameters

The most important and sensitive hyperparameter in DAGPA is the DLP sampling step size β (Equation (9)). To this end, we first find values for all other hyperparameters through preliminary experimentations then fix them, and only vary in the step size for the experiments on the synthetic binary dataset and the real-world datasets.

Some of the other important hyperparameters and their values:

DLP support logit set D : This hyperparameter controls the support logits that the model parameter θ can take during sampling. We use D = [ -2 . 0 , 0 . 0 , 2 . 0] .

LogMeanExp temperature α : This hyperparameter controls the approximation accuracy of the t-conorm operator (Equation (8)) and thus the accuracy of differentiable d-separation scores. Setting this value too large will lose approximation accuracy, while setting it too low will induce unstable and unsmooth gradients. Thus, during training or parameter update cycles, we use a α train = 0 . 01 , while during evaluation when computing the DAG selection score, we use a α eval = 1 e -5 .

DAGMA's [4] log-det acyclicity constraint hyperparameter s : This hyperparameter controls the valid region of M-matrices in which the log-det acyclicity loss is well-defined. Setting this value too large will risk model parameters stepping out of this region and causing undefined gradients, whereas setting this value too large will cause the gradient to have very small norm. In our experiments, for small graphs (include n = 10 synthetic binary dataset and both Sachs [28] and Lucas [14]) we use s = 3 . 0 , while for large graphs ( n = 50 synthetic binary dataset) we use s = 8 . 0 .

Finally, for the DLP step size β , for each dataset we choose a different range to run hyperparameter search and choose the best sampled DAGs therein according to the DAG selection score (Appendix C.3). The specific value range of β is chosen according to the acceptance rate in the DLP's Metropolis-Hastings acceptance-rejection step. Specifically, we choose the lowest β value to be the one that can roughly achieve 0.8 acceptance rate, and the highest β value to be the one that can roughly achieve 0.2 acceptance rate.

n = 10 graphs in synthetic binary dataset: β ∈ { 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 , 1 . 0 , 1 . 1 , 1 . 2 } .

n = 50 graphs in synthetic binary dataset: β ∈ { 0 . 31 , 0 . 32 , 0 . 33 , 0 . 34 , 0 . 35 , 0 . 36 , 0 . 37 , 0 . 38 } .

Sachs and Lucas: β ∈ { 0 . 76 , 0 . 78 , 0 . 80 , 0 . 82 , 0 . 84 , 0 . 86 , 0 . 88 , 0 . 90 }

D.5 Baseline model details and hyperparameters

We tested our method against 5 methods from the table in the following section. For PC and kPC, we used chi-square independence test and chose significance level threshold at 0.05, and we tested with both first and second order conditional independence tests for kPC. We used the causallearn implementation[53] for PC and GES algorithms. For GES we used the local BIC score. For NOTEARS and DAGMA linear mode, we used the l2 loss for non-linear mode, and we followed the default dimensions for the MLP layer as used in the authors' original code. Rest of parameters all followed default settings.

D.6 Compute resources used

For the baselines, we ran PC, kPC, GES and linear version of NOTEARS, DAGMA on a 64-core AMDEpyc 7662 "Rome" processor with 16 CPU cores and 32 GB memory requested. The non-linear version of NOTEARS and DAGMA were run on one A30 with same CPU and memory requirement. Every experiment is completed in 4 hours.

For DAGPA, we run all experiments on an AMD GPU cluster, equipped with 32GB MI108 and 64GB MI210 and EPYC 7V13 cpu with 64 cores. Experiments on small graphs ( n = 10 synthetic binary, Sachs, and Lucas) terminate within 1.5 hours, whereas experiments on large graphs ( n = 50 ) terminates within 12 hours.

E Related Work

The methodological evolution of causal discovery has progressed through two primary paradigms-constraint-based and score-based approaches-with recent advances bridging their complementary strengths.

Constraint-Based Methods Pioneered by [34], constraint-based methods leverage conditional independence (CI) tests to reconstruct causal graphs. The PC algorithm [33] established core principles: (1) infer conditional independencies via statistical tests, (2) eliminate edges violating d-separation rules, and (3) orient edges using collider detection. While effective in principle, finitesample reliability suffered from error propagation in high-order CI tests [9]. Extensions like FCI [46] addressed latent confounding through more sophisticated separation criteria, and Kernel CI Test (KCIT) [48] enabled nonparametric testing via Hilbert space embeddings. Modern variants like LOCI [42] demonstrated that low-order CI statements suffice for structure recovery under specific faithfulness conditions, while k-PC [18] provided theoretical guarantees for bounded-order testing.

Score-Based Methods Score-based methods reformulated causal discovery as a combinatorial optimization problem, maximizing score functions ( e.g. , BIC [29]) over DAG spaces. These methods can be broadly divided into those that search the discrete graph space and those that leverage continuous optimization. Among discrete methods, GES [6] advanced this paradigm through greedy equivalence class search, Another major family consists of Bayesian methods, such as BayesDAG [1], which define a prior distribution over DAG structures and a likelihood function. By applying Bayes' rule, they compute a posterior distribution over DAGs, typically using sampling methods like MCMC to explore the structure space and average over models. While powerful for uncertainty quantification, these methods, like greedy search, are computationally intensive and constrained by the discrete search space. The field transformed with [51]'s NOTEARS framework, which redefined acyclicity through a differentiable constraint:

where W is the weighted adjacency matrix. This enabled continuous gradient-based optimization, spawning derivatives like DAGMA [4] with improved learning stability and enhanced gradient behavior via a log-determinant characterization,

and GOLEM [23] using likelihood-based objectives. Nonlinear extensions [52] incorporated neural networks to model complex functional relationships while preserving acyclicity.

F Full Experiment Results

F.1 Analysis of CI-MCC versus graph structure metrics

In this work we primarily showcase the conditional independence Matthews Correlation Coefficient (CI-MCC) metric, along with auxiliary graph-structure-based metrics like CPDAG Arrowhead F1, CPDAG Skeleton F1, CPDAG SHD, and DAG F1. We notice, however, that the graph-structure-based metrics may pose an unfair challenging to DAGPA. In particular, we found that the graph-structure metrics may not always align with DAGPA's objective of matching the causal DAG's predicted d-separation statements with the low-order CI statements found in the dataset. There are many cases where the DAGs returned by DAGPA made few mistakes in aligning the CI statements, yet are still scored badly by the graph-structure-based metrics. We provide concrete evidence in this section.

Here, we pick one experiment run of DAGPA on one of the synthetic binary dataset generated by an ER graph with n = 10 nodes with in total N = 10000 samples. This particular run performed 1000 sampling steps, thus giving us 1000 sampled DAGs. For each sampled DAG, we compute the DAG selection score (Appendix C.3) and the CI-MCC metric score, as well as the scores of all graph-structure-based metrics. To recall, the DAG selection score is used by DAGPA to select the best DAGs out of all that were sampled during training and requires only the input data to compute. It is not a test metric. Figure 8 shows the scatter plots of comparing CI-MCC to graph-structurebased metrics, Figure 7 shows the scatter plots of comparing the DAG selection score to graph-

0.2

0.3

0.4

0.5

0.6

0.7

DAG Selection Score

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

CI-MCC

0

200

400

600

800

Sampling Step

structure-based method, and finally Figure 6 shows the relationship between the DAG selection score with CI-MCC.

From Figure 8, we can observe that, although the graph-structure-based metrics have generally positive correlation with CI-MCC, there are many samples concentrated in the bottom-right - the region where these sampled DAGs achieves high CI-MCC but low graph-structure-based metric values. Moreover, especially on regions of high CI-MCC values, the samples vary a lot on their graph-structure-based metric values. This implies that there is a misalignment between CI-MCC and the graph-structure-based metric, where performing well on matching model's d-separation with data's low-order CI statements do not translate to similarity on the graph structure.

Figure 7 reveals a similar story. In this case, the x-axis is the score actually used by DAGPA during training. The similar pattern shows that, even if DAGPA discovers a high-quality DAG with high DAG selection score, which in turn translates to making minimal false positive and false negative mistakes on the low-order CI statements (Appendix C.3), it may still have a drastically different structure than the ground-truth DAG or CPDAG, yielding a very low graph-structure-based metric value such as a low CPDAG Arrowhead F1 value. We leave the problem of proposing an alternative DAG selection score that may yield much more positive correlation with graph-structure-based metric to future research.

As a sanity check, Figure 6 shows the relationship between DAGPA's DAG selection score and the CI-MCC metric. Here, we can finally observe a strong correlation, demonstrating the validity of our proposed DAG selection score - it is correctly doing what it is designed to do, i.e., checking alignments of low-order CI statements between the model and input data.

Finally, throughout all three figures, we visualize the sampling step numbers in color scale, where the darker colors corresponds to early stages in the sampling, and lighter colors corresponds to later stages. We can observe that, in general, there exists a gradual transition from darker color to lighter color when going from bottom-left to top-right. This pattern suggests that DAGPA is indeed gradually approaching the regions in the weight space that have better and better performance, across all types of metrics.

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

CI-MCC

0.0

0.1

0.2

0.3

0.4

0.5

CPDAG Arrowhead F1

(a) CPDAG Arrowhead F1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

CI-MCC

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

CPDAG Skeleton F1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

CI-MCC

15

20

25

30

35

CPDAG SHD

(c) CPDAG SHD

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

CI-MCC

0.0

0.1

0.2

0.3

0.4

0.5

DAG F1

0

200

400

600

800

Sampling Step

(d) DAG F1

0.2

0.3

0.4

0.5

0.6

0.7

DAG Selection Score

0.0

0.1

0.2

0.3

0.4

0.5

CPDAG Arrowhead F1

(a) CPDAG Arrowhead F1

0.2

0.3

0.4

0.5

0.6

0.7

DAG Selection Score

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

CPDAG Skeleton F1

0.2

0.3

0.4

0.5

0.6

0.7

DAG Selection Score

15

20

25

30

35

CPDAG SHD

(c) CPDAG SHD

0.2

0.3

0.4

0.5

0.6

0.7

DAG Selection Score

0.0

0.1

0.2

0.3

0.4

0.5

DAG F1

0

200

400

600

800

Sampling Step

(d) DAG F1

F.2 Results on synthetic binary data

We present the full suite of results on the synthetic binary dataset in Tables 1 to 4.

F.3 Results on real-world data

We show the results with additional metrics on Sachs [28] dataset and on an additional real-world dataset Lucas [14] in Figure 9.

G Future Work Discussions

G.1 Integration with Score-Based Causal Discovery Methods

Our differentiable d-separation framework provides a novel optimization objective based on conditional independence constraints, which can potentially be integrated with existing score-based methods like NOTEARS and DAGMA as a structured regularization technique. This section briefly discusses potential strategies for integrating our approach with existing score-based methods, particularly NOTEARS [51] and DAGMA [4]. We focus on the key technical challenge: ensuring parameter compatibility between score-based and CI-constraint objectives.

The challenge is ensuring compatibility between the model parameters θ used in score-based methods and the probabilistic edge weights W ∈ [0 , 1] d × d required by our framework. For example, for NOTEARS linear models, applying sigmoid activation σ ( θ ) naturally produces valid probability measures. However, for nonlinear variants where the weighted adjacency matrix is derived from neural network parameters (e.g., √ ( θ (1) ) T θ (1) for the first MLP layer), additional normalization is needed to map unnormalized non-negative values to [0 , 1] while preserving meaningful probabilistic semantics. Once this parameterization is established, the score-based prediction loss (e.g., MSE reconstruction loss) can be added as an additional task in our multi-task optimization framework, combining functional relationship learning with explicit CI constraint enforcement. While this

(a) Synthetic binary ER, r = 2

(a) Synthetic binary ER, r = 4

(d) Synthetic binary ER, r = 4 , CPDAG SHD

(a) Synthetic binary SF, r = 2 , CI-MC

(a) Synthetic binary SF, r = 4 , CI-MC

(b) Synthetic binary SF, r = 4 , CPDAG F1 Arrowhead

(c) Synthetic binary SF, r = 4 , CPDAG F1 Skeleton

integration direction is promising and could leverage complementary strengths of both paradigms, the normalization challenge for nonlinear models and comprehensive empirical evaluation remain important future work.

G.2 Alternative Conditional Independence Measurements and Tradeoffs

Our framework uses p-values from statistical conditional independence (CI) tests as soft measures of independence strength. This section discusses the motivation for CI-based measurements and compares different CI measurement approaches.

G.2.1 Motivation: Robustness and Interpretability

Robustness to Data Preprocessing. Many statistical CI tests offer inherent robustness due to their scale-invariance properties. Standard CI tests remain reliable across different preprocessing pipelines: Fisher-z test statistics are invariant under linear transformations since partial correlations ρ XY | Z are preserved when variables are standardized; chi-square tests operate on frequency counts which are unaffected by scaling; kernel-based tests use normalized embeddings providing scale-invariance by design. We empirically verified this: DAGPA maintains CI-MCC scores with low variance of ≈ 0 . 12 across unnormalized, standardized, min-max normalized, and robust-scaled versions of the same dataset ( n = 100 , d = 10 ER-2 synthetic data), demonstrating stability that is valuable when data sources have heterogeneous scales or different preprocessing conventions.

Interpretability via Direct Hypothesis Testing. CI-based measurements provide direct interpretability through testable hypotheses. Each structural decision corresponds to specific CI tests with p-values, enabling transparent explanations. For example, when removing edge X → Y , we can justify: 'Edge removed because unconditional test shows X ̸ ⊥ ⊥ Y with p-value 0.001 and conditional test shows X ⊥ ⊥ Y | Z with p-value 0.82, indicating X influences Y likely through Z but does not have direct causal link to Y .' Domain experts can independently verify these claims by examining raw data, running alternative tests, or checking whether distributional assumptions hold. This transparency is particularly valuable in high-stakes applications (medical diagnosis, policy making) where algorithmic decisions require human oversight and regulatory approval.

G.2.2 Overview of CI Measurement Methods

Different CI tests present tradeoffs across multiple dimensions. Table 5 summarizes key characteristics:

Fisher-z Test: Tests whether partial correlation ρ XY | Z equals zero using transformation z = 1 2 √ n -| Z | -3 log 1+ˆ ρ 1 -ˆ ρ ∼ N (0 , 1) . Strengths include excellent sample efficiency (reliable with n ≥ 50 for | Z | ≤ 1 ), O ( n ) computational cost, and native p-values. Limitations: designed for Gaussian data and primarily detects linear relationships. Our default choice for continuous data in low-sample regime.

Chi-square Test: Tests independence via contingency table using χ 2 = ∑ i,j ( O ij -E ij ) 2 E ij . Strengths include no distributional assumptions, detection of arbitrary associations, and good small-sample performance. Limitations: only applicable to categorical variables, requires sufficient cell counts ( E ij ≥ 5 ), and exponential complexity with | Z | . Our default for binary/categorical data.

Kernel CI Test (KCIT): Tests independence via kernel embeddings using Hilbert-Schmidt Independence Criterion. Strengths include detecting arbitrary nonlinear dependencies and no parametric

assumptions. Limitations: O ( n 2 ) complexity, requires n > 500 for reliable results, and kernel bandwidth selection affects performance. Recommended for large datasets ( n > 1000 ) with suspected nonlinear relationships.

Randomized CI Test (RCIT): Approximates KCIT using random Fourier features, reducing complexity to O ( n log n ) . Provides middle ground between Fisher-z (fast, linear) and KCIT (slow, nonlinear) for moderate samples ( 500 < n < 5000 ).

Conditional Mutual Information (CMI): Measures I ( X ; Y | Z ) = E [ log p ( X,Y | Z ) p ( X | Z ) p ( Y | Z ) ] via k-NN or kernel density estimation. Provides information-theoretic interpretation but lacks native p-values (requires transformation like p = exp( -β · ˆ I ) ) and needs n > 1000 for reliable density estimation.

Tradeoffs and Selection Guidance. For causal discovery with d variables, total CI testing requires O ( d 2 ) order-0 tests and O ( d 3 ) order-1 tests. With d = 50 , n = 1000 : Fisher-z completes in ∼ 30 seconds, RCIT in ∼ 5 minutes, KCIT in ∼ 45 minutes. Sample efficiency also varies: Fisher-z achieves high power with n < 200 for linear relationships, while kernel methods require n > 500 but eventually match or exceed Fisher-z for nonlinear cases. Our framework is measurementagnostic-any function producing [0 , 1] scores can be substituted via our API:

We recommend Fisher-z or chi-square for most applications due to sample efficiency and computational scalability, with kernel methods reserved for large datasets with confirmed nonlinear relationships. Practitioners should conduct sensitivity analysis across multiple tests when possible, as edges appearing consistently provide higher confidence regardless of specific test assumptions.

H Licenses

In this work, we evaluated our method on two publicly available causal discovery benchmark datasets:

Sachs Dataset : The Sachs dataset contains simultaneous measurements of 11 phosphorylated proteins and phospholipids derived from thousands of individual primary immune system cells, subjected to both general and specific molecular interventions. This dataset was originally published by Sachs et al. (2005) and is widely used as a benchmark in causal discovery research. The dataset is publicly available through multiple repositories including bnlearn [30], other causal discovery toolboxes.

Availability : The dataset is publicly accessible for research purposes through various causal discovery software packages and repositories.

LUCAS Dataset : The LUCAS (LUng CAncer Simple set) dataset [14] is a synthetic benchmark dataset consisting of 12 binary variables and 2000 instances, representing 12 different causal relationships in a medical diagnosis problem for identifying patients with lung cancer. This dataset was created as part of the Causality Workbench project to provide standardized benchmarks for testing causal discovery algorithms.

Availability : The dataset is publicly available for research and educational purposes through the Causality Workbench repository.

Usage Declaration: Both datasets were used in accordance with their respective terms of use for academic research purposes. No additional permissions were required for their use in this study.

I Societal Impact Statement

Our differentiable d-separation framework for causal discovery has potential positive impacts in healthcare (identifying causal factors in disease), public policy (evaluating intervention effectiveness), scientific discovery (understanding complex systems), and algorithmic fairness (distinguishing causation from correlation in decision systems). However, several risks must be acknowledged: (1) incorrect causal discoveries could lead to harmful interventions if implemented without domain expert validation; (2) causal discovery in sensitive domains may raise privacy concerns, necessitating differential privacy techniques; (3) when applied to historically biased datasets, discovered

relationships might reflect and perpetuate these biases rather than ground truth; and (4) computational requirements might limit accessibility to well-resourced institutions. We recommend multiple mitigations: returning diverse candidate structures rather than single models, requiring expert validation before implementation, implementing privacy-preserving techniques with sensitive data, examining discovered relationships for bias, and developing more efficient implementations to improve accessibility. Causal discovery tools require responsible application and domain expertise, especially in high-stakes domains where incorrect inferences could lead to harmful consequences.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract and introduction claim that we contribute a novel framework for causal discovery that bridges constraint-based and continuous-optimization score-based methods with promising empirical results. These claims accurately reflect our work's scope and contributions as evidenced in: (1) Section 3, where we develop the novel differentiable d-separation framework with theoretical guarantees and percolation-based probabilistic interpretations; (2) Section 4, where we demonstrate one practical instantiation of this framework into the model we named DAGPA; and (3) Section 5, where we empirically validate that our approach achieves competitive performance compared to established methods across multiple datasets and settings.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We provide a dedicated Limitations paragraph in Section 7 that thoroughly discusses multiple aspects of our work's constraints and outlines specific future research directions to address them. Throughout the paper, we are transparent about the scope of our empirical validation and the provisional nature of our current implementation compared to the theoretical framework's potential.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize

that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: In our paper, all assumptions are explicitly stated before each theorem and lemma, and the complete proofs are provided in Appendix A, with every item properly numbered and consistently cross-referenced throughout the paper. In addition, we precede each theoretical result with intuitive explanations connecting the formal statements to the broader framework, and for the most important results, such as Theorem 3.2, we provide proof sketches in the main text that highlight the critical steps and insights.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We provide complete reproducibility through detailed algorithm pseudocode for DAGPA (Appendix C), code and data release (Appendix D.1), dataset creation and preprocessing procedures (Appendix D.2), and comprehensive hyperparameter specifications for both our method (Appendix D.4) and all baselines (Appendix D.5). Additionally, we document the computational resources used (Appendix D.6) and fully explain our evaluation metrics (Appendix D.3), ensuring all aspects of our experimental pipeline can be independently reproduced.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Weprovide complete code and data access through an anonymous GitHub repository (linked in Appendix D.1) that includes all implementation files, datasets (both synthetic generators and real-world data), and evaluation scripts necessary to reproduce our results. The repository contains detailed documentation with exact environment setup instructions (via Conda environment files), step-by-step commands to run all experiments, data preprocessing pipelines, and scripts to generate all figures and tables presented in the paper.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https://nips.cc/public/ guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: The main text (Section 5) provides essential experimental details including dataset characteristics, evaluation metrics, baselines, and the overall experimental design, while Appendix D contains comprehensive information on hyperparameters, model selection criteria, data creation and preprocessing, and computational resources. We explicitly describe our parameter selection process and provide justification for key design choices to ensure results can be fully understood and contextualized.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Our primary results are presented as empirical cumulative distribution functions (CDFs) over multiple (10) independent dataset instances per configuration, fully displaying performance variability across different data realizations and allowing for direct statistical comparison between methods. For tabular results, we report mean values with standard deviation error bars (1-sigma), clearly labeled as such, and calculated using standard statistical formulas across the independent trials. The sources of variability (different random DAG structures and data samples) are explicitly described in Section 5, and all statistical comparisons are appropriately referenced in the text.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Yes. We detail the computation resources used for each experiment settings in Appendix D.6.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Yes. This research fully conforms to NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We provide a comprehensive Broader Impact Statement in Appendix I that discusses both potential positive impacts (in healthcare, public policy, scientific discovery, and algorithmic fairness) and potential negative impacts (misplaced trust in discovered causal relationships, privacy concerns, potential for reinforcing biases, and accessibility challenges). For each potential negative impact, we also suggest specific mitigation strategies that practitioners can implement when applying our method.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: Our work presents a methodological framework for causal discovery that does not pose the high-risk misuse concerns typically associated with generative models or scraped datasets. We only use synthetic datasets (generated with provided code) and well-established, properly cited benchmark datasets (e.g. Sachs) from trusted scientific sources. Our method does not learn representations that could be misused for generating harmful content, creating fake profiles, or similar high-risk applications that would necessitate special safeguards beyond standard research best practices.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We properly cite all original papers for datasets and code libraries used in our work. In Appendix H, we provide detailed license information for the Sachs dataset (the only external real-world dataset used) and all third-party libraries/packages employed in our implementation, including version numbers and URLs to their repositories. For synthetic datasets, we specify that they are generated by our code which is released under an MIT license.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: We release our implementation of DAGPA via an anonymized GitHub repository (linked in Appendix D.1) with comprehensive documentation including: installation instructions, usage examples, API documentation, parameter descriptions, training/optimization/sampling procedures, and known limitations. The repository includes a structured README file, detailed code comments, and a clear MIT license statement.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or nonstandard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: This work does not involve LLMs as any component of our core methodology, algorithms, or experiments. LLMs were used solely for writing assistance, editing, and LaTeX formatting, which per the NeurIPS LLM policy does not impact the scientific methodology or originality of our work and therefore does not require declaration beyond this checklist.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Better

Better

1.0

0.8

0.6

0.4

0.2

0.0

0.2

0.3

0.9

1.0

0.4

0.5

0.6

0.7

0.8

Low-order CI Statement MCC (CI-MCC)

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

(k=1) kPC

(k=2) kPC

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(a) Sachs CI-MCC

0.3

0.4

0.5

0.6

0.7

0.8

Low-order CI Statement MCC (CI-MCC)

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

kPC (k=1)

kPC (k=2)

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(b) Lucas CI-MCC

0.000

0.001

0.002

0.003

0.004

0.005

CPDAG Arrowhead F1

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

kPC (k=1)

kPC (k=2)

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(c) Sachs CPDAG F1 Arrowhead

0.0

0.2

0.4

0.6

0.8

1.0

Low-order CI Statement MCC (CI-MCC)

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

kPC (k=1)

kPC (k=2)

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(d) Lucas CPDAG F1 Arrowhead

0.0

0.2

0.4

0.6

0.8

CPDAG Skeleton F1

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

kPC (k=1)

kPC (k=2)

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

CPDAG Skeleton F1

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

GES

kPC (k=1)

kPC (k=2)

NOTEARS (Linear)

NOTEARS (Nonlinear)

PC

(f) Lucas CPDAG F1 Skeleton

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

DAG F1

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

NOTEARS (Linear)

NOTEARS (Nonlinear)

(g) Sachs DAG F1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

DAG F1

0.0

0.2

0.4

0.6

0.8

1.0

Empirical CDF

Ours

DAGMA (Linear)

DAGMA (Nonlinear)

NOTEARS (Linear)

NOTEARS (Nonlinear)

(h) Lucas DAG F1

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors introduce DAGPA (DAG Percolation Apartness), a novel causal discovery method formulated as an optimization problem, which directly employs results from conditional independence tests. In detail, DAGPA computes the $p$-value of marginal independence tests between any two distinct variables $(x, y)$ and of conditional independence tests between any three distinct variables $(x, y, z)$. Then, the authors train the model by relating conditional independence statements with $d$-separation conditions. They inscribe this matching in a differentiable optimization problem by (i.) formulating $d$-separation as a first-order-logic formula and (ii.) applying a soft logic framework to optimize the DAG edges. The authors test empirically their work against differentiable score-based baselines (NOTEARS, DAGMA) and constraint-based baselines (PC).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths.\n\n- The method is designed to improve causal discovery performance in the small sample size regime, and experimental results hint at better results in terms of Matthews Correlation Coefficient (MCC) compared to baselines in this particular scenario.\n- The overall method is novel, and it is interesting the idea of mixing conditional independence-based methods with optimization approaches.\n- While the use of first-order-logic formulas lead to a quite convoluted presentation, the authors' effort in describing step-by-step most pivotal passages is appreciable.\n\nWeaknesses.\n\n- The method requires computing conditional independence among all possible pairs and triples of distinct variables to define the loss function of the model. This might significantly affect performance in terms of computational time.\n- Widely accepted metrics for causal discovery, such as F1 and SHD, are relegated to the Appendix where (i.) the method underperforms compared to the baselines (ii.) I would have expected the score-based baselines to have better results. However, this second point might not be a problem depending on the data-generating process adopted by the authors — please refer to the question.\n- Not really a weakness, but please notice two typos \"pioneerd\" (line 96) and \"Hammindg\" (line 295).\n\nQUESTIONS:\n- In Appendix, the method underperforms in terms of F1 and SHD. Can the authors justify this behaviour? While I agree that MCC is an interesting measure, the paper should also discuss more in detail its relation with more widely adopted metrics in the main body.\n- It seems that to compute the loss, you need to precompute all possible conditional independence tests. Is this correct? How would this compare with the number of tests performed by PC? How does DAGPA compare to PC and score-based methods in terms of computational time?\n- Differentiable score-based methods, such as NOTEARS and DAGMA, are proven to work remarkably well on non-normalized data, while struggling to recover the DAG after rescaling data (\"Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game.\", Reisach et al., 2021). How is the pre-processing handled in your experimental pipeline?\n- Can this approach be integrated as a sort of \"regularization\" technique to existing score-based differentiable methods? For instance, in the linear case, I would have imagined to also minimize the reconstruction MSE loss $\\|\\|\\mathbf{W}^\\top\\mathbf{X}-\\mathbf{X}\\|\\|_2^2$ adopted by NOTEARS et similia. Is there some fundamental reason to avoid doing so?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel method to incorporate \"(conditional) independence statements\" into a differentiable causal discovery framework. The authors first express d-separation $S_A(x,y|z)$ using logical notation (Definition 3.1) and then introduce a continuous relaxation of this definition  $\\tilde{S}_W(x,y|z)$ through soft logic. They establish a connection between the soft version and the \"expected\" d-separation statements, enabling the integration of causal inference with differentiable optimization techniques.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Novelty: This is the first time I’ve seen a natural way to combine constraint-based and score-based methods, which is a significant contribution to the field.\n\nWeaknesses\n- The method description could be presented in a more straightforward way.\n- Learning Time: The paper does not report or compare the learning time, which is crucial for evaluating the method's efficiency, especially in large-scale settings.\n- Computational Cost: The method requires performing CI tests on all pairs of variables, which can be computationally expensive and may not scale well to large networks.\n- Code is not available\n\nQUESTIONS:\nHow many CI statements are needed? If you need test all pairs, PC would be much faster? I can’t imagine a practical scenario where testing all pairs is feasible, especially when the number of nodes is large. This seems to contradict the motivation behind methods like NOTEARS, which aim to reduce computational complexity.\n\n\nComments:\n1. The notation $\\independent_D$ is somewhat unfamiliar to me. Typically, independence is discussed in the context of a joint distribution $P$, so this notation may need further clarification.\n2. Is Theorem 3.2 a theorem or a definition？As far as I know， d-separation can be directly read off from the graph $A$ and is denoted by $\\independent_A$. From my perspective, Thm.3.2 and Def.3.1  are the same thing.\n3. Lemma 3.4 appear to be redundant in this draft and could be omitted to improve clarity and conciseness.\n4. It took me some time to understand the exact method for learning the graph. It would be helpful to highlight the key method more clearly at the beginning of Section 3.\n5. The mathematical notation is dense and may be difficult to follow for readers unfamiliar with the specific formalism used. Simplifying or breaking down the notation could improve readability.\n6. Despite these points, I appreciate the idea of bridging constraint-based and score-based methods, which is a promising direction in causal discovery.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDifferentiable Constraint‐Based Causal Discovery introduces a novel hybrid framework that marries the rigorous conditional‐independence testing of constraint‐based methods with the flexibility of gradient‐based optimization. The authors first show how low‐order d‐separation and d‐connection statements can be encoded as first‐order logical formula over graph reachability, then relax these discrete predicates into continuously differentiable “percolation” measures using probabilistic soft logic (LogLTN operators). This yields differentiable lower‐bounds on expected d‐separation/d‐connection that serve as losses alongside a log‐determinant acyclicity regularizer, forming the DAGPA algorithm. Empirical results on synthetic Erdős–Rényi and Scale‐Free graphs and the real‐world Sachs protein‐signaling dataset demonstrate DAGPA’s strong robustness in small‐sample regimes—surpassing both traditional constraint‐based (PC, k‐PC) and score‐based (NOTEARS, DAGMA) baselines—and competitive performance when ample data are available.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n1. The paper is exceptionally well written. Despite introducing many novel concepts and intricate notation, the authors clearly explain each idea and present their results in a coherent manner. To further aid comprehension, I suggest including a concrete example alongside each formal definition—particularly Definition 3.1—which currently feels under‐explained.\n\n2. The motivation and core idea are both compelling and original. The authors demonstrate how to embed the d-separation/d-connection criterion as a differentiable scoring mechanism within a gradient-based optimization framework. This novel integration clearly merits an accept.\n\n**Weaknesses**\n\nAs the authors acknowledge, the current approach only handles d-separation/d-connection for singleton conditioning sets. Extending this to larger conditioning sets would require exponentially many additional constraints, leading to severe computational challenges both in computing each individual score and in managing the objectives within the multi-task optimization.\n\nQUESTIONS:\nI recommend adding the final optimization objective to the main text—showing concretely how the differentiable d-separation/d-connection losses and the acyclicity regularizer combine into a single formula will greatly improve readability.\n\nIt would be helpful if the authors could analyze the computational complexity of DAGPA and report empirical runtimes on representative benchmarks in the main paper. Are there opportunities to accelerate the method (e.g.\\ parallelizing percolation calculations, exploiting sparsity in the graph, or using approximate reachability)?\n\nThe experimental metric plotted in Figures 2 and 3 (“Low-order CI Statement MCC vs. Empirical CDF”) is not self-explanatory. Could you share more details on this part.\n\nLIMITATIONS:\nYES", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a new \"style\" of gradient-based causal discovery approach, where the gradient based optimization incorporates CI constraints as in constraints-based CD methods. This differs from the usual, existing score based approach, which encodes DAG in the constraint. The paper derives theoretical sounds showing soundness. The paper also demonstrates empirically that the method developed works particularly well in low sample settings, showing it is more robust in such settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: The paper's motivation is interesting, aiming to produce a differentiable, constraint based causal discovery method. This could potentially lead to new lines of work just like how NOTEAR opened up a lot of directions and areas for improvement. I thought the paper did a nice job building up from d-separation and S_A to a main theoretical result (Lemma 4.2). The experiments do compare fairly against known baselines like NOTEARS, GES and PC (but the experiments are specifically evaluating for recovery of d-separation statements implied by the true underlying causal structure that generated the data, which is what the method optimizes).\n\nWeakness: With that said, I think more could be said about the optimization guarantees when optimizing the five multi-task loss functions (which was done in NOTEARS). I think the paper could also say more to motivate the necessity of having explicitly encoded constraint based approaches (what are some applications where this is important and NOTEARS is not sufficient).\n\nQUESTIONS:\n1. Can the authors comment more about the convergence of the (joint) optimization of the proposed objective function? This will make the paper more complete.\n\n2. Are there other replacements for the p-values, which are used as heuristic conditional independence measures?\n\n3. Is it correct to say that the approach developed in this paper can complement any existing score based approach? Specifically it can be added on top of any gradient-based method in that one can jointly optimize this paper's criterion and the criterion of the existing score based method. If yes, would this be better and if no, why not?\n\nLIMITATIONS:\nPlease see weakness section.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-30"}