Title: Learning DAGs from Data with Few Root Causes
Venue: NeurIPS 2023
Accepted: yes
Keywords: directed acyclic graph

few root causes

structural equation models

linear SEMs

additive noise
Abstract: We present a novel perspective and algorithm for learning directed acyclic graphs (DAGs) from data generated by a linear structural equation model (SEM). First, we show that a linear SEM can be viewed as a linear transform that, in prior work, computes the data from a dense input vector of random valued root causes (as we will call them) associated with the nodes. Instead, we consider the case of (approximately) few root causes and also introduce noise in the measurement of the data. Intuitively, this means that the DAG data is produced by few data generating events whose effect percolates through the DAG. We prove identifiability in this new setting and show that the true DAG is the global minimizer of the $L^0$-norm of the vector of root causes. For data satisfying the few root causes assumption, we show superior performance compared to prior DAG learning methods.

Learning DAGs from Data with Few Root Causes

Abstract

We present a novel perspective and algorithm for learning directed acyclic graphs (DAGs) from data generated by a linear structural equation model (SEM). First, we show that a linear SEM can be viewed as a linear transform that, in prior work, computes the data from a dense input vector of random valued root causes (as we will call them) associated with the nodes. Instead, we consider the case of (approximately) few root causes and also introduce noise in the measurement of the data. Intuitively, this means that the DAG data is produced by few data generating events whose effect percolates through the DAG. We prove identifiability in this new setting and show that the true DAG is the global minimizer of the L 0 -norm of the vector of root causes. For data satisfying the few root causes assumption, we show superior performance compared to prior DAG learning methods.

1 Introduction

We consider the problem of learning the edges of an unknown directed acyclic graph (DAG) given data indexed by its nodes. DAGs can represent causal dependencies (edges) between events (nodes) in the sense that an event only depends on its predecessors. Thus, DAG learning has applications in causal discovery, which, however, is a more demanding problem that we do not consider here, since it requires further concepts of causality analysis, in particular interventions [Peters et al., 2017]. However, DAG learning is still NP-hard in general [Chickering et al., 2004]. Hence, in practice, one has to make assumptions on the data generating process, to infer information about the underlying DAG in polynomial time. Recent work on DAG learning has focused on identifiable classes of causal data generating processes. A frequent assumption is that the data follow a structural equation model (SEM) [Shimizu et al., 2006, Zheng et al., 2018, Gao et al., 2021], meaning that the value of every node is computed as a function of the values of its direct parents plus noise. Zheng et al. [2018] introduced NOTEARS, a prominent example of DAG learning, which considers the class of linear SEMs and translates the acyclicity constraint into a continuous form for easier optimization. It inspired subsequent works to use continuous optimization schemes for both linear [Ng et al., 2020] and nonlinear SEMs [Lachapelle et al., 2019, Zheng et al., 2020]. A more expansive discussion of related work is provided in Section 4.

In this paper we also focus on linear SEMs but change the data generation process. We first translate the common representation of a linear SEM as recurrence into an equivalent closed form. In this form, prior data generation can be viewed as linearly transforming an i.i.d. random, dense vector of root causes (as we will call them) associated with the DAG nodes as input, into the actual data on the DAG nodes as output. Then we impose sparsity in the input (few root causes) and introduce measurement noise in the output. Intuitively, this assumption captures the reasonable situation that DAG data may be mainly determined by few data generation events of predecessor nodes that percolate through the DAG as defined by the linear SEM. Note that our use of the term root causes is related to but different from the one in the root cause analysis by Ikram et al. [2022].

Contributions. We provide a novel DAG learning method designed for DAG data generated by linear SEMs with the novel assumption of few root causes. Our specific contributions include the following:

We provide an equivalent, closed form of the linear SEM equation showing that it can be viewed as a linear transform obtained by the reflexive-transitive closure of the DAG's adjacency matrix. In this form, prior data generation assumed a dense, random vector as input, which we call root causes.

In contrast, we assume a sparse input, or few root causes (with noise) that percolate through the DAG to produce the output, whose measurement is also subject to noise. Interestingly, Seifert et al. [2023] identify the assumption of few root causes as a form of Fourier-sparsity.

We prove identifiability of our proposed setting under weak assumptions, including zero noise in the measurements. We also show that, given enough data, the original DAG is the unique minimizer of the associated optimization problem in the case of absent noise.

We propose a novel and practical DAG learning algorithm, called SparseRC, for our setting, based on the minimization of the L 1 -norm of the approximated root causes.

We benchmark SparseRC against prior DAG learning algorithms, showing significant improvements for synthetic data with few root causes and scalability to thousands of nodes. SparseRC also performs among the best on real data from a gene regulatory network, demonstrating that the assumption of few root causes can be relevant in practice.

2 Linear SEMs and Root Causes

We first provide background on prior data generation for directed acyclic graphs (DAGs) via linear SEMs. Then we present a different viewpoint on linear SEMs based on the concept of root causes that we introduce. Based on it, we argue for data generation with few root causes, present it mathematically and motivate it, including with a real-world example.

DAG. We consider DAGs G = ( V, E ) with | V | = d vertices, E the set of directed edges, no cycles including no self-loops. We assume the vertices to be sorted topologically and set accordingly V = { 1 , 2 , ..., d } . Further, a ij ∈ R is the weight of edge ( i, j ) ∈ E , and

is the weighted adjacency matrix. A is upper triangular with zeros on the diagonal and thus A d = 0 .

Linear SEM. Linear SEMs [Peters et al., 2017] formulate a data generating process for DAGs G . First, the values at the sources of a DAG G are initialized with random noise. Then, the remaining nodes are processed in topological order: the value x j at node j is assigned the linear combination of its parents' values (called the causes of x j ) plus independent noise. Mathematically, a data vector x = ( x 1 , ..., x d ) ∈ R d on G follows a linear SEM if x j = xA : ,j + n j , where the subscript : , j denotes column j , and n j are i.i.d. random noise variables [Shimizu et al., 2006, Zheng et al., 2018, Ng et al., 2020]. Assuming n such data vectors collected as the rows of a matrix X ∈ R n × d , and the noise variables in the matrix N , the linear SEM is typically written as

2.1 Transitive closure and root causes

Equation (2) can be viewed as a recurrence for computing the data values X from N . Here, we interpret linear SEMs differently by first solving this recurrence into a closed form. We define A = A + A 2 + ... + A d -1 , which is the Floyd-Warshall (FW) transitive closure of A over the ring ( R , + , · ) [Lehmann, 1977], and I + A the associated reflexive-transitive closure of A . Since A d = 0 we have ( I -A )( I + A ) = I and thus can isolate X in (2):

Lemma 2.1. The linear SEM (2) computes data X as

In words, the data values in X are computed as linear combinations of the noise values N of all predecessor nodes with weights given by the reflexive-transitive closure I + A .

Since X is uniquely determined by N , we call the latter the root causes of X . The root causes must not be confused with the root nodes (sources) of the DAG, which are the nodes without parents. In particular, N can be non-zero in nodes that are not sources.

Few root causes. In (3) we view N as the input to the linear SEM at each node and X as the measured output at each node. With this viewpoint, we argue that it makes sense to consider a data generation process that differs in two ways from (3) and thus (2). First, we assume that only few nodes produce a relevant input that we call C , up to low magnitude noise N c . Second, we assume that the measurement of X is subject to noise N x . Formally, this yields the closed form

Multiplying both sides by ( I + A ) -1 = ( I -A ) yields the (standard) form as recurrence

i.e., N in (2) is replaced by C + N c + N x ( I -A ) , which in general is not i.i.d. Note that (4) generalizes (3), obtained by assuming zero root causes C = 0 and zero measurement noise N x = 0 .

We consider C not as an additional noise variable but as the actual information, i.e., the relevant input data at each node, which then percolates through the DAG as determined by the SEM to produce the final output data X , whose measurement, as usual in practice, is subject to noise. Few root causes mean that only few nodes input relevant information in one dataset. This assumption can be stated as:

Definition 2.2 (Few Root Causes assumption) . Weassume that the input data ( C + N c )+ N x ( I -A ) to a linear SEM are approximately sparse. Formally this holds if:

where ∥·∥ 0 counts the nonzero entries of C , ∥·∥ 1 is elementwise, and ϵ, δ are small constants.

For example, in Appendix A we show that (6) holds (in expectation) for ϵ = δ = 0 . 1 if N c , N x are normally distributed with zero mean and standard deviation σ = 0 . 01 and the ground truth DAG G has uniformly random weights in [ -1 , 1] and average degree 4 . The latter two conditions bound the amplification of N x in (6) when multiplied by I -A .

The term root cause in the root cause analysis by Ikram et al. [2022] refers to the ancestor that was the only cause of some effect(s). If there is only one root cause in our sense, characterized by a non-zero value, then determining its location is the problem considered by Ikram et al. [2022].

2.2 Example: Pollution model

Linear SEMs and associated algorithms have been extensively studied in the literature [Loh and Bühlmann, 2014, Peters and Bühlmann, 2014, Ghoshal and Honorio, 2017, Aragam and Zhou, 2015]. However, we are not aware of any real-world example given in a publication. The motivation for using linear SEMs includes the following two reasons: d -variate Gaussian distributions can always be expressed as linear SEMs [Aragam and Zhou, 2015] and linearity is often a workable assumption when approximating non-linear systems.

Our aim is to explain why we propose the assumption of few root causes. The high-level intuition is that it can be reasonable to assume that, with the view point of (3), the relevant DAG data is triggered by sparse events on the input size and not by random noise. To illustrate this, we present a linear SEM that describes the propagation of pollution in a river network.

DAG. We assume a DAG describing a river network. The acyclicity is guaranteed since flows only occur downstream. The nodes i ∈ V represent geographical points of interest, e.g., cities, and edges are rivers connecting them. We assume that the cities can pollute the rivers. An edge weight a ij ∈ [0 , 1] , ( i, j ) ∈ E , captures what fraction of a pollutant inserted at i reaches the neighbour j . An example DAG with six nodes is depicted in Fig. 1a.

Transitive closure. Fig. 1b shows the transitive closure of the DAG in (a). The ( i, j ) -th entry of the transitive closure is denoted with a ij and represents the total fraction of a pollutant at i that reaches j via all connecting paths.

0.00

0

.

5

0

.

5

0

.

8

0

.

3

0

.

7

0

.

1

(a) Original DAG

0

.

5

0

.

5

0

.

8

0

.

3

0

.

7

0

.

1

0

.

39

0

.

06

0

.

55

0

.

56

0

.

03

0

.

08

0

.

21

(b) Transitive closure

3

1.5

1.5

6.65

4.66

0.67

(c) Measurements vector

3

0

0

5

0

(d) Root causes

Data and root causes. Fig. 1c shows a possible data vector x on the DAG, for example, the pollution measurement at each node done once a day (and without noise in this case). The measurement is the accumulated pollution from all upstream nodes. Within the model, the associated root causes c in Fig. 1d then show the origin of the pollution, two in this case. Sparsity in C means that each day only a small number of cities pollute. Negligible pollution from other sources is captured by noise N c and N x models the noise in the pollution measurements (both are assumed to be 0 in Fig. 1).

We view the pollution model as an abstraction that can be transferred to other real-world scenarios. For example, in gene networks that measure gene expression, few root causes would mean that few genes are activated in a considered dataset. In a citation network where one measures the impact of keywords/ideas, few root causes would correspond to the few origins of them.

3 Learning the DAG

In this section we present our approach for recovering the DAG adjacency matrix A from given data X under the assumption of few root causes, i.e., (4) (or (5)) and (6). We first show that under no measurement noise, our proposed setting is identifiable. Then we theoretically analyze our data generating model in the absence of noise and prove that the true DAG adjacency A is the global minimizer of the L 0 -norm of the root causes. Finally, to learn the DAG in practice, we perform a continuous relaxation to obtain an optimization problem that is solvable with differentiation.

3.1 Identifiability

The rows of the data X generated by a linear SEM are commonly interpreted as observations of a random row vector X = ( X 1 , ..., X d ) [Peters and Bühlmann, 2014, Zheng et al., 2018], i.e., it is written analogous to (2) as X i = X A : ,i + N i , where N i are i.i.d. zero-mean random noise variables, or, equivalently, as X = X A + N . Given a distribution P N of the noise vector, the DAG with graph adjacency matrix A is then called identifiable if it is uniquely determined by the distribution P X of X . Shimizu et al. [2006] state that A is identifiable if all N i are non-Gaussian noise variables. This applies directly to our setting and yields the following result.

Theorem 3.1. Assume the data generation X = ( C + N c ) ( I + A ) , where C and N c are independent. Let p ∈ [0 , 1) and assume the C i are independent random variables taking uniform values from [0 , 1] with probability p , and are = 0 with probability 1 -p . The noise vector N c is defined as before. Then the DAG given by A is identifiable.

Proof. Using (5), the data generation equation can be viewed as a linear SEM (in standard recursive form) with noise variable C + N c , which is non-Gaussian due to Lévy-Cramér decomposition theorem [Lévy, 1935, Cramér, 1936], because C is non-Gaussian. The statement then follows from LiNGAM [Shimizu et al., 2006], given that the noise variables of C and N c are independent. Note that this independency would be violated if we assumed non-zero measurement noise N x , as in (4).

In Theorem 3.1, C yields sparse observations whenever p is close to zero (namely dp root causes in expectation). However, identifiability follows for all p due to non-Gaussianity.

3.2 L 0 minimization problem and global minimizer

Suppose that the data X are generated via the noise-free version of (4):

We assume C to be generated as in Theorem 3.1, i.e., with randomly uniform values from [0 , 1] with probability p and 0 with probability 1 -p , where p is small such that C is sparse. Given the data X we aim to find A by enforcing maximal sparsity on the associated C , i.e., by minimizing ∥ C ∥ 0 :

The following Theorem 3.2 states that given enough data, the true A is the unique global minimizer of (8). This means that in that case the optimization problem (8) correctly specifies the true DAG. One can view the result as a form of concrete, non-probabilistic identifiability. Note that Theorem 3.2 is not a sample-complexity result, but a statement of exact reconstruction in the absence of noise. The sample complexity of our algorithm is empirically evaluated later.

Theorem 3.2. Consider a DAG with weighted adjacency matrix A with d nodes. Given exponential (in d ) number n of samples X the matrix A is, with high probability, the global minimizer of the optimization problem (8) .

Proof.

3.3 Continuous relaxation

In practice the optimization problem (8) is too expensive to solve due its combinatorial nature, and, of course, the noise-free assumption rarely holds in real-world data. We now consider again our general data generation in (4) assuming sparse root causes C and noises N c , N x satisfying the criterion (6). To solve it we consider a continuous relaxation of the optimization objective. Typically (e.g., [Zheng et al., 2018, Lee et al., 2019, Bello et al., 2022]), continuous optimization DAG learning approaches have the following general formulation:

where ℓ ( A , X ) is the loss function corresponding to matching the data, R ( A ) is a regularizer that promotes sparsity in the adjacency matrix, usually equal to λ ∥ A ∥ 1 , and h ( A ) is a continuous constraint enforcing acyclicity.

In our case, following a common practice in the literature, we substitute the L 0 -norm from (8) with its convex relaxation [Ramirez et al., 2013], the L 1 -norm. Doing so allows for some robustness to (low magnitude) noise N c and N x . We capture the acyclicity with the continuous constraint h ( A ) = tr ( e A ⊙ A ) -d used by Zheng et al. [2018], but could also use the form of Yu et al. [2019]. As sparsity regularizer for the adjacency matrix and we choose R ( A ) = λ ∥ A ∥ 1 . Hence, our final continuous optimization problem is

We call this method SparseRC (sparse root causes).

4 Related work

The approaches to solving the DAG learning problem fall into two categories: combinatorial search or continuous relaxation. In general, DAG learning is an NP-hard problem since the combinatorial space of possible DAGs is super-exponential in the number of vertices [Chickering et al., 2004]. Thus, methods that search the space of possible DAGs apply combinatorial strategies to find an approximation of the ground truth DAG [Ramsey et al., 2017, Chickering, 2002, Tsamardinos et al., 2006].

Continuous optimization. Lately, with the advances of deep learning, researchers have been focusing on continuous optimization methods [Vowels et al., 2021] modelling the data generation process using SEMs. Among the first methods to utilize SEMs were CAM [Bühlmann et al., 2014] and LiNGAM [Shimizu et al., 2006], which specializes to linear SEMs with non-Gaussian noise. NOTEARS by Zheng et al. [2018] formulates the combinatorial constraint of acyclicity as a continuous one, which enables the use of standard optimization algorithms. Despite concerns, such as lack of scale-invariance [Kaiser and Sipos, 2022, Reisach et al., 2021], it has inspired many subsequent DAG learning methods. The current state-of-the-art of DAG learning methods for linear SEMs include DAGMA by Bello et al. [2022], which introduces a log-det acyclicity constraint, and GOLEM by Ng et al. [2020], which studies the role of the weighted adjacency matrix sparsity, the acyclicity constraint, and proposes to directly minimize the data likelihood. Zheng et al. [2020] extended NOTEARS to apply in non-linear SEMs. Other nonlinear methods for DAG learning include DAG-GNN by Yu et al. [2019], in which also a more efficient acyclicity constraint than the one in NOTEARS is proposed, and DAG-GAN by Gao et al. [2021]. DAG-NoCurl by Yu et al. [2021] proposes learning the DAG on the equivalent space of weighted gradients of graph potential functions. Pamfil et al. [2020] implemented DYNOTEARS, a variation of NOTEARS, compatible with time series data. A recent line of works considers permutation-based methods to parameterize the search space of the DAG [Charpentier et al., 2022, Zantedeschi et al., 2022]. The work in this paper considers DAG learning under the new assumption of few root causes. In Misiakos et al. [2024] we further build on this work by learning graphs from time series data.

Fourier analysis. Our work is also related to the recently proposed form of Fourier analysis on DAGs from Seifert et al. [2023, 2022], where equation (3) appears as a special case. The authors argue that (what we call) the root causes can be viewed as a form of spectrum of the DAG data. This means the assumption of few root causes is equivalent to Fourier-sparsity, a frequent assumption for common forms of Fourier transform [Hassanieh, 2018, Stobbe and Krause, 2012, Amrollahi et al., 2019]. Experiments in [Seifert et al., 2023, Misiakos et al., 2024] reconstruct data from samples under this assumption.

5 Experiments

We experimentally evaluate our DAG learning method SparseRC with both synthetically generated data with few root causes and real data from gene regulatory networks [Sachs et al., 2005]. We also mention that our method was among the winning solutions [Misiakos et al., 2023, Chevalley et al., 2023] of a competition of DAG learning methods to infer causal relationships between genes [Chevalley et al., 2022]. The evaluation was done on non-public data by the organizing institution and is thus not included here.

Benchmarks. We compare against prior DAG learning methods suitable for data generated by linear SEMs with additive noise. In particular, we consider the prior GOLEM [Ng et al., 2020], NOTEARS [Zheng et al., 2018], DAGMA [Bello et al., 2022], DirectLiNGAM [Shimizu et al., 2011], PC [Spirtes et al., 2000] and the greedy equivalence search (GES) [Chickering, 2002]. We also compared SparseRC against LiNGAM [Shimizu et al., 2006], causal additive models (CAM) [Bühlmann et al., 2014], DAG-NoCurl [Yu et al., 2021], fast greedy equivalence search (fGES) [Ramsey et al., 2017], the recent simple baseline sortnregress [Reisach et al., 2021] and max-min hill-climbing (MMHC) [Tsamardinos et al., 2006], but they where not competitive and thus we only include the results in Appendix B.

ˆ

Metrics. To evaluate the found approximation A of the true adjacency matrix A , we use common performance metrics as in [Ng et al., 2020, Reisach et al., 2021]. In particular, the structural Hamming distance (SHD), which is the number of edge insertions, deletions, or reverses needed to convert ˆ A to A , and the structural intervention distance (SID), introduced by Peters and Bühlmann [2015], which is the number of falsely estimated intervention distributions. SHD gives a good estimate of a method's performance when it is close to 0 . In contrast, when it is high it can either be that the approximated DAG has no edges in common with the ground truth, but it can also be that the approximated DAG contains all the edges of the original one together with some false positives. Thus, in such cases, we also consider the true positive rate (TPR) of the discovered edges. We also report the total number of nonzero edges discovered for the real dataset [Sachs et al., 2005] and, for the synthetic data, provide results reporting the TPR, SID, and normalized mean square error (NMSE) metrics in Appendix B. For the methods that can successfully learn the underlying graph we further

Hyperparameter

Default

Change

Varsort.

SparseRC (ours)

GOLEM

NOTEARS

DAGMA

PC

GES

1

Default settings

0

.

95

0

.

6

±

0

.

8

82

±

34

59

±

22

269

±

6

.

0

282

±

34

247

±

9

.

0

failure

2

Graph type

Scale-free

0

.

99

2

.

2

±

1

.

5

34

±

9

.

0

28

±

9

.

5

296

±

14

188

±

27

275

±

16

375

±

141

3

N

c

,

N

x

distribution

Gumbel

0

.

97

1

.

4

±

1

.

0

87

±

44

59

±

17

278

±

7

.

4

287

±

43

250

±

17

396

±

33

4

Edges / Vertices

4

10

0

.

99

46

±

7

.

5

212

±

70

243

±

26

896

±

30

1078

±

105

973

±

17

failure

5

Standardization

No

Yes

0

.

50

624

±

48

failure

failure

failure

278

±

25

247

±

18

failure

6

Larger weights in

A

(0

.

1

,

0

.

9)

(0

.

5

,

2)

1

.

00

failure

96

±

25

92

±

14

209

±

4

.

1

840

±

121

400

±

9

.

8

failure

7

N

c

,

N

x

deviation

σ

= 0

.

01

σ

= 0

.

1

0

.

97

504

±

19

98

±

14

199

±

12

238

±

13

538

±

45

255

±

11

failure

8

Dense root causes

C

p

= 0

.

1

p

= 0

.

5

0

.

98

1221

±

33

29

±

2

.

5

126

±

32

83

±

8

.

0

257

±

14

244

±

12

failure

9

Samples

n

= 1000

n

= 100

0

.

97

2063

±

92

failure

failure

328

±

13

error

351

±

12

failure

10

Fixed support

No

Yes

0

.

89

failure

failure

failure

failure

failure

379

±

37

failure

proceed on approximating the true root causes C that generated the data X via (4). With regard to this aspect, we count the number of correctly detected root causes C TPR, their false positive rate C FPR and the weighted approximation C NMSE. We also report the runtime for all methods in seconds. For each performance metric, we compute the average and standard deviation over five repetitions of the same experiment.

Our implementation. To solve (10) in practice, we implemented 1 a PyTorch model with a trainable parameter representing the weighted adjacency matrix A . This allows us to utilize GPUs for the acceleration of the execution of our algorithm (GOLEM uses GPUs, NOTEARS does not). Then we use the standard Adam [Kingma and Ba, 2014] optimizer to minimize the loss defined in (10).

5.1 Evaluation on data with few root causes

Data generating process and defaults. In the second, blue column of Table 1 we report the default settings for our experiment. We generate a random Erdös-Renyi graph with d = 100 nodes and assign edge directions to make it a DAG as in [Zheng et al., 2018]. The ratio of edges to vertices is set to 4 , so the number of edges is 400 . The entries of the weighted adjacency matrix are sampled uniformly at random from ( -b, -a ) ∪ ( a, b ) , where a = 0 . 1 and b = 0 . 9 . As in [Zheng et al., 2018, Ng et al., 2020, Bello et al., 2022] the resulting adjacency matrix is post-processed with thresholding. In particular, the edges with absolute weight less than the threshold ω = 0 . 09 are discarded. Next, the root causes C are instantiated by setting each entry either to some random uniform value from (0 , 1) with probability p = 0 . 1 or to 0 with probability 1 -p = 0 . 9 (thus, as in Theorem 3.1, the location of the root causes will vary). The data matrix X is computed according to (4), using Gaussian noise N c , N x of standard deviation σ = 0 . 01 to enforce (6). Finally, we do not standardize (scale for variance = 1 ) the data, and X contains n = 1000 samples (number of rows).

Experiment 1: Different application scenarios. Table 1 compares SparseRC to six prior algorithms using the SHD metric. Every row corresponds to a different experiment that alters one particular hyperparameter of the default setting, which is the first row with values of the blue column as explained above. For example, the second row only changes the graph type from Erdös-Renyi to scale-free, while keeping all other settings. Note that in the last row, fixed support means that the location of the root causes is fixed for every sample in the data.

Since the graphs have 400 edges, an SHD ≪ 400 can be considered as good and beyond 400 can be considered a failure. In row 3 this threshold is analogously set to 1000 . The failure cases are indicated as such in Table 1, or the SHD is shown in green if they still achieve a good TPR > 0 . 8 . In Appendix B.1 we report the TPR, SID, and the runtimes of all methods.

In the first three rows, we examine scenarios that perfectly match the condition (6) of few root causes. For the default settings (row 1), scale-free graphs (row 2), and different noise distribution (row 3) SparseRC performs best and almost perfectly detects all edges. The next experiments alter parameters that deteriorate the few root causes assumption. Row 4 considers DAGs with average degree 10 , i.e., about 1000 edges. High degree can amplify the measurement noise and hinder the root causes assumption (6). However, our method still performs best, but even larger degrees decay the performance of all methods as shown in experiment 5 below. The standardization of data (row

5) is generally known to negatively affect algorithms with continuous objectives [Reisach et al., 2021] as is the case here. Also, standardization changes the relative scale of the root causes and as a consequence affects (6). Row 6 considers edge weights > 1 which amplifies the measurement noise in (6) and our method fails. Higher standard deviation in the root causes noise (row 7) or explicitly higher density in C (row 8) decreases performance overall. However, In both cases SparseRC still discovers a significant amount of the unknown edges. For a small number of samples (row 9) most methods fail. Ours achieves a high TPR but requires more data to converge to the solution as we will see in experiment 2 below. Row 10 is out of the scope of our method and practically all methods fail.

Overall, the performance of SparseRC depends heavily on the degree to which the assumption of few root causes (6) is fulfilled. The parameter choices in the table cover a representative set of possibilities from almost perfect recovery (row 1) to complete failure (row 10).

SparseRC is also significantly faster than the best competitors GOLEM and NOTEARS with typical speedups in the range of 10 -50 × as shown in Table 2. It is worth mentioning that even though LiNGAM provides the identifiability Theorem 3.1 of our proposed setting, it is not able to recover the true DAG. While both DirectLiNGAM and LiNGAM come with theoretical guarantees for their convergence, these require conditions, such as infinite amount of data, which in practice are not met. We include a more extensive reasoning for their subpar performance in Appendix B.4 together a particularly designed experiment for LiNGAM and DirectLiNGAM.

Varsortability. In Table 1 we also include the varsortability for each experimental setting. Our measurements for Erdös-Renyi graphs (all rows except row 2) are typically 1 -2% lower than those reported in [Reisach et al., 2021, Appendix G.1] for linear SEMs, but still high in general. However, the trivial variance-sorting method sortnregress, included in Appendix B, fails overall. Note again that for fixed sparsity support (last row), all methods fail and varsortability is lower. Therefore, in this scenario, our data generating process poses a hard problem for DAG learning.

Experiment 2: Varying number of nodes or samples. In this experiment we first benchmark SparseRC with varying number of nodes (and thus number of edges) in the ground truth DAG, while keeping the data samples in X equal to n = 1000 . Second, we vary the number of samples, while keeping the number of nodes fixed to d = 100 . All other parameters are set to default. The results are shown in Fig. 2 reporting SHD, SID, and the runtime.

When varying the number of nodes (first row in Fig. 2) SparseRC, GOLEM, and NOTEARS achieve very good performance whereas the other methods perform significantly worse. As the number of nodes increases, SparseRC performs best both in terms of SHD and SID while being significantly faster than GOLEM and NOTEARS. When increasing the number of samples (second row) the overall performances improve. For low number of samples SparseRC, GOLEM and NOTEARS fail. Their performance significantly improves after 500 samples where SparseRC overall achieves the best result. The rest of the methods have worse performance. SparseRC is again significantly faster than GOLEM and NOTEARS in this case.

Experiment 3: Learning the root causes. Where our method succeeds we can also recover the root causes that generate the data. Namely, if we recover a very good estimate of the true adjacency matrix via (10), we may compute an approximation ̂ C of the root causes C , up to noise, by solving (4):

In the last row of Fig. 2 we evaluate the top-performing methods on the recovery of the root causes (and the associated values) with respect to detecting their locations ( C TPR and FPR) and recovering

20

40

60

80

100

Number of nodes

0

100

200

300

400

500

600

700

SHD

100200

500

800

1000

Number of samples

0

100

200

300

400

500

SHD

(a) SHD (

↓

)

20

40

60

80

100

Number of nodes

0

1000

2000

3000

4000

5000

6000

7000

8000

SID

100200

500

800

1000

Number of samples

0

2000

4000

6000

8000

SID

(b) SID (

↓

)

20

40

60

80

100

Number of nodes

0

100

200

300

400

500

Time (s)

100200

500

800

1000

Number of samples

0

100

200

300

400

500

Time (s)

(c) Runtime [s]

SparseRC

GOLEM

NOTEARS

DAGMA

PC

GES

20

40

60

80

100

Number of nodes

0.75

0.80

0.85

0.90

0.95

1.00

C

TPR

(d) Root causes TPR (

↑

)

20

40

60

80

100

Number of nodes

0.00

0.01

0.02

0.03

0.04

C

FPR

(e) Root causes FPR (

↓

)

20

40

60

80

100

Number of nodes

0.1

0.2

0.3

0.4

0.5

0.6

0.7

C

NMSE

(f) Root causes NMSE(

↓

)

SparseRC

GOLEM

NOTEARS

(a) SHD ( ↓ )

(b) Runtime (s)

their numerical values using the C NMSE metric. The support is chosen as the entries ( i, j ) of C that are in magnitude within 10% of the largest value in C (i.e., relatively large): c ij > 0 . 1 · C max. We consider the default experimental settings.

Experiment 4: Larger DAGs. In Table 3, we investigate the scalability of the best methods SparseRC, GOLEM, and NOTEARS from the previous experiments. We consider five different settings up to d = 3000 nodes and n = 10000 samples. We use the prior default settings except for the sparsity of the root causes, which is now set to 5% . The metrics here are SHD and the runtime . The results show that SparseRC excels in the very sparse setting with almost perfect reconstruction, far outperforming the others including in runtime. SparseRC even recovers the edge weights with an average absolute error of about 5% for each case in Table 3, as shown in Appendix B.3.

Experiment 5: Varying average degree. We evaluate the top-scoring methods on DAGs with higher average degree (dense DAGs). Note that this violates the typical assumption of sparse DAGs [Zheng et al., 2018]. In Fig. 3 we consider default settings with DAGs of average degree up to 20 . SID was not computed due to the presence of cycles in the output of the algorithms and TPR is reported instead. We conclude that for dense DAGs the performance decays, as expected.

5

10

15

20

Avg. degree

0

500

1000

1500

2000

SHD

(a) SHD (

↓

)

5

10

15

20

Avg. degree

0.4

0.5

0.6

0.7

0.8

0.9

1.0

TPR

(b) TPR (

↑

)

5

10

15

20

Avg. degree

0

50

100

150

200

250

Time (s)

(c) Runtime [s]

SparseRC

GOLEM

NOTEARS

5.2 Evaluation on a real dataset

We apply SparseRC on the causal protein-signaling network data provided by Sachs et al. [2005]. The dataset consists of 7466 samples (we use the first 853 ) for a network with 11 nodes that represent proteins, and 17 edges representing their interactions. It is small, but the task of learning it has been considered a difficult benchmark in [Ng et al., 2020, Gao et al., 2021, Yu et al., 2019, Zheng et al., 2018]. It is not known whether the assumption of few root causes holds in this case. We report the performance metrics SHD and SID for the most successful methods in Table 4. The number of total edges is used to ensure that the output of the methods are non-trivial (e.g. empty graph).

The best SID is achieved by GOLEM (equal variance), which, however, has higher SHD. NOTEARS has the best SHD equal to 11 . Overall, SparseRC performs well, achieving the closest number of edges to the real one with 16 and a competitive SHD and SID. We also mention again the good performance of SparseRC on a non-public biological dataset [Misiakos et al., 2023, Chevalley et al., 2023] as part of the competition by Chevalley et al. [2022].

6 Broader Impact and Limitations

Our method inherits the broader impact of prior DAG learning methods including the important caveat for practical use that the learned DAG may not represent causal relations, whose discovery requires interventions. Further limitations that we share with prior work include (a) Learning DAGs beyond 10000 nodes are out of reach, (b) there is no theoretical convergence guarantee for the case that includes noise, (c) empirically, the performance drops in low varsortability, (d) our method is designed for linear SEMs like most of the considered benchmarks.

A specific limitation of our contribution is that it works well only for few root causes of varying location in the dataset. This in addition implies that the noise must have low deviation and the graph to have low average degree and weights of magnitude less than one. Also, in our experiments, we only consider root causes with support that follows a multivariate Bernoulli distribution.

7 Conclusion

We presented a new perspective on linear SEMs by introducing the notion of root causes. Mathematically, this perspective translates (or solves) the recurrence describing the SEM into an invertible linear transformation that takes as input DAG data, which we call root causes, to produce the observed data as output. Prior data generation for linear SEMs assumed a dense, random valued input vector. In this paper, we motivated and studied the novel scenario of data generation and DAG learning for few root causes, i.e., a sparse input with noise, and noise in the measurement data. Our solution in this setting performs significantly better than prior algorithms, in particular for high sparsity where it can even recover the edge weights, is scalable to thousands of nodes, and thus expands the set of DAGs that can be learned in real-world scenarios where current methods fail.

References

Amrollahi, A. Zandieh, M. Kapralov, and A. Krause. Efficiently learning Fourier sparse set functions. Advances in Neural Information Processing Systems , 32, 2019.

Aragam and Q. Zhou. Concave penalized estimation of sparse Gaussian Bayesian networks. The Journal of Machine Learning Research , 16(1):2273-2328, 2015.

Bello, B. Aragam, and P. Ravikumar. DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization. Advances in Neural Information Processing Systems , 35:8226-8239, 2022.

Bühlmann, J. Peters, and J. Ernest. CAM: Causal additive models, high-dimensional order search and penalized regression. The Annals of Statistics , 42(6):2526-2556, 2014.

Charpentier, S. Kibler, and S. Günnemann. Differentiable DAG Sampling. In International Conference on Learning Representations , 2022.

Chevalley, Y. Roohani, A. Mehrjou, J. Leskovec, and P. Schwab. CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data. arXiv preprint arXiv:2210.17283 , 2022.

Chevalley, J. Sackett-Sanders, Y. Roohani, P. Notin, A. Bakulin, D. Brzezinski, K. Deng, Y. Guan, J. Hong, M. Ibrahim, et al. The CausalBench challenge: A machine learning contest for gene network inference from single-cell perturbation data. arXiv preprint arXiv:2308.15395 , 2023.

M. Chickering. Optimal structure identification with greedy search. Journal of machine learning research , 3 (Nov):507-554, 2002.

Chickering, D. Heckerman, and C. Meek. Large-sample learning of Bayesian networks is NP-hard. Journal of Machine Learning Research , 5:1287-1330, 2004.

Cramér. Über eine eigenschaft der normalen verteilungsfunktion. Mathematische Zeitschrift , 41(1):405-414, 1936.

Gao, L. Shen, and S.-T. Xia. DAG-GAN: Causal structure learning with generative adversarial nets. In International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 3320-3324. IEEE, 2021.

Ghoshal and J. Honorio. Learning identifiable Gaussian Bayesian networks in polynomial time and sample complexity. Advances in Neural Information Processing Systems , 30, 2017.

Hassanieh. The Sparse Fourier Transform: Theory and Practice , volume 19. Association for Computing Machinery and Morgan and Claypool, 2018.

Ikram, S. Chakraborty, S. Mitra, S. Saini, S. Bagchi, and M. Kocaoglu. Root cause analysis of failures in microservices through causal discovery. Advances in Neural Information Processing Systems , 35:3115831170, 2022.

Kaiser and M. Sipos. Unsuitability of NOTEARS for Causal Graph Discovery when dealing with Dimensional Quantities. Neural Processing Letters , pages 1-9, 2022.

Kalainathan and O. Goudet. Causal discovery toolbox: Uncover causal relationships in python. arXiv preprint arXiv:1903.02278 , 2019. URL https://github.com/FenTechSolutions/CausalDiscoveryToolbox .

P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Lachapelle, P. Brouillard, T. Deleu, and S. Lacoste-Julien. Gradient-based neural dag learning. In International Conference on Learning Representations , 2019.

H.-C. Lee, M. Danieletto, R. Miotto, S. T. Cherng, and J. T. Dudley. Scaling structural learning with NO-BEARS to infer causal transcriptome networks. In Pacific Symposium on Biocomputing , pages 391-402. World Scientific, 2019.

J. Lehmann. Algebraic structures for transitive closure. Theoretical Computer Science , 4(1):59-76, 1977.

Lévy. Propriétés asymptotiques des sommes de variables aléatoires enchaînées. Bull. Sci. Math , 59(84-96): 109-128, 1935.

P.-L. Loh and P. Bühlmann. High-dimensional learning of linear causal networks via inverse covariance estimation. The Journal of Machine Learning Research , 15(1):3065-3105, 2014.

Misiakos, C. Wendler, and M. Püschel. Learning Gene Regulatory Networks under Few Root Causes assumption. OpenReview , 2023. URL https://openreview.net/pdf?id=TOaPl9tXlmD .

Misiakos, V. Mihal, and M. Püschel. Learning signals and graphs from time-series graph data with few causes. In International Conference in Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2024. To appear.

Ng, A. Ghassami, and K. Zhang. On the role of sparsity and DAG constraints for learning linear DAGs. Advances in Neural Information Processing Systems , 33:17943-17954, 2020.

Pamfil, N. Sriwattanaworachai, S. Desai, P. Pilgerstorfer, K. Georgatzis, P. Beaumont, and B. Aragam. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics , pages 1595-1605. PMLR, 2020.

Peters and P. Bühlmann. Identifiability of Gaussian structural equation models with equal error variances. Biometrika , 101(1):219-228, 2014.

Peters and P. Bühlmann. Structural intervention distance for evaluating causal graphs. Neural computation , 27 (3):771-799, 2015.

Peters, D. Janzing, and B. Schölkopf. Elements of causal inference: foundations and learning algorithms . The MIT Press, 2017.

Ramirez, V. Kreinovich, and M. Argaez. Why ℓ 1 is a good approximation to ℓ 0 : A Geometric Explanation. Journal of Uncertain Systems , 7(3):203-207, 2013.

Ramsey, M. Glymour, R. Sanchez-Romero, and C. Glymour. A million variables and more: the fast greedy equivalence search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images. International journal of data science and analytics , 3(2):121-129, 2017.

Reisach, C. Seiler, and S. Weichwald. Beware of the simulated DAG! causal discovery benchmarks may be easy to game. Advances in Neural Information Processing Systems , 34:27772-27784, 2021.

Sachs, O. Perez, D. Pe'er, D. A. Lauffenburger, and G. P. Nolan. Causal protein-signaling networks derived from multiparameter single-cell data. Science , 308(5721):523-529, 2005.

Seifert, C. Wendler, and M. Püschel. Learning Fourier-Sparse Functions on DAGs. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality , 2022.

Seifert, C. Wendler, and M. Püschel. Causal Fourier Analysis on Directed Acyclic Graphs and Posets. IEEE Trans. Signal Process. , 71:3805-3820, 2023. doi: 10.1109/TSP.2023.3324988.

Shimizu, P. O. Hoyer, A. Hyvärinen, and A. Kerminen. A Linear Non-Gaussian Acyclic Model for Causal Discovery. Journal of Machine Learning Research , 7(72):2003-2030, 2006. URL http://jmlr.org/ papers/v7/shimizu06a.html .

Shimizu, T. Inazumi, Y. Sogawa, A. Hyvarinen, Y. Kawahara, T. Washio, P. O. Hoyer, K. Bollen, and P. Hoyer. Directlingam: A direct method for learning a linear non-gaussian structural equation model. Journal of Machine Learning Research-JMLR , 12(Apr):1225-1248, 2011.

Spirtes, C. N. Glymour, R. Scheines, and D. Heckerman. Causation, prediction, and search . MIT press, 2000.

Stobbe and A. Krause. Learning Fourier sparse set functions. In Artificial Intelligence and Statistics , pages 1125-1133. PMLR, 2012.

Tsagris, C. Beneki, and H. Hassani. On the folded normal distribution. Mathematics , 2(1):12-28, 2014. ISSN 2227-7390. doi: 10.3390/math2010012. URL https://www.mdpi.com/2227-7390/2/1/12 .

Tsamardinos, L. E. Brown, and C. F. Aliferis. The max-min hill-climbing Bayesian network structure learning algorithm. Machine learning , 65(1):31-78, 2006.

J. Vowels, N. C. Camgoz, and R. Bowden. D'ya like DAGs? A survey on structure learning and causal discovery. ACM Computing Surveys (CSUR) , 2021.

Yu, J. Chen, T. Gao, and M. Yu. DAG-GNN: DAG structure learning with graph neural networks. In International Conference on Machine Learning , pages 7154-7163. PMLR, 2019.

Yu, T. Gao, N. Yin, and Q. Ji. DAGs with no curl: An efficient DAG structure learning approach. In International Conference on Machine Learning , pages 12156-12166. PMLR, 2021.

Zantedeschi, J. Kaddour, L. Franceschi, M. Kusner, and V. Niculae. DAG Learning on the Permutahedron. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality , 2022.

Zheng, B. Aragam, P. K. Ravikumar, and E. P. Xing. Dags with no tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems , 31, 2018.

Zheng, C. Dan, B. Aragam, P. Ravikumar, and E. Xing. Learning Sparse Nonparametric DAGs. In International Conference on Artificial Intelligence and Statistics , pages 3414-3425. PMLR, 2020.

A Few Root Causes Assumption

Lemma A.1. Consider random row vector C ∈ R 1 × d of root causes that are generated as in Section 5.1 with probability p = 0 . 1 being non-zero and value taken uniformly at random from (0 , 1) . Also, let the noise vectors N c , N x ∈ R 1 × d be Gaussian N c , N x ∼ N ( 0 , σ I ) with σ = 0 . 01 and the DAG has average degree δ ( G ) = 4 and weights a ij ∈ [ -1 , 1] . Then we show that for ϵ = δ = 0 . 1 :

Proof. From the Bernoulli distribution for the support of the root causes we get

Also the root causes take uniform values in (0 , 1) and have expected value = 0 . 5 . Therefore,

The computation of the expected value follows from the mean of the Folded normal distribution [Tsagris et al., 2014]. Dividing the last two relations gives the required relation

B Additional experimental results

B.1 Different application scenarios

Experiment 1: Different application scenarios. We present additional experimental results to include more metrics and baselines. The results support our observations and conclusions from the main text. First, we expand Table 1 reporting the SHD metric and provide the computation of SHD, TPR, SID and runtime in the tables below. Those further include the methods LiNGAM [Shimizu et al., 2006], CAM [Bühlmann et al., 2014], DAG-NoCurl [Yu et al., 2021], fGES [Ramsey et al., 2017], sortnregress [Reisach et al., 2021] and MMHC [Tsamardinos et al., 2006] .

Finally, we have that:

B.2 Varying number of nodes or samples

Experiment 2: Varying number of nodes or samples. In addition to Fig. 2 we include the plots of Fig. 4 for the experiments that vary the number of nodes of the ground truth DAG or the number of samples in the data. The plots include the methods LiNGAM, CAM, DAG-NoCurl, fGES, sortnregress and MMHC and additionally contain the metrics TPR and total number of edges regarding the unweighted adjacency matrix and NMSE with respect to the weighted approximation of the adjacency matrix.

B.3 Larger DAGs

Experiment 3: Larger DAGs. In the large scale experiment we don't report SID as it is computationally too expensive for a large number of nodes. Since, our method achieved perfect reconstruction, we further investigate whether the true weights were recovered. Table 9 reports metrics that evaluate the weighted approximation of the true adjacency matrix. Denoting with | E | the number of edges of the ground truth DAG G = ( V, E ) , we compute:

the average L 1 loss ∥ A -̂ A ∥ 1 | E | ,

the MaxL 1 loss max i,j ∣ ∣ ∣ A ij -̂ A ij ∣ ∣ ∣ ,

the average L 2 loss ∥ A -̂ A ∥ 2 | E | and

the NMSE ∥ A -̂ A ∥ 2 ∥ A ∥ 2 .

B.4 LiNGAM's performance

The subpar performance of DirectLiNGAM arises as a contradiction to the fact that [Shimizu et al., 2006] provide the identifiability result for our setting. A possible reasoning to this can be that the corresponding linear noise variances as in (5) are non i.i.d., where as both LiNGAM and DirectLiNGAM consider i.i.d. noise distribution. In Fig. 5, we evaluate LiNGAM and DirectLiNGAM with respect to ours, using default settings, but with zero measurement noise N x = 0 , which translates (4) to a linear SEM with i.i.d. noise. We conclude that the methods cannot recover the true graph even in these settings. The reasoning we give for their failure is the following: First, LiNGAM can stay on a local optimum due to a badly chosen initial state in the ICA step of the algorithm, or even compute a wrong ordering of the variables [Shimizu et al., 2011]. Secondly, DirectLiNGAM is guaranteed to converge to the true solution only if the conditions are striclty met, among which is the infinite number of available data, which is not the case in our experiments.

B.5 Real Data

In the real dataset from [Sachs et al., 2005] we follow Lachapelle et al. [2019] and only use the first 853 samples from the dataset. Differences may occur between the reported results and the literature due to different choice of hyperparameters and the use of 853 samples, where others might utilize the full dataset.

Root causes in real data. To question whether the few root causes assumption is valid in the real dataset of [Sachs et al., 2005] we perform the following experiment. Using the ground truth matrix in the [Sachs et al.,

Table 5: SHD metric (lower is better) for learning DAGs with 100 nodes and 400 edges. Each row is an experiment. The first row is the default, whose settings are in the blue column. In each other row, exactly one default parameter is changed (change). The best results are shown in bold. Entries with SHD higher that 400 are reported as failure and in such a case the corresponding TPR is reported if it is higher than 0 . 8 .

(a)

(b)

(a)

(b)

2005] dataset we perform an approximation of the underlying root causes. As the known matrix is unweighted we choose to assign them the weights that would minimize the sparsity of the root causes ∥ C ∥ 1 . The root causes C that are computed based on this optimization are depicted in Fig. 6a. In Fig. 6b We see the distribution of the values of root causes: there are many values with low magnitude close to zero and a few with significantly high magnitude. Also many at roughly the same location (Fig. 6a).

B.6 Implementation details

Training details. In the implementation of SparceRC we initiate a PyTorch nn.Module with a single d × d weighted matrix represented as a linear layer. The model is trained so that the final result will have a weight matrix which approximates the original DAG adjacency matrix. To train our model we use the Adam optimizer [Kingma and Ba, 2014] with learning rate 10 -3 . We choose λ = 10 -3 as the coefficient for the L 1 regularizer of optimization problem (10). For NOTEARS we also choose λ = 10 -3 . The rest of the methods are used with

(a)

(b)

default parameters, except for the post-processing threshold which is always set to 0 . 09 , except to the case of larger weights (Table 1, row 6) where it is set slightly below the weights ω = 0 . 4 .

Resources. Our experiments were run on a single laptop machine with 8 core CPU with 32GB RAM and an NVIDIA GeForce RTX 3080 GPU.

Licenses. We use code repositories that are open-source and publicly available on github. All repositories licensed under the Apache 2.0 or MIT license. In particular we use the github repositories of DAGMA [Bello et al., 2022] https://github.com/kevinsbello/dagma, GOLEM [Ng et al., 2020] https://github.com/ignavierng/golem, NOTEARS [Zheng et al., 2018] https://github.com/xunzheng/notears, DAG-NoCurl [Yu et al., 2021] https://github.com/fishmoon1234/DAG-NoCurl, LiNGAM [Shimizu et al., 2006] https://github.com/cdt15/lingam the implementation of sortnregress [Reisach et al., 2021] https://github.com/Scriddie/Varsortability and the causal discovery toolbox [Kalainathan and Goudet, 2019] https://github.com/FenTechSolutions/CausalDiscoveryToolbox. Our code is licensed under the MIT license and available publicly in github.

C Global Minimizer

We provide the proof of Theorem 3.2 of the main paper. The data X are assumed to be generated via noise-less root causes as described . We denote by ̂ A the optimal solution of the optimization problem (8). We want to prove the following result.

Theorem C.1. Consider a DAG with weighted adjacency matrix A with d nodes. Given exponential (in d ) number n of samples X the matrix A is, with high probability, the global minimizer of the optimization problem (8) .

In particular what we will show the following result, from which our theorem follows directly.

20

40

60

80

100

Number of nodes

0

200

400

600

800

SHD

100200

500

800

1000

Number of samples

0

100

200

300

400

500

SHD

(a) SHD (

↓

)

20

40

60

80

100

Number of nodes

0

2000

4000

6000

8000

SID

100200

500

800

1000

Number of samples

0

2000

4000

6000

8000

10000

SID

(b) SID (

↓

)

20

40

60

80

100

Number of nodes

0

100

200

300

400

500

Time (s)

100200

500

800

1000

Number of samples

0

100

200

300

400

500

Time (s)

(c) Runtime [s]

SparseRC

GOLEM

NOTEARS

DAGMA

PC

GES

CAM

DAG-NoCurl

fGES

sortnregress

MMHC

20

40

60

80

100

Number of nodes

0.2

0.4

0.6

0.8

1.0

TPR

100 200

500

800

1000

Number of samples

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

TPR

(d) TPR (

↑

)

20

40

60

80

100

Number of nodes

0

200

400

600

800

1000

1200

NNZ

100200

500

800

1000

Number of samples

0

500

1000

1500

2000

2500

3000

3500

NNZ

(e) Total edges

20

40

60

80

100

Number of nodes

0.0

0.5

1.0

1.5

2.0

2.5

3.0

NMSE

100 200

500

800

1000

Number of samples

0.0

0.5

1.0

1.5

2.0

2.5

3.0

NMSE

(f) NMSE (

↓

)

20

40

60

80

100

Number of nodes

0

100

200

300

400

500

SHD

(a) SHD (

↓

)

20

40

60

80

100

Number of nodes

0

1000

2000

3000

SID

(b) SID (

↓

)

20

40

60

80

100

Number of nodes

0

100

200

300

400

500

Time (s)

(c) Runtime [s]

SparseRC

where k = ⌊ dp ⌋ and

Rows (samples)

Columns (nodes)

(a) Illustration of

C

T

0

500 100015002000250030003500

|

cij

|

0

500

1000

1500

2000

2500

3000

Count of values

(b) Histogram of values of C

Then with probability (1 -ϵ ) 2 the solution to the optimization problem (8) coincides with the true DAG, namely ̂ A = A .

Remark C.3 . The reason we choose k = ⌊ dp ⌋ is that because of the Bernoulli trials for the non-zero root causes, the expected value of the cardinality of the support will be exactly ⌊ dp ⌋ . Thus we expect to have more concentration on that value of k which is something we desire, since in the proof we use all possible patterns with support equal to k .

Proof sketch. Given a large (exponential), but finite number of data n we can find, with high likelihood (approaching one as n increases), pairs of sub-matrices of root causes matrices C (true) and ̂ C (corresponding to the solution of optimization), with the same support and covering all possibilities of support with k non-zero elements, due to the assumption of randomly varying support. Exploiting the support of the pairs we iteratively prove that the entries of the matrices A (true) and ̂ A (solution of the optimization) are equal. The complete proof is provided next.

We begin with some important observations and definitions.

Proof. We have that

Definition C.5. We define ̂ C = X -X ̂ A the root causes corresponding to the optimal adjacency matrix.

Definition C.6. Let S ⊂ { 1 , 2 , 3 , ..., d } be a set of indices. We say that a root causes vector c = ( c 1 , ..., c d ) has support S if c i = 0 for i ∈ [ d ] \ S .

Definition C.7. For a given support S we consider the set R ⊂ [ n ] of the rows of C that have support S . Then, C R , ̂ C R denote the submatrices consisting of the rows with indices in R .

Lemma C.8. For any row subset R ⊂ [ n ] we have that rank ( ̂ C R ) = rank ( C R )

Proof. We have that

Therefore, since both A , ̂ A are acyclic and I + ̂ A , I + A are invertible we have that

Lemma C.9. For any row subset R ⊂ [ n ] of root causes C with the same support S such that | S | = k and | R | = r ≥ k the nonzero columns of C R are linearly independent with probability 1 and therefore rank ( C R ) = k .

Proof. Assume c 1 , ..., c k are the non-zero columns of C R . Then each c i is a vector of dimension r whose each entry is sampled uniformly at random in the range (0 , 1] . Given any k -1 vectors out of c 1 , ..., c k their linear span can form a subspace of (0 , 1] r of dimension at most k -1 . However, since every c i is sampled uniformly at random from (0 , 1] r and r ≥ k > k -1 the probability that c i lies in the linear span of the other k -1 vectors has measure 0 . Therefore, the required result holds with probability 1 .

Lemma C.10. With probability (1 -ε ) 2 for every different support S with | S | = k there are rows R such that C R have support S and

Proof. Set l = 2 3 d -2 d ( d -1) and K = l ( d k ) (1 -δ ) . Also set N be the random variable representing the number of root causes c with | supp ( c ) | = k and N i the number of repetitions of the i -th k -support pattern, i = 1 , ..., ( d k ) .

We first use conditional probability.

Now we will show that P ( N ≥ K ) ≥ (1 -ϵ ) and P ( N i ≥ l, ∀ i | N ≥ k ) ≥ (1 -ϵ ) using Chernoff bounds. From Hoeffding inequality we get:

where µ = np k (1 -p ) d -k ( d k ) is the expected value of N , δ ≥ 1 p k (1 -p ) d -k ( d k ) √ 1 2 ln ( 1 ϵ ) and n ≥ 1 1 -δ K p k (1 -p ) d -k ( d k ) so

Now given that we have at least K root causes with support exactly k , we will show that, on exactly K such root causes, each different pattern will appear at least l times with probability (1 -ϵ ) . From the union bound

So we need to show that P ( N i ≤ l ) ≤ ϵ ( d k ) . We use again the Hoeffding inequality.

where the expected value is µ = K ( d k ) = l 1 -δ since now the probability is uniform over all possible k -sparsity patterns. Given δ ≥ ( d k ) √ 1 2 ln ( ( d k ) ϵ ) we derive

The result follows.

Lemma C.11. Assume that the data number n is large enough so that for every different support S we have the rows R such that C R have support S and | R | > 2 d l where

Then there exist rows ˆ R such that | ˆ R | > k , C ˆ R has support S and ̂ C ˆ R has support ˆ S with | ˆ S | = k . Moreover, both the non-zero columns of C ˆ R , ̂ C ˆ R form two linearly independent set of vectors. In words, this means that the data are many enough so we can always find pairs of corresponding submatrices of root causes with each consisting of k non-zero columns.

Proof. According to the assumption there exists a row-submatrix C R where all rows have support R and | R | > 2 d l . Group the root causes ̂ C R into all different supports, which are 2 d in number. Take any such group R ′ ⊂ R of rows of ̂ C that all have the same support S ′ . If | R ′ | ≥ k then from Lemma C.8 rank ( ̂ C R ′ ) = rank ( C R ′ ) = k so ̂ C R ′ will have at least k non-zero columns, and since the support is fixed, it will have at least k non-zero elements at each row, which means at least as many non-zeros as C R ′ . Therefore ̂ C R ′ can only

have less non-zero elements if | R ′ | < k , and in that case C R ′ has at most k ( k -1) more elements. If we count for all k = 1 , ..., d all different supports of ̂ C R ′ for all possible supports of C R this gives that ̂ C can have at most ∑ d k =2 ( d k ) 2 d k ( k -1) less non-zero elements compared to C .

Due to the pigeonhole principle, there exists ˆ R ⊂ R and | ˆ R | > l with ̂ C ˆ R all having the same support ˆ S , not necessarily equal to S . According to our previous explanation we need to have at least k non-zero columns in ̂ C ˆ R . If we had k +1 columns then this would give l more non-zero elements, but

So then ∥ ̂ C ∥ 0 > ∥ C ∥ which is a contradiction due to the optimality of ̂ A . Therefore, ̂ C ˆ R has exactly k non-zero columns which necessarily need to be linearly independent in order to have rank ( ̂ C ˆ R ) = rank ( C ˆ R ) = k as necessary. The linear independence of the columns of C ˆ R follows from Lemma C.9 since l ≫ k .

Definition C.12. Apair C R , ̂ C R constructed according to Lemma C.11 that have fixed support S, ˆ S respectively with cardinality k each and | R | large enough so that rank ( ̂ C R ) = rank ( C R ) = k , will be called a k -pair of submatrices.

Remark C.13 . For notational simplicity we will drop the index R whenever the choice of the rows according to the sparsity pattern S is defined in the context.

The complete Theorem 3.2 follows after combining the following two propositions C.16 and C.18. Before, the proof of the first proposition we need the following lemmas.

Lemma C.14. A is (strictly) upper triangular if and only if A is (strictly) upper triangular.

Proof. If A is (strictly) upper triangular then it is straightforward to show the same for A = A + A 2 + ... + A d -1 . For the other way round, remember that the inverse of an upper triangular matrix (if exists) it is an upper triangular matrix. However, the inverse of I + A is I -A which means, that if A is strictly upper triangular, then I + A and so is I -A are upper triangular which in turn reduces to A being strictly upper triangular.

Lemma C.15. Consider C , ̂ C a k -pair. Also let X be the corresponding submatrix of data. If X : ,i = 0 then

Proof.

̸

If ̂ C : ,i = 0 then i ∈ supp ( ̂ C ) , and the expression above constitutes a linear combination of the support columns with not all coefficients non-zero. This contradicts Lemma C.9. Thus C : ,i = 0 and ˆ a ji = 0 ∀ j ∈ supp ( ̂ C ) .

We are now ready to prove Prop. C.16.

Proposition C.16. If the data X are indexed such that A is (strictly) upper triangular, then so is ̂ A .

Remark C.17 . Based on lemma C.14, in our proof of Prop. C.16 next we may only care about the transitive weights, namely we will only need to show that ˆ a ji = 0 for all i < j .

Proof. We first choose a k -pair C , ̂ C such that the support S of C is concetrated to the last k columns, namely C : ,i = 0 for i = 1 , ..., d -k . Then the corresponding data submatrix X will necessarily have the same sparsity pattern since the values are computed according to predecessors, which in our case lie in smaller indices. Thus X : ,i = 0 ∀ i = 1 , ..., d -k and necessarily ̂ C will follow the same sparsity pattern. Moreover, according to Lemma C.15 we get that

We notice that the desired condition is fullfilled for i = d -k , i.e. it has zero influence from a node with larger index. We now prove the same sequentially for i = d -k -1 , d -k -2 , ..., 1 by moving each time (one step) left the leftmost index of the support S . We prove the following statement by induction:

We know that P ( d -k ) is true and P (1) gives the required relation for all indices i = 1 , ..., d -k . Now we assume that P ( l ) holds. If we pick k -pair C , ̂ C such that C has support S = { l, d -k +2 , d -k +3 ..., d } . Then X : ,i = 0 for all i < l which with Lemma C.15 gives ̂ C : ,i = 0 for i < l which means supp ( ̂ C ) ⊂ { l, l +1 , ..., d } and ˆ a ji = 0 for j ∈ supp ( ̂ C ) . Note, that by the induction hypothesis we have that ˆ a jl = 0 for all l < j . However it is true that X : ,l = C : ,l and also

Therefore l ∈ supp ( ̂ C ) and thus ˆ a li = 0 for all i < l which combined with P ( l ) gives

which is exactly P ( l -1) and the induction is complete.

Now it remains to show that ˆ a ji = 0 for d -k +1 ≤ i ≤ d and i < j . We will again proceed constructively using induction. This time we will sequentially choose support S that is concentrated on the last k +1 columns and at each step we will move the zero column one index to the right. For l = d -k +1 , ..., d let's define:

First we show the base case Q ( d -k + 1) . For this we choose a k -pair C , ̂ C such that C has support S = { d -k, d -k +2 , d -k +3 , ..., d } . It is true that X : ,i = 0 for i = 1 , ..., d -k -1 hence ̂ C : ,i = 0 for i ≤ d -k -1 and therefore the node d -k doesn't have any non-zero parents, since also ˆ a j ( d -k ) = 0 for d -k < j from previous claim. Therefore ̂ C : ,d -k = X : ,d -k = C : ,d -k . Also, for l = d -k +1

The equation from ̂ A gives:

From the linear Independence Lemma C.9 of the support of ̂ C we necessarily need to have ̂ C : ,l = 0 , a d -k,l = ˆ a d -k,l and ˆ a jl = 0 for l < j which gives the base case.

For the rest of the induction we proceed in a similar manner. We assume with strong induction that all Q ( d -k +1) , ..., Q ( l ) are true and proceed to prove Q ( l +1) . Given these assumptions we have that

Consider root causes support S = { d -k, d -k + 1 , ..., l, l + 2 , ..., d } (the ( l + 1) -th column is 0 ) for the k -pair C , ̂ C . Then we have the equations:

Where we used the linear independence lemma and sequentially proved that the root causes columns up to l are equal. The equation for the ( l +1) -th column now becomes:

where the last set of equalities follows from linear independence. This concludes the induction and the proof.

To complete the proof of Theorem 3.2 it remains to show the following proposition.

Proposition C.18. If both A , ̂ A are upper triangular then ̂ A = A .

For our proof we use the following definition.

Definition C.19. We denote by P k the set of all k -pairs C R , ̂ C R for all possible support patterns.

Now we proceed to the proof.

Proof. We will show equivalently that ̂ A = A using two inductions. First we show for l = 1 , ..., k the following statement.

P ( l ) : For all k -pairs C , ̂ C in P k the first l non-zero columns C : ,i 1 , C : ,i 2 , ..., C : ,i l and ̂ C : , ˆ i 1 , ̂ C : , ˆ i 2 , ..., ̂ C : , ˆ i l are in the same positions, i.e. i j = ˆ i j and

either they are respectively equal C : ,i j = ̂ C : ,i j

or C : ,i l is in the last possible index, namely i l = d -( l -1)

For the base case P (1) , consider a k -pair C , ̂ C and let i 1 be the position of the first non-zero root causes column of C . Then X : ,i = 0 for i < i 1 and therefore from Lemma C.15 ̂ C : ,i = 0 for i < i 1 . Hence

Therefore ̂ C : ,i 1 = C : ,i 1 and we proved P (1) , by satisfying both the positioning and the first requirement.

Assuming now that P ( l ) holds, we will show P ( l +1) . Take any k -pair of P k which we denote by C , ̂ C . Then, if C : ,i l is in the last possible position, then necessarily C : ,i l +1 is in the last possible position. Moreover, from the induction hypothesis the first l root causes columns are in the same positions. Therefore in the same manner, ̂ C : ,i l is in the last position and ̂ C : ,i l +1 , too. This case fulfills the desired statement.

If C : ,i l is not in the last position, then from induction hypothesis, the first l root causes columns are equal. If C : ,i l +1 is in the last position and the same holds ̂ C : , ˆ i l +1 , the requirement is satisfied. Otherwise ˆ i l +1 < i l +1 and the equation for ˆ i l +1 gives:

According to linear independence Lemma C.9 we necessarily derive ̂ C : , ˆ i l +1 = 0 , absurd. Thus ˆ i l +1 = i l +1 and the induction statement is fullfilled in that case.

It remains to consider the case where C : ,i l +1 is not in the last position. Since, i l +1 is not the last position there exists a k -pair C ′ , ̂ C ′ such that the column i l +1 is zero and the ( l +1) -root causes column of C ′ lies at i ′ l +1 > i l +1 . The equation at i l +1 for X ′ gives:

For the pair C ′ , ̂ C ′ the induction hypothesis holds, thus we derive:

Therefore, Lemma C.9 gives ˆ a i j , ˆ i l +1 = a i j ,i l +1 for all j = 1 , ..., l and returning back to the equation for X : ,i l +1 we derive:

which completes the induction step. Notice that P ( k ) gives that for all k -pairs, the k root causes columns are in the same position. Given this fact we will now show that ̂ A = A . We will prove by induction that for l = k -1 , ..., 1 , 0

To prove Q ( k -1) we choose all the k -pairs C , ̂ C , such that C : ,i 2 is in the last possible position, i 2 = d -k +2 . Then for i 1 ≤ d -k the columns C : ,i 1 , ̂ C : ,i 1 lie at the same position and are equal. Choosing i 1 = i and computing the equation for X : ,j where i < j ≤ d -k +1 gives:

Therefore ˆ a ij = a ij for all 1 ≤ i < j ≤ d -( k -1) and Q ( k -1) is satisfied. Next, assume that Q ( k -l ) is true. We want to show Q ( k -l -1) . Similarly to the base case we consider all k -pairs such that C : ,l +2 lies in its last possible position i l +2 = d -k + l +2 , and i l +1 ≤ d -k + l . Since the ( l +1) -th column is not in the last position, from the previous induction we have that:

The equation for d -k + l +1 gives:

By choosing all such possible k -pairs the indices i j span all possibilities 1 ≤ i < d -k + l +1 . Combining this with Q ( k -l ) we get that Q ( k -l -1) is true and the desired result follows.

Peer reviews:
comments: SUMMARY:
The paper studies a new causal discovery method, in which it assumes that the DAG data is produced by few data-generating events whose effect percolates through the DAG. They propose a simple but effective method to learn the true DAG based on the few roots assumption. The proposed method outperforms baselines in various settings.

STRENGTHS:
1. The few roots assumption is reasonable. And the paper motivates it well.
2. The proposed solution is simple and effective.
3. The paper conducts experiments on both synthetic and real-world datasets, indicating the effectiveness of the proposed method.

WEAKNESSES:
1. It is not clear why the objective function Eq.(10) contains noise. It needs more clear explanation and derivation here.
2. I wonder whether the proposed method could find the root nodes at the same time rather than just learning the DAG.
3. The proposed method could not achieve the best results on real-world datasets. Hence, I doubt the few roots assumptions satisfy the real-world scenarios. BTW, it is better to say the network is a protein network rather than a gene network.

QUESTIONS:
Please reply to the questions in the weaknesses. I am willing to raise the score if all the concerns are solved.

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-20

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers a new setting of linear DAG learning problem. Based on a linear transform of linear SEM, authors propose to study a new setting where there are few \"root causes\", with potential measurement noise in the data. Identifiability is proved and the true DAG is shown to be the global minimizer of the L0-norm of the vector of \"root causes\", under a specific distribution on the \"root cause\" variables.\n\nSTRENGTHS:\n- a new setting for the linear DAG learning problem\n- useful identification result (Thm. 3.2) with a complete proof\n\nWEAKNESSES:\n- the new setting and its motivating example are not sufficiently convincing. \n- authors only consider specific distribution on the \"root causes\" variables, making  theoretic result somewhat limited\n- some results are trivial from the literature (e.g., Thm 2.1)\n\nQUESTIONS:\n- My first concern is about the new setting of learning linear DAGs; it is not clear whether the new setting is indeed meaningful in practice. In the pollution model example, it is stated that \"the relevant DAG data is triggered by sparse events on the input size and not by random noise \", and \"We assume a DAG describing a river network. The acyclicity is guaranteed since flows only occur downstream. .. We assume that the cities can pollute the rivers.\" In this example, why do we need to learn DAGs? The graph structure can be more accurately obtained by getting the information of flows. As such, I suggest authors give more practical examples in the context of DAG learning, to make the new setting indeed meaningful. \n\n- root causes: in the DAG learning literature, \"root causes\" generally refer to the source nodes of DAGs. Not sure if it is suitable to use a (somewhat) conventional name to refer to something new in the same context. \n\n* Theorem 2.1 is not new and may be not stated as a theorem. \n* Regarding Thm 3.1: similarly, the result simple follows from the LiNGAM result, by assuming a specific distribution on the \"root causes\", so maybe consider put it as a lemma or proposition. Besides, in the experiments in the supplementary material, I can see LiNGAM failed. Can you explain why? After all, the linear SEM falls exactly into the setting of LiNGAM if there no measurement noise. \n* after Eq. 8, \"Among all possible DAG matrices, the solution of the optimization problem (8) is the one that minimizes the number of the root causes X\": can you give more details about this claim? \n* This may be a bit picky, but only sparse graphs (with edge/node=2 and 3) are considered. Please try other degrees of graphs. (But this is not very important and may be added after the rebuttal.) \n* please use \\cite, \\citet properly; e.g., line 131 line 181-182\n\nOverall, I like the new setting of learning linear DAGs, but every new setting should be validated with more examples/details. I look forward to author response.\n\nLIMITATIONS:\nAuthors discussed a limitation that the proposed method only works for few root causes in the paper. To me, another important limitation is the specific distribution assumption on the \"root causes\", as the proof heavily depends on this assumption.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers learning of linear SEMs (weight matrix) under a data generation process that differs from the common formulation. It is assumed that each sample is generated from only few number of non-zero noise variables in which the set of noise variables is stochastic. The main theoretical result is identifiability of the weight matrix via an L0 norm objective. A relaxation of the objective with an L1 norm loss is used for extensive synthetic experiments and shown to be effective for learning the weight matrix under the proposed data generation mechanism.\n\nSTRENGTHS:\n- Sparse root causes are an interesting concept to explore.\n- Experiments on synthetic data demonstrate strong performance. Specifically, the proposed algorithm is fast and seems to scale up well, and when the assumptions are violated slightly, the algorithm still yields reasonable results.\n\nWEAKNESSES:\n- While the sparse root causes assumption is interesting, it is not well-motivated. For the pollution example, if I understand correctly, the top left and bottom left nodes do not really play “causal” roles in the sense that they are deterministic mediators in the system, and the whole causal system can be represented without using those nodes. In this sense, “few root nodes” become the effective causal nodes. Am I missing something? The numerical evaluations using real data also do not provide much help for motivation. \n- NNZ is not a strong metric when SHD is already provided, and the only edge of the proposed algorithm in the real data experiments is this metric.\n- Presentation can be improved. For instance, Theorem 2.1 formulation of the linear SEM is trivial. Similarly, having $N_c$ and $N_x$ separately is superfluous; having a single small variance, not necessarily isotropic noise suffices for the description.\n\nQUESTIONS:\n- The authors can elaborate on the Weaknesses item 1.\n\n- An additional question for the presentation. In L103-105,  “The high-level intuition is that it can be reasonable to assume that, with the viewpoint of (3), the relevant DAG data is triggered by sparse events on the input size and not by random noise.” \nAt first, I thought that you only consider a fixed subset of the nodes that have “random noise” with large magnitudes at each sample. If that was the case, the data is not i.i.d, the events are sparse on the input size, but it doesn’t necessarily occur on the same set of variables and I was puzzled with the importance of the proposed mechanism. Then I realized in Theorem 3.1 that at every sample, the non-zero noise variables are chosen randomly and it made sense (please correct me if I misunderstood anything). For a paper that proposes a new data generation model, the presentation should have been cleaner.\n\n- Theorem 3.2.. Given a large enough but finite number of samples, but not a sample complexity result. This is a rather weird statement.\n\n- Note: figure 2 has Möbius as the algo name. Perhaps it’s forgotten in the main text.\n\nLIMITATIONS:\nThe limitations of this work are discussed very clearly in Section 6. I thank the authors for that paragraph.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a new formulation of linear SEM by specifying a structure on the noise variables, which seems to impose zero-inflated distrbutions to achieve the \"few roots\" modelling goal. Identifiablity is given and guarantee for $L^0$ minimization estimator is provided for a special noise-free case. The minimization problem is further formulated into continuous optimization and numeric experiments are conducted.\n\nSTRENGTHS:\n- The idea of the new formulation is interesting, based on the illustrative river pollution example.\n- The experiments consider many different setups, also including real datasets. The proposed algorithm shows comparable performance in the simulation study, though not by much.\n\nWEAKNESSES:\n- The motivation of the proposed formulation needs more elaboration, see questions. And the current definitions of \"few roots\" and \"negaligible noise\" in (6) are not formal or clear.\n- Thm 3.2 only works for noise-free setting, thus there is no guarantee for consistency of $L^0$ minimization estimator for general model (4) beyond noise-free.\n- Neither the related work and experiments discuss or compare with constraint-based methods, even the basic PC algorithm. Is it becaus we lose Markov property and faithfulness in this new formulation?\n\nQUESTIONS:\n- Thm 3.1 is proved by transforming the proposed model into a non-Gaussian DAG model. Just want to clarify: is the proposed formulation covered by the orginal DAG model? If so, are all other DAG learning approaches applicable? In that case, how do the authors justify the superiority of the new formulation beyond the empirical experiments?\n- In Figure 2(a) and (b), the SHD and SID for mobius are larger than others at the beginning but drop in the end, any explanation on that?\n\nLIMITATIONS:\nLmitations are discussed in Section 6.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-14"}