Original: L EARNING DAG S FROM F OURIER -S PARSE DATA
Revised: L EARNING DAG S FROM F OURIER -S PARSE DATA

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We present a novel perspective on learning directed acyclic graphs (DAGs) from data, leveraging a recently proposed theory of causal Fourier analysis on DAGs.
Revised: We present a novel perspective on learning directed acyclic graphs (DAGs) from data, leveraging a recently proposed theory of causal Fourier analysis on DAGs.

Original: We build on prior work that learned DAGs from data generated by a structural equation model (SEM).
Revised: We build on prior work that learned DAGs from data generated by a structural equation model (SEM).

Original: First, we show that data generated by linear SEMs can be characterized in the frequency domain as having dense spectra with random coefficients.
Revised: First, we show that data generated by linear SEMs can be characterized in the frequency domain as having dense spectra with random coefficients.

Original: Then we propose the new problem of learning DAGs from approximately
Revised: Then we propose the new problem of learning DAGs from approximately

Original: Fourier-sparse data, which we solve by minimizing the L 1 norm of the spectrum.
Revised: Fourier-sparse data, which we solve by minimizing the L 1 norm of the spectrum.

Original: We provide a motivation for this problem and compare our method to prior DAG learning methods, showing superior performance.
Revised: We provide a motivation for this problem and compare our method to prior DAG learning methods, showing superior performance.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In this work we study the problem of learning directed acyclic graphs (DAGs) from data using a
Revised: In this work we study the problem of learning directed acyclic graphs (DAGs) from data using a

Original: Fourier analysis perspective.
Revised: Fourier analysis perspective.

Original: The DAG learning problem can be stated as learning the edges of an unknown DAG given data indexed by its nodes.
Revised: The DAG learning problem can be stated as learning the edges of an unknown DAG given data indexed by its nodes.

Original: DAGs can represent causal dependencies (edges) between events (nodes) in the sense that an event only depends on its predecessors.
Revised: DAGs can represent causal dependencies (edges) between events (nodes) in the sense that an event only depends on its predecessors.

Original: Thus, DAG learning has applications in causal discovery, which, however, is a more demanding problem that we do not consider here, as it requires further concepts of causality analysis, like interventions (Peters et al., 2017).
Revised: Thus, DAG learning has applications in causal discovery, which, however, is a more demanding problem that we do not consider here, as it requires further concepts of causality analysis, like interventions (Peters et al., 2017).

Original: Despite being conceptually simpler than causal learning, learning a DAG from data is still  very challenging problem and even NP-hard (Chickering et al., 2004).
Revised: Despite being conceptually simpler than causal learning, learning a DAG from data is still a very challenging problem and even NP-hard (Chickering et al., 2004).

Original: Thus,for the problem to be solvable, one has to make assumptions onhow theobserved data relates to  the underlying DAG.
Revised: Thus, one has to make assumptions on the data generating process, to infer information about the underlying DAG.

Original: One common assumption is that the data follow a structural equation model (SEM) (Shimizu, 2014;
Revised: One common assumption is that the data follow a structural equation model (SEM) (Shimizu, 2014;

Original: Zheng et al., 2018; Gao et al., 2021), meaning that the value of every node is recursively computed as a function of the values of its direct parents and noise.
Revised: Zheng et al., 2018; Gao et al., 2021), meaning that the value of every node is recursively computed as a function of the values of its direct parents and noise.

Original: In this paper we will consider a novel form of generating data on DAGs that is based on the notion of Fourier-sparsity.
Revised: In this paper we will consider a novel form of generating data on DAGs that is based on the notion of Fourier-sparsity.

Original: In particular, our viewpoint on DAG learning is based on a recently proposed theory of causal Fourier analysis on DAGs (Seifert et al., 2022a;b).
Revised: In particular, our viewpoint on DAG learning is based on a recently proposed theory of causal Fourier analysis on DAGs (Seifert et al., 2022a;b).

Original: The theory instantiates all basic Fourier concepts including shift, filters, spectrum and Fourier transform for signals (or data) on DAGs, following the general approach of the algebraic signal processing theory (Püschel & Moura, 2006).
Revised: The theory instantiates all basic Fourier concepts including shift, filters, spectrum and Fourier transform for signals (or data) on DAGs, following the general approach of the algebraic signal processing theory (Püschel & Moura, 2006).

Original: In particular, the proposed Fourier transform for
Revised: In particular, the proposed Fourier transform for

Original: DAGs coincides with the weighted version of the Möbius transform from combinatorics (Rota, 1964).
Revised: DAGs coincides with the weighted version of the Möbius transform from combinatorics (Rota, 1964).

Original: Interestingly, as we will show first, data produced by linear SEMs can then be viewed as having a dense, random spectrum of causes (in a sense being defined) with the data generation matrix being the inverse Fourier transform.
Revised: Interestingly, as we will show first, data produced by linear SEMs can then be viewed as having a dense, random spectrum of causes (in a sense being defined) with the data generation matrix being the inverse Fourier transform.

Original: With this viewpoint we inherit a particular notion of Fourier-sparsity, i.e., data with an approximately sparse spectrum of causes.
Revised: With this viewpoint we inherit a particular notion of Fourier-sparsity, i.e., data with an approximately sparse spectrum of causes.

Original: For prior, classical forms of Fourier transforms, Fourier-sparsity has played a significant role, including of course the discrete Fourier transform (DFT) (Hassanieh, 2018), the discrete cosine transform (DCT) where it enables JPEG compression (Wallace, 1991), or the
Revised: For prior, classical forms of Fourier transforms, Fourier-sparsity has played a significant role, including of course the discrete Fourier transform (DFT) (Hassanieh, 2018), the discrete cosine transform (DCT) where it enables JPEG compression (Wallace, 1991), or the

Original: Walsh-Hadamard transform for estimating set functions Stobbe & Krause (2012).
Revised: Walsh-Hadamard transform for estimating set functions Stobbe & Krause (2012).

Original: Contributions.
Revised: Contributions.

Original: In this work we leverage the concept of Fourier-sparsity associated with the Fourier analysis from (Seifert et al., 2022b) for solving the DAG learning problem.
Revised: In this work we leverage the concept of Fourier-sparsity associated with the Fourier analysis from (Seifert et al., 2022b) for solving the DAG learning problem.

Original: Our contributions consist of the following:
Revised: Our contributions consist of the following:

Original: • We characterize data generated by linear SEMs in the spectral domain.
Revised: • We characterize data generated by linear SEMs in the spectral domain.

Original: We show that they cover the whole frequency spectrum and have random Fourier coefficients.
Revised: We show that they cover the whole frequency spectrum and have random Fourier coefficients.

Original: • Motivated by the concept of Fourier-sparsity, we pose the new problem of learning DAGs from data with approximately sparse Fourier-spectrum.
Revised: • Motivated by the concept of Fourier-sparsity, we pose the new problem of learning DAGs from data with approximately sparse Fourier-spectrum.

Original: • To solve this problem, we propose a novel linear DAG learning method, called Möbius, that has as minimization objective the L 1 norm of the approximated spectrum.
Revised: • To solve this problem, we propose a novel linear DAG learning method, called Möbius, that has as minimization objective the L 1 norm of the approximated spectrum.

Original: Analyzing Möbius experimentally, in comparison with recent linear DAG learning methods, we conclude that our method offers significant improvements on Fourier-sparse data.
Revised: Analyzing Möbius experimentally, in comparison with recent linear DAG learning methods, we conclude that our method offers significant improvements on Fourier-sparse data.

Original:  C AUSAL F OURIER A NALYSIS
Revised:  C AUSAL F OURIER A NALYSIS

Original: In this section we provide the necessary background on the causal Fourier analysis on DAGs from (Seifert et al., 2022b;a) including a motivating example.
Revised: In this section we provide the necessary background on the causal Fourier analysis on DAGs from (Seifert et al., 2022b;a) including a motivating example.

Original: 2. B ACKGROUND
Revised: 2. B ACKGROUND

Original: DAG.
Revised: DAG.

Original: We consider a DAG G =
Revised: We consider a DAG G =

Original: ( V, E ) with | V | =
Revised: ( V, E ) with | V | =

Original: d vertices and no loops.
Revised: d vertices and no loops.

Original: G induces a partial order on V : i ≤ j for i, j ∈ V whenever there is a path from i to j .
Revised: G induces a partial order on V : i ≤ j for i, j ∈ V whenever there is a path from i to j .

Original: We assume the vertices to be sorted topologically and for simplicity assume then V = { 1 , 2 , ..., d } .
Revised: We assume the vertices to be sorted topologically and for simplicity assume then V = { 1 , 2 , ..., d } .

Original: Further, we assume a weighted adjacency matrix A of the graph:
Revised: Further, we assume a weighted adjacency matrix A of the graph:

Original: [Equation]
Revised: [Equation]

Original: We consider the nodes as events and say that i is a cause of j if i ≤ j .
Revised: We consider the nodes as events and say that i is a cause of j if i ≤ j .

Original: Note that A d = 0 .
Revised: Note that A d = 0 .

Original: Signal.
Revised: Signal.

Original: A signal (or data) over G is defined as a vector s =
Revised: A signal (or data) over G is defined as a vector s =

Original: ( s i ) i ∈ V indexed by the vertices of G , where s i has been measured at i .
Revised: ( s i ) i ∈ V indexed by the vertices of G , where s i has been measured at i .

Original: Further, the model assumes that every s i is a linear combination
Revised: Further, the model assumes that every s i is a linear combination

Original: [Equation]
Revised: [Equation]

Original: of unknown values
Revised: of unknown values

Original: c j of its predecessors with associated weights w ji .
Revised: c j of its predecessors with associated weights w ji .

Original: By slight extension of terminology, c j , for j ≤ i , is called a cause of s i and w ji captures the associated strength of influence.
Revised: By slight extension of terminology, c j , for j ≤ i , is called a cause of s i and w ji captures the associated strength of influence.

Original: In matrix form, Eq.
Revised: In matrix form, Eq.

Original: (2) becomes s = W T c with W = ( w ij ) i,j .
Revised: (2) becomes s = W T c with W = ( w ij ) i,j .

Original: Note that the weights are present for all j ≤ i , not just for the edges as in A .
Revised: Note that the weights are present for all j ≤ i , not just for the edges as in A .

Original: Thus, W will be computed through a transitive closure of A as explained next.
Revised: Thus, W will be computed through a transitive closure of A as explained next.

Original: Weighted transitive closure.
Revised: Weighted transitive closure.

Original: The computation of W from A first requires a weighted transitive closure of A , denoted with A .
Revised: The computation of W from A first requires a weighted transitive closure of A , denoted with A .

Original: According to the theory in (Abdali & Saunders, 1985) there are several choices depending on the meaning of the weights in A which could be length of shortest path, level of pollution, most reliable path, or capacity, and they can be computed using a modified
Revised: According to the theory in (Abdali & Saunders, 1985) there are several choices depending on the meaning of the weights in A which could be the shortest path’s length, level of pollution, most reliable path, or capacity, and they can be computed using a modified

Original: Floyd-Warshall algorithm.
Revised: Floyd-Warshall algorithm.

Original: Two examples of weighted transitive closures are the following:
Revised: Two examples of weighted transitive closures are the following:

Original: [Equation]
Revised: [Equation]

Original: The trivial zero closure encodes a Markovian condition: only direct predecessors have an influence.
Revised: The trivial zero closure encodes a Markovian condition: only direct predecessors have an influence.

Original: The Floyd-Warshall closure can be computed by running a modified Floyd-Warshall algorithm (Lehmann, 1977).
Revised: The Floyd-Warshall closure can be computed by running a modified Floyd-Warshall algorithm (Lehmann, 1977).

Original: One intuitive interpretation of A is a pollution model.
Revised: One intuitive interpretation of A is a pollution model.

Original: Assume G is a river network and an edge weight a ij ∈ [0 , 1] , ( i, j ) ∈ E , captures what fraction of a pollutant inserted at i reaches the neighbour j .
Revised: Assume G is a river network and an edge weight a ij ∈ [0 , 1] , ( i, j ) ∈ E , captures what fraction of a pollutant inserted at i reaches the neighbour j .

Original: Then a ij is the total fraction for every i ≤ j , considering all paths from i to j .
Revised: Then a ij is the total fraction for every i ≤ j , considering all paths from i to j .

Original: It remains to determine the self-loop weights w ii , i ∈ V , which are all chosen as = 1 .
Revised: It remains to determine the self-loop weights w ii , i ∈ V , which are all chosen as = 1 .

Original: As a result,
Revised: As a result,

Original: [Equation]
Revised: [Equation]

Original: is a reflexive-transitive closure of A and invertible.
Revised: is a reflexive-transitive closure of A and invertible.

Original: One might ask the question whether there can exist multiple DAGs A that attain the same Floyd-Warshall transitive closure A .
Revised: One might ask the question whether there can exist multiple DAGs A that attain the same Floyd-Warshall transitive closure A .

Original: The answer is negative:
Revised: The answer is negative:

Original: Lemma 1.
Revised: Lemma 1.

Original: The Floyd-Warshall transitive closure A
Revised: The Floyd-Warshall transitive closure A

Original: (cid:55)→ A is a bijective mapping.
Revised: (cid:55)→ A is a bijective mapping.

Original: Proof.
Revised: Proof.

Original: It holds that
Revised: It holds that

Original: [Equation]
Revised: [Equation]

Original: 0 .0 .0 .0 .0 .0 .(a)
Revised: 0 .0 .0 .0 .0 .0 .(a)

Original: Original DAG 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .(b)
Revised: Original DAG 0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .0 .(b)

Original: Transitive closure 1.
Revised: Transitive closure 1.

Original: (c) Signal     0.
Revised: (c) Signal     0.

Original: (d) Spectrum and thus
Revised: (d) Spectrum and thus

Original: [Equation]
Revised: [Equation]

Original: Fourier transform.
Revised: Fourier transform.

Original: In summary, the linear relationship (2) between the observed signal s and its causes c becomes
Revised: In summary, the linear relationship (2) between the observed signal s and its causes c becomes

Original: [Equation]
Revised: [Equation]

Original: ) (Seifert et al., 2022a) argues that c can be interpreted as a form of spectrum of s .
Revised: ) (Seifert et al., 2022a) argues that c can be interpreted as a form of spectrum of s .

Original: This is done by providing a suitable notion of shift and associated shift-equivariant convolution whose eigenvectors are the columns of I + A T .
Revised: This is done by providing a suitable notion of shift and associated shift-equivariant convolution whose eigenvectors are the columns of I + A T .

Original: Here, we only need and state the Fourier transform obtained by inverting (6):
Revised: Here, we only need and state the Fourier transform obtained by inverting (6):

Original: [Equation]
Revised: [Equation]

Original: using the proof of Lemma 1.
Revised: using the proof of Lemma 1.

Original: Within the presented model it computes the causes from the observed data.
Revised: Within the presented model it computes the causes from the observed data.

Original: c is sparse if s has few causes.
Revised: c is sparse if s has few causes.

Original: 2. I NTUITIVE EXAMPLE
Revised: 2. I NTUITIVE EXAMPLE

Original: We expand the pollution in a river network mentioned before for an intuitive example.
Revised: We expand the pollution in a river network mentioned before for an intuitive example.

Original: DAG.
Revised: DAG.

Original: We assume a DAG describing a river network.
Revised: We assume a DAG describing a river network.

Original: The acyclicity is guaranteed since flows only occur downstream.
Revised: The acyclicity is guaranteed since flows only occur downstream.

Original: The DAG consists of 6 nodes and is depicted in Fig.
Revised: The DAG consists of 6 nodes and is depicted in Fig.

Original: 1a.
Revised: 1a.

Original: The nodes i ∈ V represent geographical points of interest, e.g. cities, and edges are rivers connecting them.
Revised: The nodes i ∈ V represent geographical points of interest, e.g. cities, and edges are rivers connecting them.

Original: We suppose that the cities can cause pollution to the rivers.
Revised: We suppose that the cities can cause pollution to the rivers.

Original: The edge weight a ij ∈ [0 , 1] describes what fraction of a pollutant at i reaches a neighbour j .
Revised: The edge weight a ij ∈ [0 , 1] describes what fraction of a pollutant at i reaches a neighbour j .

Original: Transitive closure.
Revised: Transitive closure.

Original: Fig.
Revised: Fig.

Original: 1b shows the transitive closure of the DAG in (a).
Revised: 1b shows the transitive closure of the DAG in (a).

Original: The edge weight a ij ∈ [0 , 1] now represents the total fraction of a pollutant at i  reaches j via all connecting paths.
Revised: The edge weight a ij ∈ [0 , 1] now represents the total fraction of a pollutant at i that reaches j via all connecting paths.

Original: Signal and spectrum.
Revised: Signal and spectrum.

Original: Fig.
Revised: Fig.

Original: 1c shows a possible signal s on the DAG, for example the total pollution measured at each node.
Revised: 1c shows a possible signal s on the DAG, for example the total pollution measured at each node.

Original: Within the model, the associated spectrum c in Fig.
Revised: Within the model, the associated spectrum c in Fig.

Original: 1d then shows the origin or causes of the pollution.
Revised: 1d then shows the origin or causes of the pollution.

Original: In this case the spectrum is sparse, i.e., there are few causes.
Revised: In this case the spectrum is sparse, i.e., there are few causes.

Original: Fourier-sparsity will be the key assumption in our work.
Revised: Fourier-sparsity will be the key assumption in our work.

Original: Fourier transform.
Revised: Fourier transform.

Original: Fig.
Revised: Fig.

Original: 2 shows the matrices associated with the example: the DAG’s adjacency matrix, its reflexive-transitive closure, and its inverse-transpose, which is the Fourier transform matrix.
Revised: 2 shows the matrices associated with the example: the DAG’s adjacency matrix, its reflexive-transitive closure, and its inverse-transpose, which is the Fourier transform matrix.

Original:  L INEAR D ATA ON DAG S
Revised:  L INEAR D ATA ON DAG S

Original: In this section we interpret linear SEMs from a causal Fourier analysis perspective.
Revised: In this section we interpret linear SEMs from a causal Fourier analysis perspective.

Original: In particular, we show that they can be interpreted as generating signals with a dense, random spectrum.
Revised: In particular, we show that they can be interpreted as generating signals with a dense, random spectrum.

Original: This will then
Revised: This will then

Original:   0 0 .
Revised:   0 0 .

Original:      (a) A   1 0 .
Revised:      (a) A   1 0 .

Original:      (b) I + A      0 0 − 0 .   0 0 − 0 .   

Revised:      (b) I + A      0 0 − 0 .   0 0 − 0 .

Original: 0− 0 .
Revised: 0− 0 .

Original: 8 − 0 . 

Revised: 8 − 0 .

Original: 0 − − 0 − 0 .
Revised: 0 − − 0 − 0 .

Original: 7 1 0− − 0 .
Revised: 7 1 0− − 0 .

Original: 1 0  (c) (cid:16) I
Revised: 1 0  (c) (cid:16) I

Original: + A T (cid:17) − 1 =
Revised: + A T (cid:17) − 1 =

Original: I − A T allow us to generalize the process of generating causal data to synthetically imposing (approximate)
Revised: I − A T allow us to generalize the process of generating causal data to synthetically imposing (approximate)

Original: Fourier-sparsity.
Revised: Fourier-sparsity.

Original: First, we recall the definition of a linear SEM.
Revised: First, we recall the definition of a linear SEM.

Original: Linear SEM.
Revised: Linear SEM.

Original: Consider a matrix X ∈ R n × d of n signals of dimension d indexed by the vertices of the DAG G with weighted adjacency matrix A and N ∈ R n × d a matrix of independent random noise values.
Revised: Consider a matrix X ∈ R n × d of n signals of dimension d indexed by the vertices of the DAG G with weighted adjacency matrix A and N ∈ R n × d a matrix of independent random noise values.

Original: The data X follow a linear SEM if
Revised: The data X follow a linear SEM if

Original: [Equation]
Revised: [Equation]

Original: Linear SEMs (Peters et al., 2017) first appeared in the literature of econometrics in 1930’s
Revised: Linear SEMs (Peters et al., 2017) first appeared in the literature of econometrics in 1930’s

Original: (Frisch &
Revised: (Frisch &

Original: Waugh, 1933).
Revised: Waugh, 1933).

Original: Currently, several continuous DAG learning methods are based on the assumption that data X follow a linear SEM (Shimizu, 2014; Zheng et al., 2018; Ng et al., 2020).
Revised: Currently, several continuous DAG learning methods are based on the assumption that data X follow a linear SEM (Shimizu, 2014; Zheng et al., 2018; Ng et al., 2020).

Original: They assume the following data generating process: First, the values at the sources of the DAG are initialized with random noise.
Revised: They assume the following data generating process: First, the values at the sources of the DAG are initialized with random noise.

Original: Then, the remaining nodes are processed in topological order and each node is assigned the linear combination of its parents’ values in addition with independent noise.
Revised: Then, the remaining nodes are processed in topological order and each node is assigned the linear combination of its parents’ values in addition with independent noise.

Original: Formally, the procedure is described by Algorithm 1.
Revised: Formally, the procedure is described by Algorithm 1.

Original: Computationally, Algorithm 1 operates in a similar manner to the signal model (2) in the sense that the values of a node depend on itself and its ancestors.
Revised: Computationally, Algorithm 1 operates in a similar manner to the signal model (2) in the sense that the values of a node depend on itself and its ancestors.

Original: However, the main difference is that in the linear SEM only the direct parents are considered, and the node values occur linearly from signal values instead of spectral ones.
Revised: However, the main difference is that in the linear SEM only the direct parents are considered, and the node values occur linearly from signal values instead of spectral ones.

Original: The interesting connection between linear SEMs and the causal Fourier transform is analyzed with Lemma 2.
Revised: The interesting connection between linear SEMs and the causal Fourier transform is analyzed with Lemma 2.

Original: A DAG G ( V, E ) is given with weighted adjacency matrix A . 2.
Revised: A DAG G ( V, E ) is given with weighted adjacency matrix A . 2.

Original: Initialize the columns i that correspond to sources of the graph G with random noise X : ,i = N : ,i . 3.
Revised: Initialize the columns i that correspond to sources of the graph G with random noise X : ,i = N : ,i . 3.

Original: Compute the values for every other vertex in topological order.
Revised: Compute the values for every other vertex in topological order.

Original: The value of the column corresponding to i th vertex is computed based on the values of its direct parents X : ,i =
Revised: The value of the column corresponding to i th vertex is computed based on the values of its direct parents X : ,i =

Original: X : ,Pa ( i )
Revised: X : ,Pa ( i )

Original: A Pa ( i ) ,i + N : ,i 4.
Revised: A Pa ( i ) ,i + N : ,i 4.

Original: Output the synthetic data X that satisfy X = XA + N .
Revised: Output the synthetic data X that satisfy X = XA + N .

Original: Lemma 2.
Revised: Lemma 2.

Original: The linear SEM data generation Algorithm 1 computes data X that satisfy
Revised: The linear SEM data generation Algorithm 1 computes data X that satisfy

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Using (cid:0) I + A (cid:1)
Revised: Using (cid:0) I + A (cid:1)

Original: ( I − A )
Revised: ( I − A )

Original: =
Revised: =

Original: I , we derive
Revised: I , we derive

Original: X = XA + N ⇔ X
Revised: X = XA + N ⇔ X

Original: ( I − A )
Revised: ( I − A )

Original: =
Revised: =

Original: N ⇔ X
Revised: N ⇔ X

Original: =
Revised: =

Original: N ( I − A ) − 1 ⇔ X = N (cid:0)
Revised: N ( I − A ) − 1 ⇔ X = N (cid:0)

Original: I + A (cid:1) as desired.
Revised: I + A (cid:1) as desired.

Original: In the appendix, we also include a constructive proof of the above statement based on the iterations of Algorithm 1.
Revised: In the appendix, we also include a constructive proof of the above statement based on the iterations of Algorithm 1.

Original: Lemma 2 together with the analysis of the previous section yields the following interpretation of linear SEMs.
Revised: Lemma 2 together with the analysis of the previous section yields the following interpretation of linear SEMs.

Original: Theorem 1.
Revised: Theorem 1.

Original: The linear SEM data generation Algorithm 1 is equivalent to the inverse causal Fourier transformation of the noise spectra N , given by (7), when the weights W are computed according to the Floyd-Warshall closure.
Revised: The linear SEM data generation Algorithm 1 is equivalent to the inverse causal Fourier transformation of the noise spectra N , given by (7), when the weights W are computed according to the Floyd-Warshall closure.

Original: In essence, in (7) the rows of the noise matrix N contain the spectra of the signals that constitute the rows of X .
Revised: In essence, in (7) the rows of the noise matrix N contain the spectra of the signals that constitute the rows of X .

Original: Fourier-sparse data.
Revised: Fourier-sparse data.

Original: Motivated by Theorem 1, we define a novel data generating process for
Revised: Motivated by Theorem 1, we define a novel data generating process for

Original: DAG signals based on the notion of Fourier sparsity.
Revised: DAG signals based on the notion of Fourier sparsity.

Original: As previously, we consider a topologically sorted DAG G with weighted adjacency matrix A and transitive closure A .
Revised: As previously, we consider a topologically sorted DAG G with weighted adjacency matrix A and transitive closure A .

Original: We define the matrices
Revised: We define the matrices

Original: C , N f , N s ∈ R n × d that correspond to the signal, spectra and the random noise for the frequency and signal domain respectively.
Revised: C , N f , N s ∈ R n × d that correspond to the signal, spectra and the random noise for the frequency and signal domain respectively.

Original: According to the spectra and the noise, the data X ∈ R n × d are generated via the causal Fourier transform
Revised: According to the spectra and the noise, the data X ∈ R n × d are generated via the causal Fourier transform

Original: [Equation]
Revised: [Equation]

Original: The data are characterized as approximately Fourier-sparse if
Revised: The data are characterized as approximately Fourier-sparse if

Original: [Equation]
Revised: [Equation]

Original: The L 0 norm promotes only a few spectral coefficients to be non-zero.
Revised: The L 0 norm promotes only a few spectral coefficients to be non-zero.

Original: Thus, in this context, being sparse in the Fourier domain simply translates into the signal being generated using only a few causes.
Revised: Thus, in this context, being sparse in the Fourier domain simply translates into the signal being generated using only a few causes.

Original: Note that (8) provides an alternative way to express linear SEMs.
Revised: Note that (8) provides an alternative way to express linear SEMs.

Original: Namely, for zero spectra C = 0 , zero noise on the signal domain N s = 0 and Floyd-Warshall closure
Revised: Namely, for zero spectra C = 0 , zero noise on the signal domain N s = 0 and Floyd-Warshall closure

Original: A =
Revised: A =

Original: A + A 

Revised: A + A

Original: + ...
Revised: + ...

Original: + A d − 1 , (8) specializes to (7).
Revised: + A d − 1 , (8) specializes to (7).

Original: Thus, our model (8) generalizes linear SEMs by potentially allowing non-zero spectra, signal noise, and other notions of transitive closure from (Abdali & Saunders, 1985).
Revised: Thus, our model (8) generalizes linear SEMs by potentially allowing non-zero spectra, signal noise, and other notions of transitive closure from (Abdali & Saunders, 1985).

Original: Example.
Revised: Example.

Original: Consider the river network of our previous example.
Revised: Consider the river network of our previous example.

Original: We assume that we take measurement for all d sensors of the network once in a day for n days and collect them in the matrix X ∈ R n × d .
Revised: We assume that we take measurement for all d sensors of the network once in a day for n days and collect them in the matrix X ∈ R n × d .

Original: As before, the data X represent accumulated pollution in the nodes of the river network.
Revised: As before, the data X represent accumulated pollution in the nodes of the river network.

Original: The pollution is caused at nodes (cities) of the network.
Revised: The pollution is caused at nodes (cities) of the network.

Original: Fourier-sparsity means that every day only a small number of cities pollute, which is captured by C .
Revised: Fourier-sparsity means that every day only a small number of cities pollute, which is captured by C .

Original: Negligible pollution from other sources is modeled as the noise N f .
Revised: Negligible pollution from other sources is modeled as the noise N f .

Original: N s models the noise in the sensor measurements.
Revised: N s models the noise in the sensor measurements.

Original: Thus, in this scenario we deduce that
Revised: Thus, in this scenario we deduce that

Original: C , N f , N s satisfy (9), i.e., the measurements in X are approximately Fourier-sparse.
Revised: C , N f , N s satisfy (9), i.e., the measurements in X are approximately Fourier-sparse.

Original:  M ETHOD FOR F OURIER - SPARSE D ATA
Revised:  M ETHOD FOR F OURIER - SPARSE D ATA

Original: In this section we present our proposed continuous optimization problem for learning DAGs from
Revised: In this section we present our proposed continuous optimization problem for learning DAGs from

Original: Fourier-sparse data.
Revised: Fourier-sparse data.

Original: We assume that the data matrix X ∈ R n × d is generated as the inverse Fourier transformation of sparse spectra with noise according to (8).
Revised: We assume that the data matrix X ∈ R n × d is generated as the inverse Fourier transformation of sparse spectra with noise according to (8).

Original: The Fourier-sparsity is ensured when C and the noise N f , N s satisfy the sparsity criterion (9).
Revised: The Fourier-sparsity is ensured when C and the noise N f , N s satisfy the sparsity criterion (9).

Original: Notice that for both noise matrices N f , N s we do not make any assumption (e.g., on their distribution) other than that the noise has low magnitude.
Revised: Notice that for both noise matrices N f , N s we do not make any assumption (e.g., on their distribution) other than that the noise has low magnitude.

Original: The general approach for solving the DAG learning problem via continuous optimization consists of the following objective
Revised: The general approach for solving the DAG learning problem via continuous optimization consists of the following objective

Original: [Equation]
Revised: [Equation]

Original: in which ℓ ( A , X ) is the loss function corresponding to the data, R ( A ) is a regularizer that promotes sparsity in the adjacency matrix and h ( A ) is any continuous constraint for acyclicity.
Revised: in which ℓ ( A , X ) is the loss function corresponding to the data, R ( A ) is a regularizer that promotes sparsity in the adjacency matrix and h ( A ) is any continuous constraint for acyclicity.

Original: Current approaches assume that the data are generated via a linear SEM following the equation X = XA + N and the loss function in that case takes the form
Revised: Current approaches assume that the data are generated via a linear SEM following the equation X = XA + N and the loss function in that case takes the form

Original: [Equation]
Revised: [Equation]

Original: This loss function is called the mean-squared loss approximation and achieves the minimization of the L 2 norm of the approximated noise ˆ N .
Revised: This loss function is called the mean-squared loss approximation and achieves the minimization of the L 2 norm of the approximated noise ˆ N .

Original: A popular choice for the regularizer is the L 1 norm
Revised: A popular choice for the regularizer is the L 1 norm

Original: R ( A ) = λ
Revised: R ( A ) = λ

Original: ∥
Revised: ∥

Original: A ∥ 1 to promote sparsity in the adjacency matrix.
Revised: A ∥ 1 to promote sparsity in the adjacency matrix.

Original: The acyclicity constraint can be, for example, the one proposed by (Zheng et al., 2018), namely, h ( A )
Revised: The acyclicity constraint can be, for example, the one proposed by (Zheng et al., 2018), namely, h ( A )

Original: = tr (cid:0) e A ⊙ A (cid:1) − d , or the one from (Yu et al., 2021), which is h ( A ) = tr (cid:16) ( I + λ A ⊙ A ) d (cid:17) − d .
Revised: = tr (cid:0) e A ⊙ A (cid:1) − d , or the one from (Yu et al., 2021), which is h ( A ) = tr (cid:16) ( I + λ A ⊙ A ) d (cid:17) − d .

Original: For our case we consider the following instantiation of the optimization problem (10):
Revised: For our case we consider the following instantiation of the optimization problem (10):

Original: [Equation]
Revised: [Equation]

Original: We call this method Möbius due to the fact that the Fourier transform in the causal setting coincides with the Möbius transformation for posets (Seifert et al., 2022a).
Revised: We call this method Möbius due to the fact that the Fourier transform in the causal setting coincides with the Möbius transformation for posets (Seifert et al., 2022a).

Original: Here the acyclicity constraint can be chosen freely among those previously mentioned.
Revised: Here the acyclicity constraint can be chosen freely among those previously mentioned.

Original: The choice of the L 1 norm for minimizing the approximated spectra X (cid:0) I + A (cid:1) − 1 is to promote sparsity.
Revised: The choice of the L 1 norm for minimizing the approximated spectra X (cid:0) I + A (cid:1) − 1 is to promote sparsity.

Original: If we assume that we have found the true weighted adjacency matrix A then according to the Eq.
Revised: If we assume that we have found the true weighted adjacency matrix A then according to the Eq.

Original: (8) the approximated spectra can be computed explicitly as
Revised: (8) the approximated spectra can be computed explicitly as

Original: [Equation]
Revised: [Equation]

Original: This quantity is sparse when assumption (9) holds and N s (cid:0)
Revised: This quantity is sparse when assumption (9) holds and N s (cid:0)

Original: I + A (cid:1) − 1 has entries of low magnitude.
Revised: I + A (cid:1) − 1 has entries of low magnitude.

Original: Remark.
Revised: Remark.

Original: Notice that, the optimization problem (11) becomes the conventional one in the linear
Revised: Notice that, the optimization problem (11) becomes the conventional one in the linear

Original: SEM setting if we use the Floyd-Warshall transitive closure and interchange the L 2 with the L 1 norm.
Revised: SEM setting if we use the Floyd-Warshall transitive closure and interchange the L 2 with the L 1 norm.

Original: For the linear SEM case, the mean-squared loss is the optimal loss function to consider in order to minimize the approximated noise.
Revised: For the linear SEM case, the mean-squared loss is a more compatible loss function to consider in order to minimize the approximated noise.

Original: Therefore, our method is an extension of the current DAG learning literature in order to learn DAGs from Fourier-sparse data.
Revised: Therefore, our method is an extension of the current DAG learning literature in order to learn DAGs from Fourier-sparse data.

Original: Furthermore, it could also be used with different notions of transitive closure.
Revised: Furthermore, it could also be used with different notions of transitive closure.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Graph learning.
Revised: Graph learning.

Original: We build on the idea to utilize Fourier analysis tools to learn graphs that are directed and acyclic.
Revised: We build on the idea to utilize Fourier analysis tools to learn graphs that are directed and acyclic.

Original: Related work applies graph signal processing (GSP) (Sandryhaila & Moura, 2013b;
Revised: Related work applies graph signal processing (GSP) (Sandryhaila & Moura, 2013b;

Original: Ortega et al., 2018) concepts to extract the graph structure from data (Xia et al., 2021).
Revised: Ortega et al., 2018) concepts to extract the graph structure from data (Xia et al., 2021).

Original: Kalofolias &
Revised: Kalofolias &

Original: Perraudin (2017); Kalofolias (2016) learn graphs by applying a GSP smoothness criterion, Dong et al.
Revised: Perraudin (2017); Kalofolias (2016) learn graphs by applying a GSP smoothness criterion, Dong et al.

Original: (2016) learn the Laplacian of the graph, and Egilmez et al.
Revised: (2016) learn the Laplacian of the graph, and Egilmez et al.

Original: ; Pavez et al.
Revised: ; Pavez et al.

Original: ; Koyakumaru et al.
Revised: ; Koyakumaru et al.

Original: (2021) utilize Gaussian Markov random fields for graph learning.
Revised: (2021) utilize Gaussian Markov random fields for graph learning.

Original: Prior GSP tools are not applicable to DAGs since thethe spectrum collapses (all eigenvalues of the adjacency matrix are zero) and no Fourier transform is available.
Revised: Prior GSP tools are not applicable to DAGs since the spectrum collapses (all eigenvalues of the adjacency matrix are zero) and no Fourier transform is available.

Original: The Fourier analysis from (Seifert et al., 2022b) that we use is fundamentally different from prior GSP and specifically designed for DAGs.
Revised: The Fourier analysis from (Seifert et al., 2022b) that we use is fundamentally different from prior GSP and specifically designed for DAGs.

Original: DAG learning.
Revised: DAG learning.

Original: The approaches to solve the DAG learning problem fall into two categories, using combinatorial search or continuous relaxations.
Revised: The approaches to solve the DAG learning problem fall into two categories, using combinatorial search or continuous relaxations.

Original: In combinatorics terms, DAG learning is an NP-hard problem, as the space of possible DAGs is super-exponential in the number of vertices (Chickering et al., 2004).
Revised: In combinatorics terms, DAG learning is an NP-hard problem, as the space of possible DAGs is super-exponential in the number of vertices (Chickering et al., 2004).

Original: Methods that search on the space of possible DAGs apply heuristic criterions in order to find the ground truth DAG (Ramsey et al., 2017; Chickering, 2002; Hauser & Bühlmann, 2012;
Revised: Methods that search on the space of possible DAGs apply heuristic criteria in order to find the ground truth DAG (Ramsey et al., 2017; Chickering, 2002; Hauser & Bühlmann, 2012;

Original: Tsamardinos et al., 2006).
Revised: Tsamardinos et al., 2006).

Original: Lately, with the computational advance of deep learning , researchers have been focusing on continuous optimization methods (Vowels et al., 2021).
Revised: Lately, with the computational advances of deep learning , researchers have been focusing on continuous optimization methods (Vowels et al., 2021).

Original: These methods model the data generation process using SEMs.
Revised: These methods model the data generation process using SEMs.

Original: Among the first methods to utilize SEMs were
Revised: Among the first methods to utilize SEMs were

Original: CAM (Bühlmann et al., 2014) and LINGAM (Shimizu, 2014) with the latter specializing in linear
Revised: CAM (Bühlmann et al., 2014) and LINGAM (Shimizu, 2014) with the latter specializing in linear

Original: SEMs with Gaussian noise.
Revised: SEMs with Gaussian noise.

Original: NOTEARS (Zheng et al., 2018) described the combinatorial constraint of acyclicity as a continuous one.
Revised: NOTEARS (Zheng et al., 2018) described the combinatorial constraint of acyclicity as a continuous one.

Original: Despite some vulnerabilities, like lack of scale-invariance (Kaiser & Sipos, 2022; Reisach et al., 2021), it has inspired many subsequent DAG learning methods.
Revised: Despite some vulnerabilities, like lack of scale-invariance (Kaiser & Sipos, 2022; Reisach et al., 2021), it has inspired many subsequent DAG learning methods.

Original: A continuation of it (Zheng et al., 2020) utilizes neural networks to generalize NOTEARS to non-
Revised: A continuation of it (Zheng et al., 2020) utilizes neural networks to generalize NOTEARS to non-

Original: linear
Revised: linear

Original: SEMs.
Revised: SEMs.

Original: Other nonlinear methods for DAG learning are DAG-GNN (Yu et al., 2019), in which also a more efficient acyclicity constraint than the one in NOTEARS is proposed, and
Revised: Other nonlinear methods for DAG learning are DAG-GNN (Yu et al., 2019), in which also a more efficient acyclicity constraint than the one in NOTEARS is proposed, and

Original: DAG-GAN (Gao et al., 2021).
Revised: DAG-GAN (Gao et al., 2021).

Original: DAG-NoCurl (Yu et al., 2021) proposes learning the DAG on the equivalent space of weighted gradients of graph potential functions.
Revised: DAG-NoCurl (Yu et al., 2021) proposes learning the DAG on the equivalent space of weighted gradients of graph potential functions.

Original: GOLEM (Ng et al., 2020) studies the role of the weighted adjacency matrix sparsity, the acyclicity constraint, and proposes to directly minimize the data likelihood.
Revised: GOLEM (Ng et al., 2020) studies the role of the weighted adjacency matrix sparsity, the acyclicity constraint, and proposes to directly minimize the data likelihood.

Original: The method DYNOTEARS (Pamfil et al., 2020) implements a variation of
Revised: The method DYNOTEARS (Pamfil et al., 2020) implements a variation of

Original: NOTEARS compatible with time series data.
Revised: NOTEARS compatible with time series data.

Original: A recent line of works consider permutation-based methods to parametrize the search space of the DAG (Charpentier et al., 2022; Zantedeschi et al., 2022).
Revised: A recent line of works consider permutation-based methods to parametrize the search space of the DAG (Charpentier et al., 2022; Zantedeschi et al., 2022).

Original: Our work considers DAG learning under the new assumption of Fourier-sparsity.
Revised: Our work considers DAG learning under the new assumption of Fourier-sparsity.

Original: Fourier sparsity.
Revised: Fourier sparsity.

Original: The notion of sparsity in the frequency domain has been widely used in different forms of Fourier analysis.
Revised: The notion of sparsity in the frequency domain has been widely used in different forms of Fourier analysis.

Original: For the classical DFT (Hassanieh et al., 2012b; Indyk & Kapralov, 2014) it enables faster data processing in spectrum sensing (Hassanieh et al., 2014), magnetic resonance imaging (Shi et al., 2013), or GPS locking (Hassanieh et al., 2012a).
Revised: For the classical DFT (Hassanieh et al., 2012b; Indyk & Kapralov, 2014) it enables faster data processing in spectrum sensing (Hassanieh et al., 2014), magnetic resonance imaging (Shi et al., 2013), or GPS locking (Hassanieh et al., 2012a).

Original: For the DCT it enables JPEG compression.
Revised: For the DCT it enables JPEG compression.

Original: For the WHT and other set function Fourier transforms (Püschel & Wendler, 2020) it enables estimation from samples (Stobbe & Krause, 2012;
Revised: For the WHT and other set function Fourier transforms (Püschel & Wendler, 2020) it enables estimation from samples (Stobbe & Krause, 2012;

Original: Amrollahi et al., 2019; Wendler et al., 2021).
Revised: Amrollahi et al., 2019; Wendler et al., 2021).

Original: In GSP it has enabled compression or estimation (Sandryhaila & Moura, 2013a; Ortega et al., 2018; Chen et al., 2015).
Revised: In GSP it has enabled compression or estimation (Sandryhaila & Moura, 2013a; Ortega et al., 2018; Chen et al., 2015).

Original: Our work aims to leverage Fourier-sparsity instantiated for (Seifert et al., 2022a) for DAG learning.
Revised: Our work aims to leverage Fourier-sparsity instantiated for (Seifert et al., 2022a) for DAG learning.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We experimentally evaluate our novel DAG learning method with both synthetically generated
Revised: We experimentally evaluate our novel DAG learning method with both synthetically generated

Original: Fourier-sparse data and real data from (Sachs et al., 2005).
Revised: Fourier-sparse data and real data from (Sachs et al., 2005).

Original: Baselines.
Revised: Baselines.

Original: We compare against DAG learning methods suitable for data generated by classical linear SEMs with additive noise.
Revised: We compare against DAG learning methods suitable for data generated by classical linear SEMs with additive noise.

Original: In particular, we consider the prior NOTEARS (Zheng et al., 2018),  DAG-NoCurl (Yu et al., 2021), greedy equivalence search (GES) (Chickering, 2002; Hauser & Bühlmann, 2012), fast greedy equivalence search (fGES) (Ramsey et al., 2017), max-min hillclimbing (MMHC) (Tsamardinos et al., 2006), and causal additive models (CAM) (Bühlmann et al., 2014).
Revised: In particular, we consider the prior NOTEARS (Zheng et al., 2018), GOLEM (Ng et al., 2020), the trivial baseline sortnregress (Reisach et al., 2021), DAG-NoCurl (Yu et al., 2021), greedy equivalence search (GES) (Chickering, 2002; Hauser & Bühlmann, 2012), fast greedy equivalence search (fGES) (Ramsey et al., 2017), max-min hill-climbing (MMHC) (Tsamardinos et al., 2006), and causal additive models (CAM) (Bühlmann et al., 2014).

Original: Metrics.
Revised: Metrics.

Original: We use the following performance metrics that consider the unweighted approximation ˆ U of the adjacency matrix: (a) the true positive rate (TPR), computed as the ratio of correct edges found by ˆ U , (b) the structural Hamming distance (SHD), which is the number of edge insertions, deletions or reverses needed to get from ˆ U to the ground truth U , and (c) the total number of nonzero edges (NNZ) proposed by the algorithm.
Revised: We use the following performance metrics that consider the unweighted approximation ˆ U of the adjacency matrix: (a) the true positive rate (TPR), computed as the ratio of correct edges found by ˆ U , (b) the structural Hamming distance (SHD), which is the number of edge insertions, deletions or reverses needed to get from ˆ U to the ground truth U , and (c) the total number of nonzero edges (NNZ) proposed by the algorithm.

Original: The first two metrics accurately show whether an algorithm yields a good approximation while the last one only measures whether the algorithm proposes a number of edges close to the real number.
Revised: The first two metrics accurately show whether an algorithm yields a good approximation while the last one only measures whether the algorithm proposes a number of edges close to the real number.

Original: For the weighted matrix approximation we compute the normalized mean-squared error (NMSE) as ∥ ˆ A − A ∥ 2 / ∥
Revised: For the weighted matrix approximation we compute the normalized mean-squared error (NMSE) as ∥ ˆ A − A ∥ 2 / ∥

Original: A ∥ 2 .
Revised: A ∥ 2 .

Original: 6. E VALUATION ON F OURIER - SPARSE DATA
Revised: 6. E VALUATION ON F OURIER - SPARSE DATA

Original: We perform our first evaluation on synthetic Fourier-sparse data on DAGs.
Revised: We perform our first evaluation on synthetic Fourier-sparse data on DAGs.

Original: Data generating process.
Revised: Data generating process.

Original: We first generate a random Erdös-Renyi graph with weights sampled uniformly at random from ( − b, − a ) ∪ ( a, b ) , where a, b ∈ R + are fixed.
Revised: We first generate a random Erdös-Renyi graph with weights sampled uniformly at random from ( − b, − a ) ∪ ( a, b ) , where a, b ∈ R + are fixed.

Original: This results in the true adjacency matrix A .
Revised: This results in the true adjacency matrix A .

Original: Next, the sparse DAG spectra C are instantiated by setting each entry either to 0 with probability 1 − p or to some random uniform value from (0 , 1) with probability p .
Revised: Next, the sparse DAG spectra C are instantiated by setting each entry either to 0 with probability 1 − p or to some random uniform value from (0 , 1) with probability p .

Original: A low value of p yields higher sparsity, since p is the expected fraction of non-zero coefficients.
Revised: A low value of p yields higher sparsity, since p is the expected fraction of non-zero coefficients.

Original: Note that this means that the support (locations of nonzero values) varies between spectra.
Revised: Note that this means that the support (locations of nonzero values) varies between spectra.

Original: Finally, the data matrix X is computed according to Eq.
Revised: Finally, the data matrix X is computed according to Eq.

Original: (8) where only one of the noise matrices N f , N s is non-zero.
Revised: (8) where only one of the noise matrices N f , N s is non-zero.

Original: The non-zero noise is set to have low standard deviation σ = 0 .
Revised: The non-zero noise is set to have low standard deviation σ = 0 .

Original: This together with low value of p ensures that the inequalities (9) hold and that the data are truly Fourier-sparse.
Revised: This together with low value of p ensures that the inequalities (9) hold and that the data are truly Fourier-sparse.

Original: After executing the corresponding method on X we get its approximation ˆ A of the true weighted adjacency matrix A .
Revised: After executing the corresponding method on X we get its approximation ˆ A of the true weighted adjacency matrix A .

Original: For the evaluation we take the average and standard deviation over 10 repetitions, for all methods and all corresponding performance metrics.
Revised: For the evaluation we take the average and standard deviation over 10 repetitions, for all methods and all corresponding performance metrics.

Original: Experiment 1: Comparison of the methods.
Revised: Experiment 1: Comparison of the methods.

Original: We consider first the experiment with the default experimental settings shown in the blue column of Table 1.
Revised: We consider first the experiment with the default experimental settings shown in the blue column of Table 4.

Original: We show the results in Fig.
Revised: We show the results in Fig.

Original: 3, where we see that Möbius achieves optimal performance and the rest of the methods perform worse.
Revised: 3, where we see that Möbius achieves optimal performance and the rest of the methods perform worse.

Original: The reason for this behavior is that Möbius optimization problem (11) is well suited for these settings.
Revised: The reason for this behavior is that Möbius optimization problem (11) is well suited for these settings.

Original: Both the generated data are Fourier-sparse and the DAG is sparse in the sense it has low average degree.
Revised: Both the generated data are Fourier-sparse and the DAG is sparse in the sense it has low average degree.

Original: Experiment 2: Challenging the methods.
Revised: Experiment 2: Challenging the methods.

Original: In Fig.
Revised: In Fig.

Original: 4, we challenge the algorithms by either using a small number of samples to learn the DAG or by restricting the Fourier spectra to be non-zero always the same locations (fixed support).
Revised: 4, we challenge the algorithms by either using a small number of samples to learn the DAG or by restricting the Fourier spectra to be non-zero always the same locations (fixed support).

Original: While, for a small number of samples, Möbius performance degrades, it still achieves the best TPR among all methods. 
Revised: While, for a small number of samples, Möbius performance degrades, it still achieves the best TPR among all methods. [Table] [Figure] [Table]

Revised: [Figure]

Original: [Table]
Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original: For fixed Fourier-sparsity support all algorithms fail to learn the DAG for any number of vertices.
Revised: [Table] [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Figure]

Revised: For fixed Fourier-sparsity support all algorithms fail to learn the DAG for any number of vertices.

Original: One can imagine the fixed sparsity support on the the river pollution example.
Revised: One can imagine the fixed sparsity support on the the river pollution example.

Original: Fixed support means that there is only a fixed subset of cities that cause pollution.
Revised: Fixed support means that there is only a fixed subset of cities that cause pollution.

Original: Intuitively, this will result in only giving information about the outgoing edges of these cities and not for the rest.
Revised: Intuitively, this will result in only giving information about the outgoing edges of these cities and not for the rest.

Original: This poses the question whether the problem of learning DAGs from Fourier-sparse data with fixed sparsity support is possible at all with a suitable algorithm.
Revised: This poses the question whether the problem of learning DAGs from Fourier-sparse data with fixed sparsity support is possible at all with a suitable algorithm.

Original: Experiment 3: Sensitivity analysis on the hyperparameters.
Revised: Experiment 3: Sensitivity analysis on the hyperparameters.

Original: In Table 1, we present an extensive series of experiments each of which alters a particular hyperparameter indicated in the first column.
Revised: In Table 4, we present an extensive series of experiments each of which alters a particular hyperparameter indicated in the first column.

Original: For higher standard deviation, we see worse performance on all methods except MMHC and CAM.
Revised: For higher standard deviation, we see worse performance on all methods except MMHC and CAM.

Original: Our method still performs best.
Revised: Our method still performs best.

Original: It is expected that for higher standard deviation the sparsity in the data breaks and our algorithm will not longer be efficient.
Revised: It is expected that for higher standard deviation the sparsity in the data breaks and our algorithm will not longer be efficient.

Original: Higher sparsity has no effect on our performance, whereas lower sparsity degrades our performance as expected.
Revised: Higher sparsity has no effect on our performance, whereas lower sparsity degrades our performance as expected.

Original: In contrast, imposing high sparsity is harmful for NOTEARS.
Revised: In contrast, imposing high sparsity is harmful for NOTEARS.

Original: For smaller weight bounds we see a deterioration in performance on most algorithms.
Revised: For smaller weight bounds we see a deterioration in performance on most algorithms.

Original: The best performance in this case is achieved by MMHC.
Revised: The best performance in this case is achieved by MMHC.

Original: Larger weight bounds affect our performance only by a small percentage and our method is still the best in this case.
Revised: Larger weight bounds affect our performance only by a small percentage and our method is still the best in this case.

Original: For higher average degree we also achieve the best accuracy.
Revised: For higher average degree we also achieve the best accuracy.

Original: Testing on more dense graphs would have a negative effect on methods that impose sparsity on the adjacency matrix, like ours.
Revised: Testing on more dense graphs would have a negative effect on methods that impose sparsity on the adjacency matrix, like ours.

Original: Next, the standardization of data is something that generally affects negatively algorithms with continuous objectives (Reisach et al., 2021) and is expected.
Revised: Next, the standardization of data is something that generally affects negatively algorithms with continuous objectives (Reisach et al., 2021) and is expected.

Original: Moreover, on the zero transitive closure our method is not the best but has decent performance close to the best MMHC.
Revised: Moreover, on the zero transitive closure our method is not the best but has decent performance close to the best MMHC.

Original: Finally, applying noise in the spectrum instead of the signal does not have impact our performance.
Revised: Finally, applying noise in the spectrum instead of the signal does not have impact our performance.

Original: The last two rows of the table simply repeat what we saw in the previous experiment, but with higher resolution on the TPR metric.
Revised: The last two rows of the table simply repeat what we saw in the previous experiment, but with higher resolution on the TPR metric.

Original: Overall, our method achieves the best TPR in most scenarios.
Revised: Overall, our method achieves the best TPR in most scenarios.

Original: 6. E VALUATION ON REAL DATA
Revised: 6. E VALUATION ON REAL DATA

Original: We also execute our method on the causal protein-signaling network provided by (Sachs et al., 2005).
Revised: We also execute our method on the causal protein-signaling network provided by (Sachs et al., 2005).

Original: The dataset consists of 7466 samples from a network with 11 nodes that represent proteins andedges showing the interaction between them.
Revised: The dataset consists of 7466 samples from a network with 11 nodes that represent proteins andedges showing the interaction between them.

Original: Even if the DAG of this network is relatively small, the task of learning it is rather difficult.
Revised: Even if the DAG of this network is relatively small, the task of learning it is rather difficult.

Original: This dataset has been a common benchmark for many prior DAG
Revised: This dataset has been a common benchmark for many prior DAG

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: learning methods
Revised: learning methods

Original: Ng et al.
Revised: Ng et al.

Original: ; Gao et al.
Revised: ; Gao et al.

Original: Yu et al.
Revised: Yu et al.

Original: ; Zheng et al.
Revised: ; Zheng et al.

Original: Among these methods, the best performance has SHD that does not drop below 14 which is relatively large comparing to the 17 edges in total.
Revised: Among these methods, the best performance has SHD that does not drop below ≈ 10 which is relatively large comparing to the 17 edges in total.

Original: We report the performance metrics for all methods in Table 2.
Revised: We report the performance metrics for all methods in Table 2.

Original: As seen in the synthetic experiment our method works best when the data are Fourier-sparse.
Revised: As seen in the synthetic experiment our method works best when the data are Fourier-sparse.

Original: However, for this dataset it is not clear whether this is true.
Revised: However, for this dataset it is not clear whether this is true.

Original: Here, our method has decent performance in this case.
Revised: Here, our method has decent performance.

Original: Among the other methods the best TPR is achieved by fGES, which, however,proposes significantly more edges than the true graph and thus has higher SHD.
Revised: Among the other methods the best TPR is achieved by MMHC, which, however, has higher SHD.

Original: NOTEARS together with CAM have the best SHD equal to 25 , which is slightly better than ours which is 28 .
Revised: NOTEARS together with CAM have the best SHDs equal to 11 and 12 respectively, which is slightly better than ours which is 15 .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We presented a novel form on learning DAGs from Fourier-sparse data building on a recently proposed causal Fourier analysis framework.
Revised: We presented a novel form on learning DAGs from Fourier-sparse data building on a recently proposed causal Fourier analysis framework.

Original: We analyzed linear SEMs from a Fourier analysis point of view and discovered that the data they generate have random spectra in the Fourier domain.
Revised: We analyzed linear SEMs from a Fourier analysis point of view and discovered that the data they generate have random spectra in the Fourier domain.

Original: This observation allowed us to generalize the data generating process of linear SEMs by further capturing the notion of (approximate) sparsity in the spectral domain.
Revised: This observation allowed us to generalize the data generating process of linear SEMs by further capturing the notion of (approximate) sparsity in the spectral domain.

Original: Fourier-sparsity has been a frequently exploited feature in various types of Fourier domains, and we provide an intuitive example to demonstrate viability for DAGs.
Revised: Fourier-sparsity has been a frequently exploited feature in various types of Fourier domains, and we provide an intuitive example to demonstrate viability for DAGs.

Original: To learn DAGs from Fourier-sparse data, we build on prior work to enforce acyclicity but propose an alternative L 1 -loss on the approximated spectrum.
Revised: To learn DAGs from Fourier-sparse data, we build on prior work to enforce acyclicity but propose an alternative L 1 -loss on the approximated spectrum.

Original: Empirically, we show that our method has the best performance under the Fourier-sparsity assumption compared to NOTEARS, DAG-NoCurl, and other DAG learning baselines.
Revised: Empirically, we show that our method has the best performance under the Fourier-sparsity assumption compared to NOTEARS, DAG-NoCurl, and other DAG learning baselines.

Original: Thus we hope that our contribution can serve as a valuable building block for learning DAGs in real case scenarios where current methods fail.
Revised: Thus we hope that our contribution can serve as a valuable building block for learning DAGs in real case scenarios where current methods fail.

Original: A P ROOFS
Revised: A P ROOFS

Original: Here we present an additional proof on Lemma 2 that allows us to get a better understanding of how the values are propagated from parent vertices to their children.
Revised: Here we present an additional proof on Lemma 2 that allows us to get a better understanding of how the values are propagated from parent vertices to their children.

Original: Proof.
Revised: Proof.

Original: Consider the total ordering 1 , 2 , ..., d of the vertices of the graph.
Revised: Consider the total ordering 1 , 2 , ..., d of the vertices of the graph.

Original: For each vertex i we will show by induction that X : ,i = N (cid:0) I : ,i + W : ,i + W 2: ,i + ...
Revised: For each vertex i we will show by induction that X : ,i = N (cid:0) I : ,i + W : ,i + W 2: ,i + ...

Original: + W k : ,i (cid:1)
Revised: + W k : ,i (cid:1)

Original: Starting with the sources, we have that
Revised: Starting with the sources, we have that

Original: [Equation]
Revised: [Equation]

Original: since W : ,i = 0 and W j : ,i =
Revised: since W : ,i = 0 and W j : ,i =

Original: W j − 1 W : ,i = 0 , for all j > 1 .
Revised: W j − 1 W : ,i = 0 , for all j > 1 .

Original: Now assume that the statement is true up to the i − th vertex according to the ordering.
Revised: Now assume that the statement is true up to the i − th vertex according to the ordering.

Original: Consider, now the vertex i + 1 .
Revised: Consider, now the vertex i + 1 .

Original: Then Pa ( i + 1) ⊆ { 1 , 2 , ..., i } and according to the update rule of the algorithm.
Revised: Then Pa ( i + 1) ⊆ { 1 , 2 , ..., i } and according to the update rule of the algorithm.

Original: [Equation]
Revised: [Equation]

Original: The relation
Revised: The relation

Original: [Equation]
Revised: [Equation]

Original: holds by induction and
Revised: holds by induction and

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: because W j,i +1 = 0 for all j / ∈
Revised: because W j,i +1 = 0 for all j / ∈

Original: Pa ( i + 1) .
Revised: Pa ( i + 1) .

Original: Also W k +

Revised: Also W k +

Original: =
Revised: =

Original: O since W is nilpotent.
Revised: O since W is nilpotent.

Original: The previous proof leads to conclude that the polynomial W + W 2 + ...
Revised: The previous proof leads to conclude that the polynomial W + W 2 + ...

Original: + W k works as the transitive closure W of the weighted adjacency matrix.
Revised: + W k works as the transitive closure W of the weighted adjacency matrix.

Original: In addition to Lemma 1, we prove the following lemma to provide a condition that works as a deterministic assumption for the identifiability of the graph from Fourier-sparse data.
Revised: In addition to Lemma 1, we prove the following lemma to provide a condition that works as a deterministic criterion for the identifiability of the graph from Fourier-sparse data.

Original: This lemma simply states that under assumptions on the spectrum and the noise, the generated data correspond uniquely to the unknown DAG.
Revised: This lemma simply states that under assumptions on the spectrum and the noise, the generated data correspond uniquely to the unknown DAG.

Original: Lemma 3.
Revised: Lemma 3.

Original: If the transitive closure of the graph is bijective and the data X are generated with fixed spectral data C + N f with full rank and zero signal noise, then any two graphs generating the same signal data X using zero noise are equal.
Revised: If the transitive closure of the graph is bijective and the data X are generated with fixed spectral data C + N f with full rank and zero signal noise, then any two graphs generating the same signal data X using zero noise are equal.

Original: Proof.
Revised: Proof.

Original: Indeed, let X A be generated by the graph with weighted adjacency matrix A and X B similarly by B .
Revised: Indeed, let X A be generated by the graph with weighted adjacency matrix A and X B similarly by B .

Original: Then if X A = X B we have that
Revised: Then if X A = X B we have that

Original: [Equation]
Revised: [Equation]

Revised: Notice that the previous criterion doesn’t make any assumption on the variances of the noise variables.

Revised: In the special case when the signal noise is zero, our model coincides with the linear SEM formulation and in that case we can also use standard identifiability criterions from the literature.

Revised: In that case

Revised: [Equation]

Revised: The way we generate the independent causes is the following.

Revised: Each data point c in the spectrum is set to 0 with probability (1 − p ) , otherwise set uniformly at random to a value [0 , 1] .

Revised: Then at each such point we add the corresponding noise n f .

Revised: So this means that for each node the variable c + n f has the same variance.

Revised: Therefore, this case falls in the category of equal noise variances for linear SEMs and is thus identifiable according to (Peters & Bühlmann, 2014).

Revised: This is supported empirically, by the fact that our method achieves very high accuracy on discovering the true DAG.

Revised: The scenario where we have fixed sparsity support is not generated with equal variances and thus is not justified to be identifiable.

Revised: We are currently working towards analyzing the theory for the identifiability of our model.

Revised: As preliminary result we may say that when the support of the sparsity is concentrated on the sink nodes then our model is identifiable due to the conditional variance criterion (Park, 2020).

Revised: B A DDITIONAL EXPERIMENTAL RESULTS

Revised: In addition to the TPR metric of Table 4 we provide the computation of SHD in the table below.

Revised: [Figure]

Revised: Hyperparameter

Revised: [Table]

Revised: Varsortability.

Revised: Moreover, in the table above we compute the varsortability in each different experimental setting.

Revised: We notice that we have high varsortability in general.

Revised: However, our measurements are in general lower than the measurements reported in (Reisach et al., 2021)

Revised: (Appendix G.1) for linear SEMs.

Revised: In some cases is around 5% lower than those of linear SEMs, which is a sign that in our experimental setting it is not trivial to learn the DAG.

Revised: This can also be deduced by the performance of sortnregress, which has decent performance but is not achieving the best results.

Revised: It is also worth mentioning that the fixed sparsity support case, seems like the most interesting one as all the methods fail in this case and varsortability is very low.

Revised: Therefore, there is at least one scenario where our model successfully proposes a hard problem for DAG learning.