Original: Discrete Learning Of DAGs Via BackpropagationAnonymous Author(s)
Revised: Learning Discrete Directed Acyclic Graphs via Backpropagation

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Recently continuous relaxations have been proposed in order to learn directedacyclic graphs (DAGs)  by backpropagation, instead of  combinatorial optimization.
Revised: Recently continuous relaxations have been proposed in order to learn Directed Acyclic Graphs (DAGs) from data by backpropagation, instead of using combinatorial optimization.

Original: However, a number of techniques for fully discrete backpropagation could insteadbe applied.
Revised: However, a number of techniques for fully discrete backpropagation could instead be applied.

Original: In this paper, we explore this direction and propose DAG-DB, a framework for learning DAGs by Discrete Backpropagation, based on the architecture of
Revised: In this paper, we explore that direction and propose DAG-DB, a framework for learning DAGs by Discrete Backpropagation.

Original:  Implicit Maximum Likelihood Estimation (I-MLE) [1].
Revised: Based on the architecture of Implicit Maximum Likelihood Estimation [I-MLE, 1], DAGDB adopts a probabilistic approach to the problem, sampling binary adjacency matrices from an implicit probability distribution.

Original: DAG-DB performs competitively using either of two fully discrete backpropagation techniques, I-MLEitself, or straight–through estimation [2, 3]
Revised: DAG-DB learns a parameter for the distribution from the loss incurred by each sample, performing competitively using either of two fully discrete backpropagation techniques, namely I-MLE and Straight-Through Estimation.

Original:  Introduction
Revised:  Introduction

Original: Aim.
Revised: Aim.

Original: Directed acyclic graphs (DAGs) occur in a wide range of contexts, including project management, version control systems, evolutionary biology, and Bayesian networks.
Revised: Directed Acyclic Graphs (DAGs) occur in a wide range of contexts, including project management, version control systems, evolutionary biology, and Bayesian networks.

Original: Bayesian networkshave been a particularly popular subject for machine learning, including for the problem consideredin this paper, learning a Bayesian network’s DAG  from data generated by the Bayesian network.
Revised: Bayesian networks have been a particularly popular subject for machine learning, including for the problem considered in this paper, learning a Bayesian network’s DAG structure from data.

Original: Thispaper aims to learn DAGs using fully discrete backpropagation,  avoidingthe relaxation from discreteto continuous variables which is a common part of non-combinatoric DAG learning [for example,4–16].
Revised: This paper aims to learn DAGs using fully-discrete backpropagation, i.e. avoiding continuous relaxations, which are usually used for learning DAGs from data – for example, see [2–13].

Original: Bayesian networks.
Revised: Bayesian networks.

Original: A Bayesian network associates a DAG with a random variable which hascomponents indexed by the DAG nodes.
Revised: A Bayesian network associates a DAG with a random variable which has components indexed by the DAG nodes.

Original: A directed edge i j between nodes i and j representsa dependency of the random variable’s j component on its i component.
Revised: A directed edge i j between nodes i and j represents a dependency of the random variable’s j component on its i component.

Original: Discussion of Bayesiannetworks and DAGs may be found in textbooks [e.g. 17–19]; to establish our terminology, appendix Agives a very brief outline.
Revised: Discussion of Bayesian networks and DAGs may be found in textbooks (e.g., see [14–16]). To establish our terminology, appendix A gives a very brief outline.

Original: Contribution.
Revised: Contribution.

Original: Our contribution is to show that backpropagation methods which retain the fullydiscrete nature of a directed acyclic graph can be used to predict DAGs from data.
Revised: Our contribution is to show that backpropagation methods which retain the fullydiscrete nature of a DAG (i.e., that do not rely on continuous relaxations) can be used to predict DAGs from data.

Original: We avoid thecommon approach of ‘relaxing’ digraph adjacency matrices froma binary toa real matrix.
Revised: We avoid the common approach of “relaxing” digraph adjacency matrices from binary to real matrices.

Original: Instead,we sample discrete variables probabilistically and use methods of backpropagating that do not relaxthe variables to be continuous.
Revised: Instead, we sample discrete variables probabilistically and use methods of backpropagating that do not relax the variables to be continuous.

Original: No ethical issues are raised.

Original: Related work and background

Original: Learning DAGs from data.

Original: Most ways of learning DAGs from data can classed as combinatoricmethods [reviewed in 20] or continuous optimisation methods [reviewed in 21].

Original: Combinatoricmethods may themselves be labelled as constraint–based, identifying the graph from conditionalindependence testing, and score–based, searching the space of possible graphs using a score functionto evaluate search–results [20].

Original: The PC algorithm [22] is a well–known constraint–based method, witha number of variant algorithms, for example PC-Stable [23] which reduces dependency on arbitrarynode ordering.

Original: An example of a score–based approach is the Fast Greedy Equivalence Search (FGES)

Original: x Θ
Revised: x Θ

Original: Z x (cid:5) Z ˜ x
Revised: Z x ⋄ Z ˜ x

Original: MSE( x , ˜ x ) (cid:96) =
Revised: MSE( x , ˜ x ) ℓ =

Original: MSE( x , ˜ x ) + r ( Z )
Revised: MSE( x , ˜ x ) + r ( Z )

Original: L r ( Z )
Revised: L r ( Z )

Original: [Figure] ∼ p ( Z ; Θ )
Revised: D pred ∼ p ( Z ; Θ )

Original: GFAS r f Φ
Revised: GFAS r f Φ

Original: E p  algorithm
Revised: E p  Related work and background Learning DAGs from data.

Original: [24], which is an optimised version of the Greedy Equivalence Search (GES) 
Revised: Most ways of learning DAGs from data can be classed as combinatoric methods (see [17] for a review) or continuous optimisation methods (reviewed in [18]). Combinatoric methods may themselves be labelled as constraint-based, identifying the graph from conditional independence testing, and score-based, searching the space of possible graphs using a score function to evaluate search results [17]. The PC algorithm [19] is a well-known constraint-based method, with a number of variant algorithms, such as PC-Stable [20], which reduces dependency on arbitrary node ordering. An example of a score-based approach is the Fast Greedy Equivalence Search (FGES) algorithm [21], which is an optimised version of the Greedy Equivalence Search (GES) [22, 23].

Original: Thedevelopment of NOTEARS in 2018 [4] for linearly–generated data signalled the start of widespreaddevelopment of continuous optimisation approaches, including the GOLEM method [10] which isgenerally even more successful than NOTEARS for linear data with, for example, Gaussian noise.
Revised: The development of NOTEARS in 2018 [2] for linearly-generated data signalled the start of widespread development of continuous optimisation approaches, including the GOLEM method [8] which is generally even more successful than NOTEARS for linear data with, for example, Gaussian noise.

Original: Many other continuous methods have recently been developed [for example, 5–9, 11–16] and themethod presented in this paper, which uses gradient descent, may be viewed as related to these.
Revised: Many other continuous methods have recently been developed (e.g., see [3–7, 9–13]) and the method presented in this paper, which uses gradient descent, may be viewed as related to these.

Original: However, continuous methods generally adopt some approach of ‘relaxing’ the discrete property ofan edge being present or absent into a continuous variable, whereas our DAG-DB approach maintainsedges as fully discrete, binary objects, even during our method’s training phase.
Revised: However, continuous methods generally adopt some approach of ‘relaxing’ the discrete property of an edge being present or absent into a continuous variable, whereas our DAG-DB approach maintains edges as fully discrete, binary objects, even during our method’s training phase.

Original: Our approach mightbe termed a ‘probabilistic relaxation’, and another example of this can be found in the SDI [27]framework, which focuses on contexts where data may be generated by interventions during training,whereas, here, our focus is on pre–generated ‘observational’ data.
Revised: Our approach might be termed a ‘probabilistic relaxation’, and another example of this can be found in the SDI [24] framework, which focuses on contexts where data may be generated by interventions during training, whereas, here, our focus is on pre-generated ‘observational’ data.

Revised: Recently, a Bayesian approach has also been taken (e.g., see [25–27]), focusing on predicting a distribution of DAGs, instead of, as we do, seeking to predict only the most likely.

Original: Discrete backpropagation.
Revised: Discrete backpropagation.

Original: A number of methods have also been developed to allow backpropagation without transforming discrete variables to be continuous in training.
Revised: A number of methods have also been developed to allow backpropagation without transforming discrete variables to be continuous in training.

Original: We take four such methodsas examples.
Revised: We take four such methods as examples.

Original: Straight–through estimation (STE) [2, 3] essentially ignores the discrete nature of thevariables, and, as we will see, can be surprisingly effective.
Revised: Straight-through estimation [STE, 28, 29] ignores the derivative of a discrete function during back-propagation and passes on the incoming gradient as if the function was the identity function; this simple approach, as we will see, can be surprisingly effective.

Original: Score–function estimation (SFE) [28]uses the elementary rules of dfferentiation to estimate the gradient of an expectation by samplingwith respect to a probability distribution.
Revised: Score-function estimation [SFE, 30], also referred to as REINFORCE, rewrites the gradient of an expectation in an expectation using the log-derivative trick, which can be then estimated using Monte Carlo methods.

Original: ‘Blackbox’ differentiation (BB) [29] is a way of adjustingcontinuous inputs to a combinatorial solver in order to mimic gradient descent for the discrete variable.
Revised: Black-box differentiation [BB, 31] is a way of adjusting continuous inputs to a combinatorial solver in order to mimic gradient descent for the discrete variable.

Original: Implicit maximum likelihood estimation (I-MLE) [1] incorporates noise into BB to handle discreterandom variables.
Revised: Implicit maximum likelihood estimation [I-MLE, 1] incorporates noise into BB to handle discrete random variables.

Original: This allows combinatorial solvers to be used for maximum likelihood estimation.
Revised: This allows combinatorial solvers to be used for maximum likelihood estimation.

Original: I-MLE also makes use of both STE and a seminal result by Domke [30].
Revised: I-MLE also makes use of both STE and a seminal result by Domke [32].

Original: We note that SDI [27],highlighted above, uses a particular Bernoulli–based SFE-like form of backpropagation [31].
Revised: We note that SDI [24], highlighted above, uses a particular Bernoulli-based SFE-like form of backpropagation [33].

Original: Perturb–and–MAP sampling.
Revised: Perturb-and-MAP sampling.

Original: We use the technique of perturb–and–MAP (P&M) sampling ofrandom variables [32].
Revised: We use the technique of perturb-and-MAP (P&M) sampling of random variables [34].

Original: This approximates a parametrized distribution, say p ( Z ; Θ ) , witha randommatrix variable Z and continuous parametermatrix Θ , as
Revised: This approximates a matrix distribution, say p ( Z ; Θ ) , with parameter Θ , as

Original: [Equation]
Revised: [Equation]

Original: where each element of the noise matrix Ψ is i.i.d.
Revised: where each element of the noise matrix Ψ is i.i.d.

Original:  sampled from a standard one–dimensionaldistribution, τ > 0 is a temperature, and MAP( Θ + τ Ψ ) denotes the most likely value of Z withrespect to p ( Z ; Θ + τ Ψ ) .
Revised: and sampled from a standard one-dimensional distribution, τ > 0 is a temperature, and MAP( Θ + τ Ψ ) denotes the most likely value of Z with respect to p ( Z ; Θ + τ Ψ ) .

Original: P&M is useful when sampling p ( Z ; Θ ) directly is intractable or expensive,and the MAP solver is cheap. Maximum DAG. Our random variable Z will represent the adjacency matrix of a directed graph.
Revised: P&M is useful when sampling p ( Z ; Θ ) directly is intractable or expensive, and the MAP solver is cheap, and facilitates I-MLE’s approach to backpropagation.

Original: To ﬁnd a DAG from that digraph, we will use an approximate solution to the ‘maximum DAG’problem.

Original: This considers a weighted digraph (each edge is assigned a real number) and the challengeis to remove edges from the digraph to form a DAG, whilst maximising the sum of the remainingedge–weights.

Original: We found it most efﬁcient, and necessary as the number of digraph nodes rose toward100, to use an approximate maximum DAG solver, ‘Greedy Feedback Arc Set’ (GFAS)

Original: [33], our

Original: Python implementation being adapted from (more optimised)

Original: Java code [34, 35].

Original:  Method
Revised:  Method

Original: Framework.
Revised: Framework.

Original: Figure 1 gives an overview of the DAG-DB framework.
Revised: Figure 1 gives an overview of the DAG-DB framework.

Original: Suppose we wish to learna DAG with d nodes from a dataset X ∈ R n × d of n data points x ∈ R d .
Revised: Suppose we wish to learn a DAG with d nodes from a dataset X ∈ R n × d of n data points x ∈ R d .

Original: Let R ⊂ R d × d
Revised: Let R ⊂ R d × d

Original: and
Revised: and

Original: B ⊂ { 0 , 1 } d × d be the sets of zero–diagonal, respectively real and binary, matrices.
Revised: B ⊂ { 0 , 1 } d × d be the sets of zero-diagonal, respectively real and binary, matrices.

Original: A learnable vectorΘ ∈ R parametrizes a exponential family distribution p ( Z ; Θ ) ,
Revised: A learnable vector Θ ∈ R parameterises a exponential family distribution p ( Z ; Θ ) ,

Original: [Equation]
Revised: [Equation]

Original: where Z ∈ Z ⊆ B is a discrete matrix, τ > 0 is a temperature, and A( Θ ) normalizes p ( Z ; Θ ) tosum to one.
Revised: where Z ∈ Z ⊆ B is a discrete matrix, τ > 0 is a temperature, and A( Θ ) normalizes p ( Z ; Θ ) to sum to one.

Original: Recognising the matrix form of Θ and Z , eq.
Revised: Recognising the matrix form of Θ and Z , eq.

Original: (2) uses the Frobenius scalar product,
Revised: (2) uses the Frobenius scalar product,

Original: [Equation]
Revised: [Equation]

Original: The matrix Z is interpreted as the adjacency matrix of a directed graph  with, because of its zerodiagonal, no self–loops.
Revised: The matrix Z is interpreted as the adjacency matrix of a directed graph on d nodes, with, because of its zero diagonal, no self-loops.

Original: In the forward pass, samples Z ( s ) , s = 1 , ..., S, from p ( Z ; Θ ) are takenusing the P&M sampling outlined in section 2.
Revised: In the forward pass, samples Z ( s ) , s = 1 , ..., S, from p ( Z ; Θ ) are taken using the P&M sampling outlined in section 2.

Original: For Z = B , the MAP solver sets matrix elementsof MAP( Θ ) to be one if Θ > 0 and zero otherwise.
Revised: For Z = B , the MAP solver sets matrix elements of MAP( Θ ) to be one if Θ > 0 and zero otherwise.

Original: We found empirically that it is best to use thestandard logistic distribution to generate the noise Ψ in eq.
Revised: We found empirically that it is best to use the standard logistic distribution to generate the noise Ψ in eq.

Original: This can also be theoretically justiﬁedas a good choice because of the binary nature of Z ’s matrix elements [32].
Revised: This can also be theoretically justified as a good choice because of the binary nature of Z ’s matrix elements [34].

Original: As is common for continuous optimisation methods, DAG learning proceeds bysolving the problemof predicting values of x components x j from its values
Revised: Maximum DAG. To go from the parameter Θ and a digraph Z to a DAG, we can consider the digraph edges as weighted by the corresponding matrix elements of Θ , and then solve the associated maximum directed acyclic subgraph problem. Solving this well-known “maximum DAG” problem involves identifying the directed acyclic sub-graph with the maximum sum of weights. To find the maximum DAG from Θ and Z , we devise both exact and approximate solvers.

Revised: For exactly solving the maximum DAG problem, we cast it as a constrained combinatorial optimisation problem, which we solve using MiniZinc [35, 36], a free and open-source constraint modelling language.

Original: x i at ‘parent’ nodes with edges i j .
Revised: We found it most efficient, and necessary as the number of digraph nodes rose toward 100, to use an approximate maximum DAG solver, “Greedy Feedback Arc Set” [GFAS, 37], our Python implementation being adapted from (more optimised) Java code [38, 39]. Tests for 30 nodes and fewer, for which we could use the exact MiniZinc solver, suggested that the solutions of our approximate GFAS solver were in practice exact, or close to exact, due, most likely, to regularizing (see below) Z towards being a DAG.

Revised: Whatever maximum DAG solver is chosen, it can be applied either directly as part of the MAP solver, or subsequent to the MAP solver.

Revised: We adopt latter option, which has the advantage of allowing an approach in which training is done without the maximum DAG solver, deploying it only at evaluation.

Revised: In all but one part of an experiment, that ‘evaluation only’ approach is the one we will use.

Revised: Learning As is common for continuous optimisation methods, learning proceeds by solving the problem of predicting values of x components x j from its values x i at ‘parent’ nodes with edges i j .

Original: Asset out in appendix C, this is ensured by the ‘graphiﬁcation’ operation x (cid:5) Z in the context of thelinear map f Φ .
Revised: As set out in appendix C, this is ensured by the graphification operation x ⋄ Z in the context of the linear map f Φ .

Original: Elsewhere, this technique usually employs a weighted adjacency matrix with realelements [see, e.g., 4, 8, 10]; but we use the binary adjacency matrix Z without such a relaxation.
Revised: Elsewhere, this technique usually employs a weighted adjacency matrix with real elements (see e.g., [2, 6, 8]); but we use the binary adjacency matrix Z without such a relaxation.

Original: Tothe extent we relax the problem, this is by treating the digraph probabilistically via the distributionp ( Z ; Θ ) , which, in practice, becomes concentrated around the most likely adjacency matrix.
Revised: To the extent we relax the problem, this is by treating the digraph probabilistically via the distribution p ( Z ; Θ ) , which, in practice, becomes concentrated around the most likely adjacency matrix.

Original: The parameters Θ and Φ are trained by feeding data points x ∈ X into the model batchwise.
Revised: The parameters Θ and Φ are trained by feeding data points x ∈ X into the model batch-wise.

Original: The mean–squared error between ˜ x and x is added to a regularizing function r ( Z ) to give a loss(cid:96) associated with the sample Z .
Revised: The mean-squared error between ˜ x and x is added to a regularizing function r ( Z ) to give a loss ℓ associated with the sample Z .

Original: The empirical mean of these losses, over the samples Z and thebatch members x , gives the batch loss
Revised: The empirical mean of these losses, over the samples Z and the batch members x , gives the batch loss

Original: L.
Revised: L.

Original: Our learnt parameters Φ and Θ are then updated bybackpropagation: while for Φ this is standard backpropagation, a technique needs to be chosen fordiscrete backpropagation from Z to Θ .
Revised: Our learnt parameters Φ and Θ are then updated by backpropagation: while for Φ this is standard backpropagation, a technique needs to be chosen for discrete backpropagation from Z to Θ .

Original: We found I-MLE [1] and straight–through estimation [2, 3]to be useful techniques for such backpropagation; we had less success here with score–functionestimation [28] and ‘blackbox’ differentiation [29].
Revised: We found I-MLE [1] and straight-through estimation [28, 29] to be useful techniques for such backpropagation; we had less success here with score-function estimation [30] and black-box differentiation [31].

Original: Appendix B gives details of I-MLE and straight–through estimation as used in DAG-DB.
Revised: Appendix B gives details of I-MLE and straightthrough estimation as used in DAG-DB.

Original: After the ﬁnal epoch of training, the GFAS algorithm is applied.

Original: We take the single most probabledirected graph Z = MAP( Θ ) , and then ﬁnd its maximum DAG when weighted by Θ .

Original: Testsfor 30 nodes and fewer, for which we could use an exact maximum DAG solver we built using

Original: MiniZinc [36, 37], suggested that the solutions of our solver were in practice exact, or close to exact,due, most likely, to regularizing (see below)

Original: Z towards being a DAG.

Original: Because of computational cost,for all but one of our experiments, we use GFAS only at evaluation after the ﬁnal epoch.

Original: Regularization.
Revised: Regularization.

Original: DAG-DB can employ both functional and constraint regularization.
Revised: DAG-DB can employ both functional and constraint regularization.

Original: Functionalregularization is based on that of NOTEARS [4], and provided by the function
Revised: Functional regularization is based on that of NOTEARS [2] and provided by the function

Original: [Equation]
Revised: [Equation]

Original: where ρ DAG , ρ sp > 0 are strength coefﬁcients.
Revised: where ρ DAG , ρ sp > 0 are strength coefficients.

Original: For the DAG-regularizer r DAG , we use the binaryversion of NOTEARS’ DAG regularizer,
Revised: For the DAG-regularizer r DAG , we use the binary version of the DAG regularize used by NOTEARS,

Original: [Equation]
Revised: [Equation]

Original: which has been shown [4, prop. 2] to vanish if and only if Z is the adjacency matrix for a DAG.
Revised: which has been shown [2, prop. 2] to vanish if and only if Z is the adjacency matrix for a DAG.

Original: Thesecond regularizer, r sp , is an L 1 -regularizer promoting sparsity by summing Z ’s matrix elements.
Revised: The second regularizer, r sp , is an L 1 -regularizer promoting sparsity by summing Z ’s matrix elements.

Original: The other form of regularization is to constrain Z to within a proper subset Z (cid:36) B .
Revised: The other form of regularization is to constrain Z to within a proper subset Z ⫋ B .

Original: We optionallyintroduce a maximum size  of digraph, constraining Z to have no more than a given maximum number M of non–zero entries (edges).
Revised: We optionally introduce a maximum size (i.e. number of edges) of digraph, constraining Z to have no more than a given maximum number M of non-zero entries (edges).

Original: It is here that P&M sampling becomes particularly helpful: withno such constraint, we could have used Bernoulli sampling instead of P&M; but, with a maximumsize, there is no tractable direct sampling method.

Original: For a maximum size M, the MAP solver identiﬁesthe M biggest elements of Θ , but drops any that are not positive.
Revised: For a maximum size M, the MAP solver identifies the M biggest elements of Θ , but drops any that are not positive.

Original: Matrix elements of MAP( Θ )corresponding to the resulting index set have value one, and the remainder have value zero.[Table]
Revised: Matrix elements of MAP( Θ ) corresponding to the resulting index set have value one, and the remainder have value zero.

Original:  Experiments
Revised:  Experiments

Original: Overview.
Revised: Overview.

Original: As detailed in appendix D.3, we identiﬁed useful hyperparameters settings, STE_84 ,using STE with a maximum size 84, and IMLE_None , using I-MLE with no maximum size constraint.
Revised: As detailed in appendix D.3, we identified useful hyperparameters settings, STE_84 , using STE with a maximum size 84, and IMLE_None , using I-MLE with no maximum size constraint.

Original: We then conducted ablation experiments, and tests on linearly–generated synthetic data, and on a realdataset.
Revised: We then conducted ablation experiments, and tests on linearly-generated synthetic data, and on a real dataset.

Original: The main metric we use is SHD c , often normalised to nSHD c , complemented by a precisionmeasure prec c and, sometimes also by a recall measure rec c .
Revised: The main metric we use is SHD c , often normalised to nSHD c , complemented by a precision measure prec c and, sometimes also by a recall measure rec c .

Original: Appendix D.2 gives more details onthese metrics, as part of supplementary material on the experiments in appendix D.
Revised: Appendix D.2 gives more details on these metrics, as part of supplementary material on the experiments in appendix D.

Original: Ablation experiments.
Revised: Ablation experiments.

Original: We performed further ablation experiments to identify the relative role ofthe regularizers, with results shown in appendix D.4.
Revised: We performed further ablation experiments to identify the relative role of the regularizers, with results shown in appendix D.4.

Original: We found that r DAG , and, the appropriate settingor not of a maximum size regularizer, both have a notable good effect.
Revised: We found that r DAG , and, the appropriate setting or not of a maximum size regularizer, both have a notable good effect.

Original: However, the presence of asparsity regularizer r sp , although almost always helpful, was only very relevant for STE_84 , and thenonly when the maximum size constraint was also ablated.
Revised: However, the presence of a sparsity regularizer r sp , although almost always helpful, was only very relevant for STE_84 , and then only when the maximum size constraint was also ablated.

Original: Synthetic data experiments.
Revised: Synthetic data experiments.

Original: We generated random Erdös–Rényi ‘ER k ’ and Barabási-–Albert‘SF k ’ DAGs and further generated data from these using a Gaussian equal–variance linear additivenoise model.
Revised: We generated random Erdös-Rényi ‘ER k ’ and Barabási-Albert ‘SF k ’ DAGs and further generated data from these using a Gaussian equal-variance linear additive noise model as the Bayesian network.

Original: The number of edges possible in these synthetic graphs varies considerably, so themaximum size limit for STE_84 was adjusted in proportion to the expected size.
Revised: The number of edges possible in these synthetic graphs varies considerably, so the maximum size limit for STE_84 was adjusted in proportion to the expected size.

Original: We tested ourmethods against those highlighted in section 2 on sets of 24 random graphs, with results shownin ﬁg. 2.
Revised: We tested our methods against those highlighted in section 2 on sets of 24 random graphs, with results shown in fig.

Original: We can see that the DAG-DB methods usually outperform the combinatoric approaches,and they are themselves outperformed by the other continuous methods.
Revised: We can see that the DAG-DB methods usually outperform the combinatorial approaches, and they are themselves outperformed by the other continuous methods.

Original: Of our methods, STE_84performs better, with the maximum size adjustments noted above, while IMLE_None ’s performance isfairly close, except for ER4 and SF4 with d = 100 nodes, where its metrics worsen noticeably.
Revised: Of our methods, STE_84 performs better, with the maximum size adjustments noted above, while IMLE_None ’s performance is fairly close, except for ER4 and SF4 with d = 100 nodes, where its metrics worsen noticeably.

Original: Real data experiments.
Revised: Real data experiments.

Original: We performed experiments on the Sachs cellular biochemistry dataset,representable as a DAG with 11 nodes and 17 edges [38].
Revised: We performed experiments on the Sachs cellular biochemistry dataset, where each instance can be represented as a DAG with 11 nodes and 17 edges [40].

Original: Performance noted previously [4, 10]suggests that linear models perform reasonably on Sachs’ 853-data point set of purely observationaldata.
Revised: Performance noted previously [2, 8] suggests that linear models perform reasonably on Sachs’ 853-data point set of purely observational data.

Original: Table 1 compares our results against other methods.
Revised: Table 1 compares our results against other methods.

Original: For STE_84 , the table also indicatesmaximum size constraints which have to be reset, by a proportioning method (23) and with referenceto typical, relatively successful, sizes predicted by other methods (8).
Revised: For STE_84 , the table also indicates maximum size constraints which have to be reset, by a proportioning method (23) and with reference to typical, relatively successful, sizes predicted by other methods (8).

Original: We can see that IMLE_None ismuch more successful than either STE_84 setting, and, to account for a stochastic element in DAG-DBresults, we therefore show, for I-MLE only, the results over 24 runs on the same Sachs data.
Revised: We can see that IMLE_None is much more successful than either STE_84 setting, and, to account for a stochastic element in DAG-DB results, we therefore show, for I-MLE only, the results over 24 runs on the same Sachs data.

Original: We cansee from the table that IMLE_None ’s performance is quite close to that of GOLEM and NOTEARSwhich are the best of the comparison methods.
Revised: We can see from the table that IMLE_None ’s performance is quite close to that of GOLEM and NOTEARS which are the best of the comparison methods.

Original: Standard deviations are discussed in appendix D.7.
Revised: Standard deviations are discussed in appendix D.7.

Original: We also ran a method IMLE_None_Tr , which, unlike our other DAG-DB methods, applies GFAS tomake Z the adjacency matrix of a DAG, in training .
Revised: We also ran a method IMLE_None_Tr , which, unlike our other DAG-DB methods, applies GFAS to make Z the adjacency matrix of a DAG, in training .

Original: In light of the associated compute time, we didnot undertake a full hyperparameter search for IMLE_None , and instead only vary a Domke–relatedhyperparameter to which I-MLE can be sensitive.
Revised: In light of the associated compute time, we did not undertake a full hyperparameter search for IMLE_None , and instead only vary the λ hyperparameter to which I-MLE can be sensitive.

Original: Generally digraph training performs marginallybetter on SHD c and often markedly  and 
Revised: Generally digraph training performs marginally better on SHD c and often markedly better on prec c and rec c .

Revised: This may be because it was not practical to optimise hyperparameters for DAG training.

Revised: The results can be found in appendix D.7.

Original: [Figure]
Revised: [Table]

Original: Conclusion and future work{+N one GO L E M N O T EA R S F G ES P C 0. 0.8 1.0 graph_type = SF4 | d = S T E _
+}
Revised: {+N one GO L E M N O T EA R S F G ES P C 0. 0.8 1.0 graph_type = SF4 | d = S T E _
+}

Original: For linear data, DAG-DB mostly outperforms combinatoric methods tested here, but is itself out–performed by the continuous methods.{+I M L E _ N one GO L E M N O T EA R S F G ES P C graph_type = SF4 | d = S T E _
+}
Revised: {+I M L E _ N one GO L E M N O T EA R S F G ES P C graph_type = SF4 | d = S T E _
+}

Original: The DAG-DB framework should adapt to data generated bynon–linear models and, potentially, to discrete data, or to interventional causal models [18].
Revised: I M L E _ N one GO L E M N O T EA R S F G ES P C graph_type = SF4 | d = [Table]

Original:  It wouldalso be interesting to see if the SFE-like backpropagation [31] used in SDI [27] would workfor DAG-DB. Separately, one of the I-MLE paper’s experiments [1] built a variational autoencoder (VAE),and a similar experiment could be performed with a latent space of DAGs.
Revised:  Conclusion and future work

Original: A VAE might learn asingle DAG for a given dataset, and also learn values for each data point on that DAG’s nodes. Thismight be interpretable as learning a discrete hierarchy representing characteristics of the dataset.
Revised: For linear data, DAG-DB mostly outperforms combinatorial methods tested here, but is itself outperformed by the continuous methods. The DAG-DB framework should adapt to data generated by non-linear models and, potentially, to discrete data [Table]

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’scontributions and scope? [Yes]
Revised: or to causal models [15].

Original: b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work? [N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to 
Revised: It would also be interesting to see if the SFE-like backpropagation [33] used in SDI [24] would work for DAG-DB.

Original: them?

Original: [Yes]

Original: If you are including theoretical results...

Original: a) Did you state the full set of assumptions of all theoretical results? [N/A] (b) Did you include complete proofs of all theoretical results? [N/A]
Revised: Separately, one of the I-MLE paper’s experiments [1] built a variational auto-encoder (VAE), and a similar experiment could be performed with a latent space of DAGs.

Original: If you ran experiments... a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [No]
Revised: A VAE might learn a single

Original: b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
Revised: DAG for a given dataset, and also learn values for each data point on that DAG’s nodes.

Original: [Yes]

Original: c) Did you report error bars (e

Original: g., with respect to the random seed after running experi-

Original: ments multiple times)? [Yes] d) Did you include the total amount ofcompute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
Revised: This might be interpretable as learning a discrete hierarchy representing characteristics of the dataset.

Original: [Yes]

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]

Original: (b) Did you mention the license of the assets?

Original: [Yes]

Original: c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating? [N/A]
Revised: Acknowledgments and Disclosure of Funding

Original: e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?

Original: [N/A] If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? [N/A]
Revised: Andrew J. Wren is very grateful to his co-authors who supervised him in the University College

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: London MSc Machine Learning thesis project from which this paper derives.

Original: spent on participant compensation?

Original: [N/A]

Original: A Notation and terminology
Revised: A Notation and terminology

Original: Notation.
Revised: Notation.

Original: We write vectors in bold, like x .
Revised: We write vectors in bold, like x .

Original: Matrices are in bold capitals, like Z or Θ .
Revised: Matrices are in bold capitals, like Z or Θ .

Original: Randomvariables, like U, are in non–bold capitals even if they yield vectors.
Revised: Random variables, like U, are in non–bold capitals even if they yield vectors.

Original: Graph–related objects appearin sans serif, as in D or j .
Revised: Graph–related objects appear in sans serif, as in D or j .

Original: Non–standard sets are in calligraphic capitals, such as R .
Revised: Non–standard sets are in calligraphic capitals, such as R .

Original: Components ofvectors and elements of matrices retain their bold typeface and their case, for example, z i or Θ ij .
Revised: Components of vectors and elements of matrices retain their bold typeface and their case, for example, z i or Θ ij .

Original: Graph terminology.
Revised: Graph terminology.

Original: To ﬁx terminology, a directed graph, or digraph, G = ( V , E ) comprises aset of nodes V and a set of edges E ⊂ V × V .
Revised: To fix terminology, a directed graph, or digraph, G = ( V , E ) comprises a set of nodes V and a set of edges E ⊂ V × V .

Original: An edge e = ( i ,
Revised: An edge e = ( i ,

Original: j ) ∈ E may be represented as eitheri j or j i .
Revised: j ) ∈ E may be represented as either i j or j i .

Original: We do not allow self–loops i i .
Revised: We do not allow self–loops i i .

Original: A directed cycle is an aligned sequence of edges
Revised: A directed cycle is an aligned sequence of edges

Original: C = iii· · · i c −i 0 , for some positive integer, c.
Revised: C = iii· · · i c −i 0 , for some positive integer, c.

Original: A directed acyclic graph (DAG) is a digraph with no directed cycles.
Revised: A directed acyclic graph (DAG) is a digraph with no directed cycles.

Original: A partially–directed graphextends the concept of a digraph to allow undirected edges i j .
Revised: A partially–directed graph extends the concept of a digraph to allow undirected edges i j .

Original: A partially–directed acyclic graph(PDAG) is a partially–directed graph with no directed cycles: cycles including undirected edges areallowed [see 19, p. 82].
Revised: A partially–directed acyclic graph (PDAG) is a partially–directed graph with no directed cycles: cycles including undirected edges are allowed [see 16, p. 82].

Original: A Bayesian network, M =
Revised: A Bayesian network, M =

Original: ( D , U ) consists of a DAG D
Revised: ( D , U ) consists of a DAG D

Original: =
Revised: =

Original: ( V , E ) and a vector of random variablesindexed by the nodes U
Revised: ( V , E ) and a vector of random variables indexed by the nodes U

Original: =
Revised: =

Original: ( U j ) j ∈ V , such that,
Revised: ( U j ) j ∈ V , such that,

Original: [Equation]
Revised: [Equation]

Original: so the probability for U j at a node j only directly depends on the values of U at the j ’s ‘parent’ nodes.
Revised: so the probability for U j at a node j only directly depends on the values of U at the j ’s ‘parent’ nodes.

Original: B Discrete backpropagation
Revised: B Discrete backpropagation

Original: Sampling.
Revised: Sampling.

Original: In the forward pass, via eq.
Revised: In the forward pass, via eq.

Original: (1), DAG-DB generates i.i.d.
Revised: (1), DAG-DB generates i.i.d.

Original: P&M samples Z ( s ) =
Revised: P&M samples Z ( s ) =

Original: MAP( Θ + τ Ψ ( s ) ) , s = 1 , ..., S, of the binary adjacency matrix, where τ is the exponential familydistribution temperature from eq.
Revised: MAP( Θ + τ Ψ ( s ) ) , s = 1 , ..., S, of the binary adjacency matrix, where τ is the exponential family distribution temperature from eq.

Original: For each matrix element ij , the noise samples Ψ ( s ) ij aregenerated by the standard logistic distribution.
Revised: For each matrix element ij , the noise samples Ψ ( s ) ij are generated by the standard logistic distribution.

Original: DAG-DB then uses a method such as STE or I-MLEto backpropagate the gradient of resulting empirical loss function,
Revised: DAG-DB then uses a method such as STE or I-MLE to backpropagate the gradient of resulting empirical loss function,

Original: L = S − 1 (cid:80) Ss =1 (cid:96) Z ( s ) , from thesamples Z ( s ) to Θ .
Revised: L = S − 1 (cid:80) Ss =1 ℓ Z ( s ) , from the samples Z ( s ) to Θ .

Original: Blackbox estimation (BB) [29] can be considered as a special case of I-MLE: with no P&M sampling,it considers a purely deterministic function from Θ to Z = MAP( Θ ) [1].
Revised: Blackbox estimation (BB) [31] can be considered as a special case of I-MLE: with no P&M sampling, it considers a purely deterministic function from Θ to Z = MAP( Θ ) [1].

Original: BB’s inability to delivergood results suggests that the sampling element of DAG-DB plays a useful role.
Revised: BB’s inability to deliver good results suggests that the sampling element of DAG-DB plays a useful role.

Original: Tracking Θcomponents shows that p ( Z ; Θ ) becomes more concentrated around its MAP value as trainingdevelops: an initial, broad distribution helps exploration, and, as training progresses, the distributionconcentrates around the predicted digraph Z .
Revised: Tracking Θ components shows that p ( Z ; Θ ) becomes more concentrated around its MAP value as training develops: an initial, broad distribution helps exploration, and, as training progresses, the distribution concentrates around the predicted digraph Z .

Original: The maximum size constraint show the versatility of P&M. With no maximum size constraint, P&Mlogistic noise sampling is the same as independent Bernoulli sampling for each (non–diagonal)component
Revised: The maximum size constraint shows the versatility of P&M. With no maximum size constraint, P&M logistic noise sampling is the same as independent Bernoulli sampling for each (non–diagonal) component

Original: Z ij [see, for example, 32].
Revised: Z ij [see, for example, 34].

Original: With such a constraint, Bernoulli sampling is not possiblebecause the Z ij components are no longer independent.
Revised: With such a constraint, Bernoulli sampling is not possible because the Z ij components are no longer independent.

Original: Straight–through estimation (STE).
Revised: Straight–through estimation (STE).

Original: The STE approximation for backpropagation is given by
Revised: Our STE approximation for backpropagation is given by

Original: [Equation]
Revised: [Equation]

Original: This is as in the original treatments of STE [2, 3], with the inclusion of a factor to account for thetemperature.
Revised: This is as in the original treatments of STE [28, 29], summed over the samples and with an arbitrary choice of proportionality constants.

Original: Implicit maximum likelihood estimation (I-MLE).
Revised: Implicit maximum likelihood estimation (I-MLE).

Original: I-MLE [1] uses a procedure due to
Revised: I-MLE [1] uses a procedure due to

Original: Domke [30] to set target distribution parameters, which, for DAG-DB, are, for s = 1 , .., S,
Revised: Domke [32] to set target distribution parameters, which, for DAG-DB, are, for s = 1 , .., S,

Original: [Equation]
Revised: [Equation]

Original: where λ arises from the Domke procedure.
Revised: where λ arises from the Domke procedure.

Original: In I-MLE, λ is treated a hyperparameter, with theapproximation of the loss gradient with respect to Θ then being
Revised: In I-MLE, λ is treated a hyperparameter, with the approximation of the loss gradient with respect to Θ then being

Original: [Equation]
Revised: [Equation]

Original: C Details of graphiﬁcation
Revised: C Details of graphification

Original: Each data point is combined with every sample Z
Revised: Each data point is combined with every sample Z

Original: = Z ( s ) by the ‘graphiﬁcation’
Revised: = Z ( s ) by the ‘graphification’

Original: [Equation]
Revised: [Equation]

Original: The resulting matrix is then fed into a function f Φ , where the Φ are learnable parameters.
Revised: The resulting matrix is then fed into a function f Φ , where the Φ are learnable parameters.

Original: In principlef Φ could be a multi–layer neural network, for predictions on non–linear Bayesian networks, but,for simplicity, we conﬁne attention to a linear function with no bias, having Φ ∈ R as real–matrixparameters, and 
Revised: In principle f Φ could be a multi–layer neural network, for predictions on non–linear Bayesian networks, but, for simplicity, we confine attention to a linear function with no bias, having Φ ∈ R as real–matrix parameters, and [Equation] j )

Original: [Equation]
Revised: [Equation]

Original:  for a d × d matrix M .
Revised: 11) for a d × d matrix M .

Original: Given the digraph Z , deﬁne the parents of node j as pa Z ( j ) :
Revised: Given the digraph Z , define the parents of node j as pa Z ( j ) :

Original: = { i : Z ij = 1 } .
Revised: = { i : Z ij = 1 } .

Original: Equations (10) and (11) then together imply a prediction for the value of x at node j from the valuesof x its parent nodes,
Revised: Equations (10) and (11) then together imply a prediction for the value of x at node j from the values of x its parent nodes,

Original: [Equation]
Revised: [Equation]

Original: This ensures that, in prediction, a node j ’s value will only be a function of values associated with itsparent nodes.
Revised: This ensures that, in prediction, a node j ’s value will only be a function of values associated with its parent nodes.

Original: D Details of experimentsD.1 Synthetic DAGs and data
Revised: D Details of experiments D.1 Synthetic DAGs and data

Original: Summary.
Revised: Summary.

Original: Synthetic data is often used for learning hyperparameters and testing as it can beeasily and cheaply generated.
Revised: Synthetic data is often used for learning hyperparameters and testing as it can be easily and cheaply generated.

Original: We follow the approach in the NOTEARS paper [4], also adoptedby GOLEM [10] and many other papers.
Revised: We follow the approach in the NOTEARS paper [2], also adopted by GOLEM [8] and many other papers.

Original: The approach is to generate a random DAG, based on anunderlying distribution for randomly–generating graphs or digraphs, and then to randomly–generatean appropriate linear additive noise model (LANM) on that DAG.
Revised: The approach is to generate a random DAG, based on an underlying distribution for randomly–generating graphs or digraphs, and then to randomly–generate an appropriate linear additive noise model (LANM) on that DAG.

Original: The LANM is then used to createthe synthetic dataset, the n × d matrix X .
Revised: The LANM is then used to create the synthetic dataset, the n × d matrix X .

Original: To generate synthetic DAGs and data, we used open–sourcecode from the GOLEM paper [39], itself derived from NOTEARS [40].
Revised: To generate synthetic DAGs and data, we used open–source code from the GOLEM paper [41], itself derived from NOTEARS [42].

Original: Generating DAGs.
Revised: Generating DAGs.

Original: The NOTEARS and GOLEM papers consider two types of randomly–generated
Revised: The NOTEARS and GOLEM papers consider two types of randomly–generated

Original: DAGs: Erdös–Rényi ER k DAGs, where the number of edges is binomially distributed, and inexpectation, there are dk edges; and Barabási-–Albert SF k DAGs, where the number of edges isﬁxed, at k [ d − ( k + 1) / 2] , with some nodes being preferred in terms of having more edges.
Revised: DAGs: Erdös-Rényi ER k DAGs, where the number of edges is binomially distributed, and in expectation, there are dk edges; and Barabási-Albert SF k DAGs, where the number of edges is fixed, at k [ d − ( k + 1) / 2] , with some nodes being preferred in terms of having more edges.

Original: We useboth these types in our experiments.
Revised: We use both these types in our experiments.

Original: Generating data.
Revised: Generating data.

Original: We use a standard Gaussian equal–variance linear additive noise model.
Revised: We use a standard Gaussian equal–variance linear additive noise model.

Original: Such amodel, with variance σ 2 ∈ R + , is a Bayesian network M =
Revised: Such a model, with variance σ 2 ∈ R + , is a Bayesian network M =

Original: ( D , U ) with values u ∼ U having eachnode–component u j ∈ R , and satisfying
Revised: ( D , U ) with values u ∼ U having each node–component u j ∈ R , and satisfying

Original: [Equation]
Revised: [Equation]

Original: (13) for some weights matrix Φ .
Revised: for some weights matrix Φ .

Original: The NOTEARS/GOLEM implementation, which we use, creates such amodel by sampling Φ ij i.i.d.
Revised: The NOTEARS/GOLEM implementation, which we use, creates such a model by sampling Φ ij i.i.d.

Original: uniformly randomly from [ − 2 , − 0 .
Revised: uniformly randomly from [ − 2 , − 0 .

Original: 5] ∪ [0 . 5 , 2] .
Revised: 5] ∪ [0 . 5 , 2] .

Original: Potential limitations.
Revised: Potential limitations.

Original: There have been criticisms of this approach.
Revised: There have been criticisms of this approach.

Original: In 2008, it was noted that asimilar approach to DAG generation does not generate DAGs uniformly [41, s. 3.3], and this issuealso affects the current DAG–generation method.
Revised: In 2008, it was noted that a similar approach to DAG generation does not generate DAGs uniformly [43, s. 3.3], and this issue also affects the current DAG–generation method.

Original: It has also, more recently, been pointed out thatlinear additive noise models offer clues to prediction methods which may be undesirable.
Revised: It has also, more recently, been pointed out that linear additive noise models offer clues to prediction methods which may be undesirable.

Original: These cluesare tendencies for variances and co–variances between nodes to increase as we follow the DAG’sedges [42].
Revised: These clues are tendencies for variances and co–variances between nodes to increase as we follow the DAG’s edges [44].

Original: Nonetheless these approaches remain the standard synthetic data benchmark and weretain their use, supplemented by our real data experiment, which may mitigate these concerns.
Revised: Nonetheless these approaches remain the standard synthetic data benchmark and we retain their use, supplemented by our real data experiment, which may mitigate these concerns.

Original: D.2 Metrics
Revised: D.2 Metrics

Original: CPDAGs.
Revised: CPDAGs.

Original: Different DAGs may represent the same Bayesian network, being, in general, impossibleto distinguish simply by observation — that is, by i.i.d.
Revised: Different DAGs may represent the same Bayesian network, being, in general, impossible to distinguish simply by observation — that is, by i.i.d.

Original: sampling of the Bayesian network’s randomvariable.
Revised: sampling of the Bayesian network’s random variable.

Original: The Markov Equivalence Class (MEC) of a DAG D is formed by the DAGs D (cid:48) for whichthere exists a Bayesian network which may be represented by both D and D (cid:48) .
Revised: The Markov Equivalence Class (MEC) of a DAG D is formed by the DAGs D ′ for which there exists a Bayesian network which may be represented by both D and D ′ .

Original: A MEC may berepresented by the DAG’s class partially–directed acyclic graph (CPDAG) in which some of the
Revised: A MEC may be represented by the DAG’s class partially–directed acyclic graph (CPDAG) in which some of the

Original: DAG’s edges become undirected.
Revised: DAG’s edges become undirected.

Original: All DAGs in a MEC have the same skeleton: that is, the sameedges, ignoring direction.
Revised: All DAGs in a MEC have the same skeleton: that is, the same edges, ignoring direction.

Original: Any MEC can be represented by a CPDAG: undirected edges indicate thatthe direction may differ between DAGs within the MEC.
Revised: Any MEC can be represented by a CPDAG: undirected edges indicate that the direction may differ between DAGs within the MEC.

Original: Identiﬁability For some particular Bayesian networks, however, the exact DAG can be identiﬁed.
Revised: Identifiability For some particular Bayesian networks, however, the exact DAG can be identified.

Original: Gaussian equal–variance linear noise models have this property.
Revised: Gaussian equal–variance linear noise models have this property.

Original: For real data, such as the Sachsdata we experiment on, there is no such guarantee.
Revised: For real data, such as the Sachs data we experiment on, there is no such guarantee.

Original: For consistency, we will therefore use metricsbased on CPDAGs.
Revised: For consistency, we will therefore use metrics based on CPDAGs.

Original: More discussion of identiﬁability can be found in Peters, Janzing & Schölkopf’stextbook [19, sec.
Revised: More discussion of identifiability can be found in Peters, Janzing & Schölkopf’s textbook [16, sec.

Original: Structural Hamming distance (SHD).
Revised: Structural Hamming distance (SHD).

Original: Suppose now that we have a true DAG D true =
Revised: Suppose now that we have a true DAG D true =

Original: ( V , E true )and a predicted DAG D pred =
Revised: ( V , E true ) and a predicted DAG D pred =

Original: ( V , E pred ) , on the same set of nodes V , but with potentially differingedge sets E true and E pred .
Revised: ( V , E pred ) , on the same set of nodes V , but with potentially differing edge sets E true and E pred .

Original: A given unordered pair of nodes { i , j } ’s join status will be one of thefollowing: unjoined, joined i j , or joined i j .
Revised: A given unordered pair of nodes { i , j } ’s join status will be one of the following: unjoined, joined i j , or joined i j .

Original: If we also allow undirected edges, as in CPDAGs,then this adds a fourth possibility i j .
Revised: If we also allow undirected edges, as in CPDAGs, then this adds a fourth possibility i j .

Original: SHD is the count of the ordered pairs i , j which differ in terms of join status between D true and D pred .
Revised: SHD is the count of the ordered pairs i , j which differ in terms of join status between D true and D pred .

Original: The class structural Hamming distance SHD c is thecorresponding metric for the CPDAGs associated with D true and D pred .
Revised: The class structural Hamming distance SHD c is the corresponding metric for the CPDAGs associated with D true and D pred .

Original: Dividing by the number ofnodes gives the normalized class structural Hamming distance nSHD c .
Revised: Dividing by the number of nodes gives the normalized class structural Hamming distance nSHD c .

Original: (n)SHD is found in, forexample, the NOTEARS [4] and GOLEM [10] papers, and (n)SHD c in the GOLEM paper.
Revised: (n)SHD is found in, for example, the NOTEARS [2] and GOLEM [8] papers, and (n)SHD c in the GOLEM paper.

Original: Note thatif we predicted an empty DAG D pred , with no edges, we would have nSHD c = | E true | / | V | , because a
Revised: Note that if we predicted an empty DAG D pred , with no edges, we would have nSHD c = | E true | / | V | , because a

Original: CPDAG has the same skeleton, and so the same number of edges, as any DAG it represents.
Revised: CPDAG has the same skeleton, and so the same number of edges, as any DAG it represents.

Original: Precision and recall.
Revised: Precision and recall.

Original: In the current context, precision is deﬁned as
Revised: In the current context, precision is defined as

Original: [Equation]
Revised: [Equation]

Original: [Equation] treating the edges i j and i j as distinct.
Revised: (14) treating the edges i j and i j as distinct.

Original: Similarly, recall is deﬁned as
Revised: Similarly, recall is defined as

Original: [Equation]
Revised: [Equation]

Original: In graph literature [e.g. 4, 10], precision may be replaced by the false discovery rate, which is oneminus the precision, and recall may be termed the true positive rate.
Revised: In graph literature [e.g. 2, 8], precision may be replaced by the false discovery rate, which is one minus the precision, and recall may be termed the true positive rate.

Original: As for SHD, we may also usethe CPDAGs associated with D true and D pred to deﬁne the class precision prec c and class recall rec c .
Revised: As for SHD, we may also use the CPDAGs associated with D true and D pred to define the class precision prec c and class recall rec c .

Original: In this extension to CPDAGs, we regard i j as a further distinct type of edge between i and j .
Revised: In this extension to CPDAGs, we regard i j as a further distinct type of edge between i and j .

Original: D.3 Hyperparameters
Revised: D.3 Hyperparameters

Original: Hyperparameter settings.
Revised: Hyperparameter settings.

Original: To choose hyperparameters, we sought to optimise on a randomly–generated set of six synthetic Erdös–Rényi ‘ER2’ DAGs with d = 30 nodes and a mean of 60edges.
Revised: To choose hyperparameters, we sought to optimise on a randomly– generated set of six synthetic Erdös-Rényi ‘ER2’ DAGs with d = 30 nodes and a mean ofedges.

Original: On each of these graphs, a synthetic dataset was created using a Gaussian equal–variancelinear additive noise model.
Revised: On each of these graphs, a synthetic dataset was created using a Gaussian equal–variance linear additive noise model.

Original: We assessed the viability of I-MLE, STE, SFE and BB as the discretebackpropagation method, ﬁnding I-MLE and STE to be the most promising.
Revised: We assessed the viability of I-MLE, STE, SFE and BB as the discrete backpropagation method, finding I-MLE and STE to be the most promising.

Original: In exploring hyperparameter settings, we used a combination of Optuna [43, 44], a well–known package for Bayesianoptimisation of hyperparameters, and manual adjustment, for example taking an Optuna–generatedset of hyperparameters and varying the standard probability distribution used as noise for P&M.
Revised: In exploring hyperparameter settings, we used a combination of Optuna [45, 46], a well–known package for Bayesian [Table] optimisation of hyperparameters, and manual adjustment, for example taking an Optuna–generated set of hyperparameters and varying the standard probability distribution used as noise for P&M.

Original: Wealso performed a grid search over hyperparameters, which was less successful in identifying optimalsettings.
Revised: We also performed a grid search over hyperparameters, which was less successful in identifying optimal settings.

Original: We identiﬁed two settings as particularly promising, STE_84 , using STE with a maximumsize constraint of 84 edges, and IMLE_None , using I-MLE with no maximum size constraint.
Revised: We identified two settings as particularly promising, STE_84 , using STE with a maximum size constraint of 84 edges [Table] and IMLE_None , using I-MLE with no maximum size constraint.

Original: We chose synthetic ER2 DAGs with 30 nodes, as lying roughly in the middle of the range of typesand node–numbers considered here, and in the NOTEARS and GOLEM papers.
Revised: We chose synthetic ER2 DAGs with 30 nodes, as lying roughly in the middle of the range of types and node–numbers considered here, and in the NOTEARS and GOLEM papers.

Original: Note that, followingappendix D.2, a DAG with no edges would score an expected nSHD c = 2 when compared witha ‘true’ ER2 DAG, suggesting a cut–off maximum nSHD c .
Revised: Note that, following appendix D.2, a DAG with no edges would score an expected nSHD c = 2 when compared with a ‘true’ ER2 DAG, suggesting a cut–off maximum nSHD c .

Original: With BB, we did not ﬁnd any settingswhich delivered nSHD c < 2 ; whilst for SFE, we only managed nSHD c slightly less than 2.
Revised: With BB, we did not find any settings which delivered nSHD c < 2 ; whilst for SFE, we only managed nSHD c slightly less than 2.

Original: Table 2shows the best hyperparameters for DAG-DB we found for STE_84 and IMLE_None .
Revised: Tableshows the best hyperparameters for DAG-DB we found for STE_84 and IMLE_None .

Original: D.4 Ablation of regularizers
Revised: D.4 Ablation of regularizers

Original: We can see from tables 3 and 4 that, for both STE_84 and IMLE_None settings, all ablations worsen themetrics, although, for IMLE_None , the effect of ablating r sp is very minor, as might be expected fromtable 2 which shows that its coefﬁcient ρ sp is rather small. 
Revised: We can see from tables 3 and 4 that, for both STE_84 and IMLE_None settings, all ablations worsen the metrics, although, for IMLE_None , the effect of ablating r sp is very minor, as might be expected from table 2 which shows that its coefficient ρ sp is rather small. [Table] [Table]

Original: D.5 Variation by DAG
Revised: D.5 Variation by DAG

Original: The standard box–and–whisker plots of ﬁg. 2 indicate variation over 24 test DAGs.
Revised: The standard box–and–whisker plots of fig. 2 indicate variation over 24 test DAGs.

Original: Each box coversfrom the ﬁrst to the third quartile, with an interior line indicating the median.
Revised: Each box covers from the first to the third quartile, with an interior line indicating the median.

Original: A white dot indicatesthe mean.
Revised: A white dot indicates the mean.

Original: The whiskers each extend to cover all points within a further 1.5 times the inter–quartilerange, whilst any points outside that extended range are marked individually.
Revised: The whiskers each extend to cover all points within a further 1.5 times the inter–quartile range, whilst any points outside that extended range are marked individually.

Original: We also considered whether 24 DAGs was enough to give a reasonable sample, experimentingvia a test set of 240 ER2 DAGs with 30 nodes.
Revised: We also considered whether 24 DAGs was enough to give a reasonable sample, experimenting via a test set of 240 ER2 DAGs with 30 nodes.

Original: We found that taking a 100 000 random 24 DAGsub–samples of the 240 gave standard errors for nSHD c of 0.1 for STE_84 and 0.15 for IMLE_None ,and of 0.03 for prec c with either method.
Revised: We found that taking a 100 000 random 24 DAG sub–samples of the 240 gave standard errors for nSHD c of 0.1 for STE_84 and 0.15 for IMLE_None , and of 0.03 for prec c with either method.

Original: We then examined how much results could vary in repeatedprediction on the same 24 DAGs.
Revised: We then examined how much results could vary in repeated prediction on the same 24 DAGs.

Original: We made predictions ten times for a constant set of 24 DAGs, [Table] ﬁnding variation from minimum to maximum for nSHD c of 0.03 for STE_84 and 0.05 for IMLE_None ,and of 0.01 for prec c with either method.
Revised: We made predictions ten times for a constant set of 24 DAGs, finding variation from minimum to maximum for nSHD c of 0.03 for STE_84 and 0.05 for IMLE_None , and of 0.01 for prec c with either method.

Original: These results suggest that 24 DAGs may be a sufﬁcientsample size for testing our methods.
Revised: These results suggest that 24 DAGs may be a sufficient sample size for testing our methods.

Original: D.6 Methods used for comparison
Revised: D.6 Methods used for comparison

Original: We follow the methods used for comparison in the GOLEM paper [10], with one exception.
Revised: We follow the methods used for comparison in the GOLEM paper [8], with one exception.

Original: Thecombinatoric methods we use are FGES [24] (formerly known as FGS [45]) and PC [22].
Revised: The combinatoric methods we use are FGES [21] (formerly known as FGS [47]) and PC [19].

Original: The
Revised: The

Original: GOLEM paper used the Conservative PC algorithm [46] as its version of PC, however, this producesresults in a form more general than a CPDAG, complicating comparisons.
Revised: GOLEM paper used the Conservative PC algorithm [48] as its version of PC, however, this produces results in a form more general than a CPDAG, complicating comparisons.

Original: We therefore use the PC-Stable [23] variant, which, like the original PC algorithm, returns a CPDAG.
Revised: We therefore use the PCStable [20] variant, which, like the original PC algorithm, returns a CPDAG.

Original: The continuousmethods we compare with are NOTEARS [4] and GOLEM [10].
Revised: The continuous methods we compare with are NOTEARS [2] and GOLEM [8].

Original: For NOTEARS, as was donein the GOLEM paper, we employ the NOTEARS-L1 variant with the settings given as defaultsfor linear SEMs in the NOTEARS code repository.
Revised: For NOTEARS, as was done in the GOLEM paper, we employ the NOTEARS-L1 variant with the settings given as defaults for linear SEMs in the NOTEARS code repository.

Original: GOLEM provides two variants: GOLEM-EV,suitable for data generated with equal variance, and GOLEM-NV, suitable for non–equal variancedata.
Revised: GOLEM provides two variants: GOLEM-EV, suitable for data generated with equal variance, and GOLEM-NV, suitable for non–equal variance data.

Original: Accordingly, we use GOLEM-EV for our synthetic experiments and GOLEM-NV for our realdata experiment.
Revised: Accordingly, we use GOLEM-EV for our synthetic experiments and GOLEM-NV for our real data experiment.

Original: To run experiments for these methods, we used py-causal [47] for FGES and PC, and the
Revised: To run experiments for these methods, we used py-causal [49] for FGES and PC, and the

Original: NOTEARS [40] and GOLEM [39] repos.
Revised: NOTEARS [42] and GOLEM [41] repos.

Original: Licences for these packages, and for the code we adaptedfor GFAS, are shown in table 5.[Table] [Table] [Table]
Revised: Licences for these packages, and for the code we adapted for GFAS, are shown in table 5.

Original: D.7 Supplementary results on Sachs experiments
Revised: D.7 Supplementary results on Sachs experiments

Original: See table 6 for results from I-MLE, with training using digraphs, IMLE_None , and with training using
Revised: See table 6 for results from I-MLE, with training using digraphs, IMLE_None , and with training using

Original: DAGs, IMLE_None_Tr .
Revised: DAGs, IMLE_None_Tr .

Original: Table 7 shows standard deviations, over 24 runs, for these methods. 
Revised: Table 7 shows standard deviations, over 24 runs, for these methods. The [Table] [Table]

Revised: [Table]

Revised: confirm

Original: Thevaried hyperparameter, λ, is the Domke hyperparameter discussed in appendix B. We varied this as,in I-MLE, results can be particularly sensitive to λ [1].
Revised: [Table] deviation [Table] varied hyperparameter, λ, is the Domke hyperparameter discussed in appendix B. We varied this as, in I-MLE, results can be particularly sensitive to λ [1].

Original: Table 7’s λ = 27 .
Revised: Table 7’s λ = 27 .

Original: 14 rows for IMLE_None arethe standard deviations corresponding to table 1.
Revised: 14 rows for IMLE_None are the standard deviations corresponding to table 1.

Original: GOLEM and NOTEARS results have very smallstandard deviations, whilst FGES and PC are deterministic.
Revised: GOLEM and NOTEARS results have very small standard deviations, whilst FGES and PC are deterministic.

Original: As noted, DAG training is time consuming.
Revised: As noted, DAG training is time consuming.

Original: Running time for each single prediction increases fromabout 0h15m with IMLE_None to around 1h15m with IMLE_None_Tr .
Revised: Running time for each single prediction increases from about 0h15m with IMLE_None to around 1h15m with IMLE_None_Tr .

Original: D.8 Computational resources
Revised: D.8 Computational resources

Original: Computations were done on a combination of a laptop with a GeForce RTX 3080 Mobile 16GB GPUand a computer cluster.
Revised: Computations were done on a combination of a laptop with a GeForce RTX 3080 Mobile 16GB GPU and a computer cluster.

Original: Prediction for a single ER2 graph on 30 nodes (roughly the mid–range of thegraphs shown in ﬁg. 2) took around 0h12 on the laptop, for the longest–running of all the methods inﬁg.
Revised: Prediction for a single ER2 graph on 30 nodes (roughly the mid–range of the graphs shown in fig. 2) took around 0h12 on the laptop, for the longest–running of all the methods in fig.

Original: 2, IMLE_None , giving around 72 hours to compile that element of the ﬁgure.
Revised: 2, IMLE_None , giving around 72 hours to compile that element of the figure.

Original: The corresponding
Revised: The corresponding

Original: STE_84 computation took around a quarter of that time, and the other methods we compared with werequicker, giving a total time to compute ﬁg. 2 of the order of 100 hours.
Revised: STE_84 computation took around a quarter of that time, and the other methods we compared with were quicker, giving a total time to compute fig. 2 of the order of 100 hours.

Original: Hyperparameter exploration,ablation experiments and Sachs experiments are estimated to add another 200 hours compute, whilethe experiments on variation described in appendix D.5 took around 120 hours.[Table]
Revised: Hyperparameter exploration, ablation experiments and Sachs experiments are estimated to add another 200 hours compute, while the experiments on variation described in appendix D.5 took around 120 hours.

Original: [Table]