Reliable Causal Discovery with Improved Exact Search and Weaker Assumptions

1 Introduction . Although it is often more reliable to discover causal relationships by making use of interventions or randomized experiments , they are practically challenging , expensive , or even prohibited owing to ethical considerations . Thus , causal discovery from observational data has received considerable attention in recent decades , and has been widely applied in different fields such as genetics [ 29 ] . One major class of causal discovery methods is the constraint-based methods , such as PC [ 40 ] and FCI [ 43 , 4 ] , that leverage conditional independence tests to estimate the skeleton and then perform edge orientation . These methods are guaranteed to asymptotically return the true Markov equivalence class ( MEC ) under the Markov and faithfulness assumptions . Several modifications [ 30 , 41 ] to these constraint-based methods have been developed to allow certain types of unfaithfulness , which , however , generally give rise to weaker claims and are not guaranteed to estimate the true MEC . Another popular approach is the GES [ 3 ] algorithm that searches in the space of MECs greedily by maximizing a well-defined score , such as the Bayesian information criterion ( BIC ) [ 36 ] score . It starts with an empty structure and consists of two phases : ( 1 ) adding edges until a local maximum is found , and ( 2 ) removing edges until a local maximum is reached . In spite of the greedy strategy , GES converges in the large sample limit to the true MEC under the Markov and faithfulness assumptions , similar to the aforementioned constraint-based methods . Recently , NOTEARS [ 52 ] casts the Bayesian network structure learning task into a continuous constrained optimization problem with the least squares objective , using an algebraic characterization of directed acyclic graph ( DAG ) . Subsequent work GOLEM [ 23 ] adopts a continuous unconstrained optimization formulation with a likelihood-based objective . For NOTEARS , it remains unclear 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . of the required assumptions for asymptotic correctness , whereas GOLEM adopts the generalized faithfulness assumption [ 9 ] to learn linear Gaussian DAGs , which could be converted to their MECs for causal interpretation [ 42 ] . These methods enable the application of numerical solvers and GPU acceleration , which thus are scalable to large graphs . However , they are only guaranteed to find a local optimum of the optimization problem , and therefore the quality of the solution in practice may not be guaranteed , even in the asymptotic case . Another line of research focuses on weakening the faithfulness assumption required for asymptotic correctness of the search results , since , given finite samples , approximate violations of faithfulness occur surprisingly often , especially when there is a large number of variables [ 46 ] . For instance , exact search methods find the optimal Bayesian network based on a predefined score function , such as dynamic programming ( DP ) [ 18 , 24 , 39 , 38 ] , A * [ 49 , 48 ] , and integer programming [ 1 , 5 ] . The DAGs estimated by these methods can be converted to their MECs for causal interpretation [ 42 ] . Note that the approaches based on sparsest permutation ( SP ) [ 32 ] and Boolean satisfiability solver ( SAT ) [ 15 , 16 ] can be viewed as instances of exact methods . Lu et al . [ 22 ] further demonstrated that these exact methods may produce correct results in cases where methods relying on faithfulness fail . Due to the large search space of possible DAGs [ 2 , 13 ] , exact search methods are feasible only for small structures . Therefore , super-structure has been adopted to constrain the search space [ 27 , 45 ] , which is defined to be an undirected graph that restricts the search to candidate DAGs whose skeleton is its subgraph . However , most of these methods rely on discovering the skeleton of the true DAG for use as a super-structure , utilizing estimation methods like MMPC [ 45 ] , which require the faithfulness assumption to be asymptotically correct . Under approximate violations of faithfulness , these skeleton estimation methods may miss some edges owing to unfaithful conditional independencies in the data distribution ; thus , further exact search procedures are guaranteed to miss those edges . Contributions . In this work , we introduce several strategies to improve the scalability of exact search in the linear Gaussian setting , giving rise to a more reliable causal discovery procedure . Our main contributions can be summarized as follows : • We develop a super-structure estimation method based on the support of inverse covariance matrix of the data distribution , and show that it is asymptotically correct under assumptions strictly weaker than faithfulness ( or , more specifically , than triangle-faithfulness ) . We combine this with exact search method like DP or A * to reduce search space . • To further scale up exact search , we develop a local search strategy , called Local A * , on the local clusters formed by each variable and its neighbors within two hops in the super-structure . • We demonstrate the efficacy of our super-structure estimation method and local search strategy by conducting extensive experiments , and show that it scales up to hundreds of nodes with a high accuracy . Paper organization . We review the common assumptions for causal discovery and the linear structural equation model ( SEM ) in Section 2 . In Section 3 , we establish weaker variants of faithfulness and show how they could be used to learn a sound super-structure . We further formulate an improved exact search strategy in Section 4 . The empirical studies in Section 5 validate our theoretical results and the efficacy of the proposed procedure . We then conclude our work in Section 6 . 2 Background . We first review the concepts of causal Bayesian networks and some commonly used assumptions that are related to our further analysis . We then give a brief overview of the linear SEM . 2.1 Causal Bayesian Network and Common Assumptions . Let G = ( V , E ) be a DAG with vertex set V = { X1 , . . . , Xd } in which each node Xi corresponds to a random variable . Denote X = ( X1 , . . . , Xd ) as the random vector concatenating all variables and its associated probability distribution P. Let Xpa ( i ) be the set of parental nodes of Xi in G such that there is a directed edge from Xj 2 Xpa ( i ) to Xi , or Xj ! Xi 2 E. We assume causal sufficiency , i.e. , no hidden variables , throughout the paper . In a Bayesian network , the distribution P is assumed to be Markov w.r.t . to DAG G , as defined below . Assumption 1 ( Markov ) . Given a DAG G and distribution P over the variable set V , every variable X in V is probabilistically independent of its non-descendants given its parents in G .. A causal Bayesian network can be viewed as a Bayesian network where the directed edges are provided a causal meaning , which thereby allows it to answer interventional queries [ 20 ] . In general , there are many DAGs that induce the same conditional independence ( CI ) relations with the distribution P , and are said to be Markov equivalent . The Markov equivalence class ( MEC ) consists of all DAGs that entail the same conditional independence ( CI ) relations as G does , and is uniquely determined by its skeleton and v-structures [ 25 ] . V-structure is defined to be a collider X ! Y Z where X and Z are not adjacent in G , therefore referred to as an unshielded collider . If X and Z are adjacent , then it is called a shielded collider . The following faithfulness assumption is commonly used to relate the CI relations in the distribution to the DAG , and can be thought of as the converse to the Markov assumption . Assumption 2 ( Faithfulness [ 44 ] ) . Given a DAG G and distribution P over the variable set V , P implies no CI relations not already entailed by the Markov assumption . Under the Markov and faithfulness assumptions , constraint-based methods such as PC have been shown to asymptotically output the correct MEC . However , in the finite sample regime , the faithfulness assumption is sensitive to statistical testing errors when inferring the CI relations , and its approximate violations occur surprisingly often , especially when there is a large number of variables [ 46 ] . Thus , different relaxations of faithfulness have been proposed , such as orientation-faithfulness , adjacencyfaithfulness [ 30 ] , and triangle-faithfulness [ 51 ] , which we review in Appendix A . Clearly , the triangle-faithfulness assumption is a consequence of adjacency-faithfulness . Based on these weaker assumptions , different modifications to constraint-based methods have been proposed , such as Conservative PC [ 30 ] and Very Conservative SGS [ 41 ] . As their names suggest , these methods make weaker claims about the estimated causal structure and therefore are not guaranteed to estimate the true MEC . On the other hand , Raskutti and Uhler [ 32 ] proposed the following assumption and show that it is strictly weaker than faithfulness . Assumption 3 ( Sparsest Markov representation ( SMR ) [ 32 ] ) . Given a DAG G and distribution P over the variable set V , the MEC of G is the unique sparsest MEC that satisfies the Markov assumption with P. Forster et al . [ 7 ] referred to the above assumption as the ( unique ) frugality assumption , and argued that it has multiple desirable properties compared to faithfulness . Under the SMR assumption , the SP method [ 32 ] has been shown to produce asymptotically correct results . Raskutti and Uhler [ 32 ] also conjectured that SP reaches the information-theoretic limit in the sense that the SMR assumption may be the weakest assumption guaranteeing the asymptotic correctness of any method for learning the true MEC . It is worth noting that SP can be viewed as an instance of exact score-based method . The study by Lu et al . [ 22 ] demonstrated that causal discovery methods that rely on the faithfulness assumption ( e.g. , PC , GES ) may output sub-optimal solutions in various cases , whereas exact methods ( e.g. , SP , A * , SAT ) are able to produce the correct results . Therefore , in this work , we aim at improving the scalability of exact score-based methods for reliable causal discovery , by relying on the SMR assumption and relaxing the faithfulness assumption as much as is viable . 2.2 Linear Structural Equation Model . Given a linear SEM , each random variable follows the relationship Xi = BTi X+Ni , where Bi is a coefficient vector and Ni is an exogenous noise variable associated with variable Xi . In this work we focus on the linear Gaussian model where the variables Ni ’ s follow the Gaussian distribution . The linear SEM can be written in matrix form as X = BTX +N , where B = [ B1 , B2 , · · · , Bd ] corresponds to a weighted adjacency matrix , and N = ( N1 , . . . , Nd ) is a noise vector characterized by the covariance matrix ⌦ = cov [ N ] = diag ( 21 , . . . , 2d ) . We assume that 2 i > 0 for i = 1 , . . . , d so that the distribution P has positive measure everywhere . As a standard assumption , we also assume structural minimality [ 29 ] which implies that the nonzero coefficients in B define the structure of G , i.e. , Xj ! Xi 2 E if and only if the coefficient in Bi corresponding to variable Xj is nonzero . Since one can always center the data , we assume E [ X ] = E [ N ] = 0 without loss of generality . The inverse covariance matrix ⇥ = ⌃ 1 of X is given by ⇥ = ( I B ) ⌦ 1 ( I B ) T. Note that ⇥ij = 0 if and only if Xi ? Xj |V \ { Xi , Xj } in the linear Gaussian case .

Summary: This paper proposes two steps for learning the DAG structure from linear Gaussian data. First, it is shown that under a condition weaker than several forms of faithfulness, the inverse of the observational covariance matrix can be used to find a graph guaranteed to contain all edges in the DAG. Second, another theoretical result is established that shows that an exact structure learning algorithm will find the correct edges and v-structures around X even when it sees only a particular neighbourhood around X. An algorithm is proposed that leverages both results.