Iterative Causal Discovery in the Possible Presence of Latent Confounders and Selection Bias

1 Introduction . Causality plays an important role in many sciences , such as social sciences , epidemiology , and finance ( Pearl , 2010 ; Spirtes , 2010 ) . Understanding the underlying mechanisms is crucial for tasks such as explaining a phenomenon , predicting , and decision making . Pearl ( 2009 ) provided a machinery for automating the process of answering interventional and ( retrospective ) counterfactual queries even when only observed data is available , and determining if a query can not be answered given the available data type ( identifiability ) . This requires knowledge about the true underlying causal structure ; however , in many real-world situations , this structure is unknown . There is a large body of literature on recovering causal relations from observed data—causal discovery ( Spirtes et al. , 2000 ; Peters et al. , 2017 ; Cooper & Herskovits , 1992 ; Chickering , 2002 ; Shimizu et al. , 2006 ; Hoyer et al. , 2009 ; Rohekar et al. , 2018 ; Yehezkel & Lerner , 2009 ; Nisimov et al. , 2021 ) , differing in the assumptions upon which they rely . In this work we assume a directed acyclic graph ( DAG ) for the underlying causal structure and focus on learning it from observational data . Furthermore , we assume the causal Markov and faithfulness assumptions , and consider recovering the structure by performing a series of conditional independence ( CI ) tests ( Spirtes et al. , 2000 ) . In this setting the true DAG is statistically indistinguishable from many other DAGs . Moreover , when considering the possible presence of latent confounders and selection bias ( no causal sufficiency ) , the true DAG can not be recovered . Instead , Richardson & Spirtes ( 2002 ) proposed the maximal ancestral graph ( MAG ) , which represents independence relations among observed variables , and the partial ancestral graph ( PAG ) , which is a Markov equivalence class of MAGs—a set of MAGs that can not be ruled out given the observed independence relations . Recently , causal identification was demonstrated for PAG models ( Jaber et al. , 2018 , 2019 ) , which is a more practical use of these models . That is , by using only observed data and no prior knowledge on the underlying causal relations , some identification and causal queries can be answered . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . In this paper , we address the problem of learning a PAG such that interrupting the learning process results in a correct PAG . That is , all the entailed independence and causal relations are correct , although it can be less informative . This anytime property is important in many real-world settings where it is desired to recover as many causal relations as possible under limited compute power . 2 Related Work . Causal discovery in the potential presence of latent confounders and selection bias requires placing additional assumptions . In this paper we assume the causal Markov assumption ( Pearl , 2009 ) , faithfulness ( Spirtes et al. , 2000 ) , and a DAG structure for the underlying causal relations . In this setting , several causal discovery algorithms have been proposed , FCI ( Spirtes et al. , 2000 ) , RFCI ( Colombo et al. , 2012 ) , FCI+ ( Claassen et al. , 2013 ) , and GFCI ( Ogarrio et al. , 2016 ) . Limitations of FCI have been reported previously where it tends to erroneously exclude many edges that are in the true underlying graph , and it requires many CI tests with large conditioning sets . The GFCI algorithm employs a greedy score-based approach to improve the accuracy for small data sizes ( small sample ) . However , it requires additional assumptions for justifying the score function that it uses . The RFCI algorithm alleviates computational complexity by avoiding the last stage of FCI . This stage requires many CI tests having large conditioning sets . Although it is sound ( outputs correct causal information ) , it is not complete ( some MAGs in the equivalence class can be ruled out given the data ) . Similarily to the FCI and FCI+ algorithms , we consider a procedure that is sound and complete in the large sample limit ( or when a perfect conditional independence oracle is used ) . However , these algorithms , for a MAG , treat nodes that are m-separated by adjacent nodes differently from nodes that are m-separated by a minimal separating set that includes nodes outside the neighborhood . In contrast , we treat all possible separating sets in a similar manner . We employ an iterative procedure that gradually increases the search radius on the graph for identifying minimal separating sets . This allows our method to be interrupted at any iteration , returning a correct graph , similarly to the anytime FCI algorithm ( Spirtes , 2001 ) . 3 Anytime Iterative Discovery of Causal Relations . First , we provide definitions and assumptions . Then , we describe the iterative causal discovery ( ICD ) algorithm and prove its correctness . We conclude by discussing efficiency and stability . 3.1 Preliminaries . A causal DAG , D , over nodes V = O∪S∪L is denoted byD ( O , S , L ) , where O , S , and L represent disjoint sets of observed , selection , and latent variables , respectively . We use an ancestral graph ( Richardson & Spirtes , 2002 ) to model the conditional independence relations among the observed variables O in the causal DAG D. This class of graphical models is useful since for every causal DAG there exits a unique MAG . In this setting , our method is aimed at recovering the MAG of the true underlying DAG , from the result of CI tests . A CI test is commonly a statistical hypothesis test used to determine from observed data whether two variables are statistically dependent conditioned on a set of some other variables ( a conditioning set ) . If independence is found , the conditioning set is called a separating set for the tested nodes . However , in this setting the MAG can not be fully recovered , and only a Markov equivalence class , represented by a PAG , can be recovered . In a PAG , a variant edge-mark is denoted by an empty circle ‘ —o X ’ . Namely , in the equivalence class there exists at least one MAG that has a tail edge-mark ’ —–X ’ and at least one MAG that has an arrowhead ’ — > X ’ at the same location . Definition 1 ( O-equivalence ( Spirtes et al. , 2000 ) ) . Two DAGs , Di ( O , Si , Li ) and Dj ( O , Sj , Lj ) are said to be O-equivalent if and only if X⊥ Y| ( Z ∪ Si ) in Di ⇐⇒ X⊥ Y| ( Z ∪ Sj ) in Dj , for every possible disjoint subsets X , Y , and Z of O . That is , the observed d-separation relations in two O-equivalent DAGs , Di and Dj , are identical . Spirtes ( 2001 ) defines equivalence considering an n-oracle for testing conditional independence . It returns “ dependence ” if the conditioning set size is larger than n , otherwise d-separation is tested and returned . Thus , n-O-equivalence class is defined by using the n-oracle instead of d-separation in Definition 1 . The PAG that represents this equivalence class is called n-representing ( Spirtes , 2001 ) . In a MAG , node X is in D-Sep ( A , B ) ( note the capitalization of ‘ D ’ ) if and only if X 6= A and there is an undirected path between A and X such that every node on the path , except the endpoints , is a collider and is an ancestor of A or B ( Spirtes et al. , 2000 ) . The FCI algorithm utilizes a super-set of D-Sep , called Possible-D-Sep ( A , B ) ( Spirtes et al. , 2000 ) for testing CI . It is defined for a skeleton learned by the PC algorithm1 ( Spirtes et al. , 2000 ) and its identified v-structures , referred in this paper as the first stage of FCI . This super-set , used by FCI in its second stage , includes all the nodes connected by a path to A , where every node on this path , except the end-points , is a collider or part of a triangle , hiding its orientation . We consider a smaller super-set of D-Sep for PAGs and take special interest in the path that connects any Z ∈ Possible-D-Sep ( A , B ) to A . Definition 2 ( PDS-path ) . A possible-D-Sep-path ( PDS-path ) from A to Z , with respect to B , in PAG G , denoted ΠB ( A , Z ) , is a path 〈A , . . . , Z〉 such that B is not on the path and for every sub-path 〈U , V , W 〉 of ΠB ( A , Z ) , V is a collider or { U , V , W } forms a triangle . Note that an edge in a PAG is a PDS-path . Following the definition of PDS-path , we define a tree rooted at a given node consisting of all PDS-paths from that node . Definition 3 ( PDS-tree ) . A possible-D-Sep-tree ( PDS-tree ) for A , with respect to B given PAG G , is a tree rooted at A , such that there is a path from A to Z , 〈A , V1 , . . . , Vk , Z〉 , if and only if there is a PDS-path 〈A , V1 , . . . , Vk , Z〉 , with respect to B , in G. Lastly , considering the last condition of D-Sep definition , we define a possible ancestor of a node in a given PAG . Definition 4 ( Possible Ancestor ) . In a PAG , Vi is a possible ancestor of Vi+k if there is a path 〈Vi , Vi+1 , . . . , Vi+k〉 such that , ∀j ∈ { i , . . . , i+k−1 } , on the edge ( Vj , Vj+1 ) there is no arrowhead at Vj and no tail edge-mark at Vj+1 . 3.2 Iterative Causal Discovery Algorithm . We present a single stage that is called iteratively , for recovering the underlying equivalence class , represented by a PAG , for the true underlying DAG D ( O , S , L ) . Each iteration is parameterized by r , where r ∈ { 0 , . . . , |O| − 2 } . Given a PAG returned by the previous r − 1 iteration , each pair of connected nodes A and B is tested for independence conditioned on a set Z ⊂ O . If independence is found , the connecting edge is removed . The conditioning set Z should comply with the following conditions for ( A , B ) or ( B , A ) , which we call ICD-Sep conditions . ICD-Sep conditions for the ordered pair ( A , B ) are , 1 . |Z| = r , 2 . ∀Z ∈ Z , there exists a PDS-path ΠB ( A , Z ) such that , ( a ) |ΠB ( A , Z ) | ≤ r and ( b ) every node on ΠB ( A , Z ) is in Z , 3 . ∀Z ∈ Z , node Z is a possible ancestor of A or B ( not a necessary condition ) . 1PC is a causal discovery algorithm assuming causal sufficiency ( absence of hidden confounders and selection bias ) . Conditioning sets consist of nodes only from the neighborhood of the tested nodes . Condition 1 restricts the conditioning set size , and condition 2 restricts its distance from the tested nodes . Tying these two conditions together is the key idea of the presented ICD algorithm . Condition 3 follows the last condition in the definition of D-Sep . It is not a necessary condition for the correctness of the proposed method but can reduce the number of considered conditioning sets when testing independence . Sets complying with ICD-Sep conditions are denoted ICD-Sep . In comparison to Possible-D-Sep that is utilized by the FCI algorithm , ICD-Sep is a smaller super-set of { S | S ⊂ D-Sep , |S| = r } . Nevertheless , ICD-Sep and Possible-D-Sep may be similar when considering densely connected graphs or when the recoverable equivalence class is large ( many variant edge-marks ) , such as a complete graph . However , these cases are usually rare in real-world scenarios . Next , consider a PDS-tree for A with respect to B in G. Sets complying with ICD-sep conditions can be created by traversing this tree . Condition 1 restricts the number of nodes to be included in Z to the specific value of r. Condition 2 places constraints on the nodes in Z as follows . Condition 2a limits the radius around A in which the nodes of Z reside , effectively limiting the depth of the PDS-tree . Condition 2b ensures that if a node is in Z , then every node on the PDS-path connecting it to A is also in Z . An example is given in Figure 1 , where ( a ) is the true underlying DAG and ( b ) is the corresponding PAG that represents a 2-O-equivalence class . A redundant edge between D and E exists . PDS-trees for D and E are depicted in Figure 1 ( c ) and ( d ) , respectively . An example for complying with condition 2b when constructing an ICD-Sep for ( D , E ) is as follows . From Figure 1 ( c ) if A is in the set , then B must also be in the set , as it lies on the only path from D to A . It is important to note that by parameter r , we bind the conditioning set size to its distance from the tested nodes ( ICD-Sep conditions 1 and 2a ) . That is , the conditioning set size is bounded by the shortest PDS-path length connecting its nodes to the tested nodes . In an ICD iteration r , first , all connected edges are tested for independence conditioned on ICD-Sep sets . Then the ICD iteration concludes by orienting the resulting graph . Initially , v-structures are oriented and then FCI-orientation rules ( Spirtes et al. , 2000 ; Spirtes , 2001 ) are repeatedly applied until no more edges can be oriented . For completeness , in the last ICD iteration , a complete set of orientation rules is applied ( Zhang , 2008 ) . The ICD algorithm is described in Algorithm 1 . The main loop , lines 2–4 , iterates over conditioning set sizes concurrently with the search radius on the graph . The iterative stage is described in function Iteration , lines 6–18 . This function can be viewed as an operator that maps from an ( r − 1 ) -Oequivalence class to an r-O-equivalence . Thus , ICD is anytime in the sense that the main loop , lines 2–4 , can be terminated for any value of r , resulting in a PAG that entails correct independence and causal relations . That is , terminating the loop after iteration r = n , results in a PAG that represents an n-O-equivalence class . Nevertheless , it still may not entail all relations that are entailed from the O-equivalence class of the underlying causal graph . In the first ICD iteration , the initial PAG is a complete graph and r = 0 . Thus every pair of nodes is tested for marginal independence ( conditioning sets are empty ) . In the second iteration r = 1 , where only nodes that are adjacent to the tested nodes are included in the conditioning set ( PDS-path length limit is one edge ) . Only in succeeding iterations , nodes that are outside the neighborhood of the tested nodes may be included in the conditioning set . Conditioning sets , composed with from adjacent nodes or from outside the neighborhood , are returned by the function PDSepRange . The result of PDSepRange ( X , Y , r , G ) , in Algorithm 1-line 9 , is an ordered set of possible separating sets { Zi } ` i=1 , where each Zi complies with the ICD-Sep conditions . The specific order in which these sets are used ( Algorithm 1-line 12 ) to test conditional independence in Algorithm 1-line 13 may affect the total number of CI test in practice . One possible heuristic for ordering this set is such that Zi sets are sorted based on to the average of the shortest PDS-path lengths connecting each node in Zi to the tested nodes . First , for every set Z ∈ { Zi } ` i=1 created by PDSepRange , the following value is calculated , d̂X ( Z ) = 1 |Z| ∑ W∈Z min ( |ΠY ( X , W ) | ) , ( 1 ) where ΠY ( X , W ) is the PDS-path from X to W , and | · | is path length . Then , the possible separating sets , Zi are ordered according to this value . Note that the correctness of ICD is invariant to this order . Algorithm 1 : Iterative causal discovery ( ICD algorithm ) Input : n : desired n-representing PAG ( default : |O| − 2 ) Ind : a conditional independence oracle Output : G : a PAG for n-O-equivalence class ( a completed PAG is returned for the default n = |O| − 2 ) 1 initialize : r ← 0 , G ← a complete graph with ‘ o ’ edge-marks , and done← False 2 while ( r ≤ n ) & ( done = False ) do 3 ( G , done ) ← Iteration ( G , r ) . refine G using conditioning sets of size r 4 r ← r + 1 5 return G 6 Function Iteration ( G , r ) : 7 done← True 8 for edge ( X , Y ) in edges ( G ) do 9 { Zi } ` i=1 ← PDSepRange ( X , Y , r , G ) . Zi complies with ICD-Sep conditions 10 if ` > 0 then 11 done← False 12 for i← 1 to ` do 13 if Ind ( X , Y |Zi ) then 14 remove edge ( X , Y ) from G 15 record Zi as a separating set for ( X , Y ) 16 break 17 orient edges in G 18 return ( G , done )

Summary: The authors describe an algorithm for learning causal relations in the presence of latents and selection bias. The algorithm called ICD recovers the partial ancestral graph (PAG) corresponding to "some" equivalence class of the maximal ancestral graph of the underlying DAG. Most prior work for this problem studied the problem of recovering PAG using FCI variants.   In this work, the authors outline a greedy neighborhood search-based algorithm (ICD) that can be stopped after "r" iterations, and the PAG recovered by ICD corresponds to an "r-equivalence class". By definition, an "r-equivalence class" encodes all causal relations that can be recovered using conditional independence tests that use at most "r" observable nodes. Therefore, the guarantees of this paper are weaker than the recovery guarantees of FCI which return the PAG of the equivalence class. Note that FCI uses conditioning sets that can be of size as large as the size of the total observable nodes.  FCI is an anytime algorithm and can be stopped after "r" iterations and outputs an "r-equivalence class". In the first stage of FCI, the PC algorithm returns a set of nodes S and a partial graph. Then, FCI checks subsets of S in the order of increasing size for conditional independencies. Finally, the orientation rules that are sound and complete are applied. In this paper, the authors suggest a way of bypassing this step-wise procedure, directly generating subsets of increasing size and checking for conditional independencies. Finally, they suggest using the orientation rules of FCI (when asked to terminate after "r"th iteration). Although there is no theoretical improvement in the number of conditional independence tests used, the authors empirically demonstrate that their approach is better than FCI in the number of conditional independence tests used.