Latent Hierarchical Causal Structure Discovery with Rank Constraints
Abstract
Most causal discovery procedures assume that there are no latent confoundersin the system, which is often violated in real-world problems.
In this paper, weconsider a challenging scenario for causal structure identiﬁcation, where somevariables are latent and they may form a hierarchical graph structure to generatethe measured variables; the children of latent variables may still be latent and onlyleaf nodes are measured, and moreover, there can be multiple paths between everypair of variables (i e., it is beyond tree structure).
We propose an estimation procedure that can e ﬃ ciently locate latent variables, determine their cardinalities, andidentify the latent hierarchical structure, by leveraging rank deﬁciency constraintsover the measured variables.
We show that the proposed algorithm can ﬁnd thecorrect Markov equivalence class of the whole graph asymptotically under properrestrictions on the graph structure and with linear causal relations.
Introduction
In many cases, the common assumption in causal discovery algorithms–no latent confounders–maynot hold.
For example, in complex systems, it is usually hard to enumerate and measure all taskrelated variables, so there may exist latent variables that inﬂuence multiple measured variables,the ignorance of which may introduce spurious correlations among measured variables.
Muche ﬀ ort has been made to handle the confounding problem in causal structure learning.
One researchline considers the causal structure over measured variables, including FCI and its variants [Spirteset al., 2000, Pearl, 2000, Colombo et al., 2012, Akbari et al., 2021], matrix decomposition-basedapproaches [Chandrasekaran et al., 2011, 2012, Frot et al., 2019], and over-complete ICA-basedones [Hoyer et al., 2008, Salehkaleybar et al., 2020].
Another line focuses on identifying the causal structure among latent variables, including Tetradcondition-based approaches [Silva et al., 2006, Kummerfeld and Ramsey, 2016], high-order momentsbased ones [Shimizu et al., 2009, Xie et al., 2020, Adams et al., 2021], copula model-based approach
[Cui et al., 2018], and mixture oracles-based approach [Kivva et al., 2021].
Moreover,regarding the scenario of latent hierarchical structures, previous work along this line assumes a treestructure [Pearl, 1988, Zhang, 2004, Choi et al., 2011, Drton et al., 2017], where there is one andonly one undirected path between every pair of variables.
This assumption is rather restrictive andthe structure in real-world problems could be more complex–beyond a tree.
In this paper, we consider a more challenging scenario where latent causal variables form a hierarchicalgraph structure to linearly generate measured variables—the children of latent variables may still belatent and only the leaf nodes are measured, and moreover, there can be multiple paths between everypair of variables (see the example hierarchical graphs in Figure 1).
We aim to ﬁnd out identiﬁabilityconditions of the hierarchical structure that are as mild as possible, and meanwhile, develop ane ﬃ cient algorithm with theoretical guarantees to answer the following questions.
(1) How can welocate latent parents for both measured and latent variables, as well as determining the cardinality of
L
L 2 L
L 4 L
L 6 L 7 L
L 9 L 10 X 12 X 13 X 14 X 15 X
X 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 X 9 X 10 X(a)
X
X
X X
L
L L
L L
X X
L
X 0 X
L
X X
X X
X(b)
[Figure]
Parents
[Figure]
the latent parents, by only providing the leaf nodes?
(2) How can we identify the causal relationshipsamong latent variables and that from latent variables to measured variables?
Interestingly, we can answer these questions by properly making use of rank deﬁciency constraints;ﬁnding and leveraging rank properties in speciﬁc ways enable us to identify the Markov equivalenceclass of the whole graph, under appropriate conditions.
Our contribution is mainly two-fold:
• We propose a structure identiﬁcation algorithm that can e ﬃ ciently locate latent variables (includingtheir cardinalities) and identify the latent hierarchical structure, by leveraging the rank deﬁciency.
• We show that the proposed algorithm can ﬁnd the correct graph asymptotically under mildrestrictions of the graph structure.
Roughly speaking, we show that it is su ﬃ cient to have k + 1 pure
children (which can be latent), as well as another k + 1 neighbors, to identify the latent variable set
with size k (see the detailed conditions in Deﬁnition 4 and Condition 1).
It is worth mentioning that rank constraints have been used in previous methods [Silva et al., 2006,
Kummerfeld and Ramsey, 2016], but they assume that each latent variable has three measured onesas children and each measured variable has only one latent parent.
There are also other methods forlatent structure learning; for instance, Anandkumar et al.
[2013], which uses matrix decomposition,needs 3 k measured children, and the GIN-based method [Xie et al., 2020], which makes use ofhigh-order statistics, needs 2 k measured children.
However, all those developments require that everylatent variable should have measured variables as children.
The paper is organized as follows.
In Section 2, we give formal deﬁnitions of the latent hierarchicalcausal model under investigation and give conditions that are essential to the identiﬁability of thegraph structure.
In Section 3, we propose an e ﬃ cient algorithm that makes use of rank-deﬁciencyconstraints to identify the latent hierarchical structure.
Moreover, we show theoretically in Section4 that the proposed algorithm outputs the correct Markov equivalence class of the whole graphasymptotically.
In Section 5, we empirically validate the proposed approach on synthetic data.
Acomplete version of experimental settings and results, as well as more details on the related work,identiﬁability conditions, algorithms, and theoretical results, are given in Appendix.
Notations forgraphical representations that are used in the paper are provided in Table 1.
Latent Hierarchical Causal Model
In this paper, we focus on latent hierarchical causal model with graph structure G , where bothmeasured variables X G = { X 1 , · · · , X m } and latent variables L G =
{ L 1 , · · · , L n } are generated by theirlatent parents in a directed acyclic graph (DAG) with linear relationships:
[Equation]
(1) where b ij and c jk are the causal strength from L j to X i and from L k to L j , respectively, and ε X i andε L j are noise terms that are independent of each other.
Without loss of generality, we assume that allvariables X G and L G have zero mean.
Below, we ﬁrst give the general deﬁnition of a linear latent hierarchical graphical model in Deﬁnition1.
Then we give more detailed conditions on the graph structure (Deﬁnitions 2-4) that are essential toformalize the identiﬁability condition of the latent hierarchical structure.
Deﬁnition 1 (Linear Latent Hierarchical (L 2 H) Model) .
A graphical model, with its graph G =( V G , E G ) , is a linear latent hierarchical model if:
V G = X G (cid:83) L G , where X G is the set of measured variables and L G is the set of latent variables,2.
there is at least one undirected path between every pair of variables, and3.
each variable in X G and L G are generated by the structural equation models in Eq.
Generally, without further constraints, the causal structure of the L 2 H model is hard to be identiﬁed.
It has been shown that if the underlying graph structure satisﬁes a tree [Pearl, 1988], then the structureis identiﬁable.
However, this structural constraint may be too strong to hold in many real-worldproblems.
In this paper, we give su ﬃ cient conditions that are much milder than previous ones, aswell as an e ﬃ cient search algorithm, for the identiﬁability of the causal structure.
We now give the corresponding deﬁnitions, including pure children , pure descendants , and e ﬀ ectivecardinality , that will be used in the identiﬁability condition, together with illustrative examples.
Deﬁnition 2 (Pure Children) .
Variables V are pure children of a set of latent variables L in a graph
G , if Pa G ( V ) =
L and V ∩ L = ∅ , and for any V ⊂ V (cid:48) , L ⊂ Pa G ( V (cid:48) ) .
That is, V is the maximal setthat have no other parents than L .
We denote the pure children of L by PCh G ( L ) .
Accordingly, Pure Descendants of a set of latent variables L are deﬁned as all recursive pure childrenof L (including PCh G ( L ) , PCh G ( PCh G ( L )), etc.), denoted by PDe G ( L ).
Furthermore, measuredvariables that are pure descendent of L are called Measured Pure Descendants , denoted by M G ( L ).
Example 1.
In Figure 1(a), the pure children of { L 2 , L 3 } are { L 6 , · · · , L 10 } , its pure descendants are{ L 6 , · · · , L 10 , X 1 , · · · , X 11 } , and its measured pure descendants are { X 1 , · · · , X 11 } .
Deﬁnition 3 (E ﬀ ective Cardinality) .
For a set of latent variables L , denote by C the largest subset of
PCh G ( L ) such that there is no subset C (cid:48) ⊆
C satisfying | C (cid:48)
| > | Pa G ( C (cid:48) )
| and | Pa G ( C (cid:48) )
| < | L | .
Then,the e ﬀ ective cardinality of L ’s pure children is | C | .
In the case when L and its pure children are fully connected, the e ﬀ ective cardinality is just thecardinality of the pure children of L .
However, for Figure 1(a), the e ﬀ ective cardinality of the purechildren of { L 7 , L 8 } is 3, because here the largest subset that satisﬁes the condition is { X 5 , X 6 , X 7 } .
We further deﬁne latent atomic cover that constraints the number of pure children and the number ofneighbour set of latent variables, which are essential for the structural identiﬁability.
Deﬁnition 4 (Latent Atomic Cover) .
Let L =
{ L 1 , ..., L k } be a set of latent variables in graph G , with|
L | =
k.
We say that L is a latent atomic cover if the following conditions are met:
there exists a subset of pure children C (cid:48) ⊆
PCh G ( L ) with e ﬀ ective cardinality ≥ k + 1 ;
there exists a neighbour set B to L s.t.
B (cid:84) C (cid:48)
= ∅ and | B | =
k + 1 ;
there does not exist a partition of L =
L 1 ∪ L 2 , so that both L 1 , L 2 satisfy conditions 1 and 2 and{ PCh G ( L 1 ) ∪ PCh G ( L 2 ) }\ L = PCh G ( L ) .
Example 2.
In Figure 1(a), L = { L 2 , L 3 } is a latent atomic cover with k = 2 , because (1) there existsa subset of pure children C (cid:48)
=
{ L 6 , L 7 , L 8 } with e ﬀ ective cardinality 3 = k + 1 , (2) there exists aneighbor set B =
{ L 1 , L 9 , L 10 } , s.t. B ∩ C (cid:48)
= ∅ and | B | = 3 = k
+ 1 , and (3) neither { L 2 } or { L 3 }satisﬁes the above two conditions.
We now give the conditions for structural identiﬁability from measured variables X G alone, includingthose on the structural constraints (Condition 1) and rank faithfulness assumption (Condition 2).
Condition 1 (Irreducible Linear Latent Hierarchical (IL 2 H) Graph) .
An L 2 H graph G is an IL 2 Hgraph if
every latent variable L ∈ L in G belongs to at least one latent atomic cover,
for any pair of latent atomic covers ( L A , L B ) , if PDe G ( L A ) (cid:84) PDe G ( L B ) (cid:44) ∅ , then either (a)
L A ⊂ L B or (b) L A ⊂ PDe G ( L B ) or (c) L B ⊂ L A or (d) L B ⊂ PDe G
( L A ) , and
for any three latent atomic covers L A , L B , L C , if the causal structure satisﬁes L A → L B → L C or
L A ← L B → L C , and | L B | = k
, then L B has 2 k neighbors except for L A , L C and the parents in the v structure where L B is a collider
Next, we give the faithfulness assumption, which holds for generic covariance matrices consistentwith G [Spirtes, 2013].
Condition 2 (Rank Faithfulness) .
A probability distribution P is rank faithful to a DAG G if everyrank constraint on a sub-covariance matrix that holds in P is entailed by every linear structuralmodel with respect to G .
We will show in section 4 that if the underlying latent hierarchical graph satisﬁes an IL 2 H graphand the rank faithfulness holds, then the location and cardinality of latent variables, and the causalstructure among latent atomic covers and that from latent atomic covers to measured variables, areidentiﬁable with appropriate search procedures.
Below, let us ﬁrst present the identiﬁcation procedure.
Structure Identiﬁcation with Rank-Deﬁciency Constraints
We propose an identiﬁcation algorithm (Algorithm 1) to identify the structure of IL 2 H graphs, byleveraging rank-deﬁciency constraints of measured variables.
In particular, the algorithm includesthree phases: ﬁnding causal clusters and assigning latent atomic covers in a greedy manner (“ ﬁndCausalClusters "), reﬁning incorrect clusters and covers due to the greedy search (“ reﬁneClusters "),and reﬁning edges and ﬁnding v structures (“ reﬁneEdges ").
Input : Date from a set of measured variables
X G Output: Markov equivalence class G (cid:48) 1 G (cid:48) =
ﬁndCausalClusters ( X G ) ; // find clusters and assign latent covers greedily 2 G (cid:48)
= reﬁneClusters ( G (cid:48) ) ; // refine incorrect clusters and covers from greedy search 3 G (cid:48)
= reﬁneEdges ( G (cid:48) ) ; // refine some edges and find v structures 137 Before describing the identiﬁcation algorithm, we ﬁrst give a theorem that relates the graphicalstructure of an IL 2 H graph to the rank constraints over the covariance matrix of measured variables.
Theorem 1 (Graphical Implication of Rank Constraints in IL 2 H Graphs) .
Suppose G satisﬁes an IL 2 H graph.
Under the rank faithfulness assumption, the cross-covariance matrix Σ X A , X B over measured variables X A and X B in G (with | X A | , | X B | > r) has rank r, if and only if there exists a subset of latent variables L with | L | = r such that L d-separates X A from X B , and there is no L (cid:48) with | L (cid:48) | < | L | that d-separates X A from X B .
That is,
[Equation]
For instance, in Figure 1(a), rank ( Σ { X 1 , X 2 } , { X 3 , X 4 } )
= 1, because L 6 d-separates { X 1 , X 2 } from { X 3 , X 4 }with | L 6 | = 1.
We further show that for any subset of latent variables in an IL 2 H graph, we can usethe measured variables as surrogates to estimate the rank, as indicated in the following theorem.
Theorem 2 (Measurement as a surrogate) .
Suppose G is an IL 2 H graph.
Denote by A , B ⊆ V G twosubsets of variables in G , with A ∩ B = ∅ .
Furthermore, denote by X A the set of measured variablesthat are d-separated by A from all other measures, and by X B the set of measured variables that ared-separated by B from all other measures.
Then rank ( Σ A , B ) = rank ( Σ X A , X B ) .
A special case when Theorem 2 holds is that X A and X B are the measured pure descendants of Aand B , respectively.
Note that Theorem 1 is a special case of Theorem 2.8 in Sullivant et al.
[2010]when applied to IL 2 H graphs.
Di ﬀ erent from the setting in Sullivant et al.
[2010] where access to thefull covariance matrix Σ V G , V G is assumed, we only have access to the covariance matrix Σ X G , X
G overthe measured variables X G , which we will use to infer the causal structure over the entire graph G .
For this reason, although we can infer the number of latent variables that d-separate any two sets ofmeasured variables X A , X B , we cannot directly know the exact location of these variables in the graph.
Fortunately, the structure constraints of the IL 2 H graph and Theorem 2 will allow us to reconstructthe graph with certain search procedures, as shown below.
3.1 Phase I: Finding Causal Clusters
We start to discover clusters in a recursive and greedy manner, by performing rank-deﬁciency testsover measured variables.
We denote by S a set of active variables that is under investigation; S is setto X G initially and will be updated to include latent atomic covers when rank deﬁciency is discovered.
Below, we ﬁrst give the deﬁnition about set size and the deﬁnition of atomic rank-deﬁciency set ,which will be used in Rule 1 and Algorithm 2.
Deﬁnition 5 (Set Size) .
Suppose S is a set.
We denote by |S| the cardinality of S ; that is, |S| isthe number of elements S i in S , for S i ∈ S .
We denote by (cid:107)S(cid:107) the number of variables in S , where(cid:107)S(cid:107)
=
| (cid:83) i S i | , for S i ∈ S .
For example, suppose S =
(cid:8) { L 1 } , { L 6 } , { L 7 , L 8 } , { L 9 , L 10 } (cid:9) .
Then |S| = 4 and (cid:107)S(cid:107) = 6.
Deﬁnition 6 (Atomic Rank-Deﬁciency Set) .
Given a graph G (cid:48) .
Denote by S ⊆ V G (cid:48) an active setof variables that is under investigation.
Let A ⊂ S and B = S\ A .
Denote by X A = M G (cid:48)
( A ) and
X B = M G (cid:48)
( B ) the measured pure descendants of A and B , respectively.
If (1) Σ A , B is rank deﬁcient,i.e., rank ( Σ A , B ) < min {(cid:107) A (cid:107) , (cid:107) B (cid:107)} ), and (2) no proper subset of A is rank deﬁcient, then A is calledan atomic rank-deﬁcient set, and Σ A , B can be estimated by Σ X A , X B .
For example, in Figure 1(a), suppose now we have the measured variables X G under investigation, sothe active variable set S =
X G .
A =
{ X 1 , X 2 } ⊂ S forms an atomic rank-deﬁciency set because thecross-covariance matrix of X A against all other measures has rank 1 (i.e., rank deﬁcient).
Accordingto Theorem 1, the rank deﬁciency occurs because the latent parent L 6 of { X 1 , X 2 } d-separates themfrom all other measures.
This naturally leads to the following rule that assigns a latent atomic coverover the rank-deﬁciency set:
Rule 1: If A is a rank-deﬁciency set with rank ( Σ X A , X B ) =
k , then assign a latent atomic cover L ofsize k as the parent of every variable A i ∈ A .
Later in Phase II, we will show that Rule 1 may not correctly identify the latent atomic coverin some cases, and, accordingly, we will further provide an e ﬃ cient revision procedure in Phase
II.
At the current phase, we use Rule 1, together with certain search procedures, to identify theclusters and latent atomic covers.
The detailed search procedure of Phase I is given in Algorithm2 ( ﬁndCausalClusters ), which tests for rank deﬁciency recursively to discover clusters of variablesand their latent atomic clusters.
The set of active variables S is set to X G initially (line 1) and will beupdated as the search goes on (line 13).
We start to identify the latent atomic cover with size k = 1(line 1).
We consider any subset of the latent atomic covers in S and replace them with their purechildren, resulting in ˜ S (line 4).
Then we draw a subset of variables A ⊂ ˜ S with cardinality at leastk + 1 and conduct a rank deﬁciency test of A against S\ A by estimating the rank of rank ( Σ X A , X B )(lines 6-7).
Note that here X A and X B are the measured pure descendants of A and B , respectively,in the currently learned graph.
This step is repeated until all subsets are tested (lines 5-9).
If rankdeﬁciency is found, we merge the overlapping groups into a cluster and add latent covers over thecluster (lines 10-13).
We further reset k = 1 and resume the search (line 14).
Otherwise, if no latentcover is found, we increment k
= k + 1 (line 16).
This procedure is repeated until no more clusters arefound.
Finally, we connect the elements in S into a chain structure (line 18).
Figure 2 illustrates anexample procedure of ﬁnding new clusters by applying ﬁndCausalClusters to the measured variables
X G generated from the structure in Figure 1(a).
[Figure]
G (cid:48)
[Figure]
3.2 Phase II: Reﬁning Clusters
As we have mentioned above, the naive assignment of causal clusters in Algorithm 2 ( ﬁndCausalClusters ) may not be correct in some cases.
In this section, we provide a precise characterizationof the cases where ﬁndCausalClusters incorrectly clusters variables and, accordingly, propose ane ﬃ cient algorithm to correct such cases.
Take Figure 3(a) as an example to illustrate the issue.
By applying ﬁndCausalClusters to the measuredvariables X G , at k = 2, we discovered that { X 9 , X 10 , X 11 } form a cluster and then set a 2-latent atomiccover { L (cid:48) 1 , L (cid:48) 2 } over it (Figure 3(b)).
This is not correct, because X 9 , X 10 , X 11 actually belong to threedi ﬀ erent 1-latent atomic covers, L 1 , L 2 , L 3 , respectively.
The incorrect clustering and covering arebecause when discovering the rank-deﬁciency set { X 9 , X 10 , X 11 } , the latent covers { L 4 , L 5 } , { L 6 , L 7 }have not been identiﬁed yet.
If { L 4 , L 5 } had already been discovered, we would correctly ﬁnd that{ X 9 , L 4 , L 5 } form a 1-latent atomic cover, and the algorithm would proceed correctly thereafter.
Moregenerally, ﬁndCausalClusters may set incorrect cluster for the rank-deﬁciency set { X 9 , X 10 , X 11 } , inthe case when their parents L 1 , L 2 , L 3 split the graph into two or more disconnected graphs.
Thisresult is formally stated in the following deﬁnition and theorem.
Deﬁnition 7 (Bond Set) .
Consider a set of measured variables X ⊆ X G , and a minimal set of latentvariables L ⊆
L G that d-separate X from all other measures X (cid:48) (cid:66)
X G \ X .
We say that X is a bondset if L also d-separates some partition X
A ⊂ X (cid:48) , X B ⊂ X (cid:48) from one another.
From the previous example, we have seen that with the existence of bond sets, ﬁndCausalClustersmay end up with incorrect clusters and covers.
The following theorem shows that the presence ofbond sets is the only reason of incorrect clusters and covers with ﬁndCausalClusters .
Theorem 3 (Correct Cluster Condition) .
Suppose G is an IL 2 H graph with measured variables X G .
Consider the output G (cid:48) from applying ﬁndCausalClusters over X G .
If none of the clusters in G (cid:48) is thebond set in G , then all latent atomic covers have been correctly identiﬁed.
However, without access to the true graph G , we are not able to identify which clusters formed fromﬁndCausalClusters are bond sets.
Fortunately, the following properties of the clusters formed overbond sets provide a way to remove bond sets in our discovered graph even without such knowledge,so that the clusters and latent covers can be correctly identiﬁed.
Theorem 4 (Correcting Clusters) .
Denote by G (cid:48) the output from ﬁndCausalClusters and by G thetrue graph.
For a latent atomic cover L (cid:48) in G (cid:48) , if the measured pure descendants of L (cid:48) is a bond setin the true graph G , then there exist a set of siblings S of L (cid:48) in G (cid:48) , a set of children C of L (cid:48) , and a setof grandparents P of L (cid:48) , such that M G (cid:48)
( S ∪ C ∪ P ) forms a cluster that is not a bond set in G .
The above theorem shows that whenever forming a bond set, the siblings and grandparents of thelatent cover are the key to correct the incorrect clusters and covers.
Speciﬁcally, we will remove sucha cover and use its children to form new clusters with the siblings and grandparents (see details in Algorithm 3), and Theorem 4 guarantees that these new clusters will not contain bond sets, and thus,providing correct clusters.
Example 3.
In Figure 3(b), the formed cluster { X 9 , X 10 , X 11 } is the bond set in the true graph in
Figure 3(a), and the cover { L (cid:48) 1 , L (cid:48) 2 } is not correct.
Observe that L (cid:48) 7 , L (cid:48) 8 are siblings of { L (cid:48) 1 , L (cid:48) 2 } , and ifwe were to remove { L (cid:48) 1 , L (cid:48) 2 } (and its parent { L (cid:48) 9 , L (cid:48) 10 } ) and consider its children, we would ﬁnd that L (cid:48)
7forms a size 1 cluster with X 10 , and the cluster is over measures { X 5 , X 6 , X 7 , X 8 , X 10 , X 11 } which isnot a bond set.
[Figure]
Furthermore, with Theorem 4, we give the following rule for correcting the clusters and covers.
Rule 2: For each discovered latent atomic cover L , let V =
Gp G (cid:48)
( L ) (cid:83) S ib G (cid:48)
( L ) (cid:83) Ch G (cid:48)
( L ) andapply ﬁndCausalClusters to V to reﬁne the clusters.
Before introducing the detailed reﬁning algorithm reﬁneClusters based on Rule 2, we ﬁrst introducean operator makeRoot that will be used in the algorithm, which will not change the rank deﬁciencyconstraints.
Deﬁnition 8 ( makeRoot ) .
Give a graph G (cid:48) and a latent atomic cover L in G (cid:48) .
A makeRoot operatorof L , denoted by makeRoot ( L ) , reorients all outgoing edges of L to L , such that L is a root variable.
Algorithm 3 introduces the procedure of reﬁning clusters based on Rule 2.
Starting with the output
G (cid:48) from ﬁndCausalClusters , we proceed in a breadth-ﬁrst search from the root variable (lines 1-3, 5).
Because the root is trivially not over a bond set, Theorem 4 guarantees that for each child of the root,we will form new clusters that are not over bond sets.
For each cover L , if it has only one child, wewill recursively consider the children of its child instead, since if L is over a bond set then its singlechild will also be.
We add the ﬁrst set of children into the search set V (line 4).
We construct thesearch set (line 6) and remove the covers we are reﬁning (lines 7-8).
Finally, we make L the root(line 8) and conduct the search (line 9).
The algorithm ends after reﬁning every latent cover in G (cid:48) .
With this reﬁning procedure, we will derive the correct latent covers for the graph in Figure 3(a).
Algorithm
Input Output
Reﬁned
(cid:48) refineClusters ( G (cid:48) ): (cid:48) ( Root ( G (cid:48) ))); 2 repeat
L
(); while Ch G is a single atomic do remove Ch G
L and add Ch G
Ch as children of L ; each C
Ch
3.3 Phase III: Reﬁning Edges
With ﬁndCausalClusters and reﬁneClusters , the output G (cid:48) correctly identiﬁes the latent variables.
Moreover, G (cid:48) correctly identiﬁes the following d-separation: for every L ∈ L G (cid:48) , its parents Pa G (cid:48)
( L )d-separates L and its descendants from the ancestors of Pa G (cid:48)
( L ), and thus, there cannot be any edgesfrom each L to any of its ancestors beyond its own parents.
However, G (cid:48) may still have incorrectedges locally .
Speciﬁcally, each time we create an atomic cover L in ﬁndCausalClusters , the implicitassumption is that each of its children is conditionally independent of the other children given theparents L .
However, this assumption is not necessarily true, as (i) the children of L may be directlyconnected to one another and (ii) L may only be directly connected to a subset of its children.
Inother words, previous steps did not consider condition independence relationships across clusters.
Hence, we need a further step to correct edges over each L and its children.
For example, consider the true graph in Figure 4(a), which ends up with the graph in Figure 4(b) withthe ﬁrst two phases.
However, note that the ﬁrst two phases did not consider the d-separation between
L (cid:48) 2 and L (cid:48) 3 , and thus the edges among L (cid:48) 1 , L (cid:48) 2 , L (cid:48) 3 , L (cid:48) 4 may not be correct, including the v structure.
Inparticular, in the true graph, L 1 d-separates L 2 from L 3 , while { L 1 , L 4 } does not d-separate L 2 from
L 3 , but these d-separations are not reﬂected in the discovered G (cid:48) in the ﬁrst two phases.
Thus, weneed to reﬁne the edges.
To this end, we ﬁrst set L (cid:48) 1 , L (cid:48) 2 , L (cid:48) 3 and L (cid:48) 2 , L (cid:48) 3 , L (cid:48) 4 to be fully connected, andconsider testing A
= { L (cid:48) 2 , X 1 } against B = { L (cid:48) 3 , X 2 } , where we partition the children of L (cid:48) 1 into twosets and put them in A and B , respectively.
By doing so, we force L (cid:48) 1 in G (cid:48) to be in the separating set.
Since rank ( Σ A , B ) = 1, it implies that no other variable is in the separating set, and therefore we canconclude that L (cid:48) 1 d-separates L (cid:48) 2 from L (cid:48) 3 .
This principle is captured in the following lemma.
Lemma 5 (Cross-Cover Test) .
Given a set of variables S , consider two latent atomic covers L A , L B ∈
S , and a potential separating set C =
{ L C i } ⊆
S\{ L A , L B } .
For each L C i , consider C Ai , C Bi ⊆
PCh ( L C i ) with C Ai , C Bi (cid:44) ∅ and C Ai ∩ C Bi = ∅ , and denote the cardinality k Ai (cid:66) min ( | L C i | , | C Ai | ) ,k B i (cid:66) min ( | L C
i | , | C B i | ) , respectively.
Then there is no edge between L A and L B if and only if there existsa separating set C such that rank ( Σ A , B ) < min ( | L A | + (cid:80)
i k Ai , | L B | + (cid:80) i k Bi ) , where A =
{ L A , C A 1 ,
C A 2 , ... }and B = { L B , C B 1 , C B 2 , ... } .
In this case, we say that C satisﬁes the cross-cover test of L A against L B .
Note that in order to ﬁnd rank deﬁciency when performing the cross-cover test, L C i needs to satisfythe third condition in Condition 1.
Based on Lemma 5, we use the following rule to reﬁne the edges.
Rule 3: For a pair of latent covers ( L A , L B ), let A ← { L A , C A 1 , C A 2 , ... } and B ← { L B , C B 1 , C B 2 , ... } .
If there exists such A , B such that rank G (cid:48)
( Σ A , B ) is rank deﬁcient, then remove all edgesbetween L A , L B in G (cid:48) .
Furthermore, with Rule 3, Algorithm 4a ( crossCoverTest ) in Appendix gives the procedure of reﬁningthe edges over a set of latent variables, which helps to correct the causal skeleton.
Furthermore, we aregoing to identify the v structures among latent atomic covers.
In the output G (cid:48) from phase II, for any
L (cid:48) with a child C (cid:48) i and parent P (cid:48) , it is not possible to have a collider C (cid:48)
i → L (cid:48) ← P (cid:48) in the ground-truthgraph, because in this case the cluster would not have been rank deﬁcient.
Therefore, the only vstructures in the graph are amongst the variables in L (cid:48) ∪ PCh G (cid:48)
( L (cid:48) ), and similar to crossCoverTest ,we only need to test for v structures locally.
Continue to consider the example in Figure 4.
The edge between L (cid:48) 2 and L (cid:48) 3 is missing because the rankover { L (cid:48) 2 , X 1 } and { L (cid:48) 3 , X 2 } is 1, implying that L (cid:48) 1 d-separates L (cid:48) 2 from L (cid:48) 3 , as is done in crossCoverTest .
Now, since L (cid:48) 2 − L (cid:48) 4 − L (cid:48) 3 forms an unshielded triplet, we want to test if a collider exists at L (cid:48) 4 .
We ﬁndthat the rank over A =
{ L (cid:48) 2 , L (cid:48) 4 , X 1 } and B = { L (cid:48) 3 , X 2 } is 2 > 1, and the rank over A
=
{ L (cid:48) 2 , X 1 } and
B = { L (cid:48) 3 , L (cid:48) 4 , X 2 } is 2 > 1, so L (cid:48)
→ L (cid:48) 4 ← L (cid:48) 3 .
For general cases, the rule for ﬁnding v structures areformulated in the following lemma and Algorithm 4b ( ﬁndColliders ) in Appendix.
Lemma 6 (V-Structure Test) .
For any unshielded triangle L A − L C − L B , let A , B be the set ofvariables in Lemma
such that Σ A , B was rank deﬁcient.
Let k = rank ( Σ A , B ) , k 1 = rank ( Σ A∪ L C , B ) ,and k 2 = rank ( Σ A , B∪ L C ) .
Then, L A → L B ← L C if and only if k < min( k 1 , k 2 ) .
As mentioned above, we only need to perform cross-cover test and v-structure test locally in theestimated graph G (cid:48) .
Algorithm 4 ( reﬁneEdges ) combines the search procedure of the two componentstogether with the output G (cid:48) from reﬁneClusters as the input.
Speciﬁcally, for each latent cover L (cid:48) in
G (cid:48) , we only need to consider testing for edges amongst L (cid:48) and its children C =
Ch G (cid:48)
( L (cid:48) ).
Thus, weperform a depth-ﬁrst traversal of the output graph G (cid:48) starting from Root ( G (cid:48) )
(lines 1-3), and apply thecross-cover test at each L to reﬁne the skeleton of G (cid:48) (lines 4-7, 11) and the collider test to identify thev structures (lines 10, 11).
After determining v-structures, we can ﬁnd more directions by applying
Meek’s rule (line 12), analogous to that in the PC algorithm
[Spirtes et al., 2000].
[Figure]
Theoretical Results
In this section, we show the correctness of the algorithms proposed in Section 3.
In particular, bymaking use of the rank constraints of only the measured variables, the proposed algorithms outputthe correct Markov equivalence class of the IL 2 H graph asymptotically, under the minimal-graphoperator and skeleton operator , with their deﬁnitions given below.
Deﬁnition 9 (Minimal-Graph Operator) .
Suppose G is an IL 2 H graph.
For every latent atomic cover
L in G , merge L to its parents P if the following conditions hold: (1) L is the pure children of P , (2)| L | = | P | , and (3) the pure children of L form one latent atomic cover, or the siblings of L form onelatent atomic cover.
We call such operator the minimal-graph operator and denote it by O min ( G ) .
Deﬁnition 10 (Skeleton Operator) .
Suppose G is an IL 2 H graph.
A skeleton operator of G , denoted by
O s ( G ) is deﬁned as follows: for any latent atomic cover L , draw an edge from l j ∈ L to c k ∈ PCh G ( L i ) ,if l j and c k are not directly connected in G .
Note that the minimal-graph operator and the skeleton operator will not change the rank.
Theseoperators have ready been achieved in Algorithm 1, so the output of the algorithm is the rankequivalent graph with the minimal number of latent atomic covers.
For instance, for the graph in
Figure 1(a), after applying the operators, the following edges will be added: L 3 → L 6 and L 3 → L 10 .
More details and corresponding examples are given in Section B in Appendix.
We next proceed to show that phases I-III will output a graph G (cid:48) such that O min ( O s ( G (cid:48) )) will be in thesame Markov equivalence class as O min ( O s ( G )), denoted by G (cid:48)
≈ G .
We have already shown earlier in
Theorem 3 that ﬁndCausalClusters gives correct latent covers when there is no bond set.
Furthermore,
Theorem 4 shows that even in the presence of bond sets, Phase II reﬁneClusters is able to reﬁneclusters into the one without bond sets.
Therefore, Phase I-II can correctly identify the latent atomiccovers of O min ( G ), and with only the ﬁrst two conditions in Condition 1 (see the theorem in Appendix).
Moreover, Phase III reﬁneEdges further guarantees correct skeletons and v structures.
Therefore, thefollowing theorem shows that Algorithm 1, which includes Phases I-III, can asymptotically identifythe Markov equivalence class, up to the skeleton operator and the minimal-graph operator.
Theorem 7.
Suppose G is an IL 2 H graph with measured variables X G .
Algorithm 1, including
Phases I-III, over X G can asymptotically identify the Markov equivalence class of O min ( O s ( G )) .
Experimental Results
We applied the proposed algorithm to synthetic data to learn the latent hierarchical causal graph.
Speciﬁcally, we considered di ﬀ erent types of latent graphs and di ﬀ erent sample sizes.
The causalstrength was generated uniformly from [ − 5 , − 0 .
5] ∪ [0 . 5 , 5], and the noise term was randomly chosenfrom a normal distribution with noise variance uniformly sampled from [1 , 5].
To the best of ourknowledge, this is the ﬁrst algorithm that can identify such general latent hierarchical structures, so tofairly compare with other methods, besides general IL 2 H graphs, we also considered tree structuresand measurement models.
We compared the proposed method with tree-based method–Chow-Liu
Recursive Grouping (CLRG)
[Choi et al., 2011], as well as measurement-model-based methods,including FOFC [Kummerfeld and Ramsey, 2016] and GIN [Xie et al., 2020].
Table 2 gives the estimation results evaluated on three types of latent graphs: IL 2 H graphs, treestructures, and measurement models.
The performance is measured by the percentage of correctlyidentiﬁed causal clusters over only measured variables.
Our method gives the best results on all typesof graphs, indicating that it can handle not only the tree-based and measurement-based structures,but also the latent hierarchical structure.
The CLRG algorithm does not perform well on tree-basedstructure because the metric is rather strict–even a single misclustered variable outputs an error.
Complete experimental settings and more results, including performance measured by other metricsand the case when the noise terms are uniformly distributed, are given in Section D in Appendix.
[Table]
Conclusions and Future Work
In this paper, we formulated a speciﬁc type of latent hierarchical causal model and proposed a methodto identify its graph by making use of rank deﬁciency constraints.
Theoretically, we show that theproposed algorithm can ﬁnd the correct Markov equivalence class of the whole graph asymptoticallyunder mild restrictions of the graph structure.
For more general graphs, only using the second-orderstatistics may result in rank equivalence class that contains multiple DAGs, so how to further leveragehigh-order statistics to distinguish between causal graphs within the equivalence class will be ourfuture work.
Other future research directions include allowing nonlinear causal relationships andallowing measured variables to cause latent variables (existing techniques, e.g., Adams et al.
Squires et al.
[2022], may help to mitigate this issue).
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions
and
scope?
[Yes]
We
proposed a method to identify latent hierarchical
causal
graphs by making use of rank deﬁciency constraints. Theoretically, we show thatthe proposed algorithm can ﬁnd
the correct Markov equivalence class asymptoticallyfor more general graph structures compared to previous methods
(b) Did you describe the limitations of your work?
[Yes] See Section 6 on future work.
One limitation is that here we only allowed linear models, and we will extend it to allownonlinear causal relations as well.
Another limitation is that here we did not considerdirect causal edges among the measurement variables.
This is because, for such cases,only using the second-order statistics may only identiﬁed to the rank equivalence class,which may contain multiple DAGs, so how to further leverage high-order statistics todistinguish between causal graphs within the equivalence class will be our future work.
(c) Did you discuss any potential negative societal impacts of your work?
[No] We do not
think there is any negative societal impact at the current moment.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes] We have read the ethics review guidelines.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
We clearly
stated the assumptions in Section 2.
(b) Did you include complete proofs of all theoretical results?
[Yes] We gave complete
proofs of all theoretical results in Appendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We gave the
detailed experimental setting in Appendix.
Code will be made public after acceptance.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] We speciﬁed all the details in Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section D in Appendix.(d)
Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section D in Appendix.
If you are using existing assets (e.g., code, data, models) or curating / releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N / A]
(b) Did you mention the license of the assets?
[N / A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N / A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re using / curating?
[N / A]
(e) Did you discuss whether the data you are using / curating contains personally identiﬁableinformation or o ﬀ ensive content?
[N / A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N / A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N / A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N / A]