A Versatile Causal Discovery Framework to Allow Causally-Related Hidden Variables
Authors: Xinshuai Dong
Biwei Huang
Ignavier Ng
Xiangchen Song
Yujia Zheng
Songyao Jin
Roberto Legaspi
Peter Spirtes
Kun Zhang
Venue: ICLR 2024 poster (2024)
Keywords: Causal Discovery; Latent Variables
Primary area: causal reasoning
TL;DR: Causal Discovery in the Presence of Latent Variables with Rank Constraints

Abstract: Most existing causal discovery methods rely on the assumption of no latent confounders, limiting their applicability in solving real-life problems. In this paper, we introduce a novel, versatile framework for causal discovery that accommodates the presence of causally-related hidden variables almost everywhere in the causal network (for instance, they can be effects of measured variables), based on rank information of covariance matrix over measured variables. We start by investigating the efficacy of rank in comparison to conditional independence and, theoretically, establish necessary and sufficient conditions for the identifiability of certain latent structural patterns. Furthermore, we develop a  Rank-based Latent Causal Discovery algorithm, RLCD, that can efficiently locate hidden variables, determine their cardinalities, and discover the entire causal structure over both measured and hidden ones. We also show that, under certain graphical conditions, RLCD correctly identifies the Markov Equivalence Class of the whole latent causal graph asymptotically. Experimental results on both synthetic and real-world personality data sets demonstrate the efficacy of the proposed approach in finite-sample cases. Our code will be publicly available.

Published as a conference paper at ICLR 2024
A VERSATILE CAUSAL DISCOVERY FRAMEWORK TO ALLOW GAUSALLYRRELATH HIDDEN VARIABLES
Xinshuai Dong"l Biwei Huang +2
Ignavier Ng
Xiangchen Song'
Yujia Zheng'
Songyao Jin
Roberto Legaspi
Peter Spirtes
Kun Zhang
CCarnegie Mellon University
University of California San Diego
3 KDDD Research
Mohamed bin Zayed University of Artificial Intelligence
ABSTRACT
Most existing causal discovery methods rely on the assumption of latent con- founders, limiting their applicability in solving real-life problems. In this paper, we introduce a novel, versatile framework for causal discovery that accommodates the presence of causally-related hidden variables almost everywhere in the causal network (for instance, they can effects of observed variables), based on rank in- formation of covariance matrix over observed variables. We start by investigating the efficacy of rank in comparison to conditional independence and, theoretically, establish necessary and sufficient conditions for the identifiability of certain latent structural patterns. Furthermore, we develop a Rank-based Latent Causal Discov- ery algorithm, RLCD, that can efficiently locate hidden variables, determine their cardinalities, and discover the entire causal structure over both measured and hid- den ones. We also show that, under certain graphical conditions, RLCD correctly identifies the Markov Equivalence Class of the whole latent causal graph asymp- totically. Experimental results on both synthetic and real-world personality data sets demonstrate the efficacy of the proposed approach in imite-sample cases. Our code will be publicly available.
INTRODUCTION AND RELATED WORK
Causal discovery aims finding causal relationships from observational data and has received suc- cessful applications in many fields (Spirtes et al.) 20000 2010; Pearl 2019). However, traditional methods, such as PC (Spirtes et al. 2000), GES (Chickering 2002b), and LiNGAM (Shimizu al., 2006b), generally assume that there are no latent confounders the graph, which hardly holds in many real-world scenarios. Therefore, extensive efforts have been dedicated to addressing this issue for causal structure learning.
One line of research focuses on inferring the causal structure among the observed variables, de- spite the possible existence of latent confounders. Notable approaches include FCI and its variants (Spirtes al. 2000 Pearl 2000; Colombo et al., 20122] Akbari (202]) that leverage conditional independence tests and over-complete ICA-based techniques (Hoyer et al. 2008 Salehkaleybar
al.
2020) that further leverage non-Gate -ausssianintt.
Another line of thought focuses more on uncovering the causal structure among latent variables, by assuming observed variables are not directly adjacent. This includes Tetrad condition-basse (Silva
et al.
2006,
Kummerfeld & Ramsey| 2016), high-order moments- based (Shimizu et al. 2009 Cai
al.
2019
Xie et al. 2020, Adams al. 202T Chen et al.] 2022). matrix decomposition-bases
Anandkumar al. (201)), and mixture oracles-based (Kivva al. 2021) approaches. Recently,
Huang et al. (2022) propose an approach that makes use of rank constraints to identify general latent hierarchical structures, and yet observed variables can only be leaf nodes. A Chandrasekaran et al.] (2012) allow direct causal influences within observed variables and the existence of latent variables, cannot recover the causal relationships among latent variables and has strong graphical constraints. For a more detailed discussion of related work, please refer to our Appx. C.I
"Equal contribution.

---

Published as a conference paper at ICLR 2024
An example graph that we aim to handle, where latent variables are denoted by L and observed vari- ables are denoted by X. The latent variables can act mediator for both observed variables and other latent variables. See the ap- pendix for more examples.
Figure I:
cause, effect, or
In this paper, we aim to handle a more general scenario for causal discovery with latent variables, where observed variables are allowed to be directly adjacent, and latent variables to be flexibly related to all the other variables. That is, hidden variables can serve as confounders, mediators, or effects of latent or observed variables. and even form a hierarchical structure (illustrated in Fig. DI. This setting is rather general and practically meaningful to deal with many real-world problems.
To address such challenging problem, we are confronted with three fundamental questions: (i) What information and constraints can be discovered from the observed variables to reveal the under- lying causal structure? (ii) How can we effectively and efficiently search for these constraints? (iii) What graphical conditions are needed to uniquely locate latent variables and ascertain the complete causal structure? Remarkably, these questions can be addressed by harnessing the power of rank de- ficiency constraints on the covariance of observed variables. By carefully identifying and utilizing rank properties in specific ways, we are able to determine the Markov equivalence class of the entire graph. Our contributions are mainly three-fold:
We investigate the efficacy of rank in comparison conditional independence in latent causal graph discovery, and theoretically introduce necessary and sufficient conditions for the identifia- bility of certain latent structural properties. For instance, the condition we proposed for nonadja- cency generalizes the counterpart in Spirress et al. (2000) to graphs with latent variables.
We develop RLCD, efficient three-phase causal discovery algorithm that is able to locate latent variables, determine their cardinalities, and identify the whole causal structure involving measured and latent variables, by properly leveraging rank properties. In the special case with latent variables, it asymptotically returns the same graph as the PC algorithm (Spirtes et al. 20000) does.
We provide a set of graphical conditions that are sufficient for RLCD to asymptotically identify the correct Markov Equivalence Class of the latent causal graph; notably, these graphical conditions are significantly weaker than those in previous works. Our empirical study on both synthetic and real-world datasets validates RLCD on finite samples.
PROBLEM SETTING
In this paper, we aim identify the causal structure of latent línear causal model defined as follows. Definition 1. (Latent Linear Causal Models) Suppose a directed acyclic graph G := (Vg.Eg) where each variable V E Vgg is generated follovinn a linear causal structural model:
V; Evyernot..) dijV + EV:
where Vg Lg Ugg contains a set of m random variables. with latent variables Lg [ and n observed variables Xg := {X PVg(V.) denotes the parent set of Vy, ary the causal coefficient from V, to V,, and Ev, represents the noise term.
Our objective is to identify the underlying causal structure G over all the variables Lg U Xg (de- tailed in Sec. (5) that are generated according to a latent linear causal model, given i.i.d. samples of observed variables XG only. To address this challenging problem, traditional wisdom often relies on strong graphical constraints (Pearl 1988 Zhang 2004 Huang et al.) 2022. Maeda & Shimizu 2020) (detailed in Appx. B.I with illustrative graphs). In contrast, Definition allows all the vari- ables including observed and latent variables to be very flexibly related. We basically allow the presence of edges between any two variables such that node V, no matter whether it is observed or not, can act as a cause, effect, or mediator for both observed and latent variables.
A summary of notations is in Tab. The rest of the paper is organized follows. In Sec. B] we motivate the use of rank and propose conditions for nonadjacency and the existence of latent variables. In Sec. 4 we establish the minimal identifiable substructure of linear latent graph, based on which we propose RLCD for latent variable causal discovery. In Sec. [ we introduce the identifiability of RLCD, In Sec. 6 we validate our method using both synthetic and real-life data.
2

---

Published as a conference paper at ICLR 2024
Table Graphical notations used throughout this paper.
Pa: Parents
V: Variables
V: Variable
G: The underlying graph
Ch: Children
L: Latent variables
L: Latent variable
G': Output Graph
PCh: Pure children
X: Observed variables
X: Observed variable
S: Set of covers
Sib: Siblings
MDe: Measured descendants
PDe: Pure descendants
S: Set of sets of covers
WHY USE RANK INFORMATION?
In this section, first motivate the use of rank constraints for causal discovery in the presence of latent variables, and then establish some fundamental theories about what rank implies graphically.
3.1 PRELIMINARIES ABOUT TREKS AND RANK
When there is no latent variable, a common approach for causal discovery is to use conditional independence (CI) relationships to identify d-separations in a graph; sec, e.g., the PC algorithm Spirtes et al. 2000). The following theorem illustrates this idea.
Theorem 1 (Conditional Independence and D-separation (Pearl 1988).. Under the Markov and faithfulness assumption, for disjoint sets of variables A, B and C, d-sspprratss and in graph G, iff A 1L BIC holds for every distribution in the graphical model associated to G.
As for latent linear causal models, trek-separations (I-separations) provide more information than d d-separations (for readers who are not very familiar with treks and t-separations, kindly refer to Appx. A.2 for examples). The definitions of treks and t-separation are given as follows, together with Theorem 2 showing the relations between seepaattions and d-separations.
Definition (Treks (Sullivant et al. 2010). In G, a trek from X to is an ordered pair of directed paths (P1, P2) where P1 has sink X. P2 has a sink Y, and both P1 and P2 have the same source Z. Definition 3 (T-separation (Sullivant et al., 2010)) Let A, B. CA. and C3 be four subsets V V in graph 9 (not necessarilly disjoint). (C Cn) I-separates A from B ffr every trek (2) from a vertex in A to a vertex in B, either Pi contains a vertex in CA or P2 contains a vertex in CB.
Theorem 2 (T- and D-sep (Dil 2009)). For disjoint sets A, B and C, C d-separates A and B in graph G. iff there is a partition C CA U UC such that (CA, CB) t-separates A C from BUC.
The theorem above reveals that all d-sep can be reformulated by t-sep, and thus, t-sep encompass d-sep information. Just as we use CI tests to find d-sep, t-sep can be identified by the rank of cross-covariance matrix over specific combinations of variables, which is formally stated as follows.
Theorem 3 (Rank and T-separation (Sullivant et al., 2010 Given two sets of variables A and B from a linear model with graph G, we have;
rank(E. S min([CAl + |CB| : (CA, CB) I-separates from B in 9), where DA.B is the cross-covariance over A and B, and equality generically holds.
(2)
The left-hand side of Equation 2 is about properties of the observational distribution, while the right- hand side describes properties of the graph. Equality in Equation 2 generally holds, if there are no specific ways for the parameters of the underlying causal model to be coupled (e.g., accidental path cancellations). Recall that the classical faithfulness (Spirtes et al. 2000) is assumed so that we can infer d d-separations from CI relations, here we formulate rank faithfulness in a similar spirit such that can infer t-separations from rank deficiency constraints, as follows.
Assumption (Rank Faithfulness). A probability distribution p is rank faithful to G if the inequality in Equation 2 can be replaced by equality.
This formulation of rank faithfulness differs from that of Spirtes (013)). though the spirit is shared The faithfulness assumption in causal discovery (Spirtes et al. 2000) is critical and prevalent. If faithfulness is violated (illustrated in Appx. (B2). even classical methods like PC cannot guarantee asymptotic correctness. On the other hand, with infinite data, the set of values of the SCM's free parameters s.t. rank faithfulness does not hold is of Lebesgue measure 0 (Spirtes) 2013).
Without latent variables, rank and CI are equally informative about the underlying DAG. Yet, in the presence of latent variables, t-separations inferred by rank offer more graphical information. Thus, we next show how rank constraint play a pivotal role in identifying latent causal structures.
3.2 RANK: AN INFORMATIVE GRAPHICAL INDICATOR FOR LATENT VARIABLES
In the presence of latent variables, Cl is not enough: FCI and its variants make full use of CI but only recover a representation which is not informative enough about the latent confounders. Fortunately, leveraging rank information can naturally make causal discovery results more informative.
3

---

Published as a conference paper at ICLR 2024
(a) G1
(e) Gs- Figure 2: Examples that illustrate the basic intuition for latent to be identifiable.
(b) ci skeleton of G1-
(c) G2-
(d) ci sieleton of Co-
( CT sketeton of G3-
An example highlighting the greater informativeness of rank compared to CI is as follows. Consider the graph G1 in Fig. where (X,, )2) and (X3, } are d-separated by L1, but we cannot infer that from a CI test (i.e., whether (X1: X2) 11 (X3. X1 ) L1), as L is not observed. In contrast, with rank information, we can infer that rank ( t-separated by one latent variable. The rationale behind is that the t-sep of A, B by (CA. CB) can be deduced through rank (as in Theorem 3) without observing any element in (CA, CB).
1, which implies (X1 X2) and (X3, X4} are
With this intuition in mind, below we present three theorems that characterize the graphical implica- tions of rank constraints, in scenarios where latent variables might exist: Theorem
4 ggives conditions
for observed variables to be nonadjacent, illustrated by Example 6 Theorem 5 gives conditions for the existence of latent variables, illustrated by Example 1 Theorem 6 implies how to utilize pure children as surrogates for calculating rank, illustrated with Example All proofs are in Appendix.
Theorem (Condition for Nonadjaceney). Consider a latent linear causal model. Two ob- served variables X1.X2 E Xg are not adjacent, if there exist two sets A,B S XX,,X1, X23 that are not necessarily disjoint, such that runk(2Auux))BBx(X))) = rank(_A,.) and rank(E (..
= rank(EA,,B) +
4 presents a sufficient condition for determining nonadjacency between two observed variables. In the absence of latent variables, this condition becomes a both necessary and sufficient one. Note that A and B may overlap. SGS and PC (Spirtes et al. (000) also introduced a necessary and sufficient condition for determining nonadjacency between two observed variables
Remark 1. Theorem
in the absence of latent variables: there exist a set of observed variables C S Xg. X1, X2 & C. such that X1 X2|C. Interestingly, this condition can be expressed in the form of Theorem 4 with C C= AA- B; thus Theorem 4 generalizes PC's condition to where latent variables may exitt
(detailed in Appx. [1.8). Even in the presence of latent variables, with further graphical constraints also become both necessary and sufficient one, in Proposition 7
the condition
Theorem (Condition for Existence of Latent Variable). Suppose a latent linear causal model with graph G and observed variables Xg. If there exist three disjoint sets of variables A,B, C S Xg. such that (i) [B| Z A| 2 2, (ii) V distinct A1, A E A, A1, A are adjacent in the CI skeleton over Xg with CI skeleton defined in Appx. AA.), (iii) VA E A, B E B, A, B are adjacent in the CI skeleton over (iv( C S [X] JY E A AUB. s.t. Y,Y are adjacent in the CI skeleton) (i.e., all elements in C are neighbours of element in A in the CI skeleton), (v) rank( (EAUCC,UC)))) |AA| |C), then there must exist at least one latent variable in the treks between A,B.
Remark 2. Theorem 5 provides sufficient condition for determining the existence of latent vari- ables. The intuition is that, in the absence of latent variables, rank information should align with CI skeleton; if not, there must exist latent variable. Furthermore, will show in Theorem 9 that, with further graphical constraints the condition in Theorem becooees both necessary and sufficient.
Moreover, it can be shown that observed children, even descendants of latent variables can be used as surrogates to calculate rank as stated in Theorem 66) with the definition of pure children below.
Definition 4 (Pure Children). In G. we denote the pure children set of a set of variables V, as PChg(V) = [C| Pag(C) * 0 and (Cg C V).
Theorem 6 (Pure Children as Surrogate for Rank Estimation). Let C C PChg(A) be a subset of pure children of A, and B be a set of variables such that B Deg (C) = (1. We have rank(EAA.)) 2 rank Ec,B). Moreover, if rank(EA,.))
|A], then rank (EA,B) = rank
Remark 3. Theorem @informs us that under certain conditions, we can estimate the rank of covari- ance involving latent variables by using their pure children as surrogates. Even when the pure chil- dren are not observed one can recursively examine the children's 's children until reaching observed descendants (defined in Appx. A. This enables to deduce graphical information associated with latent variables through the use of observed ones as surrogates.
4 DISCOVERING LATENT STRUCTURE THROUGH RANK CONSTRAINTS
In this section, we begin with the concept of atomic cover and explore its rank deficiency, and then develop an efficient algorithm based on mef-ciiiy for latent causal discovery, as Alg. [
4

---

Published as a conference paper at ICLR 2024
4.1 АTOМIC COVER AND RANK DEFICIENCY
Below, we introduce atomic covers and their associated rank deficiency properties, which allows us to define the minimal identifiable substructure of a graph. We start with an example in Figure thhat motivates the conditions for latent variable to be identifiable.
Example 1. Consider 9n in Fig. 2 (a). It can be shown that the latent variable L, in G1 is not identifable. We can easily find a gruph gi with no latent variable, e.g,, as in Fig. (b),, such that 9i shares the same skeleton as Fig. 2 (a), but all the observational rank information entailed by G1 is the same as by Gj they are indistinguishable. However, if X4 becomes the children of Li, as in G2 given in Fig. (c), the whole structure becomes identifiable. Specifically, the conditions in Theorem 5 holds when we take A = X2), B [X3, X4 ). and C = V. which informs the existence of latent variables. The same conditions also hold for Fig. (e) and thus L1 in G3 is also identifiable, though L1 has only two children logether with another two neighbors.
The intuition is that, for a latent variable to be identifiable, it should have enough children and enough neighbors. We next formalize this intuition into the concept of atomic cover as the minimal identifiable unit in graph. The formal defimition of an atomic cover is given in Definition 3 where effective cardinality of a set of covers V defined as | = [[Uvev
Definition 5 (Atomic Cover and p Cover). Let V S V.. where of the [V] variables are
latent, and the remaining |V|- - are observed. V is an atomic++p cover (p ' 0), if contains single observed variable, or the the following conditions hold:
(i) There exists set of atomic+00 covers C, s.t. UCECC S PChg (V) and ||C|| 2 11++pp
(ii) There exists set of covers N, s.t. (UNENN) n (Ucec
0, every element in UNENN is
neighbour of V.N.N .N and C are d-separated by V, and [|A'] 2 1++1.
(ii) There does not exist a partition of V = V1 U2, s.t., both 11 and 22 are atomic+0 covers. For brevity atomic+0 cover is often referred to atomic cover.
We allow the definition to take different values of p as we can benefit from more pure children for structure identification (as shown in Theorem D]. Yet, in this paper, we mainly focus on atomic covers (i.e., atomic+0 covers). We note that if V satisfies the definition of atomic+1 I cover, it also satisfies the definition of atomic cover. In other words, a theorem that holds under atomic covers will also hold under atomic+1 covers. In the definition above, each observed variable is treated as an atomic cover, e.g., {X1} in Figure]] [ We define atomic cover as the minimal identifiable unit with the rationale that, when two or more latent variables share exactly the same set of neighbors (e.g., Li and L2 in Fig. 6), they can never be distinguished from observational information. Hence, it is more convenient and unified consider them logether in an atomic cover. Examples of atomic covers can be found in Appx. B.3]] Based on the definition of atomic covers, we define a cluster as the set of pure children of an atomic cover, and refer k-cluster to a cluster whose parents' cardinality is k. We further define an operator Sep(X) = Uxex ((X}). We next show that every atomic cover possesses a useful rank deficiency property, which is formally stated in Theorem 7 (proof in Appx. (AII.
Theorem 7 (Rank Deficiency of an Atomic Cover). Let V X SUL L be an atomic cover where X are observed, L are latent, and [V] k. Let X = Sep(X), Xs Sep((x)), and a set of atomic covers C, satisfying (i) in Definition 5 Then rank(E Bew.xxxxxx././Dee.... min( U x1 ||x U Xg)CC.MMD (CMMDeg(C)) (MDeg denotes measured descendants).
= k and <
Example 2 (Example for atomic cover and rank deficiency). Consider an atomic cover in Fig. A(d):
V= V(X2,,Lo) L2). V is an atomic cover, because V has at least 2 pure children and has additional
neighbors, satisfying the conditions in Definition 5 If we take C =
). (( and X = {(X+}}, (xx.
we have vank(Ecux.xuxx (C. (D))) = rank(
Xx)) = = VV. Noted
that both ||C X1I || X''CC' (C)) > 2, so here the rank is deficient.
Theorem establishes the rank rank-deficiency property of an atomic cover. Furthermore, if we can build a unique connection between rank deficiency and atomic covers under certain conditions, then we can exploit rank deficiency to identify atomic covers in a graph. In the following, we present the graphical conditions to achieve identifiability and Theorem & delineates under what conditions the uniqueness of the rank-deficiency property can be ensured.
Condition 1 (Basic Graphical Conditions for Identifiability). Every latent variable belongs to at least one atomic cover in G and for each atomic cover that contains at least one latent variable, any of its children is not adjacent to any of its neighbours.
5

---

Published as a conference paper at ICLR 2024
(a) The underlying graph G. all the observed variahles of which are taken as imput to Phase! I.
(b) Given the CI skeleton from Phase I, variables are parti. tioned two groups, shown the two dashed areas.
(c) Take variables from the first dashed to Phase and 3, use the result to update the CI skeleton.
(d) Do the same thing for the second dasbed area and update all the remaining directions.
Figure 3: An illustrative example of the overall search procedure in Alg.
(a) Take x ((x3)) and
(b) Take and C = ((x)), (Xe)).
Xs)) (1))
(c) Take ((X2) and
(d) Take X and ((x)) (L2, Xeh (X:])
c= ([X)))
C= ((Xe), (.KK),(00 (Xs), (X+)).
Figure 4: An illustrative example of the process of Phase in Alg.
Theorem 8 (Uniqueness of Rank Deficiency). Suppose G satisfies Condition) 7 We farther assume (i) all the atomic covers with cardinality k' have been discovered and recorded, and (ii) there is no collider in G. If there exists a set of observed variables X and set of atomic covers C satisfying X = Sep(Xx X), C n X = 0, and ||C|| +|C||+10 ||x||| = kt + 1, such that (i) For all recorded k' cluster C², |Pag(C)) (ii) rank( rank(Eoux xx MC
n
k. then there exists atomic cover [X|, and UCECC C (V),
= BULUX X in G, with [L|
For better understanding, we provide an illustrative example in Appx. A.12 Basically, Theorem
says that under certain conditions, we can build unique connection between rank deficiency and atomic covers, and thus we can identify atomic covers in a graph by searching for combinations of C and X' that induce rank deficient property. We further introduce Theorem 9 which useful in that provides necessary and sufficient conditions for the existence of latent variables, under Condition
and Theorem [ which provides necessary and sufficient conditions for nonadjacency.
Theorem (Necessary and Sufficient Condition for Existence of Latent Variable). Ifa graph satis- fies Condition 7 then the "if" in Theorem Sannrrem@bbecomeee becomes "if and only if""
Proposition (Necessary and Sufficient Condition for Nonadjacency). In G. ifevyy latent variable belongs to at least one alomic+ I cover, then the "if" in Theorem becomes "if and only if""
Proposition I shows the further identifiability that can be established by assuming one additional pure child of atomic covers. As we mainly focus on atomic covers, the next question is, how to design search procedure that strives to fulfill these conditions, in order to cash out Theorem and Theorem 9 for identifying atomic covers, and consequently the whole latent causal structure.
4.2 METHOD RANK-BASED BASED LATENT CAUSAL DISCOVERY
In this section, we propose computationally efficient and scalable search algorithm identify the latent causal structure, referred to as Rank-based Latent Causal Discovery (RICD), that leverages the connection between graph structures and rank deficiency, as we discussed in previous sections. The search process mainly comprises three phases: (1) Phase 1: FindCISkeleton, (2) Phase 2: Find- CausalClusters, and (3) Phase 3: RefineCaausalllutten., as outlined Alg.
Specifically, Phase is to find the CI skeleton deduced by conditional independence tests over XG. and Phase and Phase 3 are designed such that the conditions in Theorem alre satisfied to the largest extent. in order make use of the unique rank deficiency property to identify the latent structure. We initiate the process by finding the CI skeleton first, for the reason as follows. According to Theorem 9 latent variables exist iff conditions (i)-(v) in Theorem 5 are satisfied, while conditions

---

Published as a conference paper at ICLR 2024
Algorithm 1: The overall procedure for Rank-based Latent Causal Discovery (RICD).
Input Samples from all observed variables X5
Output: Markov equivalence class g'
def Latent Wwariabbleasssllisseeeeeeee
Phase I: g' FindCISkeleton
(Algorithm 2
for Each 2 a group of overlapping maximal cliques, in G' do
Sct an empty graph g", X UQeo Q, No = {N 3X E Xo s.t. N, X are adjacent in } Phase 2: G" = Find[CussilCuusters("" ( (Algorithm 3
Phase 3: g" = RefineCausalClussteede
X U No) (Algorithm
Transfer the estimated DAG g" to the Markov equivalence class and update g' by g":
Orient remaining causal directions that can be inferred from structures;
return g"
(i)-iiv) can be directed inferred from the CI skeleton. Therefore, there is no need to consider all variables in Xg as inputs for Phases 2 and Instead, we make use of the CI skeleton over Xg and select some groups of observed variables as inputs into Phases 2 and 3, where variables in each group together have the potential to satisfy conditions (i)-(iv). This also benefits the computational efficiency as the Phase 2 and 3 for different groups can be done in parallel.
An example of the entire search process is illustrated in Fig- 3 After Phase 1, variables are parti- tioned into groups shown in Fig. 3 (b). For each group (dashed area in Fig. 3 (b)), we conduct Phases 2 and 3, and have the final result shown in Fig. 3 (d). Details of each phase are given below.
4.3 PHASE I: FINDING CI SKELETON
The objective of Phase 1 is to find the CI skeleton over Xg by utilizing CI relations. To this end, we employ the first stage of the PC algorithm (Spirtes et al. 2000),, with the difference that replace all CI tests with rank tests, according to the following Lemma 10 (proof in Appx. 19.).
Lemma 10 (D-separation by Rank Test). Suppose a linear latent causal model with graph G. For disjoint A, B, C E Xg. C Cd-separates A and B in graph G. if and only if
= |C]. We summarize the procedure of Phase in Alg. 2 in the appendix. Although, asymptotically, using CI and rank information will provide the same d-separation result over observed variables, we use rank instead of CI in Phase I just for the purpose of having a unified causal discovery framework with rank constraints (as Phases 2 and 3 are also based on rank).
Given the CI skeleton G' (result from Phase 1), the next step is to find the substructures in g' that might contain latent variables. Specifically, Theorem (intreemmiiffmssss us that latent variable exists iff (i)- (v) in Theorem 5 holds, where (i)-ivv can be directly inferred from the CI skeleton. Specifically, consider all the maximal cliques Q in g' (a clique is set of variables that are fully connected and a maximal clique is a clique that cannot be extended), s.t., |Q| 2 3. We then partition these cliques into groups such that two cliques Q1, Q2 are in the same group if |Q Q2 2 2. Finally, for each group of cliques Q (as in line 3 in Alg. []]. we combine them to form a set of variables Xe Q. We further determine the neighbour set for each Xº. as No = {N 3X X s.t. X are adjacent in CI skeleton G'}. It can be shown that variables that satisfy (i)-(iv) will be the same set with Xe No (examples and proof in Appx. B.4]. Therefore our next step is to take each X U No separately as input to our Phases 2 and 3, detailed as follows.
4.4 PHASE 2: FINDING CAUSAL CLUSTERS
In this section, we introduce the second phase of our algorithm, FindCausalClusterr, summarized in Alg. Bl in the appendix and illustrated in Fig. 4 The objective here to design an effective search procedure to find combinations of sets of covers C and X' (as defined in Theorem B). such that rank deficiency holds and all the conditions required in Theorem R are satisfied. To be specific, given Condition I Theorem g further requires that (i) when
searching for k-clusters, all the k'- clusters, K" < have been found and recorded, and that (ii) there is no collider in G. We next introduce the key designs for that end, accompanied by examples.
As for the requirement (i), we design our search procedure such that it starts with ho = 1. If a k- cluster is found, update the graph and reset k to 1; otherwise, we increase & by 1 (as in line 5 Alg B This ensures to a large extent that when searching for a k-cluster, all clusters such that k' k, can be found, in order to fulfill the requirement (i).
7

---

Published as a conference paper at ICLR 2024
Regarding the requirement (ii), we need to ensure that all the rank deficiencies are from atomic covers, rather than from colliders. One could directly assume the absence of colliders in the under- lying G. but this would impose rather strong structural constraints and thus limit the applicability of a discovery method. Therefore, we add a collider check function NoCollider defined in Alg. [5) together with the designed search procedure allow incorporating colliders timely such that they will not induce unexpected rank deficiency anymore. With these two designs, we only rely on a much weaker condition about colliders, as Condition under which it can be guaranteed that our search procedure will not be affected by the existence of colliders (proof in Appx. A.18).
Condition 2 (Graphical condition on colliders for identifiability). In a latent graph G. if (i) there exists a set of variables C such that every variable in C is a collider of two atomic covers V1, V2. and denote by A the minimal set of variables that d-separates V1 from V2. (ii) there is a latent variable in 11, 22, C or A, then we must have |C
A Z |1|++V
We summarize the whole process of Phase 2 in Alg. and provide an illustration in Fig. 4 where the input variables are from the left dash area in Fig. 3 (b). For a better understanding, please refer to Appx. B.5 for a detailed description of key steps and illustrative examples.
4.5 PHASE 3: REFINING CAUSAL CLUSTERS
In Phase 2, we strive to fulfill all required conditions such that we can correctly identify causal clusters and related structures. However, there still exist some rare cases where our search cannot ensure the requirement (i) in Theorem g In this situation, Phase 2 might produce big cluster in the resulting G' that should be split into smaller ones (see examples in Appx. B. Fortunately, the incorrect cluster will not do harm to the identification of other substructures in the graph, and thus we can employ Phase 3 to characterize and refine the incorrect ones, by making use of the following Theorem [1] (proof in Appx. Ap.7.A..1].
Theorem 11 (Refining Clusters). Denote by G' the output from FindCausalClusters and by G the true graph. For an atomic cover V in G'. if V is not a correct cluster in G but consists of some smaller clusters, then V can be refined into correct ones by FindCausaCClusters( X), where & is got by deleting V, all neighbors of V that are latent, and all relating edges of them, from G'.
To be specific, we search through all the atomic covers V in g', the output of Phase 2, and then perform FindCausalClusterss($, ' X), where & is defined as in Theorem 11 With this procedure, we can make sure that all the found clusters in G' are correct as in G. We summarized Phase in Alg. and provide illustrative examples in Appx. B.6
5 IDENTIFIABILITY THEORY OF CAUSAL STRUCTURE
Here show the identifiability of the proposed RLCD algorithm. Specifically, RLCD asymptot- ically produces the correct Markov equivalence class of the causal graph over both observed and latent variables under certain graphical conditions, up to the minimal-graui operator Omin and skeleton operator (-).) They are defined in Appx. A.4 following but differently with Huang et al. (2022). Omin(.) is absorb redundant latent variables under certain conditions and (.(.) is to intro- duce edges involving latent variables if certain conditions hold, and we note that the observational rank information is invariant to these two graph operators (examples in Appx. B.12 We summarize the identifiability result in Theorem 12 along with Corollary [ (proof in Appx. AA.18.
Theorem 12 (Identifiability of the Proposed Alg. D.. Suppose G is a DAG associated with a Linear
Latent Causal Model that satisfies Condition 7 and Condition - Algorithm 7 can asymptotically identify the Markov equivalence class of Omia (9))).
Corollary 1. Assume linear causal models. Asymptotically, when no latent variable exists, Alg. D output is the same as that of PC; as another special case, when there is no edge between observed variables, the output of Alg. 7 is the same as that of Hier. rank (Fuang et al. 2022).
6 EXPERIMENTS
We validate our method using both synthetic and real-life data. In finite sample cases, we employ canonical correlations (Anderson, 1984) to estimate the rank (detailed in Appx.
Synthetic Data Specifically, we considered different types of latent graphs: (i) latent tree models (Appx. B. 8. (ii) latent measurement models (Appx. B.9), and (iii) general latent models (Fig. and Appx. B. 10 The causal strength is uniformly from 1-10, 10), and the noise is either Gaussian or uniform (which is for RCD and GIN). We propose use the following two metries: (1) F1 score of
8

---

Published as a conference paper at ICLR 2024
Table 2: Fl scores (mean (standard deviation)) of compared methods on different types of latent graphs.
Fl score for skeleton among all variables Vg (both X and Lg)
Algorithm
Ours
Hier. rank
PC
FCI
GIN
RCD
2k
0.84 (0.11)
0.58 (0.01)
0.36 (0.01)
0.37 (0.01)
0.37 (0.03)
0.24 (0.04)
Latent+tree
5k
0.92 (0.05)
0.60 (0.01)
0.37 (0.00)
0.37 (0.01)
0.41 (0.03)
0.33 (0.00)
10k
0.98 (0.02)
0.60 (0.01)
0.37 (0.00)
0.38 (0.02)
0.41 (0.03)
0.33 (0.01)
2k
0.81 (0.12)
0.52 (0.05)
0.44 (0.01)
0.38 (0.02)
0.40 (0.02)
0.26 (0.03)
metent+
5k
0.88 (0.11)
0.52 (0.05)
0.49 (0.01)
0.40 (0.01)
0.46 (0.03)
0.29 (0.01)
10k
0.91 (0.09)
0.53 (0.05)
0.49 (0.01)
0,40 (0.01)
0.47 (0.05)
0.34 (0.04)
2k
0.66 (0.01)
0.44 (0.02)
0.31 (0.01)
0.25 (0.02)
0.30 (0.04)
0.32 (0.03)
Latens general
5k
0.72 (0.03)
0.45 (0.03)
0.32 (0.01)
0.28 (0.02)
0.38 (0.04)
0.34 (0.02)
10k
0.80 (0.05)
0.45 (0.04)
0.32 (0.01)
0.28 (0.02)
0.35 (0.01)
0.36 (0.01)
[05)
100
Extraversion
Neuroticlam
Openness
LOO
[04]
L6
[C4] (tep
HO
[KG] Sougg
Conscientiousness
[OS
(A20 Heateseer
Agreeableness
-JCCI
[C3R]
(CR) Y000
JVQ
Figure 5: Identified latent graph on Big Five personality dataset.
skeleton among observed variables Xg. (2) F1 score of skeleton among all variables Vg. We con- sider combinations and permutations of latent variables during evaluation (detailed in Appx. B.15 together with definition of F1 score).
We compared with many competitive baselines, including (i) Hier. rank (Huang et al. 2022). (fi) PC (Spirtes al., 2000). (iii FCI (Spirtes et al.] 2013), (iv) RCD (Maeda & Shimizu 2020), and (v) GIN (Xie et al. 2020). The results are reported in Tab. 2 and Tab. 3 where we run experiments with different random seeds and sample sizes 2k, 5k, and 10k. Our proposed RLCD gives the best results
on all types of graphs, in terms of both metrics, with a clear margin. This result serves as strong empirical support for the identifiability of latent linear causal graphs by our proposed method.
Real-World Data To further verify our proposed method, we employed real-world Big Five org// It consists of 50 personality indi- cators and close to 20,000 data points. Each Big Five personality dimension, namely, Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (O-C-E-A-N), are measured with their own 10 indicators. Data is processed to have zero mean and unit variance. We employ the proposed method to determine the Markov equivalence class and employ GIN (Xie et al. 2020 to further decide other directions between latent variables (more details in Appendix[B.11
Personality dataset
We analyzed the data using RLCD, producing a causal graph in Fig 1that exhibits interesting psy- chological properties. First, most of the variables related to the same Big Five dimension are the same cluster. Strikingly, our result reconciles two currently deemed distinct theories of personal- ity: latent personality dimensions and network theory (Cramer et al., 2012 Wright 2017). We see groups of closely connected items that are predictable from latent dimensions (LI, L2, L3), interac- tions among latents (L1-+L6>L33 L1-+22+L3 LL3), and latents influencing the same indicators (LI, L3). We also observe plausible causal links between indicators (c.g.. 02-04 and 01-08). We ar- gue that our findings are consistent with pertinent personality literature, but more importantly, offer new, plausible explanations as to the nature of human personality (detailed analysis in Appx. B.18
7 DISCUSSION AND CONCLUSION
We developed a versatile causal discovery approach that allows latent variables to be causally-related in a flexible way, by making use of rank information. We showed the proposed method can asymp- totically identify the Markov equivalence class of the underlying graph under mild conditions. One limitation of our method is that it cannot directly handle cyclic graphs, and as future work, we will extend this line of thought to cyclic graphs and to nonlinear models.
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENT
This material is based upon work supported by the Al Research Institutes Program funded by the National Science Foundation under AI Institute for Societal Decision Making (AI-SDM), Award No. 2229881. The project is also partially supported by the National Institutes of Health (NIH) under Contract R01HL159805, and grants from Apple Inc., KDDI Research Inc., Quris AI, and Infinite Brain Technology.
REFERENCES
Jeffrey Adams, Niels Hansen, and Kun Zhang. Identification of partially observed linear causal models: Graphical conditions for the non- gaussian and heterogeneous cases. Advances Neural Information Processing Systems, 34, 2021.
Raj Agrawal, Chandler Squires, Neha Prasad, and Caroline Uhler. The decamfounder: Non-linear causal discovery in the presence of hidden variables. arXiv preprini arXiv:2102.0792), 2021.
Sina Akbari, Ehsan Mokhtarian, AmirEmad Ghassami, and Negar Kiyavash. Recursive causal struc- ture learning in the presence of latent variables and selection bias. In Advances in Neural Infor- mation Processing Systems, volume 34, pp. 10119-10130, 2021.
Animashree Anandkumar, Daniel Hsu, Adel Javammard, and Sham Kakade. Learning linear bayesian networks with latent variables. In International Conference on Machine Learning, pp- 249-257, 2013.
T. W. Anderson. An Introduction to Multivariate Statistical Analysis. 2nd ed. John Wiley & Sons, 1984.
Ruichu Cai, Feng Xie, Clark Glymour, Zhifeng Hao, and Kun Zhang. Triad constraints for learning causal structure of latent variables. In Advances in Neural Information Processing Systems, pp. 12863-12872, 2019.
Chandler Squires. causaldag: creation, manipulation, and learning of causal models, 2018. (tRtsstts://qiit com/utto/aeoooooo wtt.uuraa//cauuaaaaaaaa
V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, Rank-sparsity incoherence for matrix decomposition. SIAM Journal on Optimization, 21(2):572-50 2011.
V. Chandrasekaran, P. A. Parrilo, and A. S. Willsky. Latent variable graphical model selection via convex optimization. Annals of Statistics, 40(4):1935-196)) 2012.
Zhengming Chen, Feng Xie, Jie Qiao, Zhifeng Hao, Kun Zhang, and Ruichu Cai. Identification of linear latent variable model with arbitrary distribution. In Proceedings 36th AAAI Conference Artificial Intelligence (AAAI), 2022.
David Maxwell Chickering. Learning equivalence classes of bayesian-network structures. The Journal of Machine Learning Research, 2:445-498. 2002a.
David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research, 3(Nov):507-554, 2002b.
David Maxwell Chickering. transformational characterization of equivalent bayesian network structures. arXiv preprint urXiv: 2013.
Diego Colombo, Marloes H Maathuis, Markus Kalisch, and Thomas S Richardson. Learning high- dimensional directed acyclic graphs with latent and selection variables. The Annals of Statistics, pp. 294-321, 2012.
Angélique OJ Cramer, Sophie Van der Sluis, Arjen Noordhof, Marieke Wichers, Nicole Geschwind, Steven H Aggen, Kenneth Kendler, and Denny Borsboom. Dimensions of normal personality as networks in search of equilibrium: You can't 't like parties you don't like people. European Journal of Personality, 26(4):414-431 2012.
10

---

Published as a conference paper at ICLR 2024
Petru Lucian Curseu, Remus Ilies, Delia Virgä, Laurenfiu Maricujoiu, and Florin A Sava. Person- ality characteristics that are valued in teams: Not always "more is better"? International Journal of Psychology. 54(5):638 649, 2019.
Haoyue Dai, Peter Spirtes, and Kun Zhang. Independence testing-based approach to causal discov- ery under measurement enror and linear non-gaussian models. Advances in Neural Information Processing Systems, 35:27524-27536. 2022.
Yanming Di. t-separation and d-separation for directed acyclic graphs. preprint, 2009.
Doris Entner and Patrik O Hoyer. Discovering unconfounded causal relationships using linear non-gaussian models. In JSAI International Symposium on Artificial Intelligence, pp. 181-195. Springer, 2010.
Thomas Gatzka. Aspects of openness as predictors of academic achievement. Personality and Individual Differences, 170:1 10422, 2021.
William G Graziano and Renée M Tobin. Agreeableness: Dimension of personality or social desir-
ability artifact? Journal of personality, 70(5):695-725 2002.
Pierce J Howard and Jane Mitchell Howard. The owner's manual for personality at work: How the Big Five personality traits affect performance, communication, teamwork, leadership, and sales. Center for Applied Cognitive Studies (CentACS), 2010.
Patrik O Hoyer, Shohei Shimizu, Antti J Kerminen, and Markus Palviainen. Estimation of causal effects using linear non-gann -an-gssssian causal models with hidden variables. International Journal of Approximate Reasoning, 49(2):362-33 378, 2008
Patrik O Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlin- ear causal discovery with additive noise models. In Advances in neural information processing systems, pp. 689-696 2009.
B. Huang", K. Zhang". J. Zhang, R. Sanchez-Romero, C. Glymour, and B. Schölkopf. Causal discovery from heterogeneous/noosaattonnyy data. In JMLR, volume (89))), 2020.
Biwei Huang, Charles Jia Han Low, Feng Xie, Clark Glymour, and Kun Zhang. Latent hierarchical causal structure discovery with rank constraints. arXiv preprint arXiv:2210.01798. 2022.
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. Learning latent causal graphs via mixture oracles. Advances in Neural Information Processing Systems, 34, 2021.
Erich Kummerfeld and Joseph Ramsey. Causal clustering for 1-factor measurement models. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and ACM, 2016.
Data Mining. pp. 1655 1655-1664.
Huy Le, In-Sue Oh, Steven B Robbins, Remus Ilies, Ed Holland, and Paul Westrick. Too much of a good thing: curvilinear relationships between personality traits and job performance. Journal of Applied Psychology, 96(1):113 13,, 2011.
Wendy Lord. NEO PI-R: A guide interpretation and feedback in work context. Hogrefe, 2007.
Takashi Nicholas Maeda and Shohei Shimizu. Red: Repetitive causal discovery of linear non- gaussian acyclic models with latent confounders. In International Conference Artificial Intel- ligence and Statistics, pp. 735-745. PMLR, 2020.
Christopher Meek. Causal inference and causal explanation with background knowledge. arXiv preprint urXiv:1302. 4972, 2013.
J. Pearl. Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan kaufmann, 1988
Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, New York, NY, USA, 2000. ISBN 0-521-77362-8.
11

---

Published as a conference paper at ICLR 2024
Judea Pearl. The seven tools of causal inference, with reflections machine learning. Communi- cations of the ACM, 62(3):54-00) 60, 2019.
Saber Salehkaleybar, AmirEmad Ghassami, Negar Kiyavash, and Kun Zhang. Learning linear non- gaussian causal models in the presence latent variables. Journal of Machine Learning Research, 21(39):1-24, 2020.
Shohei Shimizu, Patrik O. Hoyer, Aapo Hyvärinen, and Antti Kerminen. A linear non-gaussian acyclic model for causal discovery. J. Mach. Learn, Res., 7:2003-2030, December 2006a. ISSN
1532-4435.
Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006b.
Shohei Shimizu, Patrik O Hoyer, and Aapo Hyvärinen. Estimation of linear non con-gaussian acyclic models for latent factors. Neurocomputing, 72(7-9):2024-2022) 2009.
Ricardo Silva, Richard Scheine, Clark Glymour, and Peter Spirtes. Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7(Feb):191-246, 2006.
Peter Spirtes. An anytime algorithm for causal inference. In International Workshop on Artificial Intelligence and Statistics, pp. 278-285. PMLR, 2001.
Peter Spirtes. Calculation of entailed rank constraints in partially non-linear and cyclic models. In Proceedings of the Twenty -inth Conference on Uncertainty in Artificial Intelligence, pp. 606- 615, AUAI Press, 2013.
Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction, and search. MIT press, 2000.
Peter Spirtes, Clark Glymour, Richard Scheines, and Robert Tillman. Automated search for causal relations: Theory and practice. 2010.
Peter I. Spirtes, Christopher Meek, and Thomas S Richardson. Causal inference in the presence of latent variables and selection bias. arXiv preprint /rXiv:/302.440 2013.
Seth Sullivant, Kelli Talaska, and Jan Draisma. Trek separation for gaussian graphical models. arXiv:08/2.1938,, 2010.
Tatsuya Tashiro, Shohei Shimizu, Aapo Hyvärinen, and Takashi Washio. ParceLiNGAM: causal ordering method robust against latent confounders. Neural Computation, 26(1):57-83, 2014.
Sofia Triantafillou and loannis Tsamardinos. Constraint-based causal discovery from multiple in- terventions over overlapping variable sets. The Journal of Machine Learning Research, 16(1): 2 2147-2205, 2015.
S. Wang- Causal clustering for 1-factor measurement models data with various types. arXiv preprint arXiv:2009.086600 2020.
Aidan GC Wright. Factor analytic support for the five-factor model. The Oxford handbook of the five factor model, pp. 217-242, 2017.
Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang. Generalized independent noise condition for estimating latent variable causal graphs. In Advances in Neural Information Processing Systems, pp. 14891-14902, 2020.
Kun Zhang and Aapo Hyvärinen. On the identifiability of the post-nonlinear causal model. In Proceedings of the twenty-fithh conference on uncertainty in artificial intelligence, pp. 647-655.
AUAI Press, 2009.
Nevin I. Zhang. Hierarchical latent class models for cluster analysis. The Journal of Machine Learning Research, 5:697-723, 2004.
Yujia Zheng, Biwei Huang, Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang. Causal-leam: Causal discovery in python. arXiv preprint arXiv:2307.16444 16405, 2023.
12

---

Published as a conference paper at ICLR 2024
Appendix
Organization of Appendices:
Section A: Definitions, Examples, and Proofs
Section A.1
Subcovariance Matrix and Definition Descendants.
Section A.2
Treks, T-separations, and Examples.
Section A.3
Definition of CI Skeleton
Section A.4
Definition of Rank-invariant Graph Operator.
Section A.5
Max-Flow-Min-Cur Lemma
13 for Treks
Section A.6
Example for Theorem
Section
A.7
Example for Theorem
Section
Proof of Theorem
Section
A.9
Proof of Theorem
Section
A. 10
Proof of Theorem 0
Section
Proof of Theorem
Section
A.
Example for Theorem
8
Section
Proof Theorem
Section
A.
Proof of Theorem
Section
A.
Proof Lemma
Section
A
Proof of Theorem
Section A.
Proof of Theorem
Section
A.19
Description of the rank test that we employ.
Section B: Graphs, Illustrations of algorithms, and more information on datasets
Section B.1
Examples of Grpahs that Each Method Can Handle.
Section B.2
Example of Violation of faithfulness
Section B.3
Example of Atomic Cover.
Section B
Example for Phase
Section B.5
Detailed Description and Example for Phase 2.
Section B.6
Example for Phase
Section
B.
Graph examples for model that has only observed variables.
Section
8.8
Graph examples for latent tree model.
Section
B.9
Graph examples for latent measurement model.
Section
B
10
Graph examples for general latent model.
Section B
Example for considering colliders in Phase
Section B 12
Examples for graph operators.
Section B 13
Examples for graphical relations between covers.
Section B. 14
Discussions checking colliders completely.
Section B 15
Evaluation metric details.
Section B. 16
More details of experiments synthetic data.
Section B 17
Detailed information of the Big Five personality dataset.
Section B. 8
Detailed analysis of the result from the Big Five dataset.
Section C: Related work
Section Related work.
Section D: Additional Information (added during rebuttal)
13

---

Published as a conference paper at ICLR 2024
Table 3: F1 scores (mean (standard deviation)) of compared methods on different types of latent graphs. Fl are calculated only for edges between observed variables Xg in this graph. Hier.rank and GIN assume that observed variables are not directly adjacent so their performance reported
Fl score skeleton among X.
Algorithm
Ours
Hier. rank
PC
FCI
GIN
RCD
2k
0.79 (0.16)
0.46 (0.02)
0.47 (0.01)
0.30 (0.03)
tatent+tree
5k
0.86 (0.10)
0.44 (0.00)
0.46 (0.02)
0.38 (0.01)
10k
0.97 (0.04)
0.44 (0.00)
0.46 (0.01)
0.39 (0.02)
2k
0.84 (0.11)
0.50 (0.02)
0.53 (0.02)
0.30 (0.02)
matens+
5k
0.93
(0.08)
0.49 (0.01)
0.51 (0.02)
0.32 (0.02)
10k
0.95 (0.05)
0.48 (0.02)
0.51 (0.02)
0.42 (0.09)
2k
0.68 (0.02)
0.44 (0.01)
0.39 (0.01)
0.39 (0.06)
Latent general
5k
0.71 (0.03)
0.43 (0.01)
0.41 (0.04)
0.44 (0.05)
10k
0.78 (0.06)
0.45 (0.01)
0.41 (0.02)
0.44 (0.01)
A DEFINITIONS, EXAMPLES, AND PROOFS
A.1 SUBCOVARIANCE MATRIX AND DEFINITION OF DESCENDANTS
Ea,b B refers subcovariance over A and B. E.g., [(x,,))((X) is 2 x 1 matrix whose entry (1, 1) is Cov(X1 Xa) and entry at 1) is Cov(X2, Xa). ΣA. refers to subcovariance over A and = AU E.g., (x is also 2 x matrix whose entry at (1, 1) is Cov X3) and entry at (2, 1) is Cov(X2, X3).
B. Specifically,
As for the definition of descendants, descendant is node that can be reached by following one or more directed edges from a given node, and this definition excludes the node itself.
A.2 TREKS, T-SEPARATIONS, AND EXAMPLES
X1
Figure 6: An example graph show treks, t-separations, and rank subcovariance matrix.
L
X2
X
X
X,
X,
X=
Example 3 (Example of Treks). Figure 6 there are four treks between X3 and X4: (i) (PI, Po)((s t L,, L X4), (ii) (P,, P2)=(Xs + L2. L2 X4), (iii) Po), + L1 &- X1. X1 + L2 X4). (iv) (2),P))=(X, + L2 + X1, X1 + L1 + Xa). For adjacent variables such X1 and X2, there must exist at least trek (P1, P2) =xx, X1 X2) between them.
Example 4 (Example of Tree-separations). In Figure 6 X3 and Xa can be r-separated by (L1,,2)), (), D), as for all the treks (i)-(iv) in Example 3 either P1 contains vertex in {L1, L2] or P2 contains vertex in D. Similarly, X3 and X4 can also be r-separated by (0,, L2)). However, the most simple way i-separate X3 and Xx is by ([Xa), 0) or (0, (Xa)).
Example 5 (Example of caleulating rank). As shown in Example 4 the minimal way I-separate X3 and Xa is by (Ca, CB) = ((Xa). 0) or (0, (X+)), and thus min CAl + |al = 1. Therefore, 1. Now suppose we want calculate rank(E xxx,, minimal way 1-separate (X,,X4, Xs) and (X,, X6. X7) is by ((L2, L2), 0) (or (0, (L,,12)) the rank is (L, L2) + -0]=2.
rank(E (x(x))
As the
14

---

Published as a conference paper at ICLR 2024
Figure 7: illustrative example that highlights the motivation for using rank. When using CI, we cannot de- duce that fX,, X2} f and (Xa, X4 J d-separated by as latent, while by using rank we can.
X
L
X2
X
A.3 DEFINITION OF CI SKELETON
Definition 6. A CI skeleton of Xg is an undirected graph where the edge between X1 and X2 exists iff there does not exist set of observed variables C such that X1, X2 & C and X1 IL X2|C.
Examples of CI skeleton can be found in Example)
A.4 DEFINITION OF RANK-INVARIANT GRAPH OPERATOR
The definitions
follows with examples in Appx. B.12
Given two atomic covers L, P in G, can merge L P if the following conditions hold: (i) LiLk is the pure children of (ö) all elements of L and P are latent and [L| = [P) and (iti) the pure children L form a single atomic cover, or the siblings of form single atomic cover. We denote such an operator as minimal- minimal-graph operator O minn()
Definition 7 (Minimal-Grapp Operator (Huang et al. 2022
Definition (Skeleton Operator). Given an atomic covers V in a graph G. do the following two operations. (i) add directed edges such that V1,, V2 E V, V1 and V2 are adjacent and the whole graph still acyclic. (ii) consider as set of atomic covers such that VS
S C V. Let C = PChg(V) Uses PSh We can draw edges from elements in V to elements in C. We
denote the combination of (i) and (ii) as the skeleton operator (g()).
A.5 MAX-FLOWW-MIN- LEMMA FOR TREKS
Lemma 13 Max-Fllw-Mint for Treks (Sullivant et al. 2010 The minimal |Cal |CB s.t., (Cal, CB) t-separates A from B, equals to the maximum number of non-overlapping (no sided intersection) treks from A to B.
A.6 EXAMPLE FOR THEOREM 4
X
X
X
X₂
X₂
X₂
X₂
x
X,
X
(a) Illustrative graph G1 for The- orem
(b) Illustrative graph G2 The- orem
(c) Illustrative graph Ga The- orem
Figure 8: Illustrative figures for Theorem 4
Example 6. In Figure (a), we can employ Theorem 4 to check whether X1 and X2 are adja- cent. Specifically, let
(Xa} and = (Xa), then the condition in Theorem is satisfied, i.e., rank(E [Bu())) = rank(EA,B) and rank(EAutx. X,), [ + 2. Therefore X1 and X2 are not adjacent.
When it comes to Figure 8 (b). the condition does not hold, and thus cannot conclude that X1 and X2 are not adjacens.
Figure 8 (c) is example show that lank(EAu(x,, X-) B,XXX, rank(LA..)) + 2 in the condition is important the theorem. We need this condition to ensure than treks between A and B do not rely on X1 or Xz. Otherwise, in Figure (c), if we only check rank(E (UU rank(EA we will found that it holds if we take A (Xg} and B (Xa). Therefore, we will
mistakenly conclude that X1 and X2 are not adjacent.
15

---

Published as a conference paper at ICLR 2024
4
Ls
X1
X
X
X2
X
X;
X
%
(a) Gr.
(b) G2.
Figure 9: Illustrative figures for Theorem 6
X₁
X₂
X
X₂
X1
X2
X₁
X4
X
X
X
X
(a) G1
(b) gi-
(c) Gi'
Figure 10: Examples to show that 91 cannot be identified as 91 94 have the same observational rank information.
A.7 EXAMPLE FOR THEOREM 6
Example 7. Take Figure (a) as an example. Suppose we aim to calculate rank (EA,B), where A (L2, L3} and B (X1, X2). We can employ the pure children of A, C (X3, X7) to do B.an))=
so. Specifically, we have
rank(>) (x (x)....X));(X(,...
1.
However, in Figure (b), it is not the case. this is because the number of pure children of {L2, L3} is not enough and thus rank(EA,c) + |A]. In this case C {X3} cannot work surrogate for calculating rank(E (A.
A.8 PROOF OF THEOREM 4
Proof. Assume that the condition holds and suppose rank(EA,n) = t, which means that there exist non-overlapping treks between A and B. By rank(E Xx Bu(X,
rank(EA,R)) + 2, we further have that these treks do not necessarily travel across X1 or X2. As such, if Xı and X2 are adjacent, then must have rank(EAuRp) BJ())) + rank (EA.B), which contradicts with the condition. Therefore X₁ and X2 canmot be adjacent.
Here we show that the condition generalizes PC's condition. SGS and PC (Spirtes et 2000) pro- posed: there exist set of observed variables C C Xg. X1. X2 & C, such that X1 II. X2 C. This con- dition can expressed the form of Theorem 4 with
B, because we have X X2|C C iff rank(E (A, (X.) rank(EA.B) Plus, (B) rank (A.B) + is always true when A
x2).
A.9 PROOF OF THFOREM 5
We first introduce Lemma 14 as follows to show that when there is no latent variable, the rank information should be aligned with what CI skeleton provides.
Lemma 14. When there is no latent variable in the graph, rank and CI are equally informative about the underlying structure, i.e. the ank-eqqivalence class and the Markov equivalence class are the same when there is no latent variable.
Proof. (i) As all d-sep can be stated by rank according to Lemma I0 using rank information is able arrive the markov equivalence class. (ii) Every element in the markov equivalence class
16

---

Published as a conference paper at ICLR 2024
distributionally equivalent in terms of second order statisties. Therefore, using information from the rank of the covariance matrix cannot differentiate elements in the markove equivalence class. Taking (i) and (ii) together, we have that the rauk-equivalenee class and the Markov equivalence class the same when there is no latent variable.
Bellow is the proof of Theorem
Proof. First we assume that there latent variable and we have that the CI skeleton among observed variables is also the skeleton of the true underlying graph G. By (ii) and (ii), we have (ii") distinet A1, A2 E A, A1, A2 are adjacent in 9, (iii') AA E A, B, A, are adjacent in g. By (ii') and (iii",, it must hold that (iv") (iv). Therefore, there must exist at least one latent variable.
= | CJ. However, (iv') contradicts with
A.10 PROOF OF THEBRREM 6
Proof. By Theorem and Lemma 13 we have rank (EA,B) equals the maximum number of non- overlapping trek paths from A to B, As C are pure children of A and element in B are descen- dants C, every trek from to must travel across A. Therefore, rank(E 2 rank(EC..)) If further have rank(EA, from A to equals the maximum number of non-overlapping trek paths from C to B, and thus rank( (AA,B) rank(E.c.)).
| then the maximum number of non-overlapping trek paths
A.11 PROOF OF THEOREM 7
Proof. As none of elements in Xg C. are descendants of C, by Theorem the minimal way block every trek path between C U X' and (Ng \ (C U is by blocking all the elements of the atomic cover V and all the elements in X. As X subset of V, have (nn(Eeux.(Ax [ = k. L
A.12 EXAMPLE FOR THEOREM g
Example 8 (Example for the uniqueness of rank deficiency in Theorem In Figure if the current k = 2, and we assume that all k = clusters are are found, and no v-structure exists, the rank deficiency would uniquely map to a k 2 cluster. E.g., we take C ([Xe), (X7}) and X {[X2]), we have rank( (eux RUXGACMMMDeDCC rank deficiency uniquely relates to a cover V = X UUU. LL, where == Ux'exx' = (X2), L is latent X 1, and C ((Xo). (X+) are the pure children V.
rank(
(o,,).........
= 2. In this case, this
variable be added with [L
In contrast, if we are searching for k = 2 and a 1-cluster L4 + (X14, Xs} has not been identified, then the condition for Theorem iis satisfied and thus the uniqueness of rank deficiency does not Xoo 2, which is deficient, and yet (X13, X14, X1s} are not from a k = 2 cluster. This is because this rank deficiency is not from k cluster, rather, is from the 1-cluster (X14, X15} with parent L4 that has not been found yet.
hold: e.g., by taking C = [X13, X1a, X15} and X = ={}, we have rank (2 pXr Ku.Xs
A.13 PROOF OF THEOREM 8
Proof. We first show that (a) if ||| t= 0 and, elements from C. are pure children of two or more atomic covers, then we must have rank Lank("cux,xxuu "un (C)) = k +
Proof of (a). As there is no collider between atomic covers, we have that all the elements from C are pure children of atomic covers. Suppose C can be partitioned into CI,..., Cn. where each C, are the pure children of distinct atomic cover in G. If C; are the pure children of an atomic cover with cardinality k' k, then we have ||C,|| < k'. If C, are the pure children of an atomic cover with cardinality k' > k 2 ||C,]] (k > ||C,]| because if & < ||C,|] then all elements of C are from the same cluster), so we also have ||C,|| < K. Therefore, by Lemma 13 and the fact that each atomic cover has A + pure children and l: additional neighbors,
(C)) rank(Eç, (Ec.x; (C)) = Σn ||C,|| = + Therefore, when | = = 0, the rank deficiency property does not hold when elements of C are from different clusters.
have rank(Ecux,a)
17

---

Published as a conference paper at ICLR 2024
(b) When ||-||| = + * O, we consider new graph G", where all variables from || removed (as well as related edges). Assume elements of C are from different clusters and elements of X' are from the same atomic cover. Thus by (a), we have that the maximum number of non-overlapping treks in g" between C and Xg" \ C MDeg (C) is k + - t. Then we add X' with relating edges back to the graph and thus will have || f additional non-overlapping treks between C J X and X U Ag \ C MDeg(C). Therefore, we also have
xxux (.x (eD) = k + 1, when || = t & O and elements of are from different clusters. Similarly, if elements of from the same cluster but not all elements of X' the parents of that cluster, we also have (xx
= k +
Taking (a) and (b) together, we have that rank deficiency holds only if all elements of C are from the same cluster and all elements of X' are the parents of that cluster.
A.14 PROOF OF THEOREM 9
In the proof of Theorem 5
we have already shown the 'if" direction. Now we are going to show the sketch of the proof for the 'only if" direction.
Proof. Suppose there is latent variable in G. According to Condition T must belong to an atomic cover, say V and we suppose that V contains n latent variables in total, rest of which are observed X. According to the definition of atomic cover, has at least n + pure children and n + neighbours that are distinct with the n + 1 pure children. Assume that all of them are observed. Then we can simply take A as the pure children, B as the neighbours, C as X, and thus conditions (i)-(v) will be all satisfied. If some of the pure children neighbors of V are latent, pure children instead (if the pure children are still latent, use the pure children of the pure children and finally will find enough observed pure children/ddeseeddant,, leaf nodes). Thus the conditions (i)-(v) can also be satisfied. Therefore, if there is at least a latent variable, then there must exist disjoint A.B, and C, such that (i)-(v) hold.
simply their
latent variables cannot be
A.15 PROOF OF THEOREM 1
In the proof of Theorem 1 have already shown the 'if" direction. Now are going to show the sketch of the proof for the 'nnly if" direction.
Proof.
A.16 PROOF OF LEMMA TO
Proof. By Theorem 2 we have that for disjoint A,B and C, C d-separates A from B. iff there is a partition C CA CB such that (C),CB) t-separates U C from B U C. By Theorem 3
we have that Therefore, C d-separates A from B, iff rank (EAAUCC.B))) CC
< |C). Plus, by the definition of treks, rank([AAc..Bc)) 2
A.17 PROOF OF THEOREM
Proof. The sketch of the proof is as follows.
We first show that fake cover will not influence all other found structures except itself and its neighbors in the result g'. By Lemma 11 in (Huang et al. 2022). we have that a fake cover with observed descendants X in g' implies bond set 9 (whose definition can be found in Huang et al. (222)), ) and there is partition of the rest of the observed variables Xg X into two groups A and B such that A and B are d-separated by the bond set. Suppose this faker cover is V that corresponds to set of latent covers [L,,.., ..., L,) } in G. Then [L1, ..., L., d-separates A and B, and thus during the search we will generate two dummy covers that interact with A and B respectively, during which the rank information will not be mistaken. Then we show that in Phase 3, the fake cover can be corrected. When we refine the fake cover V. we will delete together with its neighbours and thus the two dummy covers will be deleted. Suppose now we searching for k clusters, this
18

---

Published as a conference paper at ICLR 2024
time all the remaining k' clluutes s.t. k' < k have been found, the FindCausalCluster function will not generate fake cluster anymore. Therefore, the output would be corrected.
A.18 PROOF OF THEOREM
First, we show an extension of Theorem to atomic covers.
Lemma 15 (Pure Children as Surrogate for Atomic Covers). Let A LUX be an atomic cover, C S PCha(A) be subset of pure children of A, and B BI,B2 be two sets of variables such that for all E B₁ U B2, & Deg We have rank(E (AAU
rank(Zau(xju(B.)..b.. if
||C|| + X 2 |A].
Proof. By Lemma 13 rank (A(A)((B.) J.B.) BB.) is the max number of non-overlapping treks between {A} {B1} } and B2- If |C||
X A holds, all the treks starting from {A} U [B,] can be extended to treks that start from C U same, which means rank [AAAU
X U B1). and the max number of non-overlapping treks is
Ba) rank ("c((XXX (.B
This lemma informs us that if we correctly found a cover LUX by our rules Algorithm 2 we can calculate the rank relating to A by using its pure children C together with part of the observation of A, i.e., X as surrogates, even though part of A i.e., L, cannot be observed. Note that ||||+||X| 2 |A| always holds it is required when
searching for clusters in Algorithm
Next, we show that when we are searching for combinations aging the checking function NoCollider defined in Algorithm 5 the correctness of Algorithm will not be influenced even though there might exist colliders C.
and in Algorithm by lever-
Lemma 16 (Colliders in C do not harm). Suppose there exist some collider structures in graph g, e.g., there exist two atomic covers V, 1 and V2. with A the minimal set of variables that d-separates V1 from 22, and C as a collider of V1. V2. The correctness of Algorithm 2 will not be influenced by the existence of colliders in C.
Proof. In Algorithm 2 we check whether different combinations of C, X, and M induce rank defi- ciency. Suppose we take C ([V')), (V'2), (C'),, where V S V,, V22 S V2, and C' C C and let R. be V1 V V (i) If |C'| < [R|, and rank deficiency holds, we have VV U 2 C = |C'| + |A]. There- fore, we can detect C' in C by the checking function NoCollider, because by removing C' we have |
IV' U', =
(ii) If C > R, and rank deficiency holds when checking k, we have V 1 U V U U 1 A k+1, which means VVUV2l |
1+ R |||+ CC' < k. Therefore, by the unfolding order in Algorithm C will be taken as the children of V1 and V2 first and thus will not induce incorrect rank deficiency.
Further, under Condition 2 we can show that the correctness of Algorithm will not be influenced even though there might exist colliders in N,, which is summarized in the following Lemma.
Lemma 17 (Under Condition 2 colliders in A do not harm). If Condition 2 holds, i.e., for every collider structures V1. 22 C, and A, we have |C| |A| Z V2I++ + (V2), then the correctness of Algorithm will not be influenced by the existence of colliders in N.
Proof. Consider collider structure V,, 22, C, and A. The potential existence of C in N will not
cause rank deficiency unless is when = |C] + [A]. But under Condition 2 we have |C] + [A] 2
|V1l + |V2,, so in Algorithm 2] the colliders C will be taken as the pure children of V, and V2 first. This holds for every collider structure and thus the correctness of Algorithm 2 will not be influenced.
Now are ready to prove Theorem 12 follows.
19

---

Published as a conference paper at ICLR 2024
Algorithm 2: Phasel: FindCISkeleton (Stage 1 of PC (Spirtes et al. 2000)
Input
Samples from observed variables X.g
Output: skeleton
def Stage/PC(Xg):
Initialize complete undirected graph G' on X repeat
repeat
Select an ordered pair X, that are adjacent in G", s.t,, (X)()) 2 n; Select subset S S Adjgr (X)) s.t., |S] n
If Faank(E (Exguu, (Yic)) = |C], delete the edge between X and Y from G' and record in
Sepset(X, Y) and Sepset(Y, X)
until all X, Y s.t. JAdja (X) (YH 2 n and all S C Adio (XMM), |S] = n, tested.;
n==++l;
10 until adjacent X, s.t., Addig MYYH <;;
11
return G'
Proof. As the existence of colliders in X' will enable more non-overlapping treks, the existence of colliders X' will not induce incorrect rank deficiency. Taking this and the above two Lemmas [6] the existence of colliders between atomic covers will not influence the correctness of Algorithm 2 During the search process of Phase 2, Lemma 15 allows us the rank involving partially observed atomic covers and thus we are able to ileratively find all clusters in G by making use of Theorem g with the direction of some edges undetermined. Further, Theorem 11 allows us to correct clusters induced by the violation of the assumption that when searching & clusters all &' < Is clusters have been found, by Phase 3 in Algorithm Therefore, our Algorithm including Phase 1, 2, 3 can identify the Markov equivalence class of G, up to rank invariant operations min and O,,
into consideration, under Condition aand? 2
Corollary D is directly from Theorem 12 min (O,()) is the Markov equivalence of G so asymptotically the output of RLCD is the same as that of PC.
When there is no latent, the Markov equivalence of
A.19 RANK TEST
We employ canonical correlations (Anderson) 1984) to calculate the rank of covariance matrices.
Denote by a₂ the i-th canonical correlation coefficient between two sets of variables and B, under the null hypothesis rank(CA.B) with N sample size, the statistics - (NN (p + q +
3)/2) Em maxx((A)) log(1 - af) is approximately x² distributed with ([A) - r) [ - r) degrees of freedom.
B ILLUSTRATIONS OF ALGORITHMS AND MORE DETAILS ABOUT DATASETS
B.1 EXAMPLES OF GRPAHS THAT EACH METHOD CAN HANDLE
For causal discovery in the presence of latent variables, traditional wisdom often relies strong graphical constraints for achieving the identifiability of the structure. E.g., Pearl (1988); Zhang (2004) assume that the underlying graph only follows tree structure; Huang et al. (2022) assumes more general latent hierarchical structures but edges among observed variables are not allowed; Maeda Shimizul (020) allows observed variables to be adjacent but requires that all latent vari- ables are mutually independent.
Illustrative graphs allowed by each method are shown Figure II To be specific, (a) is the illus- trative graph allowed by Pearl [1988) Zhang (2004) where each cluster has only one latent variable and observed variables are not allowed to directly related to each other. (b) is the graph allowed by Huang al. (2022) where each cluster can have multiple latent variables. However, observed variable cannot be adjacent to each other and observed variables cannot cause of latent variables.
(c) is the graph allowed by Maeda & Shimizu (2020), where all latent variables required to be
20

---

Published as a conference paper at ICLR 2024
Algorithm 3: Phase2: FindCausalClusters
Input Samples from observed variables X Output: Graph G'
def FindCausalClusters(("" XG):
Active set 8 + NG = (1x1) [Xa]) k
1/ S is set of covers
repeat
G", S. found Search(("' X2, k);
// Only when nothing can found If found then +- else & +- 1 // udner current we add k by
until & is sufficiently large;
return g¹:
def Search(G', Xa, k):
Rank deficiency
0
To store rank deficient combinations
10
for T & PowerSer(S) (from to 0) do
S' + U (UTeTPCho (ST (T)): for L = 0 do
// Unfold 8 to get s'
repeat
14
Draw a set of t observed covers X' C S' Rg;
repeat
Draw set of covers C S s.t. |[C|| + and get (XXU)) if rank(Ecc.x (NUx) and NoCollider(C, x. .V) then Add D:
until all C exhausted;
if o then
for D. E do if |Pag: (D;) k then P + (D)) UX: else Create new latent variahles L, s.t., P - L Pag (Di) U X and
21
[| = h |Pag (D,) UX
23
Update G' by taking elements D. as the pure children of P: P is atomic then Update S + (S) U P.
return S. True:
// Return to search with
until all X' exhausted;
27
return S' False
// Return to search with + k
Algorithm 4: Phase3: RefineCausaCClusters
Input Graph g'
Output: Refined graph
def ReffineCuuslCLussteese Xg):
repeat
Draw atomic cover V from G':
Delete V, neighbours of V that latent, and all relating edges from g' get 9;
g' = FindCausalClustersS
Xx);
until No more V found and all V exhausted;
return
mutually independent. (d) the graph allowed by the proposed method, where all variables are allowed to be very flexibly related to each other.
B.2 EXAMPLE OF VIOLATION OF FAITHFULNESS
Below, we provide a special example where faithfulness does not hold. Suppose the true underlying graph G is X 4 Y 4 Z and X S Z. If the corresponding SCM is parameterized with ab + 0, then the faithfulness assumption is violated. This is because, when ab + c 0, there exists another SCM with a graph G' X م K' that can generate exactly the same observational distribution that of G, and thus from observational data it is impossible to differentiate and g' (which results in rank ( = O). We note that such scenarios are very rare and classical methods like PC (Spirtes) al. 2000) cannot handle these situations either.
21

---

Published as a conference paper at ICLR 2024
Algorithm 5: Function: NoCollider
Input C. R.W
Output: Whether there exists O E s.t., collider of (O) and NoCollider(C, N): do
for c to |C|
Draw C' s.t., |C'| = repeat
if rank(Eeuxxxuu < ||C" U XX] then return False
until all C' exhausted;
return True
4
2
X
X
(a) Illustrative graph allowed by
(b) Illustrative graph allowed by Huang et al. 2022
Pearl 1988 : Zhang 2004).
X1
X
(c) Illustrative graph allowed by
(d) Illustrative graph allowed by the proposed method.
Maeda & Shimizu 2020
Figure 1I: Examples of graphs that are allowed by each method.
B.3 EXAMPLE OF ATOMIC COVER
Figure 12: An example graph for showing atomic covers.
Example 9. Take Figure 12 example. Here (X1), (X2). (X3), (X4), (Xs), (Xo) all atomic covers as they only contain single observed variable. We also have V = {L2, as an atomic cover. To show this, lets check whether conditions (i)-(üi) in Definition 5 are satisfied. For
22

---

Published as a conference paper at ICLR 2024
and ||C|| = k 3. For (ii), can let - ((L1), (Xo]. [X7]), and |M = 3 k 1- 3. For (iii), it can be shown that both {L2} and {Lا} cannot be atomic cover. Therefore. {L2, Lو} is an atomic cover.
(i), can let C = (X4)) (Xs)).
Next, let's show {L1} is also an atomic cover. As As for (i) we can take C = ([L,,L3) with ||c|| =
2 2 k- - + 2, for (ii) we can take N {(X]); {X2]} with ||MI| = 2 2 k 1 = 2, and
(iii) naturally holds as {L1} has a single element. Therefore {L,} is an atomic cover.
From the example it is natural to use a set of covers, i.e. C, to capture the pure children of cover.
that when we take an atomic cover (a set) as the unit, we need
B.4 EXAMPLE FOR PHASE!
We take Figure 3 (a) as an example. Afte Phase 1, we will find the CI skeleton G'. In g', have three maximal cliques that have cardinality2 3. They are (X1,X,, X3, Xa, Xs. Xs),
(X2, X3, X7), and (Xg, X10. X11, X12). Then we partition them into groups such that two cliques
groups of cliques Q ((X,, X2, X3, Xa, Xs, Xo}. (X2, X3, } and 22 = {[Xo, X1o, X11, X12 Given Q and O2. we get 2 Uqe Q (X1, X2, X3, Xs, Xs. X6, X7] and X e. Jqe Q (X9, X10. X11, X12), the corresponding imput to Phase 2 and 3 will be Xe, UNe. = (X1, X2, X3, X4 Xs. Xs. X7, Xs} and Xe, UNe, {Xg, X10, X11, X12, Xs} respectively.
QI, Q2 are in the same group if |Q n Q2 2 2. Thus we have
We next show that if there exist disjoint A, B, C, s.t., (i)(iv) in Theorem hoold, then AUBUCE X
If there exist disjoint A, B. C, s.t., (i)-(iv) in Theorem 5 hold. Then A itself is clique, and for all B E B, A U (B) is also a clique. Plus A and A U (B) have least two common elements as |A| 2 2. Thus, after our processing, A, B will both be subsets of same Xo. Plus, by (iv), C will
be a subset of No. Therefore, we have A AUBUCS _ X
B.5 DETAILED DESCRIPTION AND EXAMPLE FOR PIIASE 2
Our search starts with k 1 and input graph g' (could be empty) over observed variables. Every time we successfully found rank deficiency with rank = k, we update the graph and reset & to 1. On the other hand, if no rank deficiency can be found with current k. we add & by (line 5, Alg 3) as we want to ensure all k' clusters, k' < k. have been found when searching for k; as in Theorem g
During the search procedure, we maintain an active set S, which is set of covers and initialized
as the set of observed covers {(X1),... (Xn3) (line 2, Alg. 3). We test the rank deficiency over different combinations of C and X, drawn from S' where 5 is generated from S by unfolding some existing clusters (lines 10-11, Alg. 2). Introducing S and S' has several merits: (i) When we found a new atomic cover, we want to explore its relation with existing ones. This can be achieved by adding the newly found atomic cover the active set S (illustrated in Example I0].. (ii) According to Theorem g we do not want any descendants of to be on the right side of the cross-covariance matrix when testing the rank. This can be achieved by removing all the children of a newly found atomic cover from the active set S. Taking (i) and (ii) together, we update the active set by S + (S)C) P (line 24, Alg. B). (iii) We unfold S get S', and draw combinations C and X' from S' instead of S. This allows the children of existing clusters to re-appear in C' and X, to facilitate finding new clusters that share parents with existing ones, which will be discussed in detail later.
In addition, to establish a unique connection between rank deficiency and atomic covers, we need to avoid colliders and their descendants to appear in S as the existence of colliders in C or N (N + (2 U C)) might induce rank deficiency that does not indicate correct cluster (example in Appx. B.II. To this end, we take two steps: (1) Every time we found rank deficiency, we further check whether there a collider in C (by the NoCollider function described in Alg. 5) i.c., check
whether there exists O E s.t. O is a collider of CV{(O} and N (line 5 in Alg. 5). If there is a collider, then we ignore the corresponding combination of C and X' (line 17 in Alg. 3). (ii) We perform unfolding on S to get S'. That is, every time we consider T. subset of S, and get S' + (S)T) U (UTer (T)) (lines 10-11 in Alg. 3). This allows us to reconsider the pure children of existing atomic covers when choosing combinations of C and X, and thus, colliders can
23

---

Published as a conference paper at ICLR 2024
be identified (illustrated in Example 10). Taking these two steps together, under the Condition 2 it can be guaranteed that our search procedure will not be affected by the existence of colliders (proof in Appx.p....... A.18).
Example 10 (Example for Phase 2). Consider the graph in Figure 4 We start with finding atomic covers with k- = 1, and we can find that {X3} is a parent of (X&), as in Figure Ma). At this point, no more k clusters can be found, so next we search for k = 2 clusters. Then to identify collider (X7), we only need to consider {X7} and [Xa] together as the children of (X2, X3). After finding such a relationship. we arrive at Figure 4 b). and from now on the collider (X7} will not induce unfavorable rank deficiency anymore (as it is recorded). The next step is to find the relation of
(Xa). (xs)-XXx {X6] with {L2, X2), by taking X' = {{X2)} and C = ([Xa): [Xs]] or {(Xa); {Xs}} or [[xs). (xx,) and thus conclude [[X4).(Xs). (Xx) }
as the pure children of (L2,X2) as in
Fig (tc) Finally we are able to find the relationship of (X1). (.1. (22.222..- {Xa} with {L1). J. by taking
X' } and C as {{x1), 1,, X2)), ((X1), {Xa}), or {[22, [Xa]). b. as in Figure (d)
We here give more detailed example to show the procedure of Phase 2, with the underlying graph G showed in Figure (d).
Step 1. Initialize active set S as ((.,),.., (Xs]]. as 1.
Step 2. Get S' by unfolding S. Currently, S' = ((Xi), (Xa)}. Now let k: = 1 and I. = 1. Draw set of i. observed covers X' c S' n Kg. and draw set of covers C C 8" s.L., ||C|| k - t 1,
and check whether rank deficiency holds. We will find that when X' = {[Xa}} and C = [[Xa]}
rank deficiency holds and there is no collider detected. Therefore we draw a link from X3 to Xs in G', as shown in Figure aa). a). Now update the active set S as (X73).
Step 3. Continue searching with k= = 1, but no more rank deficiency can be found. Therefore, we add & by 1.
Step 4. Unfold S and get S' = ((x1).... {Xa]]. Now, kt = 2 and t = By drawing X and C, we will find that when X' = {{x)), [Xa}} and C = {[X+}}, rank deficiency holds and there is no collider detected. Therefore, we draw links from X2X3 to X7 in G", as shown in Figure 4b). Now update the active set S as {[X1)..., (Xo)).
Step 5. Reset k
and search for rank deficiency. No more rank deficiency can be found with k = 1, and thus we add k by 1.
Step 6. Get S' by unfolding S, and S' = ((Xi)),... (Xs)). When == 2 and 2, no more rank deficiency can be found. Therefore, we try h = 2 and = 1. By drawing X' and C, we will find that when X' = {[Xa}} and C = ((X)). [X:]] or ((X)), [X+]] or [[Xa]. (Xo rank deficiency holds and there is no collider detected. Therefore, we conclude that there is an atomic cover. As = 2 but ||||| || = 1, we need one additional latent variable to explain this atomic cover. Thus, we add a new node L2 to G' (the subscript index for L can be rather arbitrary long as it is not the same as an existing one), and draw links from L2X₂ to X,X5XB in G', as shown in Figure (c). Now, update the active set S as {(x1), (X2), (Xa), [L2,X2)).
= and search for rank deficiency. Unfold S and get S' = ) (x2). (X3), (22,X2])- When h 1 and = 1, no more rank deficiency can be found. Therefore we try k- = 1 and t = O. By drawing X' and C, we will find that when X' = O and ((X,,, ((X,), [X2]] or (1X2), [Xa]] or ((X1), {Xa}} or {[L2, X2)}, rank deficiency holds and there is no collider detected. Therefore, we conclude that there is an atomic cover. All the possible will be merged. As k
Step 7. Reset
C
but |||||| 0. we need one additional latent variable to explain this atomic cover. Thus, we add a new node L1 to G', and draw links from L, to X2X3XXX3 in g', as shown in Figure (d), and update the active set as ((L1)).
Step& 8. From now on, no more rank deficiency can be found, and when i is sufficiently large the procedure ends. Output G', as in Figure (d).
B.6 EXAMPLE FOR PHASE
Here, give an example (see Figure 13 where Phase 2 may result in incorrect latent covers, and thus we need Phase to characterize and refine these incorrect latent covers. Specifically, as in Figure 13), when we look for k 3 clusters, none of the atomic covers (L.).(44) L2) has
24

---

Published as a conference paper at ICLR 2024
(a) The ground truth graph g.
(b) Algorithm output after phase taken input of RefineCausalClusterr.
(c) Remove {41 L1 13} and its neighbours that (d) During FindCausalCusteers performed at (c) are latent. Then perform FindCausalClusters. first find {Lis}
(c) During FindCausalllssters performed at (c), (f) During FindCausalClusters performed at (c), we finally find (Lís. List.
we then find {Lí4}.
Example in subfigure (a) is the real graph g. After phase 2 the output graph still contains a fake cluster, as in (b). After phase 3, the output graph will be correct, as in (f).
Figure 13:
been discovered. Therefore, in Phase 2, when looking for 3 clusters, we will find combina- tion of C = }, [X2]. J. (Xa), {Xa}} and X' = O that causes rank deficiency, and thus we will mistakenly create atomic cover [L1, L2, L3} with their pure children (X1), (X2), (X3), (X4). as in Figure (b)..
Fortunately. this incorrect cluster will not affect the identification of other clusters in the graph: e.g., in Figure b3)b), the covers {Lz, Lg. L.)iiLio Li Liz) are correctly found, except that the neighbors of the wrong atomic cover [L4, L3. L3} could be incorrect. This allows us to take further look into the incorrect cluster and refine it based on Theorem (the proof of which is in Appendix [A.17).
As shown in Figure 1 the subfigure (a) is the underlying graph g. After phase the output graph g' in (b) contains incorrect cover = -5, Lg). (c), we first calculate G. which is got by deleting V, all neighbours of V that are latent, and all relating edges of them from G'. The resulting & is shown in Figure (c).
(c). After that, perform FindCausalClusters( &. X), and then the clusters (X1, X2), (an) (X4) can be correctly found, as shown in Figure 13 (f)
B.7 GRAPH EXAMPLES WITH VARIABLES ALL OBSERVED
Please refer to Figure 15
B.8 GRAPH EXAMPLES FOR LATENT TREE MODELS
Please refer to Figure T8
B.9 GRAPH EXAMPLES FOR LATENT MEASUREMENT MODELS
Please refer Figure 17
25

---

Published as a conference paper at ICLR 2024
X2
X
A
Xa
X10
X
X
X
X
X,
X
(a) The original graph g.
(b) After operators Oin and O,
Figure 14: Example to show graph operators O and O,,
1,
%
N
(a) Example
(b) Example 2.
(c) Example 3.
Figure 15: Examples graphs that have only observed variables.
B.10
GRAPH EXAMPLES FOR GENERAL LATENT MODELS
Please refer Figure T9
B.11
ILLUSTRATIVE EXAMPLE OF CONSIDERING COLLIDERS IN PHASE
For example, Figure 16 suppose that we have already found the cover L, as the parent of clus-
ter X2, and Lg as the parent of cluster XgX7. Next, search for k: 2 clusters and take = [[X3]. {X1): {L2}} and X = (}. and then we have rank deficiency rank(Eeux.N
= 2.
However, this rank deficiency does not imply correct cluster there is set of collider {L2] inside
C. Fortunately, can be detected by Algorithm 5 Specifically, we take C' = ((Xa), [X4]]. we find that rank(Ecrux
rank X
(line 5 in Algorithm 5) which
means there exists a smaller group of rank deficiency caused by removing the collider in C. Thus, O is not a correct combination and will consider them for forming a cluster (as in line I in Algorithm
conclude that {{Xa), ) {X+), {L2}} and
B.12 EXAMPLES FOR GRAPI OPERATORS
Suppose graph G of latent linear model in Figure 14 a) is G. After applying (O,)), have the graph in Figure 14).). Specifically, the O, operator adds the following edges: L2 - Xy, L3 X7, L2 L L4, and L3 - La- The Omin operator delete Lş and add edge from L1 directly to Xg. Xg. and X10- For G, such two operators will not change the rank in the infinite sample case.
B.13 GRAPHICAL RELATIONS BETWEEN COVERS AND SET OF COVERS
The relation between covers naturally follows the relation between a set of variables. For example, in Figure [a). a), the pure children of [La, Ls) is fXs. Xx)..) For the relation between sets of covers, it also follows the relationship between variables. E.g., in Figure B[a,, the parents of ([L4), {Ls)} is of nodes (L1).
26

---

Published as a conference paper at ICLR 2024
X
X
Figure 16: Example of checking colliders in N.
(a) Example 1.
(b) Example 2.
(c) Example 3.
(d) Example 4.
Figure 17: Examples of Latent Measurement Graphs.
14 DISCUSSIONS ON CHECKING COLLIDERS COMPLETELY
make sure that the existence of colliders between atomic covers in will not induce incorrect clustering results. However, we note that there still chances that colliders that the existence of colliders in ' will not induce fake clusters. In fact, there is a way to further check whether there exist colliders N. Specifically, in the line 17 of Algorithm 2 if have (Nxx = k. and NoCollider(C, N) returns True, we can further check whether there exist a set of covers A C V such that A" consists all the colliders between C and A'W" To this end, we just enumerate all the possible subsets N of N. If N' is the set of all the colliders, then it must be that (i) rank
With search procedure that checks colliders in Algorithm 5
in W. If Condition hooiiionnhhllls then
make
rank(Eeux )
k' k. and (ii) rank aark(("cuuuw..nyuu
k ||W"|]
Take Figure 16 HLLHH + [ = 2 < |V1 + V2l = [[X,,Xa) + (Xs, X
example. First, check whether Condition 2 holds. As |C| + |A|
2 does not if we take X' 0. {{Xs}, (X4).(Xs] and N
4. Condition
hold. Therefore, when checking
([x)) (x (Xs). (X7), (Xs) } in line 17 of Algorithm we will find that rank rank (Eru
((Xa) (X4), (Xs)) should be only 1. Fortunately, in this scenario, we can detect that N' ((X7), (Xx) C N is the set of all the colliders, by finding that (i) rank("Ecu, rank((cuuuu (EcuxuV,,N)Ux)) = > 1 || 3.
2, which implies an incorrect cluster as the cardinality of parents
= k 2, and (ii)
As mentioned in Section by adding this check function to our algorithm (specifically to line 17 in Algorithm 2 before adding C ID), can achieve better identifiability that relies on Condition only. However, that additional checking function is computationally inefficient.
B.15 EVALUATION METRIC DETAILS
The definition of F1 as follows. F1 Recall
Regal Precision Recall TPLFN and Precision = TP TP+FF- where TP, FP, and FN denote True Positive, False Positive, and False Negative, respectively.
For fair comparison, we need to align the latent variables in the output graph G' of method with the latent variables in the ground truth graph G. To this end, we first pad each result by adding latents that have edge any other variables to match the number of latents in the ground truth graph. On the other hand, if the number of latents is more than that of the ground truth G, all different combinations will be tried. Finally, try all different permutations of latent variables to test the
27

---

Published as a conference paper at ICLR 2024
(a) Example
(b) Example 2.
Example 3.
(d) Example
Figure 18: Examples of Latent Tree Graphs.
(a) Example 1.
(b) Example 2.
X
X
X
X,
(c) Example 3.
(d) Example 4.
Figure 19: Examples of Latent General Graphs.
Fl score. For each method, the final Fl score is taken as the best Fl score among all possible combinations and permutations.
B.16 MORE DETAILS EXPERIMENTS ON SYNTHETIC DATA
Our code is implemented with Python 3.7. Asymptotically speaking, if the ground gruth graph a DAG, then there will be no cycle in our result. However, the finite sample case, rank test results could be self-contradictary. Therefore our implementation checking whether cycles may occur every time before concluding a cluster. As different meth- ods employ different statistical tests that may perform differently, the hyperparameter a chosen from {0.1, 0.05, 0.01, 0.005) in favor of each method to ensure their best performance and thus a fair comparison. For the proposed method we employ a 0.005 for the procedure of finding latent variables, while for the first stage we empirically find that using a rather big would be better. This is because the first stage of PC good at deleting edges and thus bad recall, and the following procedure would expects input with high recall rather than high precision. We con- duct all the experiments with single Intel(R) Xeon(R) CPU E5-2470. Our proposed method and GIN (Xie al.) 2020) take around 3 hours finish all the experiments (three random seeds and three different sample sizes), and Hier. rank (Huang et al.] (002)) takes around hour. PC (Spirtes) al. 2000) and FCI (Spirtes al., 2013) take around 10 minutes, while RCD (Maeda & Shimizu 2020 takes around two days to finish the experiments. For GIN, RCD, and Hier. rank, we em- the causal-learn python package eo/en/ /at/ttts///////
explictly prevent that by
ploy their original implementation while for PC and FCI
https::/causal/ -larn. www.erraadthhoooee
The time complexity of our proposed algorithm is upper bounded by 0 k= Et (") (TTTL)) where n is the number measured variables, K the cardinality the largest cover the esti- mated graph, with K < n,, and / is the number of levels of the estimated graph, with <
28

---

Published as a conference paper at ICLR 2024
It is also possible exhaustively enumerate all possible graphs and check whether one of them may be aligned with observational rank information from data. However, that would be very computa- tionally expensive. Assume that the underlying graph consists of n measured variables and m latent variables. To conduct an exhaustive search for causal clusters, we need to enumerate all possible numbers of latent vars and then enumerate all possible structures, which results in approximate total of D 3(++n)(i+n-1)) possible combinations. In our synthetic data, which comprises erage of 15 measured variables and 4 latent variables, the mere act of enumeration already demands 1006 seconds (in our Python environment), which is computationally unacceptable.
B.17 DETAILED INFORMATION OF THE BIG FIVE PERSONALITY DATASET
Data was collected through an interactive online personality test https: // openpsychometrics.u org// Participants were informed that their responses would be recorded and used for research the beginning of the test and asked to confirm their consent at the end of the test. Items were rated on five-point scale: 1=Disagree, 2-Slightly disagree, 3=Neutral, 4=Slightly agree, 5=Agree (0=missed). Datapoints with missing values have been filtered out. Some additional information is also collected including Race, Age, and Gender but not used in our experiment. The Markov equivalence class of Figure 5 is generated by using our proposed method, while we further apply GIN (Xie al.) 2020) to determine directions between latent variables. The five personality dimensions are Openness, Conscientiousnes, Extraversion, Agreeableness, and Neuroticism (O-C-E-A-N). Below are the raw questions. E.g., El denotes the first question for the Extraversion score.
E1 am the life of the party.
E2 don't 't talk lot.
E3 feel comfortable around people.
E4 keep in the background.
E5 start conversations.
E6 have little to say.
E7 talk to lot different people at parties.
E8 don't 't like to draw attention to myself.
E9 don't 't mind being the center of attention.
E10 I am quiet around strangers.
N1 get stressed out easily.
N2 am relaxed most of the time.
N3 I worry about things.
N4 seldom feel blue.
N5 I am easily disturbed.
N6 get upset easily.
N7 I change my mood lot.
N8 have frequent mood swings.
N9 I get irritated easily.
N10 often feel blue.
AI I feel little concern for others.
A2 am interested in people.
A3 I insult people.
A41 sympathize with others' feelings.
A5 am not interested in other people's problems.
A61 have soft heart.
A7 am not really interested in others.
A8 I take time out for others.
A9 feel others' emotions.
A10 make people feel at ease.
CI am always prepared.
C2 leave my belongings around.
C3 pay attention to details.
C4 make mess of things.
C5 get chores done right away.
C6 often forget to put things back in their proper place.
29

---

Published as a conference paper at ICLR 2024
C7 like order.
C8 shirk my duties.
C9 follow schedule.
C10 am exacting in my work.
01 have rich vocabulary.
02 I have difficulty understanding abstract ideas.
03 have vivid imagination.
04 I am not interested in abstract ideas.
05 have excellent ideas.
06 I do not have good imagination.
07 am quick to understand things.
08 I use difficult words.
09 spend time reflecting on things.
010 I am full of ideas.
B.18 MORE ANALYSIS OF THE RESULTS FOR THE BIG FIVE
A prevalent theory of personality is that personality dimensions (factors or traits) are latent causes of the responses to personality inventory items, which are indicators of the latent construct. For in- stance, extraversion yields high scores for the indicators "I like to go to parties" and "I like people." Thus, the responses to inventory items are outcomes of one's position on the latent dimension. How- ever, there is also the suggestion of network perspective in which personality structure is viewed in terms of microcausal connections in complex network (Wright 2017) and personality dimensions emerge out of the connectivity structure (Cramer et al. 2012). This radical divergence from the conventional viewpoint that dimensions are causes of the relevant indicators. For instance, a network would show that instead of being two distinct markers of extraversion, one might say "I like to go to parties" because like people" (Cramer et al.] (2012); or in the case of openness, "I am full of ideas" because "I have vivid imagination". Our result in Figure [ indicates, however, that our method adheres to network perspective while identifying groups of closely connected items that are predictable under a latent dimension model. Further, it can be observed that causal links occur between latent dimensions, between observed indicators, and among latents and indicators. Following are interesting aspects of our results.
(i) LI, I.2, L3 and L.5. While L11 1.2, and 1.3 clearly delineate conscientiousness, agreeableness, and extraversion as causes of the corresponding item responses, it different in the case of openness (L5).. Openness to experience can lead to excellent ideas brought about by active imagination, reflection, and understanding of things. Moreover, those who develop a rich vocabulary will have the propensity to think critically and read more, behaviors that also birth ideas.
L1) +L3. Conscientiousness and openness are most frequently associated with achievement Gatzka 202I). In our results, those who are organized, efficient at tasks, thorough, systematic, or exacting will comprehend things quickly, demonstrate sophistication in language. or are good at introspection. These could instill sense of confidence and assurance that encourages assertive, verbal, and bold behaviors, among other extraversion markers.
(iii) L1- L33. People who score highly on conscientiousness frequently perceived as per- fectionists, high achievers, overly focused on personal goals, preoccupied with flawless task execu- tion, overly demanding, and headstrong (Le et al., 2011 (Curseu al., 2019). Our findings suggest that due to such behaviors, highly conscientious individuals would judge other people on their ac- complishments and results, without giving consideration to others' feelings. Consequently, they act distant, uncommunicative, or unsociable. These can be reasons why they have been found not to en- gage group behaviors that lead to straining relationships and tend to take criticism poorly (Curseu et al., (201)) as well as refrain from conversing about interpersonal issues because they have no bearing on achieving task objectives (Curşeu et al. 2019). On the other hand, those who are both highly conscientious and agrecable put the needs of others before their own (Lord] 2007), at times to the point of pleasing others by overlooking their mistakes, doing things for others because they can- not say no, not disclosing performance gaps and withholding dissident opinions out of aversion to conflict and a lack of competitiveness (Graziano & Tobin (002) & Howard, 2010))Curyen et al. 2019.. Thus, they will resolve problems their own, not to draw attention to themselves,
30

---

Published as a conference paper at ICLR 2024
have little to say, would rather stay in the background in order to be integrated. Those who score low on agreeableness are perceived to be unempathetic, unfriendly, and untrustworthy, consequently leading to introvertive behaviors as well.
(iv) L1 and L3 together as common causes. Conscientious individuals who care about being liked by others despite being focused, detail-oriented, and exacting, will make efforts make people feel at ease amid these behaviors. Otherwise, conscientious individuals who care not about what others think of them could be quick lambast others if they perform poorly. Low-conscientious people, those who are disorganized, messy, sloppy, and negligent, will also tend to make people at ease in order to remain in their good graces.
(v) No latent variable for neuroticism. Our method did not discover any latent variable that is sup- posed to correspond neuroticism. One possible interpretation would be that the question "[NI0]: I often feel blue." is designed so well that it fully captured the sense of neuroticism.
(vi) Responses to indicators influence other responses. It is the question items, not the la- tent dimensions, that can be perceived to have caused the succeeding responses, as in the case of N10->N8-NY N10~09, O2-04, and 01-08, all of which are plausible. Mood swings are common with depression, and frequent mood swings cause emotions to fluctuate rapidly and intensely, switching between positive and negative emotions. Some people may find themselves reflecting a lot things because they are trying to figure out what makes them frequently feel blue and how to cope with it. A person who has trouble understanding abstract concepts is unlikely to be particularly interested in them. Finally, one who has rich vocabulary will not be constrained from using unusual words that are hard to comprehend.
C RELATED WORK AND BROADER IMPACTS
C.I RELATED WORK
Causal discovery aims to identify causal relationships from observational data. Most existing ap- proaches are based the assumption that there are latent confounders (Spirtes al., 2000, Chickering 20026 Shimizu et al., 2006a] Hoyer et al., 2009 Zhang & Hyvärinen (209)). and yet this assumption barely holds for rial-life problems. Thus, causal discovery methods that can handle the existence of latent variables are crucial. Existing causal discovery methods for handling latent variables can be categorized into the following folds.
(i) Conditional independence constraints. The FCI algorithm (Spirtes et al., 2000) and its vari- ants (Colombo al., 2012] Pearl 2000; Akbari et al.) 2021). This line of work checks conditional independence over observed variables to identify the causal structure over observed variables up to a maximal ancestral graph. They can deal with both linear and nonlinear causal relationships. but there are large indeterminacies in their results, e.g., the existence of an edge and confounders. Plus, they cannot consider causal relationships between latent variables. Based on CI tests, TTiintafillou & Tssmmrddinos] (2015) proposes method that can co-analyze multiple datasets that share common variables and sort the significance tests to address conflicts from statistical errors. (ii) Tetrad condi- tion. This line of work makes use of the rank constraints of every 2 x 2 off-diagonal sub-covariance matrix to locate latent variables and thus find the causal skeleton based on linear relationships be- tween variables (Silva et al. 2006 Kummerfeld & Ramsey 2016 Wang 2020 Pearl 1988). One limitation of this line of work is that they assume each measured variable is influenced by only one latent parent, and each latent variable must have more than three pure measured children. (iii) Ma- trix decomposition. This line of work proposes to decompose the precision matrix into a low-rank matrix and sparse matrix. where the former represents the causal structure from latent variables to measured variables and the latter represents the causal structure over measured variables, under certain assumptions (Chandrasekaran al.) 2011 2012 Anandkumar et al. 2013). E.g.. Anandku- mar et al. (203)) decomposed the covariance matrix into a low-rank matrix and diagonal matrix, by assuming three times more measured variables than latent variables. (iv) Over-complete inde- pendent component analysis (ICA). Over-complete ICA allows more source signals than observed signals, and thus can be used to learn the causal structure with latent variables (Shimizu et al. 2009). and yet they normally do not consider the causal structure among latent variables. The estimation of ever-complete ICA models could be hard to reach global optimum without further assumptions (Entner& & Hoyer 2010 Tashiro al. 2014). (v) Generalized independent noise (GIN). The GIN
31

---

Published as a conference paper at ICLR 2024
Table Structural Hamming Distance (SHD) compared methods on different types of latent graphs where the values are averaged over three random seeds. The smaller the better.
SHD for skeleton among all variables Ve (both Xg and Lg)
Algorithm
Ours
Hier. rank
PC
FCI
GIN
RCD
2k
6.9
9.3
23.7
23.7
20.5
22.2
tree
5k
3.2
9.0
25.0
24.3
21.2
23.5
10k
0.7
9.0
25.1
24.3
20.0
24.0
2k
4.6
8.1
14.3
14.7
10.8
13.4
Latent+
5k
3.8
7.7
15.0
15.0
9.2
16.2
10k
2.9
7.4
15.5
14.8
9.2
16.0
2k
27.1
28.1
38.0
37.6
36.4
36.5
Latent general
5k
23.0
26.0
38.2
36.8
33.8
32.5
10k
21.4
26.0
39.0
37.1
34.1
36.1
condition is an extension of the independent noise condition when latent variables exist. Based on non-gaussianity leverages higher-order statisties identify latent structures. E.g., Xie et al. (2020) allows multiple latent parents behind every pair of observed variables and can identify causal direc- tions among latent variables, and yet requires least twice measured children as latent variables. Dai et al. (2022) proposes a transformed version of GIN to handle measurement errors. (vi) Mixture oracles-based. Kivva et al. (202I) proposes a mixture oracles-based method to identify the causal structure in the presence of latent variables where the causal relationships can be nonlinear. It is based on assumptions that the latent variables are discrete and each latent variable has measured variables children. (vii) Rank deficiency. Recently Huang et al. (2022) proposes to leverage rank deficiency of sub-covariance of observed variables to find the underlying causal structure in the presence of latent variables. Our method differs in that we consider a more general setting, i.e., we allow hidden variables can be causally related to each other, form a hierarchical structure (i.e., the children of hidden variables can still be hidden), and even serve as both confounders and inter- mediate variables for observed variables. Our graphical condition for identifiability also generalizes the condition (Huang et al., 2022) cases where edges between observed variables are allowed. (viii) Heterogeneous data. Huang* et al. (2020) considere a special type of latent confounders that can be represented as a function of domain index or a smooth function of time. This line of work makes use of domain index or time index as surrogate to remove confounders' influence and con- sequently identify causal structure over observed variables. (ix) Score based. [Agrawal et al. (202T) propose score-based method for latent variable causal discovery, by assuming additional structure among latent confounders.
The most related work to our method is Hier. Rank Huang et al. (022). Compared to Hier. Rank, our graphical conditions are not only strictly but also much weaker. Our conditions are strictly weaker in the sense that Hier. Rank can be taken as special case of the proposed method by disallowing direct edges between observed variables, which is formally captured by our Corollary Our conditions variables to be flexibly related and exist everywhere in graph, which is illustrated in Figure (b) v.s. Figure [I (d). The reason why we are able identify these latent structures that Hier. Rank cannot identify. lies in that we utilize the rank constraints in a more flexible and comprehensive fashion. Specifically, Hier. Rank only uses the part of the rank information rank (EA,B) where A n B = D. while the proposed method uses the rank (EA.n B) where A and B are rather arbitrary and thus more t-searaations can be inferred. The extra graphical information allows us to make use of, e.g., Lemma [T0 for identifying edges between observed variables, and Theorem g to identify atomic covers that are partially hidden partially observed.
much weaker in the sense that, basically we allow latent variables and observed
D ADDITIONAL INFORMATION
D.I EMPIRICAL RESULT USING SHD
In this section we further show the performance of each method using the Structural Hamming Distance (SHD). As shown in Table 4 The SHD of the proposed RLCD method to the ground truth consistently smaller than all comparative methods under all the settings, which again validates RLCD in the finite sample cases.
32

---

Published as a conference paper at ICLR 2024
X
X
X
X,
X
X
X
X
(a) In G1, Condition 1 is not satisfied, in (b) Given G1. RLCD outputs Gi, which the sense that L,ا2 do not have enough is not informative but correct as they pure children and neighbours.
rank-equivalent.
4
L
X
X
X
(c) In G2. Condition I is not satisfied, in (d) Given Gz. RLCD outputs Gy which the sense that La does not have enough is correct and informative, though not ex- pure children and neighbours.
actly the same as G2.
X1
X
Xs
X
X
X
%
X
X
Xs
X.
x,
X
X
X
(c) In Ga. the Condition 2 is not satisfied, (f) Given Ga. RLCD outputs Gy which is because (L1,L,) + + HX + incorrect but can infer from the result that conditions violated.
[[Xa]]-
Figure 20: Examples to show that even when graphical conditions are not satisfied, the proposed RLCD can provide the correct result (though may be uninformative), or infer that the condition is violated.
D.2 VIOLATION OF GRAPHICAL CONDITIONS
In this section, we shall discuss what would the output of the proposed method be when graphical conditions are not satisfied.
(i) The result is correct but uninformative. For instance, in Figure 20 (a), LبL2 do not belong to any atomic cover, as they do not have enough pure children plus neighbours. Thus, Condition is not satisfied. In this scenario, though the output of the proposed method is not informative, the result correct.
not informative the sense that the result Figure 20 (b) fails to inform us the existence of latent variables. The result is correct in the sense that it correctly outputs the CI skeleton, which is cank-equivalent to the ground truth Gr. In other words, the output graph Gi and the ground truth G1 are able to entail the same of observational rank constraints and algorithm can differentiate them solely by rank information.
(ii) The result is correct and informative, though it uses of the rank-equivalence class. An example is given in Figure 20 (c), where L4 does not belong any atomic cover (as enough pure children plus neighhours), and thus G2 does not satisfy
compact graph the representation
33

---

Published as a conference paper at ICLR 2024
%
X1O
X12
X
X
X
X1o
(a) Input latent general graph.
(b) The output of RLCD, G1.
(c) The output of Hier. Rank, G2.
(d) The output of FCI, G3-
(e) The output of GIN, G4-
Figure 21: Illustrative examples the output each method given data generated from G. We only show the skeleton here for simplicity.
Condition However, the proposed RLCD still outputs the correct and informative result Ø2- The output G3 correct in the sense that 9%2 and G2 rank-equivalent. i.e., they entail the same set of observational rank constraints, and the local violation of Condition [ does not harm the correctness of other substructures. The output is also informative as 9% is compact representation of the rank- cquivalence class that contains G2; one can easily infer many other members of the class from G.2
(iii) The result is incorrect, but from the result we can infer that the conditions are violated. An example is given in Figure 20 (e) where for the v structure in G3 we have [[,, L2H + |[a > [[X and thus Condition is not satisfied. In this scenario, RLCD will output G/z which is incorrect. However, we can easily infer that this result is abnormal, as the result is not consistent with the CI skeleton: in 9/2, there are subgroups that are not connected each other, while the CI skeleton would inform us that all the variables are directly indirectly connected.
+ [[X+]]
We note that the above analysis holds in the large sample limit. In the finite sample cases, the result of statistical tests could be incorrect and thus self-contradictory.
D.3 ILLUSTRATIVE OUTPUTS OF EACH METHOD.
Here we give illustrative outputs of each method to give an intuitive understanding of why the proposed RLCD performs the best. Here we use 10000 data points and only show the skeleton for simplicity.
The underlying graph G shown Figure 21 (a) and has observed variables and latent variables. The output of the proposed RLCD, Gi given in Figure 21 and it is nearly the same as the input G. As it is the finite sample case, there are still two edges missing due to the error of the statistic test. However, by increasing the sample size, the missing two edges also be recovered.
The output of Hier. rank, G2 is given in Figure 21 (c). introduces redundant latent variables compared the ground truth, and many edges incorrect. The underlying reason is that Hier. rank cannot handle the scenario where observed variables are directly adjacent and it does not allow edges from observed variable latent variable.
34

---

Published as a conference paper at ICLR 2024
The output of FCI, G3 is given in Figure (d). It can neither identify the cardinality nor the location of latent variables. Furthermore, many edges between observed variables are missing or incorrect. This might be due to the fact that FCI relies on CI tests conditioned on multiple variables but tests of independence conditional on large numbers of variables have very low power/Spirtes (2001)
The output of GIN, P4 is given in Figure 21 (e). The number of latent variables it discovered is correct, but they do not exactly correspond to the latent variables in the ground truth G. Plus, many edges are incorrect, and the reason is similar that for Hier. rank, i.e., GIN cannot handle direct edges between observed variables and edges from an observed variable to a latent variable
D.4 DETAILED DISCUSSION ABOUT COMBINING RESULT OF PHASES 2 & 3 WITH PHASE 1.
ii "Transfer the estimated DAG G" to Markov equivalence class" (the first part Line Al- gorithm []:
Here, we transfer the output of Phases 2 & 3, i.e., G". to Completed Partial Di- rected Acyclic Graph (CPDAG), which represents the corresponding Markov Equivalence Class. Algorithms for transferring DAG to CPDAG have been well studied (Chickering 2202bb a: 2013) with implementations available from c.g., causal-learn (Zheng et al. 2023) or causaldag (Chandler Squires 2018) python package.
(ii) "Update g' by g"" (the second part of Line 7 in Algorithm 1): Note that the input to Phases & 3 is X o U o. and let Le be the newly discovered latent variables during Phases 2 & 3. By Theorem 9 latent variables must be in treks between variables in Xo. Therefore, we just need to consider the edges among Xg Lg. Specifically, first delete all the edges among X in graph G', then add Lo to G', and finally add all the edges among X U Lo from g" to g'.
(iii "Orient remaining causal directions that can be inferred from structures" (Line Algo- rithm Given g", partially directed acyclic graph (PDAG), here we orient all remaining causal directions that can be determined. This can be easily achieved by following stage of PC (Spirtes et al., (000):: we use the CI results detected in Phase to find v-structures among observed variables and apply Meek's rule (Meek) 2013) to infer the remaining directions that can decided. After that, we then transfer this PDAG implementations available) to get the corresponding Markov equivalence class.
CPDAG (well studied in Chickering (2002b 2013) with
35


Reviews:
[{"review_id": "Tt0SIQZngA", "reviewer": "Reviewer_DkWP", "summary": "This work expands on the use of the rank of the covariance matrix to identify causal structures with latent variables in linear models. The authors give sufficient and necessary conditions under which latent variables of the causal graph can be identified using the rank. This insight is then used to devise an algorithm that can identify linear latent causal graphs upto an indeterminacy.", "questions": "- The assumptions in condition 1 are on unmeasured latent variables, how would you verify this before carrying out the graph search procedure? \n- Similarly, how can you verify assumptions in condition 2? This seems particularly strong to me, \n- Is Lemma 10 a contribution or has this been stated by previous work? If so, a reference is missing.\n- Similar to the above, is Theorem 4 a contribution?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- Very well written and a pleasure to read. The paper is well structured and the exposition is clear.\n- Claims are well justified.", "weaknesses": "- SID and SHD are more natural metrics for graphical evaluations\n- Assumptions are not justified. (this might be for the field in general)\n\t- Assumptions are made on unmeasured variables and graphical structure that are hard to verify. Justification of why these are relatively weak (compared to previous work) would make this more useful.\n- Relation to previous work does not contain enough information. It makes it hard to judge the exact contribution of this work. Some detail is given in the introduction.\n\t- Related work has been moved to the Appendix but there is not enough information about what the differences to similar works are. It would be useful if the most related works (e.g. Huang et al. 2022) are described in more detail.\n\t- For example, Hier. rank, that uses rank to discover hierarchical structures, is not described in sufficient detail. This leaves the question in the readers mind: what specifically allows for the identification of children of latents and mediatior latents etc as opposed to this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ebmjyysZso", "reviewer": "Reviewer_G6SE", "summary": "The paper presents a novel rank-based latent causal discovery algorithm to identify equivalence classes of directed acyclic graphs which can have both measured and latent variables. The causal relationships among the variables can be quite general compared to restricted patterns in the literature. They prove the discovery algorithm is asymptotically correct and degenerates to existing algorithms when certain aspects of the algorithm are simplified. Simulations and real data examples demonstrate the utility of their method.", "questions": "1. Can the author explain the minimal-graph operator and the skeleton operator in addition to their definitions? Perhaps giving some examples will be helpful for readers to understand what equivalent graphs they entail.\n2. Corollary 1 says it degenerates to PC when there is no latent variable. I suppose that is under the assumption that the causal model is linear as assumed throughout in this paper. In other words, if the truth is not linear, the PC is asymptotically correct but the proposed algorithm may not -- is it right?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The paper addresses a very important problem in causal discovery, i.e., identify graphs with latent variables\n2. The graph pattern considered in the paper is quite general compared to existing methods.\n3. The learning algorithm has theoretical guarantee in the large sample size limit. \n4. Real-world example is quite convincing. \n5. The paper is very well written.", "weaknesses": "No major weakness is found. Just a few minor ones; see the questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q38zzes2Lo", "reviewer": "Reviewer_aySa", "summary": "The authors propose a method for causal discovery in the presence of latent variables (`RLCD`), which makes use of observational rank information under the assumption of an underlying linear latent causal model, in which measured variables can be adjacent. The authors show how the rank information can be used to locate the presence of hidden variables and how their approach can reliably discover the causal structure over both measured and latent variables, asymptotically up to their Markov equivalence class.", "questions": "1. I am not convinced about the claim on page 1 that, for algorithms like FCI, \"the research relies on the assumption that all hidden variables are independent of each other\". Could you perhaps point to where that assumption is made? As far as I know and have checked, any maximal ancestral graph (MAG) is learnable from conditional independence using FCI, and a MAG is also valid if obtained from marginalizing over dependent latent variables.\n2. By \"latent ones\" do you mean latent variables? If so, I would say latent variables instead, because it is not immediately clear what \"ones\" refers to.\n3. In Theorem 7, what is the distinction between $\\mathbf{X}$ and $\\mathbf{X}_\\mathcal{G}$?\n4. Why is it so important to have a unified causal discovery framework, in the sense that rank constraints are used for finding the CI skeleton in Phase 1 of the procedure? I imagine it is more important to find an accurate CI skeleton, so do rank constraints provide more accurate *d*-separation statements than conditional independence tests? Could it also be better to mix different types of tests?\n5. I imagine FCI would perform quite poorly in terms of the skeleton F1 score for all variables (Table 2), since it does not explicitly identify any latent variables, but how do you explain a score of 0.00? Does that mean that FCI did not get any edge right at all, not even between observed variables, as Table 3 also suggests?\n6. What happens after Phase 3 in Algorithm 1? How is the information from $\\mathcal{G''}$ transferred to the Markov equivalence class? How are the rest of the orientations performed? I am also confused by the fact that $\\mathcal{G'}$ is supposed to be the skeleton on the observed variables (output of Algorithm 2 from PC), yet toward the end it becomes the MEC over both observed and latent variables (output of Algorithm 1). What am I missing here?\n7. The idea of learning part of the latent structure explicitly has important ramifications. Could the authors perhaps comment on what extra information can be ascertained relative to methods like FCI, for which the latent structure is implicit? Put a different way, does the difference lie solely in the fact that some latent variables can be identified? Will RLCD always provide more structural information than FCI or other CI-based causal discovery algorithms?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work is rigorously described for the most part and quite technical, yet the authors have managed to present intricate information in a reasonably clear manner. The approach seems to be an original extension of previous work on hierarchical structure and has potentially significant application, since it allows the user to uncover part of the latent causal structure under relatively mild assumptions. The differences between the proposed method and related work, including the PC algorithm and methods for identifying latent hierarchical structures, is clearly explained.", "weaknesses": "The paper is rigorously written for the most part, but I think there some parts are missing important details. For example, the steps in Algorithm 1 after Phases 1-3 are not described in detail, specifically how the cluster information is aggregated after the first three phases. It is also not very clear how the additional information is added into the CI skeleton.\n\nI also think a bigger focus on the evaluation in the main paper would be warranted, since the experimental section seems a bit sparse. It is not quite clear to me why and where RCLD performs so much better than the competitors, just by looking at the F1 score. It would have been helpful to see the output of these different algorithms on a running example. \n\n*Miscellaneous comments:* \n- some references are duplicated (Judea Pearl - *Probabilistic reasoning in intelligent systems*, Shohei Shimizu et al. - *A linear non-gaussian acyclic model for causal discovery*.\n- page 1, first paragraph in introduction: \"ICA-based techniques ... that further **leverage**\"\n- page 2: repetition in \"our main contributions are mainly three-fold\"\n- page 3: instead of the unusual construction \", - we basically\", I would employ a semicolon, or simply start a new sentence.\n- page 3, Section 3.2: I would not start a sentence with \"E.g\", but instead say 'For instance,'\n- page 5, Algorithm 1 is introduced too early in the paper. I would move it to Section 4.2, where it is first explained.\n- page 5, before Theorem 7: \"uesful\"\n- page 6, Figure 3 is introduced too early in the paper. I would move it to Section 4.2, where it is first used to explain Alg. 1\n- page 6, Figure 3(c) caption: \"**Take** variables from...\"\n- page 6, Condition 1: \"triangle structure\" should be defined\n- page 7, Section 4.2, second paragraph: I believe \"Condition (i)-(iv)\" is supposed to be **conditions (i)-(iv)**.\n- page 7, last paragraph before Section 4.4: \"We further determine the **neighbour** set... \" \n- page 7, last paragraph: \"we **increase** *k* by 1\"\n- page 8, Table 2 appears too early in the paper, since the experiment section is on the next page.\n- page 9, Section 7: \"causal discovery approach that allows causally-related variables\" seems to be an unfinished thought", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VqPXS8TCgm", "reviewer": "Reviewer_Rcya", "summary": "This paper presents a new framework for causal discovery in the presence of unmeasured confounders and linear structural causal models (SCMs).", "questions": "1. It’s a minor comment. I think the definition of $\\Sigma_{\\mathbf{A},\\mathbf{B}}$ should be within Theorem 3. \n2. If $\\mathbf{A},\\mathbf{B}$ are not t-separable, what does Theorem 3 imply? \n3. I am curious about the benefits of the proposed algorithm in terms of running speed and time complexity compared to other existing algorithms, such as FCI or LiNGAM.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "Overall, I like this paper, and I think the contribution made in this paper is quite huge. \n\n1. The introduction is written in a straightforward and neat manner. The contribution is expressed clearly.\n2. All the results seemed technically sound to me. \n3. Every result is accompanied by examples, which greatly helps understanding the paper. \n4. Experimental studies are conducted extensively, providing strong empirical benefits.", "weaknesses": "One minor weakness/limitation of the paper is that its method is confined to linear SCM. If the variables are mixtures of discrete and continuous random variables, or if they follow a nonlinear SCM, are there any opportunities for the proposed method to contribute?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)