Score-based Greedy Search for Structure Identification of Partially Observed Causal Models
Authors: Xinshuai Dong
Ignavier Ng
Haoyue Dai
Jiaqi Sun
Xiangchen Song
Peter Spirtes
Kun Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: Causal Discovery
Latent Variable
Primary area: causal reasoning

Abstract: Identifying the structure of a partially observed causal system is essential to various scientific fields. Recent advances have focused on constraint-based causal discovery to solve this problem, and yet in practice these methods often face challenges related to multiple testing and error propagation. These issues could be mitigated by a score-based method and thus it has raised great attention whether there exists a score-based greedy search method that can handle the partially observed scenario. In this work, we propose the first score-based greedy search method for the identification of structure involving latent variables with identifiability guarantees. Specifically, we propose Generalized N Factor Model and establish the global consistency: the true structure including latent variables can be identified up to the Markov equivalence class by using score. We then design Latent variable Greedy Equivalence Search (LGES), a greedy search algorithm for this class of model with well-defined operators, which search very efficiently over the graph space to find the optimal structure. Our experiments on both synthetic and real-life data validate the effectiveness of our method.

Published as a conference paper at ICLR 2026
SCORE-BASED GREEDY SEARCH FOR STRUCTURE IDENTIFICATION OF PARTIALLY OBSERVED CAUSAL MODELS
Xinshuai Dong"1
Ignavier Ng +1
Haoyue Dai
Jiaqi Sun'
Xiangehen Songb
Peter Spirtes* Kun Zhang!
1Carnegie Mellon University
"Mohamed bin Zayed University of Artificial Intelligence
ABSTRACT
Identifying the structure of a partially observed causal system is essential to various scientific fields. Recent advances have focused on constraint- causal discovery to solve this problem, and yet in practice these methods often face challenges related to multiple testing and error propagation. These issues could be mitigated by score-based method and thus it has raised great attention whether there exists a score-basee based greedy search method that can handle the partially observed scenario. In this work, propose the first score- based greedy search method for the identification of structure involving latent variables with identifiability guarantces. Specifically. we propose the Generalized N Factor Model and establish its global consistency: the true structure including latent variables can be identified up to the Markov equivalence class by using score. We then design Latent variable Greedy Equivalence Search (LGES), greedy search algorithm for this class models with well-defined operators, which searches very efficiently over the graph space to find the optimal structure. Our experiments
both synthetic and real-life data validate the effectiveness of our method (code will be available at https:/ /tt///giitt github....... com http://cnxxiissiiaaooooom sechhuaaiismmmmmeeee identify).
INTRODUCTION AND RELATED WORK
Causal discovery aims at identifying the causal relations from observational data and it is crucial to many scientific fields (Spirtes et al. 2001 Pearl 2009). However, traditional methods such as PC (Spirtes et al. 2001 GES (Chickering 2002). and LiNGAM (Shimizu al. 2006), rely on the causal sufficiency assumption, i.e., the absence of latent variables, which hardly holds in many real-world scenarios. Therefore, extensive efforts are being made towards structure identification of partially observed causal model.
To handle this problem, the earliest attempts make use conditional independence including Fast Causal Inference (FCT) (Spirtes et al. 2001 Zhang] 2008) and its variants (Colombo et al. 2012 Spirtes al. 2013 Claassen et al. 2013] (013]Ak0 Akbari et al. [222]] D. as well as over-complete ICA- techniques (Hoyer et al. 2008; Salehkaleybar et al. 2020). Yet, these methods only focus on identifiable relations among observed variables, and the results provide limited information about
structure among latent variables.
To this end, recent advance has been on the discovery of the entire structure including latent variables, by introducing additional parametric graphical assumptions. Typical methods include rank or tetrad constraints based under linearity assumption (Silva et al., 20033 2006 Silva & Scheines
2005 Choi al., 2011 Kummerfeld & Ramsey] 2016 Huang al. 2022 Dong et al. (024a).
high-order moments (Shimizu et al. 2009: 2 et al. [2018]]Caa Cai et al. 2019 Salehkaleybar et al.
2020 Xie al. 2020; Adams al.
2021 Dai et al. | Améndola et al.] [023]] Wang & Drton
2023 matrix decomposition (Anandkumar et al. 2013), mixture oracles (Kivva et al. (021)) and
multiple domains (Zeng al. [021]] Sturma al. 2023). Despite of the asymptotic correctness, however, these methods generally fall into the category of constraint- methods; they rely
"Equal contribution.

---

Published as a conference paper at ICLR 2026
heavily statistical tests to iteratively construct the structure and thus suffer from the problem of multiple-testing and error propogation (Spirtes 2010 Colombo et al. 2012), especially with small
sample size and large number of variables.
On the other hand, score-based causal discovery may not suffer from these issues and thus is believed to be more practially favorable (Nandy et al. 2018 Ramsey et al.] (017). One of the most classical
methods is Greedy Equivalence Search (GES) [Chickering] (2002) and yet it cannot handle the
existence of latent variables. Later, various score-based methods that allow the existence of latent
variables have been proposed (Shpitser et al. 2012 Triantatillou & Tsamardinos) 2016 Nowzohour
al., 2017: Bhattacharya et al. [2021a] Shahin & Chechik 2020;; Bernstein et al. 2020 Bellot &
van der Schaar 2021] Claassen & Bucur 2022). and yet they still focus only on relations among
observed variables, except the one by Zhang (2004) without identifiability and another with exact search by Ng et al. (2024) (a more detailed discussion between this work and the exact search (Ng
jtt al. 2024) can be found in Section (A). As a consequence, the following crucial question naturally arises: Is it possible to develop a score-based greedy search method that can efficiently recover the entire underlying structure involving latent variables with an asymptotic correctness guarantee?
To address such challenging problem, we are confronted with three fundamental questions: (i) What is the essential relations between structure identifiability and likelihood scores? (ii) What graphical assumptions are needed to uniquely recover the structure by using score? (iii) How can we design an efficient algorithm to search over the graph space to find the optimal structure? We provide answers to these questions respectively in Sections 3.1 3.2 and 4 and our contributions can be summarized as follows.
We characterize how the likelihood score can be used for the structure identification of partially observed linear causal models. Specifically, we show that the structure with the best likelihood score and minimal dimension is algebraically equivalent to the ground truth (in Theorem IT.
We propose the GNFM (in Def. 2 and accordingly establish the global consistency of score for it the whole underlying structure can be uniquely recovered up to the bMarkov Equivalence Class (MEC) by using the score (in Theorem and Corollary TT. This graphical condition is rather mild and takes the prevalent one factor model (Silva et al. 2003) as a special case.
We develop the Latent variable Greedy Equivalence Search (LGES), an efficient causal discovery algorithm for identifying structure involving latent variables. To our best knowledge, this is the first score- based greedy search with identifiability guarantees in the partially observed scenario (in Thm. Our experiments on both synthetic and real-world dataset empirically validate its effectiveness.
3.
2 PRELIMINARIES
2.1 PROBLEM SETTING
We aim to identify the structure of a partially observed linear causal model, defined as follows.
Definition (Partially Observed Linear Causal Models). Let G (Vg, Eg) be a Directed Acyelic Graph (DAG) and variables follow a linear SEM as VG F V ev where VG LG UXg- [V.) = (L.)T U (X+]i contains m latent variables and n observed variables, F = (Jy,)) is the weighted adjacency matrix and fa, + O if and only if V, is a parent of V in G, and ev, represents the Gaussian noise term V..
Our goal is to identify the underlying structure G over all the variables LgUXg, given i.i.d. samples of observed variables Xg only. Note that the name/order of latent variables can never be identified so we focus on structure identification up to permutation of latent variables. Without loss of generality, we can assume that all variables have zero mean, and thus the observational data can also be summarized as the empirical covariance matrix over observed variables, i.e., Exe- We use V and V, denote random variable and a set of variables, respectively. We drop the subscript G in Lg and Xg when the context is clear. For a matrix M, we define its support set as sM))(M 91 and G2 belong to the same MEC iff they share the same skeleton and set of v-structures (ín Definition 9 over the entire graph including latent variables.
[(i,]) : Mi. 4 0}.
2.2 LIKELIHOOD SCORE
Despite the asymptotic correctness of constraint- based causal discovery approaches, in practice these methods often suffer from the problem of multi-testing and error propagation (Spirtes) 2010 Colombo
2

---

Published as a conference paper at ICLR 2026
et al. 2012). In the finite sample case, they rely heavily statistical tests iteratively build the result, while the power of each test might be limited especially when the sample size is small and number of variables is large. On the contrary, coore-based causal discovery methods may not suffer from these problems and could be practically more favorable (Nandy et al. 2018 Ramsey et al.
2017), especially when the sample size is small (also empirically validated in Section 5.2)
A more
detailed discussion about error propagation can be found in Section C.8
To this end, in this work we aim at structure identification based on the use of likelihood scores in the partially observed scenario. In contrast to the fully observed case, in the presence of latent variables, the formulation of the likelihood is not trivial, and we provide it in what follows.
Proposition 1 (Parameterization of Population Covariance Dong et al. 2024b)). Consider the model and let F= FLL FXL
defined in Def.
FLX) ) and Ω (og-
where Se is the diagonal covariance
Dex matrix of g. Let M =(1 FLL- Fix - Fxx)=111 (xx))
(((I F
Fxx) -Fx))) -1, and Vi, M M + N NT N. Then the population covariance of X can be formulated as
Σx = (1- FLXXLLFX + NFx + Oxx +
-x-x) (1)
By making use of Proposition to parametrize Σx, the maximum log-likelihood of a given structure G and observation Σx is as follows (tr and det refers to matrix trace and determinant, respectively).
scoreML (G, Σx)=
max (F.R): supp(T)(<upp(F Fa) Reding
C.
(2)
f= (N-(N/2)(((Ex)> + log det Ex).
We next show the theoretical foundation of using maximum likelihood score for the structure identification of partially observed linear causal models.
SCORE-BASED IDENTIFIABILITY THEORY FOR PARTIALLY OBSERVED CAUSAL MODELS
3.1 ALGEBRAIC EQUIVALENCE BY SCORE AND DIMENSION
Consider a model in Def. Its structure imposes various types of equality (i.e., algebraic) con- straints on the covariance matrices (over observed variables). no matter how its parameters (F, S2) may change. The imposed equality constraints are properties of the observational distribution and contain crucial graphical information about the underlying structure G. Such constraints include conditional independence (i.e., vanishing partial correlation) constraints (Spirtes et al., 2001 rank constraints (i.e., vanishing determinant) (Spirtes et al., 2001 Sullivant et al. 2010), and possibly Verma constraints (Verma & Pearl [99I). An overview can be found in Drton 2018).
Let (g) be the set of equality constraints imposed by structure G on the generated covariance matrices over observed variables (detailed in Definition [6) G be the set of all DAG structures that has n measured variables, and H = USe BG)), where B(G) consists of the canonical and minimal set of equality constraints with respect to reduced Gröbner basis (detailed Definition 7). We say two structures Gi and G2 are algebraically equivalent, if they lead to the same equality constraints (on the observational distribution), i.e., (91) H (G2) (van Ommen & Mooij] 2017). Similar to the classical CI faithfulness assumption in causal discovery (Spirtes et al. 2001), to better relate the constraints to the underlying structure, we assume the generalized faithfulness as follows.
Assumption 1 (Generalized faithfulness (Ghassami al. 2020). A distribution Σx is said to be generalized to DAG G E T if the entries of Σx satisfy an equality constraint E H" only if E H9).
In Assumption T. it suffices to use Σx to denote the distribution, as X are jointly gaussian and mean do not contain any information about structure (Ghassami et al. 2020). Note that different types of faithfulness assumptions have been adopted causal discovery, e.g., CI faithfulness and rank faithfulness (Spirtes et al.) 2001 Ghassami et al.) 2020 Huang et al. 2022 Dong et al. 2024a) and Assumption I is the generalized version of them for linear causal models. Similar to Cl faithfulness, generalized faithfulness is justified by that the set of parameters that result in violation
has Lebesgue measure zero (Ghassami et al., 2020) and it has been widely adopted in the field
(Ng al. 2020 BBhattachryya et al. 2021b: Sethuraman al. 2023). On the other hand, without
3

---

Published as a conference paper at ICLR 2026
4
4
X2
X
X2
X₂
X
X
Xs
(a) Ground truth g"
(b) 9i by Theorem
$2 by Theorem
Figure 1: Without further graphical assumption, the algebraic equivalence class is very large and not very informative: suppose the ground truth G in (a), by Theorem we may arrive at either 91 (b) or S2 (c), both are algebraically equivalent to G
faithfulness, graphical information extracted from observations cannot be trusted, which makes structure identification extremely hard, if not impossible.
Furthermore, let di) denote the model dimension or degrees of freedom of DAG G for the marginal over the observed variables (which can also be viewed the number of free parameters of the set of distribution it can generate). In the absence of latent variables, the degrees of freedom are nothing but the sum of the number of edges and the number measured variables. However, in the presence of latent variables, it does not necessarily hold (Geiger et al. 1996] 2001]. Without any specific graphical assumption, capturing the dimension could be highly non-trivial; e-g., the analysis of the degrees of freedom for sparse factor analysis, where latent variables are independent, already involves complex techniques from algebraic statisties (Drton et al., 2023). note that the focus of this work is not to characterize the dimension of each structure. In contrast, only need to know the basic idea of dimension here and later we will show that greedy search does not necessarily rely on knowing the exact dimension of graph.
Now we are ready to present the key result of this subsection, achieving algebraic equivalence by making use of the likelihood score, captured in the following theorem.
Theorem (Algebraic Equivalence by Score and Dimension). Suppose a model follows Desinition 7 with G and distribution Vi satisfies the generalized fuithflte assumption. Given observation Σx and let G arg maxgeg" scoreML(9) Σx). If G* and 9 E arg minges. dim(g), then & and G are algebraic equivalent, i.e., (9) (9*), in the large sample limit.
Remark 1. Theorem I (partially inspired by Ng al. 2024 says that, if 9 can generate the observation Σx, and G has the smallest dimension among those graphs that can generate the observation, then & and G are algebraically equivalent. other words, we can enumerate all the graphs in the assumed graph space, and utilize Theorem [ to find graph & that is algebraically equivalent to the ground truth G" In the absence of latent variables, if 9 and g* are algebraically equivalent, G and G* belong to the same MEC, and thus we can make use of score identify the structure up to MEC. In this sense, Theorem iakes the theoretical guarantee of score, e.g., in GES Chickering 2002) as a special case and generalizes it to the partially observed scenario.
Algebraic equivalence is general sense of equivalence: if two graphs are algebraically equivalent, then we cannot differentiate them purely by observational data without any further assumption. The reason lies in that, in the linear gaussian case, all the information from the distribution just equality and inequality constraints (in Definition (). and generally inequality constraints cannot be used; we have to assume parameters that results in violation of such faithfulness is not of Lebesgue measure zero.
conpuuaitty-ontttti---t faithfulness utilize ceequullty-constrriin but the set of
At this point, we have characterized how the likelihood score can be used for structure identification in the partially observed scenario. Yet, there still exist two main challenges. First, in the partially observed scenario, without any graphical assumption, relating 9 to G" can be very challenging, as the algebraic equivalence class is very large. For example, suppose the ground truth graph is g in I (a). Even though by making use of all the equality constraints, we still just arrive some elements of the algebraic equivalence class of G*, e.g., Gn or G2 in Figure [bb) and (c). In fact, without any further graphical assumption, the cardinality of the class is infinity, as we can always add more latent variable to the structure without changing any constraints. Therefore, general recipe
Figure
4

---

Published as a conference paper at ICLR 2026
may involve identifying suitable structural assumptions that allow algebraic equivalence to translate into more fine -raieegridd notions of model equivalence, such as Markov equivalence, as in Section 3.2
Second, Theorem onnly implies search procedure that requires the exact enumeration of all possible graphs in the assumed graph space. Yet, such an exact search is impractical due to the computational overhead. To be specific, the number of possible graphs grows super-exponentially with the increase of the number of observed variables, even when we rule out a lot of structures with latent variables that cannot be identified (e.g., X1 + L1 + X2 where we can never know whether L exists or not). More specifically, if we consider the GNFM (which will be detailed in the next section with definition in Definition 2 with 10 observed variables, the number of all possible graphs that satisfy Definition 2 is more than 3 x 10a and this number grows to 2 x 10 when the number of observed variables increase only by 1. Therefore, it is crucial to design an efficient way to search over the graph space for the identification of the optimal structure, as in Section
3.2 GRAPHICAL ASSUMPTION AND GLOBAL CONSISTENCY
In this section, we propose the generalized N factor model, a graphical condition under which algebraic equivalence can be translated into Markov equivalence, defined as follows.
Definition 2 (Generalized N Factor Model). DAG satisfies the definition of generalized N factor model if observed variables are the effects of latent variables and there exists a partition of all latent variables Lg such that for each element the partition, Lp. (i) there exist at least [Lp * observed variables X2 such that for all X E Xp, Pag(X)= = Lg. (ii) if a variable V E Vg causes or is caused by another variable in Lp, then also causes or is caused by all the variables in Lp. respectively, and (iii) elements in Lp are muiually nonadjacent.
For brevity, in the rest of the paper, definition of generalized N factor model. For a better understanding of the generalized N factor model, we provide an example to elaborate each requirement in Definition 2
GONEM to denote the set of all graphs that satisfy the
Fig. 2 shows an illustrative graph that satisfies Def. 2 Specifically, all the observed variables in Figure 2 are leaf nodes and caused by only latent variables. Fur- ther, all latent variables can be partitioned into groups {L1}. { {L3}. Ls, (Ls, L7] and thus the rest re- quirements (i), (ii,, (iii) are also satisfied. For example, for I, = (Ls, L7). there exist X, (X15. X16. X17, X18} such that their parents are (Ls, L7] and X,I 2 LL x 2. These groups have the properties that the relation within a group can not be identified, and the relation between groups applies to every element in the group. Thus, re- quirements in Def. 2 are satisfied in Figure 2
Figure 2: An illustrative example of the graph that satisfies generalized N factor model in Definition 2
Notably, the graphical condition required for generalized N factor model is rather weak. It takes the prevalent one factor model assumption (Silva et al., (003) (defined in Definition and illustrated in Section C.2 for comparison) as a special case, and further allows latent variables share observed variables as children. In GNFM, we want to make very weak assumption about how latent variables are related they can be partitioned into groups, and the relation among each group of latent variables can be very flexible. At this premise, the key requirement of Lp observed children of Lp in GFNM minimal. On the other hand, if we make stronger assumption about how latent variables are related to each others than what has been made in GNFM, the requirement of having at least twice number of observed children can be relaxed.
Here we take {L3} in Fig. 2 as Lp to show why we need 2) ا observed children of Lp If we want to determine whether two Tatent groups }Li} and [La, are adjacent, we need to check whether (Lr) } and {L4, Ls} can be d-separated by (Lz). which can be done by checking whether there exists a pure observed child of {L1) (c.g.. X1) and another pure observed child of {L4, レ5} (c.g.. Xio) can be d-separated by {La} As {La} cannot be observed, we have to rely t-separation (Sullivant et al. 2010) to check whether this d-separation holds, as t-separation allows us to use the observed children of [Ly} surrogates. This can be translated into checking whether there exists two distinct groups X,, and X,, as the observed pure children of [La] such that, [X,] = X [L and (X,) } and (X10) can be t-separated by (X., X.). Thus, for each Lp we need 2 L observed children of it.
5

---

Published as a conference paper at ICLR 2026
The reason why GNFM chooses this specific trade-off point is that we believe it is more practically meaningful. In real-life problems, the way latent variables are related could be complicated and we do not want to rule out the possibility of certain structural patterns in advance. At the same time, the number of observed children is insufficient, we can still gather more relevant observations measurements of the underlying system.
Next, we show in Theorem 2 that for GNFM, the notion of algebraic equivalence leads to Markov equivalence, and thus by using score we can identify the structure up to MEC, as in Corollary [
Theorem 2 (Identifiability of Generalized N Factor Models by Equality Constraint up MEC). For G1 S2 E GGNFM, if they are algebraically equivalent, i.e., ( = H (G2), then G1 and G2 belong to the same MEC (same skeleton and v-structures over all variables).
Corollary 1 (Global Consistency by Score for Generalized N Factor Models). Suppose model follows Definition Srrioomwiiiii with 9 E GGNFM and distribution E satisfies Assumption /
Given observation
Ex and let G = arg maxgeGrow scorem: (9, Ex). 1ʃ E G* and & arg mingee- dim(g), then & and 9 are Markov equivalent in the large sample limit.
Corollary essaablishe the global consistency of using likelihood score for structure identification. Yet, it still requires impractical exact enumeration. Furthermore, it is also unclear how to calculate the exact dimension for each graph with latent variables. Fortunately, an efficient greedy search can be designed to identify the optimal structure without capturing the dimension, shown in what follows.
4
SCORE-BASED GREEDY SEARCH FOR PARTIALLY OBSERVED LINEAR CAUSAL MODELS
We begin with the general design for greedy search in Section 4.1
specify the design in LGES for generalized N factor model in Section 4.2 and establish the asymptotic correctness in Section 4.3
4.1 GENERAL DESIGN FOR GREEDY SEARCH
Our overall objective is to find a graph such that it can generate Σx equally well as the ground truth G", while having dimension that as small as possible (as in Theorem
[ and Corollary [ To efficiently search over the graph space, we follow the traditional wisdom GES
(Chickering 2002) and define three key elements for a greedy search.
(a) Smn by Def. 3. where latents are mutu- ally fully connected by undirected edges and all latents cause all observed.
A set of states.
A representation scheme for the states.
A set of operators.
A state represents a solution to the search problem and we use S to represent state and S to represent a set of states. The representation scheme defines efficient way to represent the states. As the structure can only be identified up MEC, follow GES such that each state MEC, which corresponds to an unique Completed Partially Directed Graph (CPDAG in Definition TT). Thus we also use S to refer to a CPDAG and S(G) to transform a DAG into CPDAG. Finally, the set of operators is to transform one state to another state, in order to traverse the whole graph space systematically and efficiently.
Sphwel. the output of Alg. where the number latent and the edges from latent to observed variables are determined.
For any two graphs belong to the same state (i.e., the same MEC), they share the same dimension and maximum like- lihood score (Prop. 1 in (Ng et al., 2024)). Thus, we also define scoreML for CPDAG S, scoremL (S, Σx) score (G. Ex), for all G MEC(S).
(c) Smat. the output of Alg. represents the MEC the ground truth 9" (here the MRC only contains G*).
Figure 3: LGES illustration, where (a) is the initial state of Alg. [, (b) is the output of Alg. and (c) is the final output.
Furthermore, the initial state Smm is often designed as a state that is super graph of the ground truth, and Smin can

---

Published as a conference paper at ICLR 2026
generate the observation equally well as the ground truth. For example, the phase 1 of GES is to find such an initial state and thus the phase 2 of GES can focus on the delete operation. Next, we introduce the detailed design of our novel method, LGES.
4.2 ALGORITHM: LATENT VARIABLE GREEDY EQUIVALENCE SEARCH (LGES)
In this section we discuss the detailed design for Latent variable Greedy Equivalence Search (LGES) for structure identification of generalized N factor models. We begin with the initial state nnit-
Definition 3 (Initial State for Generalized N Factor Model). Given X, Smit () outputs a CPDAG
such that it contains observed variables X and 2 latent variables, all latent variables are fully connected with undirected edge, and all latent variables cause all observed variables.
In Definition i implicitly requires that the number of observed variables is at least twice the number
of latent ones. We note that this latent-to-observe ratio is a property of the graphical assumption of GNFM in Definition 2 and thus invariant to the design of a method. An example of Sma (X) can be
found in Figure 3 where (c) is the ground truth g and (a) is the initial state Smi (X) by Definition 3
The reason why design Smi as such lies in the properties of Sma formalized in Lemma]
Lemma 1 (Properties of Initial State). Suppose model follows Definition | with G E @GNNFM and we are given observation Σx. Then Smat is supergraph of S(G*) and Smi can generate the observed distribution, i.e., score (Siit Σx)= maxgeGown sc (9, Σx) in the large sample limit.
The core spirit of LGES is that we begin with a state that can generate the observation. Each time we delete some edges and see whether the new CPDAG can still generate the observation. If so, we keep it as the new state; otherwise we try a different deletion. As an edge deletion leads to smaller or equal dimension, the process in essence finding the CPDAG with smallest dimension while keeping the ability of generating the observation. Next we discuss the operators that connect between states.
Definition 4 (Delete Operator OLx). OL returns a CPDAG that is the same as S except that all edges from L X are deleted.
Definition 5 (Delete Operator OLL). OLL L2, H) returns a CPDAG that is the same as S except that (i) all edges between Lı and L2 are deleted, (ii) for each H E H, directing the previously undirected edge between L1 and H as L1 - H and directing the previously undirected edge between L2 and H as L2 + H.
OLx is designed to delete edges from latent variables observed variables while OLL (partially inspired by CChikkring (2002) is designed to delete relations among two groups of latent variables L and Lg. Our LGES is built upon these two operators and includes two phases. Phase 1 (summarized in Alg D is to recover the structure between latent variables and observed variables. Roughly speaking, each time the algorithm chooses some latent variables and some observed variables from the current state, deletes all the edges between them to get a neighbouring state, and check whether this neighbouring state can generate the observation Ex (up to a certain tolerance level o). If so, this neighboring state is taken as the new state; otherwise the algorithm looks for a new combination for edge deletion. An example of the output of phase is in Fig 3 (b), where the structure between latent and observed are expected to be the same as the ground truth (also formalized in Lemma 2
Phase 2 (summarized in Alg 2 of LGES, which is partially inspired by Chickering (2002) aims to identify the structure among latent variables. Roughly, each time the algorithm chooses two groups of latent variables, deletes all the edges between them to get a neighbouring state; if this state can generate the observation (up to o), then takes it as the new state; otherwise looks for another two groups. d controls the sparsity level in the finite sample case: with bigger ó, deletions are more likely to be kept (ablation study in Section 5.2). An illustration of the output of phase 2 is in Fig (c). which is expected be the same as the ground truth (which is formalized in Theorem B).
As implied by Theorem our objective is search over the graph space to find graph & such that (i) 9 has the best likelihood and (ii) at the premise of ensuring (i), the dimension of should be as small as possible. Below are the key designs of LGES to ensure this goal.
We start with a graph that is guaranteed to be a super graph of the ground truth.
At each step, we try to delete some edges. Only when the likelihood after the deletion is still the best do we keep the deletion. This guarantees that (i) always holds.
7

---

Published as a conference paper at ICLR 2026
Table 1: F1 score and SHD (mean(standard error) of each compared method (LGES, FOFC, GIN,
and RLCD) across different sample sizes. 1 means the bigger the better while + the smaller the better.
FI score for skeleton 1
SHD for MEC
Sample size
LGES
FOFC
GIN
RLCD
LGES
FDFC
GIN
RLCD
100
0.60 (0.01)
0.55 (0.02)
0.27 (0.01)
0.33
20.87 (0.72)
(0.34)
(0.36)
(300)
200
0.72 (0.02)
0.50 (0.02)
0.36 (0.01)
0.49 (0.02)
14.03 (0.85)
(0.29
24.14(0043)
28.58 (2.59)
500
0.79 (0.02)
0.58 (0.02)
0.42 (0.01)
0.69 (0.02)
10.02 (0.69)
16.5 (0.31)
22.46 (0.39)
14.68 (0.96)
1000
0.82 (0.02)
0.61 (0.03)
0.47 (0.0))
0.76 (0.02)
(0.70)
(0.29)
20.88 (0.54)
11.24
Table 2: Under model misspecification, Fl score and SHD (mean error)) of each compared
method across different sample sizes. 1 means the bigger the better while + the smaller the better.
Non-gaussian
FI score for skeleson
SHD for MEC 1
Sample size
LGES 0.57 (0.03)
FOFC
GIN
RLCD
LGES
FOFC
GIN
RLCD
100
0.40 (0.03)
0.27 (0.01)
0.35 (0.01)
21.70
22.72 (0.34)
26.76 (0.36)
39,31 (7.37)
200
0.74 (0.04)
0.42 (0.03)
0.36 (0.01)
(0.03)
12.67 (1.97
(0.50
24.14 (0.45)
35.12.66.52
500
0.78 (0.02)
0.40 (0.03)
0.42 (0.01)
0.68 (0.01)
10.72 (0.93)
17.66 (0.49)
22.46 (0.39)
14.81 (0.63)
1000
0.79 (0.02)
0.39 (0.05)
0.47 (0.01)
0.75 (0.02)
10.12 (0.82)
17.82 (0.44)
20.88 (U.54)
12.45 (0.77)
Non-linear
Fl score for skeleton
SHD for MFC J.
Sample size
LGES
FOFC
GIN
RLCD
LGES
FOFC
GIN
RLCD
100
0.58 (0.02)
0,35 (0.03)
(6.2000011)
0 (0.02)
(1
21
29.28 (1.11)
(4.77)
200
0.65 (0.01)
0.33 (0.03)
0.38 (0.01)
0.45(0.02)
(L.1)
21,80 (0.41)
27.28 (0.28)
31.73 (4.33)
500
0.68 (0.02)
0.26 (0.04)
0.50 (0.02)
0.60 (U.02)
16.6
19.98 (0.46)
23.38 (0.64)
19.89 (1.08)
1000
0.71 (0.02)
0.23 (0.04)
0.52 (0.02)
0.65 (0.01)
(0.55)
(0.54)
23.32 (0.73)
17.56 (0.77)
An edge deletion operation will either keep the dimension the same or decrease the dimension. Thus, throughout our search process, the dimension will decrease monotonically.
The design of deletion operators ensure at each step, the current graph is always a super-graph of the ground truth, through out the whole process. This is because if the post-deletion graph is not super-graph of the ground truth, additional equality constraints will be introduced such that the post-deletion graph cannot reach the best likelihood and thus this deletion will not be kept. By the end of the process, LGES will arrive the ground truth. This can be proved by contradiction in a rough sense as follows (detailed in proof). Suppose the final state g' is not optimal. As it must be super graph of g", there must exist sequence of deletion to transform G' into g". Suppose the first deletion in the sequence leads to G". As G" is also a super graph of G", the score of g" is also the best, and thus the algorithm would not terminate at G', yielding a contradiction.
4.3 ASYMPTOTIC CORRECTNESS OF LGES
Here we establish the asymptotic correctness of LGES. Specifically, if the ground truth satisfies the generalized N factor model, LGES can asymptotically produce the correct Markov equivalence class over both observed and latent variables, as in Lemma and Theorem 3 (all proofs in Appendix).
Lemma 2 (Correctness of Phase of LGES). Suppose model follows Definition wwith G &GGNrM and we are given observation Σx. In the large sample limit the output Sphasel of Algorithm is CPDAG such that the number of latent variables in Sphaset is the same that of g and the edges from X to L in Sphasel is the same as that of " up to permutation of latent variables.
Theorem 3 (Correctness of LGES). Suppose a model follows Definition 7 with G &GNFY and we are given observation Σx. In the large sample limit the output Symat of Algorithms 1 and 2 is CPDAG that represent the MEC of g", up to permutation of latent variables.
EXPERIMENTS
5.1 SYNTHETIC SETTING AND EVALUATION METRIC
In our synthetic experiments, we validate LGES by comparing it with multiple latent variable causal discovery methods, including FOFC (Kummerfeld & Ramsey 2016). GIN (Xic et al.] 2020), and RLCD (Dong et al. 2024a). The ground truth model follows Def. T where each edge coefficient fzx is randomly sampled uniformly from [-5, 5) and the variance for each noise term uniformly from [0.1, 1]. As GIN assumes non-gaussianity, we use uniform distribution for the noise when testing GIN. We consider 20 randomly generated structures that satisfy Def. 2 as the ground truth structure (examples in Fig. g in Appendix). On average, each ground truth graph contains 20 variables and 5 of them are latent. For the output of each method, we first transform the output into a CPDAG and then compare it with the ground truth CPDAG by calculating Fl score over the skeleton and SHD over the MEC evaluation metrics, and consider four different sample sizes: 100, 200, 500,
8

---

Published as a conference paper at ICLR 2026
Table 3: Model fitness scores on real-life datasets to validate LGES (t the bigger the better).
Deet
Scenarios
Model
Fitness Scores
Dataset
Structures
RMSEA
CH1
TLI1
Big Five
Pigur by 1.GR
0,054
0.874
0.855
1h Golthen
0.072
0.767
0.746
T Burnout
Figure Js LGBS
0.067
0,876
0.865
By Byre
(194
0.096
0.753
0.727
Rx Rune
0.072
0.861
0.847
Guedie
M-tasking
Liaks
0.068
0.977
0.965
Figure 4: Causal structure (CPDAG) recovered by LGES Multi-tasking behavior dataset.
0.087
0.962
0.943
Extraversion
Openness
Ematiinnal Eshauttion
Depersonaization
Climato
[C1]
Agr Agreeableness
Peer Support
(M0 mutwek
Conscientiouseess
M [M
Neuroticism
De Fid
Self liff-Gdeem
Externel Locus
Pereonal Accempfiismment
XQ
Figure 5: Causal structure (CPDAG) recovered by
Figure 6: Causal structure (CPDAG) recovered by LGES on Teacher burnout dataset.
LGES on Big Five personality dataset.
1000. We adopt 10 random seeds to generate the ground truth causal model and observational data, and report the mean and standard error.
5.2 PERFORMANCE ON SYNTHETIC DATA AND ABLATION STUDY ON 6
The Fl scores of skeletons and SHDs of MECs are reported in Table I As shown, the proposed LGES achieves the best Fll score (bigger better) and SHD (smaller better) performance compared to all baselines. For example, with sample size 1k, LGES achieves a F1 score of 0.82 and SHD of 8.8, where the runner-up achieves 0.76 and 11.24 respectively. Another key observation is that score- score-based method can still work well with very small sample size, while custraint-bbsse methods may not. F.g., with only 100 datapoints, LGES still achieves a Fl score of 0.60, while constraint-based method GIN (CI-rest-based) and RLCD and 0.33 respectively. The reason why constraint-bad based methods do not work well with small sample size may lie in that with small sample size the power of the test is limited and and the null distribution may be very different from the asymptotic case, both of which aggravate the issue of error propagation. On the contrary, our score-based method may not suffer from these issues.
only achieves 0.27
Similar to the syper-paramette X in GES, in practice we can tune the value of 5 to control the sparsity level of the result (the bigger 5 the sparser). In our experiments, 8 is set as 5 = 0..55 log(N) N is the sample size. This design follows the spirit of BIC score in GES such that 5 + when N + co. The ablation study analyze the sensitivity to 8 can be found in Table where LGES is not very sensitive to small change of 5. A more detailed discussion about g can be found in Section C.5
.V' where
5.3 MISSPECIFICATION BEHAVIOR
We investigate the performance of LGES under model misspecification: violation of normality and linearity. For the setting of violation of normality, we use uniform noise terms for the underlying model, and report the result in Table 2 Specifically, when the normality is violated, LGES still performs the best compared to baselines and the result is almost the same as that of the gaussian case. For example, with 1k sample size, the Fl score of LGES under non-gaussianity is 0.79 while the counter part under the standard setting is 0.82. This is not very unexpected: the structure identifiability by score is built upon the hard constraints imposed by structure on the observational covariance matrix, which only relies on the linearity of the underlying causal model and does not rely on gaussianity. As for violation of linearity, we employ leaky ReLU (Xul 2015) to simulate piecewise linear function, as V; LR(EVEER(V)) V Ev,). LR(x) max 2z), where
= 0.8. The result is in Table 2 which shows that LGES works reasonably well under certain extent of non- -oneaiirr,,,y, and still surpasses all baselines. For example, with sample size 1k, LGES still achieves 0.71 F1 score even under nonlinearity, while the runner-up achieves 0.65.

---

Published as a conference paper at ICLR 2026
5.4 REAL-WORLD PERFORMANCE
We consider three real-life datasets. Big Five personality dataset (openpaychonetticc.. orgg). which consists of 50 questions with 19,719 datapoints. There are five dimensions: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (O-C-E-A-N), and each dimension with 10 questions (e.g.. O1 is the first question for Openness). We use the first 5 questions for each dimension and in total 25 variables. The structure by LGES is shown in Figure 5 Interestingly, without any prior knowledge, the structure recovered by LGGSS is very aligned with psychology study. To be specific, each item in our result is indeed caused by the supposed dimension (latent variable). Further, we found that some items are caused not only by one latent variable. For example, E3 (I feel comfortable around people) is caused by L4 (Extraversion), L5 (Agreeableness), and L6 (Neuroticism), which may shed new light on the existing factor- factor-analysis analysis-based psychometric study.
Teacher burnout dataset (Byme 2001). The term burnout refers to the inability to perform effectively in one's job due to job-related stress. The dataset includes 32 observed variables with 599 datapoints. Multi-tasking behavior dataset (Himi et al. (019a). To compare with the model proposed by Himi al. (2019a), we use variables of it with all its 202 datapoints. The structures produced by LGGSS for teacher burnout data and multi-tasking data are shown in Figures 4 I respecttyy Finally, we use three prevalent goodness fit statistics to validate the structure produced by LGES: RMSEA
(Steiger 1980 1990), CFI (Bentler 1990), and TLI (Tucker & Lewis 1973 Bentler & Bonett 1980) (detailed in Section C.4. We used these fit indices to compare the structure produced by our method with the well-known hypothesized structures in existing psychological studies. The result is reported
in Table 3 where the structures by LGES achieve the best scores compared to all the structures proposed psychological research, shows that the structures by LGES explain the observational data better than existing study, and validates the proposed method in real-life scenarios.
5.5 COMPARISON WITH THE EXACT SEARCH
In this section we compare the proposed L.GES the score-based exact search method (Ng et al. 2024). The exact search has to enumerate all the possible candidate graphs and thus the complexity grows super-exponentiall with the increase of the number of observed variables [Xçl (which is why we do not include the exact search in our main result in Table []. In contrast to exact search, LGES search over the candidate graph space in a properly designed way such that each time we only need to decide greedily while the asymptotic correctness can still be guaranteed.
The result is shown in Table 4 Specifically, to handle single dataset with observed variables, the exact search requires more than 100 hours while LGES only requires 17 seconds. When it comes to 9 observed variables, the exact search requires more than 1000 hours (by estimation) while LGESS only requires 19 seconds. As for the causal discovery performance measured by Fl and SHD, the exact search only performs slightly better than LGES. For example, for 7 observed variables, the Fl score of the exact search is 0.89 while the F1 of LGES is 0.88; yet, the the exact search requires nearly 400 times more time (1.5 hours v.s. 14 seconds). This empirical result demonstrates the significant computation efficiency improvement of LGES against the exact search, with only slight degradation in the performance.
5.6 IMPLEMENTATION DETAILS, RUNTIME ANALYSIS, AND EXTENDABILITY
The readers are referred to Section C.5 and Section C.6 for implementation details and runtime analysis, respectively. As for the discussion about the extendability of the proposed method to non-Gaussian or non-linear scenarios, please refer to Section C.7
6 CONCLUSION
In this paper, we first characterize how likelihood score and minimal dimension are related to the structure identifiability of partially observed linear causal models. Then we propose Generalized N Factor Model under which we prove the global consistency of using score for structure identification. Finally we propose LGES, an asymptotically correct score- greedy search method to efficiently search over the graph space and identify the causal structure.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
We would also like to acknowledge the support from NSF Award No. 2229881, Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract RO1 and grants from Quris AI, Florin Court Capital, MBZUAI-WIS Joint Program, and the Al Deira Causal Education project.
1,99881.44 AI Institute for Societal
REFERENCES
Jeffrey Adams, Niels Hansen, and Kun Zhang. Identification of partially observed linear causal models: Graphical conditions for the non- -ongsassiian and heterogeneous cases. Advances Neural Information Processing Systems, 01122222-228333 2021.
Sina Akbari, Ehsan Mokhtarian, AmirEmad Ghassami, and Negar Kiyavash. Recursive causal structure learning in the presence of latent variables and selection bias. Advances in Neural Information Processing Systems, 34:101 19-10130, 2021.
Carlos Améndola, Mathias Drton, Alexandros Grosdos, Roser Homs, and Elina Robeva. Third-order moment varieties of linear con-aaussian graphical models. Information and Inference: Journal of the IMA, 12(3):aaad007, 2023.
Animashree Anandkumar, Daniel Hsu, Adel Javanmard, and Sham Kakade. Learning linear bayesian networks with latent variables. In International Conference on Machine Learning, pp. 249-25192
PMLR, 2013,
Kevin Bello, Bryon Aragam, and Pradeep Ravikumar. DAGMA: Learning DAGs via M-matrices and a log- degemeetrminant acyclicity characterization. In Advances Neural Information Processing Systems, 2022.
Alexis Bellot and Mihaela van der Schaar. Deconfounded score method: Scoring dags with dense unobserved confounding. arXiv preprint arXiv:2103.15106, 2021.
Peter Bentler. Comparative fit indexes in structural models. Psychological bulletin, 107(2):238, 1990.
Peter M Bentler and Douglas Bonett. Significance tests and goodness of fit in the analysis of covariance structures. Psychological bulletin, 88(3):588, 1980.
Daniel Bernstein, Basil Saeed, Chandler Squires, and Caroline Uhler. Ordering- baasdd causal structure learning in the presence of latent variables. In International Conference Artificial Intelligence and Statistics, pp. 4098-4108. PMLR, 2020.
Rohit Bhattacharya, Tushar Nagarajan, Daniel Malinsky, and Ilya Shpitser. Differentiable causal discovery under unmeasured confounding. In International Conference on Artificial Intelligence and Statistics, 2021a.
Rohit Bhattacharya, Tushar Nagarajan, Daniel Malinsky, and Ilya Shpitser. Differentiable causal discovery under unmeasured confounding. In International Conference Artificial Intelligence and Statistics, pp. 2314 2322. PMLR, 2021b
Philippe Brouillard, Sébastien Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexan- dre Drouin. Differentiable causal discovery from interventional data. In Advances in Neural Information Processing Systems, 2020.
Barbara M. Byrne Burnout: testing for the validity, replication, and invariance of causal structure across elementary, intermediate, and secondary teachers. Am. Educ. Res. J., 31(3):645-677 673,, 1994. doi: 10.3102/000283) 203 1003645.
Barbara M. Byrne Structural Equation Modeling with Amos: Basic Concepts, Applications, and Programming. Multivariate Application Series. Toutledge, Taylor and Francis Group, New York London, 2nd edition, 2010. ISBN 978-0-8058-6372-7.

---

Published as a conference paper at ICLR 2026
B.M. Byrne Structural Equation Modeling With AMOS: Basic Concepts, Applications, and Pro-
gramming. Multivariate Applications Series. Taylor Francis, 2001.
Ruichu Cai, Feng Xie, Clark Glymour, Zhifeng Hao, and Kun Zhang. Triad constraints for learning causal structure of latent variables. Advances in neural information processing systems, 32, 2019.
David M. Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 30N0v):507-554, 2002.
Myung Jin Choi, Vincent YF Tan, Animashree Anandkumar, and Alan S Willsky, Learning latent tree graphical models. Journal of Machine Learning Research, 12:1771-1812, 2011.
Tom Claassen and Ioan G Bucur. Greedy equivalence search the presence of latent confounders. In Conference on Uncertainty in Artificial Intelligence, 2022.
Tom Claassen, Joris Mooij, and Tom Heskes. Learning sparse causal models is not np-hard, arXiv preprint 11x.vvv1309.8844 2013.
Diego Colombo, Marloes Maathuis, Markus Kalisch, and Thomas S Richardson. Learning high- dimensional directed acyclic graphs with latent and selection variables. The Annals of Statistics, pp. 294-321, 2012.
James Cussens. Bayesian network learning with cutting planes. arXiv preprint arXiv: 2012.
Haoyue Dai, Peter Spirtes, and Kun Zhang. Independence testing- approach causal discovery under measurement error and linear non-gaussian models. Advances in Neural Information Processing Systems, 35:27524-27536, 2022.
Chang Deng, Kevin Bello, Bryon Aragam, and Pradeep Kumar Ravikumar. Optimizing NOTEARS objectives via topological swaps. In International Conference Machine Learning, 2023.
Yanming Di. t-separation and d-separation for directed acyclic graphs. preprint, 2009.
Xinshuai Dong, Biwei Huang, Ignavier Ng, Xiangchen Song, Yujia Zheng, Songyao Jin, Roberto Legaspi, Peter Spirtes, and Kun Zhang. A versatile causal discovery framework to allow causally- related hidden variables. 2024a.
Xinshuai Dong, Ignavier Ng, Biwei Huang. Yuewen Sun, Songyao Jin, Roberto Legaspi, Peter Spirtes, and Kun Zhang. On the parameter identifiability of partially observed linear causal models. 2024b.
Xinshuai Dong, Ignavier Ng. Boyang Sun, Haoyue Dai, Guang- Yuan Hao, Shunxing Fan, Peter Spirtes, Yumou Qiu, and Kun Zhang. Permutation-basse based rank test in the presence of discretization and application in causal discovery with mixed data. In ICML, 2025.
Mathias Drton. Algebraic problems in structural equation modeling. In Advanced Studies in Pure Mathematics, pp. 35-86. Mathematical Society of Japan, 2018
Mathias Drton, Alexandros Grosdos, Irem Portakal, and Nils Sturma. Algebraic sparse factor analysis. arXiv preprint wxXi::23/2.14722 2023.
Gonçalo Rui Alves Faria, Andre Martins, and Mario A. T. Figueiredo. Differentiable causal discovery under latent interventions. In Conference on Causal Learning and Reasoning. 2022.
Joseph Felsenstein. Inferring phylogenies. In Inferring phylogenies, pp. 664 664. 2004.
Dan Geiger, David E. Heckerman, and Christopher Meek. Asymptotic model selection for directed networks with hidden variables. In Conference on Uncertainty in Artificial Intelligence, 1996.
Dan Geiger, David Heckerman, Henry King, and Christopher Meek. Stratified exponential families: Graphical models and model selection. The Annals of Statistics, 29(2):505-529, 2001.
AmirEmad Ghassami, Alan Yang. Negar Kiyavash, and Kun Zhang. Characterizing distribution equiv- alence and structure learning for cyclic and acyclic directed graphs. In International Conference Machine Learning, 2020.
12

---

Published as a conference paper at ICLR 2026
L. R. Goldberg. The structure of phenotypic personality traits. Am. Psychol, 48(1)226 1993. doi: 0003-066X. URL htttps://ssycnet apa. -p2pa..rr//aiaaaaiiaa001------.0 2F0003-066X 48.1.26
Samsad Afrin Himi, Markus Buchner, Matthias Schwaighofer, Anna Klapetek, and Sven Hilbert. Multitasking behavior and its related constructs: Executive functions, working memory capacity, relational integration, and divided attention. Cognition, 189:275-296 08 2019a.
Samsad Afrin Himi, Markus Bühner, Matthias Schwaighofer, Anna Klapetek, and Sven Hilbert. Multitasking behavior and its related constructs: Executive functtions working memory capacity, relational integration, and divided attentio. Cognition, 189:275-298, 2019b. ISSN 0010-0277. doi:
c0.10066i.o 2019.4...... 04.0 aaticclee/11//00002771990000
com wttp:reetttccocini.n.cceee science/
Patrik O Hoyer, Shohei Shimizu, Antti J Kerminen, and Markus Palviainen. Estimation of causal effects using linear non-gaussian causal models with hidden variables. International Journal of Approximate Reasoning, 49(2):362-33 2008.
Biwei Huang, Charles Low, Feng Xie, Clark Glymour, and Kun Zhang. Latent hierarchical causal structure discovery with rank constraints. In Advances in Neural Information Processing Systems, 2022.
John P Huelsenbeck, Fredrik Ronquist, Rasmus Nielsen, and Jonathan P Bollback. Bayesian inference of phylogeny and its impact on evolutionary biology. science, 29445550)2310-23144 2001.
Markus Kalisch and Peter Bühlman. Estimating high-dimensional directed acyclic graphs with the pe-algorithm. Journal of Machine Learning Research, 8(3), 2007.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2014.
Bohdan Kivva, Goutham Rajendran, Pradeep Ravikumar, and Bryon Aragam. Learning latent causal graphs via mixture oracles. Advances in Neural Information Processing Systems, 34: 18887-18101, 2021.
Mikko Koivisto and Kismat Sood. Exact bayesian structure discovery in bayesian networks. The Journal of Machine Learning Research, 5:549-573, 2004.
Erich Kummerfeld and Joseph Ramsey. Causal clustering for 1-factor measurement models. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1664, 2016.
Sébastien Lachapelle, Philippe Brouillard, Tristan Deleu, and Simon Lacoste-Julien. Gradient-based neural DAG learning. In International Conference on Learning Representations, 2020.
Hao-Chib Lee, Matteo Danieletto, Riccardo Miotto, Sarah Cherng, and Joel T. Dudley. Scaling structural learning with NO-BEARS to infer causal transcriptome networks. Pucific Symposium on Biocomputing. Pucific Symposium on Biocomputing, 25:391-402, 2019.
Phillip Lippe, Taco Cohen, and Efstratios Gavves. Efficient neural causal discovery without acyclicity constraints. In International Conference on Learning Representations, 2022.
Preetam Nandy, Alain Hauser, and Marloes Maathuis. High-dimensional consistency score-based and hybrid structure learning. The Annals of Statistics, 46(6A))3151-31888 2018.
Ignavier Ng, AmirEmad Ghassami, and Kun Zhang. On the role of sparsity and dag constraints for learning linear dags. Advances in Neural Information Processing Systems, 33: 33:17943-17954 2020.
Ignavier Ng, Séhastien Lachapelle, Nan Rosemary Ke, Simon Lacoste-Julien, and Kun Zhang. On the convergence of continuous constrained optimization for structure learning. In International Conference Artificial Intelligence and Statistics, 2022a.
Ignavier Ng, Shengyu Zhu, Zhuangyan Fang, Haoyang Li, Zhitang Chen, and Jun Wang. Masked cradient-basedd causal structure learning. In SIAM International Conference on Data Mining, 2022b.
13

---

Published as a conference paper at ICLR 2026
Ignavier Ng. Xinshuai Dong, Haoyue Dai, Biwei Huang, Peter Spirtes, and Kun Zhang. Score-based causal discovery of latent variable causal models. In Forty-first International Conference on Machine Learning, 2024.
Christopher Nowzohour, Marloes H Maathuis, Robin J Evans, and Peter Bühlmann. Distributional equivalence and structure learning for bow-free acyclic path diagrams. 2017.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary Detioo, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
Judea Pearl. Causality. Cambridge university press, 2009.
Joseph Ramsey, Madelyn Glymour, Ruben Romber, and Clark Glymour. million variables and more: the fast greedy equivalence search algorithm for learning high- high-dimensional graphical causal models, with an application to functional magnetic resonance images. International Journal of Data Science and Analytics, 3(2):121-3 2017.
Garvesh Raskutti and Caroline Uhler. Learning directed acyclic graphs based on sparsest permutations. arXiv preprint Xrviv: arXiv: 1307.3366v3 2014.
Thomas Richardson and Peter Spirtes. Ancestral graph Markov models. The Annals of Statistics, 30 (4):962-1030, 2002.
Thomas Richardson, Robin J Evans, James M Robins, and Ilya Shpitser. Nested markov properties for acyclic directed mixed graphs. The Annals of Statistics, 51(1):334-361, 2023.
Raanan Y Rohekar, Shami Nisimov, Yaniv Gurwiez, and Gal Novik. Iterative causal discovery in the possible presence of latent confounders and selection bias. Advances in Neural Information Processing Systems, 34:2454-2465, 2021.
Saber Salehkaleybar, AmirEmad Ghassami, Negar Kiyavash, and Kun Zhang. Learning linear non-gaussian causal models in the presence of latent variables. The Journal of Machine Learning Research, 21(1):1436- 1459-1459, 2020.
Mauro Scanagatta, Cassio de Campos, Giorgio Corani, and Marco Zaffalon. Learning bayesian networks with thousands of variables. Advances in neural information processing systems, 28, 2015.
Muralikrishnna G Sethuraman, Romain Lopez, Rahul Mohan, Faramarz Fekri, Tommaso Biancalani, and Jan- Christian Hütter. Nodags-flow: Nonlinear cyclic causal structure learning. In International Conference on Artificial Intelligence and Statistics, pp. 6371-6387 PMLR, 2023.
Ramy Shahin and Marsha Chechik. Automatic and efficient variability lifting of functional programs. Proceedings of the ACM on Programming Languages, 4(00PSLA)11-27,2022 2020.
Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Shohei Shimizu, Patrik O Hoyer, and Aapo Hyvärinen. Estimation of linear non- con-gaussian acyclic models for latent factors. Neurucomputing, 72(7-9):2024-2027, 2009.
Ilya Shpitser, Thomas S. Richardson, James M. Robins, and Robin Evans. Parameter and structure learning in nested Markov models. arXiv preprint
2012.
Ilya Shpitser, Robin J Evans, Thomas Richardson, and James M Robins. Introduction to nested markov models. Behaviormetrika, 41:3-39, 2014.
Ricardo Silva and Richard Scheines. Generalized measurement models. Technical report, Carnegie- Mellon Univ Pittsburgh PA School of Computer Science, 2005.
Ricardo Silva, Richard Scheines, Clark Glymour, and Peter Spirtes. Learning measurement models for unobserved variables. In Conference on Uncertainty in Artificial Intelligence, 2003.
14

---

Published as a conference paper at ICLR 2026
Ricardo Silva, Richard Scheines, Clark Glymour, and Peter Spirtes. Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7(8):191-246, 2006. URL
http://jmlz
Spirtes, C. Glymour, and Scheines. Causation, Prediction, and Search. MIT press, 2nd edition, 2001.
Peter Spirtes.
Introduction causal inference. Journal of Machine Learning Research, 1 11(5), 2010.
Peter L. Spirtes, Christopher Meek, and Thomas S Richardson. Causal inference the presence of latent variables and selection bias. arXiv preprint /302:49930 2013.
James H Steiger. Statistically based tests for the number of common factors. In Paper presented at the Annual Meeting of the Psychometric Society, lowa Cyty, 1980, 1980.
James H Steiger. Structural model evaluation and modification: An interval estimation approach. Multivariate behavioral research, 25(2):13--18)) 1990.
Nils Sturma, Chandler Squires, Mathias Drton, and Caroline Uhler. Unpaired multi-domain causal representation learning. arXiv preprint arXiv:2302.00999 2023.
Seth Sullivant, Kelli Talaska, and Jan Draisma. Trek separation for gaussian graphical models. The Annals of Statistics, 38(3):1665-1688 2010.
Boyang Sun, Yu Yao, Xinshuai Dong, Zongfang Liu, Tongliang Liu, Yumou Qiu, and Kun Zhang, A sample efficient conditional independence test in the presence of discretization. In ICML, 2025.
Sofia Triantafillou and Ioannis Tsamardinos. Score-based vs constraint-based causal learning in the presence of confounders. Cfa@ uai, pp. 59-67, 2016.
Ledyard R Tucker and Charles Lewis. A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1):1-10 1973.
Thijs van Ommen and Joris M. Mooij. Algebraic equivalence of linear structural equation models. In Conference Uncertainty Artificial Intelligence, 2017.
Thomas Verma and Judea Pearl. Equivalence and synthesis of causal models. In Conference on Uncertainty in Artificial Intelligence, 1991.
Y. Samuel Wang and Mathias Drton. Causal discovery with unobserved confounding and non- Gaussian data. Journal of Machine Learning Research, 24(271):1-61, 2023.
Dennis Wei, Tian Gao, and Yue Yu. DAGs with no fears: A closer look at continuous optimization for learning Bayesian networks. In Advances in Neural Information Processing Systems, 2020.
Feng Xie, Ruichu Cai, Biwei Huang, Clark Glymour, Zhifeng Hao, and Kun Zhang. Generalized independent noise condition for estimating latent variable causal graphs. Advances neural information processing systems, 33:1489) 1891-14991922 2020.
Bing Xu. Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.0085: 2015.
Yue Yu, Jie Chen, Tian Gao, and Mo Yu. DAG-GNN: DAG structure learning with graph neural networks. In International Conference on Machine Learning. 2019.
Changhe Yuan and Brandon Malone. Learning optimal Bayesian networks: A shortest path perspec- tive. Journal of Artificial Intelligence Research, 48(1):23-65 65, 2013.
Yan Zeng. Shohei Shimizu, Ruichu Cai, Feng Xie, Michio Yamamoto, and Zhifeng Hao. Causal discovery with 1-4. PMLR, 2021.
LiNGAM for latent factors. In Causal Analysis Workshop Series, pp.
Jiji Zhang. On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias. Artificial Intelligence, 172(16-17)11833-1188 2008
15

---

Published as a conference paper at ICLR 2026
Kun Zhang, Mingming Gong, Joseph Ramsey, K. Batmanghelich, Peter Spirtes, and Clark Gly- mour. Causal discovery with linear non- non-Gaussian models under measurement error: Structural identifiability results. In Conference Uncertainty in Artificial Intelligence, 2018.
Nevin L. Zhang. Hierarchical latent class models for cluster analysis. Journal of Machine Learning Research, 5:697-723, dec 2004.
Zhen Zhang, Ignavier Ng. Dong Gong, Yuhang Liu, Ehsan M Abbasnejad, Mingming Gong, Kun Zhang, and Javen Qinfeng Shi. Truncated matrix power iteration for differentiable DAG learning. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Zhen Zhang, Ignavier Ng, Dong Gong, Yuhang Liu, Mingming Gong, Biwei Huang, Kun Zhang, Anton van den Hengel, and Javen Qinfeng Shi. Analytic DAG constraints for differentiable DAG learning. In International Conference Learning Representations, 2025.
Xun Zheng, Bryon Aragam, Pradeep Ravikumar, and Eric Xing. DAGs with NO TEARS: Continuous optimization for structure learning. In Advances in Neural Information Processing Systems, 2018.
Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, and Eric P. Xing. Learning sparse nonparametrie DAGs, In International Conference Artificial Intelligence and Statistics, 2020.
16

---

Published as a conference paper at ICLR 2026
Algorithm Phase I of LGES
1: Input: Empirical covariance 5i over X, tolerance
2: Oufput: CPDAG Spasel
3: Let current state S = Sinit by Def. with latent variables L, let k = X; X, La = 0, and P
be an empty list;
4: while k < [x_ do
5: for each size & subset L, S La (in parallel) do
6: for each Xj E X, (in parallel) do
7:
Sij O L;, (X;3);
8:
Calculate scoremL Ex):
9:
Let scoreML Σxx= =;
10:
for each i,j s.t., |scoreMı (Sij. 5x) - s| < do
11:
S OLx(S,L L L,, {X,}) and X; = =X, {
12:
Let L' be any size & subset of L Lai
13:
for each size + subset X S X, (in parallel) do = OL, L ),,X,)
14:
Calculate scoreMI. (S). Ex))
16:
Maintain a disjoint set X' for elements in Xj,J E (i: scoremi. (Sj, -x)=scorce (8,îx)
8):
17:
for each X' E X do Let L' be any size subset of \ Ldi
19:
S (0xxSS))))) L L' X');
Ld La UL, X, X, append to P:
21:
k=k+1;
22:
Remove \ La from and let Sphasel = S;
23: return Splaset, P (which records those latent variables that really exist for the use of Phase 2)
Algorithm 2 Phase 2 of LGES
1: Input: Empirical covariance Σx X, tolerance 8, the output Sphasel and P of Algorithm
2: Output: CPDAG Siinl
3: Let S Sphasel:
4: while True do
5: for L, P (in parallel) do
6: for L, E P (in parallel) do
7:
if L. Ly or L, - Ly then
8:
Let H [L L Ly and (L - L; or L L, or <- L,);;
9:
for each subset H. C (in parallel) do
10:
Siju = OrL L;, Lj, Hk);;
Calculate scoremi (Sijk: Ex);
12:
Let i jj", A = arg max scoreML (Sigkr Ex);
13: if' (S:orcML((S..... k Ex) scOreML (S, Σx)1 < then S = S else Break;
for L; E do
15: Delete edges among L; in S;
16:
Shnal =S
17:
return Stial
A DETAILED DISCUSSION ABOUT RELATED WORK
In this section, we provide more comprehensive review of related work based on those discussed in Section [ expanding on both latent variable causal discovery and score-based causal discovery.
Latent variable causal discovery: The earliest attempts for handling latent variables in causal discovery were the Fast Causal Inference (FCI) algorithm (Spirtes et al. 2001]] Richardson & Spirtes 2002 Zhang 2008). More recently, LCD (Rohekar et al. 2021), ) iterative causal discovery method,
17

---

Published as a conference paper at ICLR 2026
was proposed for structure identification in the possible presence of latent confounders and selection bias. However, this line of work aims at identifying the Maximal Ancestral Graph. In other words, the objective is more of "deconfounding" for the identification of causal relations among observed vari- ables, and does not provide direct insight into the causal structure among latent variables. Moreover, FCT is already proved to be maximally informative under nonparametric conditional independence constraints, Therefore, to go beyond the limitations of conditional independence constraints, new statistical tools have been developed, often relying on additional parametric assumptions.
These tools have been discussed in Section 1 Among them, the most commonly used one and also the earliest developed one might be rank constraints (Sullivant et al.) 2010), which generalize the classical Tetrad representation theorem (Spirtes et al., (001) and basic conditional independence constraints. Based on these new tools, many algorithms have also been developed (Silva et al. 2003
Huang et al. 2022 Dong et al. 2024a), with tests that handle discretizations (Dong et al. 2025 (Sun et al. 2025). However, despite their theoretical advancements, most existing methods remain within the constraint-
paradigm, heavily relying on statistical tests that suffer from multiple-testing and error propagation issues. This is exactly our motivation on developing score-based algorithms for latent variable causal discovery.
Score-Based causal discovery: Score-based methods offer an alternative to constraint-based approaches, mitigating some of the issues related error propagation. These methods search for an optimal structure by maximizing a scoring function, such as the Bayesian Information Criterion (BIC) or likelihood-basee scores. Based on their search strategies, they can be broadly categorized as follows:
Exact Search Methods employ exhaustive graph traversal techniques or exploit minimal pruning strategies. such as permutation search (Raskutti & Uhler 2014). dynamic programming (Koivisto) & Sood 2004), or integer linear programming (Cussens 2012), to identify the globally optimal structure. These methods require minimal assumptions about the underlying graph structure or parametric model. However, their computational complexity grows super exponentially with the number of variables, rendering them impractical for large-scale causal discovery.
A A+-based and heuristic search methods integrate heuristic functions into the search process to guide exploration through the graph space (Yuan Malone 2013; Scanagatta et al., 2015). These methods strike a balance between computational efficiency and search completeness by prioritizing graph structures with high potential scores while avoiding exhaustive enumeration. Although more scalable than exact search, the quality of the learned structure relies heavily the effectiveness of the heuristic function.
Greedy search methods, such the widely used Greedy Equivalence Search (GES) (Chickering 2002), formulate the search problem in terms of graphical operators that iteratively modify the computationally efficient
structure by adding, deleting, reversing edges. These methods and well-suited for large-scale problems. As illustrated in (Nandy et al.) 2018), greedy search methods can often be interpreted as progressively refining the graph structure based on conditional independence or other graphical constraints, offering an intuitive connection between constraint- based and score-based paradigms.
Differentiable approaches, such as the seminal NOTEARS method (Zheng et al. 2018) recast the structure learning problem as a continuous optimization task. Subsequent works have incorporated nonlinear functional forms YYu et al.) 22019; Lachapelle et al. 2020 Zheng et al. 2020: Ng ct al.)
20226), interventional data (Brouillan lard et al. 2020 Faria et al. 2022 Lippe et al. (2022). alternative
optimization techniques (Ng et al. 2020 2022a] Bello et al. [2022] Deng et al. al.] 2023). and improved
formulations of the acyclicity constraint (Yu et al. 2019 Lee et al. 2019: Wei et al., 2020 Bello
(tt al. 2022. Zhang et al.) 2022. 2025). These methods benefit from direct compatibility with well-established numerical solvers and GPU acceleration, enabling them to efficiently handle large-scale problems.
Lastly, let us note that in the intersection of latent variable causal discovery and score-based algorithms, the only existing approach, to our knowledge, is that of Ng et al. (2024). As for the specific search procedure, their method follows an inefficient exact search paradigm. In contrast, our work is the first to introduce greedy score-based search for latent causal discovery, offering a more practical and scalable solution real-world problems while maintaining identifiability guarantees. We note that
18

---

Published as a conference paper at ICLR 2026
Table 4: Comparison of LGES with exact search across different graph sizes with 1000 sample size.
means the result is unavailable due to that the complexity of exact search grows super-exponentialyy:
Fl score for skeleton f
SHD for MEC J
Time find single graph
Graph Size
LGES
Exact Search
LGES
Exact Search
LGES
Exact Search
X
= 5
0.96 (0.06)
0.96 (0.03)
0.55 (0.43)
0.37 (0.05)
9 seconds
IS seconds
Xg 6
0.91 (0.02)
0.91 (0.02)
1.24 (0.75)
0.98 (0.54)
11 seconds
8 minutes
X5
0.88 (0.03)
0.89 (0.02)
(0.87)
2.05 (0.80)
14 seconds
1.5 hours
X
= 8
0.86 (0.02)
(0,78)
17 seconds
> 1O0 hours
Xg 9
0.85 (0.03)
4.11(1.52)
19 seconds
>> 1000 hours
Section 3.1 is highly related to Ng et al. (2024), and yet our contribution in Section 3.1 is unique, with reasons as follows.
First, to establish structure identifiability based on likelihood score and dimension, we have to assume the generalized faithfulness (the spirit of which is similar to the classical CI faithfulness). To assume the generalized faithfulness, one has to formally define several necessary notions including equality constraints, HG) (the set of equality constraints for a graph 9). B(G) (the set of canonical equality constraints with respect to reduced Grobner basis), and H (the union of B(G) over all possible graphs with n observed variables). This paper provides formal definitions of these crucial notions, which serves the basis for rigorous version of the generalized faithfulness Assumption D as contrast, these notions are not rigorously defined in Ng et al. 2024), which gives rise to some counter examples that shows the violation set of the generalized faithfulness might not be of measure zero.
Second, the identifiability theory proposed in Ng et al. (2024) is based on score where the likelihood and dimension are entangled. This actually requires a score based method (either exact search or greedy search) to be able to explicitly characterize the dimension of any candidate graph during the search procedure. However, characterizing an arbitrary latent causal structure still remains an open challenge in the field, which limits the usage of the theory. In contrast, this work explicitly disentangled the maximum likelihood score and dimension when stating the identifiability theory in Theorem 1 This gives rise to our greedy search method that does not need to explicitly characterize the dimension of each candidate graph during the search.
This work is also closely related to the RLCD algorithm (Dong et al. 2024a). Specifically, both RLCD and LGES aims to identify the underlying causal structure involving latent variables given observational distribution. The RLCD algorithm, which is constraint based method, makes use of rank constraints. Note that the set of all rank constraints is super-set of the vanishing partial correlation constraints and a sub-set of all the equality constraints, and RLCD examines the rank constraints by using statistical rank tests. In contrast, LGES is a score based method, which does not rely on statistical tests. In essence, by comparing the likelihood scores and dimensions, LGES implicitly makes use of all the equality constraints, which is a super-set of the rank constraints.
The research nested Markov model Shpptser al. 2012 2014); Richardson et al. (2023) is related to causal discovery in the presence of latent variables. This line of work is elegant in that it accomodates Verma-type constraints and contains marginal distributions given by DAG model with latent variables. However, nested Markov models follow the acyclic directed mixed graph (ADMG) framework, where the effect of latent variables are simplified into bidirected edges between observed variables. That is to say, within ADMG, only structure among observed variables is concerned. On the contrary, this paper aims to identify the whole underlying causal structure among both observed and latent variables (e.g., an edge from a latent variable to another latent variable). The research on learning phylogenetic tree [elsenstein (2004): Huelsenbeck et al. (200T) is also related as it aims to infer the evolutionary relationships among a group of organisms using observed data cata-ttypcallly and outputs a tree-structured graph. Yet, the graphical assumption in this line of work is much stronger then the GNFM considered in this paper.
B PROOFS
B. PROOF OF THEOREM II
Theorem 1 (Algebraic Equivalence by Score and Dimension). Suppose follows Definition ? with 9" and distribution Ex satisfies the generalized faithfulness assumption. Given observation E.
19

---

+ can Published as a conference paper at ICLR 2026
Table 5: Ablation study on the sensitivity of hyper- parameter 8.
of
score for skeleton
SHD for MEC
x 10"4
0.79(0.02)
10.85(0,79) 79))
x 10-3
(.81(0.02)
9.52(0.84)
2 x 10
0.82(0.02)
8.80(0.70)
4 10
0.80(0.02)
9.85(0.59)
1x 10
0.78(0.02)
11.13(0.70)
and let G arg maxgec" (G, If G* and & E arg mingeg. dim(g), then & and G are algebraic equivalent, i.e., H()) Hg*),. in the large sample limit.
The overall proof strategy below is inspired by Ghassami et al. (2020): Ng et al. 2024
Proof. In the large sample limie, we have Σx = Σx. By E G. we have that & can generate Ex and thus Σx contains all the equality and inequality constraints of &. Under the generalized faithfulness assumption, we have
H()) S H())).
Further, we have E arg gin dim As G" E G", we have di) < dim()"). Suppose by contradiction that Hg)) ? H())). This implies dim(ğ) > dim(g"), which contradicts with dim(g) dim( Therefore, we have
H()) & H())).
(5)
Taking Equations 4 and together, we have H() H Hg*).
B.2 PROOF OF THEOREM 2
Theorem 2 (Identifiability of Generalized N Factor Models by Equality Constraint up MEC). For G1, G2 E GONEM, if they are algebraically equivalent, i.e., H(G)) (G2),, then Gi and G2 belong the same MEC (same skeleton and v-structures over all variables).
Proof. We prove by showing that, by using equality constraints we can identify graph G E GONEM up to MEC.
Suppose 9 satisfies Definition and thus there exists partition of all latent variables in G that satisfies the requirement in Definition 2 as (L: For each L,, there exist at least [L,] observed variables X, such that for all X E X, (X) L, We first prove by induction that the structure from latent variables to observed variables can be identified up to MEC by equality constraints. Let 1. For those LL = k. all the pure children of L, can be identified by rank constraints: We can simply check size + combination of observed variables x and we have the variables in X are pure children of the same size latent group, iff rankex x k. given the relation between rank and t-separation in Sullivant et al. (2010). Next, suppose when the pure children of all the latent groups with L.I S have been found, we show the pure children of latent groups with |L,| also be found by rank constraint. In this case, check all size k + 2 combination of observed variables X such that ranky, xg1* = k + 1. Consider all such { and maintain a disjoint set for them such
that X; and X, belong to the same group they have least one common element. Take the union of such group, say X. If X share no common element with the pure children of any L, that has sizes k then elements in X are pure children of the same latent group L, with |L] = 1: + By induction, all the pure children of each latent group can be identified.
Further, for an observed variable, say, X, that common child of multiple latent groups, suppose its common parents are L, which is set of some groups of latent variables. Let A = X U UL,hc where PChş (L,) is a set of [Ly] pure children of L,. we have rank A El,ec [5g], and that when any edge related to X is changed, the related observed rank constraint
20

---

Published as a conference paper at ICLR 2026
will change, and thus the set of equality constraints will also change. Therefore, have that if ( = (G2), then the structure from latent to observed variables in 91 and that in G2 are the same.
Next, we show that the structure among latent groups can be identified up to MEC by equality constraints. By making of Corollary 1.3 in Di 2009,, we translate d-separation between latent groups into t-separation among the pure children of these latent groups. Specifically, for L, L i + j. we have L;, Lj, are d-separated by L S and
L, iff A = PChg (L.) U (PChg (L.)} (Lcg)(L)) U (n,ec (L)) are 1-separated by {c, 0} or (0, C), where PCh2 (La) and PCh. (Lr) refer to two disjoint groups of | pure children of L, (by definition we know such Iwo groups must exist). Given Assumption and the relation between rank and I-separation in Sullivant et al. (2010), we have that L; and L are d-separated by L, iff rankyą.. = L. This means that the d-separations among latent groups can be inferred from rank constraints on observed variables, and d d-separations can be used to identified the structure among latent variables up to MEC. As rank constraints are part of the equality constraints, we have that the structure among latent variables can be identified up to MEC by equality constraints. Taking that the structure from latent to observed variables can also be identified up to MEC by equality constraints (proved before) and in Generalized N factor models there is no direct edge between observed variables, we have that if H(t)) = H(2)), then 91 and G2 belong to the same MEC.
||4||. wheee
B.3 PROOF OF COROLLARY
Corollary 1 (Global Consistency by Score for Generalized N Factor Models), Suppose a model follows Definition D with G E GGNFM and distribution Vix satisfies Assumption Given observation Σx and let G arg maxgeGoww scoremL (G, Ex). If e G and arg minge-- dim(g), then and g are Markov equivalent in the large sample limit.
Proof. Similar to the proof of Theorem we can show that H()) Hg*)). By Theorem 2 we have 9 and G" belong to the same MEC.
B.4 PROOF OF LEMMA II
Lemma 1 (Properties of Initial State), Suppose model follows Definition 7 with G E GONFM and we are given observation Σx. Then Smat is supergraph of S(g*) and Sini can generate the observed distribution, i.e., (S Σx) maxgeGewns scoreML. (9. Σx) in the large sample limit.
Proof. By the graphical assumption in Definition 2 suppose L is the set of all latent variables in G* then there must exist at least [L| x 2 observed variables in g As such, the number of latent variables [L] should no more than | J. Thus, by Definition Sinit must be supergraph of the CPDAG of the ground truth, S(G*), up to permutation of latent variables, and thus Sinit must be able to generate the observation equally well as S(g*).
B.5 PROOF OF LEMMA 2
Lemma (Correctness of Phase of LGGS), Suppose a model modelfollows follows Desinition Twith G &GGNFM and we are given observation Σx. In the large sample limit the output Sphasel of Algorithm iss a CPDAG such that the number of latent variables in Sphasel is the same that of G and the edges from to L in Sphasel is the same as that of , up to permutation of latent variables.
We provide sketch of proof as follows.
Proof. We prove by induction. First consider & We show that all the observed variables that belong to size I latent group can be identified. Specifically, Or.x (S, L L', X,) )) in line 17 introduces a rank constraint generate the observation, then under the generalized faithfulness in Assumption I this rank constraint must also belong to H(*),, which means that variables in X, must belong to the same size latent group in 9" As such, all observed variables that belong to the same size 1 latent group can be
x(x,) = to S,. if S in line 17 of Algorithm [ can still
21

---

Published as a conference paper at ICLR 2026
identified. Now, suppose have already identified those observed variables whose parent set has cardinality t and let the set of these observed variables as X for all t I T. We show that for k T++1 all the observed variables that belong to size k latent group can be identified. Specifically, Sj introduces a rank constraint rank(Ex, x.X x = k to S,. if S in line 17 of Algorithm 1 can still generate the observation, then under the generalized faithfulness in Assumption this rank constraint must also belong to (G)). Given that X, has no common variable with any X for all S. T, we have that X; must belong the same size k latent group in g Therefore, by the end of the while in line 27 in Algorithm|1 [ all the parents of each observed variable can be identified. Till now, in the state considered in the algorithm, there might still exist some latent variables having no observed variable as children. These latent variables are removed from the current state in line 28 in Algorithm D and thus the number of latent variables in Sphusel the same as that of the ground truth and the structure between latent variables and observed variables in Sphasel is also the same as that of the ground truth up to permutation of latent variables.
B.6 PROOF OF THEOREM 3
Theorem 3 (Correctness of LGES). Suppose model follows Definition with 9 GGNnY and we are given observation Σxx In the large sample limit the output Simat of Algorithms and 2 is CPDAG that represent the MEC of g", up to permutation of latent variables.
The proof is partially inspired by the proof of Lemma 10 Chickering (0002)
Proof. First, we know that all the states must be able to generate the observation. Assume that Phase 2 terminates with a sub-optimal state S' and let G' be a DAG that belongs to S'. By Theorem 4 in Chickering (2002) we know that there must exist sequence of covered edge reversals and edge additioins that transforms G to G'. Suppose G" precedes the last edge addition in the sequence. We have that G" must also be able to generate the observation and S(G") is neighboring state of S', which means Phase 2 should not terminate at S', yielding a contradiction. Thus, by the end of Phase 2, the structure among latent variables in Simal must be the same that of the ground truth up to permutation of latent variables. Taking Lemma 2 also into consideration, we have that in the large sample limit the output Sinal of Algorithms 1 and 2 is CPDAG that represent the MEC of 9", up to permutation of latent variables.
O
C ADDITIONAL DEFINITIONS, IMPLEMENTATION DETAILS, RUNTIME ANALYSIS, AND EXAMPLES
C.1 DETAILED DEFINITION OF EQUALITY CONSTRAINTS, INEQUALITY CONSTRAINTS, HG), AND H!
Definition 6 (Definition of Equality Constraints, Inequality Constraints, and H(G).). Let be the DAG structure of Partially Observed Linear Causal Model with m latent variables and n observed variables (as in Definition 7T. The entries of the observed covariance matrix Σx are polynomial functions of model parameters 0 = (FT, Ite), where ET is the edge coefficient matrix and Ω2 is the diagonal noise variance matrix. This induces a parametric map under 9:
0g R - 12a++1//2
from parameters to observed covariance matrix, defining the system of equations:
(Exig - (g. = 0 si n).
Each equality constraint is just a polynomial equality equation consists of entries of Ex and 9, and each inequality constraint just polynomial inequality equation consists of entries of Ex and As our objective is to find information from Ex to infer the causal structure, we should focus on those equality equations that consists of only entries of Ex, which can be achieved as follows.
Let RRO0, Σx] be the polynomial ring which contains all variables for all model parameters O and all distinct covariance entries Exiy. while [x be the polynomial ring on Ex only. Define the ideal Ig C TR. 00, Ex) generated by the above equations
Ig ((Exi) (0) 0 sinssss)
22

---

Published as a conference paper at ICLR 2026
Then, (g) is the elimination ideal obtained by intersecting Ig with R[Ex], i.e.
H == 1g n R[x]]
In other words, (g) contains all equality constraints implied by 9 on the observed covariance matrix.
Definition 7 (Definition of B(g) and H".). Let >> be a fixed lexicographic monomial ordering on RiO Xx] such that all parameter variables in greater than all covariance variables in Σxx Define (g)) as follows. (i) Compute the reduced Gröbner basis of Ig following ordering >, i.e. GB(Ig, >). (ii) Retain only those polynomials that involve only variables in Ex. Formally,
B(G) =- Gn(G,, (n(l,,,)nBB 2)
Then H is defined as
H
U B(9).
GEG
In essence, (9) contains all equality constraints implied by on the observed covariance matrix, while (g) consists of canonical and minimal (owing reduced Gröbner basis) set of polynomial constraints among observed covariances that must vanish for any distribution consistent with structure G. Since the reduced Gröbner basis is unique (given fixed monomial order), B(G) serves as a standard representative of these constraints. The vanishing set of B(G) defines the smallest algebraic variety that contains all observed covariance matrices generated by the model.
C.2 DEFINITION OF ONE FACTOR MODEL BY SILVA ET AL.. 22033
COMPARISON
Definition 8 (One Factor Model (Silaa et al. 2003)). DAG G satisfies the definition of One Factor Model if each measured variable has a single latent parent, and each latent variable has at least three measured variables as children.
First, the generalized N factor model takes one factor model as special case. example of one factor model can be found in Figure (b). As a comparison, an example of generalized N factor model can be found in Figure 7 (a). Specifically, (a) differs from (b) in that, (i) (a) allows latent variables to form group and share observed variables children, e.g., {L4, L5) in (a) compared to L4 in (b), (ii) (a) allows some observed variables to common children of muiltiple groups of latent variables, e.g., X14 in (a), while (b) does not.
C.3 DEFINITION OF V-STRUCTURE AND CPDAG
Definition 9 (V-Structure / Collider). Let G (V, E) be a directed graph, where V is set of vertices (nodes) and a of directed edges. v-structure is an ordered triplet of distinct nodes (X, Y,Z) where X, Y, Z E V, that satisfies the following two conditions:
The graph G contains the directed edges X and
X and Z are not adjacent in G.
The node Y is referred to as the collider node of the v-structure.
Definition 10 (CPDAG (also called essential graph maximally oriented graphs) (Spirtes al. 2001 Chickering, 2002)). Let be the Markov equivalence class of DAG G. The CPDAG that represents & is the unique graph g* (consisting of both directed and undirected edges) such that:
Directed Edge: An edge X + Y is in G if and only if the edge X + Y exists in every DAG G E E.
Undirected Edge: An edge X is in G" if and only if there exists at least one DAG 91 E containing the edge X + Y and least other DAG G2 E E containing the edge X < Y.
C.4 GOODNESS-OF-MODEELE MEASURES
RMSEA (Steiger 1980 1990) measures the discrepancy due to the approximation per degree of
freedom. It actually badness-of-f measure and thus the lower value the better fit of the model.
23

---

Published as a conference paper at ICLR 2026
4
(a) An illustrative graph that satisfies generalized N(b) An illustrative graph satisfies one factor factor model.
model.
Figure 7: Illustrative examples compare two graphical assumptions, generalized N factor model V.s. one factor model.
The sample RMSEA is estimated as follows.
max - df, 0)
RMSEA
df(N
where x2 the chi-square statistic of the concerned model and df is the degree of freedom of the chi-square statistic.
The CFI (Bentler 1990) measures the relative improvement in terms of from the baseline model to the proposed model. The sample CFI is estimated follows:
CFI = 1 max (xf dfx: 0)
max df 0)"
where X2 and df' corresponds to the concerned model while xu and dfo corresponds to the baseline independent model that can only parameterize diagonal covariance matrix.
The TLI (Tucker & Lewis 1973 Bentler & Bonett 1980) measures relative reduction in misfit per degree of freedom. The sample estimator of TLI can given as follows:
TLI xo dfo /f/
xa/
C.5 IMPLEMENTATION DETAILS AND DISCUSSION ON THE DESIGN ò
Our code is based on Python3.7 and PyTorch Paszke al. 2017) and the optimization problem in Equation (2) is solved by Adam Kingma & Ba 2014) and LBFGS. Data is standardized to have zero mean and unit variance. The hyper parameter 8 in Algorithms and is set as 5 0.25 log(N) where N the sample size. This design follows the spirit of BIC score such that 5 + when N + In practice, found that the result is only influenced marginally by small change of 5.
Bellow we provide a further discussion on and the criterion of keeping edge removal in LGES.
Specifically, in LGES an edge removal is kept only when [LogL.cur curr LogL S Nog where 8 = k N is the tolerance level. N is the sample size, and k is hyper-parameter that controls sparsity. This design follows the spirit BIC score in GES (score BIC LogL - 0.5 log N dim).
N
Thus, the behavior of LGES is quite similar to GES both of them accommodates asymptotic consistency and finite sample performance at the same time.
In the asymptotic case, 0.5 log dim - O, and thus the likelihood term dominates the BIC score. Therefore, what GES favors is exactly the graph that (i) has the best likelihood, (ii) at the premise of (i) the dimension should be as small as possible. Similarly, in the asymptotic case - 0, and thus in LGES what is performed is precisely "Only when the likelihood after the deletion is still the best do we keep the deletion" " in the asymptotic case. Combined with other designs as discussed in our last response, LGESS also achieves the goal of finding the graph that (i) has the best likelihood, (ii)
N
24

---

Published as a conference paper at ICLR 2026
the premise of (i) the dimension should be small possible, and thus guarantees the asymptotic consistency.
In the finite sample case, there exists a problem that a supermodel always has better likelihood. To address this problem, in GES the BIC score sensibly encourages an edge removal by the term 0.5 log.N dim. As in GES each edge removal results in exactly 1 dimension decrease, the criterion N
in GES is equivalent to keeping an edge removal as long LogL. curr LogL.prev 0.5 No In LGES, an edge removal kept when [LogI_.curr - Logl..prev] S A: 1.Lox and thus it is also encouraged sensibly in LGES with finite samples.
C.6 RUNTIME ANALYSIS
Next we discuss the time complexity of LGES. Similar the classical PC and GES, our method has a worst-case complexity exponential in the number of observed variables. However, if the underlying graph is sparse, which is a common and reasonable assumption Kalisch Bühlman (2007). the complexity becomes polynomial. The intuition is as follows. Similar to PC and GES, during the process, LGES enumerates different combinations of variables and check the score to decide whether to delete some edges. Although the number of all combinations is exponential (also in GES and PC), the underlying graph is sparse, e.g., maximum degree of node is P. the algorithm will successfully find the correct combination to delete the edge before enumerating all the combinations. Thus the number of combinations that are actually enumerated only depends on the constant instead of number of variables. Therefore, the time complexity will become term polynomial in N, times term polynomial in constant P. and thus polynomial in N.
In our implementation, the computational cost is almost irrelevant to sample size, we only need to calculate the sample covariance once and cache it. Further, lines 5,6,13 in Alg and lines 5,6,9 in Alg 2 can be executed in parallel. Owe to these designs, in practice LGES is fairly efficient: on average it takes only one minute to handle a graph with 20 variables.
We conduct all the experiments with single Intel(R) (e) CPU E5-2470. Thanks to the fact that lines 5,6,13 1313mn in Algorithm 1 and lines 5,6,9 in Algorithm 2 can be executed in parallel, we employ the package joblib in python to conduct them by multi- procesppocessing. As such, on average it takes only one minute to handle a graph with 20 variables. We note that the computational cost is almost irrelevant sample size, we only need to calculate the sample covariance matrix once and cache it for further use. Compared to RLCD and GIN, LGES is faster than RLCD but slower than GIN. Specifically, it takes RLCD around minutes and GIN around 20 seconds to handle a graph with 20 variables, while around one minute for LGES. The reason why GIN is faster than LGES, is that GIN only focuses on the structure between latent variables and observed variables and does not identify structure among latent variables, while RLCD and LGES identify the whole underlying structure involving both observed and latent variables.
C.7 WHETHER THE IDENTIFAABILTYY THRORY CAN BE EXTENDED TO NON-GAUSSIAN OR NONLINEAR MODELS?
The identifiability result can be extended to both linear non-Gaussian and certain kinds of nonlinear models. (i) The key role of the score our identifiability theory is to check whether the constraints on the observed covariance matrix is subset of the constraints entailed by candidate structure. For any structure, the constraints entailed by in the non-Gaussian case is exactly the same as that of the Gaussian case. Therefore, the proposed identifiability theory still holds in the non-Gaussian scenario. (ii) For certain kinds of nonlinearity, the proposed identifiability theory still works. For example, in Nonparanormal models where there exist smooth, monotonic transformations for each variable to transform variables to be jointly Gaussian, as the monotonic transformations can be identified to trivial indeterminacy, we can still use the proposed score-based theory for structure identification.
C.8 FURTHER DISCUSSION ABOUT ERROR PROPAGATION
Our score-based greedy search method is similar to constraint- method in the sense that both starts from a complete graph and deletes edges according to some criterion. In this sense, why score-based methods are expected to suffer less from error propagation? The main reason lies in
25

---

Published as a conference paper at ICLR 2026
the difference of the used criteria. Score-based methods rely MLE score and dimension, while constraint- based methods rely on statistical tests. We provide our further analysis as follows.
Using Score the Criterion Can Have Fewer Algorithm Steps. The theoretical basis of constraint-bar based methods and score-based methods are the same checking whether the equality constraints the observational distribution graph. Thus, roughly speaking, given a dataset the total number constraints that need to be checked the same for both score-based and constraint-based methods (if we want to guarantee asymptotic correctness). However, constraint-ben test, while score-based methods can in essence examine multiple constraints together the same time. More specifically, in coore-based methods, we can introduce multiple constraints in step and examine them together at the same time, by checking whether the likelihood still maximal, As consequence, ccore-based methods require fewer algorithm steps, which mitigates the problem of error propagation. This is also aligned with our empirical observation that, although the score-based GES and the constraint-basee PC require basically the same assumptions, in practice GES often has fewer steps to finish, and can often handle 10 times more variables than PC.
aligned with the constraints entailed by candidate
methods only examine one constraint in each step by single
Using Test as the Criterion Suffers More From Small Sample Sizes. Statistical tests (except for some permutation based) reply on an approximation the asymptotic null distribution of the test statistic to calculate the p-value for controlling the type-I error. When the sample size is small, e.g.. N=100, the approximation (often based on the central limit theorem) could be far away from the true null distribution. As consequence, with small sample size the type-I error in each step of the constraint-basse consistent with our empirical results: in Table I and Table 2, LGES still performs well with a very small sample size (N=100) both with and without model mis-specification, while constraint-base methods such as RLCD and GIN does work well in these scenarios.
methods cannot be properly controlled, let along the type-ll error. This is also
D LIMITATIONS
One limitation of this work is that our theoretical results are based on the assumption of linear causal models. When data is not linear, we have also conducted experiments to see the performance and can shown that our method still performs well. Yet, theoretical analysis and identifiability guarantee for the nonlinear case are to be developed and will be the focus of future work.
E BROADER IMPACTS
The goal of this paper is to advance the field of machine learning. We do not see any potential negative societal impacts of the work.
26

---

Published as a conference paper at ICLR 2026
1
X2
X
X
X
X2
X
X
X
4
L
L
%
X
4
Lx
Xx
X
X
L
Ls
%
X 10
X
14
X
X
X
X
X
X
X,5
X1
X12
X
X
X
(a)
G1.
(b)
G2.
X
X2
X
X
X₂
X
X
X
L
L
L
%
X
4
L3
X
X
X
X
La
Ls
X
X
X
1.s
X
X
X
X
X
X
X15
X
X12
X
X
X
(c) G3-
(d) Ga.
X
Xx
4
X=
L
L
X₂
X
X
LG
X
X
X
X
X
X
X+
X
(e)
Gs-
(f) Go.
X
X
X
X
Xa
X
L
X
X
X1
X12
X
X
LG
L
L
X15
X,
X,
X,
X
X,
X
(g) GT.
(h) Gs.
Figure 8: Examples of graphs considered in our experiments.
They satisfy Definition
2
27


Reviews:
[{"review_id": "4MsSKOIqvd", "reviewer": "Reviewer_WPvf", "summary": "This work proposes a score-based observational causal discovery algorithm in the presence of latent variables for linear SCM. It provides identifiability guarantees for the causal structure involving latent variables. This paper aims to identify the whole underlying causal structure among both observed and latent variables. It also propose Generalized N Factor Model and can identify the structure up to the Markov equivalence class by score.", "questions": "- What is $V$ in line 223?\n- Lines 246-247: Why does the partitioning make the requirement of $2|\\mathbf{L}_{p}|$ observed variables being minimal?\n- What is GNF in line 248? Is it GNFM?\n- Lines 253-254 say ‘At the same time, if the number of observed children is insufficient, we can still gather more relevant observations or measurements of the underlying system.’ Do the authors mean the number of observed children is sufficient instead of insufficient?\n- Why does the CPDAG in the initial state need to explicitly list out all the latent variables in the graph?\n- Why is SALAD by Ng et al. 2024 not compared in the experiment?\n- Do all the baselines output CPDAG? Are they compared against a ground truth CPDAG in the experiment?\n- Can authors report the performance only based on the causal structure among the observed variables, including the F1 scores for arrowheads?\n- In the experiment, it says 10 random seeds, but there are 20 randomly generated DAGs. Does that mean each DAG is used 10 times to generate 10 different datasets for each sample size?\n- Which CI test is used for GIN in the experiment? What is the alphas used? \n- Why not compare the performance with the baselines when the ground truth graphs do not satisfy Definition 2 (generalized N factor models), instead of the violation of normality and linearity, as the first experiment has already used uniform noise terms?\n- Why is the performance only reported for sample sizes 500 and 1000 for the model specification case in the experiment? What about the sample size of 100?\n\n- Why is LCD [1] for latents not compared against LGES?\n\nReference:\n\n[1] Rohekar, Raanan Y., et al. \"Iterative causal discovery in the possible presence of latent confounders and selection bias.\" Advances in Neural Information Processing Systems 34 (2021): 2454-2465.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- This paper seems to extend the work by Ng et al. 2024. The major difference between this paper and Ng et al. 2024 seems to be greedy search vs. exact search. Particularly, this work proposes an algorithm named LGES, a two-phase GES-style greedy search, operating directly over MEC. This is the first score-based greedy search method for the identification of structures involving latent variables with identifiability guarantees.\n\n- By introducing the generalized N factor model, LGES proves that algebraic equivalence implies Markov equivalence, providing a clean route to global consistency up to MEC via a greedy method. That’s a nontrivial step on top of the SALAD’s algebraic equivalence by Ng et al. 2024.\n\n- The description of LGES is clear.\n\n- The paper provides some real-world datasets to motivate the design of the algorithm.\n\n- Proofs seem to be sound. \n\n- The proposed algorithm can incorporate Verma-type constraints.\n\n\nReference:\n\nNg, Ignavier, et al. \"Score-based causal discovery of latent variable causal models.\" Forty-first International Conference on Machine Learning. 2024.", "weaknesses": "- The algorithm is limited to linear Gaussian SCM. \n- Since this paper is highly related to Ng et al. 2024,  it should give a more in-depth discussion between this work and Ng et al. 2024 in the introduction or related work to highlight the differences and contributions of this work.\n- The organization of the paper needs improvement. Some terms such as equality constraints are mentioned in line 136, but it does not provide a brief explanation what that means at the same place of the paper. Rather, it refers to the appendix of the paper to see Definition 6. In terms of organization, this makes the reading more difficult. Similarly for Definition 7. Also, line 197, inequality constraints appear without any explanation about what the means. Line 214 does not provide any contexts about Definition 2 before it says ‘the number of all possible graphs that satisfy Definition 2’. Definition 2 only shows up later in the paper.\n- Besides the organization of the paper, the clarity of the paper also needs improvement. For example, GNFM and GNF are used interchangeably. \n- Lines 253-254 are very hard to follow: ‘At the same time, if the number of observed children is insufficient, we can still gather more relevant observations or measurements of the underlying system.’\n- The efficiency of the proposed algorithm lies critically on the proposed generalized N factor model. It is debatable whether this model assumption is realistic or not. In practice, there should be way more latent than observed. \n- The paper should introduce some basic terms in causal graphical models such as CPDAG.\n- It is confusing to use the same notation $\\mathcal{S}$ to represent both the state and a CPDAG.\n- The paper should cite the exact theorem number in line 299.\n\nReference:\n\n- Ng, Ignavier, et al. \"Score-based causal discovery of latent variable causal models.\" Forty-first International Conference on Machine Learning. 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S7J1ATV2cH", "reviewer": "Reviewer_JjnK", "summary": "The paper proposes a score-based causal discovery for latent variable models, more precisely \"Generalized n factor models\", that is inspired by GES. It builds upon a MLE score that is proven to be consistent. Moreover, the presented algorithm LGES also provably recovers (up to permutation) the equivalence class of the true graph.", "questions": "1. Can you clarify the relation to recent work by Ng et al. (2024) and Dong et al. (2024)? In particular, the novel aspects of Section 3.1.\n\n2. Can you comment how the approaches distinguishes itself from constraint-based methods that start with a complete graph and subsequently delete edges? How does the proposed method avoid issues with error propagation?\n\nSuggestions:\n- line 020: \"propose *the* generalized...\"\n- line 020: \"establish *its* global consistency\"\n- line 022: \"by using score\" ?\n- line 024: \"class of *models*\" and \"which *searches*\"\n- line 033: \"are being made *towards*...\"\n- line 041: \"on *the* discovery of *the* entire structure\n- line 042: \"on rank or tetrad constraints linearity assumption\" ?\n- line 048: \"statistical *tests*\"\n- line 053: \"most classical\"\n- line 060: \"As a consequence, considerable attention has been given...\" please provide references to substantiate this statement\n- line 063: \"What is the core of structure identifiability by using likelihood scores?\" I don't understand what this means\n- line 068: \"We characterize how *a* likelihood...\"\n- line 071: \"We propose *the*...\" and below \"*the* score\"\n- line 072: \"MEC\" this abbreviation has not been introduced thus far\n- line 074: \"develop *the* Latent...\"\n- line 076: \"identifiability *guarantees*\"\n- line 096: \"v-structures\" this term is not introduced\n- line 107: \"formulation of *the* likelihood\"\n- line 139: \"*algebraically* equivalent\"\n- line 159: \"*the* number of edges and *the* number *of* measured variables\n- line 180: \"by making use of score\" ?\n- line 200: \"how *the* likelihood score\"\n- line 218: \"we propose *the* generalized\"\n- line 266: establishes\n- line 281: \"we follow the traditional wisdom GES Chickering (2002)\"\n- line 317: \"In Definition 3, it implicitly requires\"\n- line 338: remove superfluous \".\"\n- line 403 and 407: Def. 1 and Def.2 (also please be consistent with abbreviations)\n- line 481 to the end: please remove or replace by a proper section or conclusion", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper studies an important topic: causal discovery without assuming causal sufficiency of the observed variables. The proposed approach for generalized n factor models is novel and the techniques non-trivial. The theorems for asymptotic consistency are sound and additionally the algorithm is evaluated on synthetic data. Generally, the writing in the paper is clear (apart from a significant number of small typos and grammatical errors).", "weaknesses": "Relation to other recent work:\nWhile the paper gives a broad overview of causal discovery (in the presence of latent variables), the papers (Ng et al, 2024) and (Dong et al, 2024) appear to be closely related. In particular for the results in Section 3.1, it is not clear to me how novel the contributions in the paper are (Theorem 1 is stated as \"inspired by Ng et al (2024)\"). The paper would benefit from distinguishing itself more clearly here.\n\nGES as a score-based algorithm in contrast to constraint-based methods:\nIn the paper it is argued that score-based methods compare favorably to constraint-based methods in particular because the latter are prone to error propagation. I agree with the sentiment, but to me it seems that the proposed algorithm suffers from exactly the same problem. If I understand correctly, the algorithm starts with a graph that (at least in some parts) is complete (similar to constraint-based methods), and then deletes edges which lead to no worsening of the MLE score. An incorrectly deleted edge can never be added back into the graph (thus errors propagate) and also there is an intuitive connection to the independence tests that the algorithm aims to avoid (the edge can be deleted asymptotically because some conditional independence is found to hold and delta could be interpreted as some cutoff value). Another thing is that \"Greedy\" is maybe an unfortunate name, because it usually suggests that each step is done to be locally optimal wrt the score but it doesn't appear to be the case here.\n\nAssumption of the generalized n factor model:\nGenerally, it is of course clear that restrictive assumptions are needed to recover latent variables. Thus, I find it fair to confine the analysis to a model such as the generalized n factor model even though it may reduce applicability. The paper could do a better job though of explaining why the constraints in Def. 2 are needed. With the proofs deferred to the appendix I find this rather opaque.\n\nCode availability:\nThe code doesn't appear to be part of the submission, which I find unfortunate. \n\nGrammatical errors and typos:\nWhile I found the paper generally easy-to-read and thus do not weigh this issue much, the paper contains a significant grammatical errors, small inconsistencies and typos, which could have been avoided by careful proofreading (a list of suggestions for improvement is under questions).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eyH6dEaLXH", "reviewer": "Reviewer_yLgV", "summary": "The paper proposes LGES (Latent variable Greedy Equivalence Search), a score-based greedy algorithm to learn the causal structure of partially observed linear SEMs with latent variables. The authors prove global consistency under some assumptions. They design an efficient two-phase search with tailored operators and show empirical results versus constraint-based baselines on synthetic and real data.", "questions": "In which real domains do you expect the 2|Lp| pure-children and group-wise uniform adjacency assumptions to hold?\nWhat happens when there are fewer than 2|Lp| pure children, or when pure children do not exist for some groups? Do you degrade gracefully (e.g., to equivalence classes over merged latents)? You argue the identifiability story extends because equality constraints over covariance remain. But then the method does not exploit higher-order moments.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Overall, the paper tackles an important gap: scalable score-based discovery for latent-variable models with identifiability guarantees. A greedy score-based method with identifiability guarantees in the presence of latent variables is nontrivial. Linking equality constraints, minimal dimension, and GNFM to MEC identifiability. Strong synthetic performance vs. constraint-based baselines.", "weaknesses": "I am not sure if the conditions, such as GNFM structure, generalized faithfulness, and linear/noise are satisfied.\nAbout experiments, I see that on Big Five and Burnout, LGES fits worse than hand-designed CFA structures (Table 3) you should discuss why and provide intuition. Also, since Ng et al. (2024) is cited as an exact search score-based approach for latent models, a direct comparison would be informative to position LGES.\nPlease clarify the notation and correct the expression in equation 3.\nTypos/edits:\n“propogation” → “propagation”;\n“eal-life” → “real-life”;\n\neal-life\n\nδ definition has conflicts in Section 5.2, Appendix C.4", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)