On the Parameter Identifiability of Partially Observed Linear Causal Models

Authors: xinshuai  dong
ignavier  ng
biwei  huang
yuewen  sun
songyao  jin
roberto  legaspi
peter  spirtes
kun  zhang
san  diego
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 19:10:15

Abstract: Linear causal models are important tools for modeling causal dependencies and yet in practice, only a subset of the variables can be observed. In this paper,  we examine the parameter identifiability of these models by investigating whether the edge coefficients can be recovered given the causal structure and partially observed data. Our setting is more general than that of prior research—we allow all variables, including both observed and latent ones, to be flexibly related, and we consider the coefficients of all edges, whereas most existing works focus only on the edges between observed variables. Theoretically, we identify three types of indeterminacy for the parameters in partially observed linear causal models. We then provide graphical conditions that are sufficient for all parameters to be identifiable and show that some of them are provably necessary. Methodologically, we propose a novel likelihood-based parameter estimation method that addresses the variance indeterminacy of latent variables in a specific way and can asymptotically recover the underlying parameters up to trivial indeterminacy. Empirical studies on both synthetic and real-world datasets validate our identifiability theory and the effectiveness of the proposed method in the finite-sample regime.

Main paper: 
 introduction and related work: 
 Causal models, which serve as a fundamental tool to capture causal relations among random variables, have achieved great success in many fields [49, 39, 40, 44] . A fundamental problem in the field is how and to what extent can we identify the underlying causal model given observational data. When all variables are observed, the problem has been well studied: the underlying structure can be identified up to the Markov equivalence class, e.g., by the PC [49] or GES [13] algorithm; when the structure is given, the causal coefficient (direct causal effect) between two variables can also be identified [8, 39] . However, in real-world systems, the variables of interest may only be partially observed. Thus, considerable efforts have been dedicated to identification of causal models in the presence of latent variables. One line of research focuses on structure learning given partially observed variables. Notable approaches include FCI and its variants [49, 38, 14, 2] , as well as ICA-based [23, 43] , tetrad-based [48, 28] , high-order moments-based [46, 11, 58, 1, 12] , and rank constraint-based [48, 24, 18] methods.In this paper, we focus on the the identification of parameters of a partially observed model. Specifically, given the causal structure of and observational data from a partially observed causal model, we are interested in identifying all the parameters, and thus the underlying causal model can be fully specified. To identify the parameters, a classical way is to project the directed acyclic graph (DAG) with latent variables to an acyclic directed mixed graph (ADMG) or partially ancestral graph [42] , without explicitly modeling the latent confounders. Based on ADMG, graphical criteria such as halftrek [20] , G-criterion [9] , and some further developments [51, 29] have been proposed to establish the *Equal contribution. 38th Conference on Neural Information Processing Systems (NeurIPS 2024). parameter identifiability. Another way is to leverage do-calculus, proxy variables, and instrumental variables [47, 39, 25] to identify the direct causal effect, which corresponds to the edge coefficient in linear causal models. For a more detailed discussion of related work, please refer to Appendix D.Despite the effectiveness of current methods for parameter identification, however, they have two main drawbacks: they require all the variables to be connected in specific ways, and only focus on identifying the edge coefficients between observed variables. To this end, in this paper we propose a novel framework that considers a more general setting for parameter identification. To be specific, we allow all variables, including both observed and latent ones, to be flexibly related, and we aim to recover the edge coefficients among all variables, even including those from a latent variable to another latent variable or another observed variable, which previous methods cannot handle. We summarize our contributions as follows.• To the best of our knowledge, we are the first to consider parameter identifiability of partially observed causal model in the most general scenario-all variables, including both observed and latent ones, are allowed be flexibly related, and edge coefficients between any pair of variables are concerned. In contrast, most existing works consider only the edges between observed variables. • Theoretically, we identify three types of parameter indeterminacy in partially observed linear causal models. We then provide graphical conditions that are sufficient for all parameters to be identifiable and show that some of them are provably necessary. These necessary conditions also offer insights into scenarios where the parameters are guaranteed to be non-identifiable. • Methodologically, we propose a novel likelihood-based parameter estimation method, which parameterizes population covariance in specific ways to address variance indeterminacy. Our empirical studies on both synthetic and real-world data validate the effectiveness of our method in the finite-sample regime, even under certain misspecification of the underlying causal model. 
problem setting: 
 In this work, we focus on partially observed linear causal models, defined as follows.Definition 1 (Partially Observed Linear Causal Models). Let G := (V G , E G ) be a DAG. Each variable V i ∈ V G follows a linear structural equation model V i = Vj ∈Pa G (Vi) f j,i V j + ϵ Vi , whereV G := L G ∪ X G = {L i } m i=1 ∪ {X i } m+ni=m+1 contains m latent variables and n observed variables. Pa G (V i ) denotes the parent set of V i , f j,i denotes the edge coefficient from V j to V i , and ϵ Vi represents the Gaussian noise term of V i .We drop the subscript G in L G and X G when the context is clear. We use V, V, and V to denote a random variable, a set of variables, and a set of sets of variables, respectively. In Definition 1, the relations between variables can also be written in the matrix form as V G = F T V G + ϵ V G , where F = (f j,i ) i,j∈ [m+n] is the weighted adjacency matrix. Here, f j,i ̸ = 0 if and only if V j is a parent of V i in G. We also writeF = L G X G L G A B X G C D and Ω = Ω L G 0 0 Ω X G ,where Ω is the diagonal covariance matrix of ϵ V G . Our objective is to identify F , the causal edge coefficients of the model, given observational data and the causal structure G. Denote by Σ L G and Σ X G the population covariance matrix of latent variables L G and observed variables X G , respectively; their precise formulations are provided in Proposition 1. We also denote by σ i,j the (i, j)-th entry of Σ X G . In this work, we assume that the noise variances ϵ L G of latent variables have unit variance, i.e., Ω L G = I, which will be justified later in Section 3.1. Note that variables are partially observed and thus we only have access to i.i.d. samples of observed variables X G . As variables are jointly Gaussian, the observations can asymptotically be summarized as the population covariance matrix Σ X G . In other words, we aim to identify F and Ω given Σ X G and G. The identification of parameters is important in that, once we identify the parameters, the underlying causal model is fully specified, and thus we can flexibly calculate causal effects, infer interventional distributions, and finally answer counterfactual queries [39] . It is worth noting that, for parameter identification, the structure G is assumed to be known, which is different from the setting of causal discovery where the goal is to identify G from data.(a) Graph G1. Its parameters can be fully identifiable.(b) Graph G2. Its parameters cannot be fully identified.(c) G1 G2 have the same ADMG in the latent projection framework.Figure 1 : Illustrations of the advantage of our framework. Within our framework, it can be shown that G 1 's parameters can be identified (up to sign) while G 2 's cannot. In contrast, the latent projection framework cannot even differentiate G 1 from G 2 as they share the same ADMG (c) after projection. Furthermore, with ADMG, any edge coefficient that involves a latent variable cannot be considered.Without latent variables, it has been shown all parameters are identifiable [8] . However, the problem becomes very challenging when latent variables exist. There are two lines of research. One focuses on the use of do-calculus, proxy variables, and instrumental variables to identify direct causal effects among observed variables [47, 39, 25] (in linear models the direct causal effect is captured by the edge coefficient). Another line addresses latent confounders by projecting a DAG with latent variables into an ADMG, where the confounding effects of latent variables are simplified and represented by correlation among noise terms [20, 9, 51, 29 ]. An example is in Figure 1 , where (a) is the original graph and (c) is the projected ADMG whose bidirected edges correspond to correlated noise terms.Compared to the two previous lines of thought, our framework has two advantages. To begin with, we additionally considers the identifiability of coefficients of edges that involve latent variables. For example, in Figure 1 , we aim to identify all the coefficients including the one from L 1 to X 3 , i.e., f 1,3 . In contrast, the proxy variable framework and the latent projection framework identify only the coefficients among observed variables: the proxy variable framework focuses only on the direct causal effect from one observed variable to another observed variable, while the latent projection framework transforms all latent variables into bidirected edges and thus can never identify the coefficient of the edge that has a latent variable as the head or tail.Furthermore, the projection framework deals with latent variables in a rather brute-force way: dense latent confounding effects among observed variables may be caused by only a small number of latent variables, but that information is lost during projection. For example, in Figure 1 , (a) and (b) share the same ADMG after projection, i.e., (c). However, as we will show later, parameters in (a) can be identified, while in (b) the parameters cannot. If we only consider the ADMG in (c), then we can never capture this nuance and thus cannot identify the coefficients that we might be able to.3 Identifiability Theory 
definition of parameter identifiability and indeterminacy: 
 We follow the notion of generic identifiability and define parameter identifiability as follows.Definition 2 (Identifiability of Parameters of Partially Observed Linear Causal Models). Let θ = (F, Ω) ∈ Θ. We say that θ is generically identifiable, if the mapping ϕ(θ) = Σ X G is injective, for almost all θ ∈ Θ with respect to the Lebesgue measure. Definition 2 indicates if parameter θ is identifiable, then there does not exist θ ′ ∈ Θ that entails the same observations as those of θ. As in the typical literature of parameter identification, we consider generic identifiability to rule out some rare cases where the parameters for that structure is generally identifiable, but with some specific parameterization, the parameters cannot be identified. This is similar to faithfulness in causal discovery [49] and we will provide an example in Example 1. We next introduce three important indeterminacies about parameter identification when latent variables exist. Then, θ and θ entail the same observations, i.e., ΣX G = Σ X G . Furthermore, we have ΣL G = ΛΣ L G Λ.(a) G1. Its structure is identifiable but its parameters are not identifiable even if the structure is given (due to orthogonal indeterminacy).(b) G2. Its structure is not identifiable but params are identifiable.(c) G3. G2's structure is not identifiable due to the existence of G3. A similar theoretical result is provided in [4] , and yet our setting is much more general and takes that of [4] as a special case: in our setting, all variables including latent and observed ones can be arbitrarily related while in [4] latent variables cannot be the effect of observed variables.Remark 1 (Implication of Theorem 1). A key implication of Theorem 1 is that, without further assumption, the edge coefficients involving latent variables, i.e., (A, B, C), can never be identified, as there always exists a diagonal matrix Λ such that θ and θ entail the same observations but ( Ã, B, C) ̸ = (A, B, C). Thus, in the rest of this paper, we assume that the noise variances ϵ L G of latent variables have unit variance, i.e., Ω L G = I. Under this assumption, we have( ΩL G ) i,i = Λ 2 i,i (Ω L G ) i,i = 1, i ∈ [m], which implies Λ i,i = ±1.As such, this assumption makes parameter identifiability possible. However, even though we fix the scaling of Ω L G , there still exists indeterminacy about the sign of parameters, captured by Theorem 2. Then, θ and θ entail the same observations, i.e., ΣX G = Σ X G , andm ≥ 1, θ = (A, B, C, D, Ω X G , Ω L G ),( ΣL G ) ii = (Σ L G ) ii , ∀i ∈ [m].Remark 2 (Remark on Theorem 2). The indeterminacy described in Theorem 2 is referred to as group sign indeterminacy for the following reason: According to the theorem, flipping the sign of S i,i is equivalent to flipping the signs of all coefficients of edges involving the latent variable L i . This transformation preserves the resulting observations Σ X G . In essence, each group consists of coefficients of edges involving a particular latent variable. Example 1 (Example for Group Sign Indeterminacy and Generic Identifiability). In Figure 2 (b), given the structure and Σ X G , by assuming Ω L G = I, the parameters are generally identifiable up to group sign indeterminacy. Specifically, there exist three equality constraints with three free parameters:f 1,2 f 1,3 = σ 2,3 , f 1,2 f 1,4 = σ 2,4 , and f 1,3 f 1,4 = σ 3,4 . The solutions are: (i) f 1,2 = σ2,3σ2,4 σ3,4 , f 1,3 = σ 2,3 /f 1,2 , f 1,4 = σ 2,4 /f 1,2 and (ii) f 1,2 = - σ2,3σ2,4 σ3,4 , f 1,3 = -σ 2,3 /f 1,2 , f 1,4 = -σ 2,4 /f 1,2 .The two solutions are different only in terms of group sign. However, if we set f 1,2 = 0, then the parameters are not identifiable (as we will encounter division where the divisor is zero). These rare cases of parameters are of zero Lebesgue measure so we rule out these cases for the definition of identifiability, as in Definition 2. Intuitively speaking, group sign indeterminacy arises because one may multiply the latent variable L i by -1 and accordingly flip the signs of all edge coefficients involving L i . Note that such an indeterminacy is rather minor for the following reason. (i) In practice, we can always anchor the sign of some edges according to our preference or prior knowledge in order to eliminate the group sign indeterminacy. For example, in Figure 4 , if we expect that L2 should be understood as Extraversion instead of non-Exterversion, we can add one additional constraint during our parameter estimation such that the edge coefficient from L2 to E1 ("I am the life of party.") will be positive (as we believe E1 should be positively related to Extraversion). (ii) On the other hand, there are some application scenarios that are not influenced by the group sign indeterminacy, such as causal effect estimations between certain variables. We note that, as the indeterminacy of group sign is rather minor, in the following if the parameters are identifiable only up to group sign indeterminacy, we still say that the parameters are identifiable. Definition 3 (Orthogonal Transformation Indeterminacy). Consider a model that follows Definition 1 with number of latent variables m ≥ 1, θ = (A, B, C, D, Ω X G , Ω L G ), and Ω L G = I. We say that there exists an orthogonal transformation indeterminacy in the identification of parameters if there exists a non-diagonal orthogonal matrix Q such that ( Ã, B, C, D, ΩX G , ΩL G ) and (A, B, C, D, Ω X G , Ω L G ) share the same support and entail the same observations, whereÃ := Q T AQ, B := Q T B, C := CQ, D := D, ΩL G := Q T Ω L G Q = I, and ΩX G := Ω X G .The orthogonal transformation indeterminacy is the major indeterminacy we consider in the presence of latent variables. Such an indeterminacy also arises in factor analysis [45, 7] , which can be viewed as a special case of the data generating procedure considered in Definition 1. Here we only give the definition and will later provide Theorem 4 with an example that captures the scenarios where such indeterminacy exists.It is worth noting that the graphical condition for structure identifiability and parameter identifiability could be very different. For example, G 1 in Figure 2 (a) is structure-identifiable, and yet the parameters are not identifiable even if the structure is given. In contrast G 2 in Figure 2 (b) is not structure-identifiable, as there exists another structure G 3 in Figure 2 (c) such that G 2 and G 3 can never be differentiated from observational distribution; and yet if G 2 is given, its parameters are identifiable (as in Example 1). Therefore, in this paper, we first consider the cases where the structure can be identified and then study which further conditions are needed for the identifiability of parameters. This will give rise to conditions under which the whole causal model can be fully specified.To explore the conditions for the whole causal model to be specified, we start with the structure identifiability of partially observed linear causal models. Recent advances have shown that if certain graphical conditions are satisfied [24, 18] , even though all variables including latent ones are allowed to be very flexibly related, the causal structure can still be identified. Next, we focus on the conditions by [18] , which takes that of [24] as special cases. Roughly speaking, the identifiability of the structure of a partially observed linear causal model is built upon the identifiability of atomic covers, defined as follows (with effective cardinality defined as ||V|| = |(∪ V∈V V)| and PCh G defined in Appendix B.2). Definition 4 (Atomic Cover [18] ). Let V ∈ V G be a set of variables, where l out of |V| are latent, and the remaining |V| -l are observed. V is an atomic cover if V contains a single observed variable, or if the following conditions hold: (i) There exists a set of atomic covers C, with ||C|| ≥ l + 1, such that ∪ C∈C C ⊆ PCh G (V).(ii) There exists a set of covers N , with ||N || ≥ l + 1, such that every element in ∪ N∈N N is a neighbour of V and (∪ N∈N N) ∩ (∪ C∈C C) = ∅. (iii) There is not a partition of V = V 1 ∪ V 2 such that both V 1 and V 2 are atomic covers. The intuition that we build structure identifiability upon the notion of atomic covers is as follows. When a set of latent variables share the same set of children and neighbors, it is impossible to differentiate these latent variables from each other, and thus we need to consider them together as the minimal identifiable group to build up the identifiability of the whole structure. Such a minimal identifiable group of variables is defined as an atomic cover. Roughly, for a group of variables to be qualified as an atomic cover, it has to have enough children and neighbors. An example is as follows.Example 2 (Example of Atomic Cover). Consider the graph in Fig. 3 . V = {L 1 , X 6 } is an atomic cover. This is because there exist C = {{X 9 }, {X 10 }} with ||C|| ≥ l + 1 = 2 such that (i) in Def. 4 is satisfied. And there exist N = {{X 11 }, {X 12 }} (or, N = {{X 4 }, {X 5 }}) with ||N || ≥ l + 1 = 2 such that (ii) in Def. 4 is satisfied. We can also find that (iii) in Def. 4 is satisfied. Thus {L 1 , X 6 } is an atomic cover. Another example would be in Figure 8 , where {L 1 , L 2 } is an atomic cover.Condition 1 (Basic Conditions for Structure Identifiability [18] ). G satisfies the basic graphical condition for identifiability, if every latent variable belongs to at least one atomic cover in G and for each atomic cover with latent variables, any of its children is not adjacent to any of its neighbours. Condition 2 (Condition on Colliders [18] ). In G, if (i) there exists sets of variables V, V 1 , V 2 , and T such that every variable in V is a collider of two atomic covers V 1 , V 2 , and T is a minimal set of variables that d-separates V 1 from V 2 , and (ii) there exists at least one latent variable inV ∪ V 1 ∪ V 2 ∪ T, then we must have |V| + |T| ≥ |V 1 | + |V 2 |.Example 3 (Example that satisfies Conditions 1 and 2). Consider Figure 3 . All latent variables in the graph belong to at least one atomic cover and thus Condition 1 is satisfied. Plus, Condition 2 is also satisfied. This is because the sets of variables V, V 1 , V 2 , and T that satisfy (i) and (ii) in Condition 2 are 3). Roughly speaking, the underlying causal structure of G can be identified except that the directions of some edges cannot be determined. Next, we will show that, given any DAG in the identified equivalence class together with Σ X G , the parameters of the model are also identifiable, if certain conditions are satisfied.V = {X 12 }, V 1 = {L 1 , X 6 }, V 2 = {L 2 },In this section we show that, given graphical Conditions 1 and 2, the causal coefficients F in Definition 1 are also identifiable, if certain conditions are satisfied. Theorem 3 (Sufficient Condition for Parameter identifiability (up to group sign)). Assume that G satisfies Conditions 1 and 2 and thus the structure can be identified up to the MEC of O min (O s (G)). For any DAG in the equivalence class, the parameters are identifiable, if both the following hold:(i) For any atomic cover V = X ∪ L, |L| ≤ 1. (ii) If an atomic cover V = X ∪ L satisfies |L| ̸ = 0 and |X| ≥ 1, then there exists X ⊆ X G such that X d-separates X and L.Theorem 3 provides a sufficient condition such that the parameters are identifiable. We note that this sufficient condition has certain extent of necessity and we will detail this point later after we introduced some necessary conditions that can guide us under which scenarios parameters are guaranteed to be not identifiable. Now, for a better understanding of Theorem 3, we provide an example of it as follows. Example 4 (Example for Theorem 3). The graph G in Figure 3 satisfies the conditions for parameter identifiability in Theorem 3. Specifically, condition (i) in Theorem 3, is satisfied as all atomic covers contain no more than one latent variable. Plus, condition (ii) in Theorem 3 is also satisfied, as the atomic cover V = X∪L = {L 1 }∪{X 6 } satisfies |L| ̸ = 0 and |X| ≥ 1 and there exist X = {X 4 , X 5 } such that X d-sep X and L. Therefore, the parameters are identifiable for the graph in Figure 3 .Next, we discuss under which conditions the parameters are guaranteed to be not identifiable. As discussed in Section 3.1, there are three kinds of indeterminacy. The first one can be solved by assuming unit variance of the noise terms of latent variables while the second one group sign indeterminacy is rather trivial such that we still consider parameters as identifiable even if group sign indeterminacy exists. Therefore, we will focus on the third one, orthogonal transformation indeterminacy, in what follows. Theorem 4 (Condition for the Existence of Orthogonal Transformation Indeterminacy). Consider the model in Definition 1. If a set of latent variables L with |L| ≥ 2, have the same parents and children, then there must exist orthogonal transformation indeterminacy regarding the edge coefficients F . In other words, F can at most be identified up to orthogonal transformation indeterminacy. Example 5 (Example for Thm. 4). Consider Fig. 8 . The graph satisfies the conditions in Thm. 4 as the parents and children of L 1 and L 2 are exactly the same. Therefore, there must exist orthogonal transformation indeterminacy for the edge coefficients F and thus the parameters are not identifiable.The Theorem 4 above indicates that, if there exist two latent variables that share the same parents and children, then the edge parameters can at most be identified up to orthogonal transformation. This directly implies a necessary condition for parameter identifiability as follows.Corollary 1 (General Necessary Condition for Parameter Identifiability). For parameters to be identifiable, every pair of latent variables has to have at least one different parent or child. Corollary 1 captures a necessary condition in the general cases such that parameters are identifiable. If we further consider the graphs that are also structure identifiable (as we need to identify the structure first to fully specified the causal model), we further have the following Corollary 2 by considering the notion of atomic covers (the proofs of both corollaries can be found in the Appendix). Corollary 2 (Necessary Condition about Atomic Covers for Parameter Identifiability). Assume G satisfies Conditions 1 and 2 and thus the structure can be identified up to the MEC of O min (O s (G)).For any DAG G in the equivalence class, for G's parameters to be identifiable, every atomic cover must contain no more than one latent variable. Now we are ready to discuss the necessity of the graphical condition proposed in Theorem 3. Remark 3 (Necessity of Conditions in Theorem 3). Theorem 3 provides a sufficient condition for parameter identifiability and it has certain extent of necessity with reasons as follows. First, condition (i) in Theorem 3 is provably necessary: by Corollary 2, for parameters to be identifiable, one has to assume (i) in Theorem 3. Second, if condition (ii) in Theorem 3 does not hold, then there are only some rare cases where parameters can be identified. We would like to mention that establishing a necessary and sufficient condition is always highly non-trivial in various tasks. For example, for the identification of linear non-Gaussian causal structure with latent variables, researchers initially developed sufficient conditions with three pure children in [46] , later relaxed to two in [11, 58] , before ultimately achieving both necessary and sufficient conditions in [1] . Similarly, for parameter identification, although the condition we proposed is not a necessary and sufficient one, it has considerable extents of necessity, and could be expected to serve as a stepping stone towards tighter and ultimately the necessary and sufficient condition for the field.Identifiability theory often focuses on the asymptotic case, i.e., we assume that we know the structure and the population covariance matrix Σ X G . However, in practice, we only have access to i.i.d. data with finite sample size and thus only have the sample covariance matrix. Therefore, in the next section, we will propose a novel method to estimate the parameters in the finite sample cases.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes]Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: The code and the data will be publicly available.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
parameter estimation method 4.1 objective: 
 Our goal is to estimate F in Definition 1, given the causal structure G and observational data. The key is to parameterize the population covariance Σ X G using θ = (F, Ω) and then maximize the likelihood of observed sample covariance ΣX G . To make this technically precise, we provide a closed-form expression of Σ X G in terms of θ in the following proposition, with a proof given in Appendix A. M := I -A -B(I -D) -1 C -1 and N := (I -A)C -1 (I -D) -B -1 .Then, the population covariance matrices of L G and X G can be formulated asEQUATIONΣ X G =(I -D) -T B T Σ L G B + Ω X G + Ω X G N B + B T N T Ω X G (I -D) -1 .(2) The formulations of Σ L G and Σ X G are rather complicated due to the general scenario we considered, i.e., latent variables can be the cause or the effect of latent and observed variables. That is, the submatrices A, B, C and D defined in the above proposition can all have nonzero entries. In most existing works, at least one of these submatrices are assumed to be zero. For instance, factor analysis assumes that A, C and D are zero, while [32] assumes that A and C are zero. Furthermore, Proposition 1 also provides insight into the indeterminacy involved when identifying the parameters, such as the indeterminacy of variance in Theorem 1 and the orthogonal transformation indeterminacy in Theorem 4.Similar to the typical factor analysis setting [45, 7, 21] , we assume that ϵ V G are all Gaussian and thus X G are jointly Gaussian. Thus, the negative log-likelihood of observational data can be formulated asEQUATIONwhere K is the number of i.i.d. observations. With the parameterized negative log-likelihood, we estimate the edge coefficients by minimizing the negative log-likelihood, asEQUATIONwhere the entries of matrix F that do not correspond to an edge in G are constrained to be zero during the optimization. Note that in Eq. 4 the constraint that the noise terms of latent variables have unit variance is crucial to deal with the variance indeterminacy defined in Theorem 1. In practice, it is also favorable to use another constraint to address the variance indeterminacy, i.e., the constraint that all the latent variables have unit variance. This leads to an alternative objective asEQUATIONwhere the entries of F that do not correspond to an edge in G are also constrained to be zero.Both objectives in Eq. 4 and Eq. 5 can be employed, and yet using the second one gives rise to edge coefficients that are easier to understand. To be concerete, if we normalize all observed variables to have unit variance, then using Eq. 5 would give rise toF such that -1 ≤ Fi,j ≤ 1, ∀i, j ∈ [m].An example can be found in Figure 4 . However, it may not be straightforward to realize the constraint in Eq. 5. To this end, in the next section we introduce a way to parameterize Σ X G using F , such that the required constraint in Eq. 5 can be automatically satisfied. Later in Section 5.2, we also empirically compare the performance of using Eq. 4 with that of using Eq. 5.In this section, we introduce how trek rules can be employed to parameterize Σ X G while the unit variance constraint on latent variables in Eq. 5 can be elegantly satisfied. We start with the definition of trek. For readers who are less familiar with treks, please refer to Appendix B.1 for examples. Definition 5 (Treks [50] ). In G, a trek from X to Y is an ordered pair of directed paths (P 1 , P 2 ) where P 1 has a sink X, P 2 has a sink Y, and both P 1 and P 2 have the same source Z, i.e., top(P 1 , P 2 ) = Z. A Trek is simple if P 1 and P 2 have no intersection except their common source Z.At this point, we are able to parameterize each entry of Σ X G using (F, {σ ii } n+m i=1 ), instead of (F, Ω), by making use of the (simple) trek rule [50] , as follows:EQUATIONwhere S(V i , V j ) is the set of all simple treks between V i and V j , and f P is the path monomial along P defined as f P := Π k→l∈P f kl . By this form of parameterization, we can simply set all entries of {σ ii } n+m i=1 as 1 (which is equivalent to requiring all variables to have unit variance), such that the constraint in Eq. 5 can be automatically satisfied. For a better understanding of how to use the simple trek rule for parameterization, we provide an example as follows. Example 6 (Example for Parameterization using Simple Trek). In Figure 7 (a), there are four simple treks between X 4 and X 5 , as shown in (b). By the simple trek rule and further assuming that all variables have unit variance, the covariance between X 4 and X 5 , σ 4,5 , can be formulated asf 1,4 f 1,5 + f 3,4 f 3,5 + f 2,1 f 1,4 f 2,3 f 3,5 + f 2,3 f 3,4 f 2,1 f 1,5 . 
experiments: 
 We validate our identifiability theory and parameter estimation method on synthetic and real-life data.We begin with our experimental setting of synthetic data. The causal strength f ij is uniformly sampled from [-2, 2] and the noise terms are Gaussian with variance uniformly from [1, 5] . We consider 20 graphs. 10 of them should be parameter-identifiable up to group sign indeterminacy according to our identifiability theory and we refer to them as GS Case (examples in Figure 10 in Appendix). Another 10 should be parameter-identifiable up to group sign and orthogonal transformation indeterminacy and we refer to them as OT Case (examples in Figure 11 in Appendix). On average each graph contains 15 variables, 3 out of them are latent. We consider three different sample sizes: 2k, 5k, and 10k. We use three random seeds to generate the causal model and report the mean performance as well as the std.As the optimization in Eq 4 is nonconvex, we will rely on 30 random starts and choose the one with the best likelihood. We report the performance of the proposed method with two different objectives. (i) Parameter Estimator with objective defined in Eq. 4, referred to as Estimator, and (ii) Parameter Estimator with objective defined in Eq. 5 and Trek Rule parameterization trick in Eq 6, referred to as Estimator-TR.It is worth noting that our setting is very general in that we allow latent variables and observed variables to be causally connected in a very flexible way, and we consider the identification of parameters of edges that can involve both observed and latent variables. Therefore, to the best of our knowledge, no current method can achieve the same goal to serve as the baseline (which also shows the novelty of the proposed method). As such, we mainly focus on comparing our estimation result with the ground truth parameters.We use two MSE-based metrics defined as follows. MSE up to group sign: suppose the ground truth parameter is F and our estimation is F . The MSE up to group sign is defined as∥|F |-| F |∥ 2 2 ∥F ∥0, where | • | takes element wise absolute value, ∥ • ∥ 2 denotes the Frobenius norm and ∥ • ∥ 0 denotes the number of nonzero entries of a matrix. MSE up to orthogonal transformation: the MSE up to orthogonal transformation is defined asEQUATIONwhere the optimization is solved by Adam [27] and the orthogonal matrix Q can be directly parameterized in PyTorch.We report the performance using synthetic data in Tab 1a and 1b, where both our Estimator and Estimator-TR achieve very good identification performance. For example, in the GS scenario with 10k samples, our Estimator achieves 0.0.0012 MSE up to group sign and our Estimator-TR achieves 0.0003 MSE up to group sign. The good performance by Estimator and Estimator-TR not only validates our estimation method, but also empirically verifies our identifiability theory.In this section, we show that the proposed estimation method still performs well, even under model misspecification: violation of normality and violation of linearity.As for violation of normality, we use uniform noise terms for the underlying model, and thus the distribution is not jointly Gaussian anymore. We aim to see to what extent can the proposed method still recover the correct parameters. The result is shown in Table 2 in the Appendix, which shows even when the normality is violated, we can still estimate the parameters pretty well. The reason lies in that our proposed asymptotic identifiability result holds true, even when we do not assume Gaussianity; as we only make use of the second-order statistics of the distribution, the additive noise in Definition 1 can follow any other continuous distribution.To simulate the violation of linearity, we employ the leaky ReLU function during the generation process, asV i = LRELU( V j ∈Pa(V i ) f ji V j + ϵ V i ), LRELU(x) = max(αx, x).When α is close to 1, the function is close to a linear one, and when α is close to 0, the model is very nonlinear. The result is shown in Table 3 and we found that our estimation method is quite robust to small violations of linearity. For example, for Estimator-TR in GS case with 10k sample size, if we set α = 0.8, we still get a small MSE of 0.001. Even when α decreases to 0.6, the MSE is around 0.005, which is still small. However, when α is decreased to 0.3, the underlying model is considerably nonlinear, and the MSE increases to 0.027.Our code is based on Python3.7 and PyTorch [37] . Data is standardized and the optimization in Eq 4, Eq 5, and Eq. 7 are solved by Adam [27] , with a learning rate of 0.02. We conduct all the experiments with single Intel(R) Xeon(R) CPU E5-2470. All experiments can be finished within 2 hours. We note that our method is very computationally efficient. First, the computational cost is almost irrelevant to sample size: we only need to calculate the sample covariance matrix once and cache it for further use during the optimization. Plus, our estimation method can handle a large number of variables. For example, the running time of our method are roughly 10 seconds, 2 minutes, and 10 minutes for 20 variables, 50 variables, and 100 variables respectively. For 300 variables, which is a considerably large number for typical experiments considered in causal discovery papers, the estimation can still be finished within around one hour.It is also worth noting that model misspecifications do not influence the computation cost of our method. We briefly discuss the efficiency of checking whether conditions in Theorem 3 hold, together with what if conditions do not hold in solving real-life problems in Appendix A.7 and A.8.In this section, we employ a famous psychometric dataset -Big Five dataset https: //openpsychometrics.org/, to validate our method. It consists of 50 indicators and close to 20,000 data points. There are five dimensions: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism (O-C-E-A-N). Each is measured with 10 indicators. Data is standardized.We employ the RLCD method [18] to determine the MEC and GIN [58] to decide the remaining directions. Then we employ the proposed Estimator-TR to estimate all the edge coefficients.The estimated edge coefficients are shown in Figure 4 . We found that our estimated coefficients are well aligned with existing psychology studies. For example, according to [16, 17] , being successful in exploratory endeavors depends on the stability to pursue them. This is illustrated in our result where L5 ----→L2) [54] . Such consistency with current psychometric studies again validates the effectiveness of the proposed method in parameter estimation of real-life systems. 
conclusion: 
 In this paper, we characterize indeterminacy of parameter identification and provide conditions for identifiability. Finally, we propose a novel estimation method and validate it by empirical study.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: See the experiment section for the setting and details.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
acknowledgement: 
 This material is based upon work supported by NSF Award No. 2229881, AI Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract R01HL159805, and grants from Salesforce, Apple Inc., Quris AI, and Florin Court Capital. Then, θ and θ entail the same observations, i.e., ΣX G = Σ X G . Furthermore, we have ΣL G = ΛΣ L G Λ.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: The synthetic experiments were supported by error bars to show statistical significance.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: It is not related to this work. No such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: All the assets are properly credited. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. 
 
 Appendix: Proof of Theorem 1. Let M and N be matrices defined as in Proposition 1, and similarly for M and Ñ . We then haveM = I -Ã -B(I -D) -1 C -1 = Λ -1 Λ -Λ -1 AΛ -(Λ -1 B)(I -D) -1 (CΛ) -1 = Λ -1 I -A -B(I -D) -1 C -1 Λ = Λ -1 M Λ and Ñ = (I -Ã) C-1 (I -D) -B -1 = (Λ -1 Λ -Λ -1 AΛ)(CΛ) -1 (I -D) -Λ -1 B -1 = (I -A)C -1 (I -D) -B -1 Λ = N Λ.By Proposition 1, the latent covariance matrix ΣL after rescaling of the parameters is given byΣL = M T ΩL M + Ñ T ΩX Ñ = (Λ T M T Λ -T )(ΛΩ L Λ)(Λ -1 M Λ) + Λ T N T Ω X N Λ = Λ(M T Ω L M + N T Ω X N )Λ = ΛΣ L Λ.This implies that the variance of each latent variable L i is scaled by Λ 2 ii . By Proposition 1, the observed covariance matrix ΣX after rescaling of the parameters is given by Then, θ and θ entail the same observations, i.e., ΣX G = Σ X G , and ΣX = (I -D) -T BT ΣL B + ΩX + ΩX Ñ B + BT Ñ T ΩX (I -D) -1 = (I -D) -T (Λ -1 B) T (ΛΣ L Λ)(Λ -1 B) + Ω X + Ω X (N Λ)(Λ -1 B) + (Λ -1 B) T (N Λ) T Ω X (I -D) -1 = (I -D) -T B T Σ L B + Ω X + Ω X N B + B T N T Ω X (I -D) -1 = Σ X .( ΣL G ) ii = (Σ L G ) ii , ∀i ∈ [m].The structure identifiability part is that if G satisfies Condition 1 and Condition 2, the structure of G can be identified up to the Markov equivalence class of O min (O s (G)), which is by Theorem 12 in [18] .Next we will focus on the proof of parameter identifiability part, i.e., for any DAG in the equivalence class, if (i) and (ii) in Theorem 3 are satisfied, the parameters are identifiable (up to group sign). Without loss of generality, we assume that all variables have unit variance and zero mean. The reason is that if we can show that the parameters are identfiable (up to group sign) under this assumption, then it is straightforward to show that they are also identifiable under the original assumption where Ω L = I.Lemma 1. Consider a graph G that satisfies (i) and (ii) in Theorem 3. For an atomic cover V in G with one latent variable, V = {L} ∪ {X i } m i=1 (m could be zero), if it has an observed pure child C and the coefficients of the edges from V to C, i.e., f L→C , f X1→C , . . . , f Xm→C , are known, then for any variable A, such that A ̸ = C and A is not a descendant of C, σ L,A can be calculated as(σ A,C -Σ m i=1 σ X i ,A f X i →C )/f L→C .Proof of Lemma 1. By the definition of atomic covers, all variables in V = {L} ∪ {X i } m i=1 are not adjacent. By trek rule and the fact that C is a pure child of V, all treks from C to A go through V, and thus by the trek rule we have σL,A f L→C + Σ m i=1 σ X i ,A f X i →C = σ A,C .Remark: This lemma implies that we can find all edge coefficients of the graph in a bottom-up fashion. Roughly speaking, for a latent variable L that belongs to an atomic cover V, once we identify f L→C , f X1→C , ..., f Xm→C where C is an observed pure child of V, we can take L as if it is observed. More specific explanations can be found in the following proof. Theorem 3 (Sufficient Condition for Parameter identifiability (up to group sign)). Assume that G satisfies Conditions 1 and 2 and thus the structure can be identified up to the MEC of O min (O s (G)).For any DAG in the equivalence class, the parameters are identifiable, if both the following hold:(i) For any atomic cover V = X ∪ L, |L| ≤ 1.(ii) If an atomic cover V = X ∪ L satisfies |L| ̸ = 0 and |X| ≥ 1, then there exists X ⊆ X G such that X d-separates X and L.Proof of Theorem 3. Consider a graph G that satisfies (i) and (ii) in Theorem 3. We first show that if all the pure children of an atomic cover V are observed, then all the edge coefficients from the atomic cover to its children are identifiable (up to group sign). To this end, we categorize the scenarios into four cases and prove them separately with illustrative examples.(a) V = {X} contains a single observed variable. The proof for this case is trivial as the edge coefficient from X to its pure child C is simply σ X,C .(b) V = {L} contains a single latent variable. By Condition 1 there must exist C 1 , C 2 , and X N , such that C 1 , C 2 are pure children of V and X N is an observed variable that has a trek to V. Then we haveσ C1,C2 = f L→C1 f L→C2 , σ C1,X N = f L→C1 σ L,X N , and σ C2,X N = f L→C2 σ L,X N .By these three equations, we can solve f L→C1 and f L→C2 . If V has more than two pure children, we can prove the identifiability similarly in a pairwise fashion.Example. Take the atomic cover {L 4 } in Figure 5 as an example. By Condition 1, it must have at least two pure children. To identify f 4,13 and f 4,14 , we need to borrow an observed variable that has at least one trek from L 4 to it, e.g, X 8 . As L must have two other neighbors, there must exist such a variable. Then we have σ 13,14 = f 4,13 f 4,14 , σ 13,8 = f 4,13 σ 4,8 , and σ 14,8 = f 4,14 σ 4,8 . By these three equations, we can solve f 4,13 and f 4,14 .(c) V = {L} ∪ {X i } m i=1 (m ≥ 1) contains a single latent variable and m observed variables, where V has at least three pure children. We assume that there exist C 1 , C 2 , and C 3 , such that C 1 , C 2 , and C 3 are pure children of V. By (ii) in Theorem 3, there exist X such that X d-separates L and {X i } m i=1 . Let σ ϵ L = t (the variance of the noise term of L). In this case, we haveEQUATIONEQUATIONEQUATIONBy these three equations, we can solve f L→C1 ,f L→C2 , and f L→C3 , with the only remaining free parameter t. In other words, we have f L→C1 (t),f L→C2 (t), and f L→C3 (t). Now we consider X = { Xj } p j=1 to solve t. We have additional pm + p free parameters that are f Xj →X i , i ≤ m, j ≤ p and f Xj →L , j ≤ p, and they can be solved by σ Xj ,X i , i ≤ m, j ≤ p and σ Xj ,L (t), j ≤ p (where σ Xj ,L (t) is calculated by Lemma A.1). To this point, all the edge coefficients among V, X, and C 1 ,C 2 ,C 3 are solved, up to the only free parameter t. Finally, we have that L has unit variance and thus we have t + p j=1 f 2 Xj →L (t) = 1. By this additional equation, t can also be solved and thus all desired edge coefficients are identified. If V has more than three pure children, we just choose all the combinations of any three and thus we can identify all the edge coefficients from V to all its pure children.Example. Take the atomic cover {L 1 , X 5 } in Figure 6 as an example. Let σ ϵ L 1 = t. We have σ X8,X9|{X3,X5} = tf L1→X8 f L1→X9 , σ X8,X10|{X3,X5} = tf L1→X8 f L1→X10 , and σ X9,X10|{X3,X5} = tf L1→X9 f L1→X10 . By these three equations, we can solve f L1→X8 (t),f L1→X9 (t), and f L1→X10 (t), up to t. Then we introduce X 3 and solve f X3→X5 and f X3→L1 by σ X3,X5 and σ X3,L1 (t). Finally, we have t + f 2 X3→L1 (t) = 1 and thus all desired edge coefficients are identified.(d) V = {L} ∪ {X i } m i=1 (m ≥ 1) contains a single latent variable and m observed variables, where V has two pure children. By Condition 1, there must exist C 1 , C 2 as the pure children of V. If there exists one additional pure child, then it is the same as (c). By (ii) in Theorem 3, there exist X = { Xj } p j=1 such that X d-separates L and {X i } m i=1 . Plus, if there only exist these two pure children, there must exist X N such that X N is a parent of V, and thus X N ∈ X. Without loss of generality, we take X N = X1 .We assume that σ ϵ L = t, and thus we haveEQUATIONEQUATIONEQUATION)We further haveσ C1,X N |( X\X N ) = f X N →L f L→C1 + m i=1 f X N →X i f X i →C1 , σ C2,X N |( X\X N ) = f X N →L f L→C2 + m i=1 f X N →X i f X i →C2 , and σ X N ,X i |( X\X N ) = f X N →X i , i ≤ m.We now have 3m + 3 edge coefficients and 3m + 3 equations, and by these equations, we can solve these edge coefficients up to t (i.e., all these edge coefficients are functions of t).N ) = { Xj } p j=2 . Specifically, we have f Xj →X i = σ Xj ,X i |( X\ Xj ), where i ≤ m, 2 ≤ j ≤ p. We also have f Xj →L = σ Xj ,L|( X\ Xj ) (t), where 2 ≤ j ≤ p. By the unit variance of L, we further have t + p j=1 f 2 Xj →L (t) = 1, by which t can be solved. Therefore, all edge coefficients among V, X, C 1 , C 2 can be identified.Example. Take the atomic cover {L 2 , X 6 , X 7 } in Figure 6 as an example. We assume that σ ϵ L 2 = t, and thus we have σ X12,X13|X3,X4,X6,X7 = tf L2→X12 f L2→X13 , and σ X12,X i |X3,X4 = σ X i |(X3,X4) f X i →X12 , 6 ≤ i ≤ 7 and σ X13,X i |X3,X4 = σ X i |(X3,X4) f X i →X13 , 6 ≤ i ≤ 7. We further have σ X12,X3|X4 = f X3→L2 f L2→X12 + 7 i=6 f X3→X i f X i →X12 , σ X13,X3|X4 = f X3→L2 f L2→X13 + 7 i=6 f X3→X i f X i →X13 , and σ X3,X i |X4 = f X3→X i , 6 ≤ i ≤ 7.We now have 9 edge coefficients and 9 equations, and thus by these equations, we can solve these edge coefficients up to t. Next we solve t by considering X 4 . Specifically, we have f X4→X i = σ X4,X i |X3 , where 6 ≤ i ≤ 7. We also have f X4→L2 = σ X4,L2|X3 (t). By these 3 equations we can solve f X4→L2 , f X4→X6 , and f X4→X7 , up to t. Till now all edge coefficients among X 3 , X 4 , L 2 , X 6 , X 7 , X 12 , X 13 can be taken as a function of t. Finally by the unit variance of L, we further have t + f 2 X3→L2 (t) + f 2 X4→L2 (t) = 1, by which t can be solved. Therefore, all edge coefficients among X 3 , X 4 , L 2 , X 6 , X 7 , X 12 , X 13 can be identified.Taking (a), (b), (c), (d) into consideration, for a graph that satisfies the conditions in Theorem 3, for an atomic cover V in the graph, if all pure children of it are observed, then all the edge coefficients from V to its pure children can be identified. Now, we will prove by induction to show that, for a graph that satisfies the conditions in Theorem 3, for any atomic cover V in the graph, all the edge coefficients from V to its children can be identified, and thus all the edge coefficients of the graph can be identified (the set of all edge coefficients in the graph is the union of the set of edge coefficients from each V to each V's children).To this end, we first index all the atomic covers by the inverse causal ordering, such that leaf nodes have smaller indexes. Then we have a sequence of atomic covers V i , i = 1, ..., C in the graph, where C is the number of atomic covers in the graph.(i) We show for V i , i = 1, all the edge coefficients from V 1 to its children can be identified. This is proved by considering (a) (b) (c) (d), as V 1 's children must be all observed; otherwise it cannot be indexed as 1.(ii) We show that, for i > 1, if for all V j , 1 ≤ j < i, all the edge coefficients from V j to V j 's children has been identified, then all the edge coefficients from V i to V i 's children can also be identified. This can be proved by combining (a) (b) (c) (d) with Lemma 1. If V i has children that are latent, then the latent children must belong to an atomic cover with a smaller index. Therefore, as all the edge coefficients from V j to V j 's children have been identified, by the use of Lemma 1, the latent children of V i can be taken as if they are observed. Therefore, all the edge coefficients from V i to V i 's children can also be identified.Taking (i) and (ii) together, all the edge coefficients of the graph can be identified.Example. An example would be in Figure 6 . As illustrated in the example in (d), edge coefficients from {L 2 , X 6 , X 7 } to {X 12 , X 13 } can be identified. By Lemma 1, L 2 can be taken as if it is observed. The reason lies in that other than X 12 and X 13 , there is no descendant of {L 2 , X 6 , X 7 }; therefore, from all other variables' perspective, e.g., X 11 , σ X11,L2 can be calculated as(σ X11,X12 -Σ 7 i=6 σ X i ,X11 f X i →X12 )/f L2→X12 .In other words, σ X11,L2 can be calculated using observational data, so L 2 can be taken as observed from X 11 's perspective. Plus, by the example in (c), we know that all edge coefficients from {L 1 , X 5 } to {X 8 , X 9 , X 10 } can be identified and thus similarly by Lemma 1, L 1 can be taken as if it can be observed. Finally, we consider edge coefficients among L 1 , X 5 , X 3 , X 4 , L 2 , X 6 , X 7 . In this situation, L 1 and L 2 can be taken as if they can be observed and the other variables are themself observed, therefore the edge coefficients among L 1 , X 5 , X 3 , X 4 , L 2 , X 6 , X 7 can be easily identified.A.4 Proof of Theorem 4 Lemma 2. Let Σ X be the observed covariance matrix entailed by A, B, C, D, Ω X , Ω L . Let Q be an orthogonal matrix, andÃ := Q T AQ, B := Q T B, C := CQ, D := D, ΩL := Q T Ω L Q, and ΩX := Ω X .Then, the matrices Ã, B, C, D, ΩX , and ΩL can also entail the covariance matrix Σ X .Proof of Lemma 2. Let M and N be matrices defined as in Proposition 1, and similarly for M and Ñ . We then haveM = I -Ã -B(I -D) -1 C -1 = Q T Q -Q T AQ -(Q T B)(I -D) -1 (CQ) -1 = Q -1 I -A -B(I -D) -1 C -1 Q -T = Q T M Q and Ñ = (I -Ã) C-1 (I -D) -B -1 = (Q T Q -Q T AQ)(CQ) -1 (I -D) -Q T B -1 = (I -A)C -1 (I -D) -B -1 Q -T = N Q.By Proposition 1, the latent covariance matrix ΣL is given byΣL = M T ΩL M + Ñ T ΩX Ñ = (Q T M T Q -T )(Q T Ω L Q)(Q -1 M Q) + Q T N T Ω X N Q = Q T (M T Ω L M + N T Ω X N )Q = Q T Σ L Q.By Proposition 1, the observed covariance matrix ΣX is given byΣX = (I -D) -T BT ΣL B + ΩX + ΩX Ñ B + BT Ñ T ΩX (I -D) -1 = (I -D) -T (Q T B) T (Q T Σ L Q)(Q T B) + Ω X + Ω X (N Q)(Q T B) + (Q T B) T (N Q) T Ω X (I -D) -1 = (I -D) -T B T Σ L B + Ω X + Ω X N B + B T N T Ω X (I -D) -1 = Σ X .This indicates that the matrices Ã, B, C, D, ΩX , and ΩL can also entail the covariance matrix Σ X .Using Lemma 2, we can prove Theorem 4. Theorem 4 (Condition for the Existence of Orthogonal Transformation Indeterminacy). Consider the model in Definition 1. If a set of latent variables L with |L| ≥ 2, have the same parents and children, then there must exist orthogonal transformation indeterminacy regarding the edge coefficients F . In other words, F can at most be identified up to orthogonal transformation indeterminacy.Proof of Theorem 4. Let S 1 , S 2 , S 3 , S 4 , and S 5 be the indices of L, their latent parents, their latent children, their measured parents, and their measured children in G, respectively. Let Q be a |L| × |L| orthogonal matrix. For matrices A, B, C, and D from matrix F , suppose that we replace A S2,S1 , A S1,S3 , C S4,S1 , and B S1,S5 with A S2,S1 Q, Q T A S1,S3 , C S4,S1 Q, and Q T B S1,S5 , respectively. Then, we will show that the entailed covariance matrix Σ X is unchanged.Let U be an (m + n) × (m + n) orthogonal matrix such that: (i) U S1,S1 = Q, (ii) the remaining diagonal entries are ones, and (iii) the remaining non-diagonal entries are zeros. Let By Lemma 2, the matrices above can entail the same covariance matrix Σ X .By construction of U , left multiplication of U T on B only affects B S1, * ; specifically, it is equivalent to replacing B S1, * with Q T B S1, * . Furthermore, only the columns of S 5 in B S1, * will be affected, because those columns correspond to the measured children of L. Therefore, all entries of B are the same as B, except that B S1,S5 is replaced with Q T B S1,S5 . Similar reasoning shows that all entries of C are the same as C, except that C S4,S1 is replaced with C S4,S1 Q.Now consider U T AU . By the reasoning above, left multiplication of U T on A only is equivalent to replacing A S1,S3 withQ T A S1,S3 . Further right multiplication of U on U T A is equivalent to replacing (U T A) S2,S1 with (U T A) S2,S1 Q. Since S 1 , S 2, and S 3 are mutually disjoint, all entries of Ã = U T AU are the same as A, except that A S2,S1 and A S1,S3 are replaced with A S2,S1 Q andQ T A S1,S3, respectively.Hence, for matrices A, B, C, and D, suppose we replace A S2,S1 , A S1,S3 , C S4,S1 , andB S1,S5 with A S2,S1 Q, Q T A S1,S3 , C S4,S1 Q, and Q T B S1,S5, respectively. By the reasoning above, this is equivalent to replacing A, B, C, and D with Ã, B, C, and D, respectively, which share the same support and entail the same covariance matrix Σ X .Corollary 1 (General Necessary Condition for Parameter Identifiability). For parameters to be identifiable, every pair of latent variables has to have at least one different parent or child.Proof of Corollary 1. Proof by contradiction. If it is not the case that every pair of latent variables has to have at least one different parent or child, then there exist L such that |L| ≥ 2 and L share the same parents and children. Therefore by Theorem 4 there must exist orthogonal transformation indeterminacy regarding F , and thus the parameters are not identifiable.Corollary 2 (Necessary Condition about Atomic Covers for Parameter Identifiability). Assume G satisfies Conditions 1 and 2 and thus the structure can be identified up to the MEC of O min (O s (G)).For any DAG G in the equivalence class, for G's parameters to be identifiable, every atomic cover must contain no more than one latent variable.Proof of Corollary 2. Proof by contradiction. If for a DAG in the equivalence class, there is an atomic cover that has more than one latent variable, then according to the definition of the concerned equivalence class, the latent variables in that atomic cover share the same parents and children. Then by Theorem 4 there must exist orthogonal transformation indeterminacy regarding F , and thus the parameters are not identifiable. Then, the population covariance matrices of L G and X G can be formulated asEQUATIONEQUATIONProof of Proposition 1. Since matrices A and D are invertible, using the formula of 2 × 2 block matrix inversion [22, Chapter 0.7], we obtain(I -F ) -1 = M -M B(I -D) -1 -(I -D) -1 CM (I -D) -1 + (I -D) -1 CM B(I -D) -1 , which implies (I -F ) -T = M T -M T C T (I -D) -T -(I -D) -T B T M T (I -D) -T + (I -D) -T B T M T C T (I -D) -T and (I-F ) -T Ω = M T Ω L -M T C T (I -D) -T Ω X -(I -D) -T B T M T Ω L (I -D) -T Ω X + (I -D) -T B T M T C T (I -D) -T Ω X .We then haveΣ L = M T Ω L M + M T C T (I -D) -T Ω X (I -D) -1 CM = M T Ω L M + N T Ω X Nand Here we want to investigate, given a structure, can we efficiently check whether the proposed sufficient conditions hold? To this end, we generate random graphs and each graph has 100 variables. According to our empirical result, such a check can be done very efficiently. Specifically, on average, given a structure with 100 variables, it only takes our Python code around 3 seconds to check whether the conditions hold. Our condition is useful in solving real-life problems. For example, in the psychometric study, we can properly design the questions with domain knowledge following the condition in Theorem 3 such that each single latent variable has enough observed variables as pure children and thus it can be ensured that all parameters are identifiable (as illustrated in our real-life data result in Figure 4 ).Σ X = (I -D) -T B T M T Ω L M B(I -D) -1 + (I -D) -T Ω X (I -D) -1 + (I -D) -1 Ω X (I -D) -1 CM B(I -D) -1 + (I -D) -T B T M T C T (I -D) -T Ω X (I -D) -1 + (I -D) -T B T M T C T (I -D) -T Ω X (I -D) -1 CM B(I -D) -1 = (I -D) -T Ω X + B T M T Ω L M B + Ω X (I -D) -1 CM B + B T M T C T (I -D) -T Ω X + B T M T C T (I -D) -T Ω X (I -D) -1 CM B (I -D) -1 = (I -D) -T Ω X + B T Σ L B + Ω X N B + B T N T Ω X (I -D) -1 .On the other hand, even though sometimes the questionnaires and data were designed not so well such that the conditions are not satisfied for the identification of parameters, our Theorem 3 is still useful. In this case, we can still make use of our conditions to check the given structure, and find some local sub-structures where our conditions are satisfied. Consequently, it can be ensured that all the parameters of some sub-structures are identifiable, and we can employ our estimation method to find all the edge coefficients of these sub-structures. In Figure 7 (a), there are four treks between X 4 and X 5 : (i) X 4 ← L 1 → X 5 , (ii) X 4 ← X 3 → X 5 , (iii) X 4 ← L 1 ← X 2 → X 3 → X 5 , and (iv) X 4 ← X 3 ← X 2 → L 1 → X 5 , illustrated inDefinition 6 (Pure Children [18] ). Y are pure children of variablesX in graph G, iff Pa G (Y) = ∪ Y i ∈Y Pa G (Y i ) = X and X ∩ Y = ∅.We denote the pure children of X in G by PCh G (X).The definitions are as follows with examples.Definition 7 (Minimal-Graph Operator [24, 18] ). Given two atomic covers L, P in G, we can merge L to P if the following conditions hold: (i) L is the pure children of P, (ii) all elements of L and P are latent and |L| = |P|, and (iii) the pure children of L form a single atomic cover, or the siblings of L form a single atomic cover. We denote such an operator as minimal-graph operator O min (G).Definition 8 (Skeleton Operator [24, 18] ). Given an atomic cover V in a graph G, for all V ∈ V, V is latent, and all C ∈ PCh G (V), such that V and C are not adjacent in G, we can draw an edge from V to C. We denote such an operator as skeleton operator O s (G).Suppose a graph G of a latent linear model in Figure 9 (a) is G. After applying O min (O s (G)), we have the graph in Figure 9 (b). Specifically, the O s operator adds an edge from L 2 to X 5 and the O min operator delete L 4 and add an edge from L 1 directly to X 6 and X 7 . For G, such two operators will not change the rank in the infinite sample case.Please refer to Figures 10 and 11 . Please refer to Tables 2and 3.Our optimization problem in Eq. 4 is solved by gradient descent using PyTorch. Our current implementation is based on CPU but it can be further accelerated by using GPU. A very related discussion can also be found in [35] .The optimization problem in Eq.3 is solved by gradient descent, which involves evaluating the LogDet and matrix inverse (for the gradient) terms (which is similar to continuous causal discovery methods based on Gaussian likelihood [35] ). According to [53] , the computational complexity is O(td 3 ), where d is the number of variables and t is the number of iterations of gradient descent respectively. Note that the computational cost is largely independent of the sample size as we only need to calculate the sample covariance once and save it for further use.It is possible to perform inference on the learned parameters in our framework. To be specific, as we use maximum likelihood estimation for the parameters, some standard techniques can be readily used. For example, bootstrap can be employed to provide standard errors on linear coefficients and Chi-square test can also be done to examine the fitness of the model.One main line of research in latent variable estimation centers on factor-analysis-based methods.Representative studies include [41, 45, 36, 56, 7, 57] . Various other techniques have also been employed for latent structure and parameter identification, including over-complete ICA-based techniques [23, 43, 1] that leverage non-Gaussianity and matrix decomposition-based approaches [3] . However, these approaches typically consider latent variables with observed children, without considering parameter identification in latent hierarchical structures. A more related work is [4] , but it considers a much simpler structure. Another direction would be to project the graph to an ADMG and the latent confounding effects are encoded by correlated noise terms. Following this idea, graphical criteria such as half-trek [20, 5] , G-criterion [9] , and some further developments [51, 29, 42, 8, 19] has been proposed. Furthermore, another line of works involve studies on causal effect estimation in the presence of latent confounders [52, 6, 30, 33, 26] , which often rely on instrumental variables or proxy variables for identification. Notice that in this task, the parameters may not be identified [30] , although the causal effect from the treatment variable to the outcome variable can be identified.Furthermore, several existing works also solve an optimization problem that involves parameterization of maximum likelihood, such as those in continuous optimization for causal discovery [35, 59, 31, 10, 34] and parameter estimation of Lyapunov models [55, 15] . Differently, our formulation involving likelihood parameterization aims to estimate parameters of partially observed linear causal models.One limitation of this work is that our theoretical results are based on the assumption of linear gaussian causal models. When data is not linear gaussian, we have also conducted experiments to see the performance of our method. It turns out that our method still performs well in the presence of certain extents of violation of normality and linearity. However, theoretical analysis under violation of linearity and normality would be interesting and the focus of future work.The goal of this paper is to advance the field of machine learning. We do not see any potential negative societal impacts of the work. NeurIPS Paper Checklist Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 


Decision: This paper investigates parameter identification in linear causal models including both observed and latent variables, and proposes necessary and sufficient conditions for parameter identifiability. This is an important problem that is clearly relevant for NeurIPS, and all reviewers have recommended acceptance.

Reviews: review: paper_summary: paper_summary: This paper introduces conditions under which DAGs can be recovered in the linear case where some nodes are observed and some are not. This DAG recovery involves computing edge weights between nodes in a causal graph. Nodes are allowed to be latent or observed, with varying types of edge weight indeterminacy depending on the structure of the causal system in question.
strength_weakness: strength_weakness: The text overall is generally well-written, with some caveats listed below. In my reading, the first half of the paper read more clearly than the second half. For example, I couldn't quite piece together from the discussion of estimation whether the estimated graph will be dense (all nodes connected to all other nodes, given the [seemingly?] continuous optimization being done, e.g., Eq 3. If all edges are connected to all other edges, then the relative usefulness seems to be weakened in that usually, investigators seek out a parsimonious representation of a causal system. 

I was also wondering what more established methods would yield as an empirical baseline (e.g., PC algorithm); currently, the Estimator-LM (estimator with Lagrange multiplers) is articulated as a baseline. This is one of the methods introduced in the paper. An external state-of-art baseline would be most informative. The authors state that "no existing method...can achieve the same goal as ours." If this is because of the latent variable aspect, one could in principle restrict the MSE calculation to edges among observed nodes. In other words, perhaps there isn't a perfect analogue method but an imperfect comparison could be better than none. We could also get a visual comparison of the DAG among observed variables used in Figure 4 from some existing baseline methods for the Appendix. 

Finally, there is little discussion of uncertainty estimation. Uncertainty estimation in the observed DAG recovery case is hard, even more so here (presumably). 

There are points where the text could, to me, use more clarity in the discussion: 

- Condition 1 seems relatively minimal and even intuitive. It seems very hard to know in practice if Condition 2 (line 193) holds or is even a minimal or very restrictive condition.

- I appreciate the author(s)' inclusion of Example 2 and Example 3. I think the logic could be made clear, perhaps with additional shadings or labelings that would help us see which sets of nodes and edges are doing what work regarding Condition 1 and Condition 2. 

- I would make the "pretty close" language in 218 and 255 a bit more precise. Also, starting line 264. I would revise this from, "it has considerable extents of necessity, and could be expected to serve as a stepping stone towards tighter and ultimately the necessary and sufficient condition for the field." to something that also is seomwhat more precise. Also, there is no guarantee that necessity+sufficiency will be found (or perhaps there is an impossibility), so would hedge this possibility somewhat. 

- Can you define what it means for "QF and F" to "share the same support" in this case? "Support" is often defined in causal inference settings as an event probability falling between 0 and 0; here, F is defined as the matrix embodying the causal edge coefficients, so I believe what is meant here is that QF and F do not share the same set of non-zero entries. Clarifying this would be helpful. I would also consider beginning with the example of indeterminacy before defining it to help the reader see your point intuitively before the formalization.

- I would definition 7 into the main text. It is an important definition used multiple times and without it, it is hard to follow the atomic cover discussion. it's also a very short definition.

Moreover, I would in general help the reader along by first explaining the concept before formally/technically defining it. Examples: 

- The term structure identifiability and parameter identifiability should be clearly defined before the terms are used. (I don't think I see a clear definition before use currently; I would move the paragraph beginning on line 171 up in the text, as it is a clear articulate of the point.) In a similar vein, I would explain what atomic covers are going to do before we jump into the definition on line 164. 

Other details would help this reader: 

- The MSE up to orthogonal transformation is an interesting metric. Some mention of how this optimization is done would be helpful.

- I would add a sentence explaining whether GPU acceleration would or would not be helpful and why.

I also noted several minor points listed here: 

- References used are inconsistent at times. Sometimes, we see reference to "condition (i)", others to Condition 1. 

- Line 212. Missing space "identified upto the" should read "identified up to the". This same occurs in line 140 ("upto" should read "up to") and in other parts of the text as well. 

- Line 147. "indetermincay" should read "indeterminacy". This typo occurs a few times in the text. 

- Line 129. Clarify what is meant by "entails the same observation as that of..." This also appears in line 142 ("Entails the same observation"). I assume this means something about implying the same probability distribution, but helping the uninitiated reader is usually appreciated. 

- Line 137. "However, if we set f1,2 = 0, then the parameters are not identifiable. These rare cases of parameters are of zero Lebesgue measure so we rule out these cases for the definition of identifiability" -> I would just say "these presumably rare cases of parameters". Probably some justification is needed to articulate why this should be rare in real causal systems. Does any prior literature speak to this?

- Line 72. I believe "the causal edge coefficient of the model" should read "the causal edge coefficients of the model".

- Capitalize "Gaussian" on line 286.
questions: questions: - What are the implications of the diagonal covariance matrix $\epsilon_{\mathbf{V}_{\mathcal{G}}}$? 

- Regarding, "As variables are jointly Gaussian, asymptotically our observation can be summarized as population covariance over observed variable". Wouldn't this statement also apply in finite samples under Gaussianity?

- A major benefit seems to be identification of edge weights. If actual interpretation of the edge weights is going to be done in practice, group sign indeterminancy would limit applicability. Would it help to anchor the sign of one edge based on prior science? Guidance? 

- In Figure 4, are circulate nodes latent and nodes denoted by [LetterNumber] observed? If so, clearly articulate this in the figure label.
limitations: limitations: I see no major negative societal impacts.
paper_summary: paper_summary: This paper investigates the parameter identifiability of partially observed linear causal models, focusing on whether edge coefficients can be recovered given the causal structure and partially observed data. It extends previous research by considering relationships between all variables, both observed and latent, and the coefficients of all edges. The authors identify three types of parameter indeterminacy in these models and provide graphical conditions for the identifiability of all parameters, with some conditions being necessary. A novel likelihood-based parameter estimation method is proposed to address the variance indeterminacy of latent variables, validated through empirical studies on synthetic and real-world datasets, showing the effectiveness of the proposed method in finite samples.
strength_weakness: strength_weakness: The paper considers a restricted setting, allowing only linear relations between observed and unobserved variables and restricting the noise to be Gaussian. The theoretical guarantees hold under the scenario when the noise covariance matrix is unknown. Nonetheless, the authors mitigate this by providing an estimation scheme for the noise covariance matrix from limited data samples. The conditions of identifiability proposed are not fully necessary and sufficient; this is left for future work.
questions: questions: I have  few questions and comments for the Authors:

*  Line 139: The authors mention that the indeterminacy of group sign is rather minor. if the parameters are identifiable only up to group sign indeterminacy, we still say that the parameters are identifiable. It would be useful to explain why group sign indeterminacy is a minor issue. It might actually depend on the underlying task, and there might be application scenarios where it is not insignificant. Some details on this would be useful in the main paper.

*  The additive noise is assumed to be Gaussian. Suppose the noise follows another continuous distribution such as Gamma or Student-t distribution. Can we use the identifiability results? If not, what is special about the Gaussian noise here compared to other continuous noise distributions?

* Referring to Proposition 1, some matrix inverses need to be calculated to compute the noise covariance matrices. Will the inverses always exist? If yes, can you elaborate on how? If no, how does that impact the application of Proposition 1?

* In the experiments section, the authors demonstrate the effect of sample size on the mean squared error (MSE) for the parameter matrix \( F \). It would be useful to also show how accurate the estimates of the noise covariance matrix depending on the sample size used.
limitations: limitations: The authors clearly describe the problem setting and assumptions. I have mentioned the main limitation of the proposed method in the weaknesses section. I don't think there are any potential negative societal impacts of this work.
paper_summary: paper_summary: This paper investigates the problem of parameter identification in linear causal models, which is important and well-studied task in causality. The authors examine models that explicitly include both observed and latent variables. The identification of parameters in such models has not been studied so far and the authors are the first to formulate this problem and provide the first results in this regard. The main achievements of the paper are the sufficient and necessary conditions for parameter identifiability.  Moreover the authors conducted empirical studies on both synthetic and real-world data to validate the proposed methods.
strength_weakness: strength_weakness: The sufficient condition for parameter identifiability assumes that G, in addition to conditions (i) and (ii) in Thm. 1, satisfies conditions 1 and 2 presented in section 3.2. I agree that conditions 1 and 2 imply that the structure G can be identified (as shown in [17]) however the  sufficient conditions formulated in this way are overall very restrictive. It would be interesting to have sufficient conditions also in the case of structures which do not satisfy conditions 1 and 2: It can happen that G can be identified even if it does not satisfy 1 and 2 or the structure is provided by a researcher / theory. The authors do not discuss if there exist cases which can be identified but which do not satisfy 1 or 2.

Also, it is not clear what is the gap between the sufficient and necessary conditions.

The next issue is that the authors do not discuss what is the computational complexity of parameter identification in linear causal models, that explicitly include both observed and latent variables.
 
It is not clear to what extent the sufficient and necessary conditions proposed in Section 3 are useful for (numerical) parameter estimation discussed in Section 4.
questions: questions: Please provide the motivation for considering edges / parameters from observed to latent variables and edges between latent variables.

See also my questions above.

In Line 69: explain that d=n+m
limitations: limitations: yes
paper_summary: paper_summary: The manuscript proposes novel methods for learning linear structural equation models from partially observed data (i.e., allowing for latent variables). The authors provide graphical identifiability conditions for such models and describe an algorithm for learning structural parameters from data via gradient descent. Results on synthetic and real-world examples suggest that the method works well in practice.
strength_weakness: strength_weakness: I was a bit confused by the discussion of necessary graphical conditions following Thm. 1. First, I would recommend avoiding terms like "pretty close to...necessary", which doesn't mean much. We get a bit more insight in Remark 1, where it's revealed that condition (i) *is* in fact necessary, but condition (ii) is not. We learn that if (ii) does not hold, then "there are...some rare cases where parameters can be identified." Oddly, we don't see any examples of such structures (either in the main text or the supplement), or learn what "rare" amounts to here (presumably *not* Lebesgue measure zero?) Specific counterexamples would help a great deal! Perhaps some sort of disjunctive graphical condition could do the trick, e.g. condition (i) + [(ii) OR (iii)], where (iii) covers those purportedly "rare" structures that violate (ii) but are still technically identifiable. Alternatively, I would cut all discussion of necessity from this section and move it instead to a discussion section later in the manuscript. (I appreciate that space is tight here, but with an extra page in a final version this could be a more satisfying solution).


Minor errata: 
-All instances of "upto" should be "up to"
-It appears that references 46 and 47 are the same
-It appears that references 5 and 19 are the same
-The word "Gaussian" is occasionally uncapitalized
questions: questions: -I take it that the identiability conditions of Thm. 1 are untestable? Or perhaps they have some testable consequences in certain settings? If so, this would be very helpful for practitioners!

-Would it be possible to perform inference on the learned parameters? For instance, could this method provide standard errors on linear coefficients? 

-Though I know the method is designed for the partially observable setting, I'm curious how it fares against competitors such as GES or PC when latent variables are absent?
limitations: limitations: Yes, in Appx. D. May be worth moving this to the main text in the final submission, however.

Comments: title: Comments
Re: Author rebuttal
Author response
Response
Re.
Rebuttal by Authors Part 2
comment: Dear Reviewer PaWo,

Thank you for the positive feedback. We are so happy that all your questions/concerns were properly addressed. Your insightful review comments have helped us further improve the quality and clarity of our paper. We wonder whether you would kindly reconsider your rating based on our responses to your questions. Your consideration is highly appreciated. 

Many thanks!

Your sincerely,

Authors of submission 11557
Dear Authors, 

thank you for your thorough answers to my questions and comments. It seems the rebuttal addresses all my concerns.
Dear Reviewer PaWo,

Thank you once again for your insightful comments and valuable feedback. We would like to know if our response has addressed your concerns and questions? If there is any remaining confusion, we are happy to address them as soon as possible.

Many thanks!

Your sincerely,

Authors of submission 11557
We would like to once again express our appreciation for your insightful comments, helpful writing suggestions, and positive feedback. If you have any further insights or questions, we would be more than happy to hear from you.

Many thanks!

Your sincerely,

Authors of submission 11557
Many thanks to the authors for their thoughtful replies to my comments. I will maintain my score and look forward to seeing the revised manuscript in the camera ready draft.
Thank you again for your helpful comments and positive feedback. It means a lot to us. We are so happy that most of your questions/hesitations were properly addressed. If you have any further insights, we would be more than happy to hear from you.

Thank you!

Your sincerely,

Authors of submission 11557
Thank you once again for your valuable comments which have helped improve the clarity of our paper. If you have any further questions or insights, we would be more than happy to hear from you. 

Thank you!

Yours sincerely,

Authors of submission 11557
Many thanks to the authors for their detailed responses. These answers clarify some of my questions/hesitations; the associated paper revisions should help bolster the contribution too. The discussion of sparsity and optimization will help readers understand the contribution better, as well as the impact of some of the required assumptions. 

Pondering the question of whether to alter the numerical rating, with the addition of some of the new baselines, I update my view of the paper from, "no major concerns w.r.t. evaluation" to "with good evaluation", and hence move my score to a "7".
Thanks for responding to questions in detail. The clarity of the paper would be improved given that the authors update the paper as they mentioned in their response to my review. I would keep my acceptance decision and score for the paper.
**Q9:**
Regarding, "As variables are jointly Gaussian, asymptotically our observation can be summarized as population covariance over observed variable". Wouldn't this statement also apply in finite samples under Gaussianity?

**A9:** In finite samples under Gaussianity, our observation can be summarized as the empirical covariance, which is an estimation of the population covariance.

**Q10:**
If actual interpretation of the edge weights is going to be done in practice, group sign indeterminancy would limit applicability. Would it help to anchor the sign of one edge based on prior science? Guidance?

**A10:** Yes, we can always anchor the sign of some edges according to our preference or prior knowledge in order to eliminate such indeterminacy. For example, in Figure 4, 
if we expect that L2 should be understood as Extraversion instead of non-Exterversion, we can add one additional constraint during our parameter estimation such that the edge coefficient from L2 to E1 ("I am the life of party.") will be positive (as we believe E1 should be positively related to Extraversion). Thank you for your insightful question and we have added a related discussion to our revision.

**Q11:**
In Figure 4, are circulate nodes latent and nodes denoted by [LetterNumber] observed? If so, clearly articulate this in the figure label

**A11:**
Yes. We have revised the caption of Figure 4 as you suggested.

Regarding the suggestions on writing such as additional shadings for examples, 
more precise statements for Remark 1,
moving definition 7 into the main text and some explanations to the front of corresponding definitions, and correction of some typos, we thank the reviewer and have revised them accordingly.
We genuinely appreciate the reviewer's effort and hope that your concerns/questions are addressed.