Identification of Latent Confounders via Investigating the  Tensor Ranks of the Nonlinear Observations
Authors: Zhengming Chen
Yewei Xia
Feng Xie
Jie Qiao
Zhifeng Hao
Ruichu Cai
Kun Zhang
Venue: ICML 2025 poster (2025)
Keywords: Causal discovery
discrete latent variables
Primary area: probabilistic_methods->structure_learning
TL;DR: We study the problem of learning causal structure of discrete latent variables

Abstract: We study the problem of learning discrete latent variable causal structures from mixed-type observational data. Traditional methods, such as those based on the tensor rank condition, are designed to identify discrete latent structure models and provide robust identification bounds for discrete causal models. However, when observed variables—specifically, those representing the children of latent variables—are collected at various levels with continuous data types, the tensor rank condition is not applicable, limiting further causal structure learning for latent variables. In this paper, we consider a more general case where observed variables can be either continuous or discrete, and further allow for scenarios where multiple latent parents cause the same set of observed variables. We show that, under the completeness condition, it is possible to discretize the data in a way that satisfies the full-rank assumption required by the tensor rank condition. This enables the identifiability of discrete latent structure models within mixed-type observational data. 
Moreover, we introduce the two-sufficient measurement condition, a more general structural assumption under which the tensor rank condition holds and the underlying latent causal structure is identifiable by a proposed two-stage identification algorithm. Extensive experiments on both simulated and real-world data validate the effectiveness of our method.

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Zhengming Chen 12
Yewei Xia
Feng Xie Jie Qiao Zhifeng Hao!
Ruichu Cai? Kun Zhang 46
Abstract
1. Introduction
We study the problem of learning discrete latent variable causal structures from mixed-type obser- vational data. Traditional methods, such as those based on the tensor rank condition, are designed to identify discrete latent structure models and provide robust identification bounds for discrete causal models. However, when observed vari- ables-specifically, those representing the chil- dren of latent variables-are collected at various levels with continuous data types, the tensor rank condition is not applicable, limiting further causal structure learning for latent variables. In this paper, we consider a more general case where observed variables can be either continuous or diserete, and further allow for scenarios where multiple latent parents cause the same set of ob- served variables. We show that, under the com- pleteness condition, it is possible to discretize the data in a way that satisfies the full-rank as- sumption required by the tensor rank condition. This enables the identifiability of discrete latent structure models within mixed-type observational data. Moreover, introduce the two-sufficient measurement condition, more general structural assumption under which the tensor rank condition holds and the underlying latent causal structure is identifiable by a proposed two-stage identifi- cation algorithm. Extensive experiments on both simulated and real-world data validate the effec- tiveness of our method.
Latent confounders, typically referred as unobserved vari- ables that influence multiple observed variables, are fre- quently encountered in various disciplines such as statistics, machine learning, and causal discovery. Ignoring latent confounders may introduce spurious correlations among observed variables, impeding the correctness and robustness of models trained from observational data. Taking into ac- count latent confounders modeling procedures remains Long-standinng challenge and has attracted wide attention in both theoretical and applied research (Zheng et al., 2023; Li et al., 2024; Miao et al., 2018; Thaden & Kneib, 2018; Wang & Blei, 2019; Bartolucci et al., 2023).
Much effort has been made to handle the latent confounding problem in causal structure learning. One line of research considers the causal structure over the measured variables, including constraint-based methods (Spirtes et al., 2000; Colombo et al., 2012; Zhou et al., 2020), functional model- based ones (Chen et al., 2021b; Hoyer et al., 2008; Salehka- leybar et al., 2020), and hybrid model approaches (Chen et al., 2021c; Ogarrio et al., 2016). Another line of research focuses on identifying the causal structure among latent vari- ables by utilizing the pure measurement model assumption, where each latent variable has certain measured variables its children. This includes linear model-basee methods (Silva al., 2006; Cai et al., 2019; Xie al., 2020; Chen et al., 2022; Jin et al., 2023; Huang et al., 2022; Xie et al., 2024) and discrete latent tree model-based approaches (Gu, 2022; Gu & Dunson, 2023; Choi et al., 2011: Zhou et al., 2020). These methods either assume linear relationships among all variables or constrain the latent structure to a specific structure. Consequently, they are unable to handle general causal structure with non-linear relationships among latent variables and observed variables.
Coolleee of Mathematics and Computer, Shantou University, Shantou, China "School of Computer Science, Guangdong Uni- versity of Technology, Guangzhou, China Department of Com- puter Science, Fudan University, Shanghai, China 'Department of Machine Learning, Mohamed Bin Zayed University of Ar- tificial Intelligence, Abu Dhabi, UAE 'Department of Ap- plied Statistics, Beijing Technology and Business University, Beijing. China 'Department of Philosophy. Carnegie Mellon University, Pittsburgh, USA. Correspondence to: Ruichu Cai <cairuichu@ com>, Kun Zhang <kunz] @ cmu.edu>.
When observed variables are continuous and nonlinearly depend on the discrete latent variables, one straightforward way to learn the causal structure among latent variables to treat it mixture model (Teicher, 1967: Tahmasebi et al., 2018; Allman et al., 2009; Ritchie et al., 2020), so that the causal structure can be learned from the recovered latent distribution (Kivva et al., 2021). However, such re- covery requires an unbiased estimation of the parameters
Proceedings of the nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
of the mixture model, which is unrealistic and impractical (Chung et al., 2004; xian Wang et al., 2004). Recent work by (Chen et al., 2024) shows that it is possible to identify causal relations among latent variables by recovering the cardinality of the support of latent variables, which is rather practical and also testable. However, it is only applicable in discrete cases, and it is still unclear whether the results hold in the presence of continuous observed variables.
V = (X, L), where X E R" is the observed variable set while L E 52 is the latent variable set, 52 is discrete space with [Ω]] r 2 2. In a causal graph, we do not allow directions from observed to latent to preserve the latent confounder structure. The data generation of observed variables is defined as
(1) where Pa(Xi) is the parent set of X, and Ex; is noise term of X,, We denote the generating function by f [ax(X)), + X,, We denote such a causal model as the nonlinear causal model with discrete latent confounders.
X; == (Pa(())),
In this paper, we investigate the problem of identifying discrete latent variables and their causal structure from non- linearly dependent observed variables. When the observed variables are also discrete, the tensor rank condition (Chen et al., 2024) implies d-separation by identifying the cardi- nality of the latent supports from observed variables. This raises the question: can the cardinality of latent supports be identified from continuously observed variables, and then can the tensor rank condition be applied to further learn the causal structure? Interestingly, we find that it is possible to identify the cardinality of latent supports from nonlin- early dependent observed variables (whether discrete or con- tinuous) by, for example, using appropriate discretization techniques. We show that, under the completeness condi- tion, the tensor rank condition can be used to identify the d-separation among observed variables that are non -on-linearly dependent given the discrete latent confounder, if there are two sufficient measured variables for each latent variable. Based on this, we apply the tensor rank condition to the dis- crete latent structure model with mixed-type observational data and develop a two-stage identification algorithm to learn more general discrete latent variable structure, where an observed variable can have multiple latent parents. We theoretically show that the latent structure of the discrete la- tent structure model is identified up to Markov equivalent class by properly utilizing the tensor rank condition.
In a causal model, certain causal assumptions are neces- sary to ensure identifiability. We assume that the distri- bution P(V) satisfies the Markov property and Faithful- ness condition with respect to G. That is, the distribution P(V) = eV P(V)))a(VV)) and the conditional inde- pendence relations imply the d-separation structure in the causal graph (Spirtes et al., 2000). Besides, to identify the discrete latent variable only from observational data, we make the following non-degeneracy assumption.
Assumption 2.1 (Non-degeneracy)) The distribution over V {X, L) satisfies:
(a) [Non-zero mass condition] For any r E S2 P(L, -)
O for all L; E L.
(b) [Full-rank condition] For any discrete conditional distri- bution contingency table P(V))Pa(V))) is full rank.
The non-degeneracy assumption has also been used in (Kivva et al., 2021; Kong et al., 2024; Chen et al., 2024) to ensure the identifiability of discrete latent variables. Further discussion on this topic can be found in (Kivva et al., 2021).
Throughout this paper, we standard notation such as Pa((V) VV - V.J. Ch((V) (V,lV - V2), Ane(V) (V ~ V.J. Des(V.) (V - Vj} to denote the set of parents, children, ancestors, descendants, and nodes of V., respectively. For a discrete variable V,, we use supp (V,) = (u E Z+ PVV = v) > 0} to denote the set of possible values of V₁, and T(x..) to denote probability tensor of discrete joint distribution IP(X,). Beside, we |Vpl denote the dimension (or cardinality) of Vp-
The contributions of this work are three-fold: (1) We first demonstrate that under the completeness condition, the ten- rank condition holds for variable sets that may be non- linearly dependent on latent confounders. (2) We develop an identification algorithm that identifies a more general discrete latent variable structure up to Markov equivalent class. (3) We conduct simulated experiments to verify its capability to handle mixed-type observational data.
2. Non-linear Causal Models with Discrete Latent Confounders
Goal. We aim to develop a statistically testable method to learn the causal structure among latent variables in the nonlinear causal model with discrete latent confounders.
In this paper, we focus on the causal graphical model (Spirtes et al., 2000) and study how to identify latent vari- ables and their causal relationships from measured variables that may be nonlinearly dependent on latent variables. We first introduce the basic notation.
3. Graphical Criteria in Non-linear Causal Models with Discrete Latents
We begin with a brief review of discrete latent structure learning methods, with a focus on the tensor rank condition. Subsequently, we demonstrate that the tensor rank condition
Consider a causal graph G = {V, EJ, it is a DAG with
2

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
can be extended to nonlinear causal models with discrete latent confounders. Thus, provide a general result for the tensor rank condition (Theorem 3.4).
Theorem 3.2 (Graphical implication of tensor rank con- dition (Chen et al., 2024)). In the discrete causal model, suppose the Markov, Faithfulness condition and full-rank condition hold. Consider an observed variable set X2 = (X1,'.,,,, (X, S X and 2 S p S n) and the corresponding p-way probability tensor T(x,)) that is the tabular representation of the joint probability mass func- tion P(X11.... ,Xp), then Rank(((x,))
3.1. Background
When the mixture model over X is identifiable, (Kivva et al., 2021) shows that the causal structure among latent variables is identifiable from an existing mixture oracle. To estimate the parameters of the mixture oracle, such as the number of components, (Kivva et al., 2021) applies a K-means algo- rithm. However, since K-means is an approximate method, it may sometimes misidentify the number of components in real-world applications, potentially leading incorrect identification of the causal structure.
(r > 1)' if and only if (i) there exists a variable set S C V with |supp(S)] = r that d-separates any pair of variables in (X,,... .... Xp). and (ä) does no exist conditional set S that satisfies [supp(S)] <r.
The full-rank assumption (in Theorem 3.2) posits that any conditional probability table ((V[Pa(V)) is full rank, aligned with the non-degeneracy condition (b). Roughly speaking, the tensor rank condition shows that tify the cardinality of latent support by the probability tensor of observed variables, for detecting the d-separation rela- tions among observed or latent variables. However, when the observed variables are continuous, which exhibit more general nonlinear dependencies, it remains unclear under what conditions the graphical implications of the tensor rank condition hold for the probability tensor over continuous variables. This is because the graphical criteria rely on the full-rank assumption, which typically only applies to discrete variables. Fortunately, we find a well-studied condi- tion, the completeness condition, under which the graphical implications of the tensor rank condition also hold for the continuous observed variables.
Example 3.1 (Incorrect identifying the cardinality of latent support). As shown in Fig. 1, the left side represents the discrete latent DAG, while the right side shows the corre- sponding mixture distribution. In the latent structure, the latent variables L1, L2 have the same support {0, 1}, and the observed variable is mixed by the Laplace distribution. One can see that the cardinality of the latent variable is in- correctly identified as three (when it is actually four) by the K-means algorithm (Kivva et al., 2021), even in a simple structure (left side in Fig. .1).
iden-
The vioualization of the clustered data.
2nd oboerved owrkelle
Condition 3.3 (Completeness condition). For any X; E X that has only one (latent) parent Lj. we assume that the conditional distribution [ is complete. That is, for all measurable real functions g such that B((g(()) < +oo, [()())xz)) 0 almost surely iif. g(l) = 0 almost surely.
Figure 1. Example of discrete latent DAG and corresponding mixture distribution, clustered by K-means algorithms.
In this case, any function g Ω + R is indeed bounded. This condition is also used in (Miao et al., 2018; Cui et al., 2023; Liu et al., 2024). It is worth noting that many commonly parametric and semiparametric mod- els such as exponential families (Newey & Powell, 2003) and location-scal families like convolution density func- tions (Hu & Shiu, 2018) satisfy the completeness condition. Further discussion can be found in Appendix.
While the identification of latent variables can be built upon the identifiability of mixture models, learning mixture mod- els is a nontrivial problem (Kivva et al., 2021). Rather than relying on approximate methods to estimate mixture models, seeking a simple and robust approach is more mean- ingful. Recently, (Chen et al., 2024) introduced statisti- cally testable tool, the tensor rank condition, for handling discrete cases with discrete latent confounders. By exam- ining the rank of the probability tensor, which reflects the cardinality of the latent support, the tensor rank condition establishes a connection between algebraic constraints and the rl d-separation structure discrete latent structure models, and can be used to identify the latent structure. Mathemati- cally, the graphical implication of the tensor rank condition is as follows (Theorem 3.2).
3.2. General Results for Tensor Rank Condition
Here, we aim show that the tensor rank condition holds in the nonlinear causal model with discrete latent confounders under the completeness condition.
'The rank of a tensor T. denoted Rank(T), is the smallest number of rank-one tensors that generate as their sum, where N-way tensor is rank-one it be written as the outer product of vectors.
3

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
X8
common model is to adopt the pure measurement model, such as the linear case (Silva et al., 2006; Cai et al., 2019; Xie et al., 2020) and discrete case (Chen et al., 2024; Gu, 2022).
85
Based on the nonlinear causal model with discrete latent con- founders and combining the discrete latent structure model from (Chen et al.. 2024), in this paper, we further consider more challenging case where the observed variable can be discrete or continuous, and allow for multi-factor structure among observed variables. We formalize this as the discrete latent structure model with mixed-type data.
Xs Xa
X10 X12
Figure 2. Example of discrete latent structure model with mixed- type observational data, where X, represents continuous observed variables X, represents discrete observed variables (purple), and L; represents discrete latent variable. Moreover, the blue edge constructs the structure model C, the black edge constructs the measurement model M.
Definition 4.1 (Discrete I Latent Structure Model with Mixed- Type Observational Data). A nonlinear causal model with latent confounders and its corresponding causal graph G is a discrete latent structure model with mixed-type data (Mixed LSM) if it satisfies the following conditions:
Theorem 3.4 (Graphical Implication in Non-linear Causal Model). In the nonlinear causal model with discrete la- tent confounders, suppose the Markov condition, faithful- ness, non non-degeneracy condition, and the completeness con- dition hold. Consider an observed variable set Xp = {X1.....Xpl Xg C X is a continuous observed vari- able set (can be empty). for its corresponding p-way (condi- tional) probability tensor Ip(pp Xxy), denoted by Tix) then Rank (Tix,
(Purity Assumption) there are direct edges between the observed variables;
(Three-Pure Child Variable Assumption) each latent variable set Lp C L (at least exist one |Lpl = 1), in which every latent variable directly causes the same set of observed variables, has at least 2 2|Lp| + 1 pure variables? as children:
r and r < p. ifand only (i) there exists a variable set $ C V with supp(S)] = such that SUX,- = S. S d-separates any pair variables in X2 (con- ditional on X, X + 0), and (ii) does no exist conditional set $ UUX X. that satisfies supp(SS] < r.
(Two-Sufficient Measurement Assumption) each latent variable set Lp C L, in which every latent variable directly causes the same set of observed variables, has at least two sufficient measured variables', which can be either continuous variables or discrete variables with larger support than the latent parents.
The above theorem shows that in the nonlinear causal model with discrete latent confounders, given an observed variable set Xp. the rank of its (conditional) probability tensor corre- sponds to the cardinality of the discrete conditional variable set that d-separates all variables in Xp. When all observed variables are discrete, Theorem 3.2 is specific case of Theorem 3.4. Moreover, we allow continuous, nonlinearly dependent observed variable sets and further consider the constraints on the (conditional) probability tensor.
Note that the mixed LSM is different from the discrete latent variable model like (Chen et al., 2024; Gu, 2022), in which they assume all variables are discrete variables. The mixed-type setting more closely aligns with real-world scenarios, where the latent variable of interest is assessed using multiple indicators at varying measurement levels. For instance, a specific disease (unobserved, discrete variable) may be reflected by various physiological indicators, such as blood oxygen levels (continuous) and heart rate (discrete).
Remark 3.5. Although computing the tensor rank for con- tinuous, nonlinearly dependent observed variables is in- tractable due to the curse of dimensionality, we will show in Section 4 that a practical implementation to discretize the continuous variables, thereby making tensor rank com- putation feasible.
Discussions on the Assumptions. In general, to study the causal structure of latent variables, the purity and three- pure-children assumption are commonly used, even in linear models (Silva et al., 2006; Kummerfeld & Ramsey, 2016; Chen et al., 2022). When the latent variable is discrete, some works achieve identification under weaker assump- tions, such as the and the multi-parent assumption (Kivva et al., 2021; Kong et al., 2024). However, these approaches often involve spe-
4. Application in Discrete Latent Structure Models
In this section, our goal is to apply the tensor rank condi- tion (Theorem 3.4) to perform causal discovery with latent variables in the nonlinear causal model with discrete latent confounders. Without additional assumptions, however, it is hard to identify the causal structure among latent variable only from observed variables X (Silva et al., 2006; Kivva et al., 2021). Generally speaking, to address this issue, one
assumption (Gu, 2022)
Pure variables denote the variables that have only one latent parent, and no ohserved parents. FFor an observed variable X with support X and latent parent L. with support Z we define a sufficient measurement as | > [[].
4

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
cific structural constraints or complex estimation algorithms, hindering their practical application in real-world scenarios. It is important to emphasize that the purity and three-pune three-pure- children assumption are only sufficient conditions for identi- fying the causal structure of latent variables, which ensures the identification algorithm is simpler and more efficient. In fact, relaxing these assumptions to a more general case is feasible. Further discussion on this extension is provided in Appendix D.
Proposition 4.2. In the mixed LSM, suppose that the Markov condition, faithfidness assumption, and complete- ness condition hold, Then, there exists discretization X; of Xs, such that P(X, has full rank.
Proof of Sketch. In the mixed LSM, each observed vari- able is caused by their latent parents, i.e., PPX Pax.) P(X, L.). Under the completeness condition, [[g(())z;) O with 9(1) 0 implies that the linear independence of functions {SL.Ix. (I= where JL, can be seen as the PMF of conditional distribution P[Lg(X)) Therefore, one can further infer that {Fx.12, (fx))2(2;;))) (=;())] are also linear independent based on the Bayes theorem, where Fx, is the CDF of IT(( It ensures that the partition of P(X;) [.) is also linearly independent.
Under the mixed-type LSM, G decomposes as the union of two subgraphs G = .M U S, where . denotes the mea- surement model graph that represents the bipartite graph of edges pointing from L to X, and S represents the structure model graph that is a DAG over the latent variable L. Such definition can be referred to previous works (Chen al., 2024; Silva et al., 2006; Cai et al., 2019; Xie et al., 2020; Gu, 2022). See Figure 2 for example.
Intuitively, the completeness condition admits linear inde- pendence within the conditional probability space and main- tains this independence for any subspace of the probability space. This induces the possibility to find a discretization that satisfies the full-rank assumption.
Based on the graphical implication of the tensor rank con- dition, one can develop a statistically testable and robust algorithm to recover the latent structure, aligning with (Chen et al., 2024). However, the challenge hindering the appli- cation of the tensor rank is that for the probability tensor involving continuous variables, it is intractable to compute the tensor rank due to the curse of dimensionality. Thus, we first provide a practical method to test the tensor rank con- dition for the continuous probability tensor by employing proper discretization techniques.
Now, we aim to address the second issue: how to implement such a discretization. However, it is a non-trivial task to find such a discretization that satisfies the full-rank condition in the mixed LSM. The challenge lies in the fact that L of [(X|L) is a latent variable, which cannot be directly measured or tested.
Remark 4.3 (The challenging of discretization). Since one can only discretize the observed data X and P(X;[T.)) is unobserved, it is possible to obtain discretized data in which L(X;(L.) not full rank. When the full-rank condition does not hold in Theorem 3.4 (or, Theorem 3.2), the rank of the probability tensor may fail to uniquely represent the d-separation relations among the observed variables. There- fore, if continuous data is randomly discretized without careful considcration, the causal structure may be incor- rectly identified using the tensor rank condition (see detailed example in Appendix).
4.1. Estimation Tensor Rank from Continuous Observed Variables
To apply the tensor rank condition (Theorem 3.4) for con- tinuous observed variables, one practical way is to properly discretize continuous variables and then test the tensor rank condition on the discretized data. However, ensure the tensor rank condition holds in the discretized data, two key issues must be addressed:
1). whether there exists discretization of continuous data such that the tensor rank condition holds, and 2). how to implement such discretization.
Although the full-rank condition cannot be directly tested since L().)La) cannot be obtained from observational data, we have found that it can be indirectly assessed by exam- ining the non-negative rank of two discrete variables. This approach is motivated by the following observation.
Resolving the first issue guarantees that suitable discretiza- tion approach can be identified to apply the tensor rank condition for causal discovery, while the second issue fo- cuses on the operable implementation procedure. We begin by addressing the first issue.
Lemma 4.4 (Upper bound after discretization). In the mixed LSM and suppose Markov condition, faithfulness assumption and full-rank condition hold. Let X, and X, be the discretization of X, and X,, then we have Xa,,(P((X,,X)) S supp(LL), where L is a latent parent that d-separates from X.- Moreover, X; and X, satisfy the full-rank assumption if and only if the above inequality holds with equality.
For the discrete data, the tensor rank condition relies on the full-rank condition, as shown in Theorem 3.2. Therefore, to apply the tensor rank condition to discretized data (derived from continuous observed variables), the task is equivalent to finding a discretization such that (Px() ) is full rank in the mixed LSM. We first show its existence.
Lemma 4.4 shows that one can achieve the upper bound of the rank of probability tensor when the discretized variables
5

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
satisfy the full-rank condition. This motivates us to dis- cretize continuous variables and then check their rank con- straints over the joint contingency table, for achieving the
there exists discretized dataset X2 of X such that the
graphical implication of tensor rank condition holds.
full-rank condition. When the cardinality of latent support is unknown, one practical way is to use a rank-stoppinneresing technique, which can indirectly detect the full-rank condi- tion.
4.2. Structure Learning in Mixed LSMs
In this section, we aim to learn the causal structure of mixed latent structure models (LSMs) from mixed-type observa- tional data. One commonly used approach is the two-stage structure learning algorithm, as seen in (Silva et al., 2006; Chen et al., 2024). Specifically, we first learn the causal clus- ters among the observed variables to determine the latent variables (Step D, and then test the d-separation relations among the latent variables using their measured variables to recover the causal structure of the latent variables (Step II).
Theorem 4.5 (Ranks' stopped increasing to implement full rank). For continuous random variable X; and discrete variable X3, denote X; be the discretization of X4. Let ppi) (Xi, xx) be the k-th discretization of contingency tables P(X,,, Xz) that satisfies
1) (ank(E()) (x)) 2 (xx,
2) Max(Rank((())) (X,, < Min(I (iin[sspp(X))) (X:)l. |spp (Xz)), then there must exist a finite order l such that
4.2.1. STEP I: FINDING CAUSAL CLUSTERS
Without any constraints, the discrete latent variable is unidentifiable. When the observed variable set shares a common latent parent, for instance, one can always replace a pair of distinct latent variables L, and L, with a single la- tent variable L2 where |spp)(L*]] simplify the identification algorithm, we assume all latent variables have the same support and that at least one set of observed variables is caused by single latent parent (see Appendix for extensions). Next, we focus on identifying these latent variables.
Rank((p(k) (X Xz) = (ank(((((t))) (X,,
That is, X, and X, satisfy the full-rank assumption. Remark 4.6 (Practical implementations). We can discretize the continuous variable by choosing suitable cut- points (bin numbers) in practice. Generally speaking, a larger bin num- ber effectively controls discretization error and more easily meets the full-rank assumption. To achieve this, we follow the heuristics proposed by (Dougherty et al., 1995) to set the cut points. Moreover, we can construct a hypothesis test for the null-hypothesis that PP( P(,;, Xj) has the rank 2 r, following the approach of (Mazaheri et al., 2023). This rank test method is based on matrix perturbation theory. More details can be found in (Ratsimalahelo, 2001).
supp(L;, 42)]. To
Since each latent variable has certain observed children as its measured variables in the mixed LSM, one can identify these measured variables causal clusters to determine the existence of latent variables. To ensure that latent variables are identified correctly and without redundancy, two issues need to be addressed:
In other words, one can discretize the data multi-times (suf- ficiently large k), to find the discretized data that satisfies the full-rank condition. Consequently, the tensor rank con- dition can then be applied to the discrete data to learn causal structure. Moreover, we can show that the conditional inde- pendence (CI) relations in the discretized data imply the true corresponding d-separation relations in the causal graph.
(i). Finding all causal clusters from the observed variables and determining latent parents for each cluster. (ii). Merging the causal clusters that share common latent parents to avoid redundancy.
For the first issue, we first define the concepts of causal clusters and one-factor (multi-factor) structures.
Let Xa, be the discretized vari- able of (X,,... ,,,) correspondingly and satisfy the full- rank condition. Then under the Markov assumption, faith- fulness assumption, the CI relations among {X1,.... . imply the true d-separation in the causal graph G.
Proposition 4.7.
Definition 4.9 (Causal cluster). In the Mixed LSM model, C = (X1, ... .xp) is a causal cluster and only if all variables in (XI.-.. .,,) share the common latent parent.
..,)p)
Due to the multi-factor model setting, we further define the observed set with a multi-factor structure.
Based on Proposition 4.7, we can derive that the tensor rank condition and its graphical implications hold in the discretized data (i.e., data obtained from the discretization of continuous variables).
Definition 4.10 (One-factor & multi-factor cluster). For a causal cluster C. if VX; E C, X; has only one latent parent in the causal graph g, C, is a one-factor cluster (or one-factor structure if |C,| = 1). Otherwise, it a multi- factor cluster (or multi- factor structure if [C]] 1).
Lemma 4.8 (Tensor rank condition in mixted-type data). In the nonlinear causal model with latent confounders, sup- pose the Markov condition, faithfulness assumption, and non don-degnneracy assumption hold, then for the continuous variable set X, that satisfies the completeness condition,
To determine the number of latent parents in a causal cluster, it is crucial to first distinguish the facti-factor structure.
6

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Algorithm 1 Finding Causal Clusters
Example 4.14. Take the Fig .2 as an example. One can see that {X7, Xa), (Xa, Xs, Xa) and (X10, X12} are identified as one-factor clusters.
Require: Data set X {.I..... }
Ensure: The set of causal clusters C' and its corresponding
latent parent number set L.
Similarly, one can identify the causal cluster from X \ X to determine the multi-factor structure.
1: Initialize the causal cluster set C = 0;
2: Initialize the dictionary C=0, 0;
Proposition 4.15 (Multi-factor cluster). Ler (X,, X,, X&} C \ X {Xa, Xj, Xx} is multi-factor causal cluster that caused by n latent parents if E X (X,, Xx), (i) Rank(P(( (Xpp Xg)) = r", where Vp, q E ki,j,k, and
3: X < Discretize all continuous variables in X by ranks'
stop increasing criteria (Theorem 4.5);
4: X + one-factor group by Proposition 4.11;
5: for X, E Xe, and X5 E X; do
(ti) Rank(( X,, Xx, X.)) =r"
6: if Proposition 4.13 hold for {X,, X,.XAx then
7: C CU (X, Xj, Xz):
Once the causal cluster identified, one can determine the existence of latent variables. However, it cannot ensure that the latent variable is identified without redundancy, e-g., two causal clusters that share the same latent parent. Thus, it is necessary to merge the causal clusters that share a common latent into one causal cluster (the second issue).
8: C ((X, X : } for recording the num-
ber of the latent parent of this causal cluster;
9: end if
10: end for
11: r f- the cardinality of latent support by Proposition
4.13;
Proposition 4.16 (Merge Rule). Let C₁ and C2 be two causal clusters identified by Proposition 4.13 or Proposition 15, then C₁ and C2 share the common latent parent if one of the following conditions holds:
12: for each set C. X X do
causal cluster by Proposition 4.15; 14: C (C+ m} with m Rk),(X,)),
13: C
15:
end for
16:
C merge the causal clusters that share the common latent parent by Proposition 4.16;
RI. CinC2 C2 # 0. or
R2. One of them is multi-factor cluster, such as C2, and (ank( U C2)) = Rank
17: return C and L:
Now, one can identify all latent variables by their causal clusters. The procedure is summarized in Algorithm 1.
Although the cardinality of the latent support is unknown, this structure can be identified by leveraging its graphical properties, i.e., tensor rank condition.
4.2.2. STEP II: STRUCTURE LEARNING FOR LATENT VARIABLES
Proposition 4.11 (Distinguish multi-factoo, structure). For X; E X. X, has only one latent parent VX; E eX (X,), Rank(P(X;, Xj)) is invariant. Otherwise, X; is caused by more than one latent parent in discrete LSM.
In Step I, identify the causal clusters and determine the number of latent variables based on whether they belong to one-factor or multi-factor clusters. In this stage, we will show how the causal structure of latent variables can be identified up to a Markov equivalent class based on the identified clusters.
Based on Proposition 4. 4.11, one can divide the observed into two groups, one group the observed variable that has only one latent parent, and another is the observed variable that has a multi-factor structure. Now, one can identify the cardinality of latent support by identifying the rank of the contingency table of any two observed in the first group. We first define such a group as the one-factor group
To this end, we follow the constraint-based method by (Chen et al., 2024) and present the conditional independence test among latent variables as follows.
Theorem 4.17 (d-separation among latent varaible). In the mixed LSM model, suppose the Markov condition, faith- fulness assumption, full-rank condition, and completeness condition hold. Let r be the dimension of latent support, given the discretized data X. then L; L; if and only if Rank(Tix... X.,1..X Xx2)) = pop,, where X, and Xj are the pure children of L; and L5. Xp1 and Xy2 are two disjoint child set of Lp that satisfy (i) VX, E X UX p2. X; is a suf- ficient measured variable and, (ii) VL, E Ly Chy, OXp1 + 0, Cha, OX2z #
Definition 4.12 (One-factor group). Let X. S X denote a subset of observed variables, where each observed X; E X, has exactly one latent parent (one-factor structure).
One can identify the causal clusters that share only one latent parent, as shown in the following result.
Proposition 4.13 (One-factor cluster). Let (X,,X) X and X E X XX, X,}, {X,, Xx, is causal cluster if VX, E (X, Xz.Xx) XzJ. (PX,, Xj, Xj,X,, X.)) = r,
where r is the cardinality of latent support that can be identified by Rank (x).
In Theorem 4.17, a sufficient measured variable X, refers to a variable that is either continuous or a discrete variable

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
with larger support than its latent parents.
Latent Omission (4)
F1 Score (1)
0.5
Example 4.18. Consider the structure in Fig .2 Suppose that the cardinality of latent support is r. To test the CI relation Lg 1 Lal Lاا L2), let X, = (X
0.9
0.6
X2),
Case
0.3
one can test the rank of the probability tensor P(Xp), where X1 and X2 is discretized variables of X1 and X2. That is, (a))()(X,)) = 1² + L3 1 L
0.2
0.2
0.0
50k
100k
500k
50k
100k
500c
Based on Theorem 4.17, one can extend the PC-TENSOR- RANK algorithm (Chen et al., 2024) into the case of mixed observational data, by discretizing all continuous variables in X by ranks' stop increasing criteria (Theorem 4.5). Due to the space limitation, the complete algorithm is provided in Appendix. By this, we further give the identification result of mixed LSM.
0.8
0.6
0.6
Case II
0.4
0.4
0.2
0.3
0.2 56k
0.0
100k
Theorem 4.19 (Identification of Mixed LSM). In the mixed LSM, assuming the Markov condition, faithfulness assump- tion, full rank-rank condition, and completeness condition hold, if all latent variables have the same support and at least one set of observed variables is caused by a single latent purent, the causul structure of the latent variables can be identified up to a Markov equivalence class.
0.6
0.6
0.5
Case III
0.4
0.5
0.3
0.2
0.4
0.1
0.3
0.0
50k
1ook
500k
50k
100k
500k
number samples
number samples
5. Simulation Experiments
Mixed-LT
TS-PC
Ours
In this section, we conduct simulation experiments ver- ify the accuracy and effectiveness of our method. Baseline approaches include
Figure 3. Performance under different setups.
SITENSOR-RANE (abbreviated as TS-PC) (Chen et al., 2024), and mixed latent tree (abbrevi- ated as Mixed-LT) (Zhou et al., 2020). To ensure fairness, since TS-PC can only handle discrete data, we directly dis- cretize the continuous data using random cut points, and then apply TS-PC. For our algorithm, we test the matrix rank using the hypothesis test by (Mazaheri et al., 2023) and the tensor rank following (Chen al., 2024).
of latent variables in advance, according to different latent structures (e.g.. SM1), then (ii) we generate the conditional contingency table of discrete observed variables (condition on their latent parent) and each component of a Gaussian mixture model for continuous observed variables, and fi- nally (iii) we sample the observed data according to the probability contingency table and Gaussian mixture model, where the dimension of latent support r is set to 2 and the dimension of all discrete observed variables support is set to 3, sample size ranged from {5k, 10k, 50k)..
Setup. In the simulation studies, we consider the different combinations of various structure models and measurement models. Specifically, for the structure model, we consider the following three typical cases: Case I: L1 + L2; Case II: L1 + L2 - Lg; Case III: the structure of latent variables is shown in Fig. 2. For the measurement model, each latent variable has three pure observed variables, i.e., L;- {Xai41, X3442 X3143). We randomly choose 1/3 of the observed variables to be continuous variables.
Evaluation Metric. For each simulation study, we ran- domly generate the dataset and apply the proposed algorithm and baselines to these data. We use latent omission to as- sess the performance of causal clusters from each algorithm, which can be referred to (Silva et al., 2006). Moreover, to assess the ability of these algorithms to discover the causal structure among latent variables, we use the F1 score as the evaluation metric. Each experiment was repeated ten times with randomly generated data.
Generation Process. To ensure that the generated data sat- isfies the completeness condition, we use mixture model as a practical and effective way to simulate data. Although the data is generated from mixture model, the resulting joint distribution still reflects nonlinear dependencies (due to the probability property of the mixture model), and thus remains consistent with the setting described in Eq. (1). In all cases, the data generation process follows the following procedure: (i) we generate the probability contingency table
Results Analysis. The results are reported in Fig. 3. Our method consistently delivers the best outcomes across most scenarios, demonstrating its capability to identify both the causal clusters and the causal structures of latent variables with mixed data types. In contrast, the TS-PC approach per- forms poorly, as random discretization may violate the full- rank assumption (see Appendix B). Additionally, the Mixed-
8

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
LT algorithm shows suboptimal performance in structure learning of latent variables due to their limitations specific structural models, such tree structures, assumptions that latent variables are binary. More experimental results and discussions are provided in the Appendix.
is often limited in practice, as the tensor rank condition re- lies on certain assumptions such as the full-rank condition or completeness condition. Notable merits of our work in- clude establishing the connection between the tensor rank condition and the causal graph pattern, providing a practi- cal method for constructing discretized data, and applying the tensor rank condition to learn the causal structure from purely observational data.
6. Conclusion
In this paper, extend the tensor rank condition discrete latent structure models with mixed-type observational data. We demonstrate that, under the completeness condition, the tensor rank condition holds for discretized data, requiring only two sufficient measured observed variables. Based on this extension, we propose structure learning algorithm that first identifies causal clusters to determine the num- ber of latent variables and then infers the causal structure among them using their measured variables. This identifi- cation result extends the identification bounds of discrete latent structure models with multi- multi-factor structure. Future work includes allowing for impure observed structures and exploring more general non-linear models.
References
Allman, E., Matias, C., and Rhodes, J. Identifiability of parameters in latent structure models with many observed variables. Annals of Statistics, 37699)3009-3113,, 2009,
Anandkumar, A., Hsu, D., Huang, F., and Kakade, S. M. Learning high-dimensional mixtures of graphical models. arXiv preprint arXiv: 203,0697, 2012.
Anandkumar, A., Ge, R., Hsu, D. J., Kakade, S. M., Telgar- sky, M., al. Tensor decompositions for learning latent variable models. J. Mach. Learn. Res., 15()):2773-8322 2014.
Acknowledgements
Bartolucci, F., Pennoni, F, and Vittadini, G. A causal latent transition model with multivariate outcomes and unobserved heterogeneity: Application human capital development. Journal of Educational and Behavioral Statistics, 48(4):387-419 2023.
Feng Xie would like to acknowledge the support by the National Natural Science Foundation of China (62306019). RC would like to acknowledge the support by by National Key R&D Program of China (2021ZD01 1501), National Science Fund for Excellent Young Scholars (62122022), Natural Science Foundation of China (61876043, 61976052, 62476163) 163), the major key project of PCL (PCL2002A12).. ZF would like to acknowledge the support by the National Natural Science Foundation of China under grants No. 62476163 and U24A20233, and the Guangdong Basic and Applied Basic Research Foundation under grant number 2023B15151 120020. KZ would like to acknowledge the support from NSF Award No. 2229881, AI Institute for Societal Decision Making (AI-SDM), the National Insti- tutes of Health (NIH) under Contract RO1 HL199805, and grants from Quris AI, Florin Court Capital, and MBZUAI- WIS Joint Program. We would like to thank the anonymous reviewers for their helpful comments.
Bollen, K. A. Structural Equations with Latent Variable.
John Wiley & Sons, 1989.
Brundy, J. M. and Jorgenson, D. W. Efficient estimation of simultancous equations by instrumental variables. The Review of Economics and Statistics, pp. 207-224, 1971.
Cai, R., Xie, F., Glymour, C., Hao, Z., and Zhang, K. Triad constraints for learning causal structure of latent variables. In NeurIPS, pp. 12863- 12872, 2019.
Chen, W., Cai, R., Zhang, K., and Hao, Z. Causal discovery in linear non-gaussian acyclic model with multiple latent confounders. IEEE Transactions on Neural Networks and Learning Systems, 33(():2816-288) 2021a.
Impact Statement
Chen, W., Cai, R., Zhang, K., and Hao, Z. Causal discovery in linear non-gaussian acyclic model with multiple latent confounders. IEEE Transactions on Neural Networks and Learning Systems, 2021b.
Learning causal structure among latent variables is essen- tial throughout the data-driven sciences and has attracted much attention. Our research focuses on learning causal structure among discrete latent variables only from their measured variables, which can be continuous variables or discrete variables. This is typically encountered in fields like social sciences, economics, public health, and neuro- science. We assess the impact of our work in the context of these fields. However, the applicability of existing methods
Chen, W., Zhang, K., Cai, R., Huang, B., Ramsey, J., Hao, Z., and Glymour, C. Fritl: A hybrid method for causal discovery in the presence of latent confounders. arXiv preprint arXiv:2103.0 14238, 2021c
Chen, Z., Xie, F., Qiao, J., Hao, Z., Zhang, K., and Cai, R. Identification of linear latent variable model with arbitrary

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
distribution. In Proceedings 36th AAAI Conference on Artificial Intelligence (AAAT), 2022.
Hoyer, O., Shimizu, S., Kerminen, A. J., and Palviainen, M. Estimation of causal effects using linear non-gaussian causal models with hidden variables. IAR, 49(2):362- 378, 2008.
Chen, Z., Xie, F,, Qiao, J., Hao, 7.,, and Cai, R. Some general identification results for linear latent hierarchical causal structure. In Proceedings of the Thirty-Second Second International Joint Conference on Artificial Intelligence, pp. 3568-33633 -1666, 2023.
Hu, Y. and Shiu, J.-L. Nonparametric identification using instrumental variables: sufficient conditions for complete- ness. Econometric Theory, 34(3):659-06 2018.
Chen, Z., Cai, R., Xie, F., Qiao, J., Wu, A., Li, Z.,, Hao, Z., and Zhang, K. Learning discrete latent variable struc- tures with tensor rank conditions. In Advances in Nearul Information Processing Systems, 2024.
Huang. B., Low, H., Xie, F., Glymour, C., and Zhang, K. Latent hierarchical causal structure discovery with rank constraints. In Advances in Neural Information Prucessing Systems, 2022.
Choi, M. J., Tan, V. Y., Anandkumar, A., and Willsky, A. S. Learning latent tree graphical models. Journal of Machine Learning Research, 12(49):177--1112 2011.
Hyvarinen, A. and Morioka, H. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics, pp. 460-469. PMLR, 2017.
Chung. H., Loken, E.,, and Schafer, J. L. Difficulties in drawing inferences with finite-mixture models: a simple example with a simple solution. The American Statisti- cian, 58(2):152-158, 2004.
Hyvarinen, A., Sasaki, H., and Turner, R. Nonlinear ica us- ing auxiliary variables and generalized contrastive learn- ing. In The 22nd International Conference on Artificial Intelligence and Statistics, pp.
PMLR, 2019.
Colombo, D., Maathuis, M. H., Kalisch, M., and Richard- son, T. S. Learning nigh-dimensional directed acyclic graphs with latent and selection variables. The Annals of 2012.
Jin, S., Xie, F., Chen, G., Huang, B., Chen, Z., Dong, X., and Zhang, K. Structural estimation of partially observed linear non-gaussian acyclic model: A practical approach with identifiability. In The Twelfth International Confer- ence on Learning Representations, 2023.
Statistics, pp.
Cui, R., Groot, P., Schauer, M., and Heskes, T. Learning the causal structure of copula models with latent variables. In Proceedings of the Thirty- Fourth Conference on Uncer- tainty in Artificial Intelligence, UAI 2018, pp. 188-197. AUAI Press, 2018.
Kargas, N. and Sidiropoulos, N. D. Learning mixtures of smooth product distributions: Identifiability and algo- rithm. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 388-396. PMLR, 2019.
Cui, Y., Pu, H., Shi, X., Miao, W., and Tchetgen Tchetgen, E. Semiparametric proximal causal inference. Journal of the American Statistical Association, pp. 1-12, 2023.
Kasza, J., Wolfe, R., and Schuster, T. Assessing the im- pact of unmeasured confounding for binary outcomes using confounding functions. International journal of epidemiology,
Dougherty, J., Kohavi, R., and Sahami, M. Supervised and unsupervised discretization of continuous features. In Ma- chine learning proceedings 1995, pp. 194-202. Elsevier, 1995.
2017.
Kim, D. and Lindsay, B. G. Empirical identifiability in finite mixture models. Annals of the Institute of Statistical Mathematics, 67:745-772, 2015.
Dziadkowiec, O. The role of confounders in statistical analysis. Journal of Obstetric, Gynecologic & Neonatal Nursing, 52(5):335-335 2023.
Kivva, B., Rajendran, G., Ravikumar, P., and Aragam, B. Learning latent causal graphs via mixture oracles. Ad- vances in Neurul Information Processing Systems, 34, 2021.
Gu, Y. Blessing of dependence: Identifiability and geometry of discrete models with multiple binary latent variables. arXiv preprint arXiv:2203. 04403,, 2022.
Kong, L.,, Chen, G., Huang, B., Xing, E. P., Chi, Y., and Zhang, K. Learning discrete concepts in latent hierarchi- cal models, arXiv preprint arXiv:2406.00519, 2024.
Gu, and Dunson, D. B. Bayesian pyramids: Identifiable multilayer discrete latent structure models for discrete data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(2):399-425 2023.
Kruskal, J. B. Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear algebra and its applica- tions, 18(2):95-338 1977.
Hackbusch, W. Tensor spaces and numerical tensor calcu- lus, volume 42. Springer, 2012.
10

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Kummerfeld, E. and Ramsey, J. Causal clustering for 1- factor measurement models. In KDD, pp. 1655-1664. ACM, 2016.
Myers, A., Rassen, J. A., Gagne, J., Huybrechts, K. F. Schneeweiss, S., Rothman, K. Joffe, M. M., and Glynn, R. J. Effects of adjusting for instrumental variables on bias and precision of effect estimates, American journal of epidemiology, 174(1 12:13322222222, 2011.
Kummerfeld, E., Ramsey, J., Yang, R., Spirtes, P., and Scheines, R. Causal clustering for 2-factor measurement models. In Proceedings of the 2014th European Confer- ence on Machine Learning and Knowledge Discovery in Databases- Volume Part II, pp. 34-49, 2014.
Newey, W. K. and Powell, J. L. Instrumental variable esti- mation of nonparametric models. Econometrica, 71(5): 1565-1578, 2003.
Li, C., Shen, X., and Pan, W. Nonlinear causal discovery with confounders. Journal of the American Statistical Association, 1196546):105--13
Ogarrio, M., Spirtes, P., and Ramsey, A hybrid causal search algorithm for latent variable models. Confer- ence on probabilistic graphical models, pp. 368-379, PMLR, 2016.
2024.
Lindsay, B. G. and Roeder, K. Uniqueness of estimation and identifiability in mixture models. Canadian Journal of Statistics, 21(2):199-147, 1993.
Ratsimalahelo, Z. Rank test based on matrix perturbation theory. Technical report, EERI Research Paper Series, 2001.
Liu, M.. Sun, X., Yu, Q. and Wang, Y. Causal discovery via conditional independence testing with proxy variables. In Forty-first International Conference on Machine Learn- ing. 2024.
Ritchie, A., Vandermeulen, R. A., and Scott, C. Consistent estimation of identifiable nonparametric mixture models from grouped observations. Advances in Neural Informa- tion Processing Systems, 33:1 1111-16668880 2020.
Liu, X. and Wang, L. The impact of measurement error and omitting confounders statistical inference of mediation effects and tools for sensitivity analysis. Psychological Methods, 26(3):327, 2021.
Rudolph, K. E., Williams, N., and Díaz, I. Using instru- mental variables to address unmeasured confounding in causal mediation analysis. Biometrics, 80(1 (0(1))))) 1:ujadd3377 2024.
Lovitz, B. and Petrov, F. A generalization of kruskal's theo- rem on tensor decomposition. In Forum of Mathematics, Sigma, volume 11, pp. e27. Cambridge University Press, 2023.
Salehkaleybar, S., Ghassami, A., Kiyavash, N., and Zhang, K. Learning linear non- non-gaussian causal models in the presence of latent variables. JMLR, 21(39); 2020.
Mazaheri, B., Gordon, S., Rabani, Y., and Schulman, L. Causal discovery under latent class confounding. arXiv preprint arXiv:2311.0745.. 2023.
Shi, X., Miao, W., Nelson, J. C., and Tchetgen Tchet- gen, E. J. Multiply robust causal inference with double- negative control adjustment for categorical unmeasured confounding. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(2):521-540, 2020.
McLachlan, G. J. and Peel, D. Finite mixture models, vol- 299. John Wiley& & Sons, 2000.
Meek, C. Causal inference and causal explanation with background knowledge. In UAI, pp. 403-410, 1995.
Silva, R., Scheine, R., Glymour, C., and Spirtes, P. Learning the structure of linear latent variable models. JMLR, (Feb):191-246, 2006.
Mena, R. H. and Walker, S. G. On the bayesian mixture model and identifiability. Journal of Computational and Graphical Statistics, 24(4): 1169, 2015.
Spirtes, P., Glymour, C., and Scheines, R. Causation, Pre- diction, and Search. MIT press, 2000.
Miao, W., Geng, Z., and Tchetgen Tchetgen, E. J. Identify- ing causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4):987-993, 2018.
Tahmasebi, B., Motahari, S. A., and Maddah-Ali, M. A. On the identifiability of finite mixtures of finite product measures. arXiv preprint arXiv: 80740044.444.. 2018.
Miao, W., Shi, X., Li, Y., and Tchetgen Tchetgen, E. J. A confounding bridge approach for double negative control inference on causal effects. Statistical Theory and Related Fields, pp. 1-12, 2024.
Tchetgen Tchetgen, E. The control outcome calibration approach for causal inference with unobserved confound- ing, American journal of epidemiology. 17965):633-640 2014.
Mourad, R., Sinoquet, C., Zhang, N. L., Liu, T., and Leray, A survey on latent tree models and applications. Journal of Artificial Intelligence Research, 47(1):157-203, 2013.
Teicher, H. Identifiability of mixtures product measures. The Annals of Mathematical Statistics, 38(4):1300- 1967.

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Thaden, H. and Kneib, T. Structural equation models for dealing with spatial confounding. The American Statisti- cian, 72(3):239-252, 2018.
Zheng, J., Wu, J., D' D'Amour, A., and Franks, A. Sensi- tivity to unobserved confounding in studies with factor- structured outcomes. Journal of the American Statistical Association, pp. 1-12, 2023.
Tofighi, D., Hsiao, Y.-Y., Kruger, E. S., MacKinnon, D. P., Lee Van Horn, M., and Witkiewitz, Sensitivity analysis of the no-omitted confounder assumption in latent growth curve mediation models. Structural equation modeling: mulridisciplinary journal, 26(1):94-109 2019.
Zheng, Y. Ng. and Zhang, K. On the identifiability of nonlinear ica: Sparsity and beyond. Advances in neural information processing systems, 35:16411-16422, 2022.
Zhou, C., Wang, X., and Guo, Learning mixed latent tree models. Journal of Machine Learning Research, 21(249): 1-35, 2020.
Valente, M. J., Pelham III W. E., Smyth, H., and MacKin- non, D. P. Confounding in statistical mediation analysis: What it is and how address it. Journal of counseling psychology, 64(6))659, 2017.
Viberg, M., Wahlberg, B., and Ottersten, Analysis of state space system identification methods based on instru- mental variables and subspace fitting. Automatica, 33(9): 1603-1616, 1997.
Wang, Y. and Blei, D. M. The blessings of multiple causes. Journal of the American Statistical Association, 114(528): 2019.
xian Wang, H., Luo, B., bing Zhang, Q.. and Wei, S. Esti- mation for the number of components a mixture model using stepwise split-and merge em algorithm. Pattern Recognition Letters, (091))99-9800000 2004.
Xie, F., Cai, R., Huang, B., Glymour, C., Hao, Z., and Zhang, K. Generalized independent noise conditionfor estimating latent variable causal graphs. In NeurlPS. pp. 14891-14902, 2020.
Xie, F., He, Y., Geng, 7., Chen, 7.,, Hou, R., and Zhang, K. Testability of instrumental variables linear non- gaussian acyclic causal models. Entropy, 24(4):512, 2022a.
Xie, F., Huang, B., Chen, Z.,, He, Y., Geng, Z., and Zhang, K. Identification linear non-gaussian latent hicrarchi- cal structure. In International Conference on Machine Learning, pp.
PMLR, 2022b.
Xie, F., Huang, B., Chen, Z., Cai, R., Glymour, C., Geng, Z., and Zhang, K. Generalized independent noise condi- tion for estimating causal structure with latent variables. Journal of Machine Learning Research, 25:1 61, 2024.
Yang, B., Fu, X., Sidiropoulos, N. D., and Huang, K. Learn- ing nonlinear mixtures: Identifiability and algorithm. IEEE Transactions on Signal Processing, 68:2857-2869, 2020.
Ying, A., Miao, W., Shi, X., and Tchetgen Tchetgen, E. J. Proximal causal inference for complex longitudinal stud- ies. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(3):884-044 2023.
12

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
A. Related Works
Latent Confounders. Unobserved confounding has long been a significant challenge for reliably drawing causal inferences and conducting statistical analyses from observational data, In the causal inference, most methods focus on addressing the problem of effect estimation in the presence of latent confounders, such as instrumental variables-based approaches (Brundy & Jorgenson, 1971; Viberg et al., 1997; Myers et al., 2011; Xie et al., 2022a; Rudolph et al., 2024), the control outcome-bass based methods (Tchetgen Tchetgen, 2014; Shi et al., 2020), confounding functions-based ones (Kasza et al., 2017; Miao et al., 2024). There are also some statistical models that account for latent confounders, such as those proposed in (Tofighi et al., 2019; Valente et al., 2017; Dziadkowiec, 2023; Liu & Wang, 2021). Most of these methods focus on the problem of estimation, rather than addressing the challenge of structure learning.
Mixture Model. Mixture models are closely related to our problem setup. Traditional methods focus on the parameter
identifiability of mixture models, as seen in (Lindsay & Roeder, 1993; McLachlan & Peel, 2000; Allman et al., 2009; Anandkumar et al., 2014; Kim & Lindsay, 2015; Mena & Walker, 2015; Yang et al., 2020; Tahmasebi et al., 2018; Kargas & Sidiropoulos, 2019). To learn the causal structure in the presence of latent confounders, (Mazaheri et al., 2023; Anandkumar et al., 2012) present a rank-based approach to infer the d-separation relations among observed variables, leveraging the identifiability of mixture models. For learning the structure of latent variables, (Kivva et al., 2021) propose a mixture oracle-based method to recover the distribution of latent variables and then perform causal discovery among them. However, the identification results depend on the existence of a mixture oracle, and it remains unclear how such an oracle can be identified.
Latent Variable Model. Latent variable graphical models have been well studied in the literature. Most work focuses on linear models, such as the linear latent variable model with Gaussian noise assumption (Silva et al., 2006; Kummerfeld & Ramsey, 2016; Kummerfeld et al., 2014; Huang et al., 2022), son-Gaussian linear latent structure models (Cai et al., 2019; Xie et al., 2020; 2022b; Chen et al., 2023; 2022; Jin et al., 2023), and copula model-based approaches (Cui et al., 2018). Less is known regarding structure learning between latent variables for nonlinear models. One typical method is based on non-linear ICA, such as (Zheng et al., 2022; Hyvarinen & Morioka, 2017: Hyvarinen al., 2019). In the discrete model domain, there are few methods, including the latent tree model (Choi et al., 201 1; Mourad et al., 2013) and pure measurement model (Gu, 2022; Gu & Dunson, 2023; Chen et al., 2024). However, these methods focus on restricted latent structures or linear assumptions and cannot handle discrete latent variable structures with mixed-type observational data.
B. More Details on the Motivating Example for Discretization
In this section, we discuss two key questions: (i) why it is necessary to design a method for discretization, and (ii) the detailed implementation of the discretization process. The first part highlights that without a careful discretization strategy, the tensor rank condition may fail to hold in the discretized data. The second part provides an illustrative example to show this procedure in practice.
We begin by discussing the first question. When we learn the causal structure using the tensor rank condition, a key assumption for applying the tensor rank condition is the full-rank assumption (Chen et al., 2024), which stipulates that the conditional probability table must have full rank. However, when continuous data are discretized arbitrarily, this assumption may be violated. potentially undermining the validity of the tensor rank condition. For example, in Fig. .4, one can see that there exists discretization of X, such that IP(X..T.) is not full rank.
In this example, we simulate a simple structure L + X, where X; is continuous variable and L is a binary variable. Our goal is to discretize X; into a discrete variable X, with support {0, 1, 3,2. Thus, To achieve this, we randomly select three cut points { n, 3r 2 }. on the marginal distribution of X, (i.e., the pink area). When examining the conditional distributions P(X,|L== = 0) and IP(X,LL~ = 1). represented by the blue and green areas, respectively, we observe that the conditional probability table is not full rank. This is because the blue and green areas are identical in each partition, thereby violating the full-rank condition. Therefore, if we discretize the continuous variable arbitrarily, the structure learned by the tensor rank condition may be incorrect. This is the reason why we need to present the ranks stopped increasing criteria (Theorem 4.5) during the discretization process.
Remark B.I. In (Kargas & Sidiropoulos, 2019), it is demonstrated that the identifiability of mixture distributions holds even after discretization, and a practical algorithm is provided to learn the conditional probability by tensor decomposition (actually, the tensor rank) for discretized data. Based on their algorithm, one can efficiently estimate the tensor rank of the
13

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
discretized probability tensor.
Next, we provide additional implementation details for the discretization methodology. Specifically, we describe how to discretize the data such that the full-rank condition holds. We generate data using the following directed acyclic graph (DAG): L X. - Y, L + Z, where X and Y are continuous variables, and Z binary discrete variable. The latent variable has the support {0, 1). The marginal distribution of X and Y is mixture of the Laplace distribution (Figure (a) and (c)). Through proper discretization of X and Y into X and Y (Figure (b) and (d)), respectively, for example, selecting cutpoints 2.06 and 0.26 for X, and cutpoints -0.94 and 0.83 for Y, we discretize the two continuous variables into two discrete variables X and Y. as shown in Figure 5 (b) and (d). In this procedure, we maximize Rank (P Y)), ensuring that L) and, hence, IPYY|L) satisfy the full-rank assumption. For instance, the joint distribution P(X, Y, Z = i) with i = 0, shown in Figure 6. We discretize this distribution into a discrete joint probability table, as shown in Figure which is rank-2 tensor, indicating that the conditional probability table is full rank. Based on these results, one can use X, Y, and Z compute the lensor rank of the probabilistic contingency table and recover the support of L.
1- paaas 2)-
P(8-3)- be
1idds
Marginal Distribution of POO
1.0
PXXL PXLL
0.8
P())
Figure 4. Example for discretized data, where the latent parent L. of X; has (T.) {0, 1}. There are three cut point { 3r that
2
lead to P(X;) with support d; = {00, 2, 3). One can see that P(X,[L) is not full rank because the green and blue areas are equal in each partition.
Probabiitt Dersity Function
Probabilty Density Function
0.50
2.00
0.35
0.30
0.4
0.25
0.3
0.20
6.6
0:5
-2
Stacretaasinn valus
(a) Marginal distribution of X.
Discretization X.
(c) Marginal distribution of Y.
(d) Discrctization of Y.
Figure 5. Exanple for illustrating the procedure of discretization. For marginal distribution X and Y (i.e., subfig (a) and subfig (c)), we discretize them into two discretized variables (i.c., subfig (b) and subfig (d)).
C. More Details on PC-TENSOR RANK Algorithm
The specific details of the PC-TENSOR RANK algorithm are provided below.
14

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
p(X, YKK,
Y=0
==1
Y
X=0
0.170
0.169
0.001
Ronc:
X=1
0.116
0.133
0.081
x=2
0.116
0.133
0.081
16
p(XYY, N, 1)
Y 0
Y
Y
x=0
0.145
0.142
0.001
X=1
0.114
0.137
0.105
Planci 2
x=
0.114
0.137
0.105
Figure 7. Probabilistic contingency table of P(X,Y, 2 with i 0, One can see that Rank(P(X, Y, = -0) = 2 and
Rank(P(X, y, Z 1)) indicating L (with |supp( 2) d-separates X, Y and Z.
X. 1ril
Figure 6. Joint distribution of P(X, i) with i 0,
Algorithm 2 SITENSOR-RANC
Input: The discretized data set X [X)..... Xm) and causal cluster C
Output: A partial DAG G.
Initialize the maximal conditions set dimension k;
2: Let L, denote as C. C, E C:
3: Form the complete undirected graph on the latent variable set L;
4: for VL4, Lj L and adjacent in G do
5: //lest the CI relations among latent variables by Theorem 4.17
6: if WLp L {L; L2} and (Lpl < k) such that Le JL Lp hold then
7: delete edge L, - L. from G;
8: end if
9: end for
Search V structures and apply meek rules (Meek, 1995).
11: return partial DAG G of latent variables.
D. Analysis the Identification of Discrete Latent Variable Models under Sparsity Condition
We now extend the results of the previous section when there are some direct edges between the observed variable (impure structure) in contrast to the three-pure children condition. Our goal is to explore under what milder conditions the discrete latent variable model remains identifiable. Under more general sparsity assumption, we provide the identification results of the discrete latent variable model with the discrete observed variables. For continuous observed variables, the identification result can be easily extended by appropriately estimating the conditional distribution.
In the following, aim to relax the purity assumption and the three-pure children assumption in the discrete latent variable model. The alternative identification condition is the sparsity assumption.
Assumption D.1 (Sparsity assumption). each latent variable set Lp C L, in which every latent variable directly causes the same set of observed variables. has at least three observed children variable X,, XY, Xg E such that (i) (X,, X, Xx 1 X (X,, X,, Xx) Lp, and (ii) there are X. C X (X,, X,, X&), X, 1 X, X& (L) (L,, X,).
Example D.2. For example, for the L3 in Fig .8, one have X13, X15 and X, satisfy the spasity assumption. Since given X14+ we have X13 X15 L X² (Xu, L3). By the tensor rank condition, the joint distribution P(X13. X1s, X²(X11) follows the graphical implication.
15

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Under the sparsity assumption, we allow edges between observed variables and do not impose the three pure children structure constraints. This leads to a more general identification result for discrete latent variable models. We define such general discrete latent variable model as the Sparse Discrete Latent Variable Model.
Definition D.3 (Sparse Discrete Latent Variable Model). A nonlinear causal model with latent confounders and its corresponding causal graph 9 is Sparse Discrete Latent Variable Model (Discrete Sparse-LSM) if all observed variables are discrete and satisfy the following conditions:
(Sparsity Assumption) each latent variable set L, C. L, in which every latent variable directly causes the same set of observed variables, satisfy the sparsity assumption (Assumption D.1);
(Two-Sufficient Measurement Assumption) each latent variable set Lp C L, in which every latent variable directly causes the same set of observed variables, has at least two sufficient measured variables, i.e., larger support than their latent parents.
X
8
X
X2
X4
X18
87
110 X11 X12
Figure& 8. A generalized discrete latent structure in the presence of mixed-type observed variables, where X; represents continuous observed variables X, represents discrete observed variables (purple), and L, represents a discrete latent variable.
To ensure the cardinality of latent support is identifiable, we require the following assumption. Assumption D.4. All latent variable has the same support, and there exists at least one observed variable set X, with |X,| 3, X, has only one latent parent.
Remark D.5 (Discussion on assumption D.4). Assumption D.4 is only sufficient condition for identifying the cardinality of the latent support. In (Mazaheri et al., 2023), the cardinality of the latent support is assumed to be known, while in (Kivva et al., 2021), the Subset condition is assumed. The Subset condition requires that the neighborhoods of two distinct latent variables do not overlap, thereby ensuring the identification of the components in the mixture model. Actually, it is possible to relax the Assumption D 9 to a more general case where the support of latent variable can be different, by using the minimal state space criteria (Chen et al., 2024).
In the following, we develop identification algorithm that identifies the latent causal structure under the Discrete Spare- LSM. We still follow the strategy that first determines the latent variable by causal cluster, and then infer the causal structure based on these clusters. Since there are edges between observed variables (called impure structure), it is necessary to find these impure structures to ensure the causal cluster can be identified correctly. In the discrete sparse-LSMM, the impure structure can be identified by performing the conditional independent test in the presence of a latent confounder.
D.1. Conditional Independent Test in the presence of Latent Confounders
We begin with an illustrative example that demonstrates the connection between the tensor rank condition and the d- separation relations among observed variables in an impure structure. Take the structures in Fig .9 as an example, one can see that, X, 1 X3l ( (X2, } hold for two structures. Suppose X1, X2, X3 has the same support {0, 1, 2} and the latent variable L1, Lg has the same support (0, 1}, let PXx, X3 X2 c) be the conditional probability contingency table, denote Pole,t PP(X,, j X2 c. In i), pyJe, P(X3 j|X2 c, Jn i), and Pije P(I, i X c), under the Markov assumption, faithfulness assumption, one can that
Pole,o Pule,1
Poje
Pale.o Pil P2ic. Pole,1
F(X,, = =) =
Pile.0
P1le,1
(3)
Pile
Pile,1 P2ie.1
P2j0.0
P2
Ding (P Xa=e))
16

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
X1
X3
X1
Figure 9. Example of the impure structure that can be identified by tensor rank condition.
Under the non-degenerate assumption (i.e., the conditional contingency table is full rank and non-zero elements in the diagonal matrix), one can see that (ank( Xa|X2 =c)) = 2. This implies that when considering the conditional distribution of 11 and X3 given the observed variable set X2 (which d-separates X1 from X3 by combining with the latent variable 11), , the rank of the conditional tensor is determined by the latent parent L1. Thus, one can test the conditional independence relations among observed variables by identifying these properties, which can help detect the impure structure among the measured variables.
Theorem D.6 (CI relations among observed variables). In the Discrete Sparse-LSM, suppose the Markov assumption, faithfulness condition, and the non-degenerate condition hold. Let r denote the cardinality of latent support, X; 1 p, p. L} hold if and only for the conditional probability tensor X x,) =
Proof. Observe that any two observed variables (if they are purely measured variables) are separated by one of their latent parents in the discrete sparse-I.M model. This result can be directly proven by combining the non-degenerate condition with the findings from (Anandkumar et al., 2012).
By theorem D.6, one can find the general pure child set for each latent variable by designing a proper search algorithm.
Lemma D.7 (Stop decreasing for probability matrix). For any pair of observed variables X, and Xj, let Li and L, be their latent parent. If there is no X, such that Rank(P(X,, Xp)) has a lower rank than any Rank((((X, Xx,),)) = r, then min() (oo((((ppPP()))) supp((((L,)) =r.
X
Proof. The proof is straightforward. Suppose supp(P((L)) supp(P(L;)) based on Theorem D.6, if find the conditional set X, in which {L, X,) is the conditional set that d-separates any pair variable in Xp. then Rank(P(X,, Xj (x,))
under the Markov assumption, faithfulness assumption, and the non-degenerate condition. Oth- erwise, Rank(P(X. Xj(X,)) > r according to the graphical implication of tensor rank condition. Therefore, if we find that Rank(P(X; X5 (X,)) is minimal (do not exist X,, such that the conditional probability tensor has lower rank), then {T,, X,) is the conditional set with the minimal cardinality of latent support, i.c., min(
supp( = =r.
Based on Lemma D.7, all d-separations among observed variables can be identified using an appropriate search algorithm. such (Anandkumar et al., 2012; Mazaheri al., 2023). Moreover, existing methods can identify whether an observed variable set is caused by latent confounder, such (Silva et al., 2006; Chen et al., 2021a). Thus, observed variable set in which each observed variable has at least one latent parent. We omit the specific algorithm and assume that all d-separation relations between any pair of measured variables X, X, are recorded in a set denoted as dset. For instance, (X,, Xj) X, implies that there exist Ih, C L, such that L, U X, d-separates X, from Xj.
determine the
D.2. Identifying the Causal Structure of Discrete Sparse-
Based on the fact that all d-separation relations among observed variables are identifiable, we can infer the identifiability of the discrete sparse latent variable model. Since its identification algorithm is hybrid and lacks elegance (e.g., hybrid algorithm from the modified PC-algorithm (Mazaheri et al., 2023), the FINDPATTERN algorithm (Silva et al., 2006) and the discrete latent structure learning algorithm (Chen et al., 2024)), we primarily focus on discussing its identifiability result.
Theorem D.8 (Identification of discrete sparse-LSMs), Given an unbiased estimation of the conditional distribution of any observed variables, in the discrete sparse latent structure model, suppose the Markov assumption, faithfulness condition. non don-degeee condition, and assumption D.4 hold. The causal structure of the latent variable can be identified up to a Markov equivalence class.
17

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Proof. To complete this proof, there are two stages of identification that need to be discussed: the causal cluster that determines the latent variable and the structure model among latent variables.
Before providing the proof, we first demonstrate that all observed variable sets with at least one latent parent can be identified using the following corollary.
Corollary D.9. For X; and x if one of them does not have a latent parent, then there exists a subset X, S X X,b such that (P xx,,,, x,))
Since one of X; and Xj does not have a latent parent, then X; and X, are d-separated by the observed variable set. According to Remark G. in (Chen et al., 2024), this result is proven. Based on this corollary, all observed variables that do not have a latent parent can be removed. Now, we can continue the proof of discrete sparse-LSM.
Stage I: Identify the cardinality of latent support. Based on Lemma D.7 and Assumption D.4, one can identify the cardinality of latent support, i.e., identifying the minimal rank for the (conditional) probability table over two observed variables that are caused by a single latent parent. We can iterate over all pairs of variables and identify the cardinality of latent support by finding the minimum rank constraint of the probability contingency table.
Stage II: Determine the latent variable. Under the sparsity assumption and the result of Theorem D.6, one can see that, {X,, x,, } is a causal cluster, if (an (x)) = with X, = (X,, x,, Xx. X,} for any X, E X \ {X,, X.), and x, U Li (L; is the latent parent of them) is the conditional set that d-separates any pair variable in (Xi, Xj. Xa). where X, _x = 0. The reason is as follows.
First, by Theorem D.6, one can get all conditional independent relations between any pair of observed variables (Mazaheri al., 2023; Anandkumar et al., 2012). We record these d-separation sets for each pair of observed variables X, and Xj- denoted by dset(X; X;) = Xp where X, 1 (,,,,,,,,3 Xp). L, is one of the parent variables of X; or Xy. For observed variable set (X,, X,. X&} and any X, E X \\  {X,, X,, Xx). let X, U..se dset(X;, Xy), i.e., the combination of the d-separation set for any pair variable in Xp. Therefore, if {X, Xj. X&) share the one latent parent variable set, denoted by L,, then one can see that {Le} U UX, is conditional set that d-separates any pair variable in Xp. Similar to the proof of (Chen et al., 2024), one can see that } U X,, is the minimal conditional set with support Thus, Rank(P(X,
Second, by repeating the above procedure, the causal cluster can be identified. Moreover, since the cardinality of the latent support is identifiable, let r represent the cardinality of the latent support. Without loss of generality, the number of latent parents can be determined as fr, where nr is the rank of probability tensor P(X,, X,, Xs, X.(,).
Third, we aim show that the causal cluster can be further merged if they share common latent parent, to avoid the redundant introduction of latent variables. Under the sparsity assumption and the result of d-separation for any pair observed variable, if for two causal clusters C₁ and C2. C₁ n * 0, then C₁ and C2 share the common latent parent, according to the result of (Chen et al., 2024) (Proposition 4.5), Moreover, one can modify the result of Proposition 4.16 to check more merging rules. Let X, be the conditional set for any pair variable in C₁ and CN, by constructing the conditional probability tensor, Proposition 4. 16 holds. By detecting the causal clusters that share a common latent parent and merging them into one causal cluster, the number of latent variables is identifiable.
Stage III: Identify the causal structure among latent variables, given the causal cluster for each latent variable. In stage II, we have recorded the d-separation set for any pair of observed variables. Thus, one construct the conditional probability tensor (X)) for any set X, C X such that any pair variable X,, X5 E X, p are d-separated by (X, U T.), T, E L. This means that, one can find that L, C L such that L, UX d-separates any pair variable in X, and there do not exist L, with (P < (Pup (L.)) also be the conditional set. According to the result of (Chen et al., 2024), the conditional independent relations among the latent variables are identified. Using the PC algorithm with latent variable (Chen et al., 2024), the causal structure among latent variables is identified up to Markov equivalent class.
D.3. Discussion of Two Pure-children Condition
We aim to demonstrate that the measurement model is identifiable under the condition that each latent variable has only two pure measured variables, assuming the latent structure is fully connected. This result, which is also shown by (Gu, 2022), is not surprising. However, compared to (Gu, 2022), our identification algorithm is simpler and more efficient.
18

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
We only need to check the rank condition of a three-way tensor, identify the causal cluster.
Theorem D.10. In the discrete sparse-LSM, suppose the Markov assumption, faithfulness assumption, non-degenerate condition, and the assumption D.4 hold. When each latent variable has least two pure measured variables and the latent structure is fully connected, the measurement model is identifiabl.
Proof. To identify the causal cluster by tensor rank condition, one can check for any X,, X, E X, VX2 E {X,, let r, then [X,,X)] a causal cluster, where is the cardinality of latent support. The proof is derived from the Proof of Proposition 4.3 in (Chen al., 2024). That is, if X,, X, are not causal cluster, e.g., L2 is the latent parent of X; and L1 is the latent parent of Xy. then there exists X2 E X X, X, such that the Ran)(P(Xp)) A r (see Fig 4 in (Chen et al., 2024)). Thus, all causal clusters can be identified by checking the rank constraints over the three-way probability tensor. Moreover, by applying the merging rule in Proposition 4.16, two causal clusters that share a common latent parent can be merged to avoid the redundant introduction of latent variables. Therefore, the identification of the measurement model is completed.
Xp = xx,
if (XXp))((X))))
Remark D.1 1 (Discussion on two-sufficient measurement). It is important to contrast this condition with the sufficient observation assumption in (Chen et al., 2024), where the cardinality of each discrete observed variable is larger than the cardinality of the latent variable. One may note that in the mixed LSM, each latent variable has at least three observed children, but only requires two of them to continuous variables, regardless of whether the third variable is discrete or continuous. Additionally, if the third variable is discrete, we do not assume any specific cardinality of its support; for instance, it can be a binary variable.
E. More Details on the Difference between Mixture Model and Tensor Rank Condition
To learn discrete latent structures solely from observed variables, a traditional approach treats structural identification as a parameter identification problem in mixture models. Numerous studies focus on this approach, including (Gu, 2022; Kivva et al., 2021; Anandkumar et al., 2012). We are particularly interested in discussing whether the identification results based on mixture models offer more general identifiability than those achieved through tensor rank conditions.
In (Gu, 2022), it is shown that for binary latent structure model, one can vectorize P(L1,... Lmm to marginal distribution PP((T,)) that the support of Thp is I supp [P((,)]- According to Kruskal's Theorem (Kruskal, 1977), the parameters of pw) and IL,,) can be uniquely recovered through tensor decomposition, under the assumption that each latent variable has at least three pure children. Based on the identified parameter matrix, the measurement model becomes identifiable. In (Kivva et al., 2021), it is shown that the measurement model can be identified under a weaker structural condition, leveraging the identification results of mixture model (Mixture Oracle). This is because the structure can be identified under weaker conditions by using simplified version of Kruskal's condition, as proposed by Lovitz and Petrov (Lovitz& & Petrov, 2023), shown below.
Lemma E.1 (Generalization of Kruskal's theorem (Lovitz & Petrov, 2023)). Let 2 and m 2 be integers, let V Vi @ ... Vm be vector space over field F and let
Fa.1 8
& Famm : a E [n] S CVV {0}
(4)
be a multiset of product tensors. For each a E [n], let va = Fa,1 ? ... & Famm- For each subset S S [n] and index j E [m]].
let
d dim paan(za. a E S}.
(5)
Tf2181 S (d - 1) + for every subset S [n|] with 2 < |S| S then Ene(n) To constites a unique tensor rank decomposition.
Lemma E.1 shows that the conditions for unique tensor decomposition can be relaxed to more general case. Specifically, by constructing a tensor that encapsulates structural information, Lemma E.1 enables a more broadly applicable identification result for the measurement model.
In summary, both (Gu, 2022) and (Kivva et al., 2021) use the unique tensor decomposition to identify the measurement model, relying on Kruskal's theorem or its generalized form.
19

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
F. More Details on Completeness Condition
Here, we present some existing results that illustrate the generality of the completeness condition, primarily based on the findings from (Hu & Shiu, 2018).
Remark F.1 (Sufficient Condition for (x]x(=(()) The sequence (xx]z(z]1)) corresponding to sequence {I: 1, .r] is linearly independent if one of the following conditions hold:
1) Ei (x((x))) 0 for all x implies C2 for all i = 1,2, ;
2) for all I, lim [(
[(=[(+1) 0 or limg-wo
f(al-11) F(ll)
0 for some z0-
In other words, many conditional densities satisfy the completeness condition. An important family of conditional distributions that exhibit completeness is the exponential family, including Gaussian, Poisson, Binomial, and certain multivariate forms of these. Another family that implies completeness consists of convolution density functions (Hu & Shiu, 2018). Additionally, (Newey& & Powell, 2003) demonstrates that the semiparametric exponential family also satisfies the completeness condition. A detailed discussion of the completeness condition can be found in the supplementary material of (Ying et al., 2023).
G. Proof and Illustratoons
Notations. Suppose X, L, X are random variables with domains X, C, X.. We denote the probability law of X as Lx == Po X-1 and denote the cumulative density function (CDF) Fx. For discrete variables X, we denote their levels 1 xx Ir etc. We use the the column vector P(8[=) == [p(ā, (a.),....((( p(21x [x the row vector P(a XX) == [p(x|āı), p(a T., )}.
and the matrix P(X[L) == [P) P(X [4,] to represent their transition probabilities. For a continuous variable X, we denote the probability density function (PDF) as fx and the conditional PDF of given L 1, as fz11, ( (r [(X]L- 4.)) We denote [1 a disjoint partition of X' with lx subspaces, such that U X; = X' and X; n X2 = 0 for
* j. We thereby define the discretized version of X as X, and denoting X == ã; to represent the event X E X, for any i. We denote (x)) as the conditional CDF of x given L = 4,. Besides, we denote F(x)((+)) = (F(x)41), F(x[4,) with
[r] : r.. We let A1 A2 == A n AG for two sets A2, We denote n as the sample size.
To simplify notation in the proof and distinguish X; and X, without using subscripts, we sometimes replace X,, X, with X, Y. This implies that Y is also observed variable measured in relation the latent variables in the mixed-LSM.
G.1. Proof of Theorem 3.4
We first give a lemma that gives a specific case proof of our theorem.
Lemma G.I. (Allman et al., 2009) Consider a bivariate distribution of the form
P(X,, X2) x2)= = Σ Tad (X)))(()))
(6)
Then P2) has rank r, if and only if. for each of j = 1, 2 the measures {ue hisser are linearly independent, where p denote the jth marginal of ps-
Remark G.2. One can see that, given IPXX,, X2 V) has the same forms of Eq. 6, the above result still holds. Besides, Let i² represent probabilistic decomposition of graphical models, PXXX, VV = =), with supp(V)] = under the Markov assumption and faithfulness condition, the above result also holds.
Remark G.3 (Mathematical Representation of Probability Tensor for Continuous Variables). To represent the probability ten- sor of continuous variable, such as X,, Xj. one can let the entry of probability tensor pi,j R 1: g= م p(ac, xz )dd,daj, where A 0, such that the probability tensor of P(X;, Xx) has size J, with 1, - co. One can see that this may result in an infinite tensor, leading to the curse of dimensionality. We will provide a practical solution for estimation rank in Sec .4.
20

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Lemma G.4 (Tensor Rank Condition in Two Variables Case). Let X1, X2 be two variables in the nonlinear causal model with discrete latent confounders, suppose the Markov assumption, faithfulness assumption, and completeness condition hold. Then F X2) has rank r (r < u) if and only if there exist a discrete variable set S with ssup([S) = r,, that d-separates X1 from X2. and there are no other conditional set S that satifies suup((S) <
Proof. 'If' part: IfS with sSpp(S) r, are the minimal conditional set that d-separates X from X2, one can rewrrite i) i)P(S i) according to the Markov assumption. Denote fx, be P(X, = i). by completeness condition, one can see that (Jx,is=
P(X1, X2) Ei=
are linearly independent. This is because [[g(L)|z} = O implies that {-==i 1 are linearly independent. By the Bayes' theorem, {fx, also are linearly independent (here are discrete latent variables set due to the model assumption).
Conversely, due to (Jx, with j = 1 are linear independent, the corresponding set of c.d.f.s (Fx,|s=))=_ are also linear independent. One may choose collections of points (t/) such that the r matrices M2 whose jntriess are Fx Fx,Is= (t2) have full rank. Then with F denoting the c.d.f. for IP, the matrix N with entries F(t!, (1)) can be expressed as
N = MI diag(P(S))M,,
and therefore has full rank. Otherwise, similar factorization arising from the expression of P using fewer than summands shows that N has rank smaller than which violates the completeness condition. Thus the rank of IP is at least and since the faithfulness assumption, it has rank exactly r.
OOll if' part: If X, and X2 are d-separated by a continuous variable set Xp. then r - n. This is because P(11,X)) = [(,,, (X,,),, for any data point. It means that for any r, the r x matrices Mg whose i,.j-entries are Ixx have full rank. Thus, there does not exist a low-rank expression of IP with r < n. Otherwise. X, degenerates into a discrete distribution that violates the faithfulness assumption. This also holds for the case that the conditional set contains continuous variable set.
Suppose P(X1, X2) has rank r with r < n, then there exist a discrete variable set S that d-separates X1 and X2. Otherwise, the rank r must close to n (for example, ,(X1, = P(X, (15))22)PP (2)))(X2) has large rank r due to X2 are a continuous conditional set). By the Markov assumption, P(X1, X2) has the form
P(X1,X)) = Σ -
(8)
According to Lemma G.1, with supp(SS) < r, it violate the linear independent of {f(X,)s IS = i)hsigr. j = 1, 2. Thus, S is the minimal conditional set with smallest cardinality of latent support.
115 ()sisrr j = =,,2, must be linear independent. If there exists a conditional set
Besides, for any no conditional set S with suup((S)) ) = PXX,, X21S, = i) is not rank-one matrix, therefore, cannot be expressed by Eq. 8 according to the Markov assumption. If there exist S1 with supp(S.)] = that is constructed from i) is rank-one matrix, it will violate the faithfulness assumption.
the parameter space, such that P X2
In summary, if (P(X,, X2) has rank with < n, then there exists conditional set S with smallest cardinarlity latent support that d-separates X, from X2.
Corollary G.5. Under the Markov assumption, faithfulness assumption, if the conditional set X1 and X2 contains continuous variable set Vn then for P(X,; X2 V,), the result of Lemma G.4 also holds.
Proof. The proof is straightforward. In the nonlinear causal model with latent confounders, all continuous variables are observational. Therefore, given P(X,, xp), if the result of Lemma G.4 does not hold, then it violates the Markov assumption and faithfulness assumption.
Now, we can prove Theorem 3.4.
21

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
Proof. When pI 2, the result holds according to Lemma G.4 and Corollary G.5, Next, we consider the case that xxp and x 0.
'If" part: if S d-separates any pair variables in Xp. according to the Markov assumption, we have
P(X,)) = Σ i
- II F(X, i),
where P(X,IS i) = fx, [s= are linear independent by the completeness condition. We further have {Fx,Is)in also linearly independent, such that the probability tensor can be expressed with entries F(t,, .. , .t)) (see a specific case in the proof of Lemma G.4). Furthermore, if there does not exist V, with (supp(V,)) p)1 < r, by the definition of tensor rank and the faithfulness assumption, F(X,) has rank
"Only if' part: if the probability tensor P(Xp) has rank r, then there does not exist X, E Xp. X2 and X, are d-separated by a continuous variable set. Otherwise, the sub-tensor with entries X,, X, has rank r that is close to n (See the proof of Lemma G.4). Thus, by the definition tensor rank, let IP(S = i) a constant and P(Xp/S S = i) is rank-one tensor, one has
P(X,) Σ P () @ P(X, =),
(10)
where any pair of
and F(X,|S = i) are linearly independent by the completeness condition. We aim to prove that S is a discrete conditional set with the smallest cardinality of support that d-separates all variables in Xp. There are three cases:
(1) If (S)) with supp(SS] = r arbitrarily constructed from the parameter space (i.e., S is not a node set in the causal graph 9), will violate the faithfulness assumption.
(2) If is not conditional set that d-separates any pair variables in Xy. without loss of generality, suppose there exists X,, X2 E X,, X, L XyyS, one can see that P(X,IS = i) is not rank-one tensor since the sub-tensor P(X;, X,, X,,X, x
c) is not a rank-one matrix according to the Markov assumption and faithfulness assumption (Hackbusch, 2012), where
X ((5)
denote X, (X,, X,). and X (G.j)
means that the values of X,) (i.J) is fixed to
(3) If S a conditional set for X, but not smallest cardinality of support, without loss of generality, let S is the conditional set with smallest cardinality of support, denoted by |supp($)] = k, k < r, one has
P(Xp) = Σ P(S 1) = 1) 2 P.X,I ii)
i=1
k
P(X, 1) P(X. =
(11)
Σ p1 8 ...
where PS = i) denote P(Š = i) = 1), and u² is a vector. According to the definition of tensor rank, the rank of P(X,) () k, which violates the preconditions that P P(X,) has rank r. Thus, S must be the conditional set with the smallest cardinality of support.
Next, consider the case of X * 0. if Xf US d-separates any pair variable in Xp. by the Markov assumption and completeness condition, one can see that (X,) has rank Without considering the X,, as the conditional set, the rank of (P(())) will be close to n. This is because there exists X,, X, E X,, X,, X, are d-connected without given X, as the conditional set. Thus, the sub-tensor with entries X; and X3 has rank & close to n (see the proof in Lemma G.4).
On the other hand, suppose PX, Xgy) has rank r, and r & n, according to the previous proof, there exists conditional set Since P(X,IX,) are the conditional
with the smallest cardinality of support r that d-separates variable
22

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
distribution of Xp given xy. then Xg also be the conditional set of Xp
Corollary G.6. If Xy is the conditional set that d-separates all variables in Xp. then the conditional probability tensor (g) has rank one.
Proof. Since Xg is the conditional set, we have
p()) = /x,Ix, ... SX,IX, -
80...
(12)
is a rank-one tensor according to the Markov assumption. Thus, the conditional probability tensor has rank one.
G.2. Proof of Proposition 4.2
Proof. We will show that, there exists a partition {X}i= with I 2 r such that the matrix P(X has rank We first show that the function (L = (=> are linear independent. Prove by contradiction. Suppose that there are r numbers {ou}f=1 that are not all zero such that Eim f (L = 1|æ) = 0. Let g(l) be the piece-wise constant function 1/)) 0 for all z, ie.. E[g(() a = O. Since g(.) * O, this contradicts the completeness assumption. Consequently, (f2xx (==) are linear independent. Besides, note that [x] 1) fLIX (laa therefore it is easy obtain the linear independence of {[x]z (3) T. 1) Since [x|z(x] T. =) = 1x1x((l I. 1), can also obtain the linear independence of [x (Similar results be found in (Liu al., 2024)).
defined by g(l) og, we then have Σ gl)
F(L=))
Given that (Fx]z(x:) = 1) are linear independent, we now show that there exists 1 2 points x, = { such that the matrix [L) has rank Denote Fx = [Fxi((1)), ... Fxx((x)) where Fxu(x) Fx r(x|L= = 1). We have
(x1)
Fx (x1)
Fxi
Fx (x2)
Rank(P(X[[))) Rank(
(13)
(x)((1))
(x1(
i.e. Rank([Fx]((x)),.. Fx (x: =r
We first discuss the case when 1 = r. The proof is based mathematical induction:
(1) Base: VZ E [r], we have 3z,, (x)( * 0. Thus, we have det( (Fx(((x))) + O.
(2) Induction hypothesis: assume for any combination 1,, 12. ... , E [r] (A E N,1 & k r 1). for l linear independent functions Fxi,, ......FFFpp.. (.), there exists points Xx = {I1," ... , such that det(MM)) o O, where
M (xx), (xx)), Fxju, (xx)
+r),,, there exists & + point X (....... Ja+1 }) such that det( & O, where M841 & [Fx (xu+1), Fx (x++1)] To this end, we first have the Laplace expansion of matrix determinant
(3) Induction: show that for any ls + linear independent functions Fxir, (.,,.., Ex
k+1
dot(M&-1)(xeH) = Σ (-1))+1+1 k+1+1 Fx(
(14)
where A/t+1 denote the submatrix obtained by removing the (I: + 1)-th row and I-th column from M That is, Aft11 = [[Fxi, (Xx), ... Fxit, (xx), Fx (xx). ... Fx (xx)] with (x)( = [Fxu(zz),.... ".Fp))(())))())))
Since Fxp,()).
Fxu, (-) are k linear independent functions, according to the induction hypothesis, there exist & points -1,".- II such that det(AAtt) +1 * O. That is, at least one of the cofactors det k++ ) in the expansion is not zero. Since Fxj, (.),.. Fx (-) are linear independent, we have - aq FxFx| = 0 iif. all a = 0 (a column expansion of matrix determinant). Thus, there exists Fk+11 such that det (x++))x+))) # for X which concludes the induction.
23

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
For the case when / > r, have (xI((x)),..- "x])))xx,xx(x)))) (x,) (where Fx(((x) (Pxu(z1),..- (x)(x))] are linear independent because (x,)(x);;... (x,) (where Fx]((x)) ( (x)(xr)] linear independent. That is, the row slice of matrix M, _ (xi),,.. "xx/"(( has the full column rank and hence the M, also has full column rank.
G.3. Proof of Lemma 4.4
Proof. Any discretization X; and X, in the discrete latent variable model has an upper bound of [supp[L]], which d-separates X2 from Xj. For convenience, we denote A, as the support of X, that X, = i. Ay is defined similarly.
P(X: = JA P(X;; Xx)dx,xx
-
supp(L)
Σ P(X, L- k= k)dX,dx,
(L
P(L = kb) / SA.
/
Σ
(15)
k=0
(L)
P(L kb) / P(Xi)L=))dl /S, -(X;IL=))dd
Σ
LL)
Σ
k)P(X; = - =-==A0 =A)P(X = =i k):
0=
where PPX; = i|L = k) = SA, P(X: L = k)dX, and P(X,- = -ih = k) = Sa, -(X,L==))d
We can rewrite P(X; = i, X, = j) as follows.
supp(L))
P(L = k)) -|-= k)P((X = -|L k)
(16)
P(X, = L) diag(P(L)) (X, = r|L).
So, one can get that
P(X,, X,) = P(X (P
(17)
Thus, if supp(L)] < |supp(X.)), and supp(L)] < suup((X;)), the upper bound of rank of (X, is
Rank(P(X,, X2)) < supp(L)]-
(18)
Moreover, one can that Rank(P( (P,(,, X,)) supp([L) if and only i(,,)) and IP(X)IL) full column rank.
G.4. Proof of Theorem 4.5
Proof. This result is proven based on Lemma 4.4. Since Rank(Ip())X;. X,), ((), and ((X)IL) are full column rank, we cannot find discretization such that Rank(p(k) (X,, > |upp(L))] ie., for the (I: + 1) time discretization, the rank stops increasing. Therefore, in this case, X, and X, satisfy the full-rank assumption.
supp(L)), and the upper bound is achieved
24

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
G.S. Proof of Proposition 4.7
Proof. In the discrete latent structure model, any two observed variables X,, Xj are conditionally independent given one latent parent of them. We first show that, by discretizing the marginal distribution of X, or Xy, the generation process of X still determined by its latent parent. One can see that
P(X = 1)
P(X)dX
Bi-1
(upp(P(())]
Σ
P[ j|PLL=
Bi-1
1=1
|supp[P(LN]
B.
E
-)P(L= j)dx
Bi-1
(19)
B,
Σ
IPLL 5)
PXX| - j)dx
B
(sLp((P(L))
P(L P(L=j)P(X = =|==))
((())
Under Markov assumption, faithfulness assumption and P(X full rank, we have
P(X) = P(L)(((T)))
(20)
which means that the generation process of X still determined by its latent parent Thus, we have
= ((L)=P(L)(((LLL))))
P(L)))(X,
(21)
where L1 parent of X, and L2 parent of X2.
Suppose L1 1L La]L in the causal graph, aim show that X, JL X2 also holds.
= -UIIII/TTPES//PPPLPLLLAEDDDEEE
(22)
PLLSI P(2..1114)
P(X2.a214)
Moreover, let (((i)L)) Σx, P(X,, ( LL) and P(Xy(D)= El, P(X2, L2) that
P(X, Σ P(X
(23)
Therefore, X, JL Xgl L also holds.
Mcanwhile, if L, L L in the causal graph, we aim to show that X, L X² L also holds. Without loss of generality, suppose (2) * (T.,]I))P([L]] T.), one can see that
25

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
X -
Σ Σ
PA = i)
L1 L2
(24)
Σ (())(L))) Σ P(X
Ly
Tn
1/2
P(X L
i) i).
Thus, we have X1 L X2/4 hold if X1 1. X2|L in the causal graph. Based on the proof above, the CI relations after discretization are consistent.
Discussion. Actually, since the CI relationship between observed variables relies on the CI relations among latent variables in the mixed-type latent structure model (LSM), the discretized variables still maintain the conditional probability decomposition in the discrete probability space. In other words, the discretization does not alter the relations within the discrete probability space.
G.6. Proof of Lemma 4.8
Proof. For continuous observational data that satisfy the completeness condition, one can discretize the continuous data into discretized data that satisfy the non-degeneracy condition by Theorem 4.5. Thus, for the discretized data Xp. it satisfies the Markov assumption, faithfulness assumption, and non-degeneracy condition according to Proposition 4.7, the tensor rank condition holds for Xz. The proof for the discrete data can be referred to (Chen et al., 2024).
G.7. Proof of Proposition 4.11
Proof. Under the assumption that each latent variable has the same cardinality of support, it is easy check if X; is only caused by one latent parent. One can prove by contradiction. If X, has more than one latent parent, by the model assumption, there exists Xj that shares the set of common latent parents with X,, thus, Rank(IP (AX,, (x), > r according to the graphical implication of the tensor rank condition, where is the cardinality of latent support. This means that if X; is not a one-factor structure, there are Rank(P(X;, X2)) > r and Rank (P(X,, X&)) = r where X₁ is any one-factor structure variable. Thus, (ank(P((,;;. X,)) is different for any X, E X (X:).
G.8. Proof of Proposition 4.13
Proof. Due to X. being a one-factor group, Xy) also be a one-factor sub-group. If {X,, Xx} is not a one-factor cluster, without loss of generality, let L, be the parent of (X,, ) and L2 be the parent of Xx, according to the model assumption, there exist X, also be the children of L2, such that (Xi, Xj. Xx, X,} are d-separated by { L2). By the graphical implication of the tensor rank condition, Rank(P(X,, (X,, Xy. Xx, et al., 2024)). Thus, {X,, X,, X is causal cluster if VX,, X XX,, Xx). Rank(IP(X,,. Xj, X X.)) =r.
p2 (The similar proof can be seen in (Chen
G.9. Proof of Proposition 4.15
Proof. By Proposition 4.1 1, the observed variable can be classified into the one-factor group and the That is, X \ X. is the set of the multi-facter group. For C = X,, let {L1, L.) ) be the latent parent set of C, one can map [In,... I.,}} } to be one latent variable L,g with cardinality r". Then this result can be proved by the extension result of different state space cases in (Chen et al., 2024).
group.
G.10. Proof of Proposition 4.16
Proof. For the first proposition, because C1 n C2 + 0, with loss of generality, assume that C₁ n C2 X Let (a(X)) denote the parent of X₁. One has Pa(Xx) be the parent of C₁ according to the definition of the causal cluster. Meanwhile, we also have Pa(X;) be the parent of C2. Therefore, C₁ and C2 share a common latent parent.
For the second proposition, since Pa(())) > IPa(C2)), and Rank(P (C1 U C2)) Rank(P(Cr)), by the graphical
26

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
implication of tensor rank, can infer Pa(Ci) also be the d-separated for Cz. Under the model assumption of díscrete mixed-LSM, one infer that Pa(C2) C Pa(Ci), and hence C₁ and C2 share the common latent parent.
G.11. Proof of Theorem 4.17
Proof. Proof of this theorem is straightforward by combining the proof of (Chen et al., 2024) and the property of tensor rank in which the tensor rank is greater than equal to the sub-tensor rank (constructed by sufficient measured variables). 0
G.12. Proof of Theorem 4.19
Proof. We prove the identification by a two-stage algorithm, including the identification of the measurement model and the identification of the structure model.
Stage I: the identification of the measurement model. In this stage, Algorithm I first discretizes all continuous variables to enable efficient testing of tensor rank. Next, it identifies the latent support and one-factor group using Proposition 4.13. By applying Propositions 4. 13 and 4,15, the algorithm identifies the causal clusters and determines the number of latent parents for each cluster. Finally, according to Proposition 4.16, the causal clusters are merged to prevent the redundant introduction of latent variables. By this, the measurement model is completely identified under the Mixed LSM.
Stage II: Identification of the Structural Model. In this stage, d-separation relations are identified using Theorem 4.17. Then, the PC algorithm (Spirtes et al., 2000) is applied. Given the measurement model, the causal structure among latent variables can be identified up to Markov equivalence class (Chen et al., 2024).
H. More details on the Experimental Results
We first introduce detailed definition of different evaluation metrics. Denote the output results of each algorithm Gout, the true graph is labeled G: The performance of the causal cluster is evaluated by following the scores:
latent omission: the number of latents in G that do not appear in Goat divided by the total number of true latents in G;
Moreover, we use the F1 score to evaluate the performance of causal structure among latent variables. Next, we describe the calculation details of Fl score. First, we introduce two metrics used basically for Fl score.
edge omission (EO): the number of edges in the structural model of G that do not appear in Gout divided by the possible number of edge omissions;
edge commission (EC): the number of edges in the structural model of Gout that do not exist in G divided by the possible number of edge commissions;
F1 2 (1-E0) (C)
(1 (E-FO) (1 EC)
H.1. Experimental Results on Real-world Dataset
In this section, we apply our algorithm to the real-world Industrialization and Political Democracy dataset (Bollen, 1989). This dataset contains various measures of political democracy and industrialization across developing countries, with 75 observations of variables. The details of each observation are summarized as follows.
yl: Expert ratings of the freedom of the press 1960
y2: The freedom of political opposition 1960
y3: The fairness of elections 1960
y4: The effectiveness of the elected legislature in 1960
y5: Expert ratings of the freedom of the press 1965
27

---

Identification of Latent Confounders via Investigating the Tensor Ranks of the Nonlinear Observations
y6: The freedom of political opposition in 1965
y7: The fairness of elections in 1965
y8: The effectiveness of the elected legislature in 1965
xl: The gross national product (GNP) per capita in 1960
x2: The inanimate energy consumption per capita in 1960
x3:
The percentage of the labor force industry 1960
In (Bollen, 1989), the authors show that there three latent factors that these observed variables. Specifically, a21, 233 are driven by single latent factor, while {y1, y2, 23, y4) and {25, y6, y7, y88 are each influenced by two-factor structure. In our implementation, we first obtain the discretized data using our discretization technique, where each observed variable is discretized into a discrete variable with four categories. We then apply a bootstrapping resampling approach to enhance the statistical properties of the data. Next, we compute the results for all possible combinations sets with four elements, sorting them according their reconstruction error (CP decomposition with rank v, more details can refer (Chen al., 2024)) and selecting the top-K items. (where K 10). Finally, our algorithm outputs the following result:
{L1} - f21, (upp((((L,)) = 3, (L2 (yl, v3, v4, y5, y6, y7, y8). (s2))()(L2)
|supp(P(Lss) 2.
One can observe that the results are consistent with the discussion in (Bollen, 1989), demonstrating the effectiveness of our algorithm, including the discretization technique. In fact, these latent variables have real-world meaning, For example, as discussed in (Bollen, 1989), yl to y4 are intended to be indicators of the latent variable political democracy in 1960, y5 to y8 are indicators political democracy 1965, and æ1 to æ3 are indicators of industrialization in 1960. Since there are common features between political democracy in 1960 and political democracy in 1965, our algorithm learns them as two-factor structure.
28


Reviews:
[{"review_id": "iz3tnT60FP", "reviewer": "Reviewer_xpF6", "summary": "This work proposes a causal discovery algorithm for some class of causal graph involving discrete latent variables and both discrete and continuous observed variables. The algorithm is essentially an extension of Chen et al. (2024) which uses rank tests on the probability tensors of observed variables to infer the graph connecting the latent variables. The novelty of the approach resides in its ability to deal with continuous observed variables by proposing a discretization scheme motivated by a theoretical analysis (Section 4.1).", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful review. We address each point below and have corrected the typos. Please feel free to reach out with any further questions (due to limited space).\n\n>W1W2: ... realistic dataset to test their approach. \n\nA: Please see Appendix H.1.\n>W3: Novelty is between low and moderate.\n\nA: Please see the **Q1A1** response of #Review gVAe.\n>Q1: Do you allow edges from observed to observed variables?\n\nA: We allow edges between observed variables. This highlights that the tensor rank condition applies beyond the pure measurement setting (e.g., impure case in Appendix D). \n>Q3: Assp. 2.1 (a): Do you mean for all $r\\in \\Omega$ here? \n\nA:  Yes, we mean that for any $r \\in \\Omega$ here — each marginal places non-zero mass on every element of $\\Omega$.\n>Q4: Assp. 2.1 (b):  What is its shape? Which dimension corresponds to which variable?\n\nA: The conditional distribution can be represented as a contingency table— a $|V_i| \\times |Pa(V_i)|$ matrix. \n>Q5.1: Theo. 3.2: ...what is meant by the rank of a tensor.\n\nA: The rank of a tensor $\\mathcal{X}$, denoted $rank(\\mathcal{X})$, is the smallest number of rank-one tensors that generate $\\mathcal{X}$ as their sum, where a $N$-way tensor is rank-one if it can be written as the outer product of $N$ vectors.\n>Q5.2: Theo. 3.2: So $X_p$ is just the set of observed variables here? Or is it a subset?\n\nA: We modify it to $\\mathbf{X}_p =\\{X_1,\\cdots,X_{p}\\} \\subseteq \\mathbf{X}, p \\leq n$.\n>Q5.3: Theo. 3.2: Can the conditional set $S$ intersect with $X_p$? Or should it be constrained to the latent variables? \n\nA: Yes, we allow $S \\cap X_p \\neq \\emptyset$ and allow observed variables included in $S$. \n>Q6: Lower case r is used both for the rank of tensor and for the cardinality of $\\Omega$. \n\nA: We will use distinct symbols for tensor rank and the cardinality of $\\Omega$ in revision.\n>Q7.1: Cond. 3.3: ...only X_i that satisfy “for all L_j, $ X_i \\bot  X\\setminus X_i | L_j$”, correct?... Could you give an example? Q7.2: ...function $g:\\Omega \\to R$ is bounded since $\\Omega$ is finite? Q7.3: ...no quantifier for $i$ and $j$ here.\n\nA: We will clarify this in the revision: for any $X_i \\in \\mathbf{X}$ that has only one (latent) parent $L_j$, we assume the conditional distribution $\\mathbb{P}(L_j|X_i)$ is complete. That is, for all measurable real function $g$ such that $\\mathbb{E}(|g(l)|) < +\\infty$， $\\mathbb{E}(g(l)|x) = 0$ almost surely iif $g(l) = 0$ almost surely. As an example, consider $X_7, L_3$ in Fig. 2. Besides, we assume that the domain $\\Omega$ is finite. In this case, any function $g:\\Omega \\to R$ is indeed bounded.\n>Q7.4: Cond. 3.3: It seems completeness is a property of the joint $P(L_j, X_i)$...Q7.5: App. F on cond. 3.3 didn’t help.\n\nA: Yes, completeness is the property of the joint distribution. Due to the Bayesian formula, the completeness on the conditional distribution $P(L_j|X_i)=P(L_j, X_i)/ P(X_i)$ involves the joint distribution. In this paper, we adopt the conditional form for consistency with prior work (e.g., Cui et al., 2023). \nIn Appendix F, we illustrate that the completeness condition is not overly restrictive and can arise naturally in practice. We’ve revised the appendix to clarify this intention.\n>Q8: Theo. 3.4: How do you define rank here? \n\nA: As noted in Remark G.3, we treat the tensor as a theoretical representation of the joint distribution, with rank defined by its minimal rank-one decomposition. We will clarify this in the revision.\n>Q9: Def. 4.1: What’s a pure variable? It’s not defined here. What’s a “sufficient measured variable”?\n\nA:  Pure variables denote the variables that have only one latent parent, and no observed parents. For an observed variable $X$ with support $\\mathcal{X}$ and latent parent $L$ with support $\\Omega$, we define a sufficient measurement as $|\\mathcal{X}| > |\\Omega|$.\n>Q10: Line 244: The “tensor rank condition” ... is not properly defined anywhere...\n\nA: The rank condition refer the rank deficiency of probability tensor. Besides, your statement is right, we would like to study when and how to use discretization to make this properity hold in continous data.\n>Q11.1: where we are assuming that each observed variable has a single latent parent? \n\nA: This assumption follows from the model definition, where we adopt the standard pure measurement setting (Silva et al., 2006), in which each observed variable has a single latent parent. We will clarify this more explicitly in the revision.\n>Q11.2: Does this proposition mean that any discretization will work?\n\nA: No, not all discretizations will work. We clarify this point in Remark 4.3 and illustrate it with an example in Appendix B. \n>Q12: Theo. 4.5: What’s k here? The iteration of some algorithm?\n\nA: k refers to the number of discretization steps, as discussed in Remark 4.6.\n>Q14: Prop. 4.15: Isn’t it weird that $X_s$ does not appear in (i)...?\n\nA:  $X_s$ does not need to appear in (i) since it may not share a common latent parent with the other variables.", "claims_and_evidence": "I'm inclined to say that this work makes reasonable claims that are backed by sufficient evidence (both theoretical and empirical). That beings said, clarity is an important issue which prevents me from recommending acceptation of this paper. I found many cases where theoretical claims are so unclear that it is very difficult to assess whether they are sound or not. More on this later.", "methods_and_evaluation_criteria": "The paper is mainly theoretical, but does present some empirical validation on synthetic data. This sort of empirical analysis is fairly standard for a theoretical work and IMO sufficient. That being said, I highly recommend the authors to find either a more realistic dataset to test their approach or at least a motivating example. Right now the approach feels a bit unmotivated.", "theoretical_claims": "I did read all the theoretical claims made in the main paper, but did not read the proofs in the appendix. The results appears believable, but clarity is a very significant issue IMO. (more on this later)", "experimental_designs_or_analyses": "See above.", "relation_to_broader_scientific_literature": "I am not following this literature closely, but it appeared to me that the contribution is properly contextualized within existing works. My understanding is that this work is an extension of Chen et al. (2024), which is made very clear and transparent in the manuscript, to deal with continuous observed variables. That being said, the delta between both works appears to be fairly small (it adds discretization strategy backed by theoretical analysis). Maybe the authors could explain a bit more how their work differs from Chen's?", "essential_references_not_discussed": "I didn't notice anything obvious.", "other_strengths_and_weaknesses": "Strengths:\n\n- I thought the topic was interesting, the title and the general theme got me curious and stimulated. I believe this is overall an interesting direction.\n- I appreciate the effort made by the authors to include a running example, which helped grounding some of the technical concepts.\n\nWeakness:\n\n- The work felt a bit unmotivated. IIRC, next to no effort is made to explain why this direction is exciting and could lead to interesting applications in the future.\n- Novelty is between low and moderate. If I understood correctly, the only algorithmic novelty comes from the discretization phase which then allows the application of Chen et al. (2024), which works for discrete variables. Section 4.1 also contains a theoretical analysis of discretization. This feels a bit marginal, but I suppose this is subjective.\n- The most important issue with this work is its lack of clarity. Below, I give a list of confusions I got as I read the manuscript, which I believe are due to poor writing.\n\nClarity:\n- “In the causal graph, we do not allow directions from observed to latent to preserve the latent confounder structure.” Do you allow edges from observed to observed variables? It seems all the figures have graphs without such edges, but you do not mention it anywhere AFAIK. Maybe these assumptions should belong to an Assumption environment given their importance? Oh I see this assumption is done much later in Definition 4.1. Overall it seems like multiple results use different assumptions. It would be useful to refer to these definitions/assumptions using \\ref directly in the theorems, otherwise it’s hard to follow which assumptions are made and where.\n- Eq (1), f should be indexed by i, no?\n- Assumption 2.1 (a): Do you mean for all $r \\in \\Omega$ here? As in, each marginal P(L_i) puts mass everywhere on $\\Omega$? If so, this is not clear here.\n- Assumption 2.1 (b): Might be useful to say explicitly what this conditional distribution contingency table is. What is its shape? Which dimension corresponds to which variable?\n- Theorem 3.2:\n    - It would be nice to have a definition of what is meant by the rank of a tensor. I would argue that this is not a broadly known notion (unlike the rank of a matrix for example)\n    - X_p = {X_1, …, X_n} but then it’s written X_p \\subseteq X… But earlier it was said that X was n-dimensional. So X_p is just the set of observed variables here? Or is it a subset? \n    - Can the conditional set S intersect with X_p? X? Or should it be constrained to the latent variables?\n- General point: Looks like there’s a clash in notation. Lower case r is used both for the rank of tensor and for the cardinality of \\Omega. \n- Condition 3.3:\n    - I’m a bit confused here. The condition concerns only the variables X_i that satisfy the condition “for all L_j, X_i \\indep X\\X_i | L_j”, correct? I couldn’t find a graph in your figures that had such a node X_i. Could you give an example of such a variable in a given graph?  \n    - Isn’t it the case that any function $g: \\Omega \\rightarrow \\mathbb{R}$ is bounded since $\\Omega$ is finite? Actually, are we assuming it’s finite? Also, am I right to assume that the codomain of $g$ is the real line?\n    - Are we saying that completeness holds for all conditionals P(L_j | X_i)? I.e. for all pairs (X_i, L_j)? That’s unclear because there’s no quantifier for i and j here.\n    - It seems completeness is a property of the joint P(L_j, X_i), not just the conditional P(L_j | X_i) since  statements like E[g(L_j) | X_i] = 0 almost surely depends on the distribution P(X_i). Same for g(L_j) = 0 a.e., which cannot be verified from P(L_j | X_i) alone (you need the marginal over L_j).\n    - Appendix F on condition 3.3 didn’t help and is poorly written, I really didn’t get the point there.\n- Theorem 3.4\n    - Same issues as in Theorem 3.2\n    - It seems we are manipulating a Tensor with n-dimensions where some of its dimensions have a continuum of indices. How do you define rank here? Also, this is a very unusual object so it should be described and explained more.\n- Definition 4.1\n    - what’s a pure variable? It’s not defined here.\n    - what’s a “sufficient measured variable”?\n- Line 244: The “tensor rank condition” is constantly mentioned, but is not properly defined anywhere. I genuinely don’t know what is meant here. I think what is actually meant here is whether we can find a discretization that will yield a tensor that satisfies the assumptions of Theorem 3.2, right?\n- Proof of Proposition 4.2: \n    - Can you point to where we are assuming that each observed variable has a single latent parent? (you use this assumption on line ~266).\n    - Does this proposition mean that any discretization will work? (as long as it’s not clustering all values in a single state?\n- Theorem 4.5: What’s k here? The iteration of some algorithm? Which algorithm?\n- Definition 4.9: Typo here? n should be p no?\n- Proposition 4.15: Isn’t it weird that X_s does not appear in (i), given the quantifier “for all X_s”?", "other_comments_or_suggestions": "Line 429: Typo", "questions_for_authors": "My questions appears above, and concern mainly the clarity of the work.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ogqJpA5z25", "reviewer": "Reviewer_e9yn", "summary": "When observational data contain both continuous and discrete variables, learning the causal structure among latent variables becomes a critical problem. Existing methods are often sensitive to parameter estimation. In this paper, the authors propose a statistically testable approach, the tensor rank condition, to address this issue. By discretizing continuous variables appropriately, the authors introduce graphical criteria for non-linear causal models with discrete latent variables, leveraging the tensor rank condition. Building on this, the authors formulate the Mixed LSMs framework and further develop a two-stage algorithm: first identifying causal clusters and then inferring the causal structure among latent variables. This approach provides a novel solution to the identification of Mixed LSMs. Experimental results demonstrate that the proposed algorithm outperforms state-of-the-art methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable comments and suggestions and thank you for your positive assessment of our work.\n\n>[W1] Estimating tensor rank in practice can be challenging, yet the paper does not provide sufficient discussion on this issue.\n\n**W1A1**: As discussed in the Sec. 5 (Lines 402–404), we estimate matrix rank using the hypothesis test proposed by Mazaheri et al. (2023), and tensor rank following the approach in Chen et al. (2024). We will add more discussion on this point to improve clarity.\n\n>[W2] The data generation process in the experiments follows a mixture model rather than directly following Eq. (1). The author should clarify this.\n\n**W2A2**: Thank you for the comment. To ensure that the generated data satisfies the completeness condition, we use a mixture model as a practical and effective way to simulate data. Although the data is generated from a mixture model, the resulting joint distribution still reflects nonlinear dependencies (due to the probability property of the mixture model), and thus remains consistent with the setting described in Eq. (1). We will clarify this point in the revised text.\n\n>[W3] The effectiveness of the discretization approach depends on accurately estimating the rank of the probability table. However, the paper does not discuss how the precision of rank estimation impacts causal structure learning.\n\n**W3A3**: In our theoretical results, once the latent support size $r$ is identified, the causal structure can be recovered through rank-decomposition with specified $r$. Otherwise, an incorrectly estimated $r$ may lead to incorrect identification of latent structure, especially in the number of latent variables. To show that the latent support can be estimated consistently, we provide additional simulation results on estimating the rank of the probability table.\n\nWe consider the following settings: $G_1$: $L_1 \\to \\{X_1, X_2, X_3, X_4\\}$; $G_2$: $L_1 \\to L_2$, $L_1 \\to \\{X_1, X_2, X_3\\}$, $L_2 \\to \\{X_4, X_5, X_6\\}$. In both settings, the latent variables have support size 2, and the observed variables have support size 3. Each experiment was repeated 1,000 times with randomly generated data. Using the hypothesis test by Mazaheri et al. (2023), we find that the rank of the probability table can be accurately estimated in most cases. This suggests that the effectiveness of our structure learning method, which relies on tensor rank, is supported by the reliability of rank estimation.\n\n\n||Accuracy|Accuracy|Accuracy|Accuracy|\n|---|---|---|---|---|\n|#samples|3000|5000|10000|30000|\n|G1|0.82(±0.05)|0.85(±0.05)|0.84(±0.05)| 0.85(±0.06)|\n|G2|0.76(±0.06)|0.79(±0.06)|0.80(±0.06)| 0.82(±0.05)|\n\n>[Q1] Previous works often assume sufficient observational support for latent variable identification. Why is the two-sufficient measurement condition enough for testing the tensor rank condition?\n\n\n**Q1A1**: Thank you for the question. For each latent variable, two sufficient measurement variable $X_i, X_j$ are enough to identify the latent support by detecting the rank deficiency in the joint distribution $\\mathbb{P}(X_i, X_j)$. Moreover, there is a key property of tensor rank: the rank of a tensor is at least as large as that of any of its subtensors, \ne.g., Rank$(\\mathbb{P}(X_i, X_j, X_k))$ $\\geq$ Rank$(\\mathbb{P}(X_i, X_j, X_k=c))$. \n\nFor example, consider an case that $L$ d-separated $X_i, X_j, X_k$, where $X_i, X_j$ are sufficient measurement variables and $L$ has the support size $r$. If the rank condition holds for a subtensor, e.g., Rank$(\\mathbb{P}(X_i, X_j, X_k=c)) = r$, then the full tensor $\\mathbb{P}(X_i, X_j, X_k)$ must also have rank $r$, regardless of whether $X_k$ satisfies the sufficient measurement condition (i.e., even if its support is smaller than $r$).\n\nTherefore, two sufficient measurements per latent variable are enough to test the tensor rank condition.\n\n\n\n\n\n>[Q2] Why does the identifiability of Mixed LSMs require that at least one set of observed variables is caused by a single latent parent? If this is a structural assumption, it should be explicitly stated in the main text.\n\n\n**Q2A2**: Thank you for pointing this out. This requirement is included in the Three-Pure-Child-Variable Assumption in Definition 4.1, and we further discuss it in Remark D.5. Broadly speaking, this assumption is used to identify the support of latent variables when considering the n-factor structure. We will make this assumption more explicit in the main text.", "claims_and_evidence": "The claims are well supported by theoretical analysis and extensive experiments.", "methods_and_evaluation_criteria": "The proposed method is reasonable and sound. The evaluation metrics chosen in this paper are appropriate for the focused problem, i.e., the identification of latent confounders.", "theoretical_claims": "The theoretical claims are well-discussed, and the proofs are presented clearly.", "experimental_designs_or_analyses": "Experimental designs and analyses are sound.", "relation_to_broader_scientific_literature": "The proposed method is a novel solution for the latent confounder identification.  The proposed solution is both interesting and novel and is more realistic in practical scenarios.", "essential_references_not_discussed": "To the best of my knowledge, all key references are well-discussed in the related work section.", "other_strengths_and_weaknesses": "Strengths:\n\n- The authors present graphical criteria for non-linear causal models with discrete latent confounders, establishing a connection between tensor rank and d-separation relations in the graph. This offers a novel methodology for studying causal structures in non-linear models.\n\n- In Appendix D, the authors provide identifiability results for discrete latent structures under a sparsity condition, relaxing the pure-child requirement from previous work. This is an interesting contribution.\n\n- The authors discuss the discretization process in detail, providing clear examples that make the methodology easier to follow.\n\nWeaknesses:\n\n- Estimating tensor rank in practice can be challenging, yet the paper does not provide sufficient discussion on this issue.\n\n- The data generation process in the experiments follows a mixture model rather than directly following Eq. (1). The author should clarify this.\n\n- The effectiveness of the discretization approach depends on accurately estimating the rank of the probability table. However, the paper does not discuss how the precision of rank estimation impacts causal structure learning.", "other_comments_or_suggestions": "NA.", "questions_for_authors": "1. Previous works often assume sufficient observational support for latent variable identification. Why is the two-sufficient measurement condition enough for testing the tensor rank condition?\n\n2. Why does the identifiability of Mixed LSMs require that at least one set of observed variables is caused by a single latent parent? If this is a structural assumption, it should be explicitly stated in the main text.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WZC4Q7DwUm", "reviewer": "Reviewer_gVAe", "summary": "This paper studies the problem of learning discrete latent variable causal structures from mixed-type observational data using the graphical criteria of tensor rank conditions. To handle continuous observed variables, the author proposes a discretization method that ensures the discretized data satisfy the full-rank assumption, thereby allowing existing results for discrete data to be directly extended. The proposed method is further applied to the task of learning discrete latent variable causal structures from mixed-type observational data.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your careful and valuable comments. We will respond to these issues point by point.\n>[W] The Mixture Oracle method, a relevant baseline, is not included in the experimental comparison, which limits the evaluation.\n\n**WQ1**: We have added experimental results using the Mixture Oracle as a baseline. Since K-means is not suitable for discrete data, we adapt the Mixture Oracle implementation by replacing K-means with the K-modes algorithm [1]. We evaluate the accuracy in identifying the latent support and include the results in the updated experiments. **As shown in Table 1 of https://anonymous.4open.science/r/TensorRank-E052/Experimental%20results.pdf**\n\nOne can observe that the Mixture Oracle method does not effectively identify the support of latent variables, even when using a discrete clustering algorithm. This may be due to the fact that clustering yields only an approximate solution and lacks theoretical guarantees for recovering the true latent structure.\n>[Q1] Could the author clarify the key differences between the tensor rank condition in non-linear causal models and the tensor rank condition in discrete latent structure models? How does this work advance previous results, particularly in relation to (Chen et al. 2024)?\n\n**Q1A1**: We would like to clarify our contributions in two main aspects—**applicability conditions and identification boundaries**—particularly in relation to the work by Chen et al. (2024).\n\n1. **Our work considers a more general setting for when the tensor rank condition holds.** In particular, we allow the observed variables to be continuous and show that the latent structure remains identifiable when each latent variable has two sufficiently informative observed variables. In contrast, Chen et al. (2024) propose the tensor rank condition when all variables  are discrete and each observed variable is assumed to be a sufficient measurement for its latent parent (i.e., the observed support has higher cardinality than the latent’s).\n\n2. **We also extend the tensor rank condition to conditional probability tables, which leads to a more general identifiability condition for latent structures**, as described in Appendix D. While Chen et al. (2024) explore rank constraints based on the joint probability table under the assumption of a three-pure-measurement-variable model, our approach considers constraints on conditional distributions. This allows us to test conditional relationships among observed variables in the presence of latent confounding (i.e., impure structures), and further, to identify the causal structure among latent variables even under impure setting (Appendix D). To the best of our knowledge, this provides a novel identifiability condition for discrete latent variable models.\n\n    To evaluate structure learning under an impure setting, we conduct the following simulation, **as shown in  Table 2 of https://anonymous.4open.science/r/TensorRank-E052/Experimental%20results.pdf**. We compare our method (Appendix D.2) with the approach proposed by Chen et al. (2024). One can see that the method by (Chen et al. 2024) cannot learn the causal structure among latent variable, this is because their method only suitable for the pure measurement model.\n    \n    Moreover, in Appendix D.3, we show that only two purely measured variables per latent variable are sufficient to identify the measurement model when the latent structure is fully connected. **This result also relaxes the structural assumptions required by previous work based on the three-pure-measurement-variable assumption.**\n>[Q2] For the case of continuous observed variables and discrete latent variables, previous work—such as the Mixture Oracle method—has also explored identifiability. What are the key differences between your approach and the Mixture Oracle method?\n\n**Q2A2**: We would like to clarify our contributions with two main points, especially in relation to the Mixture Oracle approach.\n\n1. **We offer a robust and testable method for structure learning**. Unlike the Mixture Oracle method, which relies on identifying a mixture model (as discussed in our Appendix E) and provides only an approximate method for parameter estimation—an approach that we find can be unreliable and difficult to test (see Sec. 3.1 and App. B)—our method introduces a hypothesis test for the tensor rank condition. This test not only enhances robustness compared to the Mixture Oracle but also allows us to identify the latent support with theoretical guarantees (see **W3A3** in response to Reviewer e9yn).\n\n2. **We introduce a novel and more general structural condition for the identifiability of discrete latent structures** (refer to Point 2 in **Q1A1**). In contrast, the Mixture Oracle assumes that no edges between observed variables, which is a quite restrictive assumption. \n\nWe sincerely appreciate your thoughtful inquiry and hope this clarification helps. Please feel free to reach out if you have any further questions.", "claims_and_evidence": "The claims are generally well-supported by theoretical analysis, but certain aspects, such as novelty and baseline comparisons, require further clarification.", "methods_and_evaluation_criteria": "The proposed method is reasonable and well-motivated. The evaluation metrics selected are appropriate for the problem of latent confounder identification.", "theoretical_claims": "The theoretical claims are clearly presented, and the proofs are OK.", "experimental_designs_or_analyses": "The experimental design and analyses are generally sound. However, the simulated structure appears relatively simple, which may limit the evaluation of the method’s robustness in more complex settings.", "relation_to_broader_scientific_literature": "The proposed method contributes to the problem of latent confounder identification. While the approach is interesting and relevant, its novelty and practical advantages over existing methods should be better articulated.", "essential_references_not_discussed": "The author gives a well-reviewed on the related literature.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and clearly organized.\n\n2. It provides a comprehensive review of related literature.\n\n3. The problem of learning latent structures in non-linear models is both important and challenging. The proposed approach appears to be sound and well-justified.\n\nWeaknesses:\n\n1. The paper focuses on a specific class of non-linear models, where latent variables are discrete and observed variables are continuous, which may limit general applicability.\n\n2. The tensor rank condition and its graphical criteria have been explored in prior work, so the novelty needs further clarification.\n\n3. The Mixture Oracle method, a relevant baseline, is not included in the experimental comparison, which limits the evaluation.", "other_comments_or_suggestions": "1. Could the author clarify the key differences between the tensor rank condition in non-linear causal models and the tensor rank condition in discrete latent structure models? How does this work advance previous results, particularly in relation to [1]?\n\n2. For the case of continuous observed variables and discrete latent variables, previous work—such as the Mixture Oracle method—has also explored identifiability. What are the key differences between your approach and the Mixture Oracle method?\n\nReference:\n\n[1]. Learning Discrete Latent Variable Structures with Tensor Rank Conditions. NeurIPS 2024.", "questions_for_authors": "see above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: While two reviewers knowledgeable of the field rated this work highly, one reviewer raised concerns regarding the clarity of the work. While authors addressed many concerns during the rebuttal period, this remains a paper that only only a small fraction of the ICML readership may be able to follow. I therefore recommend a weak accept.
Decision: Accept (poster)