Title: Causal Mixture Models: Characterization and Discovery
Venue: NeurIPS 2025
Accepted: yes
Keywords: causal discovery

mixture modelling
Abstract: Real-world datasets are often a combination of unobserved subpopulations that follow distinct causal generating processes. In an observational study, for example, participants may fall into unknown groups that either (a) respond effectively to a drug, or (b) show no response due to drug resistance. Not accounting for such heterogeneity then risks biased estimates of  drug effectiveness.  
 In this work, we formulate this setting through a causal mixture model,
 in which the data-generating process of each variable depends on latent group membership (a or b). Specifically, we model each variable as a   mixture of structural causal equation models, where latent categorical (mixing) variables index assignment to subpopulations. Unlike prior work, the approach allows  for multiple independent mixing variables, each affecting distinct sets of observed variables. To infer both the graph, mixing variables, and assignments jointly, we integrate mixture modeling into score-based causal discovery; show theoretically that the resulting scoring criterion is consistent; and demonstrate  empirically that the resulting causal discovery approach discovers the causal model in synthetic and real-world evaluations.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors study the problem of causal discovery using data from different distributions/environments which they frame as a causal mixture model using potentially multiple discrete mixing variables. These mixing variables are assumed to be unobserved. They propose a score-based EM-style method using BIC that jointly finds the causal ordering and the mixing variables. They provide a theoretical analysis of their approach as well as numerical experiments.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The authors study the relevant problem of causal discovery with data from different environments where the environment-assignment is unknown\n- They provide extensive experiments on simulated as well as real-world data\n- The paper is structured clearly, and easy to follow, but some of the mathematical notation can be streamlined (see points in weaknesses).\n\n**Weaknesses:**\n- The method rests on the strong assumption of linearity as recognized by the authors. The authors claim in the conclusion that the method can \"easily incorporate richer models\", I think it would have strengthened the contribution to explore this in some way, e.g. in the experiments.\n- There seems to be a strand of work missing in the related work section, namely methods that try to detect changing contexts/regimes, e.g. R-PCMCI [1].\n- The contribution of framing the problem in terms of a mixture model with multiple latent variables seems somewhat minor to me, also the proposed method seems straightforward for the problem and rests heavily on existing methods such as GES, EM, and TOPIC. This limits the novelty a bit, however I recognize that the application to the described causal discovery problem is novel.\n- In my opinion the introduction of the causal model in section 2 can be streamlined a bit. For example, I think it would help to express $\\beta$ as a function of the latent mixing variable (instead of allowing for general influence in the mechanism $f$ of $X_j$). This would show that this is the only point of influence for the latent. I also find it a bit confusing that you distinguish between the parent set of $X_j$ and the latent $La_j$: Isn't the mixing variable also a parent of some of the $X_j$s? Or do you define the parent set $Pa_j$ in line 88 as a subset of the observed variables?\n- Some of the concepts that not all readers are familiar with could be recapped/introduced in the main text, e.g. Adjusted Mutual Information, furthermore the notation $BIC_{\\hat{Z}}$ could be introduced more explicitly.\n- Typos:\n  - line 64 \"be extended the mixed setting\"\n  - line 80 resp.\n  - line 127 \"a frameworks\"\n  - line 180 distinguish instead of identify?\n\nQUESTIONS:\n- In the motivational example you provided (with the antibiotics), it is not a setting with multiple independent latent mixing variables, right? Can you give a real-world example where there are such independent variables? I think this would strengthen the motivation for this work.\n- In the description of the algorithm, what do you mean by the edge $Pa_j \\rightarrow X_j$? How does it relate to the formula in line 206? Do you mean the edge $X \\rightarrow Y$?\n\nLIMITATIONS:\nThere is no specific section addressing limitations. Some limitations are mentioned in the conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses statistical causal discovery under the assumption of unobserved subgroups. While prior methods have incorporated such assumptions, they typically rely on a single latent environment variable that affects all observed variables. In contrast, this work proposes a more realistic setting in which multiple latent mixing variables independently influence distinct subsets of observed variables. The authors present a novel method based on this assumption and demonstrate that it can be implemented as an extension of score-based causal discovery, specifically Greedy Equivalence Search (GES), using the Bayesian Information Criterion (BIC).\n\nThey show that while causal discovery is known to be non-identifiable under simple linear additive Gaussian noise models, the introduction of a mixture structure improves identifiability. The paper further proposes the use of the Expectation-Maximization (EM) algorithm for likelihood estimation within the mixture model, and argues, both intuitively and theoretically, that EM yields sufficient identifiability in practice.\n\nExperiments on simulated data demonstrate that the proposed method outperforms existing approaches in both recovering the mixture structure and the underlying causal graph. Additionally, evaluations on real-world cell signaling data show that the method achieves strong causal structure recovery performance.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThis paper presents a causal model based on a more realistic assumption in which multiple latent mixing variables independently influence different observed variables. The authors make a valuable theoretical contribution by demonstrating that this formulation can be addressed using an extension of score-based causal discovery with the Bayesian Information Criterion (BIC). This significantly enhances the practical applicability of statistical causal discovery methods. Moreover, the proposed method incorporates a practical inference procedure using the Expectation-Maximization (EM) algorithm, and the authors provide theoretical justification suggesting that EM yields sufficient identifiability in practice. The effectiveness of the proposed approach is convincingly demonstrated through both synthetic and real-world data.\n\nWeaknesses:\nWhile the assumption that each observed variable may be influenced by a distinct latent mixing variable is realistic and well-supported by both theoretical analysis and empirical evaluation, it comes at the cost of producing highly complex inferred causal structures. As a result, interpreting the estimated mixing variables in a meaningful and actionable way becomes challenging. In the context of causal inference, even when the structure is correctly recovered, models that lack interpretability tend to have limited value in practical applications. Addressing this issue, either by proposing a strategy for interpreting the inferred latent variables or by providing concrete examples from real-world datasets (e.g., the Sachs dataset) where the inferred mixing variables align with known experimental conditions or biological states, could greatly enhance the credibility and practical utility of the proposed method.\n\nQUESTIONS:\n1. Could the authors provide a discussion on how the inferred mixing variables in the Sachs dataset relate to known experimental conditions or biological states?\n\n2. Could the authors comment on possible directions for more general research into the interpretability of inferred mixing variables?\n\nLIMITATIONS:\nAs mentioned above, there is room for improvement in the discussion of the interpretability of the inferred mixing variables.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the problem of causal discovery in linear Gaussian structural equation models (SEMs) when the data consist of multiple subpopulations. Specifically, unlike previous work where data can typically be partitioned into multiple domains or environments, all observed data here are collected within a single environment. The authors first define the causal mixture model (CMM), where unobserved mixing variables $Z_i$ are introduced to index different linear coefficients or weights. Subsequently, they present identification results and algorithms. The authors show that, given an oracle maximum likelihood estimator, the structure can be uniquely identified using an adapted Bayesian Information Criterion (BIC) score. Furthermore, they propose an algorithm based on greedy equivalence search that employs the Expectation-Maximization (EM) algorithm to compute the BIC score. The effectiveness of the proposed method is demonstrated on both synthetic and real datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n1. The presentation of the paper is clear and easy to follow, although adding more explanations and examples may further aid in understanding the technical details.\n2. The problem is well-motivated and clearly defined.\n3. The authors provide a clear description of the background on greedy equivalence search (GES) and clearly state the limitations and conjectures.\n\n**Weakness**:\nThe presentation of certain technical details and theoretical results (e.g., Theorem 3.4) is difficult to understand (see the questions below).\n\nQUESTIONS:\n1. In Figure 2, what does the edge Z_1 -> X_1 mean? According to Equation (1), the mixing variable is introduced to distinguish the linear coefficients. Since $X_1$ is a source variable, I am not sure how different values of $Z_1$ would affect $X_1$.\n2. Do the identification results and the proposed algorithm rely on the prior assumption that all observed variables have equal noise variance? It is known that while linear additive Gaussian noise models are generally not identifiable, linear Gaussian models with equal noise variance are identifiable and can be identified using maximum likelihood estimation.\n3. Regarding Theorem 3.4:\n\n    (a) What does $Y$ represent? According to the notation in Section 2, $Y$ should be the parents of $X$. If it represents some type of \"outcome variable\", then I am unclear about the meaning of the conditional distribution $\\boldsymbol{X}|Y$.\n   \n   (b) In line 179, I recommend that the authors avoid using the term “Markov equivalence class” if the intent is to state that the models can be distinguished based on the observational distribution.\n\n    (c) Is Theorem B.2 the same as Theorem 3.4? If so, then it seems that Lemma 3.3 is derived from Theorem B.2 and should be considered as a corollary.\n\n4. In lines 210–212, how is the BIC score computed? Does it account for the unknown number of mixtures by employing the EM algorithm, or does it follow the conventional BIC formulation that assumes a single mixture component?\n\nLIMITATIONS:\nThe authors discuss the limitations of the proposed model and identification algorithm.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors consider the problem of performing causal discovery using observational datasets where samples come from multiple unknown distinct environments. They note that if samples come from heterogeneous subpopulations but this heterogeneity is ignored, applying traditional causal discovery methods may lead to spurious results. They observe that existing methods for addressing this setting, which use Gaussian Mixture Models (GMMs) to first split the dataset into estimated sub-populations, may lead to sub-optimal recovery of the subpopulations; in particular, GMMs may perform no-better than a random partition of the samples into subgroups. To address this issue, the authors propose Causal Mixture Modeling (CMM), which replaces the GMM-based subpopulation clustering with clustering based on a causal mixture of linear regressions (MLR).  \n\nThey consider a setting where (observed) random variables X have distribution defined by a linear function of (unknown) causal relationships, with coefficients determined by (unobserved) latent categorical variables. Conditioned on its causal ancestors, this means that X is distributed following an (MLR) model. These distributions can be associated with a DAG containing nodes for both observed variables X and latent variables Z; edges between observed variables with causal relationships; and latent variables -> observed variables. Latent variables are assumed to have no incoming edges (no observed variables or other latent variables have causal effects on latent variables). The target problem is then to discover the causal subgraph on the set of observed random variables X, as well as the set of latent variables Z and the set of edges from Z to X.\n\nOne fundamental obstacle to solving this problem is that computing the Maximum Likelihood Estimate (MLE) of an MLR model is NP-hard. The authors first show that access to some oracle for computing the MLE of the MLR model for any number of mixtures, existing score-based frameworks for causal discovery (in particular Greedy Equivalent Search (GES)) can be extended to handle the latent-variable setup described above, and that the consistency results for GES hold under this extension.\n\nTo derive a practical algorithm motivated by the above observation, the authors propose using Expectation Maximization (EM) to approximate the MLE. They note that under certain conditions, the EM algorithm can find the MLE for MLR models if initialized appropriately. They replace the oracle rule with the EM algorithm, and then use the extended score-based framework developed based on GES to discover a topological ordering over the nodes of the DAG corresponding to X’s. They call this framework Discover a Causal Mixture Model (CMM).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe authors do a good job of motivating the problem of causal discovery with unknown heterogeneous subpopulations, and they clearly point out why existing methods (based on GMMs) do not adequately address this setting.\nThe MLR model studied and set of assumptions made seem pretty typical within the field of causal discovery, and none seem too unrealistic.\n\nWeaknesses:\nThough the oracle-MLE method describes inherits consistency results from GES, it is my impression that the authors do not establish any guarantees for Algorithm 1. If my understanding is incorrect, please correct me. However I consider this a fairly minor weakness, as the connection to the oracle-rule version is made very clear in the paper, and there are strong computational barriers to guaranteeing good approximations to the MLE so the authors’ framework is ultimately a reasonable approximation to a consistent method.\n\nQUESTIONS:\nIs my understanding that the authors have not derived any guarantees on Algorithm 1 correct? If the components of the mixture are assumed to be well-separated, could the good-initialization regime for the EM algorithm described in lines 183-185 imply consistency for Algorithm 1, or would additional assumptions be necessary?\n\nIt is my understanding that the maximum number of latent variables Z identified by Algorithm 1 is capped at K_max, the user-specified maximum number of mixtures. Is this correct? Are there any heuristic rules that the authors would propose to help guide practitioners in selecting K_max and/or given multiple outputs from Algorithm 1 with different values of K_max, deciding which output is the most useful model of the data?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}