Title: A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random
Venue: NeurIPS 2025
Accepted: yes
Keywords: Variable Selection

Finite Mixture Models

Model-Based Clustering

Missing Data

Asymptotic Selection Consistency
Abstract: Model-based clustering integrated with variable selection is a powerful tool for uncovering latent structures within complex data. However, its effectiveness is often hindered by challenges such as identifying relevant variables that define heterogeneous subgroups and handling data that are missing not at random, a prevalent issue in fields like transcriptomics. While several notable methods have been proposed to address these problems, they typically tackle each issue in isolation, thereby limiting their flexibility and adaptability. This paper introduces a unified framework designed to address these challenges simultaneously. Our approach incorporates a data-driven penalty matrix into penalized clustering to enable more flexible variable selection, along with a mechanism that explicitly models the relationship between missingness and latent class membership. We demonstrate that, under certain regularity conditions, the proposed framework achieves both asymptotic consistency and selection consistency, even in the presence of missing data. This unified strategy significantly enhances the capability and efficiency of model-based clustering, advancing methodologies for identifying informative variables that define homogeneous subgroups in the presence of complex missing data patterns. The performance of the framework, including its computational efficiency, is evaluated through simulations and demonstrated using both synthetic and real-world transcriptomic datasets.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a model for variable selection in finite Gaussian mixtures. The basic idea is to partition the data vector into variables that are relevant for the clustering, variables that are linear combinations of the variables relevant to the clustering, and then irrelevant variables. To accomodate missing not at random variables, the authors append a missingness mechanism to the finite mixture model, which they call MNARz, that allows for missingness probabilities to vary within a latent class. They then use a penalized likelihood function to perform simultaneous parameter estimation and variable selection.\n\nThe authors apply their model for clustering, variable selection, and parameter inference in a simulation study, where the method demonstrates compentitive imputation accuracy and clustering performance. In their real data application, they use their model to analyze transcriptome data, where they determine scientifically relevant subgroups in the data based on gene expressions.\n\nSTRENGTHS AND WEAKNESSES:\nI think the strength of this paper is in the ability of the SRUW model to be expressed as a straightforward finite Gaussian mixture, with component-wise mean vector and covariance matrix expressed in terms of the relevancy partition that the authors originally present. As the authors mention, this enables theoretical analyses and relatively simple E-M algorithms for estimation. In addition, the idea that clustering is driven by a few of the study variables, and seeking to select these variables to provide interpretable insights, is quite attractive. Extending these ideas to MNAR study variables is quite challenging.\n\nQUESTIONS:\n1) Estimation and selection: I am quite confused about how variables are simultaneously selected and the finite GMM is estimated. The paper would benefit from clearer and higher-level algorithms describing estimation and selection. From the Finite GMM representation of the SRUW model, does a user need first to declare which variables they believe are relevant/irrelevant to the clustering? When/Where is the penalty applied? There is some mention of a ranking step followed by a parameter estimation step but this is not clear in the paper or manuscript. Where/when do the penalties come into play?\n\n2) MNARz mechanism: I have some serious doubts that the proposed MNARz model provides a missingness mechnism that is nonignorable. This needs to be clarified. For example the authors state that the missignness mechanism within any class factors as:\n\np(c_{n} \\mid z_{n_k}, \\psi_{k}) = \\prod_{d=1}^{D}\\rho(\\psi_{k})^{c_{nd}}(1-\\rho(\\psi_{k})^{1-c_{n})\n\nFor the missingness to be MNAR, missingness has to depend on unobserved missing values. In this case, this would require \\psi_{k} to depend on y_{d}, but this is not clear from how the model is presented. Also, the way this reads is that \\psi_{k} is the same for each variable y_d in latent class k. Is this correct? This seems highly improbably and is not a MNAR missigness mechanism. This needs to be clarified or modified if the authors claim their methods are useful for MNAR data. \n\n\n3) The distinction between the different \\pi_{k}s in (3) is not clear.\n\nLIMITATIONS:\n1) In general, the paper is heavy with notation that is difficult to follow\n2) Following my above questions:\n- I think there should be some outline for how model estimation and variable selection is conducted. The E-M algorithm presented in the main text only focuses on estimating the parameters in the joint model for the missingness and study variables for a fixed set of pertinent/irrelevant variables, unless I am missing something\n-I think the authors should revisit the MNARz missingness mechanism. In general, a variable is nonignorably missing if the probability of missingness depends on the underlying missing values. I don't see how this is encoded in the MNARz variables\n-Related, I also searched in the supplement for how the missingness mask is realized but couldn't find anything in the text or supplement. Is the data truly MNAR in the author's simulated experiments?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 1, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a unified framework for variable selection in model-based clustering when data contain Missing Not at Random (MNAR) entries. The key contribution is the development of a penalized likelihood approach using a joint model for both the data and the missingn mechanism. The authors propose an efficient EM algorithm for estimation and selection and demonstrate the method’s effectiveness through simulation studies and real data applications.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe paper focuses on variable selection in model-based clustering under MNAR, which is rarely addressed. The proposed method is computationally efficient and theoretically well-justified, relying on an EM-type iterative procedure specifically adapted to handle MNAR data structures. The experimental design is thoughtful and comprehensive, effectively demonstrating the superior performance of the proposed method across various missing scenarios. Furthermore, the consistency of the variable selection procedure is rigorously established under suitable regularity conditions.\n\nWeaknesses:\n\n1. One of my primary concerns regarding this manuscript is the level of novelty, as the proposed method appears to be an extension of [1] and [2]. The authors need to justify the novelty of the proposed method compared to these works, from either methodological or theoretical perspective. Therefore, I hold a conservative opinion toward the significance of the contribution made by this manuscript.\n2. The paper assumes that the missing mechanism follows parametric regression model. This can be restrictive in complex practical cases.\n3. There is no numerical studies or test of model misspecification for missing mechanism models.\n4. The method is currently limited to continuous data modelled by Gaussian mixtures. Many practical applications (e.g., survey or electronic health record data) involve categorical or mixed-type variables. This limitation is briefly acknowledged but not substantively addressed.\n5. While the paper claims that the proposed EM-based framework is computationally feasible, this claim is not supported by quantitative analysis or complexity evaluation. This is a notable limitation, particularly given the well-documented computational challenges associated with EM algorithms in high-dimensional settings or when the number of clusters is large.\n6. This paperdoes not report runtime, memory usage, or scalability evaluations, making it difficult to assess the method’s practicality on large-scale datasets. Including a detailed assessment of computational performance, such as runtime comparisons across competing methods, scalability experiments with increasing dimensionality and sample size, or at least a discussion of algorithmic complexity, would substantially enhance the empirical contribution.\n7. The method depends on several hyper-parameters (e.g., $\\lambda$, $\\rho$, and the stopping rule parameter $c$), yet no sensitivity analysis or guidance is provided for their selection.\n\n[1] Maugis-Rabusseau, C., Martin-Magniette, M. L., & Pelletier, S. (2012). SelvarClustMV: Variable selection approach in model-based clustering allowing for missing values. Journal de la Société Française de Statistique, 153(2), 21-36.\n\n[2] Celeux, G., Maugis-Rabusseau, C., & Sedki, M. (2019). Variable selection in model-based clustering and discriminant analysis with a regularization approach. Advances in Data Analysis and Classification, 13, 259-278.\n\nQUESTIONS:\nPlease see the Weaknesses.\n\nLIMITATIONS:\nPlease see the Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a variable selection framework in model-based clustering that handles data that is missing not at random (MNAR). The paper extends the SRUW model used in variable selection for model-based clustering in high-dimensional settings using the missing not at random model called MNARz which takes into account which class the data is coming from. They propose an EM style algorithm combined with a transformation of MNARz to MAR. The paper shows that under specific assumptions the proposed framework maximizes identifiability and consistency of the SURW framework and with high probability selects a consistent set of relevant features. They also experimentally show that the methods modified with MNARz handles missing not at random better than the standard methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The paper explores the variable selection in model-based clustering. They introduce a method that extends SRUW model to handle missing not at random data using the MNARz method. They show that the method they introduce satisfies identifiability and consistency under the SRUW model and with high probability, can extract a set of reliable and consistent features.\n\n2. The algorithm is generally easy to follow and uses an EM based approach. The overall organization of the paper and the arguments are clear. The paper is overall well-written.\n\n3. The paper shows the practicality of the MNARz based approaches and shows experimentally how they do against standard methods.\n\nWeaknesses\n\n1. It is unclear how the computational complexity of the algorithm scales with the parameters such as $D,N$ and $K$. \n\n2. The conditions for the correctness of the algorithm are mentioned in the supplementary material. It might be better to have at least some informal statement about them in the main body because they would be important in ascertaining the validity of the theorems.\n\nQUESTIONS:\n1. How dependent is the framework on the MNARz scheme? If the missingness is slightly different from MNARz (maybe for instance if it also relies on some other parameter as well), does that break things? Does it break things in practice?\n\n2. How do you initialize the EM algorithm? Is a random starting point good enough or do you need to prime it in a specific way?\n\n3. How does the computation scale with parameters such as $D,N$ and $K$? How scalable is the algorithm? What is the dependency of the algorithm's complexity on the degree of missingness?\n\nLIMITATIONS:\nThe paper highlights the limitations such as the restriction to Gaussian mixture models in continuous setting, in the discussion.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel unified framework for model-based clustering that effectively tackles both variable selection and missing data (specifically, MNAR) simultaneously. By integrating a data-driven penalty and explicitly modeling missingness with latent class membership, the proposed method, estimated via an extended EM algorithm, demonstrates theoretical guarantees of consistency and empirical improvements in identifying informative variables and clustering complex datasets.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The paper integrates two significant challenges in model-based clustering: variable selection and handling MNAR data. This unified approach is a notable advancement, as most existing methods typically address these issues in isolation.\n- The framework provides strong theoretical backing, including proofs for identifiability, asymptotic consistency, and selection consistency, even in the presence of missing data.\n\n### Weaknesses\n\n- As acknowledged by the authors, the current framework is limited to continuous data, specifically via Gaussian mixture models. This restricts its applicability to categorical or mixed-type variables, which are common in many real-world datasets.\n- While powerful for targeted MNARz scenarios, the framework currently does not offer an explicit mechanism for automatically identifying underlying missing data patterns in mixed real-world settings. This aspect might be a practical consideration for its broader application in highly heterogeneous missingness scenarios.\n- In the synthetic experiments the MAR and MNAR missingness regimes are simulated separately — not simultaneously. For practical usage, it would be highly valuable to see empirical results on synthetic datasets that deliberately incorporate a blend of different types or complexities of MNAR patterns. This would offer crucial insights into the algorithm's performance and robustness when facing the nuanced and heterogeneous MNAR scenarios commonly found in real-world applications.\n\nQUESTIONS:\n### Mixture of missing type\n\nWhile the proposed framework demonstrates strong performance under the assumed MNARz mechanism, its applicability to real-world datasets often characterized by an unknown mixture of MCAR, MAR, and diverse MNAR patterns could be further clarified. To enhance the paper's comprehensiveness and provide additional insights for practitioners, it would be highly beneficial to include empirical results on synthetic datasets that deliberately incorporate a blend of these various missing data patterns. Such an analysis would offer valuable understanding into the algorithm's robustness and generalizability beyond a single assumed missing data mechanism.\n  \n### Conclusion from Figure 1\nThe authors summarize the first synthetic experiment as \"Its performance declines only modestly as missingness increases, while the impute-then-cluster baselines deteriorate sharply,\" The figure is quite small but it looks like all four methods deteriorate in the same way, yet the proposed algorithm is definitely has more tolerance on missing ratio. It would be great to enlarge this figure and ensure clearer differentiation between the performance curves. Some statistical tests to quantitatively supporting the claim is welcomed to report.\n  \n### Decoupled versus joint approaches\nThe proposed framework admirably integrates variable selection with missing not at random (MNAR) modeling by extending the SRUW framework under the MNARz assumption. However, since the core innovation builds upon the MNARz mechanism, it would significantly strengthen the empirical contribution if the authors compared their method against a more explicit baseline that combines vanilla MNARz modeling with standard variable selection applied to fully observed data. Specifically, I encourage the authors to construct a hybrid baseline that uses:\n\n- MNARz modeling (e.g., a GMM with cluster-dependent missingness probabilities as in [59]) for handling missingness,\n- Followed by variable selection methods (e.g., penalized likelihood via adaptive LASSO or SRUW role assignment) applied to the internally imputed or completed data.\n\nThis would help isolate the gain provided by joint modeling of selection and missingness versus decoupled approaches. Even a simplified version of such a pipeline would provide valuable insight into whether the proposed method’s improvements stem from its structural innovations or primarily from leveraging MNARz assumptions.\n\t\nAsking for comparison with vanilla MNARz + variable selection isolates whether joint modeling (i.e., their contribution) gives significant advantage over a two-step pipeline.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}