Title: Covariate-moderated Empirical Bayes Matrix Factorization
Venue: NeurIPS 2025
Accepted: yes
Keywords: matrix factorization

non-negative matrix factorization

variational inference

empirical Bayes

spatial transcriptomics
Abstract: Matrix factorization is a fundamental method in statistics and machine learning for inferring and summarizing structure in multivariate data. Modern data sets often come with "side information" of various forms (images, text, graphs) that can be leveraged to improve estimation of the underlying structure. However, existing methods that leverage side information are limited in the types of data they can incorporate, and they assume specific parametric models. Here, we introduce a novel method for this problem, *covariate-moderated empirical Bayes matrix factorization* (cEBMF). cEBMF is a modular framework that accepts any type of side information that is processable by a probabilistic model or a neural network. The cEBMF framework can accommodate different assumptions and constraints on the factors through the use of different priors, and it adapts these priors to the data. We demonstrate the benefits of cEBMF in simulations and in analyses of spatial transcriptomics and collaborative filtering data. A PyTorch-based implementation of cEBMF with flexible priors is available at https://github.com/william-denault/cebmf_torch.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Covariate-moderated Empirical Bayes Matrix Factorization (cEBMF), a general framework that extends EBMF by allowing covariate-dependent priors on latent factors. The key idea is to learn a mapping from covariates to priors via flexible models (e.g., neural networks), enabling the integration of high-dimensional side information. Empirical results demonstrate strong performance across synthetic and real-world datasets, with clear gains over competing methods and favorable scalability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. This paper introduces cEBMF, a general framework integrating any side information (text, images, graphs) via covariate-dependent priors. This surpasses existing methods limited to specific data types (e.g., spatial coordinates).\n2. This paper allows plug-and-play priors, enabling adaptation to diverse problems.\n3. The proposed method is supported by solid theoretical foundations.\n\nWeaknesses\n1. This method can be regarded as a two-stage approach: it first uses a neural network to process side information, treating it as a prior, and then performs matrix factorization. I think the paper lacks sufficient discussion on why an end-to-end neural network is not directly used to model all types of information and accomplish the target task.\n2. The deep learning baselines (e.g., VAE, NCF) are relatively simple and may not fully represent the state of the art. The paper would benefit from a more nuanced positioning.\n3. Lacks analysis of the experiment results — why the proposed linear model outperforms nonlinear models in recommendation tasks.\n\nQUESTIONS:\n1. Add experimental validation to demonstrate that the proposed method can leverage more complex side information, such as incorporating semantic and visual information of items in the task of collaborative filtering.\n2. Add discussion on why an end-to-end neural network is not directly used to model all types of information and accomplish the target task.\n3. Add some new baselines about deep learning.\n4. Add the discussion of the tasks that Matrix Factorization (MF) are suitable for.\n\nLIMITATIONS:\nEven with the introduction of side information, matrix factorization (MF) remains a linear model, which seems to limit its applicability.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces cEBMF, a flexible and modular framework for matrix factorization. The method integrates various types of \"side information\" by incorporating them into adaptive, covariate-dependent prior distributions. cEBMF learns these priors directly from the data, enhancing the inference and summarization of multivariate data structures. The benefits of cEBMF are demonstrated through simulations and applications to spatial transcriptomics and MovieLens datasets.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- cEMBF's modular and flexible design is attractive and the novelty of incorporating any type of side information is practical. In addition, the proposed algorithm tries to learn the covariate-dependent prior distribution representing the side information is inspiring.\n\n- The cEMBF framework generalizes different matrix factorization algorithms with different regularizations under the view of covariate-dependent priors. It is simple yet inspiring to see well-established algorithms like sparse NMF can also be viewed under this framework. The scalability of the framework is also promising. \n\n### Weaknesses\n\n- The performance of cEMBF on small dataset, large $K$ regime is unclear, yet it is a very common scenario in real application\n- The choice of the latent dimension is $K$ for all experimental setup, while given the flexibility claimed in cEMBF, one should be able to incorporate Dirichlet process prior so $K$ is learnable.\n- While the performance of many ELBO optimization procedures can indeed be sensitive to the choice of initialization, the provided paper content does not explicitly detail the specific initialization strategies used for the latent matrices (L and F) or the prior parameters $\\theta$.\n\nQUESTIONS:\n### Choice of $K$\n\nThe cEBMF framework demonstrates impressive flexibility in incorporating various prior assumptions and types of side information. However, the rank $K$ is treated as a fixed hyperparameter in the current experimental setup.\n  \nGiven the modular nature of cEBMF and its stated ability to accommodate different prior families, have the authors considered extending the framework with non-parametric Bayesian approaches, such as Dirichlet Process priors, to allow for automatic inference of the optimal rank $K$ ? What are the theoretical or practical challenges in implementing such an extension, and are there plans for future work in this direction?\n\n### Limited sample scenario with large $K$\n\nA common challenge in matrix factorization models is dealing with scenarios where the number of latent factors, $K$, is relatively large compared to the available data size.\n  \nWhat is the expected behavior and potential limitations of cEBMF in such \"small data, large $K$\" regimes? Specifically, since training the function $\\pi(\\cdot, \\cdot)$ (which determines the covariate-dependent prior probabilities) in a \"small data size, large K\" regime seems to present significant challenges despite its flexibility, how might the empirical Bayes nature and the covariate moderation mechanisms of cEBMF help to mitigate issues like overfitting or unstable parameter estimation in these challenging settings? \n\n### Optimization\n\nThe cEBMF learning algorithm relies on an iterative coordinate ascent approach to maximize the ELBO. The performance and convergence of such optimization procedures can often be sensitive to the initial values of the latent factors and prior parameters.\n  \nCould the authors provide more details on the initialization strategies employed for the latent matrices ($L$ and $F$) and the parameters of the covariate-moderated priors ($\\theta$​)? How do these choices impact the final results and convergence speed, particularly in the challenging scenarios presented in the experiments? Does the framework incorporate any specific robust initialization techniques or strategies to mitigate potential issues arising from poor initialization?\n\nLIMITATIONS:\nno issue", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a covariate-moderated empirical Bayes matrix factorization framework (cEBMF) for incorporating side information into matrix factorization. The authors first extend the EBMF model by allowing the priors over latent factor entries to depend on covariates (side information), enabling the model to adaptively learn sparsity or structure guided by auxiliary data. Then, a modular variational algorithm is proposed, where matrix factorization is learned by solving covariate-modulated empirical Bayes normal means (cEBNM) subproblems iteratively. The framework supports various prior families, including spike-and-slab for sparsity or non-negativity. Experiments on simulated data, MovieLens 100K data, and spatial transcriptomics data show that cEBMF achieves superior performance and scalability compared to existing methods like EBMF, MFAI, Spatial PCA, and deep learning baselines.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\ni) The paper presents a well-founded theoretical framework with thorough derivations and proofs. The proposed algorithms are clearly described and easy to follow.\nii) Visualizations are informative, with multiple figures that effectively illustrate the experimental results. Applications to MovieLens 100K and spatial transcriptomics are particularly compelling.\niii) The incorporation of covariates (side information) into the factor model is innovative, and the extension of EBMF is both well-motivated and systematically developed.\n\nWeaknesses\ni) The paper lacks an ablation study evaluating the impact of covariates or the choice of different prior families on model performance.\ni)) The distinction between “EBMF” and “EBNMF” is somewhat unclear and may be confusing for readers unfamiliar with the naming convention.\n\nQUESTIONS:\ni) While the authors included comparisons between cEBMF and EBMF, would it be possible to include an ablation study exploring different prior families? Additional visualizations showing performance variation under different priors would help.\nii) How robust is cEBMF to noisy or misspecified covariates? A sensitivity analysis or simulation study on covariate quality would be valuable.\niii) Have the authors considered methods for automatically inferring the number of latent components (K)? For example, a Bayesian nonparametric approach using a Dirichlet Process Mixture or similar mechanism might offer a flexible alternative.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces cEBMF, a framework that integrates side-information into Empirical Bayes Matrix Factorization by learning both spike-and-slab mixing weights and slab parameters as differentiable functions of covariates.  The authors demonstrate improved reconstruction accuracy over EBMF, MFAI, Spatial PCA, and VAE-based baselines on simulated data, MovieLens 100K, and spatial transcriptomics, and show scalability to up to one million rows.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* New Prior Modeling: developed a new covariate→prior-parameter mappings in an EBMF framework;\n\n* Modular Learning  Structure: Decomposes the overall inference into repeated calls to a cEBNM solver, enabling plug-and-play of different solver backends.\n\n* Empirical Validation across Domains: Provides thorough experiments on synthetic scenarios, recommendation data, and spatial omics, demonstrating consistent gains when side information is informative, and no degradation when it is not.\n\n## Weakness \n\n* By design, cEBMF cannot capture nonlinear interactions or higher-order effects; no comparison is made to deep nonlinear baselines (e.g. self-attention recommenders).\n\n* Omits comparisons to modern graph-based or attention-based methods (e.g. GC-MC, LightGCN, SASRec, BERT4Rec), which also integrate side information.\n\n* Requires pre-specifying a family for g_1 (Gaussian, Exponential, mixtures). When the chosen family mismatches data, performance may suffer; robustness to mis-specification is not studied.\n\n* No formal analysis of identifiability, convergence rates, or generalization bounds—particularly important when covariates weakly correlate with latent factors.\n\nQUESTIONS:\n* How about the robustness to prior mis-specification? I suggest to include an ablation showing performance when the true slab distribution differs from the chosen parametric family.\n\n* The paper instantiates the covariate-modulated prior using a specific parametric family. Can the authors characterize the representational capacity of that family—i.e. what classes of true slab distributions it can approximate well? Moreover, what would it take to replace this with a fully nonparametric density estimator (e.g. a normalizing flow or kernel density network)? I recognize that this adds optimization and identifiability challenges, but such an analysis could clarify the upper bound on prior flexibility and guide future nonparametric extensions.\n\nLIMITATIONS:\nThe author does not describe the limitation in the main part of the paper but mentioned several in the checklist.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes cEBMF, a modular empirical‑Bayes matrix‑factorization framework that can ingest side information (text, images, graphs, coordinates) via covariate‑dependent prior families.\nThe framework unifies and generalizes prior methods (EBMF, NMF, Spatial PCA, MFAI, CMF) as special cases through flexible prior families and automatic hyper‑parameter learning.\nThe authors derive a simple coordinate‑ascent algorithm that reduces the learning problem to repeated covariate‑moderated empirical‑Bayes normal‑means sub‑problems optimizing a variational approximation, enabling plug‑and‑play priors and scalability.\nIn experiments, state‑of‑the‑art accuracy is shown across simulations, collaborative filtering (MovieLens 100K) and spatial transcriptomics (human DLPFC), while outperforming EBMF, MFAI, Spatial PCA, PMD, VAE/cVAE, NCF.\nAn open‑source R/Python implementation is provided including runtime benchmarks where cEBMF scales markedly better than covariate-dependent alternatives MFAI and Spatial PCA.\n\nSTRENGTHS AND WEAKNESSES:\n(+) generalizes many existing matrix factorization methods as special cases\n(+) a computationally efficient implementation is provided\n(+) The paper is written with generality in mind\n(+) The experimental evaluation is extensive\n(-) due to generality it was somewhat work for me as a reader to figure out, which priors and models have actually been used. Sometimes instantiating special cases helps clarity.\n(-) compared to EBMF, the covariate-dependent cEBMF seems like an obvious extension of known pieces. One might even argue that it is a special case of EBMF where the prior family is parametrized using covariates.\n(-) no theoretical results are provided beyond derivations of the algorithm and methods.\n\nQUESTIONS:\nHow sensitive is the performance to the choice of covariate model?\nWhat guarantees (if any) do you have about convergence or posterior consistency in this setting?\nIs your method applicable in settings where covariates are high-dimensional or partially missing?\n\nLIMITATIONS:\nNo Limitations Section is provided. Arguably, some limitations are implicitly addressed, by describing the limited scope of the methods.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-13"}