Title: DO-EM: Density Operator Expectation Maximization
Venue: NeurIPS 2025
Accepted: no
Keywords: Density Operators

Expectation-Maximization

Quantum Unsupervised Learning

Latent Variable Models
Abstract: Density operators, quantum generalizations of probability distributions, are gaining prominence in machine learning due to their foundational role in quantum computing. Generative modeling based on density operator models (**DOMs**) is an emerging field, but existing training algorithms - such as those for the Quantum Boltzmann Machine - do not scale to real-world data, such as the MNIST dataset. The Expectation-Maximization algorithm has played a fundamental role in enabling scalable training of probabilistic latent variable models on real-world datasets. *In this paper, we develop an Expectation-Maximization framework to learn latent variable models defined through **DOMs** on classical hardware, with resources comparable to those used for probabilistic models, while scaling to real-world data.* However, designing such an algorithm is nontrivial due to the absence of a well-defined quantum analogue to conditional probability, which complicates the Expectation step. To overcome this, we reformulate the Expectation step as a quantum information projection (QIP) problem and show that the Petz Recovery Map provides a solution under sufficient conditions. Using this formulation, we introduce the Density Operator Expectation Maximization (DO-EM) algorithm - an iterative Minorant-Maximization procedure that optimizes a quantum evidence lower bound. We show that the **DO-EM** algorithm ensures non-decreasing log-likelihood across iterations for a broad class of models. Finally, we present Quantum Interleaved Deep Boltzmann Machines (**QiDBMs**), a **DOM** that can be trained with the same resources as a DBM. When trained with **DO-EM** under Contrastive Divergence, a **QiDBM** outperforms larger classical DBMs in image generation on the MNIST dataset, achieving a 40–60% reduction in the Fréchet Inception Distance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies density operator models, which existing training algorithms can't scale to real data. The paper considers a new training algorithm, called Density Operator Expectation Maximization (DO-EM), which overcomes some existing issues by formulating the expectation set as QIP in a novel way. The author also proposes a new generative modeling algorithm, Quantum Interleaved Deep Boltzmann Machines (QiDBMs), and empirically evaluate it MNIST data.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n1. The presented algorithm is novel.\n2. The proof and derivation details appear to be technically sound (although I did not thoroughly verify them).\n3. The experiment includes real datasets to demonstrate practical relevance. \n\n\n**Weakness**\n1. The paper misses a detailed literature review related to quantum generative modeling algorithms, for example [1,2,3,4], and as well as those classical algorithms for generating quantum data, for example [5,6]. I believe the list is not exhaustive, as this is a fast-growing field. Presenting a clear overview of the literature and explaining in detail the advantages of the proposed algorithm compared to existing ones is essential for readers to understand the contribution and significance of this paper. \n\n2.  Missing important benchmarks in empirical validation. The experiment section only includes comparison with DBM, while potentially missing other quantum generative modeling algorithms. If these algorithms are not suitable for the task, a paragraph should at least be included to explain the reason.\n\n3. Minor practical relevance. The paper only experiments on MNIST, while the best result only achieves an FID of 20. From a practitioner's perspective, this suggests a highly inferior performance compared to those classical generative modeling algorithms. A clear explanation of why the proposed quantum algorithms should be favored over these classical algorithms should be added to the introduction to better motivate the work. Moreover, if the experiments can be conducted on CIFAR-10 or an extremely downscaled version of ImageNet (e.g., ImageNet-32 or even smaller), the quality of the paper will be greatly enhanced.\n\n\n**References**\n\n[1] Dallaire-Demers, Pierre-Luc, and Nathan Killoran. \"Quantum generative adversarial networks.\" Physical Review A 98.1 (2018): 012324.\n\n[2] Zhang, Bingzhi, et al. \"Generative quantum machine learning via denoising diffusion probabilistic models.\" Physical Review Letters 132.10 (2024): 100602.\n\n[3] Riofrio, Carlos A., et al. \"A characterization of quantum generative models.\" ACM Transactions on Quantum Computing 5.2 (2024): 1-34.\n\n[4] Kwun, Gino, Bingzhi Zhang, and Quntao Zhuang. \"Mixed-state quantum denoising diffusion probabilistic model.\" Physical Review A 111.3 (2025): 032610.\n\n[5] Zhu, Yuchen, et al. \"Quantum state generation with structure-preserving diffusion model.\" arXiv preprint arXiv:2404.06336 (2024).\n\n[6] Hibat-Allah, Mohamed, et al. \"A framework for demonstrating practical quantum advantage: comparing quantum against classical generative models.\" Communications Physics 7.1 (2024): 68.\n\nQUESTIONS:\nMost of my concerns are raised in the weakness section. However, I do have some more questions that could use some clarification, which I have listed below. \n\n1. Relationship between terms. I am a bit confused about the relationship between DO-LVM, CQ-LVM, and QiDBM, and it would be great if you could elaborate more on that. \n\n2. Similarly, what's the connection between the proposed DO-EM and the contrastive divergence algorithm used in practice? Are they the same thing when applied to QiDBM?\n\n3. Comparison with other quantum generative modeling algorithms. I understand that benchmarking on MNIST might be a standard practice in the community at this current stage of progress. However, to show that the proposed algorithm has an advantage, it should at least be compared to other quantum ML algorithms on the same task. If this can't be done, what's the major difficulty that prevents you from doing so?\n\nI am happy to increase my recommendation for the work if the experiments section is enhanced to demonstrate the superiority of the proposed method, and the related work section is added for better readability.\n\nLIMITATIONS:\nyes, limitations are addressed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a quantum inspired EM algorithm for density operators that replaces the classical E step by a quantum relative entropy projection step. \n\nThe algorithm is placed in a DO-LVM (density operator latent variable model) framework that generalizes classical latent variable models to density operators using partial traces to separate the observable and latent variables. This is a natural definition and leads also to quantum analogs of RBMs. However, these models are harder to train classically due to the novel quantum relative entropy projection and their advantage is not clear priori. Experiments to determine the advantage of the classical models are performed and improvements in quality are observed with respect to a specific metric. \n\nA quantum realization of the EM algorithm is proposed in principle, but the details are not fully worked out, so it unclear if this is a viable algorithm on a quantum computer.\n\nSTRENGTHS AND WEAKNESSES:\n- The idea for implementing the E step using quantum information projections and relating to the Petz recovery map is novel and goes beyond the existing QML literature on clustering and latent variable models.  \n\nThere has been previous work on quantum expectation maximization (arXiv:1908.06657) but progress has been limited as it quantizes the classical subroutines used in the classical EM algorithm for Gaussian mixtures. The present work however does not give an end-to-end quantum algorithm but introduces a novelty in the E-step. \n\n- The DO-LVM framework is a natural quantum framework generalizing classical LVMs and investigating advantages it may offer is worthy of further investigation. However, it is harder to train than the classical LVMs and the possible advantages are in qualtity and scalability. The experiments report a significantly improved Frechet Inception distance (FID) for the classical models (CQ-LVM) over Boltzmann machines, this is a metric that shows that the images being generated are more 'realistic' and is thus a measure of improved quality. It is not clear however, if the FID is competitive with state of the art image generators. \n\n- The end-to-end quantum EM algorithm needs to be specified in greater detail, specially the input model and the implementation of the quantum E step with SVT. A resource analysis should be carried out to determine how well this scales with qubits and circuit size to see if this is indeed a viable method for early quantum computers. \n\nThe paper introduces novel ideas that quantize one of the steps in the EM algorithm in a non trivial way and is therefore worthy of serious consideration. However, some parts need to be worked out further or explained better so I am going with a weak accept.\n\nQUESTIONS:\n- Line 172: Is it Theorem 2.2 being proved or Lemma 4.1? \n\n- It would be helpful to have more explanations on the new ideas introduced in this work like ELBO, minorant maximization and Petz recovery map in the main body of the paper. Most of the new ideas and proofs are found in the appendix due to space constraints, but I would suggest that  a sketch of the novel ideas in the E step and brief proof sketches should be added if the paper is accepted.\n\nLIMITATIONS:\nDiscussed adequately.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
The authors propose Density Operator Expectation Maximization, an expectation maximization algorithm for latent variable models. Empirically, they demonstrate their idea on the MINIST dataset, showing clear performance improvements. The results imply the better scalability of the method on Boltzmann quantum operators.

STRENGTHS AND WEAKNESSES:
Strengths: The paper has provided theoretical analysis of the algorithm. The empirical experiments on MNIST show large performance gain.

Weakness:
- My main concern of the paper is its accessibility to the audience. With the overuse of shorthand notations, the paper sometimes is difficult to follow.

- Line 87: I am not sure why "this can only discover local minima". This claim seems to be disconnected from the context above and not discussed clearly.

- Algorithm 1: It would be better if the authors can provide comments for each step, and the notations are very confusing. I am not sure what does it mean to have the target operator $\eta_V$ as the input.

- Contribution 3: why name this as CQ?

- Why do we need the contrastive divergence for the training?

- Figure 1: Why we cannot have the consistent name of the proposed method? 

- Experiments on MNIST: Why each image correspond to 6272 binary units?

- Discussion: Could we use consistent notations? For example, you can keep the capitalization of the latent variable models or not. But they should be consistent.

QUESTIONS:
I am not sure how the proposed algorithm is closely aligned with latent variable models from the algorithm block.

LIMITATIONS:
No
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript introduces a novel Density Operator Expectation-Maximization (DO-EM) algorithm aimed at training Density Operator Latent Variable Models (DO-LVMs) on classical data using quantum-inspired methods. The core contribution lies in developing an Expectation-Maximization framework based on Quantum Information Projection (QIP), overcoming challenges in classical EM methods by addressing the absence of well-defined quantum conditional probabilities. The authors present an iterative procedure that ensures non-decreasing log-likelihood across iterations, and demonstrate the practical application of their method using Quantum Interleaved Deep Boltzmann Machines (QiDBMs), a quantum analog of classical Deep Boltzmann Machines (DBMs). Empirical results on the MNIST dataset show that QiDBMs outperform classical DBMs in terms of Fréchet Inception Distance (FID).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper develops a quantum-inspired Expectation-Maximization algorithm that scales to real-world data. The introduction of Quantum Information Projection is a novel and effective approach.\n- The paper provides a rigorous theoretical analysis, contributing to a solid foundation for the proposed algorithm.\n\nWeakness:\n- The manuscript presents several theorems and lemmas related to the DO-EM algorithm, but these are not sufficiently explained or contextualized. As a reader with limited expertise in this field, it is difficult to fully grasp the purpose and implications of these theoretical results. \n- The experimental evaluation is somewhat limited. Currently, the authors only provide results on the MNIST dataset, which may not be enough to demonstrate the general effectiveness of the proposed approach across a variety of tasks and datasets.\n\nQUESTIONS:\nThe questions are included in the weakness.\n\nLIMITATIONS:\nThe limitations are included in the weakness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}