No MCMC Teaching For me: Learning Energy-Based Models via Diffusion Synergy

Authors: Shanchao Yang, WU Yanrui, Yidong Ouyang, Baoxiang Wang, Hongyuan Zha
Primary area: generative models
Keywords: energy-based models, generative modeling, sampling, diffusion models

Abstract:
Markov chain Monte Carlo (MCMC) sampling-based maximum likelihood estimation is a standard approach for training Energy-Based Models (EBMs). However, its effectiveness and training stability in high-dimensional settings remain thorny issues due to challenges like mode collapse and slow mixing of MCMC.
To address these limitations, we introduce a novel MCMC teaching-free learning framework that jointly trains an EBM and a diffusion-based generative model, leveraging the variational formulation of divergence between time-reversed diffusion paths. In each iteration, the generator model is trained to align with both the empirical data distribution and the current EBM, bypassing the need for biased MCMC sampling. The EBM is then updated by maximizing the likelihood of the synthesized examples generated through a diffusion generative process that more accurately reflects the EBM’s distribution. Moreover, we propose a novel objective function that further improves EBM learning by minimizing the discrepancy between the EBM and the generative model. Our proposed approach enhances training efficiency and overcomes key challenges associated with traditional MCMC-based methods. Experimental results on generative modeling and likelihood estimation demonstrate the superior performance of our method.

Decision: reject
Reviewer scores: 3, 3, 3

Reviews and discussion:
<thread>
creation date:
1730396363072

thread_type:
official_review

<thread_object>
creator: reviewer_xpjm

<content>
summary:
The paper introduces a method for training Energy-Based Models (EBMs) without relying on Markov Chain Monte Carlo (MCMC). In each training step, a diffusion-based sampler is learned to match the current EBM and data distribution. This sampler is then used to generate samples, enabling maximum likelihood training of the EBM. Experimental results on synthetic toy data demonstrate the method's effectiveness.

strengths:
The proposed method eliminates the need for MCMC. While it involves training an additional diffusion-based sampler, it avoids the bias issues associated with MCMC, provided the sampler is well-trained.

weaknesses:
- The proposed method is evaluated solely on 2D synthetic data. Testing it on high-dimensional datasets, such as images, would help assess its scalability.
- There are some missing baselines:
    - Variational Inference: [1] propose to estimate the partition function using variational inference, which is also MCMC-free
    - Noise Contrastive Estimation (NCE) [2]. NCE is MCMC-free and can work very well on 2d density estimation.
    - Energy Discrepancy (ED) [3] is a recently introduced method for training EBMs without MCMC. It offers compelling theoretical guarantees and has demonstrated effectiveness in tasks like density estimation and image modelling.

[1] Duvenaud D, Kelly J, Swersky K, Hashemi M, Norouzi M, Grathwohl W. No MCMC for me: Amortized samplers for fast and stable training of energy-based models. InInternational Conference on Learning Representations (ICLR) 2021.

[2] Gutmann, Michael, and Aapo Hyvärinen. "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." *Proceedings of the thirteenth international conference on artificial intelligence and statistics*. JMLR Workshop and Conference Proceedings, 2010.

[3] Schröder, Tobias, et al. "Energy discrepancies: a score-independent loss for energy-based models." *Advances in Neural Information Processing Systems* 36 (2024).

questions:
- The sampler is trained using a loss function designed to align the data distribution with the current EBM. While this approach is unbiased when the EBM is well-trained, it can lead to a biased maximum likelihood estimator if the EBM is underfitting, which is common in the early stages of training. It would be great to see how it works without the DSM loss in sampler training.
- The EBM is trained to match the data distribution and the current sampler. It would also be valuable to see the results when the sampler matching loss is omitted during EBM training.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730651800723

thread_type:
official_review

<thread_object>
creator: reviewer_7qot

<content>
summary:
The authors propose to replace the traditional MCMC sampling for learning energy-based models (EBMs) with sampling from diffusion models. Generation speed and sample quality are major bottlenecks in learning EBMs, and the experiments show part of those problems are addressed. The used sampling method from EBMs is not novel, as it follows the method from recent work by Richter & Berner (2024). While the proposed method is straightforward and reasonable, its contribution is incremental.

strengths:
The proposed method is reasonable.

weaknesses:
The contribution is not significant. It merely incrementally extends the published sampling method to learning EBMs. If the authors could address a major challenge in applications using the diffusion sampling, the contribution would be more noteworthy.

Minor comment:
Although the equations (8) through (11) were borrowed from previous literature, the authors have to explain those equations in their own words. The provided explanation regarding the diffusion sampling from previous work does not clarify why the proposed sampling should be better than MCMC.

questions:
The paper is clearly written.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730728876044

thread_type:
official_review

<thread_object>
creator: reviewer_p1jc

<content>
summary:
This paper presents a novel approach to training Energy-Based Models (EBMs) without relying on Markov Chain Monte Carlo (MCMC) methods, which are traditionally used but can be unstable and biased in high-dimensional settings. The proposed method, referred to as DiffEBM, employs a diffusion-based framework that trains an EBM and a diffusion model simultaneously, effectively eliminating the need for MCMC by leveraging divergence in time-reversed diffusion paths.

The paper identifies core limitations of MCMC, such as mode collapse and slow mixing, which hinder EBM training. To address these, DiffEBM introduces an objective function to match the EBM and diffusion model, using samples from the latter as unbiased approximations of the data distribution, sidestepping the biases associated with short-run MCMC. The diffusion model is trained using the technique proposed in [Richter & Berner, 2024]. In contrast, the EBM is updated based on synthesized data generated by the diffusion model. 

Experimentally, DiffEBM demonstrates superior performance on various benchmarks, including Gaussian mixture datasets and synthetic data distributions like 2Spirals and Swissroll. Performance is evaluated using Sinkhorn distance to compare generated samples to ground-truth distributions.

In summary, DiffEBM introduces a diffusion-driven training framework for EBMs that enhances efficiency, stability, and sample fidelity by removing MCMC-based sampling, thus providing an alternative pathway for EBM training in complex generative tasks​

strengths:
The paper's primary strength lies in its innovative approach to training Energy-Based Models (EBMs) without the reliance on Markov Chain Monte Carlo (MCMC) methods, which have known limitations in high-dimensional contexts. Traditional MCMC-based EBM training often suffers from mode collapse, slow mixing, and biased samples, especially with short-run MCMC. By introducing a diffusion-based generative model that jointly trains with the EBM, the authors successfully bypass these challenges. This joint training, which uses divergence between time-reversed diffusion paths as an objective function, eliminates the need for MCMC teaching. As a result, DiffEBM achieves higher sample quality by aligning the generative model directly with the EBM’s learned distribution, making it a valid alternative to MCMC-based methods.

weaknesses:
The method proposed in this paper, while innovative, introduces significant computational demands that undermine its practical efficiency. The core idea—training an EBM in tandem with a diffusion-based generative model to avoid the pitfalls of MCMC sampling—replaces the complexity of MCMC with an equally demanding requirement: learning a second, paired generative model that must be iteratively updated alongside the EBM. This approach involves repeatedly sampling from the diffusion model during each training step, as highlighted in Algorithm 1, line 223, where a full sequence of diffusion sampling is performed at each iteration. This reliance on diffusion sampling makes the process computationally intensive, as each update to the EBM requires a costly simulation of the diffusion process to produce high-fidelity samples, compounding the training time considerably. Moreover, the iterative nature of sampling across the full diffusion chain can easily lead to instability, especially if the parameters of the generative model diverge from the EBM, creating an oscillating learning dynamic that may fail to converge.

Another key issue arises from the purpose of training the EBM when the diffusion model, a high-capacity generative framework in its own right, is already optimized to produce accurate samples. If the diffusion model alone can capture the empirical data distribution effectively, as evidenced in the quality of generated samples, the rationale for learning an additional EBM becomes questionable. The diffusion model could theoretically fulfill the generative modeling objective by itself, rendering the EBM redundant for many practical applications. Training both models in parallel may not yield substantial benefits over simply using the diffusion model, especially given the EBM’s limited advantage in scenarios where the diffusion model is already well-aligned with the data distribution. Thus, while the framework’s goal is to leverage the EBM’s interpretability and robustness in capturing complex energy landscapes, the computational cost and redundancy associated with dual-model training suggest a misalignment between the theoretical motivation and the efficiency of the method.

Another limitation is the lack of direct comparison with standalone diffusion-based generative models, which would offer a fairer baseline for evaluating the proposed approach. Since the method relies heavily on a diffusion model, comparing it against established diffusion-only schemes—or even against samples generated solely by its own diffusion model—would help clarify whether the added complexity of training an EBM provides real benefits. Without such comparisons, it’s uncertain if the dual-model approach improves performance significantly over simpler, diffusion-based methods alone, potentially overestimating its effectiveness. 

Finally, in my opinion the considered datasets are too simplistic to claim that the proposed method really has superior performance compared to other schemes.

questions:
Given the substantial computational load and potential instability introduced by training an EBM alongside a diffusion model, have you considered alternative strategies to reduce the computational demands, such as truncated or approximate diffusion sampling, without compromising sample quality?

Since your approach integrates a high-capacity diffusion model, could you clarify the unique advantages of training an EBM in tandem? Specifically, how does the EBM contribute to the overall performance compared to using the diffusion model alone for generative tasks?

To better understand the value of the dual-model approach, would you consider evaluating your method on more complex datasets and comparing it directly against standalone diffusion-based generative models, as well as using samples from your diffusion model as a self-baseline? This would help clarify any performance gains provided by the EBM, particularly on challenging, high-dimensional data where diffusion-only methods may already perform well.

Can the authors clarify what they mean in the experimental section when they refer to Denoising Score Matching? What is the relationship with the sliced score matching mentioned in Figure 3? 

Also, a minor point, why in Fig 3 are the ground truth samples different for the two methodologies?
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a method for training energy based models without using MCMC. While the problem the paper addresses is very relevant and important, all three reviewers agreed that the paper should be rejected from ICLR. The computational demands that it introduces works to undermine its practical efficiency, which is a criticism of MCMC to being with. Also noted was that higher dimensional examples such as images would help make the method more compelling.