Paper: Perturb and Learn: Energy-Based Modelling in Discrete Spaces without MCMC (ICLR-2024)

Reviewer comment (Experiments):
All of the proposed perturbation methods and all of the experiments are only dealing with EBMs with binary variables. As a result, presenting the proposed method as an approach to learn discrete EBMs seems to be overclaiming. Moreover, it seems to me the study of this approach when applied to EBMs with discrete variables having more than two states should be within scope of this paper, as this is a very natural and straightforward extension but there might be new issues arising when applied to discrete EBMs with non-binary variables (e.g. the mean-pooling perturbation depends on the parametrizations of the variable states and this can impact the performance of such perturbations).

Author response:
Thank you for bringing this to our attention. We will make the necessary revisions in the paper and explicitly state our focus on binary EBMs. We would like to emphasise that our proposed methodology can, in principle, be extended to other types of discrete data sets such as spaces of the form $\{0, … K-1\}^d$ like coloured images or text. In this case, the Bernoulli perturbation may be replaced by a different Markov transition density that moves between states in the space $\{0, … K-1\}^d$. Similarly, the mean-pooling transform can be extended to this setting where one picks certain dimensions to be averaged, and the neighbourhood-based perturbation works as presented if an appropriate neighbourhood structure is defined on the discrete space of interest. We plan to explore these extensions in future work and conduct additional experiments to validate their effectiveness, particularly in the case of text data.

Reviewer comment (Novelty):
The paper seems to be taking a lot of the proofs from the original Energy Discrepancy paper more or less verbatim (e.g. appendix A and B). The reviewer questions why this is needed and why these cannot be replaced by a simple pointer to the original paper.

Author response:
Thank you for your feedback. We decided to adapt the proofs to the discrete case, specifically, to make them more accessible for a reader with a discrete application in mind. Theorem 2, in particular, was only proven in the Gaussian case in the original Energy Discrepancy paper. In the upcoming revision, we will omit some of the proofs as per your suggestion.

Reviewer comment (Writing):
On page 3, first line, p(x|y) should be p_ebm(x|y).

Author response:
Thank you for pointing out the typo in the first line on page 3. Apologies for the confusion in equation (6). The symbol $p_{ebm}$ in equation (6) represents the energy-based model $p_{ebm}(x) \propto \exp(-U(x))$, and $p_{ebm}(x|y)$ in the first line of page 3 denotes the recovery likelihood $p_{ebm}(x|y) \propto \exp(-U(x))q(y|x)$. We denote both distributions with the same symbol as they assume the same distribution over $x$. We will revise the paper to make it more clear.

Reviewer comment (Writing):
The notation p_ebm in equation (6) seems a bit confusing.

Author response:
Thank you for pointing out the typo in the first line on page 3. Apologies for the confusion in equation (6). The symbol $p_{ebm}$ in equation (6) represents the energy-based model $p_{ebm}(x) \propto \exp(-U(x))$, and $p_{ebm}(x|y)$ in the first line of page 3 denotes the recovery likelihood $p_{ebm}(x|y) \propto \exp(-U(x))q(y|x)$. We denote both distributions with the same symbol as they assume the same distribution over $x$. We will revise the paper to make it more clear.

Reviewer comment (Reproducibility):
Clarification is needed on how the mean pooling is calculated exactly. Is the mean taken in float and then rounded to either 0 or 1? The reviewer notes that this could cause issues when extended to non-binary variables, as the mean in float for non-binary variables is not particularly meaningful or informative in many cases.

Author response:
The mean-pooling perturbation does not involve a rounding operator as the values of the forward perturbation are allowed to take values in a space that is different from the data domain. For the mean pooling perturbation, specifically, the negative samples are generated by randomly permuting the ordering of bits within a given sliding window.

To elaborate, for a deterministic mapping $g: \mathcal{X} \rightarrow \mathcal{Y}$ (e.g., mean pooling), the contrastive potential $U_q(y)$ can be expressed in terms of the preimage of $g$ as

$$
U_q(y)=-\log \sum_{\{\tilde{x}: g(\tilde{x})=y\}} \exp(-U(\tilde{x}))= -\log \mathbb{E}_{\tilde{x} \sim \mathcal{U}(\{g^{-1}(y)\})}[\exp(-U(\tilde{x}))] + c
$$

where $g^{-1}(y) = \{\tilde{x}: \tilde{x} \in g^{-1}(y)\}\subseteq \mathcal X$ denotes the preimage of  $y$ under $g$. For the mean-pooling transform, $\{\tilde{x}: \tilde{x} \in g^{-1}(y)\}$ represents a set of points obtained from a permutation of  entries of the data point $x$ within the sliding window, because all these points average to the same value within the sliding window.

The methodology can be extended to non-binary spaces. One first chooses a suitable mapping with a tractable preimage that averages over features in the data vector. Discrete Energy Discrepancy is then computed by uniformly sampling from this preimage.

Reviewer comment (Experiments):
Mean pooling seems to be applicable only to image data. Presenting it as one of three perturbations seems odd as it is more specialized and not very generally applicable.

Author response:
Apologies if this seemed misleading. The mean pooling perturbation was indeed motivated by the idea to define perturbations for images but it is not limited to that setting; for other types of discrete data vectors, one can still define a sliding window in which the entries are pooled. In Figure 4, we demonstrate the effectiveness of mean pooling in grey-code density estimation. Additionally, we investigate the impact of the window size in figure 6. Surprisingly, we find that the mean pooling transform works better in the density estimation experiments than on images. 

We illustrated the mean-pooling transformation using image data for ease of understanding. Generally speaking, the mean pooling transform emphasises that deterministic maps are admissible perturbations in energy discrepancy, a possibility that we felt was not apparent from the original energy discrepancy paper.

Reviewer comment (Novelty):
The novelty of this paper is moderate at best. It seems like this paper is a combination of a few papers for instance: Gradient_Guaided importance sampling for learning binary energy-based models by Meng Liu, Haoran Liu, Shuiwang Ji; and Energy Discrepancies: A Score-Independent Loss for Energy-Based Models by Tobias Schröder 2023.

Author response:
**ANSWER:** Thank you for your comment. We want to clarify our contribution and motivation:

Currently, energy discrepancy has two limitations: Firstly, it only relies on Gaussian perturbations, limiting the approach to continuous settings. Secondly, the variance of the contrastive potential cannot be controlled, forcing the practitioner to use a relatively small noise scale which provably limits the expressiveness of energy discrepancy. This motivated us to look for ways to define perturbations in the discrete context and stabilise the estimation of the loss function. We propose techniques to address these limitations of the original energy discrepancy and to target the challenging problem of training EBMs on discrete domains, a research area with only a relatively small number of existing methodologies.

We believe that new methods for discrete energy-based modelling are needed. Maximum-Likelihood based techniques always rely on a sequentially operating samplers that are evaluated after a short number of steps, resulting in biased energy estimates. Gradient-Guided Ratio Matching, on the other hand, is limited to proposals that flip a single dimension. Furthermore, it has not been demonstrated that Gradient Guided Ratio Matching works on image data sets.

Discrete energy discrepancy offers a simple, effective, and cheap to evaluate alternative to these training methodologies. Specifically, we are making the following contributions:

1. **New Objective for Training Energy-Based Models in Discrete Spaces**
    
    In our work, we successfully extend energy discrepancy to train energy-based models in discrete spaces. Our suggested perturbation introduce new ways to define a perturbation, specifically via a deterministic map or via a neighbourhood structure, and allow the perturbation of multiple dimensions at once, demonstrating that perturbations can be chosen much more flexibly than the relatively limited Gaussian perturbations in the original energy discrepancy.
    
2. **Variance Reduction in Energy Discrepancy Loss Estimation**
    
    The Bernoulli perturbation introduces more noise than the one-bit perturbations used in ratio matching or GwG. For this reason, a low variance estimator of the contrastive potential $U_q$ is critical to the success of energy discrepancy. The introduction of importance sampling and gradient guidance helps the reduction of the variance in the contrastive potential in this case and introduces a novel interpretation of contrastive samples that was missing in the original work on energy discrepancies. This approach presents a more general framework compared to the original energy discrepancy paper, which relied on the symmetric properties of Gaussian perturbation for $U_q$ estimation. We anticipate that this new perspective will pave the way for future research in estimating energy discrepancy loss.

Reviewer comment (Novelty):
Seems like the theorems are not novel and are borrowed from other papers; they should have been as a proposition with reference to the original paper. Even the proofs are very similar and all could have been omitted.

Author response:
**ANSWER:** Thanks for your suggestions. The proof of theorem 2 had to be adapted to establish that the theorem holds for the newly proposed perturbations and for the gradient guidance scheme. We will remove some other proofs in our revision to make new contributions easier to identify.

Reviewer comment (Experiments):
This paper seems to be an extension of a poster. However, not much is added to the paper compared to the poster. More detailed experimental results are much needed. There is a lack of comparison between this method and other state-of-the-art approaches on diverse datasets.

Author response:
**ANSWER:** Training energy-based models in discrete spaces poses a significant challenge. The testing of discrete EBM training algorithms in previous literature generally involves four standard benchmarks: Ising models, grey-code density estimation, ego-graph generation, and discrete image modelling, see e.g. [1, 2]. Our methods achieve competitive results with a simpler and faster training routine and better theoretical guarantees in these benchmarks. Despite performance gaps compared to MCMC-based methods on image modelling tasks, one should note that our approaches do not depend on MCMC and can outperform CD algorithms up to five MCMC steps by a considerable margin.

[1] Grathwohl, Will, et al. "Oops i took a gradient: Scalable sampling for discrete distributions." International Conference on Machine Learning. PMLR, 2021.

[2] Zhang, Dinghuai et al. "Generative Flow Networks for Discrete Probabilistic Modeling" International Conference on Machine Learning. PMLR, 2022

Reviewer comment (Experiments):
Seems like sampling a large number of negative samples for each data point in every iteration can be computationally intensive, especially for high-dimensional discrete data, which can lead to slowing down the training process. Also, there are many hyperparameters involved in this algorithm such as the number of negative samples and stabilization parameters. Could the authors please elaborate on these concerns? How does the tuning work and how sensitive is it to suboptimal choices?

Author response:
**ANSWER:** Thanks, for raising these concerns, we are happy to elaborate. In this work, we use three hyperparameters: The number of negative samples, one stabilisation parameter, and the choice of perturbation. We would like to remark that the number of hyperparameters is not larger than in contrastive divergence methods which rely on tuning the MCMC subroutine.

Our approach is scalable in the number of negative samples. In all experiments, we selected the number of negative samples as $M=32$ irrespective of the dimension of the problem to maximise $M$ within the constraints of our GPU capacity. We demonstrate in Figures 7, 8, and 9 in the appendix that our results remain largely unchanged when choosing the number of negative samples as smaller or larger. To further investigate, we train energy-based models using different values of $M$ on the static MNIST dataset with ED-Grid. The results show that our results remain of comparable quality as the number of negative samples is decreased. Furthermore, our approach is easier to parallelise than the sequentially computed MCMC of contrastive divergence.

|  | M=4 | M=8 | M=16 | M=32 |
| --- | --- | --- | --- | --- |
| NLL | 90.13 | 90.37 | 89.14 | 90.61 |

Concerning the stabilisation parameter $w$ and the noise scalar of Bernoulli perturbation $\epsilon$, we conducted a comprehensive ablation study presented in figures 3, 5, 6, 7, 8, and 9. Our training method is robust in both parameters and not sensitive to suboptimal choices. Throughout all experiments, no specific techniques were employed for hyperparameter tuning and search. Further details regarding the hyperparameter setup are provided in Appendix D.

Reviewer comment (Theory):
The algorithm's exploration of the energy landscape might be constrained, particularly when the perturbation distribution (q) does not encompass diverse regions of the data space. This limited exploration could lead to models capturing only specific facets of the data distribution, potentially resulting in biased or incomplete representations. Could the authors please elaborate on that? In addition, the choice of perturbation distribution (q) seems to heavily rely on the characteristics of the training data. How can one design an effective perturbation strategy that is adaptive to different types of discrete data?

Author response:
**ANSWER:** Thanks for your comment. The selection of perturbation is indeed pivotal in training discrete EBMs with energy discrepancy. Theoretically the introduction of more noise by $q(y\vert x)$ enhances the expressiveness of energy discrepancy but concurrently destabilises training. Hence, there exists a trade-off between exploration and training stability.

The perturbation choices introduced in this work adhere to the conditions of Theorem 1 and thus lead to unbiased estimates of the data density, at least in theory. Furthermore, all perturbations are guaranteed to explore the state space close to data points and hence regions with high likelihoods are represented faithfully. This is supported by our experiments which show that simply perturbing the data and learning from the contrast without relying on heavy MCMC sampling can yield successful results, at least on those four standard benchmarks. This stands out as the primary contribution and insight of our work. Moreover, the Bernoulli perturbation has the theoretical capacity to cover all regions of the data space since it allows the flipping of all bits at random to construct negative samples.

Despite this success, we don't expect that one single choice of perturbation can work for all data sets, which is likely a trade-off that one needs to make to give up on the expensive MCMC subroutine used in most training methods for energy-based models. Generally speaking, the perturbation should be informed by the type of data and should destroy information effectively (e.g. a Markov process on non-binary discrete data), while the proposal should be adaptive to the energy and put emphasis on states that require adjustments of the energy to achieve low-variance estimates of the contrastive potential. One way to achieve the adaptive component of this strategy is using the gradient guidance sampling proposed in our paper. For example, it can be seen that ED-$\nabla$Bern exhibits significant performance gains compared to ED-Bern in Table 3 and outperforms CD-1 by a considerable margin. Exploring other means for low variance estimates of the energy discrepancy loss remain a promising avenue for future research.

Reviewer comment (Experiments):
Could the authors please discuss the scalability of the proposed methods with increasing dataset sizes? Moreover, can you provide insights into the interpretability of the learned models?

Author response:
**ANSWER:** Regarding the number of data points, the loss is computed and optimised in mini-batches, so there are no unusual constraints in its scalability. Regarding the dimension of the data set, the major constraint in high dimensions are singular data supports which do not admit a representation by a positive density or energy function. This problem is universal to energy-based models, see e.g. [3], making training of energy-based models particularly challenging compared to other types of generative models. This also poses some constraints to apply energy-based models to data sets of arbitrary complexity, for this reason we focussed on standard benchmarks in our work.

Interpreting energy-based models is challenging because the best metric to evaluate the learned model are it's log-likelihood or it's sample quality, but the two metrics often yield contradicting results. Based on the theoretical guarantees from Theorem 1 and our results on synthetic density estimation, we expect that energy discrepancies match the data density more accurately than standard methods like contrastive divergence.

[3] Arbel et al. "Generalized Energy-Based Models" ICLR 2021

Reviewer comment (Experiments):
The energy discrepancy was used to train continuous energy based models. The paper does not provide enough evidence about the limitation of continuous perturbation with continuous relaxation to train discrete energy based models. This makes the paper less motivated. An explanation about why continuous perturbation does not work or some empirical comparisons would be helpful.

Author response:
**ANSWER:** Thank you for raising this concern. We have conducted a number of experiments on binary data sets using the Gaussian perturbation, which was previously used when applying energy discrepancy to continuous data. Our experiments show that Gaussian energy discrepancies are incapable of training energy-based models on discrete data. We will include plots of the failure case in our revision. This motivates the development of new perturbations that are tailored to estimation tasks on discrete data sets.

Reviewer comment (Theory):
Although the paper provides comparisons between different perturbation and proposals, a principled criteria to choose them is missing. This is no doubt a hard question and a bit more discussion would be helpful.

Author response:
**ANSWER:** Thanks for your intriguing question. There is a trade-off between the expressiveness of energy discrepancy and training stability. The original energy discrepancy paper suggests that the more noise is introduced by $q(y\vert x)$, the more expressive the energy discrepancy loss becomes. However, introducing too much noise destabilises training at the same time.

In this work, we target both aspects of this trade-off. Firstly, we introduce three different perturbations: The grid neighbourhood perturbs only one bit at once which allows for stable training and good performance on image data sets, but leads to a relatively small exploration of the state space. The Bernoulli perturbation, in contrast, enables the flipping of multiple bits at once which is believed to result in a more expressive energy discrepancy and faster learning.

In turn, our proposition of gradient guidance reduces the variance of the estimated energy discrepancy loss without giving up its theoretical guarantees. In addition, the gradient-guided negative samples explore states with low energies more often which means that the contrastive samples are more informative about how the energy landscape needs to be adjusted to match the given data distribution. The resulting update then resembles the update of Contrastive divergence with Gibbs with Gradients [1] or Discrete Langevin [2], but with better theoretical guarantees.

To summarise, the perturbation should be chosen to introduce noise, while the proposal should be chosen to reduce the variance of the energy discrepancy loss function. We will add a short discussion about the effect of various choices for the perturbation and proposal.

Reviewer comment (Experiments):
In experiments, the proposed training algorithm is not significantly better than existing methods. I guess the main reasons are the tasks are simple and the existing algorithms already get very good results. As mentioned in the last paragraph of the paper, the evaluations on more structured data like molecules or text would be helpful.

Author response:
**ANSWER:** Thanks for your suggestions. Training energy-based models in discrete spaces poses challenging and current state-of-the-art methods, including GwG [1] and DULA [2], have also been tested on relatively simple datasets like static MNIST and Omniglot. The challenges of learning EBMs over discrete data sets persist even in synthetic tasks such as grey-code density estimation, where previous approaches like ratio matching encounter difficulties.

The experiments conducted by us are standard in the literature on discrete energy-based model training (see GwG [1], DULA [2] GNetFlow EBMs [3], and RMwGGIS [4]). Our methods exhibit particularly good performance in training Ising models, grey-code density estimation, and ego graph generation, where they produce competitive results to other training methods at a smaller computational cost. Despite performance gaps compared to MCMC-based methods in discrete image modelling, our approaches offer better theoretical properties, faster training, and can outperform CD-1 by a considerable margin.

Nevertheless, we believe that assessing our methods on more structured data such as molecules or text would be beneficial. This would not only make our methods more meaningful but also introduce additional benchmarks for discrete energy-based models. We hope that this can be achieved through better perturbation and proposal choices, and consider this as a potential avenue for future research.

Reviewer comment (Experiments):
Will using an energy based model with continuous relaxation with continuous energy discrepancy training work? How does it perform compared to current discrete framework?

Author response:
**ANSWER:** We were unable to replicate any of our presented experiments on discrete data sets with the originally proposed Gaussian energy discrepancy. We have tried multiple choices for the hyper-parameters. We will add a figure to our revision which shows our most successful attempt (Figure 5), which just barely reflects some structure of the data set the EBM was trained on.

Reviewer comment (Experiments):
Can we train a variational distribution for informed proposal? How would that compare to a gradient-based proposal?

Author response:
**ANSWER:** This is certainly an exciting idea to reduce the variance of the contrastive potential further. The biggest caveat with variational approaches are potential instabilities from the resulting minimax game. Furthermore, a variational proposal adds additional complexity to the methodology like choosing suitable architectures and optimisation schemes, as well as additional computational costs. We are interested in exploring variational approaches in a separate work.

Reviewer comment (Theory):
Whether there can be a dilemma in selecting $q(y|x)$?  

Building on my initial point, securing a tighter lower bound might necessitate a $q(y∣x)$ that significantly alters x by losing enough information. At another extreme, selecting $q(y∣x)$ that maps all samples to a normal distribution, irrespective of x (akin to infusing substantial noise as in diffusion processes), would mean the KL-contraction mirrors the original KL divergence closely. However, an excessive deviation of y from x (transforming it to what resembles random noise) could render negative samples trivially distinguishable from positive ones, potentially destabilizing the training. This seems to create a dilemma. Although in the experiments shown in the paper, the proposed simple perturbations seem to work. Given that the distributions shown in the paper are not very complex, I'm concerned about this problem in modeling more complex distributions.

Author response:
**ANSWER:** There is indeed a trade-off between the expressiveness of energy discrepancy and the training stability. The more noise is introduced by $q(y\vert x)$, the more expressive the loss becomes while destabilising training at the same time. This trade-off guides the design choices made by us in this paper. The grid-neighbourhood perturbation is a relatively small perturbation which allows stable training and produces good results. The Bernoulli perturbation is more ambitious, as it flips several dimensions at once and leads to a more discriminative albeit less stable loss function. We employ gradient guidance specifically to reduce the variance of the contrastive potential and estimated loss function, thus allowing for more complex perturbations. Our results show that gradient guidance indeed improves the performance of Bernoulli-based energy discrepancy.

Given the difficulty of training energy-based models on discrete spaces, we believe that the choice of $q$ is an acceptable trade-off to be made by the practitioner.

Reviewer comment (Theory):
The Feasibility of Abandoning MCMC Sampling for Complex Distributions  

The discussion in Section 4.1 of the paper introduces an importance sampling strategy that utilizes gradient information (as shown in equation 12) to stabilize training. This approach seems to parallel a single-step gradient-based MCMC sample that 'denoises' the perturbed data (so that the negative samples become closer to true samples). I question whether such a simplified rule is viable for more convoluted distributions, or if multiple updates might be necessary to yield valid samples, essentially reverting to the conventional MCMC technique.

Author response:
**ANSWER:** The gradient guidance is indeed similar to contrastive divergence with one step of MCMC. However, we also weight the contribution of each contrastive sample with an appropriate importance weight. This removes the biases of conventional contrastive divergence, since energy discrepancy, contrary to conventional MCMC techniques, offers theoretical guarantees. Indeed, we observe that conventional MCMC techniques only become competitive when more than seven steps of MCMC are used, a procedure that is more expensive than evaluating energy discrepancy. We report the negative likelihoods of MCMC-based techniques with k steps (CD-k) and energy discrepancy in table six (lower is better):

| Dataset \Method | CD-1 | CD-7 | CD-10 | ED-Bern | ED-$\nabla$Bern |
| --- | --- | --- | --- | --- | --- |
| Static MNIST | 182.53 | 98.07 | 88.13 | 96.11 | 90.16 |

We have not conducted experiments on more involved distributions. Approximating the optimal importance distribution with more gradient steps might also improve the performance of energy discrepancy in these cases, which we leave as an avenue for future research.

Reviewer comment (Experiments):
Sufficiency of Experimental Results to Substantiate the Algorithm's Effectiveness  

Reflecting upon the earlier points, the experiments demonstrated in this paper seem to be a bit easy to me. While models like the Ising Model and 2D examples can serve as sanity check examples, they might not pose a significant challenge with the current state of research. Similarly, the Ego-small dataset might lack complexity in its graph structure. Moreover, reference [1] seems to set a more robust benchmark. The MNIST dataset, used for image distribution analysis, is also relatively simplistic, and it appears the proposed model falls short of matching the performance of GWG. These factors collectively cast doubt on the proposed algorithm's performance and scalability when dealing with more sophisticated distributions.  

[1] Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations.

Author response:
**ANSWER:** Training energy-based models in discrete spaces is a challenging problem. Existing leading methods, such as contrastive divergence with Gibbs with Gradients [2] and Langevin for discrete distribution [3] are primarily tested on simple datasets like static MNIST and Omniglot as well. Similarly, the other experiments on Ising models, synthetic grey-code data, and ego-small graphs have been used as test cases for discrete energy-based model training, see e.g. [2] and [4]. Compared to contrastive divergence, our approach works without MCMC which means that energy discrepancy can simplify and accelerate the training of energy-based models on typically benchmarked data sets. 

The cited reference [1] uses a diffusion model, which serves different purposes compared to energy-based models as they learn a sampler for the data distribution, while energy-based models model the data generating density, directly.

We agree that our models seem to fall short compared to GwG in our experiments on image data. However, our training method outperforms contrastive divergence with up to seven steps of GwG, while being significantly cheaper to evaluate. We hope that future developments of energy discrepancy can match the performance of GwG at a lower computational cost.

Reviewer comment (Novelty):
More comprehensive review for EBM works  

While the paper primarily concentrates on modeling discrete distributions, it would be beneficial to provide a more comprehensive review of the field of Energy-Based Model (EBM) training, with particular emphasis on those studies that explore sampling methods for negative samples. For example, [1] pioneering work using cnn as energy function, [2] [3] [6] are works that use different amortized methods to reduce MCMC steps, [4] uses a replay buffer and [5] moves to the latent space to facilitate sampling.  

[1] "A theory of generative convnet." International Conference on Machine Learning. PMLR, 2016.  
[2] "Cooperative training of descriptor and generator networks." IEEE transactions on pattern analysis and machine intelligence 2018  
[3] "No MCMC for me: Amortized sampling for fast and stable training of energy-based models." ICLR 2021  
[4] "Improved contrastive divergence training of energy-based models." ICML 2021  
[5] "VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models" ICLR 2021  
[6] "A tale of two flows: Cooperative learning of langevin flow and normalizing flow toward energy-based model." ICLR 2022

Author response:
**ANSWER:** Thank you for the diverse list of references, they are a useful resources for exploring sampling methods to generate negative samples! It should be noted that most mentioned methods use negative samples for approximate maximum likelihood estimation, which suggests different methods to produce negative samples. We will include more references including these in the related work section. Thank you for your contribution.

**References:**
[1] Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations.

[2] Grathwohl, Will, et al. "Oops i took a gradient: Scalable sampling for discrete distributions." International Conference on Machine Learning. PMLR, 2021.

[3] Zhang, Ruqi, Xingchao Liu, and Qiang Liu. "A Langevin-like sampler for discrete distributions." International Conference on Machine Learning. PMLR, 2022.

[4] Liu, Meng, Haoran Liu, and Shuiwang Ji. "Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models." ICLR 2023.