Gradient-Guided Importance Sampling for Learning Discrete Energy-Based Models
Authors: Meng Liu
Haoran Liu
Shuiwang Ji
Venue: ICLR 2022 Submitted (2022)
Keywords: Discrete energy-based models
ratio matching
importance sampling
gradient

Abstract: Learning energy-based models (EBMs) is known to be difficult especially on discrete data where gradient-based learning strategies cannot be applied directly. Although ratio matching is a sound method to learn discrete EBMs, it suffers from expensive computation and excessive memory requirement, thereby resulting in difficulties for learning EBMs on high-dimensional data. In this study, we propose ratio matching with gradient-guided importance sampling (RMwGGIS) to alleviate the above limitations. Particularly, we leverage the gradient of the energy function w.r.t. the discrete data space to approximately construct the provable optimal proposal distribution, which is subsequently used by importance sampling to efficiently estimate the original ratio matching objective. We perform experiments on density modeling over synthetic discrete data and graph generation to evaluate our proposed method. The experimental results demonstrate that our method can significantly alleviate the limitations of ratio matching and perform more effectively in practice.

Under review as a conference paper at ICLR 2022
GRADIENT-GUIDED IMPORTANCE SAMPLING FOR LEARNING DISCRETE ENERGY-BASED MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Learning energy-based models (EBMs) is known to be difficult especially on dis- crete data where gradient-basee based learning strategies cannot be applied directly. Al- though ratio matching is sound method to learn discrete EBMs, it suffers from expensive computation and excessive memory requirement, thereby resulting difficulties for learning EBMs on nigh-dimensional data. In this study, we pro- pose ratio matching with maadient-guided importance sampling (RMwGGIS) alleviate the above limitations. Particularly, we leverage the gradient of the energy function w.r.f. the discrete data space to approximately construct the provable op- timal proposal distribution, which is subsequently used by importance sampling efficiently estimate the original ratio matching objective. We perform experiments density modeling over synthetic discrete data, graph generation, and training Ising models to evaluate our proposed method. The experimental results demon- strate that method can significantly alleviate the limitations of ratio matching, perform more effectively in practice, and scale to high-ddmensiioal problems.
INTRODUCTION
Energy-Based models (EBMs), also known as unnormalized probabilistic models, model distribu- tions by associating unnormalized probability densities. Such methods have been developed for
decades (Hopfield) 1982, Ackley et al. 1985 Cipra 1987 Dayan et al.. 1995 Zhu al., 1998] Hinton 2012) and are unitied as energy based models (EBMs) (LeCun et 2006) in the machine learning community. EBMs have great simplicity and flexibility since energy functions are not re- quired to integrate sum to one, thus enabling the usage of various energy functions. In practice, given different data types, we can parameterize the energy function with different neural networks needed, such as multi-layer perceptrons (MLPs), convolutional neural networks (CNNs) (LeCun) (ct al. 1998), and graph neural networks (GNNs) (Gori et al., 2005; Scarselli et al. 2008). Recently, EBMs have been drawing increasing attention and are demonstrated to be effective in various do- mains, including images (Ngiam et al. 2011 Xie et al. 2016 Du & Mordatch (019), videos (Xie
al. 2017). texts (Deng al.) 2020 3D objects (Xie al. 2018). molecules (Liu et
2021
Hataya et al. 2021). and proteins (Du et al. (0220)).
Nonetheless, learning (a.k.a,, training) EBMs is known to be challenging since we cannot compute the exact likelihood due the intractable normalization constant. As reviewed in Section 4 many approaches have been proposed to learn EBMs, such as maximum likelihood training with MCMC sampling (Hinton) (002) and score matching (Hyvärinen & Dayan 2005). However, most recent advanced methods cannot be applied to discrete data directly since they usually leverage gradients over the continuous data space. For example, for many methods based on maximum likelihood training with MCMC sampling, they use the gradient w.r.t the data space to update samples in each MCMC step. If we update discrete samples using such gradient, the resulting samples are usually invalid in the discrete space. Notably, discrete data is common in our real world, such as texts, graphs, and genome sequences. Therefore, learning EBMs on discrete data remains to be challenging and in demand.
Ratio matching (Hyvärinen] 2007 Lyu 2009 is proposed to learn discrete EBMs by matching ratios of probabilities between the data distribution and the model distribution, as detailed in Section 2.2 However, as analyzed in Section 3.1 it requires expensive computations and excessive memory ages, which is infeasible if the data is high-dimensional. In this work, we propose to use the gradient of the energy function w.r.f. the discrete data space guide the importance sampling for estimating

---

Under review as a conference paper at ICLR 2022
the original ratio matching objective. More specifically, utilize such gradient approximately construct the provable optimal proposal distribution for importance sampling. Thus, the proposed approach is termed ratio matching with gradient-guides importance sampling (RMwGGIS). Our RMwGGIS can significantly overcome the limitations of ratio matching. In addition, strated to be more effective than the original ratio matching because it can be optimized better in practice. Experimental results on synthetic discrete data, graph generation, and Ising model training demonstrate that our RMwGGIS significantly alleviates the limitations of ratio matching, achieves better performance with obvious margins, and has the ability of scaling to high- sigh-dimensional relevant problems.
demon-
2
PRELIMINARIES
2.1 ENERGY-BASHE MODELS
Let be a data point and Eg (a) E R be the corresponding energy, where 0 represents the learnable parameters of the parameterized energy function Ee(-). The probability density function of the model distribution is given as
(Eo(aa)
Po (0()))
(1)
Zo
where Ze E IR is the normalization constant (a.k.a partition function). To be specific, Ze I e dæ x: is in the continuous space and Zo = Ee (m(m)) for discrete data. Hence, com- puting Zo is usually infeasible due to the intractable integral summation. Note that Zo is variable depending on 0 but a constant w.r.l. C.
2.2 RATIO MATCHING
Ratio matching (Hyvärinen 2007) is developed for learning EBMs on discrete data by matching ratios of probabilities between the data distribution and the model distribution. Note that we focus on d-dimensional binary discrete data æ E (0, 1] in this work.
Specifically, ratio matching considers the ratio of p(z) and p(æ-:), where a_ (I1, x2, .. Te,- "....,,, Ta) denotes a point in the data space obtained by flipping the i-th dimension of x. The key idea is to force the ratios ne(w)) defined by the model distribution pe to be as close as possible to the ratios (p(2)
po pa
pn (m==) given by the data distribution pp. The benefit of consid-
ering ratios of probabilities is that they do not involve the intractable normalization constant Ze
since Pe (ax
( (a according to Eq. (1). To achieve the match between Egt
2e
ratios, Hyvärieen (207)) proposes minimize the objective function
PD PD( PD (=(x.)))
p0( po x
)1
[
PDD pp((x_)) PD(
2
Tras (0) F E (2)
9
Po (x
(2)
The sum of two square distances with the role of x and x switched is specifically designed since it is essential for the following simplification. In addition, the function g(u) 1 chosen in order to obtain the subsequent simplification. To compute the objective defined in Eq. (2)
14 is also carefully
is known that the expectation over data distribution (i.e., (x)))p(()) (a)) can be unbiasedly estimated
by the empirical mean of samples æ PD (xr).. However, to obtain the ratios between pD pr())) and PD (x= in Eq. (2). the exact data distribution is required to be known, which is usually impossible.
Fortunately, thanks to the above carefully designed objective, HHyäärnen (2007) demostrates that the objective function in Eq. (2) is equivalent to the following simplified version
Σ po Po (x
))
d Ex-pp((a)
(c Eo)t)) )}
TRAS (0) E Emppp(() (x)
(3)
which does not require the data distribution to be known and can be casily computed by evaluating the energy of x and æ x... is proved that the estimator given by Eq. (3) is consistent (Hyvärinen) 2007]. That means if it is minimized perfectly, the obtained model distribution will capture the data
2

---

Under review as a conference paper at ICLR 2022
distribution exactly. Further, Lyu 2009) reduced to
shows that the objective function of ratio matching can be
po aa-=)]] =
Eolle)))
JMM (0) E Po (x)
(4)
po (x)
It is obvious that Eq. ( and Eq. agree with each other since function g(-) decreases monotoni-
cally in [0, +0)) which aligns with the value range of probability ratios Po(m)
po( +)
can push down the energy of the training sample z and push up the energies of other data points obtained by flipping one dimension of æ. Thus. this objective faithfully expect that each training sample E has higher probability than its local neighboring points that are hamming distance from æ.
Intuitively, the objective function of ratio matching, as formulated in Eq.
THE PROPOSED METHOD
In this section, we analyze the limitations of the ratio matching method from the perspective of computational time and memory usage. Then, we describe our proposed method, ratio matching with gradient-guided importance sampling (RMwGGIS), which utilizes the gradient of the energy function w.t. the discrete input æ to guide the importance sampling for estimating the original ratio matching objective. Our approach can alleviate the limitations significantly and is shown to be more effective practice.
3.1 ANALYSIS OF RATIO MATCHING
Time-intensive computations. According to Eq. (4). for a given training sample æ, we have to compute the energies for all w-4, where 1,... d. In other words, we have O(d) evaluations of the energy function for each training sample. This computationally intensive, especially when the data dimension d is large.
Excessive memory usages. Besides the expensive computation, the memory usage of ratio match- ing is another limitation that cannot be ignored, especially when we learn the energy function using
modern GPUs with limited memory. As shown in Eq. ( the objective funetion consists of d terms for each training sample. When we do backpropagation. computing the gradient of the objec- tive function w.s.f the learnable parameters of the energy function required. Therefore, in order to compute such gradient, we have to store the whole computational graph and the intermediate tensors for all of the terms, thereby leading to excessive memory usages especially if the data dimension d is large. Hence, it is challenging to learn EBMs with ratio matching on modern devices, such as GPUs, for high-dimensional discrete data.
3.2 RATIO MATCHING WITH GRUDIENT-GUIDED IMPORTANCE SAMPLING
The key idea of our approach is to use the well-known importance sampling technique to reduce the variance of estimating JAM (0) with Monte Carlo method. The most critical and challenging part of using the importance sampling technique is choosing a good proposal distribution. In this work, propose to utilize the gradient of the energy function w.r.f. the discrete input to approximately construct the optimal proposal distribution for importance sampling. We describe the details of our method below.
The objective for each sample ær, defined by Eq. (4 can be reformulated
d
Fo(z) Folm , dE s-m(( 2) (F((m))F((a_))) ?
JRM x) =
(5)
d
where m(x for 1,..., is a discrete distribution. Thus, the objective of ratio matching
for each sample tion m(xx,). In the original ratio matching method, described in Section 2.2 compute such expectation exactly by considering all possible x x... leading to expensive computations and exces- sive memory usages analyzed in Section 3.1 Naturally, we can estimate the desired expectation
be viewed the expectation of Ee(o(_)) over the discrete distribu-
3

---

Under review as a conference paper at ICLR 2022
with Monte Carlo method by considering fewer terms sampled based m(x-1). However, such timation usually has a high variance, and empirically verified to be ineffective by our experiments in Scction[]]
Further, apply the importance sampling method to reduce the variance of Monte Carlo timation. Intuitively, certain values have more impact on the expectation than others. Hence, the estimator variance can be reduced if such important values are sampled more frequently than others. To be specific, instead of sampling based on the distribution m (x=x)), importance sampling aims to sample from another distribution n))=;), namely, proposal distribution. Formally,
m(x-1) [e (o()))) (
Fo(m ']] dE con(e)))
(O, x) = -B
The detailed derivation of Eq. 6 is given in Appendix A Afterwards, we can apply Monte Carlo estimation based the proposal distribution n(z-.). Specifically, sample s terms, denoted as x T. x L. according to the proposal distribution n(w=1). Note that s is usually chosen to be much smaller than d. Then the Monte Carlo estimation for JRM (0, x) is computed based on these terms. Formally,
1 =(x
(1) (Fo()))Bo(a"))
JRM x)n =
x (t)
n(x
n(x
is known that the estimator obtained by Monte Carlo estimation with importance sampling is an unbiased estimator, as the conventional Monte Carlo estimator. The key point of importance sampling is to choose an appropriate proposal distribution n(x=:), which determines the variance of the corresponding estimator. The optimal proposal distribution n² (æ=:). which yields the minimum variance, is given by the following theorem.
(c Egin +)
Theorem Let n² (x=s)
Ee Polem- be a discrete distribution on T-s. where
i I. Then for any discrete distribution n( on w-i, where 1,..., d, we have Var (0) Var [JnM (0.m),) m).).
Proof. The proof of Theorem is included in Appendix
B
To construct the exact optimal proposal distribution n (x=:) given by Theorem evaluate the energies of all æ-i, where i 1. ..d d. To avoid such complexity, we propose to leverage the gradient of the energy function w.t.t. the discrete input æ to approximately construct the optimal proposal distribution. Our approach only needs ((1) evaluations of the energy function to construct the proposal distribution.
still have to
It is observed by Grathwohl et al. al.] (202T) that many discrete distributions are implemented as contin- uous and differentiable functions, although they are evaluated only in discrete domains. Grathwohl jet al. (202T) further proposes a scalable sampling method for discrete distributions by utilizing the gradients of the underlying continuous functions w.r.f. the discrete input. In this study, we extend this idea to improve ratio matching. More specifically, in our case, even though our input æ is discrete, our parameterized energy function E.(.), such as neural network, is usually continuous and differentiable. Hence, we can use such gradient information to efficiently and approximately construct the optimal proposal distribution given by Theorem
The basic idea is that that x-; is close to x in the data space because they only have differences in one dimensior Formally,
approximate Eg (x= based on the Taylor series of Eo æ, given
Eo(x-:) R Eo(x) (x_ ==² (o
Thus, can approximately obtain the desired term Fa(x:) Fa in Theorem 1 using Eq. 08 Note that V Lgg (x) E nd contains the information for approximating all Eg(x) Ee (x= where
We have this assumption hecause data space is usually high-dimensional. If the number of data dimension small, can the original ratio matching method with affordable time and memory budgets.
4

---

Under review as a conference paper at ICLR 2022
Algorithm Ratio Matching with Gradient-Guided Importance Sampling (RMwGGIS)
Input: Observed dataset D = {x(m))
] m==1 parameterized energy function Ee(-), number of sam-
[P]
ples for Monte Carlo estimation with importance sampling
1: for æ ~ 'D do
> Batch training is applied in practice
2: Compute Ex(x))
3: Compute (o)
4: Compute the proposal distribution 7'' (x-
D Eq. (1p
2d 22 110
5: Sample s terms, denoted as x ---1"" according to 7" (m=
(+)
Ea(m) Eo(a
m(x
6: Compute JAM (0. .) d Ei=1
Eq.
(or Eq. (1))
7 (w0"
7: Update 0 based on VeJRM 7))
8:
end for
i 1, ... ., Hence, we can consider the following d-dimensional vector
2x 1) @ V (x)
ERR
where @ denotes element-wise multiplication. Note that we have I2 E. -1 if x; 0 and T; F. ==1it 1f; = 1, which can unified as 2; I; = 2x, - 1. Therefore, we have
Ea (x) (x R [2x 1) Vo == 1,...dd
(10)
Afterwards, we can provide a proposal distribution ñ (x_,) as an approximation of the optimal proposal distribution n² (x==:) given by Theorem 1. Formally,
2 (0(2=-1)07 1)
[+)
i= ...,d
(II)
[2222-1) Eo
Then (x=x>) is used as the proposal distribution for Monte Carlo estimation with importance
sampling, as described by Eq. (7). The overall process of our RMwGGIS method is summarized in Algorithm
3.3 COMPARISON BETWEEN RATIO MATCHING AND RMWGGIS
Time and memory. Since only (s < d) terms considered in the objective function of our RMwGGIS, as shown in Eq. (7). we have better computational efficiency and less memory require- ment compared to the original ratio matching method. To be specific, RMMGGIS only needs (s)) evaluations of the energy function compared with (d) in ratio matching, leading to linear speedup, which is significant especially when the data is high-dimensional. The improvement in terms of memory usage is similar. In Section 5.1] we compare the real running time and memory usage between ratio matching and our proposed RMWGGIS on datasets with different data dimsen- sions.
Better optimization? In Section 3.2 we propose our RMwGGIS based on the motivation to approx- imate the objective of ratio matching with fewer terms. Although our RMwGGIS can approximate the original ratio matching objective numerically, our objective only includes terms compared to d terms the original ratio matching objective; That is, only s terms are involved the computational graph, leading to different brok-propo gradients compared to the original ratio matching. In other words, the objective of ratio matching, as shown Eq. 4 intuitively pushes up the energies ......dd d, while our RMwGGIS only considers pushing up energies of terms
of all æ_; for i l....,d, among them, as formulated by Eq. (7). Thus, the following question might be raised. Why can our objective be effective for learning EBMs without pushing up the energies all d terms? In prac- tice, we even observe that our RMwGGIS achieves better density modeling performance than ratio matching. We conjecture that this is because our RMwGGIS can be optimized better in practice for the following two properties, which are empirically verified in Section
(1) RMwGGIS introduces stochasticity. Without involving all d terms in the objective function, method can introduce stochasticity, which could lead to better optimization in practice. This has
5

---

Under review as a conference paper at ICLR 2022
the same philosophy as the comparison between mini-batch gradient descent and vanilla gradient descent. The gradient is obtained based on each batch in mini- batch gradient descent, while it is computed over the entire dataset in vanilla gradient descent. is known that the mini-batch gradient descent usually performs better in practice since the stochasticity introduced by mini-batch training could help to escape from the saddle points in non-convex optimization (Ge et al., 2015). Therefore, the stochasticity introduced by sampling only terms in RMwGGIS could help the optimization especially when d large.
(2) RMWGGIS focuses on neighbors with low energies. Even though only energies of s terms are pushed up in our method, these s terms correspond to the neighboring points that have low energies. According to n" (x= given by Theorem
neighbor of æ, denoted æ --i is more likely to be sampled if its corresponding energy value is lower Hence, we choose to push up the energies of neighbors according to their current energies. The lower the energy, the more likely
to be selected. This is intuitively sound because the terms that have low energies are the most offending terms, which should have the higher priorities to be pushed up. This point has the same philosophy as hard negative mining, which pays more attention to hard negative samples during training. More detailed explanation about this connection is provided in Appendix C
Following the hard negative mining perspective, we observe that the coefficients used in Eq. 7 pro- vide smaller weights for terms with lower energies, which is intuitively less effective, as detailed in Therefore, we further propose the following biased estimation as the objective function by removing the coefficients in Eq. Formally,
Appendix D
TRAS (0, x)
blosed
Eola))
(e)
n (x (x-
(12)
This biased version is essentially an natural extension of the unbiased version. It is demonstrated to be more effective in practice. The explanation about this is discussed in Appendix D
4 RELATED WORKS
Learning EBMs has been drawing increasing attention recently. Maximum likelihood training with MCMC sampling, also known as contrastive divergence (Hinton 2002). is the most representative method. It contrasts samples from training set and samples from the model distribution. To draw samples from the model distribution, we can employ MCMC sampling approaches, such as Langevin dynamics (Welling& & Tch 2011)) and Hamiltonian dynamics (Neal et al., 2011). Such methods are further improved and shown to be effective by recent studies (Xie et al. 2016:] Gao et al.] 2018 Du & Mordatch] 2019 Nijkamp al.] 2019 Grathwohl et al.) 2019 Jacob et al. [020]] Qiu et al. 201o/
Du et al.) 2020a). These methods, however, require the gradient w.f.f. the data space to update samples in each MCMC step. Thus, they cannot be applied to discrete data directly. To enable maximum likelihood training with MCMC sampling on discrete data, we can naturally use discrete sampling methods, such as Gibbs sampling and Metropolis-Hastinne replace the above gradient-based sampling algorithms. Unfortunately. sampling from discrete distribution extremely consuming and not scalable. Recently, Dai et al. (2020) develops a learnable sampler parameterized as a local discrete search algorithm to propose negative samples for contrasting. Grathwohl et al. (222I) proposes a scalable sampling method for discrete distributions by surprisingly using the gradient w.r.f. the data space, which inspires our work a lot.
algorithm (Zanella) 2020), to
Maximum likelihood training with MCMC sampling is computationally expensive since MCMC sampling methods usually require a large number of steps to obtain reasonable samples. An alterna- tive method for learning EBMs is score matching (Hyvärinen & Dayan 2005 Vincent [011] T Song ct al. 2020 Song & Ermon 2019). where the scores, i.e., the gradients of the logarithmic probability distribution W.E.t the data space, of the energy function are forced to match the scores of the train- ing data. Ratio matching (Hyvärinen) 2007 Lyu 2009) is obtained by extending the idea of score matching discrete data. Our work is motivated by the limitations of ratio matching, as analyzed in Section[.1] Stochastic ratio matching (Dauphin & Bengio, 2013) also aims to make ratio matching more efficient by considering the sparsity of input data, while our approach uses the gradient of the energy function. Hence, our method is effective for general EBMs, but stochastic ratio matching is limited to sparse data.
2 A Athough this only strictly holds for n (a=-1,, this can serve relaxed explanation for ni' (ar(ar))) well since ' (2) is good approximation of n (1). More detailed analysis is included in Appendix C

---

Under review as a conference paper at ICLR 2022
Table I: Results on 32-dimensional synthetic discrete data in terms of MMD. The lower the better. The top two results on each dataset are highlighted as 1st and 2nd.
Method
2xpirals
8gaussions
cireles
moons
pirwweee
swissroll
checkerhoori
Rario Matching
0.01514
0.10270
0.11856
0.02901
0.31353
0.05820
0.00039
RMwGGIS (unbiased)
0.01099
0.09763
0.11017
0.03111
0.27888
0.05176
0.00050
RMwGGIS (biased)
0.00876
0,08414
0.10230
0.02787
0.26188
0.04477
0.00026
There are some other methods for learning EBMs, such as noise contrastive estimation (Gutmann Hyvärinen 2010 Bose al., 2018 Ceylan & Gutmann 2018 Gao et al.] (020) and learning
the stein discrepancy (Grathwohl et al., 2020). We recommend readers to refer to Song & Kingma
(2021) for a comprehensive introduction on learning EBMs. note that several works (Elvira
et al. 2015: Schuster] (015) use the gradient information of the target distribution to iteratively optimize the proposal distributions for adaptive importance sampling. However, compared to our method, they can only applied to continuous distributions and require expensive iterative process.
5 EXPERIMENTS
5.1 DENSITY MODELING ON SYNTHETIC DISCRETE DATA
Setup. For both quantitative results and qualitative visualization, we follow the experimental set- ting of Dai et al. (2020) for density modeling on synthetic discrete data. data points from 2D continuous space according some unknown distribution p. which can be naturally visualized. Then, we convert each 2D data point & E R² to a discrete data point {0, 1]d. where d is the desired number of data dimensions. To be specific, we transform each dimension of é. which is floating-poin number, into d bitit Gray code and concate- nate the results to obtain a d-bit vector σ. Thus, the unknown distribution in discrete space is p(x) = à GrayYoFlloat(z
firstly draw 2D
GrayToFloat(a (+1)))++))))))))))) This density modeling task challeng- ing since the transformation from & is non-linear.
To quantitatively evaluate the performance of density modeling, we adopt the maximum mean dis- crepancy (MMD) (Gretton et al. 2012) with a linear kernel corresponding to (d- (--HammingDitaanee) The MMD is commonly used to compare distributions. In case, particularly, the MMD is com- puted based on 4000 samples, drawn from the learned energy function via Gibbs sampling, and the same number of samples from the training set. Lower MMD indicates that the distribution defined by the learned energy function is closer to the unknown data distribution. In addition, in order to qualitatively visualize the learned energy function, we firstly uniformly obtain 10k data points from 2D continuous space. Afterwards, they are converted into bit vectors and evaluated by the learned energy function. Subsequently, we can visualize the obtained corresponding energies in 2D space.
The energy function is parameterized by 4-layer MLP with the Swish (Ramachandran al., 2017) activation and 256 hidden dimensions. The number of samples s, involved in the objective func- tions of our RMwGGIS method,
be 10. In the following, we compare our unbiased and biased methods, as formulated in Eq. (7) and Eq. respectively, with the original ratio matching
method (Hyvärinen 2007; Lyu 2009
Quantitative and qualitative results. The quantitative results on 32-dimensional datasets are shown in Table I Our RMwG- GIS, especially the biased ver- sion, consistently outperforms the original ratio matching by large margins, which demon- strates that is effective for our proposed gradient-guided impor- tance sampling to stochastically push up neighbors with low energies. This verifies our analysis Section 3.3 In Figure
2apirals 8gaussians
circles
moons
pinwheel
swissroll
checkerboord
Data
RMWGGIS (unbiased)
RMwGGIS (biased)
Figure 1: Visualization of learned energy functions on 32- dimensional synthetic discrete datasets.
https://en.wiikipediaaa
7

---

Under review as a conference paper at ICLR 2022
Table 2: Comparison between ratio matching and our RMwGGIS on 2spirals datasets with different dimensions in terms of running time and memory usage.
# Data Dimensions
J2
64
128
256
512
1024
2048
Memory Time
Ratio Matching
63.9ms
106.5ms
185.6ms
372.9ms
735.2ms
1,390 Ims
16884.mm
RMwGGIS
41.2ms
47.2ms
58.8ms
86.9ms
137.7ms
244.9ms
434,
Speedup
1.6x
2.3x
3.2x
4.3x
5.3x
5.7x
6.2x
Ratio Matching
957MB
1031MB
1189MB
1515MB
2315MB
1237MB
9633B
RMwGGIS
891MB
893MB
893MB
915MB
919MB
931MB
951MB
Memory Saving
6.9%
13,4%
24.9%
40.8%
60.3%
78.0%
90.1%
qualitatively visualize the learned energy functions of our proposed RMMGGIS.. It is observed that EBMs learned by our method can fit the data distribution accurately. Note that we choose d = 32 for quantitative evaluation because Gibbs sampling cannot obtain appropriate samples from the learned energy function with an affordable time budget if the data dimension is too high, thus leading to invalid MMD results. We will compare the results on nigher-dimensional data in the following by observing the qualitative visualization. To further demonstrate that the performance improvement of RMWGGIS over ratio matching brought by better optimization, we show that energy functions learned with our methods actually lead to lower value for the objective function defined by Eq. 4 The details are included in Appendix
Observations on higher-dimennional data. As analyzed in Section 3.3 the advantages of our approach can be greater on higher-dimensional data. To evaluate this, we conduct experiments on the 256-dimensional 2spirals dataset, and visualize the learned energy functions corresponding to different learning iterations. We construct a method for ablation study, named RMwRAND, which estimates the original ratio matching objective by randomly sampling s = 10 terms. The only difference between corresponding to low energies thanks to our proposed gradient-guided importance sampling.
RMwGGIS method and RMwRAND is that we focus more the terms
As shown in Figure Appendix our RMWGGIS accurately capture the data distribution, while the original ratio matching method cannot. This further verifies that our RMwGGIS can be optimized better than ratio matching especially when the data dimension is high, as analyzed in Section 3.3 In addition, although RMwRAND can also introduce stochasticity as our RMwGGIS by randomly sampling, it fails to capture the data distribution. This observation is intuitively reasonable since randomly pushing up 10 terms among d 256 terms leads to large variance and unsatisfactory performance. Instead, our RMwGGIS performs well since we focus on pushing up terms with low energies, which are the most offending terms and should be pushed up first. Overall, these experi- ments can show the superiority of RMMGGIS endowed by our proposed gradient- -uuddd importance sampling on high-dimensiioal data.
Running time and memory usage. As analyzed in Section 3.3 our RMwGGIS has better efficiency than ratio matching in terms of computational cost and memory requirement. To empirically verify this, we compare the real running time and memory usage on datasets of various dimensions. Specif- ically, we construct several 2xpirals datasets with different data dimensions and train parameterized energy functions using ratio matching and our RMwGGIS, respectively. We choose batch size to be 256. The reported time corresponds to the average training time per batch. For RMwGGIS, both the unbiased version and the biased version have almost the same running time and memory usage. Thus, report the results of the biased version.
As summarized in Table our RMWGGIS is much more efficient in terms of running time and mem- ory usage, compared with the original ratio matching method. In addition, our method can achieve more speedup and save more memory usages with the increasing of data dimension. Specifically, compared with ratio matching, our RMwGGIS can achieve 6.2 times speedup and save 90. 1 1% mem- ory usage on the 2048-dimensional dataset. This shows the efficiency of our RMwGGIS especially for high- sigh--imenssionll data.
5.2 GRAPH GENERATION
Setup. We further evaluate our RMwGGIS on graph generation using the Ego-small dataset YYou et al. 2018). It is set of hop ego graphs, where the number of nodes 4 < [V| 18, obtained from the Citeseer network (Sen et al. 2008). Following the experimental setting of You et al. (2018) and Liu et al. (2019), 80% of the graphs are used for training and the rest for testing. New graphs can
8

---

Under review as a conference paper at ICLR 2022
be generated via Gibbs sampling on the learned energy function. To evaluate the graph generation performance based on the generated graphs and the test graphs, we calculate the maximum mean discrepancy (MMD) (Gretton et al. 20122) over three statistics, i.e., degrees, clustering coefficients, and orbit counts, as proposed in You et al. (018))
We parameterize the energy function by a 5-layer R-GCN (Schlichtkrull et al., 2018) model with the Swish (Ramachandran et al. 2017) activation and 32 hid- den dimensions, whose input is the upper triangle of the graph adjacency matrix. The number of samples s used in our RMwGGIS objective is 50. We apply our biased version to learn the energy function since it is shown to be more effective in Section [5.] Besides ra- tio matching, we consider the recent proposed method
Table 3: Graph generation results in terms of MMD. The lower the better. Avg. denotes the average over three MMD results. The results of baselines are reported from You et al. (2018). 8), Liu et al. (2099), Niu et al. (2020), ,,hhi et al. (2019), and Luo al. (2021). We ob- tain the result of EBM (GWG) by us- ing their official implementation, and
EBM (GWG) (Grathwohl et al.) 202T), which develops a gradient- MCMC sampling method discrete dis- tribuion to learn discrete EBMs with mexiiuum-likelhhoo training, baseline. We also consider the recent works developed for graph generation as baselines, includ- ing GraphVAE (Simonovsky & Komodakis] 2018), Deep- GMG (Li et al.) 2018), GraphRNN (You et al. 2018).
the detailed settings is provided in Ap-
pendix G
Method
Degree
Claster
Orbit
Aug.
GraphVAE
0.130
0.170
0.050
0.117
DeepGMG
0.040
0.100
0.020
0.053
GmphRNN
0.090
0.220
0.003
0.104
GNF
0.030
0.100
0.001
0.044
EDP-GNN
0.052
0.003
0.007
0.050
GNF (Liu al.; (2099), EDP-GNN (Nuu al.
2020)
GraphAF
0.030
0.130
0.001
0.047
GraphAF (Shi et al. 2011)) and GraphDF (Luo et al.
GraphDF
EBM (GWG)
0.093
0.027
0.053
0.058
2021
Matching
0.008
0.045
Quantitative and qualitative results. As summarized in Table our RMwGGIS outperforms baselines in terms of the average over three MMD results. This shows that our method can learn EBMs to generate graphs that align with various characteristics of the training graphs. The generated samples are visualized in Figure 4 Appdendix H It can be observed that the generated samples are realistic one-hop ego graphs that have similar characteristics as the training samples.
RMwGGIS
0.044
0.050
0.013
0.039
5.3 TRAINING ISING MODELS
To further demonstrate the scaling abil- ity of our method and compare with re- cent baselines more thoroughly, we use our RMwGGIS to train the Ising model with a 2D cyclic lattice structure, fol- lowing Grathwohl
Table 4: Comparison of training Ising models in terms of RMSE and running time.
MCMC Ataap
10
25
50
100
RMMGGG
logIRMIS
EBM
Glbb))
2.0
4.94
al. (202I). We
Timofiar
BRM (GBbg)
(OWG)
202.5ms
30.3mm
C37 Jms
68. dmm
1110.205
100.8m
2581 Bas
222.8ma
8576 Ims
306.7es
189mm
compare methods in terms of the RMSE between the inferred connectivity matrix I and the true J and the running time per iteration. The experimental details are included in Appendix [ As shown in Table 4 our RMwGGIS is more effective than EBM (Gibbs) with various sample steps. The recently proposed EBM (GWG) (Grathwohl et al.) 2021 achieves better RMSF than ours. In terms of running time, our method is much more efficient than baselines since we avoid the expen- sive MCMC sampling during training. According to this experiment, one future direction could be further improving the effectiveness of RMwGGIS while preserving the efficiency advantage.
6 CONCLUSION
We propose ratio matching with gradient-guided importance sampling (RMwGGIS) for learning EBMs on discrete data. In particular, utilize the gradient of the energy function w.z.f. the discrete input space to guide the importance sampling for estimating the original ratio matching objective. Compared to ratio matching, usage, and is shown to be more effective for density modeling. We perform thorough experiments on both synthetic data density modeling and graph generation. The results demonstrate that our RMwGGIS achieves significant improvements over previous methods in terms of both effectiveness and efficiency.
RMwGGIS is more efficient in terms of computation and memory

---

Under review as a conference paper at ICLR 2022
REPRODUCIBILITY STATEMENT
We have made several efforts to guarantee the reproducibility of our work. To clearly present our technical method, in addition to the explanation in Section [ we provide the rigorous description of approach in Algorithm T and the detailed proof of the proposed Theorem 1 in Appendix [B] For experiments in Section 5 we provide the detailed description of datasets, model configurations, and evaluation metries for both synthetic data density modeling and graph generation. Our implementa- tions will be publicly available once the paper is published.
REFERENCES
David H Ackley, Geoffrey E Hinton, and Terrence J Sejnowski. A learning algorithm for boltzmann machines. Cognitive science, 91):147--66, 1985.
Avishek Joey Bose, Huan Ling, and Yanshuai Cao. Adversarial contrastive estimation. In Pro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics Linguiissss (Volume 1: Long Papers), pp. 1021-1032, 2018.
Ciwan Ceylan and Michael U Gutmann. Conditional noise-contrastive estimation of unnormalised models. In International Conference on Machine Learning, pp. 726-734. PMLR, 2018.
Barry A Cipra. An introduction to the ising model. The American Mathematical Monthly, 94(10): 937-959, 1987.
Hanjun Dai, Rishabh Singh, Dai, Charles Sutton, and Dale Schuurmans. Learning discrete energy-based models via auxiliary-variable local exploration. Advances in Neural Information Processing Systems, 33:10443-10455, 2020.
Yann Dauphin and Yoshua Bengio. Stochastic ratio matching of rbms for sparse high-dimensional inputs. Advances in Neural Information Processing Systems, 26:1340-1346 2013.
Peter Dayan, Geoffrey Hinton, Radford M Neal, and Richard Zemel. The helmholtz machine. Neural computation, 7(5):889-904, 1995.
Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc' Aurelio Ranzato. Residual energy-based models for text generation. International Conference on Learning Represen- lations, 2020.
Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. In Advances in Neural Information Processing Systems, volume 32, pp. 1608-3618, 2019.
Yilun Du, Shuang Li, Joshua Tenenbaum, and Igor Mordatch. Improved contrastive divergence training of energy based models. arXiv preprint arXiv:2012.01316 2020a.
Yilun Du, Joshua Meier, Jerry Ma, Rob Fergus, and Alexander Rives. Energy-based models for atomic-resolution protein conformations. In International Conference on Learning Representa- tions, 2020b.
Víctor Elvira, Luca Martino, David Luengo, and Jukka Corander. A gradicnt adaptive population importance sampler. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.
IEEE, 2015.
Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. Object detection with discriminatively trained bast- models. IEEE transactions on pattern analysis and ma- chine intelligence, 3229):1627-1645, 2009.
Ruiqi Gao, Yang Lu, Junpei Zhou, Song-Chun Zhu, and Ying Nian Wu. Learning generative con- vnets via multi-grid modeling and sampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9155-9164, 2018.
Ruiqi Gao, Erik Nijkamp, Diederik P Kingma, Zhen Xu, Andrew M Dai. and Ying Nian Wu. Flow contrastive estimation of energy- basedy models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7518-7528, 2020.
10

---

Under review as a conference paper at ICLR 2022
Rong Ge, Furong Huang. Chi Jin, and Yang Yuan. Escaping from saddle points online stochastic gradient for tensor decomposition. In Conference on learning theory, pp. 797- 842. PMLR, 2015.
Marco Gori, Gabriele Monfardini, and Franco Scarselli, A new model for learning in graph domains. In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, pp. 729-734. IEEE, 2005.
Will Grathwohl, Kuan-Chieh Wang, Born-Kerrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv: 2019.
Will Grathwohl, Kuan-Chieh Wang, Jörn-Henin Jacobsen, David Duvenaud, and Richard Zemel. Learning the stein discrepancy for training and evaluating energy- based models without sampling. In International Conference on Machine Learnnng pp. 3732-37477 PMLR, 2020.
Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, and Chris Maddison. Oops i took a gradient: Scalable sampling for discrete distributions. In International conference on machine learning. PMLR, 2021.
Arthur Gretton, Karsten M Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723-773) 2012.
Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference artificial intelligence and statistics, pp. 297-304. JMLR Workshop and Conference Proceedings, 2010.
Ryuichiro Hataya, Hideki Nakayama, and Kazuki Yoshizoe. Graph energy-based model for sub- structure preserving molecular design. arXiv preprini urXiv:2102.046000 2021.
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771-1800) 2002.
Geoffrey E Hinton. A practical guide training restricted boltzmann machines. In Neural networks: Tricks of the trade, pp. 599-619. Springer, 2012.
John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8):2554-2535 1982.
Aapo Hyvärinen. Some extensions of score matching. Computational statistics & data analysis, 51 (5):249922512, 2007.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score match- ing. Journal of Machine Learning Research, 6(4), 2005.
Ernst Ising. Beitrag zur Theorie des Ferromagnetismus. Zeitschrift fur Physik, 31(1):553-258, February 1925. doi: 10.10007/FF29980577.
Pierre E Jacob, John O'Leary, and Yves Atchadé. Unbiased markov chain monte carlo methods with couplings. Journal of the Royal Statistical Society: Series (Statistical Methodology), 82 (3):543-600, 2020.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEFE, (6(11):2278-2322 1998.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and F Huang. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. In International Conference Machine Learning, 2018.
11

---

Under review as a conference paper at ICLR 2022
Jenny Liu, Aviral Kumar, Jimmy Ba, Jamie Kiros, and Kevin Swersky. Graph normalizing flows. Advances in Neural Information Processing Systems, 32:13578 12.11558-13588, 2019.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. GraphEBM: Molecular graph generation with energy-based models. arXiv preprint urXiv:2102.0083 2021.
Youzhi Luo, Keqiang Yan, and Shuiwang Ji. GraphDF: A discrete flow model for molecular graph generation. International Conference on Machine Learning, pp. 7192-7203, 2021.
Siwei Lyu. Interpretation and generalization of score matching. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp. 359-366, 2009.
Radford M Neal al. Meme using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011.
Jiquan Ngiam, Zhenghao Chen, Pang Wei Koh, and Andrew Y Ng. Learning deep energy models. In Proceedings of the 28th International Conference on International Conference Machine Learning, pp. 1105-1112, 2011.
Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Ying Nian Wu. Learning non-convergent non- persistent short-run meme toward energy-basee model. Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 520255222422 5242, 2019.
Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Per- mutation invariant graph generation via score-based generative modeling. Artificial Intelligence and Statistics, 2020.
Yixuan Qiu, Lingsong Zhang, and Xiao Wang. Unbiased contrastive divergence algorithm for train- ing energy-based latent variable models. In International Conference on Learning Representa- tions, 2019.
Prajit Ramachandran, Barret Zoph, and Quoc V Le. Swish: self-gated activation function. arXiv preprint arxiv: 7:1, 2017.
Henry A Rowley Shumeet Baluja, and Takeo Kanade. Neural network-based face detection. IEEE Transactions on pattern analysis and machine intelligence, 20(1):23-38, 1998.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE transactions on neural networks, 20(1):61-80, 2008
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling, Modeling relational data with graph convolutional networks. In European Semantic Web Conference, pp. 593-607. Springer, 2018.
Ingmar Schuster. Gradient importance sampling. arXiv preprint arXiv: /507.05871. 2015.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. Al magazine, 29(3):93-93, 2008.
Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang. Graphaf: flow-based autoregressive model for molecular graph generation. In International Conference on Learning Representations, 2019.
Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region- based object detectors with online hard example mining. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 761-769, 2016.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In International Conference on Artificial Neural Networks, pp. 412- 422. Springer, 2018.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pp. 11918-1 11918-11930, 2019.
12

---

Under review as a conference paper at ICLR 2022
Yang Song and Diederik P Kingma. How to train your energy based models. arXiv preprint arXiv:2101.03288, 2021.
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Uncertainty in Artificial Intelligence, pp. 574-584. PMLR, 2020.
Tijmen Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference Machine learning, pp. 1064 1071, 2008.
Pascal Vincent. connection between matching and denoising autoencoders. Neural compu- tation, 23(7):16111167, 2011.
Max Welling and Yee W Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-I1), pp. 681-688, 2011.
Jianwen Xie, Yang Lu, Song-Chur Zhu, and Yingnian Wu. A theory of generative convnet. In International Conference on Machine Learning, pp. 2635-2644, 2016.
Jianwen Xie, Song-Chun Zhu, and Ying Nian Wu. Synthesizing dynamic patterns by spatial- temporal generative convnet. In Proceedings of the ieee conference computer vision and pattern recognition, pp. 7093-7101, 2017.
Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, and Ying Nian Wu. Learning descriptor networks for shape synthesis and analysis. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6629-8638, 2018.
Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. Graphrnn: Generat- ing realistic graphs with deep auto-regressive models. International conference on machine learning, pp. 5708-5717 PMLR, 2018.
Giacomo Zanella. Informed proposals for local meme in discrete spaces. Journal of the American Statistical Association, (55530))852-865,22 2020.
Song Chun Zhu, Yingnian Wu, and David Mumford. Filters, random fields and maximum entropy (frame): Towards unified theory for texture modeling. International Journal of Computer Vision, 1772):107-126, 1998.
13

---

Under review as a conference paper at ICLR 2022
A THE DETAILED DERIVATION OF EQ. (6)
The detailed derivation of Eq. (6 is as follows.
TRA (0, x) = de immp 1) e Eolx)-Eo (x-
m(x -ola))Eolla -oolar
d m(x -) (e))
(oo) Fol( +))2 -((x+t)
(13)
n(x=:)
m(x [e - -oo
de (_((x-))
ni
B
PROOF OF THEOREM
Proof. According to Eq. 77 we have
(eFoa))-oo Var (JRM(0,x),.) = = Var m(x=:) [o())) Fo(m .) .)12
d
(14)
n(x
Then we can compare the variance of the estimator based on n nx==))) and n(x Formally,
Var
(n).)(
(15)
d²
m((x=)) (Foo(x)-Eoloaa Eo(
Var
(16)
n
d
m(x (c Eo () 272
m(x (e (2) Eo ()) 27
En En_cnn
E Em- (m-.)
8
11 (x.
1 (x=
(17)
d²
m(x (o Eo (cc_.) ())
(O.
E (e_
(18)
n1" (xxxx=))
d
m(x 1) [e (o()))) Fx( (x 27
d²
TRAS (0, 22
n" (x=
(19)
d
n² (x=
d
d²
m (x
Fo(x) Foo()) Eo( 1)
Fo (x) Fol Fooo=_o())))
1)
Tna (0, xx)
(20)
d
km
d
JRM (0, )
ml m( -oo(e)))
(21)
d
d²
m m)(x..)) 2) (ol0)) -.ol E0 ()) 1)
Jnn
n1(xx_)) //n((= n((
(22)
d
d2
m(x [e [oo (m)
2
JRM (A, (0,z) 12
n1 (((==))
n(xx_.)
(23)
d
n(x=-1)
1=1
d²
d
m(x [e Fo(x)) (o)( Fo(m .)7
TRAS (0, x))
(2(
(24)
d
n ((
1=1
14

---

Under review as a conference paper at ICLR 2022
d²
m( (e
- Eo (0 2)
TRM (O, x0) 2
Ex io
(25)
n(x
Var URM((,22))
(26)
Eq. 18) can be derived because the estimator is unbiased no matter what proposal distribution is
applied. Eq. (20 is obtained by choosing n' (x-i) [e (e())) Ep(e-1))
( - Eq. (21 holds
since m ( -1) for 1....,d d. To derive Eq. 23 we apply the Cauchy- Schwarz inequality
E azb:) I E of) EE o2). This completes the proof of Theorem 1.
C CONNECTION WITH HARD NEGATIVE MINING
Here, we provide an additional insight understand why the second property, described in Sec- tion 3.3 can lead to better optimization, by connecting it with hard sample mining.
Hard sample mining (Felzenszwalb et al. 2009 Rowley et al. 1998) has been widely applied to train deep neural networks (Shrivastava al. 2016). Our RMwGGIS is particularly highly related to hard negative training strategies. The basic idea for hard negative mining is to pay more attention hard negative samples during training, which can usually achieve better performance since it can reduce false positives. In our setting of discrete EBMs, each training sample æ is a positive sample, and its energy should be pushed down. For each positive sample 2, all Ø- for 1,2, ... d are negative samples, and their energy should be pushed up. Our RMwGGIS with the specific proposal distribution shown in Eq. ( can approximately choose the æ x's's s that currently have low energies with larger probabilities. This has the same philosophy as hard negative mining. Specifically, in our case, x x=s's with low energies are hard negative samples since they are the most offending terms, which are close the positive sample and have low energies.
Since our proposal distribution defined in Eq. M approximates the provable optimal proposal dis- tribution given in Theorem 1, our proposal distribution thus approximately performs "hard negative mining". The natural follow-up question is how accurate is the approximation and how does it af- fect the learning process? We answer this question by analyzing the following two stages during learning, which intuitively show that our R.MGGII is technically sound.
Eo(-)
Ea(-)
1
2 2
Φ 1
(a)
(b)
An intuitive illustration of the approximation: (a) Stage and (b) Stage II. The black curves denote the energy functions. Green nodes and brown nodes represent the true energy values and approximated values of neighbors, respectively. Note that the approximated values are obtained based on Taylor series of E.(+) at æ, as shown in Eq. R For clarity, we only show neighbors of x in this figure, but this illustration can also be extended to include all neighbors.
Figure 2:
Stage I. As shown in Figure 2 (a), at the early stage of learning, the energy function is not learned well, thus the energy Ex( of positive sample is not smaller than its all neighbors. In this case, there are some neighbors of z which have lower energies than Ex(x), such as σ in Figure 2 (a). Therefore, "hard negative mining" is in demand in this stage. Under this situation, our approximated energies neighbors could help to perform "hard negative mining". To be specific, the estimated energies of neighbors are close the true energies, and the estimated energy of - is much lower
15

---

Under review as a conference paper at ICLR 2022
Table 5: Comparison of resulting objective values. The top two lowest values on each dataset highlighted as 1st and 2nd.
Method
2xpirals
8gaussians
circles
moons
pinwheel
swissroll
checkerboard
Ratio Matching
46.02
39.26
31.82
28.57
28.50
37.52
26.05
RMwGGIS (unbiased)
26.84
30.15
29.89
27.80
28.08
32.20
26.09
RMwGGIS (biased)
27.15
29.31
29.54
27.75
27.30
29.45
26.06
60000
flterations
180000
240000
Ratio
Matching
RMwRAND
RMwGGIS
(unbiased)
RMwGGIS
(biased)
Figure 3: Visualization of learned energy functions w.s. number of learning iterations on the 256- dimensional 2spirals dataset.
than the estimated energy of 2- Thus, our proposal distribution will sample æ with higher probability. This actually works as "hard negative mining" since 2 is the current most offending term, i.e., the so-called hard negative sample.
Stage IT. After learning for while, we can obtain relatively good energy function, where the positive sample x locates in the low energy area compared to its local neighbors. In this case our approximation is less accurate. Fortunately, in this case, "hard negative mining" is not that necessary since there do not exist many offending terms. Specifically, as shown in Figure 2 (b), the energies of - and x-2 are safely higher than Eg (z).
Even though the above analysis is based on a simplified example, we believe can serve as good intuitive understanding of why our RMwGGIS performs better than ratio matching.
D WHY DOES THE BIASED VERSION PERFORM BETTER?
Here, we provide an intuitive explanation on why our biased RMwGGIS usually performs better than unbiased version.
Following our analysis of the connection between our method and hard negative mining, as described in Appendix C it is obvious that both unbiased version (i.e., Eq. 77) and biased version (i.e.,
Eq. (12) perform "hard negative sampling". The difference lies in the coefficients for different
terms. Specifically, the biased version gives the same weights to all sampled terms. In contrast, the
unbiased version provides a weight (
(t) to each sampled term x =0 as shown in Eq. 77. Note
that m( 1/2 for all terms and n(x22) would be larger x has lower energy. Hence, the unbiased version provides smaller weights for terms with lower energies. In other words, among its selected offending terms (i.e., hard negative samples), it pay least attention to the most offending terms, which could be less effective than biased version with equal weights. This could explain why
our biased RMwGGIS usually performs better than unbiased version.
E COMPARISON OF ACHIEVED OBJECTIVE VALUES
Specifically, for all the learned energy functions in Table I we sample 4000 data points on each dataset and evaluate the resulting objective value defined by Eq. 44. The results summarized in Table We can observe that our unbiased and biased RMwGGIS indeed achieve lower objective values, which further demonstrates that our proposed RWGGGIS can be optimized better.
16

---

Under review as a conference paper at ICLR 2022
F VISUALIZATION OF LEARNED ENERGY FUNCTIONS ON HIGHER HIGHER- DIMENSIONAL DATA
The learned energy functions w.s.t. number learning iterations on the 256-dimensional 2spirals dataset are qualitatively visualized in Figure B
G DETAILED SETTINGS OF EBM (GWG) ON GRAPH GENERATION
We use the official open-sourced implementation of EBM (GWG) to perform its graph genera- tion experiment. We train models with persistent contrastive divergence (Tieleman) 2008 with a buffer size of 200 samples. We use the Adam optimizer Kingma & Ba 2015 with a learn- ing rate of le-4 and batch size of 200. The following hyperparameters are tuned and the fi- nally chosen ones are underlined: buffer initialization rate E 0.2 0.4, 0.6} and MCMC steps {100, 200, 500, 1000, 2000}.
H VISUALIZATION OF GENERATED GRAPHS
Generated graph samples shown in Figure 4
Training data
RMwGGIS samples
Figure 4: Visualization training data and samples drawn from the energy function learned by our RMwGGIS for graph generation.
I DETAILS OF TRAINING ISING MODELS
Lattice Ising Models. Ising models (Cipra 1987 is firstly developed to model the spin magnetic particles [sing 1925). For Ising models, our energy function can be naturally defined as
E(x) - Ja b"
(27)
where J and b are the parameters. J is the connectivity matrix which indicates the correlation across dimensions in æ. We follow one specific setting in Grathwohl et al. (202I), where all of the non-zero entries of J are identical (denoted as and J is the adjacency matrix of cyclic 2D lattice structure. Therefore,
E ( -ox Jæ DT x.
(28)
17

---

Under review as a conference paper at ICLR 2022
Setup. We follow Grathwohl et al. (2021) for our experimental setting. To be specific, cre- ate a model using a 25 lattice and a 0.25, thus leading to 625 dimensional distribution. For training the model, 2000 examples are generated via 000, 000 steps of Gibbs sampling. We apply our proposed RMwGGIS method to train the model. The number samples & used in our RMwGGIS objective is set to 10. We use Adam optimizer (Kingma & Ba 2015) with learning rate of le-4 and a batch size of 100. E1 penalty with strength 0.01 is used to encourage sparsity. In terms of baselines, trastive divengence (Tieleman] 2008). The number of steps for MCMC per training iteration is 15, 10, 25, 50, 1.0 The samplers Gibbs and Hetbs-WWithGrradie
consider the approaches which train discrete EBMs with persistent con-
(GWG) (Grathwohl al. Results of EBM (GWG) are obtained by running the official implementation from Grathwoh! Results of EBM (Gibbs) are obtained by reading from Figure in Grathwohl et al.
202
al.
222TT.
202
Evaluation. We evaluate the performance by computing the pootmeen-squaed-erroo (RMSE) be- tween the learned connectivity matrix J and the true matrix J. In addition, we compare the effi- ciency by reporting the running time for each iteration. To be specific, for comparing efficiency, we use the same batch size 100 for our method and baselines. The report time is the average over 100 iterations.
18


Reviews:
[{"review_id": "pqfU87p_zIV", "reviewer": "Reviewer_oca8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \n- The paper is well written and contains detailed method derivation and discussions. Overall, it is easy to follow.\n- Nice discussions to related works.\n- The method derivation is sound and the proposed solution is a reasonable way to alleviate the expensive computation/memory requirement of the original ratio matching approach.\n- Although motivated from computational perspective for fast approximation of an expensive quantity, the proposed method even achieves a better performance than original method in most tasks, which is a surprising result and achieves both computation and performance improvement. The intuitive explanations look reasonable to me.\n\nWeakness/Questions:\n- The proposed approach seems to heavily rely on the assumption that the data is binary, which is not applicable to more realistic data such as natural language modeling (or even graph with different types of nodes and edges?). Indeed, an efficient and effective generalization of ratio matching to non-binary discrete data seems more challenging, interesting and useful.\n- The taylor approximation in eq (8) may not be accurate enough since the difference between x and x_-i is not close to zero. So the approximation error might be larger depending on the structure of the energy function (e.g. non-smooth) and the number of dimension (for d=2, x and x_-i may be very different in the sample space).\n- In table 1, there is no standard deviation calculated from multiple random seeds. Also why biased version is better than unbiased version in most experiments?\n- After obtain the energy function, how do you generate graphs? (I suppose using Gibbs sampling?)\n- Potential issue with Table 2: I think you are using the same s (number of proposal samples) for various data dimension from 32 to 2048. However, we typically need larger s for larger dimension d to ensure good performance, and in this case, the speedup/memory saving may not be that large?\n- All experiments are \"toy\" data, possibly because the method is limited to model binary data.", "summary_of_the_paper": "The paper propose a Monte Carlo approximation for the expensive ratio matching method for learning discrete energy-based models. The key idea is to first write the ratio matching objective as an expectation wrt a uniform distribution, then use importance sampling for a more efficient estimation, where the optimal proposal distribution can be approximated by the gradient of the energy wrt the data space. Empirically, such an approximation shows even better results than the original expensive objective, possibly due to regularization from stochasticity and more focus on neighbors with low energies.", "summary_of_the_review": "Please see the detailed comments section.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Rida2dFhshb", "reviewer": "Reviewer_KK3T", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\nTraining EBMs on discrete data is a difficult task and few methods currently exist to do so in high dimensions. RM is an appealing option because it avoids the MCMC sampling typically used to train EBMs at scale. The method appears to be a clear win on top of RM (when it can be applied -- not all energy discrete functions can be differentiated). \n\nBeyond this, the method appears correct and valid. My main concerns lie in the method’s ability to scale to higher-dimensional settings than demonstrated in the paper.\n\nWeaknesses:\n\nThe main weakness of this work is in its empirical evaluation. The authors only validate the method in two settings; toy datasets and small graph datasets. Further, their only evaluation is via MMD. Evaluation of EBMs is difficult, but in small-scale settings likelihood can be estimated with AIS. Likelihood-based evaluation would allow comparison to autoregressive models or VAEs, for example.\n\nI am concerned with the comparison to Gibbs-With-Gradients training. How was this done? Did you use PCD? If so, what were your buffer size, refresh rate, and the number of steps per training iteration? PCD can be sensitive to these parameters and if not set properly the results could be very bad. The authors should include these experimental details in the paper (at least in the appendix).\n\nI am not convinced by the scale of the experiments as well. Prior work [1,2] has successfully trained discrete EBMs on datasets with thousands of dimensions. The results in this work are much smaller-scale. Does the method have difficulties scaling to these settings? If so then that is certainly a weakness of the method and it should be mentioned. You could compare results to the MNIST/Potts/Ising training experiments in [1] (no need to do all, but at least one experiment demonstrating scaling ability would be good to have). This would give a better context for the performance of this method relative to other recent approaches. \n\n----------Post Author Response-------------\nI thank the authors for their response to my feedback. I appreciate the new experiments. I think they give additional clarity on how the method performs and scales to higher-dimensional data. I am convinced the method has merits and I will raise my score to a 6. I am still concerned that the method will have difficulties scaling to more complicated problems/models and the current results do not demonstrate that. Having said that, this method is generally applicable and could be a valuable contribution to the community. \n", "summary_of_the_paper": "This paper proposes a more efficient version of ratio matching (RM) for training discrete energy-based models. The proposed method subsamples the dimensions to use in the original RM objective and then uses importance sampling to reduce variance. The importance sampling distribution is based on a Taylor-series approximation to the target energy function which approximates the minimal-variance importance sampling distribution. The authors show that this estimator is considerably more efficient than standard RM and enables RM to be applied to training EBMs in higher dimensions. The authors demonstrate their estimator on some toy datasets and to fit EBMs on graph data. The method is evaluated using MMD.", "summary_of_the_review": "The paper proposes an interesting and novel variant of RM that enables it to get around the memory/compute constraints of the original method. The method appears valid and appears to work but the empirical evaluation is lacking likelihood evaluation, experimental details, comparison to recent approaches and does not demonstrate that the method can scale to relevant problems. Unless these issues can be rectified, I cannot argue for the paper's acceptance. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "IBVe6o6tic6", "reviewer": "Reviewer_DiXS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper seems to contain interesting material. Some point must be clarified.\n\n- How to draw samples from your proposal n(x)? this is the main issue in your method.\n\n- Please clarify the relationship with the papers:\n\nV. Elvira, L. Martino, D. Luengo, J. Corander, \"A Grandient Adaptive Population Importance Sampler\", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2015.\n\nIngmar Schuster, Gradient Importance Sampling, arXiv:1507.05781, 2015.\n\n", "summary_of_the_paper": "The authors propose a new gradient-based importance sampling scheme, applied to learn parameter in energy-based models (EBMs).", "summary_of_the_review": "The paper seems to contain interesting material, however the degree  of novelty seems to be a bit incremental.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6ScPSHJn4L", "reviewer": "Reviewer_Btjs", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strength:\n\nThe paper is well motivated, clearly written, and seems technically sound. The paper does a good job in analyzing the limitations of ratio matching in expensive computation and excessive memory requirements, and in describing how it can leverage the gradient of the energy w.r.t. the discrete data space to address these limitations. The idea is simple, but can potentially be very useful in making ratio matching more practical for learning discrete energy-based models.\n\n## Weaknesses:\n\n- While the paper demonstrates improvements over vanilla ratio matching, it lacks comparison with other types of methods for learning discrete energy-based models. For example, maximum-likelihood training with MCMC sampling (e.g. with Gibbs sampling or the more advanced Gibbs-with-gradients, which the authors acknowledge is a big inspiration for RMwGGIS), can be easily applied to the same problems, and should be included as additional baselines. While the improvements over ratio matching are nice, if RMwGGIS cannot outperform other readily available methods, then it picked the wrong method to improve upon, and its improvements over ratio matching cannot be justified.\n- I am not convinced by the current arguments in the section **Better optimization?** in Sec. 3.3, and I don't think this point is verified empirically as the authors claim. As the authors mention at the end of page 2, the estimator given by Eq. (3)/(4) is consistent, and if Eq. (3)/(4) is minimized perfectly, the obtained model distribution will capture the data distribution exactly. The authors claim that RMwGGIS, with a biased estimator, performs better than ratio matching due to better optimization. Does that mean that the theta learned with the biased objective function (12) actually results in lower value for the objective function in Eq. (3)/(4)? I.e. somehow optimizing a different objective function (Eq. (12)) leads to better optimization of Eq. (3)/(4) than directly optimizing Eq. (3)/(4)? I would like to see this confirmed in experiments (which is a very simple evaluation). If it's really the case, I would like to see some analysis on why this is the case, since Eq. (12) is picked somewhat randomly, without theoretical justification as Eq. (7). If this is not the case, then there is no better optimization, which means Eq. (3)/(4) is not a good objective function, and the paper should rethink its foundation, or explain where the better performance comes from.\n", "summary_of_the_paper": "This paper proposes ratio matching with gradient-guided importance sampling (RMwGGIS) as an attempt to address the expensive computation and excessive memory requirement of ratio matching in learning discrete energy-based models, and demonstrates the advantages of RMwGGIS over ratio matching with experiments on density modeling and graph generation.", "summary_of_the_review": "While the paper is promising, given the two major concerns above, I don't think the paper can be accepted in its current form, and recommend weak reject.\n\n---------------------------\n\nWhile the authors clarified certain points, after the discussions I still have concerns over the overall presentation. Although the method demonstrates promising results, I don't feel the theory is properly explaining the improved performance, and more analysis/investigation is needed to clarify the claimed hard sample mining perspective. I still recommend rejection.\n\nThe authors have repeatedly tried to force their conclusions on me in their response. I want to emphasize that I still don't find the presentation to be convincing, and I believe the paper in its current form hasn't sufficiently analyzed an important point it touched upon.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper introduces a way of making Ratio Matching (RM) scale better to high-dimensional data when training energy-based models (EBMs). The main idea is to estimate the sum over the datapoint dimensions in the RM objective with importance sampling (IS), achieving computational savings by using fewer samples than dimensions. A key part of the method is a proposal that uses gradient information w.r.t. discrete variables to efficiently approximate the optimal (minimum variance) proposal, resulting in much better performance compared to uniform sampling. The authors also introduce a biased version of the estimator that samples from the same proposal but drops the importance weights when averaging over the samples, which, somewhat surprisingly, outperforms the unbiased version.

The idea of using Monte Carlo estimation based on importance sampling to speed up Ratio Matching is novel and sound. The use of gradient information to approximate the optimal IS proposal is also novel in this context, though the idea of using gradients this way to reduce the number of EBM energy function evaluations comes from Grathwohl et al. (2021), where it was used to speed up Gibbs sampling.

While the method is well described, the paper is insufficiently rigorous in several places, most importantly in claiming that Eq. 4 corresponds to Ratio Matching, which is not true. Eq. 4 is instead equivalent to the objective for Generalized Score Matching (GSM) given by Eq. 17 in (Lyu, 2009). Crucially, while both GSM and RM recover the true model if the model class is nonparametric/unconstrained, as is stated in (Lyu, 2009) (and thus agree with each other as well as with maximum likelihood estimation) ), they do not yield the same solution for constrained model classes such as neural networks. This means that the method in the paper implements GSM and not RM. Unlike RM, which has been used widely in the literature, GSM is essentially empirically unproven and thus is a less interesting choice. The main difference between the GSM and RM objectives is the presence of the squashing function g(u) = 1/(1+u) around the probability ratios in RM to avoid division by zero, when the probability in the denominator is vanishingly small (as is explained above Eq. 12 in (Hyvarinen, 2007)). This means GSM is likely to be prone to stability issues due to using probability ratios directly. This is one possible explanation for the puzzling empirical results in the paper, where the proposed sampling-based methods outperform the exact method they are supposed to approximate, with the biased method clearly performing best. The intuition-based arguments made in the paper to explain these results are not convincing and need to be improved upon. While, as the authors pointed out in their response, it is possible to apply the strategy in the paper to RM by applying IS to Eq. 3 instead of Eq. 4, that would be essentially a different paper.

One example of puzzling experimental results is Figure 3, which shows that the base method ("Ratio Matching") does not find the correct solution while the proposed approximate methods do. This suggests that there is something wrong either with the method (e.g. with the objective, as mentioned above) or with the experimental setup. In either case, the cause needs to be thoroughly investigated.

Currently the empirical evaluation is primarily MMD based, relying on sampling from the model using MCMC. Ensuring that MCMC chains mix sufficiently well to sample from the true distribution by visiting all of its modes is difficult, and it is important to provide some evidence that this was done. As suggested by a reviewer, the results would be substantially strengthened by reporting the log-likelihoods for the models, estimated e.g. using AIS, even if that requires including scaled-down versions of some of the experiments.

The title of the paper is misleading and should be changed because the proposed method is specific to EBMs for binary data, even if the intent is to extend it to other types of discrete data in the future.

The clarification and additional results provided by the authors to the reviewers and the AC were appreciated, but unfortunately the outstanding issues with the paper are too major to allow acceptance at this point. The main idea of the paper has substantial promise however, and the authors are encouraged to develop it to its full potential by addressing the points from this meta-review as well as the additional ones from the reviewers.

Bibliography correction: Hyvarinen is the solo author of "Estimation of Non-Normalized Statistical Models by Score Matching". Peter Dayan was the editor of that paper and not a co-author. Please correct your bibliography.
Decision: Reject