Original: Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable ModelsAnonymous Author(s)
Revised: Variational (Gradient) Estimate of the Score Function in Energy-based Latent Variable Models

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: The learning and evaluation of energy-based latent variable models (EBLVMs)without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable.
Revised: The learning and evaluation of energy-based latent variable models (EBLVMs) without any structural assumptions are highly challenging, because the true posteriors and the partition functions in such models are generally intractable.

Original: This paperpresents variational estimates of the score function and its gradient with respectto the model parameters in a general EBLVM, referred to as VaES and VaGESrespectively.
Revised: This paper presents variational estimates of the score function and its gradient with respect to the model parameters in a general EBLVM, referred to as VaES and VaGES respectively.

Original: The variational posterior is trained to minimize a certain divergenceto the true model posterior and the bias in both estimates can be bounded by thedivergence theoretically.
Revised: The variational posterior is trained to minimize a certain divergence to the true model posterior and the bias in both estimates can be bounded by the divergence theoretically.

Original: With a minimal model assumption, VaES and VaGEScan be applied to the kernelized Stein discrepancy (KSD) and score matching(SM)-based methods to learn EBLVMs.
Revised: With a minimal model assumption, VaES and VaGES can be applied to the kernelized Stein discrepancy (KSD) and score matching (SM)-based methods to learn EBLVMs.

Original: Besides, VaES can also be used to estimatethe exact Fisher divergence between the data and general EBLVMs.
Revised: Besides, VaES can also be used to estimate the exact Fisher divergence between the data and general EBLVMs.

Original:  Introduction
Revised:  Introduction

Original: An energy-based model (EBM) [17] ﬁrstly associates an energy E θ ( v ) with parameter θ to each conﬁguration of visible variables v , and then normalizes the energy to get a probability density as
Revised: An energy-based model (EBM) [17] ﬁrstly associates an energy E θ ( v ) with parameter θ to each conﬁguration of visible variables v , and then normalizes the energy to get a probability density as

Original: [Equation]
Revised: [Equation]

Original:  where ˜ p θ ( v ) is the unnormalized density and Z ( θ ) 
Revised: , where ˜ p θ ( v ) is the unnormalized density and Z ( θ ) =

Original: = (cid:82) e −E θ ( v ) d v is called the partition function.
Revised: (cid:82) e −E θ ( v ) d v is called the partition function.

Original: As for the learning and evaluation, maximum likelihood estimate (MLE) [9, 31, 36, 24, 25, 2, 3, 7] andnoise-contrastive estimation (NCE) [8, 26, 5] are commonly used strategies.
Revised: As for the learning and evaluation, maximum likelihood estimate (MLE) [9, 31, 36, 24, 25, 2, 3, 7] and noise-contrastive estimation (NCE) [8, 26, 5] are commonly used strategies.

Original: However, MLE requiresto estimate the partition function and NCE requires to design a proper noise distribution manually.
Revised: However, MLE requires to estimate the partition function and NCE requires to design a proper noise distribution manually.

Original: In comparison, Fisher divergence based methods provide an efﬁcient and effective alternative totrain [13, 29, 34, 20] and evaluate [6] such models.
Revised: In comparison, Fisher divergence based methods provide an efﬁcient and effective alternative to train [13, 29, 34, 20] and evaluate [6] such models.

Original: The calculation of the Fisher divergence is basedon the score function of an EBM, which is deﬁned as ∇ v log p θ ( v ) .
Revised: The calculation of the Fisher divergence is based on the score function of an EBM, which is deﬁned as ∇ v log p θ ( v ) .

Original: Notably, the score functionis independent of the partition function because ∇ v log pθ ( v ) = ∇ v log ˜
Revised: Notably, the score function is independent of the partition function because ∇ v [Equation] p

Original: p θ ( v ) − (cid:40)(cid:40)(cid:40)(cid:40)(cid:40)(cid:40) ∇ v log Z ( θ ) =
Revised: [Equation]

Original: ∇ v log ˜
Revised: ∇ v log ˜

Original: p θ ( v ) , and thereby tractable.
Revised: p θ ( v ) , and thereby tractable.

Original: Energy-based latent variable models (EBLVMs) [9, 11, 27, 28, 16, 19, 1] incorporate hidden variablesh to EBMs, which increases the model capacity and enables feature extraction.
Revised: Energy-based latent variable models (EBLVMs) [9, 11, 27, 28, 16, 19, 1] incorporate hidden variables h to EBMs, which increases the model capacity and enables feature extraction.

Original: Such a model deﬁnesa joint probability density over the visible variables v and the latent variables h as follows
Revised: Such a model deﬁnes a joint probability density over the visible variables v and the latent variables h as follows

Original: [Equation]
Revised: [Equation]

Original: where E θ ( v , h ) is the energy function with θ as its parameters, ˜
Revised: where E θ ( v , h ) is the energy function with θ as its parameters, ˜

Original: p θ ( v , h ) is the unnormalized densityand Z ( θ )
Revised: p θ ( v , h ) is the unnormalized density and Z ( θ )

Original: = (cid:82) e −E θ ( v , h )
Revised: = (cid:82) e −E θ ( v , h )

Original: d v d h is the partition function.
Revised: d v d h is the partition function.

Original: At the cost of increasing expressivenesspower, the posteriors of EBLVMs p θ ( h | v ) are generally intractable without a structural assumption [30, 33].
Revised: At the cost of increasing expressiveness power, the posteriors of EBLVMs p θ ( h | v ) are generally intractable without a structural assumption [30, 33].

Original: As a result, commonly used learning and evaluating algorithms in EBMs are not directlyapplicable to general nonstructural EBLVMs.
Revised: As a result, commonly used learning and evaluating algorithms in EBMs are not directly applicable to general nonstructural EBLVMs.

Original: To learn and evaluate general nonstructural EBLVMs, we propose variational estimates of the score function and its gradient w.r.t.
Revised: To learn and evaluate general nonstructural EBLVMs, we propose variational estimates of the score function and its gradient w.r.t.

Original: the model parametersin such models, referred to as VaES and VaGES .
Revised: the model parameters in such models, referred to as VaES and VaGES .

Original: Such estimates ﬁrstly introduce a tractable variational posterior q φ ( h | v ) to approximate the true one p θ ( h | v ) , and then estimate the score function∇ v log p θ ( v ) or its gradient ∂ ∇ v log p θ ( v ) ∂ θ by Monte Carlo methods based on samples from q φ ( h | v ) .
Revised: Such estimates ﬁrstly introduce a tractable variational posterior q φ ( h | v ) to approximate the true one p θ ( h | v ) , and then estimate the score function ∇ v log p θ ( v ) or its gradient ∂ ∇ v log p θ ( v ) ∂ θ by Monte Carlo methods based on samples from q φ ( h | v ) .

Original: We show that under some assumptions, the bias introduced by the variational posterior can be boundedby the square root of the KL divergence or the Fisher divergence [15] between the variational posteriorand the true posterior.
Revised: We show that under some assumptions, the bias introduced by the variational posterior can be bounded by the square root of the KL divergence or the Fisher divergence [15] between the variational posterior and the true posterior.

Original: With a minimal model assumption, VaES and VaGES can be applied to thekernelized Stein discrepancy (KSD) [22] and score matching (SM)-based methods [13, 34, 20] tolearn EBLVMs.
Revised: With a minimal model assumption, VaES and VaGES can be applied to the kernelized Stein discrepancy (KSD) [22] and score matching (SM)-based methods [13, 34, 20] to learn EBLVMs.

Original: Besides, VaES can also be used to estimate the exact Fisher divergence between thedata and general EBLVMs.
Revised: Besides, VaES can also be used to estimate the exact Fisher divergence between the data and general EBLVMs.

Original:  Method
Revised:  Method

Original: In this paper, we want to estimate the score function and its gradient w.r.t.
Revised: In this paper, we want to estimate the score function and its gradient w.r.t.

Original: the model parameters in an
Revised: the model parameters in an

Original: EBLVM.
Revised: EBLVM.

Original: Firstly recall [33] that the score function of an EBLVM can be expressed as
Revised: Firstly recall [33] that the score function of an EBLVM can be expressed as

Original: [Equation]
Revised: [Equation]

Original:  where p θ ( v ) =
Revised: (1) where p θ ( v ) =

Original: (cid:82) p θ ( v , h )
Revised: (cid:82) p θ ( v , h )

Original: d h is the marginal probability density and p θ ( h | v ) is the posteriorprobability density.
Revised: d h is the marginal probability density and p θ ( h | v ) is the posterior probability density.

Original: We then show that the gradient of the score function w.r.t.
Revised: We then show that the gradient of the score function w.r.t.

Original: the model parameterscan be decomposed into a term of covariance and a term of expectation:
Revised: the model parameters can be decomposed into a term of covariance and a term of expectation:

Original: [Equation]
Revised: [Equation]

Original: where the covariance term is the covariance between two random vectors ∇ v log ˜
Revised: where the covariance term is the covariance between two random vectors ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ,∇ θ log ˜
Revised: θ ( v , h ) , ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) with h ∼ p
Revised: θ ( v , h ) with h ∼ p

Original: θ ( h | v ) and the expectation term is the expectation of a random matrix∂ ∇ v log ˜
Revised: θ ( h | v ) and the expectation term is the expectation of a random matrix ∂ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ with h ∼ p θ ( h | v ) .
Revised: θ with h ∼ p θ ( h | v ) .

Original: A proof of Eqn.
Revised: A proof of Eqn.

Original: (3) is provided in Appendix A. For simplicity,we denote the covariance term and the expectation term as c ( v ; θ ) and e ( v ; θ ) respectively.
Revised: (2) is provided in Appendix A. For simplicity, we denote the covariance term and the expectation term as c ( v ; θ ) and e ( v ; θ ) respectively.

Original: Eqn.
Revised: Eqn.

Original: (2,3) inspire us with a Monte Carlo estimation of ∇ v log p θ ( v ) and ∂ ∇ v log p θ ( v ) ∂ θ in an EBLVM.
Revised: (1,2) inspire us with a Monte Carlo estimation of ∇ v log p θ ( v ) and ∂ ∇ v log p θ ( v ) ∂ θ in an EBLVM.

Original: However, it needs samples from p θ ( h | v ) , which requires the Markov chain Monte Carlo (MCMC)method [23] when only the energy function is accessible, leading to a high time cost.
Revised: However, it needs samples from p θ ( h | v ) , which requires the Markov chain Monte Carlo (MCMC) method [23] when only the energy function is accessible, leading to a high time cost.

Original: To efﬁcientlyget samples, we learn a variational posterior q φ ( h | v ) as an approximation of p θ ( h | v ) by
Revised: To efﬁciently get samples, we learn a variational posterior q φ ( h | v ) as an approximation of p θ ( h | v ) by

Original: [Equation]
Revised: [Equation]

Original: where D is the KL divergence when h is discrete and the Fisher divergence when h is continuousand the reasons will be speciﬁed in Sec. 
Revised: where D is the KL divergence when h is discrete and the Fisher divergence when h is continuous and the reasons will be speciﬁed in Sec. 2.2.

Original: In practice, we often need to estimate Eqn.
Revised: In practice, we often need to estimate Eqn.

Original: (2) and
Revised: (1) and

Original: Eqn.
Revised: Eqn.

Original: (3) on a minibatch of data v 1: M , where M is the batch size.
Revised: (2) on a minibatch of data v 1: M , where M is the batch size.

Original: In this case, we don’t initialize anew variational parameter φ for a new minibatch.
Revised: In this case, we don’t initialize a new variational parameter φ for a new minibatch.

Original: We only maintain one parameter φ and update φfor K times on the minibatch  by:
Revised: We only maintain one parameter φ and update φ for K times on the minibatch with a preﬁxed learning rate scheme α by:

Original: [Equation]
Revised: [Equation]

Original: where α is a preﬁxed learning rate scheme.

Original: 2.1 Variational (Gradient) Estimate of the Score Function in EBLVMs
Revised: 2.1 Variational (Gradient) Estimate of the Score Function in EBLVMs

Original: First, we consider estimating the score function.
Revised: First, we consider estimating the score function.

Original: According to Eqn.
Revised: According to Eqn.

Original: (2), a naive variational estimate is
Revised: (1), a naive variational estimate is

Original: [Equation]
Revised: [Equation]

Original: where L is the number of samples from q φ ( h | v ) .
Revised: where L is the number of samples from q φ ( h | v ) .

Original: By noticing that
Revised: By noticing that

Original: [Equation]
Revised: [Equation]

Original: we can subtract ∇ v log q φ ( h | v ) from the naive estimator without changing the value of the expectation, and the resulting variational estimate of the score function (VaES) is
Revised: we can subtract ∇ v log q φ ( h | v ) from the naive estimator without changing the value of the expectation, and the resulting variational estimate of the score function (VaES) is

Original: [Equation]
Revised: [Equation]

Original: When the variational posterior q φ ( h | v ) is equal to the true posterior p θ ( h | v ) , VaES( v ; θ , φ )
Revised: When the variational posterior q φ ( h | v ) is equal to the true posterior p θ ( h | v ) , VaES( v ; θ , φ )

Original: = [Equation] log 
Revised: = ∇ v log ˜

Original: [Equation]

Original: v
Revised: p

Original: [Equation] v log [Equation]
Revised: θ ( v , h ) p θ ( h | v ) = ∇ v log ˜

Original: p θ ( v ) is a deterministic variable and is exactly equal to the score function.
Revised: p θ ( v ) is a deterministic variable and is exactly equal to the score function.

Original: Then, we consider estimating the gradient of the score function in Eqn.
Revised: Then, we consider estimating the gradient of the score function in Eqn.

Original: As for the covariance termc ( v ; θ ) , we estimate it with the sample covariance matrix [4], and the resulting variational estimate isˆ [Equation] [Equation] θ
Revised: As for the covariance term c ( v ; θ ) , we estimate it with the sample covariance matrix [4], and the resulting variational estimate is ˆ

Original: [Equation]
Revised: [Equation]

Original: As for the the expectation term e ( v ; θ ) , we directly apply the Monte Carlo estimate and the resultingvariational estimate is
Revised: As for the the expectation term e ( v ; θ ) , we directly apply the Monte Carlo estimate and the resulting variational estimate is

Original: [Equation]
Revised: [Equation]

Original: Combing Eqn.
Revised: Combing Eqn.

Original: (9) and Eqn.
Revised: (8) and Eqn.

Original: (10), the resulting variational gradient estimate of the score function(VaGES) is
Revised: (9), the resulting variational gradient estimate of the score function (VaGES) is

Original: [Equation]
Revised: [Equation]

Original: Remark:
Revised: Remark:

Original: Although Eqn.
Revised: Although Eqn.

Original: (10) includes second derivatives, in practice we only need to estimatethe product of the second derivatives with vectors and only two backpropagations are required forcalculation (since z (cid:62) ∂ ∇ v log 
Revised: (9) includes second derivatives, in practice we only need to estimate the product of the second derivatives with vectors and only two backpropagations are required for calculation (since z (cid:62) ∂ ∇ v log ˜ p θ [Equation] v , h i [Equation]

Revised: θ

Original: [Equation]
Revised: [Equation]

Original: 2 Bounding the Bias
Revised: 2.2 Bounding the Bias

Original: Notice that VaES( v ; θ , φ ) and VaGES( v ; θ , φ ) are actually estimating E q φ ( h | v ) [ ∇ v log ˜
Revised: Notice that VaES( v ; θ , φ ) and VaGES( v ; θ , φ ) are actually estimating E q φ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )] and
Revised: θ ( v , h )] and

Original: [Equation]
Revised: [Equation]

Original: respectively and will introduce some bias.
Revised: respectively and will introduce some bias.

Original: Firstly, we show that when h is discrete, the bias of
Revised: Firstly, we show that when h is discrete, the bias of

Original: VaES( v ; θ , φ ) and VaGES( v ; θ , φ ) can be bounded by the square root of the KL divergence betweenq φ ( h | v ) and p θ ( h | v ) under some assumptions on boundedness, as characterized in Theorem 1 and
Revised: VaES( v ; θ , φ ) and VaGES( v ; θ , φ ) can be bounded by the square root of the KL divergence between q φ ( h | v ) and p θ ( h | v ) under some assumptions on boundedness, as characterized in Theorem 1 and

Original: Theorem 2.
Revised: Theorem 2.

Original: So in Eqn.
Revised: So in Eqn.

Original: (4), we choose D as the KL divergence when h is discrete to learn thevariational posterior q φ ( h | v ) .
Revised: (3), we choose D as the KL divergence when h is discrete to learn the variational posterior q φ ( h | v ) .

Original: Theorem 1.
Revised: Theorem 1.

Original: ( h is discrete, VaES, proof in Appendix B) Suppose ∇ v log ˜
Revised: ( h is discrete, VaES, proof in Appendix B) Suppose ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) is bounded w.r.t.v , h and θ , then the bias of VaES( v ; θ , φ ) can be bounded by the square root of the KL divergencebetween q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.
Revised: θ ( v , h ) is bounded w.r.t. v , h and θ , then the bias of VaES( v ; θ , φ ) can be bounded by the square root of the KL divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Theorem 2.
Revised: Theorem 2.

Original: ( h is discrete, VaGES, proof in Appendix B) Suppose ∇ v log ˜
Revised: ( h is discrete, VaGES, proof in Appendix B) Suppose ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∇ θ log ˜
Revised: θ ( v , h ) , ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h )and ∂ ∇ v log ˜
Revised: θ ( v , h ) and ∂ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ are bounded w.r.t.
Revised: θ ( v , h ) ∂ θ are bounded w.r.t.

Original: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be boundedby the square root of the KL divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.
Revised: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be bounded by the square root of the KL divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Then, we show that under extra assumptions on the Stein regularity (see Def. 2) and boundednessof the Stein factors (see Def. 3), the bias of VaES( v ; θ , φ ) and VaGES( v ; θ , φ ) can be boundedby the square root of the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) , as characterized in
Revised: Then, we show that under extra assumptions on the Stein regularity (see Def. 2) and boundedness of the Stein factors (see Def. 3), the bias of VaES( v ; θ , φ ) and VaGES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) , as characterized in

Original: Theorem 3 and Theorem 4.
Revised: Theorem 3 and Theorem 4.

Original: Although the boundedness of the Stein factors have only been veriﬁedunder some simple cases [18], and haven’t been extended to more complex cases, e.g., when ˜ p
Revised: Although the boundedness of the Stein factors have only been veriﬁed under some simple cases [18], and haven’t been extended to more complex cases, e.g., when ˜ p

Original: θ ( v , h )is parameterized by a neural network, Theorem 3 and Theorem 4 still inspire us to choose D as the
Revised: θ ( v , h ) is parameterized by a neural network, Theorem 3 and Theorem 4 still inspire us to choose D as the

Original: Fisher divergence in Eqn.
Revised: Fisher divergence in Eqn.

Original: (4) when h is continuous to learn the variational posterior q φ ( h | v ) .
Revised: (3) when h is continuous to learn the variational posterior q φ ( h | v ) .

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: [18] Suppose p is a probability density deﬁned on R n and f : R n → R is a function,we deﬁne g pf as a solution of the Stein equation S p g = f − E p f , where g : R n → R n and
Revised: [18] Suppose p is a probability density deﬁned on R n and f : R n → R is a function, we deﬁne g pf as a solution of the Stein equation S p g = f − E p f , where g : R n → R n and

Original: S p g ( v ) :
Revised: S p g ( v ) :

Original: = ∇ v log p ( v ) (cid:62) g
Revised: = ∇ v log p ( v ) (cid:62) g

Original: ( v ) +
Revised: ( v ) +

Original: Tr( ∇ v g ( v )) .
Revised: Tr( ∇ v g ( v )) .

Original: Remark.
Revised: Remark.

Original: The solution of the Stein equation exists.
Revised: The solution of the Stein equation exists.

Original: For example, let h =
Revised: For example, let h =

Original: f
Revised: f

Original: − E p f , theng
Revised: − E p f , then

Original: [Equation]
Revised: [Equation]

Original: v

Original: [Equation]

Original: =

Original: [Equation]

Original: is a solution.
Revised: is a solution.

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function,we say f satisﬁes the Stein regular condition w.r.t.
Revised: Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function, we say f satisﬁes the Stein regular condition w.r.t.

Original: p, q iff ∀ i ∈
Revised: p, q iff ∀ i ∈

Original: Z ∩ [1 , m ] , lim
Revised: Z ∩ [1 , m ] , lim

Original: || v ||→∞ q ( v )
Revised: || [Equation] ||

Original: g
Revised: [Equation] [Equation]

Original:  pf i ( v ) = 0 .
Revised: g pf i ( v ) = 0 .

Original: Deﬁnition 3.
Revised: Deﬁnition 3.

Original: [18] Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function satisfying satisﬁes the Stein regular condition w.r.t.
Revised: [18] Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function satisfying satisﬁes the Stein regular condition w.r.t.

Original: p, q , we deﬁne κ p,q f :=
Revised: p, q , we deﬁne κ p

Original: (cid:115)

Original: [Equation]
Revised: [Equation]

Original: x
Revised: :=

Original: [Equation]
Revised: [Equation]

Original: || 22 ,referred to as the Stein factor of f w.r.t.
Revised: referred to as the Stein factor of f w.r.t.

Original: p, q .
Revised: p, q .

Original: Theorem 3.
Revised: Theorem 3.

Original: ( h is continuous, VaES, proof in Appendix C) Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜
Revised: ( h is continuous, VaES, proof in Appendix C) Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )as a function of h satisﬁes the Stein regular condition w.r.t.
Revised: θ ( v , h ) as a function of h satisﬁes the Stein regular condition w.r.t.

Original: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the
Revised: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the

Original: Stein factor of ∇ v log ˜
Revised: Stein factor of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) as a function of h w.r.t.
Revised: θ ( v , h ) as a function of h w.r.t.

Original: p ( h | v ; θ ) , q ( h | v ; φ ) is bounded w.r.t.
Revised: p ( h | v ; θ ) , q ( h | v ; φ ) is bounded w.r.t.

Original: v , θand φ , then the bias of VaES( v ; θ , φ ) can be bounded by the square root of the Fisher divergencebetween q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.
Revised: v , θ and φ , then the bias of VaES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Theorem 4.
Revised: Theorem 4.

Original: ( h is continuous, VaGES, proof in Appendix C) Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜
Revised: ( h is continuous, VaGES, proof in Appendix C) Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ,∇ θ log ˜
Revised: θ ( v , h ) , ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∇ v log˜
Revised: θ ( v , h ) , ∇ [Equation] log

Revised: [Equation]

Original: p
Revised: p

Original: θ ( v , h ) ∂ log˜
Revised: [Equation] v , [Equation] log

Original: p
Revised: [Equation]

Original: θ ( v , h ) ∂ θ and ∂ ∇ v log ˜
Revised: [Equation] θ and [Equation]

Original: p θ ( v , h ) ∂ θ as functions of h satisfythe Stein regular condition w.r.t.
Revised: as functions of h satisfy the Stein regular condition w.r.t.

Original: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the Stein factors of∇ v log ˜ 
Revised: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the Stein factors of ∇ v log ˜ p

Original: p
Revised: [Equation] [Equation]

Original: θ ( v , h ) , ∇θ log ˜
Revised: [Equation] [Equation] , ∇

Original: p
Revised: [Equation]

Original: θ ( v , h ) , ∇ v log ˜ 
Revised: log ˜ [Equation]

Original: p
Revised: [Equation]

Original: θ ( v , h ) ∂ log˜
Revised: , h [Equation] log

Original: p
Revised: [Equation] [Equation]

Original: θ ( v , h ) ∂ θ and ∂ ∇ v log ˜
Revised: , h [Equation] and ∂ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ as functions ofh w.r.t.
Revised: θ ( v , h ) ∂ θ as functions of h w.r.t.

Original: p ( h | v ; θ ) , q ( h | v ; φ ) are bounded w.r.t.
Revised: p ( h | v ; θ ) , q ( h | v ; φ ) are bounded w.r.t.

Original: v , θ and φ , (3) ∇ v log ˜
Revised: v , θ and φ , (3) ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) and ∇ θ log ˜
Revised: θ ( v , h ) and ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h )are bounded w.r.t.
Revised: θ ( v , h ) are bounded w.r.t.

Original: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be bounded by the square rootof the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.
Revised: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original:  Learning EBLVMs
Revised:  Learning EBLVMs

Original: In this section, we show that VaES and VaGES can extend Kernelized Stein Discrepancy (KSD)
Revised: In this section, we show that VaES and VaGES can extend Kernelized Stein Discrepancy (KSD)

Original: [22]and methods [34, 20] based on score matching (SM) [13] to learn nonstructural EBLVMs.
Revised: [22] and methods [34, 20] based on score matching (SM) [13] to learn nonstructural EBLVMs.

Original: Besides,we empirically validate our methods and further details of experimental settings can be found in
Revised: Besides, we empirically validate our methods and further details of experimental settings can be found in

Original: Appendix E.
Revised: Appendix E.

Original: 3.1 Learning EBLVMs with KSD
Revised: 3.1 Learning EBLVMs with KSD

Original: Method.
Revised: Method.

Original: The KSD between a target probability density p t ( v ) and the model probability densityp θ ( v ) is deﬁned as
Revised: The KSD between a target probability density p t ( v ) and the model probability density p θ ( v ) is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: which properly measures the difference between p t and p θ under some mild assumptions [22].
Revised: which properly measures the difference between p t and p θ under some mild assumptions [22].

Original: To learn an EBLVM from the target density, we can use gradient-based optimization to minimize
Revised: To learn an EBLVM from the target density, we can use gradient-based optimization to minimize

Original: KSD( p t , p θ ) , where the gradient w.r.t.
Revised: KSD( p t , p θ ) , where the gradient w.r.t.

Original: θ is
Revised: θ is

Original: [Equation]
Revised: [Equation]

Original: Estimating ∇ v log ˜
Revised: Estimating ∇ v log ˜

Original: p θ ( v ) with VaES and estimating ∂ ∇ v (cid:48) log ˜ p θ
Revised: p θ ( v ) with VaES and estimating ∂ ∇ v [Equation] log ˜ p θ

Original: [Equation]
Revised: [Equation]

Original: v (cid:48) )
Revised: v [Equation]

Original:  ∂ θ with VaGES, the variationalstochastic gradient estimate of KSD (VaGES-KSD) is
Revised: ) ∂ θ with VaGES, the variational stochastic gradient estimate of KSD (VaGES-KSD) is

Original: [Equation]
Revised: [Equation]

Original: where the union of v 1: M and v (cid:48) 1: M is a minibatch from the target density, and VaES( v i ; θ , φ ) and
Revised: where the union of v 1: M and v (cid:48) 1: M is a minibatch from the target density, and VaES( v i ; θ , φ ) and

Original: VaGES( v (cid:48)
Revised: VaGES( v (cid:48)

Original: i ; θ , φ ) are independent.
Revised: i ; θ , φ ) are independent.

Original: Model and Setting.
Revised: Model and Setting.

Original: To the best of our knowledge, KSD hasn’t been shown feasible to scale upto natural images.
Revised: To the best of our knowledge, KSD hasn’t been shown feasible to scale up to natural images.

Original: So we illustrate the validity of VaGES-KSD by learning Gaussian restricted
Revised: So we illustrate the validity of VaGES-KSD by learning Gaussian restricted

Original: Boltzmann machines (GRBMs) [35, 10] on the 2-D checkerboard dataset, whose density is shown in
Revised: Boltzmann machines (GRBMs) [35, 10] on the 2-D checkerboard dataset, whose density is shown in

Original: Appendix F.1.
Revised: Appendix F.1.

Original: The energy function of a GRBM is
Revised: The energy function of a GRBM is

Original: [Equation]
Revised: [Equation]

Original: with learnable parameters θ = ( σ, W, b, c ) .
Revised: with learnable parameters θ = ( σ, W, b, c ) .

Original: Since
Revised: Since

Original: GRBM has a tractable posterior, we can directly learnit using KSD (see Eqn.
Revised: GRBM has a tractable posterior, we can directly learn it using KSD (see Eqn.

Original: We also compare another baseline where
Revised: We also compare another baseline where

Original: ˜
Revised: ˜

Original: p
Revised: p

Original: θ ( v ) is estimated by importancesampling (i.e., ˜ p θ ( v ) ≈ [Equation]
Revised: θ ( v ) is estimated by importance sampling (i.e.

Original: L L (cid:80)

Original: [Equation]
Revised: [Equation]

Original: ) ,where unif means the uniform distribution) and we callit IS-KSD.
Revised: where unif means the uniform distribution) and we call it IS-KSD.

Original: We generate 60,000 samples for trainingand 10,000 samples for testing.
Revised: We generate 60,000 samples for training and 10,000 samples for testing.

Original: We use the RBF kernelk ( v , v (cid:48) )
Revised: We use the RBF kernel k ( v , v (cid:48) )

Original: =
Revised: =

Original: exp( − || v − v (cid:48) || {+
+}
Revised: exp( − || v − v [Equation] || {+
+}

Original: 22 2 σ) and σ = 0 . 
Revised: [Equation] and σ = 0 . 1 .

Original: Result.
Revised: Result.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, VaGES-KSD outperforms IS-KSD on all L (i.e., the number of h sampledfor each v ) and is comparable to the KSD baseline (the test log-likelihood curves of VaGES-KSD arevery close to KSD when L = 5 , 10 )
Revised: 1, VaGES-KSD outperforms IS-KSD on all L (i.e., the number of h sampled for each v ) and is comparable to the KSD baseline (the test log-likelihood curves of VaGES-KSD are very close to KSD when L = 5 , 10 ).

Original: 3.2 Learning EBLVMs with Score Matching
Revised: 3.2 Learning EBLVMs with Score Matching

Original: Method.
Revised: Method.

Original: Recall that the Fisher divergence [15] between the target density p t and the model densityp θ is[Equation]
Revised: Recall that the Fisher divergence [15] between the target density p t and the model density p θ is

Original: [Equation]
Revised: [Equation]

Original: which measures the difference between the score functions of p t and p θ .
Revised: which measures the difference between the score functions of p t and p θ .

Original: The denoising scorematching (DSM) [34] minimizes the Fisher divergence between a perturbed target density and themodel density:
Revised: The denoising score matching (DSM) [34] minimizes the Fisher divergence between a perturbed target density and the model density:

Original: [Equation]
Revised: [Equation]

Original: where p σ 0 ( v ) =
Revised: where p σ 0 ( v ) =

Original: (cid:82) p t ( w ) p σ 0 ( v | w )d w is the perturbed target density, p σ 0 ( v | w ) =
Revised: (cid:82) p t ( w ) p σ 0 ( v | w )d w is the perturbed target density, p σ 0 ( v | w ) =

Original: N ( v | w , σ 20 I ) isthe Gaussian perturbation and σ 0 is a ﬁxed noise level.
Revised: N ( v | w , σ 20 I ) is the Gaussian perturbation and σ 0 is a ﬁxed noise level.

Original: The multiscale denoising score matching(MDSM) [20] is a variant of DSM which uses different levels of noise to learn an EBM in highdimensional spaces:
Revised: The multiscale denoising score matching (MDSM) [20] is a variant of DSM which uses different levels of noise to learn an EBM in highdimensional spaces:

Original: [Equation]
Revised: [Equation]

Original: where p ( σ ) is a prior distribution over the ﬂexible noise level σ and σ 0 is a ﬁxed noise level.
Revised: where p ( σ ) is a prior distribution over the ﬂexible noise level σ and σ 0 is a ﬁxed noise level.

Original: We extend the above two score matching methods (Eqn.
Revised: We extend the above two score matching methods (Eqn.

Original: (17) and Eqn.
Revised: (16) and Eqn.

Original: (18)) to learn EBLVMs.
Revised: (17)) to learn EBLVMs.

Original: Firstlywrite Eqn.
Revised: Firstly write Eqn.

Original: (17) and Eqn.
Revised: (16) and Eqn.

Original: (18) in a general form for simplicity
Revised: (17) in a general form for simplicity

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where p t ( w , v ) is the joint distribution of w and v (speciﬁcally, p t ( w , v )
Revised: where p t ( w , v ) is the joint distribution of w and v (speciﬁcally, p t ( w , v )

Original: =
Revised: =

Original: p t ( w ) p σ 0 ( v | w ) for
Revised: p t ( w ) p σ 0 ( v | w ) for

Original: Eqn.
Revised: Eqn.

Original: (17) and p t ( w , v ) =
Revised: (16) and p t ( w , v ) =

Original: (cid:82) σ p t ( w ) p ( σ ) p σ ( v | w )d σ for Eqn.
Revised: (cid:82) σ p t ( w ) p ( σ ) p σ ( v | w )d σ for Eqn.

Original: We use gradient-based optimization to minimize J ( θ ) and its gradient w.r.t.
Revised: We use gradient-based optimization to minimize J ( θ ) and its gradient w.r.t.

Original: θ is
Revised: θ is

Original: [Equation]
Revised: [Equation]

Original: Estimating ∇ v log ˜
Revised: Estimating ∇ v log ˜

Original: p θ ( v ) with VaES and estimating ∂ ∇ v (cid:48) log ˜
Revised: p θ ( v ) with VaES and estimating ∂ ∇ v (cid:48) log ˜

Original: p θ ( v (cid:48) )
Revised: p θ ( v (cid:48) )

Original: ∂ θ with VaGES, the variationalstochastic gradient estimate of the score matching methods (VaGES-SM) is[Equation]
Revised: ∂ θ with VaGES, the variational stochastic gradient estimate of the score matching methods (VaGES-SM) is

Original: [Equation]
Revised: [Equation]

Original: where ( w 1: M , v 1: M ) is a minibatch from p t ( w , v ) , and VaES( v i ; θ , φ ) and VaGES( v (cid:48)
Revised: where ( w 1: M , v 1: M ) is a minibatch from p t ( w , v ) , and VaES( v i ; θ , φ ) and VaGES( v (cid:48)

Original: i ; θ , φ ) areindependent.
Revised: i ; θ , φ ) are independent.

Original: We explicitly denote our methods as VaGES-DSM or VaGES-MDSM according towhich score matching objective is used.
Revised: We explicitly denote our methods as VaGES-DSM or VaGES-MDSM according to which score matching objective is used.

Original: Result.
Revised: Result.

Original: To validate the effectiveness of VaGES-SM, we compare VaGES-SM with the state-of-artbi-level score matching (BiSM) [1] for learning EBLVMs, which reformulates Eqn.
Revised: To validate the effectiveness of VaGES-SM, we compare VaGES-SM with the state-of-art bi-level score matching (BiSM) [1] for learning EBLVMs, which reformulates Eqn.

Original: (17) and Eqn.
Revised: (16) and Eqn.

Original: (18)as bi-level optimization problems (BiDSM and BiMDSM) to learn EBLVMs and uses the gradientunrolling technique to solve the bi-level problems.
Revised: (17) as bi-level optimization problems (BiDSM and BiMDSM) to learn EBLVMs and uses the gradient unrolling technique to solve the bi-level problems.

Original: We compare VaGES-DSM and BiDSM underdifferent settings in GRBMs (see Eqn.
Revised: We compare VaGES-DSM and BiDSM under different settings in GRBMs (see Eqn.

Original: (15)) on the Frey face dataset in Fig.
Revised: (14)) on the Frey face dataset in Fig.

Original: For BiSM, L meansthe number of h sampled for each v ; K means the times of updating parameters in the lower levelproblem given a minibatch; N means the number of steps for gradient unrolling in the higher levelproblem given a minibatch.
Revised: For BiSM, L means the number of h sampled for each v ; K means the times of updating parameters in the lower level problem given a minibatch; N means the number of steps for gradient unrolling in the higher level problem given a minibatch.

Original: Since the meaning of L and K in BiSM is similar to VaGES-SM, wecompare them in the same dimension and leave N in another dimension when ploting Fig.
Revised: Since the meaning of L and K in BiSM is similar to VaGES-SM, we compare them in the same dimension and leave N in another dimension when ploting Fig.

Original: 2 (a-d).
Revised: 2 (a-d).

Original: VaGES-DSM outperforms BiDSM ( N =0 or N =2) and has similar performance with BiDSM ( N =5).
Revised: VaGES-DSM outperforms BiDSM ( N =0 or N =2) and has similar performance with BiDSM ( N =5).

Original: Meanwhile it requires the least memory and the time consuming is similar to BiDSM ( N =2). 
Revised: Meanwhile it requires the least memory and the time consuming is similar to BiDSM ( N =2). [Equation]

Original: [Figure]
Revised: [Figure]

Original: Besides, we compare VaGES-MDSM with BiMDSM in a deep EBLVM with energy function
Revised: Besides, we compare VaGES-MDSM with BiMDSM in a deep EBLVM with energy function

Original: E ( v , h ; θ ) =
Revised: E ( v , h ; θ ) =

Original: g 3 ( g 2 ( g 1 ( v ; θ 1 ) , h ); θ 2 ) , where g 1 and g 3 are two neural networks, g 2 is an additivecoupling layer and h is continuous.
Revised: g 3 ( g 2 ( g 1 ( v ; θ 1 ) , h ); θ 2 ) , where g 1 and g 3 are two neural networks, g 2 is an additive coupling layer and h is continuous.

Original: We evaluate them on the MNIST dataset.
Revised: We evaluate them on the MNIST dataset.

Original: The BiMDSM has bettervisual quality then VaGES-MDSM.
Revised: The BiMDSM has better visual quality then VaGES-MDSM.

Original: It indicates that there are still challenges to apply VaGES-MDSMto high-dimensional data, which is analyzed in Sec. 5.
Revised: It indicates that there are still challenges to apply VaGES-MDSM to high-dimensional data, which is analyzed in Sec. 5.

Original: [Figure]
Revised: [Figure]

Original:  Evaluating EBLVMs
Revised:  Evaluating EBLVMs

Original: Method.
Revised: Method.

Original: In this setting, we are given an EBLVM p θ ( v , h ) =
Revised: In this setting, we are given an EBLVM p θ ( v , h ) =

Original: e −E ( v , h ; θ ) /
Revised: e −E ( v , h ; θ ) /

Original: Z ( θ ) and a set of samples{ v i } ni =1 from the target density p t ( v ) .
Revised: Z ( θ ) and a set of samples { v i } ni =1 from the target density p t ( v ) .

Original: We want to measure how well the model p θ ( v ) approximates the target density p t ( v ) , which needs an absolute value representing the difference between p t andp θ .This
Revised: We want to measure how well the model p θ ( v ) approximates the target density p t ( v ) , which needs an absolute value representing the difference between p t and p θ .This

Original: task is more difﬁcult than model comparison, which only needs relative values to comparebetween models.
Revised: task is more difﬁcult than model comparison, which only needs relative values to compare between models.

Original: We use the Fisher divergence of the maximum form [12, 6] to measure how well p θapproximates p t :
Revised: We use the Fisher divergence of the maximum form [12, 6] to measure how well p θ approximates p t :

Original: [Equation]
Revised: [Equation]

Original: where F is the set of functions f : R d → R d , s.t.
Revised: where F is the set of functions f : R d → R d , s.t.

Original: lim
Revised: lim

Original: || v ||→∞ p t ( v ) f
Revised: || v ||→∞ p t ( v ) f

Original: ( v ) = 0 and p ( (cid:15) ) is a noisedistribution (e.g., Gaussian distribution) introduced for computation efﬁciency [6].
Revised: ( v ) = 0 and p ( (cid:15) ) is a noise distribution (e.g., Gaussian distribution) introduced for computation efﬁciency [6].

Original: Under some mildassumptions (see Appendix D), we have D mF ( p t || p θ )
Revised: Under some mild assumptions (see Appendix D), we have D mF ( p t || p θ )

Original: =
Revised: =

Original: D F ( p t || p θ ) .
Revised: D F ( p t || p θ ) .

Original: In practice, F is approximatedby a neural network { f η : η ∈ H } , where η is the parameter and H is the parameter space.
Revised: In practice, F is approximated by a neural network { f η : η ∈ H } , where η is the parameter and H is the parameter space.

Original: Weoptimize the right hand side of Eqn.
Revised: We optimize the right hand side of Eqn.

Original: (22) and the gradient w.r.t.
Revised: (21) and the gradient w.r.t.

Original: η is
Revised: η is

Original: [Equation]
Revised: [Equation]

Original: Estimating ∇ v log ˜
Revised: Estimating ∇ v log ˜

Original: p θ ( v ) with VaES, the variational stochastic gradient estimate is
Revised: p θ ( v ) with VaES, the variational stochastic gradient estimate is

Original: [Equation]
Revised: [Equation]

Original: ∂ η − ∂ || f η ( v i ) ||∂ η , where v 1: M is a minibatch from p t ( v ) and (cid:15) 1: M is aminibatch of noise from p ( (cid:15) ) .
Revised: where v 1: M is a minibatch from p t ( v ) and (cid:15) 1: M is a minibatch of noise from p ( (cid:15) ) .

Original: We refer to this methodas VaGES-Fisher.
Revised: We refer to this method as VaGES-Fisher.

Original: In practice, Eqn.
Revised: In practice, Eqn.

Original: (22) is optimizedon the training data, validated on the validation data(if there is one) and tested on the testing data.
Revised: (21) is optimized on the training data, validated on the validation data (if there is one) and tested on the testing data.

Original: Result.
Revised: Result.

Original: We validate the effectiveness of VaGESFisher in GRBMs.
Revised: We validate the effectiveness of VaGESFisher in GRBMs.

Original: We initialize a GRBM as p t ( v ) ,perturb its weight with increasing noise and providethe Fisher divergence between the initial GRBM andthe perturbed one.
Revised: We initialize a GRBM as p t ( v ) , perturb its weight with increasing noise and provide the Fisher divergence between the initial GRBM and the perturbed one.

Original: The dimensions of v and h aresame and we experiment on dimensions of 200 and500.
Revised: The dimensions of v and h are same and we experiment on dimensions of 200 and 500.

Original: Further details of experimental settings can befound in Appendix E.
Revised: Further details of experimental settings can be found in Appendix E.

Original: The result is shown in Fig.
Revised: The result is shown in Fig.

Original: We compare our estimated Fisher divergence with theaccurate Fisher divergence.
Revised: We compare our estimated Fisher divergence with the accurate Fisher divergence.

Original: Under both dimensions,our estimated one is close to the accurate one.
Revised: Under both dimensions, our estimated one is close to the accurate one.

Original:  Challenges in High-Dimensional Spaces and Deep EBLVMs
Revised:  Challenges in High-Dimensional Spaces and Deep EBLVMs

Original: Although our methods achieve promising results when the data is simple (e.g., the checkerboarddataset) or when the model is simple (e.g., GRBMs), there are still challenges in high-dimensionalspaces or in deep EBLVMs.
Revised: Although our methods achieve promising results when the data is simple (e.g., the checkerboard dataset) or when the model is simple (e.g., GRBMs), there are still challenges in high-dimensional spaces or in deep EBLVMs.

Original: Consider the VaGES estimator, it’s a matrix of size dim( v ) × dim( θ ) .
Revised: Consider the VaGES estimator, it’s a matrix of size dim( v ) × dim( θ ) .

Original: So either more complex data or models will increase the size of the matrix, making it harder toestimate.
Revised: So either more complex data or models will increase the size of the matrix, making it harder to estimate.

Original: In our estimate, the bias might matter more than the variance, since increasing L doesn’tincrease the visual quality of samples, as shown in Fig.
Revised: In our estimate, the bias might matter more than the variance, since increasing L doesn’t increase the visual quality of samples, as shown in Fig.

Original: It is an initial hypothesis and requiresfurther experiments for validation.
Revised: It is an initial hypothesis and requires further experiments for validation.

Original: Besides, VaGES is based on the sample covariance matrix, whichis relatively simple and there are might be advanced substitutes with better properties
Revised: Besides, VaGES is based on the sample covariance matrix, which is relatively simple and there are might be advanced substitutes with better properties.

Original: A Proof of Equation (3)
Revised: A Proof of Equation (3)

Original: Proof.
Revised: Proof.

Original: Firstly we have
Revised: Firstly we have

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: ) and similarly we have E p ( h | v ; θ ) [ ∇ θ log p ( h | v ; θ )]
Revised: and similarly we have E p ( h | v ; θ ) [ ∇ θ log p ( h | v ; θ )]

Original: = 0 .
Revised: = 0 .

Original: Thereby, we have
Revised: Thereby, we have

Original: [Equation]
Revised: [Equation]

Original: ∇

Original: [Equation] θ [Equation] ] , and similarly we have ∇ θ log ˜ p ( v ; θ ) =
Revised: ] , and similarly we have ∇ θ log ˜ p ( v ; θ ) =

Original: E p ( h | v ; θ )
Revised: E p ( h | v ; θ )

Original: [ ∇ θ log ˜ p
Revised: [ ∇ θ log ˜ p

Original: ( v , h ; θ )] .
Revised: ( v , h ; θ )] .

Original: Taking derivatives to Eqn.
Revised: Taking derivatives to Eqn.

Original: (23) w.r.t.
Revised: (22) w.r.t.

Revised: θ

Revised: [Equation]

Original: θ , we have 
Revised: we have E p

Revised: [Equation]

Revised: h

Revised: [Equation]

Revised: v ; θ

Original: [Equation]
Revised: [Equation]

Original: The leftmost term of Eqn.
Revised: The leftmost term of Eqn.

Original: (24) can be written as
Revised: (23) can be written as

Original: [Equation]
Revised: [Equation]

Original: ( v , h
Revised: =

Original: [Equation]
Revised: [Equation]

Original:  θ) ∂ log p ( h
Revised: = [Equation] θ

Original: [Equation]
Revised: [Equation]

Original: θ ) ∂ θ (cid:21) − E p ( h | v ; θ ) (cid:20) ∇ v log p ( v ; θ ) ∂ log p ( h | v ; θ ) ∂ θ (cid:21) =
Revised: =

Original: E p ( h | v ; θ ) (cid:20) ∇ v log p ( v , h ; θ ) ∂ log p ( h | v ; θ ) ∂ θ (cid:21) = E p ( h | v ; θ ) (cid:20) ∇ v log p ( v , h ; θ ) ∂ log ˜ p ( v , h ; θ ) ∂ θ (cid:21) − E p
Revised: [Equation]

Original: ( h | v ; θ ) (cid:20) ∇ v log p ( v , h ; θ ) ∂log ˜ p ( v ; θ) ∂ θ (cid:21) = 
Revised: [Equation] ∂ θ = [Equation] [Equation]

Original: E p ( h | v ; θ ) (cid:20) ∇ v log p ( v , h ; θ ) ∂ log ˜ p ( v , h ; θ ) ∂ θ (cid:21) − E p
Revised: [Equation]

Original: ( h | v ; θ ) [ ∇ v log p ( v , h ; θ )]
Revised: [Equation] ) ∂ θ = [Equation]

Original: ∂ log ˜ p ( v ; θ ) ∂ θ = E p ( h | v ; θ ) (cid:20) ∇ v log p ( v , h ; θ ) ∂ log ˜ p ( v , h ; θ ) ∂ θ (cid:21)
Revised: p [Equation] [Equation] [Equation]

Original: −E p
Revised: −

Original: ( h | v ; θ ) [ ∇ v logp ( v , h ; θ )]
Revised: [Equation] [Equation] [Equation] log

Original: ∂ log ˜ p ( v ; θ ) ∂ θ = E p ( h | v ; θ ) (cid:20) ∇ v log p ( v , h ; θ ) ∂ log ˜ p ( v , h ; θ ) ∂ θ (cid:21) −
Revised: [Equation]

Original: E p ( h | v ; θ ) [ ∇ v log p ( v , h ; θ )]
Revised: v [Equation] θ ) ∂ θ [Equation]

Original: E p ( h | v ; θ ) (cid:20) ∂ log ˜ p
Revised: = [Equation] p ( h | v [Equation]

Original: ( v , h ; θ ) ∂ θ (cid:21) =
Revised: θ ) ( ∇ v [Equation]

Original: Cov p (h | v ; θ ) ( ∇ v log p ( v , h ; θ ) , ∇ θ log ˜ p
Revised: p ( v [Equation] [Equation] ∇ θ log ˜ p

Original: ( v , h ; θ )) .
Revised: ( v , h ; θ )) .

Original: Thereby, we have
Revised: Thereby, we have

Original: [Equation]
Revised: [Equation]

Original: B Proof of Theorem 1 and Theorem2
Revised: B Proof of Theorem 1 and Theorem

Original: Lemma 1.
Revised: Lemma 1.

Original: Suppose p, q are probability mass functions deﬁned on Ω , and f : Ω → R m , then theapproximation error of estimating E p f using E q f can be bounded as
Revised: Suppose p, q are probability mass functions deﬁned on Ω , and f : Ω → R m , then the approximation error of estimating E p f using E q f can be bounded as

Original: [Equation]
Revised: [Equation]

Original: where || f || ∞ :
Revised: where || f || ∞ :

Original: =
Revised: =

Original: sup ω ∈ Ω || f ( ω ) ||
Revised: sup ω ∈ Ω || f ( ω ) ||

Original: Proof.
Revised: Proof.

Original: Firstly, we have
Revised: Firstly, we have

Original: [Equation]
Revised: [Equation]

Original: According to Pinsker’s inequality [32], we have (cid:80) ω ∈ Ω | p ( ω ) − q ( ω ) | ≤ (cid:112) 2 D KL ( p || q ) . Thereby,|| E p f − E q f || 2 ≤ (cid:112) 2 D KL ( p || q ) || 
Revised: According to Pinsker’s inequality [32], we have (cid:80) ω ∈ Ω | p ( ω ) − q ( ω ) | ≤ (cid:112) 2 D KL ( p || q ) . Thereby, || E [Equation]

Original: f || ∞ .
Revised: ∞ .

Original: Theorem 1.
Revised: Theorem 1.

Original: ( h is discrete, VaES)
Revised: ( h is discrete, VaES)

Original: Suppose ∇ v log ˜
Revised: Suppose ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) is bounded w.r.t.
Revised: θ ( v , h ) is bounded w.r.t.

Original: v , h and θ , then thebias of VaES( v ; θ , φ ) can be bounded by the square root of the KL divergence between q φ ( h | v ) andp θ ( h | v ) up to multiplying a constant.
Revised: v , h and θ , then the bias of VaES( v ; θ , φ ) can be bounded by the square root of the KL divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Proof.
Revised: Proof.

Original: According to Lemma 1, we have
Revised: According to Lemma 1, we have

Original: [Equation]
Revised: [Equation]

Original: ( h | v ) || p θ

Original: [Equation]

Original: h | v )) .
Revised: | v )) .

Original: By the boundedness of ∇ v log ˜
Revised: By the boundedness of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∃ A < ∞ , ∀ v , ∀ h , ∀ θ , ||∇ v log ˜
Revised: θ ( v , h ) , ∃ A < ∞ , ∀ v , ∀ h , ∀ θ , ||∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ||
Revised: θ ( v , h ) ||

Original: 2 ≤ A .
Revised: 2 ≤ A .

Original: Let
Revised: Let

Original: C = √ 2 A , then
Revised: C = √ 2 A , then

Original: [Equation]
Revised: [Equation]

Original: Deﬁnition 1.
Revised: Deﬁnition 1.

Original: Suppose A is a matrix, we deﬁne ||
Revised: Suppose A is a matrix, we deﬁne ||

Original: A || 2 :
Revised: A || 2 :

Original: = (cid:114)(cid:80) i,j A 2 i,j .
Revised: = (cid:114)(cid:80) i,j A 2 i,j .

Original: Lemma 2.
Revised: Lemma 2.

Original: Suppose a , b are two vectors, then || ab (cid:62) ||
Revised: Suppose a , b are two vectors, then || ab (cid:62) ||

Original: 2 = ||
Revised: 2 = ||

Original: a || 2 || b
Revised: a || 2 || b

Original: || 2 .
Revised: || 2 .

Original: Proof.
Revised: Proof.

Original: ||
Revised: ||

Original: ab (cid:62) ||
Revised: ab (cid:62) ||

Original: 2 =
Revised: 2 =

Original: (cid:114)(cid:80)
Revised: (cid:114)(cid:80)

Original: i,j a 2 i b 2 j = ||
Revised: i,j a 2 i b 2 j = ||

Original: a || 2 || b
Revised: a || 2 || b

Original: || 2 .
Revised: || 2 .

Original: Theorem 2.
Revised: Theorem 2.

Original: ( h is discrete, VaGES) Suppose ∇ v log ˜
Revised: ( h is discrete, VaGES) Suppose ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∇ θ log ˜
Revised: θ ( v , h ) , ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) and ∂ ∇ v log ˜
Revised: θ ( v , h ) and ∂ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θare bounded w.r.t.
Revised: θ ( v , h ) ∂ θ are bounded w.r.t.

Original: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be bounded by the square rootof the KL divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.
Revised: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be bounded by the square root of the KL divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Proof.
Revised: Proof.

Original: According to Thm. 1, ∃ C 1 < ∞ , s.t.
Revised: According to Thm. 1, ∃ C 1 < ∞ , s.t.

Original: [Equation]
Revised: [Equation]

Original: | v ) [ ∇ v log ˜ p θ ( v , h )] − ∇ v log ˜
Revised: θ ( v ) ||

Original: p θ ( v ) || 2 ≤ C 1 (cid:113) D KL ( q φ ( h | v ) ||
Revised: 2 ≤ C 1 (cid:113) D KL ( q φ ( h | v ) ||

Original: p θ ( h | v )) .
Revised: p θ ( h | v )) .

Original: Similarly, ∃ C 2 < ∞ , s.t.
Revised: Similarly, ∃ C 2 < ∞ , s.t.

Original: [Equation]
Revised: [Equation]

Original: and ∃ C 3 < ∞ , s.t. 
Revised: and ∃ C 3 < ∞ , s.t. |

Original: [Equation]
Revised: [Equation]

Original: By the boundedness of ∇ v log ˜
Revised: By the boundedness of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) and ∇ θ log ˜
Revised: θ ( v , h ) and ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∇ v log ˜
Revised: θ ( v , h ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ log ˜
Revised: θ ( v , h ) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ is alsobounded.
Revised: θ ( v , h ) ∂ θ is also bounded.

Original: Thereby, ∃ C 4 < ∞ , s.t.[Equation] ≤
Revised: Thereby, ∃ C 4 < ∞ , s.t.

Original: [Equation]
Revised: [Equation]

Original: By the boundedness of ∇ v log ˜
Revised: By the boundedness of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) and ∇ θ log ˜
Revised: θ ( v , h ) and ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) , we can assume C < ∞ is a constantthat bounds ||∇ v log ˜
Revised: θ ( v , h ) , we can assume C < ∞ is a constant that bounds ||∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) || 2 and ||∇ θ log ˜
Revised: θ ( v , h ) || 2 and ||∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ||
Revised: θ ( v , h ) ||

Original: Then, by the triangle inequality and
Revised: Then, by the triangle inequality and

Original: Lemma.
Revised: Lemma.

Original: 2, we have
Revised: 2, we have

Original: [Equation]
Revised: [Equation]

Original: ∂ log ˜

Original: p θ ( v , h ) ∂ θ (cid:21) ||≤||
Revised: v , h ) ∂ θ (cid:21) ||≤||

Original: E q φ ( h | v ) [ ∇ v log ˜
Revised: E q φ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )] (cid:18) E q φ ( h | v ) (cid:20) ∂ log ˜
Revised: θ ( v , h )] (cid:18) E q φ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜
Revised: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21)(cid:19)
Revised: θ (cid:21)(cid:19)

Original: ||+ || (cid:0) E q φ ( h | v ) [ ∇ v log ˜
Revised: ||+ || (cid:0) E q φ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )]
Revised: θ ( v , h )]

Original: − E p θ ( h | v ) [ ∇ v log ˜
Revised: − E p θ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )] (cid:1) E p
Revised: θ ( v , h )] (cid:1) E p

Original: θ ( h | v ) (cid:20) ∂ log ˜
Revised: θ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) ||=
Revised: θ (cid:21) ||=

Original: || E q φ ( h | v ) [ ∇ v log ˜
Revised: || E q φ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )]
Revised: θ ( v , h )]

Original: || 2 ||
Revised: || 2 ||

Original: E q φ ( h | v ) (cid:20) ∂ log ˜
Revised: E q φ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜
Revised: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) ||+ || E q φ ( h | v ) [ ∇ v log ˜
Revised: θ (cid:21) ||+ || E q φ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )]
Revised: θ ( v , h )]

Original: − E p θ ( h | v ) [ ∇ v log ˜
Revised: − E p θ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )]
Revised: θ ( v , h )]

Original: || 2 ||
Revised: || 2 ||

Original: E p θ ( h | v ) (cid:20) ∂ log ˜
Revised: E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) ||≤ C
Revised: θ (cid:21) ||≤ C

Original: || E q φ ( h | v ) (cid:20) ∂ log ˜
Revised: || E q φ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜
Revised: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂
Revised: θ ( v , h ) ∂

Original: θ (cid:21) ||+
Revised: θ (cid:21) ||+

Original: C ||
Revised: C ||

Original: E q φ ( h | v ) [ ∇ v log ˜
Revised: E q φ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )]
Revised: θ ( v , h )]

Original: − E p θ ( h | v ) [ ∇ v log ˜
Revised: − E p θ ( h | v ) [ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h )]
Revised: θ ( v , h )]

Original: ||≤ CC 2 (cid:113) D KL ( q φ ( h | v ) ||
Revised: ||≤ CC 2 (cid:113) D KL ( q φ ( h | v ) ||

Original: p θ ( h | v ))
Revised: p θ ( h | v ))

Original: + CC 1 (cid:113) D KL ( q φ ( h | v ) ||
Revised: + CC 1 (cid:113) D KL ( q φ ( h | v ) ||

Original: p θ ( h | v )) =(
Revised: p θ ( h | v )) =(

Original: CC 2 + CC 1 ) (cid:113) D KL ( q φ ( h | v ) ||
Revised: CC 2 + CC 1 ) (cid:113) D KL ( q φ ( h | v ) ||

Original: p θ ( h | v )) .
Revised: p θ ( h | v )) .

Original: Thereby,|| Cov
Revised: Thereby, || Cov

Original: [Equation]
Revised: [Equation]

Original: φ

Original: [Equation]

Original: −

Original: [Equation]
Revised: [Equation]

Original: p

Original: [Equation] ≤|| E q [Equation] ( h
Revised: (cid:20)

Original: [Equation]
Revised: [Equation]

Original: ) (cid:20)
Revised: (cid:21)

Original: [Equation]
Revised: [Equation]

Original: (cid:21)
Revised: (cid:20)

Original: [Equation]
Revised: [Equation]

Original: (cid:20)
Revised: (cid:21)

Original: [Equation]
Revised: [Equation]

Original: p
Revised: |

Original: [Equation]
Revised: [Equation]

Original: (cid:21) ||
Revised: + ||

Original: [Equation]
Revised: [Equation]

Original: + || E q φ ( h | v ) [ ∇
Revised: −

Original: [Equation]
Revised: [Equation]

Original: log ˜ p
Revised: p

Original: [Equation]
Revised: [Equation]

Original: (v , h 
Revised: ( h |

Original: [Equation]
Revised: [Equation]

Original: ] E
Revised: [ ∇

Original: [Equation]
Revised: [Equation]

Original: φ
Revised: log ˜ p θ ( v , h )] E p

Original: [Equation]
Revised: [Equation]

Original:  h|
Revised: ( h

Original: [Equation]
Revised: [Equation]

Original: ) (cid:20) ∂ log ˜ p

Original: [Equation]
Revised: [Equation]

Original: v
Revised: ∂

Original: [Equation]
Revised: [Equation]

Original: ∂ θ (cid:21) − E p θ ( h | v ) [ ∇ v log ˜ p θ ( v , h )] E p θ ( h | v ) (cid:20) ∂ log ˜ p
Revised: ||

Original: [Equation]
Revised: [Equation]

Original: ( v , h ) ∂ θ (cid:21) ||

Original: [Equation]

Original: ≤
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: (cid:113) D [Equation] ( q
Revised: (cid:113) D KL ( q

Original: [Equation]
Revised: [Equation]

Original: ( h | v ) || p
Revised: h | v ) || p

Original: [Equation]
Revised: [Equation]

Original: ( h | v ))
Revised: ( h | v ))

Revised: + ( CC

Original: [Equation]
Revised: [Equation]

Original: ( CC
Revised: CC

Original: [Equation]
Revised: [Equation]

Original: CC 1 ) (cid:113) D KL ( q
Revised: [Equation] D KL ( q

Original: [Equation]
Revised: [Equation]

Original: ( h | v ) || 
Revised: h | v ) || p

Revised: [Equation]

Original: p θ ( h | v )) =(
Revised: ( h | v )) =(

Original: C 4 + CC 2 + CC 1 ) (cid:113) D 
Revised: C 4 + CC 2 + CC 1 ) (cid:113) D KL ( q φ ( h | v ) || p θ ( h | v )) .

Original: As a result
Revised: As a result

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: | v )) .

Original: C Proof of Theorem 3 and Theorem
Revised: C Proof of Theorem 3 and Theorem

Original: Deﬁnition 2.
Revised: Deﬁnition 2.

Original: Suppose p is a probability density on R n and f : R n → R n , we deﬁne S p
Revised: Suppose p is a probability density on R n and f : R n → R n , we deﬁne S p

Original: f ( v ) :
Revised: f ( v ) :

Original: =∇ v log p ( v ) (cid:62) f
Revised: = ∇ v log p ( v ) (cid:62) f

Original: ( v ) +
Revised: ( v ) +

Original: Tr( ∇ v f ( v )) .
Revised: Tr( ∇ v f ( v )) .

Original: Lemma 3.
Revised: Lemma 3.

Original: [21] Suppose p is a probability density on R n and f : R n → R n is a function satisfyinglim || v ||→∞ p ( v ) f( v ) = 0 ,
Revised: [21] Suppose p is a probability density on R n and f : R n → R n is a function satisfying lim || v ||→∞ p ( v ) f

Original:  then E p ( v ) [ S p f ( v )] = 0 .
Revised: ( v ) = 0 , then E p ( v ) [ S p f ( v )] = 0 .

Original: Proof.
Revised: Proof.

Original: [Equation]
Revised: [Equation]

Original: Thereby,0 =
Revised: Thereby,

Original: Tr( E p ( v

Original: [Equation]
Revised: [Equation]

Original: (cid:2) ∇ v f ( v ) + f ( v ) ∇ v log p ( v ) (cid:62) (cid:3) ) =

Original: E p ( v ) (cid:2)

Original: Tr( ∇ v f ( v ))

Original: + ∇ v log p ( v ) (cid:62)

Original: f ( v ) (cid:3) =

Original: E p ( v ) [ S p f ( v )] .

Original: Lemma 4.
Revised: Lemma 4.

Original: Suppose p, q are probability densities on R n and f : R n → R n satisﬁeslim || v ||→∞ q ( v )
Revised: Suppose p, q are probability densities on R n and f : R n → R n satisﬁes lim || v ||→∞ q ( v )

Original: f ( v ) =
Revised: f ( v ) =

Original: , we have
Revised: , we have

Original: [Equation]

Original: v )

Original: [Equation]
Revised: [Equation]

Original: q || p )

Original: Proof.
Revised: Proof.

Original: By Lemma 3, we have E q S q f = 0 .
Revised: By Lemma 3, we have E q S q f = 0 .

Original: Thereby,
Revised: Thereby,

Original: [Equation]
Revised: [Equation]

Original: ( v ) f ( v ) (cid:62) ( ∇ v log p ( vv ) − ∇ v log q ( v )) | ≤ E q ( v ) ||
Revised: ) − ∇ v log q ( v )) | ≤ E q ( v ) ||

Original: f ( v ) ||
Revised: f ( v ) ||

Original: ||∇ v log p ( v ) − ∇ v log q ( v ) || ≤ (cid:113) E q ( v ) ||
Revised: ||∇ v log p ( v ) − ∇ v log q ( v ) || ≤ (cid:113) E q ( v ) ||

Original: f ( v ) ||
Revised: f ( v ) ||

Original: 2 E q ( v ) ||∇ v log p ( v ) − ∇ v log q ( v ) ||= (cid:113) E q ( v ) ||
Revised: 2 E q ( v ) ||∇ v log p ( v ) − ∇ v log q ( v ) ||= (cid:113) E q ( v ) ||

Original: f ( v ) ||
Revised: f ( v ) ||

Original: (cid:112) D F ( q || p ) .
Revised: (cid:112) D F ( q || p ) .

Original: Deﬁnition 3.
Revised: Deﬁnition 3.

Original: [18] Suppose p is a probability density deﬁned on R n and f : R n → R is a function,we deﬁne g pf as a solution of the Stein equation S p g = f − E p f .
Revised: [18] Suppose p is a probability density deﬁned on R n and f : R n → R is a function, we deﬁne g pf as a solution of the Stein equation S p g = f − E p f .

Original: Remark.
Revised: Remark.

Original: The solution of the Stein equation exists.
Revised: The solution of the Stein equation exists.

Original: For example, let h =
Revised: For example, let h =

Original: f
Revised: f

Original: − E p f , then 
Revised: − E p f , then g

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: is a solution.
Revised: is a solution.

Original: Deﬁnition 4.
Revised: Deﬁnition 4.

Original: Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function,we say f satisﬁes the Stein regular condition w.r.t. p,
Revised: Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function, we say f satisﬁes the Stein regular condition w.r.t

Original: [Equation]
Revised: [Equation]

Original: Deﬁnition 5.
Revised: Deﬁnition 5.

Original: [18] Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function satisfying satisﬁes the Stein regular condition w.r.t.
Revised: [18] Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function satisfying satisﬁes the Stein regular condition w.r.t.

Original: p, q , we deﬁneκ p,q f :=
Revised: p [Equation] q , we deﬁne

Original: [Equation]
Revised: [Equation]

Original: referred to as the Stein factor of f w.r.t.
Revised: referred to as the Stein factor of f w.r.t.

Original: p, q .
Revised: p, q .

Original: Lemma 5.
Revised: Lemma 5.

Original: Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a functionsatisfying the Stein regular condition w.r.t.
Revised: Suppose p, q are probability densities deﬁned on R n and f : R n → R m is a function satisfying the Stein regular condition w.r.t.

Original: p, q , then the approximation error of estimating E p f using371 E q f can be bounded as
Revised: p, q , then the approximation error of estimating E p f using E q f can be bounded as

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: By Lemma 4, we have
Revised: By Lemma 4, we have

Original: [Equation]
Revised: [Equation]

Original: Thereby, we have
Revised: Thereby, we have

Original: [Equation]
Revised: [Equation]

Original: where κ p,q f :=
Revised: where κ p,q f :=

Original: [Equation]
Revised: [Equation]

Original: Theorem 3.
Revised: Theorem 3.

Original: ( h is continuous, VaES)
Revised: ( h is continuous, VaES)

Original: Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜
Revised: Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) as a function ofh satisﬁes the Stein regular condition w.r.t.
Revised: θ ( v , h ) as a function of h satisﬁes the Stein regular condition w.r.t.

Original: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the Stein factor of∇ v log ˜
Revised: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the Stein factor of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) as a function of h w.r.t.
Revised: θ ( v , h ) as a function of h w.r.t.

Original: p ( h | v ; θ ) , q ( h | v ; φ ) is bounded w.r.t.
Revised: p ( h | v ; θ ) , q ( h | v ; φ ) is bounded w.r.t.

Original: v , θ and φ , then thebias of VaES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence between q φ ( h | v )and p θ ( h | v ) up to multiplying a constant.
Revised: v , θ and φ , then the bias of VaES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Proof.
Revised: Proof.

Original: It can be directly derived from Lemma 5.
Revised: It can be directly derived from Lemma 5.

Original: Theorem 4.
Revised: Theorem 4.

Original: ( h is continuous, VaGES) Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜
Revised: ( h is continuous, VaGES) Suppose (1) ∀ ( v , θ , φ ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∇ θ log ˜
Revised: θ ( v , h ) , ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ,∇ v log ˜
Revised: θ ( v , h ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ log ˜
Revised: θ ( v , h ) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ and ∂ ∇ v log ˜
Revised: θ ( v , h ) ∂ θ and ∂ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ as functions of h satisfy the Stein regular condition w.r.t.
Revised: θ ( v , h ) ∂ θ as functions of h satisfy the Stein regular condition w.r.t.

Original: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the Stein factors of ∇ v log ˜
Revised: p ( h | v ; θ ) and q ( h | v ; φ ) and (2) the Stein factors of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) , ∇ θ log ˜
Revised: θ ( v , h ) , ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ,∇ v log ˜
Revised: θ ( v , h ) , ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ log ˜
Revised: θ ( v , h ) ∂ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ and ∂ ∇ v log ˜
Revised: θ ( v , h ) ∂ θ and ∂ ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) ∂ θ as functions of h w.r.t.
Revised: θ ( v , h ) ∂ θ as functions of h w.r.t.

Original: p ( h | v ; θ ) , q ( h | v ; φ ) arebounded w.r.t.
Revised: p ( h | v ; θ ) , q ( h | v ; φ ) are bounded w.r.t.

Original: v , θ and φ , (3) ∇ v log ˜
Revised: v , θ and φ , (3) ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) and ∇ θ log ˜
Revised: θ ( v , h ) and ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) are bounded w.r.t.
Revised: θ ( v , h ) are bounded w.r.t.

Original: v , h and θ ,then the bias of VaGES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence betweenq φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.
Revised: v , h and θ , then the bias of VaGES( v ; θ , φ ) can be bounded by the square root of the Fisher divergence between q φ ( h | v ) and p θ ( h | v ) up to multiplying a constant.

Original: Proof.
Revised: Proof.

Original: According to Lemma 5, ∃ C 1 < ∞ , s.t.|| E q φ ( h | v ) [ ∇ v log ˜
Revised: According to Lemma 5, ∃ C 1 < ∞ , s.t.

Original: p θ ( v , h )] − ∇ v log ˜ p θ ( v ) || 2 ≤ C 1 (cid:113) D F ( q φ ( h | v ) ||
Revised: [Equation]

Original: p θ ( h | v )) , ∃ C 2 < ∞ , s.t.|| E q φ ( h | v ) [ ∇ θ log ˜
Revised: , ∃ C 2 < ∞ , s.t. [Equation]

Original: p
Revised: ∃

Original: θ ( v , h )] − ∇ θ log ˜ p θ ( v ) || 2 ≤ C 2 (cid:113) D F ( q φ ( h | v ) || p θ ( h | v )) ,
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: and ∃ C 4 < ∞ , s.t.
Revised: and ∃ C 4 < ∞ , s.t.

Original: [Equation]
Revised: [Equation]

Original: By the boundedness of ∇ v log ˜
Revised: By the boundedness of ∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) and ∇ θ log ˜
Revised: θ ( v , h ) and ∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) , we can assume C < ∞ is a constantthat bounds ||∇ v log ˜
Revised: θ ( v , h ) , we can assume C < ∞ is a constant that bounds ||∇ v log ˜

Original: p
Revised: p

Original: θ ( v , h ) || 2 and ||∇ θ log ˜
Revised: θ ( v , h ) || 2 and ||∇ θ log ˜

Original: p
Revised: p

Original: θ ( v , h ) ||
Revised: θ ( v , h ) ||

Original: After establishing the above bounds w.r.t.the Fisher divergence, the rest proof is exactly the same as Theorem 2.
Revised: After establishing the above bounds w.r.t. the Fisher divergence, the rest proof is exactly the same as Theorem 2.

Original: For completeness , we restatethe proof as follows.
Revised: For completeness , we restate the proof as follows.

Original: By the triangle inequality and Lemma. 2, we have
Revised: By the triangle inequality and Lemma. 2, we have

Original: [Equation]
Revised: [Equation]

Original: ||

Original: [Equation]

Original: ||

Original: E

Original: [Equation]

Original: φ ( h | v ) [ ∇ v log ˜

Original: p

Original: θ ( v , h )] (cid:18) E q φ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ ( v , h ) ∂

Original: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ ( v , h ) ∂

Original: θ (cid:21)(cid:19)

Original: ||+ || (cid:0) [Equation] φ ( h | v ) [ ∇ v log ˜ p
Revised: )] ||≤ CC 2 (cid:113) D F ( q φ ( h | v ) || p

Original: θ ( v , h )]

Original: − E p θ ( h | v ) [ ∇ v log ˜

Original: p

Original: θ ( v , h )] (cid:1) E p

Original: θ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ ( v , h ) ∂

Original: θ (cid:21) ||=

Original: [Equation]

Original: |

Original: [Equation]

Original: [Equation]

Original: v ) [

Original: [Equation]

Original: log ˜

Original: p θ (v , h )]
Revised: θ ( h | v ))

Original: || 2 || E

Original: [Equation]

Original: φ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ ( v , h ) ∂

Original: θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ (v , h  )∂ θ (cid:21) || [Equation] ||
Revised: + CC 1 (cid:113) D F ( q φ ( h | v ) ||

Original: [Equation]

Original: ( h | v ) [

Original: [Equation]

Original: log ˜ p θ (v , h )]
Revised: p θ ( h | v )) =(

Original: [Equation]

Original: p θ ( h | v ) [ ∇ v log ˜

Original: p

Original: θ ( v , h )]

Original: || 2 ||

Original: E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ ( v , h ) ∂

Original: θ (cid:21) ||

Original: [Equation]

Original: ( h | v ) (cid:20) ∂ log ˜ p

Original: [Equation]

Original: ( v , h ) ∂ θ (cid:21) − E p θ ( h | v ) (cid:20) ∂ log ˜

Original: p

Original: θ ( v , h ) ∂

Original: θ (cid:21) ||

Original: [Equation]

Original: |

Original: [Equation]

Original: ( h | v ) [ ∇ v log ˜

Original: p

Original: θ ( v , h )]

Original: − E p θ ( h | v ) [ ∇ v log ˜

Original: p

Original: θ ( v , h )] ||≤ CC 2(cid:113) D F ( q φ ( h | v ) || p θ ( h | v )) + CC 1  (cid:113) D F ( q φ ( h | v ) ||
Revised: CC 2 + CC 1 ) (cid:113) D F ( q φ ( h | v ) ||

Original: p θ ( h | v )) =(

Original: CC 2 + CC 1 ) (cid:113) D F ( q φ ( h | v ) ||

Original: p θ ( h | v )) .
Revised: p θ ( h | v )) .

Original: Thereby,|| Cov q φ [Equation] [Equation] +
Revised: Thereby,

Original: [Equation]

Original: |

Original: [Equation]

Original: [Equation]

Original: ≤

Original: [Equation]

Original: ( q

Original: [Equation]

Original: ( h | v ) || p

Original: [Equation]

Original: ( h | v ))

Original: [Equation]

Original: ( CC

Original: [Equation]

Original: CC 1 )

Original: [Equation]
Revised: [Equation]

Original: D F

Original: [Equation]
Revised: [Equation]

Original: q φ

Original: [Equation]

Original: h | v ) ||

Original: p θ ( h | v )) =(

Original: C 4 + CC 2 + CC 1 ) (cid:113) D F ( q φ ( h | v ) ||

Original: p θ ( h | v )) .

Original: As a result
Revised: As a result

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: D Consistency between D mF and D F
Revised: D Consistency between D mF and D F

Original: Theorem 5.
Revised: Theorem 5.

Original: Suppose lim || v ||→∞ p t ( v )( ∇ v log p θ ( v ) − ∇ v log p t ( v ))
Revised: Suppose lim || v ||→∞ p t ( v )( ∇ v log p θ ( v ) − ∇ v log p t ( v ))

Original: = 

Revised: =

Original: and E p ( (cid:15) ) (cid:2) (cid:15)(cid:15) (cid:62) (cid:3)
Revised: and E p ( (cid:15) ) (cid:2) (cid:15)(cid:15) (cid:62) (cid:3)

Original: =
Revised: =

Original: I , then
Revised: I , then

Original: D mF ( p t || p θ )
Revised: D mF ( p t || p θ )

Original: =
Revised: =

Original: D F ( p t || p θ ) .
Revised: D F ( p t || p θ ) .

Original: Proof.
Revised: Proof.

Original: Notice that E p ( (cid:15) ) (cid:2) (cid:15) (cid:62) ∇ v f ( v ) (cid:15) (cid:3) =
Revised: Notice that E p ( (cid:15) ) (cid:2) (cid:15) (cid:62) ∇ v f ( v ) (cid:15) (cid:3) =

Original: Tr( ∇ v f ( v )) by the assumption E p ( (cid:15) ) (cid:2) (cid:15)(cid:15) (cid:62) (cid:3) =
Revised: Tr( ∇ v f ( v )) by the assumption E p ( (cid:15) ) (cid:2) (cid:15)(cid:15) (cid:62) (cid:3) =

Original: I and
Revised: I and

Original: D mF ( p t || p θ ) can be simpliﬁed as
Revised: D mF ( p t || p θ ) can be simpliﬁed as

Original: [Equation]
Revised: [Equation]

Original: Suppose f ∈ F , i.e., f is a function from R d to R d and lim
Revised: Suppose f ∈ F , i.e., f is a function from R d to R d and lim

Original: || v ||→∞ p t ( v ) f
Revised: || v ||→∞ p t ( v ) f

Original: ( v ) = 0 , by the Stein’sidentity, we have E p t ( v ) (cid:2) ∇ v log p t ( v ) (cid:62)
Revised: ( v ) = 0 , by the Stein’s identity, we have E p t ( v ) (cid:2) ∇ v log p t ( v ) (cid:62)

Original: f ( v ) +
Revised: f ( v ) +

Original: Tr( ∇ v f ( v )) (cid:3) = 0 .
Revised: Tr( ∇ v f ( v )) (cid:3) = 0 .

Original: Thereby, we have
Revised: Thereby, we have

Original: [Equation]
Revised: [Equation]

Original: v ) [Equation] The equality is achieved when f ( v ) = ∇ v log p θ ( v ) − ∇ v log p t ( v ) , which is a function in F byassumption.
Revised: The equality is achieved when f ( v ) = ∇ v log p θ ( v ) − ∇ v log p t ( v ) , which is a function in F by assumption.

Original: As a result, D m F ( p t || p θ )
Revised: As a result, D m F ( p t || p θ )

Original: =
Revised: =

Original: D F ( p t || p θ ) .
Revised: D F ( p t || p θ ) .

Original: E Experimental Settings
Revised: E Experimental Settings

Original: For experiments on GRBMs, q φ ( h | v ) is always a Bernoulli distribution parametermized by afully connected layer with the sigmoid activation and we use the Gumbel-Softmax trick [14] forreparameterization of q φ ( h | v ) with 0.1 as the temperature.
Revised: For experiments on GRBMs, q φ ( h | v ) is always a Bernoulli distribution parametermized by a fully connected layer with the sigmoid activation and we use the Gumbel-Softmax trick [14] for reparameterization of q φ ( h | v ) with 0.1 as the temperature.

Original: By default, the variational parameter φ isupdated for K = 5 times for each minibatch.
Revised: By default, the variational parameter φ is updated for K = 5 times for each minibatch.

Original: The batch size is 100 for all experiments.
Revised: The batch size is 100 for all experiments.

Original: We use the
Revised: We use the

Original: Adam optimizer for all experiments.
Revised: Adam optimizer for all experiments.

Original: E.1 Learning EBLVMs with Kernelized Stein Discrepancy
Revised: E.1 Learning EBLVMs with Kernelized Stein Discrepancy

Original: The density of the checkerboard dataset is shown in Fig.
Revised: The density of the checkerboard dataset is shown in Fig.

Original: 5 (a).
Revised: 5 (a).

Original: The dimension of h is 4.
Revised: The dimension of h is 4.

Original: The learningrate is 0.001.
Revised: The learning rate is 0.001.

Original: E.2 Learning EBLVMs with Score Matching
Revised: E.2 Learning EBLVMs with Score Matching

Original: Experiments on GRBMs: The Frey face dataset 1 consists of gray-scaled face images of size 20 ×28.
Revised: Experiments on GRBMs: The Frey face dataset 2 consists of gray-scaled face images of size 20 × 28.

Original: Following BiSM, we split 1,400 images for training, 300 images for validation and 265 imagesfor testing; the dimension of h is 400; the learning rate is 0.0002.
Revised: Following BiSM, we split 1,400 images for training, 300 images for validation and 265 images for testing; the dimension of h is 400; the learning rate is 0.0002.

Original: Experiments on deep EBLVMs: The MNIST dataset consists of gray-scaled hand-written digitsof size 28 × 28.
Revised: Experiments on deep EBLVMs: The MNIST dataset consists of gray-scaled hand-written digits of size 28 × 28.

Original: The dimension of h is 20.
Revised: The dimension of h is 20.

Original: Following BiSM, we split 60,000 samples for trainingand 10,000 samples for testing on MNIST; g 1 consists of a 12-layer ResNet and a linear layer whichaligns the output dimension of g 1 with the dimension of h ; g 3 is a fully connected layer; q φ ( h | v ) is a
Revised: Following BiSM, we split 60,000 samples for training and 10,000 samples for testing on MNIST; g 1 consists of a 12-layer ResNet and a linear layer which aligns the output dimension of g 1 with the dimension of h ; g 3 is a fully connected layer; q φ ( h | v ) is a

Original: Gaussian distribution parameterized by a 3-layer convolutional neural network (CNN).
Revised: Gaussian distribution parameterized by a 3-layer convolutional neural network (CNN).

Original: For BiMDSM,the times of updating parameters in the lower level problem is K = 5 and the number of steps forgradient unrolling in the higher level problem is N = 0 .
Revised: For BiMDSM, the times of updating parameters in the lower level problem is K = 5 and the number of steps for gradient unrolling in the higher level problem is N = 0 .

Original: E.3 Evaluating EBLVMs
Revised: E.3 Evaluating EBLVMs

Original: f η is a multilayer perceptron (MLP) with 2 hidden layers and each layer has the same width.
Revised: f η is a multilayer perceptron (MLP) with 2 hidden layers and each layer has the same width.

Original: The
Revised: The

Original: GRBM is initialized as a standard Gaussian distribution by letting b = 0 , c
Revised: GRBM is initialized as a standard Gaussian distribution by letting b = 0 , c

Original: = 0 , W = 0 , σ = 1 , sowe can get accurate samples from it.
Revised: = 0 , W = 0 , σ = 1 , so we can get accurate samples from it.

Original: We get 20,000 samples from the initial GRBM, and split 16,000samples for training, 2,000 samples for validationand 2,000 samplesfor testing. The learning rate is0.0002. The number of h sampled for each v is L = 1 .
Revised: We get 20,000 samples from the training validation samples

Original: F Additional Results 
Revised: F Additional Results F.1 Learning EBLVMs with Kernelized Stein Discrepancy

Original: [Figure] F.1 Learning EBLVMs with Kernelized Stein Discrepancy

Original: The density of the checkerboard dataset is shown in Fig.
Revised: The density of the checkerboard dataset is shown in Fig.

Original: 5 (a).
Revised: 5 (a).

Original: The densities of GRBMs learnedby KSD, VaGES-KSD and IS-KSD are shown in Fig.
Revised: The densities of GRBMs learned by KSD, VaGES-KSD and IS-KSD are shown in Fig.

Original: 5 (b-h).
Revised: 5 (b-h).

Original: Our VaGES-KSD is comparableto the KSD baseline and is better than the IS-KSD baseline.
Revised: Our VaGES-KSD is comparable to the KSD baseline and is better than the IS-KSD baseline.

Original: The result is consistent with the testlog-likelihood results in Figure 1 in the full paper.
Revised: The result is consistent with the test log-likelihood results in Figure 1 in the full paper.

Original: [Figure]
Revised: [Figure]