Original: Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model
Revised: Adaptive Multi-stage Density Ratio Estimation for Learning Latent Space Energy-based Model

Original: Abstract
Revised: Abstract

Original: This paper studies the fundamental problem of learning energy-based model (EBM) in the latent space of the generator model.
Revised: This paper studies the fundamental problem of learning energy-based model (EBM) in the latent space of the generator model.

Original: Learning such prior model typically requires running costly Markov Chain Monte Carlo (MCMC).
Revised: Learning such prior model typically requires running costly Markov Chain Monte Carlo (MCMC).

Original: Instead, we propose to use noise contrastive estimation (NCE) to discriminatively learn the EBM through density ratio estimation between the latent prior density and latent posterior density.
Revised: Instead, we propose to use noise contrastive estimation (NCE) to discriminatively learn the EBM through density ratio estimation between the latent prior density and latent posterior density.

Original: However, the NCE typically fails to accurately estimate such density ratio given large gap between two densities.
Revised: However, the NCE typically fails to accurately estimate such density ratio given large gap between two densities.

Original: To effectively tackle this issue and learn more expressive prior models, we develop the adaptive multi-stage density ratio estimation which breaks the estimation into multiple stages and learn different stages of density ratio sequentially and adaptively.
Revised: To effectively tackle this issue and learn more expressive prior models, we develop the adaptive multi-stage density ratio estimation which breaks the estimation into multiple stages and learn different stages of density ratio sequentially and adaptively.

Original: The latent prior model can be gradually learned using ratio estimated in previous stage so that the final latent space EBM prior can be naturally formed by product of ratios in different stages.
Revised: The latent prior model can be gradually learned using ratio estimated in previous stage so that the final latent space EBM prior can be naturally formed by product of ratios in different stages.

Original: The proposed method enables informative and much sharper prior than existing baselines, and can be trained efficiently.
Revised: The proposed method enables informative and much sharper prior than existing baselines, and can be trained efficiently.

Original: Our experiments demonstrate strong performances in image generation and reconstruction as well as anomaly detection.
Revised: Our experiments demonstrate strong performances in image generation and reconstruction as well as anomaly detection.

Original:  Introduction
Revised:  Introduction

Original: Deep generative model provides a powerful framework for representing complex data distributions and have seen many successful applications in image and video synthesis [21, 43, 51], representation learning [34] as well as unsupervised or semi-supervised learning [19, 37].
Revised: Deep generative model provides a powerful framework for representing complex data distributions and have seen many successful applications in image and video synthesis [21, 43, 51], representation learning [34] as well as unsupervised or semi-supervised learning [19, 37].

Original: Such model, referred to as generator model usually consists of low-dimensional latent variables that follow non-informative prior distribution and a top-down network that maps such latent vector to the observed example.
Revised: Such model, referred to as generator model usually consists of low-dimensional latent variables that follow non-informative prior distribution and a top-down network that maps such latent vector to the observed example.

Original: The informative prior model in the latent space [36, 1] can be learned to further improve the expressive power of the whole model.
Revised: The informative prior model in the latent space [36, 1] can be learned to further improve the expressive power of the whole model.

Original: Specifically, we consider learning energy-based model (EBM) in the latent space as our informative prior for the generator model.
Revised: Specifically, we consider learning energy-based model (EBM) in the latent space as our informative prior for the generator model.

Original: Learning latent space EBM can be challenging and requires iterative Markov Chain Monte Carlo (MCMC) sampling step which is computationally expensive and sensitive to hyperparameters.
Revised: Learning latent space EBM can be challenging and requires iterative Markov Chain Monte Carlo (MCMC) sampling step which is computationally expensive and sensitive to hyperparameters.

Original: In this paper, we instead propose to use noise contrastive estimation (NCE) [12] for learning EBM prior via density ratio estimation.
Revised: In this paper, we instead propose to use noise contrastive estimation (NCE) [12] for learning EBM prior via density ratio estimation.

Original: The EBM is learned discriminatively by classifying the latent vector sampled from the prior density and the latent sampled from posterior density.
Revised: The EBM is learned discriminatively by classifying the latent vector sampled from the prior density and the latent sampled from posterior density.

Original: Instead of variational learned inference [23, 41] which needs a separate inference network designed, we obtain the posterior latent sample through short-run Langevin dynamics [33] to ensure more accurate inference.
Revised: Instead of variational learned inference [23, 41] which needs a separate inference network designed, we obtain the posterior latent sample through short-run Langevin dynamics [33] to ensure more accurate inference.

Original: However, the success of NCE depends on the closeness of prior density and posterior density [18].
Revised: However, the success of NCE depends on the closeness of prior density and posterior density [18].

Original: Given large gap between two densities, NCE typically fails to accurately estimate such density ratio which leads to inaccurate EBM modeling.
Revised: Given large gap between two densities, NCE typically fails to accurately estimate such density ratio which leads to inaccurate EBM modeling.

Original: To effectively tackle the inaccurate estimation issue and further learn more expressive prior model, we develop the adaptive multi-stage density ratio estimation for latent space EBM training.
Revised: To effectively tackle the inaccurate estimation issue and further learn more expressive prior model, we develop the adaptive multi-stage density ratio estimation for latent space EBM training.

Original: The proposed model breaks the density estimation into multiple stages and learn different stages of density ratio sequentially.
Revised: The proposed model breaks the density estimation into multiple stages and learn different stages of density ratio sequentially.

Original: Thanks to the low-dimensionality of the latent space and the short-run style posterior inference, in each stage, the gap between prior and posterior density could be kept in check which makes NCE easier.
Revised: Thanks to the low-dimensionality of the latent space and the short-run style posterior inference, in each stage, the gap between prior and posterior density could be kept in check which makes NCE easier.

Original: The density ratio estimated in previous stage can be further integrated into the current prior model as a correction term to build more expressive prior density for later stage.
Revised: The density ratio estimated in previous stage can be further integrated into the current prior model as a correction term to build more expressive prior density for later stage.

Original: With such framework, the final latent space EBM prior can then be naturally formed by product of ratios in different stages on top of the initial base prior.
Revised: With such framework, the final latent space EBM prior can then be naturally formed by product of ratios in different stages on top of the initial base prior.

Original: Contributions: 1) we propose an EBM prior on generator model which is modelled through estimation of density ratios in multiple stages.
Revised: Contributions: 1) we propose an EBM prior on generator model which is modelled through estimation of density ratios in multiple stages.

Original: We develop the adaptive multi-stage noise contrastive estimation to learn different stages of ratios sequentially and adaptively.
Revised: We develop the adaptive multi-stage noise contrastive estimation to learn different stages of ratios sequentially and adaptively.

Original: The ratio estimated in previous stage can be integrated to form the more informative prior in the later stage.
Revised: The ratio estimated in previous stage can be integrated to form the more informative prior in the later stage.

Original: 3) we demonstrate strong empirical results to illustrate the proposed method.
Revised: 3) we demonstrate strong empirical results to illustrate the proposed method.

Original:  Background 2.1 Maximum likelihood learning of deep latent variable models
Revised:  Background 2.1 Maximum likelihood learning of deep latent variable models

Original: Let x ∈ R D be an observed example such as an image, and z ∈ R d be the latent variables where d < D .
Revised: Let x ∈ R D be an observed example such as an image, and z ∈ R d be the latent variables where d < D .

Original: A latent variable generative model (a.k.a, generator model ) factorize the joint distribution of ( x , z ) as
Revised: A latent variable generative model (a.k.a, generator model ) factorize the joint distribution of ( x , z ) as

Original: [Equation]
Revised: [Equation]

Original: where p ( z ) is the prior distribution over latent variables z , p θ ( x | z ) is the top-down generation model with parameters θ .
Revised: where p ( z ) is the prior distribution over latent variables z , p θ ( x | z ) is the top-down generation model with parameters θ .

Original: Usually the prior distribution is chosen to be a simple one such as N (0 , I d ) , but it can also be more expressive with learnable parameters [36].
Revised: Usually the prior distribution is chosen to be a simple one such as N (0 , I d ) , but it can also be more expressive with learnable parameters [36].

Original: The generation model is the same as that in VAE [23], i.e., x = g θ
Revised: The generation model is the same as that in VAE [23], i.e., x = g θ

Original: ( z ) + ϵ with g θ to be the decoder network and ϵ ∼ N (cid:0) 0 , σ 2 I D (cid:1) , so that p θ ( x | z ) =
Revised: ( z ) + ϵ with g θ to be the decoder network and ϵ ∼ N (cid:0) 0 , σ 2 I D (cid:1) , so that p θ ( x | z ) =

Original: N (cid:0) g θ ( z ) , σ 2 I D (cid:1) .
Revised: N (cid:0) g θ ( z ) , σ 2 I D (cid:1) .

Original: As in VAE, σ 2 takes a pre-specified value.
Revised: As in VAE, σ 2 takes a pre-specified value.

Original: Given a set of N training samples { x i , i = 1 , . . .
Revised: Given a set of N training samples { x i , i = 1 , . . .

Original: , N } from the unknown data distribution p data ( x ) , the model p θ can be trained by maximizing the log likelihood over training samples
Revised: , N } from the unknown data distribution p data ( x ) , the model p θ can be trained by maximizing the log likelihood over training samples

Original: L ( θ ) =
Revised: L ( θ ) =

Original: 1 N (cid:80) Ni
Revised: 1 N (cid:80) Ni

Original: =1 log p θ ( x i ) .
Revised: =1 log p θ ( x i ) .

Original: Maximizing the log likelihood L ( θ ) can be accomplished by gradient ascent where the gradient can be obtained from
Revised: Maximizing the log likelihood L ( θ ) can be accomplished by gradient ascent where the gradient can be obtained from

Original: [Equation]
Revised: [Equation]

Original: ∇ θ log p θ ( x , z ) can be easily computed according to the form of log p θ ( x , z ) , however, approximating the expectation requires drawing samples from p
Revised: ∇ θ log p θ ( x , z ) can be easily computed according to the form of log p θ ( x , z ) , however, approximating the expectation requires drawing samples from p

Original: θ ( z | x ) , which can be difficult.
Revised: θ ( z | x ) , which can be difficult.

Original: Sampling from the intractable posterior p θ ( z | x ) requires MCMC, and one convenient MCMC algorithm is Langevin
Revised: Sampling from the intractable posterior p θ ( z | x ) requires MCMC, and one convenient MCMC algorithm is Langevin

Original: Dynamics (LD) [30].
Revised: Dynamics (LD) [30].

Original: Given a step size s > 0 , and an initial value z 0 , the Lanegvin dynamics iterates
Revised: Given a step size s > 0 , and an initial value z 0 , the Lanegvin dynamics iterates

Original: [Equation]
Revised: [Equation]

Original: (3) where ω k ∼ N (0 , I ) .
Revised: (3) where ω k ∼ N (0 , I ) .

Original: For sufficiently small step size s , the marginal distribution of z k will converge to p θ ( z | x ) as k → ∞ .
Revised: For sufficiently small step size s , the marginal distribution of z k will converge to p θ ( z | x ) as k → ∞ .

Original: However, it is not feasible to run Langevin dynamics until convergence, and in practice the iteration in Eq.
Revised: However, it is not feasible to run Langevin dynamics until convergence, and in practice the iteration in Eq.

Original: 3 is run for finite iterations, which yields a Markov chain with an invariant distribution approximately close to the original target distribution.
Revised: 3 is run for finite iterations, which yields a Markov chain with an invariant distribution approximately close to the original target distribution.

Original: When z 0 is initialized from the noise distribution, the algorithm is called noise-initialized short-run LD [32, 33].
Revised: When z 0 is initialized from the noise distribution, the algorithm is called noise-initialized short-run LD [32, 33].

Original: 2.2 Learning EBMs with discriminative density ratio estimation
Revised: 2.2 Learning EBMs with discriminative density ratio estimation

Original: Suppose there are two distributions with density functions p ( x ) and q ( x ) from which we can sample, we can estimate the density ratio 1 r ( x )
Revised: Suppose there are two distributions with density functions p ( x ) and q ( x ) from which we can sample, we can estimate the density ratio 1 r ( x )

Original: =
Revised: =

Original: p ( x ) q ( x ) by training a classifier to distinguish samples from p and q [44].
Revised: p ( x ) q ( x ) by training a classifier to distinguish samples from p and q [44].

Original: Specifically, we can train the binary classifier D : R n → (0 , 1) by minimizing the binary cross-entropy loss
Revised: Specifically, we can train the binary classifier D : R n → (0 , 1) by minimizing the binary cross-entropy loss

Original: [Equation]
Revised: [Equation]

Original: The objective is minimized when D ( x ) =
Revised: The objective is minimized when D ( x ) =

Original: q ( x ) q ( x )+ p ( x ) [11], and denoting the classifier at optimality by
Revised: q ( x ) q ( x )+ p ( x ) [11], and denoting the classifier at optimality by

Original: D ∗ ( x ) , we have r ( x ) =
Revised: D ∗ ( x ) , we have r ( x ) =

Original: p ( x ) q ( x ) ≈ 1 − D ∗ ( x ) D ∗ ( x ) .
Revised: p ( x ) q ( x ) ≈ 1 − D ∗ ( x ) D ∗ ( x ) .

Original: Equivalently, the ratio r ( x ) = p ( x ) q ( x ) can be estimated by directly minimizing
Revised: Equivalently, the ratio r ( x ) = p ( x ) q ( x ) can be estimated by directly minimizing

Original: [Equation]
Revised: [Equation]

Original: where r ϕ ( x ) is a non-negative ratio estimating model implemented as the exponential of an unconstrained neural network with scalar output.
Revised: where r ϕ ( x ) is a non-negative ratio estimating model implemented as the exponential of an unconstrained neural network with scalar output.

Original: The minimizer ϕ ∗ satisfies r ϕ ∗ ( x ) = p ( x ) q ( x ) [12].
Revised: The minimizer ϕ ∗ satisfies r ϕ ∗ ( x ) = p ( x ) q ( x ) [12].

Original: Such a technique can be useful for training Energy-based models (EBMs).
Revised: Such a technique can be useful for training Energy-based models (EBMs).

Original: Given samples from the true data distribution p data ( x ) and a base distribution q ( x ) that we can sample from, we consider EBMs of the form p ϕ
Revised: Given samples from the true data distribution p data ( x ) and a base distribution q ( x ) that we can sample from, we consider EBMs of the form p ϕ

Original: ( x ) = 1 Z r ϕ ( x ) q ( x ) , where Z is the normalizing constant and r ϕ is an unconstrained positive function.
Revised: ( x ) = 1 Z r ϕ ( x ) q ( x ) , where Z is the normalizing constant and r ϕ is an unconstrained positive function.

Original: With this parametrization, obviously the optimal r ϕ equals the density-ratio p data ( x ) q ( x ) .
Revised: With this parametrization, obviously the optimal r ϕ equals the density-ratio p data ( x ) q ( x ) .

Original: In fact, if r ϕ ( x ) is trained with density ratio estimation, the normalizing constant Z is simply 1 .
Revised: In fact, if r ϕ ( x ) is trained with density ratio estimation, the normalizing constant Z is simply 1 .

Original: Therefore, the problem of learning an EBM becomes the problem of estimating a density-ratio, which can be solved by discriminative density ratio estimation.
Revised: Therefore, the problem of learning an EBM becomes the problem of estimating a density-ratio, which can be solved by discriminative density ratio estimation.

Original: Typically the base distribution q ( x ) is chosen to be Gaussian, resulted in so-called noise contrastive estimation (NCE) [12].
Revised: Typically the base distribution q ( x ) is chosen to be Gaussian, resulted in so-called noise contrastive estimation (NCE) [12].

Original: Although NCE provides a promising way to train EBMs without running MCMC, the accuracy of the density ratio estimation depends on the closeness between the two distributions.
Revised: Although NCE provides a promising way to train EBMs without running MCMC, the accuracy of the density ratio estimation depends on the closeness between the two distributions.

Original: The ratio estimator is often severely inaccurate when the gap between p and q is large.
Revised: The ratio estimator is often severely inaccurate when the gap between p and q is large.

Original: Rhodes et al.
Revised: Rhodes et al.

Original: [42] propose Telescoping density-Ratio Estimation (TRE), which breaks the density ratio estimation task into a collection of harder sub-tasks and show improvement over simple NCE on density ratio estimation.
Revised: [42] propose Telescoping density-Ratio Estimation (TRE), which breaks the density ratio estimation task into a collection of harder sub-tasks and show improvement over simple NCE on density ratio estimation.

Original: However, it is still difficult to apply the technique to energy-based modeling.
Revised: However, it is still difficult to apply the technique to energy-based modeling.

Original: On one hand, EBMs in high-dimensional data space such as image space can be highly complex and multimodal, making them extremely far away from simple noise distribution.
Revised: On one hand, EBMs in high-dimensional data space such as image space can be highly complex and multimodal, making them extremely far away from simple noise distribution.

Original: On the other hand, the intermediate distributions are pre-designed through linear transition, making them less effective to connect complicated target densities.
Revised: On the other hand, the intermediate distributions are pre-designed through linear transition, making them less effective to connect complicated target densities.

Original: In Rhodes et al.
Revised: In Rhodes et al.

Original: [42], TRE only obtains limited success on training EBMs through density estimation on MNIST dataset.
Revised: [42], TRE only obtains limited success on training EBMs through density estimation on MNIST dataset.

Original:  Adaptive Multi-stage Desnity Ratio Estimation
Revised:  Adaptive Multi-stage Desnity Ratio Estimation

Original: In this section, we introduce adaptive multi-stage density ratio estimation on latent space in details.
Revised: In this section, we introduce adaptive multi-stage density ratio estimation on latent space in details.

Original: 3.1 Multi-stage density ratio estimation in latent space
Revised: 3.1 Multi-stage density ratio estimation in latent space

Original: Instead of modeling directly on high-dimensional data space, it is easier to introduce low-dimensional latent variables and learn an EBM in latent space, while also learning a mapping from the latent space to the data space [3, 26].
Revised: Instead of modeling directly on high-dimensional data space, it is easier to introduce low-dimensional latent variables and learn an EBM in latent space, while also learning a mapping from the latent space to the data space [3, 26].

Original: We follow this approach and attempt to model a latent space EBM using contrastive estimation.
Revised: We follow this approach and attempt to model a latent space EBM using contrastive estimation.

Original: The latent EBM can be learned discriminatively by estimating the ratio between prior density and the posterior density.
Revised: The latent EBM can be learned discriminatively by estimating the ratio between prior density and the posterior density.

Original: Due to low-dimensionality of the latent space, such densities can be much easier to deal with than those in high dimensional data space.
Revised: Due to low-dimensionality of the latent space, such densities can be much easier to deal with than those in high dimensional data space.

Original: However, it presents new challenges.
Revised: However, it presents new challenges.

Original: Firstly, while the target density in data space is given and fixed (i.e., empirical data distribution), posterior density in latent space is driven by the prior density and the inference on the posterior can be hard.
Revised: Firstly, while the target density in data space is given and fixed (i.e., empirical data distribution), posterior density in latent space is driven by the prior density and the inference on the posterior can be hard.

Original: Secondly, while the prior is typically assumed to be un-informative and fixed (e.g., unit Gaussian), the expressiveness of the model is limited.
Revised: Secondly, while the prior is typically assumed to be un-informative and fixed (e.g., unit Gaussian), the expressiveness of the model is limited.

Original: Inspired by [42], we propose to learn the latent space EBM of the below form through multiple stages
Revised: Inspired by [42], we propose to learn the latent space EBM of the below form through multiple stages

Original: [Equation]
Revised: [Equation]

Original: where p 

Revised: where p

Original: ( z ) is the unit Gaussian base distribution, and r ϕ k is the intermediate density ratio learned in each stage.
Revised: ( z ) is the unit Gaussian base distribution, and r ϕ k is the intermediate density ratio learned in each stage.

Original: Such proposed model shares the similar root as the Product-of-Expert (PoE) [17] where r ϕ k in each stage can be treated as individual expert model, and it has the potential to produce much sharper distribution than the one with single expert model built such as [36].
Revised: Such proposed model shares the similar root as the Product-of-Expert (PoE) [17] where r ϕ k in each stage can be treated as individual expert model, and it has the potential to produce much sharper distribution than the one with single expert model built such as [36].

Original: Although the formulation of EBM in Eq. 5 is related to the TRE proposed in Rhodes et al.
Revised: Although the formulation of EBM in Eq. 5 is related to the TRE proposed in Rhodes et al.

Original: [42], our training method is fundamentally different.
Revised: [42], our training method is fundamentally different.

Original: In the next section, we will introduce the training of our model, and highlight the distinction with TRE.
Revised: In the next section, we will introduce the training of our model, and highlight the distinction with TRE.

Original: 3.2 Learning latent EBMs with adaptive multi-stage density ratio estimation
Revised: 3.2 Learning latent EBMs with adaptive multi-stage density ratio estimation

Original: Our proposed generator model specifies the distribution on joint space ( x , z ) : p θ,ϕ ( x , z ) =
Revised: Our proposed generator model specifies the distribution on joint space ( x , z ) : p θ,ϕ ( x , z ) =

Original: p ϕ ( z ) p θ ( x | z ) , where p ϕ ( z ) is the prior model specified in Eq.
Revised: p ϕ ( z ) p θ ( x | z ) , where p ϕ ( z ) is the prior model specified in Eq.

Original: 5, and ϕ = { ϕ 0 , . . .
Revised: 5, and ϕ = { ϕ 0 , . . .

Original: , ϕ m − 1 } that collects parameters for all intermediate learned ratios.
Revised: , ϕ m − 1 } that collects parameters for all intermediate learned ratios.

Original: It is tempting to apply maximum likelihood estimation (MLE) to train such model.
Revised: It is tempting to apply maximum likelihood estimation (MLE) to train such model.

Original: However, there are several challenges: (1) learning of latent EBM p
Revised: However, there are several challenges: (1) learning of latent EBM p

Original: ϕ
Revised: ϕ

Original: ( z ) needs costly and hard mixing MCMC sampling.
Revised: ( z ) needs costly and hard mixing MCMC sampling.

Original: (2) the prior p ϕ ( z ) needs to have a fixed form during training and cannot be adaptively adjusted.
Revised: (2) the prior p ϕ ( z ) needs to have a fixed form during training and cannot be adaptively adjusted.

Original: To alleviate the aforementioned limitations, we therefore break the density ratio estimation of p ϕ ( z ) into m stages, learn and build the prior sequentially and adaptively.
Revised: To alleviate the aforementioned limitations, we therefore break the density ratio estimation of p ϕ ( z ) into m stages, learn and build the prior sequentially and adaptively.

Original: Specifically, in the k th stage, we consider the generator model of the form
Revised: Specifically, in the k th stage, we consider the generator model of the form

Original: [Equation]
Revised: [Equation]

Original: where p ϕ k
Revised: where p ϕ k

Original: ( z ) = (cid:81) k − 1 i =

Revised: ( z ) = (cid:81) k − 1 i =

Original: r ϕ i ( z ) p 0 ( z ) .
Revised: r ϕ i ( z ) p 0 ( z ) .

Original: The whole training procedure iterates between the maximum likelihood estimation of generation model θ and the sequential contrastive estimation of prior ϕ .
Revised: The whole training procedure iterates between the maximum likelihood estimation of generation model θ and the sequential contrastive estimation of prior ϕ .

Original: MLE for generation model θ : The generation model can be trained by maximizing the marginal log-likelihood p θ ( x ) .
Revised: MLE for generation model θ : The generation model can be trained by maximizing the marginal log-likelihood p θ ( x ) .

Original: In k th stage, the complete data log-likelihood of the model p θ,ϕ k
Revised: In k th stage, the complete data log-likelihood of the model p θ,ϕ k

Original: ( x , z ) can be expressed as
Revised: ( x , z ) can be expressed as

Original: [Equation]
Revised: [Equation]

Original: where g θ is the decoder and C is a constant independent of θ .
Revised: where g θ is the decoder and C is a constant independent of θ .

Original: The generation model parameter θ is then updated using the gradient based on Eq.
Revised: The generation model parameter θ is then updated using the gradient based on Eq.

Original: 2 with a batch of training n samples x i :
Revised: 2 with a batch of training n samples x i :

Original: [Equation]
Revised: [Equation]

Original: where η t is the learning rate.
Revised: where η t is the learning rate.

Original: The expectation over the posterior can be approximated by running short-run Lanegvin dynamics in Eq.
Revised: The expectation over the posterior can be approximated by running short-run Lanegvin dynamics in Eq.

Original: Note that the running LD to sample from p θ ( z | x ) is equivalent to sample from p θ ( x , z ) with fixed x .
Revised: Note that the running LD to sample from p θ ( z | x ) is equivalent to sample from p θ ( x , z ) with fixed x .

Original: Adaptive multi-stage NCE for prior ϕ :
Revised: Adaptive multi-stage NCE for prior ϕ :

Original: The prior model p ϕ ( z ) can be sequentially and adaptively learned to bridge the gap between prior and posterior densities in the previous stages.
Revised: The prior model p ϕ ( z ) can be sequentially and adaptively learned to bridge the gap between prior and posterior densities in the previous stages.

Original: Specifically, in k th stage, the correction term r ϕ k can be trained to estimate the density ratio between p ϕ k
Revised: Specifically, in k th stage, the correction term r ϕ k can be trained to estimate the density ratio between p ϕ k

Original: ( z ) and its aggregated posterior q k ( z ) through contrastive estimation using Eq. 4.
Revised: ( z ) and its aggregated posterior q k ( z ) through contrastive estimation using Eq. 4.

Original: The appealing advantage of this estimator is that it simply trains a binary classifier rather than using expensive MCMC sampling.
Revised: The appealing advantage of this estimator is that it simply trains a binary classifier rather than using expensive MCMC sampling.

Original: The optimality of such logistic loss leads to the estimated r ϕ k ( z ) ≈
Revised: The optimality of such logistic loss leads to the estimated r ϕ k ( z ) ≈

Original: q k ( z ) p ϕk ( z ) .
Revised: q k ( z ) p ϕk ( z ) .

Original: The prior model in the ( k + 1) th stage can then be sequentially adapted to match the previous aggregated posterior q k ( z ) , i.e., letting p ϕ k
Revised: The prior model in the ( k + 1) th stage can then be sequentially adapted to match the previous aggregated posterior q k ( z ) , i.e., letting p ϕ k

Original: +

Revised: +

Original: ( z ) =
Revised: ( z ) =

Original: r ϕ k ( z ) p ϕ k ( z ) match q k ( z ) .
Revised: r ϕ k ( z ) p ϕ k ( z ) match q k ( z ) .

Original: Given the new prior, we similarly infer the posterior using short-run LD in Eq.
Revised: Given the new prior, we similarly infer the posterior using short-run LD in Eq.

Original: Then, the next density ratio estimator r ϕ k +1 can be learned through contrastive estimation to match the updated prior and its aggregated posterior, which is further used to adapt the prior in next stage.
Revised: Then, the next density ratio estimator r ϕ k +1 can be learned through contrastive estimation to match the updated prior and its aggregated posterior, which is further used to adapt the prior in next stage.

Original: Particularly, we have
Revised: Particularly, we have

Original: [Equation]
Revised: [Equation]

Original: where q k ( z ) is the aggregated posterior for prior p ϕ k
Revised: where q k ( z ) is the aggregated posterior for prior p ϕ k

Original: ( z ) .
Revised: ( z ) .

Original: The above telescoping product holds since the new prior is designed to match the aggregated posterior in previous stage, i.e., p ϕ k +1 ( z )
Revised: The above telescoping product holds since the new prior is designed to match the aggregated posterior in previous stage, i.e., p ϕ k +1 ( z )

Original: ≈ q k ( z ) .
Revised: ≈ q k ( z ) .

Original: Each stage estimates the ratio r ϕ k ( z )
Revised: Each stage estimates the ratio r ϕ k ( z )

Original: ≈ q k ( z ) p ϕk ( z ) via contrasive estimation.
Revised: ≈ q k ( z ) p ϕk ( z ) via contrasive estimation.

Original: Then the aggregated posterior q m − 1 ( z ) can be obtained via
Revised: Then the aggregated posterior q m − 1 ( z ) can be obtained via

Original: [Equation]
Revised: [Equation]

Original: Our final prior model can then be obtained by matching such aggregated posterior q m − 1 ( z ) which has the same form as Eq. 5.
Revised: Our final prior model can then be obtained by matching such aggregated posterior q m − 1 ( z ) which has the same form as Eq. 5.

Original: The proposed training is illustrated in Figure 1 and the algorithm is detailed in Algorithm 1.
Revised: The proposed training is illustrated in Figure 1 and the algorithm is detailed in Algorithm 1.

Original: Comparison with TRE in Rhodes et al.
Revised: Comparison with TRE in Rhodes et al.

Original: [42]: The most significant difference between our training method and TRE is that TRE assumes a fixed target distribution and construct multiple stages simultaneously via interpolation, whereas our model considers adaptive targets and learn multi-stage density ratio estimators sequentially via NCE.
Revised: [42]: The most significant difference between our training method and TRE is that TRE assumes a fixed target distribution and construct multiple stages simultaneously via interpolation, whereas our model considers adaptive targets and learn multi-stage density ratio estimators sequentially via NCE.

Original: We conduct empirical comparisons in Sec. 5.5.
Revised: We conduct empirical comparisons in Sec. 5.5.

Original: Sampling from prior p ϕ k ( z ) :
Revised: Sampling from prior p ϕ k ( z ) :

Original: The density ratio estimation of r ϕ k in each stage requires the samples from the prior p ϕ k
Revised: The density ratio estimation of r ϕ k in each stage requires the samples from the prior p ϕ k

Original: ( z ) and posterior.
Revised: ( z ) and posterior.

Original: The posterior samples are inferred through short-run Langevin dynamics which can be efficient and accurate.
Revised: The posterior samples are inferred through short-run Langevin dynamics which can be efficient and accurate.

Original: For drawing prior samples from p ϕ k ( z ) , we can either use short-run prior Langevin dynamics or persistent update.
Revised: For drawing prior samples from p ϕ k ( z ) , we can either use short-run prior Langevin dynamics or persistent update.

Original: One one hand, we could directly utilize the short-run Langevin on the p ϕ k
Revised: One one hand, we could directly utilize the short-run Langevin on the p ϕ k

Original: ( z ) to obtain prior samples.
Revised: ( z ) to obtain prior samples.

Original: On the other hand, the samples from prior can also be obtained in a persistent chain manner to avoid the prior Langevin altogether.
Revised: On the other hand, the samples from prior can also be obtained in a persistent chain manner to avoid the prior Langevin altogether.

Original: When introducing the ( k + 1) th stage of density ratio estimation, we assume that the current estimator r ϕ k ( z ) performs well in modeling the ratio between the current aggregated posterior distribution q k ( z ) and current prior p ϕ k ( z ) .
Revised: When introducing the ( k + 1) th stage of density ratio estimation, we assume that the current estimator r ϕ k ( z ) performs well in modeling the ratio between the current aggregated posterior distribution q k ( z ) and current prior p ϕ k ( z ) .

Original: Therefore, we simply use samples from q k ( z ) to approximately serve as samples from the new prior p ϕ k +1 ( z ) for the learning of r ϕ k +

Revised: Therefore, we simply use samples from q k ( z ) to approximately serve as samples from the new prior p ϕ k +1 ( z ) for the learning of r ϕ k +

Original: ( z ) .
Revised: ( z ) .

Original: In practice, it is achieved by maintaining a memory matrix that stores a posterior samples ˜
Revised: In practice, it is achieved by maintaining a memory matrix that stores a posterior samples ˜

Original: z i associated to each data point x i .
Revised: z i associated to each data point x i .

Original: Note that we only need to keep one memory matrix throughout the training, as only the posterior samples from the previous stage are needed.
Revised: Note that we only need to keep one memory matrix throughout the training, as only the posterior samples from the previous stage are needed.

Original: Test time sampling:
Revised: Test time sampling:

Original: After obtaining the density ratio estimators in each stage and form the final
Revised: After obtaining the density ratio estimators in each stage and form the final

Original: EBM prior p ϕ
Revised: EBM prior p ϕ

Original: ( z ) , we can sample latent variables z ∼ p ϕ
Revised: ( z ) , we can sample latent variables z ∼ p ϕ

Original: ( z ) and produce sample x by decoding z .
Revised: ( z ) and produce sample x by decoding z .

Original: Sampling from p ϕ
Revised: Sampling from p ϕ

Original: ( z ) can be done by either running Langvin dynamics with ∇ z log p ϕ
Revised: ( z ) can be done by either running Langvin dynamics with ∇ z log p ϕ

Original: ( z ) =
Revised: ( z ) =

Original: ∇ z (cid:16)(cid:80) m − 1 i =0 log r ϕ
Revised: ∇ z (cid:16)(cid:80) m − 1 i =0 log r ϕ

Original: i ( z ) − 12 ∥ z ∥ 2 (cid:17) , or Sampling-Importance-Resampling (SIR) techniques.
Revised: i ( z ) − 12 ∥ z ∥ 2 (cid:17) , or Sampling-Importance-Resampling (SIR) techniques.

Original:  Related Work
Revised:  Related Work

Original: Latent variable deep generative models: Our proposed method aims to improve the performance of latent variable deep generative models.
Revised: Latent variable deep generative models: Our proposed method aims to improve the performance of latent variable deep generative models.

Original: Such models consist of a decoder for generation, and require an inference mechanism to infer latent variables.
Revised: Such models consist of a decoder for generation, and require an inference mechanism to infer latent variables.

Original: VAEs [23, 46] learn the decoder network by training a tractable inference network (encoder) to approximate the intractable posterior distribution of the latent variables.
Revised: VAEs [23, 46] learn the decoder network by training a tractable inference network (encoder) to approximate the intractable posterior distribution of the latent variables.

Original: Alternatively, Han et al.
Revised: Alternatively, Han et al.

Original: [13, 14], Xie et al.
Revised: [13, 14], Xie et al.

Original: [52], Nijkamp et al.
Revised: [52], Nijkamp et al.

Original: [33] infer the latent variables by Langevin sampling from the posterior distribution without using a encoder.
Revised: [33] infer the latent variables by Langevin sampling from the posterior distribution without using a encoder.

Original: Our method follows the latter approach that uses Langevin sampling to infer latent variables.
Revised: Our method follows the latter approach that uses Langevin sampling to infer latent variables.

Original: Discriminative contrastive estimation for learning generative models: The efforts has been made to combine the discriminative and generative models [27, 20, 47], particularly, as introduced in
Revised: Discriminative contrastive estimation for learning generative models: The efforts has been made to combine the discriminative and generative models [27, 20, 47], particularly, as introduced in

Original: Section 2.2, discriminative contrastive estimation can be applied to learning EBMs.
Revised: Section 2.2, discriminative contrastive estimation can be applied to learning EBMs.

Original: Gao et al.
Revised: Gao et al.

Original: [7] use a normalizing flow [38] as the base distribution for contrastive estimation.
Revised: [7] use a normalizing flow [38] as the base distribution for contrastive estimation.

Original: Aneja et al.
Revised: Aneja et al.

Original: [1] refine the prior distribution of a pre-trained VAE by noise contrastive estimation.
Revised: [1] refine the prior distribution of a pre-trained VAE by noise contrastive estimation.

Original: However, such a method may fail if the empirical latent distribution (called aggregated posterior) is far away from the Gaussian noise.
Revised: However, such a method may fail if the empirical latent distribution (called aggregated posterior) is far away from the Gaussian noise.

Original: Rhodes et al.
Revised: Rhodes et al.

Original: [42] propose telescoping density-ratio estimation, which breaks the estimation into several sub-problems.
Revised: [42] propose telescoping density-ratio estimation, which breaks the estimation into several sub-problems.

Original: The method is connected to a range of methods leverage sequences of intermediate distributions such as [8, 29, 24].
Revised: The method is connected to a range of methods leverage sequences of intermediate distributions such as [8, 29, 24].

Original: Generator model with flexible prior: Our method trains an energy-based prior on the latent space by proposed adaptive multi-stage NCE, so our work is related to the broader line of previous papers on introducing flexible prior distribution.
Revised: Generator model with flexible prior: Our method trains an energy-based prior on the latent space by proposed adaptive multi-stage NCE, so our work is related to the broader line of previous papers on introducing flexible prior distribution.

Original: Tomczak and Welling [45] parameterized the prior based on the posterior inference model, and [2] proposed to construct priors using rejection sampling.
Revised: Tomczak and Welling [45] parameterized the prior based on the posterior inference model, and [2] proposed to construct priors using rejection sampling.

Original: Some previous work adopt a two-stage approach, which first trains a latent variable model with simple prior, and then trains a separate prior model to match the aggregated posterior distribution.
Revised: Some previous work adopt a two-stage approach, which first trains a latent variable model with simple prior, and then trains a separate prior model to match the aggregated posterior distribution.

Original: For example, 2s-VAE [4] trains another VAE in the latent space; Ghosh et al.
Revised: For example, 2s-VAE [4] trains another VAE in the latent space; Ghosh et al.

Original: [9] fit a Gaussian mixture model on latent codes.
Revised: [9] fit a Gaussian mixture model on latent codes.

Original: Additional work in this line include [34, 5, 48, 50, 39, 53, 54].
Revised: Additional work in this line include [34, 5, 48, 50, 39, 53, 54].

Original: Pang et al.
Revised: Pang et al.

Original: [36] have the closest connection to our work.
Revised: [36] have the closest connection to our work.

Original: Similar to us, they introduce an EBM on the latent space.
Revised: Similar to us, they introduce an EBM on the latent space.

Original: Both the latent space EBM and the generator network are learned jointly by maximum likelihood, and in particular the training involves short-run MCMC sampling from both the prior and posterior distributions.
Revised: Both the latent space EBM and the generator network are learned jointly by maximum likelihood, and in particular the training involves short-run MCMC sampling from both the prior and posterior distributions.

Original: In contrast, we sequentially learn a more expressive EBM with our novel adaptive multi-stage NCE, which avoids running MCMC for EBM prior.
Revised: In contrast, we sequentially learn a more expressive EBM with our novel adaptive multi-stage NCE, which avoids running MCMC for EBM prior.

Original: We also show improved results on image generation and outlier detection tasks.
Revised: We also show improved results on image generation and outlier detection tasks.

Original:  Experiments
Revised:  Experiments

Original: In this section, we present a set of experiments which highlight the effectiveness of our proposed method.
Revised: In this section, we present a set of experiments which highlight the effectiveness of our proposed method.

Original: We want to show that our method can (i) learn an generator model with expressive prior distribution from which visually realistic images can be synthesized, (ii) generalize well by faithfully reconstructing test images during training, and (iii) successfully perform anomaly detection.
Revised: We want to show that our method can (i) learn an generator model with expressive prior distribution from which visually realistic images can be synthesized, (ii) generalize well by faithfully reconstructing test images during training, and (iii) successfully perform anomaly detection.

Original: To show the performance of our method, we mainly include SVHN [31], CelebA [28] and CIFAR-10 [25] in our study.
Revised: To show the performance of our method, we mainly include SVHN [31], CelebA [28] and CIFAR-10 [25] in our study.

Original: Besides, we also include studies on the training dynamics and the Langevin sampling, as well as ablation studies to better understand our method.
Revised: Besides, we also include studies on the training dynamics and the Langevin sampling, as well as ablation studies to better understand our method.

Original: Details about the experiments, including network architecture, the choices of the model hyper-parameters and the optimization method for each dataset can be found in Appendix A.
Revised: Details about the experiments, including network architecture, the choices of the model hyper-parameters and the optimization method for each dataset can be found in Appendix A.

Original: 5.1 Image Synthesis and Reconstruction
Revised: 5.1 Image Synthesis and Reconstruction

Original: We evaluate the quality of the generated and reconstructed images.
Revised: We evaluate the quality of the generated and reconstructed images.

Original: Ideally, if the model is well-trained, the EBM prior on latent space will fit the marginal distribution of latent variables, which in turn leads to realistic samples and faithful reconstructions.
Revised: Ideally, if the model is well-trained, the EBM prior on latent space will fit the marginal distribution of latent variables, which in turn leads to realistic samples and faithful reconstructions.

Original: We benchmark our model against a variety
Revised: We benchmark our model against a variety

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: of previous methods including VAE [23], Alternating Back-propogation (ABP) [13] and Short-run
Revised: of previous methods including VAE [23], Alternating Back-propogation (ABP) [13] and Short-run

Original: Inference (SRI)
Revised: Inference (SRI)

Original: [33] which assume a simple standard Gaussian prior distribution for the latent vector, as well as recent two-stage methods such as 2-stage VAE [4], RAE [9] and NCP-VAE [1], whose prior distributions are learned with posterior samples in a second stage after the generator is trained.
Revised: [33] which assume a simple standard Gaussian prior distribution for the latent vector, as well as recent two-stage methods such as 2-stage VAE [4], RAE [9] and NCP-VAE [1], whose prior distributions are learned with posterior samples in a second stage after the generator is trained.

Original: We also compare our method with LEBM [36], which learns a EBM prior adaptively during training the generator, while the EBM prior is trained by maximum likelihood instead of density ratio estimation.
Revised: We also compare our method with LEBM [36], which learns a EBM prior adaptively during training the generator, while the EBM prior is trained by maximum likelihood instead of density ratio estimation.

Original: To make fair comparisons, we follow the protocol as in [36].
Revised: To make fair comparisons, we follow the protocol as in [36].

Original: Synthesis: We report the quantitative results of FID [16] in Table 1, where we observe that across all datasets, our proposed method achieves superior generation performance compared to baseline models based with simple or learned prior distribution.
Revised: Synthesis: We report the quantitative results of FID [16] in Table 1, where we observe that across all datasets, our proposed method achieves superior generation performance compared to baseline models based with simple or learned prior distribution.

Original: We show qualitative results of generated samples in Figure 2, where we observe that our model can generate diverse, sharp and high-quality samples.
Revised: We show qualitative results of generated samples in Figure 2, where we observe that our model can generate diverse, sharp and high-quality samples.

Original: Additional qualitative samples are presented in
Revised: Additional qualitative samples are presented in

Original: Appendix C. To test our method’s scalability, we trained a larger generator on CelebA-HQ ( 128 × 128 ) and show samples in Appendix C, the model can produce realistic samples.
Revised: Appendix C. To test our method’s scalability, we trained a larger generator on CelebA-HQ ( 128 × 128 ) and show samples in Appendix C, the model can produce realistic samples.

Original: Reconstruction: Note that the posterior Langevin dynamics should not only help to learn the latent space EBM prior model but also produce samples that approximately come from true posterior distribution p θ ( z | x ) of the generator model.
Revised: Reconstruction: Note that the posterior Langevin dynamics should not only help to learn the latent space EBM prior model but also produce samples that approximately come from true posterior distribution p θ ( z | x ) of the generator model.

Original: To verify this, we evaluate the accuracy of the posterior inference by looking at reconstruction error on test images.
Revised: To verify this, we evaluate the accuracy of the posterior inference by looking at reconstruction error on test images.

Original: We quantitatively compare reconstructions of test images with baseline models using mean square error (MSE) in Table 1.
Revised: We quantitatively compare reconstructions of test images with baseline models using mean square error (MSE) in Table 1.

Original: We observe that our method consistently obtain lower reconstruction error than competing methods do.
Revised: We observe that our method consistently obtain lower reconstruction error than competing methods do.

Original: We also provide qualitative results of reconstruction in Appendix B.
Revised: We also provide qualitative results of reconstruction in Appendix B.

Original: 5.2 Anomaly Detection
Revised: 5.2 Anomaly Detection

Original: Anomaly detection is another task to evaluate the generator model.
Revised: Anomaly detection is another task to evaluate the generator model.

Original: With a generator and an EBM prior model trained on in-distribution data, the posterior p θ ( z | x ) would have separated probability
Revised: With a generator and an EBM prior model trained on in-distribution data, the posterior p θ ( z | x ) would have separated probability

Original: [Table]
Revised: [Table]

Original: densities for in-distribution and out-of-distribution (anomalous) samples.
Revised: densities for in-distribution and out-of-distribution (anomalous) samples.

Original: In particular, we decide whether a test sample x is anomalous or not by first sampling z from the posterior p θ ( z | x ) by short-run Langevin dynamics, and then computing the joint density p θ,ϕ ( x , z )
Revised: In particular, we decide whether a test sample x is anomalous or not by first sampling z from the posterior p θ ( z | x ) by short-run Langevin dynamics, and then computing the joint density p θ,ϕ ( x , z )

Original: =
Revised: =

Original: p θ ( x | z ) p ϕ
Revised: p θ ( x | z ) p ϕ

Original: ( z ) .
Revised: ( z ) .

Original: A higher value of log joint density indicates the test sample is more likely to be a normal sample.
Revised: A higher value of log joint density indicates the test sample is more likely to be a normal sample.

Original: Some prior work on using latent variable generative model for anomaly detection includes [49, 15, 36].
Revised: Some prior work on using latent variable generative model for anomaly detection includes [49, 15, 36].

Original: Following the experimental settings in [26, 55]
Revised: Following the experimental settings in [26, 55]

Original: [Table]
Revised: [Table]

Original: we set each class in the MNIST dataset as an anomalous class and leave the other 9 classes as normal.
Revised: we set each class in the MNIST dataset as an anomalous class and leave the other 9 classes as normal.

Original: Note that it is a challenging task and all previous methods do not perform well.
Revised: Note that it is a challenging task and all previous methods do not perform well.

Original: To evaluate the performance, we use the log-posterior density to compute the area under the precision-recall curve (AUPRC) [6].
Revised: To evaluate the performance, we use the log-posterior density to compute the area under the precision-recall curve (AUPRC) [6].

Original: We compare our method with related models in Table 2, where we observe that our method obtains significant improvements.
Revised: We compare our method with related models in Table 2, where we observe that our method obtains significant improvements.

Original: 5.3 Analyzing Training Loss
Revised: 5.3 Analyzing Training Loss

Original: In Figure 4, we plot the evolution of the density ratio estimation loss (Eq. 4) for each stage of estimation during training.
Revised: In Figure 4, we plot the evolution of the density ratio estimation loss (Eq. 4) for each stage of estimation during training.

Original: Our experiment has 4 estimation stages, resulted in 4 density ratio estimators.
Revised: Our experiment has 4 estimation stages, resulted in 4 density ratio estimators.

Original: We observe that the loss for the first stage, which estimates the density ratio between unit
Revised: We observe that the loss for the first stage, which estimates the density ratio between unit

Original: Gaussian prior p 0 ( z ) and aggregated posterior is significantly lower than later stages, which estimate the ratio between the updated prior and updated posterior.
Revised: Gaussian prior p 0 ( z ) and aggregated posterior is significantly lower than later stages, which estimate the ratio between the updated prior and updated posterior.

Original: This observation is consistent with our intuition: directly discriminating between Gaussian prior and posterior is very easy, while introducing additional stages of estimation make the task more difficult, and hence the estimated density ratio is more reliable.
Revised: This observation is consistent with our intuition: directly discriminating between Gaussian prior and posterior is very easy, while introducing additional stages of estimation make the task more difficult, and hence the estimated density ratio is more reliable.

Original: Higher losses in later stages also suggests that the prior is getting close to aggregated posterior, as the discrimination becomes harder.
Revised: Higher losses in later stages also suggests that the prior is getting close to aggregated posterior, as the discrimination becomes harder.

Original: 5.4 Analyzing Langevin Dynamics
Revised: 5.4 Analyzing Langevin Dynamics

Original: In Figure 3, we visualize the transition of Langevin dynamics initialized from p 0 ( z ) towards p ϕ
Revised: In Figure 3, we visualize the transition of Langevin dynamics initialized from p 0 ( z ) towards p ϕ

Original: ( z ) on a model trained on CelebA. The LD iterates for 200 steps, which is longer than the LD for training ( 30 steps).
Revised: ( z ) on a model trained on CelebA. The LD iterates for 200 steps, which is longer than the LD for training ( 30 steps).

Original: We expect that with a well-trained p ϕ ( z ) , the trajectory of a Markov chain should transit towards samples of higher quality.
Revised: We expect that with a well-trained p ϕ ( z ) , the trajectory of a Markov chain should transit towards samples of higher quality.

Original: Indeed, we observe that the quality of synthesis improves significantly with as the LD progresses.
Revised: Indeed, we observe that the quality of synthesis improves significantly with as the LD progresses.

Original: In addition, we observe human faces with different
Revised: In addition, we observe human faces with different

Original: [Table]
Revised: [Table]

Original: identities along the LD, suggesting that the Markov chain can mix between different modes of the prior distribution.
Revised: identities along the LD, suggesting that the Markov chain can mix between different modes of the prior distribution.

Original: This indicates that the density function of learned EBM prior has a smooth geometry that allows MCMC to mix well.
Revised: This indicates that the density function of learned EBM prior has a smooth geometry that allows MCMC to mix well.

Original: 5.5 Ablation Study
Revised: 5.5 Ablation Study

Original: To better understand our proposed method, we conduct ablation study on number of density ratio estimators and training methods.
Revised: To better understand our proposed method, we conduct ablation study on number of density ratio estimators and training methods.

Original: We use CelebA for the ablation experiments.
Revised: We use CelebA for the ablation experiments.

Original: Number of stages.
Revised: Number of stages.

Original: The most important hyper-parameter of our method is the number of density ratio estimators, or equivalently, the number of training stages.
Revised: The most important hyper-parameter of our method is the number of density ratio estimators, or equivalently, the number of training stages.

Original: We present the FID score of models trained with different number of stages in the first part of Table 3.
Revised: We present the FID score of models trained with different number of stages in the first part of Table 3.

Original: The line of 0 stage means no latent
Revised: The line of 0 stage means no latent

Original: EBM at all, i.e., simply training a generator model by short-run inference and sampling from it by decoding z ∼ p 0 ( z ) .
Revised: EBM at all, i.e., simply training a generator model by short-run inference and sampling from it by decoding z ∼ p 0 ( z ) .

Original: We make the following observations.
Revised: We make the following observations.

Original: Firstly, directly sampling latent variables from p 0 leads to poor results, while any latent EBM trained by density ratio estimation significantly improves the performance, suggesting the necessity of learning the latent EBM.
Revised: Firstly, directly sampling latent variables from p 0 leads to poor results, while any latent EBM trained by density ratio estimation significantly improves the performance, suggesting the necessity of learning the latent EBM.

Original: Secondly, we see that multi-stage density ratio estimation can further significantly improve the performance of single-stage estimation.
Revised: Secondly, we see that multi-stage density ratio estimation can further significantly improve the performance of single-stage estimation.

Original: The results indicate that multi-stage density ratio estimation facilitates the training of latent EBM by gradually making the estimation task harder.
Revised: The results indicate that multi-stage density ratio estimation facilitates the training of latent EBM by gradually making the estimation task harder.

Original: We observe that the FID score does not improve for more than 4 stages, and therefore we choose 4 as the number of stages for our main experiments.
Revised: We observe that the FID score does not improve for more than 4 stages, and therefore we choose 4 as the number of stages for our main experiments.

Original: Training method: adaptive vs. non-adaptive.
Revised: Training method: adaptive vs. non-adaptive.

Original: It is important to distinguish our method from TRE in Rhodes et al.
Revised: It is important to distinguish our method from TRE in Rhodes et al.

Original: TRE assumes the target distribution to be fixed, therefore, if we adopt TRE, the posterior distribution p θ ( z | x ) will be a fixed one throughout the training.
Revised: TRE assumes the target distribution to be fixed, therefore, if we adopt TRE, the posterior distribution p θ ( z | x ) will be a fixed one throughout the training.

Original: In contrast, our training method is adaptive in the sense that the target posterior is updated by incorporating the current EBM prior into the joint distribution when a new stage is introduced.
Revised: In contrast, our training method is adaptive in the sense that the target posterior is updated by incorporating the current EBM prior into the joint distribution when a new stage is introduced.

Original: To quantitatively compare these two approaches, we also train non-adaptive version of the model and report the numbers in the second part of Table 3.
Revised: To quantitatively compare these two approaches, we also train non-adaptive version of the model and report the numbers in the second part of Table 3.

Original: We observe that models trained with non-adaptive multi-stage density ratio estimation obtain significantly worse results.
Revised: We observe that models trained with non-adaptive multi-stage density ratio estimation obtain significantly worse results.

Original: Therefore, we believe that it is crucial to learn density ratios sequentially with adaptive posterior.
Revised: Therefore, we believe that it is crucial to learn density ratios sequentially with adaptive posterior.

Original: 5.6 Parameter Efficiency
Revised: 5.6 Parameter Efficiency

Original: One potential disadvantage of our method is its parameter inefficiency from multiple estimator networks.
Revised: One potential disadvantage of our method is its parameter inefficiency from multiple estimator networks.

Original: Moreover, since the training is sequential, we cannot share parameters between estimators as done in Rhodes et al.
Revised: Moreover, since the training is sequential, we cannot share parameters between estimators as done in Rhodes et al.

Original: Fortunately, our EBM is on the latent space so the network is lightweighted.
Revised: Fortunately, our EBM is on the latent space so the network is lightweighted.

Original: For example, with 4 density ratio estimators, the number of parameters in the prior EBM is only around 1% of the number of parameters in the generator.
Revised: For example, with 4 density ratio estimators, the number of parameters in the prior EBM is only around 1% of the number of parameters in the generator.

Original: In addition, we confirm the larger number of parameters in the latent EBM is not the cause of improvements, as we train a single stage model with 4 × size and observe no improvement.
Revised: In addition, we confirm the larger number of parameters in the latent EBM is not the cause of improvements, as we train a single stage model with 4 × size and observe no improvement.

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we propose adaptive multi-stage density ratio estimation, which is an effective method for learning a EBM prior for a generator model.
Revised: In this paper, we propose adaptive multi-stage density ratio estimation, which is an effective method for learning a EBM prior for a generator model.

Original: Our method learns the latent EBMs by introducing multiple density ratio estimators that learn the density ratio between prior and posterior sequentially and adaptively.
Revised: Our method learns the latent EBMs by introducing multiple density ratio estimators that learn the density ratio between prior and posterior sequentially and adaptively.

Original: We demonstrate the effectiveness of our method by conducting comprehensive experiments, and empirical results show the advantage of our method on generation, reconstruction and anomaly detection tasks.
Revised: We demonstrate the effectiveness of our method by conducting comprehensive experiments, and empirical results show the advantage of our method on generation, reconstruction and anomaly detection tasks.

Original: As future directions, our method can potentially be applied to modeling the latent space of generator models in other domains, such as text [35] and graph.
Revised: As future directions, our method can potentially be applied to modeling the latent space of generator models in other domains, such as text [35] and graph.

Original: We also tend to develop more advanced and efficient inference schemes for posterior density estimation.
Revised: We also tend to develop more advanced and efficient inference schemes for posterior density estimation.

Original: A Experimental Details

Original: In this section, we introduce the detailed settings of our experiments.

Original: A.1 Datasets

Original: We mainly study our method with SVHN [31] (32 × 32 × 3) , CIFAR-10 [25] (32 × 32 × 3) , and

Original: CelebA [28] (64 × 64 × 3) .

Original: Following Pang et al.

Original: [36], we use the full training set of SVHN ( 73 , 257 ) and CIFAR-10 ( 50 , 000 ), and take 40 , 000 examples of CelebA as training data following [32].

Original: The training images are resized and scaled to [ − 1 , 1] .

Original: A.2 Network architectures.

Original: For experiments on SVHN, CelebA and CIFAR-10, each density ratio estimator network has a simple fully-connected structure described in Table 4.

Original: [Table]

Original: We let the generator network having a simple deconvolution structure, similar to DCGAN [40].

Original: The generator network for each dataset is depicted in Table 5.

Original: [Table]

Original: A.3 Training Hyper-parameters

Original: We introduce some hyper-parameter setting for training our model.

Original: For the main experiments, we have 4 density ratio estimation stages.

Original: We adopt the persistent approach for generating samples from prior distribution.

Original: For the posterior sampling Langevin dynamics, we use step size 0 .

Original: [Table]

Original: and run the

Original: LD for 30 steps for SVHN and CelebA

Original: [Table]

Original: and 40 steps on CIFAR-

Original: [Table]

Original: The parameters for the density ratio estimators and image generators are initialized with Xavier initialization [10].

Original: We train both the generator and density ratio estimators using Adam [22] optimizer.

Original: The learning rate for the generator is 1 e − 4 and the learning rate for the density ratio estimator is

Original: 5 e − 5 .

Original: We train the model for 100 epochs for SVHN and CelebA, where a new estimation stage is introduced every 25 epochs.

Original: For CIFAR-10, we train the model for 200 epochs for SVHN and

Original: CelebA, where a new estimation stage is introduced every 50 epochs.

Original: During test stage, we run LD on the learned EBM prior with step size 0 .

Original: 1 for 100 steps.

Original: B Reconstruction Samples

Original: In Figure 5, we provide some qualitative examples of reconstructing test images.

Original: We see that our model can reconstruct unseen images faithfully.

Original: C Additional Qualitative Results

Original: We trained our model on high-resolution 128 × 128 CelebA-HQ, samples are shown in Figure 6 We also provide additional qualitative samples from our models trained on SVHN, CelebA and CIFARin Figure 7.