Langevin Autoencoders for Learning Deep Latent Variable Models
Authors: Shohei Taniguchi
Yusuke Iwasawa
Wataru Kumagai
Yutaka Matsuo
Venue: NeurIPS 2022 Accept (2022)
Keywords: Langevin dynamics
deep latent variable models
amortized inference

Abstract: Markov chain Monte Carlo (MCMC), such as Langevin dynamics, is valid for approximating intractable distributions. However, its usage is limited in the context of deep latent variable models owing to costly datapoint-wise sampling iterations and slow convergence. This paper proposes the amortized Langevin dynamics (ALD), wherein datapoint-wise MCMC iterations are entirely replaced with updates of an encoder that maps observations into latent variables. This amortization enables efficient posterior sampling without datapoint-wise iterations. Despite its efficiency, we prove that ALD is valid as an MCMC algorithm, whose Markov chain has the target posterior as a stationary distribution under mild assumptions. Based on the ALD, we also present a new deep latent variable model named the Langevin autoencoder (LAE). Interestingly, the LAE can be implemented by slightly modifying the traditional autoencoder. Using multiple synthetic datasets, we first validate that ALD can properly obtain samples from target posteriors. We also evaluate the LAE on the image generation task, and show that our LAE can outperform existing methods based on variational inference, such as the variational autoencoder, and other MCMC-based methods in terms of the test likelihood.

Langevin Autoencoders for Learning Deep Latent Variable Models
Shohei Taniguchi
Yusuke Iwasawa
The University of Tokyo
The University of Tokyo
suniguchi@weblab u-tokookoacc ac. jp
iwasawa@reblab t...aaa...uut.. u-tokyo. ac.
Wataru Kumagai
Yutaka Matsuo
The University of Tokyo
The University of Tokyo
kumagai @weblab. uutokkoo.ase ac ji.
matsuo@weblab. u-utooyo. jc.
Abstract
Markov chain Monte Carlo (MCMC), such as Langevin dynamics, is valid for approximating intractable distributions. However, its usage is limited in the context of deep latent variable models owing to costly datapoint-wise sampling iterations and slow convergence. This paper proposes the amortized Langevin dynamics (ALD), wherein datapoint-wise MCMC iterations are entirely replaced with updates of encoder that maps observations into latent variables. This amortization enables efficient posterior sampling without datapoint-wise iterations. Despite its efficiency, prove that ALD is valid as an MCMC algorithm, whose Markov chain has the target posterior as a stationary distribution under mild assumptions. Based on the ALD, also present the Langevin autoencoder (LAE). Interestingly, the LAE can implemented by slightly modifying the traditional autoencoder. Using multiple synthetic datasets, we first validate that ALD can properly obtain samples from target posteriors. We also evaluate the LAE the image generation task, and show that our LAE outperform existing methods based on variational inference, such as the variational autoencoder, and other MCMC-based methods in terms of the test likelihood.
deep latent variable model named
Introduction
Variational inference (VI) and Markov chain Monte Carlo (MCMC) are two practical tools to approximate intractable distributions. Recently, VI has been dominantly used in deep latent variable models (DLVMs) to approximate the posterior distribution over the latent variable z given the observation x, i.e., p x). At the core of the success of VI is the invention of amortized variational inference (AVI) Kingma and Welling 2013 Rezende et al. 2014). which replaces optimization of datapoint-wis wise variational parameters with an encoder that predicts latent variables from observations.
The framework of learning DLVMs based on AVI is called the variational autoencoder (VAE), which
widely used in applications Kingma al. 2014 An and Cho 2015 Zhang et al., 2016 Eslami et al., 2018 Kumar et al.) 2018). An important advantage of AVI over traditional VI is that the optimized encoder can be used to infer latent representation even for new data in test time, expecting its generalization ability. On the other hand, the approximation power of AVI (or VI itself) is limited because it relies on tractable distributions for complex posterior approximations of DLVMs as shown in Figure ] (a). Although there have been attempts to improve their flexibility (e.g.. normalizing flows [Rezende and Mohamed] (2055] Kingma et al. 2016 Van Den Berg et al. 2018 Huang et al.)
2018 such methods typically have architectural constraints (e.g., invertibility in normalizing flows).
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

eidea points
dir points
Encodor
Encoder
Poeterfor decalty
Postarior deralty
x
vetaileed enceder model
Update /n
MONO over LaSenE
(a) Variational Inference (b) Langevin Dynamics
(c) Hoffman [217
(d) ALD (ours)
Figure 1: Comparison between existing approximated inference methods and our amortized Langevin dynamics (ALD). (a) In variational inference, posteriors are approximated using tractable distributions
(e.g., Gaussians). (b) In traditional Langevin dynamics (LD), the approximation performed by running gradient-based sampling iterations directly the latent space for each datapoint. (c) Hoffman [017)] uses an encoder that maps the observation into the latent variable to initialize traditional LD, but still relies on datapoint-wise iterations. (d) Our ALD also uses an encoder, but it treats the output of the encoder as a posterior sample, and its update is implicitly performed by updating the encoder.
Compared VI, MCMC (c.g., Langevin dynamics) can approximate complex distributions by repeating sampling from a Markov chain that has the target distribution as its stationary distribution [Liu and Liu 2001 Robert et al. 2004 Gilks et al. 1995 Geyer 1992]. However, despite its high approximation ability, MCMC has received relatively little attention in learning DLVMs. is because MCMC methods take a long time converge, making difficult to be used in the training of DLVMs. When learning DLVMs with MCMC, we need to run tine-consuming MCMC iterations for sampling from each posterior per data point, i.e., p (z x))) (i = 1,.... ,,), where n is the number of mini-batch data, shown in Figure (b). Furthermore, need to re-run the sampling procedure when we obtain new data at test time.
As in VI, there have been some attempts to introduce the concept of amortized inference to MCMC. For example, Hoffman (2017) initializes datapoint- wise MCMC using an encoder that predicts latent variables from observations as shown in Figure (c). However, as they use encoders only for the initialization MCMC, these methods still rely on datapoint-wise sampling iterations. Not only is time-consuming, but implementations of such partially amortized methods also tend to be complicated compared the simplicity of AVI. To make MCMC more suitable for the inference of DLVMs, a more sophisticated framework of amortization is needed.
This paper proposes amortized Langevin dynamics (ALD), which entirely replaces datapoint- MCMC iterations with updates of an encoder that maps the observation into the latent variable as shown in Figure (d). Since the latent variable depends on the encoder, the updates of the encoder can be regarded as implicit updates of latent variables. By replacing MCMC on the latent space with one on the encoder's parameter space, we can benefit from the encoder's generalization ability. For example, after running a Markov chain of the encoder for some mini-batch data, it is expected that the encoder can map data into high density area in the latent space; hence we can accelerate the convergence of MCMC for other data by initializing it with the encoder. Moreover, despite the simplicity, we can thcoretically guarantee that ALD has the true posterior as a stationary distribution under mild assumptions, which critical requirement for valid MCMC algorithms.
Using our ALD for sampling from the latent posterior, we derive a novel framework of learning DLVMs, which we refer to as the Langevin autoencoder (LAE). Interestingly, the learning algorithm of LAES can be regarded as a small modification of traditional autoencoders (Hinton and Salakhut dinov [006]] In our experiments, we first show that ALD can properly obtain samples from target distributions using toy datasets. Subsequently, we perform numerical experiments of the image generation task using the MNIST, SVHN, CIFAR-10, and CelebA datasets. We demonstrate that our LAE can outperform existing learning methods based on variational inference, such as the VAE, and existing MCMC-bar based methods in terms of test likelihood.

---

2 Preliminaries
2.1 Problem Definition
Consider a prohabilistic model with the observation x, the continuous latent variable z, and the model parameter 0 as follows:
p (x: 0)
- 0)=
p z: 2;
(1)
To learn this latent variable model (LVM) via maximum likelihood in a gradient-based manner, we need calculate the expectation over the posterior distribution p (z x; 0) follows:
V (x) [logp (a;; 0)]
Σ E (((00))00))))) Vo log (z) = =0000
(2)
where Pdata is the empirical distribution define by the training set, and x (n) are mini-batch data uniformly drawn from Pdata- However, this expectation cannot be calculated in a closed-form because the posterior is intractable. In this paper, we consider to use Monte Carlo approximation by obtaining samples from the posterior per data point.
2.2 Langevin Dynamies
Langevin dynamics (L.D) Neal 2011 is a sampling algorithm based on the following Langevin equation:
dz V-U (æe, z; 0) + V 2B--IdB,
where U is a Lipschitz continuous potential function that satisfies an appropriate growth condition, B an inverse temperature parameter, and B is Brownian motion. This stochastic differential equation (SDE) has pd (z x; 0) x exp (-3U (xz 2; O)) as its stationary distribution. We set 3 = 1 and define the potential as follows to obtain the target posterior p (z x; 0) its stationary distribution:
U (z, 2; 0) log p (ax 2; 0)
We can obtain samples from the posterior by simulating the SDE of Eq. using the Kuler-Maruyann method [Kloddnn and Platen 2013 as follows:
zt-1 (Z1+1 x1),
(5)
( N vV (x, 2m
(6)
where is a step size for discretization. The initial value 2u is typically sampled from the prior (z). When the step size is sufficiently small, the samples asymptotically move to the target posterior by repeating this sampling iteration. To remove the discretization eπror. an additional Metropolis- Hastings (MH) rejection step often used. In MH step, we first calculate the acceptance rate ag as follows:
0₂ min 1, exp (-U (ar, Z 0)) (z, 2441)
c-p (x, z2; 0)) (zt+1 2x)
The sample 2 is accepted with probability at, and rejected with probability 1 at If the sample is rejected, we set z4+1 latent variables, provided the potential energy is differentiable on the latent space. To obtain the posterior samples for all mini- batch data ((1), (.....")) we should perform iterations of Eq. per data point, as shown in F
LD can be applied to any posterior inference problems for continuous
After obtaining the samples, the gradient in Eq. (2) is approximated using the samples follows:
n
T
Vo Fo (log (ax; 0)
Vo log p (1) 22 ():
R
The time averaged gradient in Eq. (8) is sometimes substituted for the one calculated with the final sample as follows:
V Te (xpaaa (x) (x:
Σ V logp (())) 200002200000 :)
(9)
im1

---

Algorithm Amortized Langevin dynamics
Φ Initialize parameters
2()
((n) +-2 2
> Initialize sample sets for all n datapoints.
repeat
(Φ ()) = N (Ø': - Ei= VOU (at 2010 = Saix (2:(1) 2n
Jo ( (b)
t ø' with probability min
MH rejection step.
2 (()) 2 U {faxx
(z) 2 (N) [2ix ;
Add samples.
until convergence of parameters return ( (n)
However, in these traditional approaches, we need to run MCMC iterations from random initialization every time after updating the model parameter 0. In addition, we also need to run it from stratch to perform inference for new data in test time. This inefficiency hinders the practical use of MCMC for learning DLVMs. A naive approach to mitigate this problem is to initialize MCMC with persistent sample [Tieleman 2008 Han al. 2017] that is the final value of the previous Markov chain. However, this approach is also inefficient especially when the training set is large, because we need to store persistent samples for all training examples.
To alleviate the inefficiency of LD for LVMs, Hotiman 2017 has proposed to use a stochastic encoder (z x) initialize the datapoint wise MCMC shown in Figure (c). The encoder is typically defined as a Gaussian distribution as in the VAE:
(10) where μ and o² are mappings from the observation space into the latent space. In the training, LD iterations of Eq. (5) are initialized using sample from the distribution of Eq. (TO), and the model parameter is updated using the stochastic gradient in Eq. (9.. Along with it, the encoder is trained via
q (= x; Φ) = (z; (æ; Φ) diag (o² (æ;; +))),
maximization of the evidence lower bound in VAEs:
p (2c, (xx,2;0)
C ( Egtz(x)p(x) log
(11)
7 (z x; Φ)
Although initializing LD with the encoder can speed up the convergence, this method still relies on datapoint-wise MCMC iterations. Moreover, the encoder has to be trained using the different objective, which also makes its implementation complicated. In Section demonstrate method that entirely remove the datapoint-wi iterations by amortization.
Method
3.1 Amortized Langevin Dynamics
As an alternative to the direct simulation latent dynamics of Eq. (3. we define a deterministic encoder S2/x, which maps the observation into the latent variable, and consider an SDE over its parameter Φ as follows:
dΦ = = -VoV
(0) dt V2dB
(12)
n
V (Φ)) = Σ (2(()) (1) Falx (2( Φ :)))))))
(13)
Because the function fa/x outputs the latent variable, the stochastic dynamics on the parameter space induces dynamics on the latent space. The main idea of our amortized Langevin dynamics (ALD) is to regard the transition on this induced dynamics a sampling procedure for the posterior distributions, as shown in Figure| (d). We can use the - method to simulate Eq. I2 with discretization:
Φis1 (#++1 Φt), (Φ) Φ) = N vV (Φ) Tn
(14)
(15)

---

As in the traditional LD, the discretization error can be removed by adding a MH rejection step, calculating the acceptance rate as follows:
ae min {
1, exp (-V (#++1)) Φz+1)
(16)
exp (-V (Φr)) q (0++1 Φ+)
Through the iterations, the posterior sampling is implicitly performed by collecting outputs of the encoder for each data point as described in Algorithm 1 Note that (4) denotes a set of samples of the posterior for the i-th data (i.e., (z x())) obtained using ALD.
If this implicit update iteration has the posterior as its stationary distribution, this sampling procedure valid as an MCMC algorithm. To obtain the stationary distribution over the latent variables, we first derive the stationary distribution over the parameter & of Eq. (12). then transform it into the latent space by considering the change of random variable by the encoder faxx- Based on this strategy, derive the following theorem:
Theorem 1. Let q (Z X) = I 9 (z(0) x0)) be stationary distribution over the latent
variables that is induced by Eq. (12 When the mapping falx meets the following conditions, the
stationary distribution satisfies q Z X) exp (- -im U (x!(),2(())
1. The mapping has the form of feix (x;; 3) &g (az), where T is de x d matrix, 9 is a mapping from Rd Rd, and dx dg and are the dimensionalities of x. z and g (a)) respectively.
2. The rank of G is where G is matrix with (z(1) in row G..-
See the appendix for the solid proof. Theorem [ suggests that samples obtained by ALD asymp- totically converge to the true posterior when we construct the encoder faix with an appropriate form. Practically, we can implement such a function using a neural network whose parameters are fixed except for the last linear layer. In this implementation, the last linear layer takes role of the parameter 4, and the preceding feature extractor takes a role of the function g-
To meet the second condition, the dimensionality of the last linear layer should be larger than the batch size n. relatively easy meet this condition because the batch size is about 1,000 at most in practice.
3.2 Remarks
Remark 1: ALD completely removes datapoint-wise iterations.
Because ALD treats the encoder's outputs themselves as posterior samples, we no longer need to run time-consuming iterations of datapoint-wise MCMC, whereas existing methods, such Hoffman 2017) only use the encoder for initialization of datapoint-wist MCMC.
Remark 2: ALD is valid
MCMC algorithm.
Although the sampling procedure of ALD is quite simple, ALD has the true posterior as its stationary distribution under mild assumptions, which guarantee that ALD is valid as an MCMC algorithm. Basically, we can meet the assumptions by using a sufficiently wide neural network for the encoder. In addition, it is worth mentioning that the traditional LD can be seen as special case of ALD
where g (6) one-hot (i). In that case, ( fxx (z()), Φ) corresponds to the i-th column of d, which is equivalent running MCMC on the latent space independently for each data point.
Remark 3: The encoder may accelerate the convergence of MCMC.
After running ALD iterations of the encoder for some mini-batch data, it is expected that the encoder can map data into high density area the latent space. Therefore, the encoder can accelerate the convergence of MCMC for other data by initializing it with the encoder. This characteristics is useful not only in the training of DLVMS efficiently estimate the gradient of the model parameters but also at test time to infer the latent representation for new test data.
5

---

Algorithm 2 Langevin Autoencoders
1: 0, ②. vb Initialize parameters
2: repeat
3: (11)
x
(xx)
> Sample minibatch of examples from the training data. > Run ALD iterations.
4: for
0, 0,.....
do
5:
Vi
Σ 1 logp (x
z Dg (x(1); : :0)
6:
Φ q (6 3) = N 1 : Φ 7 V. 2np
7:
V - logp (x 2() ()
D y (a) 2b)
Φ - D with probability min {
exp(
+/
8:
MH rejection step.
expl (1)
9:
end for
10:
VT
- log (x)(0, 2(() &g ; ;b;;
11:
0 - - - Vo > V
Update the decoder.
12:
+ < Φb u 7 Σ Ve
Update the encoder.
13:
until convergence of parameters
return 0 $, b
3.3 Langevin Autoencoder
Based on ALD, we propose novel framework of learning deep latent variable models called the Langevin autoencoder (LAE). Algorithm
2 is a summary of the LAE's training procedure. First, we prepare an encoder defined by faix (z: Φ, bb) == $g (z: $ The feature extractor is typically implemented using a deep neural network. Before each update of the model parameter 0, the encoder's final linear layer + is updated using ALD for T times, and the gradient of 0 in Eq. (8) is calculated using the time average of V., Along with the update of the model parameter, the encoder's feature extractor z/ also updated using the gradient so that the encoder can map data into high density area of the posteriors. Although we describe the parameter update using a simple stochastic gradient ascent, it can be substituted for more sophisticated optimization methods, such as Adam [DP and Ba 2015].
be seen that the algorithm very similar to the one for traditional deterministic autoencoders
(Hinton and Salakhutdinov 2006]. In particular, when we skip the ALD iterations in lines 4 to 9 and the encoder's final linear layer + is also trained via gradient ascent, the algorithm is identical to the training of autoencoders regularized by the latent prior p (z). In that case, the encoder tends to shrink to the maximum a posteriori (MAP) estimate rather than the posterior; hence the gradient of the model parameter 0 would be a strongly biased estimate of the marginal log-likeli logeliiel gradient. Therefore, the additional ALD iterations can interpreted as the modification reduce the bias of the traditional autoencoder by updating the encoder's 's parameters along with the noise-injected
gradient.
4 Related Works
Amortized inference is well-investigated in the context of variational inference. is often referred to as amortized variational inference (AVI) [Rezende and Mohamed, 2015 Shu et al. 2018). The
basic idea of AVI to replace the optimization of the datapoint variational parameters with the optimization of shared parameters across all datapoints by introducing an encoder that predicts latent variables from observations. The AVI is commonly used in generative models [Kingma and Welling 2013). semi-supervised learning Kingma et al. 2014]. anomaly detection [An and Cho 2015), machine translation [Zhang et al.) 2016]. and neural rendering (Eslami et al.) [ Kumar et al. 2018). However, in the MCMC literature, there are few works such amortization. Han ct al. 2017 use traditional LD to obtain samples from posteriors for the training of deep latent variable models. Such Langevin-baa based algorithms for deep latent variable models are known as alternating back-propagation (ABP) and are widely applied in several fields [Xie et al.) 2019 Zhang
et al. 2020 Xing et al. 2018
Zhu et al. 2019). However, ABP requires datapoint- Langevin

---

4
GT
d > 72
71
d < m
Figure Effects of changing the encoder's capacity in the experiment of bivariate Gaussian examples. d denotes the dimensionality of the last linear layer's input.
iterations, causing slow convergence. Moreover, when we perform inference for new data in test time, ABP requires MCMC iterations from randomly initialized samples again. Li et al.) [2017] and Hoffmann 2017] propose use VAE-like encoder to initialize MCMC, and (Salimans et al.) [2015] also proposes to combine VAE-based inference and MCMC by interpreting each MCMC step as an auxiliary variable. However, they only amortize the initialization cost in MCMC by using an encoder; hence, they still rely on datapoint-wise MCMC iterations.
Autoencoders (AEs) [Hinton and Salakhutdinov 2006) are a special case of LAEs, wherein the ALD iterations are omitted and uniform distribution is used as the latent prior p (z). When a different distribution is used the latent prior as regularization, it is known sparse autoencoders (SAEs) [gg et al. 10 As described the previous section, the encoder of the traditional AE tends to converge to MAP estimates of the latent posterior. Therefore, the gradient of the decoder's 's parameter is biased as the gradient estimate of the marginal log-likelihood. Our LAE can modify this bias by adding the ALD iterations before each parameter update, making it valid as training of generative model.
Variational Autoencoders (VAEs) are based on AVI, wherein encoder is defined variational distribution q (z x; Φ) using a neural network. Its parameter Φ is optimized by maximizing the
Interestingly, there is a contrast between VAEs and LAES relative to when stochastic noise is used in posterior inference. In VAEs, noise is used to sample from the variational distribution in calculating the potential U, i.e., in the forward calculation. However, in LAEs, noise is used for the parameter update along with the gradient calculation V,U, i.e., in the backward calculation. This contrast characterizes their two different approaches to approximate posteriors: the optimization-base approach of VAEs and the sampling- based approach of LAES. The advantage of LABS over VAEs is that LAES can flexibly approximate complex posteriors by obtaining samples, whereas VAES' approximation ability is limited by choice of variational distribution g (z x: Φ) because it requires tractable density. Although there are several considerations in the improvement of the approximation flexibility, these methods typically have architectural constraints (c.g., invertibility and ease of Jacobian calculation in normalizing flows
evidence lower bound (ELBO), i.e., By(zla:d) log exp(-U((x,=)
((=[=:)))
[Rezende and Mohamed, 2015 Kingma et al. 2016 Van Den Berg et al. 2018 Huang et al. 2018 Titsias and Ruiz [2019). or they incur more computational costs (e.g., MCMC sampling for the reverse conditional distribution unbiased implicit variational inference [Titsias and Ruiz 2019
Experiment
In our experiment, we first test our ALD algorithm on toy examples investigate its behavior, then we show the results of its application to the training of deep generative models for image datasets.
5.1
Toy Examples
We perform numerical simulation using toy examples to demonstrate that our ALD can properly obtain samples from target distributions. First, we use LVM where the posterior density can be derived closed-form. We initially generate three synthetic data æ 22 3z where each æ; is sampled from a bivariate Gaussian distribution as follows:
(z)) = N (12 Ez), ( 2) = Nz, Ex)
(17)
7

---

GT
Mcan-field VI
Full VI
ALD (ours)
Figure 3: Visualizations of a ground truth posterior (left), an approximation by VI (center), and samples by ALD (right) in the neural likelihood example.
In this case, we can calculate the exact posterior as follows:
(
xx) = N (Mz Enix)
(18)
12/x E2ix
[Σz
Uz + xx'xaa Σ/ix (Σz² + 1x
(19)
0
0
0.7 0.6
In this experiment, we set 142
Σ,
and Σx
0.8 We simulate
0
0.7
our ALD algorithm for this setting to obtain samples from the posterior. We use a neural network of three fully connected layers of 128 units with ReLU actívation for the encoder fa|xi setting the step size to 4 10 4, and update the parameters for 3,000 steps. We omit the first 1,000 samples as burn-in steps and use the remaining 2,000 samples for qualitative evaluation. As we described in Section [3.1] we need to make the last linear layer of the encoder have sufficiently large dimensions to guarantee the convergence to the true posterior. To empirically demonstrate this theoretical finding, we change the dimensionality of the last linear layer of the encoder from (< n) to 128 (>> n).
The results are summarized in Figure 2 It can be observed that the sample quality is good when the dimensionality of the last linear layer is equal to or greater than the number of data points (i.e., d 2 n). When the dimensionality is smaller than the number of data points, the samples for some data points shrink a small area, while good samples are obtained for the remaining data points.
In addition to the simple conjugate Gaussian example, we experiment with a complex posterior, wherein the likelihood is defined with randomly initialized neural network. For comparison, also implement the variational inference (VT), in which the posterior is approximated with a Gaussian distribution. Figure 3 shows typical example, which characterizes the difference between VI and ALD. The mean-filed VI and the full VI use Gaussians with diagonal and full covariance matrices for variational distributions, respectively. The advantage of our ALD over VI is the flexibility of posterior approximation. VI methods typically approximate posteriors using variational distributions, which have tractable density functions. Hence, their approximation power is limited by the choice of variational distribution family, and they often fail to approximate such complex posteriors. In particular, the mean-field VI, which is widely used for learning DLVMs, cannot capture the correlation between dimensions due to the constraint of the variational distribution. The full VI mitigate the inflexibility, but it still cannot capture the multimodality of the true posterior. Moreover, the full VI requires computational costs proportional to the square of the dimension of the latent variable. On the other hand, ALD can capture such posteriors well. The results in other examples are summarized in the appendix.
5.2 Image Generation
To demonstrate the applicability of our LAE to the generative model training, we experiment on image generation tasks using MNIST, SVHN, CIFARIO, 10, and CelebA datasets. Note that our goal here is not to provide the state-of-the-art results on image generation benchmarks but to verify the effectiveness of our ALD as a method of approximate inference in deep latent variable models. For this aim, we compare our LAE with some baseline methods, as shown in Table [ The VAE [Kingma and Welling 2013) is one of the most popular deep latent variable models, in which the posterior distribution is approximated using the VI. The VAE-flow is an extension of VAE, which the flexibility of VI is improved using normalizing flows [Rezende and Mohamed 2015). In addition to VI-based methods, we use Hoffman| [017] as a baseline method based on Langevin dynamics (LD)). As described in uses VAE-like encoder to initialize LD, and the encoder is trained by
Section 2.2 Hoffman 2017

---

Table I: Quantitative results of the image generation for MNIST, SVHN CIFAR-IO, 10, and CelebA We report the mean and standard deviation of the negative evidence lower bound per data dimension in three different seeds. Lower is better.
MNIST
SVHN
CIFAR-10
CelebA
VAE
1.189 0.002
4.442 + 0.003
4.820 + 0.005
4.671 0.001
VAE- flow
1.183 = 0.001
4
4.828 + 0.005
4.667 + 0.005
Hoffman [2017]
1.189 = 0.002
4.440
4.831 0.005
4.662 0.011
LAE (ours)
1.177 = 0.001
4.412 + 0.002
4.773 + 0.003
4.636 = 0.003
maximizing the evidence lower bound. We use the same fully connected deep neural networks for the model construction of all methods. We set the number of ALD iterations of the LAE to 2, i.e, T = 2 Please refer the appendix for more implementation details.
in Algorithm 2
For evaluation, since the marginal log-likelihood for test data is not tractable, we substitute its evidence lower bound (ELBO) for it using proposal distribution q as follows:
p (0x, 2: 0)
log (x: 0) 2 By(x))
log
(20)
q (z)
For the baseline methods, their stochastic encoders are used for the proposal distribution, i.e., q (z) = q (= æ;; Φ). For our LAE, we use a Gaussian distribution whose mean parameter is defined by its encoder, i.e., q (z) N (z; Øg (æ; +b), o² o²I). We set 0.05 in the experiment.
The results are summarized in Table ! Note that the negative ELBO is shown in the table, so lower values indicate better results. can be observed that the LAE consistently outperforms baseline methods, demonstrating that accurate posterior approximation by ALD leads to better results in the training of DLVMs. On training speed, we observe that our LAE is 2.24 and 1.88 times slower than VAE and VAE-flow respectively. This is natural because VAE and VAE-flow do not require MCMC steps. (Hofmma (017) and our LAE are almost identical in terms of training speed.
150
T- 10
No MH stops
Negetive
1:35
125
120
Epoch
Figure 4: Learning curves comparison.
We also investigate the effect of the MH rejection step and the number of ALD iterations, i.e., T in Algorithm 2 using MNIST dataset. Figure shows comparison of the learning curves of the negative ELBO for MNIST's test set. It can seen that the number of ALD iterations has little effect on performance long as the number of steps is at least 2. In addition, we observe that the MH rejection step is important to stabilize the training.
Conclusion
This paper proposed amortized Langevin dynamics (ALD), efficient MCMC method for deep latent variable models (DLVMs). ALD amortizes the cost of datapoint-wise iterations by using an encoder that predict the latent variable from the observation. We showed that our ALD algorithm could accurately approximate posteriors with both theoretical and empirical studies. Using ALD, we derived a novel scheme of deep generative models called the Langevin autoencoder (LAE). We demonstrated that our LAE performs better than VI-based methods, such as the variational autoencoder, and existing LD-based methods in terms of the marginal log-likelihood for test sets.
This study will be the first step to further work on encoder-basee MCMC methods for latent variable models. For instance, developing algorithms based on more sophisticated Hamiltonian Monte Carlo methods is an exciting direction of future work.
One of the limitations of MCMC--bsed learning algorithms is the difficulty in choosing the number of MCMC iterations. To reduce the bias of the gradient estimate, we need to run the iterations for many times, but there are few clues to how many MCMC iterations are sufficient in advance.

---

Recently, a method to remove the bias of MCMC with couplings is proposed by Jacob et al. (020)] and it may help overcome this limitation of MCMC-based learning algorithm in the future work. Another limitation of our LAE is that there is constraint on the structure of the encoder as described in Theorem T Although the constraint is relatively minor, may be problematic when applying our method to modern DLVMs that have a hierarchical structure in the latent variables (e.g.. Vahdat and Kautz 2020 and Child (020]).
From a broader perspective, developing deep generative models that can synthesize realistic images could cause a negative impact, such as abuse of Deepfake technology. We must consider the negative aspects and take measures for them.
Acknowledgments and Disclosure of Funding
This work was supported by JSPS KAKENHI Grant Number JP21J22342 and the Mohammed bin Salman Center for Future Science and Technology for Saudi-Japan Vision 2030 at The University of Tokyo (MbSC2030). Computational resources Al Bridging Cloud Infrastructure (ABCT) provided by National Institute of Advanced Industrial Science and Technology (AIST) were used for the experiments.
References
Diederik Kingma and Max Welling. arXiv:1312.6114, 2013.
Auto-encoding variational bayes.
arXiv preprint
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference pages 1278-1288 PMLR, 2014.
conferern on machine learning.
Diederik P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. Semi-supervised learning with deep generative models. In Advances in neural information processing systems, pages 3581-3589, 2014.
Jinwon An and Sungzoon Cho. Variational autoencoder based anomaly detection using reconstruction probability. Special Lecture on IE, 2(1), 2015.
Biao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, and Min Zhang. Variational neural machine translation. arXiv preprint 1605..660 2016.
SM Ali Eslami, Danilo Jimenez Rezende, Frederic Besse, Fabio Viola, Ari S Morcos, Marta Gar- nelo, Avraham Ruderman, Andrei A Rusu, Ivo Danihelka, Karol Gregor, et al. Neural scene representation and rendering. Science, 360(6394):
2018.
Ananya Kumar, SM Eslami, Danilo J Rezende, Marta Garnelo, Fabio Viola, Edward Lockhart, and Murray Shanahan. Consistent generative query networks. arXiv preprint arXiv.1807.02033, 2018.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. International conference on machine learning, pages 1530-1538. PMLR, 2015.
Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Improved variational inference with inverse autoregressive flow, Advances in neural information processing systems, pages 4743-475: 2016.
Rianne Van Den Berg, Leonard Hasenclever, Jakub M Tomezak, and Max Welling. Sylvester normalizing flows for variational inference. In 34th Conference on Uncertainty in Artificial Intelligence 2018, UAl 2018, pages 393-402. Association For Uncertainty in Artificial Intelligence (AUAI), 2018
Chin Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive flows. In International Conference on Machine Learning, pages 2078-2087. PMLR, 2018.
Matthew D Hoffman. Learning deep latent gaussian models with markov chain monte carlo. In International conference machine learning, pages 1510-1519, PMLR, 2017.
10

---

Jun S Liu and Jun S Liu. Monte Carlo strategies in scientific computing, volume 10. Springer, 2001.
Christian P Robert, George Casella, and George Casella. Monte Carlo statistical methods, volume 2. Springer, 2004.
Walter R Gilks, Sylvia Richardson, and David Spiegelhalter. Markov chain Monte Carlo in practice. CRC press, 1995.
Charles J Geyer. Practical markov chain monte carlo. Statistical science, pages 473 373-483, 1992.
Geoffrey E Hinton and Ruslan Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(6786):504-5000 2006.
Radford Neal. Meme using hamiltonian dynamics. Handbook of Markov Chain Monte Carlo, page 113, 2011.
Peter E Kloeden and Eckhard Platen. Numerical solution of stochastic differential equations, vol- ume 23. Springer Science & Business Media, 2013.
Tijmen Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of the 25th international conference on Machine learming, pages 1064- 1071, 2008
Tian Han, Yang Lu, Song-Chun Zhu, and Ying Nian Wu. Alternating back-propagation for generator network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
Kingma DP and Jimmy Ba. Adam: method for stochastic optimization. Proc. of the 3rd
International Conference for Learning Representations (ICLRR, 2015.
Rui Shu, Hung H Bui, Shengjia Zhao, Mykel J Kochenderfer, and Stefano Ermon. Amortized infer- ence regularization. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 4398 4407, 2018.
Jianwen Xie, Ruiqi Gao, Zilong Zheng, Song-Chun Zhu, and Ying Nian Wu. Learning dynamic generator model by alternating back pock-propagation through time. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pages 5498-5507, 2019.
Jing Zhang, Jianwen Xie, and Nick Barnes. Learning noise-aware encoder-decoder from noisy labels by alternating back poo-paropagttion for saliency detection. arXiv preprint 20Xi::0077.222/111 2020.
Xianglei Xing, Ruiqi Gao, Tian Han, Song-Chun Zhu, and Ying Nian Wu. Deformable gen- erator network: Unsupervised disentanglement of appearance and geometry. arXiv preprint arXiv: 06298, 2018
Yizhe Zhu, Jianwen Xie, Bingchen Liu, and Ahmed Elgammal. Learning Ceature-to-feature translator by alternating back-propagation for generative zero-shot learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9844 9854,, 2019.
Yingzhen Li, Richard Turner, and Qiang Liu. Approximate inference with amortised meme. arXiv 2017.
preprint
Tim Salimans, Diederik Kingma, and Max Welling. Markov chain monte carlo and variational inference: Bridging the gap. In International Conference on Machine Learning, pages 1218-1226. PMLR, 2015.
Andrew Ng et al. Sparse autoencoder. CS294A Lecture notes, 72(201 1); 2011.
Michalis K Titsias and Francisco Ruiz. Unbiased implicit variational inference. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 167-176. PMLR, 2019.
Pierre E Jacob, John O'Leary, and Yves F Atchadé. Unbiased markov chain monte carlo methods with couplings. Journal of the Royal Statistical Society: Series (Statistical Methodology), 82(3): 543-600, 2020.

---

Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder. Advances in Neural Information Processing Systems, 33: 33:19667-19679 2020.
Rewon Child. Very deep vaes generalize autoregressive models and can outperform them on images. In International Conference on Learning Representations, 2020.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcn+++: Improving the pixelcnn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv: 2017.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. arXiv preprint 1rXiv:1607.06445 2016.
Danilo Jimenez Rezende and Fabio Viola. Taming vaes. arXiv preprint urXiv:1810.00597,, 2018.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, (6(11))2278-23222 1998.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] See the abstract and Section 1
(b) Did you describe the limitations of your work? [Yes] See Section 6
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
Section 6
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Theorem
(b) Did you include complete proofs of all theoretical results? [Yes] See the appendix.
3. If you ran experiments...
(a) Did you include the code. data, and instructions needed reproduce the main perimental results (either in the supplemental material or as a URI.)? [Yes] See the appendix.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See the appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See Section
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See the appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
12

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "tedOAOnCXuY", "reviewer": "Reviewer_753q", "summary": "This paper presents Amortized Langevin Dynamics (ALD), a scalable MCMC algorithm that can be used to sample from high dimensional deep latent variable models. \n\nUnlike competing MCMC algorithms that need to rely on per-datapoint iterations, ALD can sample from the correct posterior distribution by performing updates to an encoder. This gives rise to Langevin Autoencoders, a family of deep LVMs that relies on ALD. \n\nExperiments on a number of benchmark datasets show that Langevin Autoencoders outperform competing models while maintaining scalability. \n\n \n\n \n\n", "questions": "See comments in the \"weaknesses\" section.", "limitations": "Yes", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS \n\nMost applications of deep LVMs use variational methods to approximate the intractable posterior distribution.\nVariational methods are however inherently biased, since they try to approximate a complex posterior distribution with distributions from a typically simple variational family (e.g. Gaussian). \n\nThis is not an issue in sample-based MCMC methods, but they do not scale for high dimensional deep LVMs trained on large datasets, since they require expensive per-datapoint iterations. \n\nThis paper provides a novel solution to this, that relies on applying amortization ideas to MCMC methods. \nThe idea of obtaining samples from the posterior distribution by updating an encoder that data points to latent variables is very interesting to me, as it preserves the scalability of variational methods while not making any strong assumption on the approximate posterior distribution.\nAlso, theoretical and empirical results show that ALD matches the correct stationary posterior distribution even for the complex posterior distributions that are typical of DLVMs.\n\nI believe this paper can have an impact in the neurips community, since it can inspire a new research direction on enoder-based MCMC methods that use amortization and avoid datapoint-wise iterations.\n\n\nWEAKNESSES \n\nThere are some experimental improvements that I believe could further increase the impact of this paper.\n\n1. You state that the goal of the experiments is to show that ALD works, and not that of achieving SOTA results, which is ok. However, for this type of paper that presents a method that can be readily applied to any deep LVM, trying it out on a more SOTA architecture would have made the results much more compelling. The fact that you have not done it makes me wonder if there are some complications to applying it to more complex architectures.\n\n2. How can one monitor the convergence of the MCMC chain? Which convergence diagnostics have you used? \n\n3. Did you ever get in situations where the MH acceptance rate was too low/high? \n\n4. What are the training times of the different models considered in the experiments?\n\n\n Minor comments:\n\n* typo in title: variabel -> variable \n\n* line 29. The issue is not that the distribution is tractable as it is implied in the sentence. The issue is that the posterior for these types of models is complex, so the tractable distributions we use are not a good approximation. \n\n* line 41: \"in test time\" -> \"at test time\" \n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KsbFayYbKEI", "reviewer": "Reviewer_xhuG", "summary": "The paper deals with performing Langevin dynamics as a form of sampling, only using an amortisation mechanism. This reduces the computational overhead of performing the sampling. The method is then verified on synthetic and generated data sets\n", "questions": "You have used EM numerical solver, which has a lof of well defined numerical stability criterion, for example : Higham, D.J., 2001. An algorithmic introduction to numerical simulation of stochastic differential equations. SIAM review, 43(3), pp.525-546. Did you check any of them ?\n\nWhat is really fascinating about Langevin dynamics is that for a certain class you derive a shadow lemma which means you are tractable towards your “real” underlying distribution for example : https://arxiv.org/abs/1107.2967\nCan this analysis somehow improve the theorem on G\n", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is very clearly written and cleanly presented. It is worth mentioning that the authors also did a very good job introducing other related works (albeit a bit late in the text at section 4).\nMore so I think The main theorem is an important one. The idea of unfing the generated Markov chains with respect to the data set is novel.\nIn top of that using amortisation is a fantastic idea for a Markovian process since the lack of time correlation can make this efficient much like Mori Zwanzig operators untangle non markovian processes \nThat being said, there are a few things I am a bit worried about. It seems like to sample the ELBO there is a need to resample all the chains ?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "6TXYNZVS7s", "reviewer": "Reviewer_ZVTr", "summary": "The paper introduces amorized Langevin dynamics (ALD), a method for initializing Langevin dynamics in the function parameter space that makes it efficient to perform posterior sampling. Based on this method, the paper presents the Langevin autoencoder (LAE), a deep latent variable model that is easy to implement and has competitive performance compared to similar existing methods.", "questions": "- In Hoffman (2017) the encoder parameters are updated via gradient descent on the ELBO. However in this work the gradients are taken w.r.t. $V$ instead. I did not find derivations for this gradient update. How is this gradient update justified? What is the objective function in this case?\n- What do you think could be the reason behind the efficiency of performing MCMC in the parameter space instead of the data space?", "limitations": "The main limitation of the method seems to be the requirement that the size of the last linear encoder layer needs to be larger than the batch size. This is addressed in the experimental section 5.1.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The preliminaries and the core method are explained clearly. The drawbacks of existing MCMC and variational inference methods motivate the main ideas very well.\n- The algorithm of LAE is presented very clearly (Algorithm 2).\n- The idea of performing Langevin sampling in encoder parameter space seems quite novel and interesting. Proof is given on why this MCMC in the encoder parameter space will converge to the true data posterior.\n\nWeaknesses:\n- Although it doesn't get in the way of understanding the key ideas, the writing can certainly be improved at a couple of places. Examples include \"a more straightforward and sophisticated framework\" (line 48), \"replacing MCMC on the latent space into the encoder's parameter space\" (line 52) and \"we substitue it for its evidence lower bound\" (line 277, I believe it should be the other way around). Should be easy to fix.\n- I find the claim that ALD \"completely removes datapoint-wise iterations\" slightly misleading, since it does so trivially by turning datapoint-wise iterations into parameter space iterations. In essence it still relies on the encoder (up to the second to last layer) to initialize the MCMC in a high-density region, which is still pretty similar to Hoffman (2017). Maybe it can be shown that this method can have the same or better performance with less MCMC iterations than Hoffman, at which point it would be interesting to discuss why moving to parameter space gives such an advantage.\n- Without providing details on hyperparameter search (which can influence results greatly), the experimental results might not be sufficient for concluding that the LAE \"outperforms\" the existing methods. As mentioned in the last point, I think it would be really beneficial to show how the performance of the Hoffman (2017) model is influenced by the number of MCMC steps used.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept