user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Metropolis-Hastings view on variational inference and adversarial training

Paper content:
In this paper we propose to view the acceptance rate of the Metropolis-Hastings algorithm as a universal objective for learning to sample from target distribution -- given either as a set of samples or in the form of unnormalized density.
This point of view unifies the goals of such approaches as Markov Chain Monte Carlo (MCMC), Generative Adversarial Networks (GANs), variational inference.
To reveal the connection we derive the lower bound on the acceptance rate and treat it as the objective for learning explicit and implicit samplers.
The form of the lower bound allows for doubly stochastic gradient optimization in case the target distribution factorizes (i.e. over data points).
We empirically validate our approach on Bayesian inference for neural networks and generative models for images.
Bayesian framework and deep learning have become more and more interrelated during recent years.
Recently Bayesian deep neural networks were used for estimating uncertainty BID6 , ensembling BID6 and model compression BID20 .
On the other hand, deep neural networks may be used to improve approximate inference in Bayesian models BID13 .Learning
modern Bayesian neural networks requires inference in the spaces with dimension up to several million by conditioning the weights of DNN on hundreds of thousands of objects. For such
applications, one has to perform the approximate inference -predominantly by either sampling from the posterior with Markov Chain Monte Carlo (MCMC) methods or approximating the posterior with variational inference (VI) methods.MCMC methods provide the unbiased (in the limit) estimate but require careful hyperparameter tuning especially for big datasets and high dimensional problems. The large
dataset problem has been addressed for different MCMC algorithms: stochastic gradient Langevin dynamics BID28 , stochastic gradient Hamiltonian Monte Carlo , minibatch MetropolisHastings algorithms BID15 BID1 . One way to
address the problem of high dimension is the design of a proposal distribution. For example
, for the Metropolis-Hastings (MH) algorithm there exists a theoretical guideline for scaling the variance of a Gaussian proposal BID24 BID25 . More complex
proposal designs include adaptive updates of the proposal distribution during iterations of the MH algorithm BID12 BID7 . Another way
to adapt the MH algorithm for high dimensions is combination of adaptive direction sampling and the multiple-try Metropolis algorithm as proposed in BID17 . Thorough overview
of different extensions of the MH algorithm is presented in BID18 .Variational inference
is extremely scalable but provides a biased estimate of the target distribution. Using the doubly stochastic
procedure BID27 BID11 VI can be applied to extremely large datasets and high dimensional spaces, such as a space of neural network weights BID14 BID5 . The bias introduced by variational
approximation can be mitigated by using flexible approximations BID22 and resampling BID9 .Generative Adversarial Networks BID8
) (GANs) is a different approach to learn samplers. Under the framework of adversarial training
different optimization problems could be solved efficiently BID0 BID21 . The shared goal of "learning to sample" inspired
the connection of GANs with VI BID19 and MCMC BID26 .In this paper, we propose a novel perspective on
learning to sample from a target distribution by optimizing parameters of either explicit or implicit probabilistic model. Our objective is inspired by the view on the acceptance
rate of the Metropolis-Hastings algorithm as a quality measure of the sampler. We derive a lower bound on the acceptance rate and maximize
it with respect to parameters of the sampler, treating the sampler as a proposal distribution in the Metropolis-Hastings scheme.We consider two possible forms of the target distribution: unnormalized density (density-based setting) and a set of samples (sample-based setting). Each of these settings reveals a unifying property of the proposed
perspective and the derived lower bound. In the density-based setting, the lower bound is the sum of forward
and reverse KL-divergences between the true posterior and its approximation, connecting our approach to VI. In the sample-based setting, the lower bound admit a form of an adversarial
game between the sampler and a discriminator, connecting our approach to GANs.The closest work to ours is of BID26 . In contrast to their paper our approach (1) is free from hyperparameters; (
2) is able to optimize the acceptance rate directly; (3) avoids minimax problem in the density based setting.Our main contributions are as follows:1. We introduce a novel perspective on learning to sample from the target distribution
by treating the acceptance rate in the Metropolis-Hastings algorithm as a measure of sampler quality. 2. We derive the lower bound on the acceptance rate allowing for doubly stochastic
optimization of the proposal distribution in case when the target distribution factorizes (i.e. over data points). 3. For sample-based and density-based forms of target distribution we show the connection
of the proposed algorithm to variational inference and GANs.The rest of the paper is organized as follows. In Section 2 we introduce the lower bound on the AR. Special forms of target distribution
are addressed in Section 3. We validate our approach
on the problems of approximate Bayesian inference in the space
of high dimensional neural network weights and generative modeling in the space of images in Section 4. We discuss results and directions of the future work in Section 5.
This paper proposes to use the acceptance rate of the MH algorithm as the universal objective for learning to sample from some target distribution.
We also propose the lower bound on the acceptance rate that should be preferred over the direct maximization of the acceptance rate in many cases.
The proposed approach provides many ways of improvement by the combination with techniques from the recent developments in the field of MCMC, GANs, variational inference.
For example• The proposed loss function can be combined with the loss function from BID16 , thus allowing to learn the Markov chain proposal in the density-based setting.•
We can use stochastic Hamiltonian Monte Carlo for the loss estimation in Algorithm 1. •
In sample-based setting one can use more advanced techniques of density ratio estimation.Application of the MH algorithm to improve the quality of generative models also requires exhaustive further exploration and rigorous treatment.

assistant: text: Learning to sample via lower bounding the acceptance rate of the Metropolis-Hastings algorithm