Question: What is your motivation of combining Langevin flow and Normalizing flow
Evidence: Normalizing flows are a family of generative models that construct a complex distribution by transforming a simple probability density, such as Gaussian distribution, through a sequence of invertible and differentiable mappings. Due to the tractability of the exact log-likelihood and the efficiency of the inference and synthesis, normalizing flows have gained popularity in density estimation and variational inference. However, for the sake of ensuring the favorable property of closed-form density evaluation, the normalizing flows typically require special designs of the sequence of transformations, which, in general, constrain the expressive power of the models.
Energy-based models (EBMs) define an unnormalized probability density function of data, which is the exponential of the negative energy function. The energy function is directly defined on data domain, and assigns each input configuration with a scalar energy value, with lower energy values indicating more likely configurations. Recently, with the energy function parameterized by a modern deep network such as a ConvNet, the ConvNet-EBMs have gained unprecedented success in modeling large-scale data sets and exhibited stunning performance in synthesiz-ing different modalities of data, e.g., videos, volumetric shapes, point clouds and molecules. The parameters of the energy function can be trained by maximum likelihood. However, due to the intractable integral in computing the normalizing constant, the evaluation of the gradient of the log-likelihood typically requires Markov chain Monte Carlo (MCMC) sampling, e.g., Langevin dynamics, to generate samples from the current model. However, the Langevin sampling on a highly multi-modal energy function, due to the use of deep network parameterization, is generally not mixing. When sampling from a density with a multi-modal landscape, the Langevin dynamics, which follows the gradient information, is apt to get trapped by local modes of the density and is unlikely to jump out to explore other isolated modes. Relying on non-mixing MCMC samples, the estimated gradient of the likelihood can be biased, and the resulting learned energy-based model may become an invalid model, which is unable to approximate the data distribution as expected.
Recently,  propose to train an EBM with a short-run non-convergent Langevin dynamics, and show that even though the energy function is invalid, the short-run MCMC can be treated as a valid flow model that generates realistic examples. This not only provides an explanation of why EBMs with non-convergent MCMC are still capable of synthesizing realistic examples, but also suggests a more practical computationally-affordable way to learn useful generative models under the existing energy-based learning frameworks. Although EBMs have been widely applied to different domains, learning short-run MCMC in the context of EBM is still underexplored.
In this paper, we will accept the fact that MCMC sampling is not mixing in practice, and decide to give up the goal of training a valid EBM of data. Instead, we treat the short-run non-convergent Langevin dynamics, which shares parameters with the energy function, as a flow-like transformation that we call the Langevin flow because it can be considered as a noise-injected residual network. Even though implementing a short-run Langevin flow is surprisingly simple, which is nothing but a design of a bottom-up ConvNet for the energy function, it might still require a sufficiently large number of Langevin steps (each Langevin step consists of one step of gradient descent plus a diffusion step) to construct an effective Langevin flow, so that it can be expressive enough to represent and synthesize the data. Motivated by reducing the number of Langevin steps in the Langevin flow for computational efficiency, we propose the CoopFlow model that trains a Langevin flow jointly with a normalizing flow in a cooperative learning scheme, in which the normalizing flow model learns to serve as a rapid sampler to initialize the Langevin flow so that the Langevin flow can be shorter, while the Langevin flow teaches the normalizing flow by short-run MCMC transition toward the EBM so that the normalizing flow can accumulate the temporal difference in the transition to provide better initial samples. Besides, the motivation of using the normalizing flow instead of a generic generator, is because the cooperative learning framework need to incorporate the MLE algorithm of an EBM and the MLE algorithm of a generator. The MLE of a normalizing flow generator will be much more tractable than the MLE of any other generic generator. The latter might resort to either MCMC-based inference to evaluate the posterior distribution or another encoder network for variational inference. In the proposed CoopFlow, the Langevin flow can overcome expressivity limitation of the normalizing flow due to the invertibility constraint. Such a benefit also motivates us to study the CoopFlow framework. To be specific, we understand the dynamics of cooperative learning with short-run non-mixing MCMC by information geometry. We provide justifications that the CoopFlow trained in the context of EBM with non-mixing MCMC is a valid generator, because it converges to a moment matching estimator. Experiments, including image generation, image reconstruction, and latent space interpolation, are conducted to support our justification.
a. Motivated by reducing the number of Langevin steps in the Langevin flow
for computational efficiency, we propose the CoopFlow model that trains a Langevin flow jointly with a normalizing flow in a cooperative learning scheme, in which the normalizing flow model learns to serve as a rapid sampler to initialize the Langevin flow so that the Langevin flow can be shorter, while the Langevin flow teaches the normalizing flow by short-run MCMC transition toward the EBM so that the normalizing flow can accumulate the temporal difference in the transition to
provide better initial samples. Besides, the motivation of using the normalizing flow instead of a generic generator, is because the cooperative learning framework need to incorporate the MLE algorithm of an EBM and the MLE algorithm of a generator. The MLE of a normalizing flow generator will be much more tractable than the MLE of any other generic generator. The latter might resort to either MCMC-based inference to evaluate the posterior distribution or another encoder network for variational inference. In the proposed CoopFlow, the Langevin flow can overcome expressivity
limitation of the normalizing flow due to the invertibility constraint. Such a benefit also motivates us to study the CoopFlow framework.
Answer: Our major motivation has been discussed in the last paragraph in our original submission. To be specific, our paper is mainly motivated by the current short-run MCMC EBM paper [a], which realizes that the convergent MCMC can not be guaranteed in learning EBM via MLE in practice and the synthesized examples are actually generated by a flow-like generator that is constructed by a short sequence of Langevin steps, i.e., a short-run MCMC. Although the short-run MCMC generator is appealing due to its simplicity of implementation, its training is time-consuming because the short-run MCMC still needs to be built with roughly 100 Langevin steps to ensure a sufficient model capacity. To further reduce the length of the short-run MCMC, we try to use a deterministic transformation, i.e., normalizing flow, to fast initialize the short-run MCMC so that the MCMC can be much shorter.
The above main motivation has been also pointed out by Reviewer DUki: "This paper is strongly motivated by the short-run MCMC energy-based model, and tries to use a normalizing flow model for a nice initialization to shorten MCMC steps even more."
The motivation of choosing normalizing flow instead of a generic generator, is because the cooperative learning framework needs to incorporate an MLE algorithm of EBM and an MLE of generator. The MLE of a normalizing flow generator will be much more tractable than the MLE of any other generic generator. The latter might resort to either MCMC-based inference to evaluate the posterior distribution or another encoder network for variational inference.
The normalizing flow model will also benefit from its cooperation with EBM because its expressivity will be improved by the stochastic sampling process guided by the EBM. [b] has shown that stochasticity overcomes expressivity limitation of normalizing flows resulting from the invertibility constraint. Such a benefit also motivate us to propose the CoopFlow framework. Our experiments have shown that the CoopFlow indeed improves the performance of a single normalizing flow. Here we also want to grant credits to Reviewer 1uN8 who pointed out the related reference [b]. We have added it into our reference. [b] justifies advantage of the combination of stochastic Langevin flow and deterministic normalizing flow.
We have revised our paper by explicitly mentioning our motivation we discuss above. Please check the last paragraph in our revised paper. We use red color to highlight our revision. Thank you for your constructive suggestion to improve the presentation of our paper.