Paper:
Improving Explorability in Variational Inference with Annealed Variational Objectives
Despite the advances in the representational capacity of approximate distributions for variational inference, the optimization process can still limit the density that is ultimately learned. We demonstrate the drawbacks of biasing the true posterior to be unimodal, and introduce Annealed Variational Objectives (AVO) into the training of hierarchical variational methods. Inspired by Annealed Importance Sampling, the proposed method facilitates learning by incorporating energy tempering into the optimization objective. In our experiments, we demonstrate our method's robustness to deterministic warm up, and the benefits of encouraging exploration in the latent space.

Review:
POST-REBUTTAL  Having read the rebuttal, I am inclined to keep my score as is. I like the paper and the main reason why I am not giving a higher score is because I think the proposed solution AVO (an ad hoc collection of objectives) seems rather crude. I look forward to future work that improves upon the currently proposed AVO.  I still think that one needs to be careful with the "optimization process" claim. In the rebuttal, the authors stated that "both HVI and NF are highly expressive families of distribution that can provide a good approximate of a potentially highly multimodal target."  There are two problems with this line of argument:  1. The correctness of the aforementioned quote depends on the expressivity of amortization family (aka encoder size). In the limit of universal function approximation, I agree with the statement. For finite-depth networks used in practice, I do not think we have the necessarily empirical evidence to back up the claim.  2. Let's suppose the encoder has "infinite capacity." Any argument that there is an issue with the optimization process is *very difficult to support*. If one readily admits that the optimization process is suboptimal, It is difficult to say for sure that the suboptimal optimization process will necessarily result in mode-seeking behavior---even if the objective itself is known to be mode-seeking. This is because the mode-seeking characterization of the objective is only guaranteed for the optimal solution.   Ultimately, this issued is resolved via their empirical experiments, which showed consistent mode-collapsing behavior. I therefore think the authors could be more careful about the way they structured their claim.  --------------------------------------------------------------------------------  SUMMARY OF PAPER  The paper makes the interesting observation that ELBO objective encourages the variational posterior to adopt a mode-seeking behavior, and that this bias will persist even when a more expressive variational family is used due to optimization issues. To remedy this, they introduce a novel regularizer inspired by Annealed Importance Sampling that supposedly biases the variational posterior to be less mode-seeking. They demonstrate the value of the proposed regularizer on several illuminating toy examples as well as MNIST/OMNIGLOT density estimation.  --------------------------------------------------------------------------------  SUMMARY OF EVALUATION  This is quite an interesting paper that highlights the mode-seeking behavior of variational inference. While the mode-seeking behavior of variational inference is well-known, little has thus far been written in the VAE community about how this biases the generative model to adopt unimodal posteriors. I think this paper will help draw attention to this important issue.  Pros:  + Generally well-written + Addresses an important problem + Proposes an interesting solution (AVO) + Insightful experiments  Cons:  - AVO is kind of hacky - Not completely clear why AVO encourages exploration  --------------------------------------------------------------------------------  NARRATIVE  The narrative is sound; there are only a few things I want to remark:  1. Is the problem the optimization *process* or the optimization objective?  To me, optimization process suggests that the problem has to do with the choice of optimizer, and that access to an oracle optimizer will solve the issue. The issue as presented in this paper, however, is regarding the inherent mode-seeking behavior of the variational inference objective which trains the variational posterior via reverse-mode KL.  2. More discussion about the AVO   There are a couple of things that make AVO peculiar and worthy of more exposition.  First, it is interesting that AVO is not a single optimization problem, but a heuristic collection of T separate optimization problems. My guess is that this design is forced by the fact that optimization of the normalized f-tilde (w.r.t. theta) and the marginal q(z) are intractable. The fact that the AVO is an ad-hoc collection of T objective problems makes it harder to truly characterize how AVO will behave. It also raises the question how the behavior of AVO might have differed had it been a single, fully-differentiable objective trained on both inference and generative model parameters. I think these issues should be discussed in the paper so that future readers are inspired to come up with potentially better solutions than AVO.  Second, it is not obvious why AVO necessarily encourages exploration. I would appreciate a more thorough discussion on why the authors believe exploration is implied by AVO. At a very high level, OVA still more or less minimizes the KL(q || f*r), which still (arguably) biases q toward mode-seeking (w.r.t. f*r). Why then is this not a source of concern? On a related note, I think the sentence "in the case of amortized VI, we set f0 to be the prior distribution of the VAE, so each intermediate target has a more widely distributed density function, which facilitates exploration of the transition operators" should be expanded upon to emphasize the potential importance of designing f0 to be a broad distribution. In fact, I would appreciate additional experiments for Table 1 showing what would happen if f0 (and q0) is instead trainable.  3. Checking the unimodality claim  The unimodality-bias of ELBO popped up a number of times. While I agree that this is probably true, it would be nice to empirically check whether the ELBO-trained VAEs tend to end up with unimodal posteriors, and whether AVO-trained VAEs end up with non-unimodal posteriors (or something else?). If there is anyway of checking the number of modes in either the true posterior or variational posterior, this would be an interesting experiment.  --------------------------------------------------------------------------------  EXPERIMENTS  1. Show ELBO values for Figure 3  I would like to see the ELBO values for the q learned by HVI-AVO v. HVI-ELBO. Ideally, we should see that HVI-AVO has a worse ELBO.   2. Where is appendix D?  I only see appendix A and B in the supplementary file. I would like to check how the entropy of q_T(z_T) is computed to get a sense of how tightly the lower bound approximates the the log marginal.   3. Beta-annealing schedule used in 5.3  Can you describe more precisely what is the experimental set-up for 5.3? I'm afraid I don't understand the sentence "x-axis: the percentage of total training time it takes to anneal β back to 1." Is it stated anywhere that the beta-annealing schedule is also linear?  4. AVO applied to amortized VI  There are a number of ways one could potentially construct the amortized forward and backward transition operators depending on how you choose to condition on x. It would be nice if graphical model figure is provided somewhere in the paper, showing the exact choices of generative/inference model structure used in the experiments (e.g. see Auxiliary Deep Generative Model Fig. 1 for example).  5. Log-likelihood estimation for Table 1  Please state in the paper how the log-likelihood is estimated.  6. Not setting f0 to be the prior  I'm quite interested in what would happen if f0 is not set to be the prior distribution of the VAE.