Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: Summary of the paper:
This paper proposes to meta-learn a parametric divergence measure for variational inference (VI). Specifically, the paper focuses on alpha-divergence and f-divergence, aims at choosing a good alpha or f for a particular task leveraging the experience learned from previous tasks. The meta-learned divergence is applied to approximate a mixture of two Gaussians, regress sinusoid function with Bayesian neural networks, and learn recommender systems with partial VAE. 

Summary of my opinion:
I am leaning towards reject this paper because 
1) I'm not convinced by their motivation -- "this line of work has also shown that the optimal divergence can vary depending on tasks". There are many key factors to achieve good performance in VI, such as a good likelihood model, a good variational posterior, a good optimizer and so on. The divergence itself, in my opinion, is less significant. If you really care about the flexibility, the Wasserstein distance may be a better choice.
2) Choosing the alpha or f is a hyper-parameter search problem to me. What you need to do is to prepare a validation set and select the best hyper-parameters based on the performance there. It is impractical to collect M tasks and search for hyper-parameters according to some meta-losses. 
3) The experimental part is weak -- two toy problems plus an unusual recommendation system problem is not the typical way that people choose to evaluate VI methods. I suggest to evaluate on standard VAE/BNN benchmarks. 

Major comments: 
1. From eq.(4) and eq.(8), it seems one has to compute the value of the density function p(theta, D) in order to compute the gradients, which is obviously not possible for BNNs and VAEs. More details on how to perform VI with alpha-divergence and f-divergence should be covered. 
2. Can you let the meta-loss to be equal to D_eta? Meta-learning algorithms often optimize the same loss function in the inner and outer level. I couldn't see the point why you want to learn a f-divergence while setting the meta-loss to be D_0.5. 
3. Do you have a validation set for the Bayesian optimization (BO) baseline or do you use cross-validation? The details of this baseline should be elaborated. In fact, it is possible to choose the best alpha for each task using BO. Should this be compared?
4. What does VB mean in Section 4.2? If VB uses KL rather than alpha-divergence or f-divergence, the outperformance may only suggest that KL is insufficient there. 
5. Have you considered input convex neural networks (Amos et al. 2016) for implementing f? 

Minor comments:
1. The proofs of Prop 1 & 2 are missing.   
2. "However, using this KL divergence for VI has been criticized for under-estimating the uncertainty..." Any reference to this?
3. Figure 1: alpha is used before it is defined. 
4. "Other existing definitions of alpha-divergences have dis-continuous gradients at the alpha values corresponding to KL divergences" -- missing reference.
5. "When D_0.5 is in use as the meta-loss, the corresponding log f*'' is analytical" -- could you elaborate on this? 

------------------------------
After rebuttal:
I would like to thank the authors for their extremely detailed responses! The paper is now greatly improved. 
I would like to increase my score from 3 to 5, however, there is no such option this year. 
The reason why I am still hesitated to accept this paper is mainly because I am not sure how useful learning the divergence to VI is. For unsupervised learning, people use GAN or VQ-VAE to obtain realistic samples, but which is not attributed to better divergence measures. Plus, what we refer to "better" depends on the supplied meta-loss, which may not be better at all. Another unsatisfactory point of this paper in my opinion is that it only demonstrated results on toy datasets and MNIST. Although the proposed method is valid for alpha-divergence and f-divergence, I suggest the authors to find a real application which demonstrates the merit of the idea in practice. 
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: SUMMARY OF THE PAPER:
This paper proposes a way to automatically select a divergence to use in variational inference (VI) given a set of datasets (tasks). They consider searching within the alpha-
divergence and f-divergence families. The proposed algorithm works as follows: do a few gradient descent steps on the variational parameters given a fixed divergence parameter, then update the divergence parameter by taking the gradient with respect to a meta loss which is a task-specific measure of goodness of the variational distribution (like test log marginal likelihood). The latter gradient is computed through the gradient descent computation in the inner loop.
The second proposed algorithm does the same, but also learns a good initialization for the variational parameters. This initialization is good in the sense that taking one (or a few) gradient descent step on the variational parameters should give us good variational parameters (MAML style). This is done by taking a gradient with respect to this good initialization parameter through the inner loop gradient descent.

STRUCTURE:
The paper is well-written and easy to understand.

NOVELTY:
As far as I know, learning a divergence for VI using meta-learning is new. The related work is discussed well in section 5.

EXPERIMENTS:
There are experiments on three tasks of increasing complexity. In the simpler experiments, careful ablation studies are done. The results generally show that the proposed method is preferable to alternatives.

CONCLUSION:
I recommend acceptance.
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper proposes to use a meta-learning approach to learn the divergence used in variational inference and initial variational parameters. It considers two families of learnable divergences, including the alpha-divergence and f-divergence, and proposes two double loop algorithms. In both algorithms, the inner loop adjusts the variational parameters for a specific variational inference task. The outer loop adjusts the parameters in the divergence (and perhaps an initial variational parameter for all tasks) in terms of a human-designed meta-loss. Proposition 1 (following Wang et al. 2018a) shows that the gradients with respect to the meta-parameters in the f-divergence can be obtained using f''. This makes it convenient to parameterize f'' by neural networks to implicitly model f.

The experiments show that the proposed method can outperform a Bayesian optimization (BO) baseline if the meta-loss is the metric of interest in the MoG example. For the regression and recommendation system examples, it compares with a VB baseline and a p-VAE baseline and shows better results.

As for the significance of the paper, I have the following issues:

1. To apply the proposed method, we need to have a family of tasks that shares similarity at hand, which seems to be restricted. For instance, the first two examples are synthesized by hand. The recommendation system example is from real life but also crafted by splitting groups by age. If we just use all of the data by a p-VAE and fine-tune the hyperparameters a little bit (meta-learning needs more time than a p-VAE in a single configuration), can we obtain better results? If so, the last example is not appealing.

2. To apply the proposed method, we need to have some knowledge about our preference in the evaluation and encode that knowledge into the meta-loss. In the first example, the comparison with BO in the alpha=.5 case is unfair because Meta-D uses this knowledge as the meta-loss while BO does not leverage such information. Besides, if we already know the preferred divergence, it is not necessary to learn that divergence. For the other settings where Meta-D does not leverage any knowledge about the evaluation preference, it lacks stronger baselines such as BO.

3. The idea is straightforward and the most challenging part of how to model a convex function by a neural network is solved by existing work.

Based on these issues, I think the contribution of the paper is not sufficient and I tend to reject the paper. Also, note that the paper length extends 8 pages.

The motivation should be strengthened and the experiments should be more precise to improve the paper. 
review_title: Official Blind Review #2

Meta-review: The paper proposes a meta-learning algorithm to learn the divergence measure of variational inference as well as the initialization of the variational parameters (which reduces optimization steps of VI). Improved performance by the learned divergence against hand-designed ones are empirically shown on: Gaussian mixture approximation, Bayesian neural regression, and p-VAE based recommender.
Reviewers initally raised some concerns on hyperparameters selection, weakness of experiments, and motivation for the proposed scheme. The authors responded by adding additional experiments (MNIST) as well as some new sections in their appendix about details of their method or the baselines.
The reviewers greatly appreciated the response and commonly believed that the revised version is significantly improved over the initial draft  and the improvements of the draft. As a result of that, some reviewers increased their scores. However, some of their concerns did not resolve. In particular, R1 questions the impact of the work and importance of learning divergence measure (referring to GAN or VQ-VAE for obtaining realistic samples). Also R1 finds evaluation based on MNIST unsatisfactory, as it is commonly considered as a toy dataset. To motivate the method, it is suggested that the authors think about real applications which can highlight the benefits of their method in practice. Similar concerns are shared by R2 after authors' response. In particular, R2 is not convinced about motivation and the necessity of using meta-learning for learning the divergence. I suggest authors improve on issues around motivation and support the impact of their scheme in a more practical setting.
Decision: Reject