Meta-Learning for Variational Inference
Authors: Ruqi Zhang
Yingzhen Li
Chris De Sa
Sam Devlin
Cheng Zhang
Venue:  (2020)
Keywords: Variational inference
Meta-learning

Abstract: Variational inference (VI) plays an essential role in approximate Bayesian inference due to its computational efficiency and general applicability. 
Crucial to the performance of VI is the selection of the divergence measure in the optimization objective, as it affects the properties of the approximate posterior significantly. In this paper, we propose a meta-learning algorithm to learn (i) the divergence measure suited for the task of interest to automate the design of the VI method; and (ii) initialization of the variational parameters, which reduces the number of VI optimization steps drastically. We demonstrate the learned divergence outperforms the hand-designed divergence on Gaussian mixture distribution approximation, Bayesian neural network regression, and partial variational autoencoder based recommender systems.

Under review as a conference paper at ICLR 2020
META- FOR VARIATIONAL INFERENCE
Anonymous authors Paper under double-blind review
ABSTRACT
Variational inference (VT) plays an essential role approximate Bayesian infer- ence due to its computational efficiency and general applicability. Crucial to the performance VI is the selection of the divergence measure the optimization objective, as it affects the properties of the approximate posterior significantly. In this paper, we propose a meta-learning algorithm to learn (i) the divergence
measure suited for the task of interest to automate the design of the VI method; and (ii) initialization of the variational parameters, which reduces the number VI optimization steps drastically. We demonstrate the learned divergence out- performs the hand-designed divergence on Gaussian mixture distribution approx- imation, Bayesian neural network regression, and partial variational autoencoder based recommender systems.
INTRODUCTION
Bayesian inference provides powerful tool for probabilistic modelling of data, however, exact Bayesian inference often intractable. Therefore practical Bayesian inference often resort to ap- proximations. and various approximate inference methods have been developed. Variational Infer- ence (VI) (Jordan et al. 1999 Zhang et al.) 2018) approximates the intractable target distribution through optimization of tractable distribution. Compared to Markov chain Monte Carlo (MCMC), VI is biased but more computationally efficient, making it particularly suitable to large-scale models in deep learning such as Bayesian neural networks and deep generative models.
VI approximates the target distribution by minimizing a divergence objective. Different divergence metrics essentially define a different inference algorithm which leads to different properties of the approximation. Therefore the selection of this divergence is one of the crucial factors of making VI successful. The most widely used divergence measure is (L(qllp) where p the target distri- bution and q is the approximate distribution. However, using this KL divergence for VI has been criticized for under-estimating the uncertainty (Bishop, 2006) Blei et al. 22177: Wang et al. (018a), which leads to poor model performance when uncertainty estimation is essential. Many alterna- tive divergence measures have been proposed for VI to alleviate this issue (Minka et al. 2005
Hernández-Lobato et al. 2016 Li & Turner 2016 Csiszár et al.) 2004 Bamler et al. 2017
Wang
ct al.) 201 which provide better bias and variance trade-ofls and lead to better predictive results with more accurate uncertainty estimation.
0.16
0.35
a 3.0
0:54
- 1.0
0.14
0.12
a 0.1
0.25
0.10
0.93
0.38
0.65
022
8:02
12
Figure 1: An illustration of approximating distributions Gaussian mixture by different value of a (defined in (3). "std" the standard deviation of the Gaussian approximation.
However, as illustrated by Figure [ this line of work has also shown that the optimal divergence can vary depending on tasks and undesired divergence objective can lead to mediocre performance

---

Under review as a conference paper at ICLR 2020
(Li & Turner 2016 Depeweg et al. 2016). Unfortunately, choosing suitable divergence objective for a specific task Is challenging as it requires a thorough understanding of (i) the shape of the target distribution; (ii) the desirable properties of the approximate distribution (c.g. mass-covering or mode-seeking); and (iii) the bias-variance trade-off between the tightness and the variance of the Monte Carlo estimate of the variational bound. A crucial question remains to be addressed in order to make VI a success for a wide range of applications: can we automatically choose a suitable divergence which are tailored specific type of tasks?
To answer this question, we propose meta-learning for variational inference (meta-VI) which uti- lizes the advantages of meta-learning to improve approximate Bayesian inference. Meta-learning to design a learner based on several training tasks that can generalize well to future tasks (Naik & Mammone 1992 Thrun Pratt 2012 Hochreiter et al., (00T). Our meta-VI VT learns an inference algorithm that is tailored to the problem of interest. Additionally, meta-V VI can provide a good ini- tialization of the variational parameters which reduces the training time remarkably. We summarize our contributions the following:
We developed a general framework for meta-learning variational inference (Section 3.1 which can choose divergence objective automatically by meta-learning. Specifically, we derive meta-V VI first for a-divergence based VI, then extend it to f-divergence family and provided a novel parameterization of f-divergence which then used in meta-VI.
In addition to meta-lcarning the divergence objective, we further combine meta-V VI with meta-learning parameters for the variational distribution (Section 3.2). This method pro- vides initialization for fast adaptation and automatically learns the divergence to optimize.
We demonstrate the efficacy of our approach on various experimental settings (Section On both the Gaussian mixture distribution approximation and Bayesian neural network regression tasks, meta-V VI significantly out-performs all baseline methods. On a large-scale, real-world example, meta- VI also improves performance when applied to partial variational auto-encoder based recommender systems.
PRELIMINARIES
Considering a dataset D { and 6 6-parameterized model. Bayesian inference requires computing the posterior over given the dataset D: p(0]D) = 2(D10)p(0)
p(D) The exact posterior D) is generally intractable, therefore one needs to resort to approximation with a tractable approximate posterior q()) p(0)D). Typically the approximation posterior q(0) is obtained by minimizing a divergence, e.g. Variational Inference (VI) applications often minimize (L(g(0) P(()D)). This turns Bayesian inference into an optimization task (divergence minimization). In practice, due to the intractability of p(D), VI alternatively maximizes equivalent objective called the variational
lower bound:
P(D,0) = logg KL-(gll)) log
Lvi E,
((0)
Renyi's a-divergence a-divergence is a rich family that includes many common divergence as special cases (Minkal 200T Hernández-LLohato et al., 2016) [i & Turner 2016). In this paper, we focus on Renyi's isss definition (Rényi et al. 1961 Li & Turner, (2166) rather than the others (Amari) 2012 Tsallis 1988) due to its unitied expression of gradient for all finite a. The benefit of having unified expression will be seen in Section 3.1 Renyi's a-divergence is defined by
(a(p))))
log -00)"((0)'~000
0, 11
where (pllg) + (L(p)))) when + 1. Similar to maximizing the variational lower bound when the divergence objective is KL(gllp), one can maximize the variational Renyi bound (VR bound) derived from Renyi's a-divergence:
1
(
1-a7 (((0)
La La(g; D) 1
log EggE,
= D(D) Da(gllp)
(3)
(())
2

---

Under review as a conference paper at ICLR 2020
The reparameterization trick (Salimans al. 2013 Kingma & Welling 2013) is commonly used in practice for gradient ascent based optimization of the VR bound Eq.(3) where sampling 0 ~ qo(0) is conducted by first sampling e p(c) from a simple distribution independent with the variational parameter Φ (e.g. Gaussian) then parameterizing 0 = ho(c). Using the reparameterization trick,
( (p(hale).)) (o(h)(())
1-a]
the VR bound Eq.(3) becomes (gp; D) = i- log E.
The expectation is
usually computed by Monte Carlo (MC) approximation. The gradient of VR bound w.r.t. after MC approximation with K particles is
Wark V log p(ho(cx), ((ha(c)))
V Fobii
(4)
R=1
When 0 the weights Wa,k 1/K and the corresponding gradient (4) becomes an unbiased estimate of the gradient of the varia- tional lower bound (
where Wa,k
(() atha(ea))
Σ K k=1
plholee) ((hatea])
As shown in Figure approximate inference with different a-divergences results in different vari- ational distributions. Minka et al. (2005) and Li & Turner (2016) also showed the optimal a- divergence may vary for different tasks and datasets, and in practice it is difficult to choose an optimal didivergence a priori.
f-divergence f-divergence defines more general family of divergences (Csiszár et al., 2004 Minka et al., 2005). can be defined using a twice differentiable convex function f R + (Csiszár et al. 2004):
[ p(O) To )
Dy(pl)as) Egrage
f(1)
(5)
This family includes KL-divergences in both directions which can be seen by taking f(t) log for (L(gllp) and ft) = t logt for KL(pllg). It also contains a-divergences which takes f(t) = a(a-1) for a E R\ 1}. Although f-divergence family is very rich due to the usage of arbitrary twice differentiable convex function, it requires significant expertise to design suitable f function for a specific task.
META-VI
The goal of meta-learning variational inference algorithm is to learn a divergence objective, so that the resulting VI algorithm produces an approximate distribution with desired properties on a certain type of tasks. To achieve this, we first construct learnable divergence family, then design meta-loss function that gives guidance for updating the divergence.
Assume have M training tasks Ti, ..., TM sampled from underlying task distribution p( Each task has its own probabilistic model pr, (0,, Dr.). Let D,,((l-) be the learnable diver- gence parameterized by 1), then for each task the approximate posterior gø, (0,) is computed by minimizing D,, (Pr, (D, [la,, (0))). In the rest of the paper we also write D,, (g,( T.) = (p, (0.( 1aeell99. qo. (0.)) for brevity. During meta-training, we define a meta-loss function I(qo, Te) which is optimized w.r.t. the divergence parameter 77. This meta-loss function is de- signed to evaluate the desired properties of the approximate distribution, e.g. test log-likelihood. During meta-testing, a new task is sampled from p(T), and the learned divergence D,, is used to optimize the variational distribution.
Besides the above setting, we also consider few-shot learning set-up similar to the model-agnostic meta-learning (MAML) framework (Finn et al., 2017 22188] Kim et al. (2088). In this case the prob- abilistic model architecture is shared by all the tasks, and the goal of meta-training is to obtain divergence as well as an initialization of the variational parameters ø for unseen tasks. After meta- training, we will train the model with the learned divergence objective and the learned initialization of variational parameters 6 on new tasks. The above meta-learning settings are practical as demon- strated in many previous work (Finn et al., 2017 2018 Kim et al. 2018), including meta-learning for Bayesian inference (Gong et al.) 2018). Attaining common knowledge based on the previous tasks has been proved be useful for the future tasks.
3

---

Under review as a conference paper at ICLR 2020
Algorithm Meta-D
Algorithm 2 MMea--Dee D&o
Input: M: number of training tasks. 8, 7:
Input: p(T): distribution over tasks. B, 7, learning rate hyperparameters.
learning rate hyperparameters.
Initialize o; i = ,,..., M (Ø: can have different structures). loop
Initialize 6.
loop
Sample M tasks T, p(T).
for T; -- I...,, do for B times do
for all T₃ do
Update the variational parameters with the current divergence: Φi + $
Update the variational parameters with the current divergence: Φ, + - -. Drlo, end for end for Update . 7 V [io,,, end loop Output:
BV Ti).
end for Update & Φ -- -VM E; [( T3): + 1- E, J (1p:: end loop Output: 17,
3.1 META- DIVERGENCE OBJECTIVE (META-D)
We consider the first problem setting of Iearning divergence. We first present method assuming the parameteric form of D, is given. Then we will provide the details of parameterization of two divergence families: a-divergence and f-divergence and how they fit in this framework.
The idea of meta-learning divergences is that we first optimize the approximate posterior by min- imizing the current divergence, then update the divergence using the feedback from the meta-loss. Formally speaking, for each task T; we perform B gradient descent steps on the variational param- tb using the current divergence D., in the typical VI optimization:
V Pr((e.: T:)-
By doing the updated variational parameters function of the divergence parameters v/. Then we update the divergence parameters by one-step gradient descent using the meta-loss J:
1 7 - n 7 V M Σ Maos: T.)
We call meta learning divergence objective meta- and outline the algorithm in Algorithm
Meta-learning within a-divergence family The parameterization of Renyi's a-divergence is straightforward: = a. As the VR bound (3) is an equivalent optimization objective to Renyi's a-divegrence, it means Vr.Dr, - - Ca. and in practice this gradient is computed using Eq. (p(0, .:) in (q.44) is computed by p(x)))p((O). The direct computation of (q.(4) is beneficial as it continuous in (0, +x)). This is in contrast with automatic differentiation of a-divergences which has numerical issues due to the dis-continuity of a-divergences for certain values (Li & Tumnr 2016; Minka al. 2005).
Meta-learning within f-divergence family We wish to parameterize the f-divergence (5) by parameterizing the convex function using neural network. However, is less straight-forwart to specify the convexity constraint for neural networks Fortunately, the /ddivergence and its gradient can be specified through its second order derivative f" without the original f (Wang et al. 2018a).
Proposition 1. Assume Vo log
p(6) (0)
exists, then
V DyDylllaa) = -E [or (p(ha(e)) (o(h)()))
((0) Voho(c) log
)
(8)
(8)
where gf()) 2
The above proposition implics that we can define the gradient of f-divergence through f" The following proposition guarantees that any is corresponding valid f.
4

---

Under review as a conference paper at ICLR 2020
Proposition 2.
For any non-negative functioon 9 R4, there exists function f such that g(t) ("(t)t?. (t() is strictly positive, i.e. 91 (1) > 0, then (p(plg))) = 0 implies = qs-
See Wang et al. (2018a) for the proofs. Given these guarantees, we propose to parameterize implicitly by parameterizing 9J which can be any non-negative function. We turn the problem into using neural network to express a non-negative function who is strictly positive at = 1. For computational convenience, we further restrict the form of the function to be gy(()
(t) where h (t) is a neural network with parameters 1). This definition of 95 strictly positive for all which clearly satisfies the assumption of Proposition 2 Then by using Eq. (8) to compute the gradient Vo, D,, = Vo,Dr, (see appendix for details), it is clear to see that the f-divergence is learnalle through Algorithm 1
3.2 META- DIVERGENCE OBJECTIVE AND VARIATIONAL PARAMETERS
In addition to learning the divergence objective, we also consider the setting where fast adaptation of the variational parameters new tasks is desirable. Similar MAML, the probabilistic models fpr, (6, }r,}} share the same architecture, and the goal is to learn an initialization of variational parameters @; t Φ. On a specific task, ø is adapted to be Φ; according to the learnable divergence D, (which can be -La or D)
Again the updated Φ; is a function of both and o. Here we simply assume the number of gradient steps to be B and it is straightforward to extend the method to B 1. For meta-update, besides updating divergence parameters v with Eq. we also use the same meta-loss to update &z
サ の M Σ I (qo..
(10)
We call meta-VI VI with learning both the divergence objective and variational parameters' initializa- tion meta- & and summarize the algorithm in Algorithm [ Similar to the previous section, the divergence families consideration are a-divergences and f-divergences.
EXPERIMENTS
We evaluate the proposed meta-VI VI approaches on three tasks: gaussian mixture approximation, Bayesian neural network regression, and recommender system with partial variational auto-encoder. We use marginal log-likelihood as the meta-loss for all experiments except for the Gaussian mix- ture approximation task where we evaluate the method using a variety of meta-losses to directly demonstrate the ability of our method to learn the optimal divergence.
4.1 APPROXIMATE MIXTURE OF GAUSSIANS
We first verify the proposed meta- VI approach can leam good divergence by considering 1-d dis- tribution approximation problem. Each task includes approximating a mixture two Gaussians by Gaussian distribution which is attained by minimizing mins D, Ip). The mixture of Gaussian distribution p(0) (0; μ1, of) + 0.5. μ2, oo) is generated by
#1 Onif(0, 3]
Unif[0.5, 1.0];
- 02 =
Therefore each task has different target distribution but with similar properties (i.e. the distance between two modes is the same and the standard deviation of the second mode is 2 times larger than that of the first mode). The choice of the divergence affects the properties of the approximated Gaussian distribution as shown in Figure
We test the meta-Vi approach with two types of meta-loss: Do.s(qlib (p) and total variation (TV), Do.5(gli2 ()) is the metric we care when evaluating the quality of the approximation & to the target p, then a good divergence will be Do.s (plp( itself. Therefore the goal of testing with meta-loss Do.s is to verify that method is able to learn the preferred divergence given rich enough family of candidate divergences {.,} As in this case the preferred divergence is known, we can directly evaluate the learned divergence by comparing it with the known preferred divergence. In practice,
5

---

Under review as a conference paper at ICLR 2020
Table I: Meta-D on MoG: learned value of a from meta-o and BO. BO with 8 iterations has similar running time as meta-a.
Table 2: Meta-D Peta-Daaa on MoG: value of meta-loss
over 10 test tasks.
Methods
a 0.5
TV
Methods
0.5
TV
ground truth
0.0817+:00027
meta-o
0.52+0.01
0.31
meta-a
meta-f
0.0811
0.0795=10.0301
0.085510.0149
0.0806:10.0163
BO (8 iters)
0.81+00.03
0.69+0008
(8 iters)
0.0833=+0.0289
0.0879=:000433
BO (16 iters)
0.54
0.32+0003
BO (16 iters)
0.0811+000277
0.0855=0.0049
Table 3: Meta-D on MoG: rank of
meta-loss over 10 test tasks.
g= 0.5 matee with mataeoss a= 0.5
0. 1.0
mota with meta Icss TV
Methods a = 0.5
TV
meta-a
2.10:0.70
2.10=0.30
meta-f
2.10
1.00=0.000
BO (8 iters)
3.50=0.67
4.00==0.00
BO (16 iters)
2.3010.90
2.90
Table 4: Meta-Dee D&c on MoG: learned
0.2
1,5
3.2
25
value of a.
Figure 2: Meta-D on MoG: visualizing the learned log f".
Methods = 0.5 TV
meta-o&& 0.88
0.77
the desired evaluation metric for approximation quality (e.g. test log- typically does not belong to a- or f-divergence family; to test this scenario we use TV to evaluate the performance of our method when meta-loss is beyond the divergence family. TV a common distance measure for probability distributions. It is defined as
1 / lp(x) g(x))dr.
TV g) = -uup (p(x) - ((>)) =
2
For a E (0, 1]. TV is related to a-divergence by GTV² - (pllg) Gilardoni 2010).
We first test meta-learning the divergence objective (Algorithm D.. If the divergence set is a- divergence which is parameterized by a scalar, then an alternative approach is to treat the divergence learning task as a hyperparameter search problem and use Bayesian optimization (BO) (Snock al.) 2012) to solve it. Therefore, we use BO as a baseline for meta-a. We note that BO is not appli- cable when the divergence set is f-divergence which is parameterized by a neural network. We set the search region for BO to be a [0.3] which includes many common divergence such as KI, Helinger distance (a = 0.5) and =d-divergence (a = 2). We learn the divergence on M = 10 tasks and set B- = 1. BO is used to find the optimal a value for these 10 tasks (see appendix B.2 for more details about BO).
In Table I we report the learned value of from meta-c and BO. When the meta-loss is Do.s D (gll the learned a from meta-a is very close to 0.5 which demonstrates that our method can essentially learn good a. On the other hand, BO is less computationally efficient, as it needs to train model from scratch every single time when evaluating a new value of a, while our method can update a based the current model. We also consider learning f f-divergence and visualize in Figure 2 the learned log f" When Do.s(gllp) [p) is in
the meta-loss, the corresponding log for Do.s (ol(p))p)) analytical, and we see from Figure 2 a) that the learned log f" and log f + 0.8 are almost identical. This means meta-V VI has learned the optimal divergence Do.s(q (p) (for any positive a, (()) and a af(()) define the same divergence).
In the case of using TV as the meta-loss, the optimal divergence is not analytic. Therefore, we instead report in Table 2 the meta-losses on 10 test tasks, which are obtained by executing the learned divergence minimization algorithm for 2000 iterations. The error bar is large due to the large variance among different tasks, so we also report the ranking in Table 3 It clearly shows that meta-a and meta-f f are superior over BO. Moreover, meta- f outperforms meta-a when the meta- loss is TV. From Figure (b), we can see that the learned f-divergence is not inside a-divergence, showing the benefit of using larger divergence family.
Next we test meta-learning both the divergence objective and the variational parameters (Algorithm
2). We use Algorithm ? without updating divergence baseline, denoting by VB&o. During

---

Under review as a conference paper at ICLR 2020
0.16
0.16
0.14
0.14
mela-a5. o
meta-a&..
0.12
0.12
mela-/&pp
0.10
0.10
VB&$
VB&
0.08
0.08
0.08
0.08
0.04
0.04
0.02
0.2
0.00
-2
2
8
10
-2
0
4
10
(a) Meta-loss: a 0.5
(b) Meta-loss: TV
Figure 3: Meta- Meta-D&o on MoG: visualization of approximate distribution after 20 updates.
training, we sample 10 tasks each time and perform B = 20 inner loop gradient update. The learned is different from Table [ (see Table 4 We conjecture that this is related to the learned e and the horizon length. During meta-testing, we use the learned & for variational parameter initialization, and train the variational parameters with the learned divergence for 20 and 100 iterations respectively to evaluate the effect of the learned divergence in short and long horizon. We summarize the meta- loss in Table 5 and the ranking in Table [6] Our method are not only better than VB& after 20 updates but also better after 100 updates. This demonstrate the benefit of learning divergence for the tasks instead of the conventional VB. To further explore the reason of getting lower meta-loss of me we visualize the approximate distribution of all methods after 20 steps in Figure] 3 The approximate distributions obtained by meta- D& tends to the mixture of Gaussians more g globally (mass-covering) than VB& This mass-covering behaviour result in better meta-loss. Compared to learning divergence only, learning variational parameter initialization helps shorten the training time on new tasks (100 iterations v.s. 2000 iterations). Notably, meta- VI is able provide this initialization along with divergence learning without extra cost.
Table 5: Meta-D&ø on MoG: value of meta-loss over 10 test tasks.
Methods Meta-loss
0.5 (20 iters)
TV (20 iters)
a = 0.5 (IOD iters)
TV (t00s)ers)
meta-o&zó
0.2007=000000 0.0793=+0.0377 0.123710.0539
0.0982++00016
0.0879=000300 0.0784=0.0333
0.0903+0.0049
f&
0.0935+0.0152
0.0918+000151
VB&
0.1026110.0181
0.0905:10.0332
0.092610000133
Table 6: Meta-D&o on MoG: rank of meta-loss over 10 test tasks.
Methods Meta-loss
a 0.5 (20 iters)
TV (20 iters)
0.5 (100 iters)
TV (100 iters)
0.10::.554 1.20=10.60
1.80=#0.00
2.20::0.75
1.40=0.66
meta- f &oo
1.50
1.40=10.80
2.10=0.83
VB-MAML
2.70+10.46
2.70:10.46
2.40+10.49
2.50=0.50
4.2 REGRESSION TASKS WITH BAYESIAN NEURAL NETWORKS
The second test considers Bayesian neural network regression. The distribution of ground truth regression function is defined by a sinusoid function with heteroskedastic noise (which a function rf see Figure 4 (a))
A sin sin(x++)) (x+))++/+++)) cos (cz+))))) where the amplitude A E [5, 10), the phase E [0, 1 and (N 1). The heteroskedastic noise makes the uncertainty estimate crucial when compared with the sinusoid function fitting task in Finn et al. (2017): Kim et al. (2018). The model is a two-layer neural network with hidden layer size 20 and RELO nonlinearities.
For meta-learning divergence only, the training set size is 1000 and is obtained by sampling x e [-4, 4) uniformly. We use M 20, = 1, K 50 and batch size 40 of which 20 data points are for updating Φ (q and 20 points are for updating 77. We train meta-D for 1500 epochs. To evaluate the performance, we train the model with the learned divergence and VB respectively on new tasks for 1000 epochs. The quantitative results are summarized in Table 7 We can see that the test log-likelike likeliooddo of both meta-a and meta-f ff are significantly better than VB and the root mean square error (RMSE) are similar for all methods. We visualize the predictive distribution an example sinusoid function in Figure 4 All methods fit the mean well which is consistent with the RMSE results. However, VB fails capture the heteroskedastic uncertainty and instead used
7

---

Under review as a conference paper at ICLR 2020
Table Meta-D on Sin: results are over 10 test tasks (1000 epochs).
Table 8: Meta- on Sin: results are over 10 test tasks (500 epochs).
Test LL
RMSE
Test LL
RMSE
VB
0,4522:0.0196 -.66377=0.0433
VB& meta-od20 meta-
-0.6354+000059 -.44967=000647 -0.4852+0008533
0.4556=000247
meta-a
-0.859610.0887
0.4500110.0233 0.4599::0.0200
0,4562:10.0207
m
-0.4390-10.10
0,4552-0.0217
-10
-10
-50
(a) Ground Truth
(b) VB
(c) meta-a
(d) meta-
Figure 4: Meta- D on sin: the predictive distribution on sinusoid wave.
homoskedastic noise to the data. On the other hand, meta-a and f can reason about the heteroskedastic noise. This explains the results of better test log-likelihood.
For learning both divergence and variational parameters initialization, we sample 20 tasks where each task has 40 data points. We use 20 points for (9) and the other 20 points for updating divergence Eq. and the shared initialization ø (q.(1O) We set B = 1. To evaluate, we start with the learned initialization and train the variational parameters with the learned divergence for 500 epochs. Similar to the results of learning only the divergence objective, meta-a&zó and meta- f&oo are able model heteroskedastic predictive distribution while &B& cannot. The quantitative evaluation are given in Table 8 and an example of predictive distribution is given in Figure 0 (see appendix). Meta- Meta-D&z converges faster than meta- D, indicating that learning model initialization can shorten the training time new tasks.
4.3
RECOMMENDER SYSTEM WITH PARTIAL VARIATIONAL AUTO-ENCODERS
We test method rec- ommender systems with Par- tial Variational Auto-encoders (p-VAEs). P-VAE is a recently proposed model deal with partially observed data and has been used to do user rating pre- diction in recommender system (Ma et al. 201 Simi- lar to vanilla VAE (Kingma & Welling 2013), p-VAE uses the Kl--dvergence the variational objective. We apply our pro- posed method to the divergence objective in p- PAE.
-135
-15
-1.36
mata-d-p-VAF
-30
meta-oli
-141
-140
mela-f-p-VAE
P-VAE
-35
MAML P-VAE
-1422
50
100
150
200
250
300
40
69
80
100
Epoch
Iteration
(a) Meta-D
(b) Meta-D&zó
Figure 5: Test likelihod of meta- VI on MovieLens. (b) The final results of meta-a& meta- & and HAML++-VAB are -1.3855, -1.3985 and -1 1..140 respectively.
We consider MovieLens IM dataset (Harper & Konstan 2016) which contains 1,000,206 ratings of 3,952 movies from 6,040 users. We split the users into seven age groups: under 18, 18-24, 25-34, 35-44, 45-49, 50-55 and above 56, and regard predicting the ratings of users within the same age group task since the users with similar age may have similar preferences. We select four as training tasks (under 18, 25-34, 45-49, above 56) and use the remaining as test tasks.
For the setting of learning divergence only, during meta-training, we sample 100 users per task (400 users total) and use half of the observed ratings to compute (q. and the other half for computing the meta-loss. The number of training epochs is 400. During meta-testing, we use 90%/10% training-tess split for the three test tasks and train p- VAE with the learned divergence. The baseline p- VAE is directly trained on test tasks with KL-divergence. From Figure 5 (a), can see that the combination of meta-D D and p- VAE outperforms vanilla p- p-VAE terms of test log- likelihood, showing that meta-D has learn suitable divergence that leads to better test performance.
8

---

Under review as a conference paper at ICLR 2020
For learning both divergence and variational parameters, the setup of training is the same as learning divergence only except that now we also perform updates in Eq.(9). We compare our method with getting a p- VAE model initialization only (obtained by Algorithm 2 withhut updating 11). This can be regarded as a combination of MAMI. and P-VAR. During evaluation, we apply 60%/40% training- test split for the test tasks and train the learned p- p-VAE model with learned divergence. Figure (b) implies that all methods can converge quickly on the new task with only 100 iterations. Both meta- & and meta- && are better than P-VAE at the beginning, indicating that the learned divergence can help fast adaptation. Besides, meta-a&tó and meta- f& also converge better than p- p-VAE in the end. This shows the learned divergence helps in both short and long horizon.
5 RELATED WORK
Variational Inference Variational inference (VT) has advanced rapidly in recent years (Zhang iet al. (2018). Several works have introduced a new divergence family for VI (Li & Turner 2016 al., 2016 Bamler et al. 2017). Another line of work improve the importance sampling estimate of model evidence by increasing the number of importance sampling particles (Burda et al. 2015) or increasing the signal-to gegnal-o-noose ratio of gradient estimate (Rainforth et al.,
Hemanndzz-Lotee
2018). Stochastic optimization methods have also been deployed to scale up VI (Hoffman et al., 2013 Li al. 2015 Dehaene & Barthelmé 2018). Our work related to the work that improves the variational objective with alternative divergence measures; the difference is that divergence measure is learnable and can be selected an automatic fashion for a certain type of tasks.
Meta-Leaaminngfeew-shho learning Recent work has applied Bayesian modelling techniques to enhance uncertainty estimate for meta-learningffew-shhe learning (Grant et al.] 2018: Finn et al,,
2018 Kim al. 2018 Ravi & Beatson 2018). They regard the framework of MAML (Finn al.
2017 as hierarchical Bayes and conduct Bayesian inference on meta-parameters and/or task-specific parameters. Grant al. (2018) : Kim et al. (2018) applied Bayesian inference task-specific param- eters with Laplacian approximation and Stein variational gradient descent, respectively. |Finn et al. (2018) instead approximated the exact posterior over meta- mnea--arameters using variational inference but still kept point estimate for task-specific parameters. RRavi & Beatson (2018) obtain posteriors over both meta-pane peramparrseters and task-specific parameters with variational inference. Our focus is distinct from this line work in that our research is the opposite direction: leveraging the idea of meta-learning to advance Bayesian inference. Additionally, our meta- D&o without learning diver- gence can be regarded as a different Bayesian MAML method: we do not follow hierarchical Bayes but directly train variational parameters so that it can quickly adapt to new tasks.
Meta-Learning for loss functions Our meta-learning method is also related to meta-learning loss function. In reinforcement learning, Houthooft et al. (2018) meta-learned the loss function for updating policy. Xu al. (2018) heta-learned the value function to interact with the environment. Our work extends the idea of learnable loss function to Bayesian inference.
Meta-Learning for Bayesian inference algorithms A recent attempt to meta-learning stochastic gradient MCMC (SG-MCMC) is presented by Gong et al. (2018), which proposed to meta-learn the diffusion and curl matrices of the SG-MCMC's underlying stochastic differential equation. Also Wang et al. 201 applied meta-learning build efficient and generalizable block-Gibbe sampling proposals. Our work is distinct from previous work in that we apply meta-learning to improve VI, which is more scalable inference method than MCMC. To the best our knowledge, are the first to study the automatic choice and design of VI inference algorithms.
6 CONCLUSION
We propose meta-VI which automates the choice of divergence objective in VI via meta-learning. further allows meta-learning of variational parameter initialization for fast adaptation on new tasks. Within the framework of meta-VI, we consider two divergence families, a-divergence and f-divergence, and design parameterizations of divergences to enable learning via gradient descent. Experimental results on Gaussian mixture approximation, regression with Bayesian neural networks and recommender systems demonstrate the improvement of meta-Vi VI over vanilla VI, which shows the benefits of learning suitable divergence measure tailored to the specific tasks at hand.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Shun-ichi Amari.
Differential geeoetrrcl methods in statistics, volume 28. Springer Science & Business Media, 2012.
Robert Bamler, Cheng Zhang, Manfred Opper, and Stephan Mandt. Perturbative black box varia- tional inference. In Advances in Neural Information Processing Systems, pp. 5079-5088, 2017.
Christopher M Bishop. Puttern recognition and machine learning. Springer Science+ Business Media, 2006.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: review for statisti- cians. Journal of the American Statistical Association, 112(518):559-57) 2017.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. arXiv preprint arXiv:1509.005: 2015.
Imre Csiszár, Paul C Shields, et al. Information theory and statistics: A tutorial. Foundations and Trends()) Communications and Information Theory, 144::417-52, 2004.
Guillaume Dehaene and Simon Barthelmé. Expectation propagation in the large data limit. Journal of the Royal Statistical Sociery: Series B (Statistical Methodology), 80(1):199-25 111999217,4 2018.
Stefan Depeweg, José Miguel Hernández-Lobend Finale Doshi-Velez, and Steffen Udluft. Learning and policy search in stochastic dynamical systems with bayesian neural networks. arXiv preprint arXiv:1605.07127. 2016.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 1136-1135. JMLR. org, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. In Ad- vances Neural Information Processing Systems, pp. 9516-9527, 2018.
Gustavo I. Gilardoni. On pinsker's and vajda's type inequalities for csiszár's f-divergences. IEEE Transactions on Information' Theory, 56(1 1):377755386, 2010.
Wenbo Gong, Yingzhen Li, and José Miguel Hernández-Lobato. Meta-learning for stochastic gra- dient memc. arXiv preprint arXiv: 046.0.252., 2018.
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- based meta-learnigg hierarchical bayes. arXiv preprint mnXiv:1801.058 2018.
F Maxwell Harper and Joseph Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5(4):19, 2016.
José Miguel Hemmnndez-Lobtte Yingzhen Li, Mark Rowland, Daniel Hermnndez-Lobato, Thang Bui, and Richard Turner. Black-box a-divergence minimization. 2016.
Sepp Hochreiter, A Steven Younger, and Peter R Conwell. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, pp. 87-94. Springer, 2001.
Matthew D Hoffman, David M Blei, Chong Wang, and John Paisley. Stochastic variational infer- ence. The Journal of Machine Learning Research, 14(1):1303 303-133733 2013.
Rein Houthooft, Yuhua Chen, Phillip Isola, Bradly Stadie, Filip Wolski, OpenAl Jonathan Ho, and Picter Abbeel. Evolved policy gradients. In Advances in Neural Information Processing Systems, pp. 5400-5409, 2018.
Michael Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul. An introduction to variational methods for graphical models. Machine learning, 37(2))183 133-233, 1999.
Taesup Kim, Jaesik Yoon, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian model-agnostic meta-learning. arXiv preprint arXiv: (alo::806603838 2018.
10

---

Under review as a conference paper at ICLR 2020
Diederik P Kingma and Max Welling. arxiv:1312.611,, 2013.
Auto-engoding variational bayes.
arXiv preprint
Yingzhen Li and Richard E Turner. Rényi divergence variational inference. In Advances in Neural Information Processing Systems, pp. 1073-1081, 2016.
Yingzhen Li, José Miguel Hemández--obatt and Richard E Turner. Stochastic expectation propa-
gation. In Advances in neural information processing systems, pp. 2323-2331,
Chao Ma, Wenbo Gong, José Miguel Hernández- Lobatto Noam Koenigstein, Sebastian Nowozin, and Cheng Zhang. Partial vae for hybrid recommender system. In NIPS Workshop on Bayesian Deep Learning, 2018a.
Chao Ma, Sebastian Tschiatschek, Konstantina Palla, Jose Miguel Hernandez Lobato, Sebastian Nowozin, and Cheng Zhang. Eddi: Efficient dynamic discovery of high-value information with partial vac. arXiv preprint arXiv:1809.1 1142, 2018b.
Thomas Minka. Expectation propagation for approximate bayesian inference. In Proceedings of the Seventeenth conference on Uncertuinty in artificial intelligence, pp. 362-369, Morgan Kauf- mann Publishers Inc., 2001.
Tom Minka et al. Divergence measures and message passing. Technical report, Technical report, Microsoft Research, 2005.
Devang Naik and RJ Mammone. Meta-neural networks that learn by learning. In /Proceedings 1992] IJCNN International Joint Conference on Neural Networks, volume 1, pp. 437-442. IEEE, 1992.
Tom Rainforth, Adam R Kosiorek, Tuan Anh Le, Chris J Maddison, Maximilian Igl, Frank Wood, and Yee Whye Teh. Tighter variational bounds are not necessarily better. arXiv preprint WcXiv:1802.04537 2018
Sachin Ravi and Alex Beatson. Amortized bayesian meta-learning. 2018.
Alfréd Rényi et al. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions the Theory of Statistics. The Regents of the University of California, 1961.
Tim Salimans, David A Knowles, et al. Fixed-form variational posterior approximation through stochastic linear regression. Bayesian Analysis, 8(4):837-882, 2013.
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems, pp. 2951-2959, 2012.
Sebastian Thrun and Lorien Pratt. Learning to learn. Springer Science & Business Media, 2012.
Constantino Tsaliis. Possible generalization of boltzmann-gibbs statisties. Journal of statistical physics, 52(1-2):479-45 1988.
Dilin Wang, Hao Liu, and Qiang Liu. Variational inference with tail-adaptive f-divergence. In Advances in Neural Information Processing Systems, pp. 5737-5747, 2018a.
Tongzhou Wang, Yi Wu, Dave Moore, and Stuart J Russell. Meta-learning meme proposals. In Advances in Neural Information Processing Systems, pp. 4146-4156. 2018b.
Zhongwen Xu, Hado P van Hasselt, and David Silver. Meta-gradient reinforcement learning. In Advances in neural information processing systems, pp. 2396 2407, 2018
Cheng Zhang, Judith Butepage, Hedvig Kjellstrom, and Stephan Mandt. Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence, 2018.
11

---

Under review as a conference paper at ICLR 2020
A COMPUTING EQUATION (8) IN PRACTICE
With dataset D, the density ratio in f-divergence becomes D(0 p(D
We esti-
(0( 4o (1)p(D))
mate pl DD) through importance sampling and MC approximation: p(D) = Es-p(o) [p() =
(0))
p(D(0)p(0)
p(D(0))p(0k) where 0g go(0). After doing this, the density ratio Σ
(
9o(O%)
becomes p( (0x) pp(O0))p(0n) (e()
Σ
p(116))((0)) which can be regarded self-normalize (e(0k)
estimator, similar to the normalization importance weight in & Turner (2016). A self-normalized estimator generally helps stabilize the training especially at the beginning. We use this estimator for regression tasks and recommender system.
B ADDITIONAL EXPERIMENTAL RESULTS AND SETTING DETAILS
B.1 MODEL ARCHITECTURE FOR f-DIVERGENCE
On all experiments, we parameterize g(() in f-divergence by neural network with 2 hidden layers with 100 hidden units and RELU nonlincarilities.
B.2 APPROXIMATE MIXTURE OF GAUSSIANS
The expectation in Eq. and 8 is computed by MC approximation with 1000 particles. Note that p(0) is computable, since we know the parameters of p.
Bayesian optimization is implemented through public package The acquisition function is the upper confidence bound with kappa 0. We used the same data of the training tasks for BO. Specif- ically, the objective function that BO wants to minimize is the meta-loss (Du.s or TV). Every time BO selects an a, we train 10 models with that a-divergence on the support sets of 10 training tasks respectively and get the mean of log-likelihood the query sets of the 10 training tasks. Each time the model is trained for 2000 iterations.
When Do.5 in use as the meta-loss, ideally the learned f-divergence should be close to Do.5. When 10.6
the f-divergence is Do.5, the f function is f(t) = 0055 and the analytical expression of log f -1.5 log & + C with C reflecting the scaling constant in f. Figure 2, we compare the learned log (t) and the ground truth - 1.5 log t C. We found that the learned log (t) is very close to -1. log + 0.8. This means that our method has learned the optimal divergence Do.5 (because the definition of f-divergence is invariant to constant scaling of the function i.e. and e f define the same divergence).
B.3 REGRESSION TASKS WITH BAYESIAN NEURAL NETWORKS
We provide the learned value of from meta-a and meta-a&zó in Table 9 The predictive distribu- tions on an example test task are given in Fig. 66] Similar to the results of meta-D, meta-D&óó is also able model heteroskedastic noise while VB& cannot. We have tried BO this task and the later recommender system task but it appears to be much more inefficient than our methods. Given the similar runtime as our methods, BO can only conduct two search resulting in bad value of a. The results of BO are worse than KL-divergence, therefore ignored in the paper.
Table 9: Learned value of a of meta-a and mo0 on sinusoid regression.
Table 10: Learned value of a of meta-a and meta-o&tó on Moviel.nn.
meta-a meta-a&tó
meta-a meta-a&có
0.6666
0.20
0.9029
1.0602
12

---

Under review as a conference paper at ICLR 2020
Oiseration
(a) Ground Truth
(b) VB&
(c) meta-a&o
(d) meta- f&o
Figure 6: Meta-D&oó on sin: Predictive distribution on sinusoid wave.
B.4 RECOMMENDER SYSTEM
Again we provide the value learned or from meta-a and meta-a&zó in Table 10 Besides the test log-likelihood, there are other popular evaluation metric being used in recommender system and sometimes they are not consistent with each other. Therefore, our method in terms of other common metrics: test root mean square eπror (RMSE) and test mean absolute error (MAE). For both metries, our methods converge better than the baseline in the setting of learning inference algorithm and the setting of learning inference algorithm and model parameters (see Figure and S).
also evaluate the performance of
1.125
1.100
meta meta-a+p-Vel a+P-VAE
0.900
meta-&+p-VAE meta-f+ P-VAE
1.075
meta meta-ftp-VAl FAP-VAE
0.875
p-VAE
p-VAE
1.050
1.025
0.825
1.000
0.800
0.975
0.775
0.990
0.925
50
1.00
150
2.00
250
300
50
100
150
200
250
300
Epach
Epoch
Figure 7: Meta- D on ML: Comparison of meta-D D and p- VAV in terms of test RMSE and test MAE.
meta-a$.e
3.5
meta-oGu
meta &
meta flup
3.0
MAML P-VAE
MAML- + P-VAE
RMSE
2.5
2.0
1.0
20
60
80
100
20
40
60
100
Iteration
Itaration
Figure 8: Meta-D&o on ML: Comparison of meta-D&e and MAML+--VAE in terms of test RMSE and test MAE.
C VAE ON MNIST
We added an additional experiment on the real-word dataset, MNIST, to further demonstrate the effectiveness of our methods. Specifically, we trained variational auto-encoder (VAE) on MNIST and replaced the KL divergence VAE by our learned a-divergence. Each digit is regarded as task and we let the first 5 digits (0-4) to be the training tasks and the last 5 digits (5-9) to be the test tasks. For learning divergence only, during meta-training, we sample 128 images each task and use half of the images to compute (6) and the other half for computing the meta-loss. The number of training epochs is 600. During meta-testing, we train VAE with the learned divergence on the training set of the test tasks for 300 epochs and compute the marginal log-likehood on the test set of
13

---

Under review as a conference paper at ICLR 2020
Table 11: Meta-a on MNIST: marginal log- on 5 test tasks.
Digit 5
6
7
8
9
VAE -13..94
-121.74
-92.20
-145.32
-120.55
m VAE
-133.59
-120.86
-92.06
-144.84
-120.24
Table 12: Meta-a&zo on MNIST: marginal log- likkliihkhodd on 5 test tasks.
Digit 5
VAE MAML
-139.06
-129.56
-99.76
-149.76
-124,91
meta-a&c -134.75
-124.35
-92.71
-145.99
-11.774
Table 13: Learned value of of meta-o and meta-o& on MNIST.
meta-a
meta-a&tó
a 0.15
0.97
the test tasks. B 1 and K 10. The baseline VAE means the standard VAE with KL-divergence. Similar to the previous tasks, we have considered BO on this task but found that it is very inefficient. For meta-learning both divergence and the model initialization, the setup of training is the as learning divergence except that now all tasks share the same model initialization. We compare our method with getting a VAE model initialization only (obtained by A This can be regarded as a combination of MAML and VAE. During evaluation, we train the VAE with the learned divergence and the model initialization on the training set of the tasks for 200 epochs and evaluate the marginal logeliieldoood on the test set.
updating 11).
We use the same architecture (100 hidden units and 3 latent variables) and the marginal log- likelihood estimator as in Kingma & Welling (2013).
We report the test marginal log log-likelihood for each test digit in Table align with other experiments that the meta-o: and meta-o& are both better than their counterparts. Meta-a and meta-o&c are better than the vanilla VAE on all test tasks, indicating our methods have learned suitable divergence.
TTble and 12 Overall, these results
14


Reviews:
[{"review_id": "rkxIPeBQ5H", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary of the paper:\nThis paper proposes to meta-learn a parametric divergence measure for variational inference (VI). Specifically, the paper focuses on alpha-divergence and f-divergence, aims at choosing a good alpha or f for a particular task leveraging the experience learned from previous tasks. The meta-learned divergence is applied to approximate a mixture of two Gaussians, regress sinusoid function with Bayesian neural networks, and learn recommender systems with partial VAE. \n\nSummary of my opinion:\nI am leaning towards reject this paper because \n1) I'm not convinced by their motivation -- \"this line of work has also shown that the optimal divergence can vary depending on tasks\". There are many key factors to achieve good performance in VI, such as a good likelihood model, a good variational posterior, a good optimizer and so on. The divergence itself, in my opinion, is less significant. If you really care about the flexibility, the Wasserstein distance may be a better choice.\n2) Choosing the alpha or f is a hyper-parameter search problem to me. What you need to do is to prepare a validation set and select the best hyper-parameters based on the performance there. It is impractical to collect M tasks and search for hyper-parameters according to some meta-losses. \n3) The experimental part is weak -- two toy problems plus an unusual recommendation system problem is not the typical way that people choose to evaluate VI methods. I suggest to evaluate on standard VAE/BNN benchmarks. \n\nMajor comments: \n1. From eq.(4) and eq.(8), it seems one has to compute the value of the density function p(theta, D) in order to compute the gradients, which is obviously not possible for BNNs and VAEs. More details on how to perform VI with alpha-divergence and f-divergence should be covered. \n2. Can you let the meta-loss to be equal to D_eta? Meta-learning algorithms often optimize the same loss function in the inner and outer level. I couldn't see the point why you want to learn a f-divergence while setting the meta-loss to be D_0.5. \n3. Do you have a validation set for the Bayesian optimization (BO) baseline or do you use cross-validation? The details of this baseline should be elaborated. In fact, it is possible to choose the best alpha for each task using BO. Should this be compared?\n4. What does VB mean in Section 4.2? If VB uses KL rather than alpha-divergence or f-divergence, the outperformance may only suggest that KL is insufficient there. \n5. Have you considered input convex neural networks (Amos et al. 2016) for implementing f? \n\nMinor comments:\n1. The proofs of Prop 1 & 2 are missing.   \n2. \"However, using this KL divergence for VI has been criticized for under-estimating the uncertainty...\" Any reference to this?\n3. Figure 1: alpha is used before it is defined. \n4. \"Other existing definitions of \u000balpha-divergences have dis-continuous gradients at the alpha values corresponding to KL divergences\" -- missing reference.\n5. \"When D_0.5 is in use as the meta-loss, the corresponding log f*'' is analytical\" -- could you elaborate on this? \n\n------------------------------\nAfter rebuttal:\nI would like to thank the authors for their extremely detailed responses! The paper is now greatly improved. \nI would like to increase my score from 3 to 5, however, there is no such option this year. \nThe reason why I am still hesitated to accept this paper is mainly because I am not sure how useful learning the divergence to VI is. For unsupervised learning, people use GAN or VQ-VAE to obtain realistic samples, but which is not attributed to better divergence measures. Plus, what we refer to \"better\" depends on the supplied meta-loss, which may not be better at all. Another unsatisfactory point of this paper in my opinion is that it only demonstrated results on toy datasets and MNIST. Although the proposed method is valid for alpha-divergence and f-divergence, I suggest the authors to find a real application which demonstrates the merit of the idea in practice. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "SkgtKpvSYS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use a meta-learning approach to learn the divergence used in variational inference and initial variational parameters. It considers two families of learnable divergences, including the alpha-divergence and f-divergence, and proposes two double loop algorithms. In both algorithms, the inner loop adjusts the variational parameters for a specific variational inference task. The outer loop adjusts the parameters in the divergence (and perhaps an initial variational parameter for all tasks) in terms of a human-designed meta-loss. Proposition 1 (following Wang et al. 2018a) shows that the gradients with respect to the meta-parameters in the f-divergence can be obtained using f''. This makes it convenient to parameterize f'' by neural networks to implicitly model f.\n\nThe experiments show that the proposed method can outperform a Bayesian optimization (BO) baseline if the meta-loss is the metric of interest in the MoG example. For the regression and recommendation system examples, it compares with a VB baseline and a p-VAE baseline and shows better results.\n\nAs for the significance of the paper, I have the following issues:\n\n1. To apply the proposed method, we need to have a family of tasks that shares similarity at hand, which seems to be restricted. For instance, the first two examples are synthesized by hand. The recommendation system example is from real life but also crafted by splitting groups by age. If we just use all of the data by a p-VAE and fine-tune the hyperparameters a little bit (meta-learning needs more time than a p-VAE in a single configuration), can we obtain better results? If so, the last example is not appealing.\n\n2. To apply the proposed method, we need to have some knowledge about our preference in the evaluation and encode that knowledge into the meta-loss. In the first example, the comparison with BO in the alpha=.5 case is unfair because Meta-D uses this knowledge as the meta-loss while BO does not leverage such information. Besides, if we already know the preferred divergence, it is not necessary to learn that divergence. For the other settings where Meta-D does not leverage any knowledge about the evaluation preference, it lacks stronger baselines such as BO.\n\n3. The idea is straightforward and the most challenging part of how to model a convex function by a neural network is solved by existing work.\n\nBased on these issues, I think the contribution of the paper is not sufficient and I tend to reject the paper. Also, note that the paper length extends 8 pages.\n\nThe motivation should be strengthened and the experiments should be more precise to improve the paper. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HygQloVRtS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "SUMMARY OF THE PAPER:\nThis paper proposes a way to automatically select a divergence to use in variational inference (VI) given a set of datasets (tasks). They consider searching within the alpha-\ndivergence and f-divergence families. The proposed algorithm works as follows: do a few gradient descent steps on the variational parameters given a fixed divergence parameter, then update the divergence parameter by taking the gradient with respect to a meta loss which is a task-specific measure of goodness of the variational distribution (like test log marginal likelihood). The latter gradient is computed through the gradient descent computation in the inner loop.\nThe second proposed algorithm does the same, but also learns a good initialization for the variational parameters. This initialization is good in the sense that taking one (or a few) gradient descent step on the variational parameters should give us good variational parameters (MAML style). This is done by taking a gradient with respect to this good initialization parameter through the inner loop gradient descent.\n\nSTRUCTURE:\nThe paper is well-written and easy to understand.\n\nNOVELTY:\nAs far as I know, learning a divergence for VI using meta-learning is new. The related work is discussed well in section 5.\n\nEXPERIMENTS:\nThere are experiments on three tasks of increasing complexity. In the simpler experiments, careful ablation studies are done. The results generally show that the proposed method is preferable to alternatives.\n\nCONCLUSION:\nI recommend acceptance.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a meta-learning algorithm to learn the divergence measure of variational inference as well as the initialization of the variational parameters (which reduces optimization steps of VI). Improved performance by the learned divergence against hand-designed ones are empirically shown on: Gaussian mixture approximation, Bayesian neural regression, and p-VAE based recommender.
Reviewers initally raised some concerns on hyperparameters selection, weakness of experiments, and motivation for the proposed scheme. The authors responded by adding additional experiments (MNIST) as well as some new sections in their appendix about details of their method or the baselines.
The reviewers greatly appreciated the response and commonly believed that the revised version is significantly improved over the initial draft  and the improvements of the draft. As a result of that, some reviewers increased their scores. However, some of their concerns did not resolve. In particular, R1 questions the impact of the work and importance of learning divergence measure (referring to GAN or VQ-VAE for obtaining realistic samples). Also R1 finds evaluation based on MNIST unsatisfactory, as it is commonly considered as a toy dataset. To motivate the method, it is suggested that the authors think about real applications which can highlight the benefits of their method in practice. Similar concerns are shared by R2 after authors' response. In particular, R2 is not convinced about motivation and the necessity of using meta-learning for learning the divergence. I suggest authors improve on issues around motivation and support the impact of their scheme in a more practical setting.
Decision: Reject