Bayesian Meta Sampling for Fast Uncertainty Adaptation
Authors: Zhenyi Wang
Yang Zhao
Ping Yu
Ruiyi Zhang
Changyou Chen
Venue:  (2020)
Keywords: Bayesian Sampling
Uncertainty Adaptation
Meta Learning
Variational Inference

Abstract: Meta learning has been making impressive progress for fast model adaptation. However, limited work has been done on learning fast uncertainty adaption for Bayesian modeling. In this paper, we propose to achieve the goal by placing meta learning on the space of probability measures, inducing the concept of meta sampling for fast uncertainty adaption. Specifically, we propose a Bayesian meta sampling framework consisting of two main components: a meta sampler and a sample adapter. The meta sampler is constructed by adopting a neural-inverse-autoregressive-flow (NIAF) structure, a variant of the recently proposed neural autoregressive flows, to efficiently generate meta samples to be adapted. The sample adapter moves meta samples to task-specific samples, based on a newly proposed and general Bayesian sampling technique, called optimal-transport Bayesian sampling. The combination of the two components allows a simple learning procedure for the
meta sampler to be developed, which can be efficiently optimized via standard back-propagation. Extensive experimental results demonstrate the efficiency and effectiveness of the proposed framework, obtaining better sample quality and faster
uncertainty adaption compared to related methods.

Published as a conference paper at ICLR 2020
BAYESIAN META SAMPLING FOR FAST UNCERTAINTY ADAPTION
Zhenyi Wang ', Yang Zhao Ping Yu / Ruiyi Zhang Changyou Chen
1 State University of New York at Buffalo
Duke University
( (zhenyiwa, yzhao63, pingyu, changyou] @buffalo.eds
2 ryzhangecs. duke.edu
ABSTRACT
Meta learning has been making impressive progress for fast model adaptation. How- ever, limited work has been done on learning fast uncertainty adaption for Bayesian modeling. In this paper, we propose to achieve the goal by placing meta learning on the space of probability measures, inducing the concept of meta sampling for fast uncertainty adaption. Specifically, we propose a Bayesian meta sampling frame- work consisting of two main components: a meta sampler and sample adapter. The meta sampler is constructed by adopting a neural-inverse wtwaaliieeee-etteeggeeeeee -wt-ararrrssiieeeeiieeeee floww (NIAF) structure, variant of the recently proposed neural autoregressive flows, efficiently generate meta samples to be adapted. The sample adapter then pushes meta samples to task-specific samples, based on a newly proposed and general Bayesian sampling technique, called optimal-transpaa combination of the two components allows a simple learning procedure for the meta sampler to be developed, which can be efficiently optimized via standard back-proper propc-prppagtionn Extensive experimental results demonstrate the efficiency and effectiveness of the proposed framework, obtaining better sample quality and faster uncertainty adaption compared to related methods.
Bayesian sampling. The
INTRODUCTION
Meta learning (Schmidhuber, 1987 Andrychowicz et al. 2016) is an important topic in modern machine learning. The goal is to learn some abstract concepts from different but related tasks, which can then be adapted and generalized to new tasks and environments that have never been encountered during training. There has been lots of research on this topic. A recent review classifies the methods as metric-based, model-based and optimization-ba Bearnnng-to-leam seeks to learn a meta optimizer that can be applied to different models, with some task-specific information such as current gradients as input (Andrychowicz et al.] 2016). Model agnostic meta learning (MAML) aims to learn a meta paraneeeer/nodel from set of training tasks such that it can quickly adapt to models for new tasks (Finn et al., 2017). Many follow-up works have been proposed recently, including but not limited to the meta network (Munkhdalai & Yu! 2017). the meta learner (Ravi & Larochelle, (017), the Reptile model (Nichol et 2018)) and the lately
methods (Weng) 2018). Among these methods,
extensions an online setting (Finn et al. 2019). model hierarchical relation (Yao et 2019)
and sequential strategies (Ortega et al. 2019) and to its stable version Antoniou et al. (0019) and to
some theoretical analysis (Khodak et al. 2019).
It is worth noting that all the aforementioned models are designed from an optimization perspective. Bayesian modeling, in parallel with optimization, has also been gaining increasing attention and found various applications in deep learning. Recent research has extended the above meta-learning methods to Bayesian setting. For example, Bayesian MAML. (BMAML) replaces the stochastic- gradient-descen (SGD) step with Stein variational gradient descent (SVGD) for posterior sampling (Yoon et al. 2018). Probabilistic MAML (PMAML) extends standard MAML by incorporating a parameter distribution of the adapted model trained via variational lower bound (Finn et al. 201 8).
Amortized Bayesian Meta Learning extends the idea of MAML to amortized variational inference (Ravi & Beatson 2019 Choi et al. 2019). VERSA (Gordon et al.) 2019) uses amortization network to approximate the posterior predictive distributions. Meta particle flow (Chen et al.) 2019)
realizes Bayes's rule based on ODE neural operator that can be trained in meta-learning framework.

---

Published as a conference paper at ICLR 2020
Though methodologically elegant with many interesting applications, the above methods lack the ability to uncertainty propagation/adaation, in the sense that uncertainty is either not considered (e.g., in MAML) or only considered in the specific task level (e.g., BMAML). Uncertainty modeling is critical when considering especially in few labeled data setting. Lacking of such modeling, the prediction uncertainty would be inaccurate. Model prediction uncertainty is considered and calibrated in Ananya Kumar (2019).
To tackle this problem, we propose to perform meta learning on the space of probability measures, ie. instead of adapting parameters to a new task, one adapts a meta distribution to new tasks. When implementing distribution adaption in algorithms where distributions are approximated by samples, our inntriittiionadapptttiiiin
framework becomes sample-to- sample adaption. In other words, the meta parameter in standard MAMI. becomes meta samples in our method, where uncertainty can be well encoded. For this reason, we call our framework Bayesian meta sampling.
Specifically, we propose mathematically elegant framework for Bayesian meta sampling based on the theory of Wasserstein gradient flows (WGF) (Ambrosio al. 2005). Our goal is to learn meta sampler whose samples can be fast adapted to new tasks. Our framework contains two main components: mela sampler and a sample adapter. For the meta sampler, we adopt the state-of- the-art flow-based method to learn to transport noise samples to meta samples. Our meta sampler is parameterized by neural inverse- autoreersttreee flow (NIAF), extension of the recently developed neural autoregressive flows (NAFs) (Huang et al., 2018). The NIAF consists of a meta-sample generator and an autoregressive conditioner model, which outputs the parameters of the meta-sample generator. The NIAF takes some task-specific information (such as gradients of target distributions) and random noise as input and outputs meta samples from its generator. These meta samples are then quickly adapted to task-specific samples of target distributions by feeding them to the sample adapter. To ensure efficient and accurate adaptations to new task distributions, novel optimal-transport Bayesian sampling (OT-sampling scheme, based on Wasserstein gradient flows, is proposed as the adaptation mechanism of the sample adapter. The OT-sampling is general and can ensure samples to be adapted in way that makes the sample density evolve to target distribution optimally, thus endowing the property of fast uncertainty adaption. Finally, when one aims to perform specific tasks such as Bayesian classification with a task network, these samples are used to encode uncertainty into modeling. To this end, we further develop an efficient learning algorithm to optimize the task network based variational inference. Extensive experiments proposed meta-sampling framework, ranging from wynthetic-distriburt adaption and to k-shot learning in Bayesian neural networks and reinforcement learning. Our results demonstrate a better performance of the proposed model compared to related methods.
conducted to test the advantages of the to posterior- distriitoisi
NEURAL INVERSE- AABRRE--AUTOOECEESSSIVE BAYESIAN META SAMPLING
Our model combines ideas from Bayesian sampling, Wasserstein gradient flows and inverse- autoregressive flows. A detailed review of these techniques is provided in Section Af the Appendix.
2.1 THE BASIC SETUP AND OVERALL IDEA
In meta sampling, one is given a set
meta sample
noise
of related distributions, e.g.. posterior distributions of the weights of a set of Bayesian neural networks (BNNs), each of which is used for classifica- tion on a different but related dataset. With our notation, each of the network and the related dataset called a task, which is denoted as r. Meta sampling aims to learn meta sampler based set of training tasks so that samples from the meta sampler can be fast adapted to samples for an unseen new task.
task-specific
Meta Sampler
Sample Adapter
+e samples
task
task
Figure 1: Overall idea of the proposed meta-sampling frame- work. The task is denoted with The two components and specific inputs will be described in details.
For each task, the dataset D, is divided into two sets D P {X yf and D & [xyal youl }. Our model builds the MAML framework Finn et al. (017). consisting of meta initialization and task adaptation. The difference is that instead of adapting parameters as in MAML, we adapt the distribution of parameters in terms of samples. Following Grant al. (2018), the probabilistic view
2

---

Published as a conference paper at ICLR 2020
of our model can be formulated hierarchical Bayes model in equation where Wo denotes the meta initialization of network parameters, W denotes the network parameters for task т, and adapting from P(Wo DF) to P(W, (Dr for specific task is denoted P(W, D Wo):
Plyt nad XTm
D
E Wo-P(Wo ( TET
II
P(yt fd W X7 P(W, D. Wo)dW, (1)
Our overall idea solve the problem is mimic dicrarchical-samplee procedure but in much more efficient way. Specifically, we propose to decompose meta sampling into two components: meta sampler and a sample adapter. The meta sampler responsible for generating meta samples that characterize common statistics of different tasks, ie.. approximating P(W, o D ) all the training tasks T with meta samples; The sample adapter is designed for fast adaptation of meta samples to task-specific target distributions, i.e., adapt the distribution from P(W DFF) P(W, ( in terms of samples for a task т. The meta sampler parameterized as a conditional generator, and aggregates all local losses of different tasks to form final loss for optimization based optimal- transport theory. Our method allows gradients to be directly backpropagated for meta-sampler updates. The overall idea is illustrated in Figure T
Comparisons with related works We distinguish our model with two mostly related works: the meta NNSGHMC (Gong al., 2019) and the probabilistic MAML (PMAML) (Finn al.. 2018). The main differences lie in two aspects: meta representation and model architecture. terms of meta-model representation,
model adopts data/parameter samples, instead of determinstic parameters, as meta representation, and thus can be considered as sample-to-sample adaption. Meta NNSGHMC uses samples on different tasks, there is no concept of meta samples. Finally, PMAML fully relies on variational inference whose representation power could be restricted. In terms of model architecture, our model adopts the state-of-the-as autoregressive architectures, which can generate high-quality meta samples. Furthermore, our model adopts simpler way to define the objective function, which allows gradients to directly flow back for meta-sampler optimization. is worth noting that our framework reduces to MAML when only one meta sample is considered for sample adaption. Since our methods aim for Bayesian sampling. it is more similar to meta NNSGHMC (Gong et al., 2019); whereas MAML aims for point estimation of model. Finally, note that the recently proposed neural process (Garnelo et al., (018)) might also be used for meta-learning "few-shot function regression" as stated in the original paper. However, work has been done for this purpose.
knowledge, specific
2.2 THE BAYESIAN META SAMPLER
This section aims to design a meta sampler for efficiently generating meta samples. One idea is to use nonparametric model to generate meta samples such as standard stochas- tie gradient MCMC (SG-MCMC)
Task network C(=; W)
NIAF
(Welling & Teh 2011 Ma Ma al. 2015)
and the Stein variational gradient de- scent (SVGD) (Liu & Wang 2016a),, where no parameters are consider in the model. However, methods under this setting are typically slow, and the generated samples are usually highly correlated. Most importantly, it would be hard to design nonparametric sam- plers that share information be- tween different tasks. As result, propose to learn the meta sampler with a parametric model, which also denoted generator.
E2
T(
T(:
T(*
One meta sample 23
Figure 2: Roll-out architecture of the proposed NIAF as meta sampler, consisting of generator G and an autore- gressive conditional model Tb: The meta samples are then fed to task network C(:; W) to encode uncertainty into specific tasks such as classification. Please see detailed descriptions in the text.
3

---

Published as a conference paper at ICLR 2020
2.2.1 LEARNING PARAMETERIZED META SAMPLERS WITH NIAFS
There are two popular options to parameterize the meta sampler: with an explicit generator or with an implicit generator. An explicit generator parameterizes the output (i.e, meta samples) as samples from an explicit distribution with a known density form such as Gaussian, thus limiting the representation power. In the following, we propose to adopt an implicit generator for the meta sampler based on neural
flows (NIAF), an inverse extension of the recently proposed NAF (uang et al. 2018) used for density estimation. As will be seen, NIAF can incorporate task-specific information into an implicit generator and generates samples in an autoregressive manner efficiently. Finally, meta samples are used in task network to encode uncertainty for specific tasks such as Bayesian classification. The architecture of the meta sampler is illustrated in Figure 2 Note the idea of using NIAF to generate network parameter is similar to the hypemneworr (Ha et al. 2016 Krueger 2017). As an extention, [Padier et al. (2018) perform inference on a lower dimensional latent space. Using hypernetworks model posterior distributions have also been studied in (Pawlowski
et al.
et al. 2017 7 Sheikh et al. 2017).
Neural inverss-autoreggresive flows Directly adopting the NAF for sample generation is inap- propriate as it was originally designed for density evaluation. To this end, we propose the NIAF for effective meta-sample generation. Specifically, let λ denote the k-th element of a sample z to be generated; 2 denotes a sample from last round; I' denotes the task-specific information. In our case, we set r _ (2, Va p()) with p(.) denoting the target distribution (with possible hidden parameters). NIAF generates the sample z = (21, ... 2k,, ..) via an autoregressive manner, as:
Z4 G(E,, ri T rb)))
where } are noise samples; G(., :) is an invertible function (generator) parameterized by & and implemented as a DNN; and T is an autoregressive conditioner model parameterized by & to generate the parameters of the generator G' at each step k. which is itself implemented as deep sigmoidal flow or a deep dense sigmoidal flow as in (Huang et al. 2018). According to Huang et al. (2018), using strictly positive weights and strictly monotonic activation functions for G is sufficient for the entire network to be strictly monotonic, thus invertible.
The original NAFs are not designed for drawing samples, as one needs the inverse function G-1, which is not analyticully solvable when G is implemented as neural nelwork. Although it is stated Huang el al. 2018)) that G-1 can be approximated numerically, one needs repeated approxinations, making computationally prohibited. Our proposed NIAF is designed specifically for sample generation by directly transforming the noise with a flow-based network G.
Remark
The task network In addition to the NIAF, task network might be necessary for processing specific learning tasks. In particular, if one is only interested in generating samples from some target distribution, a task network is not necessary as the meta samples will be used to adapt task-specific samples. However, if one wants to do classification with uncertainty, the task network should be defined as a classification network such as an MLP or CNN. In this case, denoting the weights of the task network as W, we consider the task network as a Bayesian neural network, and propose two ways of parameterization to encode uncertainty of meta samples into the task network:
Sample parameterization: A sample of the weights of the task network is directly rep- resented by a meta sample from our meta sampler, i.e., W = (21, 22, 22p) with the parameter dimensionality.
Multiplicative parameterization: Adopting the idea of multiplicative normalizing flows (Louizos& & Welling (0177)) we define an inference network for the weights as the multipli- cation of z and a Gaussian variational distribution for W, i.e., the variational distribution is defined as the following semi-implicit distribution to approximate the true posterior distribution for W:
NIAF((s)).)) s(W 2) = N(W of Sulle og))
(3)
Here and in the following, we consider the task network parameterized for notation simplicity, although our method applies to other network structures; and we have used NIAF (aF((());)) D) to denote the output of meta samples from the NIAF
one-layer MLP
4

---

Published as a conference paper at ICLR 2020
Comparing the two parameterizations, sample parameterization directly generates weights of the task network from the meta sampler, thus is more flexible in uncertainty modeling. However, when the task network grows larger to deal with more complex data, this way of parameterization quickly becomes unstable or even intractable due to the high dimentionality of meta samples. Multiplicative parameterization overcomes this issue by associating each element of meta sample with one node of the task network, reducing the meta-sample dimensionality from O(Nin x Nou)) to O(Nin) with Nn and Nour being the input and output sizes of the task network. As result, we adopt the multiplicative parameterization when dealing with high dimensional problems in our experiments. Efficient inference for these two cases will be described in Section 2.4 Note recent work on NAF inference proposes to first sample from mean-field approximating distribution, which are then transformed by an NAF to a more expressive distribution (Webb et al., 2019). However, the approach is hard to scale to very high dimensional problems, e.g. posterior distribttions of network parameters.
2.3 THE SAMPLE ADAPTER
The output of the meta sampler contains shared (meta) information of all the tasks. Task-specific samples are expected to be adapted fast from these meta samples. This procedure is called sample adaption. Since there are potentially a large number of tasks, learning task-wise parametric models for sample adaption is impractical. Instead of using standard nonparametric samplers such SG-MCMC or SVGD, we propose a general Bayesian sampling framework based on (Villani 2008) for new task-s adaption, where back- prop-propagttion can be directly applied.
theory
A general Bayesian sampling framework based on optimal transport Let a task-specific target distribution be pr (z). indexed by т. A standard way to adapt the samples based on Markov chain whose stationary distribution equals Pr. e.g., via SG-MCMC. However, Markov-chain chase- methods might not be efficient enough in practice due to the potentially highly-correlated samples (Chen et al. 201 Furthermore, it is not obvious how to apply backpropagation (BP) in most of sampling algorithms. To deal with these problems, we follow [Cen et al. (2018b) and view Bayesian sampling from the Wasserstein- gradeintngaddn perspective (discussed in Section A.2), i.e., instead of evolving samples, we explicitly evolve the underlying sample density functions. We will see that train the proposed meta sampler efficiently via standard BP.
such a solution allows
Considering our meta-learning setting. Since we aim to adapt meta samples to new tasks, it is reasonable to define the adaptation via task-wise WGFs, i.e., for each task, there is WGF with a specific functional energy and the corresponding first variation, denoted respectively as E, and F, A BE: with the task index T. Here denotes the underlying density of the samples. Consequently, will evolve with a variant of the PDE by replacing E with Fr, in equation 9 for each task. To solve the corresponding PDE, we prove Theorem [ meta samples, which is termed optimal-transpeee Bayesian sampling (OT-Bayesian) sampling).
Ap
based discrete approximation of p with the evolved
Bayesian Sampling) Let Pt time t be approximated by parti- cles, iee, pz(z) R Nr E.Kh 6,00 (z), where (z(") } at time L = 0 are a set of metu samples from the meta generator the delta-function da(b) 1 if a = b and 0 otherwise. Then sample adaption for task T can be described with the following differential equations:
Theorem (Optimal-Transs)
(zF,(z (2 ddt, for'Vi.
dz
(4)
Based on Theorem 1 particle updates for sampling can be obtained by solving equation 4 numerically, e.g., via the Euler scheme,
R++1 z +AV
(1) (6)
where h the stepsize. Here with a little abuse of notation but for conciseness, we use z to denote
(
the numerical solution of 2!4) at time / = h& with & being the iteration. Theorem and equation 5 indicate that the derivative of E, w.r.t. (1) can be considered as a 2E (a Fr (z This is
useful result to derive a learning algorithm for the meta sampler described in Section 2.4
WWe use the bold letter 2 (or ( ) to denote the i-th meta sample evolved with equation 4 at time (or equation at iteration k). This should be distinguished from the normal unbold letter 2; defined in Section 2.2.1 which denotes the k-tth element of z.
5

---

Published as a conference paper at ICLR 2020
Energy functional design Choosing an appropriate energy function E, is important for efficient sample adaptation. To achieve this, the following conditions should be satisfied: i) E, (p) should be convex w.I.t. p: ti) The first variation F, could be calculated conveniently. A general and convenient functional family is the f-divergence, which defined, with our notation and convex function
(2(2] dz. The f-divergence f: such that f(1) O, as: E & (y(plp))) = p- (z)
p(a))
is general family of divergence metric. With different functions f. it corresponds to different divergences including the popular KL divergence, inverse- KL divergences, and the Lensen-Shamme divergence. For more details, please refer to (Nowozin et al. (016). A nice property of f-divergence that its first variation endows a convenient form as stated Proposition 2
Proposition 2 Let r(z) A p(z)
The first variation of the I-divergence endows the following form:
P-(2)
SEf 0p (p)=
Ff
In our experiments, we focus on the KL-ddivegeecc,, which corresponds to [(r) logr. r. this case, (z) Vg log p. (z) Vz log p(z). Since the density p(z) required in evaluating is not readily available due to its implicit distribution, we follow Chen et al. (2018b) and use the meta
samples {z at the k-th step for approximation. resulting in
(z) = Va (z)
Σ V
(Z 2
Σ
Σ v x2) (3) Σ k(z, x
(6)
where x(.,-) .) is a kernel function. The number of adaptation steps k should be set based on problems. For tasks that vary significantly, larger i should be chosen ensure the quality of adapted samples. To further improve the accuracy, inspired by Chen et al. (2018b), we combine equation 6 with the first variation of SVGD, resulting the following form at iteration
M
(zF+(2)
A M
log (z (m(z, 2 V
V kz 20 (0) (z)
(7)
2
where A 2 O is a hyperparameter to balance the two terms.
2.4 MODEL TRAINING AND SAMPLE GENERATION
We first describe how to train the proposed model under the two kinds of parameterization of the task network defined in Section 2.2.1
setting In this case, one only needs to optimize the Φ as parameters of other networks are directly generated. Specifically, because the energy functionals for each task are designed so that the minima correspond to the target distributions, the objective thus can be defined over the whole task distribution p(T) as:
Training in the conditional model a):
min C A Et-p(r) [Er
For notation simplicity, we will not distinguish among the adapted samples (i.e., 2 (1) for different
tasks. Since the only parameter is ab in the autoregressive conditioner model T (see Figure and note the parameters and W for the meta generator and task network do not need to be learned as they are the outputs and G, respectively), its gradient can directly calculated using chain rule:
OC dxb
ETap(r)
Σ
Oz
(
T OE.
ETap(T)
E
Σ
Oz Oxb
(
T
V
F
(9)
(8)
Oab
a dz
(4)
where (.*).. follows by the results from Section 2.3 and V 2/
(z()) (3)
[
is readily calculated with
equation 7 the term
i (x ai can be calculated by standard BP over the meta-sampler network (NIAF).
Training in the to be learned, &b and W. For th, one can optimize &p by adopting the same update equation as the pample-paramettriiattoo setting. For W, we Louizos & Welling (2017a) and adopt variational
setting In this case, two sets of parameters

---

Published as a conference paper at ICLR 2020
inference. Specifically, we first augment the task network with auxiliary network with conditional
likelihood (z W) & I M(z + parameterized by 0 _ (ū., Based on Ranganath ct al. (2016); ( Louizos & Welling (2017a); Touati et al. (2019), with the inference network detned in
(3). and writing the implicit distribution of (z(z) and the Gaussian prior distribution of W as pl (W) we arrive at the following ELBO:
[(0, 6 2b) (z (z. W) [logp(ul x, W, z) + log p W) + log ro W log d6 ( 2) log 4 (z)
Different from Louizos & Welling 2017a), we only update (0, Φ) by optimizing the above ELBO: while leave the update of sb with the gradient calculated in equation 8 which reflects gradients of samples from the NIAF Note also that in the meta learning setting, the task network needs to be adapted for new tasks. This can be done by standard MAML with the above ELBO as the new-task objective. The meta training and testing algorithms are illustrated in the Appendix B A similar ideas of multiplicative parametrization was proposed recently in (Kristiadi & Fischer 2019), which used a compound density network to quantify the predictive uncertainty.
New-task sample generation After training, samples for new task can be directly generated by feeding the task information I' and some noise to the meta sampler depicted in Figure |] Typically, one needs to run few numbers of sample-adaption steps to generate good samples for new tasks. Notably, the number of sample-adaption steps required to obtain good accuracy will be shown much less than simply starting sampler from scratch in the experiments.
3 EXPERIMENTS
We conduct series of experiments to evaluate the efficiency and effectiveness of our model, and compare it with related Bayesian sampling algorithms such as SGLD, SVGD and SGHMC. The main compared algorithms for meta learning include the PMAML (Finn et al. 2018). Amortized Bayesian
Meta-Learning (ABML) (Ravi & Beatson 2019), and NNSGHMC (Gong al.) 2201)). recently proposed meta SG-MCMC algorithm. Inspired by Finn et al.) (2017), we denote our algorithm
distribution agnostic meta sampling (DAMS). Implementation details are given in Appendix F Our code is made available at: https//gthbbooomm/eehiiiggeeeeee sampligll/atttssaligg..... git
3.1 EVALUATIONS OF OT-BAYESIAN SAMPLING
Table 1: Accuracy of BLR with different samplers.
We first demonstrate our proposed NIAe-baseed sampler is able to generate more effective samples compared to the popular Bayesian algorithms such as SVGD, SGLD and SGHMC, in non-meta-sampling setting. To this end, we apply standard Bayesian Logistic Regression (BLR) on several real datasets from the UCI repository: Australian (15 features, 690 samples), German (25 features, 1000 samples), Heart (14 features, 270 samples). We perform posterior sampling for BLR using our proposed sampler, as well as SVGD, SGLD, SGHMC. For a more detailed investigation of different components in our model, we also test the generator with different architectures, including generators with MLP (DAMS with MLP), IAF (DAMS with IAF), and NIAF (DAMS with NIAF). We follow Liu & Wang (2016a) and apply Gaussian priors for the parameters Po (w]aa = N(w; O, a- I) with Po (a) = Gamma (a, 1, 0.01). A random selection of 80% data are used for training and the remaining for testing. The testing accuracies are shown in Table []
Australian
Gerraan
Heart
SVGD
88.7
74.5
78.7
SGLD
88.4
75.2
81.5
SGHMC
88.7
77.0
88.6
DAMS with MLP
85.5
75.0
87.0
DAMS with IAF
87.0
75.5
88.9
DAMS with NIAF
89.1
77.5
90.7
It is observed that DAMS with NIAF achieves the best performance in terms of accuracy. The results also indicate the effectiveness and expressiveness of the proposed NIAF architecture in the OT-Bayesian sampling framework.
3.2 EVALUATIONS OF META-SAMPLL ADAPTABILITY
In this set of experiments, we aim to demonstrate the excellent meta-sample adaptability of our meta-sampling framework in different tasks. An additional synthetic experiment on meta posterior adaption is presented in Section D.3 of the Appendix.
Gaussian mixture model We first conduct experiments to meta-sample several challenging Gaus- sian mixture distributions. We consider mixtures of 4, 6 and 20 Gaussians. Detailed distributional forms are given in the Appendix. To setup a meta sampling scenario, we use 2, 3 and 14 Gaussian components with different means and covariance, respectively, for meta-training of the meta sampler.
7

---

Published as a conference paper at ICLR 2020
6-Gatss Mixture
20-Gouss Nixture
3:40
0.10
REGGANM
0.135
50.5
20.--1
0.190
2010
0.325
325
0.10
2.20
MMD
0.24
0.09
0090
0885
3.00
2.00
15.0
220
200
AGC
Beeations
Berations
Figure 3: Convergence of mata-sgnpling different Gaussian mixture models. SGLD-m and SVGD-m are SGLD and SVGD with the same initialization as DAMS. Other samplers are initialized randomly.
After training, meta samples are adapted to samples from target Gaussian mixture by following the tas-ttak-smml--eenneee
procedure described in Section 2.4 We plot the convergence of 1000 meta samples to a target distribution versus a number of particle (sample) updates (iterations), measured with the maximum mean discrepancy (MMD) evaluated by samples. For a fair comparison, we use the same number of samples (particles) evaluation the MMD. The results are shown in Figure 3 It is clear that our proposed meta sampler DAMS converges much faster and better than other sampling algorithms, especially on the most complicated mixture of 20-Gaussians. The reason for the fast convergence (adaption) is partially due to the learned meta sampler, which provides good initialization for sample adaption. This is further verified by inspecting how samples in the adaption process evolve, which are plotted in Figure 10
and 12 in the Appendix.
Meta Sampling for Bayesian Neural Networks Finally, we test the proposed DAMS for meta sampling of BNNs on MNIST and CIFAR-I0 (Krizhevsky) 2009). We follow the experimental setting in (Gong et al. 2019). and split the MNIST and CIFARIÓ dataset into two parts for meta training and testing (sample adaption), respectively. As we focus on fast adaptation, we show the accuracy within 200 iterations. To deal with the gigh-dimensionality issue, we adopt the method of multiplicative parameterization proposed in Section 2.2.1
We randomly pick 5 classes for training, and the remaining classes for testing. A BNN is trained only on the training data for meta sample (weights of the BNN) generation, with each sample corresponding to meta BNN. In testing, the meta BNNs are adapted based on the testing data. For the MNIST dataset, we parameterize a BNN as a CNN with two convolutional layers followed by a fully connected layer with 100 hidden units. The kernel and filter sizes of the two conv layers are 3 and 16, respectively. A similar architecture is applied for the CIFARIO dataset, but with 16 and 50 filters whose kernel sizes are 7 and 5 for the two convolutional layers, respectively. The hidden units of the fully connected layer is 300.
a) Adaptation efficiency: For this purpose, we compare our model with NNSGHMC (Gong et al. 2019), as well as with a nen-meng- method to train from scratch. To demonstrate the effectiveness of our NIAF structure for adaptive posterior sampling, we also compare it with the simple conditional version of MNF (Louizos & Welling 2017a). plots the learning curves of testing accuracy versus the number of iterations. It clearly seen that our DAMS adapts the fastest to new tasks, and is able to achieve the highest classification accuracy on all cases due to the effectiveness of uncertainty adaption. To further demonstrate the superiority of DAMS over NNSGHMC, we list the test accuracy at different adaptation steps in Table 2 The results clearly show faster adaption and higher accuracy of the proposed DAMS compared to NNSGHMC. It also interesting to see that MNF with adaptation, performs better than NNSGHMC: while method performs better than both, demonstrating the effectiveness of the proposed NIAF architecture.
b) Sample efficiency: To demonstrate sample efficiency of our framework, we compare it with both NNSGHMC and the standard Bayesian learning of DNNs with SGHMC. To this end, we use the first five classes of image
CIFAR10 as training data to learn the meta sampler and randomly select 5%, 20%, 30% of the training data in the remaining image classes of the test task to adapt the meta sampler. Figure 5 shows the corresponding test accuracies for different settings. It is observed
8

---

Published as a conference paper at ICLR 2020
Table 2: Accuracy (%) for adaptation efficiency.
Table 3: Uncertainty adaptation evaluation of
Approach
CIFAR
MNIST
Bayesian meta sampling for BNNs terms of negative log-likelihood.
20 steps
NNSGHMC
23.8
28.6
20 steps
MNF
42.3
59.4
steps
DAMS-NTAF
42.7
60.7
Approach
accuracy
NLL1/00
steps
NNSGHMC
47.2
63.6
20 steps
SGHMC
32.3
79.22
steps
MNF
75.5
20 steps
DAMS-NIAF
42.7
68.98
steps
DAMS-NIAF
61.9
78.5
50 steps
SGHMC
39.2
75.34
100 steps
NNSGHMC
58.1
80.1
50 steps
HAMS-NIAL
61.9
57.82
100 steps
MNF
67.2
85.8
100 steps
SGHMC
46.5
67.95
100 steps
DAMS-NIAF
69.5
88.0
100 steps
DAMS- NTAF
69.5
46,49
200 steps
NNSGHMC
71.7
88.6
200 steps
SGHMC
59.1
31.79
200 steps
MNF
72.7
90.8
200 steps
DAMS-NIAF
75.2
36.53
200 steps
DAMS-NIAF
75.2
92.6
0.
accuracy
6.5
eccuraCy
meta adaptation with NNSGHMC etaptesion with DAVS adaptation training from scratch with DAMS
0.-
meta adaptation with NNSGHMC
mete edeetation with DAMS edaptation war
03
trairing from scratch with DAMS
125
230
"umber iteralions
FurF ber of llerations
Figure 4: Adaptation efficiency in terms of testing accuracy on CIFARIO (left) and MNIST (right).
06
necurnry
05-
test aecuracy
test accuraccy
03
02-
SCHMC NNSGHKC
SCHMC NNSGHMC
SGHMC
01
01
NNSGHMC
125
125
number iteaaioos
number
Rerabors
number iterations
Figure 5: Sample efficiency evaluation with 5%, 20%, 30% of training data on CIFARIO, (see text).
-40
1.0
DAMS
SCHMAC
0.9
03-
0es: NLU/200
Empliiial
-120
02
0.2
-340
SGHMC
0.0
25
232 17s
200
100
123
150
275
200
0.3
0.5
1.0
25
number of derations
number of iterationns
Entropy
Figure 6: Testing accuracy (left, CIFARIO), uncertainty adaptation evaluation in terms of negative likeliheld (middle, CIFAR10) and entropy of out-of-distribution prediction (right) notMNIST.
that ours, the adaptation-based methods, obtain higher accuracies than the non-adaptive method of SGHMC. Furthermore, our method achieves the best sample efficiency among others.
с) Uncertainty evaluation: Finally, we ablate study the uncertainty estimation of our, the standard SGHMC and NNSGHMC models terms of negative log likelihoodd also show the corresponding test accuracy. The results the CIFAR 10 dataset are shown in @ggureeaand Table We follow Louizos Welling (2077a) to evaluate uncertainty via entropy of out-of-sample predictive distributions (Figure ( (right)). We observe that uncertainty estimates with DAMS are better than others, since the probability of low entropy prediction is much lower than others. Details are given Section D.4 the Appendix.

---

Published as a conference paper at ICLR 2020
Table 4: Testing accuracy of Mini- ImageNet Few Shot Classification. "N/A" means results not reported.
04
Mini Imugenei 5-way Few Shot Classincation
Approsch
Accuracy
0.40
1-hhot
S-abot N/A
0.2
%
04
PMAML
50.13 1.13:210 1,86
0.30
ABML
0.600
MAML-SGHMC
50.18 1138000000
63.94 = 0.48 %
0.2
DANS
DAMS-SGLD SGLD
50.86
0.48
%
66.23 = 0.45 %
0.2
10000
Mata Amsitor
0.2
10003
Mata Amsdoon
30000
DAMS-NIAF
51.43 0.49
66.84
Figure Learning curves of 5-way 1-shot classification.
3.3 META SAMPLING FOR BAYESIAN FEW-SHOT LEARNING
DAMS for few-shot image classification We apply our method two popular few-shot image- classification tasks on the Mini-Imagenet dataset, consisting of 64, 16, and 20 classes for training, validation and testing, respectively. We compare our method with MAML and its variants with uncertainty modeling. including the Amortized Bayesian Meta-Learning (ABML) (Ravi & Beatson 2019) and Probabilistic MAML (PMAML.) (Finn et al. 2018). To get better understanding of each component of our framework, we also conduct an ablation study with three variants of our model: MAML-SGLD, MAML-SGHMCC and DAMS-SGLD. MAML-SGLD and MAML-SGHMC correspond to the variants where SGLD and SGHMC are used to sample the parameters of the classifier, respectively; and DAMS-SGLD replaces the WGF component DAMS-NIAF with SGLD. Follow the setting in (Finn al., 2017 2018), the network architecture includes a stacked 4-layer convolutional feature extractor, followed by a meta classifier with one single colnecconeettdd layer using the multiplicative parameterization. Testing results are presented in Table 4 With method, we observe significant improvement of the classification accuracy at an early stage compared with MAML. The learning curves are plotted in Figure 7 further demonstrating the superiority of our method, which can provide an elegant initialization for the classification network. Finally, from the ablation study, the results suggest both the NIAF and the WGF components contribute the performance gain obtained by our method.
Meta sampling for reinforce- ment learning We next adapt our method for meta reinforce- ment learning. We test and com- pare the models on the same Mu- JoCo continuous control tasks (Todorov et al., 2012) as used in (Finn et al. 2017). including the
creatah, goal valeity
cheetah directionn
-100
Asscoga Nulurne
200
200
TUMMS MANL
DAMS
goal velocity task and goal direc- tion task for cheetah robots. For a fair comparison, we leverage the TRPO-RL (Schulman al.) 2015) framework for meta updat- ing following MAML method. Specifically, implement the policy network with two hidden layers with ReLu activation followed by a linear layer to produce the mean value of the Gaussian policy. The first hidden layer a fully connected layer, and we adopt the multiplicative parameterization for the second hidden layer. As shown in Figure 8 our method obtains higher rewards compared with MAML on both tasks, indicating the importance of effective uncertainty adaptation RL.
Peta Benations
230
30
Metta fecctionns
300
Figure 8: Locomotion comparison results for goal velocity task (left) and goal direction task (right).
CONCLUSION
We present Bayesian sama-sammling framework, called DAMS, consisting of a meta sampler and sample adapter for effective uncertainty adaption. Our model is based on the recently proposed neural autoregressive flows and related theory from optimal transport, enabling a simple yet effective training procedure. To make the proposed model scalable, an efficient uncertainty parameterization is proposed for the task network, which is trained by variational inference. DAMS is general and can be applied to different scenarios with ability for fast uncertainty adaptation. Experiments of tasks demonstrate the advantages of the proposed framework over other methods including the recently proposed meta SG-MCMC, in terms of both sample efficiency and fast uncertainty adaption.
series
10

---

Published as a conference paper at ICLR 2020
REFERENCES
L. Ambrosio, N. Gigli, and G. Savaré. Gradient Flows in Metric Spaces and in the Space of Probability Measures. Lectures in Mathematics ETH Zürich, 2005.
Tengyu Ma Ananya Kumar, Perey Liang. Verified uncertainty calibration. In Advances in Neural Information Processing Systems, 2019.
Marein Andrychowiez, Misha Denil, Sergio Gómez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. D. D. Lee, M. Sugiyama, U. V. Luxburg. Guyon, and Garnett (eds.), Advances in Neural Information Processing Systems, pp. 3981-3989. 2016.
Antreas Antoniou, Harrison Edwards, and Amos Storkey. How train your maml. In International Conference on Learning Representations, 2019.
Yaroslav Bulatov. Not mnist dataset. 2011. URL BtLp:t://yaroollv...bbo www.:/p.bblooppp.oooooo com/ 1011109/notmmn st datadadttteettttmm
C. Chen, C. Li, L. Chen, W. Wang, Y. Pu, and L. Carin. Continuous- flows for deep generative models. In International Conference on Machine Learning, 201 8a.
C. Chen, Zhang, W. Wang, B. Li, and L. Chen. A unified particle-optimizatinn framework for scalable Bayesian sampling. In Conference on Uncertainty in Artificial Intelligence, 2018b.
Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. Variational lossy autoencoder. In International Conference on Learning Representations, 2017.
Xinshi Chen, Hanjun Dai, and Le Song. Meta Particle Flow for Sequential Bayesian Inference. arXiv e-prints, art. 1.X.vv1902.0000. Feb 2019.
Kristy Choi, Mike Wu, Noah Goodman, and Stefano Ermon. Meta-amortized variational inference and learning. CoRR, abs/1902.01950, 2019. URL
/1902. 01950
Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In International Conference on Learning Representations, 2017.
Y. Feng, D. Wang, and Q. Liu. Learning to draw samples with amortized stein variational gradient descent. In Conference on Uncertainty in Artificial Intelligence, 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference Machine Learning, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. In Advances in Neural Information Processing Systems. 2018.
Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. In International Conference on Machine Learning, 2019.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and S. M. Ali Eslami. Conditional neural processes. In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 1704 1713, 2018.
Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin. Bayesian Data Analysis. Chapman and Hall/CRC, 2nd ed. edition, 2004.
Mathieu Germain, Karol Gregor, lain Murray, and Hugo Larochelle. Made: Masked autoencoder for distribution estimation. In International Conference on Machine Learning, 2015.
Wenbo Gong, Yingzhen Li, and Josö Miguel Hemnändez-Lobato Meta-learning for stochastic gradient meme. In International Conference Learning Representations, 2019.

---

Published as a conference paper at ICLR 2020
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E. Turner. Meta-learning probabilistic inference for prediction. In International Conference on Learning Representations, 2019.
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- based meta-learning as hierarchical bayes. In International Conference on Learning Representa- lions, 2018.
David Ha, Andrew Dai, and Quoe V. Le. Hypernetworks. In https//arriioorgad/////0....0000 2016.
Chin- Wei Huang, David Krueger, Alexandre Lacoste, and Aaron Courville. Neural autoregressive flows. In International Conference on Machine Learning, 2018.
Mikhail Khodak, Maria-Florina Balcan, and Ameet Talwalkar. Provable guarantees for gradient- meta-learning. In International Conference on Machine Learning, 2019.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference Learning Representations, 2015.
Diederik P. Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. Im- proving variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems, 2016.
Agustinus Kristiadi and Asja Fischer. Predictive uncertainty quantification with compound density networks. In htpps//arrihoorr/bb//99200000000 2019.
Alex Krizhevsky. Learning multiple layers of features from tiny images. In Technical report, 2009.
David Krueger, Chin-Wei Huang, Riashat Islam, Ryan Turner, Alexandre Lacoste, and Aaron Courville. Bayesian hypernetworks. In htpps//arzi.orggab///00004999 2017.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017.
Y. Ji, J. Hernández-Lobato, and R. E. Turner. Stochastic expectation propagation. In Advances in Neural Information Processing Systems, 2015.
Chang Liu, Jingwei Zhuo, Pengyu Cheng, Ruiyi Zhang, and Jun Zhu. Understand and accelerate particle-based variational inference. In International Conference on Machine Learning, 2019a.
Chang Liu, Jingwei Zhuo, and Jun Zhu. Understanding MCMC dynamies flows the wasserstein space. In International Conference on Machine Learning. 2019b.
Q. Liu and D. Wang. Stein variational gradient descent: A general purpose Bayesian inference algorithm. Advances in Neural Information Processing Systems, 2016a.
Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. In Advances in Neural Information Processing Systems, 2016b.
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In Proceedings the 34th International Conference on Machine Learning Volume 70, International Conference on Machine Learning, pp. 222182 2017a
Christos Louizos and Max Welling. Multiplicative normalizing flows for variational bayesian neural networks. In International Conference on Machine Learning, 2017b.
Y. A. Ma, T. Chen, and E. Fox. A complete recipe for stochastic gradient MCMC. In Advances in Neural Information Processing Systems, 2015.
Tsendsuren Munkhdalai and Hong Yu. Meta networks. In International Conference on Machine Learning, 2017.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. 2018.
12

---

Published as a conference paper at ICLR 2020
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems, 2016.
Pedro A. Ortega, Jane X. Wang, Mark Rowland, Tim Genewein, Zeb Kurth-Nelson, Razvan Pascanu, Nicolas Heess, Joel Veness, Alex Pritzel, Pablo Sprechmann, Siddhant M. Jayakumar, Tom McGrath, Kevin Miller, Mohammad Azar, Ian Osband, Neil Rabinowitz, Andras Gyorgy, Silvia Chiappa, Simon Osindero, Yee Whye Teh, Hado van Hasselt, Nando de Freitas, Matthew Botvinick. and Shane Legg. Mcta-learning of sequential strategics. Technical Report arXiv: 1905 000.03030, DeepMind, May 2019.
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. In Advances in Neural Information Processing Systems, 2017.
Nick Pawlowski, Andrew Brock, Matthew C.H. Lee, Martin Rajchl, and Ben Glocker. Implicit weight uncertainty in neural networks. In https//arziioorgad///1111119999
2017.
Melanie F. Pradier, Weiwei Pan, Jiayu Yao, Soumya Ghosh, and Finale Doshi-veler, Projected bnns: Avoiding spich pathologies by learning latent representations neural network weights. In https://arziix.or/////////// 1.0 .07006.ppff 2018.
Rajesh Ranganath, Dustin Tran, and David M. Blei. Hierarchical variational models. International Conference on Machine Learning, 2016.
Sachin Ravi and Alex Beatson. Amortized bayesian meta-learning. International Conference on Learning Representations, 2019.
Sachin Ravi and Hugo Larochelle. Optimization model for few-shot learning. In International Conference on Learning Representations, 2017.
Jurgen Schmidhuber. Evolutionary principles in self-referential learning. on learning now to learn: The meta-metta-meta...hookk Diploma thesis, Technische Universitat Munchen, Germany, May 1987. URL ttttpp//wwet idsia eh wttpaaaceeeeeeeeeeeeaeeee.a. wwwpgge//ddiloomm html
John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In International Conference on Machine Learning, pp. 1889-1897, 2015.
Abdul-Saboor Sheikh, Kashif Rasul, Andreas Merentitis, and Urs Bergmann. Stochastic maximum likelihood optimization via hypernetworks. In Advances in Neural Information Processing Systems Workshop Bayesian Deep Learning, 2017.
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5036-5033. IEEE, 2012.
Ahmed Touati, Harsh Satija, Joshua Romoff, Joelle Pineau, and Pascal Vincent. Randomized value functions via multiplicative normalizing flows. In Conference on Uncertainty in Artificial Intelligence, 2019.
C. Villani. Optimal transport: old and new. Springer Science & Business Media, 2008
Stefan Webb, Jonathan P. Chen, Martin Jankowiak, and Noah Goodman. Improving automated variational inference with normalizing flows. In 6th ICML Workshop on Automated Machine Learning, 2019.
M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient Langevin dynamics. In International Conference on Machine Learning, 2011.
Lilian Weng. Meta-lcarning: Learning to learn fast. Technical Report https://lllanwwcnggiiuubbbbbb
log/2018 8/1 1/30///tt-laaaammmaaaa leaatttaaaaaiin.ghm 2018.
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hicrarchically structured meta- meta-learning. In
International Conference Machine Learning, 2019.
13

---

Published as a conference paper at ICLR 2020
Jaesik Yoon, Taesup Kim, Ousmane Dia, Sungwoong Kim, Yoshua Bengio, and Sungjin Ahn. Bayesian -odel-agnostic meta-learning. In Advances in Neural Information Processing Systems. 2018.
R. Zhang, C. Chen, C. Li, and L. Carin. Policy optimization as wasserstein gradient flows. In International Conference on Machine Learning, 2018.
14

---

Published as a conference paper at ICLR 2020
A
BACKGROUND
This section provides a review of background on Bayesian sampling, Wasserstein gradient flows and autoregressive flows.
A.1 BAYESIAN SAMPLING
Bayesian sampling has been long-standing tool in Bayesian modeling, with wide range of applications such as uncertainty modeling (Li et al., 2013), data generation (Feng et al. 2017] Chen et al. 2018a) and reinforcement learning (Zhang et al. 2018). Traditional algorithms icclude but are not limited to Sutropolis-Hastinng algorithm, importance sampling and Gibbs sampling (Gelman et al. 2004). Modern machine learning and deep learning have been pushing forward the development of large-scale Bayesian sampling. Popular algorithms in this line of research include the
family of stochastic gradient MCMC (SG-MCMC) (Welling & Tch 2011 Ma et al. 2015) and the Stein variational gradient descent (SVGD) (Liu & Wang 2016a). Recently,
sampling framework that unifies SG-MCMC and SVGD has also been proposed (Chen al. 2018b), followed by some recent developments (Liu et al.] 2019b a). Generally speaking, all these methods target at sampling from some particular distributions such as the posterior distribution of the weights of Bayesian neural network (BNN).
On the other hand, meta learning is a recently developed concept that tries to learn some abstract information from set of different but related tasks. A natural question by considering these two is: can we design meta sampling algorithms that learns to generate meta samples, which can be adapted to samples of task-specific distribution quickly? This paper bridges this gap by proposing mathematically sound framework for Bayesian meta sampling.
A.2 OPTIMAL TRANSPORT AND WASSERSTEIN GRADIENT FLOWS
In optimal transport, a density function, Pt, evolves along time to a target distribution optimally, Le., along the shortest path on the space of probability measures P(22), with Ω being subset of TR.. The optimality is measured in the sense that pt moves along the geodesic of a Riemannian manifold induced by functional energy, E' P(l)) + I. under the 2- 2-Wasserstein distance metric. Formally, the trajectory of pt is described by the following partial differential equation (PDE):
Oepr = V. (: Vz
( A Vx (peVV
SE 8pr
where V. F f is the divergence operator; and F & 6E (pt) is called the first variation of op.
E at Pt (functional derivative on manifold in P(2)). To ensure Pt to converge to target distribution p such as the posterior distribution of the model parameters, one must design an appropriate F. such that p = arg, min (p). A common choice is the popular KL-divergence, (L p). We will consider more general setting in our framework presented later. Note the WGF framework equation 9 allows to view Bayesian sampling from a coaniity-opimizzaionn perspective. For example, recent works (Chen et al. 2018b Liu et al. 2019b a) consider approximating Pt with samples and evolve the samples according to equation 9
Parts of our model will follow this sampling setting.
A.3 AUTOREGRESSIVE FLOWS AND VARIANTS
Our model relies on the concept of autoregressive flows for meta-sampler design. We review some key concepts here. More detailed comparisons are provided in the Appendix.
normalizing flow defines an invertible transformation from one random variable to another z. flexible way implement this is to define it via implicit distributions, meaning sample generation is implemented as: 6 ~ (E:)) z; = G(e Φ), where indexes elements e and z; G represents a deep neural network (generator) pacttrattrpppaaamaeeeee parameterized by ob
The autoregressive flow (AF) parameterizes a Gaussian conditional distribution for each 2,, e.g., p( 21:11) = (21lp,, exp(a:)). where μ4; = Ia, (Z1:-1) and a; 9a, (Z1:-11) are outputs of two neural networks 9u, and 9a.- The sample generation process is: 2; = e; exp(a,) + Ar. with /4 Srg (21x-1),, ai Jo. (21--1) and e; (0, Instances of autoregressive flows include the
15

---

Published as a conference paper at ICLR 2020
Autoregressive Flow (AF) CCenn al. 2017) and Masked Autoregressive Flow (MAF) Papamakarios ct al. (2117). The inverse autoregressive flow (IAF) (Kingma et al. (2016) is an instance of normalizing flow that uses MADE (Germain et al. 2015), whose samples generated as: 2, e, exp(a:) + Pi.
with 14; = Gp. (11.-1)) 0; = Io, (61s 1) and 64 ~ (N, 1).
The neural autoregressive flow (NAF) (Huang et al. 2018 replaces the affine transformation used in the above flows with a deep neural network (DNN), ), i.e., G = f(z,, -()): where f is a DNN transforming a complex sample distribution, p(z), NAF, Yo is considered as a simple prior, and f is an invertible function represented by DNN, whose weights are generated by T. an autoregressive conditional model. Let the induced distribution of by f be (c)). f is learned by minimizing the KL-divergence between py (e) and (o
simple latent representation qo(e). In
A.4 COMPARISONS BETWEEN DIFFERENT FLOWS
Note the μ4 and a; are computed differently for AF and IAF, i.e. previous variables Zi: are used for AF and previous random noise é1:1-1 are used for IAF. AF can be used for calculating the density p(z) of any sample z one pass the network. However, drawing samples requires performing D sequential passes (D is the dimensionality of z). Thus if D is large, drawing samples will be computationally prohibited. IAF, by contrast, can draw samples and estimate densities of the generated samples with only pass of the network. However, calculating the sample density p(z) requires D passes to find the corresponding noise c. The advantage of NAP is that the mapping function f much more expressive, and density evaluation efficient. However, drawing samples much more computationally expensive. To adopt the NAF framework for sampling, we propose the neural inverse-auttoregrssiie flow (NIAF) in Section 2.2.
B DAMS WITH ELBO AS THE OBJECTIVE
Our training algorithm in the patieeiciiieprrrameeeiiaaiooooot setting includes updating the flow parameter and learning the task network with variational inference, which is described in Algorithm and testing setting in Algorithm 2
Algorithm Meta training, MAML with ELBO.
Require: p(T): distribution over tasks;
Require: B step size hyperparameter;
randomly initialize 0, Φ L/.
while not done do Sample batch of tasks T ~ p(T) for all do
Sample K datapoints D, for each task in To
Evaluate ((e.e) L( $ 1)) using ELBO and V p. Φ) with Equation
Compute adapted parameters with gradient descent:
(0', 6() = (. x) (o (0, Φ.
- 3/ a V CT, (0, Φb 1)
Sample datapoints D' for each task in T for the meta-update end for
Update (0. (0. - Noss ΣT (' end while
Algorithm 2 Meta testing, MAML with ELBO.
Require: Training data Ddr for task
Require: Meta learned initialization of Ó Evaluate (( C(O, &, bb) using ELBO and V C(,, &, b)) with Equation g
Compute adapted parameters with gradient descent:
(0', Φ.) (0.)) (o C(O. Φb
V/ = xb a C $.
16

---

Published as a conference paper at ICLR 2020
C
PROOF OF THEOREM PROPOSITION 2
Proof For each task T, we first write out the corresponding WGF as
O.p: V (2.).
(10)
Note the WGF defined in the sense of distributions Ambrosio et al. (2005), meaning that for any smooth real functions u( uzz) with compact support, equation indicates
1/" [ 0uu(z, t)p
T J Vau(z) (V2Fpi)dtdz,
or equivilently,
di / (ru(z)dz = / Veu(z) FaFpddz
Em. [u[z)] = Ep, [Vzu(z) V2F
(11)
Taking p(t) = 37 E 8 (e (z), and for each particle letting u(z) z, equation I| then reduces the following differential equation for each partiele:
(6)
d dz
)
dt
which is the particle evolution equation we need solve.
Proof [Proof of Proposition 2 First, we introduce the following result from Ambrosio et al. (2005) [page 120]:
Lemma Let F(p) & f((()))dz, then the first variation of F w.r.t. p is
5.F (p) F
ōp
In the f-divergence case, this corresponds f(p) (( Applying Lemma 3 and using the chain rule, have
8Ef 0p
(p)
Prr (p d
Pr
8 ()) f ((")=f(()))
Pr dp Pr
Pr
Pr
which completes the proof.
D EXTRA EXPERIMENTAL RESULTS
This section provides extra experimental results to demonstrate the effectiveness of our proposed method.
D.I SYNTHETIC DISTRIBUTIONS
Analytic forms of synthetic distributions are provided below.
Mog4:
p(z) Σ N(z uui 04/
121
17

---

Published as a conference paper at ICLR 2020
drembution:: aget
dretr buten: utor
drstte buttrbutoo:
-25
-10
530
-25
10
15
350
Figure 9: Densities of three Gaussian mixtures.
where z E I2, 111 -12 = =3,00, = -14 [0,3), - [1,
Mog6:
6
p(z) Σ N(z |i; //6
where z E IR2 c4cosii)))), 4 sin (nnniit/))))))) [iT/3]] o = [1,1].
Mog20:
20 p(z) Σ N(z pi, 02)/20
where z E IR2 1 pe meshgrid(z, y), linspacel (6, linspace(-8,8.4), = =],
The distributions are plotted in Figure
D.2 EXTRA EXPERIMENTAL RESULTS ON GAUSSIAN MIXTURE MODELS
For the Gaussian mixture models, provide extra experimental results showing how samples evolved the adaptation process, as illustrated in Figure 10 and
D.3 META POSTERIOR ADAPTION
We apply our DAMS for fast adaptive sampling on regression tasks. We fol-
ldd:
ldd: counts
Idd: counts
counTS
low [Huang et al. (2018) and apply DAMS to sam- ple the posterior distribu- tion of the frequency pa- rameter of a sine-wave function, given only three data points. The sine- wave function is defined as v() = sin(2 ft), with uniform prior ((10, 2]) and a Gaussian likelihood N(; (t.). 0.125).
100
100
Figure 13: Meta sampling Bayesian regression for two test tasks (top and bottom). Each row plots posteriors after meta training (left), meta testing (middle, 0.8K (top) and 0.9K (bottom) iterations) and re-taaniingg (right, 0.8K (top) and 0.9K (bottom) iterations). that
We design a meta- samplian setting to adapt the pos- terior distributions, p(f the training data,
test data. Specifically, meta training data
18

---

Published as a conference paper at ICLR 2020
Cenwerted sents p6
Cenvatal Park
Gentenc purtc er aorlt
Comrenc partc Powect
Cenerated parti 2621
Cenerated partcce Dte
Genertted purti es Powert
Conernted parts Powerit
Consrated sarti p621
Cenernted parteies Or
Generite parte es thert
Conerited parti es Doweltt
Geneersed www.eeaaaaa..... 26
Genwased 2ug
Genemtec partic Paarl)
Genernad partic Pane
Geneersed wwr.eee.. 21.2
Genwased 2ar22
Geneiec purti Pawr
Generned partic Pane
Canoald LANDAA 2-11
Genwaed campad Sryld
Gemeradd sangle Preeon
Genersed sample Srecttt
Figure 10: Comparison among different samplers on adapting to Mixture of 4-Gaussian. Top to Bottom row: DAMS, SGLD-m, SVGD-m, SVGD, SGLD and NNSGHMC
19

---

posterior with modes of Published as a conference paper at ICLR 2020
Target diartoo abor: Lin
Cenmasted Sant 273
GraPe Siibe anth
Commenal pertic wy swrth
<20
232
Taryet dwbitobboe dbr.bbtorrrtt
Consreted sants 2611
Gearee purbber Parti
Conerntd parti Fwart
42
40
42
232
Taryet dbitt dbntbaterrsso
Censreted sarti 2011
Gearne gurces Parft
Conernted parki ts switt
42
40
40
40
144
Target www.batorr
Calacled unpinn
Geta sad sangle Pr
104
401
22
Figure 11: Comparison among different samplers on adapting to Mixture of 20-Gaussian. Top to Bottom row: DAMS, SVGD, SGLD and NNSGHMC
(t, (2)5))))))))
For the first setting, meta testing consists of data ((0,0), (6/5, For the second setting, meta testing consists of data {(0, 0), (4/5, 0), (8/5,0)). Meta training data corresponds For the first setting, the test data corresponds to a posterior with three 50.0.5/6,5/3). For the second setting, the test data corresponds four modes f {0.0, 5/8, 154. We compare our DAMS with the result of re-training from scratch with the test data. Empirical distribution with samples and kernel density estimation are plotted in Figure 13 The first setting takes about 3.4K iterations to find the three modes in the posterior with re-training, while it takes about 0.8K iterations with meta adaptation. For the second setting, it takes more than 3.6K iterations to find the four modes with training from scratch, while it is about 0.9K iterations with meta adaptation. For both test tasks, the sampler with re- re-training miss at least one mode compared with the meta sampler adaptation with the same number of iterations. We can that DAMS can adapt the training posterior to the test posterior much faster than re- training from scratch due to effective uncertainty adaption, obtaining more than 3X speedups.
E (0.0,5/4] modes
20

---

Published as a conference paper at ICLR 2020
Target dierto abar Lin
Conmerted sant 2031
Gearn sitte aart!
Gemrena ae mwct
Target diento deenrbutorr sie
Censreted parts 2011
Genrre purtble Partt
Conertte parts ao wwelt
Target eierto sbar sie
Consreted sarti 2011
Generee purcees partt
Conerited partl ts worlt
Target cwribatoors
Gesecaed UPOINE
Se Streeddd campln Paci
Gesarad sangie Proc
Figure 12: Comparison among different samplers on adapting to Mixture of 6-Gaussian. Top to Bottom row: DAMS, SVGD, SGLD and NNSGHMC
D.4 UNCERTAINTY EVALUATION VIA ENTROPY OF OUT-OF- SAMPLE PREDICTIVE DISTRIBUTIONS
We show the predictive uncertainty of DAMS compared to SGHMC and NNSGHMC by exploring the posterior of neural parameters, we estimate the uncertainty for Lakshminarayann et al. (2017). We train different algorithms on the MNIST dataset, and estimate the entropy the predictive distribution on the notMNIST dataset Bulatov (011). We follow Louizos & Welling (2017a) and use the empirical CDF of entropy to evaluate the uncertainty. Since the probability of observing high confidence prediction is low, curves that are nearer to the bottom right of the figure estimates uncertainty better. The predictive distribution of the trained model is expected to be uniform over the notMNIST digits as the samples from the dataset are from unseen classes. The BNN is CNN with 16 and 50 filters whose kernel sizes are 5 and 5 for the two convolutional layers, respectively. The hidden units of the fully connected layer is 300. The uncertainty evaluation results shown the right plot of Figure
data samples
21

---

Published as a conference paper at ICLR 2020
E IMPLEMENTATION DETAILS
For all the experiments, we follow Liu & Wang 2016b) and use RBF kernel, whose the bandwidth is calculated med? /logn with med the median of the pairwise distance between all the samples.
E. SYNTHETIC DISTRIBUTIONS
The NIAF is fixed with two layers of deep sigmoidal flows (Huang et al., 2018 A small neural network of hidden layers with sigmoid units is used. The model is trained with Adam (Kingma) Ba 2015) with a learning rate of 0.005. We use 1000 particles approximate the distribution. We set the weight À in the WGF to be le-4. The meta sampler is trained for 1000 iterations. After that, the meta sampler is adapted to the target distribution for 500 iterations. The models are trained in the sample-pane papple--rramttrizatioon setting.
& Ba,
E.2 BAYESIAN LOGISTIC REGRESSION
For all the three UCI datasets, the NIAF is fixed with two layers of deep sigmoidal flows Hoang et al.) (2018). The model is trained with Adam Kingma & Ba (2015). We set the batch size to be 100, learning rate to 0.005. We use 100 particles approximate the distribution. We set the weight in the WGF to be le-4. The hidden layer of each MADE has 64 hidden unit. We use use small neural network of hidden layers with 16 sigmoid units. The accuracy selected as the best results. The models trained in the parplee-prramttrizatioon setting.
E.3 CIFAR AND MNIST
The NIAF is fixed with two layers of deep dense sigmoidal flow (Huang et al., 2018). The model is trained with Adam (Kingma & Ba 2015). We set the batch size be 128, and the learning rate be 0.002. We set the weight A in the WGF to be Ie-4. The hidden layer of each MADE has 64 hidden unit. We use use small neural network of hidden layers with 16 sigmoid units. For the auxiliary network, we follow Louizos & Welling (2017b) and use the stable updates of masked RealNVP jet al. (2017) in Inverse Autoregressive Flow (IAF) Kingma et al. (2016) with 50 hidden units of length two to parametrize (z W). The meta sampler is trained for 10 epochs. After that, the meta sampler is adapted to unseen image classes for 200 iterations. For NNSGHMC and our DAMS, 20 meta samples used for testing evaluation.
E.4 FEW SHOT LEARNING
For few shot learning, we adopt the open-source Pytorch MAML implementation https: github com/ rntreasnnt oniooooooooo htt.:///TTTTTatooooMM.MM........ proposed DAMS-NIA
We incorporate the as a meta classifier. The whole training procedure consists of two steps: 1) Train the model to reach the highest validation accuracy. shown Figure 7 a fast adaptive classifier that interacts well with the overall model is acquired. 2) Keep all the flow parameters fixed and continue training until 100 epochs.
E.5 META REINFORCEMENT LEARNING
We evaluate method goal velocity task and goal direction task for cheetah robots. We leverage HalfCheetahDir-v/ environment. We set the fast learning rate (learning rate for the inner loop 1-step gradient update of MAML) be 0.1, max-kl (maximum value for the KL constraint in TRPO) be 0,001, task batch size (batch size for each individual task) to be 20, meta batch size (number of tasks per batch) be 40. For policy network, we have two layers, the first layer is fully connected layer and the second layer is multiplicative parametrization layer following relu activation function.
22


Reviews:
[{"review_id": "SkxuSryk5B", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a Bayesian meta sampling framework consisting of two main components: a meta sampler and a sample adapter. The meta sampler adopted a NIAF structure to generate meta samples, while the sample adapter adapts the samples based on optimal-transport Bayesian sampling. \n\nWhat is the advantage of this Bayesian meta learning methods comparing to the popular none Bayesian methods?  It seems the performance of this Bayesian method is inferior to the state-of-the-art meta-learning methods. What are the specific model structures for T and G in NIAF?\n\nThe experiments can be improved.  First, the comparison methods didn’t appear in all the results. For example, why PMAML is not compared with in Figure 3, Table 1 and Table 2? \nSecond, for the standard meta-learning tasks in Table 3, it is better to compare with the state-of-the-art methods. ", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "SkWqUPoKH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Thank you for an interesting read.\n\nAs far as I understand, this paper presents DAMS which is a MAML-like algorithm but applied to posterior sampling. The idea is the following:\n1. Construct a meta-sampler that generates good proposals/initial samples for task-specific samplers;\n2. Train the meta-sampler so that the task-specific posterior sampling converges faster to the target distribution.\n\nThe meta-sampler is designed as an inverse version of the neural auto-regressive flow (NIAF), and the task-specific sampler is based on the Wasserstein gradient flow (WGF).\n\n======= novelty ======\nThe method is novel:\n1. Probabilistic/Bayesian understanding of few-shot learning/meta learning has been proposed, but to date variational inference is the main inference engine used in the literature. This paper provides a nice complement by considering fast adaptation of posterior sampling.\n2. The meta-level parameterisation method is indeed different from (probabilistic) MAML, in the sense that the initial parameters/samples z is generated from a neural network conditioned on a task, instead of using a shared initialisation across tasks. This is more inline with approaches such as hyper-networks, and I haven't seen many MAML-like approaches doing that. The meta-sampler architecture is new but improved upon NAF so I consider the architectural novelty to be minor.\n\n\n======= significance ======\nThe experimental section contains many results, with the two main category as (a) comparisons between DAMS and existing sampling methods on Bayesian inference tasks; and (b) comparisons to MAML on few-shot learning tasks. Compared with the baselines, DAMS achieves significantly better results, which is a good sign.\n\nHowever, DAMS as a whole pipeline has a lot of components, and it is not clear to me which part is the main driving force. So I think the following ablation studies will be very helpful:\n1. To see whether it is necessary to use NIAF, one can replace NIAF with a set of learned particles shared across tasks (i.e. make \\psi = {z^(1), ..., z^(N)} which is in similar spirit as MAML).\n2. To see whether WGF brings in significant improvement in DAMS, one can replace WGF in sample adapter with SVGD or SG-MCMC method such as SGHMC/preconditioned SGLD. A comparison between e.g. SGHMC vs DAMS-SGHMC vs DAMS-WGF will be helpful for this ablation.\n3. It seems to me the meta-sampler requires running a few step of NAIF updates (\\Gamma info contains previous sample and the gradients). How many steps are required here? A detailed analysis will be useful. If a lot of steps are required, the \"fast adaptation\" in meta-testing is not really the case, as both meta-sampling and sample adaptation requires evaluating gradients.\n4. The multiplicative normalising flow for BNN approximate posterior adds in another layer of complexity, as the method also perform meta-learning on this variational distribution. So a baseline which removes the NIAF part (i.e. also learn particles for masks z, see point 1) on few-shot learning tasks would be useful. As far as I understand this baseline approach is different from ABML/PMAML that has been reported in the paper.\n\n======= clarity =======\nThe presentation needs to be improved. \nTo me, section 3 seems to be overwhelmed by details, e.g. the long discussion of task networks and WGF. For example, to what extent do the WGF construction details matter for understanding the whole DAMS pipeline? I think the general idea works for any valid posterior sampler in the fast adaptation step. \nI would suggest the following structure for section 3 instead:\n1. write down the whole pipeline in a more abstract way, e.g. say the meta-sampler is any generator conditioned on task information, and the sample adaptation as generic posterior sampling;\n2. discuss the training algorithm, what are the loss functions, etc;\n3. discuss in meta-testing how DAMS is deployed; \n3. discuss the detail implementation of the meta-sampler and sample adaptation method.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "HJeI7rOAFB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary of paper:\nThe authors propose a neural sampler for probabilistic models in the meta-learning setting. \nTheir main claim is that their model captures uncertainty in samples better than competing methods and does so at lower cost.\nIn particular, they propose a scheme which separates sampling variables for a task tau into two components: a meta-sampler and a sample adapter.\nThe meta-sampler intuitively plays the role of  a learned conditional distribution over the target variables.\nThe sample adapter is a sampler which is seeded with samples from the meta-sampler and moves them towards the desired data-distribution based on a technique called optimal-transport Bayesian sampling.\nCrucially, the meta-sampler is based on neural inverse autoregressive flows to have adequate representational capacity.\nThe authors use the proposed algorithm (denoted DAMS for distribution agnostic meta sampling) for a variety of tasks: \nFirst, parameters for logistic regression models are generated and evaluated on various UCI tasks.\nSecond, meta-learning over Gaussian mixture models (GMMs) is performed.\nThird, posterior adaption in a toy regression task where the frequency parameter of a sine wave is estimated.\nLast, the authors train neural networks with DAMS and test both classification accuracy as a function of test-time inference for test datasets of held out classes on cifar10, mnist and a few-shot training example on Mini-Imagenet, while also testing their method on meta-reinforcement learning Mujoco tasks.\n\n\nMain Comments to authors:\n\nPros:\n-interesting combination of techniques (IAF flows and WGF/Stein inference) to do meta-sampling\n-empirically appealing results as pertaining to raw performance metrics like accuracy\n\nWeaknesses:\n- The evaluation is focused on #of steps during testing, but not on # of datapoints required for eval on new domains.\n-> this is only half of what we care about when saying a sampler is \"fast\". The other half would be sample efficiency, which is typically the main motivation for Bayesian models and accurate uncertainty estimates, for instance when performing Bayesian Optimization. In fact, when making claims about uncertainty estimates as the goal of the paper, it would be most interesting to see how much test data the method needs to ingest before producing calibrated estimates. The method as currently presented only evaluates speed in terms of computation, but ignores sample efficiency entirely. As such, it is unclear from the given experiments to evaluate the main claim of the paper: that DAMS improves uncertainty adaptation.\n-> the uncertainty is barely evaluated except in the low-d and toy sine wave illustration, which probably can be done equally well or better with regular posterior inference on D = D_train union D_test, i.e. using HMC. My suggestion to the authors would be to consider comparing to regular Bayesian inference (i.e. Monte Carlo/HMC/SVI) based on train_data and test data to compare to a ground truth estimate they might want.\n\n-In Sec. 3.3 Eq. 6 and 7 a kernel is used and then not discussed much further. Kernels on high dimensional data (such as the weights of a NN)  are problematic to be used due to the curse of dimensionality. For the meta-sampler, even in the case of the multiplicative parametrization which lowers dimensionality, this would indicate that the kernel part of the objective might not be doing much work at all as in high enough dimensions all distances become even. If that were to be the case, the model might just look for the mode in any high-d example instead of actually sampling from a posterior and producing uncertainty estimates. Any empirical analysis and discussion on this is entirely missing here, unfortunately. As presented, the reader just has to accept that the objective functions make sense because the final product of putting all of the components together produces high accuracies. I would appreciate more details and careful analysis.\n\n-Please also show the performance of meta-sampler with and without sample adapter in this case to clarify the effects of performing sample adaptation versus just using the meta-sampler, as this also is never compared in the paper. I.e. how good of a conditional model is the autoregressive flow? How much work does the sampler have to do? Would another conditional model do as well or worse? Why this choice of conditional model in particular if in the end sampling is put on top of it?\n\n-The ELBO in Sec.3.4 involves an inference network over NN parameters (or potentially latent Zs per feature when using multiplicative parametrization). This object is highly nontrivial and not analyzed in terms of performance at all here.  Inference networks over neural networks are hard to get right and worthy of entire publications.\n-Please clarify the prior used for BNN models.\n-Please consider using HMC as a baseline for BNN models per task in terms of LLK to compare to DAMS.\n\nBaselines:\n- A lot of this paper relies on comparison to baselines, which are chosen to be mostly from the meta-learning field.\nHowever, in practice the goal of the paper is Bayesian Inference in a particular class of models.\nHence:\n- please consider adding conditional MNF as a baseline. This would clarify if a simple conditional version of MNF would suffice here compared to the involved scheme proposed in this paper and might show the advantage of DAMS over MNF (effectively the main driver for most of the experiments here).\n-Similarly, please consider using conditional NAI flow as a baseline to see how far that gets the reader. \n\n\nPresentation Suggestions:\n-You might want to consider establishing a formal relationship to a hierarchical probabilistic model with plates and show that this is just a way to perform sampling the posterior in a model like: P(y|x, tau) = integral_w P(y|x, w_t) P(w_t|tau) d_w\n-This might help the flow of the paper by setting the stage early, as currently I had to read through it halfway to really understand the task before starting from the beginning to absorb the details of the proposed techniques.\n-figures only readable in pdf, not in printout. Please enlarge fonts to give 'old school' readers a chance\n-page 6  \"..could be calculated effectively..\" What does effectively mean here? Efficiently?\n-page 6 under Theorem 1 typo: 'via the Eulaer scheme' -> Euler scheme\n\nRelated Work Suggestions:\n- Consider citing \"Predictive Uncertainty Quantification with Compound Density Networks\" by Kristiadi et al, as it uses a conditional model with multiplicative parametrization successfully. I understand this is per data-point and the meta-learning scenario is focused on the per-dataset setting, but I find them related enough to consider a discussion.\n- With regards to inference networks on BNNs, please cite \"Latent Projection BNNs: Avoiding weight-space pathologies by learning latent representations of neural network weights\" by Pradier et al, which attempts to do this and also discusses related work in more detail than this paper here. It is a hard task to be done well.\n-The general form of the ELBO shown here is an instance of \"Hierarchical Variational Models\" by Ranganath et al, which should also be cited.\n-Last but not least a recent paper in an ICML workshop on automatic machine learning  (https://sites.google.com/view/automl2019icml/accepted-papers) had a paper on \"Improving Automated Variational Inference with Normalizing Flows\" by Webb et al. This method looks a lot like a baseline method for this paper before task adaptation and WGF is considered and would effectively subsume the first batch of experiments entirely. I would propose the authors cite and discuss differences in detail.\n\n\nDecision:\nThe paper uses a variety of 'puzzle pieces' that are quite involved on their own right. Putting them together and making it work is nontrivial and the authors demonstrate in their experiments that they get strong performance metrics. However, unfortunately, systematic ablation experiments and detailed analysis for the individual components used here and systematic comparisons to simple baselines are not performed. In addition, the paper is presented as a method for adaptive uncertainty quantification, which as argued above is not demonstrated empirically or else. What the paper does achieve is build a pipeline that gets high predictive performance on a meta-learning setting with lower computational requirements during testing than competing methods. I would suggest the authors focus on that aspect and add the required baselines that would clarify what ablations would do to the system and how the components interact.\nAs currently presented, I would argue for rejection since I am not sure of the scientific value of the interplay of components here as regarding uncertainty quantification. However, I think this paper is promising for a slightly different story with small experimental adjustments and would encourage the authors to consider that route.\n\n\n(Edit Post Rebuttal:  revising score given the author response which addressed some of my concerns)", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a meta-learning algorithm that represents uncertainty both at the meta-level and at the task-level. The approach contains an interesting combination of techniques. The reviewers raised concerns about the thoroughness of the experiments, which were resolved in a convincing way in the rebuttal. Concerns about clarity remain, and the authors are *strongly encouraged* to revise the paper throughout to make the presentation more clear and understandable, including to readers who do not have a meta-learning background. See the reviewer's comments for further details on how the organization of the paper and the presentation of the ideas can be improved.
Decision: Accept (Poster)