Year: 2020

Abstract:
Bayesian deep learning is a promising approach towards improved uncertainty quantification and sample efficiency. Due to their complex parameter space, choosing informative priors for Bayesian Neural Networks (BNNs) is challenging. Thus, often a naive, zero-centered Gaussian is used, resulting both in bad generalization and poor uncertainty estimates when training data is scarce. In contrast, meta-learning aims to extract such prior knowledge from a set of related learning tasks. We propose a principled and scalable algorithm for meta-learning BNN priors based on PAC-Bayesian bounds. Whereas previous approaches require optimizing the prior and multiple variational posteriors in an interdependent manner, our method does not rely on difficult nested optimization problems, and moreover, it is agnostic to the variational inference method in use. Our experiments show that the proposed method is not only computationally more efficient but also yields better predictions and uncertainty estimates when compared to previous meta-learning methods and BNNs with standard priors.

heading: 1 INTRODUCTION
text: Bayesian Neural Networks (BNNs) offer a probabilistic interpretation of deep learning by inferring distributions over the model’s weights (Neal, 1996). With the potential of combining the scalability and performance of neural networks (NNs) with a framework for uncertainty quantification, BNNs have lately received increased attention (Blundell et al., 2015; Gal & Ghahramani, 2016). In particular, their ability to express epistemic uncertainty makes them highly relevant for applications such as active learning (Hernández-Lobato & Adams, 2015) and reinforcement learning (Riquelme et al., 2018).
However, BNNs face two major issues: 1) the intractability of posterior inference and 2) the difficulty of choosing good Bayesian priors. While the former has been addressed in an extensive body of literature on variational inference (e.g. Blundell et al., 2015; Blei et al., 2016; Mishkin et al., 2018; Liu & Wang, 2016), the latter has only received limited attention (Vladimirova et al., 2019; Ghosh & Doshi-Velez, 2017). Choosing an informative prior for BNNs is particularly difficult due to the high-dimensional and hardly interpretable parameter space of NNs. Due to the lack of good alternatives, often a zero-centered, isotropic Gaussian is used, reflecting (almost) no a priori knowledge about the problem at hand. This does not only lead to poor generalization when data is scarce, but also renders the Bayesian uncertainty estimates poorly calibrated (Kuleshov et al., 2018).
Meta-learning (Schmidhuber, 1987; Thrun & Pratt, 1998) acquires inductive bias in a data-driven way, thus, constituting an alternative route for addressing this issue. In particular, meta-learners attempt to extract shared (prior) knowledge from a set of related learning tasks (i.e., datasets), aiming to learn in the face of a new, related task. Our work develops a principled and scalable algorithm for metalearning BNN priors. We build on the PAC-Bayesian framework (McAllester, 1999), a methodology from statistical learning theory for deriving generalization bounds. Previous PAC-Bayesian bounds for meta-learners (Pentina & Lampert, 2014; Amit & Meir, 2018) require solving a difficult optimization problem, involving the optimization of the prior as well as multiple variational posteriors in a nested manner. Aiming to overcome this issue, we present a PAC-Bayesian bound that does not rely on nested optimization and, unlike (Rothfuss et al., 2020), can be tractably optimized for BNNs. This makes the resulting meta-learner, referred to as PACOH-NN, not only much more computationally efficient and scalable than previous approaches for meta-learning BNN priors (Amit & Meir, 2018), but also agnostic to the choice of approximate posterior inference method which allows us to combine it freely with recent advances in MCMC (e.g. Chen et al., 2014) or variational inference (e.g. Wang et al., 2019).
Our experiments demonstrate that the computational advantages of PACOH-NN do not result in degraded predictive performance. In fact, across several regression and classification environments, PACOH-NN achieves a comparable or better predictive accuracy than several popular meta-learning approaches, while improving the quality of the uncertainty estimates. Finally, we showcase how metalearned PACOH-NN priors can be used in a real-world bandit task concerning the development of vaccines, suggesting that many other challenging real-world problems may benefit from our approach.

heading: 2 RELATED WORK
text: Bayesian Neural Networks. The majority of research on BNNs focuses on approximating the intractable posterior distribution (Graves, 2011; Blundell et al., 2015; Liu & Wang, 2016; Wang et al., 2019). In particular, we employ the approximate inference method of Liu & Wang (2016). Another crucial question is how to select a good BNN prior (Vladimirova et al., 2019). While the majority of work (e.g. Louizos & Welling, 2016; Huang et al., 2020) employs a simple zero-centered, isotropic Gaussian, Ghosh & Doshi-Velez (2017) and Pearce et al. (2020) have proposed other prior distributions for BNNs. In contrast, we go the alternative route of choosing priors in a data-driven way.
Meta-learning. A range of popular methods in meta-learning attempt to learn the “learning program” in form of a recurrent model (Hochreiter et al., 2001; Andrychowicz et al., 2016; Chen et al., 2017), learn an embedding space shared across tasks (Snell et al., 2017; Vinyals et al., 2016) or the initialization of a NN such that it can be quickly adapted to new tasks (Finn et al., 2017; Nichol et al., 2018; Rothfuss et al., 2019b). A group of recent methods also uses probabilistic modeling to also allow for uncertainty quantification (Kim et al., 2018; Finn et al., 2018; Garnelo et al., 2018). Although the mentioned approaches are able to learn complex inference patterns, they rely on settings where metatraining tasks are abundant and fall short of performance guarantees. In contrast, we provide a formal assessment of the generalization properties of our algorithm. Moreover, PACOH-NN allows for principled uncertainty quantification, including separate treatment of epistemic and aleatoric uncertainty. This makes it particularly useful for sequential decision algorithms (Lattimore & Szepesvari, 2020).
PAC-Bayesian theory. Previous work presents generalization bounds for randomized predictors, assuming a prior to be given exogenously (McAllester, 1999; Catoni, 2007; Germain et al., 2016; Alquier et al., 2016). More recent work explores data-dependent priors (Parrado-Hernandez et al., 2012; Dziugaite & Roy, 2016) or extends previous bounds to the scenario where priors are meta-learned (Pentina & Lampert, 2014; Amit & Meir, 2018). However, these meta-generalization bounds are hard to minimize as they leave both the hyper-posterior and posterior unspecified, which leads to nested optimization problems. Our work builds on the results of Rothfuss et al. (2020) who introduce the methodology to derive the closed-form solution of the PAC-Bayesian meta-learning problem. However, unlike ours, their approach suffers from (asymptotically) non-vanishing terms in the bounds and relies on a closedform solution of the marginal log-likelihood. By contributing a numerically stable score estimator for the generalized marginal log-likelihood, we are able to overcome such limitations, making PACBayesian meta-learning both tractable and scalable for a much larger array of models, including BNNs.

heading: 3 BACKGROUND: THE PAC-BAYESIAN FRAMEWORK
text: Bayesian Neural Networks. Consider a supervised learning task with data S = {(xj , yj)}mj=1 drawn from unknown distribution D. In that, X = {xj}mj=1 ∈ Xm denotes training inputs and Y = {yj}mj=1 ∈ Ym the targets. For brevity, we also write zj := (xj , yj) ∈ Z . Let hθ : X → Y be a function parametrized by a NN with weights θ ∈ Θ. Using the NN mapping, we define a conditional distribution p(y|x, θ). For regression, we set p(y|x, θ) = N (y|hθ(x), σ2), where σ2 is the observation noise variance. For classification, we choose p(y|x, θ) = Categorical(softmax(hθ(x))). For Bayesian inference, one presumes a prior distribution p(θ) over the model parameters θ which is combined with the training data S into a posterior distribution p(θ|X,Y) ∝ p(θ)p(Y|X, θ). For unseen test data points x∗, we form the predictive distribution as p(y∗|x∗,X,Y) = ∫ p(y∗|x∗, θ)p(θ|X,Y)dθ.
The Bayesian framework presumes partial knowledge of the data-generating process in form of a prior distribution. However, due to the practical difficulties in choosing an appropriate BNN prior, the prior is typically strongly misspecified (Syring & Martin, 2018). As a result, modulating the influence of the prior relative to the likelihood during inference typically improves the empirical performance of BNNs and is thus a common practice (Wenzel et al., 2020). Using such a ”tempered” posterior pτ (θ|X,Y) ∝ p(θ)p(Y|X, θ)τ with τ > 0 is also referred to as generalized Bayesian learning (Guedj, 2019).
The PAC-Bayesian Framework. In the following, we introduce the most relevant concepts of PAC-Bayesian learning theory. For more details, we refer to Guedj (2019). Given a loss function l : θ ×Z → R, we typically want to minimize the generalization error L(θ,D) = Ez∗∼D l(θ, z∗). Since D is unknown, the empirical error L̂(θ, S) = 1m ∑m i=1 l(θ, zi) is usually employed, instead. In the PAC-Bayesian framework, we are concerned with randomized predictors, i.e., probability measures on the parameter space Θ, allowing us to reason about epistemic uncertainty. In particular, we consider two such probability measures, a prior P ∈ M(Θ) and a posterior Q ∈ M(Θ). In here, byM(Θ), we denote the set of all probability measures on Θ. While in Bayesian inference, the prior and posterior are tightly coupled through Bayes’ theorem, the PAC-Bayesian framework only requires the prior to be independent of the data S. Using the definitions above, the so-called Gibbs error for a randomized predictor Q is defined as L(Q,D) = Eh∼Q L(h,D). Similarly, we define its empirical counterpart as L̂(Q,S) = Eh∼Q L̂(h, S). The PAC-Bayesian framework provides upper bounds for the unknown Gibbs error in the following form: Theorem 1. (Alquier et al., 2016) Given a data distribution D, a prior P ∈ M(Θ), a confidence level δ ∈ (0, 1], with probability at least 1− δ over samples S ∼ Dm, we have:
∀Q ∈M(Θ) : L(Q,D) ≤ L̂(Q,S) + 1√ m
[ DKL(Q||P ) + ln 1
δ + Ψ(
√ m) ] (1)
where Ψ( √ m) = lnEθ∼PES∈Dm exp [√ m ( L(θ,D)− L̂(θ, S) )] .
In that, Ψ( √ m) is a log moment generating function that quantifies how strong the empirical error deviates from the Gibbs error. By making additional assumptions about the loss function l, we can bound Ψ( √ m) and thereby obtain tractable bounds. For instance, if l(θ, z) is bounded in [a, b], we obtain Ψ( √ m) ≤ ((b− a)2)/8 by Hoeffding’s lemma. For unbounded loss functions, it is common to assume bounded moments. For instance, a loss is considered sub-gamma with variance factor s2 and scale parameter c, under a prior P and data distribution D, if its deviations from the mean can be characterized by random variable V := L(h,D)− l(h, z) whose moment generating function is upper bounded by that of a Gamma distribution Γ(s, c) (Boucheron et al., 2013). In such case, we obtain Ψ( √ m) ≤ s2/(2− 2c√
m )).
Connecting the PAC-Bayesian framework and generalized Bayesian learning. In PACBayesian learning we aim to find the posterior that minimizes the bound in (1) which is in general a challenging optimization problem over the space of measuresM(Θ). However, to our benefit, it can be shown that the Gibbs posterior is the probability measure that minimizes (1). For details we refer to Lemma 2 in the Appendix or Catoni (2007) and (Germain et al., 2016). In particular, this gives us
Q∗(θ) := arg min Q∈M(Θ)
√ mL̂(Q,S) +DKL(Q||P ) = P (θ)e− √ mL̂(θ,S)/Z(S, P ) , (2)
where Z(S, P ) is a normalization constant. In a probabilistic setting, our loss function is the negative log-likelihood, i.e. l(θ, zi) := − log p(zi|θ). In this case, the optimal Gibbs posterior coincides with the generalized Bayesian posterior Q∗(θ;P, S) ∝ P (θ)p(S|θ)1/ √ m/Z(S, P ) where
Z(S, P ) = ∫
Θ P (θ)p(S|θ)1/
√ m dθ is the generalized marginal likelihood of the data sample S.

heading: 4 PAC-BAYESIAN BOUNDS FOR META-LEARNING
text: This section describes the PAC-Bayesian meta-learning setup and discusses how we can obtain generalization bounds that can be transformed into practically useful meta-learning objectives. In that, we draw on the methodology of Rothfuss et al. (2020) which allows us to derive a closed form solution of the PAC-Bayesian meta-learning problem.
In the standard supervised learning setup (see Sec. 3), the learner has prior knowledge in the form of a prior distribution P , given exogenously, over the hypothesis spaceH. When the learner faces a new task, it uses the evidence, provided as a dataset S, to update the prior into a posterior distribution Q. We formalize such a base learner Q(S, P ) as a mapping Q : Zm ×M(H)→M(H) that takes in a dataset and prior and outputs a posterior.
In contrast, in meta-learning we aim to acquire such a prior P in a data-driven manner, that is, by consulting a set of n statistically related learning tasks {τ1, ..., τn}. We follow the setup of Baxter (2000) in which all tasks τi := (Di, Si) share the same data domain Z := X × Y , parameter space Θ and loss function l(θ, z), but may differ in their (unknown) data distributions Di and the number of points
mi in the corresponding dataset Si ∼ Dmii . For our theoretical expositions, we assume w.l.o.g that m = mi ∀i. Further, each task τi ∼ T is drawn i.i.d. from an environment T , a probability distribution over data distributions and datasets. The goal is to extract knowledge from the observed datasets which can then be used as a prior for learning on new target tasks τ ∼ T (Amit & Meir, 2018). To extend the PAC-Bayesian analysis to the meta-learning setting, we again consider the notion of probability distributions over hypotheses/parameters. However, while the object of learning has previously been the NN parameters θ, it is now the prior distribution P ∈M(Θ). Accordingly, the meta-learner presumes a hyper-priorP ∈M(M(H)), i.e., a distribution over priors P . Combining the hyper-prior P with the datasets S1, ..., Sn from multiple tasks, the meta-learner then outputs a hyper-posterior Q over priors. The hyper-posterior’s performance/quality is measured in form of the expected Gibbs error when sampling priors P from Q and applying the base learner, the so-called transfer-error:
L(Q, T ) := EP∼Q [ E(D,m)∼T [ES∼Dm [L(Q(S, P ),D)]] ] While the transfer error is unknown in practice, we can estimate it using the empirical multi-task error
L̂(Q, S1, ..., Sn) := EP∼Q [ 1
n n∑ i=1
L̂ (Q(Si, P ), Si) ] .
Similar to the PAC-Bayesian guarantees for single task learning, we can bound the transfer error by its empirical counterpart L̂(Q, S1, ..., Sn) plus several tractable complexity terms: Theorem 2. Let Q : Zm ×M(H) →M(H) be a base learner and P ∈ M(M(H)) some fixed hyper-prior. For all hyper-posteriors Q ∈M(M(H)) and δ ∈ (0, 1],
L(Q, T ) ≤ L̂(Q, S1, ..., Sn) + ( 1
n √ m + 1√ n
) DKL(Q||P)
+ 1
n n∑ i=1 1√ m EP∼Q [DKL(Q(Si, P )||P )] + C(δ, n,m)
(3)
holds with probability 1− δ. If the loss function is bounded, that is l : H×Z → [a, b], the above inequality holds with C(δ, n,m) = (bk−ak) 2
8 √ m
+ (bk−ak) 2
8 √ n − 1√ n ln δ. If the loss function is sub-gamma
with variance factor s2I and scale parameter cI for data distributions Di and s2II, cII for the task distribution T , the inequality holds with C(δ, n,m) = s 2 I
2( √ m−cI) + s2II 2( √ n−cII) − 1√ n ln δ .
Under bounded loss assumption, Theorem 2 provides a structurally similar, but tighter bound than Pentina & Lampert (2014) and Rothfuss et al. (2020). In particular, by using a improved proof methodology, we are able to forgo a union bound argument, allowing us to reduce the negative influence of confidence parameter δ. Compared to Rothfuss et al. (2020), the DKL(Q,P) term has an improved decay rate, that is, 1/(n √ m) + 1/ √ n as opposed to 1/ √ m+ 1/ √ n. Importantly, the bound in (3) is consistent, i.e. C(δ, n,m) → 0 as n,m → ∞. Unlike Pentina & Lampert (2014) and Amit & Meir (2018), the theorem also provides guarantees for unbounded loss functions under moment constraints (see Appendix A.1 for details). This makes Theorem 2 particularly relevant for probabilistic models such as BNNs in which the loss function coincides with the inherently unbounded negative log-likelihood.
Amit & Meir (2018) propose to meta-learn NN priors by directly minimizing a bound similar to (3). However, the posterior inference for BNNs, i.e. obtainingQi = Q(Si, P ), is a stochastic optimization problem in itself whose solution in turn depends on P . Hence, minimizing such meta-level bound w.r.t. P constitutes a computationally infeasible two-level optimization problem. To circumvent this issue, they jointly optimize P and n approximate posteriors Qi that correspond to the different datasets Si, leading to an unstable and poorly scalable meta-learning algorithm.
To overcome these issues, we employ the methodology of Rothfuss et al. (2020) and assume the Gibbs posterior Q∗(Si, P ) as a base learner. As discussed in Section 3, Q∗(Si, P ) does not only constitute a generalized Bayesian posterior but also minimizes the PAC-Bayesian bound. Thus, the resulting bound in Corollary 1 is tighter than (3). More importantly, the bound can be stated in terms of the partition function Z(Si, P ) which allows us to forgo the explicit reliance on the task posteriors Qi. This makes the bound much easier to optimize as a meta-learning objective than previous bounds (e.g. Pentina & Lampert, 2014; Amit & Meir, 2018), since it no longer constitutes a two-level optimization problem. Moreover, it renders the corresponding meta-learner agnostic
to the choice of approximate inference method used to approach the Gibbs/Bayes posterior Q∗(S, P ) when performing inference on a new task. Corollary 1. When choosing the Gibbs posteriorQ∗(Si, P ) := P (θ) exp(− √ mL̂(Si, θ))/Z(Si, P ) as a base learner, under the same assumptions as in Theorem 2, it holds with probability 1− δ that
L(Q, T ) ≤ − 1 n n∑ i=1 1√ m EP∼Q [lnZ(Si, P )] + ( 1√ n + 1 n √ m ) DKL(Q||P) + C(δ, n,m) (4)
wherein Z(Si, P ) = Eθ∼P [ exp(−√mL̂(Si, θ)) ] is the generalized marginal log-likelihood.
A natural way to obtain a PAC-Bayesian meta-learning algorithm could be to minimize (4) w.r.t. Q. Though, in general, this is a hard problem since it would require a minimization overM(M(H)), the space of all probability measures over priors. Following Rothfuss et al. (2020), we exploit once more the insight that the minimizer of (4) can be written as Gibbs distribution (c.f. Lemma 2), allowing us to to derive such minimizing hyper-posterior Q∗, i.e. the PACOH, in closed form: Proposition 1. (PAC-Optimal Hyper-Posterior) Given a hyper-prior P and datasets S1, ..., Sn, the hyper-posterior Q that minimizes the PAC-Bayesian meta-learning bound in (4) is given by
Q∗(P ) = P(P ) exp
( 1√
nm+1
∑n i=1 lnZ(Si, P ) ) Z II(S1, ..., Sn,P)
(5) with the partition function defined as Z II = EP∼P [ exp ( 1√
nm+1
∑n i=1 lnZ(Si, P ) )] .
We refer to Q∗(P ) as PAC-optimal since, among all meta-learners, it gives us the best possible PAC-Bayesian guarantees induced by Theorem 2.

heading: 5 PACOH-NN: A SCALABLE ALGORITHM FOR LEARNING BNN PRIORS
text: We discuss our main contribution, that is, how to translate the PACOH (Prop. 1) into a practical algorithm for meta-learning BNN priors. To this end, we first specify various components of the generic meta-learning setup presented in Sec. 4 and then discuss how to obtain a tractable approximation of Q∗. The setup. First, we define a family of priors {Pφ : φ ∈ Φ} over the NN parameters θ. For computational convenience, we employ Gaussian priors with diagonal covariance matrix, i.e. Pφ = N (µP , diag(σ2P )) with φ := (µP , log σP ). Note that we represent σP in the log-space to avoid additional positivity constraints. In fact, any parametric distribution such as normalizing flows (Rezende & Mohamed, 2015) that allows for re-parametrized sampling and has a tractable log-density could be used. As typical in the Bayesian framework, our loss function is the negative log-likelihood, i.e. l(θ, z) = − ln p(y|x, θ) for which we assume an additive Gaussian noise model p(y|x, θ) = N (y;hθ(x), σ2y) in regression and a categorical softmax distribution in case of classification. Moreover, we use a zero-centered, spherical Gaussian P := N (0, σ2PI) as a hyper-prior over the parameters φ that specify the prior. In our setup, the hyper-prior acts a form of meta-level regularization that penalizes complex priors.
Approximating the hyper-posterior. Given the hyper-prior and (level-I) log-partition function lnZ(Si, P ), we can compute the PACOH Q∗ up to the normalization constant Z II. Such a setup lends itself to approximate inference methods (Blei et al., 2016). In particular, we employ Stein Variational Gradient Descent (SVGD) (Liu & Wang, 2016) which approximates Q∗ as a set of particles Q̂ = {φ1, ..., φK}. Initially, the particles φK ∼ P (i.e. the priors’ parameters) are sampled randomly. Subsequently, the method iteratively transports the set of particles to match Q∗, by applying a form of functional gradient descent that minimizes DKL(Q̂|Q∗) in the reproducing kernel Hilbert space induced by a kernel function k(·, ·). In each iteration, the particles are updated by φk ← φk + ηψ∗(φk) with the step size η and
ψ∗(φ) = 1
K K∑ k′=1 [ k(φk′ , φ)∇φk′ lnQ∗(φk′) +∇φk′k(φk′ , φ) ] . (6)
In that,∇φk lnQ∗(φk) = ∇φk lnP(φk) + 1√nm̃+1 ∑n i=1∇φk lnZ(Si, Pφk) is the score of Q∗.
Approximating the generalized marginal log-likelihood. The last remaining issue towards a viable meta-learning algorithm is the intractable generalized marginal likelihood lnZ(Si, Pφ) =
lnEθ∼Pφe− √ miL̂(θ,Si). Estimating and optimizing lnZ(Si, Pφ) is not only challenging due to the high-dimensional expectation over Θ but also due to numerical instabilities inherent in computing e− √ miL̂(θ,Si) when mi is large. Aiming to overcome these issues, we compute numerically stable Monte Carlo estimates of ∇φ lnZ(Si, Pφk) by combining the LogSumExp (LSE) with the reparametrization trick (Kingma & Welling, 2014). In particular, we draw L samples θl := f(φ, l) = µP + σP l, l ∼ N(0, I) and compute the forward pass as follows:
ln Z̃(Si, Pφ) := ln 1
L L∑ l=1 e− √ miL̂(θl,Si) = LSELl=1 ( −√miL̂(θl, Si) ) − lnL , θl ∼ Pφ (7)
The corresponding gradients follow as softmax-weighted average of score gradients:
∇φ ln Z̃(Si, Pφ) = − √ mi L∑ l=1 e− √ miL̂(θl,Si)∑L
l=1 e −√miL̂(θl,Si)︸ ︷︷ ︸ softmax ∇φf(φ, l)>︸ ︷︷ ︸ re-param. Jacobian ∇θlL̂(θl, Si)︸ ︷︷ ︸ score
(8)
Note that ln Z̃(Si, Pφ) is a consistent but not an unbiased estimator of lnZ(Si, Pφ). The following proposition ensures us that we still minimize a valid bound (see Appx. B.3 for details).
Proposition 2. In expectation, replacing lnZ(Si, Pφ) in (4) by the estimate ln Z̃(Si, P ) still yields a valid upper bound of the transfer error L(Q, T ) for any L ∈ N. Moreover, by the law of large numbers, we have that ln Z̃(Si, P )
a.s.−−→ lnZ(Si, P ) as L→∞, that is, for large sample sizes L, we recover the original PAC-Bayesian bound in (4). In the opposite edge case, i.e. L = 1, the boundaries between tasks vanish meaning that the meta-training data {S1, ..., Sn} is treated as if it were one large dataset ⋃ i Si (see Appx. B.3 for further discussion).
The algorithm. Algorithm 1 in Appendix B summarizes the proposed meta-learning method which we henceforth refer to as PACOH-NN. To estimate the score ∇φk′ lnQ∗(φk′) in (6), we can even use mini-batching on the task level. This mini-batched version, outlined in Algorithm 2, maintains K particles to approximate the hyper-posterior, and in each forward step samples L NN-parameters (of dimensionaly |Θ|) per prior particle that are deployed on a mini-batch of nbs tasks to estimate the score of Q∗. As a result, the total space complexity is in the order of O(|Θ|K + L) and the computational complexity of the algorithm for a single update (c.f. (6)) is O(K2 +KLnbs). A key advantage of PACOH-NN over previous methods for meta-learning BNN priors (e.g. Pentina & Lampert, 2014; Amit & Meir, 2018) is that it turns the previously nested optimization problem into a much simpler stochastic optimization problem. This makes meta-learning not only much more stable but also more scalable. In particular, we do not need to explicitly compute / maintain the task posteriors Qi and can do mini-batching over tasks. As a result, the space and compute complexity do not depend on the number of tasks n. In contrast, MLAP (Amit & Meir, 2018) has a memory footprint of O(|Θ|n) making meta-learning prohibitive for more than 50 tasks. A central feature of PACOH-NN is that is comes with principled meta-level regularization in form of the hyper-prior P which combats overfitting to the meta-training tasks (Qin et al., 2018). As we show in our experiments, this allows us to successfully perform meta-learning with as little as 5 tasks. This is unlike the the majority of the popular meta-learners (Finn et al., 2017; Kim et al., 2018; Garnelo et al., 2018, e.g.) which rely on a large number of tasks to generalize well on the meta-level (Qin et al., 2018; Rothfuss et al., 2020).

heading: 6 EXPERIMENTS
text: We empirically evaluate the method introduced in Section 5, in particular, two variants of the algorithm: PACOH-NN-SVGD with K = 5 priors as particles and the edge case K = 1 which
coincides with a maximum-a-posteriori (MAP) approximation of Q∗, thus referred to as PACOHNN-MAP. In order to evaluate the quality of the meta-learned prior, i.e. meta-testing, we need to do approximate the BNN posterior Q∗(S, P ) for which we use SVGD with 5 particles, too.
Comparing it to various NN-based meta-learning approaches on various regression and classification environments, we demonstrate that PACOH-NN (i) outperforms previous meta-learning algorithms in terms of predictive accuracy, (ii) improves the quality of uncertainty estimates and (iii) is much more scalable than previous PAC-Bayesian meta-learners. Finally, we showcase how meta-learned PACOHNN priors can be harnessed in a real-world bandit task concerning peptide-based vaccine development.

heading: 6.1 META-LEARNING BNN PRIORS FOR REGRESSION AND CLASSIFICATION
text: Figure 1 illustrates BNN predictions on a sinusoidal regression task with a standard Gaussian prior as well as a PACOH-NN prior meta-learned on sinusoidal functions of varying amplitude, phase and slope (details can be found in Appendix C.1). In Figure 1a we can see that the standard Gaussian prior provides poor inductive bias, not only leading to bad mean predictions away from the testing points but also to poor 95% confidence intervals (blue shaded areas). In contrast, the meta-learned PACOH-NN prior encodes useful inductive bias towards sinusoidal function shapes, leading to good predictions and uncertainty estimates, even in face of minimal training data (i.e. 1 training point).
Meta-learning benchmark. In the following, we present a comprehensive benchmark study. First, we use a BNN with a zero-centered, spherical Gaussian prior and SVGD posterior inference (Liu & Wang, 2016) as a baseline. Second, we compare our proposed approach against various popular meta-learning algorithms, including model-agnostic meta-learning (MAML) (Finn et al., 2017), its first-order version (FOMAML) (Nichol et al., 2018), Bayesian MAML (BMAML) (Kim et al., 2018) and two variants of the PAC-Bayesian approach by Amit & Meir (2018) (MLAP). For experiments with regression tasks, we also include into our comparison neural processes (NPs) (Garnelo et al., 2018) and the GP based meta-learner of Rothfuss et al. (2020) (PACOH-GP). The latter, is similar to our method as it also approximates a form of the PAC-optimal Hyper-Posterior with SVGD. However, unlike PACOH-NN it uses Gaussian Processes (GPs) as base learners and relies on a closed-form marginal log-likelihood. Among all, MLAP is the most similar to our approach as it is neural network based and minimizes PAC-Bayesian bounds of the transfer error. Though, unlike PACOH-NN, it relies on nested optimization of the task posteriors Qi and the hyper-posterior Q. Regression environments. In our experiments, we consider one synthetic and four real-world metalearning environments for regression. As a synthetic environment we follow Rothfuss et al. (2020), employing a 2-dimensional mixture of Cauchy distributions plus random GP functions. As real-world environments, we use datasets corresponding to different calibration sessions of the Swiss Free Electron Laser (SwissFEL) (Milne et al., 2017; Kirschner et al., 2019b), as well as data from the PhysioNet 2012 challenge, which consists of time series of electronic health measurements from
intensive care patients (Silva et al., 2012), in particular the Glasgow Coma Scale (GCS) and the hematocrit value (HCT). Here, the different tasks correspond to different patients. Moreover, we employ the Intel Berkeley Research lab temperature sensor dataset (Berkeley-Sensor) (Madden, 2004) where the tasks require auto-regressive prediction of temperatures measurements corresponding to sensors installed in different locations of the building. Further details can be found in Appendix C.1.
Table 1 reports the results of our benchmark study in terms of the root mean squared error (RMSE) on unseen test tasks. Among the approaches, PACOH-NN consistently performs best or is among the best two methods, demonstrating that the introduced meta-learning framework is not only sound, but also endows us with an algorithm that works well in practice. Only for low-dimensional and small-scale regression environments like Physionet, we find that the GP-based meta-learner PACOH-GP which is build on a similar theoretical foundation as our method works better than PACOH-NN.
Further, we hypothesize that by acquiring the prior in a principled data-driven manner (e.g., with PACOH-NN), we can improve the quality of the BNN’s uncertainty estimates. To investigate the effect of meta-learned priors on the uncertainty estimates of the BNN, we compute the probabilistic predictors’ calibration error, reported in Table 2. The calibration error measures the discrepancy between predicted confidence regions and actual frequencies of test data in the respective areas (Kuleshov et al., 2018). Note that, since MAML only produces point predictions, the concept of calibration does not apply to it. First, we observe that meta-learning priors with PACOH-NN consistently improves the standard BNN’s uncertainty estimates. For meta-learning environments where the task similarity is high, i.e. SwissFel and Berkeley-Sensor, the improvement is substantial. Surprisingly, while improving upon the standard BNN in terms of the RMSE, we find that NPs consistently yields worse-calibrated predictive distributions than the BNN without meta-learning. This may be due to meta-level overfitting as NPs lack any form of meta-level regularization (cf. Qin et al., 2018; Rothfuss et al., 2020).
Classification environments. We conduct experiments with the multi-task classification environment Omniglot (Lake et al., 2015), consisting of handwritten letters across 50 alphabets. Unlike previous work (e.g. Finn et al., 2017) we do not perform data-augmentation and do not re-combine letters of different alphabets, preserving the data’s original structure. In particular, one task corresponds to 5-way classification of letters within an alphabet. This leaves us with much fewer tasks (i.e. 30 train and 20 test tasks), making the environment more challenging and more interesting for uncertainty quantification. This also allows us to include MLAP in the experiment which hardly scales to more than 50 tasks.
In Table 3, we report both the accuracy and calibration error for 2-shot and 5-shot classification on test tasks. Again, PACOHNN yields the most accurate classification results and the lowest calibration error. Note that MAML fails to improve upon the standard BNN, i.e. demonstrating negative transfer. This is consistent with previous work (Qin et al., 2018; Rothfuss et al., 2020) raising concerns about overfitting to the meta-training tasks and observing that MAML requires a large number of tasks to generalize well. In contrast, by its very construction on meta-generalization
bounds, PACOH-NN is able to achieve positive transfer even when the meta-training tasks are diverse and small in number.
Scalability. Unlike the MLAP (Amit & Meir, 2018), PACOH-NN does not need to maintain posteriors Qi for the meta-training tasks and can use mini-batching on the task level. As a result, it is computationally much faster and more scalable than previous PAC-Bayesian meta-learners. This is reflected in its computation and memory complexity, discussed in Section 5. Figure 2 showcases this computational advantage during meta-training with PACOH-NN-MAP and MLAP-S in the Sinusoids environment with varying number of tasks, reporting the maximum memory requirements, as well as the training time. While MLAP’s memory consumption and compute time grow linearly and becomes prohibitively large even for less than 100 tasks, PACOH-NN maintains a constant memory and compute load as the number of tasks grow.

heading: 6.2 META-LEARNING FOR BANDITS - VACCINE DEVELOPMENT
text: We showcase how a relevant real-world application such as vaccine design can benefit from our proposed method. In particular, the goal is to discover peptide sequences which bind to major histocompatibility complex class-I molecules (MHC-I). MHC-I molecules present fragments of proteins from within a cell to T-cells, allowing the immune system to distinguish between healthy and infected cells. Following the bandit setup of Krause & Ong (2011), each task corresponds to searching for maximally binding peptides, a vital step in the design of peptide-based vaccines. The tasks differ in their targeted MHC-I allele, i.e., correspond to different genetic variants of the MHC-I protein. We use data from Widmer et al. (2010) which contains the binding affinities (IC50values) of many peptide candidates to the MHC-I alleles. The peptide candidates are encoded as 45-dimensional feature vector and the binding affinities were standardized.
We use 5 alleles (tasks) to meta-learn a BNN prior with PACOH-NN and leave the most genetically dissimilar allele (A-6901) for our bandit task. In each iteration, the experimenter (i.e. bandit algorithm) chooses to test one peptide among the pool of more than 800 candidates and receives its binding affinity as a reward feedback. In particular, we employ UCB (Lattimore & Szepesvari, 2020) and ThompsonSampling (TS) (Thompson, 1933) as bandit algorithms, comparing the BNN-based bandits with meta-learned prior (PACOH-UCB/TS) against a zero-centered Gaussian BNN prior
(BNN-UCB/TS) and a Gaussian process (GP-UCB) (Srinivas et al., 2009).
Figure 3 reports the respective average regret and simple regret over 50 iterations. Unlike the bandit algorithms with standard BNN/GP prior, PACOH-UCB/TS reaches near optimal regret within less than 10 iterations and after 50 iterations still maintains a significant performance advantage. This highlights the importance of transfer (learning) for solving real-world problems and demonstrates the effectiveness of PACOH-NN to this end. While the majority of meta-learning methods rely on a large number of meta-training tasks (Qin et al., 2018), PACOH-NN allows us to achieve promising positive transfer, even in complex real-world scenarios with only a handful (in this case 5) tasks.

heading: 7 CONCLUSION
text: Based on PAC-Bayesian theory, we present a novel, scalable algorithm for meta-learning BNN priors, that overcomes previous issues of nested optimization, by employing the closed-form solution of the PAC-Bayesian meta-learning problem. Experiments show that our method, PACOH-NN, does not only come with computational advantages, but also achieves comparable or better predictive accuracy than several popular meta-learning approaches, while improving the quality of the uncertainty estimates – a key aspect of our approach. The benefits of our principled treatment of uncertainty – showcased in the real-world vaccine development bandit task – are particularly amenable to interactive machine learning systems. This makes the integration of PACOH-NN with Bayesian optimization and reinforcement learning a potentially promising avenue to pursue. While our experiments are limited to diagonal Gaussian priors and SVGD as approximate inference method, we hope that future work will build on the added flexibility of our framework to possibly explore more recent approaches in variational inference (e.g. Wang et al., 2019) or consider more expressive priors such as normalizing flows (Rezende & Mohamed, 2015).

heading: APPENDIX

heading: A PROOFS AND DERIVATIONS

heading: A.1 PROOF OF THEOREM 2
text: Lemma 1. (Change of measure inequality) Let f be a random variable taking values in a set A and let X1, ..., Xl be independent random variables, with Xk ∈ A with distribution µk. For functions gk : A× A→ R, k = 1, ..., l, let ξk(f) = EXk∼µk [gk(f,Xk)] denote the expectation of gk under µk for any fixed f ∈ A. Then for any fixed distributions π, ρ ∈M(A) and any λ > 0, we have that
Ef∼ρ
[ l∑
k=1
ξk(f) − gk(f,Xk) ] ≤ 1 λ ( DKL(ρ||π) + lnEf∼π [ eλ (∑l k=1 ξk(f)−gk(f,Xk) )]) .
(9)
Proof of Theorem 2 To prove the Theorem, we need to bound the difference between transfer error L(Q, T ) and the empirical multi-task error L̂(Q, S1, ..., Sn). To this end, we introduce an intermediate quantity, the expected multi-task error:
L̃(Q,D1, ...,Dn) = EP∼Q [ 1
n n∑ i=1 ES∼Dmii [L(Q(S, P ),Di)] ]
(10)
In the following we invoke Lemma 1 twice. First, in step 1, we bound the difference between L̃(Q,D1, ...,Dn) and L̂(Q, S1, ..., Sn), then, in step 2, the difference between L(Q, T ) and L̃(Q,D1, ...,Dn). Finally, in step 3, we use a union bound argument to combine both results.
Step 1 (Task specific generalization) First, we bound the generalization error of the observed tasks τi = (Di,mi), i = 1, ..., n, when using a learning algorithm Q : M×Zmi → M, which outputs a posterior distribution Qi = Q(Si, P ) over hypotheses θ, given a prior distribution P and a dataset Si ∼ Dmii of size mi. In that, we define m̃ := ( ∑n i=1m −1 i ) −1 as the harmonic mean of sample sizes.
In particular, we apply Lemma 1 to the union of all training sets S′ = ⋃n i=1 Si with l = ∑n i=1mi. Hence, each Xk corresponds to one data point, i.e. Xk = zij and µk = Di. Further, we set f = (P, h1, ..., hn) to be a tuple of one prior and n base hypotheses. This can be understood as a two-level hypothesis, wherein P constitutes a hypothesis of the meta-learning problem and hi a hypothesis for solving the supervised task τi. Correspondingly, we take π = (Q, Qn) = P · ∏n i=1 P
and ρ = (Q, Qn) = Q ·∏ni=1Qi as joint two-level distributions and gk(f,Xk) = 1nmi l(hi, zij) as summand of the empirical multi-task error. We can now invoke Lemma 1 to obtain that (11) and (14)
1
n n∑ i=1 EP∼Q [L(Qi, Di)] ≤ 1 n n∑ i=1 EP∼Q [L(Qi, Si)] + 1 λ ( DKL [(Q, Qn)||(P, Pn)]
+ lnEP∼PEh∼P [ e λ n ∑n i=1(L(h,Di)−L̂(h,Si) ]) (11) Using the above definitions, the KL-divergence term can be re-written in the following way:
DKL [(Q, Qn)||(P, Pn)] = EP∼Q [ Eh∼Qi [ ln Q(P )∏ni=1∏Qi(h) P(P )∏ni=1 P (h) ]] (12)
= EP∼Q [ ln Q(P ) P(P ) ] + n∑ i=1 EP∼Q [ Eh∼Qi [ ln Qi(h) P (h) ]] (13)
= DKL(Q||P) + n∑ i=1 EP∼Q [DKL(Qi||P )] (14)
Using (11) and (14) we can bound the expected multi-task error as follows:
L̃(Q,D1, ...,Dn) ≤ L̂(Q, S1, ..., Sn) + 1
λ DKL(Q||P) +
1
λ n∑ i=1 EP∼Q [DKL(Qi||P )]
+ 1
λ lnEP∼PEh∼P
[ e λ n ∑n i=1(L(h,Di)−L̂(h,Si)) ] ︸ ︷︷ ︸
ΥI(λ)
(15)
Step 2 (Task environment generalization) Now, we apply Lemma 1 on the meta-level. For that, we treat each task as random variable and instantiate the components as Xk = τi, l = n and µk = T . Furthermore, we set ρ = Q, π = P , f = P and gk(f,Xk) = 1nL(Qi,Di). This allows us to bound the transfer error as
L(Q, T ) ≤ L̃(Q,D1, ...,Dn) + 1
β DKL(ρ||π) + ΥII(β) (16)
wherein ΥII(β) = 1β lnEP∼P [ e β n ∑n i=1 E(D,S)∼T [L(Q(P,S),D)]−L(Q(P,Si),Di) ] .
Combining (15) with (16), we obtain L(Q, T ) ≤ L̂(Q, S1, ..., Sn) + ( 1
β +
1
λ
) DKL(Q||P)
+ 1
λ n∑ i=1 EP∼Q [DKL(Qi||P )] + ΥI(λ) + ΥII(β) (17)
Step 3 (Bounding the moment generating functions)
e(Υ I(λ)+ΥII(β)) =EP∼P [ e β n ∑n i=1 E(D,S)∼T [L(Q(P,S),D)]−L(Q(P,Si),Di) ]1/β ·
EP∼PEh∼P [ e λ n ∑n i=1(L(h,Di)−L̂(h,Si) ]1/λ =EP∼P
[ n∏ i=1 e( β nE(D,S)∼T [L(Q(P,S),D))]−L(Q(P,Si),Di) ]1/β ·
EP∼PEh∼P [ n∏ i mi∏ i e λ nmi (L(h,Di)−l(hi,zij))
]1/λ (18)
Case I: bounded loss
If the loss function l(hi, zij) is bounded in [ak, bk], we can apply Hoeffding’s lemma to each factor in (18), obtaining:
eΥ I(λ)+ΥII(β) ≤ EP∼P [ e β2 8n (bk−ak)2 ]1/β · EP∼PEh∼P [ e λ2 8nm̃ (bk−ak)2 ]1/λ
(19)
= e( β 8n+ λ 8nm̃ )(bk−ak)2 (20)
Next, we factor out √ n from λ and β, obtaining
eΥ I(λ)+ΥII(β) = ( eΥ I(λ √ n)+ΥII(β √ n) ) 1√ n (21)
Using
ET ED1 ...EDn [ eΥ I(λ √ n)+ΥII(β √ n) ] ≤ e ( β 8 √ n + λ 8 √ nm̃ ) (bk−ak)2 (22)
we can apply Markov’s inequality w.r.t. the expectations over the task distribution T and data distributions Di to obtain that
ΥI(λ) + ΥII(β) ≤ β 8n
(bk − ak)2︸ ︷︷ ︸ ΨI(β)
+ λ
8nm̃ (bk − ak)2︸ ︷︷ ︸ ΨII(λ)
− 1√ n ln δ (23)
with probability at least 1− δ. Case II: sub-gamma loss
First, we assume that, ∀i = 1, ..., n the random variables V Ii := L(h,Di)− l(hi, zi,j) are sub-gamma with variance factor s2I and scale parameter cI under the two-level prior (P, P ) and the respective data distribution Di. That is, their moment generating function can be bounded by that of a Gamma distribution Γ(s2I , cI):
Ez∼DiEP∼PEh∼P [ eγ(L(h,Di)−l(h,z)) ] ≤ exp ( γ2s2I
2(1− cIγ)
) ∀γ ∈ (0, 1/cI) (24)
Second, we assume that, the random variable V II := E(D,S)∼T [L(Q(P, S),D)]− L(Q(P, Si),Di) is sub-gamma with variance factor s2II and scale parameter cII under the hyper-prior P and the task distribution T . That is, its moment generating function can be bounded by that of a Gamma distribution Γ(s2II, cII):
E(D,S)∼T EP∼P [ eγ E(D,S)∼T [L(Q(P,S),D)]−L(Q(P,S),D) ] ≤ exp ( γ2s2II
2(1− cIIγ)
) ∀γ ∈ (0, 1/cII)
(25)
These two assumptions allow us to bound the expectation of (18) as follows:
E [ eΥ I(λ)+ΥII(β) ] ≤ exp ( λs2I
2nm̃(1− cIλ/(nm̃)
) · exp ( βs2II
2n(1− cIIβ/n)
) (26)
Next, we factor out √ n from λ and β, obtaining
eΥ I(λ)+ΥII(β) = ( eΥ I(λ √ n)+ΥII(β √ n) ) 1√ n (27)
Finally, by using Markov’s inequality we obtain that
ΥI(λ) + ΥII(β) ≤ λs 2 I
2nm̃(1− cIλ/(nm̃)︸ ︷︷ ︸ ΨI(β)
+ βs2II
2n(1− cIIβ/n)︸ ︷︷ ︸ ΨII(λ)
− 1√ n ln δ (28)
with probability at least 1− δ. To conclude the proof, we choose λ = n √ m̃ and β = √ n.

heading: A.2 PROOF OF COROLLARY 1
text: Lemma 2. (Catoni, 2007) Let A be a set, g : A → R a function, and ρ ∈ M(A) and π ∈ M(A) probability densities over A. Then for any β > 0 and ∀a ∈ A,
ρ∗(a) := π(a)e−βg(a)
Z =
π(a)e−βg(a) Ea∼π [ e−βg(a) ] (29) is the minimizing probability density
arg min ρ∈M(A)
βEa∼ρ [g(a)] +DKL(ρ||π) . (30)
Proof of Corollary 1 When we choose the posterior Q as the optimal Gibbs posterior Q∗i := Q∗(Si, P ), it follows that
L̂(Q, S1, ..., Sn) + 1
n n∑ i=1 1√ m̃ EP∼Q [DKL(Q∗i ||P )] (31)
= 1
n n∑ i=1 ( EP∼QEh∼Q∗i [ L̂(h, Si) ] + 1√ m̃ (EP∼Q [DKL(Q∗i ||P )]) )
(32)
= 1
n n∑ i=1 1√ m̃ ( EP∼QEh∼Q∗i [√ m̃L̂(h, Si) + ln Q∗i (h) P (h) ]) (33)
= 1
n n∑ i=1 1√ m̃
EP∼QEh∼Q∗i  1√
m̃ m∑ j=1 l(h, zi) + ln P (h)e − 1√ m̃ ∑m j=1 l(h,zi) P (h)Z(Si, P )  (34) = 1
n n∑ i=1 1√ m̃ (−EP∼Q [lnZ(Si, P )]) . (35)
This allows us to write the inequality in (3) as
L(Q, T ) ≤ − 1 n n∑ i=1 1 mi EP∼Q [lnZ(Si, P )] + ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃) .
(36)
According to Lemma 2, the Gibbs posterior Q∗(Si, P ) is the minimizer of (33), in particular ∀P ∈ M(H),∀i = 1, ..., n :
Q∗(Si, P ) = P (h)e−
√ m̃L̂(h,Si)
Z(Si, P ) = arg min Q∈M(H)
Eh∼Q [ L̂(h, Si) ] +
1√ m̃ DKL(Q||P ) . (37)
Hence, we can write
L(Q, T ) ≤− 1 n n∑ i=1 1√ m̃ EP∼Q [lnZ(Si, P )] + ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃)
(38)
= 1
n n∑ i=1 EP∼Q [ min Q∈M(H) L̂(Q,Si) + 1√ m̃ DKL(Q||P ) ] (39)
+ ( 1√ n + 1 n √ m̃ ) + C(δ, n, m̃) (40)
≤ 1 n n∑ i=1 EP∼Q [ L̂(Q,Si) + 1√ m̃ DKL(Q||P ) ] (41)
+ ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃) (42)
= L̂(Q, S1, ..., Sn) + (
1√ n + 1 n √ m̃
) DKL(Q||P) (43)
+ 1
n n∑ i=1 1√ m̃ EP∼Q [DKL(Qi||P )] + C(δ, n, m̃) , (44)
which proves that the bound for Gibbs-optimal base learners in (36) and (4) is tighter than the bound in Theorem 2 which holds uniformly for all Q ∈M(H).
Metalearner
zj <latexit sha1_base64="ab/qOKHCfuOUApKDfjguVaHpgzw=">AAAFqXicjZRLb9NAEMenAUMJj7Zw5BIRISFRIttJH8eKZw8gtUDaiFJVtrNxTfyS7ZS2UT4CV/hsfBcO/Ge8VGlK0thaz+zs7G8ea9tNwyAvTPP3QuXGTePW7cU71bv37j9YWl55uJcng8xTbS8Jk6zjOrkKg1i1i6AIVSfNlBO5odp3+694ff9EZXmQxJ+Ls1QdRo4fB73AcwqYPp0ffTtarpsNU67aVcXSSp30tZOsVLbpK3UpIY8GFJGimAroITmU4z4gi0xKYTukIWwZtEDWFY2oir0DeCl4OLD28fQxO9DWGHNm5rLbQ5QQI8POGj3FeCtEF94cVUHPIf9gnIvNnxphKGTO8AzSFWLJ/ICVgo7hc93eSHuO5t7JdRXUo02pJ0CGqVi4Uu+C8xorGWx9WanRG/H0wXBlfoIexJBtZMB9/keoSc1dSEekEkqsiQ54GST3n/OZXp0LfiLROWI+45yGeGbQU1TNp3oqvtf1jXsUaWoM+3fdtUTyjeAXiuT+jjB7PzabRT+99IaUntyFnvRPja1yhAC7jhH5BWoMQVQS6yO9o5eiWbSOe5VsvL+ltGlDZz2N2pX3sT+V2QR1FYOZLDchZxPH8/SlP3yik5lugFRm2BRuE/N5M51GXdO5rkuuLdzzZ8rE7gTPviCugVXK5txZ/p/IJ8J1l+dT1lzFP8ya/GNdVfbshtVs2Lut+lZL/80W6TE9oWfSzy3aph18YR5q+kE/6Zfx3Ng1OsaX0rWyoPc8okuX4f0FSRYRkQ==</latexit>
Base learner
T <latexit sha1_base64="SkUiQETBG6bz1dMyyF2dYE49Fj0=">AAAFsXicjVRLb9NAEJ4GDCW8WjhyiYiQOJQosdPHsSqvHkAqqGmL0gpsZ+NYfsp2+iDKz+AKv4v/woFvxkuVpiTNWuuZnZ355ptZe5009POi2fy9VLl127hzd/le9f6Dh48er6w+OciTYeaqjpuESXbk2LkK/Vh1Cr8I1VGaKTtyQnXoBK95//BUZbmfxPvFRapOItuL/b7v2gVM3ePILgauHY72x19X6s1GU0btutLSSp302EtWK7t0TD1KyKUhRaQopgJ6SDbleLrUoialsJ3QCLYMmi/7isZURewQXgoeNqwB3h5WXW2NsWbMXKJdZAkxM0TW6AXmO0F04M1ZFfQc8g/md7F5MzOMBJkZXkA6glhifsROQQP43BQbac/xwpFcV0F92pJ6fDBMxcKVupc4b7CTwRbITo3eiqcHDEfWp+hBDNkBA+7zP4Sa1NyDtEUqQYk1og28DJL7z3xmV+cAP5HsnDGfc04jvDPoKarmUz0X35v6xj2KNGoM+5nuWiJ8I/iFIrm/Y6w+TKzmoZ9f+UJKT+5CX/qnJnY5g4+oATK/Qo0hEJXk+kzvaUe0Fm3gWSMT328pTdrUrGeh9uR7DGZiWkBdw2RMlluQ8xEneXrSHz7RaaabQCoZWoJrYb0o01mo65rrhnBt41mcKSP2pvDMS8R1YJXSWpjl/xH5RLju8nzKmqu4w1rTN9Z15cBstKyG+ald327r22yZntFzein93KZd2sMfxmx+0E/6ZVjGF+Ob4ZSulSUd85SuDCP4C7jQFSA=</latexit>
P <latexit sha1_base64="Oc0IVe+W4R0Ipzd3Jgq7DjWG0M0=">AAAFsXicjVRLb9NAEJ4GDCU82sKRS0SExKFEjp0+jhXPHkAKiLRFaQW2s3Gs+CXbKS1RfgZX+F38Fw58M16qNCVp1lrP7OzMN9/M2uumYZAXpvl7pXLjpnHr9uqd6t179x+srW88PMiTUeapjpeESXbkOrkKg1h1iqAI1VGaKSdyQ3XoDl/y/uGpyvIgiT8V56k6iRw/DvqB5xQwdY8jpxh4TjhuT76s182GKaN2VWlqpU56tJONyj4dU48S8mhEESmKqYAekkM5ni41yaQUthMaw5ZBC2Rf0YSqiB3BS8HDgXWIt49VV1tjrBkzl2gPWULMDJE1eor5RhBdeHNWBT2H/IP5XWz+3AxjQWaG55CuIJaY77FT0AA+18VG2nOydCTXVVCfdqWeAAxTsXCl3gXOK+xksA1lp0avxdMHhivrU/QghuyAAff5H0JNau5BOiKVoMQa0QFeBsn9Zz7zq3OBn0h2zpgvOKcx3hn0FFXzqZ6J73V94x5FGjWG/ZvuWiJ8I/iFIrm/E6zeTa0WoZ9d+kJKT+5CX/qnpnY5Q4CoATI/R40hEJXk+khv6YVoTdrGs0kWvt9SWrSjWc9D7cn3OJyLaQN1E5MxWe5CLkac5ulLf/hEZ5nuAKlkaAuujfWyTOehbmmu28K1hWd5pozYm8GzLhC3gFVKe2mW/0fkE+G6y/Mpa67iDmvO3lhXlQOr0bQb1odWfa+lb7NVekxP6Jn0c4/2qY0/jNn8oJ/0y7CNz8ZXwy1dKys65hFdGsbwL6PAFRw=</latexit>
Di <latexit sha1_base64="0vdE+4ItTL4j6o7I/9Oc7Zy+tA4=">AAAFs3icjZRLb9NAEMenKYYSXi0cuURESBxKlNhJ22NVCvQAUkGkrZRGle1sEit+YTulJcrn4Aofi+/Cgf+MlypNcRpb65mdnf3NY207se+lWb3+e6W0ese4e2/tfvnBw0ePn6xvPD1Ko3HiqrYb+VFy4tip8r1QtTMv89VJnCg7cHx17Ize8PrxuUpSLwq/ZJex6gb2IPT6nmtnMHVPAzsburY/2Z+eeWfr1XqtLlflptLQSpX0dRhtlA7olHoUkUtjCkhRSBl0n2xKcXeoQXWKYevSBLYEmifriqZUxt4xvBQ8bFhHeA4w62hriDkzU9ntIoqPkWBnhV5ivBOiA2+OqqCnkH8wvottUBhhImTO8BLSEWLO/IiVjIbwuW1voD2nS+/kujLq047U4yHDWCxcqXvF2cdKAttIVir0VjwHYDgyP0cPQsg2MuA+/yNUpOYepC1SCSXURBu8BJL7z/kUV+eAH0l0jpguOKcJngn0GFXzqV6I72194x4FmhrC/k13LZJ8A/j5Irm/U8w+zMwW0S+uvSG5J3ehL/1TM6scwcOuISK/Ro0+iEpifab3tCdag7Zwb5KJ9zeXJm3rrIuoPXkfR4VMC9RNDGay3IFcTJzNcyD94ROdz3QbpDxDS7gW5stmWkRt6Vy3JNcm7uUzZWJvjmdeEVtg5dJaOsv/E/lEuO78fPKay/iHNeb/WDeVI7PWsGrmp2Z1t6n/Zmv0nF7QK+nnLh3QIb4wl77SD/pJv4yW0TEco5e7llb0nmd07TKCv9vXFew=</latexit> Si <latexit sha1_base64="oWc1+CsVEVqiz1tUDjtmKWHVQYI=">AAAFqXicjZRLb9NAEMenAUMJr5YeuURESEiUKI7Tx7Eqrx5AailpI0pV2c7GteKXbKe0RPkIXOGz8V048J/xUqUpTmNrPbOzs795rG0nCfwsbzZ/L1Ru3Tbu3F28V73/4OGjx0vLTw6yeJi6quPGQZx2HTtTgR+pTu7ngeomqbJDJ1CHzuA1rx+eqTTz4+hzfpGo49D2Ir/vu3YO0/7+iX+yVG82mnLVriumVuqkr914ubJDX6lHMbk0pJAURZRDD8imDPcRmdSkBLZjGsGWQvNlXdGYqtg7hJeChw3rAE8PsyNtjTBnZia7XUQJMFLsrNFzjHdCdODNURX0DPIPxnexeaURRkLmDC8gHSEWzI9YyekUPjftDbXneO6dXFdOfdqUenxkmIiFK3UvOW+wksI2kJUavRVPDwxH5mfoQQTZQQbc53+EmtTcg7RFKqFEmmiDl0Jy/zmf8uoc8GOJzhGzGec0wjOFnqBqPtVz8b2pb9yjUFMj2L/prsWSbwi/QCT3d4zZh4nZLPr5lTek8OQu9KV/amKVI/jYdYrIr1BjAKKSWJ/oPW2LZtI67lVq4f0tZIs2dNZl1J68j4NSpgXqKgYzWW5CziZO5ulJf/hEpzPdAKnI0BKuhfm8mZZR13Su65JrG/f8mTKxN8VrXRLXwCqkNXeW/yfyiXDdxfkUNVfxDzOn/1jXlYNWw7Qarb12faut/2aL9JSe0Qvp5xbt0C6+MBc1/aCf9Mt4aewZXeNL4VpZ0HtW6MpluH8BdkERaQ==</latexit>
Q <latexit sha1_base64="pC61m5Ks5vLJ6HWTyUscWahNHlw=">AAAFsXicjVRLb9NAEJ4GDCW8WjhyiYiQOJQosdPHsSqvHkBqEWmL0gpsZ+NYfsp2+iDKz+AKv4v/woFvxkuVpiTNWuuZnZ355ptZe5009POi2fy9VLl127hzd/le9f6Dh48er6w+OciTYeaqjpuESXbk2LkK/Vh1Cr8I1VGaKTtyQnXoBK95//BUZbmfxJ+Li1SdRLYX+33ftQuYuseRXQxcOxztj7+u1JuNpozadaWllTrpsZesVnbpmHqUkEtDikhRTAX0kGzK8XSpRU1KYTuhEWwZNF/2FY2pitghvBQ8bFgDvD2sutoaY82YuUS7yBJiZois0QvMd4LowJuzKug55B/M72LzZmYYCTIzvIB0BLHE/IidggbwuSk20p7jhSO5roL6tCX1+GCYioUrdS9x3mAngy2QnRq9FU8PGI6sT9GDGLIDBtznfwg1qbkHaYtUghJrRBt4GST3n/nMrs4BfiLZOWM+55xGeGfQU1TNp3ouvjf1jXsUadQY9jPdtUT4RvALRXJ/x1h9mFjNQz+/8oWUntyFvvRPTexyBh9RA2R+hRpDICrJ9Yne045oLdrAs0Ymvt9SmrSpWc9C7cn3GMzEtIC6hsmYLLcg5yNO8vSkP3yi00w3gVQytATXwnpRprNQ1zXXDeHaxrM4U0bsTeGZl4jrwCqltTDL/yPyiXDd5fmUNVdxh7Wmb6zryoHZaFkNc79d327r22yZntFzein93KZd2sMfxmx+0E/6ZVjGF+Ob4ZSulSUd85SuDCP4C6kEFR0=</latexit>
Q <latexit sha1_base64="/glzvYfQGIYnAOGohHqeOS81re8=">AAAFp3icjZRLb9NAEMenAUMJj7Zw5BIRIYFUIsdOH8eKZw8gGkTSSKVCtrNJrfgl2yktUT4BV/hwfBcO/Ge8VGmK09haz+zs7G8ea9tNAj/LTfP3SuXGTePW7dU71bv37j9YW9942M3iceqpjhcHcdpznUwFfqQ6uZ8HqpekygndQB26o1e8fniq0syPo8/5eaKOQ2cY+QPfc3KY2u2v63WzYcpVu6o0tVInfR3EG5V9+kJ9ismjMYWkKKIcekAOZbiPqEkmJbAd0wS2FJov64qmVMXeMbwUPBxYR3gOMTvS1ghzZmay20OUACPFzho9xXgrRBfeHFVBzyD/YHwX27A0wkTInOE5pCvEgvkBKzmdwOe6vaH2nC69k+vKaUC7Uo+PDBOxcKXeBec1VlLYRrJSozfiOQTDlfkpehBBdpAB9/kfoSY19yEdkUookSY64KWQ3H/Op7w6F/xYonPEbME5TfBMoSeomk/1THyv6xv3KNTUCPZvumux5BvCLxDJ/Z1i9n5mtoh+dukNKTy5CwPpn5pZ5Qg+dp0g8gvUGICoJNYnekcvRWvSNu5NsvD+FtKiHZ11GbUv7+OolGmDuonBTJa7kIuJs3kOpT98ovOZ7oBUZGgL18Z82UzLqFs6123JtYV7+UyZ2J/jWRfELbAKaS+d5f+JfCJcd3E+Rc1V/MOa83+sq0rXajTthtVu1fda+m+2So/pCT2Tfu7RPh3gC/MQ9wf9pF/Gc+Oj0TV6hWtlRe95RJcuw/kL/MsQiw==</latexit> P <latexit sha1_base64="ssfIKALpa2PH8lgOQNW1ip9pNr8=">AAAFp3icjZRLb9NAEMenAUMJj7Zw5BIRIYFUIsdOH8eKZw8gUkTSSKVCtrNJrfgl2yktUT4BV/hwfBcO/Ge8VGmK09haz+zs7G8ea9tNAj/LTfP3SuXGTePW7dU71bv37j9YW9942M3iceqpjhcHcdpznUwFfqQ6uZ8HqpekygndQB26o1e8fniq0syPo8/5eaKOQ2cY+QPfc3KYDtpf1+tmw5SrdlVpaqVO+mrHG5V9+kJ9ismjMYWkKKIcekAOZbiPqEkmJbAd0wS2FJov64qmVMXeMbwUPBxYR3gOMTvS1ghzZmay20OUACPFzho9xXgrRBfeHFVBzyD/YHwX27A0wkTInOE5pCvEgvkBKzmdwOe6vaH2nC69k+vKaUC7Uo+PDBOxcKXeBec1VlLYRrJSozfiOQTDlfkpehBBdpAB9/kfoSY19yEdkUookSY64KWQ3H/Op7w6F/xYonPEbME5TfBMoSeomk/1THyv6xv3KNTUCPZvumux5BvCLxDJ/Z1i9n5mtoh+dukNKTy5CwPpn5pZ5Qg+dp0g8gvUGICoJNYnekcvRWvSNu5NsvD+FtKiHZ11GbUv7+OolGmDuonBTJa7kIuJs3kOpT98ovOZ7oBUZGgL18Z82UzLqFs6123JtYV7+UyZ2J/jWRfELbAKaS+d5f+JfCJcd3E+Rc1V/MOa83+sq0rXajTthnXQqu+19N9slR7TE3om/dyjfWrjC/MQ9wf9pF/Gc+Oj0TV6hWtlRe95RJcuw/kL94gQig==</latexit>
j = 1, . . . , mi <latexit sha1_base64="PWeL3EKnKlUzhya6VClT367J2wQ=">AAAFuXicjZRLb9NAEMenAUMJrwSOXCwiJA4hip2+JIRU8ewBpIJIW6mtItvZJG78ku2UligfhSt8Jr4LB/4zXqo0JWlsbWZ2dvY3j93YTQI/y5vN3yulGzeNW7dX75Tv3rv/4GGl+mgvi0epp9peHMTpgetkKvAj1c79PFAHSaqc0A3Uvjt8w+v7pyrN/Dj6mp8n6jh0+pHf8z0nh6lTqZ6Yr0yrbh514zyrm2HH71RqzUZTHvOqYmmlRvrZjaulHTqiLsXk0YhCUhRRDj0ghzK8h2RRkxLYjmkMWwrNl3VFEypj7wheCh4OrEP89jE71NYIc2ZmsttDlAAjxU6TnmG8F6ILb46qoGeQfzC+i60/N8JYyJzhOaQrxIL5CSs5DeBz3d5Qe06W3sl15dSjLanHR4aJWLhS74LzFispbENZMemdePbBcGV+ih5EkG1kwH3+RzCl5i6kI1IJJdJEB7wUkvvP+cyvzgU/lugcMVtwTmP8ptATVM2neia+1/WNexRqagT7N921WPIN4ReI5P5OMPs4NVtEP7t0QwpP7kJP+qemVjmCj10DRH6BGgMQlcT6Qh/otWgWbeCtk437W0ibNnXW86hduY/DucwWqHUMZrLcglxMnM6zL/3hE53NdBOkIsOWcFuYL5vpPOq6znVDcl3Du3ymTOzO8OwL4jpYhWwtneX/iXwiXHdxPkXNZXzDrNkv1lVlz25YrYb9ea22Xddfs1V6Qk/pufRzm3ZoF/8wD7fzB/2kX8ZLwzEGxknhWlrRex7TpcfI/gIhUxX+</latexit>
i = 1, . . . , n <latexit sha1_base64="EtJp9oXmut0eetutyNioJBfjBHA=">AAAFt3icjZRLb9NAEMenoYYSHm3hyMUiQuIQIjvpiwNSxbOHIhVE2kptVdnOJrHiF7ZTWqJ8Eq7wofguHPjPeKnSFKextZnZ2dnfPNZZNwn8LLes3wuVW4vG7TtLd6v37j94uLyy+mg/i4epp9peHMTpoetkKvAj1c79PFCHSaqc0A3UgTt4w+sHZyrN/Dj6kl8k6iR0epHf9T0nh+l0Zdk3X5l23TzuxHlWN2GpWQ1LHvO6YmulRvrZi1crO3RMHYrJoyGFpCiiHHpADmV4j8gmixLYTmgEWwrNl3VFY6pi7xBeCh4OrAP89jA70tYIc2ZmsttDlAAjxU6TnmG8F6ILb46qoGeQfzC+i61XGmEkZM7wAtIVYsH8iJWc+vC5aW+oPcdz7+S6curSltTjI8NELFypd8l5i5UUtoGsmPROPHtguDI/Qw8iyDYy4D7/I5hScwfSEamEEmmiA14Kyf3nfMqrc8GPJTpHzGac0wi/KfQEVfOpnovvTX3jHoWaGsH+TXctlnxD+AUiub9jzHYnZrPo51e+kMKTu9CV/qmJVY7gY1cfkV+gxgBEJbE+0wd6LZpNG3jr1MT3W8gmbeqsy6gd+R4HpcwWqHUMZrLcgpxNnMyzJ/3hE53OdBOkIsOWcFuYz5tpGXVd57ohua7hnT9TJnameM1L4jpYhWzNneX/iXwiXHdxPkXNVdxh9vSNdV3ZbzbsVqP5aa22Xde32RI9oaf0XPq5TTu0h38Y32o/6Cf9Ml4ap0bX6BeulQW95zFdeYyvfwGqHxUi</latexit>
ŷk <latexit sha1_base64="vce++Hk9Rmuq+HjAwXfMtjQBAfE=">AAAFr3icjZRLb9NAEMenKYYSHm3hyCUiQuJQojhJH8eqvHoAqSDSFLWl2M7GsfyU7ZSGKJ+CK3wwvgsH/jNeqjQlaWxtZnZ29jeP3dhOAi/L6/XfS6XlW8btOyt3y/fuP3i4urb+6DCLB6mj2k4cxOmRbWUq8CLVzr08UEdJqqzQDlTH9l/yeudcpZkXR5/yYaJOQ8uNvJ7nWDlMn0/6Vj4ajs/8s7VqvVaXp3JdMbVSJf0cxOulfTqhLsXk0IBCUhRRDj0gizK8x2RSnRLYTmkEWwrNk3VFYypj7wBeCh4WrD5+XcyOtTXCnJmZ7HYQJcBIsbNCzzDeCNGGN0dV0DPIPxjfxebOjDASMmc4hLSFWDDfYyWnPnxu2htqz/HCO7munHq0I/V4yDARC1fqXHJeYSWFzZeVCr0WTxcMW+bn6EEE2UYG3Od/hIrU3IW0RCqhRJpogZdCcv85n9nV2eDHEp0jZnPOaYTfFHqCqvlUL8T3pr5xj0JNjWD/prsWS74h/AKR3N8xZu8mZvPoF1duSOHJXehJ/9TEKkfwsKuPyC9QYwCiklgf6S3tiWbSFt4NauD+FrJB2zrrWdSu3Ed/JrMJ6gYGM1nuQM4nTubpSn/4RKcz3QapyLAp3Cbmi2Y6i7qpc92SXFt4F8+Uid0pXuOSuAlWIZsLZ/l/Ip8I112cT1FzGd8wc/qLdV05bNTMZq3xoVXdbemv2Qo9oaf0XPq5S/t0gH+Yg3v3g37SL8M0OsYX42vhWlrSex7Tlcfw/gLMghRe</latexit>
x⇤k <latexit sha1_base64="EY5X55MHj+HDXHRXkeZ3aNmmaMM=">AAAFq3icjVTbbtNAEJ0GDCVc2sIjLxEREkJtlMTp5bHi2geQyiVtRVsq29m4lq+ynZIS5Rt4hU/jX3jgzHip0hSnWWs9s7MzZ87M2msngZflzebvhcqNm8at24t3qnfv3X+wtLzycC+LB6mjuk4cxOmBbWUq8CLVzb08UAdJqqzQDtS+7b/k/f0zlWZeHH3OzxN1HFpu5PU9x8ph6g6/Pj/xT5brzUZTRu2q0tJKnfTYjVcqO3REPYrJoQGFpCiiHHpAFmV4DqlFTUpgO6YRbCk0T/YVjamK2AG8FDwsWH28XawOtTXCmjEziXaQJcBMEVmjp5hvBNGGN2dV0DPIP5jfxeaWZhgJMjM8h7QFscB8j52cTuFzXWyoPcdzR3JdOfVpS+rxwDARC1fqXOC8wk4Kmy87NXotni4wbFmfoQcRZBcMuM//EGpScw/SEqkEJdKIFvBSSO4/8ymvzgZ+LNk5YzbjnEZ4p9ATVM2nOhTf6/rGPQo1agT7N921WPiG8AtEcn/HWL2bWM1CH176QgpP7kJf+qcmdjmDh6hTZF5DjQEQleT6SG/phWgt2sCzSm18v4Vs06ZmXYbak+/RL8U0gbqKyZgstyBnI07ydKU/fKLTTDeBVDA0BdfEel6mZajrmuuGcO3gmZ8pI/am8NoXiOvAKqQ5N8v/I/KJcN3F+RQ1V3GHtaZvrKvKXrvRMhvtD536dkffZov0mJ7QM+nnNu3QLv4wB6x+0E/6ZawZn4wvxlHhWlnQMY/o0jDUX2NCEiw=</latexit>
h <latexit sha1_base64="RFUpTDLWeEeCizO9lNa3D53q/a4=">AAAFp3icjZRLb9NAEMenAUMJj7Zw5BIRIYFUothJH8eKZw8gWkTSSKVCtrNxrPgl2yktUT4BV/hwfBcO/Ge8VGmK09haz+zs7G8ea9tJAj/Lm83fK5UbN41bt1fvVO/eu/9gbX3jYTeLx6mrOm4cxGnPsTMV+JHq5H4eqF6SKjt0AnXkjF7x+tGpSjM/jj7n54k6CW0v8ge+a+cwHQ6/rtebjaZctauKqZU66esg3qjs0xfqU0wujSkkRRHl0AOyKcN9TCY1KYHthCawpdB8WVc0pSr2juGl4GHDOsLTw+xYWyPMmZnJbhdRAowUO2v0FOOtEB14c1QFPYP8g/FdbF5phImQOcNzSEeIBfMDVnIawue6vaH2nC69k+vKaUC7Uo+PDBOxcKXuBec1VlLYRrJSozfi6YHhyPwUPYggO8iA+/yPUJOa+5C2SCWUSBNt8FJI7j/nU16dA34s0TlituCcJnim0BNUzad6Jr7X9Y17FGpqBPs33bVY8g3hF4jk/k4xez8zW0Q/u/SGFJ7chYH0T82scgQfu4aI/AI1BiAqifWJ3tFL0Uzaxr1JFt7fQlq0o7Muo/blfRyVMlugbmIwk+Uu5GLibJ6e9IdPdD7THZCKDFvCbWG+bKZl1C2d67bk2sa9fKZM7M/xrAviFliFbC2d5f+JfCJcd3E+Rc1V/MPM+T/WVaVrNcxWwzps1/fa+m+2So/pCT2Tfu7RPh3gC3MR9wf9pF/Gc+Oj0TV6hWtlRe95RJcuw/4Ldd8Qog==</latexit>
S⇤ <latexit sha1_base64="F0htXLtDjhuAAO1RDdhj8IPXAfI=">AAAFqXicjZRLb9NAEMenAUMJrxaOXCIiJAQlcuL0cax49gBSS0kbUVpkOxvXil+yndIS5SNwhc/Gd+HAf8ZLlaY4ja31zM7O/uaxtp0k8LPcNH8vVK5dN27cXLxVvX3n7r37S8sP9rJ4mLqq48ZBnHYdO1OBH6lO7ueB6iapskMnUPvO4BWv75+oNPPj6FN+lqjD0PYiv++7dg7T7u7Rs69LdbNhylW7rDS1Uid9bcfLlS36Qj2KyaUhhaQoohx6QDZluA+oSSYlsB3SCLYUmi/risZUxd4hvBQ8bFgHeHqYHWhrhDkzM9ntIkqAkWJnjZ5gvBWiA2+OqqBnkH8wvovNK40wEjJneAbpCLFgfsBKTsfwuWpvqD3Hc+/kunLq04bU4yPDRCxcqXvOeY2VFLaBrNTojXh6YDgyP0EPIsgOMuA+/yPUpOYepC1SCSXSRBu8FJL7z/mUV+eAH0t0jpjNOKcRnin0BFXzqZ6K71V94x6FmhrB/k13LZZ8Q/gFIrm/Y8zeT8xm0U8vvCGFJ3ehL/1TE6scwceuY0R+gRoDEJXE+kjv6KVoTVrDvUItvL+FbNG6zrqM2pP3cVDKtEBdwWAmyw3I2cTJPD3pD5/odKbrIBUZWsK1MJ830zLqqs51TXJt454/Uyb2pnitc+IqWIW05s7y/0Q+Ea67OJ+i5ir+Yc3pP9ZlZa/VaFqN1k67vtnWf7NFekSP6an0c5O2aBtfmIuaftBP+mU8N3aMrvG5cK0s6D0P6cJluH8BJXERKQ==</latexit> D <latexit sha1_base64="DLsDpLQo2cPUDCQ1OgvqmzMNJac=">AAAFsXicjVRLb9NAEJ4GDCW8WjhyiYiQOJQosdPHsSoFegCpINIWpRXYziax/JTt9EGUn8EVfhf/hQPfjJcqTXGatdYzOzvzzTez9jpJ4GV5s/l7qXLrtnHn7vK96v0HDx89Xll9cpDFo9RVHTcO4vTIsTMVeJHq5F4eqKMkVXboBOrQ8V/z/uGpSjMvjj7nF4k6Ce1B5PU9185h6h6Hdj507WC8O/m6Um82mjJq15WWVuqkx368WtmjY+pRTC6NKCRFEeXQA7Ipw9OlFjUpge2ExrCl0DzZVzShKmJH8FLwsGH18R5g1dXWCGvGzCTaRZYAM0VkjV5gvhVEB96cVUHPIP9gfhfboDTDWJCZ4QWkI4gF5gfs5DSEz02xofacLBzJdeXUpy2pxwPDRCxcqXuJs4udFDZfdmr0RjwHwHBkfYoeRJAdMOA+/0OoSc09SFukEpRII9rASyG5/8ynvDoH+LFk54zZnHMa451CT1A1n+q5+N7UN+5RqFEj2M9012LhG8IvEMn9nWD1fmo1D/38yhdSeHIX+tI/NbXLGTxEDZH5FWoMgKgk1yd6RzuitWgDzxqZ+H4LadKmZl2G2pPv0S/FtIC6hsmYLLcg5yNO8xxIf/hEZ5luAqlgaAmuhfWiTMtQ1zXXDeHaxrM4U0bszeCZl4jrwCqktTDL/yPyiXDdxfkUNVdxh7Vmb6zryoHZaFkN82O7vt3Wt9kyPaPn9FL6uU17tI8/jNn8oJ/0y7CML8Y3wylcK0s65ildGYb/F2SQFRA=</latexit>
k = 1, . . . , m⇤ <latexit sha1_base64="ZkPtQP1Ab9ue5cyu8Lv65SN1qeU=">AAAFuXicjZTbbtNAEIanAUMJhyZwyY1FhIRQiGKnJwkhVRx7AVJBpK3Ulsp2NomJT7Kd0hLlUbiFZ+JduOCf8VKlKUljazOzs7PfHHZjNwn8LG82fy+Vrl03btxcvlW+fefuvZVK9f5uFg9TT7W9OIjTfdfJVOBHqp37eaD2k1Q5oRuoPXfwitf3TlSa+XH0OT9L1FHo9CK/63tODtNxpTowX5hW3TzsxHlWN8MvT48rtWajKY95WbG0UiP97MTV0jYdUodi8mhIISmKKIcekEMZ3gOyqEkJbEc0gi2F5su6ojGVsXcILwUPB9YBfnuYHWhrhDkzM9ntIUqAkWKnSY8x3grRhTdHVdAzyD8Y38XWmxlhJGTO8AzSFWLB/ICVnPrwuWpvqD3HC+/kunLq0qbU4yPDRCxcqXfOeY2VFLaBrJj0Rjx7YLgyP0EPIsg2MuA+/yOYUnMH0hGphBJpogNeCsn953xmV+eCH0t0jpjNOacRflPoCarmUz0V36v6xj0KNTWC/ZvuWiz5hvALRHJ/x5i9n5jNo59euCGFJ3ehK/1TE6scwceuPiI/Q40BiEpifaJ39FI0i9bx1snG/S2kTRs661nUjtzHwUxmC9Q6BjNZbkLOJ07m2ZP+8IlOZ7oBUpFhS7gtzBfNdBZ1Tee6Lrmu4l08UyZ2pnj2OXENrEK2Fs7y/0Q+Ea67OJ+i5jK+Ydb0F+uysms3rFbD/rha26rrr9kyPaRH9ET6uUXbtIN/mIfb+YN+0i/jueEYfeNr4Vpa0nse0IXHyP4C1ccVvw==</latexit>
l = 1, . . . , m̃ <latexit sha1_base64="OxG7gX4xGsXcs9jmMUoaIEZPO3s=">AAAFv3icjZRLb9NAEMenAUMJj6Zw4MDFIkLiEKI46euCVN49gFQQaSu1VeXHxrXil2ynNET5NFzhA/FdOPCf8VKlKUljaz2zs7O/eaxtJw2DvGi1fi9Vbtw0bt1evlO9e+/+g5Xa6sO9PBlkruq6SZhkB46dqzCIVbcIilAdpJmyIydU+07/Da/vn6ksD5L4azFM1XFk+3HQC1y7gOmk9jg0X5pWwzzykiKHKILQU6NofFKrt5otucyriqWVOulrN1mt7NAReZSQSwOKSFFMBfSQbMpxH5JFLUphO6YRbBm0QNYVjamKvQN4KXjYsPbx9DE71NYYc2bmsttFlBAjw06TnmG8F6IDb46qoOeQfzC+i82fGWEkZM5wCOkIsWR+wkpBp/C5bm+kPccL7+S6CurRltQTIMNULFype8F5i5UMtr6smPROPH0wHJmfoQcxZBcZcJ//EUyp2YO0RSqhxJpog5dBcv85n9nVOeAnEp0j5nPOaYRnBj1F1Xyq5+J7Xd+4R5GmxrB/011LJN8IfqFI7u8Ys48Ts3n080tvSOnJXehJ/9TEKkcIsOsUkV+gxhBEJbG+0Ad6LZpFG7gb1Mb7W8o2beqsZ1E9eR/7M5kdUBsYzGS5BTmfOJmnL/3hE53OdBOkMsOOcDuYL5rpLOq6znVDcl3DvXimTPSmeO0L4jpYpewsnOX/iXwiXHd5PmXNVfzDrOk/1lVlr920Os3257X6dkP/zZbpCT2l59LPbdqhXXxh/FX+oJ/0y3hl+EZspKVrZUnveUSXLmP4FwvVGNo=</latexit>
z̃l <latexit sha1_base64="wsDRgP2sdOza37NZO5mTAX/uqIc=">AAAFsXicjVRLb9NAEJ6mGEp4tIUjl4gIiUOJEjt9HKvy6gGkgkhblFbFdjaplfVDtlPaRvkZXOF38V848M14qdKUpFlrPbOzM998M2uvl+ggy+v13wulxTvW3XtL98sPHj56vLyy+mQ/iwepr1p+rOP00HMzpYNItfIg1+owSZUbelodeP3XvH9wptIsiKMv+UWijkO3FwXdwHdzmNpHeaA7ang5OtEnK9V6rS6jclNpGKVKZuzFq6VdOqIOxeTTgEJSFFEOXZNLGZ42NahOCWzHNIQthRbIvqIRlRE7gJeChwtrH+8eVm1jjbBmzEyifWTRmCkiK/QC850gevDmrAp6BvkH81JsvakZhoLMDC8gPUEsMD9iJ6dT+NwWGxrP0dyRXFdOXdqSegIwTMTClfpXOG+wk8LWl50KvRXPHjA8WZ+hBxFkCwy4z/8QKlJzB9IVqQQlMogu8FJI7j/zmV6dB/xYsnPGbMY5DfFOoSeomk/1XHxv6xv3KDSoEezfTddi4RvCT4vk/o6w+jC2moV+fu0LKTy5C13pnxrb5QwBok6R+RVq1EBUkuszvacd0Rq0gWeNbHy/hbRp07CehtqR77E/FdMB6homY7LcgpyNOM6zJ/3hE51kugmkgqEjuA7W8zKdhrpuuG4I1yae+ZkyYmcCz75CXAdWIZ25Wf4fkU+E6y7Op6i5jDusMXlj3VT27VrDqdmfmtXtprnNlugZPaeX0s9t2qU9/GHM5gf9pF+WY321vlle4VpaMDFP6dqw+n8BkXQVSQ==</latexit> S̃
<latexit sha1_base64="pg7YEdyBpUoRaNMIRlnOROA0g7c=">AAAFr3icjZRLb9NAEMenAUMJrxaOXCIiJA4lip30caxaHj2AVB5pitpCbWeTruKXbKe0RPkUXOGD8V048J/xUqUpSWNrM7Ozs7957MZeEugsr9d/L5Ru3LRu3V68U7577/6Dh0vLj/ayeJD6quXHQZzue26mAh2pVq7zQO0nqXJDL1Btr7/N6+1TlWY6jj7l54k6Ct1epLvad3OYPh/mOuio4cfR16VqvVaXp3JVsY1SJfPsxsulHTqkDsXk04BCUhRRDj0glzK8B2RTnRLYjmgIWwpNy7qiEZWxdwAvBQ8X1j5+e5gdGGuEOTMz2e0jSoCRYmeFnmG8FqIHb46qoGeQfzC+i603NcJQyJzhOaQnxIL5Dis5ncDnur2h8RzNvZPryqlLG1KPRoaJWLhS/4LzEispbH1ZqdAr8eyB4cn8FD2IIFvIgPv8j1CRmjuQrkgllMgQXfBSSO4/5zO9Og/8WKJzxGzGOQ3xm0JPUDWf6pn4Xtc37lFoqBHs30zXYsk3hF8gkvs7wuzt2GwW/ezSDSk8uQtd6Z8aW+UIGrtOEPkFagxAVBLrA72hLdFsWsO7Qg7ubyEdWjdZT6N25D72pzIboK5gMJPlBuRs4niePekPn+hkpusgFRk2hNvAfN5Mp1FXTa5rkmsT7/yZMrEzwXMuiKtgFbIxd5b/J/KJcN3F+RQ1l/ENsye/WFeVPadmN2rO+2Z1s2m+Zov0hJ7Sc+nnJu3QLv5hPu7dD/pJvyzbaltfrOPCtbRg9jymS4+l/wI+JRRD</latexit>
meta-training target training target testing
Figure S1: Overview of our meta-learning framework with environment T , meta-task distributions Di, target task distribution D, hyper-prior P , hyper-posterior Q, target prior P , target posterior Q, datasets S, and data points z = (x, y).

heading: A.3 PROOF OF PROPOSITION 1: PAC-OPTIMAL HYPER-POSTERIOR
text: An objective function corresponding to (4) reads as J(Q) = −EQ [
1√ nm̃+ 1 n∑ i=1 lnZ(Si, P )
] +DKL(Q||P) . (45)
To obtain J(Q), we omit all additive terms from (4) that do not depend on Q and multiply by the scaling factor n √ m̃√
m̃n+1 . Since the described transformations are monotone, the minimizing distribution
of J(Q), that is, Q∗ = arg min
Q∈M(M(H)) J(Q) , (46)
is also the minimizer of (4). More importantly, J(Q) is structurally similar to the generic minimization problem in (30). Hence, we can invoke Lemma 2 with A = M(H), g(a) = −∑ni=1 lnZ(Si, P ), β = 1√
nm̃+1 , to show that the optimal hyper-posterior is
Q∗(P ) = P(P ) exp
( 1√
nm̃+1
∑n i=1 lnZ(Si, P ) ) Z II(S1, ..., Sn,P) , (47)
wherein
Z II(S1, ..., Sn,P) = EP∼P [ exp ( 1√
nm̃+ 1 n∑ i=1 lnZ(Si, P )
)] .
Technically, this concludes the proof of Proposition 1. However, we want to remark the following result:
If we choose Q = Q∗, the PAC-Bayes bound in (4) can be expressed in terms of the meta-level partition function Z II, that is,
L(Q, T ) ≤ − (
1√ n + 1 n √ m̃
) lnZ II(S1, ..., Sn,P) + C(δ, n, m̃) . (48)
We omit a detailed derivation of (48) since it is similar to the one for Corollary 1.

heading: B PACOH-NN: A SCALABLE ALGORITHM FOR LEARNING BNN PRIORS
text: In this section, we summarize and further discuss our proposed meta-learning algorithm PACOH-NN. An overview of our proposed framework is illustrated in Figure S1. Overall, it consists of two stages meta-training and meta-testing which we explain in more details in the following.

heading: B.1 META-TRAINING
text: The hyper-posterior distribution Q that minimizes the upper bound on the transfer error is given by
Q∗(P ) = P(P ) exp
( 1√
nm̃+1
∑n i=1 ln Z̃(Si, P ) ) Z II(S1, ..., Sn,P)
(49)
Provided with a set of datasets S1, ..., S2, the meta-learner minimizes the respective meta-objective, in the case of PACOH-SVGD, by performing SVGD on the Q∗. Algorithm 1 outlines the required steps in more detail.
Algorithm 1 PACOH-NN-SVGD: meta-training Input: hyper-prior P , datasets S1, ..., Sn, kernel k(·, ·), step size η, number of particles K {φ1, ..., φK} ∼ P // Initialize prior particles while not converged do
for k = 1, ...,K do {θ1, ..., θL} ∼ Pφk // sample NN-parameters from prior for i = 1, ..., n do
ln Z̃(Si, Pφk)← LSELl=1 ( −√miL̂(θl, Si) ) − lnL // estimate generalized MLL
∇φkQ̃∗(φk)← ∇φk lnP(φk) + 1√nm̃+1 ∑n i=1∇φk ln Z̃(Si, Pφk) // compute score
∀k ∈ [K] : φk ← φk + ηK ∑K k′=1 [ k(φk′ , φk)∇φk′ ln Q̃∗(φk′) +∇φk′k(φk′ , φk) ] // SVGD
Output: set of priors {Pφ1 , ..., PφK}
Alternatively, to estimate the score of ∇φkQ̃∗(φk) we can use mini-batching at both the task and the dataset level. Specifically, for a given meta-batch size of nbs and a batch size of mbs, we get Algorithm 2.
Algorithm 2 PACOH-NN-SVGD: mini-batched meta-training Input: hyper-prior P , datasets S1, ..., Sn Input: kernel function k(·, ·), SVGD step size η, number of particles K {φ1, ..., φK} ∼ P // Initialize prior particles while not converged do {T1, ..., Tnbs} ⊆ [n] // sample nbs tasks uniformly at random for i = 1, ..., nbs do
S̃i ← {z1, ..., zmbs} ⊆ STi // sample mbs datapoints from STi uniformly at random for k = 1, ...,K do {θ1, ..., θL} ∼ Pφk // sample NN-parameters from prior for i = 1, ..., nbs do
ln Z̃(S̃i, Pφk)← LSELl=1 ( −√miL̂(θl, S̃i) ) − lnL // estimate generalized MLL
∇φkQ̃∗(φk)← ∇φk lnP(φk) + 1√nm̃+1 n nbs ∑nbs i=1∇φk ln Z̃(Si, Pφk) // compute score
φk ← φk + ηK ∑K k′=1 [ k(φk′ , φk)∇φk′ ln Q̃∗(φk′) +∇φk′k(φk′ , φk) ] ∀k ∈ [K] // update
part. Output: set of priors {Pφ1 , ..., PφK}

heading: B.2 META-TESTING
text: The meta-learned prior knowledge is now deployed by a base learner. The base learner is given a training dataset S̃ ∼ D pertaining to an unseen task τ = (D,m) ∼ T . With the purpose of approximating the generalized Bayesian posterior Q∗(S, P ), the base learner performs (normal) posterior inference. Algorithm 3 details the steps of the approximating procedure – referred to as target training – when performed via SVGD. For a data point x∗, the respective predictor outputs a
probability distribution given as p̃(y∗|x∗, S̃) ← 1K·L ∑K k=1 ∑L l=1 p(y ∗|hθkl (x ∗)). We evaluate the quality of the predictions on a held-out test dataset S̃∗ ∼ D from the same task, in a target testing phase (see Appendix C.2).
Algorithm 3 PACOH-NN-SVGD: meta-testing
Input: set of priors {Pφ1 , ..., PφK}, target training dataset S̃, evaluation point x∗ Input: kernel function k(·, ·), SVGD step size ν, number of particles L for k = 1, ...,K do {θk1 , ..., θkL} ∼ Pφk // initialize NN posterior particles from k-th prior while not converged do
for l = 1, ..., L do ∇θkl Q ∗(θkl ))← ∇θkl lnPφk(θ k l )) + √ m ∇θkl L(l, S̃) // compute score
θkl ← θkl + νL ∑L l′=1 [ k(θkl′ , θ k l )∇θk l′ lnQ∗(θkl′) +∇θk l′ k(θkl′ , θ k l ) ] ∀l ∈ [L] // update
particles Output: a set of NN parameters ⋃K k=1{θk1 ..., θkL}

heading: B.3 PROPERTIES OF THE SCORE ESTIMATOR
text: Since the marginal log-likelihood of BNNs is intractable, we have replaced it by a numerically stable Monte Carlo estimator ln Z̃(Si, Pφ) in (7), in particular
ln Z̃(Si, Pφ) := ln 1
L L∑ l=1 e− √ miL̂(θl,Si) = LSELl=1 ( −√miL̂(θl, Si) ) − lnL , θl ∼ Pφ . (50)
Since the Monte Carlo estimator involves approximating an expectation of an exponential, it is not unbiased. However, we can show that replacing lnZ(Si, Pφ) by the estimator ln Z̃(Si, Pφ), we still minimize a valid upper bound on the transfer error (see Proposition 3).
Proposition 3. In expectation, replacing lnZ(Si, Pφ) in (4) by the Monte Carlo estimate ln Z̃(Si, P ) := ln 1 L ∑L l=1 e
−√miL̂(θl,Si), θl ∼ P still yields an valid upper bound of the transfer error. In particular, it holds that
L(Q, T ) ≤ − 1 n n∑ i=1 1√ m̃ EP∼Q [lnZ(Si, P )] + ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃) (51)
≤ − 1 n n∑ i=1 1√ m̃ EP∼Q [ Eθ1,...,θL∼P [ ln Z̃(Si, P ) ]] + ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃).
(52)
Proof. Firsts, we show that:
Eθ1,...,θL∼P [ ln Z̃(Si, P ) ] = Eθ1,...,θL∼P [ ln 1
L L∑ l=1 e− √ miL̂(θl,Si)
]
≤ ln 1 L L∑ l=1 Eθl∼P [ e− √ miL̂(θl,Si) ] = lnEθ∼P [ e− √ miL̂(θ,Si)
] = lnZ(Si, P ) (53)
which follows directly from Jensen’s inequality and the concavity of the logarithm. Now, Proposition 3 follows directly from (53).
In fact, by the law of large numbers, it is straightforward to show that as L→∞, the ln Z̃(Si, P ) a.s.−−→ lnZ(Si, P ), that is, the estimator becomes asymptotically unbiased and we recover the original PACBayesian bound (i.e. (52) a.s.−−→ (51)). Also it is noteworthy that the bound in (52) we get by our estimator is, in expectation, tighter than the upper bound when using the naive estimator
ln Ẑ(Si, P ) := − √ mi 1
L L∑ l=1 L̂(θl, Si) θl ∼ Pφ
which can be obtained by applying Jensen’s inequality to lnEθ∼Pφ [ e− √ miL̂(θ,Si) ] . In the edge case
L = 1 our LSE estimator ln Z̃(Si, P ) falls back to this naive estimator and coincides in expectation with E[ln Ẑ(Si, P )] = − √ mi Eθ∼P L̂(θl, Si). As a result, we effectively minimize the looser upper bound
L(Q, T ) ≤ 1 n n∑ i=1 Eθ∼P [ L̂(θ, Si) ] + ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃). (54)
= Eθ∼P  1 nm̃ n∑ i=1 mi∑ j=1 − ln p(yij |xij , θ) + ( 1√ n + 1 n √ m̃ ) DKL(Q||P) + C(δ, n, m̃)
(55)
As we can see from (55), the boundaries between the tasks vanish in the edge case of L = 1, that is, all data-points are treated as if they would belong to one dataset. This suggests that L should be chosen greater than one. In our experiments, we used L = 5 and found the corresponding approximation to be sufficient.

heading: C EXPERIMENTS

heading: C.1 META-LEARNING ENVIRONMENTS
text: In this section, we provide further details on the meta-learning environments used in Section 6. Information about the numbers of tasks and samples in the respective environments can be found in Table S1.
Sinusoid Cauchy SwissFEL Physionet Berkeley n 20 20 5 100 36 mi 5 20 200 4 - 24 288
Table S1: Number of tasks n and samples per task mi for the different meta-learning environments.

heading: C.1.1 SINUSOIDS
text: Each task of the sinusoid environment corresponds to a parametric function
fa,b,c,β(x) = β ∗ x+ a ∗ sin(1.5 ∗ (x− b)) + c , (56) which, in essence, consists of an affine as well as a sinusoid function. Tasks differ in the function parameters (a, b, c, β) that are sampled from the task environment T as follows:
a ∼ U(0.7, 1.3), b ∼ N (0, 0.12), c ∼ N (5.0, 0.12), β ∼ N (0.5, 0.22) . (57) Figure S2a depicts functions fa,b,c,β with parameters sampled according to (57). To draw training
samples from each task, we draw x uniformly from U(−5, 5) and add Gaussian noise with standard deviation 0.1 to the function values f(x):
x ∼ U(−5, 5) , y ∼ N (fa,b,c,β(x), 0.12) . (58)
4 2 0 2 4 x
1
2
3
4
5
6
7
8
y
(a) Sinusoid tasks
6 4 2 0 2 4 x
1.0
1.5
2.0
2.5
3.0
y
(b) Cauchy tasks
Figure S2: Depiction of tasks (i.e., functions) sampled from the Sinusoid and Cauchy task environment, respectively. Note that the Cauchy task environment is two-dimensional (dim(X ) = 2), while (b) displays a one-dimensional projection.

heading: C.1.2 CAUCHY
text: Each task of the Cauchy environment can be interpreted as a two dimensional mixture of Cauchy distributions plus a function sampled from a Gaussian process prior with zero mean and SE kernel function k(x, x′) = exp ( ||x−x′||22
2l
) with l = 0.2. The (unnormalized) mixture of Cauchy densities
is defined as: m(x) =
6
π · (1 + ||x− µ1||22) +
3
π · (1 + ||x− µ2||22) , (59)
with µ1 = (−1,−1)> and µ2 = (2, 2)>. Functions from the task environments are sampled as follows:
f(x) = m(x) + g(x) , g ∼ GP(0, k(x, x′)) . (60)
Figure S2b depicts a one-dimensional projection of functions sampled according to (60). To draw training samples from each task, we draw x from a truncated normal distribution and add Gaussian noise with standard deviation 0.05 to the function values f(x):
x := min{max{x̃, 2},−3} , x̃ ∼ N (0, 2.52) , y ∼ N (f(x), 0.052) . (61)

heading: C.1.3 SWISSFEL
text: Free-electron lasers (FELs) accelerate electrons to very high speed in order to generate shortly pulsed laser beams with wavelengths in the X-ray spectrum. These X-ray pulses can be used to map nanometer scale structures, thus facilitating experiments in molecular biology and material science. The accelerator and the electron beam line of a FEL consist of multiple magnets and other adjustable components, each of which has several parameters that experts adjust to maximize the pulse energy (Kirschner et al., 2019a). Due do different operational modes, parameter drift, and changing (latent) conditions, the laser’s pulse energy function, in response to its parameters, changes across time. As a result, optimizing the laser’s parameters is a recurrent task.
Overall, our meta-learning environment consists of different parameter optimization runs (i.e., tasks) on the SwissFEL, an 800 meter long laser located in Switzerland (Milne et al., 2017). A picture of the SwissFEL is shown in Figure S3. The input space, corresponding to the laser’s parameters, has 12 dimensions whereas the regression target is the pulse energy (1-dimensional). For details on the individual parameters, we refer to Kirschner et al. (2019b). For each run, we have around 2000 data points. Since these data-points are generated with online optimization methods, the data are non-i.i.d. and get successively less diverse throughout the optimization. As we are concerned with meta-learning with limited data and want to avoid issues with highly dependent data points, we only take the first 400 data points per run and split them into training and test subsets of size 200. Overall,
Figure S3: Accelerator of the Swiss Free-Electron Laser (SwissFEL).
we have 9 runs (tasks) available. 5 of those runs are used for meta-training and the remaining 4 runs are used for meta-testing.

heading: C.1.4 PHYSIONET
text: The 2012 Physionet competition (Silva et al., 2012) published an open-access dataset of patient stays on the intensive care unit (ICU). Each patient stay consists of a time series over 48 hours, where up to 37 clinical variables are measured. The original task in the competition was binary classification of patient mortality, but due to the large number of missing values (around 80 % across all features), the dataset is also popular as a test bed for time series prediction methods, especially using Gaussian processes (Fortuin et al., 2019).
In this work, we treat each patient as a separate task and the different clinical variables as different environments. We use the Glasgow coma scale (GCS) and hematocrit value (HCT) as environments for our study, since they are among the most frequently measured variables in this dataset. From the dataset, we remove all patients where less than four measurements of CGS (and HCT respectively) are available. From the remaining patients we use 100 patients for meta-training and 500 patients each for meta-validation and meta-testing. Here, each patient corresponds to a task. Since the number of available measurements differs across patients, the number of training points mi ranges between 4 and 24.

heading: C.1.5 BERKELEY-SENSOR
text: We use data from 46 sensors deployed in different locations at the Intel Research lab in Berkeley (Madden, 2004). The dataset contains 4 days of data, sampled at 10 minute intervals. Each task corresponds to one of the 46 sensors and requires auto-regressive prediction, in particular, predicting the next temperature measurement given the last 10 measurement values. In that, 36 sensors (tasks) with data for the first two days are use for meta-training and whereas the remaining 10 sensors with data for the last two days are employed for meta-testing. Note, that we separate meta-training and -testing data both temporally and spatially since the data is non-i.i.d. For the meta-testing, we use the 3rd day as context data, i.e. for target training and the remaining data for target testing.

heading: C.2 EXPERIMENTAL METHODOLOGY
text: In the following, we describe our experimental methodology and provide details on how the empirical results reported in Section 6 were generated. Overall, evaluating a meta-learner consists of two phases, meta-training and meta-testing, outlined in Appendix B. The latter can be further sub-divided into target training and target testing. Figure S1 illustrates these different stages for our PAC-Bayesian meta-learning framework.
The outcome of the training procedure is an approximation for the generalized Bayesian posterior Q∗(S, P ) (see Appendix ), pertaining to an unseen task τ = (D,m) ∼ T from which we observe a dataset S̃ ∼ D. In target-testing, we evaluate its predictions on a held-out test dataset S̃∗ ∼ D from the same task. For PACOH-NN, NPs and MLAP the respective predictor outputs a probability distribution p̂(y∗|x∗, S̃) for the x∗ in S̃∗. The respective mean prediction corresponds to the expectation
of p̂, that is ŷ = Ê(y∗|x∗, S̃). In the case of MAML, only a mean prediction is available. Based on the mean predictions, we compute the root mean-squared error (RMSE):
RMSE = √√√√ 1 |S̃∗| ∑ (x∗,y∗)∈S∗ (y∗ − ŷ)2 . (62)
and the calibration error (see Appendix C.2.1). Note that unlike e.g. Rothfuss et al. (2019a) who report the test log-likelihood, we aim to measure the quality of mean predictions and the quality of uncertainty estimate separately, thus reporting both RMSE and calibration error.
The described meta-training and meta-testing procedure is repeated for five random seeds that influence both the initialization and gradient-estimates of the concerned algorithms. The reported averages and standard deviations are based on the results obtained for different seeds.

heading: C.2.1 CALIBRATION ERROR
text: The concept of calibration applies to probabilistic predictors that, given a new target input xi, produce a probability distribution p̂(yi|xi) over predicted target values yi. Corresponding to the predictive density, we denote a predictor’s cumulative density function (CDF) as F̂ (yj |xj) = ∫ yj −∞ p̂(y|xi)dy. For confidence levels 0 ≤ qh < ... < qH ≤ 1, we can compute the corresponding empirical frequency
q̂h = |{yj | F̂ (yj |xj) ≤ qh, j = 1, ...,m}|
m , (63)
based on dataset S = {(xi, yi)}mi=1 of m samples. If we have calibrated predictions we would expect that q̂h → qh as m→∞. Similar to (Kuleshov et al., 2018), we can define the calibration error as a function of residuals q̂h − qh, in particular,
calib-err = 1
H H∑ h=1 |q̂h − qh| . (64)
Note that we while (Kuleshov et al., 2018) reports the average of squared residuals |q̂h − qh|2, we report the average of absolute residuals |q̂h−qh| in order to preserve the units and keep the calibration error easier to interpret. In our experiments, we compute (64) withK = 20 equally spaced confidence levels between 0 and 1.

heading: C.3 HYPER-PARAMETER SELECTION
text: For each of the meta-environments and algorithms, we ran a separate hyper-parameter search to select the hyper-parameters. In particular, we use the hyperopt1 package (Bergstra et al., 2013) which performs Bayesian optimization based on regression trees. As optimization metric, we employ the average log-likelihood, evaluated on a separate validation set of tasks.
The scripts for reproducing the hyper-parameter search are included in our code repository2 For the reported results, we provide the selected hyper-parameters and detailed evaluation results under [Link will be made added upon acceptance]

heading: C.4 META-LEARNING FOR BANDITS - VACCINE DEVELOPMENT
text: In this section, we provide additional details on the experiment in Section 6.2.
We use data from Widmer et al. (2010) which contains the binding affinities (IC50 values) of many peptide candidates to seven different MHC-I alleles. Peptides with IC50 > 500nM are considered non-binders, all others binders. Following Krause & Ong (2011), we convert the IC50 values into negative log-scale and normalize them such that 500nM corresponds to zero, i.e. r := − log10(IC50) + log10(500) with is used as reward signal of our bandit.
1http://hyperopt.github.io/hyperopt/ 2[Link will be made added upon acceptance]
Allele A-0202 A-0203 A-0201 A-2301 A-2402 mi 1446 1442 3088 103 196
Table S2: MHC-I alleles used for meta-training and their corresponding number of meta-training samples mi.
We use 5 alleles to meta-learn a BNN prior. The alleles and the corresponding number of data points, available for meta-training, are listed in Table S2. The most genetically dissimilar allele (A-6901) is used for our bandit task. In each iteration, the experimenter (i.e. bandit algorithm) chooses to test one peptide among the pool of 813 candidates and receives r as a reward feedback. Hence, we are concerned with a 813-arm bandit wherein the action at ∈ {1, ..., 813} = A in iteration t corresponds to testing at-th peptide candidate. In response, the algorithm receives the respective negative log-IC50 as reward r(at).
As metrics, we report the average regret
Ravg.T := max a∈A r(a)− 1 T T∑ t=1 r(at)
and the simple regret RsimpleT := max
a∈A r(a)− max t=1,...,T r(at)
To ensure a fair comparison, the prior parameters of the GP for GP-UCB and GP-TS are meta-learned by minimizing the GP’s marginal log-likelihood on the five meta-training tasks. For the prior, we use a constant mean function and tried various kernel functions (linear, SE, Matern). Due to the 45-dimensional feature space, we found the linear kernel to work the best. So overall, the constant mean and the variance parameter of the linear kernel are meta-learned.

Review summaries:
One of the main issues that BNNs have to face is the choice of good informative priors in order to provide precise information about the uncertainty of predictions. The present work connects BNNs and PAC theory to formulate a new system to obtain a general-purpose approach for obtaining significative priors for Bayesian NNs. This is done by employing the closed-form solution for the PAC-Bayesian meta-learning problem. The meta learner here learns a hyper-posterior over the priors of the parameters of the BNN by using the closed-form expression PACOH  (PAC Optimal Hyper-Posterior). This is applied in the context of NNs, where the priors are to be defined over the NN parameters. Extensive experiments are carried out to show the performance both in regression and classification datasets, as well as its scalability. In all of these regards, the system seems to be competitive, improving on the results of previous state-of-the-art methods and producing promising results in real-world problems. 

The paper proposes a method for learning BNN priors based on optimising a PAC-Bayes bound for meta-learning, which they call PACOH-NN. This extends previous work on optimising PAC-Bayes bounds (PACOH), and is attractive for being principled in construction and effective in practice. The main claims are that fewer tasks are needed to achieve good performance in terms of both utility and calibration, and that the algorithm comes with performance guarantees.

The paper addresses the problem of learning data-driven priors for Bayesian neural networks. Assuming zero-centered Gaussian priors for BNNs often results in poor generalization and uncertainty quantification, whereas choosing informative priors is challenging due to limited interpretability of network weights. To be able to overcome these issues, the authors propose a meta-learning framework based on PAC-Bayesian theory, in which they optimize a PAC bound called PACOH in the space of possible posterior distributions of BNN weights. Unlike the previous approaches in the literature, their approach doesn’t rely on nested optimization schemes, instead they directly minimize PAC bound via a variational algorithm called PACOH-NN which is based on SVGD and reparameterization trick.