M ETA -L EARNING B AYESIAN N EURAL N ETWORK P RIORS B ASED ON PAC-B AYESIAN T HEORY
Anonymous authors
Paper under double-blind review
A BSTRACT
Bayesian deep learning is a promising approach towards improved uncertainty quantiﬁcation and sample efﬁciency.
Due to their complex parameter space, choosing informative priors for Bayesian Neural Networks (BNNs) is challenging.
Thus, often a naive, zero-centered Gaussian is used, resulting both in bad generalization and poor uncertainty estimates when training data is scarce.
In contrast, meta-learning aims to extract such prior knowledge from a set of related learning tasks.
We propose a principled and scalable algorithm for meta-learning
BNN priors based on PAC-Bayesian bounds.
Whereas previous approaches require optimizing the prior and multiple variational posteriors in an interdependent manner, our method does not rely on difﬁcult nested optimization problems, and moreover, it is agnostic to the variational inference method in use.
Our experiments show that the proposed method is not only computationally more efﬁcient but also yields better predictions and uncertainty estimates when compared to previous meta-learning methods and BNNs with standard priors.
I NTRODUCTION
Bayesian Neural Networks (BNNs) offer a probabilistic interpretation of deep learning by inferring distributions over the model’s weights (Neal, 1996).
With the potential of combining the scalability and performance of neural networks (NNs) with a framework for uncertainty quantiﬁcation, BNNs have lately received increased attention (Blundell et al., 2015; Gal & Ghahramani, 2016).
In particular, their ability to express epistemic uncertainty makes them highly relevant for applications such as active learning (Hern ´ andez-Lobato & Adams, 2015) and reinforcement learning (Riquelme et al., 2018).
However, BNNs face two major issues: 1) the intractability of posterior inference and 2) the difﬁculty of choosing good Bayesian priors.
While the former has been addressed in an extensive body of literature on variational inference (e.g. Blundell et al., 2015; Blei et al., 2016; Mishkin et al., 2018; Liu & Wang, 2016), the latter has only received limited attention (Vladimirova et al., 2019;
Ghosh & Doshi-Velez, 2017).
Choosing an informative prior for BNNs is particularly difﬁcult due to the high-dimensional and hardly interpretable parameter space of NNs.
Due to the lack of good alternatives, often a zero-centered, isotropic Gaussian is used, reﬂecting (almost) no a priori knowledge about the problem at hand.
This does not only lead to poor generalization when data is scarce, but also renders the Bayesian uncertainty estimates poorly calibrated (Kuleshov et al., 2018).
Meta-learning (Schmidhuber, 1987; Thrun & Pratt, 1998) acquires inductive bias in a data-driven way, thus, constituting an alternative route for addressing this issue.
In particular, meta-learners attempt to extract shared (prior) knowledge from a set of related learning tasks (i.e., datasets), aiming to learn in the face of a new, related task.
Our work develops a principled and scalable algorithm for metalearning BNN priors.
We build on the PAC-Bayesian framework (McAllester, 1999), a methodology from statistical learning theory for deriving generalization bounds.
Previous PAC-Bayesian bounds for meta-learners (Pentina & Lampert, 2014; Amit & Meir, 2018) require solving a difﬁcult optimization problem, involving the optimization of the prior as well as multiple variational posteriors in a nested manner.
Aiming to overcome this issue, we present a PAC-Bayesian bound that does not rely on nested optimization and, unlike (Rothfuss et al., 2020), can be tractably optimized for BNNs .
This makes the resulting meta-learner, referred to as PACOH-NN , not only much more computationally efﬁcient and scalable than previous approaches for meta-learning BNN priors (Amit & Meir, 2018), but also agnostic to the choice of approximate posterior inference method which allows us to combine it freely with recent advances in MCMC (e.g
Our experiments demonstrate that the computational advantages of PACOH-NN do not result in degraded predictive performance.
In fact, across several regression and classiﬁcation environments,
PACOH-NN achieves a comparable or better predictive accuracy than several popular meta-learning approaches, while improving the quality of the uncertainty estimates .
Finally, we showcase how metalearned PACOH-NN priors can be used in a real-world bandit task concerning the development of vaccines, suggesting that many other challenging real-world problems may beneﬁt from our approach.
R ELATED WORK
Bayesian Neural Networks.
The majority of research on BNNs focuses on approximating the intractable posterior distribution (Graves, 2011;
Blundell et al., 2015; Liu & Wang, 2016;
Wang et al., 2019).
In particular, we employ the approximate inference method of Liu & Wang (2016).
Another crucial question is how to select a good BNN prior (Vladimirova et al., 2019).
While the majority of work (e.g. Louizos & Welling, 2016; Huang et al., 2020) employs a simple zero-centered, isotropic Gaussian, Ghosh & Doshi-Velez (2017) and Pearce et al.
(2020) have proposed other prior distributions for BNNs.
In contrast, we go the alternative route of choosing priors in a data-driven way.
Meta-learning.
A range of popular methods in meta-learning attempt to learn the “learning program” in form of a recurrent model (Hochreiter et al., 2001; Andrychowicz et al., 2016; Chen et al., 2017), learn an embedding space shared across tasks (Snell et al., 2017; Vinyals et al., 2016) or the initialization of a NN such that it can be quickly adapted to new tasks (Finn et al., 2017; Nichol et al., 2018;
Rothfuss et al., 2019b).
A group of recent methods also uses probabilistic modeling to also allow for uncertainty quantiﬁcation (Kim et al., 2018; Finn et al., 2018; Garnelo et al., 2018).
Although the mentioned approaches are able to learn complex inference patterns, they rely on settings where metatraining tasks are abundant and fall short of performance guarantees.
In contrast, we provide a formal assessment of the generalization properties of our algorithm.
Moreover, PACOH-NN allows for principled uncertainty quantiﬁcation, including separate treatment of epistemic and aleatoric uncertainty.
This makes it particularly useful for sequential decision algorithms (Lattimore & Szepesvari, 2020).
PAC-Bayesian theory.
Previous work presents generalization bounds for randomized predictors, assuming a prior to be given exogenously (McAllester, 1999; Catoni, 2007;
Germain et al., 2016;
Alquier et al., 2016).
More recent work explores data-dependent priors (Parrado-Hernandez et al., 2012; Dziugaite & Roy, 2016) or extends previous bounds to the scenario where priors are meta-learned (Pentina & Lampert, 2014; Amit & Meir, 2018).
However, these meta-generalization bounds are hard to minimize as they leave both the hyper-posterior and posterior unspeciﬁed, which leads to nested optimization problems.
Our work builds on the results of Rothfuss et al.
(2020) who introduce the methodology to derive the closed-form solution of the PAC-Bayesian meta-learning problem.
However, unlike ours, their approach suffers from (asymptotically) non-vanishing terms in the bounds and relies on a closedform solution of the marginal log-likelihood.
By contributing a numerically stable score estimator for the generalized marginal log-likelihood, we are able to overcome such limitations, making PACBayesian meta-learning both tractable and scalable for a much larger array of models, including BNNs.
B ACKGROUND : T HE PAC-B AYESIAN F RAMEWORK
Bayesian Neural Networks.
Consider a supervised learning task with data S = { ( x j , y j ) } mj
= drawn from unknown distribution D .
In that, X = { x j } mj =1 ∈
X m denotes training inputs and Y =
{ y j } mj =1 ∈ Y m the targets.
For brevity, we also write z j :=
( x j , y j ) ∈ Z .
Let h θ : X → Y be a function parametrized by a NN with weights θ ∈ Θ .
Using the NN mapping, we deﬁne a conditional distribution p ( y | x, θ ) .
For regression, we set p ( y | x, θ ) =
N ( y | h θ ( x ) , σ 2 ) , where σ 2 is the observation noise variance.
For classiﬁcation, we choose p ( y | x, θ ) =
Categorical(softmax( h θ ( x ))) .
For Bayesian inference, one presumes a prior distribution p ( θ ) over the model parameters θ which is combined with the training data S into a posterior distribution p ( θ | X , Y )
∝ p ( θ ) p ( Y | X , θ ) .
For unseen test data points x ∗ , we form the predictive distribution as p ( y ∗ | x ∗ , X , Y ) =
(cid:82) p ( y ∗ | x ∗ , θ ) p ( θ | X , Y ) dθ .
The Bayesian framework presumes partial knowledge of the data-generating process in form of a prior distribution.
However, due to the practical difﬁculties in choosing an appropriate BNN prior, the prior is typically strongly misspeciﬁed (Syring & Martin, 2018).
As a result, modulating the inﬂuence of the prior relative to the likelihood during inference typically improves the empirical performance of BNNs and is thus a common practice (Wenzel et al., 2020).
Using such a ”tempered” posterior p τ ( θ | X , Y ) ∝ p ( θ ) p
( Y | X , θ ) τ with τ > 0 is also referred to as generalized Bayesian learning (Guedj, 2019).
The PAC-Bayesian Framework.
In the following, we introduce the most relevant concepts of
PAC-Bayesian learning theory.
For more details, we refer to Guedj (2019).
Given a loss function l : θ × Z → R , we typically want to minimize the generalization error L ( θ, D ) =
E z ∗ ∼D l ( θ, z ∗ ) .
Since D is unknown, the empirical error ˆ L ( θ, S )
= 1 m (cid:80) mi =1 l ( θ, z i ) is usually employed, instead.
In the PAC-Bayesian framework, we are concerned with randomized predictors , i.e., probability measures on the parameter space Θ , allowing us to reason about epistemic uncertainty .
In particular, we consider two such probability measures, a prior P ∈ M (Θ) and a posterior Q ∈ M (Θ) .
In here, by
M (Θ) , we denote the set of all probability measures on Θ .
While in Bayesian inference, the prior and posterior are tightly coupled through Bayes’ theorem, the PAC-Bayesian framework only requires the prior to be independent of the data S .
Using the deﬁnitions above, the so-called Gibbs error for a randomized predictor Q is deﬁned as L ( Q, D ) =
E h ∼ Q L ( h, D ) .
Similarly, we deﬁne its empirical counterpart as ˆ L ( Q, S ) =
E h ∼ Q ˆ L ( h, S ) .
The PAC-Bayesian framework provides upper bounds for the unknown Gibbs error in the following form:
Theorem 1.
(Alquier et al., 2016)
Given a data distribution D , a prior P ∈ M (Θ) , a conﬁdence level δ ∈ (0 , 1] , with probability at least 1 − δ over samples S ∼ D m , we have:
[Equation]
where Ψ( √ m )
=
ln
E θ ∼ P E S ∈D m exp (cid:104)
[Equation]
In that, Ψ( √ m ) is a log moment generating function that quantiﬁes how strong the empirical error deviates from the Gibbs error.
By making additional assumptions about the loss function l , we can bound Ψ( √ m ) and thereby obtain tractable bounds.
For instance, if l ( θ, z ) is bounded in [ a, b ] , we obtain Ψ( √ m ) ≤ (( b − a ) 2 ) / 8 by Hoeffding’s lemma.
For unbounded loss functions, it is common to assume bounded moments.
For instance, a loss is considered sub-gamma with variance factor s 2 and scale parameter c , under a prior P and data distribution D , if its deviations from the mean can be characterized by random variable V :
= L ( h, D ) − l ( h, z ) whose moment generating function is upper bounded by that of a Gamma distribution Γ( s, c ) (Boucheron et al., 2013).
In such case, we obtain Ψ( √ m ) ≤ s 2 / (2 − 2 c √ m )) .
Connecting the PAC-Bayesian framework and generalized Bayesian learning.
In PACBayesian learning we aim to ﬁnd the posterior that minimizes the bound in (1) which is in general a challenging optimization problem over the space of measures M (Θ) .
However, to our beneﬁt, it can be shown that the Gibbs posterior is the probability measure that minimizes (1).
For details we refer to Lemma 2 in the Appendix or Catoni (2007) and (Germain et al., 2016).
In particular, this gives us
[Equation]
where Z ( S, P ) is a normalization constant.
In a probabilistic setting, our loss function is the negative log-likelihood, i.e. l ( θ, z i ) :
= − log p ( z
[Equation]
| θ ) .
In this case, the optimal Gibbs posterior coincides with the generalized Bayesian posterior Q ∗ ( θ ; P, S )
∝ P ( θ ) p ( S | θ ) 1 / √ m /Z
( S, P ) where
Z ( S, P ) =
(cid:82) Θ P ( θ ) p ( S | θ ) 1 / √ m dθ is the generalized marginal likelihood of the data sample S .
PAC-B AYESIAN BOUNDS FOR M ETA -L EARNING
In the standard supervised learning setup (see Sec. 3), the learner has prior knowledge in the form of a prior distribution P , given exogenously, over the hypothesis space H .
When the learner faces a new task, it uses the evidence, provided as a dataset S , to update the prior into a posterior distribution
Q .
We formalize such a base learner Q ( S, P ) as a mapping Q : Z m × M ( H ) → M ( H ) that takes in a dataset and prior and outputs a posterior.
In contrast, in meta-learning we aim to acquire such a prior P in a data-driven manner , that is, by consulting a set of n statistically related learning tasks { τ 1 , ..., τ n } .
We follow the setup of Baxter (2000) in which all tasks τ i :=
( D i , S i ) share the same data domain Z :=
X × Y , parameter space Θ and loss function l ( θ, z ) , but may differ in their (unknown) data distributions
D i and the number of points m i in the corresponding dataset S
i ∼ D
m i i .
For our theoretical expositions, we assume w.l.o.g that m =
m i ∀ i .
Further, each task τ i ∼ T is drawn i.i.d.
from an environment T , a probability distribu- tion over data distributions and datasets.
The goal is to extract knowledge from the observed datasets which can then be used as a prior for learning on new target tasks τ ∼ T (Amit & Meir, 2018).
To extend the PAC-Bayesian analysis to the meta-learning setting, we again consider the notion of probability distributions over hypotheses/parameters.
However, while the object of learning has previously been the NN parameters θ , it is now the prior distribution P ∈ M (Θ) .
Accordingly, the meta-learner presumes a hyper-prior P ∈ M ( M ( H )) , i.e., a distribution over priors P .
Combining the hyper-prior
P with the datasets S 1 , ..., S n from multiple tasks, the meta-learner then outputs a hyper-posterior
Q over priors.
The hyper-posterior’s performance/quality is measured in form of the expected Gibbs error when sampling priors P from Q and applying the base learner, the so-called transfer-error:
[Equation]
While the transfer error is unknown in practice, we can estimate it using the empirical multi-task error
[Equation]
Similar to the PAC-Bayesian guarantees for single task learning, we can bound the transfer error by its empirical counterpart ˆ L ( Q , S 1 , ..., S n ) plus several tractable complexity terms:
Theorem 2.
Let Q : Z m × M ( H ) →
M ( H ) be a base learner and P ∈ M ( M ( H ))
some ﬁxed hyper-prior.
For all hyper-posteriors
Q ∈ M ( M ( H )) and δ ∈ (0 , 1] ,
[Equation]
holds with probability 1 − δ .
If the loss function is bounded, that is l : H × Z
→
[ a, b ] , the above inequality holds with C ( δ, n, m ) =
( b k − a k ) 2 8 √ m
[Equation]
1 √ n ln δ .
If the loss function is sub-gamma with variance factor s 2 I and scale parameter c
I for data distributions D i and s 2 II , c II for the task distribution T , the inequality holds with C ( δ, n, m
[Equation]
=
[Equation]
Under bounded loss assumption, Theorem 2 provides a structurally similar, but tighter bound than
Pentina & Lampert (2014) and Rothfuss et al.
In particular, by using a improved proof methodology, we are able to forgo a union bound argument, allowing us to reduce the negative inﬂuence of conﬁdence parameter δ .
Compared to Rothfuss et al.
, the D KL ( Q , P ) term has an improved decay rate, that is, 1 / ( n √ m ) + 1 / √ n as opposed to 1 / √ m + 1 / √ n ) .
Importantly, the bound in (3) is consistent, i.e. C ( δ, n, m )
→ 0 as n, m → ∞ .
Unlike Pentina & Lampert (2014) and Amit & Meir (2018), the theorem also provides guarantees for unbounded loss functions under moment constraints (see Appendix A.1 for details).
This makes Theorem 2 particularly relevant for probabilistic models such as BNNs in which the loss function coincides with the inherently unbounded negative log-likelihood.
Amit & Meir (2018) propose to meta-learn NN priors by directly minimizing a bound similar to (3).
However, the posterior inference for BNNs, i.e. obtaining Q i =
Q ( S i , P ) , is a stochastic optimization problem in itself whose solution in turn depends on P .
Hence, minimizing such meta-level bound w.r.t.
P constitutes a computationally infeasible two-level optimization problem.
To circumvent this issue, they jointly optimize P and n approximate posteriors Q i that correspond to the different datasets
S i , leading to an unstable and poorly scalable meta-learning algorithm.
To overcome these issues, we employ the methodology of Rothfuss et al.
and assume the
Gibbs posterior Q ∗ ( S i , P ) as a base learner.
As discussed in Section 3, Q ∗ ( S i , P ) does not only constitute a generalized Bayesian posterior but also minimizes the PAC-Bayesian bound.
Thus, the resulting bound in Corollary 1 is tighter than (3).
More importantly, the bound can be stated in terms of the partition function Z ( S i , P ) which allows us to forgo the explicit reliance on the task posteriors
Q i .
This makes the bound much easier to optimize as a meta-learning objective than previous bounds (e.g. Pentina & Lampert, 2014; Amit & Meir, 2018), since it no longer constitutes a two-level optimization problem.
Moreover, it renders the corresponding meta-learner agnostic to the choice of approximate inference method used to approach the Gibbs/Bayes posterior Q ∗ ( S, P ) when performing inference on a new task.
Corollary 1.
When choosing the Gibbs posterior Q ∗ ( S i , P ) :
= P
[Equation]
) as a base learner, under the same assumptions as in Theorem 2, it holds with probability 1 − δ that
[Equation]
wherein Z ( S i , P ) =
E θ ∼ P (cid:104) exp( −√ m ˆ L ( S i , θ ))
(cid:105) is the generalized marginal log-likelihood.
A natural way to obtain a PAC-Bayesian meta-learning algorithm could be to minimize (4) w.r.t.
Q .
Though, in general, this is a hard problem since it would require a minimization over M ( M ( H )) , the space of all probability measures over priors.
Following Rothfuss et al.
(2020), we exploit once more the insight that the minimizer of (4) can be written as Gibbs distribution (c.f. Lemma 2), allowing us to to derive such minimizing hyper-posterior Q ∗ , i.e. the PACOH , in closed form:
Proposition 1.
(PAC-Optimal Hyper-Posterior)
Given a hyper-prior P and datasets S 1 , ..., S n , the hyper-posterior Q that minimizes the PAC-Bayesian meta-learning bound in (4) is given by
[Equation]
with the partition function deﬁned as Z II = E P ∼P (cid:104) exp
[Equation]
We refer to Q ∗ ( P ) as PAC-optimal since, among all meta-learners, it gives us the best possible
PAC-Bayesian guarantees induced by Theorem 2.
PACOH-NN: A SCALABLE ALGORITHM FOR LEARNING BNN PRIORS
We discuss our main contribution, that is, how to translate the PACOH (Prop. 1) into a practical algorithm for meta-learning BNN priors.
To this end, we ﬁrst specify various components of the generic meta-learning setup presented in Sec. 4 and then discuss how to obtain a tractable approximation of Q ∗ .
The setup.
First, we deﬁne a family of priors { P φ : φ ∈ Φ } over the NN parameters θ .
For computational convenience, we employ Gaussian priors with diagonal covariance matrix, i.e.
P φ = N ( µ P , diag ( σ 2 P )) with φ := ( µ P , log σ P ) .
Note that we represent σ P in the log-space to avoid additional positivity constraints.
In fact, any parametric distribution such as normalizing ﬂows (Rezende & Mohamed, 2015) that allows for re-parametrized sampling and has a tractable log-density could be used.
As typical in the Bayesian framework, our loss function is the negative log-likelihood, i.e. l ( θ, z )
= − ln p ( y | x, θ ) for which we assume an additive Gaussian noise model p ( y | x, θ ) =
N ( y ; h θ ( x ) , σ 2 y ) in regression and a categorical softmax distribution in case of classiﬁcation.
Moreover, we use a zero-centered, spherical Gaussian P := N (0 , σ 2 P I ) as a hyper-prior over the parameters φ that specify the prior.
In our setup, the hyper-prior acts a form of meta-level regularization that penalizes complex priors.
Approximating the hyper-posterior.
Given the hyper-prior and (level-I) log-partition function ln Z ( S i , P ) , we can compute the PACOH Q ∗ up to the normalization constant Z II .
Such a setup lends itself to approximate inference methods (Blei et al., 2016).
In particular, we employ Stein
Variational Gradient Descent (SVGD) (Liu & Wang, 2016) which approximates Q ∗ as a set of particles ˆ Q = { φ 1 , ..., φ K } .
Initially, the particles φ K ∼ P (i.e. the priors’ parameters) are sampled randomly.
Subsequently, the method iteratively transports the set of particles to match
Q ∗ , by applying a form of functional gradient descent that minimizes D KL ( ˆ Q|Q ∗ ) in the reproducing kernel Hilbert space induced by a kernel function k ( · , · ) .
In each iteration, the particles are updated by φ k ← φ k + ηψ ∗ ( φ k ) with the step size η and
[Equation]
In that, ∇ φ k ln
[Equation]
is the score of Q
[Equation]
Approximating the generalized marginal log-likelihood.
The last remaining issue towards a viable meta-learning algorithm is the intractable generalized marginal likelihood ln Z ( S i , P φ ) =
ln E θ ∼ P φ e −√ m i ˆ L ( θ,S i ) .
Estimating and optimizing ln Z ( S i , P φ ) is not only challenging due to the high-dimensional expectation over Θ but also due to numerical instabilities inherent in computing e −√
m i ˆ L ( θ,S i ) when m i is large.
Aiming to overcome these issues, we compute numerically stable
[Figure]
y
[Figure]
Monte Carlo estimates of ∇ φ ln Z ( S i , P φ k ) by combining the LogSumExp (LSE) with the reparametrization trick (Kingma & Welling, 2014).
In particular, we draw L samples θ l := f
( φ, (cid:15) l ) = µ P + σ P (cid:12) (cid:15) l , (cid:15) l ∼ N (0 , I ) and compute the forward pass as follows:
[Equation]
The corresponding gradients follow as softmax-weighted average of score gradients:
[Equation]
Note that ln ˜ Z ( S i , P φ ) is a consistent but not an unbiased estimator of ln Z ( S i , P φ ) .
The following proposition ensures us that we still minimize a valid bound (see Appx. B.3 for details).
Proposition 2.
In expectation, replacing ln Z ( S i , P φ ) in (4) by the estimate ln ˜ Z ( S i , P ) still yields a valid upper bound of the transfer error
L ( Q , T ) for any L ∈ N .
Moreover, by the law of large numbers, we have that ln ˜ Z ( S i , P )
a.s. −−→
ln Z ( S i , P ) as L → ∞ , that is, for large sample sizes L , we recover the original PAC-Bayesian bound in (4).
In the opposite edge case, i.e. L = 1 , the boundaries between tasks vanish meaning that the meta-training data { S 1 , ..., S n } is treated as if it were one large dataset (cid:83) i S i (see Appx.
B.3 for further discussion).
The algorithm.
Algorithm 1 in Appendix B summarizes the proposed meta-learning method which we henceforth refer to as PACOH-NN .
To estimate the score ∇ φ k (cid:48)
ln Q ∗ ( φ k (cid:48) ) in (6), we can even use mini-batching on the task level.
This mini-batched version, outlined in Algorithm 2, maintains K particles to approximate the hyper-posterior, and in each forward step samples L NN-parameters (of dimensionaly | Θ | ) per prior particle that are deployed on a mini-batch of n bs tasks to estimate the score of Q ∗ .
As a result, the total space complexity is in the order of O ( | Θ | K + L ) and the computational complexity of the algorithm for a single update (c.f. (6)) is
O ( K 2 + KLn bs ) .
A key advantage of PACOH-NN over previous methods for meta-learning BNN priors (e.g. Pentina & Lampert, 2014; Amit & Meir, 2018) is that it turns the previously nested optimization problem into a much simpler stochastic optimization problem.
This makes meta-learning not only much more stable but also more scalable.
In particular, we do not need to explicitly compute / maintain the task posteriors Q i and can do mini-batching over tasks.
As a result, the space and compute complexity do not depend on the number of tasks n .
In contrast, MLAP (Amit & Meir, 2018) has a memory footprint of O ( | Θ | n ) making meta-learning prohibitive for more than 50 tasks.
A central feature of PACOH-NN is that is comes with principled meta-level regularization in form of the hyper-prior P which combats overﬁtting to the meta-training tasks (Qin et al., 2018).
As we show in our experiments, this allows us to successfully perform meta-learning with as little astasks.
This is unlike the the majority of the popular meta-learners (Finn et al., 2017; Kim et al., 2018;
Garnelo et al., 2018, e.g.) which rely on a large number of tasks to generalize well on the meta-level (Qin et al., 2018; Rothfuss et al., 2020).
E XPERIMENTS
We empirically evaluate the method introduced in Section 5, in particular, two variants of the algorithm: PACOH-NN-SVGD with K = 5 priors as particles and the edge case K = 1 which coincides with a maximum-a-posteriori (MAP) approximation of Q ∗ , thus referred to as PACOHNN-MAP .
In order to evaluate the quality of the meta-learned prior, i.e. meta-testing , we need to do approximate the BNN posterior Q ∗ ( S, P ) for which we use SVGD with 5 particles, too.
[Table] [Table]
Comparing it to various NN-based meta-learning approaches on various regression and classiﬁcation environments, we demonstrate that PACOH-NN (i) outperforms previous meta-learning algorithms in terms of predictive accuracy, (ii) improves the quality of uncertainty estimates and (iii) is much more scalable than previous PAC-Bayesian meta-learners.
Finally, we showcase how meta-learned PACOHNN priors can be harnessed in a real-world bandit task concerning peptide-based vaccine development.
6. M ETA -L EARNING BNN PRIORS FOR REGRESSION AND CLASSIFICATION
Figure 1 illustrates BNN predictions on a sinusoidal regression task with a standard Gaussian prior as well as a PACOH-NN prior meta-learned on sinusoidal functions of varying amplitude, phase and slope (details can be found in Appendix C.1).
In Figure 1a we can see that the standard Gaussian prior provides poor inductive bias, not only leading to bad mean predictions away from the testing points but also to poor 95% conﬁdence intervals (blue shaded areas).
In contrast, the meta-learned
PACOH-NN prior encodes useful inductive bias towards sinusoidal function shapes, leading to good predictions and uncertainty estimates, even in face of minimal training data (i.e. 1 training point).
Meta-learning benchmark.
In the following, we present a comprehensive benchmark study.
First, we use a BNN with a zero-centered, spherical Gaussian prior and SVGD posterior inference (Liu & Wang, 2016) as a baseline.
Second, we compare our proposed approach against various popular meta-learning algorithms, including neural processes ( NPs ) (Garnelo et al., 2018), model-agnostic meta-learning ( MAML )
(Finn et al., 2017), its ﬁrst-order version ( FOMAML ) (Nichol et al., 2018),
Bayesian MAML ( BMAML ) (Kim et al., 2018) and two variants of the PAC-Bayesian approach by Amit & Meir (2018) ( MLAP ).
In that, MLAP is the most similar to our approach as it also minimizes PAC-Bayesian bounds of the transfer error.
However, unlike PACOH-NN , it relies on nested optimization of the task posteriors Q i and the hyper-posterior Q .
Regression environments.
In our experiments, we consider one synthetic and four real-world metalearning environments for regression .
As a synthetic environment we follow Rothfuss et al.
(2020), employing a 2-dimensional mixture of Cauchy distributions plus random GP functions.
As real-world environments, we use datasets corresponding to different calibration sessions of the Swiss Free
Electron Laser ( SwissFEL ) (Milne et al., 2017; Kirschner et al., 2019b), as well as data from the
PhysioNet 2012 challenge, which consists of time series of electronic health measurements from intensive care patients (Silva et al., 2012), in particular the Glasgow Coma Scale ( GCS ) and the hematocrit value ( HCT ).
Here, the different tasks correspond to different patients.
Moreover, we employ the Intel Berkeley Research lab temperature sensor dataset ( Berkeley-Sensor ) (Madden, 2004) where the tasks require auto-regressive prediction of temperatures measurements corresponding to sensors installed in different locations of the building.
Further details can be found in Appendix C.1.
Table 1 reports the results of our benchmark study in terms of the root mean squared error (RMSE) on unseen test tasks.
PACOH-NN consistently outperforms the other meta-learning approaches and the standard BNN, demonstrating that the introduced meta-learning framework is not only sound, but also endows us with an algorithm that works well in practice.
Further, we hypothesize that by
[Table]
acquiring the prior in a principled data-driven manner (e.g., with PACOH-NN ), we can improve the quality of the BNN’s uncertainty estimates.
To investigate the effect of meta-learned priors on the uncertainty estimates of the BNN, we compute the probabilistic predictors’ calibration error, reported in Table 2.
The calibration error measures the discrepancy between predicted conﬁdence regions and actual frequencies of test data in the respective areas (Kuleshov et al., 2018).
Note that, since MAML only produces point predictions, the concept of calibration does not apply to it.
First, we observe that meta-learning priors with PACOH-NN consistently improves the standard BNN’s uncertainty estimates.
For meta-learning environments where the task similarity is high, i.e. SwissFel and Berkeley-Sensor , the improvement is substantial.
Surprisingly, while improving upon the standard BNN in terms of the RMSE, we ﬁnd that NPs consistently yields worse-calibrated predictive distributions than the BNN without meta-learning.
This may be due to meta-level overﬁtting as NPs lack any form of meta-level regularization (cf.
Qin et al., 2018; Rothfuss et al., 2020).
Classiﬁcation environments.
We conduct experiments with the multi-task classiﬁcation environment
Omniglot (Lake et al., 2015), consisting of handwritten letters across 50 alphabets.
Unlike previous work (e.g. Finn et al., 2017) we do not perform data-augmentation and do not re-combine letters of different alphabets, preserving the data’s original structure.
In particular, one task corresponds to 5-way classiﬁcation of letters within an alphabet.
This leaves us with much fewer tasks (i.e. 30 train andtest tasks), making the environment more challenging and more interesting for uncertainty quantiﬁcation.
This also allows us to include MLAP in the experiment which hardly scales to more than 50 tasks.
[Figure]
E x e c u t i o n t i m e ( i n h
o u r s ) [Figure] In Table 3, we report both the accuracy and calibration error for 2-shot and 5-shot classiﬁcation on test tasks.
Again, PACOH-NN yields the most accurate classiﬁcation results and the lowest calibration error.
Note that MAML fails to improve upon the standard BNN, i.e. demonstrating negative transfer.
This is consistent with previous work (Qin et al., 2018; Rothfuss et al., 2020) raising concerns about overﬁtting to the meta-training tasks and observing that MAML requires a large number of tasks to generalize well.
In contrast, by its very construction on meta-generalization bounds, PACOH-NN is able to achieve positive transfer even when the meta-training tasks are diverse and small in number.
Scalability.
Unlike the MLAP (Amit & Meir, 2018), PACOH-NN does not need to maintain posteriors Q i for the meta-training tasks and can use mini-batching on the task level.
As a result, it is computationally much faster and more scalable than previous PAC-Bayesian meta-learners.
This is reﬂected in its computation and memory complexity, discussed in Section 5.
Figure 2 showcases this computational advantage during meta-training with PACOH-NN-MAP and MLAP-S in the Sinusoids environment with varying number of tasks, reporting the maximum memory requirements, as well as the training time.
While MLAP’s memory consumption and compute time grow linearly and becomes prohibitively large even for less than 100 tasks, PACOH-NN maintains a constant memory and compute load as the number of tasks grow.
6. M ETA -L EARNING FOR B ANDITS - V ACCINE D EVELOPMENT
We showcase how a relevant real-world application such as vaccine design can beneﬁt from our proposed method.
In particular, the goal is to discover peptide sequences which bind to major histocompatibility complex class-I molecules (MHC-I).
MHC-I molecules present fragments of proteins from within a cell to T-cells, allowing the immune system to distinguish between healthy and infected cells.
Following the bandit setup of Krause & Ong (2011), each task corresponds to searching for maximally binding peptides, a vital step in the design of peptide-based vaccines.
The tasks differ in their targeted MHC-I allele, i.e., correspond to different genetic variants of the MHC-I protein.
We use data from Widmer et al.
(2010) which contains the binding afﬁnities (IC 50 values) of many peptide candidates to the MHC-I alleles.
The peptide candidates are encoded as 45-dimensional feature vector and the binding afﬁnities were standardized.
[Figure]
We use 5 alleles (tasks) to meta-learn a BNN prior with PACOH-NN and leave the most genetically dissimilar allele (A-6901) for our bandit task.
In each iteration, the experimenter (i.e. bandit algorithm) chooses to test one peptide among the pool of more thancandidates and receives its binding afﬁnity as a reward feedback.
In particular, we employ UCB (Lattimore & Szepesvari, 2020) and ThompsonSampling (TS) (Thompson, 1933) as bandit algorithms, comparing the BNN-based bandits with meta-learned prior ( PACOH-UCB/TS ) against a zero-centered Gaussian BNN prior ( BNN-UCB/TS ) and a Gaussian process ( GP-UCB ) (Srinivas et al., 2009).
Figure 3 reports the respective average regret and simple regret over 50 iterations.
Unlike the bandit algorithms with standard BNN/GP prior, PACOH-UCB/TS reaches near optimal regret within less than 10 iterations and after 50 iterations still maintains a signiﬁcant performance advantage.
This highlights the importance of transfer (learning) for solving real-world problems and demonstrates the effectiveness of PACOH-NN to this end.
While the majority of meta-learning methods rely on a large number of meta-training tasks (Qin et al., 2018), PACOH-NN allows us to achieve promising positive transfer, even in complex real-world scenarios with only a handful (in this case 5) tasks.
C ONCLUSION
Based on PAC-Bayesian theory, we present a novel, scalable algorithm for meta-learning BNN priors, that overcomes previous issues of nested optimization, by employing the closed-form solution of the PAC-Bayesian meta-learning problem.
Experiments show that our method, PACOH-NN , does not only come with computational advantages, but also achieves comparable or better predictive accuracy than several popular meta-learning approaches, while improving the quality of the uncertainty estimates – a key aspect of our approach.
The beneﬁts of our principled treatment of uncertainty – showcased in the real-world vaccine development bandit task – are particularly amenable to interactive machine learning systems.
This makes the integration of PACOH-NN with Bayesian optimization and reinforcement learning a potentially promising avenue to pursue.
While our experiments are limited to diagonal Gaussian priors and SVGD as approximate inference method, we hope that future work will build on the added ﬂexibility of our framework to possibly explore more recent approaches in variational inference (e.g. Wang et al., 2019) or consider more expressive priors such as normalizing ﬂows (Rezende & Mohamed, 2015).
A PPENDIX A P ROOFS AND D ERIVATIONS A. P ROOF OF T HEOREM
Lemma 1.
(Change of measure inequality)
Let f be a random variable taking values in a set A and let X 1 , ...,
X l be independent random variables, with X k ∈ A with distribution µ k .
For functions g k : A × A → R , k = 1 , ..., l , let ξ k ( f ) =
E X k ∼ µ k
[ g k ( f, X k )] denote the expectation of g k under µ k for any ﬁxed f ∈ A .
Then for any ﬁxed distributions π, ρ ∈ M ( A ) and any λ > 0 , we have that
[Equation]
Proof of Theorem 2 To prove the Theorem, we need to bound the difference between transfer error L ( Q , T ) and the empirical multi-task error ˆ L ( Q , S 1 , ..., S n ) .
To this end, we introduce an intermediate quantity, the expected multi-task error :
[Equation]
In the following we invoke Lemma 1 twice.
First, in step 1, we bound the difference between ˜ L ( Q , D 1 , ..., D n ) and ˆ L ( Q , S 1 , ..., S n ) , then, in step 2, the difference between L ( Q , T ) and ˜
L ( Q , D 1 , ..., D n ) .
Finally, in step 3, we use a union bound argument to combine both results.
Step 1 (Task speciﬁc generalization)
First, we bound the generalization error of the observed tasks τ
i =
( D i , m i ) , i = 1 , ..., n , when using a learning algorithm Q : M ×
Z m i
→ M , which outputs a posterior distribution Q i = Q ( S i , P ) over hypotheses θ , given a prior distribution P and a dataset S i ∼ D
m i i of size m i .
In that, we deﬁne ˜ m :=
( (cid:80) ni
=1 m − 1 i ) − 1 as the harmonic mean of sample sizes.
In particular, we apply Lemma 1 to the union of all training sets S (cid:48)
= (cid:83) ni
=1 S i with l = (cid:80) ni
=
m i .
Hence, each X k corresponds to one data point, i.e. X k = z ij and µ k
= D i .
Further, we set f =
( P, h 1 , ..., h n ) to be a tuple of one prior and n base hypotheses.
This can be understood as a two-level hypothesis, wherein P constitutes a hypothesis of the meta-learning problem and h i a hypothesis for solving the supervised task τ i .
Correspondingly, we take π = ( Q , Q n ) =
P · (cid:81) ni =1 P and ρ =
( Q , Q n ) =
Q · (cid:81) ni
=1 Q i as joint two-level distributions and g k ( f, X k ) =
1 nm i l ( h i , z ij ) as summand of the empirical multi-task error.
We can now invoke Lemma 1 to obtain that (11) and (14)
[Equation]
Using the above deﬁnitions
[Equation]
the KL-divergence term can be re-written in the following way:
[Equation]
Using (11) and (14) we can bound the expected multi-task error as follows:
[Equation]
Step 2 (Task environment generalization)
Now, we apply Lemma 1 on the meta-level.
For that, we treat each task as random variable and instantiate the components as X k
= τ
i , l = n and µ k = T .
Furthermore, we set ρ = Q , π = P , f = P and g k ( f, X k ) =
1 n L ( Q i , D i ) .
This allows us to bound the transfer error as
[Equation]
(16) wherein Υ II
[Equation]
Combining (15) with (16), we obtain
[Equation]
Step 3 (Bounding the moment generating functions)
[Equation]
Case I: bounded loss
If the loss function l ( h i , z ij ) is bounded in [ a k
[Equation]
b k ]
[Equation]
we can apply Hoeffding’s lemma to each factor in (18), obtaining:
[Equation]
Next
[Equation]
we factor out √ n from λ and β , obtaining
[Equation] [Equation]
Using
[Equation] [Equation]
we can apply Markov’s inequality w.r
[Equation]
t. the expectations over the task distribution T and data distributions D i to obtain that
[Equation] [Equation]
with probability at least 1 − δ .
Case II: sub-gamma loss
First, we assume that, ∀ i = 1 , ..., n the random variables V
I i :=
L ( h, D i ) − l
( h i , z i,j ) are sub-gamma with variance factor s 2 I and scale parameter c
I under the two-level prior ( P , P ) and the respective data distribution D i .
That is, their moment generating function can be bounded by that of a Gamma distribution Γ( s
I , c I ) :
[Equation]
Second, we assume that, the random variable V II :=
E ( D,S ) ∼T
[ L ( Q ( P, S ) , D )] − L ( Q ( P, S i ) , D i ) is sub-gamma with variance factor s 2 II and scale parameter c II under the hyper-prior P and the task distribution T .
That is, its moment generating function can be bounded by that of a Gamma distribution Γ( s 2 II , c II ) :
[Equation]
These two assumptions allow us to bound the expectation of (18) as follows:
[Equation]
Next, we factor out √ n from λ and β , obtaining
[Equation]
Finally, by using Markov’s inequality we obtain that
[Equation]
with probability at least 1 − δ .
To conclude the proof, we choose λ = n √ ˜ m and β = √ n .
A. P ROOF OF C OROLLARY
Lemma 2.
(Catoni, 2007)
Let A be a set, g : A → R a function, and ρ ∈ M ( A ) and π ∈ M ( A ) probability densities over A .
Then for any β > 0 and
[Equation] [Equation]
is the minimizing probability density
[Equation]
Proof of Corollary
When we choose the posterior Q as the optimal Gibbs posterior Q ∗ i :
=
Q ∗ ( S i , P )
[Equation]
it follows that
[Equation]
This allows us to write the inequality in (3) as
[Equation]
According to Lemma 2, the Gibbs posterior Q ∗ ( S i , P ) is the minimizer of (33), in particular ∀ P ∈
M ( H ) , ∀ i = 1 , ..., n :
[Equation]
Hence, we can write
[Equation]
(cid:88)
[Equation] [Equation]
[Equation]
(40) ≤
[Equation]
n
[Equation]
(cid:88)
[Equation]
| P
[Equation]
(cid:21) (41) + (cid:18)
[Equation]
n
[Equation]
n
[Equation]
˜
[Equation]
(cid:19)
D KL ( Q||P )
+ C ( δ
[Equation]
n
[Equation]
˜ m
[Equation]
=
ˆ L ( Q
[Equation]
[Equation] [Equation]
n
[Equation]
(cid:18)√
[Equation]
which proves that the bound for Gibbs-optimal base learners in (36) and (4) is tighter than the bound in Theorem 2 which holds uniformly for all Q ∈ M ( H ) .
[Table] [Table]
+hkpusgFRk2hNvAfN5Mp1FXTa5rkmsT7/yZMrEzwXMuiKtgFbIxd5b/J/KJcN3F+RQ1l/ENsye/WFeVPadmN2rO+2Z1s2m+Zov0hJ7Sc+nnJu3QLv5hPu7dD/pJvyzbaltfrOPCtbRg9jymS4+l/wI+JRRD</latexit> meta-training target training target testing
A. P ROOF OF P ROPOSITION 1: PAC-O PTIMAL H YPER -P OSTERIOR
An objective function corresponding to (4) reads as
[Equation]
To obtain J ( Q ) , we omit all additive terms from (4) that do not depend on Q and multiply by the scaling factor n √ ˜ m √ ˜
mn +1 .
Since the described transformations are monotone, the minimizing distribution of J ( Q ) , that is,
[Equation]
[Equation]
is also the minimizer of (4).
More importantly, J ( Q ) is structurally similar to the generic minimization problem in (30).
Hence, we can invoke Lemma 2 with A = M ( H ) , g ( a ) = − (cid:80) ni
=1 ln Z ( S i , P ) , β = 1 √ n ˜ m +1 , to show that the optimal hyper-posterior is
[Equation]
wherein
[Equation]
Technically, this concludes the proof of Proposition 1.
However, we want to remark the following result:
If we choose Q = Q ∗ , the PAC-Bayes bound in (4) can be expressed in terms of the meta-level partition function Z II , that is,
[Equation]
We omit a detailed derivation of (48) since it is similar to the one for Corollary 1.
B PACOH-NN: A SCALABLE ALGORITHM FOR LEARNING BNN PRIORS
In this section, we summarize and further discuss our proposed meta-learning algorithm PACOH-NN .
An overview of our proposed framework is illustrated in Figure S1.
Overall, it consists of two stages meta-training and meta-testing which we explain in more details in the following.
B. M ETA - TRAINING
The hyper-posterior distribution Q that minimizes the upper bound on the transfer error is given by
[Equation]
Provided with a set of datasets S 1 , ..., S 2 , the meta-learner minimizes the respective meta-objective, in the case of PACOH-SVGD , by performing SVGD on the Q ∗ .
Algorithm 1 outlines the required steps in more detail.
Input: hyper-prior P , datasets S 1 , ..., S n , kernel k ( · , · ) , step size η , number of particles K { φ 1 , ..., φ K } ∼ P // Initialize prior particles while not converged do for k = 1 , ..., K do { θ 1 , ..., θ L } ∼ P φ k // sample NN-parameters from prior for i = 1 , ..., n do
[Equation]
// estimate generalized MLL
[Equation]
// compute score
[Equation]
// SVGD
Alternatively, to estimate the score of ∇ φ k ˜ Q ∗ ( φ k ) we can use mini-batching at both the task and the dataset level.
Speciﬁcally, for a given meta-batch size of n bs and a batch size of m bs , we get
Algorithm 2.
Algorithm
Input: hyper-prior P , datasets S 1 , ..., S n
Input: kernel function k ( · , · ) , SVGD step size η , number of particles K { φ 1 , ..., φ K } ∼ P // Initialize prior particles while not converged do { T 1 , ..., T n bs } ⊆
[ n ] // sample n bs tasks uniformly at random for i = 1 , ..., n bs do ˜
S i ← { z 1 , ..., z m bs } ⊆
S T
i // sample m bs datapoints from S T
i uniformly at random for k = 1 , ..., K do { θ 1 , ..., θ L } ∼ P φ k // sample NN-parameters from prior for i = 1 , ..., n bs do ln
[Equation]
ln L // estimate generalized MLL
[Equation]
// compute score
[Equation]
[Table] [Table]
B. M ETA - TESTING
The meta-learned prior knowledge is now deployed by a base learner.
The base learner is given a training dataset ˜ S ∼ D pertaining to an unseen task τ
=
( D , m ) ∼ T .
With the purpose of approximating the generalized Bayesian posterior Q ∗ ( S, P ) , the base learner performs (normal) posterior inference.
Algorithm 3 details the steps of the approximating procedure – referred to as target training – when performed via SVGD.
For a data point x ∗ , the respective predictor outputs a probability distribution given as ˜ p
( y ∗ | x ∗ , ˜ S )
← 1 K · L (cid:80)
Kk =1 (cid:80)
Ll =1 p ( y ∗ | h
θ kl ( x ∗ )) .
We evaluate the quality of the predictions on a held-out test dataset ˜ S ∗ ∼ D from the same task, in a target testing phase (see Appendix C.2).
Algorithm
( · , · ) , SVGD
L for k = 1 , ..., K do { θ k 1 , ..., θ kL } ∼ P φ k // initialize NN posterior particles from k -th prior while not converged do for l = 1 , ...,
L do
[Equation]
, ˜ S ) //
compute score
[Equation]
] // update particles
B. P ROPERTIES OF THE SCORE ESTIMATOR
Since the marginal log-likelihood of BNNs is intractable, we have replaced it by a numerically stable
Monte Carlo estimator ln ˜ Z ( S i , P φ ) in (7),
in particular ln ˜ Z
( S
[Equation]
, P
[Equation] [Equation]
L
[Equation]
(cid:88)
[Equation]
Since the Monte Carlo estimator involves approximating an expectation of an exponential, it is not unbiased.
However, we can show that replacing ln Z ( S i , P φ ) by the estimator
ln ˜ Z ( S i , P φ ) , we still minimize a valid upper bound on the transfer error (see Proposition 3).
Proposition 3.
In expectation, replacing ln Z ( S i , P φ ) in (4) by the Monte Carlo estimate ln ˜ Z ( S i , P ) :
= ln 1 L (cid:80) Ll =1 e −√ m i
ˆ L ( θ l ,S i ) , θ l ∼ P still yields an valid upper bound of the transfer error.
In particular, it holds that
[Equation]
Proof.
Firsts, we show that:
[Equation]
which follows directly from Jensen’s inequality and the concavity of the logarithm.
Now, Proposition 3 follows directly from (53).
In fact, by the law of large numbers, it is straightforward to show that as L → ∞ , the ln ˜ Z ( S i , P ) a.s. −−→ ln Z ( S i , P ) , that is, the estimator becomes asymptotically unbiased and we recover the original PACBayesian bound (i.e. (52) a.s. −−→ (51)).
Also it is noteworthy that the bound in (52) we get by our estimator is, in expectation, tighter than the upper bound when using the naive estimator
[Equation]
which can be obtained by applying Jensen’s inequality to ln E
[Equation]
In the edge case
L = 1 our LSE estimator ln ˜ Z ( S i , P ) falls back to this naive estimator and coincides in expectation with E [ln ˆ Z ( S i , P )]
=
−√
m i E θ ∼ P ˆ L ( θ l , S i ) .
As a result, we effectively minimize the looser upper bound
[Equation]
As we can see from (55), the boundaries between the tasks vanish in the edge case of L = 1 , that is, all data-points are treated as if they would belong to one dataset.
This suggests that L should be chosen greater than one.
In our experiments, we used L = 5 and found the corresponding approximation to be sufﬁcient.
C E XPERIMENTS C. M ETA -L EARNING E NVIRONMENTS
In this section, we provide further details on the meta-learning environments used in Section 6.
Information about the numbers of tasks and samples in the respective environments can be found in
Table S1.
[Table]
C.1. S INUSOIDS
Each task of the sinusoid environment corresponds to a parametric function
[Equation]
56) which, in essence, consists of an afﬁne as well as a sinusoid function.
Tasks differ in the function parameters ( a, b, c, β ) that are sampled from the task environment T as follows:
[Equation]
Figure S2a depicts functions f a,b,c,β with parameters sampled according to (57).
To draw training samples from each task, we draw x uniformly from U ( − 5 , 5) and add Gaussian noise with standard deviation 0 .
1 to the function values f ( x ) :
[Equation]
4 x        y (a) Sinusoid tasks     4 x 1.
y (b) Cauchy tasks
C.1. C AUCHY
Each task of the Cauchy environment can be interpreted as a two dimensional mixture of Cauchy distributions plus a function sampled from a Gaussian process prior with zero mean and SE kernel function k
( x, x (cid:48) )
=
exp (cid:16) ||
x − x (cid:48)
||
22 2 l (cid:17) with l = 0 . 2 .
The (unnormalized) mixture of Cauchy densities is deﬁned as:
[Equation]
(59) with µ 1 = ( − 1 , − 1) (cid:62) and µ 2 = (2 , 2) (cid:62) .
Functions from the task environments are sampled as follows:
[Equation]
Figure S2b depicts a one-dimensional projection of functions sampled according to (60).
To draw training samples from each task, we draw x from a truncated normal distribution and add Gaussian noise with standard deviation 0 .
05 to the function values f ( x ) :
[Equation] [Equation]
C.1. S WISS FEL
Free-electron lasers (FELs) accelerate electrons to very high speed in order to generate shortly pulsed laser beams with wavelengths in the X-ray spectrum.
These X-ray pulses can be used to map nanometer scale structures, thus facilitating experiments in molecular biology and material science.
The accelerator and the electron beam line of a FEL consist of multiple magnets and other adjustable components, each of which has several parameters that experts adjust to maximize the pulse energy (Kirschner et al., 2019a).
Due do different operational modes, parameter drift, and changing (latent) conditions, the laser’s pulse energy function, in response to its parameters, changes across time.
As a result, optimizing the laser’s parameters is a recurrent task.
Overall, our meta-learning environment consists of different parameter optimization runs (i.e., tasks) on the SwissFEL, an 800 meter long laser located in Switzerland (Milne et al., 2017).
A picture of the SwissFEL is shown in Figure S3.
The input space, corresponding to the laser’s parameters, has 12 dimensions whereas the regression target is the pulse energy (1-dimensional).
For details on the individual parameters, we refer to Kirschner et al.
(2019b).
For each run, we have arounddata points.
Since these data-points are generated with online optimization methods, the data are non-i.i.d.
and get successively less diverse throughout the optimization.
As we are concerned with meta-learning with limited data and want to avoid issues with highly dependent data points, we only take the ﬁrst 400 data points per run and split them into training and test subsets of size 200.
Overall, we have 9 runs (tasks) available.
5 of those runs are used for meta-training and the remaining are used for meta-
C.1. P HYSIO N ET
The 2012 Physionet competition (Silva et al., 2012) published an open-access dataset of patient stays on the intensive care unit (ICU).
Each patient stay consists of a time series over 48 hours, where up to 37 clinical variables are measured.
The original task in the competition was binary classiﬁcation of patient mortality, but due to the large number of missing values (around 80 % across all features), the dataset is also popular as a test bed for time series prediction methods, especially using Gaussian processes (Fortuin et al., 2019).
In this work, we treat each patient as a separate task and the different clinical variables as different environments.
We use the Glasgow coma scale (GCS) and hematocrit value (HCT) as environments for our study, since they are among the most frequently measured variables in this dataset.
From the dataset, we remove all patients where less than four measurements of CGS (and HCT respectively) are available.
From the remaining patients we use 100 patients for meta-training and 500 patients each for meta-validation and meta-testing.
Here, each patient corresponds to a task.
Since the number of available measurements differs across patients, the number of training points m i ranges betweenand 24.
C.1. B ERKELEY -S ENSOR
We use data from 46 sensors deployed in different locations at the Intel Research lab in Berkeley (Madden, 2004).
The dataset contains 4 days of data, sampled at 10 minute intervals.
Each task corresponds to one of the 46 sensors and requires auto-regressive prediction, in particular, predicting the next temperature measurement given the last 10 measurement values.
In that, 36 sensors (tasks) with data for the ﬁrst two days are use for meta-training and whereas the remaining 10 sensors with data for the last two days are employed for meta-testing.
Note, that we separate meta-training and -testing data both temporally and spatially since the data is non-i.i.d.
For the meta-testing, we use the 3rd day as context data, i.e. for target training and the remaining data for target testing.
C. E XPERIMENTAL M ETHODOLOGY
In the following, we describe our experimental methodology and provide details on how the empirical results reported in Section 6 were generated.
Overall, evaluating a meta-learner consists of two phases, meta-training and meta-testing , outlined in Appendix B.
The latter can be further sub-divided into target training and target testing .
Figure S1 illustrates these different stages for our PAC-Bayesian meta-learning framework.
The outcome of the training procedure is an approximation for the generalized Bayesian posterior
Q ∗ ( S, P ) (see Appendix ), pertaining to an unseen task τ =
( D , m ) ∼ T from which we observe a dataset ˜ S ∼ D .
In target-testing , we evaluate its predictions on a held-out test dataset ˜ S ∗ ∼ D from the same task.
For PACOH-NN, NPs and MLAP the respective predictor outputs a probability distribution ˆ p ( y ∗ | x ∗ , ˜ S ) for the x ∗ in ˜ S ∗ .
The respective mean prediction corresponds to the expectation of ˆ p , that is ˆ y
=
ˆ E ( y ∗ | x ∗ , ˜ S ) .
In the case of MAML, only a mean prediction is available.
Based on the mean predictions, we compute the root mean-squared error (RMSE) :
[Equation]
62) and the calibration error (see Appendix C.2.1).
Note that unlike e.g. Rothfuss et al. (2019a) who report the test log-likelihood, we aim to measure the quality of mean predictions and the quality of uncertainty estimate separately, thus reporting both RMSE and calibration error.
The described meta-training and meta-testing procedure is repeated for ﬁve random seeds that inﬂuence both the initialization and gradient-estimates of the concerned algorithms.
The reported averages and standard deviations are based on the results obtained for different seeds.
C.2. C ALIBRATION E RROR
The concept of calibration applies to probabilistic predictors that, given a new target input x i , produce a probability distribution ˆ p ( y i | x i ) over predicted target values y i .
Corresponding to the predictive density, we denote a predictor’s cumulative density function (CDF) as ˆ F ( y j | x j ) =
(cid:82)
y
j −∞ ˆ p ( y | x i ) dy .
For conﬁdence levels 0 ≤ q h < ... <
q H ≤ 1 , we can compute the corresponding empirical frequency
[Equation]
(63) based on dataset S
= { ( x i , y i ) } mi =1 of m samples.
If we have calibrated predictions we would expect that ˆ q h → q h as m → ∞ .
Similar to (Kuleshov et al., 2018), we can deﬁne the calibration error as a function of residuals ˆ q h − q h , in particular,
[Equation]
Note that we while (Kuleshov et al., 2018) reports the average of squared residuals | ˆ q h − q h | 2 , we report the average of absolute residuals | ˆ q h − q h | in order to preserve the units and keep the calibration error easier to interpret.
In our experiments, we compute (64) with K = 20 equally spaced conﬁdence levels between 0 and 1.
C. H YPER -P ARAMETER S ELECTION
For each of the meta-environments and algorithms, we ran a separate hyper-parameter search to select the hyper-parameters.
In particular, we use the hyperopt 1 package (Bergstra et al., 2013) which performs Bayesian optimization based on regression trees.
As optimization metric, we employ the average log-likelihood, evaluated on a separate validation set of tasks.
The scripts for reproducing the hyper-parameter search are included in our code repository
For the reported results, we provide the selected hyper-parameters and detailed evaluation results under [Link will be made added upon acceptance]
C. M ETA -L EARNING FOR B ANDITS - V ACCINE D EVELOPMENT
In this section, we provide additional details on the experiment in Section 6.2.
We use data from Widmer et al.
(2010) which contains the binding afﬁnities (IC 50 values) of many peptide candidates to seven different MHC-I alleles.
Peptides with IC 50 > 500 nM are considered non-binders, all others binders.
Following Krause & Ong (2011), we convert the ICvalues into negative log-scale and normalize them such that 500nM corresponds to zero, i.e. r := − log 10 ( IC 50 )
+ log 10 (500) with is used as reward signal of our bandit.
[Table]
We use 5 alleles to meta-learn a BNN prior.
The alleles and the corresponding number of data points, available for meta-training, are listed in Table S2.
The most genetically dissimilar allele (A-6901) is used for our bandit task.
In each iteration, the experimenter (i.e. bandit algorithm) chooses to test one peptide among the pool of 813 candidates and receives r as a reward feedback.
Hence, we are concerned with a 813-arm bandit wherein the action a t ∈ { 1 , ..., 813 } =
A in iteration t corresponds to testing a t -th peptide candidate.
In response, the algorithm receives the respective negative log-ICas reward r ( a t ) .
As metrics, we report the average regret
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
and the simple regret
[Equation]
To ensure a fair comparison, the prior parameters of the GP for GP-UCB and GP-TS are meta-learned by minimizing the GP’s marginal log-likelihood on the ﬁve meta-training tasks.
For the prior, we use a constant mean function and tried various kernel functions (linear, SE, Matern).
Due to the 45-dimensional feature space, we found the linear kernel to work the best.
So overall, the constant mean and the variance parameter of the linear kernel are meta-learned.