PACOH: Bayes-Optimal Meta-Learning with PAC-Guarantees
Abstract
Meta-learning can successfully acquire useful inductive biases from data, especially when a large number of meta-tasks are available.
Yet, its generalization properties to unseen tasks are poorly understood.
Particularly if the number of meta-tasks is small, this raises concerns about overﬁtting.
We provide a theoretical analysis using the PAC-Bayesian framework and derive novel generalization bounds for meta-learning with unbounded loss functions and Bayesian base learners.
Using these bounds, we develop a class of PAC-optimal meta-learning algorithms with performance guarantees and a principled meta-regularization .
When instantiating our PACoptimal hyper-posterior (PACOH) with Gaussian processes as base learners, the resulting approach consistently outperforms several popular metalearning methods, both in terms of predictive accuracy and the quality of its uncertainty estimates.
1. Introduction
Meta-learning aims to extract prior knowledge from data, accelerating the learning process in light of new learning tasks (Schmidhuber, 1987; Thrun and Pratt, 1998).
Most existing meta-learning approaches focus on situations where the number of tasks is large (e g. Finn et al., 2017; Garnelo et al., 2018).
In many practical settings, however, the number of tasks available for meta-training is rather small .
In those settings, there is a risk of overﬁtting to the meta-training tasks (meta-overﬁtting, c.f., Qin et al., 2018), thus impairing the performance on yet unseen target tasks.
Hence, a key question is how to regularize the meta-learner in order to ensure that it generalizes to unseen tasks.
Especially when dealing with small amounts of data, reasoning about the uncertainty of a prediction model is crucial (Gal, 2016; Rothfuss et al., 2019a).
In this setting, Bayesian methods are a popular choice.
Moreover, they lend themselves easily to the meta-learning setting, since they offer a principled way to include prior knowledge into the learner (McNeish, 2016).
However, the associated uncertainty es- timates are often not well-calibrated due to model misspeciﬁcation and poorly chosen priors (Kuleshov et al., 2018).
The PAC-Bayesian framework provides a rigorous way to reason about the generalization performance of learners (McAllester, 1999).
However, initial PAC-Bayesian analyses of meta-learners (Pentina and Lampert, 2014; Amit and
Meir, 2018) only consider bounded loss functions which makes them hardly applicable to regression or probabilistic inference.
Moreover, they pose a nested optimization problem, which is computationally much more expensive than standard meta-learning approaches.
To overcome these issues, we derive the ﬁrst PAC-Bayesian bound for meta-learners with unbounded loss functions , which also does not rely on nested optimization .
For Bayesian learners, we further tighten our PAC-Bayesian bounds, relating them directly to the marginal log-likelihood of the Bayesian model.
This allows us to derive the
PAC-optimal hyper-posterior (PACOH) , which promises strong performance guarantees and a principled meta-level regularization.
Most importantly, it can be approximated using standard variational methods (Blei et al., 2016), giving rise to an entire range of meta-learning algorithms.
We instantiate our framework with Gaussian processes (GPs) as base learners and empirically evaluate the resulting approach across several synthetic and real-world regression environments.
We show that the meta-level regularization imposed by PACOH effectively prevents meta-overﬁtting.
This allows us to obtain substantial improvements over standard GPs, even when only ﬁve tasks are available for meta-training.
In our empirical study, PACOH consistently outperforms several popular meta-learning approaches, yielding more accurate predictions as well as improved uncertainty estimates.
Overall, PACOH not only provides optimal PAC-guarantees, but also constitutes an easily applicable meta-learning framework with strong empirical performance.
Especially in settings where data is scarce and good uncertainty estimates are crucial, PACOH offers a promising alternative to hand-designed priors.
In summary, our main contributions are the following:
• We present the ﬁrst PAC-Bayesian generalization bound for meta-learning with unbounded loss, which does not rely on nested optimization.
• For Bayesian learners, we provide a tighter metalearning bound, which relates to the model’s marginal log-likelihood.
• Based on this bound, we derive a PAC-optimal hyperposterior (PACOH) which can be tractably approximated with standard variational techniques.
• We provide strong empirical evidence that our proposed meta-learning method outperforms popular metalearning approaches on a range of environments.
2. Related work
Meta-learning.
One category of meta-learners directly attempts to learn the “learning program" as a recurrent model (Hochreiter et al., 2001; Andrychowicz et al., 2016;
Santoro et al., 2016; Chen et al., 2017).
Another popular approach is to meta-learn the initialization of a neural network such that it can be adapted to new tasks in a few gradient-steps (Finn et al., 2017; Nichol et al., 2018;
Rothfuss et al., 2019b) or to amortize the inference of a stochastic process (Garnelo et al., 2018).
Although they are able to learn complex inference patterns, these methods rely on a setting where meta-tasks are abundant and do not provide any performance guarantees.
The associated risk of meta-overﬁtting has previously been noted (Qin et al., 2018;
Fortuin and Rätsch, 2019), but still lacks a rigorous formal analysis under realistic assumptions (e.g., with unbounded loss functions).
We provide this principled treatment by studying the generalization properties of these algorithms within the PAC-Bayesian framework.
PAC-Bayesian theory.
Previous work presents generalization bounds for randomized predictors under various assumptions (McAllester, 1999; Catoni, 2007;
Alquier et al., 2016).
Such generalization guarantees have been extended to meta-learning (Pentina and Lampert, 2014; Amit and
Meir, 2018), but only for bounded loss functions.
Moreover, these meta-generalization bounds are hard to minimize as they leave both the hyper-posterior and posterior unspeciﬁed, which leads to nested optimization problems.
In contrast, our bounds also hold for unbounded losses and give rise to a tractable meta-learning objective.
In our derivations, we build on connections between PAC-Bayesian and classical
Bayesian inference, as described by Germain et al.
Hierarchical Bayes and Hyper-parameter Learning.
Our meta-learning setup, based on Baxter (2000), shares many aspects with hierarchical modeling and hyperparameter learning (MacKay, 1991).
However, work in this area either focuses on learning priors for a single task (Ong et al., 2005; Wilson et al., 2016; Reeb et al., 2018), makes stronger assumptions on how tasks are statistically related (Yu et al., 2005;
Bonilla et al., 2008), or lacks guarantees (e.g., Grant et al., 2018; Yoon et al., 2018).
3. Background: PAC-Bayesian Framework 3.1. Preliminaries and notation
A learning task is characterized by an unknown data distribution D , deﬁned over a domain Z , from which we are given a set of m observations S
= { z i } mi =1 , z i ∼ D .
By
S ∼ D
m we denote the i.i.d.
sampling of m data points.
In supervised learning, we are typically concerned with pairs
z i =
( x i , y i ) , where x i ∈
X are observed input features and
y i ∈ Y are target labels.
Given a sample S , our goal is to ﬁnd a hypothesis h ∈ H , typically a function h : X → Y in some hypothesis space
H , that enables us to make predictions on unseen input features x ∗ ∼ D x .
The quality of the predictions are measured by a loss function l : H × Z → R .
Accordingly, we want to minimize the expected error under the data distribution, that is, L ( h, D ) =
E z ∗ ∼D l ( h, z ∗ ) .
Since D is unknown, we typically use the empirical error ˆ
L ( [Equation] , S ) =
[Equation]
m (cid:80) m i =1 l ( h, z i ) instead.
In the PAC-Bayesian framework, we are concerned with randomized predictors , formally deﬁned as probability measures on the hypothesis space H .
This allows us to reason about the predictor’s (epistemic) uncertainty, resulting from the fact that only a ﬁnite number of data points are available for training.
We consider two such probability measures, the prior P ∈ M ( H ) and the posterior Q ∈ M ( H ) .
Here,
M ( H ) denotes the set of probability measures over the set
H .
Note that in Bayesian inference, the prior and posterior are assumed to be tightly connected through Bayes’ theorem.
In contrast, the PAC-Bayesian framework makes fewer assumptions and only requires the prior to be independent of the observed data, while the posterior may depend on it.
For a detailed treatment of the PAC-Bayesian methodology, we refer to Guedj (2019).
In the following, we overload the notation by also denoting the respective probability densities as Q and P .
Moreover, we assume that the Kullback-Leibler (KL) divergence D KL ( Q (cid:107) P ) exists.
Based on the error definitions above, we can deﬁne the so-called Gibbs error for a randomized predictor Q as L ( Q, D ) =
E h ∼ Q L ( h, D ) and its empirical counterpart as ˆ L ( Q, S ) =
E h ∼ Q ˆ L ( h, S ) .
3.2. PAC-Bayesian bounds
In practice, the generalization error L ( Q, D ) is unknown.
Thus, one typically resorts to empirical risk minimization (ERM), that is, optimizing ˆ L ( Q, S ) as a proxy for the true objective.
However, this may result in overﬁtting and poor generalization performance.
Naturally, the question arises whether we can bound the unknown generalization error based on its empirical estimate.
The PAC-Bayesian framework provides such a guarantee with high probability:
Theorem 1.
(Alquier et al., 2016)
Given a data distribution
D , a hypothesis space H , a loss function l : H × Z → R , a prior distribution P ∈ M ( F ) , a conﬁdence level δ ∈ (0 , 1] , and β > 0 , with probability at least− δ over samples
S ∼ D m , we have for all Q ∈ M ( H ) :
[Table]
<latexit [Table] +hkpusgFRk2hNvAfN5Mp1FXTa5rkmsT7/yZMrEzwXMuiKtgFbIxd5b/J/KJcN3F+RQ1l/ENsye/WFeVPadmN2rO+2Z1s2m+Zov0hJ7Sc+nnJu3QLv5hPu7dD/pJvyzbaltfrOPCtbRg9jymS4+l/wI+JRRD</latexit>
[Figure]
[Equation]
where Ψ( β, m )
[Equation]
Note that the prior distribution P must be independent of the data S .
Since Ψ( β, m ) contains L ( h, D ) which is unknown in practice, Theorem 1 cannot be directly applied.
However, if we make additional assumptions about the loss function l , we can bound Ψ( β, m ) and thereby obtain useful PACBayesian bounds.
In the following, we brieﬂy discuss three such assumptions and the resulting bounds.
For detailed explanations and derivations we refer to Appendix A.1.
Bounded loss function.
When l : H × Z → [ a, b ] is bounded, we can use Hoeffding’s lemma to obtain
[Equation]
Sub-gamma loss.
A loss function l is considered subgamma with variance factor s 2 and scale parameter c , under a prior π and data distribution D , if it can be described by a sub-gamma random variable V :=
L ( h, D ) − l ( h, z ) .
That is, its moment generating function is upper bounded by that of a Gamma distribution Γ( s, c ) .
For details see Boucheron et al.
and Germain et al.
We can use the sub-gamma assumption to bound Ψ( β, m ) as follows
[Equation]
Sub-gaussian loss.
A sub-gaussian loss with variance scan be considered the limit case c
→ 0 + of the sub-gamma assumption.
In this case, we obtain Ψ( β, m ) ≤ β [Equation] m .
3.3. Connections between the PAC-Bayesian framework and Bayesian Inference
Typically, we are interested in a posterior distribution Q that promises us the lowest generalization error.
In this sense, it seems natural to use the Q ∈ M ( H ) that minimizes the bound in (1).
The following lemma gives us the closed form solution to such a minimization problem over M ( H ) .
Lemma 1.
(Catoni, 2007)
Let H be a set, g : H → R a (loss) function and
Q ∈ M ( H ) and P ∈ M ( H ) probability densities over
H .
Then for any β > 0 and h ∈ H ,
[Equation]
is the minimizing probability density of arg
[Equation]
The respective minimizing distribution is known as optimal
Gibbs posterior Q ∗ (Catoni, 2007; Lever et al., 2013).
As a direct consequence of Lemma 1, for ﬁxed P, S, m , δ , and β
= m we can write the minimizer of (1) as
[Equation]
where Z ( S, P ) is a normalization constant.
In a probabilistic setting, we would deﬁne the loss function l ( · ) as the negative log-likelihood of the data, that is, l ( h, z i ) :
= − log p ( z i | h ) .
Thus, the optimal Gibbs posterior coincides with the Bayesian posterior
[Equation]
where Z ( S, P )
= (cid:82) H P ( h ) (cid:81) mj
=1 p ( z j | h ) dh is called the marginal likelihood of the sample S .
4. PAC-Bayesian bounds for Meta-Learning
We now present our main theoretical contributions.
An overview of our proposed framework is depicted in Figure 1.
4.1. Meta-Learning
In the standard supervised learning setup (see Section 3), we assumed that the learner has prior knowledge in the form of a prior distribution P .
When the learner faces a new task, it uses the evidence, observed in the form of a dataset S , to update the prior into a posterior distribution
Q .
We formalize such a base learner Q ( S, P ) as a mapping
Q : Z m ×M ( H ) → M ( H ) that takes in a dataset and prior and outputs a posterior.
Note that the number of samples m may vary between datasets.
While the prior P is arbitrary but ﬁxed with respect to S , it can have a major impact on the posterior chosen by the learning procedure Q ( S, P ) and the KL-divergence in the
PAC-Bayesian bound in (1).
Thus, the question arises how to properly choose P .
Importantly, for the PAC-Bayes bounds to hold, the choice of P cannot be based on our data set S .
However, we may consult data with similar properties to improve our choice of P .
This is the central idea of meta-learning (Vilalta and Drissi, 2002), which aims to learn inductive bias (e.g., in form of a prior) based on a set of statistically related tasks.
In the remainder of this section, we follow the setup of Baxter (2000) and Pentina and Lampert (2014).
So far, we have only considered a single learning task with distribution D .
To extend this to the meta-learning setting, we consider different tasks τ
i =
( D i , m i ) .
All tasks share the same data domain Z , hypothesis space H and loss function l : H × Z → R , but differ in their data distributions
D i and the number of samples m i drawn from it.
The metalearner observes n training sets S 1 , ..., S n corresponding to n different tasks.
Each dataset S i ∼ D
m i
i is assumed to be sampled i.i.d.
from its respective task distribution D i .
We further assume that each task τ i ∼ T is drawn i.i.d.
from an environment T , which is a probability distribution over data distributions and sample sizes.
The goal is to extract knowledge from the observed tasks which can then be used as prior knowledge for learning on new (yet unobserved) target tasks τ ∼ T (Amit and Meir, 2018).
The prior knowledge is represented as a prior distribution P ∈ M ( H ) over learning hypotheses h which the base learner Q ( S, P ) utilizes for inference on new tasks.
A meta-learner acquires such a prior distribution P in a datadriven way.
In order to be able to extend the PAC-Bayesian analysis to the meta-learning setting, we again consider the notion of probability distributions on function spaces.
The object of interest, which has previously been h ∈ H , is now the prior distribution P ∈ M ( H ) .
In the meta-learning PAC-Bayes framework, the metalearner presumes a hyper-prior P ∈ M ( M ( H )) , that is, a distribution over priors P .
Observing data sets S 1 , ..., S n from multiple tasks, the meta-learner then updates the hyperprior to a hyper-posterior Q .
The performance of this hyperposterior is measured as the expected Gibbs error when sampling priors P from Q and applying the base learner, the so-called transfer-error:
[Equation]
While L ( Q , T ) is unknown in practice, we can estimate it using the empirical multi-task error
[Equation]
4.2. PAC-Bayesian Meta-Learning bounds
In the following, we contribute a novel bound on the transfererror of a meta-learner.
The proofs and derivations can be found in Appendix A.3.
Theorem 2.
Let Q : Z m × M ( H ) →
M ( H ) be a base learner,
P ∈ M
[Equation]
M
[Equation]
H ))
some ﬁxed hyper-prior and ˜ m = (cid:16) 1 n (cid:80) ni
=1 1 m i (cid:17) − 1 the harmonic mean of dataset sizes.
For any conﬁdence level δ ∈ (0 , 1] the inequality L
[Equation]
holds uniformly over all hyper-posteriors Q ∈ M ( M ( H )) with probability− δ .
If the loss function is bounded, that is l : H × Z
→
[ a, b ] , the above inequality holds for
[Equation]
If the loss function is sub-gamma with variance factor sand scale parameter c , under the two-level prior ( P , P ) and the data distribution ( T , D ) , the inequality holds for
[Equation]
Proof.
See Appendix A.3.
The theorem uniformly bounds the true transfer error
L ( Q , T ) by the empirical multi-task error
ˆ L ( Q , S 1 , ..., S n ) plus several tractable complexity terms.
Similar to the metalearning PAC-Bayes bounds of Amit and Meir (2018), our proof consists of two main steps.
Firstly, we bound the base learner’s expected generalization error for each task when given a prior P ∼ Q and m i data samples (i.e., the error caused by observing only a ﬁnite number of samples).
Secondly, we bound the transfer-error on the meta-learning which is due to the fact that the meta-learning only receives ﬁnitely many tasks τ i from the task-environment T .
Finally, we employ a union bound argument to obtain the result stated in Theorem 2.
While previous PAC-Bayesian bounds for meta-learning (Pentina and Lampert, 2014; Amit and Meir, 2018) assume bounded loss functions, Theorem 2 also provides guarantees for unbounded loss functions.
This makes it particularly relevant for regression tasks where unbounded loss functions such as the negative log-likelihood or MSE are often used.
The results for unbounded losses require regularity conditions in the form of the sub-gamma assumption, which ensures that the distribution of losses is not heavy-tailed.
For instance in a Bayesian linear regression setup, Germain et al.
(2016) show that the negative log-likelihood loss is indeed sub-gamma.
Note that, in case of a sub-gamma loss, the bound always maintains a gap s 2− c between empirical multi-task and transfer error which does not decay to zero as n, m → ∞ .
While asymptotically consistent bounds (e.g., Pentina and Lampert (2014)) are theoretically more appealing, in practice they were shown to provide good guarantees only for large samples sizes.
In particular, sub-gamma
PAC-Bayesian bounds have been shown to be much tighter in simple regression scenarios with limited data ( m (cid:46) 10 4 )
(Germain et al., 2016).
The tradeoff between computational tractability and asymptotic consistency for this particular bound is further discussed in Appendix A.5.
The scale and variance parameters in (11) can be understood as implicit measures of task-environment complexity.
If the tasks τ ∼ T are very similar and simple, a relatively small c and s 2 will sufﬁce to bound the moment generating function of the empirical multi-task error distribution while dissimilar and harder tasks would require s 2 1 − c to be large.
Note that in order to compute the exact value of the bound, c and s 2 would have to be estimated for every environment, which is challenging for complex environments (Germain et al., 2016).
However, the bound can be efﬁciently minimized for meta-learning purposes without knowledge of these constants.
We thus leave the problem of estimating them for future work.
While Theorem 2 holds for any base learner Q ( S, P ) , in practice, we would preferably want to use the learner that gives us optimal performance guarantees.
As discussed in
Section 3.3, the Gibbs posterior not only minimizes PACBayesian error bounds, but also constitutes a generalization of the Bayesian posterior.
Under the assumption that we use the Gibbs posterior as base learner, the bound in (9) can be re-stated in terms of the partition function Z ( S i , P ) :
Corollary 1.
When choosing the Gibbs posterior
Q ∗ ( S i , P ) := P ( h ) exp( − m i
ˆ L ( S i , h )) /Z
( S i , P ) as a base learner, under the same assumptions as in Theorem 2, with probability 1 − δ
it holds that
[Equation]
Proof.
See Appendix A.4.
Since this bound assumes a PAC-optimal base learner, it is tighter than the bound in (9), which holds for any (potentially sub-optimal) Q ∈ M ( H ) .
If we choose the negative log-likelihood as the loss function, then ln Z ( S i , P ) coincides with the marginal log-likelihood, which is tractable for many popular learning models, such as GPs.
Accordingly, the bound in (12) consists of the expected marginal log-likelihood under the hyper-posterior
Q as well as the KL-divergence term which serves as a regularization on the meta-level.
As the number of training tasks n grows, the relative weighting of the KL term in (12) shrinks.
This is consistent with the general notion that regularization should be strong if only little data is available and vanish asymptotically as n, m → ∞ .
Theorem 2 and previous meta-level PAC-Bayes bounds (Pentina and Lampert, 2014; Amit and Meir, 2018) explicitly depend on the task speciﬁc posteriors Q i =
Q ( S i , P ) .
However, determining the Q i can itself be a non-trivial optimization problem.
Thus, employing such bounds as a meta-learning objective typically results in nested optimization problems (see Amit and Meir, 2018).
In contrast, the bound in (12) no longer depends on Q i , making it much easier to optimize as a meta-learning objective.
5. Meta-Learning the Hyper-Posterior 5.1. Optimizing the PAC-Bayes bound
In our PAC-Bayesian framework, the meta-learner is given a hyper-prior and data from several tasks, based on which it chooses a hyper-posterior Q ( P ) .
So far, we have not made any assumptions on how such a hyper-posterior is chosen.
Building on the generalization bounds stated in the previous section, we now discuss how a meta-learner should select Q .
Intuitively, we want a meta-learner that minimizes the transfer-error.
Since we cannot directly compute the transfererror, we may resort to minimizing its upper-bound in (12) with respect to Q .
In general, this is a hard problem since it would require a minimization over M ( M ( H )) , the space of all probability measures over priors.
However, by invoking
Lemma 1 on a monotone transformation of the PAC-Bayes bound in (12), we are able to derive the minimizing distribution Q ∗ in closed form.
Proposition 1.
(PAC-Optimal Hyper-Posterior)
Given a hyper-prior P ∈ M ( M ( H ))
and datasets S 1 , ..., S n , the hyper-posterior
Q ∈ M ( M ( H )) that optimizes the PAC- Bayesian meta-learning bound in (12) is given by
[Equation]
wherein the partition function Z II ( S 1 , ..., S n , P ) is deﬁned as Z
[Equation]
Proof.
See Appendix A.
[Equation] This gives us a tractable expression for the PAC-optimal hyper-posterior (PACOH) Q ∗ ( P ) up to the (level-II) partition function Z II , which is constant with respect to
P .
We refer to Q ∗ as PAC-optimal, since it provides the best possible meta-generalization guarantees among all meta-learners in the sense of Theorem 2.
5.2. Approximating the PAC-Optimal Hyper-Posterior
If the (level-I) log-partition function ln Z ( S i , P ) is tractable (e.g., in case of Bayesian linear regression or Gaussian process base learners), we can compute the PACOH Q ∗ up to the normalization constant Z II .
Such a setup lends itself to classical approximate inference methods (Andrieu et al., 2003; Blei et al., 2016; Liu and Wang, 2016).
Thus, Proposition 1 yields an entire class of possible meta-learning methods.
We now brieﬂy discuss several tractable approximations of Q ∗ which we evaluate empirically in Section 6.
Maximum A Posteriori (MAP).
This is the simplest and most crude method, which approximates Q ∗ ( P ) by a Dirac measure δ P ( P ∗ ) on the prior P ∗ that maximizes Q ∗ :
[Equation]
Variational Inference (VI).
In case of VI (Blei et al., 2016), we restrict the space of considered hyper-posteriors to a variational family F ⊂ M ( M ( H )) and aim to ﬁnd the posterior in
F that minimizes the KL-divergence to Q ∗ , that is,
[Equation]
It can be shown that the minimizing distribution ˜ Q in (15) is the same as the minimizer of the bound in (12) under the constraint Q ∈ F (see Appendix A.7 for proof).
Consequently, we can directly use (12) as an optimization objective.
Stein Variational Gradient Descent (SVGD).
SVGD (Liu and Wang, 2016) approximates Q ∗ as a set of particles
ˆ Q = { P 1 , ..., P K } .
Initially, particles P k ∼ P (which in our case are priors) are sampled randomly.
Then, the method iteratively transports the set of particles to match
Q ∗ , by applying a form of functional gradient descent that minimizes D KL ( ˆ Q|Q ∗ ) in the reproducing kernel Hilbert space induced by a kernel function k ( · , · ) .
In each iteration, the particles are updated by P k ← P k + η t φ ∗ ( P k ) with φ ∗ ( P ) =
1 K
K (cid:88) l =
[ k ( P l , P ) ∇ P l ln Q ∗ ( P l ) + ∇ P l k
( P l , P )] where η t is a (potentially time-dependent) step size.
6. Empirical Study: PAC-Bayesian MetaLearning of Gaussian Process Priors
In this section, we instantiate PACOH , our generic
PAC-Bayesian meta-learning framework from Section 5, with a speciﬁc learning model, namely Gaussian processes (GPs).
For a review of GP regression, see Appendix B and
Rasmussen and Williams (2006).
In particular, we metalearn the GP prior GP ( m θ ( x ) , k θ ( x, x (cid:48) )) , i.e. the mean and kernel function, parametrized by neural networks.
For details, we refer to Appendix C. We empirically evaluate the proposed meta-learning techniques on multiple simulated and real-world regression environments.
In particular, we provide quantitative and qualitative evidence that overﬁtting to meta-tasks (meta-overﬁtting) is a problem that can be alleviated using PACOH , and that our method improves upon previous approaches in terms of predictive accuracy and uncertainty calibration .
An implementation of PACOH for GPs is publicly available as open source code .
6.1. Meta-Learning environments
In our experiments, we consider two synthetic and three real-world meta-learning environments for regression.
Similar to Harrison et al.
(2018) and Fortuin and Rätsch (2019)
we use sinusoid functions that differ in their amplitude, phase-shift, slope and intercept.
As a second synthetic environment, we employ the density of 2-dimensional mixtures of Cauchy distributions plus random functions sampled from a GP-prior with squared exponential (SE) kernel.
As real-world environments, we use datasets corresponding to different calibration sessions of the Swiss Free Electron Laser ( SwissFEL ) (Milne et al., 2017; Kirschner et al., 2019b) as well as data from the PhysioNet 2012 challenge, which consists of time series of electronic health measurements from intensive care patients (Silva et al., 2012).
In case of PhysioNet , each task corresponds to one patient and the target is to predict a patient’s health trajectory based on a few measurements, in particular the Glasgow Coma Scale ( GCS ) and the hematocrit value ( HCT ).
Since our work focuses on meta-learning with limited data, we restrict the number of tasks and samples drawn from the different environments.
Note that in case of PhysioNet, the number of measurements available for training differs across patients.
Further details on the environments can be found in Appendix D.1.
6.2. PACOH combats meta-overﬁtting
The majority of previous work on meta-learning (e.g., Finn et al., 2017; Garnelo et al., 2018) assumes that tasks τ ∼ T are of large or inﬁnite supply during meta-training.
As a result, when presented with only a limited number of tasks, most existing meta-learning algorithms suffer from severe overﬁtting on the meta-training tasks, which adversely impacts their performance on unseen tasks from the same environment (Qin et al., 2018).
In contrast to the majority of literature on meta-learning, our proposed class of algorithms based on PACOH offers a principled treatment of overﬁtting.
Particularly, the KL-divergence D KL ( Q||P ) between
[Figure]
[Table]
Cauchy
SwissFEL
[Table]
To demonstrate the issue of meta-overﬁtting and the importance of meta-level regularization we compared the performance of our proposed regularization (PACOH-MAP) to meta-learning based on the marginal log-likelihood (MLL) (Fortuin and Rätsch, 2019) which uses an identical
GP model without regularization.
Figure 2 shows that
the
MLL method performs signiﬁcantly better on the meta-train tasks than on the meta-test tasks in both of our synthetic environments.
Our method, in contrast, does not exhibit this discrepancy and performs better than MLL on both kinds of tasks.
As expected, this effect is particularly pronounced when the number of meta-training tasks is small (i e., less than 100).
Similar results for other meta-learning methods as well as observations regarding the inﬂuence of the regularization strength can be found in the appendix (Sec. D.4).
This demonstrates the issue of meta-overﬁtting, underpins the importance of meta-level regularization, and shows that our proposed framework can be an effective measure to alleviate this problem.
6.3. PACOH improves the predictive performance
We now assess the empirical performance of our proposed
PAC-Bayesian algorithms and compare them against popular meta-learning methods across ﬁve regression environments.
As described in Section 5, we evaluate three approximations of the PACOH Q ∗ , namely the MAP estimate ( PACOHMAP ), VI within the family of fully-factorized Gaussians ( PACOH-VI ), and SVGD with 10 particles ( PACOH-SVGD ).
We compare our methods against a standard GP with SE kernel (no meta-learning), a GP with neural-network-based mean and kernel function, meta-learned by maximizing the marginal log-likelihood ( MLL ) (Fortuin and Rätsch, 2019), a neural process ( NP ) (Garnelo et al., 2018), model-agnostic meta-learning ( MAML )
(Finn et al., 2017), and the PACbound by Amit and Meir (2018) which requires nested optimization (MLAP).
Table 1 reports the results of our study in terms of the root mean squared error (RMSE) on unseen test tasks.
In addition, Table S2 in Appendix D.4 shows respective test log-likelihoods.
In the majority of environments, all our proposed methods outperform the other meta-learning approaches.
Notably, the SVGD approximation consistently performs best on all tasks.
This provides further evidence that the introduced meta-learning framework is not only sound, but also yields a class of algorithms that work well in practice.
Especially when only few training tasks are available (e.g., in case of the SwissFEL), the performance difference to previous meta-learning methods like NPs and
MAML is substantial.
As already observed in the metaoverﬁtting experiments, the performance gap between our method and the baselines decreases with an increasing number of meta-training tasks (see also Fig.
S5 in the appendix).
[Table]
Note that the MLL method is similar to PACOH-MAP, except that it does not incorporate a hyper-prior as metalevel regularization.
The fact that PACOH-MAP improves upon MLL in all environments indicates that MLL suffers from meta-overﬁtting and demonstrates that incorporating a hyper-prior, as suggested by the PAC-Bayesian theory, constitutes an effective way of regularization.
This is consistent with our ﬁndings in Section 6.2.
6.4. PACOH improves the uncertainty estimates
Reasoning about the predictive uncertainty of models is crucial in many machine learning applications (Chaloner and
Verdinelli, 1995; Shahriari et al., 2016).
While Bayesian methods (such as GPs) provide a general framework for quantifying uncertainty, the associated uncertainty estimates are often not well-calibrated due to model misspeciﬁcation and poorly chosen priors (Kuleshov et al., 2018).
We hypothesize that by acquiring the prior in a principled data-driven manner (e.g., with PACOH), we can improve the quality of the predictors’ uncertainty estimates, that is, their calibration.
Calibration refers to the consistency between the distributional forecasts and the distribution of the actual observations (Gneiting et al., 2007).
For example in regression, calibration means that data points y j should fall into the 90% conﬁdence interval of our predictor approximately 90% of the time (Kuleshov et al., 2018).
The calibration error measures the discrepancy between predicted conﬁdence regions and actual frequencies of test data in the respective areas.
We follow the deﬁnition of Kuleshov et al.
but report the square root of the calibration error for ease of interpretability.
For details, we refer to Appendix D.2.1.
Table 2 reports the calibration errors for the different metalearning methods and environments.
Note that the concept of calibration is not applicable to MAML since it only produces point estimates during meta-testing.
Compared to the standard GP with a ﬁxed prior, meta-learning a ﬂexible
GP prior with PACOH improves the calibration signiﬁcantly.
In contrast, due to its insufﬁcient regularization, MLL produces over-conﬁdent predictions, resulting in substantially higher calibration errors than even the standard GP baseline.
Once more, the SVGD approximation consistently achieves the best results across all environments.
Overall, this demonstrates that PAC-Bayesian meta-learning can be a practical alternative to hand-designed priors, yielding improved predictions as well as better uncertainty estimates.
7. Conclusion
We presented new PAC-Bayesian generalization bounds for meta-learning, including the ﬁrst ones for unbounded loss functions.
By utilizing the speciﬁc structure of the base learner, we were able to obtain meta-level generalization bounds that do not depend on the individual task posteriors
Q i and are tighter than the general bounds.
Additionally, we derived the PAC-optimal hyper-posterior that promises the best performance guarantees.
Overall, our contributions transform PAC-Bayesian meta-learning from a previously nested optimization problem (c.f., Amit and Meir (2018)) into standard approximate inference on Q ∗ .
When employing PACOH on GPs, we achieved strong empirical results.
Particularly in settings where meta-training data is scarce and good uncertainty estimates are crucial, PACOH offers a practical alternative to hand-designed priors.
Appendix A. Proofs and Derivations A.1. Bounding Ψ( β, k ) in Theorem
Below we state a generic version of Theorem 1 that uses notation which will make it easier for us to incorporate its results in later proofs.
Theorem A1 (Alquier et al. (2016)) .
Given a data distribution D ∈ M ( Z ) , a hypothesis space F , a loss function l : F × Z → R , a prior distribution π ∈ M ( F ) , a real number δ ∈ (0 , 1] , and a real number β > 0 , with probability at least 1 − δ over samples S ∼ D k , we have ∀ ρ ∈ M ( F ) :
[Equation]
where
[Equation]
Bounded loss function When the loss function is bounded, that is, l : F × Z → [ a, b ] , we can use Hoeffding’s lemma to bound Ψ( β, n ) .
In particular, we deﬁne the random variable l j = l ( h, z j ) ∈ [ a, b ] with expected value E [ l j ] =
L ( h, D ) and write
[Equation]
Sub-gamma loss A loss function l is considered sub-gamma with variance factor s 2 and scale parameter c , under a prior π and data distribution D , if it can be described by a sub-gamma random variable V :
= L ( h, D ) − l ( h, z ) , that is, its moment generating function is upper bounded by that of a Gamma distribution Γ( s, c ) :
[Equation]
For details see Boucheron et al.
and Germain et al.
We can use the sub-gamma assumption to bound Ψ( β, k ) as follows
[Equation]
Sub-gaussian loss
A sub-gaussian loss function with variance s 2 can be considered as a limit case of the previously discussed sub-gamma assumption when c → 0 + .
As direct consequence, Ψ( β, n ) can be bounded by
[Equation]
A.2. Proof of Lemma
In this section, we provide a proof for a generic version of Lemma 1 that uses general symbols for the different entities of interest.
This will make it easier to invoke the lemma in later proofs.
Lemma A1.
(Catoni, 2007)
Let A be a set, g : A → R a function, and ρ ∈ M ( A ) and π ∈ M ( A ) probability densities over A .
Then for any β > 0 and ∀ a ∈ A ,
[Equation]
[Equation]
is the minimizing probability density
[Equation]
For ease of exposition, we only provide a proof for the case when A = { a 1 , ..., a K } is a ﬁnite set.
An extension to the general case is straightforward and can be found in Zhang (2006) and Catoni (2007).
When A is countable, we can express probability densities as categorical distributions, that is, ρ ( a k ) =
ρ k and π ( a k ) =
π k ∀ k , where (cid:80)
Kk
=
ρ k = (cid:80) Kk =
π k = 1 , π k , ρ k > 0 .
This allows us to write the task of ﬁnding ρ ∗ as a constrained optimization problem:
[Equation]
The respective Lagrangian is
[Equation]
with the respective partial derivatives:
[Equation]
From (24) we get that
[Equation]
) which we insert in (25) to identify the Lagrange multiplier as
[Equation]
Finally, using (27) in (26) we obtain
[Equation]
which concludes the proof.
Note that ρ ∗ k fulﬁlls the constraint ρ ∈ [0 , 1] K since π k ≥ 0 and (cid:80)
Kk =
π k e − βg
( a k ) >
π k e − βg
( a k ) .
Hence, the Hessian ∇ 2 ρ J ( ρ ) =
diag ( ρ − 1 1 , ..., ρ − 1 K ) is positive semi-deﬁnite ∀ ρ ∈ [0 , ∞ ) K and ρ ∗ is the global minimizer of
J ( ρ ) within the K -dimensional probability simplex.
[Equation]
A.3. Proof of Theorem
Step 1 (Task speciﬁc generalization)
First, we bound the generalization error of each of the observed tasks τ
i =
( D i , m i ) , when using a learning algorithm Q : M×Z m i → M , which outputs a posterior distribution Q = Q ( S i , P ) over hypotheses θ , given a prior distribution P and a data set S i ∼ D
m i i of size m i .
In particular, we use Theorem A1 with the following instantiations.
The samples are Z k
= S i with k :=
m i and distribution
D := D i .
Further, we deﬁne f :=
( P, h ) as a tuple of a prior distribution P and hypothesis h .
This can be understood as a two-level hypothesis, wherein P constitutes a hypothesis of the meta-learning problem and h a hypothesis for solving the supervised task at hand.
In a similar manner, we deﬁne two-level priors and posteriors, that is, π =
( P , P ) and ρ =
( Q , Q ( S i , P )) ,
denoting the distribution of ﬁrst sampling P from Q and then [Table]
Using the above deﬁnitions, the KL-divergence term can be re-written in the following way:
[Equation]
Finally, we can bound the task speciﬁc generalization error based on Theorem A1 with β := m i , obtaining
L ( Q , D i ) ≤ E
[Equation]
which holds over all choices of ( Q , Q ) ∈ M
( M ( H ))
× M ( H ) with probability at least 1 − δ i .
Step 2 (Task environment generalization)
In the next step, we bound the generalization on the task-environment level.
Let τ i =
( D i , m i ) ∼ T be tasks drawn i.i.d.
from the task-environment distribution T .
We set f := P , π := P and ρ := Q and deﬁne the meta-level empirical loss function as l ( P, S )
=
ˆ L ( P, M ) = 1 n (cid:80) ni
=1 L ( P, D i ) .
Let P denote the hyper-prior and Q the hyper-posterior.
As a result from Theorem A1 with β := n , we obtain
[Equation]
over all Q ∈ M ( M ( H ))
with probability at least 1 − δ
Step 3 (Union bound)
Finally, we have to combine the results from Step 1 and 2.
For that, we bound the probability of the intersection of the events in (29) and (30) with a union bound argument.
In particular, for any δ > 0 we set δ i :=
δ 2 n and δ
=
δ 2 .
Further, we deﬁne ˜ m
= (cid:16) 1 n (cid:80) ni
=1 1 m i (cid:17) − 1 as the harmonic mean of the sample sizes m i .
[Equation]
[-+ ln 2 n δ + Ψ ( m i , m i ) (cid:19) (33) +n (cid:18) D KL ( Q||P ) + ln 
-]In that
[Equation]
we deﬁned C ( δ, n, ˜ m ) as
[Equation]
Bounded loss function If we assume that the loss function is bounded, that is l :
H × Z → [ a, b ] , we can use (2) to bound
C ( δ, n, ˜ m ) as follows:
[Equation]
Sub-gamma loss function If we assume that the loss function is sub-gamma with variance factor s 2 and scale parameter c , under the two-level prior ( P , P ) and the data distribution ( T , D ) , we can use (3) to bound C ( δ, n, ˜ m ) as follows:
[Equation]
A
Proof of Corollary 1: PAC-Bayes Meta-Learning Bound with Marginal Likelihood
When we choose the posterior Q as the optimal Gibbs posterior Q ∗ i :
= Q ∗ ( S i , P ) , it follows that
[Equation]
This allows us to write the inequality in (9) as
[Equation] [Equation]
According to Lemma A
[Equation]
the Gibbs posterior Q ∗ ( S i , P ) is the minimizer of (45), in particular Q ∗ ( S i , P
[Equation]
= P
[Equation]
Hence
[Equation]
we can write
[Equation]
[Equation]
which proves that the bound for Gibbs-optimal base learners in (48) and (12) is tighter than the bound in Theorem 2 which holds uniformly for all Q ∈ M ( H ) .
[Equation]
A.5. Alternative PAC-Bayesian bounds for meta-learning Choice of β in Theorem
In this section, we discuss alternative bounds to Theorem 2 and Corollary 1 that arise due to small modiﬁcations of the proofs.
In fact, if we choose β = √ m i in Step 1 and β = √ n in Step 2 of the proof, we obtain the following results:
Theorem 3.
Let Q : Z m × M ( H ) →
M ( H ) be a base learner, P ∈ M ( M ( H ))
some ﬁxed hyper-prior and √ ˜ m :
=n (cid:80) ni
=1 1 √ m i .
For any conﬁdence level δ ∈ (0 , 1] the inequality
[Equation]
holds uniformly over all hyper-posteriors Q ∈ M ( M ( H )) with probability 1 − δ .
If the loss function is bounded, that is l : H × Z
→
[ a, b ] , the above inequality holds for
[Equation]
If the loss function is sub-gamma with variance factor s 2 and scale parameter c , under the two-level prior ( P , P ) and the data distribution ( T , D ) , the inequality holds for
[Equation]
Proof.
Analogous to the proof in A.3 with β = √
m i in Step 1 and β = √ n Step 2.
Corollary 2.
When choosing the Gibbs posterior Q ∗ ( S i , P ) :
=
P ( h ) exp( −√ m i ˆ L ( S i , h )) /
˜ Z ( S i , P ) with
˜ Z
( S i , P ) = (cid:82)
H P ( h ) exp( −√ m i ˆ L ( S i , h )) dh as a base learner, under the same assumptions as in Theorem 3, with probability 1 − δ it holds that
[Equation]
Proof.
Analogous to the proof in Appendix A.4.
In contrast to Theorem 2, the constants in ˜ C
( δ, √ n, √ ˜ m ) now vanish to 0 as n and m i grow,
i.e. lim n,m i →∞ ˜ C
( δ, √ n, √ ˜ m )
= 0 .
However, this desirable property comes with two drawbacks.
First, the KL -terms in (54) vanish more slowly with n and m i than the similar terms in Theorem 2.
Second, and more importantly, ln ˜ Z ( S i , P ) in Corollary (2) no longer coincides with the Bayesian marginal log-likelihood when l ( h, z )
= − p ( z | h ) .
In contrast, ln Z ( S i , P ) in Corollary 1 is tractable for various Bayesian base learners such as GPs, thus, playing an instrumental role towards converting the bounds into scalable algorithms (c.f. Section 5).
A.6. Proof of Proposition 1: PAC-Optimal Hyper-Posterior In
this section, we derive the hyper-posterior distribution Q ∗ ∈ M ( M ( H )) which, given a hyper-prior P ∈ M ( M ( H )) and datasets S 1 , ..., S n , minimizes the PAC-Bayesian meta-learning bound in (12).
An objective function corresponding to (12) reads as
[Equation]
[Equation] To obtain J ( Q ) , we omit all additive terms from (12) that do not depend on Q and multiply by the scaling factor ˜ mn ˜ m + n .
Since the described transformations are monotone, the minimizing distribution of J ( Q ) , that is, [Equation]
[Equation]
is also the minimizer of (12).
More importantly, J ( Q ) is structurally similar to the generic minimization problem in (21).
Hence, we can invoke Lemma A1 with A = M ( H ) , g ( a ) = − (cid:80) ni =1 1 m i
ln Z ( S i , P ) , β = ˜ m ˜ m + n , to show that the optimal hyper-posterior is
[Equation]
wherein
[Equation]
Technically, this concludes the proof of Proposition 1.
However, we want to remark the following interesting result:
If we choose Q = Q ∗ , the PAC-Bayes bound in (12) can be expressed in terms of the meta-level partition function Z II , that is,
[Equation]
We omit a detailed derivation of (61) since it is similar to the one in Appendix A.4.
A.7. Proof of the Equivalence of Variational Inference and Minimization of the PAC-Bayes Meta-Learning Bound
We can write the optimal variational distribution ˜ Q with respect to Q ∗ as
[Equation]
[Equation]
Here, we multiplied (65) with (cid:0) 1 n + 1˜ m (cid:1) to obtain (66).
Now it is straightforward to see that (66) is the same as the meta-learning PAC-Bayes bound in (12) up to the constant C ( δ, n, ˜ m ) .
Hence, we can conclude that variational inference with respect to Q ∗ is equivalent to minimizing (12) over the same variational family F .
B. Gaussian process regression
In GP regression, each data point corresponds to a feature-target tuple
z i,j = ( x i,j , y i,j
) ∈ R d × R .
For the i -th dataset, we write S i =
( X i , y i ) , where X i = ( x i, 1 , ..., x i,m i ) (cid:62) and y i =
( y i, 1 , ..., y i,m i ) (cid:62) .
GPs are a Bayesian method in which the prior P θ ( h ) =
GP ( h | m θ ( x ) , k θ ( x, x (cid:48) )) is speciﬁed by a positive deﬁnite kernel
k θ : X × X
→ R and a mean function m θ : X → R .
The empirical loss under the GP posterior Q ∗ coincides with the negative log-likelihood of regression targets y i , that is, ˆ L ( Q ∗ , S i ) =
− 1 m i ln p
( y i |
X i ) .
Under a Gaussian likelihood p ( y | h ) =
N ( y ; h ( x ) , σ 2 I ) , the marginal log-likelihood ln Z ( S i , P θ ) =
ln p
( y i |
X i , θ ) can be computed in closed form as
[Equation]
(67) where ˜ K X ,θ = K X ,θ + σ 2 I , with the kernel matrix K X ,θ = ( k θ ( x l , x k )) m i l,k =1 , observation noise variance σ 2 , and mean vector m X ,θ
= ( m θ ( x 1 ) , ..., m θ ( x m i ))
(cid:62) .
Previous work on Bayesian model selection in the context of GPs argues that the log-determinant 12 ln | ˜ K X ,θ | in the marginal log-likelihood (67) acts as a complexity penalty (Rasmussen and Ghahramani, 2001; Rasmussen and Williams, 2006).
However, we suspect that this complexity regularization is only effective if the class of considered priors is restrictive, for instance if we only optimize a small number of parameters such as the length- and output scale of a squared exponential kernel.
If we consider expressive classes of GP priors (e.g., our setup where the mean and kernel function are neural networks), such a complexity penalty could be insufﬁcient to avoid meta-overﬁtting.
Indeed, this is what we also observe in our experiments (see Sec. 6).
C. PACOH-based Meta-Learning of GP priors
In this section, we provide further details on the three variants of PACOH, introduced in Section 5 of the paper and employed in our experiments.
Following Section 6, we instantiate our framework with GP base learners.
Since we are interested in meta-learning, we deﬁne the mean and kernel function both as parametric functions.
Similar to Wilson et al.
(2016) and
Fortuin and Rätsch (2019), we instantiate m θ and k θ as neural networks, where the parameter vector θ can be meta-learned.
To ensure the positive-deﬁniteness of the kernel, we use the neural network as feature map Φ θ ( x ) on top of which we apply a squared exponential (SE) kernel.
Accordingly, the parametric kernel reads as k θ ( x, x (cid:48) )
= 12 exp (cid:0) −|| Φ θ ( x ) − Φ θ ( x (cid:48) )
|| 22 (cid:1) .
Both m θ ( x ) and Φ θ ( x ) are fully-connected neural networks with 4 layers with each 32 neurons and tanh non-linearities.
The parameter vector θ represents the weights and biases of both neural networks.
As hyper-prior we choose a zero-mean isotropic Gaussian, that is, P ( θ ) =
N (0 , σ 2 P I ) .
C.1. PACOH-MAP
A maximum a-posteriori (MAP) approximation of Q ∗ is the simplest way to obtain a practical meta-learning algorithm from our PAC-Bayesian theory.
In particular, it approximates the Q ∗ ( P ) by a Dirac measure δ P ( P ∗ ) on the prior P ∗ that maximizes Q ∗ :
[Equation]
We can restate (68) as minimizing the following objective:
[Equation]
When we use GP base learners, as described above, the meta-learning objective reduces to the GP’s marginal log-likelihood plus L 2 -regularization.
[Equation]
To minimize J MAP ( θ ) , we use mini-batch gradient descent.
Note that here, mini-batches are created on the meta-level.
That is, we sample mini-batches of four tasks to compute the gradients of J MAP ( θ ) and use all data-points in the respective datasets S i .
Since weight-decay is equivalent to the L 2 -regularization in (70), we use AdamW as optimizer (Loshchilov and
Hutter, 2019) to optimize the ﬁrst term in (70).
C.2. PACOH-VI
When aiming to approximate Q ∗ via variational inference, we consider a family F = {Q γ | γ ∈ Γ } ⊂ M ( M ( H )) of parametric distributions with parameter vector γ and try to solve the following minimization problem:
[Equation]
In our setup, where P θ is the GP-prior, this optimization problem can be re-stated as minimizing the negative ELBO:
[Equation]
In particular, we use fully-factorized Gaussians as the variational family, that is, we meta-learn γ = ( µ Q , σ Q ) .
To estimate the gradients of J VI ( γ ) with respect to γ , we employ a pathwise gradient estimator, also known as reparametrization trick.
This means that we sample a set of K prior parameters θ k := µ Q + σ Q (cid:15) k , (cid:15) k ∼ N (0 , I ) as well as a mini-batch of H datasets S 1 , ..., S H and compute an unbiased gradient estimate of (72) as follows:
[Equation]
During gradient descent with ∇ γ J VI ( γ ) , we employ the adaptive learning rate method Adam.
Due to the double stochasticity (mini-batches of tasks and mini-batches of θ k ∼ Q γ ), we found that in practice the gradient estimates of the marginal log-likelihood term in (73) are very noisy whereas the second and third term (meta-level KL-divergence) are subject to less variance.
As a result, the less noisy gradients of the KL-divergence dominate during gradient-descent, pushing the
VI posterior towards the prior which in turn leads to a higher entropy of Q γ and even noisier gradient estimates for the marginal log-likelihood term.
To counteract this explosion in hyper-posterior entropy, we add a weight 0 < λ < 1 in front of ln P ( θ ) − ln Q γ ( θ ) which effectively down-scales the effect of D KL ( Q γ ||P ) and improves results signiﬁcantly.
Here, we treat λ as a hyper-parameter.
C.3. PACOH-SVGD
SVGD (Liu and Wang, 2016) approximates Q ∗ as a set of particles
ˆ Q = { P 1 , ..., P K } .
In our described setup, each particle corresponds to the parameters of the GP prior, that is, ˆ Q = { θ 1 , ..., θ K } .
Initially, we sample random priors θ k ∼ P from our hyper-posterior.
Then, the SVGD iteratively transports the set of particles to match Q ∗ , by applying a form of functional gradient descent that minimizes D KL ( ˆ Q|Q ∗ ) in the reproducing kernel Hilbert space induced by a kernel function k ( · , · ) .
We choose a squared exponential kernel with length scale (hyper-)parameter (cid:96) , that is, k ( θ, θ (cid:48) )
=
exp (cid:16) − || θ − θ (cid:48)
||
22 2 (cid:96) (cid:17) .
In each iteration, the particles are updated by
[Equation]
φ [Equation] Here, we can again estimate ∇ θ l ln Q ∗ ( θ l ) with a mini-batch of H datasets S 1 , ..., S H :
[Equation]
Importantly, ∇ θ l ln Q ∗ ( θ l ) does not depend on Z II which makes SVGD tractable.
C.4. Meta-Testing / Target-Training with PACOH
Meta-learning with PACOH, as described above, gives us an approximation of Q ∗ .
In target-testing (see Figure 1), the base learner is instantiated with the meta-learned prior P θ , receives a dataset ˜ S =
( ˜ X , ˜ y )
from an unseen task D ∼ T and
Sinusoid
Cauchy
SwissFEL
Physionet n    m i   4 -
4 x        y
[Figure]
outputs a posterior Q as product of its inference. In our GP setup, Q is the GP posterior and the predictive distribution ˆ p ( y ∗ | x ∗ , ˜ X , ˜ y , θ ) is a Gaussian (for details see Rasmussen and Williams, 2006).
Since the meta-learner outputs a distribution over priors, that is, the hyper-posterior Q , we may obtain different predictions for different priors P θ ∼ Q , sampled from the hyper-posterior.
To obtain a predictive distribution under our meta-learned hyper-posterior, we empirically marginalize Q .
That is, we draw a set of prior parameters θ 1 , ..., θ K ∼ Q from the hyper-posterior, compute their respective predictive distributions ˆ p ( y ∗ | x ∗ , ˜ X ,
˜ y , θ k )
and form an equally weighted mixture:
[Equation]
Since we are concerned with GPs, (74) coincides with a mixture of Gaussians.
As one would expect, the mean prediction under Q (i.e., the expectation of (74)), is the average of the mean predictions corresponding to the sampled prior parameters θ 1 , ..., θ K .
In case of PACOH-VI, we sample K = 100 priors from the variational hyper-posterior ˜ Q .
For PACOH-SVGD, samples from the hyper-posterior correspond to the K = 10 particles.
PACOH-MAP can be viewed as a special case of
SVGD with K = 1 , that is, only one particle.
Thus, ˆ p ( y ∗ | x ∗ , ˜ X , ˜ y , Q )
≈ ˆ p ( y ∗ | x ∗ , ˜ X ,
˜ y , θ MAP ) is a single Gaussian.
D. Experiments D.1. Meta-Learning Environments
In this section, we provide further details on the meta-learning environments used in Section 6.
Information about the numbers of tasks and samples in the respective environments can be found in Table S1.
D.1.1. S INUSOIDS
Each task of the sinusoid environment corresponds to a parametric function
[Equation]
which, in essence, consists of an afﬁne as well as a sinusoid function.
Tasks differ in the function parameters ( a, b, c, β ) that are sampled from the task environment T as follows:
[Equation]
Figure S1a depicts functions f a,b,c,β with parameters sampled according to (76).
To draw training samples from each task, we draw x uniformly from U ( − 5 , 5) and add Gaussian noise with standard deviation 0 .
1 to the function values f ( x ) :
[Equation]
D.1.2. C AUCHY
Each task of the Cauchy environment can be interpreted as a two dimensional mixture of Cauchy distributions plus a function sampled from a Gaussian process prior with zero mean and SE kernel function k
( x, x (cid:48) )
=
exp (cid:16) ||
x − x (cid:48)
||
22 2 l (cid:17) with l = 0 . 2 .
The (unnormalized) mixture of Cauchy densities is deﬁned as:
[Equation]
(78) with µ 1 = ( − 1 , − 1) (cid:62) and µ 2 = (2 , 2) (cid:62) .
Functions from the task environments are sampled as follows:
[Equation]
Figure S1b depicts a one-dimensional projection of functions sampled according to (79).
To draw training samples from each task, we draw x from a truncated normal distribution and add Gaussian noise with standard deviation 0 .
05 to the function values f ( x ) :
[Equation]
D.1.3. S WISS FEL
Free-electron lasers (FELs) accelerate electrons to very high speed in order to generate shortly pulsed laser beams with wavelengths in the X-ray spectrum.
These X-ray pulses can be used to map nanometer scale structures, thus facilitating experiments in molecular biology and material science.
The accelerator and the electron beam line of a FEL consist of multiple magnets and other adjustable components, each of which has several parameters that experts adjust to maximize the pulse energy (Kirschner et al., 2019a).
Due do different operational modes, parameter drift, and changing (latent) conditions, the laser’s pulse energy function, in response to its parameters, changes across time.
As a result, optimizing the laser’s parameters is a recurrent task.
Overall, our meta-learning environment consists of different parameter optimization runs (i.e., tasks) on the SwissFEL, an 800 meter long laser located in Switzerland (Milne et al., 2017).
A picture of the SwissFEL is shown in Figure S2.
The input space, corresponding to the laser’s parameters, has 12 dimensions whereas the regression target is the pulse energy (1-dimensional).
For details on the individual parameters, we refer to Kirschner et al.
(2019b).
For each run, we have around 2000 data points.
Since these data-points are generated with online optimization methods, the data are non-i.i.d.
and get successively less diverse throughout the optimization.
As we are concerned with meta-learning with limited data and want to avoid issues with highly dependent data points, we only take the ﬁrst 400 data points per run and split them into training and test subsets of size 200.
Overall, we have 9 runs (tasks) available.
5 of those runs are used for meta-training and the remaining 4 runs are used for meta-testing.
D.1.4. P HYSIO N ET
The 2012 Physionet competition (Silva et al., 2012) published an open-access data set of patient stays on the intensive care unit (ICU).
Each patient stay consists of a time series over 48 hours, where up to 37 clinical variables are measured.
The original task in the competition was binary classiﬁcation of patient mortality, but due to the large number of missing values (around 80 % across all features), the data set is also popular as a test bed for time series prediction methods, especially using Gaussian processes (Fortuin and Rätsch, 2019; Fortuin et al., 2019).
In this work, we treat each patient as a separate task and the different clinical variables as different environments.
We use the Glasgow coma scale (GCS) and hematocrit value (HCT) as environments for our study, since they are among the most frequently measured variables in this data set.
From the dataset, we remove all patients where less than four measurements of CGS (and HCT respectively) are available.
From the remaining patients we use 100 patients for meta-training andpatients each for meta-validation and meta-testing.
Here, each patient corresponds to a task.
Since the number of available measurements differs across patients, the number of training points m i ranges between 4 and 24.
D.2. Experimental Methodology
In the following, we describe our experimental methodology and provide details on how the empirical results reported in
Section 6 were generated.
Overall, evaluating a meta-learner consists of two phases, meta-training and meta-testing .
The latter can be further sub-divided into target training and target testing .
Figure 1 illustrates these different stages for our
PAC-Bayesian meta-learning framework.
Meta-training The meta-learner is provided with a set of datasets S 1 , ..., S 2 and optimizes its respective meta-objective for instance in case of PACO-VI, the negative ELBO with respect to Q ∗
Meta-testing This phase aims to evaluate the empirical generalization properties of the meta-learned prior knowledge.
In particular, we evaluate how well the base learner, instatiated with the prior knowledge, performs on multiple unseen tasks τ =
( D , m ) ∼ T .
For that, two steps are required:
In the target-training phase, the base learner is given a training dataset ˜
S ∼ D and performs (normal) inference.
Then, in target-testing , we evaluate its predictions on a test dataset ˜ S ∗ ∼ D from the same task.
For PACOH, MLL, GP, and NP, the respective predictor outputs a probability distribution ˆ p ( y ∗ | x ∗ , ˜ S ) for the x ∗ in ˜ S ∗ .
The respective mean prediction corresponds to the expectation of ˆ p , that is ˆ y
=
ˆ E ( y ∗ | x ∗ , ˜ S ) .
In the case of
MAML, only a mean prediction is available.
Based on the mean predictions, we compute the root mean-squared error (RMSE) :
[Equation]
Similarly, we compute the average log-likelihood :
[Equation]
and the calibration error (see Appendix D.2.1).
The described meta-training and meta-testing procedure is repeated for ﬁve random seeds that inﬂuence both the initialization and gradient-estimates of the concerned algorithms.
The averages and standard deviations reported in Figure 2 as well as
Tables 1, 2 and 2 are based on the results obtained for different seeds.
D.2.1. C ALIBRATION E RROR
The concept of calibration applies to probabilistic predictors that, given a new target input x i , produce a probability distribution ˆ p ( y i | x i ) over predicted target values y i .
Corresponding to the predictive density, we denote a predictor’s cumulative density function (CDF) as ˆ F ( y j | x j ) =
(cid:82)
y
j −∞ ˆ p ( y | x i ) dy .
For conﬁdence levels 0 ≤ q h < ... <
q H ≤ 1 , we can compute the corresponding empirical frequency
[Equation]
based on dataset S = { ( x i , y i ) } mi =1 of m samples.
If we have calibrated predictions we would expect that ˆ q h → q h as m → ∞ .
Similar to (Kuleshov et al., 2018), we can deﬁne the calibration error as a function of residuals ˆ q h − q h , in particular, calib-err = (cid:118)(cid:117)(cid:117)(cid:116)
[Equation]
In our experiments, we compute (84) with K = 20 equally spaced conﬁdence levels between 0 and 1.
D.3. Hyper-Parameter Selection
For each of the meta-environments and algorithms, we ran a separate hyper-parameter search to select the hyper-parameters.
In particular, we use the hyperopt 2 package (Bergstra et al., 2013) which performs Bayesian optimization based on regression trees.
As optimization metric, we employ the average log-likelihood, evaluated on a separate validation set of tasks.
The scripts for reproducing the hyper-parameter search are included in our code repository
For the results, reported in the
Tables 1, 2, and S2, we provide the selected hyper-parameters and detailed evaluation results under [Link will be added upon acceptance]
D.4. Further Experimental Results
Meta-overﬁtting To further demonstrate the issue of meta-overﬁtting, we conducted experiments where we varied the regularization intensity and analyzed its impact on the meta-test performance.
In particular, we consider the two synthetic environments as well as real-world data from the SwissFEL (see Sec. 6.1).
We report results for PACOH-MAP with a zero-mean isotropic Gaussian hyper-prior P ( θ ) =
N (0 , σ 2 P I ) .
In this case, the regularization imposed by D KL ( Q||P ) coincides with weight decay (that is, L 2 -regularization) on the neural network parameters θ .
Its strength, controlled by the weight-decay multiplier, corresponds to σ 2 P (Hanson and Pratt, 1989).
We show the optimal regularization parameters for
PACOH-MAP on the different environments in Figure S3.
The fact that these optima are non-zero for all environments provides additional evidence of the effect of meta-overﬁtting.
[Figure]
In order to investigate whether the phenomenon of meta-overﬁtting, which we have observed consistently for PACOH-MAP and MLL, is also relevant to other meta-learning methods, we also report the meta-train test error and the meta-test test error across different numbers of tasks.
The results, analogous to Figure 2, are plotted in Figure S4, showing a signiﬁcant difference between the meta-train and meta-test error that vanishes as the number of tasks becomes larger.
Once more, this supports our claim that meta-overﬁtting is a relevant issue and should be addressed in a principled manner.
[Figure]
Benchmark comparison across different numbers of tasks In addition to the performance comparison in Table 1, we assessed the performance of the different methods on our synthetic environments as a function of the number of meta-training tasks (Fig. S5).
As expected and already seen in the meta-overﬁtting experiments, the performance gap between the PACOH methods and the baselines is especially pronounced for a small number of meta-training tasks, when meta-overﬁtting is a more severe problem.
However, even for larger numbers of tasks, PACOH-MAP and PACOH-SVGD seem to outperform most baselines.
Due to the high computational complexity of MLAP, getting increasingly burdensome as the number of task grows, we did not include MLAP in this experiment.
[Figure]
Benchmark comparison in terms of the test log-likelihood Complementary to the the tables in Section 6, we report average test log-likelihoods in Table S2.
These results reﬂect the same ﬁndings as discussed in Section 6.
Comparison in computation complexity / runtime We compare the runtime of the three PACOH methods to MLAP (Amit and Meir, 2018) on the sinusoid environment ( n = 20 , m i = 5 ). Figure S6 reports the average runtime per metatraining iteration and the runtime for meta-test inference.
In that, meta-training is performed with full meta-batch size ( H = n = 20 ) and K = 5 samples SVI samples / SVGD particles. For MLAP we use 1000 gradient steps for meta-testing. The runtimes where recorded on an Intel Core i7-8550U CPU. Overall, the PACOH methods are 2-5 times faster than MLAP
[Table]
per meta-training step. Additionally, since MLAP jointly optimizes Q and Q 1 , ..., Q n in an interdependent manner, we observe that it needs ca. 2-3 times more iterations to converge than e.g. PACOH-VI.
The meta-test inference for PACOH is ca. 100 - 1000 times faster than for MLAP. This is due to the fact, that PACOH is able to use the closed-form GP solution of Q i whereas MLAP needs to gradient-descent on the single-task part of the PAC bound in (Amit and Meir, 2018). This once more stresses the practical relevance of our PAC-Bayesian meta-learning bounds that forego nested optimization, thus, allowing us to make use of closed for solutions for both the log partition function ln Z and the posteriors Q i .
[Figure]