Title: RECASTING GRADIENT-BASED META-LEARNING AS HIERARCHICAL BAYES

Abstract: Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. ( 2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm's operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.

1. INTRODUCTION:

A remarkable aspect of human intelligence is the ability to quickly solve a novel problem and to be able to do so even in the face of limited experience in a novel domain. Such fast adaptation is made possible by leveraging prior learning experience in order to improve the efficiency of later learning. This capacity for meta-learning also has the potential to enable an artificially intelligent agent to learn more efficiently in situations with little available data or limited computational resources (Schmidhuber, 1987; Bengio et al., 1991; Naik & Mammone, 1992) .

In machine learning, meta-learning is formulated as the extraction of domain-general information that can act as an inductive bias to improve learning efficiency in novel tasks (Caruana, 1998; Thrun & Pratt, 1998) . This inductive bias has been implemented in various ways: as learned hyperparameters in a hierarchical Bayesian model that regularize task-specific parameters (Heskes, 1998) , as a learned metric space in which to group neighbors (Bottou & Vapnik, 1992) , as a trained recurrent neural network that allows encoding and retrieval of episodic information (Santoro et al., 2016) , or as an optimization algorithm with learned parameters (Schmidhuber, 1987; Bengio et al., 1992) .

The model-agnostic meta-learning (MAML) of Finn et al. (2017) is an instance of a learned optimization procedure that directly optimizes the standard gradient descent rule. The algorithm estimates an initial parameter set to be shared among the task-specific models; the intuition is that gradient descent from the learned initialization provides a favorable inductive bias for fast adaptation. However, this inductive bias has been evaluated only empirically in prior work (Finn et al., 2017) .

In this work, we present a novel derivation of and a novel extension to MAML, illustrating that this algorithm can be understood as inference for the parameters of a prior distribution in a hierarchical Bayesian model. The learned prior allows for quick adaptation to unseen tasks on the basis of an implicit predictive density over task-specific parameters. The reinterpretation as hierarchical Bayes gives a principled statistical motivation for MAML as a meta-learning algorithm, and sheds light on the reasons for its favorable performance even among methods with significantly more parameters.

More importantly, by casting gradient-based meta-learning within a Bayesian framework, we are able to improve MAML by taking insights from Bayesian posterior estimation as novel augmentations to the gradient-based meta-learning procedure. We experimentally demonstrate that this enables better performance on a few-shot learning benchmark.

2. META-LEARNING FORMULATION:

The goal of a meta-learner is to extract task-general knowledge through the experience of solving a number of related tasks. By using this learned prior knowledge, the learner has the potential to quickly adapt to novel tasks even in the face of limited data or limited computation time.

Formally, we consider a dataset D that defines a distribution over a family of tasks T . These tasks share some common structure such that learning to solve a single task has the potential to aid in solving another. Each task T defines a distribution over data points x, which we assume in this work to consist of inputs and either regression targets or classification labels y in a supervised learning problem (although this assumption can be relaxed to include reinforcement learning problems; e.g., see Finn et al., 2017) . The objective of the meta-learner is to be able to minimize a task-specific performance metric associated with any given unseen task from the dataset given even only a small amount of data from the task; i.e., to be capable of fast adaptation to a novel task.

In the following subsections, we discuss two ways of formulating a solution to the meta-learning problem: gradient-based hyperparameter optimization and probabilistic inference in a hierarchical Bayesian model. These approaches were developed orthogonally, but, in Section 3.1, we draw a novel connection between the two.

2.1. META-LEARNING AS GRADIENT-BASED HYPERPARAMETER OPTIMIZATION:

A parametric meta-learner aims to find some shared parameters θ that make it easier to find the right task-specific parameters φ when faced with a novel task. A variety of meta-learners that employ gradient methods for task-specific fast adaptation have been proposed (e.g., Andrychowicz et al., 2016; Li & Malik, 2017a; b; Wichrowska et al., 2017) . MAML (Finn et al., 2017) is distinct in that it provides a gradient-based meta-learning procedure that employs a single additional parameter (the meta-learning rate) and operates on the same parameter space for both meta-learning and fast adaptation. These are necessary features for the equivalence we show in Section 3.1.

To address the meta-learning problem, MAML estimates the parameters θ of a set of models so that when one or a few batch gradient descent steps are taken from the initialization at θ given a small sample of task data x j 1 , . . . , x j N ∼ p T j (x) each model has good generalization performance on another sample x j N +1 , . . . , x j N +M ∼ p T j (x) from the same task. The MAML objective in a maximum likelihood setting is

EQUATION

where we use φ j to denote the updated parameters after taking a single batch gradient descent step from the initialization at θ with step size α on the negative log-likelihood associated with the task T j . Note that since φ j is an iterate of a gradient descent procedure that starts from θ, each φ j is of the same dimensionality as θ. We refer to the inner gradient descent procedure that computes φ j as fast adaptation. The computational graph of MAML is given in Figure 1 (left).

2.2. META-LEARNING AS HIERARCHICAL BAYESIAN INFERENCE:

An alternative way to formulate meta-learning is as a problem of probabilistic inference in the hierarchical model depicted in Figure 1 (right). In particular, in the case of meta-learning, each task-specific parameter φ j is distinct from but should influence the estimation of the parameters {φ j | j = j} from other tasks. We can capture this intuition by introducing a meta-level parameter θ on which each task-specific parameter is statistically dependent. With this formulation, the mutual dependence of the task-specific parameters φ j is realized only through their individual dependence

θ -log p( x j n | θ ) p T j (x) p D (T ) φ j -log p( x j N +m | φ j ) -log p( X | θ ) ∇ θ J N M θ x j n φ j N J Figure 1: (Left)

The computational graph of the MAML (Finn et al., 2017) algorithm covered in Section 2.1.

Straight arrows denote deterministic computations and crooked arrows denote sampling operations. (Right) The probabilistic graphical model for which MAML provides an inference procedure as described in Section 3.1. In each figure, plates denote repeated computations (left) or factorization (right) across independent and identically distributed samples.

on the meta-level parameters θ. As such, estimating θ provides a way to constrain the estimation of each of the φ j .

Given some data in a multi-task setting, we may estimate θ by integrating out the task-specific parameters to form the marginal likelihood of the data. Formally, grouping all of the data from each of the tasks as X and again denoting by x j 1 , . . . , x j N a sample from task T j , the marginal likelihood of the observed data is given by

p ( X | θ ) = j p x j 1 , . . . , x j N | φ j p φ j | θ dφ j .

(2)

Maximizing ( 2) as a function of θ gives a point estimate for θ, an instance of a method known as empirical Bayes (Bernardo & Smith, 2006; Gelman et al., 2014) due to its use of the data to estimate the parameters of the prior distribution.

Hierarchical Bayesian models have a long history of use in both transfer learning and domain adaptation (e.g., Lawrence & Platt, 2004; Yu et al., 2005; Gao et al., 2008; Daumé III, 2009; Wan et al., 2012) . However, the formulation of meta-learning as hierarchical Bayes does not automatically provide an inference procedure, and furthermore, there is no guarantee that inference is tractable for expressive models with many parameters such as deep neural networks.

3. LINKING GRADIENT-BASED META-LEARNING & HIERARCHICAL BAYES:

In this section, we connect the two independent approaches of Section 2.1 and Section 2.2 by showing that MAML can be understood as empirical Bayes in a hierarchical probabilistic model. Furthermore, we build on this understanding by showing that a choice of update rule for the taskspecific parameters φ j (i.e., a choice of inner-loop optimizer) corresponds to a choice of prior over task-specific parameters, p( φ j | θ ).

3.1. MODEL-AGNOSTIC META-LEARNING AS EMPIRICAL BAYES:

In general, when performing empirical Bayes, the marginalization over task-specific parameters φ j in (2) is not tractable to compute exactly. To avoid this issue, we can consider an approximation that makes use of a point estimate φj instead of performing the integration over φ in (2). Using φj as an estimator for each φ j , we may write the negative logarithm of the marginal likelihood as

EQUATION

Setting φj = θ + α ∇ θ log p( x j 1 , . . . , x j N | θ ) for each j in (3) recovers the unscaled form of the one-step MAML objective in (1). This tells us that the MAML objective is equivalent to a maximization with respect to the meta-level parameters θ of the marginal likelihood p( X | θ ), where a point estimate for each task-specific parameter φ j is computed via one or a few steps of gradient descent. By taking only a few steps from the initialization at θ, the point estimate φj trades off

Algorithm MAML-HB(D) Initialize θ randomly while not converged do Draw J samples T 1 , . . . , T J ∼ p D (T ) Estimate E x∼p T 1 (x) [-log p( x | θ )], . . . , E x∼p T J (x) [-log p( x | θ )] using ML-• • • Update θ ← θ -β ∇ θ j E x∼p T j (x) [-log p( x | θ )]

end Algorithm 2: Model-agnostic meta-learning as hierarchical Bayesian inference. The choices of the subroutine ML-• • • that we consider are defined in Subroutine 3 and Subroutine 4.

Subroutine ML-POINT(θ, T )

Draw N samples x 1 , . . . , x N ∼ p T (x) Initialize φ ← θ for k in 1, . . . , K do Update φ ← φ + α ∇ φ log p( x 1 , . . . , x N | φ ) end Draw M samples x N +1 , . . . , x N +M ∼ p T (x) return -log p( x N +1 , . . . , x N +M | φ )

Subroutine 3: Subroutine for computing a point estimate φ using truncated gradient descent to approximate the marginal negative log likelihood (NLL).

minimizing the fast adaptation objective -log p( x j 1 , . . . , x j N | θ ) with staying close in value to the parameter initialization θ.

We can formalize this trade-off by considering the linear regression case. Recall that the maximum a posteriori (MAP) estimate of φ j corresponds to the global mode of the posterior p( φ j | x j 1 , . . . x j N , θ ) ∝ p( x j 1 , . . . x j N | φ j )p( φ j | θ ). In the case of a linear model, early stopping of an iterative gradient descent procedure to estimate φ j is exactly equivalent to MAP estimation of φ j under the assumption of a prior that depends on the number of descent steps as well as the direction in which each step is taken. In particular, write the input examples as X and the vector of regression targets as y, omit the task index from φ, and consider the gradient descent update

EQUATION

for iteration index k and learning rate α ∈ R + . Santos (1996) shows that, starting from φ (0) = θ, φ (k) in (4) solves the regularized linear least squares problem

min y -Xφ 2 2 + θ -φ 2 Q (5)

with Q-norm defined by z Q = z T Q -1 z for a symmetric positive definite matrix Q that depends on the step size α and iteration index k as well as on the covariance structure of X. We describe the exact form of the dependence in Section 3.2. The minimization in (5) can be expressed as a posterior maximization problem given a conditional Gaussian likelihood over y and a Gaussian prior over φ.

The posterior takes the form

EQUATION

Since φ (k) in ( 4) maximizes ( 6), we may conclude that k iterations of gradient descent in a linear regression model with squared error exactly computes the MAP estimate of φ, given a Gaussian-noised observation model and a Gaussian prior over φ with parameters µ 0 = θ and Σ 0 = Q. Therefore, in the case of linear regression with squared error, MAML is exactly empirical Bayes using the MAP estimate as the point estimate of φ.

In the nonlinear case, MAML is again equivalent to an empirical Bayes procedure to maximize the marginal likelihood that uses a point estimate for φ computed by one or a few steps of gradient descent. However, this point estimate is not necessarily the global mode of a posterior. We can instead understand the point estimate given by truncated gradient descent as the value of the mode of an implicit posterior over φ resulting from an empirical loss interpreted as a negative log-likelihood, and regularization penalties and the early stopping procedure jointly acting as priors (for similar interpretations, see Sjöberg & Ljung, 1995; Bishop, 1995; Duvenaud et al., 2016) .

The exact equivalence between early stopping and a Gaussian prior on the weights in the linear case, as well as the implicit regularization to the parameter initialization the nonlinear case, tells us that every iterate of truncated gradient descent is a mode of an implicit posterior. In particular, we are not required to take the gradient descent procedure of fast adaptation that computes φ to convergence in order to establish a connection between MAML and hierarchical Bayes. MAML can therefore be understood to approximate an expectation of the marginal negative log likelihood (NLL) for each task T j as

E x∼p T j (x) [-log p ( x | θ )] ≈ 1 M m -log p x j N +m | φj using the point estimate φj = θ + α ∇ θ log p( x j n | θ ) for single-step fast adaptation.

The algorithm for MAML as probabilistic inference is given in Algorithm 2; Subroutine 3 computes each marginal NLL using the point estimate of φ as just described. Formulating MAML in this way, as probabilistic inference in a hierarchical Bayesian model, motivates the interpretation in Section 3.2 of using various meta-optimization algorithms to induce a prior over task-specific parameters.

3.2. THE PRIOR OVER TASK-SPECIFIC PARAMETERS:

From Section 3.1, we may conclude that early stopping during fast adaptation is equivalent to a specific choice of a prior over task-specific parameters, p( φ j | θ ). We can better understand the role of early stopping in defining the task-specific parameter prior in the case of a quadratic objective.

Omit the task index from φ and x, and consider a second-order approximation of the fast adaptation objective (φ) = -log p( x 1 . . . , x N | φ ) about a minimum φ * :

EQUATION

where the Hessian H = ∇ 2 φ (φ * ) is assumed to be positive definite so that ˜ is bounded below. Furthermore, consider using a curvature matrix B to precondition the gradient in gradient descent, giving the update

EQUATION

)

If B is diagonal, we can identify (8) as a Newton method with a diagonal approximation to the inverse Hessian; using the inverse Hessian evaluated at the point φ (k-1) recovers Newton's method itself.

On the other hand, meta-learning the matrix B matrix via gradient descent provides a method to incorporate task-general information into the covariance of the fast adaptation prior, p( φ | θ ). For instance, the meta-learned matrix B may encode correlations between parameters that dictates how such parameters are updated relative to each other.

Formally, taking k steps of gradient descent from φ (0) = θ using the update rule in (8) gives a φ (k) that solves

EQUATION

)

The minimization in (9) corresponds to taking a Gaussian prior p( φ | θ ) with mean θ and covariance Santos, 1996) where B is a diagonal matrix that results from a simultaneous diagonalization of H and B as O T HO = diag(λ 1 , . . . , λ n ) = Λ and Golub & Van Loan, 1983 ). If the true objective is indeed quadratic, then, assuming the data is centered, H is the unscaled covariance matrix of features, X T X.

Q for Q = OΛ -1 ((I -BΛ) -k -I)O T (

O T B -1 O = diag(b 1 , . . . , b n ) = B with b i , λ i ≥ 0 for i = 1, . . . , n (Theorem 8.7.1 in

4. IMPROVING MODEL-AGNOSTIC META-LEARNING:

Identifying MAML as a method for probabilistic inference in a hierarchical model allows us to develop novel improvements to the algorithm. In Section 4.1, we consider an approach from Bayesian parameter estimation to improve the MAML algorithm, and in Section 4.2, we discuss how to make this procedure computationally tractable for high-dimensional models.

4.1. LAPLACE'S METHOD OF INTEGRATION:

We have shown that the MAML algorithm is an empirical Bayes procedure that employs a point estimate for the mid-level, task-specific parameters in a hierarchical Bayesian model. However, the use of this point estimate may lead to an inaccurate point approximation of the integral in (2) if the posterior over the task-specific parameters, p( φ j | x j N +1 , . . . , x j N +M , θ ), is not sharply peaked at the value of the point estimate. The Laplace approximation (Laplace, 1986; MacKay, 1992b; a) is applicable in this case as it replaces a point estimate of an integral with the volume of a Gaussian centered at a mode of the integrand, thereby forming a local quadratic approximation.

We can make use of this approximation to incorporate uncertainty about the task-specific parameters into the MAML algorithm at fast adaptation time. In particular, suppose that each integrand in (2) has a mode φ * j at which it is locally well-approximated by a quadratic function. The Laplace approximation uses a second-order Taylor expansion of the negative log posterior in order to approximate each integral in the product in (2) as

p X j | φ j p φ j | θ dφ j ≈ p X j | φ * j p φ * j | θ det(H j /2π) -1 2 (10)

where H j is the Hessian matrix of second derivatives of the negative log posterior.

Classically, the Laplace approximation uses the MAP estimate for φ * j , although any mode can be used as an expansion site provided the integrand is well enough approximated there by a quadratic. We use the point estimate φj uncovered by fast adaptation, in which case the MAML objective in (1) becomes an appropriately scaled version of the approximate marginal likelihood

EQUATION

)

The term log p( φj | θ ) results from the implicit regularization imposed by early stopping during fast adaptation, as discussed in Section 3.1. The term 1 /2 log det(H j ), on the other hand, results from the Laplace approximation and can be interpreted as a form of regularization that penalizes model complexity.

4.2. USING CURVATURE INFORMATION TO IMPROVE MAML:

Using (11) as a training criterion for a neural network model is difficult due to the required computation of the determinant of the Hessian of the log posterior H j , which itself decomposes into a sum of the Hessian of the log likelihood and the Hessian of the log prior as

H j = ∇ 2 φ j -log p X j | φ j + ∇ 2 φ j -log p φ j | θ .

In our case of early stopping as regularization, the prior over task-specific parameters p( φ j | θ ) is implicit and thus no closed form is available for a general model. Although we may use the quadratic approximation derived in Section 3.2 to obtain an approximate Gaussian prior, this prior is not diagonal and does not, to our knowledge, have a convenient factorization. Therefore, in our experiments, we instead use a simple approximation in which the prior is approximated as a diagonal Gaussian with precision τ . We keep τ fixed, although this parameter may be cross-validated for improved performance.

Subroutine ML-LAPLACE(θ, T )

Draw N samples x 1 , . . . , x N ∼ p T (x) Initialize φ ← θ for k in 1, . . . , K do Update φ ← φ + α ∇ φ log p( x 1 , . . . , x N | φ ) end Draw M samples x N +1 , . . . , x N +M ∼ p T (x) Estimate quadratic curvature Ĥ return -log p( x N +1 , . . . , x N +M | φ ) + η log det( Ĥ)

Subroutine 4: Subroutine for computing a Laplace approximation of the marginal likelihood.

Similarly, the Hessian of the log likelihood is intractable to form exactly for all but the smallest models, and furthermore, is not guaranteed to be positive definite at all points, possibly rendering the Laplace approximation undefined. To combat this, we instead seek a curvature matrix Ĥ that approximates the quadratic curvature of a neural network objective function. Since it is well-known that the curvature associated with neural network objective functions is highly non-diagonal (e.g., Martens, 2016) , a further requirement is that the matrix have off-diagonal terms.

Due to the difficulties listed above, we turn to second order gradient descent methods, which precondition the gradient with an inverse curvature matrix at each iteration of descent. The Fisher information matrix (Fisher, 1925) has been extensively used as an approximation of curvature, giving rise to a method known as natural gradient descent (Amari, 1998) . A neural network with an appropriate choice of loss function is a probabilistic model and therefore defines a Fisher information matrix. Furthermore, the Fisher information matrix can be seen to define a convex quadratic approximation to the objective function of a probabilistic neural model (Pascanu & Bengio, 2014; Martens, 2014) . Importantly for our use case, the Fisher information matrix is positive definite by definition as well as non-diagonal.

However, the Fisher information matrix is still expensive to work with. Martens & Grosse (2015) developed Kronecker-factored approximate curvature (K-FAC), a scheme for approximating the curvature of the objective function of a neural network with a block-diagonal approximation to the Fisher information matrix. Each block corresponds to a unique layer in the network, and each block is further approximated as a Kronecker product (see Van Loan, 2000) of two much smaller matrices by assuming that the second-order statistics of the input activation and the back-propagated derivatives within a layer are independent. These two approximations ensure that the inverse of the Fisher information matrix can be computed efficiently for the natural gradient.

For the Laplace approximation, we are interested in the determinant of a curvature matrix instead of its inverse. However, we may also make use of the approximations to the Fisher information matrix from K-FAC as well as properties of the Kronecker product. In particular, we use the fact that the determinant of a Kronecker product is the product of the exponentiated determinants of each of the factors, and that the determinant of a block diagonal matrix is the product of the determinants of the blocks (Van Loan, 2000) . The determinants for each factor can be computed as efficiently as the inverses required by K-FAC, in O(d 3 ) time for a d-dimensional Kronecker factor.

We make use of the Laplace approximation and K-FAC to replace Subroutine 3, which computes the task-specific marginal NLLs using a point estimate for φ. We call this method the Lightweight Laplace Approximation for Meta-Adaptation (LLAMA), and give a replacement subroutine in Subroutine 4.

5. EXPERIMENTAL EVALUATION:

The goal of our experiments is to evaluate if we can use our probabilistic interpretation of MAML to generate samples from the distribution over adapted parameters, and futhermore, if our method can be applied to large-scale meta-learning problems such as miniImageNet. during fast adaptation. LLAMA requires the prior precision term τ as well as an additional parameter η ∈ R + that weights the regularization term log det Ĥ contributed by the Laplace approximation. We fix τ = 0.001 and selected η = 10 -6 via cross-validation; all other parameters are set to the values reported in Finn et al. (2017) .

We find that LLAMA is practical enough to be applied to this larger-scale problem. In particular, our TensorFlow implementation of LLAMA trains for 60,000 iterations on one TITAN Xp GPU in 9 hours, compared to 5 hours to train MAML. As shown in Table 1 , LLAMA achieves comparable performance to the state-of-the-art meta-learning method by Triantafillou et al. (2017) . While the gap between MAML and LLAMA is small, the improvement from the Laplace approximation suggests that a more accurate approximation to the marginalization over task-specific parameters will lead to further improvements.

6. RELATED WORK:

Meta-learning and few-shot learning have a long history in hierarchical Bayesian modeling (e.g., Tenenbaum, 1999; Fei-Fei et al., 2003; Lawrence & Platt, 2004; Yu et al., 2005; Gao et al., 2008; Daumé III, 2009; Wan et al., 2012) . A related subfield is that of transfer learning, which has used hierarchical Bayes extensively (e.g., Raina et al., 2006) . A variety of inference methods have been used in Bayesian models, including exact inference (Lake et al., 2011) , sampling methods (Salakhutdinov et al., 2012) , and variational methods (Edwards & Storkey, 2017) . While some prior works on hierarchical Bayesian models have proposed to handle basic image recognition tasks, the complexity of these tasks does not yet approach the kinds of complex image recognition problems that can be solved by discriminatively trained deep networks, such as the miniImageNet experiment in our evaluation (Mansinghka et al., 2013) .

Recently, the Omniglot benchmark Lake et al. ( 2016) has rekindled interest in the problem of learning from few examples. Modern methods accomplish few-shot learning either through the design of network architectures that ingest the few-shot training samples directly (e.g., Koch, 2015; Vinyals et al., 2016; Snell et al., 2017; Hariharan & Girshick, 2017; Triantafillou et al., 2017) , or formulating the problem as one of learning to learn, or meta-learning (e.g., Schmidhuber, 1987; Bengio et al., 1991; Schmidhuber, 1992; Bengio et al., 1992) . A variety of inference methods have been used in Bayesian models, including exact inference (Lake et al., 2011) , sampling methods (Salakhutdinov et al., 2013) , and variational methods (Edwards & Storkey, 2017) .

Our work bridges the gap between gradient-based meta-learning methods and hierarchical Bayesian modeling. Our contribution is not to formulate the meta-learning problem as a hierarchical Bayesian 1 Improved performance on miniImageNet has been reported by several works (Mishra et al., 2017; Munkhdalai & Yu, 2017; Sung et al., 2017) by making use of a model architecture with significantly more parameters than the methods in Table 1 . Since we do not explore variations in neural network architecture in this work, we omit such results from the table.

model, but instead to formulate a gradient-based meta-learner as hierarchical Bayesian inference, thus providing a way to efficiently perform posterior inference in a model-agnostic manner.

7. CONCLUSION:

We have shown that model-agnostic meta-learning (MAML) estimates the parameters of a prior in a hierarchical Bayesian model. By casting gradient-based meta-learning within a Bayesian framework, our analysis opens the door to novel improvements inspired by probabilistic machinery.

As a step in this direction, we propose an extension to MAML that employs a Laplace approximation to the posterior distribution over task-specific parameters. This technique provides a more accurate estimate of the integral that, in the original MAML algorithm, is approximated via a point estimate. We show how to estimate the quantity required by the Laplace approximation using Kroneckerfactored approximate curvature (K-FAC), a method recently proposed to approximate the quadratic curvature of a neural network objective for the purpose of a second-order gradient descent technique.

Our contribution illuminates the road to exploring further connections between gradient-based metalearning methods and hierarchical Bayesian modeling. For instance, in this work we assume that the predictive distribution over new data-points is narrow and well-approximated by a point estimate. We may instead employ methods that make use of the variance of the distribution over task-specific parameters in order to model the predictive density over examples from a novel task. Furthermore, it is known that the Laplace approximation is inaccurate in cases where the integral is highly skewed, or is not unimodal and thus is not amenable to approximation by a single Gaussian mode. This could be solved by using a finite mixture of Gaussians, which can approximate many density functions arbitrarily well (Sorenson & Alspach, 1971; Alspach & Sorenson, 1972) . The exploration of additional improvements such as this is an exciting line of future work.

:



Figures and Tables:

One-shot classification performance on the miniImageNet test set, with comparison methods ordered by one-shot performance. All results are averaged over 600 test episodes, and we report 95% confidence intervals. Results reported byRavi & Larochelle (2017). We report test accuracy for a comparable architecture.1 * * * We report test accuracy for models matching train and test "shot" and "way".

Table content: <table><tr><td>5-way acc. (%)</td></tr></table>

Peer review (iclr2018):
 MAML (Finn+ 2017) is recast as a hierarchical Bayesian learning procedure. In particular the inner (task) training is initially cast as point-wise max likelihood estimation, and then (sec4) improved upon by making use of the Laplace approximation. Experimental evidence of the relevance of the method is provided on a toy task involving a NIW prior of Gaussians, and the (benchmark) MiniImageNet task. Casting MAML as HB seems a good idea. The paper does a good job of explaining the connection, but I think the presentation could be clarified. The role of the task prior and how it emerges from early stopping (ie a finite number of gradient descent steps) (sec 3.2) is original and technically non-trivial, and is a contribution of this paper. The synthetic data experiment sec5.1 and fig5 is clearly explained and serves to additionally clarify the proposed method. Regarding the MiniImageNet experiments, I read the exchange on TCML and agree with the authors of the paper under review. However, I recommend including the references to Mukhdalai 2017 and Sung 2017 in the footnote on TCML to strengthen the point more generically, and show that not just TCML but other non-shallow architectures are not considered for comparison here. In addition, the point made by the TCML authors is fair ("nothing prevented you from...") and I would also recommend mentioning the reviewed paper*s authors* decision (not to test deeper architectures) in the footnote. This decision is in order but needs to be stated in order for the reader to form a balanced view of methods at her disposal. The experimental performance reported Table 1 remains small and largely within one standard deviation of competitor methods. I am assessing this paper as "7" because despite the merit of the paper, the relevance of the reformulation of MAML, and the technical steps involved in the reformulation, the paper does not eg address other forms (than L-MAML) of the task-specific subroutine ML-..., and the benchmark improvements are quite small. I think the approach is good and fruitful. # Suggestions on readability * I have the feeling the paper inverts from their use in Finn 2017 (step size for meta- vs task-training). This is unfortunate and will certainly confuse readers; I advise carefully changing this throughout the entire paper (eg Algo 2,3,4, eq 1, last eq in sec3.1, eq in text below eq3, etc) * I advise avoiding the use of the symbol f, which appears in only two places in Algo 2 and the end of sec 3.1. This is in part because f is given another meaning in Finn 2017, but also out of general parsimony in symbol use. (could leave the output of ML-... implicit by writing ML-...(.theta, T)_j in the ; if absolutely needed, use another symbol than f) * Maybe sec3 can be clarified in its structure by re-ordering points on the quadratic error function and early stopping (eg avoiding to split them between end of 3.1 and 3.2). * sec6 "Machine learning and deep learning": I would definitely avoid this formulation, seems to tail in with all the media nonsense on "what*s the difference between ML and DL ?". In addition the formulation seems to contrast ML with hierarchical Bayesian modeling, which does not make sense/ is wrong and confusing. # Typos * sec1 second parag: did you really mean "in the architecture or loss function"? unclear. * sec2: over a family * "common structure, so that" (not such that) * orthgonal * sec2.1 suggestion: clarify that .theta and .phi are in the same space * sec2.2 suggestion: task-specific parameter is distinct from ... parameters .phi_j$ (I would avoid writing just .phi altogether to distinguish in usage from .theta) * Gaussian-noised * approximation of the it objective * before eq9: "that solves": well, it doesn*t really "solve" the minimisation, in that it is not a minimum; reformulate this? * sec4.1 innaccurate * well approximated * sec4.2 an curvature * (Amari 1989) * For the the Laplace * O(n^3) : what is n ? * sec5.2 (Ravi and L 2017) * for the the