Original: G ENERALIZED I NNER L OOP M ETA -L EARNING
Revised: G ENERALIZED I NNER L OOP M ETA -L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Many (but not all) approaches self-qualifying as “meta-learning” in deep learning and reinforcement learning ﬁt a common pattern of approximating the solution to a nested optimization problem.
Revised: Many (but not all) approaches self-qualifying as “meta-learning” in deep learning and reinforcement learning ﬁt a common pattern of approximating the solution to a nested optimization problem.

Original: In this paper, we give a formalization of this shared pattern, which we call G IMLI , prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches.
Revised: In this paper, we give a formalization of this shared pattern, which we call G IMLI , prove its general requirements, and derive a general-purpose algorithm for implementing similar approaches.

Original: Based on this analysis and algorithm, we describe a library of our design, unnamedlib , which we share with the community to assist and enable future research into these kinds of meta-learning approaches.
Revised: Based on this analysis and algorithm, we describe a library of our design, unnamedlib , which we share with the community to assist and enable future research into these kinds of meta-learning approaches.

Original: We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate.
Revised: We end the paper by showcasing the practical applications of this framework and library through illustrative experiments and ablation studies which they facilitate.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Although it is by no means a new subﬁeld of machine learning research (see e.g. Schmidhuber, 1987; Bengio, 2000; Hochreiter et al., 2001), there has recently been a surge of interest in metalearning (e.g. Maclaurin et al., 2015; Andrychowicz et al., 2016;
Revised: Although it is by no means a new subﬁeld of machine learning research (see e.g. Schmidhuber, 1987; Bengio, 2000; Hochreiter et al., 2001), there has recently been a surge of interest in metalearning (e.g. Maclaurin et al., 2015; Andrychowicz et al., 2016;

Original: Finn et al., 2017).
Revised: Finn et al., 2017).

Original: This is due to the methods meta-learning provides, amongst other things, for producing models that perform well beyond the conﬁnes of a single task, outside the constraints of a static dataset, or simply with greater data efﬁciency or sample complexity.
Revised: This is due to the methods meta-learning provides, amongst other things, for producing models that perform well beyond the conﬁnes of a single task, outside the constraints of a static dataset, or simply with greater data efﬁciency or sample complexity.

Original: Due to the wealth of options in what could be considered “meta-” to a learning problem, the term itself may have been used with some degree of underspeciﬁcation.
Revised: Due to the wealth of options in what could be considered “meta-” to a learning problem, the term itself may have been used with some degree of underspeciﬁcation.

Original: However, it turns out that many meta-learning approaches, in particular in the recent literature, follow the pattern of optimizing the “meta-parameters” of the training process by nesting one or more inner loops in an outer training loop.
Revised: However, it turns out that many meta-learning approaches, in particular in the recent literature, follow the pattern of optimizing the “meta-parameters” of the training process by nesting one or more inner loops in an outer training loop.

Original: Such nesting enables training a model for several steps, evaluating it, calculating or approximating the gradients of that evaluation with respect to the meta-parameters, and subsequently updating these meta-parameters.
Revised: Such nesting enables training a model for several steps, evaluating it, calculating or approximating the gradients of that evaluation with respect to the meta-parameters, and subsequently updating these meta-parameters.

Original: This paper makes three contributions.
Revised: This paper makes three contributions.

Original: First, we propose a formalization of this general process, which we call G eneralized I nner Loop M eta- L earn i
Revised: First, we propose a formalization of this general process, which we call G eneralized I nner Loop M eta- L earn i

Original: ng (G IMLI ), and show that it subsumes several recent approaches.
Revised: ng (G IMLI ), and show that it subsumes several recent approaches.

Original: The proposed formalismprovides tooling for analysing the provable requirements of the meta-optimization process, and allows for describing the  process in general terms.
Revised: The proposed formalism allows us to describe the meta-optimization process in general terms and analyse its requirements.

Original: Second, we derive a general algorithm that supports the implementation of various kinds of metalearning ﬁtting within the G IMLI framework and its requirements.
Revised: Second, we derive a general algorithm that supports the implementation of various kinds of meta-learning ﬁtting within the G IMLI framework and its requirements.

Original: Third, based on this analysis and algorithm, we describe a lightweight PyTorch library that enables the straightforward implementation of any meta-learning approach that ﬁts within the G IMLI framework in canonical PyTorch, such that existing codebases require minimal changes, supporting third party module implementations and a variety of optimizers.
Revised: Third, based on this analysis and algorithm, we describe a lightweight PyTorch library that enables the straightforward implementation of any meta-learning approach that ﬁts within the G IMLI framework in canonical PyTorch, such that existing codebases require minimal changes, supporting third party module implementations and a variety of optimizers.

Original: Through a set of indicative experiments, we showcase the sort of research directions that are facilitated by our formalization and the corresponding library.
Revised: Through a set of indicative experiments, we showcase the sort of research directions that are facilitated by our formalization and the corresponding library.

Original: The overarching aim of this paper is not—emphatically—to purport some notion of ownership or precedence in any sense over existing efforts by virtue of having proposed a unifying formulation.
Revised: The overarching aim of this paper is not—emphatically—to purport some notion of ownership or precedence in any sense over existing efforts by virtue of having proposed a unifying formulation.

Original: Rather, in pointing out similarities under such uniﬁcation, it provides theoretical and practical tools for facilitating further research in this exciting domain.
Revised: Rather, in pointing out similarities under such uniﬁcation, it provides theoretical and practical tools for facilitating further research in this exciting domain.

Original:  G ENERALIZED I NNER L OOP M ETA -L EARNING
Revised:  G ENERALIZED I NNER L OOP M ETA -L EARNING

Original: Whereby “meta-learning” is taken to mean the process of “learning to learn”, we can describe it as a nested optimization problem according to which an outer loop optimizes meta-variables controlling the optimization of model parameters within an inner loop.
Revised: Whereby “meta-learning” is taken to mean the process of “learning to learn”, we can describe it as a nested optimization problem according to which an outer loop optimizes meta-variables controlling the optimization of model parameters within an inner loop.

Original: The aim of the outer loop should be to improve the meta-variables such that the inner loop produces models which are more suitable according to some criterion.
Revised: The aim of the outer loop should be to improve the meta-variables such that the inner loop produces models which are more suitable according to some criterion.

Original: In this section, we will formalize this process, which we call G eneralized
Revised: In this section, we will formalize this process, which we call G eneralized

Original: I nner Loop M eta- L earn i ng (G IMLI ).
Revised: I nner Loop M eta- L earn i ng (G IMLI ).

Original: We will deﬁne our terms in Section 2.1, give a formal description of the inner and outer loop’s respective optimization problems in Section 2.2 and 2.3, followed by a description and proof of the requirements under which the process can be run in Section 2.4.
Revised: We will deﬁne our terms in Section 2.1, give a formal description of the inner and outer loop’s respective optimization problems in Section 2.2 and 2.3, followed by a description and proof of the requirements under which the process can be run in Section 2.4.

Original: Finally, in Section 2.5 we describe an algorithm which permits an efﬁcient and exact implementation of G IMLI , agnostic to the model and optimizers used (subject to the requirements of Section 2. being satisﬁed).
Revised: Finally, in Section 2.5 we describe an algorithm which permits an efﬁcient and exact implementation of G IMLI , agnostic to the model and optimizers used.

Original: 2. D EFINITIONS
Revised: 2. D EFINITIONS

Original: Without loss of generality, let us assume we wish to train a model parameterized by θ .
Revised: Let us assume we wish to train a model parameterized by θ .

Original: Let ϕ describe some collection of possibly unrelated “meta-parameters” describing some aspect of the process by which we train our model.
Revised: Let ϕ describe some collection of possibly unrelated “meta-parameters” describing some aspect of the process by which we train our model.

Original: These could be, for example the learning rate or some other real-valued hyperparameters, task loss weights in multi-task learning, or perhaps the initialization of the model weights at the beginning of training.
Revised: These could be, for example the learning rate or some other real-valued hyperparameters, task loss weights in multi-task learning, or perhaps the initialization of the model weights at the beginning of training.

Original: For disambiguation, we will refer to the subset of ϕ describing optimization parameters by ϕ opt , and those meta-parameters which parameterize aspects of the training loss calculation at every step (e.g. the loss itself, the task mixture, regularization terms, etc) by ϕ loss , i.e. ϕ = ϕ opt ∪ ϕ loss .
Revised: For disambiguation, we will refer to the subset of ϕ describing optimization parameters by ϕ opt , and those meta-parameters which parameterize aspects of the training loss calculation at every step (e.g. the loss itself, the task mixture, regularization terms, etc) by ϕ loss , i.e. ϕ = ϕ opt ∪ ϕ loss .

Original: We will give several concrete examples in Section 3.
Revised: We will give several concrete examples in Section 3.

Original: Let L train be a function of θ and ϕ corresponding to the overall training objective we seek to minimize directly or indirectly (e.g. via an upper bound, a Monte Carlo estimate of an expectation, etc).
Revised: Let L train be a function of θ and ϕ corresponding to the overall training objective we seek to minimize directly or indirectly (e.g. via an upper bound, a Monte Carlo estimate of an expectation, etc).

Original: Other elements of the objective such as the dataset, and hyperparameters or other aspects of training not covered by ϕ , are assumed to be implicit and held ﬁxed in L train for notational simplicity.
Revised: Other elements of the objective such as the dataset, and hyperparameters or other aspects of training not covered by ϕ , are assumed to be implicit and held ﬁxed in L train for notational simplicity.

Original: With this, the process by which we train a model to obtain test-time parameters θ ∗ can be formalized as shown in Equation 1 1 .
Revised: With this, the process by which we train a model to obtain test-time parameters θ ∗ can be formalized as shown in Equation 1 1 .

Original: [Equation]
Revised: [Equation]

Original: We assume that this search will be performed using an iterative gradient-based method such as some form of gradient descent as formalized in Section 2.2, and thus that the derivative ∇ θ L train ( θ, ϕ ) exists, as an obvious requirement.
Revised: We assume that this search will be performed using an iterative gradient-based method such as some form of gradient descent as formalized in Section 2.2, and thus that the derivative ∇ θ L train ( θ, ϕ ) exists, as an obvious requirement.

Original: Furthermore,and without loss of generality, let L val be a function exclusively of θ (except the speciﬁc case where ϕ describes some aspect of the model, e.g. initialization, as is done in MAML (Finn et al., 2017)) describing some objective we wish to measure the post-training performance of the model against.
Revised: Furthermore, let L val be a function exclusively of θ (except the speciﬁc case where ϕ describes some aspect of the model, e.g. initialization, as is done in MAML (Finn et al., 2017)) describing some objective we wish to measure the post-training performance of the model against.

Original: This could be a validation set from the same task the model was trained on, from a collection of other tasks, or some other measure of the quality of the model parameterized by θ ∗ .
Revised: This could be a validation set from the same task the model was trained on, from a collection of other tasks, or some other measure of the quality of the model parameterized by θ ∗ .

Original: Typically, the aforementioned iterative process is not run to convergence, but rather to the point where an intermediate set of model parameters is considered satisfactory according to some criterion (e.g. best model under L val for a ﬁxed budget of training steps, or after no improvement is seen for a certain number of training steps, etc.), and these intermediate parameters will serve as θ ∗ .
Revised: Typically, the aforementioned iterative process is not run to convergence, but rather to the point where an intermediate set of model parameters is considered satisfactory according to some criterion (e.g. best model under L val for a ﬁxed budget of training steps, or after no improvement is seen for a certain number of training steps, etc.), and these intermediate parameters will serve as θ ∗ .

Original: 2. T RAINING
Revised: 2. T RAINING

Original: The process by which we typically estimate θ ∗ against L train decomposes into a sequence of updates of θ .
Revised: The process by which we typically approximate θ ∗ against L train decomposes into a sequence of updates of θ .

Original: At timestep t , we compute θ t +1 from θ t by ﬁrst computing a step-speciﬁc loss (cid:96) traint ( θ t , ϕ loss ) .
Revised: At timestep t , we compute θ t +1 from θ t by ﬁrst computing a step-speciﬁc loss (cid:96) traint ( θ t , ϕ loss ) .

Original: Note that this loss may itself also be a function of step-speciﬁc factors, such as the training data used for that timestep, which we leave implicit here for notational simplicity.
Revised: Note that this loss may itself also be a function of step-speciﬁc factors, such as the training data used for that timestep, which we leave implicit here for notational simplicity.

Original: Following this, we typically compute or estimate (e.g. as in reinforcement learning)
Revised: Following this, we typically compute or approximate (e.g. as in reinforcement learning)

Original: the gradients ∇ θ t (cid:96) traint ( θ t , ϕ loss ) of this loss with regard to θ t , by using algorithms such as backpropagation (Rumelhart et al., 1985).
Revised: the gradients ∇ θ t (cid:96) traint ( θ t , ϕ loss ) of this loss with regard to θ t , by using algorithms such as backpropagation (Rumelhart et al., 1985).

Original: We then typically use an optimizer to “apply” these gradients to the parameters θ t to obtain updated parameters θ t +1 .
Revised: We then typically use an optimizer to “apply” these gradients to the parameters θ t to obtain updated parameters θ t +1 .

Original: As such, optimization processes such as e.g. Adagrad (Duchi et al., 2011) may be stateful, in that they exploit gradient history to produce adaptive “local” learning rates.
Revised: As such, optimization processes such as e.g. Adagrad (Duchi et al., 2011) may be stateful, in that they exploit gradient history to produce adaptive “local” learning rates.

Original: Because [Equation] [Equation]
Revised: Because some aspects of the optimization process may be covered by our

Original: [Equation]
Revised: [Equation]

Original: some aspects of the optimization process may be covered by our choice of ϕ , we denote this optimization step at time-step t by opt t as shown in Equation 2, where timestep-speciﬁc attributes of the optimizer are left implicit by our use of the step subscript.
Revised: choice of ϕ , we denote this optimization step at time-step t by opt t as shown in Equation 2, where timestep-speciﬁc attributes of the optimizer are left implicit by our use of the step subscript.

Original: [Equation]
Revised: [Equation]

Original: For example, where we might use SGD as an optimizer, with the learning-rate being a meta-variable ϕ opt , we could instantiate equation 2 as follows:
Revised: For example, where we might use SGD as an optimizer, with the learning-rate being a meta-variable ϕ opt , we could instantiate equation 2 as follows:

Original: [Equation]
Revised: [Equation]

Revised: where G t = ∇ θ t (cid:96) traint ( θ t , ϕ loss )

Original: The estimation of θ ∗ from Equation 1 using T + 1 training updates according to Equation 2 yields a double recurrence in both θ t and G t (as it is a function of θ t , as outlined in Equation 3).
Revised: The estimation of θ ∗ from Equation 1 using T + 1 training updates according to Equation 2 yields a double recurrence in both θ t and G t (as it is a function of θ t , as outlined in Equation 3).

Original: [Equation]
Revised: [Equation]

Original: where P T = opt T − 1 ( θ T − 1 , ϕ opt , G T − 1 ) = opt T − 1 ( P T [Equation] 1 , ϕ opt , ∇ θ T 
Revised: where P T = opt T − 1 ( θ T − 1 , ϕ opt , G T − 1 ) = opt T − 1 ( P T − 1 , ϕ opt , ∇ θ T − 1 (cid:96) trainT − 1 ( P T − 1 , ϕ loss )) ... where P 1 = opt 0 ( θ 0 , ϕ opt , G

Original: [Equation]
Revised: [Equation]

Original: 1 (cid:96) trainT − 1 ( P T [Equation] [Equation] ϕ
Revised: = opt

Original: [Equation]
Revised: [Equation]

Original: where P 1 = opt 0 ( θ 0 , ϕ opt , G 0 )

Original: = opt 0 ( θ 0 , ϕ opt , ∇

Original: [Equation]
Revised: [Equation]

Original: 0 (cid:96)

Original: [Equation]
Revised: [Equation]

Original: From this we see that the optimization process used to train a variety of model types, with a variety of optimization methods, can be described as a function yielding test-time model parameters θ ∗ potentially as a function of parameter history θ 1 , . .
Revised: From this we see that the optimization process used to train a variety of model types, with a variety of optimization methods, can be described as a function yielding test-time model parameters θ ∗ potentially as a function of parameter history θ 1 , . .

Original: , θ T and of meta-parameters ϕ
Revised: , θ T and of meta-parameters ϕ

Original: (if any exist).
Revised: (if any exist).

Original: While this may seem like a fairly trivial formalization of a ubiquitous training process, we will see in Section 2.4 that if a few key requirements are met, this process can be nested as the inner loop within an outer loop containing—amongst other things—a meta-training process.
Revised: While this may seem like a fairly trivial formalization of a ubiquitous training process, we will see in Section 2.4 that if a few key requirements are met, this process can be nested as the inner loop within an outer loop containing—amongst other things—a meta-training process.

Original: From this process, described in Section 2.3, we estimate values of ϕ which improve our training process against some external metric.
Revised: From this process, described in Section 2.3, we estimate values of ϕ which improve our training process against some external metric.

Original: 2. M ETA -T RAINING
Revised: 2. M ETA -T RAINING

Original: We now describe the outer loop optimization problem which wraps around the inner loop described in Section 2.2.
Revised: We now describe the outer loop optimization problem which wraps around the inner loop described in Section 2.2.

Original: Through this process, we seek a value ϕ
Revised: Through this process, we seek a value ϕ

Original: ∗
Revised: ∗

Original: which ensures that the training process
Revised: which ensures that the training process

Original: L train ( θ, ϕ ∗ ) produces parameters θ ∗ which perform best against some metric L val ( θ ∗ ) which we care about.
Revised: L train ( θ, ϕ ∗ ) produces parameters θ ∗ which perform best against some metric L val ( θ ∗ ) which we care about.

Original: The formalization and decomposition of this “meta-training process” into nested optimization problems is shown in Equation 4.
Revised: The formalization and decomposition of this “meta-training process” into nested optimization problems is shown in Equation 4.

Original: [Equation]
Revised: [Equation]

Original: In this section, we introduce a formalization of an iterative process allowing us to approximate this nested optimization process.
Revised: In this section, we introduce a formalization of an iterative process allowing us to approximate this nested optimization process.

Original: Furthermore, we describe a general iterative algorithm by which the process in Equation 4 can be approximated by gradient-based methods while jointly estimating θ ∗ according to the process in Equation 1.
Revised: Furthermore, we describe a general iterative algorithm by which the process in Equation 4 can be approximated by gradient-based methods while jointly estimating θ ∗ according to the process in Equation 1.

Original: An iterative process by which we can estimate ϕ ∗ given Equation 4 following the sort of decomposition of the training process described Equation 1 into the training process described in Equations 2– 3 is described below.
Revised: An iterative process by which we can estimate ϕ ∗ given Equation 4 following the sort of decomposition of the training process described Equation 1 into the training process described in Equations 2– 3 is described below.

Original: Essentially, an estimate of θ ∗ is obtained following T + 1 steps training as outlined in Equation 3, which is then evaluated against L val .
Revised: Essentially, an estimate of θ ∗ is obtained following T + 1 steps training as outlined in Equation 3, which is then evaluated against L val .

Original: The gradient of this evaluation, ∇ ϕ L val ( θ ∗ ) is then obtained through backpropagation, and used to update ϕ .
Revised: The gradient of this evaluation, ∇ ϕ L val ( θ ∗ ) is then obtained through backpropagation, and used to update ϕ .

Original: Using τ as time-step counter to symbolize the time-scale being different from that used in the “inner loop”, we formalize this update in Equation 5 where metaopt τ denotes the optimization process used to update ϕ τ using M τ at that time-step.
Revised: Using τ as time-step counter to symbolize the time-scale being different from that used in the “inner loop”, we formalize this update in Equation 5 where metaopt τ denotes the optimization process used to update ϕ τ using M τ at that time-step.

Original: [Equation]
Revised: [Equation]

Original: = metaopt [Equation] ) where P T = opt T − 1 ( P T − 1 , ϕ opt , ∇ θ T − 1 (cid:96) trainT − 1 ( P T − 1 , ϕ loss )) ... where P 1 = opt 0 ( θ 0 , ϕ opt , G 0 )
Revised: τ ( ϕ τ , ∇ ϕ τ L val ( opt T ( P T , ϕ opt , G T ))) where P T = opt T − 1 ( P T − 1 , ϕ opt , ∇ θ T − 1 (cid:96) trainT − 1 ( P T − 1 , ϕ loss )) ... where P 1 = opt 0 ( θ 0 , ϕ opt , G 0 )

Original: 2. K EY R EQUIREMENTS
Revised: 2. K EY R EQUIREMENTS

Original: For gradient-based meta-learning as formalized in Section 2.3—in particular through the process formalized in Equation 5—to be possible, a few key requirements must be met.
Revised: For gradient-based meta-learning as formalized in Section 2.3—in particular through the process formalized in Equation 5—to be possible, a few key requirements must be met.

Original: We enumerate them here, and then discuss the conditions under which they are met, either analytically or through choice of model, loss function, or optimizer.
Revised: We enumerate them here, and then discuss the conditions under which they are met, either analytically or through choice of model, loss function, or optimizer.

Original: I. L val is a differentiable function of its input,
Revised: I. L val is a differentiable function of its input,

Original: i
Revised: i

Original: e. the derivative ∇ θ ∗ L val ( θ ∗ ) exists.
Revised: e. the derivative ∇ θ ∗ L val ( θ ∗ ) exists.

Original: II.
Revised: II.

Original: The optimization process opt in Equation 2 is a differentiable 2 function of θ and G  .
Revised: The optimization process opt in Equation 2 is a differentiable 2 function of θ and G 3 .

Original: III.
Revised: III.

Original: Either or both of the following conditionshold:
Revised: Either or both of the following conditions

Original:  (a) there exist continuous optimization hyperparameters (e
Revised: hold: (a) there exist continuous optimization hyperparameters (e

Original: g. a learning rate α ) covered by ϕ opt (e.g. α ⊆ ϕ opt ) and opt in Equation 2 is a differentiable function of ϕ opt , or (b) the gradient G t for one or more time-steps in Equations 1–3 is a function of ϕ loss (i.e. the derivative ∇ ϕ loss (cid:96) train ( θ, ϕ loss ) exists).
Revised: g. a learning rate α ) covered by ϕ opt (e.g. α ⊆ ϕ opt ) and opt in Equation 2 is a differentiable function of ϕ opt , or (b) the gradient G t for one or more time-steps in Equations 1–3 is a function of ϕ loss (i.e. the derivative ∇ ϕ loss (cid:96) train ( θ, ϕ loss ) exists).

Original: In the remainder of this section, we show that based on the assumptions outlined in Section 2.1, namely the existence of gradients on the validation objective L val with regard to model parameters θ and of gradients on the training objective L train with regard to both model parameters θ and metaparameters ϕ , there exist gradients on the validation objective L val with regard to ϕ .
Revised: In the remainder of this section, we show that based on the assumptions outlined in Section 2.1, namely the existence of gradients on the validation objective L val with regard to model parameters θ and of gradients on the training objective L train with regard to both model parameters θ and metaparameters ϕ , there exist gradients on the validation objective L val with regard to ϕ .

Original: We will then, in the next section, demonstrate how to implement this process by specifying an update algorithm for meta-variables.
Revised: We will then, in the next section, demonstrate how to implement this process by specifying an update algorithm for meta-variables.

Original: Implementing an iterative process as described in Equation 5 will exploit the chain rule for partial derivatives in order to run backpropagation.
Revised: Implementing an iterative process as described in Equation 5 will exploit the chain rule for partial derivatives in order to run backpropagation.

Original: The structure of the recurrence means we need to ensure that ∇ θ t L val ( θ ∗ ) exists for t ∈ { 0 , . .
Revised: The structure of the recurrence means we need to ensure that ∇ θ t L val ( θ ∗ ) exists for t ∈ { 0 , . .

Original: , T } in order to compute, for all such θ t , gradient paths ( ∇ θ t L val ( θ ∗ )) · ( ∇ ϕ θ t ) .
Revised: , T } in order to compute, for all such θ t , gradient paths ( ∇ θ t L val ( θ ∗ )) · ( ∇ ϕ θ t ) .

Original: We can prove this exists in virtue of the chain rule for partial derivatives and the requirements above:
Revised: We can prove this exists in virtue of the chain rule for partial derivatives and the requirements above:

Original: By the chain rule, ∇ θ t L val ( θ ∗ ) =
Revised: By the chain rule, ∇ θ t L val ( θ ∗ ) =

Original: ( ∇ θ ∗ L val ( θ ∗ )) · ( ∇ θ t θ ∗ ) exists if ∇ θ ∗ L val ( θ ∗ ) exists (and it does, by Requirement I) and ∇ θ t θ ∗ =
Revised: ( ∇ θ ∗ L val ( θ ∗ )) · ( ∇ θ t θ ∗ ) exists if ∇ θ ∗ L val ( θ ∗ ) exists (and it does, by Requirement I) and ∇ θ t θ ∗ =

Original: ∇ θ t θ T +1 exists.
Revised: ∇ θ t θ T +1 exists.

Original: Idem , ∇ θ t
Revised: Idem , ∇ θ t

Original: θ
Revised: θ

Original: T
Revised: T

Original: +1 =
Revised: +1 =

Original: ( ∇ θ t +1 θ T +1 ) · ( ∇ θ t θ t +1 ) exists by recursion over t if for all i ∈ [1 , T ] , ∇ θ i θ
Revised: ( ∇ θ t +1 θ T +1 ) · ( ∇ θ t θ t +1 ) exists by recursion over t if for all i ∈ [1 , T ] , ∇ θ i θ

Original: i +1 exists, which is precisely what Requirement II guarantees.
Revised: i +1 exists, which is precisely what Requirement II guarantees.

Original: Therefore ∇ θ t L val ( θ ∗ ) exists for t ∈ { 0 , . .
Revised: Therefore ∇ θ t L val ( θ ∗ ) exists for t ∈ { 0 , . .

Original: , T } , leaving us to demonstrate that ∇ ϕ θ t is deﬁned for all relevant values of t as a consequence of requirements I–III.
Revised: , T } , leaving us to demonstrate that ∇ ϕ θ t is deﬁned for all relevant values of t as a consequence of requirements I–III.

Original: We separately consider the case of ϕ loss and ϕ opt as deﬁned in Section 2.1:
Revised: We separately consider the case of ϕ loss and ϕ opt as deﬁned in Section 2.1:

Original: For ∇ ϕ opt θ t , the gradients trivially exist as a consequence of Requirement IIIa.
Revised: For ∇ ϕ opt θ t , the gradients trivially exist as a consequence of Requirement IIIa.

Original: For ∇ ϕ loss θ t , by the chain rule, ∇ ϕ loss θ t = ( ∇ G t − 1 θ t ) · ( ∇ ϕ loss G t − 1 ) .
Revised: For ∇ ϕ loss θ t , by the chain rule, ∇ ϕ loss θ t = ( ∇ G t − 1 θ t ) · ( ∇ ϕ loss G t − 1 ) .

Original: From Requirement II, it follows that ∇ G t − 1 θ t exists, and from Requirement IIIb, it follows that ∇ ϕ loss G t − 1 exists, therefore so does ∇ ϕ loss θ t .
Revised: From Requirement II, it follows that ∇ G t − 1 θ t exists, and from Requirement IIIb, it follows that ∇ ϕ loss G t − 1 exists, therefore so does ∇ ϕ loss θ t .

Original: Putting these both together, and having covered the union of ϕ opt and ϕ loss by exhaustion, we have shown that the gradients ∇ ϕ L val ( θ ∗ ) can be obtained by composition over the gradient paths ( ∇ θ t L val ( θ ∗ ))
Revised: Putting these both together, and having covered the union of ϕ opt and ϕ loss by exhaustion, we have shown that the gradients ∇ ϕ L val ( θ ∗ ) can be obtained by composition over the gradient paths ( ∇ θ t L val ( θ ∗ ))

Original: ( ∇ ϕ θ t ) for all t ∈ [1 , T ] .
Revised: ( ∇ ϕ θ t ) for all t ∈ [1 , T ] .

Original: In Section 2.5 we show how to implement the exact and efﬁcient calculation of ∇ ϕ L val ( θ ∗ ) .
Revised: In Section 2.5 we show how to implement the exact and efﬁcient calculation of ∇ ϕ L val ( θ ∗ ) .

Original: To complete this section, we indicate the conditions under which requirements I–III hold in practice.
Revised: To complete this section, we indicate the conditions under which requirements I–III hold in practice.

Original: Requirement I is simply a function of the choice of evaluation metric used to evaluate the model after training as part of L val .
Revised: Requirement I is simply a function of the choice of evaluation metric used to evaluate the model after training as part of L val .

Original: If this is not a differentiable function of θ ∗ , e.g. BLEU (Papineni et al., 2002) in machine translation, then a proxy metric can be selected for meta-training (e.g. negative log-likelihood of held out data), or gradient estimation methods such as REINFORCE (Williams, 1992) can be used.
Revised: If this is not a differentiable function of θ ∗ , e.g. BLEU (Papineni et al., 2002) in machine translation, then a proxy metric can be selected for meta training g log-likelihood of held out data), or gradient estimation methods such as REINFORCE (Williams, 1992) can be used.

Original: Requirement II is a function of the choice of optimizer, but is satisﬁed for most popular optimizers.
Revised: Requirement II is a function of the choice of optimizer, but is satisﬁed for most popular optimizers.

Original: We directly prove that this requirement holds for SGD (Robbins & Monro, 1951) and for A DAGRAD (Duchi et al., 2011) in Appendix A, and prove it by construction for a wider class of common optimizers in the implementation and tests of the software library described in Section 4.
Revised: We directly prove that this requirement holds for SGD (Robbins & Monro, 1951) and for A DAGRAD (Duchi et al., 2011) in Appendix A, and prove it by construction for a wider class of common optimizers in the implementation and tests of the software library described in Section 4.

Original: Requirement IIIa is a function of the choice of hyperparameters and optimizer, but is satisﬁed for at least the learning rate in most popular optimizers.
Revised: Requirement IIIa is a function of the choice of hyperparameters and optimizer, but is satisﬁed for at least the learning rate in most popular optimizers.

Original: Requirement IIIb is a function of the choice of loss function (cid:96) train (or class thereof), in that ∇ θ t (cid:96) traint ( θ t , ϕ loss ) needs to exist and be a differentiable function of ϕ .
Revised: Requirement IIIb is a function of the choice of loss function (cid:96) train (or class thereof), in that ∇ θ t (cid:96) traint ( θ t , ϕ loss ) needs to exist and be a differentiable function of ϕ .

Original: Usually, this requirement is held where ϕ is a multiplicative modiﬁer of θ t .
Revised: Usually, this requirement is held where ϕ is a multiplicative modiﬁer of θ t .

Original: For algorithms such as Model Agnostic Meta-Learning (Finn et al., 2017, MAML), this requirement is equivalent to saying that the Hessian of the loss function with regard to the parameters exists.
Revised: For algorithms such as Model Agnostic Meta-Learning (Finn et al., 2017, MAML), this requirement is equivalent to saying that the Hessian of the loss function with regard to the parameters exists.

Original: 2. T HE G IMLI UPDATE ALGORITHM
Revised: 2. T HE G IMLI UPDATE ALGORITHM

Original: In Algorithm 1, we present the algorithm which permits the computation of updates to metavariables through the nested iterative optimization process laid out above.
Revised: In Algorithm 1, we present the algorithm which permits the computation of updates to metavariables through the nested iterative optimization process laid out above.

Original: To clearly disentangle different gradient paths, we employ the mathematical ﬁction that is the “stop-gradient” operator, which is deﬁned as an operator which maintains the truth of the following expression:
Revised: To clearly disentangle different gradient paths, we employ the mathematical ﬁction that is the “stop-gradient” operator, which is deﬁned as an operator which maintains the truth of the following expression:

Original: [Equation]
Revised: [Equation]

Original: As will be shown below, this will allow us to decompose the computation of updates to the metavariables through an arbitrarily complex training process, agnostic to the models and optimizers used (subject to the requirements of Section 2.4 being satisﬁed), into a series of local updates passing gradient with regard to the loss back through the inner loop steps.
Revised: As will be shown below, this will allow us to decompose the computation of updates to the metavariables through an arbitrarily complex training process, agnostic to the models and optimizers used (subject to the requirements of Section 2.4 being satisﬁed), into a series of local updates passing gradient with regard to the loss back through the inner loop steps.

Original: This is akin to backpropagation through time (BPTT; Rumelhart et al., 1985), a method which has been adapted to other nested optimization processes with various constraints or restrictions (Andrychowicz et al., 2016; Franceschi et al., 2017; 2018).
Revised: This is akin to backpropagation through time (BPTT; Rumelhart et al., 1985), a method which has been adapted to other nested optimization processes with various constraints or restrictions (Andrychowicz et al., 2016; Franceschi et al., 2017; 2018).

Original: Require: Current model parameters θ t , meta-parameters ϕ
Revised: Require: Current model parameters θ t , meta-parameters ϕ

Original: τ
Revised: τ

Original: Require: Number of meta-parameter updates I , length of unrolled inner loop J 1: for i
Revised: Require: Number of meta-parameter updates I , length of unrolled inner loop J 1: for i

Original: = 

Revised: =

Original: to I − 1 do 2:
Revised: to I − 1 do 2:

Original: Segment meta-parameters ϕ opti , ϕ lossi ← split ( ϕ τ + i ) 3:
Revised: Segment meta-parameters ϕ opti , ϕ lossi ← split ( ϕ τ + i ) 3:

Original: Copy model state θ (cid:48) 

Revised: Copy model state θ (cid:48)

Original: ← θ t , [Table] state opt (cid:48) 

Revised: ← θ t , optimizer state opt (cid:48)

Original: ← opt t 4: for j = 0 to J − 1 do 5:
Revised: ← opt t 4: for j = 0 to J − 1 do 5:

Original: Compute inner gradient G j ← ∇ θ (cid:48)
Revised: Compute inner gradient G j ← ∇ θ (cid:48)

Original: j (cid:96) traint + j ( θ (cid:48) j , ϕ lossi ) and retain gradient graph state 6:
Revised: j (cid:96) traint + j ( θ (cid:48) j , ϕ lossi ) and retain gradient graph state 6:

Original: Update inner model θ (cid:48)
Revised: Update inner model θ (cid:48)

Original: j + [Table] ← opt (cid:48)
Revised: j +1 ← opt (cid:48)

Original: j ( θ (cid:48)
Revised: j ( θ (cid:48)

Original: j , ϕ opti , G j ) 7: end for 8:
Revised: j , ϕ opti , G j ) 7: end for 8:

Original: Initialize accumulators:
Revised: Initialize accumulators:

Original: A opti ← zerosLike ( ϕ opti ) ; A lossi ← zerosLike ( ϕ lossi ) 9:
Revised: A opti ← zerosLike ( ϕ opti ) ; A lossi ← zerosLike ( ϕ lossi ) 9:

Original: Compute B J ← ∇ θ (cid:48)
Revised: Compute B J ← ∇ θ (cid:48)

Original: J L val ( θ (cid:48) J ) 10: for j (cid:48)
Revised: J L val ( θ (cid:48) J ) 10: for j (cid:48)

Original: =
Revised: =

Original: J − 1 to 0 do 11:
Revised: J − 1 to 0 do 11:

Original: Compute optimizer-gradient derivative O j (cid:48) ← ∇ G j (cid:48) opt (cid:48)
Revised: Compute optimizer-gradient derivative O j (cid:48) ← ∇ G j (cid:48) opt (cid:48)

Original: j (cid:48)
Revised: j (cid:48)

Original: ( θ (cid:48)
Revised: ( θ (cid:48)

Original: j (cid:48) , ϕ opti , G j (cid:48) ) 12:
Revised: j (cid:48) , ϕ opti , G j (cid:48) ) 12:

Original: Update A opti ← A opti + B j (cid:48) 
Revised: Update A opti ← A opti + B j (cid:48) +1 ·

Original: +1 · ( ∇ ϕ opti opt(cid:48)
Revised: [Table] ∇ [Table] opt

Original: j (cid:48) ( (cid:74) θ (cid:48)
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: for
Revised: opti

Original: [Table]
Revised: [Table]

Revised: Update A lossi ← A lossi + B j

Original: [Table] 
Revised: [Table] + [Table]

Revised: · O j

Original: [Table]
Revised: [Table]

Original: for
Revised: j [Table] [Table] lossi

Original: [Table]
Revised: [Table]

Original: Updated
Revised: Compute B [Table] 15: end for 16: Update meta

Original: [Table]
Revised: [Table]

Original: parameters
Revised: parameters

Original: [Table]
Revised: [Table]

Revised: 17: end for 18: return Updated meta-parameters

Revised: [Table]

Original:  Each iteration through the loop deﬁned by lines 1–17 does one gradient-based update of metaparameters ϕ using the optimizer employed in line 16.
Revised: I Each iteration through the loop deﬁned by lines 1–17 does one gradient-based update of metaparameters ϕ using the optimizer employed in line 16.

Original: Each such iteration, we ﬁrst (line 3) copy the model and optimizer state (generally updated through some outer loop within which this update loop sits).
Revised: Each such iteration, we ﬁrst (line 3) copy the model and optimizer state (generally updated through some outer loop within which this update loop sits).

Original: We then (lines 4–7) compute a series of J updates on a copy of the model, preserving the intermediate gradient computations G 0 , . . .
Revised: We then (lines 4–7) compute a series of J updates on a copy of the model, preserving the intermediate gradient computations G 0 , . . .

Original: , G J − 1 , intermediate model parameters θ (cid:48) 0 , . . .
Revised: , G J − 1 , intermediate model parameters θ (cid:48) 0 , . . .

Original: , θ (cid:48)
Revised: , θ (cid:48)

Original: J (sometimes confusingly referred to as “fast weights”, following Hinton & Plaut (1987), within the meta-learning literature), and all associated activations.
Revised: J (sometimes confusingly referred to as “fast weights”, following Hinton & Plaut (1987), within the meta-learning literature), and all associated activations.

Original: These will be reused in the second stage (lines 10–15) to backpropagate higher-order gradients of the meta-loss, computed on linethrough the optimization process that was run in lines 4–7.
Revised: These will be reused in the second stage (lines 10–15) to backpropagate higher-order gradients of the meta-loss, computed on linethrough the optimization process that was run in lines 4–7.

Original: In particular, in lines 12 and 13, local (i.e. time step-speciﬁc) gradient calculations compute part of the gradient of ∇ ϕ opti L val ( θ (cid:48) J ) and ∇ ϕ lossi L val ( θ (cid:48) J ) , which is stored in accumulators which contain the exact respective gradients by the end of loop.
Revised: In particular, in lines 12 and 13, local (i.e. time step-speciﬁc) gradient calculations compute part of the gradient of ∇ ϕ opti L val ( θ (cid:48) J ) and ∇ ϕ lossi L val ( θ (cid:48) J ) , which is stored in accumulators which contain the exact respective gradients by the end of loop.

Original: What permits this efﬁcient local computation is the dynamic programming calculation of the partial derivative B j (cid:48)
Revised: What permits this efﬁcient local computation is the dynamic programming calculation of the partial derivative B j (cid:48)

Original: =
Revised: =

Original: ∇ θ (cid:48)
Revised: ∇ θ (cid:48)

Original: j (cid:48) as function of only B j (cid:48)
Revised: j (cid:48) as function of only B j (cid:48)

Original: +1 and timestep-speciﬁc gradients, implementing a second-order variant of BPTT.
Revised: +1 and timestep-speciﬁc gradients, implementing a second-order variant of BPTT through reverse-mode differentiation.

Original:  E XAMPLES AND R ELATED W ORK
Revised:  E XAMPLES AND R ELATED W ORK

Original: In this section, we highlight some instances of meta-learning which are instances of G IMLI , before discussing related approaches involving support for nested optimization, with applications to similar problems.
Revised: In this section, we highlight some instances of meta-learning which are instances of G IMLI , before discussing related approaches involving support for nested optimization, with applications to similar problems.

Original: The aim is not to provide a comprehensive literature review, which space would not permit.
Revised: The aim is not to provide a comprehensive literature review, which space would not permit.

Original: Rather, in pointing out similarity under our G IMLI formulation, we aim to showcase that rich and diverse research has been done using this class of approaches, where yet more progress indubitably remains to be made.
Revised: Rather, in pointing out similarity under our G IMLI formulation, we aim to showcase that rich and diverse research has been done using this class of approaches, where yet more progress indubitably remains to be made.

Original: This is, we believe, the strongest motivation for the development of libraries such as the one we present in Section 4 to support the implementation of algorithms that fall under the general algorithm derived in Section 2.
Revised: This is, we believe, the strongest motivation for the development of libraries such as the one we present in Section 4 to support the implementation of algorithms that fall under the general algorithm derived in Section 2.

Original: 3. E XAMPLES
Revised: 3. E XAMPLES

Original: Many of the papers referenced below contain excellent and thorough reviews of the literature most related to the type of meta-learning they approach.
Revised: Many of the papers referenced below contain excellent and thorough reviews of the literature most related to the type of meta-learning they approach.

Original: In the interest of brevity, we will not attempt such a review here, but rather focus on giving examples of a few forms of meta-learning that ﬁt the G IMLI framework (and thus are supported by the library presented in Section 4), and brieﬂy explain why.
Revised: In the interest of brevity, we will not attempt such a review here, but rather focus on giving examples of a few forms of meta-learning that ﬁt the G IMLI framework (and thus are supported by the library presented in Section 4), and brieﬂy explain why.

Original: One popular meta-learning problem is that of learning to optimize hyperparameters through gradient-based methods (Bengio, 2000; Maclaurin et al., 2015;
Revised: One popular meta-learning problem is that of learning to optimize hyperparameters through gradient-based methods (Bengio, 2000; Maclaurin et al., 2015;

Original: Luketina et al., 2016; Franceschi et al., 2017), as an alternative to grid/random search (Bergstra & Bengio, 2012) or Bayesian Optimization (Mockus et al., 1978; Pelikan et al., 1999; Bergstra et al., 2011; Snoek et al., 2012).
Revised: Luketina et al., 2016; Franceschi et al., 2017), as an alternative to grid/random search (Bergstra & Bengio, 2012) or Bayesian Optimization (Mockus et al., 1978; Pelikan et al., 1999; Bergstra et al., 2011; Snoek et al., 2012).

Original: Here, select continuous-valued hyperparameters are meta-optimized against a meta-objective, subject to the differentiability of the optimization step, and, where relevant, the loss function.
Revised: Here, select continuous-valued hyperparameters are meta-optimized against a meta-objective, subject to the differentiability of the optimization step, and, where relevant, the loss function.

Original: This corresponds to Requirements II and IIIa of Section 2.4 being met, i.e. G IMLI being run with select optimizer hyperparameters as part of ϕ opt .
Revised: This corresponds to Requirements II and IIIa of Section 2.4 being met, i.e. G IMLI being run with select optimizer hyperparameters as part of ϕ opt .

Revised: To give a simple concrete example, in the approaches of Bengio (2000) and Maclaurin et al. (2015), the only meta-variable is the learning rate α , i.e. ϕ =

Revised: ϕ opt =

Original:  A related problem is that of learning the optimizer wholesale as a parametric model (Hochreiter et al., 2001;
Revised: α . A related problem is that of learning the optimizer wholesale as a parametric model (Hochreiter et al., 2001;

Original: Andrychowicz et al., 2016; Duan et al., 2016), typically based on recurrent architectures.
Revised: Andrychowicz et al., 2016; Duan et al., 2016), typically based on recurrent architectures.

Original: Again, here the optimizer’s own parameters are the optimizer hyperparameters, and constitute the entirety of ϕ opt as used within G IMLI .
Revised: Again, here the optimizer’s own parameters are the optimizer hyperparameters, and constitute the entirety of ϕ opt as used within G IMLI .

Original: Requirements II and IIIa are trivially met through the precondition that such optimizers models have parameters with regard to which their output (and losses that are a function thereof) is differentiable.
Revised: Requirements II and IIIa are trivially met through the precondition that such optimizers models have parameters with regard to which their output (and losses that are a function thereof) is differentiable.

Revised: As a concrete example, in the work of Andrychowicz et al. (2016), an RNN with parameters φ is meta-learned, and models the updates made to parameters during training.

Revised: In our formalism, this would correspond to setting as meta-variable the parameters of this update network, i.e. ϕ opt = φ .

Original: More recently, meta-learning approaches such as MAML (Finn et al., 2017; Antoniou et al., 2018) and its variants/extensions have sought to use higher-order gradients to meta-learn model/policy initializations in few-shot learning settings.
Revised: More recently, meta-learning approaches such as MAML (Finn et al., 2017; Antoniou et al., 2018) and its variants/extensions have sought to use higher-order gradients to meta-learn model/policy initializations in few-shot learning settings.

Original: In G IMLI , this corresponds to setting θ 0 = ϕ loss , which then is not an explicit function of (cid:96) train in Equation 3, but rather is implicitly its argument through updates to the inner model over the unrolled optimization.
Revised: In G IMLI , this corresponds to setting θ 0 = ϕ loss , which then is not an explicit function of (cid:96) train in Equation 3, but rather is implicitly its argument through updates to the inner model over the unrolled optimization.

Original: All requirements in Section 2.4 must be satisﬁed (save IIIa, with Requirement IIIb further entailing that (cid:96) train be deﬁned such that the second derivative of the function with regard to θ exists (i.e. is non-zero).
Revised: All requirements in Section 2.4 must be satisﬁed (save IIIa, with Requirement IIIb further entailing that (cid:96) train be deﬁned such that the second derivative of the function with regard to θ exists (i.e. is non-zero).

Original: Finally, recent work by Chebotar et al.
Revised: Finally, recent work by Chebotar et al.

Original: (2019) has introduced the ML 3 framework for learning unconstrained loss functions as parametric models, through exploiting second-order gradients of a meta-loss with regard to the parameters of the inner loss.
Revised: (2019) has introduced the ML 3 framework for learning unconstrained loss functions as parametric models, through exploiting second-order gradients of a meta-loss with regard to the parameters of the inner loss.

Original: This corresponds, in G IMLI , to learning a parametric model of the loss parameterized by ϕ loss .
Revised: This corresponds, in G IMLI , to learning a parametric model of the loss parameterized by ϕ loss .

Original: 3. R ELATED W ORK
Revised: 3. R ELATED W ORK

Original: In a sense, many if not all of the approaches discussed in Section 3.1 qualify as “related work”, but here we will brieﬂy discuss approaches to the general problem of formalizing and supporting implementations of problems that ﬁt within the nested optimization speciﬁed by G IMLI .
Revised: In a sense, many if not all of the approaches discussed in Section 3.1 qualify as “related work”, but here we will brieﬂy discuss approaches to the general problem of formalizing and supporting implementations of problems that ﬁt within the nested optimization speciﬁed by G IMLI .

Original: The ﬁrst is work by Franceschi et al.
Revised: The ﬁrst is work by Franceschi et al.

Original: (2018) which describes how several meta-learning and hyperparameter optimization approaches can be cast as a bi-level optimization process, akin to our own formalization in 2.3.
Revised: (2018) which describes how several meta-learning and hyperparameter optimization approaches can be cast as a bi-level optimization process, akin to our own formalization in 2.3.

Original: This fascinating and relevant work is highly complementary to the formalization and discussion presented in our paper.
Revised: This fascinating and relevant work is highly complementary to the formalization and discussion presented in our paper.

Original: Whereas we focus on the requirements according to which gradient-based solutions to approaches based on nested optimization problems can be found in order to drive the development of a library which permits such approaches to be easily and scalably implemented, their work focuses on analysis of the conditions under which exact gradient-based solutions to bi-level optimization processes can be approximated, and what convergence guarantees exist for such guarantees.
Revised: Whereas we focus on the requirements according to which gradient-based solutions to approaches based on nested optimization problems can be found in order to drive the development of a library which permits such approaches to be easily and scalably implemented, their work focuses on analysis of the conditions under which exact gradient-based solutions to bi-level optimization processes can be approximated, and what convergence guarantees exist for such guarantees.

Original: In this sense, this is more relevant to those who wish to analyze and extend alternatives to ﬁrst-order approximations of algorithms such as MAML, e.g. see work of Nichol &
Revised: In this sense, this is more relevant to those who wish to analyze and extend alternatives to ﬁrst-order approximations of algorithms such as MAML, e.g. see work of Nichol &

Original: Schulman (2018) or Rajeswaran et al.
Revised: Schulman (2018) or Rajeswaran et al.

Original: On the software front, the library learn2learn (Arnold et al., 2019)3 addresses similar problems to that which we will present in Section 4.
Revised: On the software front, the library learn2learn (Arnold et al., 2019) addresses similar problems to that which we will present in Section 4.

Original: This library focuses primarily on providing implementations of existing meta-learning algorithms and their training loops that can be extended with new models.
Revised: This library focuses primarily on providing implementations of existing meta-learning algorithms and their training loops that can be extended with new models.

Original: In contrast, the library we present in Section 4 is “closer to the metal”, aiming to support the development of new meta-learning algorithms ﬁtting the G IMLI deﬁnitions with as little resort to non-canonical PyTorch as possible.
Revised: In contrast, the library we present in Section 4 is “closer to the metal”, aiming to support the development of new meta-learning algorithms ﬁtting the G IMLI deﬁnitions with as little resort to non-canonical PyTorch as possible.

Original: A recent parallel effort, Torchmeta (Deleu et al., 2019) also provides a library aiming to assist the implementation of meta-learning algorithms, supplying useful data-loaders for meta-training.
Revised: A recent parallel effort, Torchmeta (Deleu et al., 2019) also provides a library aiming to assist the implementation of meta-learning algorithms, supplying useful data-loaders for meta-training.

Original: However, unlike our approach described in 4, it requires re-implementation of models using their functional/stateless building blocks, and for users to reimplement the optimizers in a differentiable manner.
Revised: However, unlike our approach described in 4, it requires re-implementation of models using their functional/stateless building blocks, and for users to reimplement the optimizers in a differentiable manner.

Original:  T HE unnamedlib LIBRARY
Revised:  T HE unnamedlib LIBRARY

Original: In this section, we provide a high-level description of the design and capabilities of unnamedlib ,a PyTorch (Paszke et al., 2017) library aimed at enabling implementations of G IMLI with as little reliance on non-vanilla PyTorch as possible.
Revised: In this section, we provide a high-level description of the design and capabilities of unnamedlib ,a PyTorch (Paszke et al., 2017) library aimed at enabling implementations of G IMLI with as little reliance on non-vanilla PyTorch as possible.

Original: In this section, we ﬁrst discuss the obstacles that would prevent us from implementing this in popular deep learning frameworks, how we overcame these in
Revised: In this section, we ﬁrst discuss the obstacles that would prevent us from implementing this in popular deep learning frameworks, how we overcame these in

Original: PyTorch to implement G IMLI .
Revised: PyTorch to implement G IMLI .

Original: Additional features, helper functions, and other considerations when using/extending the library are provided in its documentation.
Revised: Additional features, helper functions, and other considerations when using/extending the library are provided in its documentation.

Original: 4. O BSTACLES
Revised: 4. O BSTACLES

Original: Many deep learning frameworks offer the technical functionality required to implement G IMLI , namely the ability to take gradients of gradients.
Revised: Many deep learning frameworks offer the technical functionality required to implement G IMLI , namely the ability to take gradients of gradients.

Original: However, there are two aspects of how we implement and train parametric models in such frameworks which inhibit our ability to ﬂexibly implement
Revised: However, there are two aspects of how we implement and train parametric models in such frameworks which inhibit our ability to ﬂexibly implement

Original: Algorithm 1.
Revised: Algorithm 1.

Original: The ﬁrst obstacle is that models are typically implemented statefully (e.g. torch.nn in PyTorch, keras.layers in Keras (Chollet et al., 2015), etc.), meaning that the model’s parameters are encapsulated in the model implementation, and are implicitly relied upon during the forward pass.
Revised: The ﬁrst obstacle is that models are typically implemented statefully (e.g. torch.nn in PyTorch, keras.layers in Keras (Chollet et al., 2015), etc.), meaning that the model’s parameters are encapsulated in the model implementation, and are implicitly relied upon during the forward pass.

Original: Therefore while such models can be considered as functions theoretically, they are not pure functions practically, as their output is not uniquely determined by their explicit input, and equivalently the parameters for a particular forward pass typically cannot be trivially overridden or supplied at call time.
Revised: Therefore while such models can be considered as functions theoretically, they are not pure functions practically, as their output is not uniquely determined by their explicit input, and equivalently the parameters for a particular forward pass typically cannot be trivially overridden or supplied at call time.

Original: This prevents us from tracking and backpropagating over the successive values of the model parameters θ within the inner loop described by Equation 3, through an implicit or explicit graph.
Revised: This prevents us from tracking and backpropagating over the successive values of the model parameters θ within the inner loop described by Equation 3, through an implicit or explicit graph.

Original: The second issue is that, as discussed at the end of Section 2.4 and in Appendix A, even though the operations used within popular optimizers are mathematically differentiable functions of the parameters, gradients, and select hyperparameters, these operations are not tracked in various framework’s implicit or explicit graph/gradient tape implementations when an optimization step is run.
Revised: The second issue is that, as discussed at the end of Section 2.4 and in Appendix A, even though the operations used within popular optimizers are mathematically differentiable functions of the parameters, gradients, and select hyperparameters, these operations are not tracked in various framework’s implicit or explicit graph/gradient tape implementations when an optimization step is run.

Original: This is with good reason: updating model parameters in-place is memory efﬁcient, as typically there is no need to keep references to the previous version of parameters.
Revised: This is with good reason: updating model parameters in-place is memory efﬁcient, as typically there is no need to keep references to the previous version of parameters.

Original: Ignoring the gradient dependency formed by allowing backpropagation through an optimization step essentially makes it safe to release memory allocated to historical parameters and intermediate model states once an update has been completed.
Revised: Ignoring the gradient dependency formed by allowing backpropagation through an optimization step essentially makes it safe to release memory allocated to historical parameters and intermediate model states once an update has been completed.

Original: Together, these obstacles essentially prevent us from practically satisfying Requirement II of Section 2.4.
Revised: Together, these obstacles essentially prevent us from practically satisfying Requirement II of Section 2.4.

Original: 4. M AKING STATEFUL MODULES STATELESS
Revised: 4. M AKING STATEFUL MODULES STATELESS

Original: As we wish to track and backpropagate through intermediate states of parameters during the inner loop, we keep a record of such states which can be referenced during the backward pass stage of the outer loop in Algorithm 1.
Revised: As we wish to track and backpropagate through intermediate states of parameters during the inner loop, we keep a record of such states which can be referenced during the backward pass stage of the outer loop in Algorithm 1.

Original: The typical way this is done in implementations of meta-learning algorithms such as MAML is to rewrite a “stateless” version of the inner loop’s model, permitting the use, in each invocation of the model’s forward pass, of weights which are otherwise tracked on the gradient graph/tape.
Revised: The typical way this is done in implementations of meta-learning algorithms such as MAML is to rewrite a “stateless” version of the inner loop’s model, permitting the use, in each invocation of the model’s forward pass, of weights which are otherwise tracked on the gradient graph/tape.

Original: While this addresses the issue, it is an onerous and limiting solution, as exploring new models within such algorithms invariably requires their reimplementation in a stateless style.
Revised: While this addresses the issue, it is an onerous and limiting solution, as exploring new models within such algorithms invariably requires their reimplementation in a stateless style.

Original: This typically prevents the researcher from experimenting with third-party codebases, complicated models, or those which requiring loading pre-trained weights, without addressing a signiﬁcant and unwelcome engineering challenge.
Revised: This typically prevents the researcher from experimenting with third-party codebases, complicated models, or those which requiring loading pre-trained weights, without addressing a signiﬁcant and unwelcome engineering challenge.

Original: A more generic solution, permitting the use of existing stateful modules (including with pre-loaded activations), agnostic to the complexity or origin of the code which deﬁnes them, is to modify the run-time instance of the model’s parent class to render them effectively function, a technique often referred to as “monkey-patching”.
Revised: A more generic solution, permitting the use of existing stateful modules (including with pre-loaded activations), agnostic to the complexity or origin of the code which deﬁnes them, is to modify the run-time instance of the model’s parent class to render them effectively function, a technique often referred to as “monkey-patching”.

Original: The high-level function unnamedlib.monkeypatch() does this by taking as argument a torch.nn.
Revised: The high-level function unnamedlib.monkeypatch() does this by taking as argument a torch.nn.

Original: Module instance and the structure of its nested sub-modules.
Revised: Module instance and the structure of its nested sub-modules.

Original: As it traverses this structure, it clones the parent classes of submodule instances, leaving their functionality intact save for that of the forward method which implements the forward pass.
Revised: As it traverses this structure, it clones the parent classes of submodule instances, leaving their functionality intact save for that of the forward method which implements the forward pass.

Original: Here, it replaces the call to the forward method with one which ﬁrst replaces the stateful parameters of the submodule with ones provided as additional arguments to the patched forward , before calling the original class’s bound forward method, which will now used the parameters provided at call time.
Revised: Here, it replaces the call to the forward method with one which ﬁrst replaces the stateful parameters of the submodule with ones provided as additional arguments to the patched forward , before calling the original class’s bound forward method, which will now used the parameters provided at call time.

Original: This method is generic and derived from ﬁrst-principles analysis of the torch.nn.
Revised: This method is generic and derived from ﬁrst-principles analysis of the torch.nn.

Original: Module implementation, ensuring that any ﬁrst or third-party implementation of parametric models which are subclasses of torch.nn.
Revised: Module implementation, ensuring that any ﬁrst or third-party implementation of parametric models which are subclasses of torch.nn.

Original: Module and do not abuse the parent class at runtime will be supported by this recursive patching process.
Revised: Module and do not abuse the parent class at runtime will be supported by this recursive patching process.

Original: 4. M AKING OPTIMIZERS DIFFERENTIABLE
Revised: 4. M AKING OPTIMIZERS DIFFERENTIABLE

Original: Again, as part of our need to make the optimization process differentiable in order to satisfy Requirement II of Section 2.4, the typical solution is to write a version of SGD which does not modify parameters in-place, but treated as a differentiable function of the input parameters and hyperparameters akin to any other module in the training process.
Revised: Again, as part of our need to make the optimization process differentiable in order to satisfy Requirement II of Section 2.4, the typical solution is to write a version of SGD which does not modify parameters in-place, but treated as a differentiable function of the input parameters and hyperparameters akin to any other module in the training process.

Original: While this, again, is often considered a satisfactory solution in the meta-learning literature due to its simplicity, it too is limiting.
Revised: While this, again, is often considered a satisfactory solution in the meta-learning literature due to its simplicity, it too is limiting.

Original: Not only does the inability to experiment with other inner loop optimizers prevent research into the applicability of meta-learning algorithms to other optimization processes, the restriction to SGD also means that existing state-of-the-art methods used in practical domains cannot be extended using meta-learning methods such as those described in Section 3, lest they perform competitively when trained with SGD.
Revised: Not only does the inability to experiment with other inner loop optimizers prevent research into the applicability of meta-learning algorithms to other optimization processes, the restriction to SGD also means that existing state-of-the-art methods used in practical domains cannot be extended using meta-learning methods such as those described in Section 3, lest they perform competitively when trained with SGD.

Original: Here, while less generic, the solution provided by the high-level function unnamedlib.get diff optim() is to render a PyTorch optimizer instance differentiable by mapping its parent class to a differentiable reimplementation of the instance’s parent class.
Revised: Here, while less generic, the solution provided by the high-level function unnamedlib.get diff optim() is to render a PyTorch optimizer instance differentiable by mapping its parent class to a differentiable reimplementation of the instance’s parent class.

Original: The reimplementation is typically a copy of the optimizer’s step logic, with in-place operations being replaced with gradient-tracking ones (a process which is syntactically simple to execute in PyTorch).
Revised: The reimplementation is typically a copy of the optimizer’s step logic, with in-place operations being replaced with gradient-tracking ones (a process which is syntactically simple to execute in PyTorch).

Original: To this, we add wrapper code which copies the optimizer’s state, and allows safe branching off of it, to permit “unrolling” of the optimization process within an inner loop (cf.
Revised: To this, we add wrapper code which copies the optimizer’s state, and allows safe branching off of it, to permit “unrolling” of the optimization process within an inner loop (cf.

Original: the recurrence from Equation 3) without modifying the initial state of the optimizer (e.g. to permit several such unrolls, or to preserve state if inner loop optimizer is used elsewhere in the outer loop).
Revised: the recurrence from Equation 3) without modifying the initial state of the optimizer (e.g. to permit several such unrolls, or to preserve state if inner loop optimizer is used elsewhere in the outer loop).

Original: Most of the optimizers in torch.optim are covered by this method.
Revised: Most of the optimizers in torch.optim are covered by this method.

Original: Here too, a runtime modiﬁcation of the parent optimizer class could possibly be employed as was done for torch.nn.
Revised: Here too, a runtime modiﬁcation of the parent optimizer class could possibly be employed as was done for torch.nn.

Original: Module s, but this would involve modifying Python objects at a far ﬁner level of granularity.
Revised: Module s, but this would involve modifying Python objects at a far ﬁner level of granularity.

Original: We ﬁnd that supporting a wide and varied class of optimizers is a sufﬁcient compromise to enable further research.
Revised: We ﬁnd that supporting a wide and varied class of optimizers is a sufﬁcient compromise to enable further research.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we brieﬂy present some results of experiments using select existing methods from
Revised: In this section, we brieﬂy present some results of experiments using select existing methods from

Original: Section 3.1, to show case how unnamedlib can be used to simply implement ablation studies and searches over model architectures, optimizers, and other aspects of an experiment.
Revised: Section 3.1, to show case how unnamedlib can be used to simply implement ablation studies and searches over model architectures, optimizers, and other aspects of an experiment.

Original: This could, naturally, be done without appeal to the library.
Revised: This could, naturally, be done without appeal to the library.

Original: However, in such cases, changing the model architecture or optimizer requires re-implementing the model functionally or optimization step differentiably.
Revised: However, in such cases, changing the model architecture or optimizer requires re-implementing the model functionally or optimization step differentiably.

Original: Here such changes require writing no new code (excluding the line where the model is deﬁned).
Revised: Here such changes require writing no new code (excluding the line where the model is deﬁned).

Original: 5. M ETA - LEARNING LEARNING RATES WITH HIGHER GRANULARITY
Revised: 5. M ETA - LEARNING LEARNING RATES WITH HIGHER GRANULARITY

Original: As the ﬁrst application for unnamedlib , we set up a simple meta-learning task where we metaoptimize the learning rate.
Revised: As the ﬁrst application for unnamedlib , we set up a simple meta-learning task where we metaoptimize the learning rate.

Original: Employing a handcrafted annealing schedule for learning rates has been the de facto approach to improving a learning algorithm.
Revised: Employing a handcrafted annealing schedule for learning rates has been the de facto approach to improving a learning algorithm.

Original: While scheduled annealing can help to boost the ﬁnal performance, it usually requires signiﬁcant hand-tuning of the schedule.
Revised: While scheduled annealing can help to boost the ﬁnal performance, it usually requires signiﬁcant hand-tuning of the schedule.

Original: In contrast, we adjust learning rates automatically using meta-learning, which unnamedlib enables for arbitary models and optimizers.
Revised: In contrast, we adjust learning rates automatically using meta-learning, which unnamedlib enables for arbitary models and optimizers.

Original: We take an image classiﬁcation model DenseNet-BC(k=12) (Huang et al., 2016), that provides competitive state-of-the-art results on CIFAR10, and modify its training procedure by replacing the multi-step annealing schedule of learning rate with a meta-optimizer.
Revised: We take an image classiﬁcation model DenseNet-BC(k=12) (Huang et al., 2016), that provides competitive state-of-the-art results on CIFAR10, and modify its training procedure by replacing the multi-step annealing schedule of learning rate with a meta-optimizer.

Original: Speciﬁcally, we treat learning rate for each inner optimizer’s parameter group as a separate meta-parameter that we will metalearn.
Revised: Speciﬁcally, we treat learning rate for each inner optimizer’s parameter group as a separate meta-parameter that we will metalearn.

Original: Doing so allows individual model parameters to have ﬁner learning rates, that are adjusted automatically.
Revised: Doing so allows individual model parameters to have ﬁner learning rates, that are adjusted automatically.

Original: The G IMLI algorithm provides for a clean implementation of such training procedure.
Revised: The G IMLI algorithm provides for a clean implementation of such training procedure.

Original: [Figure]
Revised: [Figure]

Original: We ﬁrst train two baseline variants of DenseNet-BC(k=12) using setup from (Huang et al., 2016).
Revised: We ﬁrst train two baseline variants of DenseNet-BC(k=12) using setup from (Huang et al., 2016).

Original: In the ﬁrst variant we keep the learning rate ﬁxed to 0 .
Revised: In the ﬁrst variant we keep the learning rate ﬁxed to 0 .

Original: 1 for all 300 epochs, while the second conﬁguration takes advantage of a manually designed multi-step annealing schedule, which drops learning rate by 10 after 150 and 225 epochs.
Revised: 1 for all 300 epochs, while the second conﬁguration takes advantage of a manually designed multi-step annealing schedule, which drops learning rate by 10 after 150 and 225 epochs.

Original: For the meta-learning variant, we split the training set into two disjoint pieces, one for training and another for validation, in proportion of 99 : 1 .
Revised: For the meta-learning variant, we split the training set into two disjoint pieces, one for training and another for validation, in proportion of 99 : 1 .

Original: We then use per parameter group learning rates ( 299 in total) for meta-optimization, initializing each learning rate to 0 .
Revised: We then use per parameter group learning rates ( 299 in total) for meta-optimization, initializing each learning rate to 0 .

Original: We perform one meta-update step after each epoch, where we unroll inner optimizer for  15 steps using batches from the trainingset, and compute meta-test error on the validation setover 10 batches from the validation set. We use batch size of 16 for meta-update, rather than 64 as in
Revised: We perform one meta-update step after each epoch, where we unroll inner optimizer for [Table] 15 steps using batches from the training set

Original: [Table]
Revised: [Table]

Original:  the base training loop.
Revised: and compute meta-test error on the validation set over 10 batches from the validation set. We use batch size of 16 for meta-update, rather than 64 as in the base training loop.

Original: We use Adam (Kingma & Ba, 2014) with default parameters as a choice for meta-optimizer.
Revised: We use Adam (Kingma & Ba, 2014) with default parameters as a choice for meta-optimizer.

Original: We average results over 3 random seeds.
Revised: We average results over 3 random seeds.

Original: Figure 1 demonstrates that our method is able reach the state-of-the-art performance faster
Revised: Figure 1 demonstrates that our method is able reach the state-of-the-art performance faster

Original: [Table]
Revised: [Table]

Original: 5. A BLATING MAML’ S MODEL ARCHITECTURE AND INNER OPTIMIZER
Revised: 5. A BLATING MAML’ S MODEL ARCHITECTURE AND INNER OPTIMIZER

Original: The unnamedlib library enables the exploration of new MAML-like models and inner-loop optimizers, which historically has required non-trivial implementations of the fast-weights for the model parameters and inner optimizers as done in Antoniou et al.
Revised: The unnamedlib library enables the exploration of new MAML-like models and inner-loop optimizers, which historically has required non-trivial implementations of the fast-weights for the model parameters and inner optimizers as done in Antoniou et al.

Original: ; Deleu et al.
Revised: ; Deleu et al.

Original: These ablations can be important for squeezing the last few bits of accuracy on well-established tasks and baselines that are already near-optimal as shown in Chen et al.
Revised: These ablations can be important for squeezing the last few bits of accuracy on well-established tasks and baselines that are already near-optimal as shown in Chen et al.

Original: and is even more important for developing new approaches and tasks that deal with different kinds of data and require adaptation to be done over non-standard operations.
Revised: and is even more important for developing new approaches and tasks that deal with different kinds of data and require adaptation to be done over non-standard operations.

Original: To illustrate how easy unnamedlib makes these ablations, in this section we take a closer look at different model architecture and optimizer choices for the MAML++ approach (Antoniou et al., 2018).
Revised: To illustrate how easy unnamedlib makes these ablations, in this section we take a closer look at different model architecture and optimizer choices for the MAML++ approach (Antoniou et al., 2018).

Original: MAML++ uses a VGG network with a SGD inner optimizer for the the Omniglot (Lake et al., 2015) and Mini-Imagenet (Vinyals et al., 2016; Ravi & Larochelle, 2016) tasks.
Revised: MAML++ uses a VGG network with a SGD inner optimizer for the the Omniglot (Lake et al., 2015) and Mini-Imagenet (Vinyals et al., 2016; Ravi & Larochelle, 2016) tasks.

Original: We start with the ofﬁcial MAML++ code and evaluation procedure and use unnamedlib to ablate across
Revised: We start with the ofﬁcial MAML++ code and evaluation procedure and use unnamedlib to ablate across

Original: VGG, ResNet, and DenseNet models and SGD and Adam optimizers.
Revised: VGG, ResNet, and DenseNet models and SGD and Adam optimizers.

Original: We provide more details about these in Appendix B.
Revised: We provide more details about these in Appendix B.

Original: We denote the combination of model and inner optimizer choice with <model>+<opt> .
Revised: We denote the combination of model and inner optimizer choice with <model>+<opt> .

Original: One ﬁnding of standalone interest is that we have kept most of the features from MAML++, except we signiﬁcantly increase the base VGG+SGD performance by using batch normalization in training mode everywhere as in (Finn et al., 2017) instead of using per-timestep statistics and parameters as MAML++ proposes.
Revised: One ﬁnding of standalone interest is that we have kept most of the features from MAML++, except we signiﬁcantly increase the base VGG+SGD performance by using batch normalization in training mode everywhere as in (Finn et al., 2017) instead of using per-timestep statistics and parameters as MAML++ proposes.

Original: In theory, this enables more inner optimization steps to be rolled out at test time, which otherwise is not possible with MAML++ because of the pertimestep information, for simplicity in this paper we have not explored this and keep every algorithm, optimizer, and mode to ﬁve inner optimization steps.
Revised: In theory, this enables more inner optimization steps to be rolled out at test time, which otherwise is not possible with MAML++ because of the pertimestep information, for simplicity in this paper we have not explored this and keep every algorithm, optimizer, and mode to ﬁve inner optimization steps.

Original: When using Adam as the inner optimizer, we initialize the ﬁrst and second moment statistics to the statistics from the outer optimizer, which is also Adam, and learn per-parameter-group learning rates and rolling average β coefﬁcients.
Revised: When using Adam as the inner optimizer, we initialize the ﬁrst and second moment statistics to the statistics from the outer optimizer, which is also Adam, and learn per-parameter-group learning rates and rolling average β coefﬁcients.

Original: Our results in Table 1 show that we are able to push the accuracy of MAML++ slightly up with this ablation.
Revised: Our results in Table 1 show that we are able to push the accuracy of MAML++ slightly up with this ablation.

Original: We note that this pushes the performance of MAML++ closer to that of state-of-the-art methods such as LEO (Rusu et al., 2018).
Revised: We note that this pushes the performance of MAML++ closer to that of state-of-the-art methods such as LEO (Rusu et al., 2018).

Original: Appendix B shows our full experimental results, and we note that in some cases Adam slightly outperforms SGD for a particular model.
Revised: Appendix B shows our full experimental results, and we note that in some cases Adam slightly outperforms SGD for a particular model.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: To summarize, we have presented G IMLI , a general formulation of a wide class of existing and potential meta-learning approaches, listed and proved the requirements that must be satisﬁed for such approaches to be possible, and speciﬁed a general algorithmic formulation of such approaches.
Revised: To summarize, we have presented G IMLI , a general formulation of a wide class of existing and potential meta-learning approaches, listed and proved the requirements that must be satisﬁed for such approaches to be possible, and speciﬁed a general algorithmic formulation of such approaches.

Original: We’ve described a lightweight library, unnamedlib , which extends PyTorch to enable the easy and natural implementation of such meta-learning approaches at scale.
Revised: We’ve described a lightweight library, unnamedlib , which extends PyTorch to enable the easy and natural implementation of such meta-learning approaches at scale.

Original: Finally we’ve demonstrated some of its potential applications.
Revised: Finally we’ve demonstrated some of its potential applications.

Original: We hope to have made the case not only for the use of the mathematical and software tools we present here, but have also provided suitable encouragement for other researchers to use them and explore the boundaries of what can be done within this broad class of meta-learning approaches.
Revised: We hope to have made the case not only for the use of the mathematical and software tools we present here, but have also provided suitable encouragement for other researchers to use them and explore the boundaries of what can be done within this broad class of meta-learning approaches.

Original: A O PTIMIZER D IFFERENTIABILITY A. SGD
Revised: A O PTIMIZER D IFFERENTIABILITY A. SGD

Original: [Equation]
Revised: [Equation]

Original: where α is the learning rate
Revised: where α is the learning rate

Original: [Equation]
Revised: [Equation]

Original: and ∇ ϕ α is deﬁned iff α ⊆ ϕ else G t ∇ ϕ α
Revised: and ∇ ϕ α is deﬁned iff α ⊆ ϕ else G t ∇ ϕ α

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: A. A DAGRAD
Revised: A. A DAGRAD

Original: [Equation]
Revised: [Equation]

Original: where η is the global learning rate
Revised: where η is the global learning rate

Original: [Equation]
Revised: [Equation]

Original: and ∇ ϕ η is deﬁned iff η ⊆ ϕ else
Revised: and ∇ ϕ η is deﬁned iff η ⊆ ϕ else

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: B MAML++ EXPERIMENTS : A DDITIONAL INFORMATION
Revised: B MAML++ EXPERIMENTS : A DDITIONAL INFORMATION

Original: Table 2 shows all of the architectures and optimizers we ablated.
Revised: Table 2 shows all of the architectures and optimizers we ablated.

Original: The table is missing some rows as we only report the results that successfully completed running three seeds within three days on our cluster.
Revised: The table is missing some rows as we only report the results that successfully completed running three seeds within three days on our cluster.

Original: We use the following model architectures, which closely resemble the vanilla PyTorch examples for these architectures but are modiﬁed to be smaller for the few-shot classiﬁcation setting:
Revised: We use the following model architectures, which closely resemble the vanilla PyTorch examples for these architectures but are modiﬁed to be smaller for the few-shot classiﬁcation setting:

Original: • vgg is the VGG architecture (Simonyan & Zisserman, 2014) variant used in MAML++ (Antoniou et al., 2018) • resnet-N is the ResNet architecture (He et al., 2016).
Revised: • vgg is the VGG architecture (Simonyan & Zisserman, 2014) variant used in MAML++ (Antoniou et al., 2018) • resnet-N is the ResNet architecture (He et al., 2016).

Original: resnet-4 corresponds to the four blocks having just a single layer, and resnet-8 and resnet-12 have respectively 2 and 3 layers in each block • densenet-8 is the DenseNet architecture (Huang et al., 2017) with 2 layers in each block
Revised: resnet-4 corresponds to the four blocks having just a single layer, and resnet-8 and resnet-12 have respectively 2 and 3 layers in each block • densenet-8 is the DenseNet architecture (Huang et al., 2017) with 2 layers in each block

Original: We
Revised: We

Original: follow TADAM (Oreshkin et al., 2018) and do
Revised: follow TADAM (Oreshkin et al., 2018) and do

Original: not use the initial convolutional projection layer common in the full-size variants of the ResNet and DenseNet.
Revised: not use the initial convolutional projection layer common in the full-size variants of the ResNet and DenseNet.

Original: We can also visualize how the learning rates and rolling momentum terms (with Adam) change over time when they are being learned.
Revised: We can also visualize how the learning rates and rolling momentum terms (with Adam) change over time when they are being learned.

Original: We show this in Figure 2 and Figure 3 for the 20-way 1-shot Omniglot experiment with the resnet-4 architecture, which is especially interesting as
Revised: We show this in Figure 2 and Figure 3 for the 20-way 1-shot Omniglot experiment with the resnet-4 architecture, which is especially interesting as

Original: Adam outperforms SGD in this case.
Revised: Adam outperforms SGD in this case.

Original: We ﬁnd that most of the SGD learning rates are decreased to near-zero except for a few select few that seem especially important for the adaptation.
Revised: We ﬁnd that most of the SGD learning rates are decreased to near-zero except for a few select few that seem especially important for the adaptation.

Original: Adam exhibits similar behavior with the learning rates where most except for a select for are zeroed for the adaptation, and the rolling moment coefﬁcients are particularly interesting where the ﬁrst moment coefﬁcient
Revised: Adam exhibits similar behavior with the learning rates where most except for a select for are zeroed for the adaptation, and the rolling moment coefﬁcients are particularly interesting where the ﬁrst moment coefﬁcient

Original: β 1 becomes relatively evenly spread throughout the space while β 2 splits many parameter groups between low and high regions of the space.
Revised: β 1 becomes relatively evenly spread throughout the space while β 2 splits many parameter groups between low and high regions of the space.

Original: [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]