Decoupling Adaptation from Modeling with Meta-Optimizers for Meta Learning
Authors: Sébastien M.R. Arnold
Shariq Iqbal
Fei Sha
Venue:  (2020)
Keywords: meta-learning
MAML
analysis
depth
meta-optimizers

Abstract: Meta-learning methods, most notably Model-Agnostic Meta-Learning (Finn et al, 2017) or MAML, have achieved great success in adapting to new tasks quickly, after having been trained on similar tasks.
The mechanism behind their success, however, is poorly understood.
We begin this work with an experimental analysis of MAML, finding that deep models are crucial for its success, even given sets of simple tasks where a linear model would suffice on any individual task.
Furthermore, on image-recognition tasks, we find that the early layers of MAML-trained models learn task-invariant features, while later layers are used for adaptation, providing further evidence that these models require greater capacity than is strictly necessary for their individual tasks.
Following our findings, we propose a method which enables better use of model capacity at inference time by separating the adaptation aspect of meta-learning into parameters that are only used for adaptation but are not part of the forward model.
We find that our approach enables more effective meta-learning in smaller models, which are suitably sized for the individual tasks.


Under review as a conference paper at ICLR 2020
DECOUPLING ADAPTATION FROM MODELING WITH META-OPTIMIZERS FOR META LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Meta-learning methods, most notably Model-Agnostic Meta-Learning (Finn et al. 2017) or MAML, have achieved great success in adapting new tasks quickly, after having been trained on similar tasks. The mechanism behind their success, however, poorly understood. We begin this work with an experimental analysis of MAML., finding that deep models are crucial for its success, even given sets of simple tasks where linear model would suffice on any individual task. Further- more, on image-recognition tasks, we find that the early layers of MAML-trained models learn task-invariant features, while later layers are used for adaptation, pro- viding further evidence that these models require greater capacity than is strictly necessary for their individual tasks. Following our findings, we propose a method which enables better of model capacity at inference time by separating the adaptation aspect of meta-learning into parameters that are only used for adap- tation but are not part of the forward model. We find that our approach enables more effective meta-learning in smaller models, which are suitably sized for the individual tasks.
INTRODUCTION
Meta-learning or learning to learn is an appealing notion due to its potential in addressing important challenges when applying machine learning to real-world problems. In particular, learning from prior tasks but being able to to adapt quickly to new tasks improves learning efficiency, model
robustness, etc. A promising set of techiques, Model-Agnostic Meta-Learning (Finn et al. 2017)
or MAML, and its variants, have received lot of interest (Nichol al. 2018 Lee & Choi 2018
Grant et al.) 2018). However, despite several efforts, understanding of how MAML works, either theoretically in practice, has been lacking (Finn & Levine 2018 Fallah et al. 2019).
For a model that meta-learns, its parameters need to encode not only the common knowledge ex- tracted from the tasks has seen, which form task-general inductive bias, but also the capability to adapt to new test tasks (similar to those it has seen) with task-specific knowledge. This begs the question: how are these two sets of capabilities represented in single model and how do they work together?
In the case of deep learning models, one natural hypothesis is that while knowledge represented distributedly in parameters, they can be localized - for instance, lower layers encode task-general inductive bias and the higher layers encode adaptable task task-specific inductive bias. This hypothesis consistent with one of deep learning's 's advantages in learning representations (or feature extractors) using its bottom layers.
Then we must ask, in order for deep learning model to meia-learn, does it need more depth than needs for solving the target tasks? In other words, is having a large capacity to encode knowledge that is unnecessary post-adaptation the price has to pay in order to be adaptable? Is there way to have a smaller (say, less deep) meta-learnable model which still adapts well? This question is of both scientific interest and practical importance a smaller model has smaller (memory) footprint, faster inference and consumes less resources.
In this work, through empirical studies both synthetic datasets and benchmarks used in the litera- ture, we investigate these questions by analyzing how well different learning models can meta-learn and adapt. We choose to focus on MAML due to its popularity.

---

Under review as a conference paper at ICLR 2020
Our observations suggest depth is indeed necessary for meta-leariin, despite the tasks being solv- able using a shallower model. Thus, applying MAML to shallower models does not result in suc- cessful meta-learning models that can adapt well. Moreover, studies also show that higher layers are responsible more for adapting to new tasks while the lower layers are responsible for learning task-general features.
Our findings prompt us to propose a new method for meta-learning. The new approach introduces meta-optimizer which learns guide the (parameter) optimization process of a small model. The small model is used for solving the tasks while the optimizer bears the burden of extracting the knowledge of how to adapt. Empirical results show that despite using smaller models, the proposed algorithm with small models attains similar performance to larger models which use MAML to meta-learn and adapt.
We note that a recent and concurrent work to ours addresses questions in this line of inquiry (Raghu jtt al. 2019). They reach similar conclusions through different analysis and likewise, they propose different approach for improving MAML. We believe our work is complementary to theirs.
2 RELATED WORK
Meta-leaming, or learning-to-leams is a vibrant research area with long and rich history, lying at the intersection of psychology (Maudsley) 1980 Biggs 1985), neuroscience (Hasselmo & Bower 1993), and computer ccience(Schiichuber 1987 Thrun & Prati 1998) Of particular interest to this
manuscript is the line of work concerned with optimization-bann meta- meta-leerring (OBML) algorithms
in the few-shot regime, of which MAML is particular instance. (Finn & Levine 2018 Finn al.
2017
Finn 2018)
Since its inception, MAML has been widely applied to tackle the few-shot learning challenge. in
domains such as computer vision (Lee et al. (019), natural language processing (Gu et al.) 2018)) and robotics (Nagabandi et al.] 2018). It is also the basis of extensions for continual learning (Finn) iet al. 2019). single- and multi-agent reinforcement learning (Rothfuss et al. 2018 |018||Al-Shee [Al-Shedivat et al.) 2017). objective learning (Chebotar et al.) 2019), transfer learning (Kirsch al. 2019), and domain adaptation (Li et al., 2018). Due to its generality, the adaptation procedure MAML introduces which the focus of our analysis has recently been branded as Generalized Inner Loop Meta- Learning. (Grefenstette et al. 2019)
While popular in practical applications, relatively few works have analysed the convergence and modelling properties of those algorithms. Finn & Levine (2018) showed that, when combined with deep architectures, OBML is able to approximate arbitrary meta-learning schemes. [Fallah et al.) (019) recently provided convergence guarantees for MAML approximate first-order stationary points for non-convex loss surfaces, under some assumptions on the availability and distribution of the data. Other analyses (empirical
theoretical) have attempted explain the generalization ability of OBML (Guiroy et al.] 2219); Nichol et al.) [2018), the bias induced by restricting the num- ber of adaptation steps (Wu et al. 2018)) the effect of higher-order terms in the meta-gradient estimationnFoeerter et af. 2018 Rothfuss et al. 2019)
Closely related our proposed methods are works attempting improve the adaptation mecha- nisms of OBML. Meta-SGD et (017)) meta-learns per-parameter learning rates, while Alpha MAML (Behl al.) 2019) adapts those learning rates during adaptation via gradient-descent, Meta- Curvature (Park & Oliva 2019) propose to learn a Kronecker-factore. pre-conditioning matrix to compute fast-adaptation updates. Their resulting algorithm where the linear transformation is not updated during adaptation. Another way of constructing pre- conditioning matrices is to explicitly decompose all weight matrices of the model in two separate components, as done in 'T-Nets (Lee & Choil 2018). The first component is only updated via the evaluation loss, while the second is also updated during fast-adaptation. Warped Gradient Descent (Flennerhag et al. 2019) further extends T-Nets by allowing both components to be non-linear func- tions. Instead of directly working with gradients, (Chebotar et al.) 2019: Xu et al., 2018) suggest to directly learn loss function which is differentiated during adattatat and results in faster and better learning. Additionally, meta-optimizers have also been used for meta-descent (Sutton [981
special case of one of our methods,
Jacobs 1988 Sutton| 1992b a Schraudolph 1999). They can be learned during pre-training phase
2

---

Under review as a conference paper at ICLR 2020
Andrychowicz al., [0166] Li & Malik 2017a bj Metz et al. 2019b c Wichrowska et al. 2017) or online (Kearney et al. 2018 Jacobsen et al., 2019 Ravi & Larochelle 2017).
Our work differentiates from the the above by diagnosing and attempting address the entangle- ment between modelling and adaptation in the meta-learning regime. We uncover a failure mode of MAML with linear and smaller models, and propose effective solution in the form of expressive meta-optimizers.
3 ANALYSIS OF MAML
3.1 BACKGROUND
In MAML and its many variants (Lee & Choi, 2018 Nichol et al., 2018 Li al., 2017). we have a model whose parameters are denoted by 6. We would like to optimize such that the resulting model can adapt new and unseeen tasks fast. To this end, we are given a of (meta)training tasks, indexed by r. For each such task, we associate with a loss (0))
Distinctively, MAML minimizes the expected task loss after adaptation phase, consisting of few steps of gradient descent from the model's current parameters. Since we do not have access to the target tasks to which we wish to adapt to, we use the expected loss over the training tasks,
META
E m)p(())
(0-
where the expectation is taken with respect to the distribution of the training tasks. is the learning rate for the adaptation phase. The right-hand- uses only one step gradient descent such that the aim is adaptfast in one step, we would like reduce the loss as much as possible. In practice, few more steps are often used.
3.2 ANALYSIS
Shallow models can be hard to meta-learn Many intuitive explanations for why MAML. works exist. One appealing suggestion is that the minimizer of the meta-learning loss META is chosen in such a way that it provides a very good initialization for the adaptation phase; however, the model is shallow such that L, is convex in its parameters, then any initialization that is good for fast adapting to one subset of tasks could be bad for another subset of tasks since all the tasks have precisely one global minimizer and those minimizers can be arbitrarily far from each other. When the test tasks are distributed similar to the training tasks, the ideal initialization point has to be the "mean" of the minimizers of the training tasks the precise definition of the mean is not important, as we will see below.
We illustrate a surprising challenge by studying MAML on a synthetic dataset and the Omniglot task (Lake et al. 2..3)) Specifically, for the former study, we construct a set of binary classification tasks by first randomly sampling datapoints w 100 from a standard Gaussian and use each of them to define linear decision boundary of a binary classification task. We assume the boundaries pass through the origin and we sample training, validation and testing samples by randomly sampling data points from both sides of the decision boundaries.
By construction, a linear model such as logistic regression is sufficient to achieve very high accuracy on any task. But can MAML learn logistic regression model from subset of training tasks that is able to adapt quickly to the test tasks?
Note that due to the random sampling of the training tasks, the average of the minimizers (ie, the samples from the Gaussian distribution) is the origin. Likewise, for a set of test tasks randomly sampled the same way, the origin provides the best initialization by not favoring any particular task.
Figure reports the 1-step post-adaptation accuracy on the test tasks for the meta-learned logistic regression model. Surprisingly, the model fails to perform better than chance. Despite the simplicity of the tasks, logistic regression models are unable to find the origin as an initialization that adapts quickly to a set of test tasks that similar to the training tasks.
SecureTalss reports how being deep can drastically change the behavior of MAML. There, we add 4-layer linear network (LinNet) the logistic regression model (before the sigmoid activation).
3

---

Under review as a conference paper at ICLR 2020
Synthetic
Omniglot
1.0
1.0
Post-Adaptation Accuracy
LR MAML
0.8
0.8
LinNet w/ MAML
0.6
Accuracy
0.6
0.4
0.4
0.2
LR MAML
0.2
LinNet w/ MAML
0.0
0.0
0.5
1.0
1.5 le4
0
2
6
Iterations
Iterations
1e4
Figure 1: Meta learning results on two datasets: Synthetic binary classification (left) and Omniglot (right). The models to he meta-learned are shallow ones ( logistiic regression models for binary classification and multi- nomial logistic regression) and deep ones the shallow models While the shallow models do not adapt and achieve chance level accuracies on average, the overparameterized versions attain significantly better results after adaptation.
with linear networks (LinNet) added to them.
Note that while the model has the same representational capacity as a linear logistic regression, it is overparameterized and has many local optimizers. As such, MAML can train this model such that its 1-step adaptation accuracy reaches 92% on average.
We observe the same phenomena on meta-learning with MAML on the Omniglot dataset (details of the dataset are given in the Section 55). The shallow logistic regression model achieves 45% accuracy on average (for the 5-way classification tasks) after 2-step adaptation from the meta-learned initial- ization. However, with linear network, the adapted model achieves significantly higher accuracy 70% on average, while having the same modeling capacity as the logistic regression.
In summary, these experiments suggest that even for tasks that are solvable with shallow models, a model needs to have enough depth in order to be meta-learnable and to adapt. We postpone the description of our experiments on nonlinear models to section where we also show that having sufficient depth is crucial for models to be meta-learnable, even when the tasks require fewer layers.
A natural question arises: if being deep is so important for meta- meta-learning what different roles, if any, do different layers of deep network play?
very simple tasks,
Depth enables task-general feature learning and fast adaptation We hypothesize that for deep models meta-learned with MAML, lower layers learn task-invariant features while higher layers are responsible for fast-adaptation.
To examine this claim, meta-train model consisting of four convolutional layers (C1 C4) and final fully-connected layer (FC) on Omniglot (Lake et al., 2015) and CIFAR-FS (Bertinetto et al., 2019). (Experimental setups are detailed in Section[5) Once the model has finished meta-training, we perform a layer-wise ablation to study each layer's s effect on adaptation. In particular, we iterate over each layer and perform two sets of experiments. In the first, we freeze the weights of the layer such that it does not get updated during fast- ansttadaptation we call it freezing only this. In the second experiment, we freeze all layers but the layer such that this layer is updated during fast-adaptation we call it adapting only this. The left two plots in Figure 2 report the average accuracy over 100 testing tasks from both datasets.
We observe that freezing only the first few lower layers (CI-C3) does not cause noticeable degra- dation to the post-adaptation accuracy. In fact, as long as the last convolutional layer (C4) is not frozen, post-adaptation accuracy remains unaffected. This indicate that C1-C3 provide information that is task- invariant, while C4 is crucial for adaptation.
This does not mean FC is not important since adapting C4 requires gradients passing through the FC layer, it cannot be arbitrary. In fact, in the rightmost plot of the figure, C1-C3 are held fixed during adaptation. While C4 and FC are allowed to adapt, and FC is perturbed with noise. When
4

---

Under review as a conference paper at ICLR 2020
Omniglet
CIFAR.FS
Freezing C1-C3, Perturbing FC
1.0
1.0
1.0
PostAdigtatio Accunaty
Post-Adupttion Accuary
0.5
Adapting (only thi)
Freezing (only this)
Post Post-Adapttioon Accunity
0.8
0.0
0.6
0.6
Freezing (only this)
Adapting (only the)
0.4
0.4
0.2
0.2
0.2
Omniglot
CIFAR FS
0.0
C1
C2
C3 Layer
C4
FC
C1
C2
C3 Layer
04
FC
2
2-1
Noise Staling Factor
Figure 2: Layer-wise study of each layer's effect on adaptation, the left and the middle plots. The rightmost plot shows that both C4 and FC have to work together to adapt. See texts for details.
the noise is strong, the performance degrades significantly. Thus, conclude that both C4 and FC play important roles in the mechanism for fast adaptation. We note that the recent work by Raghu al. 2019) concurently reached similar conclusions on the mini-ImageNet dataset, using feature similanity-basee based analyses of the model's 's representations.
These observations highlight a fundamental issue: the property of being learnalle entails more model capacity than being learnable for a specific task. Thus, MAML can fail on models that lack the capacity to encode both task- kask-general features and adaptation information, even when the mod- els themselves are powerful enough to perform well each of the tasks used for the meta-learning procedure. For example, with linear models (e.g. logistic regression), the parameters are forced to overlap and serve both modelling and adaptation purposes. However, soon the models are over- parameterized, the extra layers enable veta-learnabillty. In section 5 we show that this observation also applies to nonlinear models where MAML-traine models quickly lose their performance when the number of layers is reduced.
4 META-OPTIMIZEI FOR META LEARNING
The previous section has shown that MAML, when applied to deep learning models, meta-learns both task-general features and task-specific adaption parameters the same time. Since both represented a single model where strong dependencies exist between the lower layers and the higher layers, the model needs be large enough and all its parameters have to be used to solve the test tasks. In some cases, the model has a modelling capacity that is bigger than what the test tasks require. For model where linear network layers exist, the layers can be collapsed after adaptation so that the actual model used for inference on teh test tasks is small. However, it is not clear how to do so for typical deep models where nonlincarities prevent collapsing the model into smaller ones.
Can we have different adaptation mechanism such that a smaller model, whose modelling ca- pacity still adequate for the test tasks, can be adapted to find the minimizers of its loss on the aforementioned tasks?
In this section, we describe a new approach of learning meta-optimizers for meta-learning, The meta-optinizer aims to separate modelling from adaptation. The meta-optimizer transforms the parameter update process for the model so the model can converge fast to minimizer. How to transform is, however, learned from meta-training tasks. Because of this separation, the model for the task does not have to know how to adapt; it only needs to have enough capacity (roughly, big enough to represent the target task).
We note that classical tools could be used to compress large model for given task. However, in the scenario where meta-learning is often used, such tools unlikely effective. For example, distillation requires a lot labeled data from the target task, which not suitable for few-shot learning tasks. Pruning often degrades performance.
5

---

Under review as a conference paper at ICLR 2020
4.1 LEARNABLE OETTA-OPTMMIERS
meta-optimizer or learnable optimizer - is a parameterized function Ug defining the model's parameter updates. For example, a linear meta-optimizer might be defined
Ug(g) Ag b
where ç (A is the set of parameters of the linear transformation. The objective is to jointly learn the model and optimizer's parameters in the hope of accelerating the minimization of C,. Concretely, we alternate between model and optimizer updates such that their values at adaptation step t + are given by:
0.+1 = 0: - Ue. (V., Cr (0.) 6++1 = E1 a v. f, (0t+1)-
(3)
While this change to the classical optimization pipeline might seem innocuous at first, it bears its own set of computational challenges. First, it is common for modern machine learning models to have millions of parameters. In this case, simply storing the optimizer's parameters becomes infeasible even for the simplest of models, such as the linear one outlined above. Because this dimensionality issue most of the current literature considers per-parameter independent update function, thus greatly limiting the expressivity of the meta-optimizer. We propose to address the issue of parameter dimensionality via factorizations of the optimizer's parameters, which detail in the following subsection.
4.2 FACTORIZED META- OPTIMIZERS
To tackle the issue parameter dimensionality we propose to factorize the optimizer's parameters. To demonstrate this, let us revisit the linear optimizer example above: we assume R* to be the vectorization of matrix gradient E R=*m such that vee(G) ) = g and m n By expressing A R L as the Kronecker product of small matrices R E pm and L R=x", the linear update above can be efficiently computed as: (Bernstein) 2018 Section 9.1)
Ug(g) vec vec (L-G- G)))RR))4 b
(5)
where b R and & b.). In the best case scenario where factorization requires O (k) memory as opposed to O (k?2) for the non-factored case. Similarly, the matrix-product takes O kvk time complexity, while the non-factored version takes O (A²).
Vk, the above
Note that this linear transformation can be used as a building block implement more expressive and non-linear meta-optimizers. For example, a fulne-co network meta-optimizer Ue is the composition of linear transformations Uw, interleaved with activation function σ. the weight matrices (W,.... W₂) of the network admit Fronecker-factorizztion W; R T,, the fully- connected meta-optimizer is imlemented as:
Ug(g) = Uw, o o...ow og vec (Lio o (L, G (.)....)......
(6)
where & (Wi.... W.). Such scheme can be used to implement arbitrary meta-optimizers such as convolutional or recurrent ones - so long as the architecture involves composition of linear maps. We refer the reader to Appendix A.3 for pseudo-code and schematics of the model-optimizer loop.
We emphasize that the choice of a Kronecker factorization is arbitrary; many matrix decompositions work equivalently well and result different modeling and computational trade-offs. For example, using a low-rank Cholesky factorization A L.T" where T. R allows to interpolate between

---

Under review as a conference paper at ICLR 2020
Table Accuracies and standard deviation of Meta-Learning of Linear Models
Dataset
LR w/ MAML
LR + LINNET w/ MAML
LR w/ KLIN
LR w/ KFC
Synthetic
51.38% + 3.91
95.77% 0.41
99.00% % + 0.08
98.98% 1 0.10
Omniglot
46.50% 2.80
71.31% 4.52
63.88% 3.93
71.56% % 1 4.40
computational complexity and decomposition rank by tuning the additional Cholesky decomposition might be preferable the Kronecker one in memory-constraieen applica- tions, since r can be used to control the memory requirements of the meta-optimizer, Moreover, such decomposition imposes symmetry and positiveness on A, which might be desirable when approximating the Hessian or its inverse.
r. The
In this work, we prefered the Kronecker decomposition the computational and memory cost of the Kronecker-product are acceptable, (2) R 6 L. is full- rank whenever L, R are full-rank, and (3) the identity matrix lies in the span of Kronecker-factored matrices. In particular, this last motivation allows meta-optimizers to recover the gradient descent update by letting R, L be the identity.
alternatives for three reasons: (1)
EXPERIMENTS
In our experiments, we complement our empirical studies and analysis from Section 3 with further results and address the following question: can we meta-learn smaller (or shallower) models that perform as well as larger (or deeper) ones? To this end, we apply the proposed approach of learning meta-optimizers the example synthetic dataset, as well as popular benchmark datasets: Omniglot (Lake et al. 2015), mini- Imageee (Ravi & Larochelle| 2017]. and CIFAR-FS (Bertinetto et al., 2019). All parer-paraeeters and additional experimental details are available in Appendix(A.1]
Unless otherwise noted, all models are trained using MAML. For models that use learnable deep optimizers, we meta-learn both model and optimizer parameters. Similarly, both sets of parameters are adapted during dast-adaptation.
To differentiate, we add the name of the optimizer to the model. For example, LR w/ KLIN cor- responds to a logisitic regression model with the Kronecker-factorizze linear transformation from the previous section. Similarly, w/ KFC indicates a model optimized by Kronecker-factores Fully- Connected network in the previous section.
5.1 META-LEARNING LINEAR MODELS
We adopt the same setting in section 3 where we study the synthetic data and the Omniglot dataset. For the Omniglot dataset, we consider the 1-shot, 5-way setup of Finn al. (2017) and let the model adapt for 2 steps. Recall that the logistic regression model does not meta-learn while overparameterizee logistic regression models (ie, models with linear networks) do.
Table D reports the final post-adaptation testing accuracies and their standard deviations. As can be observed, while logistic regression (LR) cannot be meta-learned, overparameterized models (LINNET) as well as LR with our meta-optimizers (LR w/ KLIN and LR w/ KFC) can all meta- learn while our approaches lead to similar or better results compared to LinNet.
Note that since linear networks can always be collapsed into single weight matrix and bias term, any increase in performance is the consequence of better adaptation, rather than better modelling.
5.2 NON-LINEAR MODELS
We examine whether our meta-optimizers can meta-learn successfully with smaller but deep nonlin- ear models to match bigger deep models. We focus two datasets: the Omniglot where the setting 10-way classification with 5-shots and 4 adaptation steps, using the original 4-layer convolutional network (CNN) of Finn et al. (2017), and the CIFAR-FS dataset (Bertinetto et al. 2019), doing 10- way classification with 3-shots and 2 adaptation steps. The model is a CNN similar to the one for the Omniglot, but only uses 32 hidden units opposed to 64.
7

---

Under review as a conference paper at ICLR 2020
Omniglot
CIFAR-FS
1.0
Post-Adaptation Accuracy
Post-Adaptation Accuracy
0.8
0.5
0.6
0.4
0.4
CNN MAML KFC
0.3
CNN w/ MAML
CNN w/ KFC
4
2 Num. Layers
3
1
Num. Layers
Figure 3: The effect of the number of convolutional layers adaptation performance. While the model becomes smaller, the performance drops. However, approach degrades in a slower pace, as the adaptation shared hy both the model and the meta-optimizer, thus leaving more freedom for the model represent the tasks.
Table 2: Post-adaptation accuracies by different meta-optimizzrs adapting a small CNN.
Meta-I.camer Architecture
Task
UPPER BOUND
w/ MAML
w/ MSGD
w/ MC
w/ KFC
Omniglot
95.09% 1: 2.32
60.94% + 1.54
69.25% 1 0.81
85.70% 1 0.91
91.13% 1 0.93
CIFAR-FS
63.94% + 3.89
38.44% 1 0.87
45.96% 1.73
52.89%
1.43
52.94% 1 1.33
mini-ImageNet
58.45% + 5.58
18.55% 0.46
26.59% 1.43
29.74%
0.74
27.40% + 0.60
We report numerical results in detail in Table A2 in the Appendix, which are graphically presented in Figure g We vary the number of convolutional layers. As the number of the layers decrease, the approach
adaptation performance by both MAML and our approach (KFC) decrease. However, has a much slower degradation rate. In fact, our approach is generally able to adapt smaller model to the same level of performance MAML-trriied model with an additional layer.
5.3 COMPARISON OTHER META-OPTIMIZER
We also compare our approach of learning meta-optimizers to other approaches, notably Meta-SGD (Li al., 2017) and Meta- Cett--uuratture (Park Oliva, 2019). Both of these methods attempt to ap- proximate the expected task-loss landscape curvature and can be seen as ablations of our methods; Meta-Curvature corresponds to KLin without adapting the optimizer, while Meta-SGD approxi- mates Meta-Curee Curvature with diagonal matrix.
To ease the computational burden, we use a smaller CNN with 2 convolutional layers (SCNN) model for adaptation. As upper bound adaptation performance, we train the model individual tasks to convergence and report the average final accuracy. Additionally, we include results on the mini-Image dataset for all methods, in the 10- 1-shot with 5 adaptation steps setting (Ravi & Larochlle 2017).
Table reports post-adaptation testing accuracies. All methods of learning optimizers able to adapt better than SCNN w/ MAML. In particular, our approachs perform the best and comes closest to approaching the upper bound of performance (SCNN).
CONCLUSION
We introduce approach by analyzing the success and failure modes of optimization-base meta- learning methods. Namely, we find that, when successful, these methods tend to learn task-general features in early layers and adaptable parameters/update functions in the later layers. Moreover, find that this learning fails when model size is reduced, indicating that optimization-ban based meta- learning methods rely the ability to encode task-general features and/or adaptable parameters,
8

---

Under review as a conference paper at ICLR 2020
when the model itself is adequate for learning the individual tasks. As such, introduce our method for decomposing modelling from adaptation using factored meta-optimizers. These metaa-pttimizers enable the forward model to more capacity learning task-specifie features, while the expressiveness of their updates allows the forward model to adapt quickly to different tasks. We find that approach is able enable successful meta-learning in models that do not work with traditional optimization-ert based meta-learning methods.

---

Under review as a conference paper at ICLR 2020
REFERENCES
Maruan Al-Shedivat, Trapit Bansal, Yuri Burda, Ilya Sutskever, Igor Mordatch, and Pieter Abbeel. Continuous adaptation via meta-learning in nonstationary and competitive environments, arXiv preprint arXiv: wtXiv:/710.03441 2017.
Marcin Andrychowicz, Misha Denil, Sergio Gómez, Matthew W Hoffmann, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient de- scent by gradient descent. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 3981- . cttpppppprs.iis...ppppppeeeeeeee
3989. Curran Associates, Inc., 2016. URL
6461
pdf
Harkirat Singh Behl, Atilmm Güneş Baydin, and Philip H S Torr. Alpha MAML: Adaptive Model- Agnostic Meta-Learning arXiv preprint arXiv:1905.07433. May 2019.
Dennis S Bernstein. Scalar, Vector, and Matrix Mathematics: Theory, Facts, and Formulas Revised and Expanded Edition. Princeton University Press, revised, expanded edition edition, February 2018.
Luca Bertinetto, Joao Henriques, Philip Torr, and Andrea Vedaldi. Meta-learning with differen- tiable closed-form solvers. In International Conference Learning Representations, 2019. URL
.et/ / forum? i HdHHxx22hott77
J JB Biggs. THE ROLE OF METALEARNING IN STUDY PROCESSES. Br. J. Educ. Psychol., 55 (3):1852212, November 1985.
Yevgen Chebotar, Artem Molchanov, Sarah Bechtle, Ludovic Righetti, Franziska Meier, and Gau- rav S. Sukhatme, Meta-learning via learned loss. CoRR, abs/1906.05377 2019. URL http:
7/ wwxiv.orgaabs/13ee 05374
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of Gradient- Based Model-Agnostic Meta-Learning algorithms. arXiv preprint arXiv:1908.10400, August 2019.
Chelsea Finn. Learning to Learn with Gradients. PhD thesis, UC Berkeley, July 2018.
Chelsea Finn and Sergey Levine. Meta-learning and universality: Deep representations and gradi- ent descent approximate any learning algorithm. In International Conference Learning Representations, 2018. URL sttpa://openzeviie net/ htt.eee//eoommmmd///a/51553/1-/
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-Agnostic Meta-Learning for fast adap- tation of deep networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1126-1135, International Convention Centre, Sydney, Australia, 2017. PMLR.
Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th Interna- tional Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Re- search, pp. 1920-1930, Long Beach, California, USA. 09-15 Jun 2019. PMLR. URL http: //proceedings.. mmr.nngsmmrr...... mlr Swtres/v97/finnnn html
Sebastian Flennerhag, Andrei A Rusu, Razvan Pascanu, Hujun Yin, and Raia Hadsell. Meta- Learning with warped gradient descent. arXiv preprint wrXiv:199900002) August 2019.
Jakob Foerster, Gregory Farquhar, Maruan Al-Shedivat, Tim Rocktäschel, Eric Xing, and Shimon Whiteson. Dice: The infinitely differentiable monte carlo estimator. In International Conference on Machine Learning, pp. 1524-1533, 2018.
Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. Recasting gradient- based meta-learning as hierarchical bayes. In International Conference on Learning Representa- fions, 2018. URL /tt.itttpp:://peneevieee wwwp:/eeeefoommutmcmmmmm dd:.J
10

---

Under review as a conference paper at ICLR 2020
Edward Grefenstette, Brandon Amos, Denis Yarats, Phu Mon Htut, Artem Molchanov, Franziska Meier, Douwe Kiela, Kyunghyun Cho, and Soumith Chintala. Generalized inner loop meta- learning, 2019.
Jiatao Gu, Yong Wang, Yun Chen, Victor O. K. Li, and Kyunghyun Cho. Meta-leamiing for low-resource neural machine translation. In Proceedings of the 2018 Conference Empir- ical Methods in Natural Language Processing, pp. 3622-3631, Brussels, Belgium, October- November 2018. Association for Computational Linguistics. doi: 10.18653/11/DI8-13399 URL
www.wwwwwweb.oo.... http:///www...ollbbbb.ooroobaaaa
DD10/111822 1398
Simon Guiroy, Vikas Verma, and Christopher Pal. Towards understanding generalization in Gradient-Basee Meta-Learning. arXiv preprint arXiv:1907.07287 July 2019.
M E. Hasselmo and J M Bower. Acetylcholine and memory. Trends Neurosci,, 16(6):218-222, June
1993.
Robert A Jacobs. Increased rates of convergence through learning rate adaptation. Neural Netw., (4):295-307, January 1988.
Andrew Jacobsen, Matthew Schlegel, Cameron Linke, Thomas Degris, Adam White, and Martha White. Meta-descent for online, continual prediction. In AAAI Conference on Artificial Intelli- gence, 2019.
Alex Kearney, Vivek Veeriah, Jaden B Travnik, Richard S Sutton, and Patrick M Pilarski. TIDBD: Adapting temporal-difference step-sizes through stochastic meta-descent. arXiv preprint NrXiv::804.033338 April 2018.
Louis Kirsch, Sjoerd van Steenkiste, and Jürgen Schmidhuber. Improving generalization in meta reinforcement learning using learned objectives. arXiv preprint /9/0v:/9/00000990 2019.
Brenden Lake, Ruslan Salakhutdinov, and Joshua B Temenbaum. Human-level concept learning through probabilistic program induction. Science, 1332--26)):332113333 December 2015.
Kwonjoon Lee, Subhransu Maji, Avinash Ravichandran, and Stefano Soatto. Meta-learning with differentiable convex optimization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10657-10665, 2019.
Yoonho Lee and Seungjin Choi. Gradient-based meta-learinn with learned layerwise metric and subspace. Jennifer Dy and Andreas Krause (eds.). Proceedings of the 35th Interna- tional Conference Machine Learning, volume 80 of Proceedings of Machine Learning Re- search, pp. 2927-2936, Stockholmsmssan, Stockholm Sweden, 10-15 Jul 2018. PMLR. URL http::/proceediin
.wl.. pwwps/v80 1ete://eeeeaa.... html
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta- learning for domain generalization. In Thirty-Second AAAI Conference Artificial Intelligence, 2018.
Ke Li and Jitendra Malik. Learning to optimize. In International Conference Learning Repre- sentations, 2017a. URL httlttttt///:ptnreeet net httpa//wwwee.eeeeoooooooeeerer.o
Ke Li and Jitendra Malik. Learning optimize neural nets. arXiv preprint arXhv:1703.0044 March 2017b.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-SGD: Learning to learn quickly for Few-Shot learning. arXiv preprint 1XXiv:707.09980 July 2017.
Donald B Maudsley. A THEORY OF META-LEARNING AND PRINCIPLES OF FACILITA- TION: AN ORGANISMIC PERSPECTIVE, 1980.
Luke Metz, Niru Maheswaranathan. Brian Cheung, and Jascha Sohl-Dickstein. Meta-learning up- date rules for unsupervised representation learning. International Conference on Learning Representations, 2019a.

---

Under review as a conference paper at ICLR 2020
Luke Metz, Niru Maheswaranathan, Jeremy Nixon, Daniel Freeman, and Jascha Sohl-Dickstein. Understanding and correcting pathologies in the training of learned optimizers. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings
the 36th International Conference on Machine Learning, volume 97 of Proceddings of Machine Learning Research, pp. 4556-4565, mmr..
Long Beach, California, USA, 09-15 Jun 2019b. PMLR.
press Stess//v77/metz2aa.no
Luke Metz, Niru Maheswaranathan, Jonathon Shlens, Jascha Sohl-Dickstein, and Ekin D Cubuk. Using learned optimizers to make models robust to input noise. arXiv preprint 140iv:1906.3333 June 2019c.
Anusha Nagabandi, Ignasi Clavera, Simin Liu, Ronald S Fearing, Pieter Abbeel, Sergey Levine, and Chelsea Finn. Learning adapt in dynamic, real-world environments through meta- reinforcement learning. arXiv preprint arXiv:/8033 3.0. 1347, 2018.
Alex Nichol, Joshua Achiam, and John Schulman. On First-Order Meta-Leaming algorithms. arXiv preprint arXiv:1803.008 80.99999,, March 2018.
Eunbyung Park and Junier B Oliva. Meta-Curvature, arXiv preprint arXiv:1902.03356 February 2019.
Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid learning or feature reuse? towards understanding the effectiveness of MAML. arXiv preprint (00i:0:900.0915555 September 2019.
Sachin Ravi and Hugo Larochelle. Optimization as model for few-shot learning. In International Conference on Learning Representations, 2017, 2017.
Jonas Rothfuss, Dennis Lee, Ignasi Clavera, Tamim Asfour, and Pieter Abbeel. Promp: Proximal meta-policy search. arXiv preprint arXiv:1810.06784, 2018.
Jonas Rothfuss, Dennis Lee, Ignasi Clavera, Tamim Asfour, and Pieter Abbeel. ProMP: Proxi- mal meta-policy search. In International Conference on Learning Representations, 2019. URL
https://openrevieww net/ www.ee/ooormmmi-skkX0000000
Juergen Schmidhuber. Evolutionary Principles in Self- Reff-eefreetial Learning. PhD thesis, 1987.
Nicol N Schraudolph. Local gain adaptation in stochastic gradient descent. 1999.
Rich Sutton. Adaptation of learning rate parameters. Goal Seeking Components for Adaptive Intel- ligence: An Initial Assessment, 1981.
Richard Sutton. Gain adaptation beats least squares. In Proceedings of the 7th Yale workshop on adaptive and learning systems, volume 161 168, 1992a.
Richard S. Sutton. Adapting bias by gradient descent: An incremental version of delta-bar-delta. In AAAI, pp. 171-176, 1992b. URL http:://tw/// aaai
aaai 92 027.027.. 027.php
Sebastian Thrun and Lorien Pratt (eds.). Learning to Learn. Kluwer Academic Publishers, Norwell, MA, USA, 1998.
Olga Wichrowska, Niru Maheswaranathan, Matthew W. Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Nando de Freitas, and Jascha Sohl-Dickstein. Learned optimizers that scale and generalize. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 3751-3760, International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR. URL http::/proceedai wttpm.pee///////iiiiiiooiio..
ha.ml
Yuhuai Wu. Mengye Ren, Renjie Liao, and Roger Grosse. Understanding short-horizon bias in stochastic meta-optimizztiin. In International Conference on Learning Representations, 2018.
URL UtW.hhttpp:://ppeeevveee http://pameeeeeeee.eeeerrrrrrrro MczcgR-
12

---

Under review as a conference paper at ICLR 2020
Zhongwen Xu, Hado P van Hasselt, and David Silver. Meta-gradient reinforcement learn- ing. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Besa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems 31, pp. 2396- 2407. Curran Associates, Inc., 2018. URL
n ps. cc cc/paper/
13

---

Under review as a conference paper at ICLR 2020
A APPENDIX
A.1
EXPERIMENTAL DETAILS AND HYPER-PARAMETERS
The following paragraphs describe the data data-generation process, model architectures, and learning for experiments in Section Meta-optimizers's parameter matrices always initialized to the identity.
A.1.1 LINEAR MODEL EXPERIMENTS
Synthetic Binary Classification The task decision boundaries
N(O.0, (100) sampled from standard Gaussian, Each one of the 1,000 data point x ~ NO.0, 1100) also sampled from the standard Gaussian, and is assigned a label from {-1, 1} based on the sign of w x. All models trained for 30,000 iterations of stochastic gradient descent (without momentum) so as to minimize the 1-step MAML loss _MAMI, The task loss C()) the binary cross-entropy. Each stochastic gradient is estimated on single task. (i.e. the meta-batch size is 1.)
For the logistic regression (LR) results, we used meta and adapatation learning rates 0.01 and 0.5. The linear network (LR+LinNet) consists of 3 hidden layers of 64 units, and is trained with both meta- and fast-adaptation learning rates set 1.9. Experiments using the cronecker-factore linear meta-optimizer also use the same meta- and fast-adaptation learning rate, set to 3.33. The meta- optimizer itself consists of 10,001 parameters. (m 100, n = 1) The Kronecker- connected meta-optimizer uses single hidden layer with rectified linear-units (ReL.U) activation functions, for total of 20,002 parameters. (m = 100, n = 1) has both learning rates set to 3.9. None of the meta-optimizers use a bias term.
fully-
Omniglot Our Omniglot experiments exactly replicate the setup of |Fin et al. (2017). Of the 1623 classes, we designate 1,200 for meta-training, 100 for validation, and 423 for testing. We then generate 20,000 training tasks, 600 validation/testing tasks, each consisting of five classes containing a single character image, possibly rotated by 90, 180, or 270 degrees. All models are trained using Adam for 60,000 iterations, an iteration consisting of 32 tasks for which the model is allowed steps of aasttadaptation.
The logistic regression model uses meta-learning rate of 0.0005 and an adaptation learning-rate set to 1.0. The linear network uses hidden layers of 256, 128, 64, and 64 units and flattens the character images into 784-dimensional vector before processing them. The meta-learning rate was set to 0.0005 and the adaptation learning rate to 0.08. For the KLin experiment (LR w/ KLin), we meta-learing rate of 0.003 and the adaptation learning rate to 0.9. The KLin optimizer consists of 614k parameters, with m = 784, 5. The KFC meta-optimizer (LR w/ KFC) consists of 4 hidden layers (2.5M parameters) with ReLU activations, whose meta and adaptation learning rates are set to 0.001 and 0.01, respectively. Again, none of the meta-optimizer use biases.
A. NONLINEAR MODEL EXPERIMENTS
For the non-linear model experiments, we learn one KFC optimizer per layer of the model. Each KFC optimizer is based on the same architecture, but learns its own set of parameters. Moreover, we separate processing of the magnitude and direction the model's gradient as follows: the gradient is initially scaled so as to have unit norm before being fed to the KFC network. Once the normalized gradient has been processed, it is rescaled by the initial normalizing constant times a learnable factor. (This learnable factor is initialized to 1.) We found this architecture to help in situations where the model allowed several adaptation steps. For convolutional models, we flatten the height and width weight dimensions such that the optimizer is shared across filters of same layer. (i.e. if the convolutional layer has shape NxCxHx W, we have C, n HW.) Note that, again, none of the meta-optimizers use bias term
Omniglot The dataset replicates the processing of linear model experiments, detailed above. In- stead of ways and 1 shot, we use 10 ways and 5 shots. All models are allowed steps of fast adaptation.
14

---

Under review as a conference paper at ICLR 2020
The 4-layer CNN network (CNN w/ MAML) uses meta-learning rate of 0.003, and adaptation learning rate of 0.5. Its architecture replicates the one described in Finn et al. (2017). The 2-layer CNN (SCNN w/ MAML) uses the same first two layers and same input dimensionality to the FC layer as the 4-layer CNN, with meta and adapation learning rates set to 0.003 and 0.8, respectively. The 4-layer CNN has total of 112,586 parameters, while the 2-layer only 38,474. We used the same learning rates for the Meta-SGD, Meta-Curvature of 0.003, and adaptation learning rate of 0.5, The KFC architecture consists of 4 layers, such that the meta-optimizers contains a total of 134,171 parameters for the 2-layer CNN model and 267 ,451 parameters for the 4-layer CNN.
and KFC optimizers: a meta learning rate
We obtained the splits created by Bertinetto al. (2019) and exactly reproduced their preprocessing setting for our experiments on CIFAR-FS, We also split the 100 classes 64 training, 16 validation, and 20 testing classes and generate 20,000 training tasks, and 600 validation and testing tasks. We consider the 10-ways 3-shots setup with fast-adaptation steps. As opposed to Bertinetto al.] (2019), our model closely resembles the one of Omniglot experiments. Its main difference with the model described in prior work is the absence of max max-pooling layers, and averaging of the last two convolutional feature dimensions before the fully-connected layer. Doing so allows us to conduct our ablation studies on the effect of our meta-optimizen with different number of convolutional layers, while keeping the input dimensionality of the fully-connected layer constant.
CIFAR-FS
The and 2-layer CNNs use meta- and adaptatinn learning rates 0.003 and 0.7, respectively. The 4-layer CNN has a total of 29,226 parameters, while the 2-layer only 10,602. Meta-SGD, Meta- Curvature, and KFC also use 0.003 meta-learning rate, but adaptation learning rate is decreased to 0.5. The KFC optimizer consists 4 layers with ReLU activations, for a total of 69,325 parameters for the 4-layer CNN model and 35,117 parameters for the 2-layer CNN.
mini- As for the Omniglot experiment, replicate the mini- setting and model from Finn et al. (017)) We consider the 10-ways, 1-shot and 5 fast-adaptation steps set- ting.
The 4-layer CNN and 2-layer CNN both use a meta-learning rate of 0.0005 and fast-adaptation learning rate of 0.07. The 4-layer CNN has total of 36,906 parameters, while the 2-layer only 18,282. The Meta-SGD, Meta-Curvature, and KFC meta-optimizer use the same meta-learning rate, but smaller fast-adaptation learning rate set to 0.1 for Meta-SGD. 0.09 for 0,005 for KFC. For the 2-layer CNN model, the l-yyr KFC optimizer consists of approximately 2.60M parameters, while for the 4-layer CNN model it consists of 2.63M parameters.
and
15

---

Under review as a conference paper at ICLR 2020
A.2
ADDITIONAL EXPERIMENTAL RESULTS
This section provides additional experimental evidence supporting the claims in the main text.
A.2.1
EFFECT OF WIDTH ON ADAPTABILITY
Synthetic
layer
Synthetic
2 loyers
Synthetic
3 layers
Synthetic
4 layers
ndaattddtttiin Acoursyy
1.0
1.0
1.0
NuttAduglation Acouncy
1.0
2.8
0.8
0.n
6.
W-2
9.6
M-16
0.6
W-16
9.6
0.6
8.4
0.4
0.4
0.4
M8-12
128
W-G4
02-
252
0.2
9.2
125
0.
w-125
0- 256
252
0.0
0.0
0.0
0.0
Iterations 104
Iterations
164
Iterations
herattonss
Figure A.1: Effect of width on adaptability on the synthetic binary classification setting. We vary the width and depth of linear networks and report their post-adaptation accuracy. As long as the model has one or more hidden layers, the model is able to adapt to the tasks regardless of the layers' width.
In Section 3 we argue that depth is curcial ingredient enable gradient- based meta-learning. In particular, we show that even on simple, linearly separable binary classification tasks a logistic regression model (i.e. with no hidden layer) fails to solve the meta-learning problem. Using the same toy experiment, we now argue that width plays a relatively less important role with respect to adaptability.
In Figure A.2 we plot the post-adaptation accuracy for different number of hidden layers, and vary their width w = 2, ....... 256. As can be seen, all models are able to eventually solve the meta- learning problem regardless of width, so long as the model has one or more hidden layers.
A.2.2 DO LAYERS C1-C3 BEHAVE SIMILARLY TO FC? ?
Stalig
Wegh
Scllng Fact
Scaling C:
Scalin C2
Scaling C3
Scaling C4
Scalin PC
Figure A.2: Weight scaling experiment for the meta-trained models on Omniglot (top) and CIFAR- FS (bottom). We observe that the modelling layers (CI-C3) are insensitive to parameter scaling regardless of whether it is applied pre- or post-adaptation, indicating they serve modelling purposes. On the other hand, parameter scaling on the adaptation layers (C4 & FC) quickly degrades post- adaptation accuracy if applied pre-adaptation, but has virtually no effect if applied post-adaptation. In turn, this underlines the importance of these layers for adaptation.
In Section 3 we claim that early layers in deep meta- meta-trained trained models are purposed for modelling, while latter layers are responsible for fast-adaptation. This conclusion is reached through the
16

---

Under review as a conference paper at ICLR 2020
gument that (a) layers C1-C3 can be completely omitted from the adaptation dynamics, and (b) perturbing the last FC layer results in large cost-adaptation accuracy degradation.
We now present further evidence that both layer groups (i.e. C1-C3 and C4-FC) indeed serve differ- ent purposes. For each layer of a meta-trained trained model, we multiply the weights of the selected layer by constant factor either pre- post-adaptation. Intuitively, layers that affect the fast-adaptation update will incur a large post-adaptation accuracy penalty as the scaling magnitude increases, due to the value of their weights playing an important part in the computation of the adaptation gradient. On the other hand, a constant scaling of the weights should have little effect on the modelling ability of the network. due to the relative magnitude of activation being maintained. (Note that this intuition is somewhat blurred for our models, as they make use of batch normalization.)
Figure A.2 reports post-adaptation accuracy for those experiments. As predicted. scaling the weights of C1-C3 has little impact on accuracy. whether applied pre- or post-adaptation, Clearly, those layers do not affect the computation of the fast adaptation update. Similarly, scaling the weights of C4-FC post-adaptation does not impact the modelling ability of the network, and post-adaptation accuracy does not suffer. However, the same rescalings prove to be disastrous when applied pre-adaptation; the model is unable to adapt and accuracy drops by as much as 85% for Omniglot and 40% for CIFAR-FS, Evidently, those layers are central to successful fast-adaptation.
A.2.3 ADDITIONAL METHOODS FOR THE NON-LINEAR MODEL EXPERIMENTS
In this section, we detail additional methods and apply them to the non-linear models of Section
Cholesky-factored Meta-Optimizers As alluded in Section 4 the choice of Kronecker decom- position is arbitrary. We now describe an alternative implementation of meta-optimizerss, based on low-rank Chholesky decomposition. The rank Cholesky decomposition consists of ma- trix L Raxr, such that a rank r symmetric positive definite matrix A R=** is expressed as A = LL Using this decomposition, we can write the linear transformation of Section as:
Ag + b (ILL + b= L(I. +b.
where b E [Rnx1 the bias term and g E 1 is the vectorization of weight's gradient. As for the Kronecker product, the latter expression of the linear transformation does not require to construct large n X n matrix. Additionally, multiple linear transformations can be chained or interleaved by activation functions in order to obtain more expressive and adaptable meta-optimizers.
A major attribute of the Cholesky decomposition is its flexibility in terms of memory requirements. 1 force the meta-optimizer have as many weights the model.
For example, by letting This attribute can in turn be a disadvantage: as we show in our experiments below, r plays an important role in the expressivity of the meta-optimizer, and should be treated an additional to tune. A second inconvenience stems from the choice of initialization scheme of the meta-optimizer, for the Kronecker product, we initialize the Kronecker factors to the identity, thus recovering hypergradient descent in the first meta- training batch. However, since the identity matrix In does not lie in the span of low-rank Cholesky factorizable matrices, the initialization scheme becomes another decision left to the practitioner. For our subsequent experiments, we find that letting T. follow a Glorot initialization (i.e. Lij ~ (0, gain / H)) to work well.
Table A1: Post-adaptation accuracies by different meta-optimizers adapting small CNN.
Meta-Learner Architecture
Task
w/ MAML
w/ PRUNE
w/ CFC1
w/ CFC10
W/ KFC
Omniglot
60.94% + 1.54
70.06% 1 8.87
72.62% 1 1.12
79.56% + 1.24
91.13%
0.93
CIFAR-FS
38.44% 0.87
10.00%
0.05
32.11% 1 0.98
47.81 0.91
52.94%
1.33
ming-ena
18.55% 1 0.46
12.20% 0.13
16.13%
1 1.29
21.48% 1 1.57
27.40%
0.60
We compare the low-rank Chotesky-factored meta-optimizers to the Kronecker ones in Table A1 We denote by CFC1 the rank-1 Cholesky-factored fully-connected network, while CFC10 indicates
17

---

Under review as a conference paper at ICLR 2020
the rank-10 10 version. CFC10 contains upproximately many parameters KFC. (c.f. Table ??) While models trained with CFC do not outperform their KFC counter-parts, they offer a competitive alternative to existing methods.
Effect of Pruning We study the effect of post-adaptation pruning of a meta-trained model. Ide- ally, pruning provides an attractive alternative to explicitly separating adaptation from modelling for learning lean models: practitioners could meta-train a larger model, adapt it to the desired target task, and finally prune it. We implement this pipeline by pruning weights smaller than a predefined threshold and report results in Figure A.3] As can be observed, pruning the 4-layer CNN models to obtain the equivalent number of parameters to a 2-layer SCNN can be catastrophic. On Omniglot, the pruned model reaches 70.06% post-adaptation accuracy. more than 20% lower than using meta- optimizers. On CIFAR- FS and mini- ImageNe,,,,,t pruning can barely outperform chance prediction. reaching on average 10.00% and 12.20%, respectively. These results also available in Table A1 in the W/ PRUNE column.
Omniglot
CIFAR-FS
mini- mini-lmageNNet
1.0
1.0
1.0
Pruning
Pruning
0.8
Post-Adaptation Accuracy
Post-Adaptation Accuracy
0.6
0.6
0.4
0.4
0.4
Praning %
Adaptadatiionnn Accuracy
0.0
0.0
0.15 Weight Pruning Threshold
0.35
0.0
0.4
0.05
0.1
0.15
Weight Pruning Threshold
Weight Pruning Threshold
Figure A.3: Effect of pruning weights smaller than given threshold on the 4-layer networks for Omniglot (left), CIFAR-FS (middle), and mini-ImageNet (right). The respective pruning thresholds to obtain the same number of parameters as the SCNN are 0.30 (Omniglot), 0.5 (CIFAR-FS), and 0.075 ImageNe, ). For those pruning values, the accuracies are much lower than the ones obtained by separating the adaptation mechanism from the model.
18

---

Under review as a conference paper at ICLR 2020
A.3 ADDITIONAL VISUALIZATIONS
In this section, we provide visualizations of experimental results complementary to the ones pre- sented in the main text.
Loss C
VpC(0)
V22(0)
&
Optliizrrr
(b) Kronecker product.
(a) Modeel-Lsss-Optimiize Loop.
Figure A.4: Schematic of the model-loss-optimizzr loop and of the Kronecker product.
Synthetic
Omniglot
1.0
1.0
Post-Adaptation Accuracy
0.8
0.8
Accuracy
0.6
0.6
0.4
0.4
LR+ w/ MAML
LR+LinNet w/
0.2
LR w/ KLin
0.2
LR w/ KLin
w/ KFC
LR w/ KFC
0.0
0.0
0.5 Iterations
1.0
1.5
1e4
Iterations
1e4
Figure A.5: Evolution of test accuracies along training for our methods with linear models the synthetic data binary classification (left) and Omniglot (right).
Table A2: Accuracies and standard deviation of Meta-Learning of Non-Linear Models
Omniglot
CIFAR-FS
Num. Layers
CNN
CNN w/ KFC
CNN
CNN w/ KFC
1
29.20% + 1.13
66.45%
+ 1.42
24.86% + 0.47
42.02% + 0.52
2
60.94% + 1.54
91.13% + 0.93
38.44% 0.87
52.94% + 1.33
3
91.69% 0.70
96.57% 0.86
50.91% 0.80
53.07%
0.58
4
97.56% 0.45
98.49% 0.41
55.03%
1.23
55.97% + 0.59
19

---

Under review as a conference paper at ICLR 2020
Omniglot
Omniglot
1.0
0.8
Accuracy
0.6
Accuracy
CNN w/ MAML
CNN w/ MAML
0.4
SCNN w/ MAML
SCNN w/ MAML
SCNN w/ MSGD
SCNN w/ MSGD
0.2
SCNN w/ MC.
0.2
SCNN w/ MC
SCNN w/ KFC
SCNN w/ KFC
0.0
0.0
Iterations
1e4
Iterations
1e4
(a) Omniglot Train Tasks
(b) Omniglot
Test Tasks
Figure A.6: Evolution for the training (left) and testing (right) accuracies of the non-linear models on Omniglot.
CIFAR-FS
CIFAR-FS
1.0
1.0
CNN w/ MAML
0.8
0.8
SCNN w/ MAML
SCNN W/ MSGD
Accuracy
0.6
Accuracy
SCNN W/ MC.
CNN w/ MAML
SCNN KFC
0.4
SCNN w/ MAML
SCNN w/ MSGD
0.2
SCNN w/ MC
SCNN w/ KFC
0.0
Iterations
1e4
Iterations
(a) CIFAR-FS Train Tasks
(b) CIFAR-FS
Test Tasks
Evolution for the training (left) and testing (right) accuracies of the non-linear models on CIFAR-FS.
Figure A.7:
mini-ImageNett
mini-ImageNet
1.0
1.0
CNN w/ MAML
CNN w/ MAML
0.8
SCNN w/ MAML
0.8
SCNN MAML
SCNN w/ MSGD
SCNN w/ MSGD
Accuracy
0.6
SCNN w/ MC
Accuracy
0.6
SCNN w/ MC
SCNN
0.4
2
Iterations
1e4
Iterations
(a) minilmageNet- Train Tasks
(b) minilmageNet Test Tasks
Figure A.8: Evolution for the training (left) and testing (right) accuracies of the non- -ineir models on mini- Imageeem
20

---

Under review as a conference paper at ICLR 2020
Algorithm Meta-Learning with Meta-Optimizers
Require: Fast learning rate a, Initial parameters ((iit) and g(init) Optimizer Opt
1: while (fiii)) c(init) not converged do
:S
2: Sample task T ~ P(T)
3: 01 = (fonit)
((mit)
4: for step I,.... do
5:
Compute loss C, (0.)
6:
Compute gradients Ve, Vo,C, (0t) and Ve,
7:
Update the meta-optimizer parameters fr+1 (V,
8:
Compute the model update U (Vo, (V (0.))
9:
Update the model parameters 8(+1 = Usc4s oV (0.)) end for
10:
Update model and meta-optimizer initializations
12:
B(init) & ((init) Opt V gemns L- (0r)) erinit) g(init) Opt gini (TT))
14:
end while
A.4
COMPUTATIONAL METRICS
This section provides computational metrics for the non-linear experiments Table A3 All metrics are measured using single NVIDIA Titan XP.
Table A3: Computational metrics for each of the methods. The timing metrics measure the time compute one meta meta-gradie graddiega step on an entire meta- neta-batch. Inference time is identical across methods for models of the same size.
Task
Method
Model Parameters
Optimizer Parameters
Time
Accuracy
Omniglot
CNN
W/ MAML
112,586
1.09s
97.56%
W/ KFC
112,586
267,451
10.35s
98.49 %
SCNN
W/ MAMI.
38,474
0.58s
60.94%
w/ KFC
38,474
134,171
5.44s
91.13%
W/ CFC1
38,474
8,905
4.30s
72.62%
W/ CFC10
38,474
66,379
4.36s
79.56%
CIFAR-FS
CNN
w/ MAML
29,226
0.63s
55.03%
w/ KFC
29,226
69,325
2.04s
55.97
SCNN
W/ MAML
10,602
0.25s
38.44%
W/ KFC
10,602
35,117
1.08s
52.94%
CFCI
10,602
4,185
0.83s
32.11% 1%
W/ CFCIO
10,602
34,317
0.84s
47.81 %
mini-ImageNNt
CNN
w/ MAML
36,906
0.25s
29.20%
w/ KFC
36,906
2.63M
1.75s
30.81 %
SCNN
W/ MAML
18,282
0.14s
18.55%
W/ KFC
18,282
2.60M
0.94s
27.40%
W/ CFC1
18,282
42,585
0.69s
16.13%
w/ CFC10
18,282
349,197
0.71s
21.48%
21


Reviews:
[{"review_id": "rygbXAUK9B", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents an experimental study of gradient based meta learning models and most notably MAML. The results suggest that modeling and adaptation are happening on different parts of the network leading to an inefficient use of the model capacity which explains the poor performance of MAML on linear (or small networks) models. To tackle this issue they proposed a kronecker factorization of the meta optimizer.\n\nThe paper is well motivated and well written in terms of clarity in the message and being easy to follow.\n\nOne major issue is that the experimental study is not that comprehensive to support the claim of the paper. Especially, in analyzing the failure case of linear models.For example, one may try small (but nonlinear networks) and compare its performance with larger (possibly overparameterized) ones on at least 2 standard network architectures. But, it doesn't mean that I don't like the paper at its current state. The paper yet has a message and it's delivered clearly.\n\nI wonder if the overparameterized is just related to depth or overparameterization in width would work too? If not then it might be the \"nonlinearity\" that is doing the work\n\nIn section 3.2 (Figure 2, left) and (Figure2, mid) show that FC follows the pattern of C1-C3. t\nThen the authors proposed the experiment related to perturbing FC (Figure 2, right) to show that FC is actually not similar to C1-C3 and is important to adaptation. However, one can do similar experiments for C1-C3 and claim they are also important to adaptation. It seems that FC and C4 are really different.\n\nFor a non-expert reader it's not readily clear that how the kronecker factorization of A leads to equation 5. An explanation can help. Also, a few sentences or schematic demonstration of kronecker product makes the paper self-contained. \n\nThere are a few typos in the paper that can be removed after a thorough proofreading. ", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "rygbTkp-5r", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper analyzes the popular MAML (Model-Agnostic Meta-Learner) method, and thereafter proposes a new approach to meta-learning based on observations from empirical studies. The key idea of the work is to separate the base model and task-specific adaptation components of MAML. This decoupling of adaptation and modeling reduces the burden on the model, thus enabling smaller memory efficient deep learning models to adapt and give high performance on meta learning tasks. The paper proposes a learnable meta-optimizer consisting of a parametrized function U such that the knowledge of adaptation is embedded into its parameters (A,b), instead of forward model parameters. The computational challenges posed by the proposed method are addressed by expressing the parameter matrix A as a Knonecker product of small matrices which is more efficient from memory and time complexity view point. The results on Omniglot and CIFAR-FS are promising, and the paper shows that the proposed meta-optimize is \"more expressive\", as well as can adapt a shallower model to the same level of performance as MAML.\n\n+ves:\n+ The discussion on the deficiency of MAML combined with shallow models is well-supported experimentally. \n+ The idea to leverage the parameters of a meta-optimizer for adaptation instead of using model parameters is novel and interesting.\n+ The paper is well-written and easy to follow. It motivates its choices well, both in the proposed method and the experiments.\n+ The paper presents fair comparison in all experiments with appropriately chosen baseline models, and the proposed approach is validated for both linear as well as non linear models using benchmark datasets.\n\nConcerns:\n- While MAML was a seminal work and is widely followed, there have been many follow-ups of MAML, including another widely used method Reptile (Nichol et al, On First-Order Meta-Learning Algorithms). How is the proposed method relevant more broadly to this genre of methods? Some discussion of this would have been useful to understand the generalizability of the idea.\n\n- The choice of the Kronecker product to handle the dimensionality of the meta-optimizer is supported by the paper, but is not very convincing. How important is this choice? What if other decompositions were used? \n\n- The paper seems to state that shallow models are convex (Sec 3.2); however, weight symmetry induces non-convexity even in shallow models. This perspective of the problem may not be very well-justified.\n\n- In Sec 3.2, the paper compares the 1-step adaptation accuracy of a shallow network and a deeper 4 layered linear network and claim that shallow networks underperform. However this underperformance might be due to the difference in required number of steps to reach optimal performance by the two models, and may not be a fair comparison. Why is this conclusive inference? Considering these inferences motivate the full paper, this is important.\n\n- All the presented results are on small CNNs. The paper motivates this as “easing the computational burden”. The original MAML work shows results on state-of-the-art convolutional and recurrent models. It may be important to show results on deeper models to be more confident about its applicability.\n\n- Although one can obtain smaller meta-learned models using the proposed method, training via this method will incur a higher computational burden than MAML-trained deep models. The paper does not talk about this additional complexity at all. Comparisons of wall-clock times or asymptotic analysis of the proposed method w.r.t. MAML would have greatly helped understand the pros and cons of the method.\n\nI am on the borderline on this work - it is a well-written paper with a clear objective and support. But lack of rigorous analysis of the proposed method in terms of the method (how important is the Kronecker factorization?), experiments (with deeper architectures) and a more generalizable understanding of the proposed idea seems to be limiting the work's impact. \n\n========POST-REBUTTAL COMMENTS===============\nI thank the authors for their response, and all the efforts in the updated manuscript. Some of the clarifications sought were answered clearly. However, unfortunately, I continue to remain on the borderline on this work for the reasons below. (I would be willing to increase my rating to 4 or 5, which however are not available on the drop down, but perhaps not beyond).\n\n- The response to AnonReviewer1 says that \"there have been multiple empirical and theoretical works dedicated solely to the study of MAML [7-11]\", hence supporting this work dedicating its focus to MAML alone. However, on close observation, most of these efforts are not published on peer-reviewed avenues and are only on arXiv at this time. Ref [7] (Finn and Levine, 2017) is published but has significantly stronger contributions. Considering the largely empirical nature of this work, showing its generalizability would be required, in my opinion, to make the conclusions of this work useful to the audience. Expecting that it would naturally hold for other methods like REPTILE may not be sufficient. In my opinion, this is a significant limitation.\n\n- I personally remained unconvinced about the response to the question on number of adaptation steps, as well as on the lack of deeper models in the empirical studies.\n\nI once again appreciate the authors for all the additional efforts, it may just be good for the work to be more comprehensive to be relevant and useful.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJg63_1CYB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper investigated the effect of depth on the meta-learning model.   \nThe paper mainly studies through experimental means and does not have mathematical analysis to demonstrate. In this way of analysis, a large number of experiments are necessary. In addition to ensuring a large number of experiments, it is necessary to ensure the diversity of methods. This article only studied MAML, therefore, the conclusion of the experimental inquiry cannot convince me.\nFor the experimental part, I am afraid the results are also weak. For example, please notice that many meta-learning models have proposed. I believe authors should compare more existing works to demonstrate the superiority of the proposed one.\n\n[Update after rebuttal period]\nIt may seem reasonable that depth enables task-general feature learning. However, in fact, it is not true. The major reason for people to think that the receptive field becomes very large after multiple pooling operation. This is true but not the reason for good performance in feature learning. Because of back-propagation, the feature extraction layers can be trained well to extract features from objects of different scales. The major reason for poor performance in feature learning is that the header that creates an object template is not well trained for objects of different scales. As a result, I still keep the confusion in terms of the effectiveness of the proposed method.  \n\n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a number of experiments involving the Model-Agnostic Meta-Learning (MAML) framework, both for the purpose of understanding its behavior and motivating specific enhancements.  With respect to the former, the paper argues that deeper networks allow earlier layers to learn generic modeling features that can be adapted via later layers in a task-specific way.  The paper then suggests that this implicit decomposition can be explicitly formulated via the use of meta-optimizers for handling adaptations, allowing for simpler networks that may not require generic modeling-specific layers.

At the end of the rebuttal and discussion phases, two reviewers chose rejection while one preferred acceptance.  In this regard, as AC I did not find clear evidence that warranted overriding the reviewer majority, and consistent with some of the evaluations, I believe that there are several points whereby this paper could be improved.

More specifically, my feeling is that some of the conclusions of this paper would either already be expected by members of the community, or else would require further empirical support to draw more firm conclusions.  For example, the fact that earlier layers encode more generic features that are not adapted for each task is not at all surprising (such low-level features are natural to be shared).  Moreover, when the linear model from Section 3.2 is replaced by a deep linear network, clearly the model capacity is not changed, but the effective number of parameters which determine the gradient update will be significantly expanded in a seemingly non-trivial way.  This is then likely to be of some benefit.

Consequently, one could naturally view the extra parameters as forming an implicit meta-optimizer, and it is not so remarkable that other trainable meta-optimizers might work well.  Indeed cited references such as (Park & Oliva, 2019) have already applied explicit meta-optimizers to MAML and few-shot learning tasks.  And based on Table 2, the proposed factorized meta-optimizer does not appear to show any clear advantage over the meta-curvature method from (Park & Oliva, 2019).  Overall, either by using deeper networks or an explicit trainable meta-optimizer, there are going to be more adaptable parameters to exploit and so the expectation is that there will be room for improvement.  Even so, I am not against the message of this paper.  Rather it is just that for an empirically-based submission with close ties to existing work, the bar is generally a bit higher in terms of the quality and scope of the experiments.

As a final (lesser) point, the paper argues that meta-optimizers allow for the decomposition of modeling and adaptation as mentioned above; however, I did not see exactly where this claim was precisely corroborated empirically.  For example, one useful test could be to recreate Figure 2 but with the meta-optimizer in place and a shallower network architecture. The expectation then might be that general features are no longer necessary.
Decision: Reject