R OLE OF T WO L EARNING R ATES IN C ONVERGENCE OF M ODEL - AGNOSTIC M ETA - LEARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Model-agnostic meta-learning (MAML) is known as a powerful meta-learning method.
However, MAML is notorious for being hard to train because of the existence of two learning rates.
Therefore, in this paper, we derived a sufﬁcinent condition of inner learning rate (cid:11) and meta-learning rate (cid:12) for a simpliﬁed MAML to locally converge to local minima from any point in the vicinity of the local minima.
We ﬁnd that the upper bound of (cid:12) depends on (cid:11) .
Moreover, we show that the threshold of (cid:12) increases as (cid:11) approaches its own upper bound.
This result is veriﬁed by experiments on various few-shot tasks and architectures; speciﬁcally, we perform sinusoid regression and classiﬁcation of Omniglot and MiniImagenet datasets with a multilayer perceptron and a convolutional neural network.
Based on this outcome, we present a guideline for determining the learning rates: ﬁrst, search for the largest possible (cid:11) ; next, tune (cid:12) based on the chosen value of (cid:11) .
I NTRODUCTION
A pillar of human intelligence is the ability to learn and adapt to unseen tasks quickly and based on only a limited quantity of data.
Although machine learning has achieved remarkable results, many recent models require massive quantities of data and are designed for solving particular tasks.
Meta-learning, one of the ways of tackling this problem, tries to develop a model that can adapt to new tasks quickly by learning to learn new concepts from few data points (Schmidhuber, 1987;
Thrun & Pratt, 1998).
Among meta-learning algorithms, model-agnostic meta-learning (MAML), a gradient-based metalearning method proposed by Finn et al.
(2017), has recently been extensively studied.
For example, MAML is used for continual learning (Finn et al., 2019;
Jerfel et al., 2019; Spigler, 2019; Al-Shedivat et al., 2018), reinforcement learning (Finn et al., 2017; Al-Shedivat et al., 2018;
Gupta et al., 2018; Deleu & Bengio, 2018; Liu & Theodorou, 2019) and probablistic inference (Finn et al., 2018; Yoon et al., 2018; Grant et al., 2018).
The reason why MAML is widely used is because MAML is simple but efﬁcient and applicable to a wide range of tasks independent of model architecture and the loss function.
However, MAML is notorious for being hard to train (Antoniou et al., 2019).
One of the reasons why training MAML is hard is the existence of two learning rates in MAML: the inner learning rate (cid:11) and meta-learning rate (cid:12) .
A learning rate is known to be one of the most important parameters, and tuning this parameter may be challenging even if the simple gradient descent (GD) method is used.
Nevertheless, we do not yet know the relationship between these two learning rates and have little guidance on how to tune them.
Hence, guidelines for choosing these parameters are urgently needed.
In this paper, we investigate the MAML algorithm and propose a guideline for selecting the learning rates.
First, in Section 2 we brieﬂy explain by using an approximation how MAML can be regarded as optimization with the negative gradient penalty.
Because the gradient norm is related to the shape of the loss surface, a bias towards a larger gradient norm can make training unstable.
Next, based on the approximation explained in Section 2, in Section 3, we derive a sufﬁcinent condition of (cid:11) and (cid:12) for a simpliﬁed MAML to locally converge to local minima from any point in the neighborhood of the local minima.
Furthermore, by removing a constraint, we derive a sufﬁcient condition for local convergence with fewer simpliﬁcations as well.
We ﬁnd that the upper bound (cid:12) c of meta-learning rate depends on inner learning rate (cid:11) .
In particular, (cid:12) c of (cid:11) (cid:25) (cid:11) c is larger than that of (cid:11)
= 0 , where (cid:11) c is the upper bound of (cid:11) .
This is veriﬁed by experiments in Section 5.
These results imply a guideline for selecting the learning rates: ﬁrst, search for the largest possible (cid:11) ; next, tune (cid:12) .
MAML AS OPTIMIZATION WITH NEGATIVE GRADIENT PENALTY 2. MAML
The goal of MAML is to ﬁnd a representation that can rapidly adapt to new tasks with a small quantity of data.
In other words, MAML performs optimization for parameters (cid:18) 2 R d that the optimizer can use to quickly reach the optimal parameter (cid:18) (cid:3) (cid:28) for task (cid:28) with few data points.
To this end, MAML takes the following steps to update (cid:18) .
First, it samples a batch of tasks from task distribution P ( (cid:28) ) and updates (cid:18) for each task (cid:28) with stochastic gradient descent (SGD).
Although
MAML allows multiple-step being taken to update (cid:18) , we will consider the case only one step being taken for simplicity.
The update equation is as follows:
[Equation]
[Equation] ) where (cid:11) is a step size referred to as the inner learning rate, L [Equation] ( (cid:18) ) is the loss of (cid:28) , and ∇ (cid:18) L (cid:28) ( (cid:18) ) is an estimate of the true gradient.
The data used for this update is called training data.
Next, MAML resamples data, referred to as test data, from each (cid:28) and computes the loss at the updated parameters (cid:18) ′ (cid:28) , obtaining L (cid:28) ( (cid:18) ′ (cid:28) ) for each task.
Finally, to determine (cid:18) that can be adapted to (cid:18) ′ (cid:28) for all tasks, (cid:18) is updated with the gradient of a sum of loss values L (cid:28) ( (cid:18) ′ (cid:28) ) over all tasks.
In other words,
[Equation]
where (cid:12) is the learning rate called the meta-learning rate and ∇ (cid:18)
∑ (cid:28) (cid:24) P ( (cid:28) )
L (cid:28) ( (cid:18) ′ (cid:28) ) is an estimate of the true gradient by using the test data.
Though learning rates (cid:11) and (cid:12) can be tuned during training or different for each task in practice, we will think them as ﬁxed scalar hyperparameters.
2. N EGATIVE GRADIENT PENALTY
Unless otherwise noted, we will consider the case of only one step being made per update, and the data are not resampled to compute the loss for updating (cid:18) .
The case of multiple steps and that of training data and test data being separated are considered in Appendix A.
The gradient of the loss at (cid:18) ′ (cid:28) is g (cid:28) ( (cid:18) ′ (cid:28) )
= ∇ (cid:18) L (cid:28) ( (cid:18) ′ (cid:28) )
= ∇ (cid:18) (cid:18)
′ (cid:28) @L (cid:28) @ (cid:18) ′ (cid:28) , where g ( (cid:1) ) is the gradient of L ( (cid:1) ) with respect to (cid:18) .
If (cid:11) is small, we can assume that I @L (cid:28) @ (cid:18) ′ (cid:28) =
g (cid:28) ( (cid:18) ) ; this seems to hold since (cid:11) is usually small (Finn et al., 2017).
Then,
[Equation]
The result but not the procedure with the approximation is the same as that with the well-known ﬁrstorder approximation as long as data are not resampled and only one step being taken.
The ﬁrst order approximation has been mentioned by Finn et al.
(2017) and
extensively studied by Nichol et al.
(2018) and Fallah et al.
It is known that the error induced by the ﬁrst-order approximation does not degrade the performance so much in practice (Finn et al., 2017).
Also, Fallah et al.
(2019) theoretically proved that the ﬁrst-order approximation of MAML does not affect convergence result when each task is similar to each other or (cid:11) is small enough.
For simplicity, we will assume that only one task is considered during training, omitting task index (cid:28) .
Therefore, instead of ∑ (cid:28) (cid:24) P ( (cid:28) )
L (cid:28) ( (cid:18) ′ (cid:28) )
, we will consider L ( (cid:18) ′ )
as the loss of the simpliﬁed MAML.
Since the MAML loss is just a sum of task-speciﬁc loss, extension to the case of multiple tasks being considered is straightforward, as provided in Appendix A.1.1.
Because ∇ (cid:18) ( g ( (cid:18) )
⊤ g ( (cid:18) ))
=
2 H ( (cid:18) )
g ( (cid:18) ) , if we deﬁne ~ L ( (cid:18) ) = L ( (cid:18) ′ ) ,
[Equation]
[Equation]
The above means that the simpliﬁed MAML can be regarded as optimization with the negative gradient penalty.
We will analyze this simpliﬁed MAML loss in Section 3.
It can also be interpreted as a Taylor series expansion of the simpliﬁed MAML loss for the ﬁrst-order term, up to scale:
[Equation]
The fact that the simpliﬁed MAML is optimization with the negative gradient penalty is worth keeping in mind.
Because the goal of gradient-based optimization is to ﬁnd a point where the gradient is zero, a bias that favors a larger gradient is highly likely to make training unstable; this can be a cause of instability of MAML (Antoniou et al., 2019).
In fact, as shown in Fig. 1, the gradient norm becomes larger during training, as do the gradient inner products, as Guiroy et al.
(2019) observed.
[Figure]
L EARNING RATE FOR LOCAL CONVERGENCE
In this section, we will derive a sufﬁcient condition of learning rate (cid:11) and (cid:12) for local convergence from any point in the vicinity of the local minima.
To this end, we will assume that only one step is taken for update and training data and test data are not distinguished as we did in Section 2.
Also, we do not consider SGD but steepest GD to derive the condition.
In other words, same training data are assumed to be used continuously for updating parameters during training.
First, we will consider the case of only single task being considered.
Next, we will consider the case of multiple tasks being considered.
3. S INGLE TASK 3.1. C ONDITION FOR INNER LEARNING RATE (cid:11)
First, we derive the sufﬁcient condition of learning rate (cid:11) .
To this end, we will consider the sufﬁcinet condition that a ﬁxed point is a local minimum.
Taking the Taylor series for the second-order term at a ﬁxed point (cid:18)
[Equation]
, the simpliﬁed MAML loss is
[Equation]
(8) where ~
H = H (cid:0) (cid:11) ( T g + H 2 ) is the Hessian matrix of ~ L ( (cid:18) ) at (cid:18) (cid:3) and g = ∇ (
cid:18) L ( (cid:18) (cid:3) )
2 R d ,
H = ∇ 2 (cid:18) L ( (cid:18) (cid:3) )
2 R d (cid:2) d , and T = ∇ 3 (cid:18) L ( (cid:18) (cid:3) )
2 R d (cid:2) d (cid:2) d .
The calculation of ~ H is presented in
Appendix B. We calculated the magnitudes of T g and H 2 numerically and observed that T g was much smaller than H 2 in practice.
Hence, we will ignore T g while deriving the condition and will thus assume that ~ H = H (cid:0) (cid:11) H 2 .
Further details are provided in Appendix C, and the case of T g being considered is provided in Appendix D. Since P (cid:3) ~
H P ⊤ =
P [ (cid:3) H (cid:0) (cid:11) (cid:3) 2 H ] P ⊤ where (cid:3) ~
H is a diagonal matrix with entries that are eigenvalues of ~ H and P is a matrix with rows that are eigenvectors of ~ H , the sufﬁcinet condition of (cid:11) for (cid:18) (cid:3) to be a local minimum is
[Equation]
Note that (cid:21) ( A ) i represents the i th eigenvalue of matrix A .
Hence, the sufﬁcient condition of (cid:11) for (cid:18) (cid:3) to be a local minimum is
[Equation]
Therefore, (cid:11) c is the inverse of the largest eigenvalue of H .
3.1. C ONDITION FOR META - LEARNING RATE (cid:12)
Next, we derive the sufﬁcient condition of meta-learning rate (cid:12) for the simpliﬁed MAML to locally converge to the local minima discussed above from any point in the vicinity of the local minimum.
This is an extension of research of LeCun et al.
Since P P ⊤ =
I , the simpliﬁed MAML loss can be written as
[Equation]
By using the simpliﬁed loss deﬁned in Eq. 8, the update equation of the parameter (cid:18) with GD is
[Equation]
where t ( t = 0 ; :::; M ) is iteration and M is the total number of iterations.
Hence, (cid:18) ( t + 1) (cid:0) (cid:18) (cid:3)
=
( I (cid:0) (cid:12) ~ H )( (cid:18) ( t ) (cid:0) (cid:18) (cid:3) ) .
If we denote P ⊤ ( (cid:18) (cid:0) (cid:18) (cid:3) ) by v , the simpliﬁed MAML loss in Eq.
12 is ~ L ( v ) (cid:25) ~
L (0)
+ 12 v ⊤ (cid:3) ~
H v .
Because the gradient of ~ L ( v ) for v is ∇ v ~ L ( v ) = (cid:3) ~
H v , the update equation of v is
[Equation]
where v ( t ) is the value of v during iteration t .
Assuming that Eq.
11 holds, the sufﬁcinet condition of (cid:12) is as follows: for all i ,
[Equation]
[Equation] [Equation] (cid:12) [Equation]
[Equation]
{+(cid:0) (cid:11)(cid:21) ( H ) 2 i ) < 1 ( ∵ (cid:21) ( H ) i (cid:0) (cid:11)(cid:21) ( H ) 
+}
i > 0 holds because of Eq. 11 ) (17) )
(cid:12) <
(cid:21) ( H ) i (cid:0) (cid:11)(cid:21) ( H ) 2 i :
[Equation]
Consequently, the sufﬁcient condition for the simpliﬁed MAML to locally converge to local minima from any point in the vicinity of the local minima is as follows:
[Equation]
Vanilla GD with learning rate (cid:12) corresponds to MAML if (cid:11) =
In this case, (cid:12) < 2 (cid:21) max is the condition of (cid:12) , where (cid:21) max is the largest eigenvalue of H , because 2 =(cid:21) max is smaller than any other 2 =(cid:21) i (LeCun et al., 1998).
Though this holds for the simpliﬁed MAML as well, this is not the case if (cid:11) is close to (cid:11) c .
The reason is that (cid:12) c diverges as (cid:11) approaches
[Equation]
( H ) i , or (cid:11) c as Eq.indicates.
Hence, for the simpliﬁed MAML we must consider not only the largest but also other eigenvalues and in particular, the second-largest eigenvalue.
In short, unlike when vanilla GD is employed, (cid:12) c depends on (cid:11) in the case of the simpliﬁed MAML, and (cid:12) c is expected to be larger if (cid:11) is close to (cid:11) c , as shown in Fig. 2.
This ﬁnding is validated by experiments presented in Section 5.
3. M ULTIPLE TASKS
We discussed the case of only one task being available during training in Section 3.1.
In this section, we derive upper bounds of (cid:11) and (cid:12) that apply if multiple tasks (cid:28) (cid:24) P ( (cid:28) ) are considered.
Assumptions, except that multiple tasks are considered, are the same as those in Section 3.1.
. C ONDITION FOR (cid:11)
Now
[Equation]
we deﬁne the simpliﬁed meta-objective as a sum of task-speciﬁc objectives: ^ L ( (cid:18) )
=
∑ (cid:28) (cid:24) P ( (cid:28) )
~ L (cid:28) ( (cid:18) ) .
Then, at a ﬁxed point (cid:18) (cid:3) ,
[Equation]
where ^ H is the Hessian matrix of ^ L ( (cid:18) ) at (cid:18) .
Note that we ignore T (cid:28) g (cid:28) as we did in Section 3.1.1.
Since H (cid:28) (cid:0) (cid:11) H 2 (cid:28) is not necessarily simultaneously diagonalizable for each task (cid:28) , we cannot exactly express eigenvalues of ^ H as function of (cid:21) ( H (cid:28) ) and (cid:11) .
Therefore, instead of the exact value of (cid:11) c , we will derive an upper bound of (cid:11) c .
If (cid:18) (cid:3) is a local minimum for all ~ L (cid:28) ( (cid:18) (cid:3) ) , (cid:18) (cid:3) is a local minimum for ^ L ( (cid:18) ) as well.
Hence, if all eigenvalues of ~ H (cid:28) =
H (cid:28) (cid:0) 2 (cid:11) H 2 (cid:28) are positive for all tasks, those of ^ H are positive as well.
Therefore, if the inequality
[Equation]
holds, it guarantees that the condition that (cid:18) (cid:3) is a local minimum of ^ L ( (cid:18) ) is satisﬁed.
Note that this is a sufﬁcient condition if (cid:18) (cid:3) is a local minimum for all ~ L (cid:28) ( (cid:18) (cid:3) ) .
Since (cid:18) (cid:3) can be a local minimum of ^ L ( (cid:18) ) even when it is not a local minimum for all ~ L (cid:28) ( (cid:18) (cid:3) ) , the upper bound of (cid:11) seems to be larger than that in Eq.
23 in practice.
3.2. C ONDITION FOR (cid:12)
The analysis for (cid:12) is also basically the same as that performed in Section 3.1.2.
Denoting P ⊤ (cid:28) ( (cid:18) (cid:0) (cid:18) (cid:3) ) by v (cid:28) , we obtain ∑
[Equation]
Since ~ H (cid:28) is not always simultaneously diagonalizable for each task, as mentioned in Section 3.2.1,
P (cid:28) differs from task to task in general.
Hence, we will consider an upper bound of (cid:12) c as we did for (cid:11) c in Section 3.2.1.
Accordingly, if both Eq. 23 and
[Equation]
hold for any eigenvalue (cid:21) i of any task (cid:28) , it guarantees that (cid:12) satisﬁes the condition for local convergence.
Therefore, a sufﬁcient for the simpliﬁed MAML to locally converge to local minima from any point in the neighborhood of the local minima in the case of multiple tasks is as follows:
[Equation]
R ELATED W ORKS
Sevral papers have investigated MAML and proposed various algorithms (Nichol et al., 2018;
Guiroy et al., 2019; Eshratifar et al., 2018; Antoniou et al., 2019; Fallah et al., 2019; Khodak et al., [Figure] [Figure] 2019; Vuorio et al., 2018; Finn et al., 2019; Deleu & Bengio, 2018;
Liu & Theodorou, 2019;
Deleu & Bengio, 2018; Grant et al., 2018).
Nichol et al.
(2018) studied the ﬁrst-order MAML family in detail and showed that the MAML gradient could be decomposed into two terms: a term related to joint training and a term responsible for increasing the inner product between gradients for different tasks.
Guiroy et al.
(2019) investigated the generalization ability of MAML.
The researchers observed that generalization was correlated with the average gradient inner product and that ﬂatness of the loss surface, often thought to be an indicator of strong generalizability in normal neural network training, was not necessarily related to generalizability in the case of MAML.
Eshratifar et al.
(2018) also noted that the average gradient inner product was important.
Hence, the authors proposed an algorithm that considered the relative importance of each parameter based on the magnitude of the inner product between the task-speciﬁc gradient and the average gradient.
Although the above studies were cognizant of the importance of the inner product of the gradients, they did not explicitly insert the negative gradient inner product, which is the negative squared gradient norm with simpliﬁcations, as a regularization term.
To consider the simpliﬁed MAML as optimization with a regularization term is a contribution of our study.
Antoniou et al.
(2019) enumerated ﬁve factors that could cause training MAML to be difﬁcult.
Then, they authors proposed an algorithm to address all of these problems and make training MAML easier and more stable.
Behl et al.
(2019), like us, pointed out that tuning the inner learning rate (cid:11) and meta-learning rate (cid:12) was troublesome.
The authors approached this problem by proposing an algorithm that tuned learning rates automatically during training.
Fallah et al.
(2019) studied the convergence theory of MAML.
They proposed a method for selecting meta-learning rate by approximating smoothness of the loss.
Based on this result, they proved that
MAML can ﬁnd an " -ﬁrst-order stationary point after sufﬁcient number of iterations.
On the other hand, we studied the relationship between the sufﬁcient conditions of inner learning rate (cid:11) and meta-learning rate (cid:12) and showed that how the largest possible (cid:12) is affected by the value of (cid:11) .
E XPERIMENTS
In this section, we will present the results of experiments to conﬁrm our expectation that MAML allows larger (cid:12) if (cid:11) is close to its upper bound.
First, we will show the result of linear regression with simpliﬁcations used in Section 3.1.
Because linear regression is convex optimization, the result is expected to exactly match the theory presented in Section 3.1.
Second, to check if our expectation is conﬁrmed in practice as well, we will present results of the practical case without any simpliﬁcation.
In particular, we conducted sinusoid regression and classiﬁcation of Omniglot and MiniImagenet
[Table]
3e-21e1.
[Figure]
L INEAR REGRESSION
We performed a linear regression, where the task is to regress a linear function with scale parameter in the range of [0, 5.0] and bias parameter in the range of [0, 5.0] based on data points in the range of [-5.0, 5.0].
The true function has the same architecture as that of the model.
We employed the steepest gradient descent method to minimize the mean squared loss, where 1 step was taken during update.
Only one task was considered during training and the same data was used to update the task-speciﬁc parameter and the meta parameter as we did in Section 3.1.
Using these settings, we computed the training loss after 500 iterations with (cid:11) in the range of [1e-5, 9e-2] and (cid:12) in the range of [5e-3, 9e+0].
The eigenvalues are those of the Hessian matrix of the training loss at the end of the training, where (cid:11) =
5e-2 and (cid:12) =
7e-1.
We chose this training loss because it was thought to be the closest to minima.
Fig.
3 shows the training losses at various values of (cid:11) and (cid:12) .
Horizontal axis indicates (cid:11) and vertical axis indicates (cid:12) .
The curves are (cid:12) c of two eigenvalues and the dashed line shows (cid:11) c .
In the case of linear regression with simpliﬁcations used in Section 3.1, the result of numerical experiment shows good agreement with upper bounds of (cid:12) c and (cid:11) c that we derived in
Section 3.1, as shown in Fig. 3.
S INUSOID R EGRESSION
We conducted a sinusoid regression, where each task is to regress a sine wave with amplitude in the range of [0.1, 5.0] and phase in the range of [0, (cid:25) ] based on data points in the range of [-5.0, 5.0].
A multilayer perceptron with two hidden units of size 40 and ReLU was trained with SGD.
The batch size of data was 10, the number of tasks was 100, and 1 step was taken for update.
Using these settings, we computed the training loss after 500 iterations with (cid:11) in the range of [1e-4, 9e-1] and (cid:12) in the range of [1e-2, 9e-0].
Fig.
4 (a) shows the training losses with various values of (cid:11) and (cid:12) .
The dashed line indicates (cid:12) of (cid:11) = 0 over which training loss diverges.
According to Fig.
4 (a), if (cid:11) is close to the value above which the losses diverge, a larger (cid:12) can be used.
As explained above, we did not put any simpliﬁcation as we did in Section 3 and 5.1 in the experiment, meaning that we used different data for updating task-speciﬁc parameter and meta parameter, considered multiple tasks, and employed not steepest GD but SGD as optimizer.
Despite simpliﬁcations, surprisingly, this result conﬁrms the expectation that MAML allows larger (cid:12) if (cid:11) is close to (cid:11) c .
5. C LASSIFICATION
We performed classiﬁcation of the Omniglot and MiniImagenet datasets (Lake et al., 2011; ravi & Larochelle, 2017), which are benchmark datasets for few-shot learning.
The model used was essentially the same as that Finn et al.
(2017), and hence, Vinyals et al.
(2016) used.
The task is a ﬁve-way one-shot classiﬁcation, where the query size is 15, the number of update steps is two, and the task batch size is 32 for Omniglot and four for MiniImagenet.
In this setup, we computed the MiniImagenet dataset with various values of (cid:11) and (cid:12) ; for Omniglot, (cid:11) was in the range of [1e-3, 9e-0] and (cid:12) was in the range of [1e-1, 9e+1], and for MiniImagenet, (cid:11) was in the range of [1e-4, 9e-1] and (cid:12) was in the range of [1e-2, 9e-0].
Fig.
4 (b) and (c) show the training losses of classiﬁcation task at various values of (cid:11) and (cid:12) .
The dashed line indicates (cid:12) of (cid:11) = 0 above which training loss diverges.
As shown in Fig.
4 (b) and (c), the maximum (cid:12) is larger at large (cid:11) .
Even though the model architecture is composed of convolutional layer, max-pooling, and batch normalization (Ioffe & Szegedy, 2015) and practical dataset is used for training, our expectation is conﬁrmed in this case as well.
This result conﬁrms that our theory is applicable in practice.
Our experimental result conﬁrms that larger (cid:11) is good for stabilizing MAML training.
According to Fig.
4 (a), (b) and (c), moreover, while large (cid:12) does not necessarily make training loss smaller, employing large (cid:11) leads to smaller training loss comparatively.
These result has a practical implication for tuning the learning rates: ﬁrst, the largest possible (cid:11) should be identiﬁed, and (cid:12) may be subsequently tuned based on the value of (cid:11) .
Once you identify (cid:11) c , MAML is likely to work well even if meta-learning rate is roughly chosen.
Taking large (cid:11) is desirable for the goal of MAML as well.
The aim of MAML is to ﬁnd a good initial parameter which quickly adapt to new tasks and the quickness is determined by inner learning rate (cid:11) because (cid:11) determines the step size from initial parameter to task-speciﬁc parameter when model is ﬁne-tuned.
Therefore, identifying (cid:11) c is not only good for robustifying the model against divergence but also good for ﬁnding good initial parameter.
C ONCLUSIONS
We regard a simpliﬁed MAML as training with the negative gradient penalty.
Based on this formulation, we derived the sufﬁcient condition of the inner learning rate (cid:11) and the meta-learning rate (cid:12) for the simpliﬁed MAML to locally converge to local minima from any point in the vicinity of the local minima.
We showed that the upper bound of (cid:12) required for the simpliﬁed MAML to locally converge to local minima depends on (cid:11) .
Moreover, we found that if (cid:11) is close to its upper bound (cid:11) c , the maximum possible meta-learning rate (cid:12) c is larger than that used while training with ordinary
SGD. This ﬁnding is validated by experiments, conﬁrming that our theory is applicable in practice. According to this result, we propose a guideline for determining (cid:11) and (cid:12) ; ﬁrst, search for (cid:11) close to (cid:11) c ; next, tune (cid:12) based on the selected value of (cid:11) . A N EGATIVE GRADIENT PENALTY : G ENERAL CASE
In Section 2, we explained the simplest case of training data and test data being the same, only one task being considered and only one step being taken for update.
In this section, we analyze the cases of waiving one of these simpliﬁcations.
Here, we will interpret the simpliﬁed MAML loss as a Taylor series expansion of the MAML loss for the ﬁrst-order term.
A. W HEN THE TRAINING DATA AND TEST DATA ARE DIFFERENT
When using different data to compute the losses for updating the task-speciﬁc parameters and the meta parameters as done in practical applications, the simpliﬁed loss is as follows:
[Equation]
where L train and L test represent the loss values for training and test data, respectively, and g train and g test are gradients of these losses.
A.1. W HEN MULTIPLE TASKS ARE CONSIDERED
When multiple tasks are considered, as MAML does, the meta-loss ^ L ( (cid:18) ) is just a sum of losses ~
L (cid:28) ( (cid:18) ) for all tasks (cid:28) :
[Equation]
[Equation]
A. W HEN k STEPS ARE
If task-speciﬁc parameters are updated with k-step SGD, the loss can be written as follows;
[Equation]
Note that L i ( (cid:18) ) is the loss computed with the data at the i th step, and g i ( (cid:18) ) is the gradient of the loss.
B C ALCULATION OF ~ H
Because ~ H is the Hessian matrix of ~
L at (cid:18) (cid:3) , we derive the Hessian of Eq. 8.
Then,
[Equation]
C M AGNITUDE OF T g AND H
We conducted a sinusoid regression with essentially the same condition that we explain in Section 5.2 except that the total number of iterations is 50000 and learnig rates are ﬁxed.
Parameters (cid:11) and (cid:12) are 1e-2 and 1e-3 respectively.
We calculated T g numerically with the training error at the end of the training.
As we showed in Section 3, especially large eigenvalues of ~ H are important for the upper bounds of learning rates.
Therefore, if (cid:21) ( T g + H 2 ) max (cid:25) (cid:21) ( H 2 ) max , we can ignore T g when deriving the condition.
We calculate the maximum and the second-largest eigenvalues of T g ,
H 2 and T g + H 2 of the trained model.
As shown in Fig.
5 (a), (cid:21) ( T g + H 2 ) max is almost equal to
(cid:21) ( H 2 ) max , and (cid:21) ( T g ) max is by far smaller than them.
Therefore, ignoring (cid:21) ( T g ) max is reasonable when the conditions are derived.
Furthermore, we calculate the Frobenius norm of T g and H 2 .
As
Fig.
5 (b) indicates, the Frobenius norm of T g is much smaller than that of H 2 , meaning that T g is negligible in the sense of the magnitude of the norm as well.
These results conﬁrm that we can neglect T g when considering ~ H .
[Figure]
D C ONVERGENCE CONDITION WHEN T g IS CONSIDERED
We assumed that T g was negligible in Section 3.
In this section, we derive a sufﬁcient condition of (cid:11) and (cid:12) for the simpliﬁed MAML to locally converge to local minima from any point in the vicinity of the local minima under some assumptions when T g is considered.
D. C ONDITION FOR INNER LEARNING RATE (cid:11) TO SATISFY
When T g is considered, the Hessian matrix of the simpliﬁed MAML loss ~ L is ~ H = H (cid:0) (cid:11) (
T g + H 2 ) .
Because H (cid:0) (cid:11) ( T g + H 2 ) is a real symmetric matrix, it can be diagonalized.
Then, the sufﬁcient condition that a ﬁxed point (cid:18) (cid:3) is a local minimum is
[Equation]
Note that T g and H are not simultaneously diagonalizable, so we cannot decompose (cid:21) ( ~ H ) i into eigenvalues of each matrix as we did in Section 3.
Therefore, we have to consider the relationship among (cid:21) ( ~ H ) i , (cid:21) ( H ) i and (cid:21)
( T g )
i .
In general, it is known that n (cid:2) n Hermitian matrices A and B satisfy the following equation (Bhatia, 2001):
[Equation]
37) where (cid:21) ( A ) represents a vector with elements that are eigenvalues of A , " indicates the operation of sorting a vector in the ascending order, and # indicates that in the descending order.
If two real vectors x ; y 2 R d are related in the following way, x is said to be majorized by y and the relationship is written as x ≺ y :
[Equation]
[Equation]
We deﬁne A = H (cid:0) (cid:11) H 2 and B = (cid:0) (cid:11) T g ; then,
[Equation]
A +
[Equation]
[Equation]
[Equation]
Let us suppose that for top k eigenvalues of A , T g , and A + B , conditions (cid:21) ( A + B ) i (cid:25) c
A + B , (cid:21) ( T g ) i (cid:25) c
T g and (cid:21) ( A ) i (cid:25)
c
A hold, where
c A + B , c
A and c T g are some constant values.
Then, (cid:21) ( A ) i (cid:0) (cid:11)(cid:21)
( T g )
i
(cid:20) (cid:21) (
A + B ) i for 1 (cid:20) i (cid:20)
k
[Equation]
H
[Equation]
H [Equation] [Equation] (cid:11)(cid:21)
[Equation]
T g [Equation] (cid:20) [Equation]
H [Equation] ( T g + H [Equation]
[Equation]
for 1 (cid:20) i (cid:20) k: (44)
The sufﬁcient condition for a ﬁxed point (cid:18) (cid:3) to be a local minimum is
[Equation]
Now, we assume that all eigenvalues of H below a threshold are 0 ; (cid:21) ( H ) i (cid:25) 0 for k < i
(cid:20) n .
In fact, many eigenvalues of the loss computed by a trained model such as a deep neural network are known to be very small.
Thus, if the following condition is satisﬁed, it is enough to say that the condition is satisﬁed: {
[Equation]
In other words, if the smaller d (cid:0) k eigenvalues of T g are not negative values, the ﬁxed point is not guaranteed to be a minimum, and if they are negative, satisfying Eq.
49 is sufﬁcient.
D. C ONDITION FOR META LEARNING RATE (cid:12) TO SATISFY
Next, we derive a sufﬁcient condition of meta-learning rate (cid:12) .
Our analysis will be based on the assumptions identical to those mentioned above.
If the inequalities in Eq.
49 hold, it is sufﬁcient for (cid:12) to satisfy
[Equation]
for the condition of local convergence.
E E XPERIMENT WITH A DAM
We conducted the same experiment as that explained in Section 5.2 and 5.3, but this time, we used
Adam optimizer instead of SGD for the meta-optimizer (Kingma & Ba, 2014).
The results are different from those we showed in Section 5.
For training with Adam, MAML no longer allows us to use larger (cid:12) c .
Because Adam has more parameters that we have to consider than SGD does, this result is not at all surprising.
Nonetheless, it is important to keep these facts in mind when Adam is used for MAML optimization.
[Figure]