Robust Meta-learning with Noise via Eigen-Reptile
Authors: Dong Chen
Lingfei Wu
Siliang Tang
Fangli Xu
Juncheng Li
Chang Zong
Chilie Tan
Yueting Zhuang
Venue:  (2021)
Keywords: meta-learning
few-shot learning
generalization

Abstract: Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-learner's initial model is prone to meta-overfit, as there are only a few available samples with sampling noise. Besides, when handling the data sampled with label noise for FSL, meta-learner could be extremely sensitive to label noise. To address these two challenges that FSL with sampling and label noise. In particular, we first cast the meta-overfitting problem (overfitting on sampling and label noise) as a gradient noise problem since few available samples cause meta-learner to overfit on existing examples (clean or corrupted) of an individual task at every gradient step. We present Eigen-Reptile (ER) that updates the meta-parameters with the main direction of historical task-specific parameters to alleviate gradient noise. Specifically, the main direction is computed by a special mechanism for the parameter's large size. Furthermore, to obtain a more accurate main direction for Eigen-Reptile in the presence of label noise, we propose Introspective Self-paced Learning (ISPL) that constructs a plurality of prior models to determine which sample should be abandoned. We have proved the effectiveness of Eigen-Reptile and ISPL, respectively, theoretically and experimentally. Moreover, our experiments on different tasks demonstrate that the proposed methods outperform or achieve highly competitive performance compared with the state-of-the-art methods with or without noisy labels.

Under review as a conference paper at ICLR 2021
ROBUST META-LEARNING WITH NOISE VIA EIGEN-REPTILE
Anonymous authors Paper under double-bling review
ABSTRACT
Recent years have seen a surge of interest in meta-learning techniques for tackling the few-shot learning (FSL) problem. However, the meta-lcarner's 's initial model is prone to meta-overfit, as there are only a few available samples with sampling noise. Besides, when handling the data sampled with label noise for FSL, meta- learner could be extremely sensitive to label noise, To address these two challenges that FSL with sampling and label noise. In particular, we first cast the meta meta-overfitting problem (overfitting on sampling and label noise) as a gradient noise problem since few available samples cause meta-learner to overfit existing examples (clean corrupted) of an individual task at every gradient step. We present Eigen-Reptile (ER) that updates the meta- meta-parameten with the main direction of historical task- specific parameters to alleviate gradient noise. Specifically, the main direction is computed by a special mechanism for the parameter's large size. Furthermore, to obtain a more accurate main direction for Eigen-Reptile in the presence of label noise, we propose Introspective Self-paced Learning (ISPL) that constructs a plurality of prior models to determine which sample should be abandoned. We have proved the effectiveness of Figen-Reptile and ISPI., respectively, theoretically and experimentally. Moreover, our experiments on different tasks demonstrate that the proposed methods outperform or achieve highly competitive performance compared with the state-of-the-ar methods with without noisy labels.
INTRODUCTION
Meta-learning. also known as learning to learn, is the key for few-shot learning (FSL) (Vinyals et al. 2016 Wang et al., 2019a). One of the meta-learning methods is the gradient-based method, which usually optimizes meta-part parameters as initialization that can fast adapt to new tasks with few samples. However, fewer samples mean a higher risk of meta-overfitting, as the ubiquitous sampling noise in mini-batch cannot be ignored. Moreover, existing gradient-based meta-learning methods are fragile with few samples. For instance, a popular recent method, Reptile (Nichol et al. 2018), updates the meta-parameter towards the inner loop direction, which is from the current initialization to the last task-specific parameters. Nevertheless, as shown by the bold line of Reptile in Figure [ with the gradient update at the last step, the update direction of meta-parameters has significant disturbance, as sampling noise leads the meta-parametenn to overfit on the few trained samples at gradient steps. Many prior works have proposed different solutions for the meta-overfitting problem, such as using dropout (Bertinetto et al. 2018 Lee et al. 2020), and modifying the loss function (Jamal & Qi 2019) etc., which stay at the model level. This paper casts the meta-overfitting problem as a gradient noise problem that from sampling noise while performing gradient update (Wu et al. 2019) Neelakantan al. 2015)) ,ete have proved that adding additional gradient noise can improve the generalization of neural networks with large samples. However, it can be seen from the model complexity penalty that the generalization of the neural network will increase when the number of samples larger. To certain extent adding gradient noise is equivalent increasing the sample size. As for FSL, there are only a few samples of each task. In that case, the model will not only remembers the contents that need to be identified but also overfits the noise (Zhang et al. 2016).
High-quality manual labeling data often time- consuming and expensive. Low-cost approaches to collect low-quality annotated data, such as from search engines, will introduce label noise. Moreover, training meta-learner requires large number of tasks, so that it is not easy to guarantee the quality of data. Conceptually, the initialization learned by existing meta-learning algorithms severely

---

Under review as a conference paper at ICLR 2021
Gradient noise
Alleviate gradient noise
Sampling noise
Label noise
Update with main direction
eigenvector
Cat 1
Cat
Nolsy label Dog
eigenvector
Reptile
FSL with noisy labels
Eigen-Reptile
Meta-parameters
Task-specific parameters
Parameters generated by the naisy labels
Inner loop steps of Reptile, Eigen-Reptile. Reptile updates meta- peta-paraeeters towards the last task-specific parameters, which is biased. Eigen-Reptile considers all samples more fair. Note that the main direction the eigenvector corresponding to the largest eigenvalue.
Figure 1:
degrade in the presence of noisy labels. Intuitively, as shown in FSL with noisy labels of Figure [1 noisy labels cause a large random disturbance in the update direction. It means that label noise (Frénay & Verleysen 2013) leads the meta-learner to overfit on wrong samples, which can be seen as further aggravating the influence of gradient noise. Furthermore, conventional algorithms about learning with noisy labels require much data for each class (Hendrycks et al. 2018, Patrini et al.) 2017). Therefore, these algorithms cannot be applied noisy FSL problem, since few available samples per class. So it is crucial to propose a method to address the problem of noisy FSL.
In this paper, we propose Eigen-Reptin (ER). In particular, as shown in Figure T Eigen- updates the meta- perameersssseess with the main direction of task-specific parameters that can effectively alleviate gradient noise. Due to the large scale neural network parameters, it is unrcalistic to compute historical parameters' eigenvectors. We introduce the process of fast computing the main direction into FSL, which computes the eigenvectors of the inner loop step scale matrix instead the parameter scale matrix. Furthermore, we propose Introspective Self-paced Learning (ISPL), which constructs multiple prior models with randomly sampling. Then prior models will discard high loss samples from the dataset. We combine Reptile with ISPL address the noisy FSL problem, as ISPL can improve the main direction computed with noisy labels.
Experimental results show that Eigen-Reptile significantly outperforms the baseline model by 5.35% and 3.66% on corrupted Mini-ImageNet of 5-way 1-shot and clean Mini-ImageNe of 5-way 5- shot, respectively. Moreover, the proposed algorithms outperform or are highly competitive with state-of-the-ar methods on few-shot classification tasks.
The main contributions of this paper can be summarized as follows:
We cast the overfitteniitiing issue (overfitting sampling and label noise) as gradient noise issue under the meta-learning framework.
We propose Eigen-Reptile that can alleviate gradient noise effectively. Besides, we propose ISPL, which improves the performance of Eigen-Reptile in the presence of noisy labels.
The proposed methods outperform or achieve highly competitive performance compared with the state-of-the the-art methods on few-shot classification tasks.
RELATED WORK
There are three main types of meta-learning approaches: metric-based meta-learning approaches
(Ravi & Larochelle, 2016: Hochreiter al. 2001: Andrychowicz et al.
2016
Liu et al. 2018
Santoro et al.] 2016), model--ased meta-learning approaches ( (Vinyals et al.
2016 Koch et al. 2015
Mordatch| 2018 Sung et al. 2018 Snell al. 2017 Oreshkin et al. 2018
Shyam al. 201 77) and
2

---

Under review as a conference paper at ICLR 2021
gradient-based meta-learning approaches (Finn et al. 2017: Nichol et al. 2018 J Jamal & Qi 2019 Zintgraf et al.] 2018: Li et al., [017]] Rajeswaran et al. [019]] Finn et al. 2018 In this paper, we focus on gradient-based meta- veeta-lerning approaches which can be viewed as the bi- livle loop. The goal of the outer loop is to update the vneta-parameterss on a variety of tasks, while task-specific parameters learned through only small amount of data in the inner loop. In addition, some algorithms achieve state-of-the-e of-t results by additionally training a 64-way classification task on meta meta-training
set (Yang al. 2020 Hu et al. 2020), we do not compare these algorithms for fairness.
Meta-Learning with overfitting, Due to too few samples, meta-leamer inevitably tends to overfit in FSL. Zintgraf et al. (2018) introduces additional context parameters to the model's parameters. Furthermore, |Bertinetto et al. (2018) find that regularization
which can prevent meta- such as dropout can alleviate meta-overfitting their prior work; Jamal & Qi 2019) propose a novel paradigm of Task-Agnostic Meta-Learnin (TAML., which uses entropy or other approaches to minimize the inequality of initial losses beyond the classification tasks to improve the generalizability of meta-learnet All these methods stay at the model level. However, problem from the gradient aspect. We propose Eigen- Reptile, which updates the meta-parameters by the main direction of task-specific parameters to alleviate meta-learner overfit noise.
solve the meta-overfitiing
Learning with noisy labels. Learning with noisy labels has been a long-standing problem. There many approaches to solve it, such studying the denoise loss function (Hendrycks et al. 2018
Patrini et al. 2017: 10177000 Jindal et al. 2016 Patrini et al. 2017: Wang et al. 2019b] Arazo et al. 2019).
relabeling (Lin al. 20144, and on. Nevertheless, most of these methods require much data for each class, Gao et al. (2019) proposes model for noisy few-shot relation classification but without good transferability. For noisy FSL, a gradient-based meta-learner is trained to optimize an initialization on various tasks with noisy labels. As there are few samples of each class, the traditional algorithms for noisy labels cannot be applied. When the existing gradient- based meta- meta-learning algorithms, such as Reptile, update meta peeeta-rr,,,ers, they focus on the samples that generate the last gradient step. And these samples may be corrupted, which makes the parameters learned by meta-learner susceptible to noisy labels. For noisy FSL., we propose ISPI. based on the idea of Self-paced Learning (SPL) (Kumar et al. 2010: Khan et al. 2011 Basu & Christensen] 2013 Tang et al. (012)) to learn more accurate main direction for
ISPL constructs prior models to decide which sample should be discarded when train task-specific models, and this process can In contrast, the model with SPL learns the samples gradually from easy to complex, and the model itself decides the order.
be regarded as the introspective process of
3 PRELIMINARIES
Gradient-basse meta- leeanne aims learn set of initialization parameters that can be adapted to new tasks after few iterations. The dataset D is usually divided into the meta-training set D Dmeta-train and meta-testing set Dmeta-test> which are used to optimize meta-parameters and evaluate its generalization, respectively. For meta-training, we have tasks {TT) drawn from task distribution P(T), each task has its own train set D Dtrain and test set Dteat, and the tasks in meta- testing D metaa-te test are defined in the same way. Note that there are only small number of samples for each task in FSL. Specifically, the N-way K-shot classification task refers to K examples for each of the N classes. Generally, the number of shots used in meta-training should match the one used at meta test-time to obtain the best performance (Cao et al. 2019). In this paper, we will increase the sample size appropriately to get the main direction of individual tasks during meta-training. To minimize the test loss T. (Dreet: Φ) of an individual task, meta- meta- parameters need to be updated n times to get good task-specific parameters क. That is minimizing
B
[(testt: 3)
Σ
E
1 K
Σ
logg (i (1=((i==zz000)
o U (""((rrain +) (I)
N
(D.v)
where 7T represents inner loop steps through gradient descent Adam (Kingma & Ba 2014 on
x, Φ T) is the predictive distribution. When considering updating the meta- pattmpaareeeers in the outer loop, different algorithms have different rules. the case of Reptile, after n inner loop steps, the meta- sitittapprmmterrss can be updated as: & +- & + B(Ø - Φ), where 3 is a scalar stepsize hyperparameter that controls the update rate of meta-parameters.
batches from Dtrain: 9
3

---

Under review as a conference paper at ICLR 2021
4 EIGEN-REPTILE FOR CLEAN AND CORRUPTED DATA
The proposed Rigen-Reptile alleviates gradient noise to alleviate meta-learner overfitting on sampling and label noise. Furthermore, ISPL improves the performance of Eigen-Reptile in noisy FSL.
4.1 THE
ALGORITHM
To alleviate gradient noise to improve the generalizability of meta-learner, we propose Reptile, which updates d meta-paraaa perameesssssters with the main direction of task-specific parameters. We train the task-specific model with n inner loop steps that start from the meeaa-arameters & with few examples. Let i-th column W., Rdxx of parameter matrix W E Rd be the parameters after i-th gradient update, e.g.. W. U (Ø). And treat W., as d-dimensional parameter point w; in the parameter space. e E pdx1 is unit vector that represents the main direction of parameter points in W. Intuitively, projecting all parameter points onto e should retain the most information.
We represent the parameter points by a straight line of the form straight line passes through the mean point w and the signed distance of a point w from w is I. Then + L,e w 11 And determine the signed
+ le, which shows that the
we get the loss function ((,,, 42. distance of each point by partially differentiating J with respect to 4, we get I = (w w. Plugging in this expression for 1 in J, we get
.n.e) e) Ei=1
J(e) - E T (w
w www w
e
w w 1| = - Se Σ wi w
1|
(2)
where S - (w; w) (w; w is scatter matrix. According to Eq minimizing J is equivalent to maximizing: e Se. Note that needs to be roughly consistent with the gradient update direction V in the process of learning task-specific parameters. Use Lagrange multiplier method as
n/
Ve>0 0 Te===
1
max T Se
s.t.
where
V
1n/2
04-i41
w;;
(3)
We get the objective function
g(M, A, 1) E eTSe Se aeeJe- e 1) -(-Ve
where A
(4)
then partially differentiating g in Eq with respect to μ, e, X, 7,
-Ve n²² =0
28e 2Xe nV 0
(5)
eT e 1 0
2pm 0
According 5 if 1) O, then V and are orthogonal, which obviously does not meet expectations. So we get 7 * O, and μ = 0. Then Se = >e. We can see is the eigenvector of S corresponding the largest eigenvalue X, Eq Bis is not directly related to the e, in ER, Eq must retained because it determines the update direction of the outer-loop. Otherwise, the algorithm will not converge.
need the main direction. It should be noted that
concerned question about Se = Xe is that the scatter matrix $ E Raxd grows quadratically with the number of parameters d. As the high dimensionality of parameters typically used in neural networks, computing cigenvalue and eigenvector of S could come at a prohibitive cost (the worst-case complexity is O (d*) ). Centralize W by subtracting the mean W. and scatter matrix
WWT
To avoid calculating the eigenvector of S directly, we focus on W W. As W Wē = āě. Multiply both sides of the equation with W.
WW We
X
We
It can be found from 6 that W T W Ruxn and TWT T E Radd have the same eigenvalue,
A À. Furthermore, we get the eigenvector of WWT We. The main advantage of Eg 6
4

---

Under review as a conference paper at ICLR 2021
that the intermediate matrix W W now grows quadratically with the inner loop steps. As interested in FSL, n is very small. It will be much easier to compute the eigenvector O (n²). which can be ignored (detailed analysis refer to Appendix B Then get the eigenvector of W w based on Moreover, we project parameter update vectors W1+1 - w 1, 2, n nn- on to get the corresponding update stepsize so meta-parameten ø can be updated as
W W,
+ 3iee
(7)
where 33 is a scalar stepsize hyperparameter that controls the update rate of partamarsssssrss... The Figen-Reptil algorithm is summarized in Algorithm of Appendix A. To illustrate the validity of
theoretically, we present Theorem [ as follow:
Theorem Assume that the gradient noise variable follows Gaussian distribution (Hu et al. 2017 Jastrzebski et al. 2017- Mandt et al. 20166) x ~ N (0, o²). Furthermore, I and neural network parameter variable assumed to be uncorrelated. The observed covariance matrix C equals noiseless covariance matrix C₁ plus gradient noise covariance matrix Cz. Then, we get
(8) where Pt and P are the orthonormal eigenvector malrices of Ct and C respectively, At and A are the corresponding diagonal eigenvalue matrices, and I an identity malrix. It can be seen from Eq that C and C₁ has the same eigenvectors. We defer the proof to the Appendix C
C= = C = PA + As)P. = P,(A, + o. = P.AP, P.AP, PAP
Theorem shows that eigenvectors not affected by gradient noise. So Eigen-Reptile can find a more generalizable starting point for new tasks without overfitting on noise.
4.2 THE INTROSPECTIVE SELF-PACED LEARNING
Self-paced learning (SPL) learns the samples from low losses to high losses, which is proven beneficial in achieving a better generalization result (Khan et al.) 2011: Basu & Christensen
2013 TTang et al. 2012). Besides, losses of samples are determined by the model itself. Nevertheless, in meta-learning setting, meta-learner is trained on various tasks, and initial model may have lower losses for trained classes and higher losses for unseen classes or noisy samples. For this reason, cannot train the task-specifie model by the same way as SPL to solve noisy FSL problem. In this paper, we multiple prior models ples and decide which should be abandoned. As shown Figure 2, even though the two categories of yellow and green show an excellent distribution that can be well separated, some samples are marked wrong. To address this noisy label problem, we build three prior models. Specially, we randomly sample three times, and model I is trained with corrupted label. Due to different samples learned by prior models, building multiple models to vote on the data will obtain more accurate losses. Such learning process is similar to human introspection, and we call it Introspective Self- Learning (ISPL). Morcover, samples with losses above a certain threshold will be discarded. Furthermore, we imitate SPL to add the hidden variable 0 or 1 that is decided by Q prior models before the loss of each sample to control whether the sample should be abandoned. So we get the task-specific loss as
Noisy labels
True samples
Sample model
| Sample for model
Sample model
Figure 2: Randomly sample examples build different prior models.
LISPL (Φ,
Σ L (#i, Vs. Φ), where Us arg min
Lj (Z1, Bis Φ2) -ou
(9)
1=1
where ht is the number of samples from dataset Dirain. 7 is the sample selection parameter, which gradually decreases, parameter of model j is Φb (U"(D,,)), D, Dtrain- Note that we update the meta- peetappasseeerss with the model trained on samples from Dtrasn. The ISPL is summarized in Algorithm of Appendix A. Intuitively. it is difficult to say whether discarding high-loss samples containing correct samples and wrong samples will improve the accuracy of eigenvector, so we will Theorem 2 to prove the effectiveness of ISPL.
5

---

Under review as a conference paper at ICLR 2021
ER of iteration
(b) ER of iteration 15000
(c) of iteration 30000
(d) Reptile of iteration
(e) Reptile of iteration 15000
(f) Reptile of iteration 30000
Figure 3: Bigen-Reptice and Reptile training process the regression toy test. (a), (b), (c) and (d),
(c), (f) show that after the gradient update 0, 8, 16, 24, 32 times based on initialization learned by Sigen-Reptile and Reptile respectively.
Theorem Let W. be the parameter matrix generated by the corrupted samples. Compute the eigenvalues and eigenvectors of the expected observed parameter matrix
1 P Po( A
Ao 1 -aa P P.
da
(10)
X-
where Ctr is the covariance matrix generated by true samples, À is the observed largest eigenvalue, No is the largest eigenvalue in the corrupted diagonal eigenvalue matrix Ag. According to Eq if >oo> is smaller, the observed eigenvector is more accurate. Assume that the discarded high loss samples have the same contributions main directional variance, respectively. Note that these two kinds of data have the same effect on the gradient updating of the model, this assumption is relatively reasonable. Furthermore, it easy to find that (No (A - E) is smaller than Xo/X.
and Xo, representing the observed and corrupted
Theorem 2 shows that discard the high loss samples can help improve the accuracy of the ob- served eigenvector of parameter matrix learned with corrupted labels. So ISPL improve the performance of Eigen-Reptile Reptile, as it discards high loss samples in Dirain.
5 EXPERIMENTAL RESULTS AND DISCUSSION
In our experiments, we aim to (1) evaluate the effectiveness of Eigen-Reptile to alleviate gradient noise (sampling and label noise), (2) determine whether Eigen-Reptile can alleviate gradient noise in realistic problem, (3) evaluate the improvement of ISPL to Reeetlee in the presence of noisy labels, (4) validate theoretical analysis through numerical simulations. The code and data for the proposed model are provided for research purposes
5.1 META-LFARNING WITH NOISE ON REGRESSION
In this experiment, we evaluate Eigen-Reptile by the ID sine wave K-shot regression problem (Nichol et al. 2018 Each task is defined by sine curve y(z) Asin(x + b), where the amplitude A ~ C(0.1 5.0)) and phase ~ U(0, 2m]). The amplitude and phase are varied between tasks. The goal of each task is fit sine curve with the data points sampled from the corresponding y(x). We calculate loss in &2 using 50 equally-spacce points from the whole interval [-5.0, 5.0) for each task. The loss is
(())
(x) 2| dx
(11)
550
Cooee included in the supplemental material, Code will he released upon the paper acceptance.

---

Under review as a conference paper at ICLR 2021
Table 1:
Few Shot Classification on Mini-Imagenet N-way K-shot accuracy. The shows 95% confidence interval over tasks.
Algorithm
5-way 1-shot
5-way 5-shot
MAML (Finn et al., 2017
48.70 + 1.84%
63.11 = 0.92%
Relation Network (Sung el al. 2018
50.44 + 0.82%
65.32 + 0.70%
CAML (512) Zintgraf et al. 20180
51.82 + 0.65%
65.85 + 0.55%
TAML (VL Meta- Jamal & Q 2019
51.77 1.86%
6 0.93%
Meta-dropout(Lee al. 2019
51.93 0.67%
67.42
Warp-MAML (Flennerhag et al.
2019
52.30 + 0.8%
68.4
MC (128) Park & Oliva 2019
54.08 + 0.93%
67.99 0.73%
ARML
Yao et al. 20200)
50.42 + 1.73%
ModGrad (64)
(Simon et al.
2020)
53.20 + 0.86%
69.17 + 0.69
Reptile
et al.
2018
49.97 + 0.32%
65.99 0.58%
Eign-Reptile (32)
51.80 + 0.9%
68.10 + 0.50%
Fign-Reptile (64)
53.25 + 0.45%
69.85 + 0.85%
where ((x) is the predicted function that start from the initialization learned by meta-learner.
The K'shot regression task fits a selected sine curve through K points, here K 10. For the regressor, we use a small neural network, which is the same as Nichol et al. (2018), except that the activation functions are Tanh. Specifically, the small network includes input layer of size 1, followed by two hidden layers of size 64, and then an output layer of size 1. In this part, we mainly compare Reptile and RepeneR, Both meta-learners 30000 iterations with inner loop steps 5, batch size 10, and fixed inner loop learning rate = 0.02.
the same regressor and are trained for
We report the results of Reptile and Eigen-Reptile in Figure 3] It can be seen that the curve fitted by Sigen-Reptike is closer to the true green curve, which shows that Rigen-Reptile performs better. According to Jamal & Qi (2019), the initial model that has a larger entropy before adapting to new tasks would better alleviate meta-overfiin meta-ovvrriiting. As shown in Figure 3. from 30000 iterations, Sigen-Reptile is more generalizable than Reptile as the initial blue line of Eigen-Reptile is closer to a straight line, which shows that the initialization learned by Eigen-Reptile is less affected by gradient noise. Furthermore, Figure 4 converges faster and gets
2.5
8 3.5-
5.5
landlanu
shows that lower loss than Reptile.
Figure Loss of the 10-shot regression.
5.2 META-LEARNING IN REALISTIC PROBLEM
We evaluate our method on a popular few-shot classification dataset: Mini-ImageNet (Vinyals et al. 2016). The Mini-Imagenet dataset contains 100 classes, each with 600 images. We follow Ravi & Larochelle (016) to divide the dataset into three disjoint subsets: meta-training set, meta-validation set, and meta-testing set with 64 classes, 16 classes, and 20 classes, respectively. And we follow the few-shot learning protocols from prior work (Vinyals et al. 2016). except that the number of the meta-training shot 15, which is still much smaller than the number of samples required by traditional tasks. Moreover, we run our algorithm on the dataset for the different number of shots and compare our results to the state-of-the-art results. What needs to be reminded that approaches that use deeper, residual networks can achieve higher accuracies (Gidaris & Komodakis 2018), so for a fair comparison, we only compare algorithms that use convolutional networks as Reptile does. Specifically, our model follows follows|Nichol Nichol et al. (2018), which has 4 modules with 3 x convolutions and 64 filters, 2 x max-pon maxx-pooling etc.. The images are downsampled to 84 x 84, and the loss function the cross-entropy error. We use the Adam optimizer with 18 in the inner loop. Our model
7

---

Under review as a conference paper at ICLR 2021
Table 2: Average test accuracy of 5-way 1-shot on the Mini-Imagenet with symmetric label noise.
Algorithm
p=0.0
p=0.1
p=0.2
p=0.5
Reptile Nichol al. 2018
47.64%
46.08%
43.49%
23.33%
Reptil++IPPL
47.23%
46.50%
43.70%
21.83%
Rign-Reptile
47.87%
47.18%
45.01%
27.23%
Figen-Reptil++ISPL.
47.26%
47.20%
45.49%
28.68%
is trained for 100000 iterations with fixed inner loop learning rate 0.0005, and 7 inner-loop steps. Regarding some hyperparameters analysis, defer to Appendix
The results of Eigen-Reptile and other meta-learning approaches are summarized in Table The pro- posed Figen-Reptile (64 filters) outperforms and achieves highly competitive performance compared with other algorithms for 5-shot and 1-shot classification problems, respectively. More speeifically, for I-shot, the result of MC (128 filters) with a higher capacity network is better than that of Eigen- Reptile. However, as a second-order optimization algorithm, the computational cost of MC will be much higher than Eigen-Reptile. And obviously, the result of Eigen-Reptile is much better than Reptile for each task. Compared with Reptile, Reeenne uses the main direction to update the meta--arameetrs to alleviate the meta-overfitting caused by gradient noise. More importantly, Eigen-Reptile outperforms the state-of-the-art meta-overfitting preventing method Meta-dropout (Lee) al. 2019), which is based regularization. This result shows that the effectiveness of addressing the meta-overfitting problem from the perspective of alleviating gradient noise.
5.3 META-LEARNING WITH LABEL NOISE
We conduct the 5-way 1 1-shot experiment with noisy labels generated by corrupting the original labels of Mini- Imagenet.... More specifically, in this section, we only focus on symmetric label noise, as correct labels are flipped to other labels with equal probability, i.e., in the case of symmetric noise of ratio p. a sample retains the correct label with probability with probability P/ 1). An example of symmetric noise is shown in Figure Furthermore, the asymmetric label noise experiment is conducted in Appendix [F] Note that we only introduce noise in the train set during meta-training. where the meta-training shot is 30. Moreover, all meta-learners with 32 filters are trained for 10000 iterations, with a learning rate of 0.001 in the inner loop. The sample selection parameter 7 10 that decreases by 0.6 every 1000 iterations. The other settings of this experiment are the same as in section 5.2
p. It becomes some other label
As shown in Table 3 with the increase of the ratio p. the performance of Reptile decreases rapidly. When 0.5, the mitialization point learned by Reptile can hardly meet the requirements of quickly adapting to new tasks with few samples. On the other hand, Bigen-Reptile labels than Reptile, especially when the noise ratio is high, i.e., p 0.5. The experimental results also verify the effectiveness of ISPL, as when p * 0. It also can be seen that ISPL plays more significant role when p is higher. However, when p = 0, ISPL harms Sigen-Reptile as ISPL only discards correct samples. In addition, ISPL does not significantly improve Reptile, especially when p 0.5. This is because too many high-loss samples are removed, causing Reptile to fail to converge quickly with the same number of iterations. These experimental results show that Rigen-Reptile and ISPL effectively alleviate the gradient noise problem caused by noisy labels, thereby alleviating the meta-overfitting on corrupted samples.
less affected by noisy
achieves better results than Figen-Reptile
CONCLUSION
In this paper, we cast the meta-overfitinng problem (overfitting on sampling and label noise) as a gradient noise problem. Then, we propose gradient-based meta-learning algorithm Sigen-Reptile updates the meta perammersssters through the main direction, which has been proven by theory and experiments that it can alleviate the gradient noise effectively. Furthermore, get closer situations, we introduce noisy labels into the meta-training dataset, and the proposed ISPL constructs prior models to select samples for Rigen-Reptile to get more accurate main direction.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Marcin Andrychowiez, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando De Freitas. Learning to learn by gradient descent by gradient descent. In Advances in neural information processing systems, pp. 3981-3989, 2016.
Eric Arazo, Diego Ortego, Paul Albert, Noel E O' Connor, and Kevin McGuinness. Unsupervised label noise modeling and loss correction. arXiv preprint arXiv:1904.11238. 2019.
Sumit Basu and Janara Christensen. Teaching classification boundaries to humans. In Twenty-Sevnnth
AAAI Conference on Artificial Intelligence, 2013.
Luca Bertinetto, Joao Henriques, Philip HS Torr, and Andrea Vedaldi. Meta-learning with differen- tiable closed-form solvers. arXiv preprint 18Xxv:8805.088 2018.
Tianshi Cao, Mare Law, and Sanja Fidler. A theoretical analysis of the number of shots in few-shot learning. arXiv preprint 1rXiv:1909.11722 2019.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning-Vor Volume 70, pp. 1 1126-1122 1135. JMLR. org, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic model-agnostic meta-learning. In Advances in Neural Information Processing Systems, pp. 9516-9527, 2018.
Sebastian Flennerhag, Andrei A Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell. Meta-learning with warped gradient descent. arXiv preprint arXiv:1909.0000 2019.
Benoît Frénay and Michel Verleysen. Classification in the presence of label noise: survey. IEEE transactions neural networks and learming systems, 25(5):845-869) 2013.
Tianyu Gao, Xu Han, Zhiyuan Liu, and Maosong Sun. Hybrid attention-based prototypical networks for noisy few-shot relation classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 6407-64 2019.
Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 4367-4375, 2018.
Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks labels corrupted by severe noise. In Advances in neural information processing systems, pp. 10456-10465, 2018.
Sepp Hochreiter, A Steven Younger, and Peter R Conwell. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, pp. 87-94. Springer, 2001.
Shell Xu Hu, Pablo G Moreno, Yang Xiao, Xi Shen, Guillaume Obozinski, Neil D Lawrence, and Andreas Damianou. Empirical bayes transductive meta-learning with synthetic gradients. arXiv preprint 2xXi::2004.12696 2020.
Wenging Hu, Chris Junchi Li, Lei Li, and Jian-Guo Liu. On the diffusion approximation of nonconvex stochastic gradient descent. arXiv preprint arXiv: (70500/20 2017.
Muhammad Abdullah Jamal and Guo-Jun Qi. Task agnostic meta-learning for few-shot learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 11719- 11727, 2019.
Stanislaw Jastrzębski. Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. Three factors influencing minima in sgd. arXiv preprint arXiv: 171106671..002.3 2017.
Ishan Jindal, Matthew Nokleby, and Xuewen Chen. Learning deep networks from noisy labels with dropout regularization. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 967-972. IEBE, 2016.

---

Under review as a conference paper at ICLR 2021
Faisal Khan, Bilge Mutlu, and Jerry Zhu. How do humans teach: On curriculum learning and teaching dimension. In Advances in neural information processing systems, pp. 1449-1457, 2011.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint 14Xiv 69800 2014.
Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2. Lille, 2015.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009,
M Pawan Kumar, Benjamin Packer, and Daphne Koller. Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems, pp. 1189-1197, 2010.
Hae Beom Lee, Taewook Nam, Eunho Yang, and Sung Ju Hwang. Meta dropout: Learning to perturb latent features for generalization. In International Conference on Learning Representations, 2019.
HB Lee, T Nam, E Yang, and SJ Hwang. Meta dropout: Learning to perturb latent features for generalization. In International Conference on Learning Representations, 2020.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint urXiv:1707.09835 2017.
Christopher H Lin, Daniel S Weld, et al. To (label), on human computation and crowdsourcing, 2014.
re (label), In Second AAAI conference
Yanbin Liu, Juho Lee, Minseop Park, Saehoon Kim, Eunho Yang, Sung Ju Hwang, and Yi Yang. Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv:1805, 10002, 2018.
Stephan Mandt, Matthew Hoffman, and David Blei. A variational analysis of stochastic gradient algorithms. In International conference on machine learning, pp. 354-363, 2016.
Igor Mordatch. Concept learning with energy-based models. arXiv preprint arXiv:1811.02485 2018.
Arvind Neelakantan, Luke Vilnis, Quoe V Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, and James Martens. Adding gradient noise improves learning for very deep networks. arXiv preprint arXiv:1511.0680) 2015.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint arXiv: (xX.v99803.0299, 2018
Boris Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. Tadam: Task dependent adaptive metric for improved few-shot learning. In Advances in Neural Information Processing Systems, pp. 721-731, 2018
Eunbyung Park and Junier B Oliva. Meta-curvature. In Advances in Neural Information Processing Systems, pp. 3314-3324, 2019.
Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 1944-1952 2017.
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with implicit gradients. In Advances in Neural Information Processing Systems, pp. 1 113-124, 2019.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.
Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Few-shot learning with embedded class models and shot-free meta training. In Proceedings of the IEEE International Conference on Computer Vision, pp. 331-339, 2019.
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta- learning with neemory-augmente neural networks. In International conference on machine learning, pp. 1842-1888 1850, 2016.
10

---

Under review as a conference paper at ICLR 2021
Victor Garcia Satorras and Joan Bruna Estrach. Few-shot learning with graph neural networks. In International Conference on Learning Representations. 2018.
Pranav Shyam, Shubham Gupta, and Ambedkar Dukkipati. Attentive recurrent comparators. In Proceedings of the 34th International Conference on Machine Learning-Volat Voluume 70, pp. 3173-3181. JMLR. org, 2017.
Christian Simon, Piotr Koniusz, Richard Nock, and Mehrtash Harandi. On modulating the gradient for meta-learning European Conference on Computer Vision, pp. 556-572 Springer, 2020.
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. In Advances in neural information processing systems, pp. 4077-4087, 2017.
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pp. 1199-1208, 2018.
Ye Tang, Yu-Bin Yang, and Yang Gao. Self-paced dictionary learning for image classification. In Proceedings of the 20th ACM international conference Multimedia, pp. 833-836, 2012.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, al. Matching networks for one shot learning. In Advances in neural information processing systems, pp. 3630-3638, 2016.
Yaqing Wang, Quanming Yao, J Kwok, and LM Ni. Few-shot learning: A survey, arXiv preprint arXiv:1904.05044 05046, 2019a.
Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jinfeng Yi, and James Bailey, Symmetric cross entropy for robust learning with noisy labels. In Proceedings of the IEEE International Conference on Computer Vision, pp. 322-330, 2019b.
Jingfeng Wu, Wenqing Hu, Haoyi Xiong, Jun Huan, Vladimir Braverman, and Zhanxing Zhu. On the noisy gradient descent that generalizes sgd. 2019.
Ling Yang, Liangliang Li, Zilun Zhang, Xinyu Zhou, Erjin Zhou, and Yu Liu. Dpgn: Distribution propagation graph network for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13390-13399, 2020.
Huaxiu Yao, Ying Wei, Junzhou Huang, and Zhenhui Li. Hierarchically structured meta-learning.
arXiv preprini arXiv: 1905.0000001,... 2019.
Huaxiu Yao, Xian Wu, Zhiqiang Tao, Yaliang Li, Bolin Ding, Ruirui Li, and Zhenhui Li. Automated relational meta-learning. arXiv preprint arXiv:2001.00745 2020.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint 1rXiv:1611.0355 2016.
Luisa M Zintgraf, Kyriacos Shiarlis, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Caml:
Fast context adaptation via meta-learning. 2018.

---

Under review as a conference paper at ICLR 2021
A PSEUDO-CODE
Algorithm Eigen-Reptile
Require:
Distribution over tasks P(T), outer step size B
1: Initialize meta- parameertss Φ
2: while not converged do
3: W [ =0
4: Sample batch of tasks ( P(T)
5: for each task T; do
6:
Φ; Φ
7:
Sample train set Dtrain of Ti
8:
for j = 1, 3, .... do
9:
Φ (D train. Φ;)
10:
W appends wi = flatten(d)), Wy E Rdx1
11:
end for
12:
Mean centering, W W w. w E Rix1
13:
Compute cigenvalue matrix A and eigenvector matrix P of seatter matrix W W
14:
Eigenvalues À 2 > > An in A Compute eigenvector matrix of WWT, P
15:
16:
Let the eigenvector corresponding to A be a unit vector e 1 do
for j = 1,2, ....
18:
V/ = == (W )e end for
20:
e Em=1 Am
A xe
21:
Calculate the approximate direction of task-specific gradient update V: [n/2]
22:
V
n
i=1
W -nni++1
W 1
23:
if
V
< 0 then
24:
=
e
25:
end if
26:
end for
Average the main directions to get -1/B) 8
27:
28:
Update meta-parameters eb &- Φ + 3 x
end while
Algorithm 2 Introspective Self-paced Learning
Require: Dataset Dtrains initialization Φ, batch size b, selection parameter y, attenuation coefficient
μ., the number of prior models Q
1: Initialize network parameters <b Φ for sampled task
2: for j = 1,2,3, ........ ,Q do
3: Sample examples Dy from Drrain for training modelj, +; U ( x*)
4: end for
5: Train task-specific parameters:
6: for i= ==
1,3, ... do
7: Compute hidden variable vector V:
8: = arg min, vy Eg Eq= ti where Ly = Eg L, (I No. Φ5)
9:
Update task-specific parameters o=:
s arg ming- LIsrL (Φ",v)
11: 7=1
end for
B ALGORITHM COMPLEXITY ANALYSIS
As for Eigen-Reptile, the cost single gradient descent in the inner-loop is O(d). where d is the number of network parameters. The cost of the covariance matrix computations is O(n2d).
12

---

Under review as a conference paper at ICLR 2021
where is the number of inner-loop. Moreover, the worst-case complexity of computing eigenvalue decomposition is O(n²). Finally, the computational complexity of restoring eigenvector ( We set the maximal number of outer-loop n + nd + nd)). ). As in FSL,, n. is usually less than 10 (for this paper n. = 77, ) so the overall time complexity is O(Td). As for Reptile, the computational complexity is also (Td)), which means that the time complexity of both Reptile and Repenle is much lower than the second-order optimization algorithms.
T. Hence the overall time complexity is (
As for spatial complexity, Eigen-Reptile needs to store a d x a matrix and a rt x n matrix. The overall space complexity (d), while the spatial complexity of Reptile is (d), too.
It can be seen that, compared to Reptile, Eigen-Reptie is the same in spatial complexity and time complexity. Still, its accuracy much higher than that of Reptile.
C THEOREM 1
Gradient update always with gradient noise inserted at every iteration, which caused Reptile, MAML, etc. cannot find accurate directions to update meta-paramettrs. In this section, we will prove that Eigen-Reptile can alleviate gradient noise.
Theorem Assume that the gradient noise variable follows Gaussian distribution (Hu et al. 2017 Jastrzebski er al. 2017 Mandt al. 2016), x N o0, Furthermore, and neural network parameter variable are assumed to be uncorrelated. The observed covariance matrix C equals noiseless covariance matrix Ct plus gradient noise covariance malrix Cz. Then, we get
C=C++C, = P
P, I'P)P, = PAP P.AP.
(12)
where Pt and P are the orthonormal eigenvector matrices of C₁ and C respectively, At and A are the corresponding diagonal eigenvalue matrices, and an identity matrix. It can be seen from T2 that C and C, has the same eigenvectors.
Proof C.1 In the following proof, we assume that the probability density function of gradient noise variable x follows Gaussian distribution, N (0, o²). Treat the parameters in the neural network as variables, and the parameters obtained by each gradient update as samples. Furthermore, gradient noise and neural network parameters are assumed be uncorrelated.
For observed parameter matrix W E Rdxn there are n. samples, let W.. E Rlxx be the observed values of the i-th variable W,, and W = [WT 1.: WT noiseless parameter matrix by W [W(..)
W d.: Similarly, we denote the
1,3
(W!.)
(W..) and
W W +
(13)
Where X variable by
[TT X ... XI] T is the dataset of noise variables. Then, centralize each
1 R WR.(1)
W. & We
(14)
n
W I... Suppose W is also centralized by the same way and get
So we get W
W
W
W
Wi T. Then, we have:
W W +
(15)
Computing the covariance matrix of W:
WW W
W X ( XT)
(16)
(W'Wi' T + XWi. + +xX])
13

---

Under review as a conference paper at ICLR 2021
Since W and X are uncorrelated, W W'X" and X XWi are approximately zero matrices. Thus:
C= W +
(17)
The component C2 (i,) is the correlation between X; and X; which corresponds to the i-th j-th rows of X. As the two noise variables are not related to each other, if + J. then Ca 0. So C2 E Raxd is diagonal matrix with diagonal elements o². Decompose C₁ as:
C. P.A.P.
(18)
where Pt is the noiseless orthonormal eigenvector matrix and De is the noiseless diagonal eigenvalue matrix, then
C₂ A.P.P P.A.P. P.C.P. where A 0² and I is the identity matrix. Thus,
(19)
C=C++
= P.A,P. + P.A.P.
(20)
= +A(A)+A))P)
PPAP
where A = A, A,. It can be seen from 8g(20that C and C, has the same eigenvector matrix. In other words, eigenvector is not affected by gradient noise.
D THEOREM 2
In this section, will prove that discarding high loss samples will result in more accurate main direction in noisy FSL.
Theorem
Let Wo be the parameter matrix generated by the corrupted samples. Compute the eigenvalues and eigenvectors of the expected observed parameter matrix
1 =(CC..)e-Po(t- Po P e No I)P. - A
Ao 2 PP.1I-6--51 2-
(21)
where Ctr is the covariance matrix generated by true samples. À is the observed largest eigenvalue, Xo is the largest eigenvalue in the corrupted diagonal eigenvalue matrix A.. According to Eq[2] if Xa/X X is smaller, the observed eigenvector is more accurate. Assume that the discarded high loss samples have the same contributions to and Xo, representing the observed and corrupted main directional variance, respectively. Note that these two kinds of data have the same effect on the gradient updating of the model, this assumption is relatively reasonable. Furthermore, it is easy to find that (ha {) - {) smaller than Xo/>
Proof D.I Here, we convenience, let w be generated by a single sample, w [dxx1 Then the parameter matrix can be obtained,
represent the parameter point obtained after gradient update. For
W W= wh
wi
(22)
where wD represents the parameters generated by the corrupted sample, and w² represents the parameters generated by the true sample. Furthermore, there are n parameter points generated by n samples. Moreover, there are m corrupted parameter points generated by m corrupted samples. Mean centering W, and show the observed covariance matrix C
C WW
n
w
(w))
T
wh
(23)
n
(w
1
- (w (w +... w]( T
w (w))) ... "w. (wx
14

---

Under review as a conference paper at ICLR 2021
It can be seen from the decomposition of C that the required eigenvector is related the parameters obtained from the true samples and the parameters obtained from the noisy samples. For a single parameter point
ww
0i
Q1
aj
aal
0dd
(24)
41
0102
0201
a3
a2dd
0gg01
0g02
As we discard all high loss samples that make the model parameters change significantly, and the randomly generated noisy labels may cause the gradient to move in any direction, we assume that the variance of corrupted parameter point variables is 5. Compute the expectations of all variables in the corrupted parameter point
+ E(aa)²
B(a1a2)
E(ajad) E(azad)
E ((az1))
03 E(a2)
E(ww")
$2
(25)
((ada1)
B(aaaz)
53 + E(aa)}
Let the of all corrupted "[(ww' 1) be Sor then
mo + 2ihh B(agi)²
1-1 E(ajiaça)
B(azaaj))
j=1 T Ra
? n
(26)
Σラ= E(azaazi)
m82
Σラ= B(aza)?
And let the of all true n wwwT be Ctr- So the expectation of C can be written as,
E(C) B(C++) + St Treat eigenvector and eigenvalue as definite values, we get
(27)
(S, + ((Ct))e Xe
(28)
where is the observed eigenvector, X is the corresponding eigenvalue. Divide both sides of the
equation by
E(Crr)e= (I- So)e
-Pa(I X P.
(29)
Po Po 1
where No is the largest eigenvalue in the corrupted diagonal eigenvalue matrix A., P is the orthonormal eigenvector matrix of St According to 2q if >o/> is smaller, more accurate. Discard some samples with the largest losses, which may contain true samples and noisy samples. Assume that the discarded high loss samples have the same contributions' E to X and X as these two kindy of data have the same effect on the gradient updating of the model. Compare the ratio of eigenvalues before and after discarding, get
X
X
E
A-E E
hefore
after
(30)
{(> Aa)
0
1(1
15

---

Under review as a conference paper at ICLR 2021
Obviously, X Xg. and if we don't discard all samples, then X So 3q O. which means discarding high loss samples could reduce Xo!>. Therefore, discarding high loss samples can improve the accuracy of eigenvector in the presence of noisy labels.
For further analysis, we assume that any two variables are independently and identically distributed, the expectation of variable a, B(a) Thus,
82
2
2
2
(31)
82 +
where p is the proportion of noisy labels, np m. can be seen from Eq if pfp /X 2 then
31
D x is diagonal matrix. According to proof. C.I L
the observed eigenvector e is unaffected by
noisy labels with the corresponding eigenvalue
E HYPERPARAMETERS OF EIGEN-
In this section, follows Lee al. 2019) Cao al. (2099) to vary the number of inner-loops and the number of corresponding training shots to show the robustness of Eigen-Reptile. Besides, other hyperparameters
the same section 5.2 As shown in Figure after the number of inner-loops i reaches 7, the test accuracy tends to be stable, which shows that changing the number of inner-loops within certain range has little effect on Eigen-Reptile. That is, Rigen-Reptile robust to this hyperparameter. As for train shot, to make the trained task-specifi parameters as unbiased as + 1, where N is the number of classes. So when i 7, the number of train shots is 15. It is important note that in our experiments, Reptile uses the original implementation hyperparameters, the number of inner-loops 8, the number of train shots is 15 and the corresponding accuracy is 65.99%.
possible, we specify train shot roughly satisfies iixbatch size
N
0.8
validation
ER
0.7
0.6
Accuracy(%
0.5
0.3
0.2
0.1
0.0
11
Inner-loop steps
Figure 5: The number of inner-loop and accuracy of 5-way 5-shot task Mini- Mini-Imagenet.
F THE ASYMMETRIC LABEL NOISE EXPERIMENT
Since asymmetric noise is common symmetric noise, focus asymmetric noise in this section. As illustrated in asymmetric noise of Figure 6 we randomly flip the labels of one class to the labels of another class without duplication in the meta-training dataset (64 classes).
16

---

Under review as a conference paper at ICLR 2021
0,8
0.05
0.05
0.05
0.06
0.0
0.2
0.05
0.08
0.06
0.05
0.05
0.06
0
0.8
0.2
0.08
(a) symmetric noise
(b) asymmetric noise
Figure 6: Examples of noise transition matrix (taking classes and noise ratio 0.2 as an example).
We compare our algorithms with Reptile in 3 We observe that meta-learning algorithms with asymmetric noise have closer results compared symmetric noise. As meta-leamnrr is trained on tasks with the same noise transition matrix, which allows the meta-learner to learn more useful information, the results are higher than that with symmetric noise. Similar to the symmetric noise results, Eigen-Reptile outperforms Reptile in all tasks, and ISPL plays more significant role when higher. The experimental results also show that when the number of iterations is constant, ISPL does not significantly improve even degrades the results of Reptile. On the contrary, ISPL can provide Eigen- Reptile with a more accurate main direction in difficult tasks, e.g., p = 0.2, 0.5.
Table 3:
Average test accuracy of 5-way 1-shot on the Mini-Imagenn with asymmetric label noise.
Algorithm
p=0.1
p=0.2
p=0.5
Reptile (Nichol al. 2018
47.30%
45.51%
42.03%
Ieptile+ISPL 47.00%
45.42%
41.09%
Eign-Reptile
47.42%
46.50%
42.29%
Eigen-Reptile+ISPL.
47.24%
46.83%
43.71%
G META-LEARNING ON CIFAR-FS
Bertinetto et al. (018) propose CIFAR-FS (CIFARI00 few-shots), which randomly sampled from CIFAR- 100 (Krizhevsky et al. 2009), containing images of size 32 32. The settings of Eigen-Reptibe in this experiment are the same as in 5.2 Moreover, we do not compare algorithms with additional tricks, such higher way (Bertinetto et al. 2018 Snell al. 2017). It can be from Table 4 that on CIFAR-FS, the performance of Eigen-Reptile is still far better than Reptile without any parameter adjustment.
Table 4:
Few Shot Classification on CIFAR-FS N-way K-shot accuracy. The + shows 95% confidence interval over tasks.
Algorithm
5-way 1-shot
5-way 5-shot
MAML Finn et al. PROTO NET Snell of al. 2017
2017
58.90 + 1.90%
71.50 + 1.00%
55.50 0.70%
72.00 = 0.60%
GNN & Estrach
2018
61.90% 55.14 + 0.48%
75.30%
Embedded Class Models[Ravichandire et al. 2019
71.66 0.39%
Reptile (Nichol et al. 2018)
58.30 + 1.20%
75.45 + 0.55%
Eign-Reptile (64)
61.90 1.40%
78.30 0.50%
Eign-Reptia (128)
62.30 + 1.40%
78.55 + 0.45%
17

---

Under review as a conference paper at ICLR 2021
H NEURAL NETWORK ARCHITECTURES
This section will show the performance Eigen-Reptile on Mini-lmagenet when using a larger network CAML, etc. Note that we only compare our algorithm with meta-learning algorithms based on the gradient in this section. As shown in Table 3] when Eigen-Reptile uses a larger convolutional neural network (CNN), higher accuracy can be obtained, which shows that Eigen-Reptile benefits from increasing model expressiveness.
Table 5:
Few Shot Classification Mini-Imagene' N-way K-shot accuracy. The + shows 95% confidence interval over tasks.
Algorithm
Backbone
5-way 1-shot
5-way 5-shot
MAML Finn et al.
2017)
Conv-4-64
48.7 = 1.8%
63.1 + 0.9%
Meta-SGD (Li et
2017
Conv-4-64
50.47 + 1.87%.
64.03 + 0.94%
CAML Ziinggral et al.
2018
Conv-4-32
47.24 0.65%
59.05 + 0.54%
CAML Zintgraf et
2018)
Conv-4-512
51.82 0.65%
65.85 0.55%
iMAMI (
(Rajeswaran et al. 2019
Conv-4-64
49.30 1.88%
MC (128) (Park & Oliva
2019
Conv-4-128
54.08
0.93%
67.99 0.73%
HSML
Yao et al.
2019)
Conv-4-32
1.85% 50.38
Reptile(Nichol al. Eign-Reptile
2018
Conv-4-32
49.97 0.32%
65.99 + 0.58%
Conv-4-32
51.80 0.90%
68.10 0.50%
Eign-Reptile
Conv-4-64
53.25 + 0.45%
69.65 0.85%
18


Reviews:
[{"review_id": "yiJZf2LUvFl", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a novel and effective meta-learning algorithm using the so-called Eigen-Reptile to update the meta-parameters and address the gradient noises. To further improve the accuracy of the main direction of Eigen-Reptile, the authors employ self-paced learning algorithm to construct several prior models determining which samples should be abandoned. The experiments demonstrate strong robustness against label noises.\n\nStrength:\n- The proposed algorithm is technically sound. The authors provide the solution to address the high cost of computing eigenvalue and eigenvector in the ER process, and also offer the theorem to demonstrate that the eigenvectors will not be affected by gradient noises. \n- The experiments are solid to demonstrate the usefulness of the Eigen-Reptile algorithm. Applying ER in regression, few-shot classification tasks achieve superior performance and robustness compared to the selected baseline models.\n\nWeakness:\n- Missing analyses on the training and testing overhead, e. .g, runtime, running memory.\n- Missing comparisons to more recent meta-learning and few-shot learning approaches. \n- ER is built upon a CONV-4 backbone. Is it applicable to more complicated architectures, e. g. ResNet-12 considering both the performance and overhead.", "extra_scores": {}, "extra_text": {}}, {"review_id": "j80tTQaK_hD", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a reptile-based meta-learning algorithm called Eigen-Reptile for few shot learning with sampling and label nosing. When Eigen-Reptile updates meta-parameters, it leverages not only the gradient direction of different task, but also the direction of eigenvector related to parameters matrix. Besides, authors propose Introspective Self-paced Learning (ISPL) for label noise problem. N-way-K-shot experiments on Mini-Imagenet demonstrate the effectiveness of Eigen-Reptile and 5-way-1shot with symmetric label noise experiments on Mini-Imagenet show that ISPL could alleviate the noising problem in some degree.\n\nStrength:\n1. The idea of Eigen-Reptile to alleviate gradient noise by eigenvector of parameters-related matrix is interesting and authors prove the effectiveness of the idea in theory.\n2. A clever, avoiding high time complexity way to obtain the eigenvector of parameters-related matrix\n3. By idea of self-paced learning with prior model to solve noisy few shot problem is reasonable and ingenious with theoretical proof.\n\nWeakness:\n1. The writing is confusing and not clear enough. For example, Para. 4.1, line 6. What is the specific meaning of “main direction of n points”, and what is suitable mathematical expression of “the unit vector e” ?\n2. In Mini-Imagenet N-way K-shot experiments, authors didn’t show specific numbers of filter of the most important comparison object,  Reptile, and the final experiments results are not particularly outstanding compared with recent papers, like DPGN[1], SIB[2].\n3. For label noise experiments, it is hard to say the ISPL is indeed effective as results showed in line 1,2,3 of Table 2 in paper.\n\n\n[1] Yang, Ling, et al. \"DPGN: Distribution Propagation Graph Network for Few-shot Learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020\n[2] Hu, Shell Xu, et al. \"Empirical Bayes Transductive Meta-Learning with Synthetic Gradients.\" ICLR (2020)..", "extra_scores": {}, "extra_text": {}}, {"review_id": "b6mEzMZ0JA0", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper is concerned about the update of meta-parameters in gradient-based meta-learning approaches. Developed upon the recent method called Reptile (Nichol et al., 2018), this work aims to make it better deal with the sampling and label noises in few-short learning. The motivation is that Reptile updates meta-parameters towards the last task-specific parameters, but this could lead to biased updating. Instead, this work proposes to update the meta-parameters by the main direction of historical task-specific parameters to reduce the variation on gradients caused by the sampling and label noises. Theoretical analysis is conducted to show that the main direction of historical task-specific parameters is just the leading eigenvector of the covariance matrix computed upon the task-specific parameters obtained in the $n$ inner loop steps. This method is called Eigen-Reptile in this paper accordingly. \n\nFurther, this work proposes Introspective Self-paced Learning (ISPL) to improve the precision of the estimate of the main direction above. It creates multiple prior models by randomly sampling the training data and discards the samples with high loss values. Theoretical analysis is conducted to justify the proposed ISPL method. \n\nExperimental study is conducted on a synthetic dataset and the Mini-ImageNet benchmark in few-shot classification tasks. The result shows that the proposed method can achieve better performance than Reptile and other state-of-the-art methods. \n\nOverall, the issue researched in this paper has its significance, and the idea is sound. Also, experimental study demonstrates the effectiveness of the proposed method. \n\nThe main weaknesses of this work lie at its theoretical part, as detailed as follows.\n\n1. Essentially, this work can be regarded as a kind of denoise method to better update the meta-parameters. Different from Reptile, this work proposes to update the meta-parameters by the \"main direction\" of task-specific parameters. Why is the \"main direction\" the best choice? The simplest choice could be the \"mean\" of the $n$ task-specific parameters. How is this option compared with the main direction? Please clarify. \n\n2. The theoretical analysis in Section 4.1 is not novel. It is largely a derivation of principal component analysis which can be founded in textbooks. The only difference is that a linear constraint $\\bar{V}e>0$ is imposed in Eq.(3). However, if understood correctly, the derivation seems to be flawed. This linear constraint does not have any effect to the optimal solution of Eq.(3), which is still proved as the leading eigenvector of $S$. Eq.(3) is a linear constrained quadratic programming. The leading eigenvector of $S$ does not necessarily satisfy the linear constraint related to an arbitrary $\\bar{V}$. This may be related to some issue in the KKT condition listed in Eq.(5). Please double check or clarify it. \n\n3. The trick in Eq.(6) is not new in machine learning and therefore cannot be regarded as a technical contribution. This trick has long been used to compute the eigensystems of the covariance matrix of \"thin\" or \"fat\" data matrices, for example, in the seminal work [R1]. \n\n4. The proof in Appendix B is largely routine. It mainly shows the properties of a covariance matrix when data are corrupted by white noise (that is, following a Gaussian distribution).\n\n5. The proposed ISPL is interesting. However, the theoretical proof for Theorem 2 in the Appendix needs to be made clear. Particularly, the statements between Eq(29) and (30) are vague. Also, ISPL seems to be more related to ensemble learning. This paper may utilize existing ensemble learning theories to better justify this idea. \n\nIn short, this work is well motivated and the overall idea is sound. Experimental study supports the effectiveness of the proposed methods. Particularly, this paper makes efforts to provide theoretical justification for each of the methods, which is appreciated. However, these theoretical justifications currently are not strong or novel enough. Some of them need to be double checked and some need to be further clarified.  \n\n[R1] M. Turk; A. Pentland (1991). \"Face recognition using eigenfaces\" (PDF). Proc. IEEE Conference on Computer Vision and Pattern Recognition. pp. 586–591.\n\n--- Thank the authors for the detailed response. After reading the response and the comments of peer reviewers, it is still felt that this work needs to better clarify some key issues and strengthen both theoretical and experimental study. In light of these, the rating is maintained as follows. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "Yd9kBINboEn", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents a method about robust meta-learning by analyzing eigenvalues and eigenvectors. The ultimate goal is to learn the new task and denoise the gradients when updating the model parameters for the specific task.  There are two aspects in the proposed approach to alleviate gradient noise: 1). Parameters updates with eigen vectors. 2). introspective self-paced learning to correct the loss of noisy samples. \n\n\nStrengths:\n- The problem of meta-learning with noisy labels is introduced in this work with some experiments to show the robustness of the proposed approach.\n- The analysis with various noise percentages shows that meta-learning suffers from noisy labels.\n \nWeaknesses:\n- The manuscript is very hard to follow especially some important information is located in Appendix. For instance, in the main paper it is not very clear how the algorithm works and implemented, this important information is provided in Appendix (Algorithm 1 and 2).\n- It is not very clear how the inner-update is performed because U(.) can be any meta-learning algorithm but the work only focuses on Reptile. The inner-loop should be explicitly defined. \n- This work has no analysis about the limit of noise that is acceptable to the algorithm (especially related to the eigenvalues). \n- The assumption of Theorem 1 is strong that the gradient contains Gaussian noise from the noisy labels. Is there any empirical evidence/previous work providing the fact or the proof?\n- The experiment settings are very artificial and the problem is not well-motivated. How is it possible to have a lot of noisy labels when only a few data is provided? Is there any valid reason or practical example to justify the setup in this work?\n- The literature review of this work is very poor:\n1. Meta-learning with differentiable closed-form solvers is proposed by Bertinetto et al. [1]. Is there any reason to cite the technical report (reimplementation) by Devos et al.?\n2. Gradient noise in meta-learning has been investigated by Simon et al. [2] but it is not discussed and compared in the manuscript.\n3. A recent method called Meta-Curvature [3] which is a closely correlated topic to the proposed method is not discussed and compared.\n\n- The experiments are not convincing to show the superiority of the proposed method:\n1. The dataset is limited to one dataset for classification tasks. This paper only provides the mini-ImageNet dataset.\n2. A rigorous experiment is needed by varying the parameters (e.g. the number of shot, the number of inner-loop, and the number of way) to show the robustness are not shown. This is important because the setup is new but the experiments do not show various conditions.\n3. There is no comparison with the existing methods for combating noisy labels. Some approaches [4, 5, 6] can be adopted to the loss function for inner-loop.\n\n\nThere are some questions that need to be addressed:\n1. In Eq. 3, is it possible that \\bar{V}.e is zero? Because the inner-loop can reach a stationary point.\n2. Is ISPL applicable to any learning condition? What is the special thing of this loss function in meta-learning?\n\n\n[1] L. Bertinetto, J. F. Henriques,  P. Torr, and A. Vedaldi, \"Meta-learning with differentiable closed-form solvers,\" ICLR, 2018.\n\n[2] C. Simon, P. Koniusz, R. Nock, M. Harandi, \"On modulating the gradient for meta-learning,\" ECCV, 2020. \n\n[3] E. Park and J.B. Olivia, \"Meta-curvature,\" Neurips, 2019.\n\n[4] E. Arazo, D. Ortego, P. Albert, N. E. O'Connor, K. McGuiness,\"Unsupervised Label Noise Modeling and Loss Correction,\" ICML, 2019.\n\n[5] Y.Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, J. Bailey, \"Symmetric Cross Entropy for Robust Learning with Noisy Labels,\"ICCV, 2019.\n\n[6] G. Patrini, A. Rozza, A. K. Menon, R. Nock, L. Qu,\"Making Deep Neural Networks Robust to Label Noise:\na Loss Correction Approach,\" CVPR, 2017.\n\n\n\n--------------------------------------------------------------------------------------------------------------------------------------------------\n\nI want to thank the authors for the revision and the rebuttal. Even though the proposed method is promising for meta-learning with noisy gradients, the setup is not backed up with strong arguments/facts. Moreover, I do not fully agree to the statements in the response. For instance, these arguments are not fully observable neither by experiments nor theories:\n\n1. \" Specifically, the meta-learner is prone to meta-overfit, as there are only a few available samples with sampling noise, even data is clean.\". There is no proof that the meta-learner is prone to meta-overfit. The 'meta-overfit' term is not defined well. What I understand is that the learned meta-parameter is prone to overfitting in this context.\n2. \"Due to the small amounts of samples, FSL is more easily affected by data noise, especially considering that human annotators are likely to make mistakes as training meta-learner requires a large number of classes. Besides, we are not the first to propose noisy FSL, [10] propose hybrid attention-based prototypical networks for the noisy few-shot relation classification task, as human annotators are also likely to make mistakes in language tasks.\". A common sense is that a few data is more managable and less vulnerable to mislabeling.\n3. \"Our noisy labels experiments aim to verify the robustness of Eigen-Reptile to noisy labels. Denoise is not the focus of our research. At the same time, ISPL is not a denoise algorithm in the traditional sense.\". The statement is vague and not aligned to the introduction, abstract, and title of this work.\n\nThe theoretical part is mostly trivial as mentioned by R2, and the contribution (Algorithm) is hidden in Appendices. I acknowledge the author's response by increasing the score but I think the paper needs to be further revised.\n\nI suggest to build strong background of this topic (meta-learning + noise/label noise) and relevant experiments to show the effectiveness of the proposed approach. Furthermore, some theoretical analysis about the limit to the noise can be verified for the future direction. For experiments with noisy gradients, I suggest to compare with the baseline on meta-learning + noise, e.g. the work by Simon et al. [2]. If the focus was noisy labels then it would be better to compare with other methods for noisy labels (not necessarily in the meta-learning setup).", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper was evaluated by four reviewers. After rebuttal, several concerns remained, e.g. Rev. 1 is interested in more thorough comparisons even if the model is claimed to be backbone-agnostic. Rev. 2 is concerned about re-print of some theories and authors' response that 'contribution is not in theoretical innovation'. Rev. 3 is overall not impressed with the clarity of the paper. Finally, Rev. 4 also remains unconvinced after rebuttal due to several somewhat loose explanations provided by authors.

At this point, AC agrees with reviewers that the paper requires more clear-cut theoretical contributions, ablations and improvements in writing clarity. While some reviewers might have been more inspired by the aspect of noisy labels, even ignoring this aspect, the overall consensus among all reviewers stands.
Decision: Reject