Title: DROPGRAD: GRADIENT DROPOUT REGULARIZATION FOR META-LEARNING

Abstract: With the growing attention on learning-to-learn new tasks using only a few examples, meta-learning has been widely used in numerous problems such as few-shot classification, reinforcement learning, and domain generalization. However, metalearning models are prone to overfitting when there are no sufficient training tasks for the meta-learners to generalize. Although existing approaches such as Dropout are widely used to address the overfitting problem, these methods are typically designed for regularizing models of a single task in supervised training. In this paper, we introduce a simple yet effective method to alleviate the risk of overfitting for gradient-based meta-learning. Specifically, during the gradient-based adaptation stage, we randomly drop the gradient in the inner-loop optimization of each parameter in deep neural networks, such that the augmented gradients improve generalization to new tasks. We present a general form of the proposed gradient dropout regularization and show that this term can be sampled from either the Bernoulli or Gaussian distribution. To validate the proposed method, we conduct extensive experiments and analysis on numerous tasks, demonstrating that the gradient dropout regularization mitigates the overfitting problem and improves the performance upon various gradient-based meta-learning frameworks.

1. INTRODUCTION:

In recent years, significant progress has been made in meta-learning, which is also known as learning to learn. One common setting is that, given only a few training examples, meta-learning aims to learn new tasks rapidly by leveraging the past experience acquired from the known tasks. It is a vital machine learning problem due to the potential for reducing the amount of data and time for adapting an existing system. Numerous recent methods successfully demonstrate how to adopt meta-learning algorithms to solve various learning problems, such as few-shot classification (Finn et al., 2017; Santoro et al., 2016; Snell et al., 2017) , reinforcement learning (Gupta et al., 2018; Rakelly et al., 2019) , and domain generalization (Balaji et al., 2018; Li et al., 2018) .

Despite the demonstrated success, meta-learning frameworks are prone to overfitting (Kim et al., 2018) when there do not exist sufficient training tasks for the meta-learners to generalize. For instance, few-shot classification on the mini-ImageNet (Vinyals et al., 2016) dataset contains only 64 training categories. Since the training tasks can be only sampled from this small set of classes, meta-learning models may overfit and fail to generalize to new testing tasks.

Significant efforts have been made to address the overfitting issue in the supervised learning framework, where the model is developed to learn a single task (e.g., recognize the same set of categories in both training and testing phase). The Dropout (Srivastava et al., 2014) method randomly drops (zeros) intermediate activations in deep neural networks during the training stage. Relaxing the limitation of binary dropout, the Gaussian dropout (Wang & Manning, 2013 ) scheme augments activations with noise sampled from a Gaussian distribution. Numerous methods (Ghiasi et al., 2018; Larsson et al., 2017; Tompson et al., 2015; Wan et al., 2013; Zoph et al., 2018) further improve the Dropout method by injecting structural noise or scheduling the dropout process to facilitate the training procedure. Nevertheless, these methods are developed to regularize the models to learn a single task, which may not be effective for meta-learning frameworks.

In this paper, we address the overfitting issue (Kim et al., 2018) in gradient-based meta-learning. As shown in Figure 1 (a), given a new task, the meta-learning framework aims to adapt model pa-rameters θ to be θ via the gradients computed according to the few examples (support data X s ). This gradient-based adaptation process is also known as the inner-loop optimization. To alleviate the overfitting issue, one straightforward approach is to apply the existing dropout method to the model weights directly. However, there are two sets of model parameters θ and θ in the inner-loop optimization. As such, during the meta-training stage, applying normal dropout would cause inconsistent randomness, i.e., dropped neurons, between these two sets of model parameters. To tackle this issue, we propose a dropout method on the gradients in the inner-loop optimization, denoted as DropGrad, to regularize the training procedure. This approach naturally bridges θ and θ , and thereby involves only one randomness for the dropout regularization. We also note that our method is model-agnostic and generalized to various gradient-based meta-learning frameworks such as (Antoniou et al., 2019; Finn et al., 2017; Li et al., 2017) . In addition, we demonstrate that the proposed dropout term can be formulated in a general form, where either the binary or Gaussian distribution can be utilized to sample the noise, as demonstrated in Figure 1(b) .

To evaluate the proposed DropGrad method, we conduct experiments on numerous learning tasks, including few-shot classification on the mini-ImageNet (Vinyals et al., 2016) , reinforcement learning (Finn et al., 2017) , and online object tracking (Park & Berg, 2018) , showing that DropGrad can be applied to and improve different tasks. In addition, we present comprehensive analysis by using various meta-learning frameworks, adopting different dropout probabilities, and explaining which layers to apply gradient dropout. To further demonstrate the generalization ability of DropGrad, we perform a challenging cross-domain few-shot classification task, in which the meta-training and meta-testing sets are from two different distributions, i.e., mini-ImageNet and CUB (Welinder et al., 2010) . We show that with the proposed method, the performance is significantly improved under the cross-domain setting.

In this paper, we make the following contributions:

• We propose a simple yet effective gradient dropout approach to improve the generalization ability of gradient-based meta-learning frameworks.

• We present a general form for gradient dropout and show that both binary and Gaussian sampling schemes mitigate the overfitting issue.

• We demonstrate the effectiveness and generalizability of the proposed method via extensive experiments on numerous tasks.

2. RELATED WORK:

Meta-Learning. Meta-learning aims to adapt the past knowledge learned from previous tasks to new tasks with few training instances. Most meta-learning algorithms can be categorized into three groups: Nevertheless, these approaches are designed for deep neural networks that aim to learn a single task, e.g., learning to recognize a fixed set of categories. In contrast, our algorithm aims to regularize the gradient-based meta-learning frameworks that suffer from the overfitting issue on the task-level, e.g., introducing new tasks.

3. GRADIENT DROPOUT REGULARIZATION:

Before introducing details of our proposed dropout regularization on gradients, we first review the gradient-based meta-learning framework.

3.1. PRELIMINARIES FOR META-LEARNING:

In meta-learning, multiple tasks T = {T 1 , T 2 , ..., T n } are divided into meta-training T train , metavalidation T val , and meta-testing T test sets. Each task T i consists of a support set D s = (X s , Y s ) and a query set D q = (X q , Y q ), where X and Y are a set of input data and the corresponding ground-truth. The support set D s represents the set of few labeled data for learning, while the query set D q indicates the set of data to be predicted.

Given a novel task and a parametric model f θ , the objective of a gradient-based approach during the meta-training stage is to minimize the prediction loss L q on the query set D q according to the signals provided from the support set D s , and thus the model f θ can be adapted. Figure 1(a) show an overview of the MAML (Finn et al., 2017) method, which offers a general formulation of gradientbased frameworks. For each iteration of the meta-training phase, we first randomly sample a task T = {D s , D q } from the meta-training set T train . We then adapt the initial parameters θ to be task-specific parameters θ via gradient descent:

EQUATION

where α is the learning rate for gradient-based adaptation and is the operation of element-wise product, i.e., Hadamard product. The term g in equation 1 is the set of gradients computed according to the objectives of model f θ on the support set D s = (X s , Y s ):

EQUATION

We call the step of equation 1 as the inner-loop optimization and typically, we can do multiple gradient steps for equation 1, e.g., smaller than 10 in general. After the gradient-based adaptation, the initial parameters θ are optimized according to the loss functions of the adapted model f θ on the query set D q = (X q , Y q ):

EQUATION

where η is the learning rate for meta-training. During the meta-testing stage, the model f θ is adapted according to the support set D s and the prediction on query data X q is made without accessing the ground-truth Y q in the query set. We note that several methods are built upon the above formulation introduced in MAML. For example, the learning rate α for gradient-adaptation is viewed as the optimization objective (Antoniou et 

3.2. GRADIENT DROPOUT:

The main idea is to impose uncertainty to the core objective during the meta-training step, i.e., the gradient g in the inner-loop optimization, such that θ receives gradients with noises to improve the generalization of gradient-based models. As described in Section 3.1, adapting the model θ to θ involves the gradient update in the inner-loop optimization described in equation 2. Based on this observation, we propose to randomly drop the gradient in equation 2, i.e., g, during the inner-loop optimization, as illustrated in Figure 1 . Specifically, we augment the gradient g as follows:

EQUATION

)

where n is a noise regularization term sampled from a pre-defined distribution. With the formulation of equation 4, in the following we introduce two noise regularization strategies via sampling from different distributions, i.e., Bernoulli and Gaussian distributions.

Binary DropGrad. We randomly zero the gradient with the probability p, in which the process can be formulated as:

EQUATION

where the denominator 1 -p is the normalization factor. Note that, different from the Dropout (Srivastava et al., 2014) method which randomly drops the intermediate activations in a supervised learning network under a single task setting, we perform the dropout on the gradient level.

Gaussian DropGrad. One limitation of the Binary DropGrad scheme is that the noise term n b is only applied in a binary form, which is either 0 or 1 -p. To address this disadvantage and better provide a better regularization with uncertainty, we extend the Bernoulli distribution to the Gaussian formulation. Since the expectation and variance of the noise term n b in the Binary DropGrad method are respectively E(n b ) = 1 and σ 2 (n b ) = p 1-p , we can augment the gradient g with noise sampled from the Gaussian distribution:

EQUATION

As a result, two noise terms n b and n g are statistically comparable with the same dropout probability p. In Figure 1 (b), we illustrate the difference between the Binary DropGrad and Gaussian DropGrad approaches. We also show the process of applying the proposed regularization using the MAML (Finn et al., 2017) 

4. EXPERIMENTAL RESULTS:

In this section, we evaluate the effectiveness of the proposed DropGrad method by conducting extensive experiments on three learning problems: few-shot classification, reinforcement learning, and online object tracking. In addition, for the few-shot classification experiments, we analyze the effect of using binary and Gaussian noise, which layers to apply DropGrad, and performance in the cross-domain setting. The source code and trained models will be made available to the public.

Algorithm 1: Applying DropGrad on MAML (Finn et al., 2017) Require: a set of training tasks T train , adaptation learning rate α, meta-learning rate η randomly initialize θ while training do

randomly sample a task T = {D s (X s , Y s ), D q (X q , Y q )} from T train g = θ L s (f θ (X s ), Y s )

compute g according to equation 5 or equation 6 // Apply DropGrad Similar to recent studies, e.g., (Rusu et al., 2019) , we also pre-train the feature extractor of ResNet-18 by minimizing the classification loss on the 64 training categories from the mini-ImageNet dataset for the MetaSGD method, which is denoted by MetaSGD*. For all the experiments, we use the default hyper-parameter settings provided by the original implementation and select the model according to the validation performance for evaluation.

θ = θ -α × g θ = θ -η θ L q (f θ (X q ), Y q ) end

Comparison between Binary and Gaussian DropGrad. We first evaluate how the proposed Binary and Gaussian DropGrad methods perform on the MAML framework with different values of the dropout probability p. Figure 2 shows that both methods are effective especially when the dropout rate is in the range of [0.1, 0.2], while setting the dropout rate to 0 is to turn the proposed DropGrad method off. Since the problem of learning from only one instance (1-shot) is more complicated, the overfitting effect is less severe compared to the 5-shot setting. As a result, applying the Drop-Grad method with a dropout rate larger than 0.3 degrades the performance. Moreover, the Gaussian DropGrad method consistently outperforms the binary case on both 1-shot and 5-shot tasks, due to a better regularization term n g with uncertainty. We then apply the Gaussian DropGrad method with the dropout rate of 0.1 or 0.2 in the following experiments.

Comparison with existing dropout methods. To show that the proposed DropGrad method is effective for gradient-based meta-learning frameworks, we compare it with two existing dropout schemes applied on the network activations in both f θ and f θ . We choose the Dropout (Srivastava et al., 2014) and SpatialDropout (Tompson et al., 2015) methods, since the former is a commonlyused approach while the latter is shown to be effective for applying to 2D convolutional maps. The performance of MAML on 5-shot classification on the mini-ImageNet dataset is: DropGrad 69.42± 0.73%, SpatialDropout 68.09 ± 0.56%, and Vanilla Dropout 67.44 ± 0.57%. This demonstrates the benefit of using the proposed DropGrad method, which effectively tackles the issue of inconsistent randomness between two different models f θ and f θ in the inner-loop optimization of gradient-based meta-learning frameworks.

Overall performance on the mini-ImageNet dataset. Table 1 shows the results of applying the proposed Gaussian DropGrad method to different frameworks. The results validate that the proposed regularization scheme consistently improves the performance of various gradient-based metalearning approaches. In addition, we present the curve of validation loss over training episodes from MAML and MetaSGD on the 5-shot classification task in Figure 3 . We observe that the overfitting problem is more severe in training the MetaSGD method since it consists of more parameters to be Layers to apply DropGrad. We study which layers in the network to apply the DropGrad regularization in this experiment. The backbone ResNet-18 model contains a convolutional layer (Conv) followed by 4 residual blocks (Block1, Block2, Block3, Block4) and a fully-connected layer (FC) as the classifier. We perform the Gaussian DropGrad method on different parts of the ResNet-18 model for MAML on the 5-shot classification task. The results are presented in Table 2 . We find that it is more critical to drop the gradients closer to the output layers (e.g., FC and Block4+FC).

Applying the DropGrad method to the input side (e.g., Block1+Conv and Conv), however, may even negatively affect the training and degrade the performance. This can be explained by the fact that features closer to the output side are more abstract and thus tend to overfit. As using the DropGrad regularization term only increases a negligible overhead, we use the Full model, where our method is applied to all the layers in the experiments unless otherwise mentioned.

4.2. CROSS-DOMAIN FEW-SHOT CLASSIFICATION:

To further evaluate how the proposed DropGrad method improves the generalization ability of gradient-based meta-learning models, we conduct a cross-domain experiment, in which the meta-testing set is from an unseen domain. We use the cross-domain scenario introduced by Chen et al. (Chen et al., 2019) , where the meta-training step is performed on the mini-ImageNet dataset while the meta-testing evaluation is conducted on the CUB dataset (Hilliard et al., 2018) . Note that, different from Chen et al. (Chen et al., 2019) who select the model according to the validation performance on the CUB dataset, we pick the model via the validation performance on the mini-ImageNet dataset for evaluation. The reason is that we target at analyzing the generalization ability to the unseen domain, and thus we do not utilize any information provided from the CUB dataset.

Table 3 shows the results using the Gaussian DropGrad method. Since the domain shift in the crossdomain scenario is larger than that in the intra-domain case (i.e., both training and testing tasks are sampled from the mini-ImageNet dataset), the performance gains of applying the proposed Drop-Grad method reported in Table 3 are more significant than those in Table 1 . The results demonstrate that the DropGrad scheme is able to effectively regularize the gradients and transfer them for learning new tasks in an unseen domain. 

4.3. REINFORCEMENT LEARNING:

We adopt the few-shot reinforcement learning (RL) setting as in (Finn et al., 2017) , which aims to make the system adapt to new experiences and learn the corresponding policy quickly with limited prior experience (i.e., trajectories). In this setting, the support set D s contains few trajectories and the corresponding rewards, while the query set D q is formed by a set of new trajectories sampled from the running policy. We conduct the experiment with the locomotion tasks simulated by the MuJoCo (Todorov et al., 2012) simulator. Two environments are considered in this experiment: HalfCheetah robot and Ant robot with forward/backward movement, i.e., HalfCheetah-Dir and Ant-Dir.

Implementation Details. We adopt the MAML-TPRO (Finn et al., 2017) framework as the baseline method. Since the rewards are usually not differentiable, policy gradients are calculated for adapting the RL models to new experiences in both inner-and outer-loop optimization. For applying the proposed DropGrad scheme in the RL framework, we augment the policy gradients calculated according to rewards in the support set during the inner-loop optimization. We use a public Pytorch implementation with the default hyper-parameter setting in the experiments.3 

Reinforcement Learning Results. In Figure 4 , we present the rewards after the model is optimized with the few trajectories, i.e., in each iteration we perform one-step policy gradient update for the inner-loop optimization. In both environments, the training process with the proposed DropGrad regularization method converges to favorable rewards compared to the original training without the proposed regularization. This improvement could be attributed by the uncertainty on gradients that provide a better exploration of the policy.

4.4. ONLINE OBJECT TRACKING:

Visual object tracking targets at localizing one particular object in a video sequence given the bounding box annotation in the first frame. To adapt the model to the subsequent frames, one approach is to apply online adaptation during tracking. Object Tracking Results. The results of online object tracking on the OTB2015 dataset are presented in Table 4 . The one-pass evaluation (OPE) protocol without restarts at failures is used in the experiments. We measure the precision and success rate based on the center location error and the bounding-box overlap ratio, respectively. The precision is calculated with a threshold 20, and the success rate is the averaged value with the threshold ranging from 0 to 1 with a step of 0.05. We show that applying the proposed DropGrad method consistently improves the performance in precision and success rate on both MetaCREST and MetaSDNet trackers. for MAML (Finn et al., 2017) . Specifically, we conduct an ablation study on the learning rate α and the number of inner-loop optimizations n inner in MAML. As shown in Table 6 , the proposed DropGrad method improves the performance consistently under different sets of hyper-parameters. 

A.4 OBJECT ONLINE TRACKING:

We present sample results of object online tracking in Figure 5 . We apply the proposed DropGrad method on the MetaCREST and MetaSDNet methods and evaluate these models on the OTB2015 dataset. Compared with the original MetaCREST and MetaSDNet, models trained with the Drop-Grad method track objects more accurately.

A.5 COMPARISON TO SIMULATED ANNEALING:

The proposed DropGrad algorithm is also related to simulated annealing (SA) (Kirkpatrick et al., 1983) . While conceptually similar to a certain extent, the goals and formulations are significantly different. SA modulates gradients by exploring uncertain solutions with the goal to escape from the local minimum during the training stage. On the other hand, our DropGrad method, drops the inner gradient to introduce uncertainty in the forward pass of the gradient-based meta-learning framework.

:

https://github.com/wyharveychen/CloserLookFewShot

https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch

https://github.com/tristandeleu/pytorch-maml-rl

CONCLUSIONSIn this work, we propose a simple yet effective gradient dropout approach for regularizing the training of gradient-based meta-learning frameworks. The core idea is to impose uncertainty by augmenting the gradient in the adaptation step during meta-training. We propose two forms of noise regularization terms, including Bernoulli and Gaussian distributions, and demonstrate that the proposed DropGrad improves the model performance in three learning tasks. In addition, extensive analysis and studies are provided to further understand the benefit of our method. One study on cross-domain few-shot classification is also conducted to show that the DropGrad method is able to mitigate the overfitting issue under a larger domain gap.

https://github.com/silverbottlep/meta trackers

https://github.com/wyharveychen/CloserLookFewShot

Figures and Tables:

Figure 2: Comparison between the proposed Binary and Gaussian DropGrad methods. We compare the 1-shot (left) and 5-shot (right) performance of MAML (Finn et al., 2017) trained with two different forms of DropGrad under various dropout rates on mini-ImageNet. The proposed DropGrad method is particularly effective with the dropout rate in [0.1, 0.2]. Moreover, the Gaussian DropGrad method consistently obtains better results compared to the Binary DropGrad scheme.

Figure 3: Validation loss over training epochs. We show the validation curves of the MAML (left) and MetaSGD (right) frameworks trained on the 5-shot mini-ImageNet dataset. The curves validate that the proposed DropGrad method alleviates the overfitting problem.

Figure 4: Few-shot reinforcement learning results. Two settings, HalfCheetah-Dir (left) and Ant-Dir (right), are considered in our experiments using the MAML-TPRO framework. We show the reward curves after the model is updated with the few trajectories and both rewards converge favorably against the original training.

Figure 5: Qualitative results of object online tracking on the OTB2015 dataset. Top and bottom rows show the sample results of MetaCREST and MetaSDNet, respectively. Red boxes are the ground truth, yellow boxes represent the original results, and green boxes stand for the results where the DropGrad method is applied.

al., 2019; Li et al., 2017), and the initial parameters θ are not generic but conditional on the support set D s (Rusu et al., 2019).

Table content: <table/>

method in Algorithm 1, while similar procedures can be applied to other gradient-based meta-learning frameworks, such as MetaSGD(Li et al., 2017) and MAML++ (Antoniou et al., 2019).

Table content: <table/>

Few-shot classification results on mini-ImageNet. The Gaussian DropGrad method improves the performance of gradient-based models on 1-shot and 5-shot classification tasks.

Table content: <table><tr><td/><td>Model</td><td/><td/><td/><td/><td/><td>1-shot</td><td/><td>5-shot</td></tr><tr><td/><td colspan="3">MAML (Finn et al., 2017)</td><td/><td/><td colspan="2">49.61 ± 0.92%</td><td colspan="2">65.72 ± 0.77%</td></tr><tr><td/><td colspan="5">MAML w/ Gaussian DropGrad (p = 0.1)</td><td colspan="4">52.35 ± 0.86% 69.42 ± 0.73%</td></tr><tr><td/><td colspan="3">MetaSGD (Li et al., 2017)</td><td/><td/><td colspan="2">51.51 ± 0.87%</td><td colspan="2">69.67 ± 0.75%</td></tr><tr><td/><td colspan="5">MetaSGD w/ Gaussian DropGrad (p = 0.1)</td><td colspan="4">53.38 ± 0.93% 71.14 ± 0.72%</td></tr><tr><td/><td colspan="2">MetaSGD*</td><td/><td/><td/><td colspan="2">60.44 ± 0.87%</td><td colspan="2">72.55 ± 0.54%</td></tr><tr><td/><td colspan="9">MetaSGD* w/ Gaussian DropGrad (p = 0.1) 61.69 ± 0.84% 73.33 ± 0.57%</td></tr><tr><td/><td colspan="4">MAML++ (Antoniou et al., 2019)</td><td/><td colspan="2">50.21 ± 0.50%</td><td colspan="2">68.66 ± 0.46%</td></tr><tr><td/><td colspan="5">MAML++ w/ Gaussian DropGrad (p = 0.2)</td><td colspan="4">51.13 ± 0.50% 69.80 ± 0.46%</td></tr><tr><td/><td>1.6</td><td/><td>MAML</td><td/><td/><td>1.6</td><td colspan="2">MetaSGD</td><td/></tr><tr><td/><td>1.4</td><td/><td/><td/><td/><td>1.4</td><td/><td/><td/></tr><tr><td>Validation Loss</td><td>0.8 1 1.2</td><td/><td/><td/><td>Validation Loss</td><td>0.8 1 1.2</td><td/><td/><td/></tr><tr><td/><td>0.6</td><td/><td/><td/><td/><td>0.6</td><td/><td/><td/></tr><tr><td/><td>0.4</td><td/><td/><td/><td/><td>0.4</td><td/><td/><td/></tr><tr><td/><td>0</td><td>400</td><td>800</td><td>1200</td><td>1600</td><td>0</td><td>400</td><td>800</td><td>1200</td><td>1600</td></tr><tr><td/><td/><td/><td>Training Epoch</td><td/><td/><td/><td colspan="2">Training Epoch</td><td/></tr><tr><td/><td/><td/><td/><td/><td/><td/><td>w/o DropGrad</td><td/><td colspan="2">w/ DropGrad</td></tr></table>

Performance of applying DropGrad to different layers. We conduct experiments on the 5-shot classification task using MAML on mini-ImageNet. It is more helpful in improving the performance by dropping the gradients closer to the output layers (e.g., FC and Block4+FC).

Table content: <table><tr><td>Origin</td><td>FC</td><td>Block4 + FC</td><td>Full</td><td>Block1 + Conv</td><td>Conv</td></tr><tr><td>65.72 ± 0.77%</td><td>68.93 ± 0.55%</td><td>69.02 ± 0.57%</td><td>69.42 ± 0.73%</td><td>64.96 ± 0.80%</td><td>65.53 ± 0.75%</td></tr></table>

Cross-domain performance for few-shot classification. We use the mini-ImageNet and CUB datasets for the meta-training and meta-testing steps, respectively. The improvement of applying the proposed DropGrad method is more significant in the cross-domain cases than the intra-

Table content: <table><tr><td>domain ones.</td><td/><td/></tr><tr><td>Model</td><td>1-Shot</td><td>5-Shot</td></tr><tr><td>MAML (Finn et al., 2017)</td><td>31.52 ± 0.52%</td><td>45.56 ± 0.51%</td></tr><tr><td>MAML w/ DropGrad (p = 0.1)</td><td colspan="2">33.20 ± 0.67% 51.05 ± 0.56%</td></tr><tr><td>MetaSGD (Li et al., 2017)</td><td>34.52 ± 0.63%</td><td>49.22 ± 0.58%</td></tr><tr><td>MetaSGD w/ DropGrad (p = 0.1)</td><td colspan="2">36.77 ± 0.72% 55.13 ± 0.72%</td></tr><tr><td>MetaSGD*</td><td>43.98 ± 0.77%</td><td>57.95 ± 0.81%</td></tr><tr><td colspan="3">MetaSGD* w/ Gaussian DropGrad (p = 0.1) 45.33 ± 0.81% 59.94 ± 0.82%</td></tr><tr><td>MAML++ (Antoniou et al., 2019)</td><td>40.73 ± 0.49%</td><td>60.57 ± 0.49%</td></tr><tr><td>MAML++ w/ DropGrad (p = 0.2)</td><td colspan="2">44.27 ± 0.50% 63.79 ± 0.48%</td></tr><tr><td colspan="3">optimized. The DropGrad regularization method mitigates the overfitting issue and facilitates the</td></tr><tr><td>training procedure.</td><td/><td/></tr></table>

Precision and success rate on the OTB2015 dataset. The DropGrad method can be applied to visual tracking and improve the tracking performance.

Table content: <table><tr><td>Model</td><td colspan="2">Precision Success rate</td></tr><tr><td>MetaCREST (Park &amp; Berg, 2018)</td><td>0.7994</td><td>0.6029</td></tr><tr><td colspan="2">MetaCREST w/ DropGrad (p = 0.2) 0.8172</td><td>0.6145</td></tr><tr><td>MetaSDNet (Park &amp; Berg, 2018)</td><td>0.8673</td><td>0.6434</td></tr><tr><td>MetaSDNet w/ DropGrad (p = 0.2)</td><td>0.8746</td><td>0.6520</td></tr></table>

The Meta-Tracker (Park & Berg, 2018) method uses meta-learning to improve two state-of-the-art online trackers, including the correlation-based CREST (Song et al., 2017) and the detection-based MDNet (Nam & Han, 2016), which are denoted as MetaCREST and MetaSDNet. Based on the error signals from future frames, the Meta-Tracker updates the model during offline meta-training, and obtains a robust initial network that generalizes well over future frames. We apply the proposed DropGrad method to train the MetaCREST and MetaSDNet models with evaluation on the OTB2015 (Wu et al., 2015) dataset. Implementation Details. We train the models using the original source code. 4 For metatraining, we use a subset of a large-scale video detection dataset (Russakovsky et al., 2015), and the 58 sequences from the VOT2013 (Kristan et al., 2013), VOT2014 (Kristan et al., 2014) and VOT2015 (Kristan et al., 2015) datasets, excluding the sequences in the OTB2015 database, based on the same settings in the Meta-Tracker (Park & Berg, 2018). We use the default hyper-parameter settings and evaluate the performance with the models at the last training iteration.

Table content: <table/>

5-shot classification results of MAML under various hyper-parameter settings. We study the learning rate α and number of iterations n inner in the inner-loop optimization of MAML using mini-ImageNet dataset.

Table content: <table><tr><td>α, n inner</td><td>0.01, 5 (original)</td><td>0.1, 5</td><td>0.001, 5</td><td>0.01, 3</td><td>0.01, 7</td></tr><tr><td>MAML (Finn et al., 2017)</td><td>65.72 ± 0.77%</td><td>65.98 ± 0.79%</td><td>58.55 ± 0.80%</td><td>64.84 ± 0.80%</td><td>68.11 ± 0.74%</td></tr><tr><td>MAML w/ DropGrad (p = 0.1)</td><td>69.42 ± 0.73%</td><td>67.78 ± 0.73%</td><td>64.05 ± 0.79%</td><td>65.42 ± 0.80%</td><td>69.65 ± 0.70%</td></tr></table>

Peer review (iclr2020):
This paper introduces a regularization scheme for gradient-based meta-learning. The scheme proposed is DropGrad, where they randomly scale the values of the gradient by a Bernoulli or Gaussian distribution effectively preventing overfitting and enhancing generalization. The authors evaluate the proposed approach extensively in domains ranging from supervised learning to reinforcement learning, and with different variations of their method. The overall paper is well written and motivated. The experiment section is extensive and they show in different domains how their method performs better than without regularization. However, I feel that the overall contribution is not enough. First, their approach is not novel enough. Even dropout has not been applied to meta-learning, it has been applied to different methods and fields successfully. Second, there is no insight why this approach works or should be used. The authors just reference to the success of dropout on other works. However, applying the dropout to a gradient is inherently different than doing it to the activations. For instance, it can result in a direction that is not of descent. Finally, while the experimentation is exhaustive, the results do not show the proposed approach works substantially better than without regularization. In section 4.1 and 4.2, almost all the results are within a standard deviation of each other. Moreover, in section 4.1, you are applying gradient dropout to 	heta and 	heta’, I wonder why that choice since it seems that the proper way of doing it would be to just apply the dropout to 	heta’. In this section, the authors should also compare against other kinds of regularization, for instance gradient penalty, L2, etc… In section 4.3, the results shown are just one random seed, which given the noisiness of reinforcement experiments renders them not very meaningful. Furthermore, the main motivation of the paper is to avoid overfitting, but here, there are just two tasks. Therefore, regularization should not help unless the improvement is coming from somewhere else. The authors claim that the “improvement could be attributed by the uncertainty on gradients that provide a better exploration of the policy”, but they do not have evidence for corroborating that claim. The inner update on MAML-RL comes for vanilla policy gradient, which already has high variance. It would be interesting (not only in this experiment but in supervised learning as well) to see the variance of your gradient estimator and w/o the regularization. At this stage, this paper does not have enough delta on novelty, insights why the proposed method works (or should be used), nor results.