Title: META LEARNING WITH FAST/SLOW LEARNERS

Abstract: Meta-learning has recently achieved success in many optimization problems. In general, a meta learner g(.) could be learned for a base model f (.) on a variety of tasks, such that it can be more efficient on a new task. In this paper, we make some key modifications to enhance the performance of meta-learning models. (1) we leverage different meta-strategies for different modules to optimize them separately: we use conservative "slow learners" on low-level basic feature representation layers and "fast learners" on high-level task-specific layers; (2) Furthermore, we provide theoretical analysis on why the proposed approach works, based on a case study on a two-layer MLP. We evaluate our model on synthetic MLP regression, as well as low-shot learning tasks on Omniglot and ImageNet benchmarks. We demonstrate that our approach is able to achieve state-of-the-art performance.

1. INTRODUCTION:

The current success of deep learning hinges on the ability to apply gradient-based optimization in high-capacity models. It has achieved impressive results on many large-scale supervised tasks such as image classification (Krizhevsky et al. (2012) ; He et al. (2016) ; Szegedy et al. (2017) ) and games (et.al. (2016; 2013) ). Notably, although these models achieve superior performance, they require huge amount of data and iterations. In contrast, humans can rapidly learn a novel concept from only a few examples. The ability of fast knowledge acquisition might be related to the meta learning process in the human brain (Harlow (1949) ).

To improve learning efficiency, a seminal work named "learning-to-learn", or "meta-learning" (Andrychowicz et al. (2016) ) has been proposed to accelerate the learning process. Suppose in a machine learning task, we aim to optimize an objective f (θ) defined over some domain θ ∈ Θ. At step t, rather than applying stochastic gradient descent (SGD) as θ t+1 = θ t -α t ∇f (θ t ), a meta learner g can be learned to guide the training process of f :

EQUATION

An interesting phenomenon has been observed in (Andrychowicz et al. (2016) ): although successful on MNIST, meta-learning does not perform as well on CIFAR-10. This issue becomes more severe in case of large-scale learning tasks such as ImageNet Training (Russakovsky & et.al. (2015) ). To alleviate this issue, a more constrained form of parameter update model is introduced in (Wichrowska et al. (2017) ) to improve generalization.

The reason of failure, in our opinion, may lie in that, in a CNN, the low-level convolutional and high-level fully-connected modules play different roles in visual recognition. The former tend to learn generic features across categories, while the latter are generally more task-specific. This phenomenon of feature transferability has been studied in (Yosinski et al. (2014) ; Azizpour et al. (2016) ). If we apply the same learning strategy for all modules, some parameters may not converge well. Accordingly, we explore an improved multiple meta-strategies model. Drawing inspiration from the common trick of "fine-tune" across tasks, we leverage a conservative "slow-learner" for low-level modules and a "fast-learner" for high-level modules to enable rapid learning.

Notably, another factor enabling fast learning is that human-being could rely on prior knowledge and memories. Low-level perceptional modules could be inheritable characteristics preservable in natural selection over successive generations, undergoing a lifelong learning process (Briscoe & Chittka. (2001) ). Accordingly, we adopt a lifelong learning scheme for the slow-learner with parameter shared across new meta-learning tasks. The assumption we make here is that the generic low-level representation is a good prior "knowledge", which could in turn enables us to learn "smarter". A high level view of our fast/slow meta-learning is shown in Figure 1 .

1.1. CONTRIBUTIONS:

The main contributions of our work are three-folds:

1. We propose a fast/slow meta-learning approach to optimize a convolutional neural network. By applying different learners on different modules, our model could further improve the meta-learning performance method; 2. We provide theoretical study on why the proposed meta-learning on a two-layer MLP; 3. Experiments on the synthetic, Omniglot and ImageNet datasets validate our model.

2. RELATED WORK:

Early works on learning-to-learn or meta-learning can be dated back to (Mitchell & Thrun (1993) ; Hochreiter et al. (2001) ). The problem can generally be formulated as a two-level learning task: a meta-level model across tasks and a specific-level model within each task. The former acquires generic knowledge, ideally transferrable to the latter. The specific and meta-level models could be framed as a single learner or separate learners. Along this line, a series of work Vilalta & Drissi. (2002); Baxter. (1998) were proposed. Perhaps, the most general framework is (Schmidhuber. (1993)), which considers a neural network able to modify their own weights.

Recently, a seminal work (Andrychowicz et al. (2016) ) leverages one recurrent neural network ("optimizer") to train another base neural network ("optimizee"). The proposed model achieves better learning curves than hand-designed optimizers such as SGD and ADAM (Kingma & Ba (2014) ). Our model improves upon their work by explicitly applying fast/slow meta-learners on different modules in the base neural network. (Li & Malik (2017) ) relied on policy search to compute metaparameters. (Wichrowska et al. (2017) ) constructed a hierarchical RNN to capture different levels of dependencies.

In particular, meta-learning has shown strong performance in one-shot or low-shot learning tasks, due to its ability to directly generate weights for the base neural network (Woodward & Finn. (2017) ). Meta learning with LSTM learners (Ravi & Larochelle. (2017) ; Munkhdalai & Yu (2017) ) and Model-Agnostic learners (Mishra et al. (2017); Finn et al. (2017b; a) ) both achieve SOA performance. Additionally, memory-augmented meta-learning (Santoro et al. (2016) ; Kaiser et al. (2017) ; Vinyals et al. (2016) ; Munkhdalai & Yu (2017) ) also produce superior result on low-shot benchmarks.

The most related work might be (Munkhdalai & Yu (2017) ), where slow/fast weights are also proposed. The main differences are: (1) our model applies lifelong slow learners with parameter sharing; (2) for a specific parameter, we apply either a slow or a fast learner; in contrast, both learners are applied and combined in (Munkhdalai & Yu (2017) ). Our model is closely related to MAML (Finn et al. (2017a) ) where a good initialized representation is learned and shared across tasks to adapt to new domains.

Meta-learning has also shown promising performance on various tasks such as reinforcement learning (Wang et al. (2017) ; Duan et al. (2017); Finn et al. (2017b) ). It can be even generalized to the scenario where gradients are not available (Chen et al. (2017) ).

3 META-LEARNING WITH SLOW/FAST LEARNERS Following (Andrychowicz et al. (2016) ), we briefly introduce the notations. Suppose we try to optimize our "optimizee" f (θ, θ ), where θ and θ stand for different modules in f . In our paper we focus on f as a CNN, where θ and θ are bottom convolutional and top fully-connected modules.

We aim to train a pair of optimizers θ * (f, φ) and θ * (f, φ ) parametrized by φ and φ . Then, given a distribution of task functions f , the expected loss can be formulated as: We sketch our algorithm in Table .1. A typical setup of our meta-optimizer on a CNN is shown in Figure 2 : the modules are decoupled into bottom convolutional θ = {W conv , ...} and fullyconnected ones θ = {W f c , ...}. We apply a lifelong-slow strategy on the former, and fast metalearning on the latter separately. The intuition is that, the more "knowledgeable" θ gets, the better the fast meta-learner adapts to a new concept θ. Next, we will carry out a case study on a synthetic shallow network to demonstrate why our approach works. 

L(φ, φ ) = E f {f [θ * (f, φ), θ * (f, φ )]} (2)

θj+1 = θj + g( ∂L ∂θ | (θ j ,θ j ) , φ) θ j+1 = θ j -δ ∂L ∂θ | (θ j+1 ,θ j ) Update the parameter φ in fast learner g(.):φi+1 = φi -δ ∂ i L i (φ)

∂φ

Table 1 : The proposed Fast/Slow Meta-Learning model. The generic slow modules θ is only initialized once and shared across tasks. During testing, we fix the parameter θ and {φ, φ }, and apply the learned fast-learner g(., φ) to update θ.

by sampling a new W ∈ R 1×d as our ground truth. We aim to predict {W f , W s } from noisy labels y = W x + , where is a Gaussian noise with zero mean.

With batch size k, we denote the inputs and outputs as

X = [x 1 , x 2 , ..., x k ] and Y = W X + = [y 1 , ..., y k ].

Then, denoting the parameters of first and second linear modules as W s ∈ R d×d and W f ∈ R 1×d , our regression task can be formulated as:

L = 1 k [ 1 2 i (W f W s x i -y i ) 2 ]

and ideally the miracle solution W s and W f satisfying:

W f = W W -1

s could be learned on a new task W as soon as possible. At convergence, we should have:

W f W s = W = (XX T ) -1 XY T = Σ -1

x XY T which is an second-order optimizer. If we are constrained to apply a first-order optimizer (SGD) to optimize W s and W f , the solution undergoes a zig-zag path and takes many steps to converge, depending on the conditioned number of the covariance matrix Σ x .

3.1.1. A PERFECT:

W s MAKES W f CONVERGE WITHIN 1-STEP

To simplify notations, we denote S = W T s W s for all the following analysis. Lemma 1 If the following condition satisfies:

S ∝ Σ -1

x then suppose we initialize W f with 0, applying one-step SGD on W f could make the optimization converge s.t.

W f W s = W .

Proof 1 The gradient of L with respect to W f is:

∂L ∂W f = 1 k i (W f W s x i -y i )x T i W s

If we initialize with W f = 0, we have:

∂L ∂W f = -E{x T W T s y} = -E{ W xx T W T s } (∵ y = W x + ) = -E{ W ΣW T s } = -E{ W ΣW T s W s W -1 s } ∝ -E{ W W -1 s } (∵ S = W T s W s ∝ Σ -1 x )

The direction of the gradient perfectly aligns with the ground truth solution W f = W W -1 s . With an appropriate choice of learning rate for W f , SGD makes W f converge in one step.

We can interpret Lemma 1 as: if we have a miracle representation h = W * s x as a whitening operator, the top module W f can be learned by SGD within one step, regardless of the task target W .

3.1.2. FAST/SLOW META-LEARNING MAKES W s CONVERGE TO MIRACLE:

Since a miracle W s (i.e., W s is the whitening matrix of input x) makes W f adapt to new tasks, it is of interest if we could learn the miracle W * s ? Next, we demonstrate that the proposed fast/slow meta-learning model could converge to the good solution.

Following previous analysis, the gradient of our top module W f with respect to loss L given a batch {X 1 , Y 1 } of batch size k is:

∂L ∂W f = 1 k (W f W s X 1 -Y 1 )X T 1 W T s

Suppose we always initialize W f = 0. Then suppose the fast-learner is a SGD with learning rate η, we have the updated W f as:

W f = η k Y 1 X T 1 W T s (3)

To improve generality, we ideally expect the new W f perform well on a new sampled batch {X 2 , Y 2 }:

L = 1 2 (W f W s X 2 -Y 2 ) 2 = 1 2 ( η k Y 1 X T 1 W T s W s X 2 -Y 2 ) 2 (∵ Eqn(3)) = 1 2 ( η k Y 1 X T 1 SX 2 -Y 2 ) 2 (∵ W T s W s = S)

Lemma 2 We denote A = E( W W T ) as the expectation of weight covariance matrix across tasks.

If we follow the fast/lifelong-slow meta-learning strategy, at convergence S = W T s W s satisfies:

S -1 = η(k + 1)(Σ x + tr(Σ x + A) + 2 k + 1 A -1 ) (4)

where k is the batch size of {X 2 , Y 2 }.

Proof 2

E( ∂L ∂S ) =ηE{X 1 Y T 1 ( η k Y 1 X T 1 SX 2 -W X 2 -)X T 2 } =ηE{X 1 Y T 1 ( η k Y 1 X T 1 S -W )X 2 X T 2 } =ηE{X 1 (X T 1 W T + )( η k ( W X 1 + )X T 1 S -W )}Σ x = η 2 k E{X 1 (X T 1 W T W X + 2 )X T 1 )SΣ x } - η k E{X 1 X T 1 W T W }Σ x =η((k + 1)Σ x AΣ x ) + (tr(Σ x A) + 2 )Σ x )SΣ x -ηΣ x AΣ x (Lemma -3) =ηΣ x A(η((k + 1)Σ x + (tr(Σ x A) + 2 )A -1 )S -I)Σ x

At convergence, an equilibrium should be reached where E{ ∂L ∂S } = 0, then:

S -1 = η(k + 1)(Σ x + tr(Σ x + A) + 2 k + 1 A -1 )

In the proof, we need the following lemma:

Lemma 3 If x ∼ N (0), then for a matrix A, we have:

E(xx T Axx T ) = Σ x AΣ x + Σ x A T Σ x + tr(Σ x A)Σ x (5)

and if X = [x 1 , x 2 , ...] with each column sampled from an i.i.d. Gaussian x i ∼ N (0, Σ x ), then:

E(XX T AXX T ) = k(k + 1)Σ x AΣ x + Σ x A T Σ x + ktr(Σ x A)Σ x (6)

Proof 3 The moment-generating function of x is φ(t) = exp( t T Σxt 2 ), then for index {i, j, k, l} we have:

E(x i x j x k x l ) = ∂ 4 φ(t) ∂t i ∂t j ∂t k ∂t l = σ i σ j σ k σ l (7)

Hence,

EQUATION

E(XX T AXX T ) =( k i=1 x i x T i )A( k j=1 x j x T j ) = k i=1 k j=1 x i x T i Ax j x T j =k(k -1)Σ x AΣ x + 2kΣ x AΣ x + ktr(Σ x A)Σ x =k(k + 1)Σ x AΣ x + ktr(Σ x A)Σ x

3.2. THE INSIGHT OF THE CASE-STUDY:

The above analysis provides why the fast/slow meta-learning works:

1. A miracle knowledge of input signal x (i.e., W s whitens x), makes top module W f learnable in one SGD step for a new task; (Lemma 1) 2. Alternating lifelong-slow W s and fast W f learning, we are able to learn W s close to the miracle knowledge of x with a regularization term. (Lemma 2)

4. EXPERIMENT:

We evaluate our algorithm on three datasets: synthetic two-layer neural network, the Omniglot lowshot benchmark and the large-scale ImageNet dataset.

4.1. SYNTHETIC DATASET:

Our first experiment is to verify our theoretical analysis on the two-layer MLP as mentioned in the last section. We set the dimension as d = 10, such that input data x ∈ R 10 , each sampled new task has a ground truth W ∈ R 1×10 , W s ∈ R 10×10 and W f ∈ R 1×10 . The input x is sampled from an i.i.d. Gaussian x ∼ N (0, Σ x ) with Σ x fixed across all tasks; each new task is sampled from another Gaussian W ∼ N (0, I). For each task, we set the batch size as k = 100.

For the slow learner g (.), we use a conservative SGD with the learning rate δ = .001; for the fast learner g(.), we use a large-step SGD with a learning rate η = 0.2. In Figure 4 , we plot the learningcurve of the regression problem. As shown in Figure 4 (a), we show the performance gain defined as the relative error of 1-step SGD update:

L1 L0 = ||Y -WsW f,1 X|| 2 ||Y -WsW f,0 X|| 2 .

We can see that as the slow learner gradually captures the distribution of input signal x, the fast learner could be very efficient (L 1 /L 0 decreases from 0.8 to 0). In Figure 4 (b), we evaluate how well W s captures the knowledge of x throughout our fast/slow meta learning process. The metric we use is the Frobenius-norm between W T s W s and the true whitening matrix:

||(W T s W s ) -1 η Σ -1 x || 2 F rob

We can see clearly that W T s W s approach the true precision matrix Σ -1 /η, i.e., the slow learner evolves to capture the input distribution "magically" after 100,000 iterations.

Summary::

The experiments aligns perfectly with our theoretical analysis in above sections: the combination of a slow learner and a fast learner achieves state-of-the-art performance.

4.2. VISUAL RECOGNITION:

We briefly introduce the setup of our fast/slow meta-learning on visual classification tasks. We divide the datasets into 3-split: on C base classes, we first pre-train a base CNN f (θ, θ ) ; then on C meta , we train our fast meta-learner g(φ) to generate θ and fine-tune our lifelong slow parameters θ with a slow-learner (SGD); on C test we test our model. For our fast learner g(., φ), we use a coordinate-wise LSTM with 10-steps.

To evaluate our approach in comparison to prior meta-learning and low-shot visual algorithms, we apply our algorithm on the Omniglot benchmark (Lake et al. ( 2011)) and MiniImageNet, which are most common recently-used low-shot benchmarks. Omniglot contains C base = 964 characters in the training set, and we divide the 1,200 categories in the testing set into C meta = 600 and C test = 600. The MiniImagenet task proposed by (Ravi & Larochelle. (2017) ) contains 100 categories with C base = 64, C meta = 16 and C test = 20 respectively. We follow the experimental protocol of (Vinyals et al. (2016) ) such as multiple 90 degree rotation for Omniglot.

For a fair comparison, we follow the same model architecture as in Matching Network (Vinyals et al. (2016) ) and MAML (Finn et al. (2017a) ). For Omniglot, we use a simple yet powerful CNN, containing 4 modules with 3 × 3 convolution with 64 filters followed by batch-normalization (Ioffe & Szegedy (2015) ), ReLU and 2 × 2 max-pooling. For MiniImagenet, we use 32 filters per layer to avoid overfitting as done in (Ravi & Larochelle. (2017); Finn et al. (2017a) ). A top fully-connected layer θ = {W f c } followed by a softmax non-linearity is applied to define the Baseline Classifier.

The performance comparison of different approaches is shown in Table 2 . Our approach is highly competitive with the State-Of-the-Art (SOA) on both Omniglot and MiniImagenet.

FullImageNet::

Finally, it is of interest to test how our algorithm combined with SOA powerful CNN on the most challenging real-life scenarios-Full ImageNet. We use the Inception-ResNet-V2 model (Szegedy et al. (2017) ) as our base model. The task contains 21,841 categories (non-empty synsets) and the total number of images up to 14,197,122. Generally, the classification performance is tested on the 1,000 base classes, with 1,280,000 images. We adopt as our base CNN model and test our algorithm in the one-shot scenario on the remaining 20,841 unseen categories. We use half of the data for meta-learning training and the other half for testing.

Model:

Omniglot (20-way, 1-shot) MiniImagenet (5-way, 1-shot) Pixel-KNN 26.7% -MANN (Santoro et al. (2016)) 36.4% -CNN (Our implementation) 85.0% 28.8% Siamese (Koch et al. (2015)) 88.1% 40.1% Feature Shrinking (Hariharan & Girshick (2017)) 89.5% 44.0% Matching Network (Vinyals et al. (2016)) 93.8% 46.6% Learning to remember (Kaiser et al. (2017)) 95.0% -MAML (Finn et al. (2017a)) 95.8% 48.7% Prototypical Net (Snell et al. (2017) 96.0% -Meta Networks (Munkhdalai & Yu (2017)) 97.0% -Temporal Conv (Mishra et al. (2017)) 97.6% -Fast/Slow (Ours) 96.8% 48.8%

Table 2 : Performance comparison of our algorithms and the state-of-the-art on Omniglot and Mini-ImageNet. As shown in Figure 5 , we plot the 1-shot classification accuracy on the novel classes with respect to class number (for example, |class| = 10 means we carries out a 1-shot 10-way task). We can see our algorithm (red) steadily outperforms the strong baseline (CNN with metric learning).

5. CONCLUSION:

In this paper, we propose a fast/slow meta-learning approach to optimize a base convolutional neural network. By applying different strategies on bottom-generic/top-task-specific modules, our approach further improve the meta-learning performance. Theoretical analysis on a two-layer MLP regression problem, and extensive experiments on the synthetic, Omniglot one-shot, as well as Ima-geNet benchmarks validate the effectiveness of our approach.

6. LIMITATIONS AND FUTURE WORK:

A strong assumption made in our theoretical analysis is that the input data observes the same distribution across tasks x ∼ N (0, Σ). However, this does not necessarily hold true. In case the data from some new categories observe a different distribution, we might further extend our model to a hierarchy:

x i ∼ P z (θ ) z ∈ {1, ..., m} where we assume the input signal x i observes a mixture model and the latent variable z could be regarded as part of the slow module z ∈ θ . In the future, we will explore how to extend our model to the scenario of a mixture model.

:



Figures and Tables:

Figure 1: The computational graph of our fast/slow learning model: to optimize the cost function f (θ, θ ), we apply different strategies on θ, θ separately. We leverage a lifelong-slow learning scheme (SGD) on θ to acquire generic knowledge and a fast learning scheme g(.) on task-specific θ. [Photo: inspired by (Andrychowicz et al. (2016))]

Figure 3: Case study: a two-layer linear MLP with two FC layers.

Figure 4: Experiments on the two-layer linear MLP. (a) x-axis: fast/slow training epochs; verticalaxis: Relative loss L(W f,1 )/L(W f,0 ) after one-step fast module update with a large learning rate η = 0.2; (b) x-axis: fast/slow training epochs; vertical-axis: Frobenius-norm of the estimation difference between slow module θ = {W s } and the true whitening precision matrix Σ -1x .

Figure 5: FullImageNet: 1-shot Classification Accuracy w.r.t. class number. Red line: our metalearning; blue line: CNN metric-learning.

Peer review (iclr2019):
 [Summary:] This paper presents a meta-learning architecture where the slow learner is trained by SGD and the fast learner is trained according to what the meta-learner guides. CNN is split into two parts: (1) bottom conv layers devoted to learn meaningful representation, which is referred to as slow learner; (2) top-fully connected layers involving task-specific fast learners. As in [Andrychowicz et al., 2016], the meta-learner guides the training of task-specific learners. In addition, slow learners are trained by SGD. The motivation is that low-level features should be meaningful everywhere while high-level features should vary wildly. They introduce “miracle representations” and prove that fast/slow learning on a two-layer linear network should converge to somewhere near this miracle representation. They evaluate on few-shot classification benchmarks to evaluate how well this fast/slow meta-learning approach works. [Strengths:] The paper has a clear motivation. It is easy to read. Training slow/fast learners using different strategies is an interesting idea. [Weaknesses:] - The technique used in this work is a mix of SGD and [Andrychowicz et al., 2016]. - The analysis is limited to a simple two-layer linear network. It is not clear whether this analysis is carried over to the proposed deep nets. - Quantitative results did not compare to recent results such as Reptile[1] or MT-Nets[2]. [Specific comments:] - The current work is an improvement over [Andrychowicz et al., 2016], claiming that training conv layers and fully-connected layers with different strategies improves the generalization. I am wondering why the comparison to [Andrychowicz et al., 2016] is missing. You can use (fully) pre-trained CNN (which already learns meaningful representation using a huge amount of data) in the framework of [Andrychowicz et al., 2016]. -As one of the points of the paper is that this meta-learning strategy enables life-long learning, it would have been nice to see an experiment using this, where the distribution of tasks changes as time goes on. -The paper says SOA(State Of the Art); I think the term SOTA(State Of The Art) is more commonly used. -The use of the term “miracle” keeps changing(miracle solution, miracle representation, miracle W, miracle knowledge); the paper would be clearer if only one “miracle X” was defined and used as these are all essentially saying the same thing. References [1]https://arxiv.org/abs/1803.02999 [2]https://arxiv.org/abs/1801.05558