Title: INCREMENTAL FEW-SHOT LEARNING WITH ATTENTION ATTRACTOR NETWORKS

Abstract: Machine learning classifiers are often trained to recognize a set of pre-defined classes. However, in many real applications, it is often desirable to have the flexibility of learning additional concepts, without re-training on the full training set. This paper addresses this problem, incremental few-shot learning, where a regular classification network has already been trained to recognize a set of base classes; and several extra novel classes are being considered, each with only a few labeled examples. After learning the novel classes, the model is then evaluated on the overall performance of both base and novel classes. To this end, we propose a meta-learning model, the Attention Attractor Network, which regularizes the learning of novel classes. In each episode, we train a set of new weights to recognize novel classes until they converge, and we show that the technique of recurrent back-propagation can back-propagate through the optimization process and facilitate the learning of the attractor network regularizer. We demonstrate that the learned attractor network can recognize novel classes while remembering old classes without the need to review the original training set, outperforming baselines that do not rely on an iterative optimization process.

1. INTRODUCTION:

The success of deep learning stems from the availability of large scale datasets with detailed annotation, such as ImageNet (Russakovsky et al., 2015) . The need for such a large dataset is however a limitation, since its collection requires intensive human labor. This is also strikingly different from human learning, where new concepts can be learned from very few examples. One line of work that attempts to bridge this gap is few-shot learning (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017) , where a model learns to output a classifier given only a few labeled examples of the unseen classes. While this is a very promising line of work, its usability in practice can be a concern, because few-shot models only focus on learning novel classes, ignoring the fact that many common classes are readily available in large datasets.

An approach that aims to enjoy the best of both worlds, the ability to learn from large datasets for common classes with the flexibility of few-shot learning for others, is incremental few-shot learning (Gidaris & Komodakis, 2018) . This combines incremental learning where we want to add new classes without catastrophic forgetting (McCloskey & Cohen, 1989) , with few-shot learning when the new classes, unlike the base classes, only have a small amount of examples. One use case to illustrate the problem is a visual aid system. Most objects of interest are common to all users, e.g., cars, pedestrian signals; however, users would also like to augment the system with additional personalized items or important landmarks in their area. Such a system needs to be able to learn new classes from few examples, without harming the performance of the original classes and without access to the original dataset used to train these classes.

In this work we present a novel method for incremental few-shot learning where during metalearning we optimize a regularizer that reduces catastrophic forgetting from the incremental few-shot learning. Our proposed "attention attractor network" regularizer is inspired by attractor networks (Zemel & Mozer, 2001) and can be thought of as a learned memory of the base classes. We also show how this regularizer can be optimized, using recurrent back-propagation (Liao et al., 2018; Almeida, 1987; Pineda, 1987) to back-propagate through the few-shot optimization stage. Finally, we show empirically that our proposed method can produce state-of-the-art results in incremental few-shot learning on the challenging mini-ImageNet (Vinyals et al., 2016) and tiered-ImageNet (Ren et al., 2018) tasks.

2. RELATED WORK:

Recently, there has been a surge in interest in few-shot learning (Koch et al., 2015; Vinyals et al., 2016; Snell et al., 2017; Lake et al., 2011) , where a model for novel classes is learned with only a few labeled examples. One family of approaches for few-shot learning, including Deep Siamese Networks (Koch et al., 2015) , Matching Networks (Vinyals et al., 2016) and Prototypical Networks (Snell et al., 2017) , follows the line of metric learning. In particular, these approaches use deep neural networks to learn a function that maps the input space to the embedding space where examples belonging to the same category are close and those belonging to different categories are far apart. Recently, Garcia & Bruna (2018) propose a graph neural networks based method which captures the information propagation from the labeled support set to the query set. Ren et al. (2018) extends Prototypical Networks to leverage unlabeled examples while doing few-shot learning. Despite the simplicity, these methods are very effective and often competitive with the state-of-the-art.

Another class of approaches try to learn models which can, unlike metric methods, adapt to the episodic tasks. In particular, Ravi & Larochelle (2017) treat the long short-term memory (LSTM) as a meta learner such that it can learn to predict the parameter update of a base learner, e.g., a convolutional neural network (CNN). MAML (Finn et al., 2017) instead learns the hyperparameters or the initial parameters of the base learner by back-propagating through the gradient descent steps. Santoro et al. (2016) use a read/write augmented memory, and Mishra et al. (2018) combine soft attention with temporal convolutions which enables retrieval of information from past episodes.

Methods described above belong to the general class of meta-learning models. First proposed in Schmidhuber (1987); Naik & Mammone (1992) ; Thrun (1998) , meta-learning is a machine learning paradigm where the meta-learner tries to improve the base learner using the learning experiences from multiple tasks. Meta-learning methods typically learn the update policy yet lack an overall learning objective in the few-shot episodes. Furthermore, they could potentially suffer from shorthorizon bias (Wu et al., 2018) , if at test time the computation graph is unrolled for longer steps. To address this problem, Bertinetto et al. (2018) propose to use fast convergent models like logistic regression (LR), which can be back-propagated via a closed form update rule. Compared to Bertinetto et al. (2018) , our proposed method using recurrent back-propagation (Liao et al., 2018; Almeida, 1987; Pineda, 1987) is more general as it does not require a closed-form update rule, and the inner loop solver can employ any existing continuous optimizers.

Our work is also related to incremental learning or continual learning, a setting where information is arriving continuously while prior knowledge needs to be transferred. A key challenge here is catastrophic forgetting (McCloskey & Cohen, 1989; McClelland et al., 1995) , i.e., the model forgets the learned knowledge. Various forms of memory-based models have since been proposed, which either store training examples explicitly (Rebuffi et al., 2017; Sprechmann et al., 2018; Castro et al., 2018) , regularize the parameter updates (Kirkpatrick et al., 2017) , or learn a generative model (Kemker & Kanan, 2018) . However, in these studies, incremental learning typically starts from scratch, and usually perform worse than a regular model that is trained with all available classes together since the network has not learned a good representation yet while dealing with catastophic forgetting.

To make incremental learning more useful, Hariharan & Girshick (2017) ; Wang et al. (2018) ; Gidaris & Komodakis (2018) combine few-shot learning and incremental learning, where they start off with a pre-trained network on a set of base classes, and try to augment the classifier with a batch of new classes that has not been seen during training. This is also the setting we adopt in this paper. Hariharan & Girshick (2017) propose the squared gradient magnitude loss, which makes the learned classifier from the few-shot examples have a smaller gradient value when learning on all examples. Wang et al. (2018) propose the prototypical matching networks, a combination of prototypical network and matching network. The paper also adds hallucination, which generates new examples.

Both papers assume that the few-shot learning stage has access to the base classes data, which can be inflexible and inefficient due to the large size of the original training set. Gidaris & Komodakis (2018) remove this assumption and propose an attention based model which generates weights for novel categories. They also promote the use of cosine similarity between feature representations and weight vectors to classify images.

In contrast, during each few-shot episode, we directly learn a linear classifier that is solved till convergence, unlike Gidaris & Komodakis (2018) which directly output the prediction. Since in our settings the model cannot see base class data in each few-shot learning episode, different from Hariharan & Girshick (2017) ; Wang et al. (2018) , it is challenging to jointly classify both base and novel categories using a vanilla logistic regression. Towards this end, we propose to add a learned regularizer, which is learned by differentiating through few-shot learning iterations. We found that, despite the simplicity, using an iterative solver with the learned regularizer significantly improves the linear classifier model on the task of incremental few-shot learning.

3. ATTENTION ATTRACTOR NETWORK FOR INCREMENTAL FEW-SHOT LEARNING:

In this section, we first define the setup of incremental few-shot learning, and then we introduce our new model, the Attention Attractor Network, which attends to the set of base classes according to the few-shot training data, and learns the attractor energy function as an additional regularizing term for the few-shot episode. Figure 1 illustrates the high-level model diagram of our method.

3.1. INCREMENTAL FEW-SHOT LEARNING:

We first define the task of incremental few-shot learning in this section. In this paper, we consider the two stage learning model proposed by Gidaris & Komodakis (2018) .

Stage 1: We learn a base model for the regular supervised classification task on dataset D a = {(x a,i , y a,i )} Na i=1 where x a,i is the i-th example from dataset a and its labeled class y a,i ∈ {1, 2, ..., K}. The purpose of this stage is to learn both a good base classifier and a good representation. We denote parameters of the top fully connected layer as W a ∈ R D×K where D is the dimension of our learned representation.

Stage 2: A few-shot dataset D b is presented, from which we can sample few-shot learning episodes E. For each N -shot K -way episode, there are K novel classes disjoint from the base classes. Each novel class has N and M images from the support set S b and the query set Q b respectively. Therefore, we have

E = (S b , Q b ), S b = (x S b,i , y S b,i ) N ×K i=1 , Q b = (x Q b,i , y Q b,i ) M ×K i=1

where y b,i ∈ {K + 1, ..., K + K }. S b and Q b can be regarded as the training set and the validation set in a regular supervised learning setting. To evaluate the performance on a joint prediction of both base and novel classes, i.e., a

(K + K )-way classification, a mini-batch Q a = {(x a,i , y a,i )} M ×K i=1 from D a is also added to Q b to form Q a+b = Q a ∪ Q b .

During this stage, the learning algorithm has only access to samples from S b , i.e., few-shot examples from the new classes. But its performance is evaluated on the joint query set Q a+b . For simplicity, the base model including W a is fixed so that only W b ∈ R D×K is learned.

In meta-training, we iteratively sample few-shot episodes E and try to optimize the few-shot learning algorithm, i.e., stage 2, in order to produce good results on Q a+b . In this work, we learn a regularizer R(

•; θ E ) such that W * b = arg min W b Loss(W b , S b ) + R(W b ; θ E )

, and the meta-learner tries to learn θ E such that the resulting W * b performs well on Q a+b .

3.1.1. JOINT PREDICTION ON OLD AND NEW CLASSES:

When solving a few-shot episode, we consider learning a linear classifier on the novel classes in the support set S b . For notational convenience, we augment the hidden representation h as below such that the 0-th dimension of the weights is the original bias.

EQUATION

We use the concatenation of W a and W b as the fully-connected layer for classification,

ŷb,i = softmax([W a , W b ] h(x b,i )).

(2)

Here we emphasize that learnable parameters of ŷb,i is W b . During the learning stage of each fewshot episode, we treat it as a classification problem and aim to minimize the following regularized cross-entropy objective on the support set S,

EQUATION

3.2. ATTENTION ATTRACTOR NETWORKS:

Directly learning the few-shot episode, e.g., by setting R(W b ; θ E ) to be zero or simple weight decay, can cause catastrophic forgetting on the old classes. This is because W b is trying to minimize the probability of all the old classes, as it is trained only on new classes. In this section, we introduce Attention Attractor Networks to address this problem.

The attractor network adds an energy term E to the few-shot learning objective, R(W

EQUATION

where the optimized parameters W * b are functions of energy function parameters θ E and few-shot samples S b . During meta-learning, θ E are updated to minimize an expected loss of the query set Q a+b which contains both old and new classes, averaging over all few-shot learning episodes.

EQUATION

θ * E = arg min

EQUATION

Conceptually, the energy function regularizes the learning of W b so that it is compatible with W a during the joint prediction. In our proposed model each base class in W a has a learned attractor U k stored in the knowledge base matrix U = [U 1 , ..., U K ]. When a novel class k is seen, its classifier is regularized towards its attractor u k which is a weighted sum of U k . The weighting can be seen as an attention mechanism where each new class attends to the old classes according to the level of interference. Specifically, the attention mechanism is implemented by the cosine similarity function:

EQUATION

)

For each image in the support set, we compare it with the set of base weights W a , average over each support class and apply a softmax function,

EQUATION

where h j are the representations of the inputs in the support set S b and τ is a learnable temperature scalar. a k ,k encodes a normalized pairwise attention matrix between the novel classes and the base classes. The attention vector is then used to compute a linear weighted sum of entries in the knowledge base matrix U ,

EQUATION

)

The final energy function is defined as a sum of squared L 2 distance from the attractors,

EQUATION

)

where γ is a learnable vector, which defines a diagonal distance metric over the hidden dimensions.

Our proposed attention attractor network takes inspiration from attractor networks (Mozer, 2009; Zemel & Mozer, 2001) , where for each base class we learn an "attractor" that stores the relevant memory regrading that class.

In summary, θ E is a tuple of (U, U 0 , γ, τ ). The number of parameters is on the same order as a fully connected layer

W a . It is important to note that E(W b ; θ E ) is convex in W b , so the optimum W * b (θ E , S b

) is guaranteed to be unique and achievable.

3.3. LEARNING AN ENERGY FUNCTION USING RECURRENT BACKPROPAGATION:

As there is no closed-form solution to the regularized linear classification problem shown above, in each training step, we need to solve for L S to obtain W * b through an iterative optimizer. The question is how to efficiently compute ∂W * b ∂θ E , back-propagating through the optimization. For simple energy functions, we can unroll the iterative optimization process in the computation graph and use the backpropagation through time (BPTT) algorithm (Werbos, 1990) . However, the number of iterations for a gradient-based optimizer to converge can be on the order of thousands, and BPTT can be computationally prohibitive. Another way is to use the truncated BPTT (Williams & Peng, 1990) (T-BPTT) algorithm that optimizes for the initial T steps of gradient-based optimization, and is commonly used in meta-learning problems. However, when T is small the training objective is biased compared to the objective that considers the optimal W b .

Alternatively, the recurrent backpropagation algorithm (Almeida, 1987; Pineda, 1987; Liao et al., 2018) allows us to backpropagate through fixed-point iterations efficiently without unrolling the computation graph and storing intermediate activations. Consider a vanilla gradient descent process on W b with step size α. The difference between two steps Φ can be written as Φ(W

EQUATION

where J F,W * b denotes the Jacobian matrix of the mapping F evaluated at W * b .

Damped Neumann RBP To compute the matrix-inverse vector product (I -J ) -1 v in Equation 12, Liao et al. (2018) propose to use the Neumann series:

EQUATION

)

Algorithm 1 Learning an Energy Function for Incremental Few-Shot Learning

Require: θ0, Da, D b , N , K , h Ensure: θE 1: θE ← θ0; 2: for t = 1 ... T do 3: {(x S b , y S b )}, {(x Q b , y Q b )} ← SampleEpisode(D b , N, K ); 4: {x Q a+b , y Q a+b } ← SampleMiniBatch(Da, N K ) ∪ {(x Q b , y Q b )}; 5: repeat 6: ŷS b,j ← softmax([Wa, W b ] h(x S b,j )); 7: L S ← 1 N K i y S b,i log ŷS b,i + λ W b 2 2 + E(W b ; θE); 8: W b ← OptimizerStep(W b , ∇W b L S ); 9: until W b converges 10: ŷQ a+b,j ← softmax([Wa, W b ] h(x Q a+b,j )), ∀j; 11: L Q ← 1 2N K j y Q a+b,j

log ŷQ a+b,j ; // Backprop through the above converging process // A dummy gradient descent step 12:

W b ← W b -α∇W b L S ; 13: J ← ∂W b ∂W b ; v ← ∂L Q ∂W b ; g ← v; 14: repeat 15: v ← J v -v; g ← g + v; 16:

until g converges 17:

θE ← OptimizerStep(θE, g ∂W b ∂θ E ) 18: end for Note that the matrix vector product J v can be computed by standard backpropagation. Nonetheless, we empirically observe that directly applying the Neumann RBP algorithm leads to numerical instability. Therefore, we propose "damped Neumann RBP", which adds a damping term 0 < < 1 to I -J . It results in the following update: ṽ(n) = (J -I) n v. The damping constant can also be interpreted as an additional weight decay term of the energy function during the backpropagation steps

E B (W b ) = E(W b ) + 2α W b 2 2 .

In practice, we found the damping term with = 0.1 helps alleviate the issue significantly.

Algorithm 1 outlines the key steps for learning an energy function using RBP in the incremental few-shot learning setting.

4. EXPERIMENTS:



4.1. DATASETS:

We used few-shot image classification benchmarks, mini-ImageNet Vinyals et al. ( 2016) and tiered-ImageNet Ren et al. (2018) , and made modifications of the datasets to accommodate the incremental few-shot learning settings. For meta-learning we need to split between training base classes, and training classes from which we sample few-shot episodes. In tiered-ImageNet we split the 351 training classes to 200 base classes and 151 novel classes used to train the meta-learner. In mini-ImageNet this is problematic as there are only 64 training classes, therefore at each meta-learning episode we randomly erase 5 classes, considering the 59 left as our base classes and sampling a few-shot episode from the removed 5 classes. For validation/test we sample from the validation/test set of the base classes and sample episodes from separate validation and test classes. Further details about these datasets and the class splits are in the supplementary material.

4.2. EXPERIMENTAL SETUP:

In the first stage, we use a standard ResNet backbone (He et al., 2016) to learn the feature representation through supervised training. For mini-ImageNet experiments, we follow Mishra et al. (2018) and use a modified version of ResNet. For tiered-ImageNet, we use the standard ResNet-18 (He et al., 2016) , but replace all batch normalization (Ioffe & Szegedy, 2015) layers with group normalization (Wu & He, 2018) , as there is a large distributional shift from training to testing in tiered-ImageNet due to categorical splits. We used standard data augmentation, with random crops and random horizonal flips. All of our models in our implementations use the same pretrained model as the starting point for the second stage.

In the second stage of learning as well as the final evaluation, we sample a few-shot episode from the D b , together with a regular mini-batch from the D a . The base class images are added to the query set of the few-shot episode. The base and novel classes are maintained in equal proportion in our experiments. For all the experiments, we consider 5-way classification with 1 or 5 support examples (i.e. shots); the total number of query images per episode is 75×2 =150.

We use an L-BFGS (Zhu et al., 1997) to solve the inner loop of our models to make sure W b converges. For BPTT baselines, we unroll the computation graph using vanilla gradient descent with a fixed learning rate 1e-2. We use the ADAM (Kingma & Ba, 2015) optimizer for metalearning with a learning rate of 1e-3, which decays by a factor of 10 after 4,000 steps, for a total of 8,000 steps. We fix the RBP to 20 iterations and = 0.1.

4.3. EVALUATION METRICS:

We consider the following evaluation metrics: 1) overall accuracy on individual query sets and the joint query set ("Base", "Novel", and "Both"); and 2) decrease in performance during joint prediction within the base and novel classes, considered separately ("∆ a " and "∆ b "). Finally we take the average ∆ = 1 2 (∆ a + ∆ b ) as a key measure of the overall decrease in accuracy.

4.4. RESULTS:

We present the few-shot benchmark in Table 1 and2. On mini-ImageNet, we compare our models to other incremental few-shot learning methods as well as pure few-shot learning methods. Note that our model uses the same backbone architecture as Mishra et al. (2018) and Gidaris & Komodakis (2018) , and is directly comparable with their results. The "1st stage" baseline is a CNN trained on the base classes, where we train the new classes using logistic regression with weight decay.

In the incremental few-shot learning setting, we implemented and compared to two methods. First, we adapted ProtoNet (Snell et al., 2017) to incremental few-shot settings. For each base class we store a base representation, which is the average representation over all images belonging to the base class. During the few-shot learning stage, we again average the representation of the fewshot classes and add them to the bank of base representations. Finally, we retrieve the nearest neighbor by comparing the representation of a test image with entries in the representation store. In summary, both W a and W b are stored as the average representation of all images seen so far that belong to a certain class. We also compare to Dynamic Few-Shot Learning without Forgetting (LwoF) (Gidaris & Komodakis, 2018) . Here the base weights W a are learned regularly through supervised pre-training, and W b are computed using prototypical averaging. We implemented the most advanced variants proposed in the paper, which involves a class-wise attention mechanism.

The main difference to this work is that we use an iterative optimization to compute W b .

Shown in Table 1 and2, our proposed method consistently outperform the two approaches described above, particularly in "∆". Our method is significantly improved during the second stage training, compared to a baseline that has only been pre-trained on base classes in the first stage.

4.5. COMPARISON TO TRUNCATED BPTT:

An alternative way to learn the energy function is to unroll the inner optimization for a fixed number of steps in a differentiable computation graph, and then we can back-propagate through time. In fact, truncated BPTT is a popular learning algorithm in recent meta-learning approaches (Andrychowicz et al., 2016; Ravi & Larochelle, 2017; Finn et al., 2017; Sprechmann et al., 2018) .

In Table 6 , we report the performance of the truncated BPTT versions of our attention attractor model on 1-shot learning. Results on 5-shot are in the supplementary material. We unrolled the computation graph by 20 or 100 steps. In comparison, the RBP version also runs for only 20 truncation Neumann steps. At test time, we also test the BPTT learned model by solving the learned energy function until convergence, just like the RBP model. We can see that truncated BPTT versions have worse performance than RBP, since solving the fully connected layer weights requires more iterations than the number of unrolling steps. When we tried to solve the optimization to convergence at test time, we found that the BPTT model performance greatly decayed (colored in red), as they are only guaranteed to work well for a certain number of steps, and failed to learn a good energy function. In contrast, learning the energy function with RBP finds a good solution upon convergence, regardless of the optimizer being used.

4.6. ABLATION STUDIES:

To understand the effectiveness of each part of the proposed model, we consider the following baselines as control experiments:

• Plain logistic regression ("None") solves a logistic regression problem at each few-shot episode, without any form of weight decay other regularizers. • Scalar weight decay ("WD") has a constant λ for weight decay as the regularizer. The λ is chosen according to validation performance. • Learnable γ ("Gamma") learns γ, which affects the size of the attractor basin, while keeping the center of the attractor u at zero. "w/o W a " variant does not fine-tune W a in the second stage. • Fixed attractor learns a fixed attractor center, in addition to learning γ. No attention mechanism is employed here.

Experimental results are reported in Table 4 . While models have similar performance on individually predicting base and novel classes, since they are using the same pre-trained backbone, there are significant differences in terms of joint prediction. Adding γ, fine-tuning W a , adding attractors, and using our attention mechanism all contribute towards the final performance. In particular, on ∆, the full attention attractor model is 80% better compared to no regularization, and 10-18% relative to to the fixed attractor.

:

& $ % &

CONCLUSION AND FUTURE WORKIncremental few-shot learning, the ability to jointly predict based on a set of pre-defined concepts as well as additional novel concepts, is an important step towards making machine learning models more flexible and usable in everyday life. In this work, we propose an attention attractor model, which outputs an additional energy function by attending to the set of base classes. We show that recurrent back-propagation is an effective and modular tool for learning energy functions in a general meta-learning setting, whereas truncated back-propagation through time fails to learn functions that converge well. Our attention attractor network is an iterative model that learns to remember the base classes without needing to review examples from the original training set, outperforming baselines that only do one-step inference. Future directions of this work include sequential iterative learning of few-shot novel concepts, and hierarchical memory organization.

Figures and Tables:

Figure 1: Our proposed attention attractor network for incremental few-shot learning. A: In stage 1, we learn W a and the feature extractor CNN backbone through supervised pretraining. B: In stage 2 we learn W b on a few-shot episode through an iterative solver, to minimize cross entropy plus an additional energy term predicted by attending to the base class representation W a . C: The attention attractor network is learned end-to-end to minimize the expected query loss.

mini-ImageNet 64+5-way few-shot classification results

Table content: <table><tr><td>1-shot</td><td>5-shot</td></tr></table>

tiered-ImageNet 200+5-way few-shot classification results

Table content: <table><tr><td>1-shot</td><td>5-shot</td></tr></table>

Learning an attention attractor network using damped Neumann RBP vs. truncated BPTT. Models are on evaluated on the validation set of tiered-ImageNet. During evaluation, "*" models are solved till convergence using a second-order optimizer.

Table content: <table><tr><td>1-shot</td><td colspan="2">Base Base+</td><td>∆a</td><td>Novel</td><td>Novel+</td><td>∆b</td><td>Both</td><td>∆</td></tr><tr><td>BPTT 20 Steps</td><td colspan="2">61.41 61.16</td><td>-0.25</td><td colspan="4">42.09 ± 0.40 37.80 ± 0.37 -4.28 49.44</td><td>-2.27</td></tr><tr><td>BPTT 100 Steps</td><td colspan="2">61.41 61.08</td><td>-0.33</td><td colspan="4">44.49 ± 0.42 40.43 ± 0.38 -4.06 50.75</td><td>-2.20</td></tr><tr><td>BPTT 200 Steps</td><td colspan="2">61.43 60.86</td><td>-0.57</td><td colspan="4">45.49 ± 0.42 42.40 ± 0.40 -3.09 51.63</td><td>-1.83</td></tr><tr><td>BPTT 20 Steps *</td><td>61.65</td><td>6.02</td><td colspan="6">-55.63 27.08 ± 0.26 21.99 ± 0.24 -5.08 14.01 -30.36</td></tr><tr><td>BPTT 100 Steps *</td><td>61.46</td><td>5.23</td><td colspan="6">-56.22 27.70 ± 0.29 19.59 ± 0.26 -8.11 12.41 -32.17</td></tr><tr><td>BPTT 200 Steps *</td><td>61.52</td><td>5.62</td><td colspan="6">-55.89 29.31 ± 0.32 21.09 ± 0.30 -8.22 13.36 -32.06</td></tr><tr><td colspan="3">Ours (RBP 20 Steps) 61.52 61.11</td><td>-0.41</td><td colspan="4">48.72 ± 0.45 47.04 ± 0.45 -1.68 54.08</td><td>-1.05</td></tr></table>

Ablation studies on tiered-ImageNet. "Base+" and "Novel+" are the prediction accuracies on Base and Novel classes within a joint query set.

Table content: <table><tr><td>1-shot</td><td>Base Base+</td><td>∆a</td><td>Novel</td><td>Novel+</td><td>∆b</td><td>Both</td><td>∆</td></tr><tr><td>None</td><td colspan="4">61.63 59.61 -2.02 43.68 ± 0.40 34.59 ± 0.34</td><td>-9.10</td><td>47.10</td><td>-5.56</td></tr><tr><td>Best WD (λ=0)</td><td colspan="4">61.63 59.61 -2.02 43.68 ± 0.40 34.59 ± 0.34</td><td>-9.10</td><td>47.10</td><td>-5.56</td></tr><tr><td>Gamma Random</td><td colspan="7">61.84 61.28 -0.56 44.29 ± 0.40 18.66 ± 0.29 -25.63 39.97 -13.09</td></tr><tr><td>Gamma w/o Wa</td><td colspan="6">61.97 61.57 -0.40 48.05 ± 0.43 32.16 ± 0.38 -15.89 46.86</td><td>-8.14</td></tr><tr><td>Gamma</td><td colspan="4">61.68 60.43 -1.25 46.17 ± 0.42 43.39 ± 0.41</td><td>-2.77</td><td>51.91</td><td>-2.01</td></tr><tr><td>Fixed Attractor</td><td colspan="4">61.74 61.06 -0.67 45.96 ± 0.42 44.30 ± 0.42</td><td>-1.66</td><td>52.68</td><td>-1.17</td></tr><tr><td>Full Model</td><td colspan="4">61.52 61.11 -0.41 48.72 ± 0.45 47.04 ± 0.45</td><td>-1.68</td><td>54.08</td><td>-1.05</td></tr><tr><td>5-shot</td><td>Base Base+</td><td>∆a</td><td>Novel</td><td>Novel+</td><td>∆b</td><td>Both</td><td>∆</td></tr><tr><td>None</td><td colspan="4">61.75 52.74 -9.01 61.00 ± 0.39 59.82 ± 0.38</td><td>-1.18</td><td>56.28</td><td>-5.09</td></tr><tr><td colspan="5">Best WD (λ=5e-4) 61.62 58.48 -3.14 61.59 ± 0.39 57.93 ± 0.37</td><td>-3.65</td><td>58.21</td><td>-3.39</td></tr><tr><td>Gamma Random</td><td colspan="4">61.78 59.42 -2.36 61.20 ± 0.38 55.66 ± 0.36</td><td>-5.54</td><td>57.54</td><td>-3.95</td></tr><tr><td>Gamma w/o Wa</td><td colspan="4">62.00 57.99 -4.01 61.18 ± 0.40 59.00 ± 0.39</td><td>-2.18</td><td>58.49</td><td>-3.09</td></tr><tr><td>Gamma</td><td colspan="4">61.46 59.87 -1.59 61.72 ± 0.39 59.80 ± 0.39</td><td>-1.92</td><td>59.83</td><td>-1.76</td></tr><tr><td>Fixed Attractor</td><td colspan="4">61.78 60.82 -0.97 61.33 ± 0.39 60.06 ± 0.39</td><td>-1.28</td><td>60.44</td><td>-1.12</td></tr><tr><td>Full Model</td><td colspan="4">61.80 61.28 -0.53 63.40 ± 0.39 62.09 ± 0.39</td><td>-1.31</td><td>61.68</td><td>-0.92</td></tr></table>

Peer review (iclr2019):
 This work addresses incremental few-shot learning that learns novel classes without forgetting old classes, which is interesting and different from conventional few-shot learning that considers only the few-shot learning task of interest. This problem is also related closely to the important problem of life-long learning. This work presents an interesting framework based on meta-learning by learning to learn how to attend to the old classes using an attention mechanism. Experimental results also show improvement over two related works on incremental few-shot learning. The writing is quite clear. Some concerns, especially its novelty, are listed below. 1. The novelty appears to be limited. The presented framework looks quite similar to the recent work Spyros Gidaris and Nikos Komodakis. Dynamic few-shot visual learning without forgetting. CVPR*18 that addresses the same problem in a similar manner: 1) learn a base feature extractor and classifier; and then 2) attend to old classes also via meta-learning and attention mechanism. As mentioned by the authors, "The main difference to this work is that we use an iterative optimization to compute W_b". More discussions on the iterative optimization and why it matters may be helpful. Another related work is "Deep Meta-Learning: Learning to Learn in the Concept Space", Arxiv*18, that also relies on an external base classes for few-shot learning. Similar to the proposed research, it also learns a feature extractor and a classifier from the base classes, which are used to regularize the learning of novel classes, in an end-to-end meta-learning manner. Extending it for the incremental setting seems natural. 2. To learn a few novel classes, all U_k on old classes are relearned, which seems quite time-consuming with a large vocabulary of base classes. 3. To learn a few novel classes, old data on base classes are still required, which seems different from how humans learn -- humans learn novel concepts solely from a few examples without forgetting old concepts, without requiring examples on old concepts.