Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: ## Summary
Following Finn et al. 2019, this paper aims at solving incremental meta-learning problems. A new setup is proposed as illustrated in Fig 1, which is motivated by learning a model that is capable of generalizing to a new task with decreasing number of shots. For the offline and online settings of incremental meta-learning, this paper proposes the MAML-VS algorithm and the FTML-VS algorithm, which are based on Finn et al. 2017 and Finn et al. 2019. Offline and online experiments are conducted on 4 benchmarks showing good performance comparing to baselines, where the Contextual MiniImageNet is a new a dataset proposed by this paper. 

## Contributions
1. Proposed a practical setup for few-shot learning when the tasks are non-mutually exclusive.
2. Proposed the learning rate scaling method for variable shots.

## Issues
1. It is not clear when and why zero-shot should work if no information about that task is revealed. More details should be elaborated on this point. 
2. In online meta-learning, each task as a different loss function, while in Theorem 1, the result is based on the assumption that all loss functions are the same. Can your result as well as the learning rate scaling method extend to the general setting? 
3. The performance of MAML-VS on Omniglot is worse than MAML. Can you try MAML-VS on other mutually exclusive datasets such as MiniImageNet? 
review_title: A well-written paper and potentially a new setup for few-shot learning

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors aim to tackle the meta learning problem in online settings, with both training and testing samples received in a streaming fashion. The authors proposed both offline and online methods, MAML-VS and FTML-VS, built upon MAML and FTML, respectively. The key contribution is to learn meta-learning rates besides the initial network parameters. Overall, the proposed approach sounds.

The contributions of this paper are as follows:

1.	The proposed online and offline meta learning algorithms aim to tackle the few-shot meta-learning problems with variable amounts of data received in a stream.

2.	The paper shows the limitation of MAML and FTML in the online setting. It also provides theoretical results on the meta learning rates in Theorem 1.

3.	The authors have done extensive experiments to evaluate the effectiveness of the proposed approaches in various datasets, e.g., RAINBOW MNIST, Contextual Mini-Imagenet, etc.

To the current status of the paper, I have a few concerns below.

First, the proposed solutions build upon MAML and FTML, which seems incremental. The differences include (i) a modification on the objective function in eq. (1) for the streaming/online setting, and (ii) a meta-learned learning rate for better model convergence property. 

Second, in table 1 (offline setting), it shows that MAML-VS outperforms baselines in fewer-shot cases with Rainbow MNIST, and in 10/20-shot cases with Contextual imageNet. It is unclear how to explain this inconsistency. Overall, the proposed MAML-VS does not perform better than baselines in offline settings.

Moreover, it would be interesting to also show how the proposed approach works in reinforcement learning settings.

Minors:

In the pseudo-code in section5.2 (line 3), it should be $j=t$.

review_title: An Interesting work with meta-learned learning rate.

final_score: rating: 6: Marginally above acceptance threshold
review_content: ## Summary

The authors propose a meta-learning algorithm for online incremental settings where not only the tasks but also the points belonging to each task arrive in a sequential order. In order to effectively minimise the total amount of supervision required by the method (both for meta-training and learning each new tasks), the author extend a previous K-shots learning method to be able to deal with the variable-shots learning scenario that naturally arise when considering the sequential order in which the task datapoints are observed. In the experimental section they compare with standard meta-learning methods in a variable-shots task both offline and online. They also show that in the online case that their method is more efficient than empirical risk minimization.   

## Comments

The paper is well motivated and the writing is clear although the mathematical notation could be improved.

The key observation of the paper (which is quite interesting) is that in many real world scenario tasks and datapoints arrive in sequential order, and in that case, it make sense to treat the problem as a variable shot learning problem where the aim is to minimize the total amount of supervision. Based on this observation, the authors proposal is very simple: to modify standard meta-learning algorithms (e.g. MAML offline and FTML online) by rescaling the learning rate in the inner loop according to the variable mini-batch size. The relation between the learning rate and the mini-batch size is given by a function parameterized by two additional parameters, where the functional form is derived in theorem 1. Despite its simplicity, it seems to improve the performance especially in the online setting.

As for the experimental section, the offline experiments seems a bit artificial and it is not very clear what the application would be. Also, it is unclear why the results for MAML-VL do not appear in the main manuscript, specially since it seems to perform better. On the other hand, the online experiments are more compelling and demonstrate improvement with respect to the baseline. Here I am missing a discussion about the sensitivity of the algorithm to the hyper-parameter M. Finally, while the problem tackled in the paper is different from the online meta-learning with online algorithm in the inner loop (as the authors point out in the related work) it would be interesting to add to the experimental section some baselines from these family of methods since they can be used to solve the problem setting described in the paper. 

## Minors

* In the preliminaries a task is defined as a dataset, while I think it should be defined as a distribution from which a dataset is sampled by iid sampling a set of examples. 
* A task dataset is defined as \mathcal{D}=\{x_i, y_i\} implying that x_i represent the covariates of all the examples in the dataset. However, later on at the beginning of page 3 x_j is used as a single datapoint. I would recommend using x_{ij} to avoid ambiguities.
* At the end of section 2, in the equation for \theta_{t+1}, I think the minimization should be over \theta (not over \theta_j). Same in equation (2).
* At beginning of section 4.3 it should be added what VL stands for
* Appendix A: VS-Meta-Update receives "s" as input. However, "s" does not appear inside the function




review_title: Official review: Variable-Shot Adaptation for Incremental Meta-Learning

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper defined a new problem called “variable-shot adaptation for incremental meta-learning”. In the proposed problem, the data within each task arrives one data point at a time, and the goal of the model is to minimize the cumulative regret summed over all of the tasks. It also proposed an algorithm that aimed to address the problem by using a scaling rule for the learning rate that scales with the number of shots. This algorithm is evaluated on four datasets.

Pros:

1- The idea of using a scaling rule for the learning rate that scales with the number of shots is interesting. The authors also provided the proof in the appendix.

2- The method is technically sound, and the experiment results can show its efficiency in some of the settings.

Cons:

1- About the problem formulation. This paper defined a new problem named “variable-shot adaptation for incremental meta-learning”. For its formulation, I have the following questions. (a) The system receives one data point at each step (Section 3, Page 3). However, for most incremental learning systems, a batch of training samples arrives at each time, e.g., in [A] and [B]. I think the latter one is more realistic as the training samples in vision systems aren’t often collected one by one. So why do you use a one-by-one setting instead of the common setting used in [A] and [B]? (b) In Section 3, for the Regret_T, you use theta_t instead of theta_T. It means the system is only evaluated on the current task, and you don’t care whether the model forgets the previous knowledge or not. Instead, most existing class-incremental learning and continual learning methods [A, B, I] evaluated the last model theta_T on all previous tasks, i.e., f_t(U_t(theta_T, alpha, min{s, M})) for t=1,...,T. It is commonly agreed that an incremental learning system should have the ability to retain the knowledge for all previous tasks instead of only the current one. If the proposed system is used for learning 100 different tasks, it will need to store 100 different models, which is not realistic given the memory budget in incremental learning. Again, my question is why did this paper choose this different setting (instead of the commonly agreed/used one in related works)?

2- About the baselines. This paper used MAML and FTML as the baselines. It claimed that these baselines don’t work well on variable-shot cases, e.g., can not meta-train different initialization weights for different shot numbers using MAML. While, if so, why not use metric-based methods like [C] and [D]? These methods have proved to be very effective in recent few-shot learning papers [E] and incremental learning papers [B]. Besides, it can be definitely applied to variable-shot learning cases (and in a direct manner).

3- About the novelty. The idea of meta-learning the base learning rates is not novel. It has been widely applied in many related papers, e.g., [F], [G], and [H]. [F] meta-learns base-learning rates for all base-learner parameters. [G] meta-learns layer-wise learning rates. [H] meta-learns a deep model to generate task-specific base learning rates. As a summary, the contribution of this submission is incremental, as it only simplifies the original MAML-VL to MAML-VS for the variable-shot settings. 

[A] Tao, Xiaoyu, et al. "Few-Shot Class-Incremental Learning." CVPR 2020.

[B] Rebuffi, Sylvestre-Alvise, et al. "icarl: Incremental classifier and representation learning." CVPR 2017.

[C] Snell, Jake, et al. "Prototypical networks for few-shot learning." NeurIPS 2017.

[D] Vinyals, Oriol, et al. "Matching networks for one shot learning." NeruIPS 2016.

[E] Chen, Yinbo, et al. "A new meta-baseline for few-shot learning." arXiv preprint arXiv:2003.04390 (2020).

[F] Li, Zhenguo, et al. "Meta-sgd: Learning to learn quickly for few-shot learning." arXiv preprint arXiv:1707.09835 (2017).

[G] Antoniou, Antreas, et al. "How to train your MAML." ICLR 2019.

[H] Liu, Yaoyao, et al. "An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning." ECCV 2020.

[I] Li, Zhizhong and Derek Hoiem. "Learning without forgetting." IEEE TPAMI 2017.

review_title: The "new" task is not well-motivated; A more reasonable baseline could already tackled the "new" task; Meta-learning LR is not novel.

Meta-review: For meta-learning with variable shot, this paper proposes a method for adapting the learning rate by a function of the number of training examples. The functional form is theoretically derived, and the method is simple and effective. However, meta-learning methods that adapt learning rates have been proposed, and the novelty is not high enough.
Decision: Reject