Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a setup of few-shot continual active learning (foCAL), where the agent continually learns with an unlabeled data and limited amount of labeling budget. This problem setup is meaningful, because previous continual learning methods focused on the scenario in which the data of the current task are available and labeled. In contrast, the new setup assumes that the robot doesn't have access to the labeled data, and needs to ask human teacher for supervision with limited availability (hence, active learning). This paper proposes FoCAL for online learning for image classification task. The framework is well summarized in the Figure 1. The features extracted from unlabeled objects go through the acquisition function a(,) to get the most informative samples (k), which are labeled by oracle. These labeled samples from oracle are used to update GMM representations. Pseudorehearsal process is used to replay old data. In the final stage of classifier, the samples from old classes and labeled features are used. Framework design description is given, and the experiment validations are presented. 
strengths_and_weaknesses: - Strength of this paper is the novelty of using GMM (Gaussian mixture model) based continual learning. The authors argue that the model system should recognize how different an incoming object (how new class it is) compared to other previously learned object classes with ease. To address this, they use a clustering-based approach; however, clustering-based approach normally uses mean feature vectors to represent object classes; instead, they use GMM to model each class data. Once the oracle gives label on the selected k feature vectors, the GBCL is applied to learn GMMs for the class. The updates of centroid is done by weighted mean of previous centroids and new inputs; the updates of covariance matrix is being done similarly. 
- Another notable part of the paper is the section 2.2. active learning using GMMs. Especially, FoCAL employs two techniques for active learning. First technique is the use of prediction entropy as the acquisition function. Second technique is the use of viewpoint consistency; inconsistent predictions for the different view of the same objects generates a high reward for acquiring the label of the object, as opposed to the consistent predictions.
- Weak points: Is there an ablation studies for using these two techniques? 
- Experiments and evaluations of FoCAL on Core-50 dataset, and evaluations on Pepper robot are presented. 
- Comparison baselines are sufficient (FT, LWF, EWC, CWR, etc) 
questions: Q1. I think the section 2.2. active learning using GMMs is the important part of the paper, and there are two essential techniques of this algorithm, namely (1) prediction entropy as acquisition function (2) viewpoint consistency. Is there any ablation studies for this? such as using just technique 1 in the first experiment, and just using technique 2 in the second experiment, comparing the performance with the algorithm.

Q2. Choice of hyperparameters. There are parameters P and \delta that affects the performance of the algorithm. Authors mentioned that P=0.2 and \delta=0.7 were chosen, and these values were chosen based on cross validation. Is there a graph or table that shows the performance of the GBCL algorithm based on different sets of those hyperparameters? How is the trend?

limitations: As authors mentioned, the main limitation of this work is that the experiments are quite limited and simple. It would've been a better paper if they had added more domain and more environments or different dataset. Some more experimental analysis on parameters and ablation studies would strengthen this paper.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper operates in the few-shot continual active learning (FoCAL) setting as exemplified by visual object category learning, but without concerning itself with continual representation learning. Instead, attention is mainly given to active, sample efficient learning of a classifier on top of pre-trained representations. A human-in-the-loop interaction model motivates the FoCAL setup, hence the paper does not only propose a solution for continual classifier learning, but it also motivates choices for the acquisition function used to select which samples to be labelled during the active continual learning process. Experimental validation is done on CORe-50 and a custom collected dataset using a robot.
strengths_and_weaknesses: Strengths:
* The paper identifies an important application of open-world learning in the particular case of visual concept learning with the constraints of a real-world robotic setup.
* The few-shot active classifier learning problem is interesting even outside of the continual learning paradigm.

Weaknesses:
* The paper does not focus on continual representation learning, since the feature extractor is pre-trained and fixed. This means reduced significance, since there exist several approaches for “shallow” continual learning, e.g. ELLA [Ruvolo et al. 2013].


FoCAL is not only about visual concept learning, but realistic robotic settings should also address issues of control, for which pre-training or hard-coding of behaviours are major limitations. The paper should limit claims to visual category classifier learning; this detracts from the clarity of the paper at the moment.

It would also help clarity if the work positioned itself better with respect to other continual learning works and made its assumptions explicit, see for context: [Hadsell et al. 2020, Mundt et al. 2020, 2022]

Several baselines [LWF, EWC, iCaRL] are designed for continual representation learning under different assumptions, and this should be discussed, also to improve clarity.

Few-shot meta-learning with pre-trained representations is well known to be effective [6], so it is difficult to argue for the novelty of the approach or the experimental setup.

It is difficult to argue for the high significance of the work due to not addressing open problems, such as continual representation learning. It is not surprising that pre-trained visual representations are useful in few-shot settings with novel visual categories, see the few-shot meta-learning works.

The paper should mention few-shot meta-learning works which have explored several of the concepts used, e.g. see the citations in [6]. There are too few references to works on open-world learning, a related field.



### References:

[Ruvolo et al. 2013] Paul Ruvolo, Eric Eaton. ELLA: An Efficient Lifelong Learning Algorithm. Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):507-515, 2013.

[Hadsell et al. 2020] Raia Hadsell, Dushyant Rao, Andrei A. Rusu, Razvan Pascanu. Embracing Change: Continual Learning in Deep Neural Networks. TiCS 2020.

[Mundt et al. 2020] Martin Mundt, Yong Won Hong, Iuliia Pliushch, and Visvanathan Ramesh. A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning. CoRR 2020.

[Mundt et al. 2022] Martin Mundt, Steven Lang, Quentin Delfosse & Kristian Kersting. CLEVA-COMPASS: A CONTINUAL LEARNING EVALUATION ASSESSMENT COMPASS TO PROMOTE RESEARCH TRANSPARENCY AND COMPARABILITY. ICLR 2022.

questions: * Is there no need for further representation learning? Is such an assumption realistic?
* How are limitations of pre-trained representations addressed by the machinery on top?

limitations: Limitations should be discussed at length in the main text, not the appendix.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposed a new task combining few-shot learning, active learning, and continual learning, in the context of robots perceiving and interacting with unseen objects. In this task, new instances of different object categories emerge and the robot needs to identify the unseen ones and ask a human for labels. To tackle this problem, the paper proposed GMM Based Continual Learning (GBCL), where Gaussian mixture models are learned continually for each object, given a pretrained and fixed feature extractor. The confidence and consistency predicted by GMMs are used to select novel object instances for active learning. The experiments are performed on the CORe-50 dataset and on a real robot. The results demonstrate the effectiveness of the proposed method over baselines. 

strengths_and_weaknesses: Strengths:
* The proposed task is very interesting and has a huge value for real-world applications of vision and robotic systems. Robots deployed in the real world need to interact with potentially unseen objects every day. This task is a combination of few-shot, active and continual learning. I think this paper formulates this task in a decent way. The experiments are performed both on a pure static dataset (CORe-50) and on real robots with human labeling in the loop. 
* The application of GMM in continual and active learning in novel and the results prove to exceed the state-of-the-art. 

Weaknesses:
* It’s unclear whether the comparison with other continual learning baselines is fair or not. 
    * How is the memory usage of different continual learning? How many images or latent features are stored for each method? Which method stores a checkpoint of the network and which does not?
    * What’s the number of Gaussian mixtures during the learning of GBCL? How does this compare to the memory usage of other CL methods?
    * Are the networks for different CL methods all pretrained in the same way? During continual learning on CORE-50, do they all use the fixed feature extractor or the entire network is trained continually?
* The feature extractor for each object is pretrained and fixed. The object detector used in the robot experiment is also fixed. It is possible that the feature extractor is not capable enough and can achieve better performance if they are learning on new objects observed. 
* It would be better if some baseline methods can be also tested on the robot experiment and compared to GBCL.

questions: * What is “GBCL-curiosity” in Line 240-241?
* How will a baseline that stores all feature vectors from all increments perform?
* Is the network doing classification on object-level or instance-level? 

limitations: I appreciate the authors’ discussion on limitations in the conclusion section. In addition, I think the usage of a pretrained and fixed feature extractor and the detector is another major limitation of this paper.
review_title: null

Meta-review: metareview: All reviewers appreciated the importance of the problem being tackled, and the effectiveness of the proposed method. There were a number of concerns about ablations and use of pre-trained feature extractors, but these have been sufficiently addressed in the authors' rebuttal. I agree with the reviewers in recommending acceptance. 
confidence: Certain
Decision: Accept