Few-Shot Continual Active Learning by a Robot
Authors: Ali Ayub
Carter Fendley
Venue: NeurIPS 2022 Accept (2022)
Keywords: Continual Learning
Catastrophic Forgetting
Active Learning
Human-Robot Intearction

Abstract: In this paper, we consider a challenging but realistic continual learning problem, Few-Shot Continual Active Learning (FoCAL), where a CL agent is provided with unlabeled data for a new or a previously learned task in each increment and the agent only has limited labeling budget available. Towards this, we build on the continual learning and active learning literature and develop a framework that can allow a CL agent to continually learn new object classes from a few labeled training examples. Our framework represents each object class using a uniform Gaussian mixture model (GMM) and uses pseudo-rehearsal to mitigate catastrophic forgetting. The framework also uses uncertainty measures on the Gaussian representations of the previously learned classes to find the most informative samples to be labeled in an increment. We evaluate our approach on the CORe-50 dataset and on a real humanoid robot for the object classification task. The results show that our approach not only produces state-of-the-art results on the dataset but also allows a real robot to continually learn unseen objects in a real environment with limited labeling supervision provided by its user.

Few-Shot Continual Active Learning by a Robot
Ali Ayub University of Waterloo Waterloo, ON N2L3GI, Canada a9ayab@uxaterooo.cc
Carter Fendley Capital One New York, NY 10017, USA ccf5164@psu.cc edu
Abstract
this paper, we consider a challenging but realistic continual learning problem, Few-Shot Continual Active Learning (FoCAL), where CL agent is provided with unlabeled data for a new or a previously learned task in each increment and the agent only has limited labeling budget available. Towards this, build on the continual learning and active learning literature and develop framework that can allow CL agent to continually learn new object classes from few labeled training examples. Our framework represents each object class using a uniform Gaussian mixture model (GMM) and uses pseudo-rchearsal mitigate catastrophic forgetting. The framework also uses uncertainty measures on the Gaussian representations of the previously learned classes to find the most informative samples be labeled in an increment. We evaluate our approach on the CORe-50 dataset and on a real humanoid robot for the object classification task. The results show that our approach not only produces state-of-the art results on the dataset but also allows a real robot to continually learn unseen objects in real environment with limited labeling supervision provided by its use
1 Introduction
4 5 6 has emerged as popular area of research recent years because of its limitless real-world applications, such domestic robots, autonomous cars, etc. Most continual machine learning models 7 g 9 10 however, are developed for constrained task-based continual learning setups, where CL model continually learns a sequence of tasks, one a time, with all the data of the current task labeled and available in an increment. Real world systems, particularly autonomous robots, do not have the luxury of getting large amount of labeled data for each task. In contrast, robots operating in real-world environments mostly have to learn from supervision provided by their users [I] [2]. Human teachers, however, would be unwilling answer large number of questions or label a large amount of data for the robot. It would therefore be useful for robots self-supervise their learning, and ask the human teachers label the most informative training samples from the environment, in each increment. In this paper, we focus on this challenging problem, termed Few-Shot Continual Active Learning (FoCAL).
Continual learning (CL)
One of the main problems faced by continual machine learning models is catastrophic forgetting, in which the CL model forgets the previous learned tasks when learning new knowledge. In recent years, several works in CL have focused on mitigating the catastrophic forgetting problem [3 3 6 Most of these works, however, are developed for the task-based continual learning setup, where the model assumes that all the data for a task is available in an increment and it is fully labeled. These constraints are costly and limit the real-world application of CL models on robots. Active learning has emerged as an area of research in recent years, where machine learning models can choose the most informative samples be labeled from large corpus of unlabeled data, thus reducing the labelling
A preliminary version 2 of this work was presented workshops in RoMan 2020 and ICRA 2021.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Oracle
Labels for Objects
Active Sample Selection
f(0',
f(0
GBCL Distributions
((:00
a(z', M)
Pswedorebearsal
C(:; W)
Unlabeled Objects
Classifler
Extractor
Our overall framework for FoCAL. In each increment t, the features extracted for unlabeled objects [(x'; 0) are passed through the acquisition function a(z", (M) to get most informative samples x²,, which are labeled by the oracle. The labeled feature vectors used update the GMM representation of the learned classes Y" Pseudo-rehearsal rehearsal is used replay old class data, and the classifier model C(;; ; W)) is trained on the pseudo-samples of the old classes and the labeled feature vectors the tth inerement.
Figure I:
Most active learning techniques use uncertainty sampling to request labels for the most uncertain objects [5 14, [6]. These techniques, however, do not learn continually and thus would suffer from catastrophic forgetting. These issues related to the development of "colse-wordd" techniques for continual learning, active learning and open-set recognition have been explored in detail in 17).
effort [1] 15
In this paper, we consider FoCAL for the online continual learning scenario for the image classification task. In this setup, CL model (applied on robot) receives small amount of unlabeled image data of objects from the environment an increment, where the objects can belong to the previously learned classes by the model, new classes. The model is allowed to get a small number of object samples to be labeled by the user. As the model continues to learn from new training samples, it does not have access to the raw image data of the previously learned objects. Overall, FoCAL is a combination of multiple challenging problems in machine learning. mainly Few-Shot Class Incremental Learning (FSCIL) [9 10,, Active Learning [4 [8], and online continual learning [19]. To solve FoCAL, we get inspiration from the continual learning and active learning literature, to develop protocols for continual learning models that they actively choose informative samples in an increment. Particularly, we take inspiration from FSCIL literature to develop a new FoCAL model, in which learn and preserve the feature representation of the previously learned objects classes by modelling them as Gaussian mixture models. To mitigate catastrophic forgetting, we use pseudo-rehearsal (20) using the samples generated from the Gaussian distributions of the old classes, thus removing the need to store raw data for the classes. Further, to choose most informative samples from an unlabeled set, we use combination of predictive entropy [2] [8] and viewpoint consistency metrics [14] [6] on the GMM representation of the previously learned classes. We perform extensive evaluations of our proposed approach on the CORc-50 dataset [22]. and on real humanoid robot in an indoor environment. Our approach outperforms state-of oftte-of-the-art the-art (SOTA) continual learning approaches for FoCAL on the CORe-50 datast with significant margins. Further, our approach can also be integrated on a humanoid robot, and allow the robot to learn a large number of common household objects over long period of time with limited supervision provided by the user. Finally, as part of this work, we also release the object dataset collected by our robot as benchmark for future evaluations for FoCAL (available here: https://tinyuzz: www 2vuwv8ye)
2 Few-Shot Continual Active Learning
We define the Few-Shot Continual Active Learning (FoCAL) problem as follows: Suppose that an AI agent (e-g. robot) gets a stream of unlabeled data sets D pooot: D2 Droot ..., Dt poot over increments,
{ [ 1D In each increment, continual learning model M with parameters θ can only obtain small number (Re < (ooo of samples to be labeled. Given the model M, an acquisition function a(x', MM), where x² E Dt poot: used by the Al agent to find the most informative samples to be labeled in increment t: x" argmax pool alx M). Therefore, in each increment
where D poot
t, the model M gets trained small subsets of labeled data D {xf]", v0)
(()f".())) [4*)
where of y

---

represents the class label of प and Y is the set of classes in the t-th increment. Note that unlike most continual learning setups, Y' n Y # 2, Vi * j. After training on D', the model M is tested to recognize all the encountered classes so far YI Y ..... Y The main challenges of FoCAL are three-fold: (1) avoid catastrophie forgetting, (2) prevent overfitting on the few training samples, (3) efficiently choose most informative samples in each increment.
For FoCAL for the task of object classification, we consider the model .M CNN) a composition of a feature extractor f(;;0) 0) with parameters 0 and a classification model with weights W. The feature extractor transforms the input images into feature space F E R". The classification model takes the features generated by the feature extractor, and generates an output vector followed by softmax function to generate multi-class probabilities. In this paper, we use a pre-trained feature extractor, therefore parameters & are fixed. Thus, we incrementally finetune the classification model on D D2 ... and get parameters W1, W : In an increment I, we expand the output layer by |Y neurons to incorporate new classes. Note that this setup does not alleviate the three challenges of FoCAL mentioned above. The subsections below describe the main components of our framework (Figure D to transform this setup for FoCAL.
2.1 GMM Based Continual Learning (GBCL)
We aim to develop a model that not only helps the system with continual learning but is also motivated by the newness of an object. To accomplish this, we must evaluate how different an incoming object is from previously learned object classes, ideally without any additional supervision. To accomplish this, we consider a clustering-
approach to represent the distribution of object classes. Unlike previous works on clustering-basee approaches for continual learning [IO] [1] that represent the object classes as mean feature vectors (centroids), we estimate the distribution of the each object class using a uniform Gaussian mixture model (GMM). We believe that representing each class data as GMM may better represent the true distribution of the data rather than assuming that the distribution is circular. We call our complete algorithm for continually learning GMMs of multiple object classes as GMM based continual learning (GBCL).
Once the k feature vectors (D') selected by acquisition function (Section 2.2 as most informative samples are labeled by the oracle increment E, GBCL is applied learn GMMs for the classes Yr. For each ith feature vector z! in D' labeled as y/. if y is a new class never seen by the model before, we initialize a new Gaussian distribution N(x O) for class y with xf as the mean (centroid) and a zero matrix (O) as the covariance matri Otherwise, if yf is a known class, we find the probabilities M(x Ic", of) .........((... (atlee
..., ...,..((G N( c%,,,%)) for = belong to all the previously learned Gaussian distributions for class where n is the total number of mixture components in the GMM for class y, and di and of represent the centroid and covariance matrix for the jth mixture component of class respectively. If the maximum probability among the calculated probabilities for all the distributions is higher than a pre-defined probability threshold P, x is used update the parameters (centroid and covariance matrix) of the most probable distribution (N(c), oy))) in class y. The updated centroid ey is calculated as weighted mean between the previous centroid oj and a!:
w x
+ x
/ +1
where, w is the number of images already clustered in the jth (most probable) Gaussian distribution. The updated covariance matrix 6y is calculated based on the procedure described in (3)
w" wg²
6y-
(at - -3)
(2)
where, of is the previous covariance matrix and (-f (c))) between of and dy. If, on the other hand, the maximum probability among the calculated probabilities
(g)) represents the covariance
"W do not describe mixing coefficients here, as they will always be 1/n for uniform GMM, where is the number of mixture components.
3

---

for all the distributions is lower than P. a new Gaussian distribution (x), O) is created for class with 2 as the centroid and O as the covariance matrix.
The result of this process is set of N' uniform GMMs with parameters o² 0² ..... L: for N classes learned up till increment t. Note that instead of using the number of mixture components as a hyperparameter,
use the probability threshold. This way model for each class (a single mixture component) and add more mixture components only when the new images of the class are too dissimilar from the previous mixture components, and thus cannot be modeled by the GMM. Therefore, the total number of mixture components for each class can be different based the similarity between the images of the class. In section 2.2 the same idea of dissimilarity between an unlabeled image and a GMM to predict most informative samples.
start with simple distribution
2.1.1 Pseudo-rchearsal and Classifier Training
To avoid catastrophic forgetting, we pseudo-rchearsal 20 to replay the old classes when learning from new data in increment t. For pscudo-rehearsal, we sample the Gaussian distributions in the GMMs of all the previously learned classes to generate set of pseudo-feature vectors. Note that we also store the total number images clustered in each Gaussian distribution of the classes (w") during the GMM learning phase (Section 2.I). Therefore, we generate the same number of pseudo- feature vectors as the original number of images for each class. After generating the pseudo-feature feature vectors, the classifier model CC;; W) is trained using the labeled dataset D in increment I, and the pseudo-feature vectors of the previous classes.
For classification of a test image x. it is first passed through the feature extractor (ar, 0) and then through the classifier C(f(z, 0), W). Softmax function (o) is then applied on the output to generate class probabilities, and the class y" with the maximum probability is predicted the label for the test image: y² = argmax, o oWV f(x,0) 0)).
2.2 Active Learning using GMMs
We quantify the novelty of an object in terms of how much the model is uncertain about the object. Unlike most active learning setups [8 14), in FoCAL the model does not have access to a training set in each increment for training the model to predict uncertain object classes. Further, even if the model does get trained to predict unknown object classes in the first increment, it will catastrophically forget the criterion of novelty as it continually learns new object classes in the subsequent increments (unknown classes in the first increment become known to the model in the subsequent increments). Therefore, we do not train our model for active learning, and instead use the GMM representations of the previously learned object classes to predict the most uncertain objects.
Considering the FoCAL setup (as described in Section 2). in an increment 6, the Al agent gets an unlabeled dataset D poot and it must find k Dipoal most informative object samples from the dataset be labeled. To develop acquisition function for this, learning techniques applied to the GMM representations of the previously learned object classes.
combination of two active
First, we use the prediction entropy Hy" [a]]] of an object as the acquisition function 21
N1-1
Hy |a
Σ p = =
For an unlabeled data point xf E Dj Ppoot we find the predictive probability of x using the GMM representation of the object classes learned in the previous increments. The predictive probability p(at(o") for of z to belong to the GMM of class y can be defined as:
1 n
TE Σ Nodilc,,3)
(4)
Intuitively, if sample of is already learned by the Al agent, then its probability to belong to one of the previously learned class GMMs must be high, and thus entropy for at must be low. Therefore, top l samples with the highest entropy can be chosen the most informative samples.

---

For the second technique, we use the idea of viewpoint consisteney used in active learning [1 [6] The main idea is that if an object is already learned by the agent before, then consistent predictions must be produced for the object under different viewpoints (see Figure 3 in the supplementary file for an example). We again use the GMM representations of the previously learned classes for this acquisition function. Let's consider that there are multiple (?) viewpoints z/, ...; ......... arj of an unlabeled object j available to the AI agent in increment t. We find the predictive probability of each of to belong to the GMM representations of the previously learned classes (equation (6)). Using the maximum predictive probability over all the classes we find the class prediction yf for each 2 as u! aargmaxy=1....hh N/ Next, we count the total number of times each of the N classes
" represent is predicted among the different viewpoints of the object j. Let S = (.))...,.. ... o
the total number of times each class is predicted among the different viewpoints of the object. We normalize set S by dividing each s by such that Σy=1 the maximum value in Sj (1/maxS,), which represents the inverse of the highest percentage of viewpoints of object j that are predicted consistently. Thus, we use this term as the inconsistency score of the object j.
NN11 sy L. We then take the inverse of
We use a combination of predictive entropy and viewpoint consistency to generate the final acquisition function for our framework. Before combining the two metrics, transform the predictive entropy for multiple viewpoints of the same objects as it is originally designed for individual data samples. For multiple viewpoints of an object j, we find the predictive entropy of all the viewpoints individually and then take an average to get the overall predictive entropy of object j. Therefore, we maximize the following combined function to get the most informative samples in increment t:
5 1
1
H 1
+ (1 8) maxS;
(5)
where, 8 is a hyperparameter that controls the contribution of predictive entropy and viewpoint consistency towards the overall uncertainty score of an object. Note that the viewpoint consistency can only be applied when the agent has access to multiple viewpoints of the same object. In cases when multiple viewpoints are unavailable, we can use the predictive entropy alone. In our experiments (Section B. however, we use multiple viewpoints of all the objects.
3 Experiments
We first evaluate our approach for FoCAL on the Core-50 dataset [22] and then using the Pepper robot. We begin by presenting the implementation details and then compare approach against SOTA continual learning approaches on the CORe-50 dataset. Finally, we present the evaluations of approach on Pepper. Details about the CORe-50 dataset are in the supplementary file.
3.1 Implementation Details
We used the Pytorch deep learning framework 24 for implementation and training of all neural network models. We used R8 (2 pre-trained on the ImageNet dataset (26] as feature extractor for GBCL. The same pre-trained network was also used in all the other continual learning approaches for fair comparison. For GBCL, we only used the diagonal entries of the covariance matrix to keep the memory budget from growing drastically.
For FoCAL experiments on the CORe-50 dataset, we develop new experimental setup, as the standard active learning and continual learning experimental setups are not sufficient to test FoCAL (see Supplementary File for more details). In this experimental setup for FoCAL on CORe-50, we combined all 8 training sessions in CORe-50 to generate 400 training object instances. In each increment, we randomly sampled m object instances (from 400 instances) and allowed the model to learn the label of k = 1 out of 5 object instances, making it a challenging problem to learn from single object instance each increment. Once an object was learned by the model, was removed from the complete set of objects and, thus, was not available to the model in later increments. Hence, we allowed the model to learn all objects 400 increments with one object learned in each increment. We used the test set accuracy at each increment as the evaluation metric. After training
5

---

100
90
GBCL(63.60)
@BCLraadomm550.5 15)
Methods
Increments
CBCU/57
FT
23
70
FLB(62.97)
LWF
23
Accuracy (%)
FT(10.00)
(WR(17.65)
EWC
24
NCM(S5 93)
iCaRL
18
ICaRL127 54)
LWFF10000
NCM
17
30
EWC(10.00)
CWR
21
FLB
19
CBCL
17
GBCL-rand
22
Increments
GBCL
15
Figure 2: Comparison of our method (red curve) to SOTA approaches in terms of classification accuracy on the Core-50 50 dataset. Average incremental accuracy is reported in parenthesis. The curves show average and standard deviation of 5 runs with random seeds.
Table 1: Total number of increments
needed by each method learn 10 classes C0.e-50.
in each increment, the model was tested on the complete test set of the CORe-50 dataset. We only report the accuracy for 70 increments for this experiment because all the approaches learn all 10 classes by 70 increments and the final accuracy starts to saturate.
For robustness, all the experiments were performed times with random seeds. We report the average and standard deviation of the accuracies. Hyperparameters and ó were chosen using cross-validation and were set 0.2 and 0.7, respectively for all increments. For the shallow neural network used for classification in GBCL. we used a linear layer of the same size as the feature vectors generated by the ResNet (512x1) 1). The shallow network was trained for 25 epochs using the cross-entropy loss optimized with stochastic gradient descent (with 0.9 as momentum). A fixed learning rate of 0.01 and minibatches of size 64 were used for training.
3.2
Experiments on the CORe-50 Dataset
We compare our approach against 6 continual learning approaches (LWF 277. EWC CWR 22
iCaRL 33, NCM (11) CBCL (10), finetuning (FT) and few-shot batch learning baseline (FLB) 288 FLB uses pre-trained CNN to extract feature vectors for images and trains a linear classifier using cross entropy loss. FLB is trained on the complete training data of the previous increments and the current increment. In other words, FLB does not learn continually and therefore should have an advantage over other continual learning approaches (including ours). FT the same architecture as FLB but FT is trained only on the data of the current increment. FT suffers from catastrophic forgetting and therefore should produce lower accuracy than other continual learning approaches. To the best of our knowledge. none of the eight approaches can be directly applied to FoCAL. Hence, for FLB, FT, LWF, EWC and CWR we used noftmax-based uncertainty sampling find most uncertain objects. For the softmax-based uncertainty scores, we found the softmax output of each image for an object instance. Then, we took the average the maximum probability in the softmax output of each image of the object. The object with the highest average probability score was chosen to be labeled. Because iCaRL, NCM, and CBCL use centroids for classification, we used the average distance of the images of the new object from the centroids as the uncertainty score. A brief description of all six continual learning approaches is provided in Supplementary File.
3.2.1 Comparison with SOTA Approaches
Figure compares our GBCL approach against random sampling and other SOTA approaches in terms of classification accuracy on the fixed test set for 10 classes. Our approach (GBCL) and FLB produce similar results for all 70 increments and GBCL produces slightly better accuracy than FLB in some increments. FLB, however, uses the data in all the previous increments at each new increment, while our approach only uses the data in the current increment and pseudo-feature vectors of the previous classes. We believe the reason for similar results for the two approaches is that GBCL. generates and uses pseudo-feature vectors for previous classes mitigate catastrophic forgetting,

---

Increments
40
80
120
160
200
240
Accuracy (%)
56.7
76.5
84.1
87.5
87.9
88.3
No. of Classes
18
20
20
20
20
20
Table 2: Test accuracy and number of classes learned by Pepper over 240 increments.
and it also learns the most informative samples in each increment. As expected, FT suffers from catastrophic forgetting since finetunes the network on new data in each increment. Regularization techniques (LWF and EWC) also suffer from catastrophic forgetting and produce the same accuracy as FT. CWR and iCaRL produce slightly better accuracy, however they also suffer from catastrophic forgetting. NCM produces much better accuracy than the other approaches with the help of centroids to remember past classes and prioritize objects to learn. However, NCM's accuracy is significantly inferior to FLB and GBCL. CBCL produces the best accuracy among the SOTA methods because learns multiple centroids per class. However, CBCL's accuracy is still lower than GBCL.
Note that after 400 increments, FLB produces 89% final accuracy and GBCL produces 888 (1% lower) accuracy. The reason is that after 400 increments both models learned all the training samples in the dataset, and therefore the advantage of informative sampling in GBCL fades away. Further, unlike FLB, GBCL does not have access to all the data the previous increments, and suffers from slight forgetting compared to batch learning (FLB).
In terms of memory storage, GBCL stores 239 clusters/Gaussian distributions (centroids and covari- ance matrices) for all the classes. To avoid huge memory storage, we only use the diagonal entries of the covariance matrix for all the experiments. Therefore, GBCL stores 33982=478 vectors of size 512 in memory after learning over 70 increments on the CORe-50 dataset, which requires only 0.97 MB. In contrast, FLB (batch learning approach) stores all 21000 features vectors for all the classes, which requires 43.08 MB (44 times more than GBCL). For other approaches, FT does not store any data, while LWF, EWC store minimal information, such as the predicted labels for the previous classes and fisher matrix to capture weight importance. CWR stores an extra weight matrix for the classification layer which requires 0.02 MB of storage space. iCaRL. stores 2000 raw images for the previous classes, which requires 393 MB. NCM stores only 10 centroids for the previous classes, which requires 0.02 MB. Finally, CBCL stores 315 centroids which requires 0.64 MB. Note that all of the approaches also store ResNet-18 18 model trained on the previous classes which requires 83 MB of space. This analysis shows that GBCL provides the best trade-off between memory storage and the overall performance in comparison with the other approaches.
3.2.2 Comparison with Random Sampling
To provide more insight into our approach, tested our GBCL approach without active learning (GBCL-random). In this case, GBCL gets a random object to be labeled in each increment. GBCL- random produced significantly lower accuracy than FLB and GBCL. for all the increments, especially in earlier increments. These results depict the contribution of our active learning approach to efficiently choose which objects to learn. In comparison with other SOTA approaches, GBCL-random beats all the approaches (except CBCL) in later increments but it is below NCM and CBCL in earlier increments. These results also demonstrate the effectiveness of GBCL (Section 2.1 of our approach to mitigate catastrophic forgetting.
3.2.3 Comparison of Learning Efficiency
To further evaluate the efficiency of of our approach for FoCAL, we report the total number of increments taken by all the approaches to learn all 10 classes (see Table [ GBCL requires the least number of increments (15) to learn the 10 classes. FLB takes 19 increments to learn all the 10 classes because it uses the softmax based sampling. NCM and iCaRL also take similar number of increments because they use centroids to calculate the uncertainty scores but do not have access to all the data of the previous increments. CBCL is the closest to our approach and learns all the classes in 17 increments. FT, LWF and EWC all use softmax- based sampling without using any previous data, so they take a large number of increments to learn all 10 classes. Lastly, random sampling takes the most number of increments (except for FT, LWF, and EWC) to learn all the 10 classes. These results clearly show that our approach is the most efficient at finding the unknown objects and learning them in small number of increments.
7

---

3.3 Experiment on the Pepper Robot
In this experiment we used the Pepper robot learn 240 houschold objects belonging 20 classes (12 objects per class). We performed the experiment in an indoor lab environment with objects placed at four different locations in the lab with different backgrounds, At each location we placed different objects on a table (Figure 2 (a, b) in the supplementary file). For this setup, the robot first localized the 5 individual objects in the image. We use RetinaNet [29] pre-trained on MS COCO dataset [30] for object detection and localization. The robot then captures images of the indívidual objects from multiple viewpoints (see Figure 2 (c) in the supplementary file for an example). These images are used by the active learning module of GBCI. to find the most informative object. The user (cxperimenter) then provides the class label of the chosen object to the robot using a keyboard. The newly labeled object is then used to update GBCL. Details about our complete system for the Pepper robot are in the Supplementary File.
This experiment tried to reproduce some of the challenges that might arise in realistic household environments. For example, a large number of objects were learned at different times, over multiple days (8 weeks), depicting true lifelong learning robot in real-world environments. Different objects were used in the training and test sets. Moreover, the background of the objects was non-ideal since many parts of the background could be viewed as objects by the object detector. For example, consider Figure in the supplementary file where the air-conditioner and the window could be viewed as objects by the object detector. Objects also varied in size and some were almost transparent. Further, the lighting conditions were not ideal. Some images were taken at night, while some during the day with different variations of sun light coming through the windows in the environment (see Figure and Figure 6 for examples. More examples in the Supplementary File).
To evaluate this experiment, we created test set of 60 objects (3 objects per class) that were different from objects in the training set. We tested Pepper's 's ability to classify the objects in the test set after 40, 80, 120, 160, 200, and 240 increments of learning new objects. We also report the number of object classes learned by Pepper after each of these increments (Table 2). After only increments, Pepper learned all 20 object classes and achieved 76.5% accuracy on the fixed test set. For the rest. of the increments the increase in accuracy was minimal, especially in later increments (120-240), similar to the results on the CORe-50 dataset. The reason is that in the earlier increments Pepper learned objects belonging to a large number of classes quickly which increased the overall accuracy. However, in the later increments Pepper only learned about more objects of the previously learned classes, thus the increase in overall accuracy was minimal. We should also note that even after 240 increments (same the total number of training objects) Pepper only learned 197 objects. The reason is that the object localization module was not perfect, and it failed to detect many objects (43 objects).
4 Ablation Studies
We performed two ablation studies to examine the contribution of different components of our approach and the effect of hyperparameter values on GBCL's performance. This set of experiments were performed on the dataset collected by the Pepper robot (we call it Pepper dataset this paper). As GBCL's accuracy saturates by 160 increments (Table 2 we performed all of these experiments for 160 increments, with a single object learned in each increment. We report accuracy for increments 0, 40, 60, 80, 120, 140, 160, and the average incremental accuracy. The rest of the experimental setup was the same as in the experiments for the CORe-50 dataset (Section 3.2).
For the first ablation study, we investigate the effect of hyperparameter & on choosing the two acquisition functions (entropy and viewpoint consistency) for active learning. Figure 3 (a) depicts the impact of choosing different values ranging from 0 to 1, for the hyperparameter 5. For the extreme values i.c. 0 and 1, our model chooses only one of the acquisition functions for active learning. Therefore, the model produces the lowest accuracy for 5 values 0 and 1. However, there is no significant difference in accuracy when choosing either of the acquisition functions. For other delta values close to 0 or 1, the accuracy increases slightly, but the best accuracy is achieved with 8 = 0.7. However, for other values of 8 close to 0.7 (such as 0.6), there is not a significant difference in accuracy (only 0.07%), which shows that our approach is not highly sensitive to a large range of possible values of 8. These results also confirm that choosing a combination of the two acquisition functions produces better performance than choosing either of them alone.
8

---

100
100
80
70-
10
Accuntty
60
60
prob=0
40
Gatt, 00fa--0000 0. (71.02)
40
GBCL, prob=0.25 (69.82)
dett=
prob (71.02)
30-
GBCL, detaa0020 (67.27)
30
CBCL, prob (67.60)
del.m00500
20
prob=0010 (55.24)
GBCL, delt3 (66.571
CBCL, prob=0.70 (70.21)
(65.733
10
prob=0.00
0-
0
20
40
60
50
100
120
140
160
20
40
100
120
140
160
Increment
Number
Figure 3: The effects of varying 6, and probability threshold on the classification accuracy on the test set Pepper dataset. While changing one the two parameters, the other parameter's values was set be 0.7, P = 0.2 for the best results
For the second ablation study, we investigate the effect of hyperparameter P that determines the number of clusters (or Gaussian distributions) generated for each object class. Figure (b) shows the impact of P on GBCL's performance. For P = 0, GBCL simply stores a single cluster for each class which might not be enough to capture the complex distributions of the object classes. Therefore, for P values close to 0, there is significant drop in GBCL's accuracy (~ (~18%). As the P value increases, the model starts to assign more clusters for each object class, and thus is able to capture the complex distributions of the object data. The average incremental accuracy of the model increase significantly when P = 0.15 with 41 total clusters for all 20 object classes. The best accuracy is achieved for P = 0.2 (83 total clusters), however, for large range of P values the average incremental accuracy stays within range of ~3%. As the P value continues to increase, the model starts to assign more clusters per class, therefore for higher values of P the model uses more storage space. Further, for P values close to 1, the model starts to recruit more clusters per class with only a few features (or single feature) assigned to each cluster. Thus, the model starts resemble more with the batch learning case when all the features of the previously learned classes can be stored. Because of this, we start to see an increase accuracy again when number of clusters stored for P = 0.2). Overall, these results show that our model is not highly sensitive probability threshold values within a large range, and values close P 0.2 provide nice trade-off between memory storage and model's performance.
0.7 (379 clusters, 4.5 times more than the
Finally, note that found both hyperparameters using coss-valiiotion the CORe-50 dataset and not on the Pepper dataset. However, even on the Pepper dataset the same hyperparameter values produce the best results. This shows that the chosen hyperparameter values are not dependent single dataset.
Related Work
Most continual learning approaches are designed for class-incremental setting in which model (often a neural network) learns from training data of different classes in different increments and then evaluated the test data of all the classes learned far (3] Most existing class-incremental learning (CIL) methods avoid catastrophic forgetting by storing a portion of the training samples from previous classes and retraining the model on mixture of the stored data and new data BTI However, this approach does not scale as additional data exhausts memory capacity limiting performance in real-world applications. To avoid this problem, some CL approaches use regularization techniques 27 Although these approaches solve the memory storage issues, their performance is significantly inferior approaches that store old class data. few works have been proposed for FSCIL, to continually learn from a few-examples per class using limited memory [10 9 32 33). However, as mentioned in 17 34], one significant limitation of all CL approaches that they require the complete training data of each class to be labeled and available in single increment, however in real-worid robotic applications data is available streaming manner (online learning (19) and it might be mostly unlabcled.

---

For object learning, many researchers have presented active learning techniques using uncertainty sampling B5 18
36 [6]. Most of these approaches train deep neural networks with special loss terms such that the networks can predict the most uncertain samples. All of these approaches, however, are trained for batch learning setting and will thus suffer from catastrophic forgetting when attempting to learn continually. Further, active learning approaches can predict unknown classes in the first increment after batch training. However, in a continual learning setting these approaches lose their ability to recognize unknown classes in subsequent increments because the model will assign the unknown classes to the newly learned classes (learned in the previous increments) that were unknown in the first increment.
14
Another related field active learning is open-set recognition (or out of the distribution detection (OOD)) [37 38 38, [9] in which a model that is trained on a set of tasks/classes might face samples belonging previously unknown classes in the test set. Therefore, during the testing/deplyyment phase, the model must be able to detect which of the samples belong to unknown classes. Although methods for OOD
have been developed to detect unknown classes, these models do not train on the newly encountered unknown samples. Mundt et al. [7 present thorough analysis of previous continual and active learning, and OOD works, and suggest a common viewpoint with open-set recognition acting natural interface between continual and active learning. Towards this goal, we present an approach that allows an Al agent to actively select the most informative (unknown) samples from unlabeled data, and continually learn from the actively selected object samples.
40 41
6 Conclusion
This paper presents and evaluates a novel method for challenging problem: Few-Shot Continual Active Learning (FoCAL) for object classification. Experimental results demonstrate that our ap- proach for FoCAL is highly efficient and helps the model learn the most uncertain objects continually without forgetting earlier classes. Our approach not only outperforms the SOTA approaches on benchmark dataset but also allows a robot to actively learn objects in a real environment over a long period of time. Finally, we have also released the dataset collected by the Pepper robot benchmark for future evaluations of FoCAL models.
The work presented here has few limitations. 1) The experimental setup is still quite simple compared to a real household environment. 2) We assume that correct object labels are provided by the human assistant to the robot. 3) We did not recruit real participants to teach the robot, but the experimenter acted as user for the robot. 4) We used a fixed feature extractor and object detector. In the future, we hope to work on these limitations and test our system with real human participants. Particularly, we hope to develop techniques that can allow feature representations to be learned from a few samples over large number of increments.
There are a number of positive and negative long-term outcomes of this work. With respect to positive outcomes, the improved performance for continual learning by our approach may one day lead to robots that can tailor their behavior and learning to the needs of a person. We envision home service robot [42) that uses continual learning to learn person's 's food preferences before serving them breakfast, or a domestic service robot that learns how a person prefers their home to be cleaned and adjusts its behavior to their preferences. In terms of negative outcomes, one potential downside is that an active learning robot might learn unnecessary objects which are never going to be used by the robot, Learning such objects can potentially decrease the recognition performance objects that are required by the robot. Using FoCAL to learn unnecessary objects can also take valuable time that might be better used by the robot to perform necessary actions. For example, consider a dish washing robot which only needs to know kitchen utensils. The robot does not need to learn mechanical tools or children's 's toys but uncertainty-basee sampling will cause the robot to learn these unnecessary objects. In such cases, some constraints might be added to FoCAL system in order to avoid the robot from learning unnecessary objects.
necessary
Acknowledgements
The authors acknowledge helpful comments of Alan R. Wagner for preliminary version of this work.
10

---

References
[1] Ali Ayuh and Alan R. Wagner. Learning novel objects continually through curiosity. In IEER ICRA 2021 (Worsshpp titled. Towards Curious Robots: Modern Approaches for parinsically-Motivvae Intelligent Behavior)), 2021.
[2] Ali Ayub and Alan R. Wagner. Online learning of objects through curiosity-driven active learning. In IEEE RoMan (Workshop on Lifelong Learning for Long-term Human- Robott Interaction), 2020.
[3] Sylvestre-Alviss Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. iCaRL: Incre- mental classifier and representation learning. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
[4] James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 114(13):3521 3526, 2017.
[5] Ali Ayub and Alan R. Wagner. Tell me what this is: Few-shot incremental object learning by robot. JEEE/RSJ International Conference Intelligent Robots and Systems (IROS), 2020.
[6] Ali Ayub and Alan Wagner. EEC: Learning to encode and regenerate images for continual learning. In International Conference on Learning Representations, 2021.
[7] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rehalancing. The IEFE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[8] Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning, In The IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
[9] Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[10] Ali Ayub and Alan R. Wagner. Segnitivelyiinspiee model for incremental learning using few examples. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2020.
[11] Dehghan, Z. Zhang, M. Siam, J. Jin, Petrich, and Jagersand. Online object task learning via human robot interaction. In 2019 International Conference on Robotics and Automation (ICRA), pages 2132-2138, May 2019.
[12] Ali Ayub and Alan R. Wagner. What am i allowed to do here?: Online lcarning of context-speciff norms by pepper. In Alan R. Wagner, David Feil-Seifer, Kerstin S. Haring, Silvia Rossi, Thomas Williams, Hongsheng Hc, and Shuzhi Sam Ge, editors, Social Robotics, pages 220-231, Cham, 2020. Springer International Publishing.
[13] Robert M. French. Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference. Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, pages 335-340, 2019.
[14] Yawar Siddiqui, Julien Valentin, and Mathhas Niessner. Viewal: Active learning with viewpoint entropy semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[15] Donggeun Yoo and In So Kweon. Learning loss for active learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[16] Devendra Singh Chaplot, Helen Jiang, Saurabh Gupta, and Abhinav Gupta. Semantic curiosity for active visual leaming. Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision ECCV 2020, pages 309-326, Cham, 2020. Springer International Publishing.
[17] Martin Mundt, Yong Won Hong, Iuliia Pliushch, and Visvanathan Ramesh. A wholistic view of continual learning with deep neural networks: Forgotten lessons and the bridge to active and open world learning. CoRR, 0tss2009.01799 2020.

---

[18] Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In Proceedings the 34th International Conference Machine Learning Volume 70, page 1183-1192. JMLR.org, 2017.
[19] Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In The IEFE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
[20] Anthony Robins. Catastrophic forgetting, rehearsal and pseudorchearsal. Connection Science, 7(2):123-
146, 1995.
[21] Claude Elwood Shannon. A mathematical theory of communication. University of Illinois Press, 1949.
[22] Vincenzo Lomonaco and Davide Maltoni. Core50: new dataset and benchmark for continuous object recognition. In Proceedings of the 1st Annual Conference on Robot Learning, volume 78, pages 17-26. 2017.
[23] Sanjoy Dasgupta and Daniel Hsu. On-line estimation with the multivariate gaussian distribution. In Twentieth Annual Conference on Learning Theory, volume 4539, page 278-292, June 2007.
[24] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024 8035, Curran Associates, Inc., 2019.
[25] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In The IEEE Conference on Computer Vision and Pottern Recognition (CVPR), June 2016.
[26] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, pages 1097-1105. 2012.
[27] Z. Li and D. Hoiem. Learning without forgetting. IEEE Transactions on Puttern Analysis and Machine Intelligence, 40(12):2935- Dec 2018.
[28] Yu Chen, Yen-Cheng Liu, 7solt Kira, Yu-Chiang Frank Wang, and Jia-Bin Huang. A closer look few-shot classification. In International Conference on Learning Representations, 2019.
[29] Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
[30] Tsung-Yr Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.
[31] Francisco M. Castro, Manuel J. Mitin-Jimenez. Nicolas Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In The European Conference on Computer Vision (ECCV), September 2018.
[32] Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, and Yihong Gong. Topology-preservv class- incremental learning. In Computer Vision ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIX, page 4700 Springer-Vertee 2020.
[33] Ali Ayub and Alan R. Wagner. F-siol-310: A robotic dataset and benchmark for few-shot incremental object learning. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 13496-13502, 2021.
[34] Martin Mundt, Steven Lang, Quentin Delfosse, and Kristian Kersting, CLEVA VEVA-compas:: A continual learn- ing evaluation assessment compass to promote research transparency and comparability. In International Conference on Learning Representations, 2022.
[35] William H. Beluch, Tim Genewein, Andreas Nümberger, and Jan M. Köhler. The power ensembles for active learning in image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
[36] Tingke Shen, Amlan Kar, and Sanja Fidler. Learning to caption images through a lifetime by asking questions. In Proceedings of the IEFE/CVF International Conference on Computer Vision (ICCV), Octoher 2019.
[37] Terrance E Boult, Steve Cruz, Akshay Raj Dhamija, Manuel Gunther, James Henrydoss, and Walter J Scheirer. Learning and the unknown: Surveying steps toward open world recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 9801-9800 2019.
12

---

[38] Abhijit Bendale and Terrance Boult. Towards open world recognition. In The IEEE Conference Computer Vision and Puttern Recognition (CVPR), June 2015.
[39] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out- of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018
[40] Walter J Scheirer, Lalit P Jain, and Terrance E Boult. Probability models for open recognition. IEEE transactions on pattern analysis and machine intelligence, 36(1 1):2317 22222, 2014.
[41] Ryota Yoshihashi, Wen Shao, Rei Kawakami, Shaodi You, Makoto lida, and Takeshi Naemura. Classffication-recnsttte
learning for open-set recognition. Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 4016-4025, 2019.
[42] Ayub, Chrystopher L. Nehaniv, Kerstin Dautenhahn. Don't forget to buy milk: Contextually aware grocery reminder household robot. arXiv:2207.00054 2022.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent obtained from people whose data you'
using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "brcdAE7VBVd", "reviewer": "Reviewer_ACp7", "summary": "This paper proposed a new task combining few-shot learning, active learning, and continual learning, in the context of robots perceiving and interacting with unseen objects. In this task, new instances of different object categories emerge and the robot needs to identify the unseen ones and ask a human for labels. To tackle this problem, the paper proposed GMM Based Continual Learning (GBCL), where Gaussian mixture models are learned continually for each object, given a pretrained and fixed feature extractor. The confidence and consistency predicted by GMMs are used to select novel object instances for active learning. The experiments are performed on the CORe-50 dataset and on a real robot. The results demonstrate the effectiveness of the proposed method over baselines. \n", "questions": "* What is “GBCL-curiosity” in Line 240-241?\n* How will a baseline that stores all feature vectors from all increments perform?\n* Is the network doing classification on object-level or instance-level? \n", "limitations": "I appreciate the authors’ discussion on limitations in the conclusion section. In addition, I think the usage of a pretrained and fixed feature extractor and the detector is another major limitation of this paper. \n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The proposed task is very interesting and has a huge value for real-world applications of vision and robotic systems. Robots deployed in the real world need to interact with potentially unseen objects every day. This task is a combination of few-shot, active and continual learning. I think this paper formulates this task in a decent way. The experiments are performed both on a pure static dataset (CORe-50) and on real robots with human labeling in the loop. \n* The application of GMM in continual and active learning in novel and the results prove to exceed the state-of-the-art. \n\nWeaknesses:\n* It’s unclear whether the comparison with other continual learning baselines is fair or not. \n    * How is the memory usage of different continual learning? How many images or latent features are stored for each method? Which method stores a checkpoint of the network and which does not?\n    * What’s the number of Gaussian mixtures during the learning of GBCL? How does this compare to the memory usage of other CL methods?\n    * Are the networks for different CL methods all pretrained in the same way? During continual learning on CORE-50, do they all use the fixed feature extractor or the entire network is trained continually?\n* The feature extractor for each object is pretrained and fixed. The object detector used in the robot experiment is also fixed. It is possible that the feature extractor is not capable enough and can achieve better performance if they are learning on new objects observed. \n* It would be better if some baseline methods can be also tested on the robot experiment and compared to GBCL.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "SWqVULGF5U4", "reviewer": "Reviewer_JcVg", "summary": "This paper proposes a setup of few-shot continual active learning (foCAL), where the agent continually learns with an unlabeled data and limited amount of labeling budget. This problem setup is meaningful, because previous continual learning methods focused on the scenario in which the data of the current task are available and labeled. In contrast, the new setup assumes that the robot doesn't have access to the labeled data, and needs to ask human teacher for supervision with limited availability (hence, active learning). This paper proposes FoCAL for online learning for image classification task. The framework is well summarized in the Figure 1. The features extracted from unlabeled objects go through the acquisition function a(,) to get the most informative samples (k), which are labeled by oracle. These labeled samples from oracle are used to update GMM representations. Pseudorehearsal process is used to replay old data. In the final stage of classifier, the samples from old classes and labeled features are used. Framework design description is given, and the experiment validations are presented. ", "questions": "Q1. I think the section 2.2. active learning using GMMs is the important part of the paper, and there are two essential techniques of this algorithm, namely (1) prediction entropy as acquisition function (2) viewpoint consistency. Is there any ablation studies for this? such as using just technique 1 in the first experiment, and just using technique 2 in the second experiment, comparing the performance with the algorithm.\n\nQ2. Choice of hyperparameters. There are parameters P and \\delta that affects the performance of the algorithm. Authors mentioned that P=0.2 and \\delta=0.7 were chosen, and these values were chosen based on cross validation. Is there a graph or table that shows the performance of the GBCL algorithm based on different sets of those hyperparameters? How is the trend?\n", "limitations": "As authors mentioned, the main limitation of this work is that the experiments are quite limited and simple. It would've been a better paper if they had added more domain and more environments or different dataset. Some more experimental analysis on parameters and ablation studies would strengthen this paper. ", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "- Strength of this paper is the novelty of using GMM (Gaussian mixture model) based continual learning. The authors argue that the model system should recognize how different an incoming object (how new class it is) compared to other previously learned object classes with ease. To address this, they use a clustering-based approach; however, clustering-based approach normally uses mean feature vectors to represent object classes; instead, they use GMM to model each class data. Once the oracle gives label on the selected k feature vectors, the GBCL is applied to learn GMMs for the class. The updates of centroid is done by weighted mean of previous centroids and new inputs; the updates of covariance matrix is being done similarly. \n- Another notable part of the paper is the section 2.2. active learning using GMMs. Especially, FoCAL employs two techniques for active learning. First technique is the use of prediction entropy as the acquisition function. Second technique is the use of viewpoint consistency; inconsistent predictions for the different view of the same objects generates a high reward for acquiring the label of the object, as opposed to the consistent predictions.\n- Weak points: Is there an ablation studies for using these two techniques? \n- Experiments and evaluations of FoCAL on Core-50 dataset, and evaluations on Pepper robot are presented. \n- Comparison baselines are sufficient (FT, LWF, EWC, CWR, etc) ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "OE-fiT_6A5m", "reviewer": "Reviewer_6wg4", "summary": "This paper operates in the few-shot continual active learning (FoCAL) setting as exemplified by visual object category learning, but without concerning itself with continual representation learning. Instead, attention is mainly given to active, sample efficient learning of a classifier on top of pre-trained representations. A human-in-the-loop interaction model motivates the FoCAL setup, hence the paper does not only propose a solution for continual classifier learning, but it also motivates choices for the acquisition function used to select which samples to be labelled during the active continual learning process. Experimental validation is done on CORe-50 and a custom collected dataset using a robot.", "questions": "* Is there no need for further representation learning? Is such an assumption realistic?\n* How are limitations of pre-trained representations addressed by the machinery on top?\n", "limitations": "Limitations should be discussed at length in the main text, not the appendix.", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* The paper identifies an important application of open-world learning in the particular case of visual concept learning with the constraints of a real-world robotic setup.\n* The few-shot active classifier learning problem is interesting even outside of the continual learning paradigm.\n\nWeaknesses:\n* The paper does not focus on continual representation learning, since the feature extractor is pre-trained and fixed. This means reduced significance, since there exist several approaches for “shallow” continual learning, e.g. ELLA [Ruvolo et al. 2013].\n\n\nFoCAL is not only about visual concept learning, but realistic robotic settings should also address issues of control, for which pre-training or hard-coding of behaviours are major limitations. The paper should limit claims to visual category classifier learning; this detracts from the clarity of the paper at the moment.\n\nIt would also help clarity if the work positioned itself better with respect to other continual learning works and made its assumptions explicit, see for context: [Hadsell et al. 2020, Mundt et al. 2020, 2022]\n\nSeveral baselines [LWF, EWC, iCaRL] are designed for continual representation learning under different assumptions, and this should be discussed, also to improve clarity.\n\nFew-shot meta-learning with pre-trained representations is well known to be effective [6], so it is difficult to argue for the novelty of the approach or the experimental setup.\n\nIt is difficult to argue for the high significance of the work due to not addressing open problems, such as continual representation learning. It is not surprising that pre-trained visual representations are useful in few-shot settings with novel visual categories, see the few-shot meta-learning works.\n\nThe paper should mention few-shot meta-learning works which have explored several of the concepts used, e.g. see the citations in [6]. There are too few references to works on open-world learning, a related field.\n\n\n\n### References:\n\n[Ruvolo et al. 2013] Paul Ruvolo, Eric Eaton. ELLA: An Efficient Lifelong Learning Algorithm. Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):507-515, 2013.\n\n[Hadsell et al. 2020] Raia Hadsell, Dushyant Rao, Andrei A. Rusu, Razvan Pascanu. Embracing Change: Continual Learning in Deep Neural Networks. TiCS 2020.\n\n[Mundt et al. 2020] Martin Mundt, Yong Won Hong, Iuliia Pliushch, and Visvanathan Ramesh. A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning. CoRR 2020.\n\n[Mundt et al. 2022] Martin Mundt, Steven Lang, Quentin Delfosse & Kristian Kersting. CLEVA-COMPASS: A CONTINUAL LEARNING EVALUATION ASSESSMENT COMPASS TO PROMOTE RESEARCH TRANSPARENCY AND COMPARABILITY. ICLR 2022.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept