Few-Shot Continual Active Learning by a Robot
Anonymous Author(s)
Abstract
In this paper, we consider a challenging but realistic continual learning problem,
Few-Shot Continual Active Learning (FoCAL), where a CL agent is provided withunlabeled data for a new or a previously learned task in each increment and the agentonly has limited labeling budget available.
Towards this, we build on the continuallearning and active learning literature and develop a framework that can allow a CLagent to continually learn new object classes from a few labeled training examples.
Our framework represents each object class using a uniform Gaussian mixturemodel (GMM) and uses pseudo-rehearsal to mitigate catastrophic forgetting.
Theframework also uses uncertainty measures on the Gaussian representations ofthe previously learned classes to find the most informative samples to be labeledin an increment.
We evaluate our approach on the CORe-50 dataset and on areal humanoid robot for the object classification task.
The results show that ourapproach not only produces state-of-the-art results on the dataset but also allows areal robot to continually learn unseen objects in a real environment with limitedlabeling supervision provided by its user.
Introduction
Continual learning (CL)
[1, 2, 3] has emerged as a popular area of research in recent years because ofits limitless real-world applications, such as domestic robots, autonomous cars, etc.
Most continualmachine learning models [4, 5, 6, 7], however, are developed for constrained task-based continuallearning setups, where a CL model continually learns a sequence of tasks, one at a time, with allthe data of the current task labeled and available in an increment.
Real world systems, particularlyautonomous robots, do not have the luxury of getting a large amount of labeled data for each task.
In contrast, robots operating in real-world environments mostly have to learn from supervisionprovided by their users [8, 3].
Human teachers, however, would be unwilling answer a large numberof questions or label a large amount of data for the robot.
It would therefore be useful for robotsto self-supervise their learning and ask the human teachers to label the most informative trainingsamples from the environment, in each increment.
In this paper, we focus on this challenging problem,termed as Few-Shot Continual Active Learning (FoCAL).
One of the main problems faced by continual machine learning models is catastrophic forgetting,in which the CL model forgets the previous learned tasks when learning new knowledge.
In recentyears, several works in CL have focused on mitigating the catastrophic forgetting problem [9, 1].
Most of these works, however, are developed for the task-based continual learning setup, where themodel assumes that all the data for a task is available in an increment and it is fully labeled.
Theseconstraints are costly and limit the real-world application of CL models on robots.
Active learning hasemerged as an area of research in recent years, where machine learning models can choose the mostinformative samples to be labeled from a large corpus of unlabeled data, thus reducing the labellingeffort [10, 11].
Most active learning techniques use uncertainty sampling to request labels for themost uncertain objects [11, 10, 12].
These techniques, however, do not learn continually and thus would suffer from catastrophic forgetting.
These issues related to the development of "close-world“techniques for continual learning, active learning and open-set recognition have been explored indetail in [13].
In this paper, we consider FoCAL for the online continual learning scenario for the image classificationtask.
In this setup, a CL model (applied on a robot) receives a small amount of unlabeled image dataof objects from the environment in an increment, where the objects can belong to the previouslylearned classes by the model, or new classes.
The model is allowed to get a small number of objectsamples to be labeled by the user.
As the model continues to learn from new training samples,it does not have access to the raw image data of the previously learned objects.
Overall, FoCALis a combination of multiple challenging problems in machine learning, mainly Few-Shot Class
Incremental Learning (FSCIL) [6, 7], Active Learning [10, 14], and online continual learning [15].
To solve FoCAL, we get inspiration from the continual learning and active learning literature, todevelop protocols for continual learning models so that they can actively choose informative samplesin an increment.
Particularly, we take inspiration from FSCIL literature to develop a new FoCALmodel, in which we learn and preserve the feature representation of the previously learned objectsclasses by modelling them as Gaussian mixture models.
To mitigate catastrophic forgetting, we usepseudo-rehearsal [16] using the samples generated from the Gaussian distributions of the old classes,thus removing the need to store raw data for the classes.
Further, to choose most informative samplesfrom an unlabeled set, we use a combination of predictive entropy [17, 14] and viewpoint consistencymetrics [10, 12] on the GMM representation of the previously learned classes.
We perform extensiveevaluations of our proposed approach on the CORe-50 dataset [18], and on a real humanoid robotin an indoor environment.
Our approach outperforms state-of-the-art (SOTA) continual learningapproaches for FoCAL on the CORe-50 datast with significant margins.
Further, our approach canalso be integrated on a humanoid robot, and allow the robot to learn a large number of commonhousehold objects over a long period of time with limited supervision provided by the user.
Finally,as a part of this work, we also release the object dataset collected by our robot as a benchmark forfuture evaluations for FoCAL.
Few-Shot Continual Active Learning
We define the Few-Shot Continual Active Learning (FoCAL) problem as follows:
Suppose that an AIagent (e.g. a robot) gets a stream of unlabeled data sets D 1 pool , D 2 pool , ..., D tpool , ... over t increments,where D tpool = { x ti } | D tpool | i = .
In each increment, a continual learning model M with parameters Θcan only obtain a small number ( k t < | D tpool | ) of samples to be labeled.
Given the model M , anacquisition function a ( x t , M ) , where x t ∈ D t pool , is used by the AI agent to find the most informativesamples to be labeled in an increment t : x t ∗ = argmax D tpool a ( x t , M )
Therefore, in each increment t , the model M gets trained on small subsets of labeled data D t ={ ( x t ∗ i , y ti ) } | k t | i =1 , where y ti ∈ Y t represents the class label of x t ∗ i and Y t is the set of classes in the t -th increment.
Note that unlike most continual learning setups, Y i ∩ Y j ̸ = ∅ , ∀ i ̸
= j .
After training on
D t , the model M is tested to recognize all the encountered classes so far Y 1 , Y 2 , ..., Y t .
The mainchallenges of FoCAL are three-fold: (1) avoid catastrophic forgetting, (2) prevent overfitting on thefew training samples, (3) efficiently choose most informative samples in each increment.
For FoCAL for the task of object classification, we consider the model M (a CNN) as a compositionof a feature extractor f ( . ; θ ) with parameters θ and a classification model with weights W .
Thefeature extractor transforms the input images into a feature space F ∈ R n .
The classification modeltakes the features generated by the feature extractor, and generates an output vector followed by asoftmax function to generate multi-class probabilities.
In this paper, we use a pre-trained featureextractor, therefore parameters θ are fixed.
Thus, we incrementally finetune the classification modelon D 1 , D 2 , ... and get parameters W 1 , W 2 , ... .
In an increment t , we expand the output layer by | Y t |neurons to incorporate new classes.
Note that this setup does not alleviate the three challenges of
FoCAL mentioned above.
The subsections below describe the main components of our framework(Figure 1) to transform this setup for FoCAL.
2.1 GMM Based Continual Learning (GBCL)
We aim to develop a model that not only helps the system with continual learning but is also motivatedby the newness of an object.
To accomplish this, we must evaluate how different an incoming objectis from previously learned object classes, ideally without any additional supervision.
To accomplishthis, we consider a clustering-based approach to represent the distribution of object classes.
Unlikeprevious works on clustering-based approaches for continual learning [7, 8] that represent the objectclasses as mean feature vectors (centroids), we estimate the distribution of the each object class usinga uniform Gaussian mixture model (GMM).
We believe that representing each class data as a GMMmay better represent the true distribution of the data rather than assuming that the distribution iscircular.
We call our complete algorithm for continually learning GMMs of multiple object classes as
GMM based continual learning (GBCL).
Once the k feature vectors ( D t ) selected by acquisition function (Section 2.2) as most informativesamples are labeled by the oracle in increment t , GBCL is applied to learn GMMs for the classes
Y t .
For each i th feature vector x t ∗ i in D t labeled as y ti , if y ti is a new class never seen by the modelbefore, we initialize a new Gaussian distribution
N ( x ti , O ) for class y with x ti as the mean (centroid)and a zero matrix ( O ) as the covariance matrix 1 .
Otherwise, if y ti is a known class, we find theprobabilities N ( x ti | c y 1 , σ y 1 ) , N ( x ti | c y 2 , σ y 2 ) , ..., N ( x ti | c yj , σ yj ) , ..., N ( x ti |
c
yn y , σ yn y ) for x yi to belong toall the previously learned Gaussian distributions for class y , where n y is the total number of mixturecomponents in the GMM for class y , and c yj and σ yj represent the centroid and covariance matrix forthe j th mixture component of class y , respectively.
If the maximum probability among the calculatedprobabilities for all the distributions is higher than a pre-defined probability threshold P , x ti is used toupdate the parameters (centroid and covariance matrix) of the most probable distribution ( N ( c yj , σ yj ) )in class y .
The updated centroid ˆ c yj is calculated as a weighted mean between the previous centroidc yj and x ti :
[Equation]
where, w yj is the number of images already clustered in the j th (most probable) Gaussian distribution.
The updated covariance matrix ˆ σ yj is calculated based on the procedure described in [19]):
[Equation]
where, σ y j is the previous covariance matrix and ( x ti − ˆ ( c ) y j ) T ( x ti − ˆ c y j ) represents the covariancebetween x ti and ˆ c yj .
If, on the other hand, the maximum probability among the calculated probabilities for all the distributions is lower than P , a new Gaussian distribution N ( x ti , O ) is created for class ywith x ti as the centroid and O as the covariance matrix.
The result of this process is a set of N t uniform GMMs with parameters ϕ 1 , ϕ 2 , ..., ϕ N t for N tclasses learned up till increment t .
Note that instead of using the number of mixture components as ahyperparameter, we use the probability threshold.
This way we can start with a simple distributionmodel for each class (a single mixture component) and add more mixture components only when thenew images of the class are too dissimilar from the previous mixture components, and thus cannotbe modeled by the GMM.
Therefore, the total number of mixture components for each class can bedifferent based on the similarity between the images of the class.
In section 2.2, we use the same ideaof dissimilarity between an unlabeled image and a GMM to predict most informative samples.
2.1.1 Pseudo-rehearsal and Classifier Training
To avoid catastrophic forgetting, we use pseudo-rehearsal [16] to replay the old classes when learningfrom new data in increment t .
For pseudo-rehearsal, we sample the Gaussian distributions in the
GMMs of all the previously learned classes to generate a set of pseudo-feature vectors.
Note thatwe also store the total number of images clustered in each Gaussian distribution of the classes ( w yj )during the GMM learning phase (Section 2.1).
Therefore, we generate the same number of pseudofeature vectors as the original number of images for each class.
After generating the pseudo-featurevectors, the classifier model C ( . ; W ) is trained using the labeled dataset D t in increment t , and thepseudo-feature vectors of the previous classes.
For classification of a test image x , it is first passed through the feature extractor f ( x, θ ) and thenthrough the classifier C
( f ( x, θ ) , W ) .
Softmax function ( σ ) is then applied on the output to generateclass probabilities, and the class y ∗ with the maximum probability is predicted as the label for thetest image: y ∗ = argmax y σ ( W T f ( x, θ )) .
2.2 Active Learning using GMMs
We quantify the novelty of an object in terms of how much the model is uncertain about the object.
Unlike most active learning setups [14, 10], in FoCAL the model does not have access to a trainingset in each increment which can be used for training the model to predict uncertain object classes.
Further, even if the model does get trained to predict unknown object classes in the first increment, itwill catastrophically forget the criterion of novelty as it continually learns new object classes in thesubsequent increments (unknown classes in the first increment become known to the model in thesubsequent increments).
Therefore, we do not train our model on a specific loss function for activelearning, and instead use the GMM representations of the previously learned object classes to predictthe most uncertain objects.
Considering the FoCAL setup (as described in Section 2), in an increment t , the AI agent gets anunlabeled dataset D tpool and it must find k <
| D tpool | most informative object samples from thedataset to be labeled.
To develop an acquisition function for this, we use a combination of two activelearning techniques applied to the GMM representations of the previously learned object classes.
For the first technique, we use the prediction entropy H
[ y ∗ | x ti ] of an object as the acquisition function[17]:
[Equation]
For an unlabeled data point x ti ∈ D tpool , we find the predictive probability of x ti using the GMMrepresentation of the object classes learned in the previous increments.
The predictive probabilityp ( x ti | ϕ y ) for of x ti to belong to the GMM of a class y can be defined as:
[Equation]
Intuitively, if a sample x ti is already learned by the AI agent, then its probability to belong to one ofthe previously learned class GMMs must be high, and thus entropy for x ti must be low.
Therefore,top k samples with the highest entropy can be chosen as the most informative samples.
For the second technique, we use the idea of viewpoint consistency used in active learning [10, 12].
The main idea is that if an object is already learned by the agent before, then consistent predictionsmust be produced for the object under different viewpoints (see Figure 3 in the supplementary filefor an example).
We again use the GMM representations of the previously learned classes for thisacquisition function.
Let’s consider that there are multiple ( l ) viewpoints x t 1 , ..., x ti , ..., x tl of anunlabeled object j available to the AI agent in increment t .
We find the predictive probability of eachx ti to belong to the GMM representations of the previously learned classes (equation (6)).
Using themaximum predictive probability over all the classes we find the class prediction y t ∗ i for each x ti asy t ∗ i = argmax y =1 ,...,N t p ( x ti | ϕ y ) .
Next, we count the total number of times each of the N t − 1 classesis predicted among the different viewpoints of the object j .
Let S j
= { s 1 j , ..., s yj , ..., s N t − 1 j } representthe total number of times each class is predicted among the different viewpoints of the object.
Wenormalize set S j by dividing each s yj by l such that (cid:80) N t − 1 y =1 s yj = 1 .
We then take the inverse ofthe maximum value in S j ( 1 / max S j ), which represents the inverse of the highest percentage ofviewpoints of object j that are predicted consistently.
Thus, we use this term as the inconsistencyscore of the object j .
We use a combination of predictive entropy and viewpoint consistency to generate the final acquisitionfunction for our framework.
Before combining the two metrics, we transform the predictive entropyfor multiple viewpoints of the same objects as it is originally designed for individual data samples.
Formultiple viewpoints of an object j , we find the predictive entropy of all the viewpoints individuallyand then take an average to get the overall predictive entropy of object j .
Therefore, we maximizethe following combined function to get the most informative samples in increment t : [Equation]
where, δ is a hyperparameter that controls the contribution of predictive entropy and viewpointconsistency towards the overall uncertainty score of an object.
Note that the viewpoint consistencycan only be applied when the agent has access to multiple viewpoints of the same object.
In caseswhen multiple viewpoints are unavailable, we can use the predictive entropy alone.
In our experiments(Section 3), however, we use multiple viewpoints of all the objects.
Experiments
We first evaluate our approach for FoCAL on the Core-50 dataset [18] and then using the Pepperrobot.
We begin by presenting the implementation details and then compare our approach against
SOTA continual learning approaches on the CORe-50 dataset.
Finally, we present the evaluations ofour approach on Pepper.
Details about the CORe-50 dataset are in the supplementary file.
3.1 Implementation Details
We used the Pytorch deep learning framework [20] for implementation and training of all neuralnetwork models.
We used ResNet-18 [21] pre-trained on the ImageNet dataset [22] as a featureextractor for GBCL.
The same pre-trained network was also used in all the other continual learningapproaches for a fair comparison.
For GBCL, we only used the diagonal entries of the covariancematrix to keep the memory budget from growing drastically.
For FoCAL experiments on the CORe-50 dataset, we develop a new experimental setup, as thestandard active learning and continual learning experimental setups are not sufficient to test FoCAL(see Supplementary File for more details).
In this experimental setup for FoCAL on CORe-50, wecombined all 8 training sessions in CORe-50 to generate 400 training object instances.
In eachincrement, we randomly sampled m = 5 object instances (from 400 instances) and allowed themodel to learn the label of k = 1 out of 5 object instances, making it a challenging problem to learnfrom a single object instance in each increment. Once an object was learned by the model, it was Methods Increments
FT
LWF EWC iCaRL NCM CWR
FLB
CBCL
GBCL-rand
GBCL removed from the complete set of objects and, thus, was not available to the model in later increments.
Hence, we allowed the model to learn all objects in 400 increments with one object learned in eachincrement.
We used the test set accuracy at each increment as the evaluation metric.
After trainingin each increment, the model was tested on the complete test set of the CORe-50 dataset.
We onlyreport the accuracy for 70 increments for this experiment because all the approaches learn all 10classes by 70 increments and the final accuracy starts to saturate.
For robustness, all the experiments were performed 5 times with random seeds.
We report the averageand standard deviation of the accuracies.
Hyperparameters P and δ were chosen using cross-validationand were set to 0.2 and 0.7, respectively for all increments.
For the shallow neural network used forclassification in GBCL, we used a linear layer of the same size as the feature vectors generated bythe ResNet (512 × 1).
The shallow network was trained for 25 epochs using the cross-entropy lossoptimized with stochastic gradient descent (with 0.9 as momentum).
A fixed learning rate of 0.01and minibatches of size 64 were used for training.
3.2 Experiments on the CORe-50 Dataset
We compare our approach against 6 continual learning approaches (LWF [23], EWC [2], CWR [18],iCaRL [1], NCM [8], CBCL [7]), finetuning (FT) and a few-shot batch learning baseline (FLB) [24].
FLB uses a pre-trained CNN to extract feature vectors for images and trains a linear classifier usingcross entropy loss.
FLB is trained on the complete training data of the previous increments and thecurrent increment.
In other words, FLB does not learn continually and therefore should have anadvantage over other continual learning approaches (including ours).
FT uses the same architectureas FLB but FT is trained only on the data of the current increment.
FT suffers from catastrophicforgetting and therefore should produce lower accuracy than other continual learning approaches.
Tothe best of our knowledge, none of the eight approaches can be directly applied to FoCAL.
Hence, for
FLB, FT, LWF, EWC and CWR we used softmax-based uncertainty sampling to find most uncertainobjects.
For the softmax-based uncertainty scores, we found the softmax output of each image for anobject instance.
Then, we took the average of the maximum probability in the softmax output of eachimage of the object.
The object with the highest average probability score was chosen to be labeled.
Because iCaRL, NCM, and CBCL use centroids for classification, we used the average distance ofthe images of the new object from the centroids as the uncertainty score.
A brief description of all sixcontinual learning approaches is provided in Supplementary File.
3.2.1 Comparison with SOTA Approaches
Figure 2 compares our GBCL approach against random sampling and other SOTA approaches interms of classification accuracy on the fixed test set for 10 classes.
Our approach (GBCL) and FLBproduce similar results for all 70 increments and GBCL produces slightly better accuracy than FLB insome increments.
FLB, however, uses the data in all the previous increments at each new increment, while our approach only uses the data in the current increment and pseudo-feature vectors of theprevious classes.
We believe the reason for similar results for the two approaches is that GBCLgenerates and uses pseudo-feature vectors for previous classes to mitigate catastrophic forgetting,and it also learns the most informative samples in each increment.
As expected, FT suffers fromcatastrophic forgetting since it finetunes the network on new data in each increment.
Regularizationtechniques (LWF and EWC) also suffer from catastrophic forgetting and produce the same accuracyas FT.
CWR and iCaRL produce slightly better accuracy, however they also suffer from catastrophicforgetting.
NCM produces much better accuracy than the other approaches with the help of centroidsto remember past classes and prioritize objects to learn.
However, NCM’s accuracy is significantlyinferior to FLB and GBCL.
CBCL produces the best accuracy among the SOTA methods because itlearns multiple centroids per class.
However, CBCL’s accuracy is still lower than GBCL.
Note that after 400 increments, FLB produces ∼ 89% final accuracy and GBCL produces ∼ 88% (1%lower) accuracy.
The reason is that after 400 increments both models learned all the training samplesin the dataset, and therefore the advantage of informative sampling in GBCL fades away.
Further,unlike FLB, GBCL does not have access to all the data in the previous increments, and it suffers fromslight forgetting compared to batch learning (FLB).
In terms of memory storage, GBCL stores 239 clusters/Gaussian distributions (centroids and covariance matrices) for all the classes.
To avoid huge memory storage, we only use the diagonal entries ofthe covariance matrix for all the experiments.
{+Therefore, GBCL stores 239 × 2=478 vectors of size512 in memory after learning over 70 increments on the CORe-50 dataset, which requires only 0.
+}
MB.
In contrast, FLB (batch learning approach) stores all 21000 features vectors for all the classes,which requires 43.08 MB (44 times more than GBCL). For other approaches, FT does not storeany data, while LWF, EWC store minimal information, such as the predicted labels for the previousclasses and fisher matrix to capture weight importance. CWR stores an extra weight matrix for theclassification layer which requires 0.02 MB of storage space. iCaRL stores 2000 raw images for theprevious classes, which requires 393 MB.
NCM stores only 10 centroids for the previous classes,which requires 0.02 MB.
Finally, CBCL stores 315 centroids which requires 0.64 MB.
{+Note that allof the approaches also store a ResNet-18 model trained on the previous classes which requires 
+}
MB of space.
This analysis shows that GBCL provides the best trade-off between memory storageand the overall performance in comparison with the other approaches.
3.2.2 Comparison with Random Sampling
To provide more insight into our approach, we tested our GBCL approach without active learning(GBCL-random).
In this case, GBCL gets a random object to be labeled in each increment.
GBCLrandom produced significantly lower accuracy than FLB and GBCL for all the increments, especiallyin earlier increments.
These results depict the contribution of our active learning approach toefficiently choose which objects to learn.
In comparison with other SOTA approaches, GBCL-randombeats all the approaches (except CBCL) in later increments but it is below NCM and CBCL in earlierincrements.
These results also demonstrate the effectiveness of GBCL (Section 2.1) of our approachto mitigate catastrophic forgetting.
3.2.3 Comparison of Learning Efficiency
To further evaluate the efficiency of of our approach for FoCAL, we report the total number ofincrements taken by all the approaches to learn all 10 classes (see Table 1).
GBCL requires the leastnumber of increments (15) to learn the 10 classes.
FLB takes 30 increments (15 more than GBCL)to learn all the 10 classes because it uses the softmax based sampling.
NCM and iCaRL also takesimilar number of increments because they use centroids to calculate the uncertainty scores but do nothave access to all the data of the previous increments.
CBCL is the closest to our approach and learnsall the classes in 23 increments (still 8 more than GBCL).
FT, LWF and EWC all use softmax-basedsampling without using any previous data, so they take a large number of increments to learn all 10classes.
Lastly, random sampling takes the most number of increments (65) to learn all the 10 classes.
These results clearly show that our approach is the most efficient at finding the unknown objects andlearning them in a small number of increments. [Table]
3.3 Experiment on the Pepper Robot
In this experiment we used the Pepper robot to learn 240 household objects belonging to 20 classes(12 objects per class).
We performed the experiment in an indoor lab environment with objectsplaced at four different locations in the lab with different backgrounds.
At each location we placed 5different objects on a table (Figure 2 (a, b)in the supplementary file).
For this setup, the robot firstlocalized the 5 individual objects in the image.
We use RetinaNet [25] pre-trained on MS COCOdataset [26] for object detection and localization.
The robot then captures images of the individualobjects from multiple viewpoints (see Figure 2 (c) in the supplementary file for an example).
Theseimages are used by the active learning module of GBCL to find the most informative object.
The user(experimenter) then provides the class label of the chosen object to the robot using a keyboard.
Thenewly labeled object is then used to update GBCL.
Details about our complete system for the Pepperrobot are in the Supplementary File.
This experiment tried to reproduce some of the challenges that might arise in realistic householdenvironments.
For example, a large number of objects were learned at different times, over multipledays (8 weeks), depicting a true lifelong learning robot in real-world environments.
Different objectswere used in the training and test sets.
Moreover, the background of the objects was non-ideal sincemany parts of the background could be viewed as objects by the object detector.
For example, consider
Figure 2 where the air-conditioner and the window could be viewed as objects by the object detector.
Objects also varied in size and some were almost transparent.
Further, the lighting conditions werenot ideal.
Some images were taken at night, while some during the day with different variations ofsun light coming through the windows in the environment (see Figure 2 and Figure 6 for examples.
More examples in the Supplementary File).
To evaluate this experiment, we created a test set of 60 objects (3 objects per class) that were differentfrom objects in the training set.
We tested Pepper’s ability to classify the objects in the test set after40, 80, 120, 160, 200, and 240 increments of learning new objects.
We also report the number ofobject classes learned by Pepper after each of these increments (Table 2).
After only 40 increments,
Pepper learned 19 object classes and achieved 70.0% accuracy on the fixed test set.
For the rest of theincrements the increase in accuracy was minimal, especially in later increments (120-240), similar tothe results on the CORe-50 dataset.
The reason is that in the earlier increments Pepper learned objectsbelonging to a large number of classes quickly which increased the overall accuracy.
However, in thelater increments Pepper only learned about more objects of the previously learned classes, thus theincrease in overall accuracy was minimal.
We should also note that even after 240 increments (sameas the total number of training objects) Pepper only learned 197 objects.
The reason is that the objectlocalization module was not perfect, and it failed to detect many objects (43 objects).
Related Work
Most continual learning approaches are designed for class-incremental setting in which a model(often a neural network) learns from training data of different classes in different increments and thenevaluated on the test data of all the classes learned so far [1].
Most existing class-incremental learning(CIL) methods avoid catastrophic forgetting by storing a portion of the training samples from previousclasses and retraining the model on a mixture of the stored data and new data [1, 27].
However,this approach does not scale as additional data exhausts memory capacity limiting performance inreal-world applications.
To avoid this problem, some CL approaches use regularization techniques[23, 2].
Although these approaches solve the memory storage issues, their performance is significantlyinferior to approaches that store old class data.
A few works have been proposed for FSCIL, tocontinually learn from a few-examples per class using limited memory [7, 6, 28].
However, asmentioned in [13, 29], one significant limitation of all CL approaches is that they require the completetraining data of each class to be labeled and available in a single increment, however in real-world robotic applications data is available in a streaming manner (online learning [15]) and it might bemostly unlabeled.
For object learning, many researchers have presented active learning techniques using uncertaintysampling [30, 14, 10, 11, 31, 12].
Most of these approaches train deep neural networks with specialloss terms such that the networks can predict the most uncertain samples.
All of these approaches,however, are trained for batch learning setting and will thus suffer from catastrophic forgetting whenattempting to learn continually.
Further, active learning approaches can predict unknown classes inthe first increment after batch training.
However, in a continual learning setting these approacheslose their ability to recognize unknown classes in subsequent increments because the model willassign the unknown classes to the newly learned classes (learned in the previous increments) thatwere unknown in the first increment. Another related field to active learning is open-set recognition (or out of the distribution detection(OOD)) [32, 33, 33, 34], in which a model that is trained on a set of tasks/classes might face samplesbelonging previously unknown classes in the test set. Therefore, during the testing/deployment phase,the model must be able to detect which of the samples belong to unknown classes.
Although methodsfor OOD [35, 36] have been developed to detect unknown classes, these models do not train on thenewly encountered unknown samples. Mundt et al. [13] present a thorough analysis of previouscontinual and active learning, and OOD works, and suggest a common viewpoint with open-setrecognition acting as a natural interface between continual and active learning. Towards this goal,we present an approach that allows an AI agent to actively select the most informative (unknown)samples from unlabeled data, and continually learn from the actively selected object samples.
Conclusion
This paper presents and evaluates a novel method for a challenging problem: Few-Shot Continual
Active Learning (FoCAL) for object classification.
Experimental results demonstrate that our approach for FoCAL is highly efficient and helps the model learn the most uncertain objects continuallywithout forgetting earlier classes.
Our approach not only outperforms the SOTA approaches on abenchmark dataset but also allows a robot to actively learn objects in a real environment over a longperiod of time.
We also commit to releasing the dataset collected by the Pepper robot (after the paperis published) as a benchmark for future evaluations of FoCAL models.
The work presented here has a few limitations. The experimental setup is still quite simplecompared to a real household environment. We assume that correct object labels are provided bythe human assistant to the robot. We did not recruit real participants to teach the robot, but theexperimenter acted as a user for the robot.
We used a fixed feature extractor and object detector.
Inthe future, we hope to work on these limitations and test our system with real human participants. Particularly, we hope to develop techniques that can allow feature representations to be learned froma few samples over a large number of increments. There are a number of positive and negative long-term outcomes of this work, beyond those thatcould be applied to any improved machine learning application. With respect to positive outcomes,the improved performance for continual learning by our approach may one day lead to robots thatcan tailor their behavior and learning to the needs of a person.
We envision a home service robotthat uses continual learning to learn a person’s food preferences before serving them breakfast, ora domestic service robot that learns how a person prefers their home to be cleaned and adjusts itsbehavior to their preferences.
Although FoCAL can help improve autonomous robots in real-world situations, uncertainty-basedsampling can have some downsides as well. One potential downside is that an autonomous robotmight learn unnecessary objects which are never going to be used by the robot. Learning such objectscan potentially decrease decrease the recognition performance on necessary objects that are requiredby the robot. Using FoCAL to learn unnecessary objects can also take valuable time that might bebetter used by the robot to perform necessary actions.
For example, consider a dish washing robotwhich only needs to know kitchen utensils.
The robot does not need to learn mechanical tools orchildren’s toys but uncertainty-based sampling will cause the robot to learn these unnecessary objects.
In such cases, some constraints might be added to a FoCAL system in order to avoid the robot fromlearning unnecessary objects.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes](b) Did you describe the limitations of your work?
[Yes](c)
Did you discuss any potential negative societal impacts of your work?
[Yes](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A](b)
Did you include complete proofs of all theoretical results?
[N/A]3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes](b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes](c)
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]