Wandering within a world: Online contextualized few-shot learning

We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online , continual setting . In this setting , episodes do not have separate training and testing phases , and instead models are evaluated online while learning novel classes . As in the real world , where the presence of spatiotemporal context helps us retrieve learned skills in the past , our online few-shot learning setting also features an underlying context that changes throughout time . Object classes are correlated within a context and inferring the correct context can lead to better performance . Building upon this setting , we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world . Furthermore , we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past . 1 1 INTRODUCTION . In machine learning , many paradigms exist for training and evaluating models : standard train-thenevaluate , few-shot learning , incremental learning , continual learning , and so forth . None of these paradigms well approximates the naturalistic conditions that humans and artificial agents encounter as they wander within a physical environment . Consider , for example , learning and remembering peoples ’ names in the course of daily life . We tend to see people in a given environment—work , home , gym , etc . We tend to repeatedly revisit those environments , with different environment base rates , nonuniform environment transition probabilities , and nonuniform base rates of encountering a given person in a given environment . We need to recognize when we do not know a person , and we need to learn to recognize them the next time we encounter them . We are not always provided with a name , but we can learn in a semi-supervised manner . And every training trial is itself an evaluation trial as we repeatedly use existing knowledge and acquire new knowledge . In this article , we propose a novel paradigm , online contextualized few-shot learning , that approximates these naturalistic conditions , and we develop deep-learning architectures well suited for this paradigm . In traditional few-shot learning ( FSL ) ( Lake et al. , 2015 ; Vinyals et al. , 2016 ) , training is episodic . Within an isolated episode , a set of new classes is introduced with a limited number of labeled examples per class—the support set—followed by evaluation on an unlabeled query set . While this setup has inspired the development of a multitude of meta-learning algorithms which can be trained to rapidly learn novel classes with a few labeled examples , the algorithms are focused solely on the few classes introduced in the current episode ; the classes learned are not carried over to future episodes . Although incremental learning and continual learning methods ( Rebuffi et al. , 2017 ; Hou et al. , 2019 ) address the case where classes are carried over , the episodic construction of these frameworks seems artificial : in our daily lives , we do not learn new objects by grouping them with five other new objects , process them together , and then move on . 1Our code and dataset are released at : https : //github.com/renmengye/oc-fewshot-public T=2 T=3 T=4 T=1 t=1 t=2 t=3 t=4 To break the rigid , artificial structure of continual and few-shot learning , we propose a new continual few-shot learning setting where environments are revisited and the total number of novel object classes increases over time . Crucially , model evaluation happens on each trial , very much like the setup in online learning . When encountering a new class , the learning algorithm is expected to indicate that the class is “ new , ” and it is then expected to recognize subsequent instances of the class once a label has been provided . When learning continually in such a dynamic environment , contextual information can guide learning and remembering . Any structured sequence provides temporal context : the instances encountered recently are predictive of instances to be encountered next . In natural environments , spatial context— information in the current input weakly correlated with the occurrence of a particular class—can be beneficial for retrieval as well . For example , we tend to see our boss in an office setting , not in a bedroom setting . Human memory retrieval benefits from both spatial and temporal context ( Howard , 2017 ; Kahana , 2012 ) . In our online few-shot learning setting , we provide spatial context in the presentation of each instance and temporal structure to sequences , enabling an agent to learn from both spatial and temporal context . Besides developing and experimenting on a toy benchmark using handwritten characters ( Lake et al. , 2015 ) , we also propose a new large-scale benchmark for online contextualized few-shot learning derived from indoor panoramic imagery ( Chang et al. , 2017 ) . In the toy benchmark , temporal context can be defined by the co-occurrence of character classes . In the indoor environment , the context—temporal and spatial—is a natural by-product as the agent wandering in between different rooms . We propose a model that can exploit contextual information , called contextual prototypical memory ( CPM ) , which incorporates an RNN to encode contextual information and a separate prototype memory to remember previously learned classes ( see Figure 4 ) . This model obtains significant gains on few-shot classification performance compared to models that do not retain a memory of the recent past . We compare to classic few-shot algorithms extended to an online setting , and CPM consistently achieves the best performance . The main contributions of this paper are as follows . First , we define an online contextualized few-shot learning ( OC-FSL ) setting to mimic naturalistic human learning . Second , we build three datasets : 1 ) RoamingOmniglot is based on handwritten characters from Omniglot ( Lake et al. , 2015 ) ; 2 ) RoamingImageNet is based on images from ImageNet ( Russakovsky et al. , 2015 ) ; and 3 ) RoamingRooms is our new few-shot learning dataset based on indoor imagery ( Chang et al. , 2017 ) , which resembles the visual experience of a wandering agent . Third , we benchmark classic FSL methods and also explore our CPM model , which combines the strengths of RNNs for modeling temporal context and Prototypical Networks ( Snell et al. , 2017 ) for memory consolidation and rapid learning . 2 RELATED WORK . In this section , we briefly review paradigms that have been used for few-shot learning ( FSL ) and continual learning ( CL ) . Few-shot learning : FSL ( Lake et al. , 2015 ; Li et al. , 2007 ; Koch et al. , 2015 ; Vinyals et al. , 2016 ) considers learning new tasks with few labeled examples . FSL models can be categorized as based on : metric learning ( Vinyals et al. , 2016 ; Snell et al. , 2017 ) , memory ( Santoro et al. , 2016 ) , and gradient adaptation ( Finn et al. , 2017 ; Li et al. , 2017 ) . The model we propose , CPM , lies on the boundary between these approaches , as we use an RNN to model the temporal context but we also use metric-learning mechanisms and objectives to train . Several previous efforts have aimed to extend few-shot learning to incorporate more natural constraints . One such example is semi-supervised FSL ( Ren et al. , 2018 ) , where models also learn from a pool of unlabeled examples . While traditional FSL only tests the learner on novel classes , incremental FSL ( Gidaris & Komodakis , 2018 ; Ren et al. , 2019 ) tests on novel classes together with a set of base classes . These approaches , however , have not explored how to iteratively add new classes . In concurrent work , Antoniou et al . ( 2020 ) extend FSL to a continual setting based on image sequences , each of which is divided into stages with a fixed number of examples per class followed by a query set . It focuses on more flexible and faster adaptation since the models are evaluated online , and the context is a soft constraint instead of a hard separation of tasks . Moreover , new classes need to be identified as part of the sequence , crucial to any learner ’ s incremental acquisition of knowledge . Continual learning : Continual ( or lifelong ) learning is a parallel line of research that aims to handle a sequence of dynamic tasks ( Kirkpatrick et al. , 2017 ; Li & Hoiem , 2018 ; Lopez-Paz & Ranzato , 2017 ; Yoon et al. , 2018 ) . A key challenge here is catastrophic forgetting ( McCloskey & Cohen , 1989 ) , where the model “ forgets ” a task that has been learned in the past . Incremental learning ( Rebuffi et al. , 2017 ; Castro et al. , 2018 ; Wu et al. , 2019 ; Hou et al. , 2019 ; He et al. , 2020 ) is a form of continual learning , where each task is an incremental batch of several new classes . This assumption that novel classes always come in batches seems unnatural . Traditionally , continual learning is studied with tasks such as permuted MNIST ( Lecun et al. , 1998 ) or split-CIFAR ( Krizhevsky , 2009 ) . Recent datasets aim to consider more realistic continual learning , such as CORe50 ( Lomonaco & Maltoni , 2017 ) and OpenLORIS ( She et al. , 2019 ) . We summarize core features of these continual learning datasets in Appendix A . First , both CORe50 and OpenLORIS have relatively few object classes , which makes meta-learning approaches inapplicable ; second , both contain images of small objects with minimal occlusion and viewpoint changes ; and third , OpenLORIS does not have the desired incremental class learning . In concurrent work , Caccia et al . ( 2020 ) proposes a setup to unify continual learning and meta-learning with a similar online evaluation procedure . However , there are several notable differences . First , their models focus on a general loss function without a specific design for predicting new classes ; they predict new tasks by examining if the loss of exceeds some threshold . Second , the sequences of inputs are fully supervised . Lastly , their benchmarks are based on synthetic task sequences such as Omniglot or tiered-ImageNet , which are less naturalistic than our RoamingRooms dataset . Online meta-learning : Some existing work builds on early approaches ( Thrun , 1998 ; Schmidhuber , 1987 ) that tackle continual learning from a meta-learning perspective . Finn et al . ( 2019 ) proposes storing all task data in a data buffer ; by contrast , Javed & White ( 2019 ) proposes to instead learn a good representation that supports such online updates . In Jerfel et al . ( 2019 ) , a hierarchical Bayesian mixture model is used to address the dynamic nature of continual learning . Connections to the human brain : Our CPM model consists of multiple memory systems , consistent with claims of cognitive neuroscientists of multiple memory systems in the brain . The complementary learning systems ( CLS ) theory ( McClelland et al. , 1995 ) suggests that the hippocampus stores the recent experience and is likely where few-shot learning takes place . However , our model is more closely related to contextual binding theory ( Yonelinas et al. , 2019 ) , which suggests that long-term encoding of information depends on binding spatiotemporal context , and without this context as a cue , forgetting occurs . Our proposed CPM contains parallels to human brain memory components ( Cohen & Squire , 1980 ) . Long term statistical learning is captured in a CNN that produces a deep embedding . An RNN holds a type of working memory that can retain novel objects and spatiotemporal contexts . Lastly , the prototype memory represents the semantic memory , which consolidates multiple events into a single knowledge vector ( Duff et al. , 2020 ) . Other deep learning researchers have proposed multiple memory systems for continual learning . In Parisi et al . ( 2018 ) , the learning algorithm is heuristic and representations come from pretrained networks . In Kemker & Kanan ( 2018 ) , a prototype memory is used for recalling recent examples and rehearsal from a generative model allows this knowledge to be integrated and distilled into a long-term memory .

Summary: This work aims to make a realistic learning setting by combining few-shot learning and continual learning in the online setting. Similar to few-shot learning, the model needs to adapt to new classes with a few samples (at least in the beginning). Similar to continual learning, the model needs to learn new classes over time while being tested on the older classes as well. When encountering a new class, the model is expected to recognize that. Similar to the online setting, model evaluation happens on each trial, after which the model can be updated with that data (labeled or unlabeled). This new paradigm is called Online Contextualize Few-Shot Learning.