Wandering within a world: Online contextualized few-shot learning

We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online , continual setting . In this setting , episodes do not have separate training and testing phases , and instead models are evaluated online while learning novel classes . As in the real world , where the presence of spatiotemporal context helps us retrieve learned skills in the past , our online few-shot learning setting also features an underlying context that changes throughout time . Object classes are correlated within a context and inferring the correct context can lead to better performance . Building upon this setting , we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world . Furthermore , we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past . 1 1 INTRODUCTION . In machine learning , many paradigms exist for training and evaluating models : standard train-thenevaluate , few-shot learning , incremental learning , continual learning , and so forth . None of these paradigms well approximates the naturalistic conditions that humans and artificial agents encounter as they wander within a physical environment . Consider , for example , learning and remembering peoples ’ names in the course of daily life . We tend to see people in a given environment—work , home , gym , etc . We tend to repeatedly revisit those environments , with different environment base rates , nonuniform environment transition probabilities , and nonuniform base rates of encountering a given person in a given environment . We need to recognize when we do not know a person , and we need to learn to recognize them the next time we encounter them . We are not always provided with a name , but we can learn in a semi-supervised manner . And every training trial is itself an evaluation trial as we repeatedly use existing knowledge and acquire new knowledge . In this article , we propose a novel paradigm , online contextualized few-shot learning , that approximates these naturalistic conditions , and we develop deep-learning architectures well suited for this paradigm . In traditional few-shot learning ( FSL ) ( Lake et al. , 2015 ; Vinyals et al. , 2016 ) , training is episodic . Within an isolated episode , a set of new classes is introduced with a limited number of labeled examples per class—the support set—followed by evaluation on an unlabeled query set . While this setup has inspired the development of a multitude of meta-learning algorithms which can be trained to rapidly learn novel classes with a few labeled examples , the algorithms are focused solely on the few classes introduced in the current episode ; the classes learned are not carried over to future episodes . Although incremental learning and continual learning methods ( Rebuffi et al. , 2017 ; Hou et al. , 2019 ) address the case where classes are carried over , the episodic construction of these frameworks seems artificial : in our daily lives , we do not learn new objects by grouping them with five other new objects , process them together , and then move on . 1Our code and dataset are released at : https : //github.com/renmengye/oc-fewshot-public T=2 T=3 T=4 T=1 t=1 t=2 t=3 t=4 To break the rigid , artificial structure of continual and few-shot learning , we propose a new continual few-shot learning setting where environments are revisited and the total number of novel object classes increases over time . Crucially , model evaluation happens on each trial , very much like the setup in online learning . When encountering a new class , the learning algorithm is expected to indicate that the class is “ new , ” and it is then expected to recognize subsequent instances of the class once a label has been provided . When learning continually in such a dynamic environment , contextual information can guide learning and remembering . Any structured sequence provides temporal context : the instances encountered recently are predictive of instances to be encountered next . In natural environments , spatial context— information in the current input weakly correlated with the occurrence of a particular class—can be beneficial for retrieval as well . For example , we tend to see our boss in an office setting , not in a bedroom setting . Human memory retrieval benefits from both spatial and temporal context ( Howard , 2017 ; Kahana , 2012 ) . In our online few-shot learning setting , we provide spatial context in the presentation of each instance and temporal structure to sequences , enabling an agent to learn from both spatial and temporal context . Besides developing and experimenting on a toy benchmark using handwritten characters ( Lake et al. , 2015 ) , we also propose a new large-scale benchmark for online contextualized few-shot learning derived from indoor panoramic imagery ( Chang et al. , 2017 ) . In the toy benchmark , temporal context can be defined by the co-occurrence of character classes . In the indoor environment , the context—temporal and spatial—is a natural by-product as the agent wandering in between different rooms . We propose a model that can exploit contextual information , called contextual prototypical memory ( CPM ) , which incorporates an RNN to encode contextual information and a separate prototype memory to remember previously learned classes ( see Figure 4 ) . This model obtains significant gains on few-shot classification performance compared to models that do not retain a memory of the recent past . We compare to classic few-shot algorithms extended to an online setting , and CPM consistently achieves the best performance . The main contributions of this paper are as follows . First , we define an online contextualized few-shot learning ( OC-FSL ) setting to mimic naturalistic human learning . Second , we build three datasets : 1 ) RoamingOmniglot is based on handwritten characters from Omniglot ( Lake et al. , 2015 ) ; 2 ) RoamingImageNet is based on images from ImageNet ( Russakovsky et al. , 2015 ) ; and 3 ) RoamingRooms is our new few-shot learning dataset based on indoor imagery ( Chang et al. , 2017 ) , which resembles the visual experience of a wandering agent . Third , we benchmark classic FSL methods and also explore our CPM model , which combines the strengths of RNNs for modeling temporal context and Prototypical Networks ( Snell et al. , 2017 ) for memory consolidation and rapid learning . 2 RELATED WORK . In this section , we briefly review paradigms that have been used for few-shot learning ( FSL ) and continual learning ( CL ) . Few-shot learning : FSL ( Lake et al. , 2015 ; Li et al. , 2007 ; Koch et al. , 2015 ; Vinyals et al. , 2016 ) considers learning new tasks with few labeled examples . FSL models can be categorized as based on : metric learning ( Vinyals et al. , 2016 ; Snell et al. , 2017 ) , memory ( Santoro et al. , 2016 ) , and gradient adaptation ( Finn et al. , 2017 ; Li et al. , 2017 ) . The model we propose , CPM , lies on the boundary between these approaches , as we use an RNN to model the temporal context but we also use metric-learning mechanisms and objectives to train . Several previous efforts have aimed to extend few-shot learning to incorporate more natural constraints . One such example is semi-supervised FSL ( Ren et al. , 2018 ) , where models also learn from a pool of unlabeled examples . While traditional FSL only tests the learner on novel classes , incremental FSL ( Gidaris & Komodakis , 2018 ; Ren et al. , 2019 ) tests on novel classes together with a set of base classes . These approaches , however , have not explored how to iteratively add new classes . In concurrent work , Antoniou et al . ( 2020 ) extend FSL to a continual setting based on image sequences , each of which is divided into stages with a fixed number of examples per class followed by a query set . It focuses on more flexible and faster adaptation since the models are evaluated online , and the context is a soft constraint instead of a hard separation of tasks . Moreover , new classes need to be identified as part of the sequence , crucial to any learner ’ s incremental acquisition of knowledge . Continual learning : Continual ( or lifelong ) learning is a parallel line of research that aims to handle a sequence of dynamic tasks ( Kirkpatrick et al. , 2017 ; Li & Hoiem , 2018 ; Lopez-Paz & Ranzato , 2017 ; Yoon et al. , 2018 ) . A key challenge here is catastrophic forgetting ( McCloskey & Cohen , 1989 ) , where the model “ forgets ” a task that has been learned in the past . Incremental learning ( Rebuffi et al. , 2017 ; Castro et al. , 2018 ; Wu et al. , 2019 ; Hou et al. , 2019 ; He et al. , 2020 ) is a form of continual learning , where each task is an incremental batch of several new classes . This assumption that novel classes always come in batches seems unnatural . Traditionally , continual learning is studied with tasks such as permuted MNIST ( Lecun et al. , 1998 ) or split-CIFAR ( Krizhevsky , 2009 ) . Recent datasets aim to consider more realistic continual learning , such as CORe50 ( Lomonaco & Maltoni , 2017 ) and OpenLORIS ( She et al. , 2019 ) . We summarize core features of these continual learning datasets in Appendix A . First , both CORe50 and OpenLORIS have relatively few object classes , which makes meta-learning approaches inapplicable ; second , both contain images of small objects with minimal occlusion and viewpoint changes ; and third , OpenLORIS does not have the desired incremental class learning . In concurrent work , Caccia et al . ( 2020 ) proposes a setup to unify continual learning and meta-learning with a similar online evaluation procedure . However , there are several notable differences . First , their models focus on a general loss function without a specific design for predicting new classes ; they predict new tasks by examining if the loss of exceeds some threshold . Second , the sequences of inputs are fully supervised . Lastly , their benchmarks are based on synthetic task sequences such as Omniglot or tiered-ImageNet , which are less naturalistic than our RoamingRooms dataset . Online meta-learning : Some existing work builds on early approaches ( Thrun , 1998 ; Schmidhuber , 1987 ) that tackle continual learning from a meta-learning perspective . Finn et al . ( 2019 ) proposes storing all task data in a data buffer ; by contrast , Javed & White ( 2019 ) proposes to instead learn a good representation that supports such online updates . In Jerfel et al . ( 2019 ) , a hierarchical Bayesian mixture model is used to address the dynamic nature of continual learning . Connections to the human brain : Our CPM model consists of multiple memory systems , consistent with claims of cognitive neuroscientists of multiple memory systems in the brain . The complementary learning systems ( CLS ) theory ( McClelland et al. , 1995 ) suggests that the hippocampus stores the recent experience and is likely where few-shot learning takes place . However , our model is more closely related to contextual binding theory ( Yonelinas et al. , 2019 ) , which suggests that long-term encoding of information depends on binding spatiotemporal context , and without this context as a cue , forgetting occurs . Our proposed CPM contains parallels to human brain memory components ( Cohen & Squire , 1980 ) . Long term statistical learning is captured in a CNN that produces a deep embedding . An RNN holds a type of working memory that can retain novel objects and spatiotemporal contexts . Lastly , the prototype memory represents the semantic memory , which consolidates multiple events into a single knowledge vector ( Duff et al. , 2020 ) . Other deep learning researchers have proposed multiple memory systems for continual learning . In Parisi et al . ( 2018 ) , the learning algorithm is heuristic and representations come from pretrained networks . In Kemker & Kanan ( 2018 ) , a prototype memory is used for recalling recent examples and rehearsal from a generative model allows this knowledge to be integrated and distilled into a long-term memory .

Summary: The paper proposes a new learning paradigm that combines both few-shot learning(FSL)  and continual learning (CL) to provide a more realistic learning environment rather than the traditional train-test-retrain approach in FSL. Two environments are proposed, along with a novel dataset. The evaluation seems to be thorough, with strong baselines (conventional approaches adapted to the proposed setting). A novel approach is proposed based on augmenting ProtoNets with contextual memory and is shown to have consistently strong performance compared to the baselines on both tasks.