Original: Wandering Within a World: Online Contextualized Few-Shot Learning
Revised: Wandering Within a World: Online Contextualized Few-Shot Learning

Original: Anonymous Authors

Original: Abstract
Revised: Abstract

Original: We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting.
Revised: We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting.

Original: In this setting, episodes do not have separate training and testing phases, and instead models are evaluated online while learning novel classes.
Revised: In this setting, episodes do not have separate training and testing phases, and instead models are evaluated online while learning novel classes.

Original: As in the real world, where the presence of spatiotemporal context helps us retrieve learned skills in the past, our online few-shot learning setting also features an underlying context that changes throughout time.
Revised: As in the real world, where the presence of spatiotemporal context helps us retrieve learned skills in the past, our online few-shot learning setting also features an underlying context that changes throughout time.

Original: Object classes are correlated within a context and inferring the correct context can lead to better performance.
Revised: Object classes are correlated within a context and inferring the correct context can lead to better performance.

Original: Building upon this setting, we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world.
Revised: Building upon this setting, we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world.

Original: Furthermore, we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past.
Revised: Furthermore, we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past.

Original: 1. Introduction
Revised: 1. Introduction

Original: In machine learning, many paradigms exist for training and evaluating models:
Revised: In machine learning, many paradigms exist for training and evaluating models:

Original: standard train-then-evaluate, fewshot learning, incremental learning, continual learning, and so forth.
Revised: standard train-then-evaluate, fewshot learning, incremental learning, continual learning and so forth.

Original: None of these paradigms well approximates the naturalistic conditions that humans and artiﬁcial agents encounter as they wander within a physical environment.
Revised: None of these paradigms well approximates the naturalistic conditions that humans and artiﬁcial agents encounter as they wander within a physical environment.

Original: Consider, for example, learning and remembering peoples’ names in the course of daily life.
Revised: Consider, for example, learning and remembering peoples’ names in the course of daily life.

Original: We tend to see people in a given environment—work, home, gym, etc.
Revised: We tend to see people in a given environment—work, home, gym, etc.

Original: We tend to repeatedly revisit those environments, with different environment base rates, nonuniform environment transition probabilities, and nonuniform base rates of encountering a  given person in a given environment.
Revised: We tend to repeatedly revisit those environments, with different environment base rates, nonuniform environment transition probabilities and nonuniform base rates of encountering a (s given person in a given environment.

Original: We need to recognize when we do not know a person, and we need to learn to recognize them the next time we encounter them.
Revised: We need to recognize when we do not know a person, and we need to learn to recognize them the next time we encounter them.

Original: We are not always provided with a name, but we can learn in a semi-supervised manner.
Revised: We are not always provided with a name, but we can learn in a semi-supervised manner.

Original: And every training trial is itself an evaluation trial as we repeatedly use existing knowledge and acquire new knowledge.
Revised: And every training trial is itself an evaluation trial as we repeatedly use existing knowledge and acquire new knowledge.

Original: In this article, we propose a novel paradigm, online contextualized few-shot learning , that approximates these naturalistic conditions, and we develop deep-learning architectures well suited for this paradigm.
Revised: In this article, we propose a novel paradigm, online contextualized few-shot learning , that approximates these naturalistic conditions, and we develop deep-learning architectures well suited for this paradigm.

Original: In traditional few-shot learning (FSL) (Lake et al., 2015;
Revised: In traditional few-shot learning (FSL) (Lake et al., 2015;

Original: Vinyals et al., 2016), training is episodic.
Revised: Vinyals et al., 2016), training is episodic.

Original: Within an isolated episode, a set of new classes is introduced with a limited number of labeled examples per class—the support set— followed by evaluation on an unlabeled query set.
Revised: Within an isolated episode, a set of new classes is introduced with a limited number of labeled examples per class—the support set— followed by evaluation on an unlabeled query set.

Original: While this setup has inspired the development of a multitude of meta-learning algorithms which can be trained to rapidly learn novel classes with a few labeled examples, the algorithms are focused solely on the few classes introduced in the current episode; the classes learned are not carried over to future episodes.
Revised: While this setup has inspired the development of a multitude of meta-learning algorithms which can be trained to rapidly learn novel classes with a few labeled examples, the algorithms are focused solely on the few classes introduced in the current episode; the classes learned are not carried over to future episodes.

Original: Although incremental learning and continual learning methods (Rebufﬁ et al., 2017; Hou et al., 2019) address the case where classes are carried over, the episodic construction of these frameworks seems artiﬁcial: in our daily lives, we do not learn new objects by grouping them with ﬁve other new objects, process them together, and then move on.
Revised: Although incremental learning and continual learning methods (Rebufﬁ et al., 2017; Hou et al., 2019) address the case where classes are carried over, the episodic construction of these frameworks seems artiﬁcial: in our daily lives, we do not learn new objects by grouping them with ﬁve other new objects, process them together, and then move on.

Original: To break the rigid, artiﬁcial structure of continual and fewshot learning, we propose a new continual few-shot learning setting where environments are revisited and the total number of novel object classes increases over time.
Revised: To break the rigid, artiﬁcial structure of continual and fewshot learning, we propose a new continual few-shot learning setting where environments are revisited and the total number of novel object classes increases over time.

Original: Crucially, model evaluation happens on each trial, very much like the setup in online learning.
Revised: Crucially, model evaluation happens on each trial, very much like the setup in online learning.

Original: When encountering a new class, the learning algorithm is expected to indicate that the class is “new,” and it is then expected to recognize subsequent instances of the class once a label has been provided.
Revised: When encountering a new class, the learning algorithm is expected to indicate that the class is “new,” and it is then expected to recognize subsequent instances of the class once a label has been provided.

Original: When learning continually in such a dynamic environment, contextual information can guide learning and remembering.
Revised: When learning continually in such a dynamic environment, contextual information can guide learning and remembering.

Original: Any structured sequence provides temporal context : the instances encountered recently are predictive of instances to be encountered next.
Revised: Any structured sequence provides temporal context : the instances encountered recently are predictive of instances to be encountered next.

Original: In natural environments, spatial context —information in the current input weakly correlated with the occurrence of a particular class—can be beneﬁcial for retrieval as well.
Revised: In natural environments, spatial context —information in the current input weakly correlated with the occurrence of a particular class—can be beneﬁcial for retrieval as well.

Original: For example, we tend to see our boss in an ofﬁce setting, not in a bedroom setting.
Revised: For example, we tend to see our boss in an ofﬁce setting, not in a bedroom setting.

Original: Human memory retrieval beneﬁts from both spatial and temporal context (Howard, 2017; Kahana, 2012).
Revised: Human memory retrieval beneﬁts from both spatial and temporal context (Howard, 2017; Kahana, 2012).

Original: In our online few-shot learning setting, we provide spatial context in the presentation of each instance and temporal structure to sequences, enabling an agent to learn from both spatial and temporal context.
Revised: In our online few-shot learning setting, we provide spatial context in the presentation of each instance and temporal structure to sequences, enabling an agent to learn from both spatial and temporal context.

Original: Besides developing and experimenting on a toy benchmark using handwritten characters (Lake et al., 2015), we also propose a new large-scale benchmark for online contextualized few-shot learning derived from indoor panoramic imagery (Chang et al., 2017).
Revised: Besides developing and experimenting on a toy benchmark using handwritten characters (Lake et al., 2015), we also propose a new large-scale benchmark for online contextualized few-shot learning derived from indoor panoramic imagery (Chang et al., 2017).

Original: In the toy benchmark, temporal context can be deﬁned by the co-occurrence of character classes.
Revised: In the toy benchmark, temporal context can be deﬁned by the co-occurrence of character classes.

Original: In the indoor environment, the context— temporal and spatial—is a natural by-product as the agent wandering in between different rooms.
Revised: In the indoor environment, the context— temporal and spatial—is a natural by-product as the agent wandering in between different rooms.

Original: We propose a model that can exploit contextual information, called contextual prototypical memory ( CPM ), which incorporates an RNN to encode contextual information and a separate prototype memory to remember previously learned classes (see Figure 4).
Revised: We propose a model that can exploit contextual information, called contextual prototypical memory ( CPM ), which incorporates an RNN to encode contextual information and a separate prototype memory to remember previously learned classes (see Figure 4).

Original: This model obtains signiﬁcant gains on few-shot classiﬁcation performance compared to models that do not retain a memory of the recent past.
Revised: This model obtains signiﬁcant gains on few-shot classiﬁcation performance compared to models that do not retain a memory of the recent past.

Original: We compare to classic few-shot algorithms (Vinyals et al., 2016; Allen et al., 2019;
Revised: We compare to classic few-shot algorithms (Vinyals et al., 2016; Allen et al., 2019;

Original: Snell et al., 2017; Javed & White, 2019; Santoro et al., 2016) extended to an online setting, and CPM consistently achieves the best performance.
Revised: Snell et al., 2017; Javed & White, 2019; Santoro et al., 2016) extended to an online setting, and CPM consistently achieves the best performance.

Original: The main contributions of this paper are as follows.
Revised: The main contributions of this paper are as follows.

Original: First, we deﬁne an online contextualized few-shot learning (OC-FSL) setting to mimic naturalistic human learning.
Revised: First, we deﬁne an online contextualized few-shot learning (OC-FSL) setting to mimic naturalistic human learning.

Original: Second, we build two datasets.
Revised: Second, we build two datasets.

Original: The RoamingOmniglot dataset is based on handwritten characters from Omniglot (Lake et al., 2015) and the RoamingRooms dataset is our new few-shot learning dataset based on indoor imagery (Chang et al., 2017), which resembles the visual experience of a wandering agent.
Revised: The RoamingOmniglot dataset is based on handwritten characters from Omniglot (Lake et al., 2015) and the RoamingRooms dataset is our new few-shot learning dataset based on indoor imagery (Chang et al., 2017), which resembles the visual experience of a wandering agent.

Original: Third, we benchmark classic FSL methods and also explore our CPM model, which combines the strengths of RNNs for modeling temporal context and Prototypical Networks
Revised: Third, we benchmark classic FSL methods and also explore our CPM model, which combines the strengths of RNNs for modeling temporal context and Prototypical Networks

Original: (Snell et al., 2017) for memory consolidation and rapid learning.
Revised: (Snell et al., 2017) for memory consolidation and rapid learning.

Original: 2. Related Work
Revised: 2. Related Work

Original: In this section, we brieﬂy review paradigms that have been used for few-shot learning (FSL) and continual learning (CL).
Revised: In this section, we brieﬂy review paradigms that have been used for few-shot learning (FSL) and continual learning (CL).

Original: Table 1 compares these paradigms based on various properties of the task;
Revised: Table 1 compares these paradigms based on various properties of the task;

Original: To denote properties that are incorporated in some preliminary form, we use to denote properties that are not fully implemented but have some preliminary form.
Revised: To denote properties that are incorporated in some preliminary form, we use to denote properties that are not fully implemented but have some preliminary form.

Original: For example, the class-incremental learning paradigm evaluates models after each task is completed, which is similar to our online evaluation in spirit but still not the same as the evaluation does not take place after
Revised: For example, the class-incremental learning paradigm evaluates models after each task is completed, which is similar to our online evaluation in spirit but still not the same as the evaluation does not take place after

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: each example.
Revised: each example.

Original: Our proposed online contextual few-shot learning (OC-FSL) spans the complete set of features of the other paradigms.
Revised: Our proposed online contextual few-shot learning (OC-FSL) spans the complete set of features of the other paradigms.

Original: We also review relevant models and their relationship to our CPM.
Revised: We also review relevant models and their relationship to our CPM.

Original: Few-shot learning: FSL (Lake et al., 2015; Li et al., 2007;
Revised: Few-shot learning: FSL (Lake et al., 2015; Li et al., 2007;

Original: Koch et al., 2015; Vinyals et al., 2016) considers learning new tasks with few labeled examples.
Revised: Koch et al., 2015; Vinyals et al., 2016) considers learning new tasks with few labeled examples.

Original: FSL models can be categorized as based on: metric learning (Vinyals et al., 2016;
Revised: FSL models can be categorized as based on: metric learning (Vinyals et al., 2016;

Original: Snell et al., 2017), memory (Santoro et al., 2016), and gradient adaptation (Finn et al., 2017; Li et al., 2017).
Revised: Snell et al., 2017), memory (Santoro et al., 2016), and gradient adaptation (Finn et al., 2017; Li et al., 2017).

Original: The model we propose, CPM, lies on the boundary between these approaches, as we use an RNN to model the temporal context but we also use metric-learning mechanisms and objectives to train.
Revised: The model we propose, CPM, lies on the boundary between these approaches, as we use an RNN to model the temporal context but we also use metric-learning mechanisms and objectives to train.

Original: Several previous efforts have aimed to extend few-shot learning to incorporate more natural constraints.
Revised: Several previous efforts have aimed to extend few-shot learning to incorporate more natural constraints.

Original: One such example is semi-supervised FSL (Ren et al., 2018), where models learn not only from a few labeled examples but also from a pool of unlabeled examples.
Revised: One such example is semi-supervised FSL (Ren et al., 2018), where models learn not only from a few labeled examples but also from a pool of unlabeled examples.

Original: While traditional FSL only tests the learner on novel classes, incremental FSL (Gidaris & Komodakis, 2018; Ren et al., 2019) tests on novel classes together with a set of base classes.
Revised: While traditional FSL only tests the learner on novel classes, incremental FSL (Gidaris & Komodakis, 2018; Ren et al., 2019) tests on novel classes together with a set of base classes.

Original: These approaches, however, have not explored how to iteratively add new classes.
Revised: These approaches, however, have not explored how to iteratively add new classes.

Original: In concurrent work, Antoniou et al.
Revised: In concurrent work, Antoniou et al.

Original: (2020) extend FSL to a continual setting based on image sequences, each of which is divided into stages with a ﬁxed number of examples per class followed by a query set.
Revised: (2020) extend FSL to a continual setting based on image sequences, each of which is divided into stages with a ﬁxed number of examples per class followed by a query set.

Original: Our paradigm focuses on more ﬂexible and faster adaptation since the models are evaluated online, and context is a soft constraint instead of a hard separation of tasks.
Revised: Our paradigm focuses on more ﬂexible and faster adaptation since the models are evaluated online, and context is a soft constraint instead of a hard separation of tasks.

Original: Moreover, new classes need to be identiﬁed as part of the sequence, crucial to any learner’s incremental acquisition of knowledge.
Revised: Moreover, new classes need to be identiﬁed as part of the sequence, crucial to any learner’s incremental acquisition of knowledge.

Original: Continual learning: Continual (or lifelong) learning is a parallel line of research that aims to handle a sequence of dynamic tasks (Kirkpatrick et al., 2017; Li & Hoiem, 2018; Lopez-Paz & Ranzato, 2017; Yoon et al., 2018).
Revised: Continual learning: Continual (or lifelong) learning is a parallel line of research that aims to handle a sequence of dynamic tasks (Kirkpatrick et al., 2017; Li & Hoiem, 2018; Lopez-Paz & Ranzato, 2017; Yoon et al., 2018).

Original: A key challenge here is catastrophic forgetting (McCloskey & Cohen, 1989; French, 1999), where the model “forgets” 
Revised: A key challenge here is catastrophic forgetting (McCloskey & Cohen, 1989; French, 1999), where the model “forgets” [Figure]

Original: A) with and classes
Revised: T= T= … T=

Original: [Figure]
Revised: [Figure]

Revised: Agent

Revised: Env 1: Kitchen

Revised: Env 3: Office

Revised: Env 2: Living room

Revised: Env 3: Office

Revised: Env 1: Kitchen v

Revised: World

Revised: Cls v

Revised: Evaluation

Revised: New

Revised: Cls

Revised: Cls

Revised: Cls

Original:  a task that has been learned in the past.
Revised: T= New New Cls t= t= t= t= Env 1: Kitchen a task that has been learned in the past.

Original: Incremental learning (Hou et al., 2019; Rebufﬁ et al., 2017; Kemker & Kanan, 2018) is a form of continual learning, where each task is an incremental batch of several new classes.
Revised: Incremental learning (Hou et al., 2019; Rebufﬁ et al., 2017; Kemker & Kanan, 2018) is a form of continual learning, where each task is an incremental batch of several new classes.

Original: This assumption that novel classes always come in batches seems unnatural.
Revised: This assumption that novel classes always come in batches seems unnatural.

Original: Traditionally, continual learning is studied with tasks such as permuted MNIST (Lecun et al., 1998) or split-CIFAR
Revised: Traditionally, continual learning is studied with tasks such as permuted MNIST (Lecun et al., 1998) or split-CIFAR

Original: (Krizhevsky, 2009).
Revised: (Krizhevsky, 2009).

Original: Recent datasets aim to consider more realistic continual learning, such as CORe50 (Lomonaco & Maltoni, 2017) and OpenLORIS
Revised: Recent datasets aim to consider more realistic continual learning, such as CORe50 (Lomonaco & Maltoni, 2017) and OpenLORIS

Original: (She et al., 2019).
Revised: (She et al., 2019).

Original: We summarize core features of these continual learning datasets in the Appendix.
Revised: We summarize core features of these continual learning datasets in the Appendix.

Original: First, both datasets have relatively few object classes, which makes meta-learning approaches inapplicable; second, both datasets contain images of small objects with minimal occlusion and viewpoint changes; and third, OpenLORIS does not have the desired incremental class learning.
Revised: First, both datasets have relatively few object classes, which makes meta-learning approaches inapplicable; second, both datasets contain images of small objects with minimal occlusion and viewpoint changes; and third, OpenLORIS does not have the desired incremental class learning.

Original: Online meta-learning: Some existing work builds on early approaches (Thrun, 1998; Schmidhuber, 1987) that tackle continual learning from a meta-learning perspective.
Revised: Online meta-learning: Some existing work builds on early approaches (Thrun, 1998; Schmidhuber, 1987) that tackle continual learning from a meta-learning perspective.

Original: Finn et al.
Revised: Finn et al.

Original: (2019) propose storing all task data in a data buffer to perform inner loop gradient descent; by contrast,
Revised: (2019) propose storing all task data in a data buffer to perform inner loop gradient descent; by contrast,

Original: Javed & White (2019) propose to directly update the inner loop with the current input and instead learn a good representation that supports such online updates.
Revised: Javed & White (2019) propose to directly update the inner loop with the current input and instead learn a good representation that supports such online updates.

Original: In Jerfel et al.
Revised: In Jerfel et al.

Original: (2019), a hierarchical Bayesian mixture model is used to address the dynamic nature of continual learning.
Revised: (2019), a hierarchical Bayesian mixture model is used to address the dynamic nature of continual learning.

Original: To evaluate the performance of online meta-learning methods, the papers above also created a few synthetic continual learning tasks which are less realistic than ours (see Table 1). 
Revised: To evaluate the performance of online meta-learning methods, the papers above also created a few synthetic continual learning tasks which are less realistic than ours (see Table 1). Connections to the human brain : Our

Original: Connections to the human brain: Our CPM model consists of multiple memory systems, consistent with claims of cognitive neuroscientists of multiplememory systems in the brain.
Revised: model consists of multiple memory systems, consistent with claims of cognitive neuroscientists of multiple

Original:  The complementary learning systems (CLS) theory (Mcclelland et al., 1995) suggests that the hippocampus stores the recent experience and supports fast recall of novel concepts, which is likely where few-shot learning takes place.
Revised: in the brain. The complementary learning systems (CLS) theory (Mcclelland et al., 1995) suggests that the hippocampus stores the recent experience and supports fast recall of novel concepts, which is likely where few-shot learning takes place.

Original: When the same content has been retrieved several times, the neocortex changes synaptic weights slowly to learn a more robust representation.
Revised: When the same content has been retrieved several times, the neocortex changes synaptic weights slowly to learn a more robust representation.

Original: The hippocampal system can be further divided into an episodic memory for individual spatiotemporal events and a semantic memory for a concrete piece of knowledge (Cohen & Squire, 1980).
Revised: The hippocampal system can be further divided into an episodic memory for individual spatiotemporal events and a semantic memory for a concrete piece of knowledge (Cohen & Squire, 1980).

Original: Our proposed CPM contains parallels to these components.
Revised: Our proposed CPM contains parallels to these components.

Original: Long term statistical learning is captured in a CNN that produces a deep embedding.
Revised: Long term statistical learning is captured in a CNN that produces a deep embedding.

Original: An RNN holds a type of working memory that can retain novel objects and spatiotemporal contexts.
Revised: An RNN holds a type of working memory that can retain novel objects and spatiotemporal contexts.

Original: Lastly, the prototype memory represents the semantic memory, which consolidates multiple events into a single knowledge vector.
Revised: Lastly, the prototype memory represents the semantic memory, which consolidates multiple events into a single knowledge vector.

Original: Other deep learning researchers have proposed multiple memory systems for continual learning.
Revised: Other deep learning researchers have proposed multiple memory systems for continual learning.

Original: In Parisi et al.
Revised: In Parisi et al.

Original: (2018), the learning algorithm is heuristic and representations come from pretrained networks.
Revised: (2018), the learning algorithm is heuristic and representations come from pretrained networks.

Original: In
Revised: In

Original: Kemker & Kanan (2018), a prototype memory is used for recalling recent examples and rehearsal from a generative model allows this knowledge to be integrated and distilled into a long-term memory.
Revised: Kemker & Kanan (2018), a prototype memory is used for recalling recent examples and rehearsal from a generative model allows this knowledge to be integrated and distilled into a long-term memory.

Original: 3. Online Contextualized Few-Shot Learning
Revised: 3. Online Contextualized Few-Shot Learning

Original: In this section, we introduce our new online contextualized few-shot learning (OC-FSL) setup in the form of a sequential decision problem, and then introduce our new benchmark datasets.
Revised: In this section, we introduce our new online contextualized few-shot learning (OC-FSL) setup in the form of a sequential decision problem, and then introduce our new benchmark datasets.

Original: Continual few-shot classiﬁcation as a sequential decision problem: In traditional few-shot learning, an episode is constructed by a support set S and a query set Q .
Revised: Continual few-shot classiﬁcation as a sequential decision problem: In traditional few-shot learning, an episode is constructed by a support set S and a query set Q .

Original: A few-shot learner f is expected to predict the class of each example in the query set x Q based on the support set information: ˆ y Q = f
Revised: A few-shot learner f is expected to predict the class of each example in the query set x Q based on the support set information: ˆ y Q = f

Original: ( x Q ; ( x S 1 , y S 1 ) , . . .
Revised: ( x Q ; ( x S 1 , y S 1 ) , . . .

Original: , ( x SN , y SN )) .
Revised: , ( x SN , y SN )) .

Original: This setup is not a natural ﬁt for continual learning, since it is unclear when to insert a query set into the sequence.
Revised: This setup is not a natural ﬁt for continual learning, since it is unclear when to insert a query set into the sequence.

Original: Inspired by the online learning literature, we can convert continual few-shot learning into a sequential decision problem, where every input example is also part of the evaluation: ˆ y t = f
Revised: Inspired by the online learning literature, we can convert continual few-shot learning into a sequential decision problem, where every input example is also part of the evaluation: ˆ y t = f

Original: ( x t ; ( x 1 , ˜ y 1 ) , . . .
Revised: ( x t ; ( x 1 , ˜ y 1 ) , . . .

Original: , ( x t − 1 , ˜ y t − 1 )) , for t = 1 . . .
Revised: , ( x t − 1 , ˜ y t − 1 )) , for t = 1 . . .

Original: T , where ˜ y here further allows that the sequence of inputs to be semi-supervised: ˜
Revised: T , where ˜ y here further allows that the sequence of inputs to be semi-supervised: ˜

Original: y equals y t if labeled, or otherwise − 1 .
Revised: y equals y t if labeled, or otherwise − 1 .

Original: The setup in Santoro et al.
Revised: The setup in Santoro et al.

Original: (2016) and Kaiser et al.
Revised: (2016) and Kaiser et al.

Original: (2017) is similar; they train RNNs using such a temporal sequence as input.
Revised: (2017) is similar; they train RNNs using such a temporal sequence as input.

Original: However, their evaluation relies on another “query set” at the end of the sequence.
Revised: However, their evaluation relies on another “query set” at the end of the sequence.

Original: We instead evaluate online while learning.
Revised: We instead evaluate online while learning.

Original: Figure 1-A illustrates these features, using an example of an input sequence where an agent is learning about new objects in a kitchen.
Revised: Figure 1-A illustrates these features, using an example of an input sequence where an agent is learning about new objects in a kitchen.

Original: The model is rewarded when it correctly predicts a known class or when it indicates that the item has yet to be assigned a label.
Revised: The model is rewarded when it correctly predicts a known class or when it indicates that the item has yet to be assigned a label.

Original: Contextualized environments: Typical continual learning consists of a sequence of tasks, and models are trained sequentially for each task.
Revised: Contextualized environments: Typical continual learning consists of a sequence of tasks, and models are trained sequentially for each task.

Original: This feature is also preserved in many incremental learning settings (Rebufﬁ et al., 2017).
Revised: This feature is also preserved in many incremental learning settings (Rebufﬁ et al., 2017).

Original: For instance, the split-CIFAR task divides CIFAR-100 into 10 learning environments, each with 10 classes, presented sequentially.
Revised: For instance, the split-CIFAR task divides CIFAR-100 into 10 learning environments, each with 10 classes, presented sequentially.

Original: In our formulation, the sequence returns to earlier environments (see Figure 1-B), which enables assessment of long-term durability of knowledge.
Revised: In our formulation, the sequence returns to earlier environments (see Figure 1-B), which enables assessment of long-term durability of knowledge.

Original: Although the ground-truth environment identity is not provided, we structure the task such that the environment itself provides contextual cues which can constrain the correct class label.
Revised: Although the ground-truth environment identity is not provided, we structure the task such that the environment itself provides contextual cues which can constrain the correct class label.

Original: Spatial cues in the input help distinguish one environment from another.
Revised: Spatial cues in the input help distinguish one environment from another.

Original: Temporal cues are implicit because the sequence tends to switch environments infrequently, allowing recent inputs to be beneﬁcial in guiding the interpretation of the current input.
Revised: Temporal cues are implicit because the sequence tends to switch environments infrequently, allowing recent inputs to be beneﬁcial in guiding the interpretation of the current input.

Original: RoamingOmniglot: The Omniglot dataset (Lake et al., 2015) contains 1623 handwritten characters from 50 different alphabets.
Revised: RoamingOmniglot: The Omniglot dataset (Lake et al., 2015) contains 1623 handwritten characters from 50 different alphabets.

Original: We split the alphabets into 31 for training, 5 for validation, and 13 for testing.
Revised: We split the alphabets into 31 for training, 5 for validation, and 13 for testing.

Original: We augment classes by 90 degree rotations to create 6492 classes in total.
Revised: We augment classes by 90 degree rotations to create 6492 classes in total.

Original: Each contextualized few-shot learning image sequence contains 150 images, drawn from a random sample of 5-10 alphabets, for a total of 50 classes per sequence.
Revised: Each contextualized few-shot learning image sequence contains 150 images, drawn from a random sample of 5-10 alphabets, for a total of 50 classes per sequence.

Original: These classes are randomly assigned to 5 different environments; within an environment, the characters are distributed according to a
Revised: These classes are randomly assigned to 5 different environments; within an environment, the characters are distributed according to a

Original: Chinese restaurant process (Aldous, 1985) to mimic the imbalanced long-tail distribution of naturally occurring objects.
Revised: Chinese restaurant process (Aldous, 1985) to mimic the imbalanced long-tail distribution of naturally occurring objects.

Original: We switch between environments using a Markov switching process; i.e., at each step there is a constant probability of switching to another environment.
Revised: We switch between environments using a Markov switching process; i.e., at each step there is a constant probability of switching to another environment.

Original: An example sequence is shown in Figure 2-A.
Revised: An example sequence is shown in Figure 2-A.

Original: RoamingRooms: As none of the current few-shot learning datasets provides the natural online learning experience that we would like to study, we created our own dataset using simulated indoor environments.
Revised: RoamingRooms: As none of the current few-shot learning datasets provides the natural online learning experience that we would like to study, we created our own dataset using simulated indoor environments.

Original: We formulate this as a few-shot instance learning problem, which could be a use case for a home robot: it needs to quickly recognize and differentiate novel object instances, and large viewpoint variations can make this task challenging (see examples in
Revised: We formulate this as a few-shot instance learning problem, which could be a use case for a home robot: it needs to quickly recognize and differentiate novel object instances, and large viewpoint variations can make this task challenging (see examples in

Original: Figure 2-B).
Revised: Figure 2-B).

Original: There are over 7,000 unique instance classes in the dataset, making it suitable to meta-learning approaches.
Revised: There are over 7,000 unique instance classes in the dataset, making it suitable to meta-learning approaches.

Original: Our dataset is derived from the Matterport3D dataset (Chang et al., 2017), which has 90 indoor worlds captured using panoramic depth cameras.
Revised: Our dataset is derived from the Matterport3D dataset (Chang et al., 2017), which has 90 indoor worlds captured using panoramic depth cameras.

Original: We split these into 60 worlds for training, 10 for validation and 20 for testing.
Revised: We split these into 60 worlds for training, 10 for validation and 20 for testing.

Original: We use MatterSim (Anderson et al., 2018) to load the simulated world and collect camera images and use HabitatSim (Savva et al., 2019) to simulate 3D mesh and align instance segmentation labels onto 2D image space.
Revised: We use MatterSim (Anderson et al., 2018) to load the simulated world and collect camera images and use HabitatSim (Savva et al., 2019) to simulate 3D mesh and align instance segmentation labels onto 2D image space.

Original: We created a random walking agent to collect the virtual visual experience.
Revised: We created a random walking agent to collect the virtual visual experience.

Original: For each viewpoint in the random walk, we randomly sample one object from the image sensor and highlight it with the available instance segmentation, forming an input frame .
Revised: For each viewpoint in the random walk, we randomly sample one object from the image sensor and highlight it with the available instance segmentation, forming an input frame .

Original: Each viewpoint provides environmental context—the other objects present in the room with the highlighted object.
Revised: Each viewpoint provides environmental context—the other objects present in the room with the highlighted object.

Original: Figure 3-A shows the object instance distribution.
Revised: Figure 3-A shows the object instance distribution.

Original: We see strong temporal correlation, as 30% of the time the same instance appears in the next frame (Figure 3-B), but there is also a signiﬁcant proportion of revisits.
Revised: We see strong temporal correlation, as 30% of the time the same instance appears in the next frame (Figure 3-B), but there is also a signiﬁcant proportion of revisits.

Original: On average, there are three different viewpoints per 100-image sequence (Figure 3-C).
Revised: On average, there are three different viewpoints per 100-image sequence (Figure 3-C).

Original: 4. Contextual Prototypical Memory Networks
Revised: 4. Contextual Prototypical Memory Networks

Original: In the online contextualized few-shot learning setup, the fewshot learner can potentially improve by modeling the temporal context.
Revised: In the online contextualized few-shot learning setup, the fewshot learner can potentially improve by modeling the temporal context.

Original: Metric learning approaches (Vinyals et al., 2016;
Revised: Metric learning approaches (Vinyals et al., 2016;

Original: Allen et al., 2019; Snell et al., 2017) typically ignore temporal relations and directly compare the similarity between training and test samples.
Revised: Allen et al., 2019; Snell et al., 2017) typically ignore temporal relations and directly compare the similarity between training and test samples.

Original: Gradient-based approaches (Finn et al., 2017; Javed & White, 2019), on the other hand, have the ability to adapt to new contexts, but they do not naturally handle new and unlabeled examples.
Revised: Gradient-based approaches (Finn et al., 2017; Javed & White, 2019), on the other hand, have the ability to adapt to new contexts, but they do not naturally handle new and unlabeled examples.

Original: We instead propose a simple yet effective approach that augments the classic
Revised: We instead propose a simple yet effective approach that augments the classic

Original: Prototypical Network with a temporal contextual encoder using an RNN, shown in Figure 4.
Revised: Prototypical Network with a temporal contextual encoder using an RNN, shown in Figure 4.

Original: Next, we describe our approach in detail.
Revised: Next, we describe our approach in detail.

Original: Prototype memory: We start describing our model with the prototype memory, which is an online version of the
Revised: Prototype memory: We start describing our model with the prototype memory, which is an online version of the

Original: Prototypical Network (or ProtoNet )
Revised: Prototypical Network (or ProtoNet )

Original: (Snell et al., 2017).
Revised: (Snell et al., 2017).

Original: ProtoNet can be viewed as a knowledge base memory, where
Revised: ProtoNet can be viewed as a knowledge base memory, where

Original: [Figure]
Revised: [Figure]

Original: Table

Original: [Figure] [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: p i c t u r e c h a i r li g h t i n g c u s h
Revised: p i c t u r e c h a i r li g h t i n g c u s h

Original: i o n t a b l e p l a n t c h e s t
Revised: i o n t a b l e p l a n t c h e s t

Original: o f
Revised: o f

Original: d r a w e r s t o w e l
Revised: d r a w e r s t o w e l

Original: s
Revised: s

Original: o
Revised: o

Original: f a b e d a pp li a n c e s s t oo l t v m o n i t
Revised: f a b e d a pp li a n c e s s t oo l t v m o n i t

Original: o r c l o t
Revised: o r c l o t

Original: h e s t o il e t f
Revised: h e s t o il e t f

Original: i r e p l a c e f u r n i t u r e b a t h t u b g y m e q u
Revised: i r e p l a c e f u r n i t u r e b a t h t u b g y m e q u

Original: i p m e n t b li n d s b o a r d p a n e l
Revised: i p m e n t b li n d s b o a r d p a n e l

Original: [Figure]
Revised: [Figure]

Original: each object class k is represented by a prototype vector p [ k ] , computed as the mean vector of all the support instances of the class in a sequence.
Revised: class k is represented by a prototype vector p [ k ] , computed as the mean vector of all the support instances of the class in a sequence.

Original: It can also be applied to our task of online few-shot learning naturally, with some modiﬁcations.
Revised: It can also be applied to our task of online few-shot learning naturally, with some modiﬁcations.

Original: Suppose that at time-step t we have already stored a few classes in the memory, each represented by their current prototype p t [ k ] , and we would like to query the memory using the current input feature h t .
Revised: Suppose that at time-step t we have already stored a few classes in the memory, each represented by their current prototype p t [ k ] , and we would like to query the memory using the current input feature h t .

Original: We model our prototype memory as
Revised: We model our prototype memory as

Original: [Equation]
Revised: [Equation]

Original: where (cid:107)·(cid:107) 2 M t is the squared Mahalanobis distance for some to-be-speciﬁed, time-varying hyperparameter matrix M t .
Revised: where (cid:107)·(cid:107) 2 M t is the squared Mahalanobis distance for some to-be-speciﬁed, time-varying hyperparameter matrix M t .

Original: To predict whether an example is of a new class, we can use a separate novelty output ˆ u rt with sigmoid activation, similar to the approach introduced in (Ren et al., 2018), where β rt and γ rt are yet-to-be-speciﬁed thresholding hyperparameters (the superscript r stands for read):
Revised: To predict whether an example is of a new class, we can use a separate novelty output ˆ u rt with sigmoid activation, similar to the approach introduced in (Ren et al., 2018), where β rt and γ rt are yet-to-be-speciﬁed thresholding hyperparameters (the superscript r stands for read):

Original: [Equation]
Revised: [Equation]

Original: Memory consolidation with online prototype averaging:
Revised: Memory consolidation with online prototype averaging:

Original: Traditionally, ProtoNet uses the average representation of
Revised: Traditionally, ProtoNet uses the average representation of

Original: [Figure]
Revised: [Figure]

Original: a class across all support examples.
Revised: a class across all support examples.

Original: Here, we must be able to adapt the prototype memory incrementally at each step.
Revised: Here, we must be able to adapt the prototype memory incrementally at each step.

Original: Fortunately, we are able to recover the same prototypes as a regular ProtoNet computes ofﬂine by using online averaging.
Revised: Fortunately, we are able to recover the same prototypes as a regular ProtoNet computes ofﬂine by using online averaging.

Original: For each prototype k , we store a count scalar c [ k ] t to indicate the number of examples that have been added to this prototype up to time t .
Revised: For each prototype k , we store a count scalar c [ k ] t to indicate the number of examples that have been added to this prototype up to time t .

Original: When the current example is unlabeled, y t is encoded as − 1 , and the model’s own prediction ˆ y wt will determine which prototype to update; in this case, the model must also determine a strength of belief,
Revised: When the current example is unlabeled, y t is encoded as − 1 , and the model’s own prediction ˆ y wt will determine which prototype to update; in this case, the model must also determine a strength of belief,

Original: ˆ u wt , that the current unlabeled example should be treated as a new class.
Revised: ˆ u wt , that the current unlabeled example should be treated as a new class.

Original: Given ˆ u wt and y t , the model can then update a prototype:
Revised: Given ˆ u wt and y t , the model can then update a prototype:

Original: [Equation]
Revised: [Equation]

Original: As-yet-unspeciﬁed hyperparameters β wt and γ wt are required.
Revised: As-yet-unspeciﬁed hyperparameters β wt and γ wt are required.

Original: (The superscript w in β wt and γ wt is for write.)
Revised: (The superscript w in β wt and γ wt is for write.)

Original: These parameters for the online-updating novelty output ˆ u wt are distinct from β rt and γ rt in Equation 2.
Revised: These parameters for the online-updating novelty output ˆ u wt are distinct from β rt and γ rt in Equation 2.

Original: The intuition is that for “selfteaching” to work, the model potentially needs to be more conservative in creating new classes (avoiding corruption of prototypes) than in predicting an input as being a new class.
Revised: The intuition is that for “selfteaching” to work, the model potentially needs to be more conservative in creating new classes (avoiding corruption of prototypes) than in predicting an input as being a new class.

Original: Contextual RNN: Instead of directly using the features from the CNN h CNN t as input features to the prototype memory, we would also like to use contextual information from the recent past.
Revised: Contextual RNN: Instead of directly using the features from the CNN h CNN t as input features to the prototype memory, we would also like to use contextual information from the recent past.

Original: Above we introduced threshold hyperparameters β rt , γ rt , β wt , γ wt as well as the metric parameter M t .
Revised: Above we introduced threshold hyperparameters β rt , γ rt , β wt , γ wt as well as the metric parameter M t .

Original: We let the contextual RNN output these additional control parameters, so that the unknown thresholds and metric function can adapt based on the information in the context.
Revised: We let the contextual RNN output these additional control parameters, so that the unknown thresholds and metric function can adapt based on the information in the context.

Original: The
Revised: The

Original: RNN produces the context vector h RNN t and other control parameters conditioned on h CNN t : 
Revised: RNN produces the context vector h RNN t and other control parameters conditioned on h CNN t : (cid:2) z t [Equation] h RNN t

Original: [Equation]
Revised: [Equation]

Original: where z t is the recurrent state of the RNN, and m t is the diagonal vector of M t .
Revised: where z t is the recurrent state of the RNN, and m t is the diagonal vector of M t .

Original: The context, h RNN t , serves as an additive bias on the state vector used for FSL: h t = h CNN t + h RNN t .
Revised: The context, h RNN t , serves as an additive bias on the state vector used for FSL: h t = h CNN t + h RNN t .

Original: Loss function: The loss function is computed after an entire sequence ends and all network parameters are learned end-to-end.
Revised: Loss function: The loss function is computed after an entire sequence ends and all network parameters are learned end-to-end.

Original: The loss is composed of two parts.
Revised: The loss is composed of two parts.

Original: The ﬁrst is binary cross-entropy (BCE), for telling whether each example has been assigned a label or not, i.e., prediction of new classes.
Revised: The ﬁrst is binary cross-entropy (BCE), for telling whether each example has been assigned a label or not, i.e., prediction of new classes.

Original: Second we use a multi-class cross-entropy for classifying among the known ones.
Revised: Second we use a multi-class cross-entropy for classifying among the known ones.

Original: We can write down the overall loss function as follows:
Revised: We can write down the overall loss function as follows:

Original: [Equation]
Revised: [Equation]

Original: 5. Experiments
Revised: 5. Experiments

Original: In this section, we show experimental results for our online contextualized few-shot learning paradigm, using RoamingOmniglot and RoamingRooms (see Sec. 3) to evaluate our model, CPM, and other state-of-the-art methods.
Revised: In this section, we show experimental results for our online contextualized few-shot learning paradigm, using RoamingOmniglot and RoamingRooms (see Sec. 3) to evaluate our model, CPM, and other state-of-the-art methods.

Original: For Omniglot, we apply an 8 × 8 CutOut (Devries & Taylor, 2017) to each image to make the task more challenging.
Revised: For Omniglot, we apply an 8 × 8 CutOut (Devries & Taylor, 2017) to each image to make the task more challenging.

Original: Implementation details: For the RoamingOmniglot experiment we used the common 4-layer CNN for few-shot learning with 64 channels in each layer.
Revised: Implementation details: For the RoamingOmniglot experiment we used the common 4-layer CNN for few-shot learning with 64 channels in each layer.

Original: For the RoamingRooms experiment we resize the input to 120 × 160 and we use the ResNet-12 architecture (Oreshkin et al., 2018) with {32,64,128,256} channels per block.
Revised: For the RoamingRooms experiment we resize the input to 120 × 160 and we use the ResNet-12 architecture (Oreshkin et al., 2018) with {32,64,128,256} channels per block.

Original: To represent the feature of the input image with an attention mask, we concatenate the global average pooled feature with the attention ROI feature, resulting in a 512d feature vector.
Revised: To represent the feature of the input image with an attention mask, we concatenate the global average pooled feature with the attention ROI feature, resulting in a 512d feature vector.

Original: For the contextual
Revised: For the contextual

Original: RNN, in both experiments we used an LSTM (Hochreiter &
Revised: RNN, in both experiments we used an LSTM (Hochreiter &

Original: Schmidhuber, 1997) with a 256d hidden state.
Revised: Schmidhuber, 1997) with a 256d hidden state.

Original: We use the Adam optimizer (Kingma & Ba, 2015) with initial learning rate 1e-3 for all experiments.
Revised: We use the Adam optimizer (Kingma & Ba, 2015) with initial learning rate 1e-3 for all experiments.

Original: For Omniglot we train the network for 40k steps with batch size of 32 with maximum sequence length 150 across 2 GPUs and learning rate decay by 0.1 × at 20k and 30k steps.
Revised: For Omniglot we train the network for 40k steps with batch size of 32 with maximum sequence length 150 across 2 GPUs and learning rate decay by 0.1 × at 20k and 30k steps.

Original: For Matterport 3D we train for 20k steps with batch size 8 with maximum sequence length 100 across 4 GPUs and learning rate decay by 0.1 × at 8k and 16k steps.
Revised: For Matterport 3D we train for 20k steps with batch size 8 with maximum sequence length 100 across 4 GPUs and learning rate decay by 0.1 × at 8k and 16k steps.

Original: We use BCE coeffcient λ
Revised: We use BCE coeffcient λ

Original: =for all experiments.
Revised: =for all experiments.

Original: In semisupervise experiments, around 30% examples are labeled.
Revised: In semisupervise experiments, around 30% examples are labeled.

Original: Evaluation metrics:
Revised: Evaluation metrics:

Original: In order to compute a single number that characterizes the learning ability over sequences, we propose to use average precision (AP) to combine the prediction on old and new classes.
Revised: In order to compute a single number that characterizes the learning ability over sequences, we propose to use average precision (AP) to combine the prediction on old and new classes.

Original: Concretely, all predictions are sorted by their old vs. new scores, and we compute
Revised: Concretely, all predictions are sorted by their old vs. new scores, and we compute

Original: AP using the area under the precision-recall curve.
Revised: AP using the area under the precision-recall curve.

Original: True positive is deﬁned as the correct prediction of a multi-class classiﬁcation among known classes.
Revised: True positive is deﬁned as the correct prediction of a multi-class classiﬁcation among known classes.

Original: We also compute the “ N - shot” accuracy; i.e., the average accuracy after seeing the label N times in the sequence.
Revised: We also compute the “ N -shot” accuracy; i.e., the average accuracy after seeing the label N times in the sequence.

Original: Note that these accuracy scores only reﬂect the performance on known class predictions.
Revised: Note that these accuracy scores only reﬂect the performance on known class predictions.

Original: All numbers are reported with an average over 2, sequences and for N -shot accuracy standard error is also included.
Revised: All numbers are reported with an average over 2, sequences and for N -shot accuracy standard error is also included.

Original: Comparisons: To evaluate the merits of our proposed model, we implement classic few-shot learning and online meta-learning methods.
Revised: Comparisons: To evaluate the merits of our proposed model, we implement classic few-shot learning and online meta-learning methods.

Original: • OML (Javed & White, 2019):
Revised: • OML (Javed & White, 2019):

Original: This is an online version of MAML (Finn et al., 2017).
Revised: This is an online version of MAML (Finn et al., 2017).

Original: It performs one gradient descent step for each labled input image, and slow weights are learned via backpropagation through time.
Revised: It performs one gradient descent step for each labled input image, and slow weights are learned via backpropagation through time.

Original: [Table]
Revised: [Table]

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: • LSTM (Hochreiter & Schmidhuber,1997) & DNC (Graves et al., 2016):
Revised: • LSTM (Hochreiter & Schmidhuber, DNC (Graves et al., 2016):

Original: We include RNN methods for comparison as well.
Revised: We include RNN methods for comparison as well.

Original: Differentiable neural computer (DNC) is an improved version of memory augmented neural network (MANN) (Santoro et al., 2016).
Revised: Differentiable neural computer (DNC) is an improved version of memory augmented neural network (MANN) (Santoro et al., 2016).

Original: • Online MatchingNet (Vinyals et al., 2016), IMP (Allen et al., 2019) & ProtoNet (Snell et al., 2017): We used the same negative Euclidean distance as the similarity function for these three metric learning based approaches.
Revised: • Online MatchingNet (Vinyals et al., 2016), IMP (Allen et al., 2019) & ProtoNet (Snell et al., 2017): We used the same negative Euclidean distance as the similarity function for these three metric learning based approaches.

Original: In particular, MatchingNet stores all examples and performs nearest neighbor matching, which can be memory inefﬁcient.
Revised: In particular, MatchingNet stores all examples and performs nearest neighbor matching, which can be memory inefﬁcient.

Original: Note that Online ProtoNet is a variant of our method without the contextual RNN.
Revised: Note that Online ProtoNet is a variant of our method without the contextual RNN.

Original: Main results: Our main results are shown in Table 2 and 3, including both supervised and semi-supervised settings.
Revised: Main results: Our main results are shown in Table 2 and 3, including both supervised and semi-supervised settings.

Original: Our approach achieves the best performance on AP con-
Revised: Our approach achieves the best performance on AP con-

Original: [Table]
Revised: [Table]

Original: sistently across all settings.
Revised: sistently across all settings.

Original: Online ProtoNet is a direct comparison without our contextual RNN and it is clear that
Revised: Online ProtoNet is a direct comparison without our contextual RNN and it is clear that

Original: CPM is signiﬁcantly better.
Revised: CPM is signiﬁcantly better.

Original: Our method is slightly worse than Online MatchingNet in terms of 3-shot accuracy on the
Revised: Our method is slightly worse than Online MatchingNet in terms of 3-shot accuracy on the

Original: RoamingRooms semisupervised benchmark.
Revised: RoamingRooms semisupervised benchmark.

Original: This can be explained by the fact that MatchingNet stores all past seen examples, whereas CPM only stores one prototype per class.
Revised: This can be explained by the fact that MatchingNet stores all past seen examples, whereas CPM only stores one prototype per class.

Original: Per timestep accuracy is plotted in Figure 5, and the decaying accuracy is due to the increasing number of classes over time.
Revised: Per timestep accuracy is plotted in Figure 5, and the decaying accuracy is due to the increasing number of classes over time.

Original: In RoamingOmniglot, CPM is able to closely match or even sometimes surpass the ofﬂine classiﬁer, which re-trains at each step, which uses all prior images in a sequence.
Revised: In RoamingOmniglot, CPM is able to closely match or even sometimes surpass the ofﬂine classiﬁer, which re-trains at each step, which uses all prior images in a sequence.

Original: This is reasonable as our model is able to leverage information from the current context.
Revised: This is reasonable as our model is able to leverage information from the current context.

Original: Effect of spatiotemporal context: To answer the question whether the gain in performance is due to spatiotemporal reasoning, we conduct the following experiment compar-
Revised: Effect of spatiotemporal context: To answer the question whether the gain in performance is due to spatiotemporal reasoning, we conduct the following experiment compar-

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: ing CPM with online ProtoNet.
Revised: ing CPM with online ProtoNet.

Original: We allow the CNN to have the ability to recognize the context in RoamingOmniglot by adding a texture background image using the Kylberg texture dataset (Kylberg, 2011)
Revised: We allow the CNN to have the ability to recognize the context in RoamingOmniglot by adding a texture background image using the Kylberg texture dataset (Kylberg, 2011)

Original: (see Figure 6 left).
Revised: (see Figure 6 left).

Original: As a control, we can also destroy the temporal context by shufﬂing all the images in a sequence.
Revised: As a control, we can also destroy the temporal context by shufﬂing all the images in a sequence.

Original: We train four different models on dataset controls with or without the presence of spatial or temporal context, and results are shown in Figure 6.
Revised: We train four different models on dataset controls with or without the presence of spatial or temporal context, and results are shown in Figure 6.

Original: First, both online ProtoNet and CPM beneﬁt from the inclusion of a spatial context.
Revised: First, both online ProtoNet and CPM beneﬁt from the inclusion of a spatial context.

Original: This is understandable as the CNN has the ability to learn spatial cues, which re-conﬁrms our main hypothesis that successful inference of the current context is beneﬁcial to novel object recognition.
Revised: This is understandable as the CNN has the ability to learn spatial cues, which re-conﬁrms our main hypothesis that successful inference of the current context is beneﬁcial to novel object recognition.

Original: Second, only our
Revised: Second, only our

Original: CPM model beneﬁts from the presence of temporal context, and it receives distinct gains from spatial and temporal contexts.
Revised: CPM model beneﬁts from the presence of temporal context, and it receives distinct gains from spatial and temporal contexts.

Original: Ablation studies: We ablate each individual module we introduce.
Revised: Ablation studies: We ablate each individual module we introduce.

Original: Results are shown in Tables 4 and 5.
Revised: Results are shown in Tables 4 and 5.

Original: Tablestudies different ways we use the RNN, including the context vector h RNN , the predicted threshold parameters β ∗ t , γ ∗ t , and the predicted metric scaling vector m t .
Revised: Tablestudies different ways we use the RNN, including the context vector h RNN , the predicted threshold parameters β ∗ t , γ ∗ t , and the predicted metric scaling vector m t .

Original: Table 5 studies various ways to learn from unlabeled examples, where we separately disable the RNN update, prototype update, and distinct write-threshold parameters β wt , γ wt (vs. using readthreshold parameters).
Revised: Table 5 studies various ways to learn from unlabeled examples, where we separately disable the RNN update, prototype update, and distinct write-threshold parameters β wt , γ wt (vs. using readthreshold parameters).

Original: We verify that each component has a positive impact on the performance.
Revised: We verify that each component has a positive impact on the performance.

Original: 6. Conclusion
Revised: 6. Conclusion

Original: We proposed online contextualized few-shot learning, OCFSL, a paradigm for machine learning that emulates a human or artiﬁcial agent interacting with a physical world.
Revised: We proposed online contextualized few-shot learning, OCFSL, a paradigm for machine learning that emulates a human or artiﬁcial agent interacting with a physical world.

Original: It combines multiple properties to create a challenging learning task: every input must be classiﬁed or ﬂagged as novel, every input is also used for training, semi-supervised learning can potentially improve performance, and the temporal distribution of inputs is non-IID and comes from a generative model in which input and class distributions are conditional on a latent environment with Markovian transition probabilities.
Revised: It combines multiple properties to create a challenging learning task: every input must be classiﬁed or ﬂagged as novel, every input is also used for training, semi-supervised learning can potentially improve performance, and the temporal distribution of inputs is non-IID and comes from a generative model in which input and class distributions are conditional on a latent environment with Markovian transition probabilities.

Original: We proposed the RoamingRooms dataset to simulate an agent wandering within a physical world.
Revised: We proposed the RoamingRooms dataset to simulate an agent wandering within a physical world.

Original: We also proposed a new model, CPM, which uses an RNN to extract spatiotemporal context from the input stream and to provide control settings to a prototype-based FSL model.
Revised: We also proposed a new model, CPM, which uses an RNN to extract spatiotemporal context from the input stream and to provide control settings to a prototype-based FSL model.

Original: In the context of naturalistic domains like RoamingRooms,
Revised: In the context of naturalistic domains like RoamingRooms,

Original: CPM is able to leverage contextual information to attain performance unmatched by other state-of-the-art FSL methods.
Revised: CPM is able to leverage contextual information to attain performance unmatched by other state-of-the-art FSL methods.