Original: W ANDERING W ITHIN A W ORLD : O NLINE C ONTEXTUALIZED F EW -S HOT L EARNING
Revised: W ANDERING W ITHIN A W ORLD : O NLINE C ONTEXTUALIZED F EW -S HOT L EARNING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting.
Revised: We aim to bridge the gap between typical human and machine-learning environments by extending the standard framework of few-shot learning to an online, continual setting.

Original: In this setting, episodes do not have separate training and testing phases, and instead models are evaluated online while learning novel classes.
Revised: In this setting, episodes do not have separate training and testing phases, and instead models are evaluated online while learning novel classes.

Original: As in the real world, where the presence of spatiotemporal context helps us retrieve learned skills in the past, our online few-shot learning setting also features an underlying context that changes throughout time.
Revised: As in the real world, where the presence of spatiotemporal context helps us retrieve learned skills in the past, our online few-shot learning setting also features an underlying context that changes throughout time.

Original: Object classes are correlated within a context and inferring the correct context can lead to better performance.
Revised: Object classes are correlated within a context and inferring the correct context can lead to better performance.

Original: Building upon this setting, we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world.
Revised: Building upon this setting, we propose a new few-shot learning dataset based on large scale indoor imagery that mimics the visual experience of an agent wandering within a world.

Original: Furthermore, we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past.
Revised: Furthermore, we convert popular few-shot learning approaches into online versions and we also propose a new contextual prototypical memory model that can make use of spatiotemporal contextual information from the recent past.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: In machine learning, many paradigms exist for training and evaluating models: standard train-thenevaluate, few-shot learning, incremental learning, continual learning, and so forth.
Revised: In machine learning, many paradigms exist for training and evaluating models: standard train-thenevaluate, few-shot learning, incremental learning, continual learning, and so forth.

Original: None of these paradigms well approximates the naturalistic conditions that humans and artiﬁcial agents encounter as they wander within a physical environment.
Revised: None of these paradigms well approximates the naturalistic conditions that humans and artiﬁcial agents encounter as they wander within a physical environment.

Original: Consider, for example, learning and remembering peoples’ names in the course of daily life.
Revised: Consider, for example, learning and remembering peoples’ names in the course of daily life.

Original: We tend to see people in a given environment—work, home, gym, etc.
Revised: We tend to see people in a given environment—work, home, gym, etc.

Original: We tend to repeatedly revisit those environments, with different environment base rates, nonuniform environment transition probabilities, and nonuniform base rates of encountering a given person in a given environment.
Revised: We tend to repeatedly revisit those environments, with different environment base rates, nonuniform environment transition probabilities, and nonuniform base rates of encountering a given person in a given environment.

Original: We need to recognize when we do not know a person, and we need to learn to recognize them the next time we encounter them.
Revised: We need to recognize when we do not know a person, and we need to learn to recognize them the next time we encounter them.

Original: We are not always provided with a name, but we can learn in a semi-supervised manner.
Revised: We are not always provided with a name, but we can learn in a semi-supervised manner.

Original: And every training trial is itself an evaluation trial as we repeatedly use existing knowledge and acquire new knowledge.
Revised: And every training trial is itself an evaluation trial as we repeatedly use existing knowledge and acquire new knowledge.

Original: In this article, we propose a novel paradigm, online contextualized few-shot learning , that approximates these naturalistic conditions, and we develop deep-learning architectures well suited for this paradigm.
Revised: In this article, we propose a novel paradigm, online contextualized few-shot learning , that approximates these naturalistic conditions, and we develop deep-learning architectures well suited for this paradigm.

Original: In traditional few-shot learning (FSL) (Lake et al., 2015; Vinyals et al., 2016), training is episodic.
Revised: In traditional few-shot learning (FSL) (Lake et al., 2015; Vinyals et al., 2016), training is episodic.

Original: Within an isolated episode, a set of new classes is introduced with a limited number of labeled examples per class—the support set—followed by evaluation on an unlabeled query set.
Revised: Within an isolated episode, a set of new classes is introduced with a limited number of labeled examples per class—the support set—followed by evaluation on an unlabeled query set.

Original: While this setup has inspired the development of a multitude of meta-learning algorithms which can be trained to rapidly learn novel classes with a few labeled examples, the algorithms are focused solely on the few classes introduced in the current episode; the classes learned are not carried over to future episodes.
Revised: While this setup has inspired the development of a multitude of meta-learning algorithms which can be trained to rapidly learn novel classes with a few labeled examples, the algorithms are focused solely on the few classes introduced in the current episode; the classes learned are not carried over to future episodes.

Original: Although incremental learning and continual learning methods (Rebufﬁ et al., 2017; Hou et al., 2019) address the case where classes are carried over, the episodic construction of these frameworks seems artiﬁcial: in our daily lives, we do not learn new objects by grouping them with ﬁve other new objects, process them together, and then move on.
Revised: Although incremental learning and continual learning methods (Rebufﬁ et al., 2017; Hou et al., 2019) address the case where classes are carried over, the episodic construction of these frameworks seems artiﬁcial: in our daily lives, we do not learn new objects by grouping them with ﬁve other new objects, process them together, and then move on.

Original: To break the rigid, artiﬁcial structure of continual and few-shot learning, we propose a new continual few-shot learning setting where environments are revisited and the total number of novel object classes increases over time.

Original: Crucially, model evaluation happens on each trial, very much like the setup in online learning.

Original: When encountering a new class, the learning algorithm is expected to indicate that the class is “new,” and it is then expected to recognize subsequent instances of the class once a label has been provided.

Original: [Figure]
Revised: [Figure]

Original: T=
Revised: T=

Original: T= …
Revised: T= …

Original: T=
Revised: T=

Original: B) Context switching
Revised: B) Context switching

Original: Agent
Revised: Agent

Original: Env 1: Kitchen
Revised: Env 1: Kitchen

Original: Env 3: Office
Revised: Env 3: Office

Original: Env 2: Living room
Revised: Env 2: Living room

Original: Env 3: Office
Revised: Env 3: Office

Original: Env 1: Kitchen v
Revised: Env 1: Kitchen v

Original: World
Revised: World

Original: Cls v
Revised: Cls v

Original: Evaluation
Revised: Evaluation

Original: New
Revised: New

Original: Cls
Revised: Cls

Original: Cls Cls
Revised: Cls Cls

Original: T=
Revised: T=

Original: New New
Revised: New New

Original: Cls t= t= t= t=
Revised: Cls t= t= t= t=

Original: Env 1: Kitchen
Revised: Env 1: Kitchen

Revised: To break the rigid, artiﬁcial structure of continual and few-shot learning, we propose a new continual few-shot learning setting where environments are revisited and the total number of novel object classes increases over time.

Revised: Crucially, model evaluation happens on each trial, very much like the setup in online learning.

Revised: When encountering a new class, the learning algorithm is expected to indicate that the class is “new

Revised: ” and it is then expected to recognize subsequent instances of the class once a label has been provided.

Original: When learning continually in such a dynamic environment contextual information can guide learning and remembering.
Revised: When learning continually in such a dynamic environment, contextual information can guide learning and remembering.

Original: Any structured sequence provides temporal context : the instances encountered recently are predictive of instances to be encountered next.
Revised: Any structured sequence provides temporal context : the instances encountered recently are predictive of instances to be encountered next.

Original: In natural environments, spatial context — information in the current input weakly correlated with the occurrence of a particular class—can be beneﬁcial for retrieval as well.
Revised: In natural environments, spatial context — information in the current input weakly correlated with the occurrence of a particular class—can be beneﬁcial for retrieval as well.

Original: For example, we tend to see our boss in an ofﬁce setting, not in a bedroom setting.
Revised: For example, we tend to see our boss in an ofﬁce setting, not in a bedroom setting.

Original: Human memory retrieval beneﬁts from both spatial and temporal context (Howard, 2017; Kahana, 2012).
Revised: Human memory retrieval beneﬁts from both spatial and temporal context (Howard, 2017; Kahana, 2012).

Original: In our online few-shot learning setting, we provide spatial context in the presentation of each instance and temporal structure to sequences, enabling an agent to learn from both spatial and temporal context.
Revised: In our online few-shot learning setting, we provide spatial context in the presentation of each instance and temporal structure to sequences, enabling an agent to learn from both spatial and temporal context.

Original: Besides developing and experimenting on a toy benchmark using handwritten characters (Lake et al., 2015), we also propose a new large-scale benchmark for online contextualized few-shot learning derived from indoor panoramic imagery (Chang et al., 2017).
Revised: Besides developing and experimenting on a toy benchmark using handwritten characters (Lake et al., 2015), we also propose a new large-scale benchmark for online contextualized few-shot learning derived from indoor panoramic imagery (Chang et al., 2017).

Original: In the toy benchmark, temporal context can be deﬁned by the co-occurrence of character classes.
Revised: In the toy benchmark, temporal context can be deﬁned by the co-occurrence of character classes.

Original: In the indoor environment, the context—temporal and spatial—is a natural by-product as the agent wandering in between different rooms.
Revised: In the indoor environment, the context—temporal and spatial—is a natural by-product as the agent wandering in between different rooms.

Original: We propose a model that can exploit contextual information, called contextual prototypical memory ( CPM ), which incorporates an RNN to encode contextual information and a separate prototype memory to remember previously learned classes (see Figure 4).
Revised: We propose a model that can exploit contextual information, called contextual prototypical memory ( CPM ), which incorporates an RNN to encode contextual information and a separate prototype memory to remember previously learned classes (see Figure 4).

Original: This model obtains signiﬁcant gains on few-shot classiﬁcation performance compared to models that do not retain a memory of the recent past.
Revised: This model obtains signiﬁcant gains on few-shot classiﬁcation performance compared to models that do not retain a memory of the recent past.

Original: We compare to classic few-shot algorithms extended to an online setting, and CPM consistently achieves the best performance.
Revised: We compare to classic few-shot algorithms extended to an online setting, and CPM consistently achieves the best performance.

Original: The main contributions of this paper are as follows.
Revised: The main contributions of this paper are as follows.

Original: First, we deﬁne an online contextualized few-shot learning (OC-FSL) setting to mimic naturalistic human learning.
Revised: First, we deﬁne an online contextualized few-shot learning (OC-FSL) setting to mimic naturalistic human learning.

Original: Second, we build two datasets.
Revised: Second, we build three datasets:

Revised: 1) RoamingOmniglot is based on handwritten characters from Omniglot (

Original: The
Revised: Lake

Original: RoamingOmniglot dataset is based on handwritten characters from Omniglot (Lake et al., 2015) and the RoamingRoomsdataset is our new few-shot learning dataset based on indoor imagery (Chang et al., 2017), which resembles the visual experience of a wandering agent.
Revised: et al. ; 2) RoamingImageNet is based on images from ImageNet (Russakovsky et al. ; and 3) RoamingRooms is our new few-shot learning dataset based on indoor imagery (Chang et al., 2017), which resembles the visual experience of a wandering agent.

Original: Third, we benchmark classic FSL methods and also explore our CPM model, which combines the strengths of RNNs for modeling temporal context and Prototypical Networks
Revised: Third, we benchmark classic FSL methods and also explore our CPM model, which combines the strengths of RNNs for modeling temporal context and Prototypical Networks

Original: (Snell et al., 2017) for memory consolidation and rapid learning.
Revised: (Snell et al. , 2017) for memory consolidation and rapid learning.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: In this section, we brieﬂy review paradigms that have been used for few-shot learning (FSL) and continual learning (CL). 
Revised: In this section, we brieﬂy review paradigms that have been used for few-shot learning (FSL) and continual learning (CL). [Figure]

Original: Few-shot learning: FSL (Lake et al., 2015; Li et al., 2007;
Revised: Few-shot learning: FSL (Lake et al., 2015; Li et al., 2007;

Original: Koch et al., 2015; Vinyals et al., 2016) considers learning new tasks with few labeled examples.
Revised: Koch et al., 2015; Vinyals et al., 2016) considers learning new tasks with few labeled examples.

Original: FSL models can be categorized as based on: metric learning (Vinyals et al., 2016;
Revised: FSL models can be categorized as based on: metric learning (Vinyals et al., 2016;

Original: Snell et al., 2017), memory (Santoro et al., 2016), and gradient adaptation (Finn et al., 2017; Li et al., 2017).
Revised: Snell et al., 2017), memory (Santoro et al., 2016), and gradient adaptation (Finn et al., 2017; Li et al., 2017).

Original: The model we propose, CPM, lies on the boundary between these approaches, as we use an RNN to model the temporal context but we also use metric-learning mechanisms and objectives to train.[Figure]
Revised: The model we propose, CPM, lies on the boundary between these approaches, as we use an RNN to model the temporal context but we also use metric-learning mechanisms and objectives to train.

Original: Several previous efforts have aimed to extend few- learning to incorporate  natural constraints.
Revised: Several previous efforts have aimed to extend few-shot learning to incorporate more natural constraints.

Original: One such example is semi-supervised FSL (Ren et al., 2018), where models also learn from a pool of unlabeled examples.
Revised: One such example is semi-supervised FSL (Ren et al., 2018), where models also learn from a pool of unlabeled examples.

Original: While traditional FSL only tests the learner on novel classes, incremental
Revised: While traditional FSL only tests the learner on novel classes, incremental

Original: FSL (Gidaris & Komodakis, 2018;
Revised: FSL (Gidaris & Komodakis, 2018;

Original: Ren et al., 2019) tests on novel classes together with a set of base classes.
Revised: Ren et al., 2019) tests on novel classes together with a set of base classes.

Original: These approaches, however, have not explored how to iteratively add new classes.
Revised: These approaches, however, have not explored how to iteratively add new classes.

Original: In concurrent work, Antoniou et al.
Revised: In concurrent work, Antoniou et al.

Original: (2020) extend FSL to a continual setting based on image sequences, each of which is divided into stages with a ﬁxed number of examples per class followed by a query set.
Revised: (2020) extend FSL to a continual setting based on image sequences, each of which is divided into stages with a ﬁxed number of examples per class followed by a query set.

Original: focuses on more ﬂexible and faster adaptation since the models are evaluated online, and context is a soft constraint instead of a hard separation of tasks.
Revised: focuses on more ﬂexible and faster adaptation since the models are evaluated online, and context is a soft constraint instead of a hard separation of tasks.

Original: Moreover, new classes need to be identiﬁed as part of the sequence, crucial to any learner’s incremental acquisition of knowledge.
Revised: Moreover, new classes need to be identiﬁed as part of the sequence, crucial to any learner’s incremental acquisition of knowledge.

Original: Continual learning: Continual (or lifelong) learning is a parallel line of research that aims to handle a sequence of dynamic tasks (Kirkpatrick et al., 2017; Li & Hoiem, 2018; Lopez-Paz &
Revised: Continual learning: Continual (or lifelong) learning is a parallel line of research that aims to handle a sequence of dynamic tasks (Kirkpatrick et al., 2017; Li & Hoiem, 2018; Lopez-Paz &

Original: Ranzato, 2017; Yoon et al., 2018).
Revised: Ranzato, 2017; Yoon et al., 2018).

Original: A key challenge here is catastrophic forgetting (McCloskey & Cohen, 1989), where the model “forgets” a task that has been learned in the past.
Revised: A key challenge here is catastrophic forgetting (McCloskey & Cohen, 1989), where the model “forgets” a task that has been learned in the past.

Original: Incremental learning (Rebufﬁ et al., 2017; Castro et al., 2018; Wu et al., 2019;
Revised: Incremental learning (Rebufﬁ et al., 2017; Castro et al., 2018; Wu et al., 2019;

Original: Hou et al., 2019;
Revised: Hou et al., 2019;

Original: He et al., 2020) is a form of continual learning, where each task is an incremental batch of several new classes.
Revised: He et al., 2020) is a form of continual learning, where each task is an incremental batch of several new classes.

Original: This assumption that novel classes always come in batches seems unnatural.
Revised: This assumption that novel classes always come in batches seems unnatural.

Original: Traditionally, continual learning is studied with tasks such as permuted MNIST (Lecun et al., 1998) or split-CIFAR
Revised: Traditionally, continual learning is studied with tasks such as permuted MNIST (Lecun et al., 1998) or split-CIFAR

Original: (Krizhevsky, 2009).
Revised: (Krizhevsky, 2009).

Original: Recent datasets aim to consider more realistic continual learning, such as CORe50 (Lomonaco & Maltoni, 2017) and OpenLORIS
Revised: Recent datasets aim to consider more realistic continual learning, such as CORe50 (Lomonaco & Maltoni, 2017) and OpenLORIS

Original: (She et al., 2019).
Revised: (She et al., 2019).

Original: We summarize core features of these continual learning datasets in Appendix A. First, both CORe50 and
Revised: We summarize core features of these continual learning datasets in Appendix A. First, both CORe50 and

Original: OpenLORIS have relatively few object classes, which makes meta-learning approaches inapplicable; second, both contain images of small objects with minimal occlusion and viewpoint changes; and third, OpenLORIS does not have the desired incremental class learning.
Revised: OpenLORIS have relatively few object classes, which makes meta-learning approaches inapplicable; second, both contain images of small objects with minimal occlusion and viewpoint changes; and third, OpenLORIS does not have the desired incremental class learning.

Original: In concurrent work, Caccia et al.
Revised: In concurrent work, Caccia et al.

Original: (2020) proposes a setup to unify continual learning and meta-learning with a similar online evaluation procedure.
Revised: (2020) proposes a setup to unify continual learning and meta-learning with a similar online evaluation procedure.

Original: However, there are several notable differences.
Revised: However, there are several notable differences.

Original: First, their models focus on a general loss function without a speciﬁc design for predicting new classes; they predict new tasks by examining if the loss of exceeds some threshold.
Revised: First, their models focus on a general loss function without a speciﬁc design for predicting new classes; they predict new tasks by examining if the loss of exceeds some threshold.

Original: Second, the sequences of inputs are fully supervised.
Revised: Second, the sequences of inputs are fully supervised.

Original: Lastly, their benchmarks are based on synthetic task sequences such as
Revised: Lastly, their benchmarks are based on synthetic task sequences such as

Original: Omniglotor tiered-ImageNet, which are less naturalistic than our RoamingRooms dataset.
Revised: Omniglot or tiered-ImageNet, which are less naturalistic than our RoamingRooms dataset.

Original: Online meta-learning: Some existing work builds on early approaches (Thrun, 1998; Schmidhuber, 1987) that tackle continual learning from a meta-learning perspective.
Revised: Online meta-learning: Some existing work builds on early approaches (Thrun, 1998; Schmidhuber, 1987) that tackle continual learning from a meta-learning perspective.

Original: Finn et al.
Revised: Finn et al.

Original: (2019) proposes storing all task data in a data buffer; by contrast, Javed & White (2019) proposes to instead learn a good representation that supports such online updates.
Revised: (2019) proposes storing all task data in a data buffer; by contrast, Javed & White (2019) proposes to instead learn a good representation that supports such online updates.

Original: In Jerfel et al.
Revised: In Jerfel et al.

Original: (2019), a hierarchical Bayesian mixture model is used to address the dynamic nature of continual learning.
Revised: (2019), a hierarchical Bayesian mixture model is used to address the dynamic nature of continual learning.

Original: Connections to the human brain: Our CPM model consists of multiple memory systems, consistent with claims of cognitive neuroscientists of multiple memory systems in the brain.
Revised: Connections to the human brain: Our CPM model consists of multiple memory systems, consistent with claims of cognitive neuroscientists of multiple memory systems in the brain.

Original: The complementary learning systems (CLS) theory (McClelland et al., 1995) suggests that the hippocampus stores the recent experience and is likely where few-shot learning takes place.
Revised: The complementary learning systems (CLS) theory (McClelland et al., 1995) suggests that the hippocampus stores the recent experience and is likely where few-shot learning takes place.

Original: However, our model is more closely related to contextual binding theory (Yonelinas et al., 2019), which suggests that long-term encoding of information depends on binding spatiotemporal context, and without this[Figure] context as a cue, forgetting occurs.
Revised: However, our model is more closely related to contextual binding theory (Yonelinas et al., 2019), which suggests that long-term encoding of information depends on binding spatiotemporal context, and without this context as a cue, forgetting occurs.

Original: Our proposed CPM contains parallels to human brain memory components (Cohen & Squire, 1980).
Revised: Our proposed CPM contains parallels to human brain memory components (Cohen & Squire, 1980).

Original: Long term statistical learning is captured in a CNN that produces a deep embedding.
Revised: Long term statistical learning is captured in a CNN that produces a deep embedding.

Original: An RNN holds a type of working memory that can retain novel objects and spatiotemporal contexts.
Revised: An RNN holds a type of working memory that can retain novel objects and spatiotemporal contexts.

Original: Lastly, the prototype memory represents the semantic memory, which consolidates multiple events into a single knowledge vector (Duff et al., 2020).
Revised: Lastly, the prototype memory represents the semantic memory, which consolidates multiple events into a single knowledge vector (Duff et al., 2020).

Original: Other deep learning researchers have proposed multiple memory systems for continual learning.
Revised: Other deep learning researchers have proposed multiple memory systems for continual learning.

Original: In Parisi et al.
Revised: In Parisi et al.

Original: (2018), the learning algorithm is heuristic and representations come from pretrained networks.
Revised: (2018), the learning algorithm is heuristic and representations come from pretrained networks.

Original: In Kemker & Kanan (2018), a prototype memory is used for recalling recent examples and rehearsal from a generative model allows this knowledge to be integrated and distilled into a long-term memory.
Revised: In Kemker & Kanan (2018), a prototype memory is used for recalling recent examples and rehearsal from a generative model allows this knowledge to be integrated and distilled into a long-term memory.

Original:  O NLINE C ONTEXTUALIZED F EW -S HOT L EARNING
Revised:  O NLINE C ONTEXTUALIZED F EW -S HOT L EARNING

Original: In this section, we introduce our new online contextualized few-shot learning (OC-FSL) setup in the form of a sequential decision problem, and then introduce our new benchmark datasets.
Revised: In this section, we introduce our new online contextualized few-shot learning (OC-FSL) setup in the form of a sequential decision problem, and then introduce our new benchmark datasets.

Original: Continual few-shot classiﬁcation as a sequential decision problem: In traditional few-shot learning, an episode is constructed by a support set S and a query set Q .
Revised: Continual few-shot classiﬁcation as a sequential decision problem: In traditional few-shot learning, an episode is constructed by a support set S and a query set Q .

Original: A few-shot learner f is expected to predict the class of each example in the query set x Q based on the support set information: ˆ y Q = f
Revised: A few-shot learner f is expected to predict the class of each example in the query set x Q based on the support set information: ˆ y Q = f

Original: ( x Q ; ( x S 1 , y S 1 ) , . . .
Revised: ( x Q ; ( x S 1 , y S 1 ) , . . .

Original: , ( x SN , y SN )) , where x ∈ R D , and y ∈ [1 . . .
Revised: , ( x SN , y SN )) , where x ∈ R D , and y ∈ [1 . . .

Original: K ] .
Revised: K ] .

Original: This setup is not a natural ﬁt for continual learning, since it is unclear when to insert a query set into the sequence.
Revised: This setup is not a natural ﬁt for continual learning, since it is unclear when to insert a query set into the sequence.

Original: Inspired by the online learning literature, we can convert continual few-shot learning into a sequential decision problem, where every input example is also part of the evaluation: ˆ y t = f
Revised: Inspired by the online learning literature, we can convert continual few-shot learning into a sequential decision problem, where every input example is also part of the evaluation: ˆ y t = f

Original: ( x t ; ( x 1 , ˜ y 1 ) , . . .
Revised: ( x t ; ( x 1 , ˜ y 1 ) , . . .

Original: , ( x t − 1 , ˜ y t − 1 )) , for t = 1 . . .
Revised: , ( x t − 1 , ˜ y t − 1 )) , for t = 1 . . .

Original: T , where ˜ y here further allows that the sequence of inputs to be semi-supervised: ˜
Revised: T , where ˜ y here further allows that the sequence of inputs to be semi-supervised: ˜

Original: y equals y t if labeled, or otherwise − 1 .
Revised: y equals y t if labeled, or otherwise − 1 .

Original: The setup in Santoro et al.
Revised: The setup in Santoro et al.

Original: (2016) and Kaiser et al.
Revised: (2016) and Kaiser et al.

Original: (2017) is similar; they train RNNs using such a temporal sequence as input.
Revised: (2017) is similar; they train RNNs using such a temporal sequence as input.

Original: However, their evaluation relies on a “query set” at the end.
Revised: However, their evaluation relies on a “query set” at the end.

Original: We instead evaluate online while learning.
Revised: We instead evaluate online while learning.

Original: Figure 1-A illustrates these features, using an example of an input sequence where an agent is learning about new objects in a kitchen.
Revised: Figure 1-A illustrates these features, using an example of an input sequence where an agent is learning about new objects in a kitchen.

Original: The model is rewarded when it correctly predicts a known class or when it indicates that the item has yet to be assigned a label.
Revised: The model is rewarded when it correctly predicts a known class or when it indicates that the item has yet to be assigned a label.

Original: Contextualized environments: Typical continual learning consists of a sequence of tasks, and models are trained sequentially for each task.
Revised: Contextualized environments: Typical continual learning consists of a sequence of tasks, and models are trained sequentially for each task.

Original: This feature is also preserved in many incremental learning settings (Rebufﬁ et al., 2017).
Revised: This feature is also preserved in many incremental learning settings (Rebufﬁ et al., 2017).

Original: For instance, the split-CIFAR task divides CIFAR-100 intolearning environments, each with 10 classes, presented sequentially.
Revised: For instance, the split-CIFAR task divides CIFAR-100 intolearning environments, each with 10 classes, presented sequentially.

Original: In our formulation, the sequence returns to earlier environments (see Figure 1-B), which enables assessment of long-term durability of knowledge.
Revised: In our formulation, the sequence returns to earlier environments (see Figure 1-B), which enables assessment of long-term durability of knowledge.

Original: Although the ground-truth environment identity is not provided, we structure the task such that the environment itself provides contextual cues which can constrain the correct class label.
Revised: Although the ground-truth environment identity is not provided, we structure the task such that the environment itself provides contextual cues which can constrain the correct class label.

Original: Spatial cues in the input help distinguish one environment from another.
Revised: Spatial cues in the input help distinguish one environment from another.

Original: Temporal cues are implicit because the sequence tends to switch environments infrequently, allowing recent inputs to be beneﬁcial in guiding the interpretation of the current input.
Revised: Temporal cues are implicit because the sequence tends to switch environments infrequently, allowing recent inputs to be beneﬁcial in guiding the interpretation of the current input.

Original: RoamingOmniglot: The Omniglot dataset (Lake et al., 2015) contains 1623 handwritten characters from 50 different alphabets.
Revised: RoamingOmniglot: The Omniglot dataset (Lake et al., 2015) contains 1623 handwritten characters from 50 different alphabets.

Original: We split the alphabets into 31 for training, 5 for validation, andfor testing.
Revised: We split the alphabets into 31 for training, 5 for validation, andfor testing.

Original: We augment classes by 90 degree rotations to create 6492 classes in total.
Revised: We augment classes by 90 degree rotations to create 6492 classes in total.

Original: Each contextualized few-shot learning image sequence contains 150 images, drawn from a random sample of 5-10 alphabets, for a total of 50 classes per sequence.
Revised: Each contextualized few-shot learning image sequence contains 150 images, drawn from a random sample [Figure] RNN RNN

Revised: CNN

Revised: CNN

Original: These classes are randomly assigned to 5 different environments; within an environment, the characters are distributed according to a
Revised: [Figure] Temporal contextual features are extracted from an RNN. The prototype memory stores one vector per class and does online averaging.

Revised: Examples falling outside the radii of all prototypes are classiﬁed as “new.”

Revised: of 5-10 alphabets, for a total of 50 classes per sequence.

Revised: These classes are randomly assigned to 5 different environments; within an environment, the characters are distributed according to a

Original: Chinese restaurant process (Aldous, 1985) to mimic the imbalanced long-tail distribution of naturally occurring objects.
Revised: Chinese restaurant process (Aldous, 1985) to mimic the imbalanced long-tail distribution of naturally occurring objects.

Original: We switch between environments using a Markov switching process; i.e., at each step there is a constant probability of switching to another environment.
Revised: We switch between environments using a Markov switching process; i.e., at each step there is a constant probability of switching to another environment.

Original: An example sequence is shown in Figure 2-A.
Revised: An example sequence is shown in Figure 2-A.

Original: [Figure] RoamingRooms: As none of the current few-shot learning datasets provides the natural online learning experience that we would like to study, we created our own dataset using simulated indoor environments.
Revised: RoamingRooms: As none of the current few-shot learning datasets provides the natural online learning experience that we would like to study, we created our own dataset using simulated indoor environments.

Original: We formulate this as a few-shot instance learning problem, which could be a use case for a home robot: it needs to quickly recognize and differentiate novel object instances, and large viewpoint variations can make this task challenging (see examples in Figure 2-B).
Revised: We formulate this as a few-shot instance learning problem, which could be a use case for a home robot: it needs to quickly recognize and differentiate novel object instances, and large viewpoint variations can make this task challenging (see examples in Figure 2-B).

Original: There are over 7,000 unique instance classes in the dataset, making it suitable to meta-learning approaches.
Revised: There are over 7,000 unique instance classes in the dataset, making it suitable to meta-learning approaches.

Original: Our dataset is derived from the Matterport3D dataset (Chang et al., 2017), which has 90 indoor worlds captured using panoramic depth cameras.
Revised: Our dataset is derived from the Matterport3D dataset (Chang et al., 2017) with 90 indoor worlds captured using panoramic depth cameras.

Original: We split these into 60 worlds for training, 10 for validation and 20 for testing.
Revised: We split these into 60 worlds for training, 10 for validation and 20 for testing.

Original: We use MatterSim (Anderson et al., 2018)  to load the simulated world and collect camera images anduse HabitatSim (Savva et al., 2019)  to simulate 3D mesh and align instance segmentation labels onto 2D image space.
Revised: MatterSim (Anderson et al., 2018) is used to load the simulated world and collect camera images and HabitatSim (Savva et al., 2019) is used to project instance segmentation labels onto 2D image space.

Original: We created a random walking agent to collect the virtual visual experience.
Revised: We created a random walking agent to collect the virtual visual experience.

Original: For each viewpoint in the random walk, we randomly sample one object from the image sensor and highlight it with the available instance segmentation, forming an input frame .
Revised: For each viewpoint in the random walk, we randomly sample one object from the image sensor and highlight it with the available instance segmentation, forming an input frame .

Original: Each viewpoint provides environmental context—the other objects present in the room with the highlighted object.
Revised: Each viewpoint provides environmental context—the other objects present in the room with the highlighted object.

Original: Figure 3-A shows the object instance distribution.
Revised: Figure 3-A shows the object instance distribution.

Original: We see strong temporal correlation, as 30% of the time the same instance appears in the next frame (Figure 3-B), but there is also a signiﬁcant proportion of revisits.
Revised: We see strong temporal correlation, as 30% of the time the same instance appears in the next frame (Figure 3-B), but there is also a signiﬁcant proportion of revisits.

Original: On average, there are three different viewpoints per 100-image sequence (Figure 3-C).
Revised: On average, there are three different viewpoints per 100-image sequence (Figure 3-C).

Original: Details and other statistics of our proposed datasets are included in the Appendix.
Revised: Details and other statistics of our proposed datasets are included in the Appendix.

Revised: RoamingImageNet: To make the classiﬁcation problem more challenging, we also report results on

Revised: RoamingImageNet, which uses the same online sequence sampler as RoamingOmniglot but applied on the tiered -ImageNet dataset (Ren et al., 2018), a subset of the original ImageNet dataset (Russakovsky et al., 2015).

Revised: It contains 608 classes with a total of 779K images of size 84 × 84.

Revised: We use the same split of classes as Ren et al.

Revised: The high-level categories are used for sampling different “environments” just like the notion of alphabets in RoamingOmniglot.

Original:  C ONTEXTUAL P ROTOTYPICAL M EMORY N ETWORKS
Revised:  C ONTEXTUAL P ROTOTYPICAL M EMORY N ETWORKS

Original: In the online contextualized few-shot learning setup, the few-shot learner can potentially improve by modeling the temporal context.
Revised: In the online contextualized few-shot learning setup, the few-shot learner can potentially improve by modeling the temporal context.

Original: Metric learning approaches (Snell et al., 2017) typically ignore temporal contextual relations and directly compare the similarity between training and test samples.
Revised: Metric learning approaches (Snell et al., 2017) typically ignore temporal contextual relations and directly compare the similarity between training and test samples.

Original: Gradient-based approaches (Javed & White, 2019), on the other hand, have the ability to adapt to new contexts, but they do not naturally handle new and unlabeled examples.
Revised: Gradient-based approaches (Javed & White, 2019), on the other hand, have the ability to adapt to new contexts, but they do not naturally handle new and unlabeled examples.

Original: Inspired by the contextual binding theory of human memory (Yonelinas et al., 2019), we propose a simple yet effective approach that uses an RNN to transmit spatiotemporal context and control signals to a prototype memory (Figure 4).
Revised: Inspired by the contextual binding theory of human memory (Yonelinas et al., 2019), we propose a simple yet effective approach that uses an RNN to transmit spatiotemporal context and control signals to a prototype memory (Figure 4).

Original: Prototype memory: We start describing our model with the prototype memory, which is an online version of the Prototypical Network (or ProtoNet )
Revised: Prototype memory: We start describing our model with the prototype memory, which is an online version of the Prototypical Network (or ProtoNet )

Original: (Snell et al., 2017).
Revised: (Snell et al., 2017).

Original: ProtoNet can be viewed as a knowledge base memory, where each object class k is represented by a prototype vector p [ k ] , computed as the mean vector of all the support instances of the class in a sequence.
Revised: ProtoNet can be viewed as a knowledge base memory, where each object class k is represented by a prototype vector p [ k ] , computed as the mean vector of all the support instances of the class in a sequence.

Original: It can also be applied to our task of online few-shot learning naturally, with some modiﬁcations.
Revised: It can also be applied to our task of online few-shot learning naturally, with some modiﬁcations.

Original: Suppose that at time-step t we have already stored a few classes in the memory, each represented by their current prototype p t [ k ] , and we would like to query the memory using the input feature h t .
Revised: Suppose that at time-step t we have already stored a few classes in the memory, each represented by their current prototype p t [ k ] , and we would like to query the memory using the input feature h t .

Original: We model our prototype memory as ˆ y t,k = softmax( − d m t ( h t , p t [ k ])) , (e.g. squared Euclidean distance or cosine dissimilarity) parameterized by a vector m t that scales each dimension with a Hadamard product.
Revised: We model our prototype memory as ˆ y t,k = softmax( − d m t ( h t , p t [ k ])) , (e.g. squared Euclidean distance or cosine dissimilarity) parameterized by a vector m t that scales each dimension with a Hadamard product.

Original: To predict whether an example is of a new class, we can use a separate novelty output ˆ u rt  with sigmoid activation, similar to the approach introduced in Ren et al.
Revised: To predict whether an example is of a new class, we can use a separate novelty output ˆ u rt [Table] [Table] with sigmoid activation, similar to the approach introduced in Ren et al.

Original: (2018), where β rt and γ rt are yet-to-be-speciﬁed thresholding hyperparameters (the superscript r stands for read): 
Revised: (2018), where β rt and γ rt are yet-to-be-speciﬁed thresholding hyperparameters (the superscript r stands for read): [Equation]

Original: [Equation]
Revised: [Equation]

Original: Memory consolidation with online prototype averaging: Traditionally, ProtoNet uses the average representation of a class across all support examples.
Revised: Memory consolidation with online prototype averaging: Traditionally, ProtoNet uses the average representation of a class across all support examples.

Original: Here, we must be able to adapt the prototype memory incrementally at each step.
Revised: Here, we must be able to adapt the prototype memory incrementally at each step.

Original: Fortunately, we can recover the computation of a ProtoNet by performing a simple online averaging:
Revised: Fortunately, we can recover the computation of a ProtoNet by performing a simple online averaging:

Original: [Equation]
Revised: [Equation]

Original: where h is the input feature, and p is the prototype, and c is a scalar indicating the number of examples that have been added to this prototype up to time t .
Revised: where h is the input feature, and p is the prototype, and c is a scalar indicating the number of examples that have been added to this prototype up to time t .

Original: The online averaging function A can also be made more ﬂexible to allow more plasiticity, modeled by a gated averaging unit (GAU): 
Revised: The online averaging function A can also be made more ﬂexible to allow more plasiticity, modeled by a gated averaging unit (GAU): A GAU ( h t

Original: [Table] [Table]
Revised: [Equation]

Original: A GAU ( h t ; p t − 1 ) = (1 − f t ) · p t − 1 + f t · h t , where f t = σ ( W f [ h t , p t − 1 ] + b f ) ∈ R . 
Revised: p t − 1 ) = (1 − f t ) · p t − 1 + f t · h t , where f t = σ ( W f [ h t , p t − 1 ] + b f ) ∈ R . [Equation]

Original: When the current example is unlabeled, y t is encoded as − 1 , and the model’s own prediction ˆ y t will determine which prototype to update; in this case, the model must also determine a strength of belief, ˆ u wt , that the current unlabeled example should be treated as a new class.
Revised: When the current example is unlabeled, y t is encoded as − 1 , and the model’s own prediction ˆ y t will determine which prototype to update; in this case, the model must also determine a strength of belief, ˆ u wt , that the current unlabeled example should be treated as a new class.

Original: Given ˆ u wt and ˆ y t , the model can then update a prototype:
Revised: Given ˆ u wt and ˆ y t , the model can then update a prototype:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: As-yet-unspeciﬁed hyperparameters β wt and γ wt are required (the superscript w is for write).
Revised: As-yet-unspeciﬁed hyperparameters β wt and γ wt are required (the superscript w is for write).

Original: These parameters for the online-updating novelty output ˆ u wt are distinct from β rt and γ rt in Equation 1.
Revised: These parameters for the online-updating novelty output ˆ u wt are distinct from β rt and γ rt in Equation 1.

Original: The intuition is that for “self-teaching” to work, the model potentially needs to be more conservative in creating new classes (avoiding corruption of prototypes) than in predicting an input as being a new class.
Revised: The intuition is that for “self-teaching” to work, the model potentially needs to be more conservative in creating new classes (avoiding corruption of prototypes) than in predicting an input as being a new class.

Original: Contextual RNN: Instead of directly using the features from the CNN h CNN t as input features to the prototype memory, we would also like to use contextual information from the recent past.
Revised: Contextual RNN: Instead of directly using the features from the CNN h CNN t as input features to the prototype memory, we would also like to use contextual information from the recent past.

Original: Above we introduced threshold hyperparameters β rt , γ rt , β wt , γ wt as well as the metric parameter M t .
Revised: Above we introduced threshold hyperparameters β rt , γ rt , β wt , γ wt as well as the metric parameter M t .

Original: We let the contextual RNN output these additional control parameters, so that the unknown thresholds and metric function can adapt based on the information in the context.
Revised: We let the contextual RNN output these additional control parameters, so that the unknown thresholds and metric function can adapt based on the information in the context.

Original: The RNN produces the context vector h RNN t and other control parameters conditioned on h CNN t:
Revised: The RNN produces the context vector h RNN t and other control parameters conditioned on h CNN t

Original: [Equation]
Revised: [Equation]

Original:  where z t is the recurrent state of the RNN, and m t is the scaling factor in the dissimilarity score.
Revised: [Table] where z t is the recurrent state of the RNN [Table] and m t is the scaling factor in the dissimilarity score.

Original: The context, h RNN t , serves as an additive bias on the state vector used for FSL: h t = h CNN t + h RNN t .
Revised: The context, h RNN t , serves as an additive bias on the state vector used for FSL: h t = h CNN t + h RNN t .

Original: This addition operation in the feature space can help contextualize prototypes based on temporal proximity, and is also similar to how the human brain leverages spatiotemporal context for memory storage (Yonelinas et al., 2019).
Revised: This addition operation in the feature space can help contextualize prototypes based on temporal proximity, and is also similar to how the human brain leverages spatiotemporal context for memory storage (Yonelinas et al., 2019).

Original: Loss function: The loss function is computed after an entire sequence ends and all network parameters are learned end-to-end.
Revised: Loss function: The loss function is computed after an entire sequence ends and all network parameters are learned end-to-end.

Original: The loss is composed of two parts.
Revised: The loss is composed of two parts.

Original: The ﬁrst is binary crossentropy (BCE), for telling whether each example has been assigned a label or not, i.e., prediction of new classes.
Revised: The ﬁrst is binary crossentropy (BCE), for telling whether each example has been assigned a label or not, i.e., prediction of new classes.

Original: Second we use a multi-class cross-entropy for classifying among the known ones.
Revised: Second we use a multi-class cross-entropy for classifying among the known ones.

Original: We can write down the overall loss function as follows:
Revised: We can write down the overall loss function as follows:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: Training and evaluation: During training, we sample learning sequences and for each sequence, we perform one iterative update to minimize the loss function (Eq. 9). At the beginning of each sequence, the memory is reset.

Original: [Figure] [Figure]
Revised: During training, the model learns from a set of training classes. During test time, the model recognizes new classes that have never been seen during training.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we show experimental results for our online contextualized few-shot learning paradigm, using RoamingOmniglot and RoamingRooms (see Sec. 3) to evaluate our model, CPM, and other state-of-the-art methods.
Revised: In this section, we show experimental results for our online contextualized few-shot learning paradigm, using RoamingOmniglot and RoamingRooms (see Sec. 3) to evaluate our model, CPM, and other state-of-the-art methods.

Original: For Omniglot, we apply an 8 × 8 CutOut (Devries & Taylor, 2017) to each image to make the task more challenging.
Revised: For Omniglot, we apply an 8 × 8 CutOut (Devries & Taylor, 2017) to each image to make the task more challenging.

Original: Implementation details: For the RoamingOmniglot experiment we used the common 4-layer CNN for few-shot learning with 64 channels in each layer.
Revised: Implementation details: For RoamingOmniglot, we use the common 4-layer CNN for few-shot learning with 64 channels in each layer.

Original: For  the RoamingRooms experiment we resize the input to 120 × 160 and we use the ResNet-12 architecture (Oreshkin et al., 2018).
Revised: For RoamingImageNet, we also use ResNet-12 with input resolution 84 × 84 (Oreshkin et al., 2018). For the RoamingRooms, we resize the input to 120 ×and use ResNet-12.

Original: To represent the feature of the input image with an attention mask, we concatenate the global average pooled feature with the attention ROI feature, resulting in a 512d feature vector.
Revised: To represent the feature of the input image with an attention mask, we concatenate the global average pooled feature with the attention ROI feature, resulting in a 512d feature vector.

Original: For the contextual RNN, in both experiments we used an LSTM (Hochreiter & Schmidhuber, 1997) with a 256d hidden state.
Revised: For the contextual RNN, in both experiments we used an LSTM (Hochreiter & Schmidhuber, 1997) with a 256d hidden state.

Original: The best CPM model is equipped using GAU and cosine similarity for querying prototypes.
Revised: The best CPM model is equipped using GAU and cosine similarity for querying prototypes.

Original: Logits based on cosine similarity are multiplied with a learned scalar initialized at 10.0 (Oreshkin et al., 2018).
Revised: Logits based on cosine similarity are multiplied with a learned scalar initialized at 10.0 (Oreshkin et al., 2018).

Original: We include additional training details in Appendix B. Evaluation metrics:
Revised: We include additional training details in Appendix B. Evaluation metrics:

Original: In order to compute a single number that characterizes the learning ability over sequences, we propose to use average precision (AP) to combine the prediction on old and new classes.
Revised: In order to compute a single number that characterizes the learning ability over sequences, we propose to use average precision (AP) to evaluate both with respect to old versus new and the speciﬁc class predictions.

Original: Concretely, all predictions are sorted by their old vs. new scores, and we compute AP using the area under the precision-recall curve.
Revised: Concretely, all predictions are sorted by their old vs. new scores, and we compute AP using the area under the precision-recall curve.

Original: True positive is deﬁned as the correct prediction of a multi-class classiﬁcation among known classes.
Revised: A true positive is deﬁned as the correct prediction of a multi-class classiﬁcation among known classes.

Original: We also compute the “ N -shot” accuracy; i.e., the average accuracy after seeing the label N times in the sequence.
Revised: We also compute the “ N -shot” accuracy; i.e., the average accuracy after seeing the label N times in the sequence.

Original: Note that these accuracy scores only reﬂect the performance on known class predictions.
Revised: Note that these accuracy scores only reﬂect the performance on known class predictions.

Original: All numbers are reported with an average over 2,000 sequences and for N -shot accuracy standard error is also included.
Revised: All numbers are reported with an average over 2,000 sequences and for N -shot accuracy standard error is also included.

Revised: Further explanation of these metrics is in Appendix A.3.

Original: Comparisons: To evaluate the merits of our proposed model, we implement classic few-shot learning and online meta-learning methods.
Revised: Comparisons: To evaluate the merits of our proposed model, we implement classic few-shot learning and online meta-learning methods.

Original: More implementation and training details of these baseline methods can be found in Appendix B.  • OML (Javed & White, 2019):
Revised: More implementation and training details of these baseline methods can be found in Appendix B. [Figure] [Figure] • OML (Javed & White, 2019):

Original: This is an online version of MAML (Finn et al., 2017).
Revised: This is an online version of MAML (Finn et al., 2017).

Original: It performs one gradient descent step for each labeled input image, and slow weights are learned via backpropagation through time.
Revised: It performs one gradient descent step for each labeled input image, and slow weights are learned via backpropagation through time.

Original: On top of OML, we added an unknown predictor ˆ u t = 1 − max k ˆ y t,k( OML-U ).
Revised: On top of OML, we added an unknown predictor ˆ u t = 1 − max k ˆ y t,k( OML-U ).

Original: We also found that using cosine classiﬁer without the last layer ReLU is much better than using the original dot-product classiﬁer, and this improvement is denoted as OML-U++ .
Revised: We also found that using cosine classiﬁer without the last layer ReLU is usually better than using the original dot-product classiﬁer, and this improvement is denoted as OML-U++ .

Original: • LSTM (Hochreiter & Schmidhuber, 1997) & DNC (Graves et al., 2016):
Revised: • LSTM (Hochreiter & Schmidhuber, 1997) & DNC (Graves et al., 2016):

Original: We include RNN methods for comparison as well.
Revised: We include RNN methods for comparison as well.

Original: Differentiable neural computer (DNC) is an improved version of memory augmented neural network (MANN) (Santoro et al., 2016).
Revised: Differentiable neural computer (DNC) is an improved version of memory augmented neural network (MANN) (Santoro et al., 2016).

Original: • Online MatchingNet (O-MN)
Revised: • Online MatchingNet (O-MN)

Original: (Vinyals et al., 2016), IMP (O-IMP) (Allen et al., 2019) & ProtoNet (O-PN)
Revised: (Vinyals et al., 2016), IMP (O-IMP) (Allen et al., 2019) & ProtoNet (O-PN)

Original: (Snell et al., 2017):
Revised: (Snell et al., 2017):

Original: We used the same negative Euclidean distance as the similarity[Table] [Table] function for these three metric learning based approaches.
Revised: We used the same negative Euclidean distance as the similarity function for these three metric learning based approaches.

Original: In particular, MatchingNet stores all examples and performs nearest neighbor matching, which can be memory inefﬁcient.
Revised: In particular, MatchingNet stores all examples and performs nearest neighbor matching, which can be memory inefﬁcient.

Original: Note that Online ProtoNet is a variant of our method without the contextual RNN.
Revised: Note that Online ProtoNet is a variant of our method without the contextual RNN.

Original: Main results: Our main results are shown in Table 1 and 2, including both supervised and semisupervised settings.
Revised: Main results: Our main results are shown in Table 1, 2 and 3, including both supervised and semi-supervised settings.

Original: Our approach achieves the best performance on AP consistently across all settings.
Revised: Our approach achieves the best performance on AP consistently across all settings.

Original: Online ProtoNet is a direct comparison without our contextual RNN and it is clear that CPM is signiﬁcantly better.
Revised: Online ProtoNet is a direct comparison without our contextual RNN and it is clear that CPM is signiﬁcantly better.

Original: Our method is slightly worse than Online MatchingNet in terms of 3-shot accuracy on the RoamingRooms semisupervised benchmark.
Revised: Our method is slightly worse than Online MatchingNet in terms of 3-shot accuracy on the RoamingRooms semisupervised benchmark.

Original: This can be explained by the fact that
Revised: This can be explained by the fact that

Original: MatchingNet stores all past seen examples, whereas CPM only stores one prototype per class.
Revised: MatchingNet stores all past seen examples, whereas CPM only stores one prototype per class.

Original: Per timestep accuracy is plotted in Figure 5, and the decaying accuracy is due to the increasing number of classes over time.
Revised: Per timestep accuracy is plotted in Figure 5, and the decaying accuracy is due to the increasing number of classes over time.

Original: In RoamingOmniglot, CPM is able to closely match or even sometimes surpass the ofﬂine classiﬁer, which re-trains at each step and uses all images in a sequence except the current one.
Revised: In RoamingOmniglot, CPM is able to closely match or even sometimes surpass the ofﬂine classiﬁer, which re-trains at each step and uses all images in a sequence except the current one.

Original: This is reasonable as our model is able to leverage information from the current context.
Revised: This is reasonable as our model is able to leverage information from the current context.

Original: Effect of spatiotemporal context: To answer the question whether the gain in performance is due to spatiotemporal reasoning, we conduct the following experiment comparing CPM with online
Revised: Effect of spatiotemporal context: To answer the question whether the gain in performance is due to spatiotemporal reasoning, we conduct the following experiment comparing CPM with online

Original: ProtoNet.
Revised: ProtoNet.

Original: We allow the CNN to have the ability to recognize the context in RoamingOmniglot by adding a texture background image using the Kylberg texture dataset (Kylberg, 2011) (see Figureleft).
Revised: We allow the CNN to have the ability to recognize the context in RoamingOmniglot by adding a texture background image using the Kylberg texture dataset (Kylberg, 2011) (see Figureleft).

Original: As a control, we can also destroy the temporal context by shufﬂing all the images in a sequence.
Revised: As a control, we can also destroy the temporal context by shufﬂing all the images in a sequence.

Original: We train four different models on dataset controls with or without the presence of spatial or temporal context, and results are shown in Figure 6.
Revised: We train four different models on dataset controls with or without the presence of spatial or temporal context, and results are shown in Figure 6.

Original: First, both online ProtoNet and CPM beneﬁt from the inclusion of a spatial context.
Revised: First, both online ProtoNet and CPM beneﬁt from the inclusion of a spatial context.

Original: This is understandable as the CNN has the ability to learn spatial cues, which re-conﬁrms our main hypothesis that successful inference of the current context is beneﬁcial to novel object recognition.
Revised: This is understandable as the CNN has the ability to learn spatial cues, which re-conﬁrms our main hypothesis that successful inference of the current context is beneﬁcial to novel object recognition.

Original: Second, only our CPM model beneﬁts from the presence of temporal context, and it receives distinct gains from spatial and temporal contexts. 
Revised: Second, only our CPM model beneﬁts from the presence of temporal context, and it receives distinct gains from spatial and temporal contexts. [Table] [Table] [Table]

Original: Ablation studies: We ablate each individual module we introduce.
Revised: Effect of forgetting: As the number of learned classes increases, we expect the average accuracy to drop. To further investigate this forgetting effect, we measure the average accuracy in terms of the number of time steps the model has last seen the label of a particular class. It is reported in Table 4 and in Appendix C Table 13, 14, where we directly compare CPM and OPN to see the effect of temporal context. CPM is signiﬁcantly better than OPN on 1-shot within a short interval, which suggests that the contextual RNN makes the recall of the recent past much easier.

Revised: On RoamingImageNet, OPN eventually surpasses CPM on longer horizon, and this can be explained by the fact that OPN has more stable prototypes, whereas prototypes in CPM could potentially be affected by the ﬂuctuation of the contextual RNN over a longer horizon.

Revised: Ablation studies: We ablate each individual module we introduce.

Original: Results are shown in Tablesand 4.
Revised: Results are shown in Tablesand 6.

Original: Table 3 studies different ways we use the RNN, including the context vector h RNN , the predicted threshold parameters β ∗ t , γ ∗ t , and the predicted metric scaling vector t .
Revised: Table 5 studies different ways we use the RNN, including the context vector h RNN , the predicted threshold parameters β ∗ t , γ ∗ t , and the predicted metric scaling vector t .

Original: Table 4 studies various ways to learn from unlabeled examples, where we separately disable the RNN update, prototype update, and distinct write-threshold parameters β wt , γ wt (vs. using read-threshold parameters), which makes it robust to potential mistakes made in semi-supervised learning.
Revised: Table 6 studies various ways to learn from unlabeled examples, where we separately disable the RNN update, prototype update, and distinct write-threshold parameters β wt , γ wt (vs. using read-threshold parameters), which makes it robust to potential mistakes made in semi-supervised learning.

Original: We verify that each component has a positive impact on the performance.
Revised: We verify that each component has a positive impact on the performance.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We proposed online contextualized few-shot learning, OC-FSL, a paradigm for machine learning that emulates a human or artiﬁcial agent interacting with a physical world.
Revised: We proposed online contextualized few-shot learning, OC-FSL, a paradigm for machine learning that emulates a human or artiﬁcial agent interacting with a physical world.

Original: It combines multiple properties to create a challenging learning task: every input must be classiﬁed or ﬂagged as novel, every input is also used for training, semi-supervised learning can potentially improve performance, and the temporal distribution of inputs is non-IID and comes from a generative model in which input and class distributions are conditional on a latent environment with Markovian transition probabilities.
Revised: It combines multiple properties to create a challenging learning task: every input must be classiﬁed or ﬂagged as novel, every input is also used for training, semi-supervised learning can potentially improve performance, and the temporal distribution of inputs is non-IID and comes from a generative model in which input and class distributions are conditional on a latent environment with Markovian transition probabilities.

Original: We proposed the RoamingRooms dataset to simulate an agent wandering within a physical world.
Revised: We proposed the RoamingRooms dataset to simulate an agent wandering within a physical world.

Original: We also proposed a new model, CPM, which uses an RNN to extract spatiotemporal context from the input stream and to provide control settings to a prototype-based FSL model.
Revised: We also proposed a new model, CPM, which uses an RNN to extract spatiotemporal context from the input stream and to provide control settings to a prototype-based FSL model.

Original: In the context of naturalistic domains like RoamingRooms, CPM is able to leverage contextual information to attain performance unmatched by other state-of-the-art FSL methods.
Revised: In the context of naturalistic domains like RoamingRooms, CPM is able to leverage contextual information to attain performance unmatched by other state-of-the-art FSL methods.

Revised: Acknowledgments: We thank Fei Sha, James Lucas and Eleni Triantaﬁllou for helpful feedback on an earlier draft of the manuscript.

Revised: Resources used in preparing this research were provided, in part, by the Province of Ontario, the Government of Canada through CIFAR, and companies sponsoring the Vector Institute ( www.vectorinstitute.ai/#partners ).

Revised: This project is supported by

Revised: NSERC and the Intelligence Advanced Research Projects Activity (IARPA) via Department of

Original:  A D ATASET D ETAILS A. B ENCHMARK COMPARISON
Revised: Interior/Interior Business Center (DoI/IBC) contract number D16PC00003. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the ofﬁcial policies or endorsements, either expressed or implied, of IARPA, DoI/IBC, or the U.S. Government. A D ATASET D ETAILS A. B ENCHMARK COMPARISON

Original: We include Table 6 to compare existing continual and few-shot learning paradigms.
Revised: We include Table 8 to compare existing continual and few-shot learning paradigms.

Original: A. R OAMING O MNIGLOT  D ETAILS
Revised: A. R OAMING O MNIGLOT & R OAMING I MAGE N ET S AMPLER D ETAILS

Original: For the RoamingOmniglot  experiments, we use sequences with maximum 150 images, fromenvironments.
Revised: For the RoamingOmniglot and the RoamingImageNet experiments, we use sequences with maximum 150 images, from 5 environments.

Original: For individual environment, we use a Chinese restaurant process to sample the class distribution.
Revised: For individual environment, we use a Chinese restaurant process to sample the class distribution.

Original: In particular, the probability of sampling a new class is:
Revised: In particular, the probability of sampling a new class is:

Original: [Equation]
Revised: [Equation]

Original: where k is the number of classes that we have already sampled in the environment, and m is the total number of instances we have in the environment.
Revised: where k is the number of classes that we have already sampled in the environment, and m is the total number of instances we have in the environment.

Original: α is set to 0.2 and θ is set to 1.0 in all experiments.
Revised: α is set to 0.2 and θ is set to 1.0 in all experiments.

Original: The environment switching is implemented by a Markov switching process.
Revised: The environment switching is implemented by a Markov switching process.

Original: At each step in the sequence there is a constant probability p switch that switches to another environment.
Revised: At each step in the sequence there is a constant probability p switch that switches to another environment.

Original: For all experiments, we set p switch to 0.2.
Revised: For all experiments, we set p switch to 0.2.

Original: We truncate the maximum number of appearances per class to 6.
Revised: We truncate the maximum number of appearances per class to 6.

Original: If the maximum appearance is reached, we will sample another class.
Revised: If the maximum appearance is reached, we will sample another class.

Original: A. A DDITIONAL R OAMING R OOMS S TATISTICS
Revised: A. M ETRICS

Revised: Average precision: We chose to use AP (average precision or area under the precision-recall curve) as a way of integrating two aspects of performance: the binary accuracy of whether an instance belongs to a known or unknown class (KU-Assign for short), and 2. the accuracy of assigning an instance the correct class label given it is from a known class (Class-Assign for short).

Revised: The procedure to calculate AP is as follows.

Revised: We ﬁrst sort all the KU-Assign, Class-Assign predictions across all sequences in descending order based on KU-Assign probability, where the high ranked predictions should be known (not novel) classes.

Revised: For the N top ranked instances in the sorted list, we compute:

Revised: 1. precision@N =

Revised: correct(Class-Assign)@N / N 2.

Revised: recall@N = correct(Class-Assign)@N / K,

Revised: where K is the true number of known instances and correct(Class-Assign)@N is the count of the number of correct class assignments among the top N. (The class assignment for an unknown instance is always incorrect.)

Revised: To obtain the AP, we compute the integral of the function (y=precision@N, x=recall@N) across all N’s.

Revised: N-shot accuracy: We deﬁne N-shot accuracy as the number of times an instance that has been seen

Revised: N times thus far in the sequence is classiﬁed correctly.

Revised: We compute the mean and standard error of this over all sequences.

Revised: A. A DDITIONAL R OAMING R OOMS S TATISTICS

Original: Statistics of the RoamingRooms are included in Table 5, in comparison to other few-shot and continual learning datasets.
Revised: Statistics of the RoamingRooms are included in Table 7, in comparison to other few-shot and continual learning datasets.

Original: Note that since RoamingRooms is collected from a simulated environment, withindoor worlds consisting of 1.2K panorama images and 1.22M video frames.
Revised: Note that since RoamingRooms is collected from a simulated environment, withindoor worlds consisting of 1.2K panorama images and 1.22M video frames.

Original: The dataset contains about 6.9K random walk sequences with a maximum of 200 frames per sequence.
Revised: The dataset contains about 6.9K random walk sequences with a maximum of 200 frames per sequence.

Original: For training we randomly crop 100 frames to form a training sequence.
Revised: For training we randomly crop 100 frames to form a training sequence.

Original: There are 7.0K unique instance classes.
Revised: There are 7.0K unique instance classes.

Original: Plots of additional statistics of RoamingRooms are shown in Figure 7.
Revised: Plots of additional statistics of RoamingRooms are shown in Figure 7.

Original: In addition to the ones shown in the main paper, instances and viewpoints also follow long tail distributions.
Revised: In addition to the ones shown in the main paper, instances and viewpoints also follow long tail distributions.

Original: The number of objects in each frame follows an exponential distribution. 
Revised: The number of objects in each frame follows an exponential distribution. [Table]

Original: A. R OAMING R OOMS S IMULATOR D ETAILS
Revised: A. R OAMING R OOMS S IMULATOR D ETAILS

Original: We generate our episodes with a two-stage process using two simulators – HabitatSim (Savva et al., 2019) and MatterSim (Anderson et al., 2018) – because HabitatSim is based on 3D meshes and using
Revised: We generate our episodes with a two-stage process using two simulators – HabitatSim (Savva et al., 2019) and MatterSim (Anderson et al., 2018) – because HabitatSim is based on 3D meshes and using

Original: HabitatSim alone will result in poor image quality due to incorrect mesh reconstruction.
Revised: HabitatSim alone will result in poor image quality due to incorrect mesh reconstruction.

Original: Therefore we sacriﬁced the continuous movement of agents within HabitatSim and base our environment navigation on the discrete viewpoints in MatterSim, which is based on real panoramic images.
Revised: Therefore we sacriﬁced the continuous movement of agents within HabitatSim and base our environment navigation on the discrete viewpoints in MatterSim, which is based on real panoramic images.

Original: The horizontal ﬁeld of view is set to 90 degrees for HabitatSim and 100 degrees for MatterSim, and we simulate with 800 × 600 resolution.
Revised: The horizontal ﬁeld of view is set to 90 degrees for HabitatSim and 100 degrees for MatterSim [Table] and we simulate with 800 × 600 resolution.

Original: The ﬁrst stage of generation involves randomly picking a sequence of viewpoints on the connectivity graph within MatterSim.
Revised: The ﬁrst stage of generation involves randomly picking a sequence of viewpoints on the connectivity graph within MatterSim.

Original: For each viewpoint, the agent scans the environment along the yaw and pitch axes for a random period of time until a navigable viewpoint is within view.
Revised: For each viewpoint, the agent scans the environment along the yaw and pitch axes for a random period of time until a navigable viewpoint is within view.

Original: The time spent in a single viewpoint follows a Gaussian distribution with mean 5.0 and standard deviation 1.0.
Revised: The time spent in a single viewpoint follows a Gaussian distribution with mean 5.0 and standard deviation 1.0.

Original: At the start of each new viewpoint, the agent randomly picks a direction to rotate and takes 12.5 degree steps along the yaw axis, and with 95% probability, a 5 degree rotation along the pitch axis is applied in a randomly chosen direction.
Revised: At the start of each new viewpoint, the agent randomly picks a direction to rotate and takes 12.5 degree steps along the yaw axis [Table] and with 95% probability, a 5 degree rotation along the pitch axis is applied in a randomly chosen direction.

Original: When a navigable viewpoint is detected, the agent will navigate to the new viewpoint and reset the direction of scan.
Revised: When a navigable viewpoint is detected, the agent will navigate to the new viewpoint and reset the direction of scan.

Original: When multiple navigable viewpoints are present, the agent uniformly samples one.
Revised: When multiple navigable viewpoints are present, the agent uniformly samples one.

Original: In the second stage, an agent in HabitatSim retraces the viewpoint path and movements of the ﬁrst stage generated by MatterSim, collecting mesh-rendered RGB and instance segmentation sensor data.
Revised: In the second stage, an agent in HabitatSim retraces the viewpoint path and movements of the ﬁrst stage generated by MatterSim, collecting mesh-rendered RGB and instance segmentation sensor data.

Original: The MatterSim RGB and HabitatSim RGB images are then aligned via FLANN-based feature matching (Muja & Lowe, 2009), resulting in an alignment matrix that is used to place the MatterSim
Revised: The MatterSim RGB and HabitatSim RGB images are then aligned via FLANN-based feature matching (Muja & Lowe, 2009), resulting in an alignment matrix that is used to place the MatterSim

Original: RGB and HabitatSim instance segmentation maps into alignment.
Revised: RGB and HabitatSim instance segmentation maps into alignment.

Original: The sequence of these MatterSim
Revised: The sequence of these MatterSim

Original: RGB and HabitatSim instance segmentation maps constitute an episode.
Revised: RGB and HabitatSim instance segmentation maps constitute an episode.

Original: [Table] We keep objects of the following categories: picture, chair, lighting, cushion, table, plant, chest of drawers, towel, sofa, bed, appliances, stool, tv monitor, clothes, toilet, fireplace, furniture, bathtub, gym equipment, blinds, board panel .
Revised: We keep objects of the following categories: picture, chair, lighting, cushion, table, plant, chest of drawers, towel, sofa, bed, appliances, stool, tv monitor, clothes, toilet, fireplace, furniture, bathtub, gym equipment, blinds, board panel .

Original: We initially generate 600 frames per sequence and remove all the frames with no object.
Revised: We initially generate 600 frames per sequence and remove all the frames with no object.

Original: Then we store every 200 image frames into a separate ﬁle.
Revised: Then we store every 200 image frames into a separate ﬁle.

Original: During training and evaluation, each video sequence is loaded, and for each image we go through each object present in the image.
Revised: During training and evaluation, each video sequence is loaded, and for each image we go through each object present in the image.

Original: We create the attention map using the segmentation groundtruth of the selected object.
Revised: We create the attention map using the segmentation groundtruth of the selected object.

Original: The attention map and the image together form a frame in our model input.
Revised: The attention map and the image together form a frame in our model input.

Original: For training, we randomly crop 100 frames from the sequence, and for evaluation we use the ﬁrstframes for deterministic results.
Revised: For training, we randomly crop 100 frames from the sequence, and for evaluation we use the ﬁrstframes for deterministic results.

Original: Please visit our released code repository to download the RoamingRooms dataset.
Revised: Please visit our released code repository to download the RoamingRooms dataset.

Original: A. S EMI - SUPERVISED L ABELS :
Revised: A. S EMI - SUPERVISED L ABELS :

Original: Here we describe how we sample the labeled vs. unlabeled ﬂag for each example in the semisupervised sequences in both RoamingOmniglot and RoamingRooms datasets.
Revised: Here we describe how we sample the labeled vs. unlabeled ﬂag for each example in the semisupervised sequences in both RoamingOmniglot and RoamingRooms datasets.

Original: Due to the imbalance in our class distribution (from both the Chinese restaurant process and real data collection), directly masking the label may bias the model to ignore the rare seen classes.
Revised: Due to the imbalance in our class distribution (from both the Chinese restaurant process and real data collection), directly masking the label may bias the model to ignore the rare seen classes.

Original: Ideally, we would like to preserve at least one labeled example for each class.
Revised: Ideally, we would like to preserve at least one labeled example for each class.

Original: Therefore, we designed the following procedure. 
Revised: Therefore, we designed the following procedure. [Table]

Original: First, for each class k , suppose m k is the number of examples in the sequence that belong to the class.
Revised: First, for each class k [Table] suppose m k is the number of examples in the sequence that belong to the class.

Original: Let α be the target label ratio.
Revised: Let α be the target label ratio.

Original: Then the class-speciﬁc label ratio α k is:
Revised: Then the class-speciﬁc label ratio α k is:

Original: [Equation]
Revised: [Equation]

Original: We then for each class k , we sample a binary Bernoulli sequence based on Ber( α k ) .
Revised: We then for each class k [Table] we sample a binary Bernoulli sequence based on Ber( α k ) .

Original: If a class has all zeros in the Bernoulli sequence, we ﬂip the ﬂag of one of the instances to 1 to make sure there is at least one labeled instance for each class.
Revised: If a class has all zeros in the Bernoulli sequence [Table] we ﬂip the ﬂag of one of the instances to 1 to make sure there is at least one labeled instance for each class.

Original: For all experiments, we set α = 0 .
Revised: For all experiments, we set α = 0 .

Original: A. D ATASET S PLITS
Revised: A. D ATASET S PLITS

Original: We include details about our dataset splits in Table 7 and 8.
Revised: We include details about our dataset splits in Table 9 and 10.

Original: B E XPERIMENT D ETAILS B. N ETWORK A RCHITECTURE
Revised: B E XPERIMENT D ETAILS B. N ETWORK A RCHITECTURE

Original: For the RoamingOmniglot experiment we used the common 4-layer CNN for few-shot learning with 64 channels in each layer, resulting in a 64-d feature vector (Snell et al., 2017).
Revised: For the RoamingOmniglot experiment we used the common 4-layer CNN for few-shot learning with 64 channels in each layer, resulting in a 64-d feature vector (Snell et al., 2017).

Original: For the RoamingRooms experiment we resize the input to 120 × 160 and we use the ResNet-12 architecture (Oreshkin et al., 2018) with {32,64,128,256} channels per block.
Revised: For the RoamingRooms experiment we resize the input to 120 × 160 and we use the ResNet-12 architecture (Oreshkin et al., 2018) with {32,64,128,256} channels per block.

Original: To represent the feature of the input image with an attention mask, we concatenate the global average pooled feature with the attention ROI feature, resulting in a 512d feature vector.
Revised: To represent the feature of the input image with an attention mask, we concatenate the global average pooled feature with the attention ROI feature, resulting in a 512d feature vector.

Original: For the contextual RNN, in both experiments we used an
Revised: For the contextual RNN, in both experiments we used an

Original: LSTM (Hochreiter & Schmidhuber, 1997) with a 256d hidden state.
Revised: LSTM (Hochreiter & Schmidhuber, 1997) with a 256d hidden state.

Original: We use a linear layer to map from the output of the RNN to the features and control variables.
Revised: We use a linear layer to map from the output of the RNN to the features and control variables.

Original: We obtain γ r,w by adding 1.0 to the linear layer output and then applying the softplus activation.
Revised: We obtain γ r,w by adding 1.0 to the linear layer output and then applying the softplus activation.

Original: The bias units for β r,w are initialized to 10.0.
Revised: The bias units for β r,w are initialized to 10.0.

Original: We also apply the softplus activation to from the linear layer output.[Table]
Revised: We also apply the softplus activation to from the linear layer output.

Original: B. T RAINING P ROCEDURE
Revised: B. T RAINING P ROCEDURE

Original: We use the Adam optimizer (Kingma & Ba, 2015) for all of our experiments, with a gradient cap of 5.0.
Revised: We use the Adam optimizer (Kingma & Ba, 2015) for all of our experiments, with a gradient cap of 5.0.

Original: For RoamingOmniglot we train the network for 40k steps with a batch size 32 and maximum sequence length 150 across 2 GPUs and an initial learning rate 2e-3 decayed by 0.1 × at 20k and 30k steps.
Revised: For RoamingOmniglot we train the network for 40k steps with a batch size 32 and maximum sequence length 150 across 2 GPUs and an initial learning rate 2e-3 decayed by 0.1 × at 20k and 30k steps.

Original: For RoamingRooms we train for 20k steps with a batch size 8 and maximum sequence length 100 across 4 GPUs and an initial learning rate 1e-3 decayed by 0.1 × at 8k and 16k steps.
Revised: For RoamingRooms we train for 20k steps with a batch size 8 and maximum sequence length 100 across 4 GPUs and an initial learning rate 1e-3 decayed by 0.1 × at 8k and 16k steps.

Original: We use the
Revised: We use the

Original: BCE coefﬁcient λ = 1 for all experiments.
Revised: BCE coefﬁcient λ = 1 for all experiments.

Original: In semi-supervised experiments, around 30% examples are labeled when the number of examples grows large ( α = 0 .
Revised: In semi-supervised experiments, around 30% examples are labeled when the number of examples grows large ( α = 0 .

Original: 3 , see Equation 11).
Revised: 3 , see Equation 11).

Original: Early stopping is used in RoamingRooms experiments where the checkpoint with the highest validation AP score is chosen.
Revised: Early stopping is used in RoamingRooms experiments where the checkpoint with the highest validation AP score is chosen.

Original: For RoamingRooms, we sample Bernoulli sequences on unlabeled inputs to gradually allow semi-supervised writing to the prototype memory and we ﬁnd it helps training stability.
Revised: For RoamingRooms, we sample Bernoulli sequences on unlabeled inputs to gradually allow semi-supervised writing to the prototype memory and we ﬁnd it helps training stability.

Original: The probability starts with 0.0 and increase by 0.2 every 2000 training steps until reaching 1.0. 
Revised: The probability starts with 0.0 and increase by 0.2 every 2000 training steps until reaching 1.0. [Table] [Table]

Original: B. D ATA A UGMENTATION
Revised: B. D ATA A UGMENTATION

Original: For RoamingOmniglot, we pad the 28 × 28 image to 32 × 32 and then apply random cropping.
Revised: For RoamingOmniglot [Table] we pad the 28 × 28 image to 32 × 32 and then apply random cropping.

Original: For RoamingRooms, we apply random cropping in the time dimension to get a chunk of 100 frames per input example.
Revised: For RoamingRooms, we apply random cropping in the time dimension to get a chunk of 100 frames per input example.

Original: We also apply random dropping of 5% of the frames.
Revised: We also apply random dropping of 5% of the frames.

Original: We pad the 120 ×images to 126 × 168 and apply random cropping in each image frame.
Revised: We pad the 120 ×images to 126 × 168 and apply random cropping in each image frame.

Original: We also randomly ﬂip the order of the sequence (going forward or backward).
Revised: We also randomly ﬂip the order of the sequence (going forward or backward).

Original: B. S PATIOTEMPORAL CONTEXT EXPERIMENT DETAILS
Revised: B. S PATIOTEMPORAL CONTEXT EXPERIMENT DETAILS

Original: We use the Kylberg texture dataset (Kylberg, 2011) without rotations.
Revised: We use the Kylberg texture dataset (Kylberg, 2011) without rotations.

Original: Texture classes are split into train, val, and test, deﬁned in Table 10.
Revised: Texture classes are split into train, val, and test, deﬁned in Table 12.

Original: We resize all images ﬁrst to 256 × 256.
Revised: We resize all images ﬁrst to 256 × 256.

Original: For each Omniglot[Table] [Table] image, a 28 × 28 patch is randomly cropped from a texture image to serve as background.
Revised: For each Omniglot image, a 28 × 28 patch is randomly cropped from a texture image to serve as background.

Original: Random
Revised: Random

Original: Gaussian noises with mean zero and standard deviation 0.1 are added to the background images. 
Revised: Gaussian noises with mean zero and standard deviation 0.1 are added to the background images. [Table]

Original:  For spatial background experiments, we added an additional learnable network of the same size as the main network to take the background image as input, and output the same sized embedding vector.
Revised: [Figure] For spatial background experiments, we added an additional learnable network of the same size as the main network to take the background image as input, and output the same sized embedding vector.

Original: This embedding vector is further concatenated with the main embedding vector to form the ﬁnal embedding of the input.
Revised: This embedding vector is further concatenated with the main embedding vector to form the ﬁnal embedding of the input.

Original: We also found that using spatially overlayed images with a single CNN can achieve similar performance as well.
Revised: We also found that using spatially overlayed images with a single CNN can achieve similar performance as well.

Original: The ﬁnal numbers are reported using the concatenation approach since it is less prone to overlay noises and is more similar to the implementation we use in the RoamingRooms experiments.[Table] [Table]
Revised: The ﬁnal numbers are reported using the concatenation approach since it is less prone to overlay noises and is more similar to the implementation we use in the RoamingRooms experiments.

Original: B. B ASELINE IMPLEMENTATION DETAILS
Revised: B. B ASELINE IMPLEMENTATION DETAILS

Original: Online meta-learning (OML): The OML model performs one gradient descent step for each input.
Revised: Online meta-learning (OML): The OML model performs one gradient descent step for each input.

Original: In order for the model to predict unknown, we use the probability output from the softmax layer summing across the unused units.
Revised: In order for the model to predict unknown, we use the probability output from the softmax layer summing across the unused units.

Original: For example, if the softmax layer has 40 units and we have only seen 5 classes so far, then we sum the probability from the 6th to the last units.
Revised: For example, if the softmax layer has 40 units and we have only seen 5 classes so far, then we sum the probability from the 6th to the last units.

Original: This summed probability is separately trained with a binary cross entropy, same as in Equation 9.
Revised: This summed probability is separately trained with a binary cross entropy, same as in Equation 9.

Original: The inner learning rate is set to 1e-2 and we truncate the number of unrolled gradient descent steps to 5/20 (RoamingOmniglot/RoamingRooms), in order to make the computation feasible.
Revised: The inner learning rate is set to 1e-2 and we truncate the number of unrolled gradient descent steps to 5/20 (RoamingOmniglot/RoamingRooms), in order to make the computation feasible.

Original: For
Revised: For

Original: RoamingOmniglot, the network is trained with a batch size 32 across 2 GPUs, for a total of 20k steps, with an initial learning rate 2e-3 decayed by 0.1 at 10k and 16.7k steps.
Revised: RoamingOmniglot, the network is trained with a batch size 32 across 2 GPUs, for a total of 20k steps, with an initial learning rate 2e-3 decayed by 0.1 at 10k and 16.7k steps.

Original: For RoamingRooms, the network is trained with a batch size 8 across 4 GPUs, for a total of 16k steps, with an initial learning rate 1e-3 decayed by 0.1 at 6.4k and 12.8k steps.
Revised: For RoamingRooms, the network is trained with a batch size 8 across 4 GPUs, for a total of 16k steps, with an initial learning rate 1e-3 decayed by 0.1 at 6.4k and 12.8k steps.

Original: Long short-term memory (LSTM): We apply a stacked two layer LSTM with 256 hidden dimensions.
Revised: Long short-term memory (LSTM): We apply a stacked two layer LSTM with 256 hidden dimensions.

Original: Inputs are h CNN t concatenated with the label one-hot vector.
Revised: Inputs are h CNN t concatenated with the label one-hot vector.

Original: If an example is unlabeled, then the label vector is all-zero.
Revised: If an example is unlabeled, then the label vector is all-zero.

Original: We directly apply a linear layer on top of the LSTM to map the LSTM memory output into classiﬁcation logits, and the last logit is the binary classiﬁcation logit reserved for unknown.
Revised: We directly apply a linear layer on top of the LSTM to map the LSTM memory output into classiﬁcation logits, and the last logit is the binary classiﬁcation logit reserved for unknown.

Original: The training procedure is the same as our CPM model.
Revised: The training procedure is the same as our CPM model.

Original: Differentiable neural computer (DNC):
Revised: Differentiable neural computer (DNC):

Original: In order to make the DNC model work properly, we found that it is sometimes helpful to pretrain the CNN weights.
Revised: In order to make the DNC model work properly, we found that it is sometimes helpful to pretrain the CNN weights.

Original: Simply initializing from scratch and train CNN+DNC end-to-end sometimes results in poor performance.
Revised: Simply initializing from scratch and train CNN+DNC end-to-end sometimes results in poor performance.

Original: We hypothesize that the attention structure in the DNC model is detrimental to representation learning.
Revised: We hypothesize that the attention structure in the DNC model is detrimental to representation learning.

Original: Therefore,  for RoamingOmniglot experiments, we use pretrained ProtoNet weights for solving 1-shot 5-way episodes to initialize the CNN, and we keep ﬁnetuning the CNN weights with 10% of the full learning rate.
Revised: Therefore, [Table] for RoamingOmniglot experiments, we use pretrained ProtoNet weights for solving 1-shot 5-way episodes to initialize the CNN, and we keep ﬁnetuning the CNN weights with 10% of the full learning rate.

Original: For RoamingRooms experiments, we train the full model end-to-end from scratch.
Revised: For RoamingRooms experiments, we train the full model end-to-end from scratch.

Original: The DNC is also modiﬁed so that it is more effective using the label information from the input.
Revised: The DNC is also modiﬁed so that it is more effective using the label information from the input.

Original: In the original MANN paper (Santoro et al., 2016) for one-shot learning, the input features h CNN t and the label one-hot ID are simply concatenated to feed into the LSTM controller of MANN.
Revised: In the original MANN paper (Santoro et al., 2016) for one-shot learning, the input features h CNN t and the label one-hot ID are simply concatenated to feed into the LSTM controller of MANN.

Original: We ﬁnd that it is beneﬁcial to directly add label one-hot vector as an input to the write head that generates the write attention and the write content.
Revised: We ﬁnd that it is beneﬁcial to directly add label one-hot vector as an input to the write head that generates the write attention and the write content.

Original: Similar to the LSTM model, the memory readout is also sent to a linear layer in order to get the ﬁnal classiﬁcation logits, and the last logit is the binary classiﬁcation logit reserved for the unknowns.
Revised: Similar to the LSTM model, the memory readout is also sent to a linear layer in order to get the ﬁnal classiﬁcation logits, and the last logit is the binary classiﬁcation logit reserved for the unknowns.

Original: Finally we remove the linkage prediction part of the DNC due to training instability.
Revised: Finally we remove the linkage prediction part of the DNC due to training instability.

Original: The controller LSTM has 256 hidden dimensions, and the memory has 64 slots each withdimensions.
Revised: The controller LSTM has 256 hidden dimensions, and the memory has 64 slots each withdimensions.

Original: There are 4 read heads and 4 write heads.
Revised: There are 4 read heads and 4 write heads.

Original: The training procedure is the same as CPM.
Revised: The training procedure is the same as CPM.

Original: Online ProtoNet: Online ProtoNet is our modiﬁcation of the original ProtoNet (Snell et al., 2017).
Revised: Online ProtoNet: Online ProtoNet is our modiﬁcation of the original ProtoNet (Snell et al., 2017).

Original: It is similar to our CPM model without the contextual RNN.
Revised: It is similar to our CPM model without the contextual RNN.

Original: The feature from the CNN is directly written to the prototype memory.
Revised: The feature from the CNN is directly written to the prototype memory.

Original: In addition, we do not predict the control hyperparameters β { r,w } t , γ { r,w } t from the RNN and they are learned as regular parameters.
Revised: In addition, we do not predict the control hyperparameters β { r,w } t , γ { r,w } t from the RNN and they are learned as regular parameters.

Original: The training procedure is the same as CPM.
Revised: The training procedure is the same as CPM.

Original: Online MatchingNet: Online MatchingNet is our modiﬁcation of the original MatchingNet (Vinyals et al., 2016).
Revised: Online MatchingNet: Online MatchingNet is our modiﬁcation of the original MatchingNet (Vinyals et al., 2016).

Original: We do not consider the context embedding in the MatchingNet paper since it was originally designed for the entire episode using an attentional RNN encoder.
Revised: We do not consider the context embedding in the MatchingNet paper since it was originally designed for the entire episode using an attentional RNN encoder.

Original: It is similar to online ProtoNet but instead of doing online averaging, it directly stores each example and its class.
Revised: It is similar to online ProtoNet but instead of doing online averaging, it directly stores each example and its class.

Original: Since it is an example-based storage, we did not extend it to learn from unlabeled examples, and all unlabeled examples are skipped.
Revised: Since it is an example-based storage, we did not extend it to learn from unlabeled examples, and all unlabeled examples are skipped.

Original: We use a similar decision rule to determine whether an example belongs to a known cluster by looking at the distance to the nearest exemplar stored in the memory, shifted by β and scaled by 1 /γ .
Revised: We use a similar decision rule to determine whether an example belongs to a known cluster by looking at the distance to the nearest exemplar stored in the memory, shifted by β and scaled by 1 /γ .

Original: Note that online MatchingNet is not efﬁcient at memory storage since it scales with the number of steps in the sequence.
Revised: Note that online MatchingNet is not efﬁcient at memory storage since it scales with the number of steps in the sequence.

Original: In addition, we use the negative
Revised: In addition, we use the negative

Original: Euclidean distance as the similarity function.
Revised: Euclidean distance as the similarity function.

Original: The training procedure is the same as CPM.
Revised: The training procedure is the same as CPM.

Original: Online inﬁnite mixture prototypes (IMP): Online IMP is proposed as a mix of prototype and example-based storage by allowing a class to have multiple clusters.
Revised: Online inﬁnite mixture prototypes (IMP): Online IMP is proposed as a mix of prototype and example-based storage by allowing a class to have multiple clusters.

Original: If an example is classiﬁed as unknown or it is unlabeled, we will assign its cluster based on our prediction, which either assigns it to one of the existing clusters or creates a new cluster, depending on its distance to the nearest cluster.
Revised: If an example is classiﬁed as unknown or it is unlabeled, we will assign its cluster based on our prediction, which either assigns it to one of the existing clusters or creates a new cluster, depending on its distance to the nearest cluster.

Original: If a cluster with an unknown label later is assigned with an example with a known class, then the cluster label will also be updated.
Revised: If a cluster with an unknown label later is assigned with an example with a known class, then the cluster label will also be updated.

Original: We use the same decision rule as online ProtoNet to determine whether an example belongs to a known cluster by looking at the distance to the nearest cluster, shifted by β and scaled by 1 /γ .
Revised: We use the same decision rule as online ProtoNet to determine whether an example belongs to a known cluster by looking at the distance to the nearest cluster, shifted by β and scaled by 1 /γ .

Original: As described above, online IMP has the capability of learning from unlabeled examples, unlike online MatchingNet.
Revised: As described above, online IMP has the capability of learning from unlabeled examples, unlike online MatchingNet.

Original: However similar to online MatchingNet, online
Revised: However similar to online MatchingNet, online

Original: IMP is also not efﬁcient at memory storage since in the worst case it also scales with the number of steps in the sequence.
Revised: IMP is also not efﬁcient at memory storage since in the worst case it also scales with the number of steps in the sequence.

Original: Again, the training procedure is the same as CPM.
Revised: Again, the training procedure is the same as CPM.

Original:  C A DDITIONALV ISUALIZATION OF E XPERIMENTAL R ESULTS C. E MBEDDING V ISUALIZATION
Revised: [Table] [Table] C A DDITIONAL E XPERIMENTAL R ESULTS C. E FFECT OF F ORGETTING

Revised: We report the effect of forgetting of RoamingRooms and RoamingImageNet in Table 13 and 14.

Revised: C. E MBEDDING V ISUALIZATION

Original: Figure 8 shows the learned embedding of each example in Online ProtoNet vs. our CPM model in RoamingOmniglot sequences, where colors indicate environment IDs.
Revised: Figure 8 shows the learned embedding of each example in Online ProtoNet vs. our CPM model in RoamingOmniglot sequences, where colors indicate environment IDs.

Original: In Online ProtoNet, the example features does not reﬂect the temporal context, and as a result, colors are scattered across the space.
Revised: In Online ProtoNet, the example features does not reﬂect the temporal context, and as a result, colors are scattered across the space.

Original: By contrast, in the CPM embedding visualization, colors are clustered together and we see a smoother transition of environments in the embedding space.
Revised: By contrast, in the CPM embedding visualization, colors are clustered together and we see a smoother transition of environments in the embedding space

Original:  C. C ONTROL P ARAMETERS VS . T IME
Revised: . C. C ONTROL P ARAMETERS VS . T IME

Original: Finally we visualize the control parameter values predicted by the RNN in Figure 9.
Revised: Finally we visualize the control parameter values predicted by the RNN in Figure 9.

Original: We verify that we indeed need two sets of β and γ for read and write operations separately as they learn different values.
Revised: We verify that we indeed need two sets of β and γ for read and write operations separately as they learn different values.

Original: β w is smaller than β r which means that the network is more conservative when writing to prototypes.
Revised: β w is smaller than β r which means that the network is more conservative when writing to prototypes.

Original: γ w grows larger over time, which means that the network prefers a softer slope when writing to prototypes since in the later stage the prototype memory has already stored enough content and it can grow faster, whereas in the earlier stage, the prototype memory is more conservative to avoid embedding vectors to be assigned to wrong clusters.
Revised: γ w grows larger over time, which means that the network prefers a softer slope when writing to prototypes since in the later stage the prototype memory has already stored enough content and it can grow faster, whereas in the earlier stage, the prototype memory is more conservative to avoid embedding vectors to be assigned to wrong clusters.

Original: Online ProtoNet
Revised: Online ProtoNet

Original: CPM (Ours)  0 0 1 1  2 1   3 3    22 2  21 4 0 0 013 1 4  6   6 0   

Revised: CPM (Ours)  0 0 1 1  2 1   3 3    22 2  21 4 0 0 013 1 4  6   6 0

Original: [Figure]
Revised: [Figure]