Unsupervised Progressive Learning and the STAM Architecture

1 INTRODUCTION . We start by posing a challenging problem , referred to as Unsupervised Progressive Learning ( UPL ) ( see Figure 1 ) . In the UPL problem , the agent observes a sequence ( or stream ) of unlabeled data vectors { xt } t∈N with xt ∈ Rn . Each vector xt is associated with a class k ( xt ) and the vectors of class k follow a distribution Fk . The class information , however , is hidden from the agent . Occasionally , the agent may be given a small number of labeled examples of one or more classes . These examples are meant to associate “ names ” ( i.e. , class labels ) with the learned representations enabling classification tasks in which the set of output classes stays constant ( “ persistent tasks ” ) or increases ( “ expanding tasks ” ) . We denote asLt the set of class labels the agent has seen up to time t. This set is gradually increasing , meaning that the agent progressively learns about more classes . In the UPL context , the goal is to learn in an online manner salient representations of the unlabeled input stream so that the agent can , at any point in time t , classify a given set of test data based on the set of classes Lt it knows about so far . We require an online learner for pragmatic reasons : it would not be possible or desirable in practice to store and/or process all previously seen data and learn a new model offline every time there is a change in Lt . The online nature of the problem constraints the solution space : methods that require multiple passes over the training data and/or randomly sampled minibatches are not applicable in the UPL context . We assume that the distribution Fk associated with class k may also change with time – but this is a slow and gradual process so that an online learner can track changes in Fk . Abrupt changes would require that the agent forgets what was previously learned about class k – we do not consider that possibility in this paper . We do not add any further constraints on the structure of the data sequence . For instance , it is possible that the learner first observes a labeled example of class k at time t ( and so k ∈ Lt ) even though it has not seen any unlabeled examples of that class prior to t – this would require a transfer-learning capability so that the learner can classify k based on representations it has previously learned from other classes . Another interesting scenario is when the unlabeled data arrive in separated class phases , which are unknown to the agent , so that each phase includes data from only few new classes – this is a challenging task from the perspective of catastrophic forgetting because the learner should not forget previously learned classes for which it does not see any new examples . We consider such UPL scenarios in Section 3 . It is plausible that UPL represents how animals learn , at least in the case of perceptual learning ( Goldstone , 1998 ) : they observe their environment , which is predominantly “ unlabeled ” , and so they learn to gradually distinguish between a growing number of different object categories even when they do not have a way yet to name them . Later , some of those classes may be associated with words ( in the case of humans ) ( Ashby and Maddox , 2005 ) , or more generally , with a specific taste , odor , reward , fear , etc . ( Watanabe et al. , 2001 ) . 1.1 UPL VERSUS SIMILAR LEARNING PARADIGMS . 1 . Unsupervised and self-supervised learning : There have been great strides recently in learning data representations ( Bengio et al. , 2013 ) via clustering ( Caron et al. , 2018 ) , generative models ( Jiang et al. , 2017 ; Eslami et al. , 2016 ; Kosiorek et al. , 2018 ; 2019 ) , and information theory ( Hjelm et al. , 2019 ; Ji et al. , 2019 ) . While these methods can learn representations without data labels , they still require prior information about the number of classes present in a given dataset ( to set the number of cluster centroids or class outputs ) and , in the continual learning case , they suffer from catastrophic forgetting unless some form of replay is used . 2 . Few-shot learning ( FSL ) and Meta-learning : Such methods attempt to recognize object classes not seen in a training set with only a single ( or handful ) of labeled examples ( Fei-Fei et al. , 2006 ; Snell et al. , 2017 ; Finn et al. , 2017 ; Ren et al. , 2018 ) . FSL requires labeled data to learn good representations - whereas UPL only requires labeled data to associate already learned representations with new classes . 3 . Semi-supervised learning ( SSL ) : SSL addresses scarcity of available labeled data for model training by leveraging large amounts of unlabeled training data to boost performance ( Springenberg , 2015 ; Oliver et al. , 2018 ; Miyato et al. , 2018 ; Kingma et al. , 2014 ) . SSL requires both labeled and unlabeled data during the training process and in most cases it needs to store and process the labeled data repeatedly during the training process . 4 : Continual learning ( CL ) : Most CL methods rely on labeled data and knowledge of which tasks are learned or performed at any point in time ( Hsu et al. , 2018 ; Parisi et al. , 2019 ; Kemker et al. , 2018 ; van de Ven and Tolias , 2019 ; Lopez-Paz and Ranzato , 2017 ) . Further , the most effective CL mechanisms address catastrophic forgetting using stored examples or generative replay ( Gepperth and Karaoguz , 2017 ; Shin et al. , 2017 ) . 5 . Transfer learning ( TL ) : Such methods require pre-training on a large labeled dataset and so they are not applicable to UPL ( Yosinski et al. , 2014 ) . 6 . Progressive learning/networks : In ( labeled ) progressive learning , a supervised classification model must be able to learn in an online manner without prior knowledge of the number of classes ( Venkatesan and Er , 2016 ; Rusu et al. , 2016 ) . However , the existing approaches in this area require supervision when a new class appears , and they suffer from catastrophic forgetting if they do not see data from a previous class for long periods of time . 2 STAM ARCHITECTURE AND LEARNING ALGORITHM . The learning approach that we pursue in this work is based on online clustering , novelty detection , two separate short-term and long-term memories , and storing only prototypical representations rather than specific examples . In the following , we describe the STAM architecture as a sequence of its seven major components . The notation is summarized in Appendix A . The image preprocessing pipeline is minimal and is described in Appendix B . The reasons we decided to not pursue a deep learning approach are discussed in Appendix C. 1 . Hierarchy of increasing receptive fields : An input vector xt ∈ Rn ( an image in all subsequent examples ) is analyzed through a hierarchy of Λ layers . Instead of neurons or hidden-layer units , each layer consists of STAM units – in its simplest form a STAM unit functions as an online clustering module . Each STAM processes one ρl × ρl patch ( subvector ) of the input at that layer . The patches are overlapping , with a small stride ( set to one pixel in our experiments ) , to accomplish translation invariance ( similar to CNNs ) . The patch dimension ρl increases in higher layers – the idea is that the first layer learns the smallest and most elementary patterns while the top layer learns the largest and most complex patterns . 2 . Online clustering : Every patch of each layer is clustered , in an online manner , to a set of centroids . These time-varying centroids form the prototypical patterns that the STAM architecture gradually learns at that layer . All STAM units of layer l share the same set of centroids Cl ( t ) – again for translation invariance.1 Given the m ’ th input patch xl , m at layer l , the nearest centroid of Cl selected for xl , m is cl.j = arg min c∈Cl d ( xl , m , c ) ( 1 ) where d ( xl , m , c ) is the Euclidean distance between the patch xl , m and centroid c.2 The selected centroid is updated based on a learning rate parameter α , as follows : cl , j = αxl , m + ( 1− α ) cl , j , 0 < α < 1 ( 2 ) A higher α value makes the learning process faster but less predictable . We do not use a decreasing value of α because the goal is to keep learning in a non-stationary environment rather than convergence to a stable centroid . If the centroid cl , j is selected by more than one patches of the same input , the centroid is updated based on the closest patch to that centroid . An online clustering algorithm that is similar to our approach ( and asymptotically equivalent to k-means ) can be implemented with a simple recurrent neural network of excitatory and inhibitory spiking neurons using strictly Hebbian learning , as shown recently ( Pehlevan et al. , 2017 ) . 3 . Novelty detection : When an input patch xl , m at layer l is significantly different than all centroids at that layer ( i.e. , its distance to the nearest centroid is a statistical outlier ) , a new centroid is created in Cl based on xl , m . We refer to this event as Novelty Detection ( ND ) . This function is necessary so that the architecture can learn centroids associated with new classes after they appear in the unlabeled data stream . To do so , we estimate in an online manner the distribution of the Euclidean distance between input patches and their nearest centroid ( separately for each layer ) . We sample a randomly chosen patch from each input vector , only considering the last 1000 inputs . The novelty detection threshold at layer l is denoted by D̂l and it is defined as the 95-th percentile ( β = 0.95 ) of the distance distribution . 1We drop the time index t from this point on but it is still implied that the centroids are dynamically learned over time . 2We have also experimented with the L1 distance metric with only minimal differences . 4 . Dual-memory organization : New centroids are stored temporarily in a Short-Term Memory ( STM ) of limited capacity ∆ ( separate for each layer ) . Every time a centroid is selected as the nearest neighbor of an input patch , it is updated based on ( 2 ) . If an STM centroid cl , j is selected sl , j > θ times , it is copied to the Long-Term Memory ( LTM ) for that layer . We refer to this event as memory consolidation . The LTM has ( practically ) unlimited capacity and the learning rate is much smaller ( in our experiments , set to zero ) . This memory organization is inspired by the Complementary Learning Systems framework ( Kumaran et al. , 2016 ) , where the STM role is played by the hippocampus and the LTM role by the cortex . This dual-memory scheme is necessary to distinguish between infrequently seen patterns that can be forgotten , and new patterns that are frequently seen after they first appear . We initialize the pool of STM centroids at each layer using randomly sampled patches from the unlabeled stream ( a single patch from each image to maximize diversity ) . When the STM pool of centroids at a layer is full , the introduction of a new centroid ( created through novelty detection ) causes the removal of an earlier centroid . We use the Least-Recently Used ( LRU ) policy to remove atypical centroids that have not been recently selected by any input . Figure 2 illustrates this dual-memory organization . 5 . Associating centroids with classes : Suppose that we have seen some labeled examples XL ( t ) from a set of classes L ( t ) up to time t. In the UPL context , we only use such labeled examples to associate existing LTM centroids at time t ( learned strictly from unlabeled data ) with the set of classes in L ( t ) . Given a labeled example of class k , suppose that there is a patch x in that example for which the nearest centroid is c. That patch contributes the following association between centroid c and class k : fx , c ( k ) = e −d ( x , c ) /D̄l ( 3 ) where D̄l is a normalization constant ( calculated as the average distance between input patches and centroids ) . The class-association vector gc between centroid c and any class k is computed aggregating all such associations , across all labeled examples in XL : gc ( k ) = ∑ x∈XL ( k ) fx , c ( k ) ∑ k′∈L ( t ) ∑ x∈XL ( k′ ) fx , c ( k ′ ) , k = 1 . . . L ( t ) ( 4 ) Note that ∑ k gc ( k ) =1 . 6 . Class informative centroids : If a centroid is associated with only one class k ( gc ( k ) = 1 ) , only labeled examples of that class select that centroid . At the other extreme , if a centroid is equally likely to be selected by examples of any labeled class , ( gc ( k ) ≈ 1/|L ( t ) | ) , the selection of that centroid does not provide any significant information for the class of the corresponding input . We identify the centroids that are Class INformative ( CIN ) as those that are associated with at least one class more than expected by chance . Specifically , a centroid c is CIN if max k∈L ( t ) gc ( k ) > 1 |L ( t ) | + γ ( 5 ) where 1|L ( t ) | is the chance term and γ is an additional significance term . 7 . Classification using a hierarchy of centroids : At test time , we are given an input x of class k ( x ) and infer its class as k̂ ( x ) . The classification task is a “ biased voting ” process in which every patch of x , at any layer , votes for a single class as long as that patch selects a CIN centroid . Specifically , if a patch xl , m of layer l selects a CIN centroid c , then that patch votes vl , m ( k ) = maxk∈L ( t ) gc ( k ) for the class k that has the highest association with c , and zero for all other classes . If c is not a CIN centroid , the vote of that patch is vl , m ( k ) = 0 for all classes . The vote of layer l for class k is the average vote across all patches in layer l ( as illustrated in Figure 3 ) : vl ( k ) = ∑ m∈Ml vl , m ( k ) |Ml| ( 6 ) where Ml is the set of patches in layer l. The final inference for input x is the class with the highest cumulative vote across all layers : k̂ ( x ) = arg max k′ Λ∑ l=1 vl ( k ) ( 7 )

Summary: This paper presents an "Unsupervised progressive learning" (UPL) problem, where a model is exposed to data in an non-iid manner, and each training example is presented once. Simple to continual learning, but a little more explicit in the connections to the way biological agents learn. They present a model that uses clustering and long-term memory (buffered) and compare on a few UPL tasks with additional supervision signal (classification) or unsupervised (clustering).