Unsupervised Progressive Learning and the STAM Architecture

1 INTRODUCTION . We start by posing a challenging problem , referred to as Unsupervised Progressive Learning ( UPL ) ( see Figure 1 ) . In the UPL problem , the agent observes a sequence ( or stream ) of unlabeled data vectors { xt } t∈N with xt ∈ Rn . Each vector xt is associated with a class k ( xt ) and the vectors of class k follow a distribution Fk . The class information , however , is hidden from the agent . Occasionally , the agent may be given a small number of labeled examples of one or more classes . These examples are meant to associate “ names ” ( i.e. , class labels ) with the learned representations enabling classification tasks in which the set of output classes stays constant ( “ persistent tasks ” ) or increases ( “ expanding tasks ” ) . We denote asLt the set of class labels the agent has seen up to time t. This set is gradually increasing , meaning that the agent progressively learns about more classes . In the UPL context , the goal is to learn in an online manner salient representations of the unlabeled input stream so that the agent can , at any point in time t , classify a given set of test data based on the set of classes Lt it knows about so far . We require an online learner for pragmatic reasons : it would not be possible or desirable in practice to store and/or process all previously seen data and learn a new model offline every time there is a change in Lt . The online nature of the problem constraints the solution space : methods that require multiple passes over the training data and/or randomly sampled minibatches are not applicable in the UPL context . We assume that the distribution Fk associated with class k may also change with time – but this is a slow and gradual process so that an online learner can track changes in Fk . Abrupt changes would require that the agent forgets what was previously learned about class k – we do not consider that possibility in this paper . We do not add any further constraints on the structure of the data sequence . For instance , it is possible that the learner first observes a labeled example of class k at time t ( and so k ∈ Lt ) even though it has not seen any unlabeled examples of that class prior to t – this would require a transfer-learning capability so that the learner can classify k based on representations it has previously learned from other classes . Another interesting scenario is when the unlabeled data arrive in separated class phases , which are unknown to the agent , so that each phase includes data from only few new classes – this is a challenging task from the perspective of catastrophic forgetting because the learner should not forget previously learned classes for which it does not see any new examples . We consider such UPL scenarios in Section 3 . It is plausible that UPL represents how animals learn , at least in the case of perceptual learning ( Goldstone , 1998 ) : they observe their environment , which is predominantly “ unlabeled ” , and so they learn to gradually distinguish between a growing number of different object categories even when they do not have a way yet to name them . Later , some of those classes may be associated with words ( in the case of humans ) ( Ashby and Maddox , 2005 ) , or more generally , with a specific taste , odor , reward , fear , etc . ( Watanabe et al. , 2001 ) . 1.1 UPL VERSUS SIMILAR LEARNING PARADIGMS . 1 . Unsupervised and self-supervised learning : There have been great strides recently in learning data representations ( Bengio et al. , 2013 ) via clustering ( Caron et al. , 2018 ) , generative models ( Jiang et al. , 2017 ; Eslami et al. , 2016 ; Kosiorek et al. , 2018 ; 2019 ) , and information theory ( Hjelm et al. , 2019 ; Ji et al. , 2019 ) . While these methods can learn representations without data labels , they still require prior information about the number of classes present in a given dataset ( to set the number of cluster centroids or class outputs ) and , in the continual learning case , they suffer from catastrophic forgetting unless some form of replay is used . 2 . Few-shot learning ( FSL ) and Meta-learning : Such methods attempt to recognize object classes not seen in a training set with only a single ( or handful ) of labeled examples ( Fei-Fei et al. , 2006 ; Snell et al. , 2017 ; Finn et al. , 2017 ; Ren et al. , 2018 ) . FSL requires labeled data to learn good representations - whereas UPL only requires labeled data to associate already learned representations with new classes . 3 . Semi-supervised learning ( SSL ) : SSL addresses scarcity of available labeled data for model training by leveraging large amounts of unlabeled training data to boost performance ( Springenberg , 2015 ; Oliver et al. , 2018 ; Miyato et al. , 2018 ; Kingma et al. , 2014 ) . SSL requires both labeled and unlabeled data during the training process and in most cases it needs to store and process the labeled data repeatedly during the training process . 4 : Continual learning ( CL ) : Most CL methods rely on labeled data and knowledge of which tasks are learned or performed at any point in time ( Hsu et al. , 2018 ; Parisi et al. , 2019 ; Kemker et al. , 2018 ; van de Ven and Tolias , 2019 ; Lopez-Paz and Ranzato , 2017 ) . Further , the most effective CL mechanisms address catastrophic forgetting using stored examples or generative replay ( Gepperth and Karaoguz , 2017 ; Shin et al. , 2017 ) . 5 . Transfer learning ( TL ) : Such methods require pre-training on a large labeled dataset and so they are not applicable to UPL ( Yosinski et al. , 2014 ) . 6 . Progressive learning/networks : In ( labeled ) progressive learning , a supervised classification model must be able to learn in an online manner without prior knowledge of the number of classes ( Venkatesan and Er , 2016 ; Rusu et al. , 2016 ) . However , the existing approaches in this area require supervision when a new class appears , and they suffer from catastrophic forgetting if they do not see data from a previous class for long periods of time . 2 STAM ARCHITECTURE AND LEARNING ALGORITHM . The learning approach that we pursue in this work is based on online clustering , novelty detection , two separate short-term and long-term memories , and storing only prototypical representations rather than specific examples . In the following , we describe the STAM architecture as a sequence of its seven major components . The notation is summarized in Appendix A . The image preprocessing pipeline is minimal and is described in Appendix B . The reasons we decided to not pursue a deep learning approach are discussed in Appendix C. 1 . Hierarchy of increasing receptive fields : An input vector xt ∈ Rn ( an image in all subsequent examples ) is analyzed through a hierarchy of Λ layers . Instead of neurons or hidden-layer units , each layer consists of STAM units – in its simplest form a STAM unit functions as an online clustering module . Each STAM processes one ρl × ρl patch ( subvector ) of the input at that layer . The patches are overlapping , with a small stride ( set to one pixel in our experiments ) , to accomplish translation invariance ( similar to CNNs ) . The patch dimension ρl increases in higher layers – the idea is that the first layer learns the smallest and most elementary patterns while the top layer learns the largest and most complex patterns . 2 . Online clustering : Every patch of each layer is clustered , in an online manner , to a set of centroids . These time-varying centroids form the prototypical patterns that the STAM architecture gradually learns at that layer . All STAM units of layer l share the same set of centroids Cl ( t ) – again for translation invariance.1 Given the m ’ th input patch xl , m at layer l , the nearest centroid of Cl selected for xl , m is cl.j = arg min c∈Cl d ( xl , m , c ) ( 1 ) where d ( xl , m , c ) is the Euclidean distance between the patch xl , m and centroid c.2 The selected centroid is updated based on a learning rate parameter α , as follows : cl , j = αxl , m + ( 1− α ) cl , j , 0 < α < 1 ( 2 ) A higher α value makes the learning process faster but less predictable . We do not use a decreasing value of α because the goal is to keep learning in a non-stationary environment rather than convergence to a stable centroid . If the centroid cl , j is selected by more than one patches of the same input , the centroid is updated based on the closest patch to that centroid . An online clustering algorithm that is similar to our approach ( and asymptotically equivalent to k-means ) can be implemented with a simple recurrent neural network of excitatory and inhibitory spiking neurons using strictly Hebbian learning , as shown recently ( Pehlevan et al. , 2017 ) . 3 . Novelty detection : When an input patch xl , m at layer l is significantly different than all centroids at that layer ( i.e. , its distance to the nearest centroid is a statistical outlier ) , a new centroid is created in Cl based on xl , m . We refer to this event as Novelty Detection ( ND ) . This function is necessary so that the architecture can learn centroids associated with new classes after they appear in the unlabeled data stream . To do so , we estimate in an online manner the distribution of the Euclidean distance between input patches and their nearest centroid ( separately for each layer ) . We sample a randomly chosen patch from each input vector , only considering the last 1000 inputs . The novelty detection threshold at layer l is denoted by D̂l and it is defined as the 95-th percentile ( β = 0.95 ) of the distance distribution . 1We drop the time index t from this point on but it is still implied that the centroids are dynamically learned over time . 2We have also experimented with the L1 distance metric with only minimal differences . 4 . Dual-memory organization : New centroids are stored temporarily in a Short-Term Memory ( STM ) of limited capacity ∆ ( separate for each layer ) . Every time a centroid is selected as the nearest neighbor of an input patch , it is updated based on ( 2 ) . If an STM centroid cl , j is selected sl , j > θ times , it is copied to the Long-Term Memory ( LTM ) for that layer . We refer to this event as memory consolidation . The LTM has ( practically ) unlimited capacity and the learning rate is much smaller ( in our experiments , set to zero ) . This memory organization is inspired by the Complementary Learning Systems framework ( Kumaran et al. , 2016 ) , where the STM role is played by the hippocampus and the LTM role by the cortex . This dual-memory scheme is necessary to distinguish between infrequently seen patterns that can be forgotten , and new patterns that are frequently seen after they first appear . We initialize the pool of STM centroids at each layer using randomly sampled patches from the unlabeled stream ( a single patch from each image to maximize diversity ) . When the STM pool of centroids at a layer is full , the introduction of a new centroid ( created through novelty detection ) causes the removal of an earlier centroid . We use the Least-Recently Used ( LRU ) policy to remove atypical centroids that have not been recently selected by any input . Figure 2 illustrates this dual-memory organization . 5 . Associating centroids with classes : Suppose that we have seen some labeled examples XL ( t ) from a set of classes L ( t ) up to time t. In the UPL context , we only use such labeled examples to associate existing LTM centroids at time t ( learned strictly from unlabeled data ) with the set of classes in L ( t ) . Given a labeled example of class k , suppose that there is a patch x in that example for which the nearest centroid is c. That patch contributes the following association between centroid c and class k : fx , c ( k ) = e −d ( x , c ) /D̄l ( 3 ) where D̄l is a normalization constant ( calculated as the average distance between input patches and centroids ) . The class-association vector gc between centroid c and any class k is computed aggregating all such associations , across all labeled examples in XL : gc ( k ) = ∑ x∈XL ( k ) fx , c ( k ) ∑ k′∈L ( t ) ∑ x∈XL ( k′ ) fx , c ( k ′ ) , k = 1 . . . L ( t ) ( 4 ) Note that ∑ k gc ( k ) =1 . 6 . Class informative centroids : If a centroid is associated with only one class k ( gc ( k ) = 1 ) , only labeled examples of that class select that centroid . At the other extreme , if a centroid is equally likely to be selected by examples of any labeled class , ( gc ( k ) ≈ 1/|L ( t ) | ) , the selection of that centroid does not provide any significant information for the class of the corresponding input . We identify the centroids that are Class INformative ( CIN ) as those that are associated with at least one class more than expected by chance . Specifically , a centroid c is CIN if max k∈L ( t ) gc ( k ) > 1 |L ( t ) | + γ ( 5 ) where 1|L ( t ) | is the chance term and γ is an additional significance term . 7 . Classification using a hierarchy of centroids : At test time , we are given an input x of class k ( x ) and infer its class as k̂ ( x ) . The classification task is a “ biased voting ” process in which every patch of x , at any layer , votes for a single class as long as that patch selects a CIN centroid . Specifically , if a patch xl , m of layer l selects a CIN centroid c , then that patch votes vl , m ( k ) = maxk∈L ( t ) gc ( k ) for the class k that has the highest association with c , and zero for all other classes . If c is not a CIN centroid , the vote of that patch is vl , m ( k ) = 0 for all classes . The vote of layer l for class k is the average vote across all patches in layer l ( as illustrated in Figure 3 ) : vl ( k ) = ∑ m∈Ml vl , m ( k ) |Ml| ( 6 ) where Ml is the set of patches in layer l. The final inference for input x is the class with the highest cumulative vote across all layers : k̂ ( x ) = arg max k′ Λ∑ l=1 vl ( k ) ( 7 )

Summary: This paper proposes a method called Self-Taught Associative Memory (STAM) for Unsupervised Progressive Learning (UPL) , i.e., learning salient representation from streams of mostly unlabeled data with occasional class labels, where the number of class increases over time. The motivation of this paper is quite interesting in that the authors try to mimic how animals learn. The surrounding environments of animals are considered to be unlabeled, and animals gradually learn to distinguish between objects without explicit information. The model shed light on the problem of catastrophic forgetting by introducing dual-memory organization. To be specific, Short-Term Memory contains a set of centroids associated with the unlabeled data, whereas Long-Term Memory stores the prototypical centroids, which are frequently seen patterns. In addition, the model utilizes novelty detection technique to introduce new centroids to each layer of the model, and it prepares the newly created centroids to be associated with new classes.